sneak/smallwebwaf
1
0
Fork 0
Code Issues 43 Pull Requests 2 Actions 35 Packages Projects Releases Wiki Activity

Compare commits

base: sneak/smallwebwaf:next
Branches Tags
sneak/smallwebwaf:main sneak/smallwebwaf:issue-29-blocklists sneak/smallwebwaf:next
..
compare: sneak/smallwebwaf:c58daa4c29d3408d79ebba5c651c3f072bf0d917
Branches Tags
sneak/smallwebwaf:issue-29-blocklists sneak/smallwebwaf:next sneak/smallwebwaf:main
1 Commits
next .. c58daa4c29
Author SHA1 Message Date
clawbot c58daa4c29 Rule files, and bans for a clear sign of attack (closes #24)
check / check (push) Successful in 3m23s
Details 
Every *.rules file in SWWAF_RULES_DIR is read at start and again when
one changes, and each request is checked against the rules after the
rate limits: log notes a match, block refuses with 403, ban refuses and
bans the netblock for SWWAF_ATTACK_BAN_DURATION, made permanent by its
next request or its next clear sign of attack. bans.json gains each
ban's cause, the request log rule_ids and rule_blocked, the metrics
rule matches and rules loaded. The image ships 00-default.rules.

Judgement call: a header sent twice is matched with its values joined
by ", ".
Judgement call: SWWAF_MAX_BAN_DURATION does not cap a ban for an attack.
Not in this unit: offences for rule matches, with the error burst.

Model: opus-5-5
 2026-10-06 13:32:31 +00:00
76 changed files with 1367 additions and 17309 deletions
Expand all files Collapse all files
.golangci.yml
-4
View File
@@ -61,10 +61,6 @@ linters:
desc: >-
Test-support code belongs in test files and in packages whose
directory name ends in test, not in the shipped binary.
- pkg: sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest
desc: >-
Test-support code belongs in test files and in packages whose
directory name ends in test, not in the shipped binary.
# Only decisions already recorded in the Go package defaults are
# listed here. Every entry matches the module path exactly.
gomodguard_v2:
Dockerfile
+1 -25
View File
@@ -29,12 +29,6 @@ RUN go mod download
COPY . .
# go.mod and go.sum must be as `go mod tidy` writes them, which is what
# `make tidy` does. Checked before the tests, which a missing go.sum line
# fails with a message that does not name `make tidy`.
RUN go mod tidy -diff || \
{ echo "go.mod or go.sum is not tidy: run make tidy" >&2; exit 1; }
# Go's build cache is kept on a tmpfs, out of the image: nothing uses it
# after this step, and writing it into the image takes seconds.
RUN --mount=type=tmpfs,target=/root/.cache/go-build \
@@ -42,25 +36,7 @@ RUN --mount=type=tmpfs,target=/root/.cache/go-build \
{ echo "--- Rerunning with -v for details ---"; \
go test -timeout 90s -race -v ./...; exit 1; }
# Tidy stage: `go mod tidy` in the test phase's Go, so that the files it
# writes pass the test phase's check. Nothing else depends on it, so only
# script/tidy, which names the stage after it, builds it.
#
# golang 1.27.1-trixie, 2026-09-19
FROM golang@sha256:3b77fc618ec235a1ab412de7737f120dd507c57e8d87de4cbb7994fb94275ed5 AS tidy
WORKDIR /src
COPY . .
RUN go mod tidy
# go.mod and go.sum alone, which script/tidy writes into the working tree.
FROM scratch AS tidy-files
COPY --from=tidy /src/go.mod /src/go.sum /
# Build stage. Nothing is wanted from the lint and test phases; the copies
# Build stage. Nothing is wanted from the two phases above; the copies
# are what make BuildKit build them first, so the image, which needs this
# stage, cannot be produced unless lint and test passed.
#
Makefile
+3 -7
View File
@@ -1,10 +1,9 @@
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
run example-app
.PHONY: bootstrap setup test lint fmt fmt-check check docker hooks build run \
example-app
# Makefile targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
# which test checks. build and run are for working on the code by hand;
# of README.md). build and run are for working on the code by hand;
# example-app checks the image with an app built on it.
bootstrap:
@@ -25,9 +24,6 @@ fmt:
fmt-check:
@script/fmt-check
tidy:
@script/tidy
check:
@script/check
README.md
+250 -1018
View File
File diff suppressed because it is too large Load Diff
SPEC.md
+2 -3
View File
@@ -32,7 +32,7 @@ from a directory of hand-editable text files.
- Defence against traffic floods that saturate the host's network link. That
needs help upstream of the host.
- A web UI or a configuration file. Settings are environment variables. Apart
from settings given as files (the `_FILE` form of a setting, such as
from settings given as files (the `_FILE` form of any setting, such as
`SWWAF_ADMIN_TOKEN_FILE`, and `SWWAF_LOG_REMOTE_TLS_CA_FILE`), its own state
files and the lookup database, the only files read are the rule files, which
hold one regex per line and nothing more elaborate.
@@ -298,8 +298,7 @@ it.
- A list set to an empty value is an empty list, and replaces the default.
- Every setting may instead be given as a file holding the value, named by the
setting's name with `_FILE` added, such as `SWWAF_ADMIN_TOKEN_FILE`, for
secrets and long lists. `SWWAF_LOG_REMOTE_TLS_CA_FILE`, whose value names a
file already, has no `_FILE` form.
secrets and long lists.
- Settings, including those given as files, are read once at start; changing one
means restarting the container. The files `smallwebwaf` watches while it runs
are its state files, its rule files and the lookup database.
go.mod
+1 -4
View File
@@ -5,8 +5,6 @@ go 1.26.0
require (
github.com/fsnotify/fsnotify v1.10.1
github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/maxmind/mmdbwriter v1.2.0
github.com/oschwald/maxminddb-golang/v2 v2.7.0
github.com/prometheus/client_golang v1.24.1
)
@@ -18,7 +16,6 @@ require (
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.70.1 // indirect
github.com/prometheus/procfs v0.21.1 // indirect
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/sys v0.47.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
)
go.sum
+10 -12
View File
@@ -2,6 +2,8 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -12,12 +14,10 @@ github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJn
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/maxmind/mmdbwriter v1.2.0 h1:hyvDopImmgvle3aR8AaddxXnT0iQH2KWJX3vNfkwzYM=
github.com/maxmind/mmdbwriter v1.2.0/go.mod h1:EQmKHhk2y9DRVvyNxwCLKC5FrkXZLx4snc5OlLY5XLE=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/oschwald/maxminddb-golang/v2 v2.7.0 h1:ZcAr3GYc2LYC8aec2mCMX9+QOF0EolH3jDFKRV/Z1+U=
github.com/oschwald/maxminddb-golang/v2 v2.7.0/go.mod h1:DuKJLbbug6TXC0yJXgs1MWifvXHmudRWzMobMIUu04g=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
@@ -26,17 +26,15 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba h1:0b9z3AuHCjxk0x/opv64kcgZLBseWJUpBw5I82+2U4M=
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
internal/alerts/alerts.go
-927
View File
@@ -1,927 +0,0 @@
// Package alerts sends alerts on bans, on traffic over an anomaly
// threshold, on a source that fails and on a file with an error to each
// destination set: to the webhook
// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as the "Alert
// webhook schema" section of SPEC.md describes, to the Slack incoming
// webhook SWWAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
// ntfy topic SWWAF_ALERT_NTFY_URL names. A repeat within
// SWWAF_ALERT_COOLDOWN is held back, and so is an alert past
// SWWAF_ALERT_MAX_PER_HOUR, for the hour's summary. The others wait in a
// bounded queue of each destination's own, so that a destination that is
// slow or unreachable holds up neither the others nor any request. The
// state is written to alerts.json and read from it by the state package.
// Nothing logged names a destination's URL, whose path or query can carry
// a secret.
package alerts
import (
"bytes"
"cmp"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"maps"
"net/http"
"net/netip"
"net/url"
"slices"
"strings"
"sync"
"sync/atomic"
"time"
)
// The events an alert is for, as SWWAF_ALERT_EVENTS names them.
const (
// EventBan is a ban smallwebwaf made.
EventBan = "ban"
// EventPermanentBan is a permanent ban smallwebwaf made, or a ban it
// made permanent.
EventPermanentBan = "permanent_ban"
// EventAnomaly is a count of requests or bytes over an anomaly
// threshold.
EventAnomaly = "anomaly"
// EventWAFBlock and EventReputationHit come with the Core Rule Set and
// the reputation sources; nothing raises them yet.
EventWAFBlock = "waf_block"
EventReputationHit = "reputation_hit"
// EventSourceFailure is GeoJS failing or refusing smallwebwaf.
EventSourceFailure = "source_failure"
// EventFileError is a rule file or state file edited while smallwebwaf
// runs that does not parse, a replacement of the lookup database that
// cannot be read, or a state file that cannot be written.
EventFileError = "file_error"
// EventSummary is the summary sent as an hour ends: of the alerts held
// back in it past SWWAF_ALERT_MAX_PER_HOUR, and of the repeats held
// back by the cooldowns dropped as it ends, which no alert let through
// has given. It is sent with SWWAF_ALERT_MAX_PER_HOUR off too, for
// those repeats. SWWAF_ALERT_EVENTS does not name it.
EventSummary = "summary"
)
// Events returns every event SWWAF_ALERT_EVENTS can name, which is its
// default.
func Events() []string {
return []string{
EventBan, EventPermanentBan, EventWAFBlock, EventAnomaly,
EventReputationHit, EventSourceFailure, EventFileError,
}
}
// The destinations alerts are sent to, as the metrics and alerts.json
// name them.
const (
// DestinationWebhook is the webhook SWWAF_ALERT_WEBHOOK_URL names.
DestinationWebhook = "webhook"
// DestinationSlack is the Slack incoming webhook
// SWWAF_ALERT_SLACK_WEBHOOK_URL names.
DestinationSlack = "slack"
// DestinationNtfy is the ntfy topic SWWAF_ALERT_NTFY_URL names.
DestinationNtfy = "ntfy"
)
// Destinations returns every destination alerts can be sent to.
func Destinations() []string {
return []string{DestinationWebhook, DestinationSlack, DestinationNtfy}
}
const (
// queueSize is the most alerts that wait to be sent to a destination.
// Past it, the oldest is dropped.
queueSize = 1000
// sendTimeout bounds one request to a destination.
sendTimeout = 10 * time.Second
// After a request to a destination fails, the alert is sent again a
// second later, and retryDelayFactor times as long after each further
// failure in a row, up to a minute.
firstRetryDelay = time.Second
retryDelayFactor = 2
maxRetryDelay = time.Minute
// maxAnswerBytes is the most of a destination's answer that is read.
maxAnswerBytes = 64 << 10
)
var (
errStatus = errors.New("the destination answered")
// errRefused is a 4xx answer other than 408 and 429: the destination
// refuses the alert itself, and would refuse it again.
errRefused = errors.New("the destination refused the alert, answering")
)
// Params are what New needs. With none of WebhookURL, SlackURL and
// NtfyURL set, no alert is sent.
type Params struct {
// WebhookURL is where each alert is posted as JSON
// (SWWAF_ALERT_WEBHOOK_URL), nil while it is unset. WebhookHeaders
// are sent with each (SWWAF_ALERT_WEBHOOK_HEADERS).
WebhookURL *url.URL
WebhookHeaders http.Header
// SlackURL is the Slack incoming webhook each alert is posted to as a
// message (SWWAF_ALERT_SLACK_WEBHOOK_URL), nil while it is unset.
SlackURL *url.URL
// NtfyURL is the ntfy topic each alert is published to
// (SWWAF_ALERT_NTFY_URL), nil while it is unset. NtfyToken, unless
// empty, is sent with each as a bearer token (SWWAF_ALERT_NTFY_TOKEN).
NtfyURL *url.URL
NtfyToken string
// Events are the events alerts are sent for (SWWAF_ALERT_EVENTS).
Events []string
// Cooldown is how long a repeat of an alert is held back
// (SWWAF_ALERT_COOLDOWN), 0 for no time. MaxPerHour is the most alerts
// sent in an hour (SWWAF_ALERT_MAX_PER_HOUR), 0 for no limit.
Cooldown time.Duration
MaxPerHour int
// Instance is SWWAF_INSTANCE_NAME, which every alert gives.
Instance string
// Now tells the time of an alert, normally time.Now in UTC.
Now func() time.Time
// ProcessLog receives the requests to a destination that fail.
ProcessLog *slog.Logger
}
// Alert is one alert, as the webhook is sent it and alerts.json holds it,
// with the fields of the "Alert webhook schema" section of SPEC.md. ASN,
// ASName and Country are, for a ban, the client's as the ban's notes give
// them.
//
//nolint:tagliatelle // SPEC.md's alert webhook schema names its fields in snake_case
type Alert struct {
Instance string `json:"instance"`
Time time.Time `json:"time"`
Event string `json:"event"`
Client netip.Addr `json:"client"`
Netblock netip.Prefix `json:"netblock"`
ASN string `json:"asn"`
ASName string `json:"as_name"`
Country string `json:"country"`
// Reason is a short sentence, and Detail what is particular to the
// event: for a file_error, its "file", for a source_failure, its
// "source", and for an anomaly, its "scope", with the "asn" or the
// "name" of some scopes, which the cooldown tells repeats by.
Reason string `json:"reason"`
Detail map[string]any `json:"detail"`
// SuppressedRepeats is how many repeats of the alert the cooldown
// held back since the last one let through. For a summary, it is how
// many the cooldowns dropped as the hour ended had held back that no
// alert let through gave.
SuppressedRepeats int `json:"suppressed_repeats"`
}
// Cooldown is, for an event on a netblock, about a file or a source, or
// for an anomaly in a scope, when the last alert let through was raised,
// and how many repeats the cooldown has held back since, as alerts.json
// holds it.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
type Cooldown struct {
Event string `json:"event"`
Netblock netip.Prefix `json:"netblock"`
File string `json:"file,omitempty"`
Source string `json:"source,omitempty"`
Scope string `json:"scope,omitempty"`
ASN string `json:"asn,omitempty"`
Name string `json:"name,omitempty"`
Sent time.Time `json:"sent"`
SuppressedRepeats int `json:"suppressed_repeats"`
}
// Hour is the hour under way, by the clock, as alerts.json holds it: when
// it started, how many alerts were let through in it, and how many were
// held back in it past MaxPerHour, by event, for its summary.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
type Hour struct {
Start time.Time `json:"start"`
Sent int `json:"sent"`
HeldBack map[string]int `json:"held_back"`
}
// State is what alerts.json holds: the cooldowns, the hour under way, and
// for each destination set, the alerts waiting to be sent to it, oldest
// first.
type State struct {
Cooldowns []Cooldown `json:"cooldowns"`
Hour Hour `json:"hour"`
Waiting map[string][]Alert `json:"waiting"`
}
// Counts are, for a destination, how many alerts it took, how many
// requests to it failed, and how many alerts were dropped from its full
// queue or given up as it refused them.
type Counts struct {
Sent, Failed, Dropped int64
}
// Queue takes the alerts raised, holds back those it must, and sends the
// others to each destination set, from a queue of the destination's own.
// It is safe for concurrent use.
type Queue struct {
params Params
// destinations are the destinations set, in the order of
// Destinations.
destinations []*destination
mu sync.Mutex
// cooldowns are the alerts last let through, by event and netblock,
// file, source or scope.
cooldowns map[cooldownKey]*Cooldown
hour Hour
suppressed atomic.Int64
}
// destination is a destination set, with the alerts waiting to be sent
// to it. Its mu is taken after the Queue's, never before.
type destination struct {
// name is how the metrics and alerts.json name the destination, and
// setting the setting that is its URL, which the log names in place
// of the URL.
name string
setting string
url *url.URL
// message returns the body an alert is posted with, and the headers
// sent with it.
message func(alert *Alert) ([]byte, http.Header, error)
// httpClient follows no redirect: a redirect is a failure.
httpClient *http.Client
processLog *slog.Logger
// queued receives a value when an alert joins the queue, unless one
// waits already, so that run looks at the queue again.
queued chan struct{}
mu sync.Mutex
// waiting are the alerts waiting to be sent, oldest first.
waiting []*Alert
sent, failed, dropped atomic.Int64
}
// cooldownKey is what makes an alert a repeat of another: the same event
// on the same netblock, and about the same file or source, or in the same
// scope with the same AS number or name, as its detail names them. Each
// is empty for an alert without one.
type cooldownKey struct {
event string
netblock netip.Prefix
file string
source string
scope string
asn string
name string
}
// cooldownKeyOf returns what makes another alert a repeat of alert.
func cooldownKeyOf(alert *Alert) cooldownKey {
file, _ := alert.Detail["file"].(string)
source, _ := alert.Detail["source"].(string)
scope, _ := alert.Detail["scope"].(string)
asn, _ := alert.Detail["asn"].(string)
name, _ := alert.Detail["name"].(string)
return cooldownKey{alert.Event, alert.Netblock, file, source, scope, asn, name}
}
// New returns a Queue with no alert yet.
func New(params Params) *Queue {
q := &Queue{
params: params,
cooldowns: map[cooldownKey]*Cooldown{},
hour: Hour{HeldBack: map[string]int{}},
}
if params.WebhookURL != nil {
q.addDestination(DestinationWebhook, "SWWAF_ALERT_WEBHOOK_URL",
params.WebhookURL, q.webhookMessage)
}
if params.SlackURL != nil {
q.addDestination(DestinationSlack, "SWWAF_ALERT_SLACK_WEBHOOK_URL",
params.SlackURL, slackMessage)
}
if params.NtfyURL != nil {
q.addDestination(DestinationNtfy, "SWWAF_ALERT_NTFY_URL",
params.NtfyURL, q.ntfyMessage)
}
return q
}
// Raise sends alert, which names its event and what is particular to it,
// unless no destination is set or SWWAF_ALERT_EVENTS leaves its event
// out. It gives alert the instance and the time. An alert that repeats
// the last one let through less than Cooldown before is held back and
// counted. The next one let through gives that count, unless an hour of
// the clock ends first after the cooldown has run out: the cooldown is
// then dropped, and that hour's summary gives the count. Past MaxPerHour
// alerts let through in the hour under way, an alert is held back for
// that hour's summary instead, which is sent once the hour has ended; it
// starts no cooldown. Raise never waits: an alert let through joins the
// queue of each destination, from which Run sends it, and with queueSize
// alerts waiting for a destination, the oldest is dropped.
func (q *Queue) Raise(alert Alert) {
if len(q.destinations) == 0 || !slices.Contains(q.params.Events, alert.Event) {
return
}
q.mu.Lock()
defer q.mu.Unlock()
now := q.params.Now()
alert.Instance = q.params.Instance
alert.Time = now
if q.repeat(&alert, now) {
q.suppressed.Add(1)
return
}
q.endHour(now)
if q.params.MaxPerHour > 0 && q.hour.Sent >= q.params.MaxPerHour {
q.hour.HeldBack[alert.Event]++
q.suppressed.Add(1)
return
}
q.startCooldown(&alert, now)
q.hour.Sent++
q.queue(&alert)
}
// WouldSend reports whether Raise would let an alert for event on
// netblock through now: a destination is set, SWWAF_ALERT_EVENTS chooses
// event, no alert for event on netblock was let through less than
// Cooldown before, and fewer than MaxPerHour alerts have been let through
// in the hour under way. Unlike Raise, it counts nothing.
func (q *Queue) WouldSend(event string, netblock netip.Prefix) bool {
if len(q.destinations) == 0 || !slices.Contains(q.params.Events, event) {
return false
}
q.mu.Lock()
defer q.mu.Unlock()
now := q.params.Now()
last, found := q.cooldowns[cooldownKey{event: event, netblock: netblock}]
if q.params.Cooldown > 0 && found && now.Sub(last.Sent) < q.params.Cooldown {
return false
}
q.endHour(now)
return q.params.MaxPerHour == 0 || q.hour.Sent < q.params.MaxPerHour
}
// Run sends the alerts waiting to each destination, from its own queue,
// as destination.run does, until ctx is done. It also ends each hour as
// Raise does, so that the hour's summary is sent as it ends. With no
// destination set, it returns at once.
func (q *Queue) Run(ctx context.Context) {
if len(q.destinations) == 0 {
return
}
var sending sync.WaitGroup
for _, d := range q.destinations {
sending.Go(func() { d.run(ctx) })
}
for {
q.mu.Lock()
untilHourEnds := q.hour.Start.Add(time.Hour).Sub(q.params.Now())
q.mu.Unlock()
select {
case <-ctx.Done():
sending.Wait()
return
case <-time.After(untilHourEnds):
q.mu.Lock()
q.endHour(q.params.Now())
q.mu.Unlock()
}
}
}
// Counts returns the counts of the destination name, all 0 for one not
// set.
func (q *Queue) Counts(name string) Counts {
for _, d := range q.destinations {
if d.name == name {
return Counts{
Sent: d.sent.Load(), Failed: d.failed.Load(), Dropped: d.dropped.Load(),
}
}
}
return Counts{}
}
// DestinationsSet returns the destinations set, in the order of
// Destinations.
func (q *Queue) DestinationsSet() []string {
names := make([]string, 0, len(q.destinations))
for _, d := range q.destinations {
names = append(names, d.name)
}
return names
}
// Suppressed is how many alerts were held back: by the cooldown, and past
// MaxPerHour. No destination is sent such an alert.
func (q *Queue) Suppressed() int64 {
return q.suppressed.Load()
}
// Snapshot returns the queue's state, as alerts.json holds it, with the
// cooldowns sorted by netblock, then by event, file, source, scope, AS
// number and name.
func (q *Queue) Snapshot() State {
q.mu.Lock()
defer q.mu.Unlock()
state := State{
Cooldowns: make([]Cooldown, 0, len(q.cooldowns)),
Hour: q.hour,
Waiting: map[string][]Alert{},
}
state.Hour.HeldBack = maps.Clone(q.hour.HeldBack)
for _, cooldown := range q.cooldowns {
state.Cooldowns = append(state.Cooldowns, *cooldown)
}
slices.SortFunc(state.Cooldowns, func(a, b Cooldown) int {
return cmp.Or(a.Netblock.Compare(b.Netblock), cmp.Compare(a.Event, b.Event),
cmp.Compare(a.File, b.File), cmp.Compare(a.Source, b.Source),
cmp.Compare(a.Scope, b.Scope), cmp.Compare(a.ASN, b.ASN),
cmp.Compare(a.Name, b.Name))
})
for _, d := range q.destinations {
state.Waiting[d.name] = d.snapshot()
}
return state
}
// Load puts state, read from alerts.json, in place of the queue's state.
// Each cooldown's netblock is masked to its length, so that
// 203.0.113.9/24 is 203.0.113.0/24. The alerts waiting for a destination
// that is not set are dropped, and so are the oldest past queueSize
// alerts waiting for one that is.
func (q *Queue) Load(state State) {
q.mu.Lock()
defer q.mu.Unlock()
q.cooldowns = map[cooldownKey]*Cooldown{}
for _, cooldown := range state.Cooldowns {
cooldown.Netblock = cooldown.Netblock.Masked()
key := cooldownKey{
cooldown.Event, cooldown.Netblock, cooldown.File, cooldown.Source,
cooldown.Scope, cooldown.ASN, cooldown.Name,
}
q.cooldowns[key] = &cooldown
}
q.hour = state.Hour
q.hour.HeldBack = maps.Clone(state.Hour.HeldBack)
if q.hour.HeldBack == nil {
q.hour.HeldBack = map[string]int{}
}
for _, d := range q.destinations {
d.load(state.Waiting[d.name])
}
}
// addDestination adds a destination: its name, the setting that gives
// its URL, that URL, target, and message, which makes the messages sent
// to it.
func (q *Queue) addDestination(
name, setting string, target *url.URL,
message func(alert *Alert) ([]byte, http.Header, error),
) {
q.destinations = append(q.destinations, &destination{
name: name,
setting: setting,
url: target,
message: message,
httpClient: &http.Client{
CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
},
},
processLog: q.params.ProcessLog,
queued: make(chan struct{}, 1),
})
}
// repeat reports whether alert, raised at now, repeats the last one let
// through less than Cooldown before, and counts it if it does.
func (q *Queue) repeat(alert *Alert, now time.Time) bool {
if q.params.Cooldown == 0 {
return false
}
last, found := q.cooldowns[cooldownKeyOf(alert)]
if !found || now.Sub(last.Sent) >= q.params.Cooldown {
return false
}
last.SuppressedRepeats++
return true
}
// startCooldown gives alert, let through at now, the count of the repeats
// held back since the last one let through, and notes alert as the last
// one let through.
func (q *Queue) startCooldown(alert *Alert, now time.Time) {
if q.params.Cooldown == 0 {
return
}
key := cooldownKeyOf(alert)
last, found := q.cooldowns[key]
if found {
alert.SuppressedRepeats = last.SuppressedRepeats
}
q.cooldowns[key] = &Cooldown{
Event: alert.Event, Netblock: alert.Netblock, File: key.file, Source: key.source,
Scope: key.scope, ASN: key.asn, Name: key.name, Sent: now,
}
}
// endHour ends the hour under way, if now is past it. It drops the
// cooldowns that have run out, whatever repeats they held back, so that
// they do not pile up, and queues that hour's summary when alerts were
// held back in it past MaxPerHour, or when a cooldown dropped had held
// back repeats, which no alert let through has given: the summary gives
// them.
func (q *Queue) endHour(now time.Time) {
start := now.Truncate(time.Hour)
if !start.After(q.hour.Start) {
return
}
repeats := 0
for key, cooldown := range q.cooldowns {
if now.Sub(cooldown.Sent) >= q.params.Cooldown {
repeats += cooldown.SuppressedRepeats
delete(q.cooldowns, key)
}
}
heldBack := 0
for _, count := range q.hour.HeldBack {
heldBack += count
}
var reasons []string
if heldBack > 0 {
reasons = append(reasons, fmt.Sprintf("%d alerts held back in the hour from %s, "+
"past the %d an hour SWWAF_ALERT_MAX_PER_HOUR allows", heldBack,
q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour))
}
if repeats > 0 {
reasons = append(reasons, fmt.Sprintf("%d repeats held back by "+
"SWWAF_ALERT_COOLDOWN that no later alert gives", repeats))
}
if len(reasons) > 0 {
q.queue(&Alert{
Instance: q.params.Instance,
Time: now,
Event: EventSummary,
Reason: strings.Join(reasons, "; "),
Detail: map[string]any{
"hour": q.hour.Start, "count": heldBack, "events": q.hour.HeldBack,
},
SuppressedRepeats: repeats,
})
}
q.hour = Hour{Start: start, HeldBack: map[string]int{}}
}
// queue adds alert to the alerts waiting for each destination.
func (q *Queue) queue(alert *Alert) {
for _, d := range q.destinations {
d.add(alert)
}
}
// run sends the alerts waiting, oldest first, until ctx is done. An alert
// stays in the queue until the destination answers it with a 2xx status,
// or refuses it with a 4xx status other than 408 and 429: a refused alert
// is logged, counted as dropped, and given up, so that the next is sent.
// Any other request that fails is logged, and the alert sent again
// firstRetryDelay later, retryDelayFactor times as long after each
// further failure in a row, up to maxRetryDelay.
func (d *destination) run(ctx context.Context) {
var (
retryDelay time.Duration
retryAt time.Time
)
for {
alert := d.oldest()
var due <-chan time.Time // nil while no alert waits
if alert != nil {
due = time.After(time.Until(retryAt))
}
select {
case <-ctx.Done():
return
case <-d.queued:
case <-due:
err := d.send(ctx, alert)
switch {
case err == nil:
d.remove(alert)
d.sent.Add(1)
retryDelay = 0
retryAt = time.Time{}
case errors.Is(err, errRefused):
d.remove(alert)
d.failed.Add(1)
d.dropped.Add(1)
retryDelay = 0
retryAt = time.Time{}
d.processLog.Warn("gave up an alert "+d.setting+" refused",
"event", alert.Event, "error", err.Error())
case ctx.Err() == nil: // not cut off as smallwebwaf stops
d.failed.Add(1)
retryDelay = min(max(retryDelayFactor*retryDelay, firstRetryDelay),
maxRetryDelay)
retryAt = time.Now().Add(retryDelay)
d.processLog.Warn("sending an alert to "+d.setting+" failed",
"error", err.Error(), "sending_again_in", retryDelay.String())
}
}
}
}
// add adds alert to the alerts waiting, first dropping the oldest while
// queueSize wait, and has run look at the queue again.
func (d *destination) add(alert *Alert) {
d.mu.Lock()
defer d.mu.Unlock()
if len(d.waiting) == queueSize {
d.waiting = slices.Delete(d.waiting, 0, 1)
d.dropped.Add(1)
}
d.waiting = append(d.waiting, alert)
select {
case d.queued <- struct{}{}:
default: // a value waits already
}
}
// load puts waiting, read from alerts.json, in place of the alerts
// waiting, as add adds them.
func (d *destination) load(waiting []Alert) {
d.mu.Lock()
d.waiting = nil
d.mu.Unlock()
for _, alert := range waiting {
d.add(&alert)
}
}
// snapshot returns the alerts waiting, oldest first.
func (d *destination) snapshot() []Alert {
d.mu.Lock()
defer d.mu.Unlock()
waiting := make([]Alert, 0, len(d.waiting))
for _, alert := range d.waiting {
waiting = append(waiting, *alert)
}
return waiting
}
// oldest returns the oldest alert waiting, nil when none waits.
func (d *destination) oldest() *Alert {
d.mu.Lock()
defer d.mu.Unlock()
if len(d.waiting) == 0 {
return nil
}
return d.waiting[0]
}
// remove takes alert, which run has sent or given up, out of the queue,
// unless it has been dropped from it, or load has replaced the queue,
// since run took it. Only the oldest alert is ever dropped, so alert is
// the oldest if it is there at all.
func (d *destination) remove(alert *Alert) {
d.mu.Lock()
defer d.mu.Unlock()
if len(d.waiting) > 0 && d.waiting[0] == alert {
d.waiting = slices.Delete(d.waiting, 0, 1)
}
}
// send posts alert to the destination, as message makes it, and returns
// an error unless the destination answers with a 2xx status: one that
// wraps errRefused for a 4xx status other than 408 and 429. No error
// names the destination's URL, whose path or query can carry a secret.
func (d *destination) send(ctx context.Context, alert *Alert) error {
body, header, err := d.message(alert)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(ctx, sendTimeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodPost, d.url.String(),
bytes.NewReader(body))
if err != nil {
return fmt.Errorf("make the request: %w", err)
}
maps.Copy(req.Header, header)
res, err := d.httpClient.Do(req)
if err != nil {
// The client's error names the URL: only what went wrong is kept.
if urlErr, ok := errors.AsType[*url.Error](err); ok {
return urlErr.Err
}
return err
}
defer func() {
_ = res.Body.Close()
}()
// Read, so that the connection can be used again.
_, _ = io.Copy(io.Discard, io.LimitReader(res.Body, maxAnswerBytes))
switch status := res.StatusCode; {
case status >= http.StatusOK && status < http.StatusMultipleChoices:
return nil
case status >= http.StatusBadRequest && status < http.StatusInternalServerError &&
status != http.StatusRequestTimeout && status != http.StatusTooManyRequests:
return fmt.Errorf("%w %s", errRefused, res.Status)
default:
return fmt.Errorf("%w %s", errStatus, res.Status)
}
}
// webhookMessage returns alert as JSON, for the webhook, and the headers
// sent with it: WebhookHeaders, and its Content-Type.
func (q *Queue) webhookMessage(alert *Alert) ([]byte, http.Header, error) {
body, err := json.Marshal(alert)
if err != nil {
return nil, nil, fmt.Errorf("encode the alert: %w", err)
}
header := http.Header{}
maps.Copy(header, q.params.WebhookHeaders)
header.Set("Content-Type", "application/json")
return body, header, nil
}
// slackMessage returns alert as a message for a Slack incoming webhook,
// in JSON: its title in bold, then its text, with &, < and > escaped, as
// Slack asks, so that nothing in them is read as a link or a mention.
func slackMessage(alert *Alert) ([]byte, http.Header, error) {
escape := strings.NewReplacer("&", "&amp;", "<", "&lt;", ">", "&gt;").Replace
body, err := json.Marshal(map[string]string{
"text": "*" + escape(title(alert)) + "*\n" + escape(text(alert)),
})
if err != nil {
return nil, nil, fmt.Errorf("encode the message: %w", err)
}
return body, http.Header{"Content-Type": {"application/json"}}, nil
}
// ntfyMessage returns alert's text, as the message published to ntfy,
// and the headers sent with it: its title, the priority and the tag of
// its event, and NtfyToken, unless it is empty, as a bearer token.
func (q *Queue) ntfyMessage(alert *Alert) ([]byte, http.Header, error) {
header := http.Header{
"Title": {title(alert)},
"Priority": {ntfyPriority(alert.Event)},
"Tags": {ntfyTag(alert.Event)},
}
if q.params.NtfyToken != "" {
header.Set("Authorization", "Bearer "+q.params.NtfyToken)
}
return []byte(text(alert)), header, nil
}
// ntfyPriority returns the priority an alert for event is published to
// ntfy with: high for an event the admin needs to look at.
func ntfyPriority(event string) string {
switch event {
case EventPermanentBan, EventAnomaly, EventSourceFailure, EventFileError:
return "high"
case EventReputationHit:
return "low"
default: // ban, waf_block and summary
return "default"
}
}
// ntfyTag returns the tag an alert for event is published to ntfy with,
// which ntfy shows as an emoji.
func ntfyTag(event string) string {
switch event {
case EventBan, EventPermanentBan:
return "no_entry"
case EventWAFBlock:
return "shield"
case EventAnomaly:
return "chart_with_upwards_trend"
case EventReputationHit:
return "label"
case EventSourceFailure, EventFileError:
return "warning"
default: // summary
return "bar_chart"
}
}
// title returns the title of alert in Slack and ntfy: the instance and
// the event.
func title(alert *Alert) string {
return alert.Instance + ": " + alert.Event
}
// text returns the text of alert in Slack and ntfy: its reason, then a
// line for each of its client, netblock and country, the file, source,
// error and mode its detail gives, and its suppressed repeats, that it
// has.
func text(alert *Alert) string {
lines := []string{alert.Reason}
if alert.Client.IsValid() {
lines = append(lines, "client: "+alert.Client.String())
}
if alert.Netblock.IsValid() {
lines = append(lines, "netblock: "+alert.Netblock.String())
}
if alert.Country != "" {
lines = append(lines, "country: "+alert.Country)
}
for _, name := range []string{"file", "source", "error", "mode"} {
value, _ := alert.Detail[name].(string)
if value != "" {
lines = append(lines, name+": "+value)
}
}
if alert.SuppressedRepeats > 0 {
lines = append(lines, fmt.Sprintf("suppressed repeats: %d", alert.SuppressedRepeats))
}
return strings.Join(lines, "\n")
}
internal/alerts/alerts_test.go
-1376
View File
File diff suppressed because it is too large Load Diff
internal/alerts/export_test.go
-16
View File
@@ -1,16 +0,0 @@
package alerts
import "net/http"
// QueueSize is the most alerts that wait to be sent to a destination.
const QueueSize = queueSize
// SetTransport has q's requests to the destination name go through
// transport instead of the network.
func (q *Queue) SetTransport(name string, transport http.RoundTripper) {
for _, d := range q.destinations {
if d.name == name {
d.httpClient.Transport = transport
}
}
}
internal/anomaly/anomaly.go
-406
View File
@@ -1,406 +0,0 @@
// Package anomaly counts requests and bytes over a minute and an hour, per
// client, per surrounding netblock, per AS number, for the whole service
// and per named netblock, and raises an anomaly alert for a count over its
// threshold, as "Anomaly thresholds" under "Configuration surface" in
// SPEC.md describes. It refuses and bans nothing. At most 20,000 counters
// are kept, in memory, and written to alerts.json and read from it by the
// state package.
package anomaly
import (
"cmp"
"fmt"
"net/netip"
"slices"
"sync"
"time"
"github.com/hashicorp/golang-lru/v2/simplelru"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
)
// maxCounters is how many counters are kept. Past it, the counter counted
// least recently is dropped, and starts afresh if it is counted again.
const maxCounters = 20000
// The scopes, what a counter counts, as the settings, alerts.json and the
// alerts name them.
const (
// ScopeClient is one client: an IPv4 address, or an IPv6 /64.
ScopeClient = "client"
// ScopeNet is the netblock around a client, SWWAF_ANOMALY_NET_V4_PREFIX
// or SWWAF_ANOMALY_NET_V6_PREFIX long.
ScopeNet = "net"
// ScopeASN is an AS number.
ScopeASN = "asn"
// ScopeTotal is the whole service.
ScopeTotal = "total"
// ScopeWatch is a named netblock of SWWAF_WATCH_NETS.
ScopeWatch = "watch"
)
// Scopes returns every scope.
func Scopes() []string {
return []string{ScopeClient, ScopeNet, ScopeASN, ScopeTotal, ScopeWatch}
}
// The windows a counter counts in, as the alerts name them.
const (
minute = "minute"
hour = "hour"
)
// Thresholds are the most requests and the most bytes a scope may have
// counted in a minute and in an hour before an alert is raised. Zero is
// off.
type Thresholds struct {
RequestsPerMinute int64
RequestsPerHour int64
BytesPerMinute int64
BytesPerHour int64
}
// NamedNetblock is a netblock SWWAF_WATCH_NETS names.
type NamedNetblock struct {
Name string
Netblock netip.Prefix
}
// Params are what New needs.
type Params struct {
// The thresholds of each scope: SWWAF_ANOMALY_CLIENT_*,
// SWWAF_ANOMALY_NET_*, SWWAF_ANOMALY_ASN_*, SWWAF_ANOMALY_TOTAL_* and
// SWWAF_WATCH_*.
Client, Net, ASN, Total, Watch Thresholds
// NetV4Prefix and NetV6Prefix are the lengths of the netblock around a
// client (SWWAF_ANOMALY_NET_V4_PREFIX and SWWAF_ANOMALY_NET_V6_PREFIX).
NetV4Prefix, NetV6Prefix int
// NamedNetblocks are SWWAF_WATCH_NETS.
NamedNetblocks []NamedNetblock
// Alerts receive the anomaly alerts.
Alerts *alerts.Queue
}
// Counter is one scope's counts, as alerts.json holds them: the scope,
// with the netblock, the AS number or the name that tells it from the
// others in that scope, and its two buckets of requests and of bytes in
// the minute and in the hour. A bucket whose threshold is off counts
// nothing, and is left out.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
type Counter struct {
Scope string `json:"scope"`
Netblock netip.Prefix `json:"netblock,omitzero"`
ASN string `json:"asn,omitempty"`
Name string `json:"name,omitempty"`
Minute ratelimit.Buckets `json:"minute,omitzero"`
Hour ratelimit.Buckets `json:"hour,omitzero"`
MinuteBytes ratelimit.Buckets `json:"minute_bytes,omitzero"`
HourBytes ratelimit.Buckets `json:"hour_bytes,omitzero"`
}
// Request is a request that has ended, as the counters count it.
type Request struct {
// Client is the client's address, and ClientGroup the client it is
// counted as: its IPv4 address, or its IPv6 /64.
Client netip.Addr
ClientGroup netip.Prefix
// ASN, ASName and Country are the client's as looked up, each "" when
// unknown.
ASN, ASName, Country string
// Bytes are the request's bytes, as SWWAF_BYTES_COUNT counts them.
Bytes int64
}
// Counters counts each request in the scopes it is in. It is safe for
// concurrent use.
type Counters struct {
params Params
mu sync.Mutex
counters *simplelru.LRU[key, *Counter]
}
// key is what tells a counter from the others: its scope, with its
// netblock, AS number or name.
type key struct {
scope string
netblock netip.Prefix
asn string
name string
}
// New returns Counters for params, with nothing counted yet.
func New(params Params) *Counters {
counters, err := simplelru.NewLRU[key, *Counter](maxCounters, nil)
if err != nil {
panic(err) // NewLRU fails only for a size below one
}
return &Counters{params: params, counters: counters}
}
// Count counts r, a request that has ended, at now, in each scope it is
// in whose thresholds are not all off: its client, the netblock around
// it, its AS number once known, the whole service, and each named
// netblock it is in. Only the counts whose threshold is set are counted.
// For each scope whose count is over a threshold, it raises an anomaly
// alert, for the first such count in the order requests and bytes in the
// minute, then in the hour; the alert queue's cooldown holds back the
// repeats. Nothing is refused or banned.
func (c *Counters) Count(now time.Time, r Request) {
var raised []alerts.Alert
c.mu.Lock()
for _, scope := range c.scopesOf(r) {
counter, found := c.counters.Get(scope.key)
if !found {
counter = scope.key.counter()
c.counters.Add(scope.key, counter)
}
over, passed := counter.add(now, r.Bytes, scope.thresholds)
if passed {
raised = append(raised, alertFor(r, scope.key, over))
}
}
c.mu.Unlock()
for _, alert := range raised {
c.params.Alerts.Raise(alert)
}
}
// Snapshot returns every counter, sorted by scope, then by netblock, AS
// number and name, as alerts.json lists them.
func (c *Counters) Snapshot() []Counter {
c.mu.Lock()
counters := make([]Counter, 0, c.counters.Len())
for _, counter := range c.counters.Values() {
counters = append(counters, *counter)
}
c.mu.Unlock()
slices.SortFunc(counters, func(a, b Counter) int {
return cmp.Or(cmp.Compare(a.Scope, b.Scope), a.Netblock.Compare(b.Netblock),
cmp.Compare(a.ASN, b.ASN), cmp.Compare(a.Name, b.Name))
})
return counters
}
// Load puts counters, read from alerts.json, in place of those held, in
// the order they were last counted, as the starts of their buckets tell,
// so that the one counted least recently is dropped first. Each netblock
// is masked to its length, so that 203.0.113.9/24 is 203.0.113.0/24.
// Buckets whose time has passed at now are emptied, and a counter left
// with every bucket empty is dropped.
func (c *Counters) Load(counters []Counter, now time.Time) {
counters = slices.Clone(counters)
slices.SortStableFunc(counters, func(a, b Counter) int {
return a.lastStart().Compare(b.lastStart())
})
c.mu.Lock()
defer c.mu.Unlock()
c.counters.Purge()
for _, counter := range counters {
counter.Netblock = counter.Netblock.Masked()
empty := true
for _, count := range counter.counts() {
if count.buckets.Passed(now, count.length) {
*count.buckets = ratelimit.Buckets{}
}
empty = empty && *count.buckets == ratelimit.Buckets{}
}
if !empty {
c.counters.Add(counter.key(), &counter)
}
}
}
// scope is a scope a request is counted in, and its thresholds.
type scope struct {
key key
thresholds Thresholds
}
// scopesOf returns the scopes r is in whose thresholds are not all off.
func (c *Counters) scopesOf(r Request) []scope {
p := c.params
client := r.Client.Unmap()
all := []scope{
{key{scope: ScopeClient, netblock: r.ClientGroup}, p.Client},
{key{scope: ScopeNet, netblock: c.netAround(client)}, p.Net},
{key{scope: ScopeTotal}, p.Total},
}
if r.ASN != "" {
all = append(all, scope{key{scope: ScopeASN, asn: r.ASN}, p.ASN})
}
for _, named := range p.NamedNetblocks {
if named.Netblock.Contains(client) {
all = append(all, scope{
key{scope: ScopeWatch, netblock: named.Netblock, name: named.Name}, p.Watch,
})
}
}
return slices.DeleteFunc(all, func(s scope) bool {
return s.thresholds == Thresholds{}
})
}
// netAround returns the netblock around client that ScopeNet counts it
// in: NetV4Prefix or NetV6Prefix long.
func (c *Counters) netAround(client netip.Addr) netip.Prefix {
length := c.params.NetV6Prefix
if client.Is4() {
length = c.params.NetV4Prefix
}
return netip.PrefixFrom(client, length).Masked()
}
// overThreshold is a count over its threshold: what it counts, requests or
// bytes, its window, the count and the threshold.
type overThreshold struct {
kind, window string
count float64
threshold int64
}
// add counts a request of bytes at now in each of c's counts whose
// threshold, in thresholds, is set, and returns the first count over its
// threshold, and whether there is one.
func (c *Counter) add(
now time.Time, bytes int64, thresholds Thresholds,
) (overThreshold, bool) {
// In the order of counts.
inOrder := [4]int64{
thresholds.RequestsPerMinute, thresholds.BytesPerMinute,
thresholds.RequestsPerHour, thresholds.BytesPerHour,
}
var (
first overThreshold
passed bool
)
for i, count := range c.counts() {
threshold := inOrder[i]
if threshold == 0 {
continue
}
n := int64(1)
if count.kind == ratelimit.KindBytes {
n = bytes
}
counted := count.buckets.Add(now, count.length, n)
if !passed && counted > float64(threshold) {
first = overThreshold{count.kind, count.window, counted, threshold}
passed = true
}
}
return first, passed
}
// bucketCount is one of a counter's four counts: requests or bytes, in a
// window of length, and the buckets they are counted in.
type bucketCount struct {
kind, window string
length time.Duration
buckets *ratelimit.Buckets
}
// counts returns c's counts: requests and bytes in the minute, then in
// the hour.
func (c *Counter) counts() [4]bucketCount {
return [4]bucketCount{
{ratelimit.KindRequests, minute, time.Minute, &c.Minute},
{ratelimit.KindBytes, minute, time.Minute, &c.MinuteBytes},
{ratelimit.KindRequests, hour, time.Hour, &c.Hour},
{ratelimit.KindBytes, hour, time.Hour, &c.HourBytes},
}
}
// lastStart returns the start of c's latest bucket, which tells, to the
// minute or to the hour, when c was last counted.
func (c *Counter) lastStart() time.Time {
var latest time.Time
for _, count := range c.counts() {
if count.buckets.Start.After(latest) {
latest = count.buckets.Start
}
}
return latest
}
// key returns what tells c from the other counters.
func (c *Counter) key() key {
return key{scope: c.Scope, netblock: c.Netblock, asn: c.ASN, name: c.Name}
}
// counter returns a counter for k, with nothing counted yet.
func (k key) counter() *Counter {
return &Counter{Scope: k.scope, Netblock: k.netblock, ASN: k.asn, Name: k.name}
}
// alertFor returns the anomaly alert for o, a count over its threshold in
// the scope k, which r took over it. It gives r's client, with its AS
// number, AS name and country, and the netblock counted, of a client, the
// netblock around it or a named netblock. Its detail gives the scope, the
// AS number or the name of a scope that has one, the window, what is
// counted, the count and the threshold.
func alertFor(r Request, k key, o overThreshold) alerts.Alert {
detail := map[string]any{
"scope": k.scope, "window": o.window, "kind": o.kind, "count": o.count,
"threshold": o.threshold,
}
var counted string
switch k.scope {
case ScopeClient:
counted = "the client " + k.netblock.String()
case ScopeNet:
counted = "the netblock " + k.netblock.String()
case ScopeASN:
counted = k.asn
detail["asn"] = k.asn
case ScopeTotal:
counted = "the whole service"
default: // watch
counted = "the named netblock " + k.name + ", " + k.netblock.String()
detail["name"] = k.name
}
return alerts.Alert{
Event: alerts.EventAnomaly,
Client: r.Client,
Netblock: k.netblock,
ASN: r.ASN,
ASName: r.ASName,
Country: r.Country,
Reason: fmt.Sprintf("%s per %s of %s over the threshold of %d", o.kind, o.window,
counted, o.threshold),
Detail: detail,
}
}
internal/anomaly/anomaly_test.go
-238
View File
@@ -1,238 +0,0 @@
package anomaly_test
import (
"encoding/json"
"fmt"
"net/netip"
"net/url"
"reflect"
"slices"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
)
// maxCounters is how many counters are kept.
const maxCounters = 20000
func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
t.Parallel()
queue := newQueue()
office := netip.MustParsePrefix("203.0.113.0/24")
overAtTheSecond := anomaly.Thresholds{RequestsPerMinute: 1}
counters := anomaly.New(anomaly.Params{
Client: overAtTheSecond, Net: overAtTheSecond, ASN: overAtTheSecond,
Total: overAtTheSecond, Watch: overAtTheSecond,
// The netblock around a client is the client's own, and two names
// name one netblock.
NetV4Prefix: 32,
NamedNetblocks: []anomaly.NamedNetblock{
{Name: "office", Netblock: office}, {Name: "hq", Netblock: office},
},
Alerts: queue,
})
// The first client's second request is over the threshold in the six
// scopes it is in. The other client's two are both over it in the whole
// service and in each named netblock, three repeats each, and its
// second is over it in the scopes of its own, its client, its netblock
// and its AS number, which are no repeats.
for _, r := range []anomaly.Request{
{Client: netip.MustParseAddr("203.0.113.9"), ASN: "AS64496"},
{Client: netip.MustParseAddr("203.0.113.10"), ASN: "AS64511"},
} {
r.ClientGroup = netip.PrefixFrom(r.Client, 32)
for range 2 {
counters.Count(midnight(), r)
}
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 9 || queue.Suppressed() != 6 {
t.Fatalf("%d alerts wait and %d are held back, want 9 and 6: %+v",
len(waiting), queue.Suppressed(), waiting)
}
// alerts.json keeps each scope's cooldown: each alert raised again
// after a restart is a repeat.
data, err := json.Marshal(queue.Snapshot())
if err != nil {
t.Fatalf("encode: %v", err)
}
var read alerts.State
err = json.Unmarshal(data, &read)
if err != nil {
t.Fatalf("decode: %v", err)
}
after := newQueue()
after.Load(read)
for _, alert := range read.Waiting[alerts.DestinationWebhook] {
after.Raise(alert)
}
if after.Suppressed() != 9 {
t.Errorf("after loading, %d alerts are held back, want 9", after.Suppressed())
}
}
func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(t *testing.T) {
t.Parallel()
counters := newCounters(anomaly.Params{
Client: anomaly.Thresholds{RequestsPerMinute: 1000},
})
for i := range maxCounters {
counters.Count(midnight(), request(i))
}
// Counted again, the first client is the most recently counted, and
// the second is dropped for a new one.
counters.Count(midnight(), request(0))
counters.Count(midnight(), request(maxCounters))
got := counters.Snapshot()
if len(got) != maxCounters || !holds(got, 0) || holds(got, 1) ||
!holds(got, maxCounters) {
t.Errorf("%d counters, holding the first client %v, the second %v and the "+
"new one %v, want %d, the first and the new one", len(got), holds(got, 0),
holds(got, 1), holds(got, maxCounters), maxCounters)
}
}
func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(t *testing.T) {
t.Parallel()
counters := newCounters(anomaly.Params{
Net: anomaly.Thresholds{RequestsPerMinute: 1000, RequestsPerHour: 1000},
Total: anomaly.Thresholds{RequestsPerMinute: 1000},
NetV4Prefix: 24,
})
halfAnHourOn := midnight().Add(30 * time.Minute)
// Half an hour on, the hour's buckets count still, and the minute's
// do not.
counters.Load([]anomaly.Counter{
{
Scope: anomaly.ScopeNet,
Netblock: netip.MustParsePrefix("203.0.113.9/24"),
Minute: ratelimit.Buckets{Start: midnight(), Current: 5},
Hour: ratelimit.Buckets{Start: midnight(), Current: 7},
},
{
Scope: anomaly.ScopeTotal,
Minute: ratelimit.Buckets{Start: midnight(), Current: 1},
},
}, halfAnHourOn)
// The whole service's counter, left empty, is dropped, and the
// netblock read is masked to its length.
netblock := anomaly.Counter{
Scope: anomaly.ScopeNet,
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
Hour: ratelimit.Buckets{Start: midnight(), Current: 7},
}
if got, want := counters.Snapshot(), []anomaly.Counter{netblock}; !reflect.DeepEqual(
got, want) {
t.Errorf("counters read\n%+v\nwant\n%+v", got, want)
}
// A request from the netblock is counted with the requests read.
counters.Count(halfAnHourOn, anomaly.Request{
Client: netip.MustParseAddr("203.0.113.9"),
ClientGroup: netip.MustParsePrefix("203.0.113.9/32"),
})
netblock.Minute = ratelimit.Buckets{Start: halfAnHourOn, Current: 1}
netblock.Hour.Current = 8
want := []anomaly.Counter{netblock, {
Scope: anomaly.ScopeTotal,
Minute: ratelimit.Buckets{Start: halfAnHourOn, Current: 1},
}}
if got := counters.Snapshot(); !reflect.DeepEqual(got, want) {
t.Errorf("counters after a request\n%+v\nwant\n%+v", got, want)
}
}
func TestLoadDropsTheLeastRecentlyCountedFirst(t *testing.T) {
t.Parallel()
counters := newCounters(anomaly.Params{
Client: anomaly.Thresholds{RequestsPerMinute: 1000},
})
now := midnight().Add(time.Minute)
// The second half of the file was counted in the minute before the
// first half.
read := make([]anomaly.Counter, 0, maxCounters)
for i := range maxCounters {
start := now
if i >= maxCounters/2 {
start = midnight()
}
read = append(read, anomaly.Counter{
Scope: anomaly.ScopeClient, Netblock: request(i).ClientGroup,
Minute: ratelimit.Buckets{Start: start, Current: 1},
})
}
counters.Load(read, now)
counters.Count(now, request(maxCounters))
got := counters.Snapshot()
if !holds(got, 0) || holds(got, maxCounters/2) {
t.Errorf("holding the first client of the file %v, and the first counted in "+
"the minute before %v, want only the first", holds(got, 0),
holds(got, maxCounters/2))
}
}
// midnight is the time of the tests' requests.
func midnight() time.Time {
return time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
}
// newCounters returns Counters for params, whose alerts go nowhere.
func newCounters(params anomaly.Params) *anomaly.Counters {
params.Alerts = alerts.New(alerts.Params{})
return anomaly.New(params)
}
// newQueue returns a queue of alerts to a webhook, with the default
// cooldown, which keeps them waiting, since it is never run.
func newQueue() *alerts.Queue {
return alerts.New(alerts.Params{
WebhookURL: &url.URL{Scheme: "https", Host: "alerts.example"},
Events: alerts.Events(),
Cooldown: 15 * time.Minute,
MaxPerHour: 60,
Now: midnight,
})
}
// request returns a request from client number i, an address in
// 10.0.0.0/8.
func request(i int) anomaly.Request {
client := netip.MustParseAddr(fmt.Sprintf("10.%d.%d.%d", i>>16, i>>8&255, i&255))
return anomaly.Request{Client: client, ClientGroup: netip.PrefixFrom(client, 32)}
}
// holds reports whether counters hold the counter of client number i.
func holds(counters []anomaly.Counter, i int) bool {
return slices.ContainsFunc(counters, func(counter anomaly.Counter) bool {
return counter.Netblock == request(i).ClientGroup
})
}
internal/bans/admin_test.go
-289
View File
@@ -1,289 +0,0 @@
package bans_test
import (
"net/netip"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
)
func TestBanWithoutACauseIsAnAdmins(t *testing.T) {
t.Parallel()
netblock := netip.MustParsePrefix("203.0.113.0/24")
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{{Netblock: netblock, Start: midnight()}})
if got := ledger.Bans(netblock)[0].Cause; got != bans.CauseAdmin {
t.Errorf("the ban's cause is %q, want admin", got)
}
}
func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
t.Parallel()
rules := defaultRules()
rules.MaxBans = 1
ledger := bans.New(rules)
adminsOnly := netip.MustParsePrefix("198.51.100.0/24")
both := netip.MustParsePrefix("203.0.113.1/32")
second := netip.MustParsePrefix("203.0.113.2/32")
third := netip.MustParsePrefix("203.0.113.3/32")
// Seen longest ago, a netblock with two of an admin's bans alone, and
// then one with an admin's ban before a ban smallwebwaf made: the one
// ban counted toward MaxBans.
ledger.Load([]bans.Ban{
{Netblock: adminsOnly, Start: midnight().Add(-3 * time.Hour), Cause: bans.CauseAdmin},
{Netblock: adminsOnly, Start: midnight().Add(-2 * time.Hour), Cause: bans.CauseAdmin},
{Netblock: both, Start: midnight().Add(-time.Hour), Cause: bans.CauseAdmin},
{
Netblock: both,
Start: midnight(),
Expires: midnight().Add(time.Hour),
Cause: bans.CauseLimit,
},
})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 2})
// A new ban drops the ban smallwebwaf made, and only that one.
ledger.BanForLimit(second, midnight(), bans.Notes{})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1})
if ledger.Bans(both)[0].Cause != bans.CauseAdmin {
t.Errorf("%s kept %+v, want the admin's ban", both, ledger.Bans(both))
}
// And the next drops that one.
ledger.BanForLimit(third, midnight(), bans.Notes{})
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1})
}
func TestReasonOfTheBansSmallwebwafMakes(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultRules())
limit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"})
byteLimit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.3/32"),
midnight(), bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"})
attack, _ := ledger.BanForAttack(netip.MustParsePrefix("203.0.113.2/32"), midnight(),
bans.Notes{RuleID: "git-dir", Target: "path"})
for _, tc := range []struct{ got, want string }{
{limit.Reason, "requests per minute over the limit of 1000"},
{byteLimit.Reason, "bytes per hour over the limit of 10737418240"},
{attack.Reason, "matched the rule git-dir"},
} {
if tc.got != tc.want {
t.Errorf("the reason is %q, want %q", tc.got, tc.want)
}
}
}
func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
t.Parallel()
// An hour's ban lifted ten minutes after it started.
netblock := netip.MustParsePrefix("203.0.113.9/32")
lifted := bans.Ban{
Netblock: netblock,
Start: midnight(),
Expires: midnight().Add(time.Hour),
Cause: bans.CauseLimit,
Lifted: midnight().Add(10 * time.Minute),
}
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{lifted})
// While it would still last, it refuses nothing, and a limit broken
// bans for an hour, as a first broken limit does; the lifted ban is
// kept, and counted among the earlier bans.
now := midnight().Add(30 * time.Minute)
_, banned, _ := ledger.Check(netblock.Addr(), now)
if banned {
t.Error("the lifted ban refuses")
}
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
}
held := ledger.Bans(netblock)
if len(held) != 2 || held[0] != lifted {
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
}
}
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
t.Parallel()
// A permanent ban for a clear sign of attack, lifted.
netblock := netip.MustParsePrefix("203.0.113.9/32")
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{{
Netblock: netblock,
Start: midnight(),
Cause: bans.CauseAttack,
Lifted: midnight().Add(time.Hour),
}})
now := midnight().Add(2 * time.Hour)
_, banned, _ := ledger.Find(netblock.Addr(), now)
if banned {
t.Error("the lifted ban refuses")
}
active, permanent := ledger.Count(now)
if active != 0 || permanent != 0 {
t.Errorf("%d bans are active and %d permanent, want none", active, permanent)
}
// The next clear sign of attack bans for seven days, as a first does.
ban, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
if ban.Expires.Sub(ban.Start) != 7*day {
t.Errorf("the next ban for an attack ends at %s, want seven days on", ban.Expires)
}
}
func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultRules())
made, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
bans.Notes{})
atStart := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
Start: midnight(),
}
// The bans read at the start were made before it.
ledger.Load([]bans.Ban{made, atStart})
if got := ledger.Made(bans.CauseAdmin); got != 0 {
t.Fatalf("%d bans made by an admin after the start's, want none", got)
}
// The admin keeps the ban smallwebwaf made, keeps the one read at the
// start, and adds one without a cause: that one alone is made.
kept := made
kept.Cause = bans.CauseAdmin
added := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.3/32"),
Start: midnight(),
}
ledger.LoadEdit([]bans.Ban{kept, atStart, added})
if ledger.Made(bans.CauseAdmin) != 1 || ledger.Made(bans.CauseLimit) != 1 {
t.Errorf("%d bans made by an admin and %d for a limit, want 1 of each",
ledger.Made(bans.CauseAdmin), ledger.Made(bans.CauseLimit))
}
}
func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
t.Parallel()
netblock := netip.MustParsePrefix("203.0.113.0/24")
ledger := bans.New(defaultRules())
// An hour's ban for a broken limit.
ledger.BanForLimit(netblock, midnight(), bans.Notes{})
wantChanged(t, ledger, true)
// A minute later an admin bans the netblock for good, named by an
// address in it: that ban is made, and counts the other among the
// earlier bans.
now := midnight().Add(time.Minute)
want := bans.Ban{
Netblock: netblock,
Start: now,
Cause: bans.CauseAdmin,
Reason: "probes for logins",
Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 1}},
}
got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
"probes for logins")
if got != want {
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
}
wantChanged(t, ledger, true)
if made := ledger.Made(bans.CauseAdmin); made != 1 {
t.Errorf("%d bans made by an admin, want 1", made)
}
// It refuses once the ban for the limit has ended.
ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
if !banned || ban != want {
t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
ban, banned, want)
}
}
func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
t.Parallel()
client := netip.MustParseAddr("203.0.113.9")
own := netip.MustParsePrefix("203.0.113.9/32")
wide := netip.MustParsePrefix("203.0.113.0/24")
other := netip.MustParsePrefix("203.0.113.10/32")
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{
// Ended an hour ago.
{
Netblock: own, Start: midnight().Add(-2 * time.Hour),
Expires: midnight().Add(-time.Hour), Cause: bans.CauseLimit,
},
// Active, on the client's address and on its /24.
{
Netblock: own, Start: midnight(), Expires: midnight().Add(time.Hour),
Cause: bans.CauseLimit,
},
{Netblock: wide, Start: midnight(), Cause: bans.CauseAdmin},
// Another client's.
{Netblock: other, Start: midnight(), Cause: bans.CauseAdmin},
})
now := midnight().Add(time.Minute)
lifted := ledger.Lift(client, now)
if len(lifted) != 2 || lifted[0].Lifted != now || lifted[1].Lifted != now {
t.Errorf("lifted %+v, want the two active bans covering the client", lifted)
}
wantChanged(t, ledger, true)
if _, banned, _ := ledger.Check(client, now); banned {
t.Error("the client is still banned")
}
if _, banned, _ := ledger.Check(other.Addr(), now); !banned {
t.Error("the other client's ban was lifted")
}
// The lifted bans are kept, and the one that had ended is not lifted.
covering := ledger.Covering(client)
if len(covering) != 3 || covering[0].Netblock != wide ||
!covering[1].Lifted.IsZero() || covering[2].Lifted != now {
t.Errorf("the bans covering the client are %+v, want the /24's and both "+
"of its own, the earlier not lifted", covering)
}
// With none active, nothing is lifted or changed.
if lifted = ledger.Lift(client, now); len(lifted) != 0 {
t.Errorf("lifted %+v again", lifted)
}
wantChanged(t, ledger, false)
}
internal/bans/bans.go
+102 -418
View File
@@ -1,13 +1,11 @@
// Package bans is the ban ledger: the bans smallwebwaf makes on the
// netblocks of clients that break a rate limit or a byte limit or show a
// clear sign of attack, and those an admin makes, with their notes, as
// the "Bans" section of SPEC.md describes. The bans are kept in memory,
// and written to bans.json and read from it by the state package.
// netblocks of clients that break a rate limit or show a clear sign of
// attack, with their notes, as the "Bans" section of SPEC.md describes.
// The bans are kept in memory, and written to bans.json and read from it
// by the state package.
package bans
import (
"fmt"
"math"
"net/netip"
"slices"
"strings"
@@ -17,15 +15,13 @@ import (
"github.com/hashicorp/golang-lru/v2/simplelru"
)
// The causes of bans.
// The causes of the bans smallwebwaf makes. A ban an admin adds to
// bans.json may have no cause.
const (
// CauseLimit is a ban smallwebwaf made for a broken limit.
// CauseLimit is a ban for a broken limit.
CauseLimit = "limit"
// CauseAttack is a ban smallwebwaf made for a clear sign of attack.
// CauseAttack is a ban for a clear sign of attack.
CauseAttack = "attack"
// CauseAdmin is a ban an admin made, or one smallwebwaf made that an
// admin keeps. It is never dropped.
CauseAdmin = "admin"
)
// repeatFactor is how many times as long as the netblock's last ban a ban
@@ -50,10 +46,9 @@ type Rules struct {
// AttackBanDuration is how long a first ban for a clear sign of attack
// lasts.
AttackBanDuration time.Duration
// MaxBans is the most bans held whose cause is not CauseAdmin, at
// least one. Past it, the earliest such ban of the netblock that has
// gone longest without a request is dropped. Bans whose cause is
// CauseAdmin are held besides, and never dropped.
// MaxBans is the most bans held, at least one. Past it, the earliest
// ban of the netblock that has gone longest without a request is
// dropped.
MaxBans int
}
@@ -63,16 +58,10 @@ type Ban struct {
Start time.Time
// Expires is when the ban ends, zero for a permanent ban.
Expires time.Time
// Cause is CauseLimit, CauseAttack or CauseAdmin.
// Cause is CauseLimit or CauseAttack, or "" for a ban an admin added
// without one.
Cause string
// Reason is a short text: for a ban smallwebwaf made, the limit broken
// or the rule that matched; for an admin's, what the admin wrote.
Reason string
// Lifted is when an admin lifted the ban, zero while no admin has. A
// lifted ban refuses nothing, and does not make the netblock's next
// ban longer.
Lifted time.Time
Notes Notes
Notes Notes
}
// Permanent reports whether the ban never runs out.
@@ -80,10 +69,9 @@ func (b Ban) Permanent() bool {
return b.Expires.IsZero()
}
// ActiveAt reports whether the ban refuses requests at now: it has not
// been lifted, and has not run out.
// ActiveAt reports whether the ban refuses requests at now.
func (b Ban) ActiveAt(now time.Time) bool {
return b.Lifted.IsZero() && (b.Permanent() || now.Before(b.Expires))
return b.Permanent() || now.Before(b.Expires)
}
// Notes are what an admin needs to decide whether to lift a ban. The
@@ -91,51 +79,30 @@ func (b Ban) ActiveAt(now time.Time) bool {
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
type Notes struct {
// ASN, ASName and Country are the client's AS number, AS name and
// country, when they were looked up: when the request that caused the
// ban was made, or when GeoJS answered about the client afterwards.
ASN string `json:"asn"`
ASName string `json:"as_name"`
// Country is the client's country, when it was looked up.
Country string `json:"country"`
// Kind, Limit, Window and Count are, for a ban for a broken limit,
// what the limit was on, "requests" for a rate limit or "bytes" for a
// byte limit, the limit that was broken, its window, "minute", "hour"
// or "day", and the count reached: the client's requests, or bytes, in
// the window, those of the request that broke the limit included.
// These are what counted toward the ban, and the window is the time
// over which they came.
Kind string `json:"kind,omitempty"`
// Limit, Window and Count are, for a ban for a broken limit, the limit
// that was broken, its window, "minute", "hour" or "day", and the
// count reached: the client's requests in the window, the one that
// broke the limit included. These are the requests that counted
// toward the ban, and the window is the time over which they came.
Limit int64 `json:"limit,omitempty"`
Window string `json:"window,omitempty"`
Count float64 `json:"count,omitempty"`
// LimitPercent and LimitPercentSetting are, for a ban for a limit a
// biased threshold lowered, the client's percentage of that kind of
// limit, of which Limit is the result, and the setting that gave it.
// Both are left out for a limit that was not lowered.
LimitPercent *int64 `json:"limit_percent,omitempty"`
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
// RuleID and Target are, for a ban for a clear sign of attack, the id
// of the rule file rule that matched, and its target.
RuleID string `json:"rule_id,omitempty"`
Target string `json:"target,omitempty"`
// Request is the request that broke the limit, or whose bytes broke
// it, or that was the clear sign of attack.
// Request is the request that broke the limit, or that was the clear
// sign of attack.
Request Request `json:"request"`
// Requests is how many requests the netblock has sent since it was
// first seen, and Refused how many of them the ban has refused so
// far. Both go up with each request the ban refuses.
Requests int64 `json:"requests"`
Refused int64 `json:"refused"`
// EarlierBans is how many bans the netblock had before this one, by
// cause.
EarlierBans EarlierBans `json:"earlier_bans"`
}
// EarlierBans counts a netblock's bans before a ban, by cause.
type EarlierBans struct {
Limit int `json:"limit"`
Attack int `json:"attack"`
Admin int `json:"admin"`
// EarlierBans is how many bans the netblock had before this one.
EarlierBans int `json:"earlier_bans"`
}
// Request is a request in a ban's notes. Each text is cut to 256 bytes.
@@ -163,12 +130,9 @@ type Ledger struct {
// netblocks holds each banned netblock's bans, oldest first. Check and
// Find make each netblock they find the most recently seen.
netblocks *simplelru.LRU[netip.Prefix, *[]Ban]
// held is how many bans netblocks holds whose cause is not CauseAdmin,
// at most rules.MaxBans.
// held is how many bans netblocks holds, at most rules.MaxBans.
held int
// made is how many bans have been made since the start, by cause: by
// the ledger, and by an admin, through BanForAdmin or in an edit of
// bans.json.
// made is how many bans the ledger has made since the start, by cause.
made map[string]int
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
// netblocks that have been banned. Check looks for a ban at each of
@@ -180,10 +144,9 @@ type Ledger struct {
// New returns a Ledger with no ban yet.
func New(rules Rules) *Ledger {
// The ledger drops bans itself, and never those whose cause is
// CauseAdmin, however many there are, so the LRU has no limit of its
// own: it keeps the netblocks in the order they were last seen.
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](math.MaxInt, nil)
// Every netblock held has a ban, so there are never more netblocks
// than rules.MaxBans, and the LRU never drops one itself.
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](rules.MaxBans, nil)
if err != nil {
panic(err) // NewLRU fails only for a size below one
}
@@ -196,9 +159,9 @@ func New(rules Rules) *Ledger {
}
}
// Changed receives a value after a ban is made, lifted or made permanent,
// so that bans.json can be written. Several changes before it is read
// leave one value.
// Changed receives a value after a ban is made or made permanent, so that
// bans.json can be written. Several changes before it is read leave one
// value.
func (l *Ledger) Changed() <-chan struct{} {
return l.changed
}
@@ -207,43 +170,39 @@ func (l *Ledger) Changed() <-chan struct{} {
// a ban on a netblock client is in is active, and returns that ban, with
// the request counted among those it refused. A ban for a clear sign of
// attack is made permanent by the request: the netblock is malicious.
// The last result reports whether the request made the ban permanent.
func (l *Ledger) Check(client netip.Addr, now time.Time) (Ban, bool, bool) {
func (l *Ledger) Check(client netip.Addr, now time.Time) (Ban, bool) {
l.mu.Lock()
defer l.mu.Unlock()
ban := l.active(client, now)
if ban == nil {
return Ban{}, false, false
return Ban{}, false
}
ban.Notes.Requests++
ban.Notes.Refused++
madePermanent := ban.Cause == CauseAttack && !ban.Permanent()
if madePermanent {
if ban.Cause == CauseAttack && !ban.Permanent() {
ban.Expires = time.Time{}
l.markChanged()
}
return *ban, true, madePermanent
return *ban, true
}
// Find is Check without counting the request among those the ban
// refused, and without making the ban permanent: in observe mode a ban
// refuses nothing. The last result reports whether Check would have made
// the ban permanent.
func (l *Ledger) Find(client netip.Addr, now time.Time) (Ban, bool, bool) {
// refused: in observe mode a ban refuses nothing.
func (l *Ledger) Find(client netip.Addr, now time.Time) (Ban, bool) {
l.mu.Lock()
defer l.mu.Unlock()
ban := l.active(client, now)
if ban == nil {
return Ban{}, false, false
return Ban{}, false
}
return *ban, true, ban.Cause == CauseAttack && !ban.Permanent()
return *ban, true
}
// activeBan returns the ban in bans, a netblock's bans oldest first, that
@@ -261,155 +220,23 @@ func activeBan(bans []Ban, now time.Time) *Ban {
}
// BanForLimit bans netblock at now for a broken limit, with notes, and
// returns the ban, and true. A first ban lasts LimitBanDuration. A ban
// made within LimitBanRepeatWindow after the netblock's ban that ended
// last, other than one for a clear sign of attack or a lifted one, lasts
// repeatFactor times as long as that one. A ban that would be longer
// than MaxBanDuration is permanent instead. If a ban on netblock is still
// active, as when two of its requests break a limit at once, that ban is
// returned with false, and no other is made. The ledger fills in the
// notes' Refused and EarlierBans itself, and gives the ban the reason
// "<Kind> per <Window> over the limit of <Limit>", from the notes, such
// as "requests per minute over the limit of 1000".
func (l *Ledger) BanForLimit(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, CauseLimit, limitReason(notes), notes, true)
}
// WouldBanForLimit returns what BanForLimit would, without making the ban:
// what observe mode would have done.
func (l *Ledger) WouldBanForLimit(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, CauseLimit, limitReason(notes), notes, false)
// returns the ban. A first ban lasts LimitBanDuration. A ban made within
// LimitBanRepeatWindow after the netblock's ban that ended last, other
// than one for a clear sign of attack, lasts repeatFactor times as long as
// that one. A ban that would be longer than MaxBanDuration is permanent
// instead. If a ban on netblock is still active, as when two of its
// requests break a limit at once, that ban is returned and no other is
// made. The ledger fills in the notes' Refused and EarlierBans itself.
func (l *Ledger) BanForLimit(netblock netip.Prefix, now time.Time, notes Notes) Ban {
return l.ban(netblock, now, CauseLimit, notes)
}
// BanForAttack bans netblock at now for a clear sign of attack, with
// notes, and returns the ban, and whether it made it, as BanForLimit
// does. A first ban lasts AttackBanDuration; once the netblock has had
// one that was not lifted, the next is permanent. Its reason is "matched
// the rule <RuleID>".
func (l *Ledger) BanForAttack(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, CauseAttack, attackReason(notes), notes, true)
}
// WouldBanForAttack returns what BanForAttack would, without making the
// ban: what observe mode would have done.
func (l *Ledger) WouldBanForAttack(
netblock netip.Prefix, now time.Time, notes Notes,
) (Ban, bool) {
return l.ban(netblock, now, CauseAttack, attackReason(notes), notes, false)
}
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit
// or CauseAttack, made at now would be permanent, as BanForLimit or
// BanForAttack would make it. It works out nothing else of the ban.
func (l *Ledger) WouldBePermanent(
netblock netip.Prefix, now time.Time, cause string,
) bool {
l.mu.Lock()
defer l.mu.Unlock()
var held []Ban
if bans, found := l.netblocks.Peek(netblock); found {
held = *bans
}
if cause == CauseAttack {
return l.attackExpiry(held, now).IsZero()
}
return l.limitExpiry(held, now).IsZero()
}
// limitReason is the reason of a ban for a broken limit, with notes.
func limitReason(notes Notes) string {
return fmt.Sprintf("%s per %s over the limit of %d",
notes.Kind, notes.Window, notes.Limit)
}
// attackReason is the reason of a ban for a clear sign of attack, with
// notes.
func attackReason(notes Notes) string {
return "matched the rule " + notes.RuleID
}
// BanForAdmin bans netblock at now for an admin, with reason, until
// expires, or for good when expires is zero, and returns the ban, whose
// cause is CauseAdmin. Unlike BanForLimit and BanForAttack, it makes the
// ban even while another on netblock is active, since the admin asked
// for this one. The ledger fills in the notes' EarlierBans, and counts
// the ban among those made.
func (l *Ledger) BanForAdmin(
netblock netip.Prefix, now, expires time.Time, reason string,
) Ban {
l.mu.Lock()
defer l.mu.Unlock()
ban := Ban{
Netblock: netblock.Masked(), Start: now, Expires: expires, Cause: CauseAdmin,
Reason: reason,
}
held, found := l.netblocks.Get(ban.Netblock)
if found {
ban.Notes.EarlierBans = earlierBans(*held)
}
l.add(ban)
l.made[CauseAdmin]++
l.markChanged()
return ban
}
// Lift lifts, at now, every ban active then on a netblock client is in,
// as an admin does, and returns those bans. A lifted ban is kept, refuses
// nothing, and does not make the netblock's next ban longer.
func (l *Ledger) Lift(client netip.Addr, now time.Time) []Ban {
l.mu.Lock()
defer l.mu.Unlock()
var lifted []Ban
for _, bans := range l.covering(client) {
for i := range *bans {
ban := &(*bans)[i]
if ban.ActiveAt(now) {
ban.Lifted = now
lifted = append(lifted, *ban)
}
}
}
if len(lifted) > 0 {
l.markChanged()
}
return lifted
}
// Covering returns every ban held on a netblock client is in, active or
// not, sorted by netblock, and each netblock's bans oldest first. It is
// not a request from client, and leaves when the netblocks were last seen
// unchanged.
func (l *Ledger) Covering(client netip.Addr) []Ban {
l.mu.Lock()
defer l.mu.Unlock()
var held []Ban
for _, bans := range l.covering(client) {
held = append(held, *bans...)
}
slices.SortStableFunc(held, func(a, b Ban) int {
return a.Netblock.Compare(b.Netblock)
})
return held
// notes, and returns the ban, as BanForLimit does. A first ban lasts
// AttackBanDuration; once the netblock has had one, the next is
// permanent.
func (l *Ledger) BanForAttack(netblock netip.Prefix, now time.Time, notes Notes) Ban {
return l.ban(netblock, now, CauseAttack, notes)
}
// Bans returns the bans held on netblock, oldest first. It is not a
@@ -426,32 +253,8 @@ func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
return slices.Clone(*bans)
}
// AddLookup gives the notes of netblock's bans that have no AS number, AS
// name or country yet those of a client in it, as the lookup answered
// about it. It is not a request from netblock, and leaves when it was last
// seen unchanged. It does not have bans.json written at once: the notes
// are written with its next write, as the counts in them are.
func (l *Ledger) AddLookup(netblock netip.Prefix, asn, asName, country string) {
l.mu.Lock()
defer l.mu.Unlock()
bans, found := l.netblocks.Peek(netblock)
if !found {
return
}
for i := range *bans {
notes := &(*bans)[i].Notes
if notes.ASN == "" && notes.ASName == "" && notes.Country == "" {
notes.ASN, notes.ASName, notes.Country = asn, asName, country
}
}
}
// Made returns how many bans for cause have been made since the start:
// for CauseLimit and CauseAttack, by the ledger; for CauseAdmin, by an
// admin, with BanForAdmin or in an edit of bans.json, as LoadEdit counts
// them. The bans read from bans.json at the start are not among them.
// Made returns how many bans for cause the ledger has made since the
// start; bans read from bans.json are not among them.
func (l *Ledger) Made(cause string) int {
l.mu.Lock()
defer l.mu.Unlock()
@@ -460,7 +263,7 @@ func (l *Ledger) Made(cause string) int {
}
// Count returns how many of the bans held are active at now, and how many
// of those are permanent. A lifted ban is neither.
// are permanent.
func (l *Ledger) Count(now time.Time) (int, int) {
l.mu.Lock()
defer l.mu.Unlock()
@@ -469,12 +272,10 @@ func (l *Ledger) Count(now time.Time) (int, int) {
for _, bans := range l.netblocks.Values() {
for _, ban := range *bans {
if !ban.ActiveAt(now) {
continue
if ban.ActiveAt(now) {
active++
}
active++
if ban.Permanent() {
permanent++
}
@@ -502,84 +303,37 @@ func (l *Ledger) Snapshot() []Ban {
return held
}
// Load puts bans read from bans.json at the start into the ledger, in
// place of the bans it holds, in the order they started, so that a
// netblock whose last ban started latest counts as the most recently
// seen. A ban without a cause is an admin's, and gets CauseAdmin. Each
// netblock is masked to its length, so that 203.0.113.9/24 is
// 203.0.113.0/24, and each text in the notes is cut to 256 bytes. Past
// MaxBans the earliest bans whose cause is not CauseAdmin are dropped, as
// when they are made.
// Load puts bans read from bans.json into the ledger, in place of the
// bans it holds, in the order they started, so that a netblock whose last
// ban started latest counts as the most recently seen. Each netblock is
// masked to its length, so that 203.0.113.9/24 is 203.0.113.0/24, and
// each text in the notes is cut to 256 bytes. Past MaxBans the earliest
// bans are dropped, as when they are made.
func (l *Ledger) Load(bans []Ban) {
l.mu.Lock()
defer l.mu.Unlock()
l.load(bans)
}
// LoadEdit is Load for an admin's edit of bans.json, taken in while
// smallwebwaf runs. Each ban in it whose cause is CauseAdmin, and which
// the ledger did not hold, with the same netblock and start, is one the
// admin made, and is counted among the bans made.
func (l *Ledger) LoadEdit(bans []Ban) {
l.mu.Lock()
defer l.mu.Unlock()
l.made[CauseAdmin] += l.load(bans)
}
// load does what Load describes, and returns how many of bans are bans
// whose cause is CauseAdmin that the ledger did not hold before.
func (l *Ledger) load(bans []Ban) int {
bans = slices.Clone(bans)
added := 0
for i := range bans {
ban := &bans[i]
ban.Netblock = ban.Netblock.Masked()
ban.Notes.Request = ban.Notes.Request.cut()
if ban.Cause == "" {
ban.Cause = CauseAdmin
}
if ban.Cause == CauseAdmin && !l.holds(ban.Netblock, ban.Start) {
added++
}
}
slices.SortStableFunc(bans, func(a, b Ban) int {
return a.Start.Compare(b.Start)
})
l.mu.Lock()
defer l.mu.Unlock()
l.netblocks.Purge()
l.held = 0
l.v4Lengths, l.v6Lengths = nil, nil
for _, ban := range bans {
ban.Netblock = ban.Netblock.Masked()
ban.Notes.Request = ban.Notes.Request.cut()
l.add(ban)
}
return added
}
// holds reports whether the ledger holds a ban on netblock that started
// at start.
func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
bans, found := l.netblocks.Peek(netblock)
return found && slices.ContainsFunc(*bans, func(ban Ban) bool {
return ban.Start.Equal(start)
})
}
// ban bans netblock at now for cause, with reason and notes, as
// BanForLimit and BanForAttack describe, and returns the ban, and whether
// it made it. Unless keep is true, the ban is not made, only returned: it
// is the ban that would have been made.
// ban bans netblock at now for cause, with notes, as BanForLimit and
// BanForAttack describe, and returns the ban.
func (l *Ledger) ban(
netblock netip.Prefix, now time.Time, cause, reason string, notes Notes, keep bool,
) (Ban, bool) {
netblock netip.Prefix, now time.Time, cause string, notes Notes,
) Ban {
l.mu.Lock()
defer l.mu.Unlock()
@@ -590,15 +344,18 @@ func (l *Ledger) ban(
if found {
active := activeBan(*bans, now)
if active != nil {
return *active, false
return *active
}
held = *bans
notes.EarlierBans = earlierBans(held)
// The netblock's first ban held counts the bans it had before that
// one, since dropped to make room, and each ban held adds one.
notes.EarlierBans = (*bans)[0].Notes.EarlierBans + len(*bans)
}
notes.Request = notes.Request.cut()
ban := Ban{Netblock: netblock, Start: now, Cause: cause, Reason: reason, Notes: notes}
ban := Ban{Netblock: netblock, Start: now, Cause: cause, Notes: notes}
if cause == CauseAttack {
ban.Expires = l.attackExpiry(held, now)
@@ -606,36 +363,11 @@ func (l *Ledger) ban(
ban.Expires = l.limitExpiry(held, now)
}
if !keep {
return ban, true
}
l.add(ban)
l.made[cause]++
l.markChanged()
return ban, true
}
// earlierBans returns how many bans a netblock with the bans held, oldest
// first, has had, by cause: the first ban held counts the bans the
// netblock had before that one, since dropped to make room, and each ban
// held adds one.
func earlierBans(held []Ban) EarlierBans {
earlier := held[0].Notes.EarlierBans
for _, ban := range held {
switch ban.Cause {
case CauseLimit:
earlier.Limit++
case CauseAttack:
earlier.Attack++
case CauseAdmin:
earlier.Admin++
}
}
return earlier
return ban
}
// markChanged has Changed receive a value, unless one is waiting already.
@@ -669,32 +401,10 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
return nil
}
// covering returns the bans of each netblock held that client is in,
// leaving when the netblocks were last seen unchanged.
func (l *Ledger) covering(client netip.Addr) []*[]Ban {
lengths := l.v6Lengths
if client.Is4() {
lengths = l.v4Lengths
}
var found []*[]Ban
for _, length := range lengths {
bans, ok := l.netblocks.Peek(netip.PrefixFrom(client, length).Masked())
if ok {
found = append(found, bans)
}
}
return found
}
// add adds ban to its netblock's bans, after the last, and makes its
// netblock the most recently seen. With MaxBans held, it drops one first,
// unless ban's cause is CauseAdmin, which does not count toward MaxBans.
// netblock the most recently seen. With MaxBans held, it drops one first.
func (l *Ledger) add(ban Ban) {
counted := ban.Cause != CauseAdmin
if counted && l.held == l.rules.MaxBans {
if l.held == l.rules.MaxBans {
l.dropOne()
}
@@ -707,10 +417,7 @@ func (l *Ledger) add(ban Ban) {
}
*bans = append(*bans, ban)
if counted {
l.held++
}
l.held++
lengths := &l.v6Lengths
if ban.Netblock.Addr().Is4() {
@@ -725,17 +432,16 @@ func (l *Ledger) add(ban Ban) {
// limitExpiry returns when a ban for a broken limit made at now ends, or
// zero when it is permanent. held are the netblock's bans, none of them
// active, of which the one that ended last, other than a ban for a clear
// sign of attack or a lifted one, can make the new ban longer. A ban an
// admin adds to bans.json can start after another and end before it, so
// that one is looked for among them all.
// sign of attack, can make the new ban longer. A ban an admin adds to
// bans.json can start after another and end before it, so that one is
// looked for among them all.
func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
length := l.rules.LimitBanDuration
var last *Ban
for i, ban := range held {
if ban.Cause != CauseAttack && ban.Lifted.IsZero() &&
(last == nil || ban.Expires.After(last.Expires)) {
if ban.Cause != CauseAttack && (last == nil || ban.Expires.After(last.Expires)) {
last = &held[i]
}
}
@@ -760,11 +466,11 @@ func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
// attackExpiry returns when a ban for a clear sign of attack made at now
// ends. held are the netblock's bans, none of them active: if one of them
// is for a clear sign of attack too, and was not lifted, the new ban is
// permanent, and its end zero; otherwise it ends AttackBanDuration later.
// is for a clear sign of attack too, the new ban is permanent, and its
// end zero; otherwise it ends AttackBanDuration later.
func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time {
for _, ban := range held {
if ban.Cause == CauseAttack && ban.Lifted.IsZero() {
if ban.Cause == CauseAttack {
return time.Time{}
}
}
@@ -772,39 +478,17 @@ func (l *Ledger) attackExpiry(held []Ban, now time.Time) time.Time {
return now.Add(l.rules.AttackBanDuration)
}
// dropOne drops the earliest ban whose cause is not CauseAdmin of the
// netblock that has gone longest without a request, of those that hold
// such a ban, and the netblock with it if that was its only ban. It is
// called with at least one such ban held. It looks at each netblock once
// at most, and drops nothing when none holds such a ban.
// dropOne drops the earliest ban of the netblock that has gone longest
// without a request, and the netblock with it if that was its only ban.
func (l *Ledger) dropOne() {
for range l.netblocks.Len() {
netblock, bans, _ := l.netblocks.GetOldest()
i := slices.IndexFunc(*bans, func(ban Ban) bool {
return ban.Cause != CauseAdmin
})
if i < 0 {
// Its bans are all an admin's, and never dropped. Get makes
// it the most recently seen, so that the next netblock is
// looked at; when it was seen matters only for dropping a
// ban, and a ban added to it makes it the most recently seen
// anyway.
l.netblocks.Get(netblock)
continue
}
if len(*bans) == 1 {
l.netblocks.Remove(netblock)
} else {
*bans = slices.Delete(*bans, i, i+1)
}
l.held--
return
netblock, bans, _ := l.netblocks.GetOldest()
if len(*bans) == 1 {
l.netblocks.Remove(netblock)
} else {
*bans = slices.Delete(*bans, 0, 1)
}
l.held--
}
// cut returns r with each text cut to maxTextBytes and copied, so that
internal/bans/bans_test.go
+53 -194
View File
@@ -21,12 +21,11 @@ func TestRepeatsTripleUntilPermanent(t *testing.T) {
// Each ban is followed by another as soon as it ends: 1, 3, 9, 27 and
// 81 hours.
for i, hours := range []int{1, 3, 9, 27, 81} {
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
ban := ledger.BanForLimit(netblock, now, bans.Notes{})
length := time.Duration(hours) * time.Hour
if !ban.Expires.Equal(now.Add(length)) ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: i}) {
t.Fatalf("ban %d lasts %s with earlier bans %+v, want %d hours and %d for a limit",
if !ban.Expires.Equal(now.Add(length)) || ban.Notes.EarlierBans != i {
t.Fatalf("ban %d lasts %s with %d earlier bans, want %d hours and %d",
i+1, ban.Expires.Sub(now), ban.Notes.EarlierBans, hours, i)
}
@@ -35,12 +34,12 @@ func TestRepeatsTripleUntilPermanent(t *testing.T) {
// The sixth would last 243 hours, more than seven days: it is
// permanent, and never ends.
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
ban := ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Permanent() {
t.Fatalf("sixth ban ends at %s, want a permanent one", ban.Expires)
}
_, banned, _ := ledger.Check(netblock.Addr(), now.Add(100*365*day))
_, banned := ledger.Check(netblock.Addr(), now.Add(100*365*day))
if !banned {
t.Error("a permanent ban ended")
}
@@ -64,12 +63,11 @@ func TestRepeatWindowRunsOut(t *testing.T) {
ledger := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
second, _ := ledger.BanForLimit(netblock, first.Expires.Add(tc.gap), bans.Notes{})
first := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
second := ledger.BanForLimit(netblock, first.Expires.Add(tc.gap), bans.Notes{})
if second.Expires.Sub(second.Start) != tc.want ||
second.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("second ban lasts %s with earlier bans %+v, want %s and 1 for a limit",
if second.Expires.Sub(second.Start) != tc.want || second.Notes.EarlierBans != 1 {
t.Errorf("second ban lasts %s with %d earlier bans, want %s and 1",
second.Expires.Sub(second.Start), second.Notes.EarlierBans, tc.want)
}
})
@@ -83,7 +81,7 @@ func TestFirstBanLongerThanTheMaximumIsPermanent(t *testing.T) {
rules.LimitBanDuration = rules.MaxBanDuration + time.Hour
ledger := bans.New(rules)
ban, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(),
ban := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(),
bans.Notes{})
if !ban.Permanent() {
t.Errorf("first ban ends at %s, want a permanent one", ban.Expires)
@@ -103,7 +101,7 @@ func TestLongestBanSetFarOffDoesNotOverflow(t *testing.T) {
now := midnight()
for i := range 14 {
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
ban := ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Expires.After(ban.Start) {
t.Fatalf("ban %d starts at %s and ends at %s", i+1, ban.Start, ban.Expires)
}
@@ -111,7 +109,7 @@ func TestLongestBanSetFarOffDoesNotOverflow(t *testing.T) {
now = ban.Expires
}
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
ban := ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Permanent() {
t.Errorf("15th ban ends at %s, want a permanent one", ban.Expires)
}
@@ -123,22 +121,12 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
ledger := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
first, made := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
if !made {
t.Error("the first ban was not made")
}
first := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
again := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
if made || again != first || len(ledger.Bans(netblock)) != 1 {
t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+
"want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first)
}
again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{})
if made || again != first {
t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made",
again, made, first)
if again != first || len(ledger.Bans(netblock)) != 1 {
t.Errorf("a limit broken during a ban gave %+v and %d bans, want %+v and 1",
again, len(ledger.Bans(netblock)), first)
}
}
@@ -147,21 +135,21 @@ func TestCheckRefusesWhileTheBanLastsAndCountsTheRefusals(t *testing.T) {
ledger := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
ban := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
for range 3 {
got, banned, _ := ledger.Check(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
got, banned := ledger.Check(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
if !banned || got.Start != ban.Start {
t.Fatalf("check during the ban gives %+v and %t", got, banned)
}
}
_, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.10"), midnight())
_, banned := ledger.Check(netip.MustParseAddr("203.0.113.10"), midnight())
if banned {
t.Error("another netblock is banned")
}
_, banned, _ = ledger.Check(netblock.Addr(), ban.Expires)
_, banned = ledger.Check(netblock.Addr(), ban.Expires)
if banned {
t.Error("the ban did not end")
}
@@ -179,14 +167,14 @@ func TestFindCountsNothing(t *testing.T) {
ledger := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
ban := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
got, banned := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
if !banned || got != ban {
t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban)
}
_, banned, _ = ledger.Find(netblock.Addr(), ban.Expires)
_, banned = ledger.Find(netblock.Addr(), ban.Expires)
if banned {
t.Error("the ban did not end")
}
@@ -208,7 +196,7 @@ func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(t *testing.T) {
d := netip.MustParsePrefix("2001:db8::/64")
now := midnight()
first, _ := ledger.BanForLimit(a, now, bans.Notes{})
first := ledger.BanForLimit(a, now, bans.Notes{})
ledger.BanForLimit(b, now, bans.Notes{})
ledger.BanForLimit(c, now, bans.Notes{})
@@ -243,14 +231,13 @@ func TestFullLedgerDropsTheEarlierBanOfTheNetblockBannedAgain(t *testing.T) {
ledger := bans.New(rules)
netblock := netip.MustParsePrefix("203.0.113.9/32")
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
first := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
second := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
held := ledger.Bans(netblock)
if len(held) != 1 || held[0] != second ||
held[0].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("the ledger holds %+v, want only the second ban, "+
"with 1 earlier ban for a limit", held)
if len(held) != 1 || held[0] != second || held[0].Notes.EarlierBans != 1 {
t.Errorf("the ledger holds %+v, want only the second ban, with 1 earlier ban",
held)
}
}
@@ -261,7 +248,7 @@ func TestRequestDuringAnAttackBanMakesItPermanent(t *testing.T) {
netblock := netip.MustParsePrefix("203.0.113.9/32")
notes := bans.Notes{RuleID: "env-file", Target: "path"}
ban, _ := ledger.BanForAttack(netblock, midnight(), notes)
ban := ledger.BanForAttack(netblock, midnight(), notes)
if !ban.Expires.Equal(midnight().Add(7*day)) || ban.Cause != bans.CauseAttack ||
ban.Notes.RuleID != "env-file" || ledger.Made(bans.CauseAttack) != 1 ||
ledger.Made(bans.CauseLimit) != 0 {
@@ -272,31 +259,23 @@ func TestRequestDuringAnAttackBanMakesItPermanent(t *testing.T) {
wantChanged(t, ledger, true)
// In observe mode the ban refuses nothing, and stays as it is, while
// Find tells that the request would have made it permanent.
got, _, wouldMakePermanent := ledger.Find(netblock.Addr(), midnight().Add(time.Hour))
if got.Permanent() || ledger.Bans(netblock)[0].Permanent() || !wouldMakePermanent {
t.Fatalf("a request found under the ban left it %+v, would have made it "+
"permanent %t, want it as it was, and true", got, wouldMakePermanent)
// In observe mode the ban refuses nothing, and stays as it is.
got, _ := ledger.Find(netblock.Addr(), midnight().Add(time.Hour))
if got.Permanent() {
t.Fatal("a request found under the ban made it permanent")
}
wantChanged(t, ledger, false)
// A request it refuses makes it permanent, says so, and makes
// bans.json due.
got, _, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(time.Hour))
if !madePermanent || !got.Permanent() || !ledger.Bans(netblock)[0].Permanent() {
t.Fatalf("after a request during the ban, it is %+v, made permanent %t, "+
"want it made permanent", got, madePermanent)
// A request it refuses makes it permanent, and bans.json due.
got, _ = ledger.Check(netblock.Addr(), midnight().Add(time.Hour))
if !got.Permanent() || !ledger.Bans(netblock)[0].Permanent() {
t.Fatalf("after a request during the ban, it is %+v, want it permanent", got)
}
wantChanged(t, ledger, true)
// The next request finds it permanent already.
_, banned, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(100*365*day))
if !banned || madePermanent {
t.Errorf("a later request is banned %t, and made the ban permanent %t, "+
"want banned by the permanent ban", banned, madePermanent)
_, banned := ledger.Check(netblock.Addr(), midnight().Add(100*365*day))
if !banned {
t.Error("the permanent ban ended")
}
}
@@ -307,8 +286,8 @@ func TestAttackAfterAnAttackBanHasEndedBansPermanently(t *testing.T) {
netblock := netip.MustParsePrefix("203.0.113.9/32")
// A ban for a broken limit before does not count.
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
second, _ := ledger.BanForAttack(netblock, first.Expires, bans.Notes{})
first := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
second := ledger.BanForAttack(netblock, first.Expires, bans.Notes{})
if second.Expires.Sub(second.Start) != 7*day {
t.Fatalf("the first ban for an attack lasts %s, want 7 days",
@@ -317,96 +296,15 @@ func TestAttackAfterAnAttackBanHasEndedBansPermanently(t *testing.T) {
// Once that has run out without a request, the netblock is served, and
// its next clear sign of attack bans it for good.
_, banned, _ := ledger.Check(netblock.Addr(), second.Expires)
_, banned := ledger.Check(netblock.Addr(), second.Expires)
if banned {
t.Fatal("the ban did not end")
}
// Its notes show the earlier ban for an attack that makes it permanent,
// beside the one for a limit.
third, _ := ledger.BanForAttack(netblock, second.Expires.Add(30*day), bans.Notes{})
if !third.Permanent() ||
third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Attack: 1}) {
third := ledger.BanForAttack(netblock, second.Expires.Add(30*day), bans.Notes{})
if !third.Permanent() || third.Notes.EarlierBans != 2 {
t.Errorf("the next ban for an attack is %+v, want a permanent one, "+
"with 1 earlier ban for a limit and 1 for an attack", third)
}
}
func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
wantChanged(t, ledger, true)
// While the first ban lasts, none would be made.
during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{})
if would || during != first {
t.Errorf("during the first ban, would ban %t with %+v, want false with %+v",
would, during, first)
}
// As it ends, a clear sign of attack would ban for seven days, and a
// limit broken again for three hours, but neither is made.
limitNotes := bans.Notes{Kind: "requests", Limit: 1, Window: "minute"}
attack, wouldAttack := ledger.WouldBanForAttack(netblock, first.Expires,
bans.Notes{RuleID: "git-dir"})
limit, wouldLimit := ledger.WouldBanForLimit(netblock, first.Expires, limitNotes)
if !wouldAttack || !attack.Expires.Equal(first.Expires.Add(7*day)) ||
attack.Reason != "matched the rule git-dir" || !wouldLimit ||
!limit.Expires.Equal(first.Expires.Add(3*time.Hour)) ||
limit.Reason != "requests per minute over the limit of 1" {
t.Errorf("would ban with %+v and %+v, want seven days for the attack and "+
"three hours for the limit", attack, limit)
}
if len(ledger.Bans(netblock)) != 1 || ledger.Made(bans.CauseLimit) != 1 ||
ledger.Made(bans.CauseAttack) != 0 {
t.Errorf("the ledger holds %+v, want the first ban alone", ledger.Bans(netblock))
}
wantChanged(t, ledger, false)
// The ban made is the one that would have been.
made, _ := ledger.BanForLimit(netblock, first.Expires, limitNotes)
if made != limit {
t.Errorf("the ban made is %+v, want %+v", made, limit)
}
}
func TestWouldBePermanentAnswersAsTheBanWouldBeMade(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
now := midnight()
// Five bans for a limit in a row, of 1, 3, 9, 27 and 81 hours, are not
// permanent. The sixth, of 243 hours, would be, while a first ban for
// an attack would not.
for i := range 5 {
if ledger.WouldBePermanent(netblock, now, bans.CauseLimit) {
t.Fatalf("ban %d for a limit would be permanent", i+1)
}
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
now = ban.Expires
}
if !ledger.WouldBePermanent(netblock, now, bans.CauseLimit) {
t.Error("the sixth ban for a limit would not be permanent")
}
if ledger.WouldBePermanent(netblock, now, bans.CauseAttack) {
t.Error("a first ban for an attack would be permanent")
}
// Once a first ban for an attack has ended, the next would be permanent.
attack, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
if !ledger.WouldBePermanent(netblock, attack.Expires, bans.CauseAttack) {
t.Error("a second ban for an attack would not be permanent")
"with 2 earlier bans", third)
}
}
@@ -418,16 +316,16 @@ func TestAttackBanDoesNotLengthenTheNextBanForALimit(t *testing.T) {
// Three times the seven days would be permanent; a limit broken as the
// ban for an attack ends bans for an hour, as a first broken limit does.
attack, _ := ledger.BanForAttack(netblock, midnight(), bans.Notes{})
limit, _ := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{})
attack := ledger.BanForAttack(netblock, midnight(), bans.Notes{})
limit := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{})
if limit.Expires.Sub(limit.Start) != time.Hour || limit.Cause != bans.CauseLimit {
t.Errorf("the ban for a limit is %+v, want one of an hour", limit)
}
// And a request during the ban for a limit leaves it as it is.
got, _, madePermanent := ledger.Check(netblock.Addr(), limit.Start)
if got.Permanent() || madePermanent {
got, _ := ledger.Check(netblock.Addr(), limit.Start)
if got.Permanent() {
t.Error("a request during a ban for a limit made it permanent")
}
}
@@ -442,7 +340,7 @@ func TestRequestTextsAreCutTo256Bytes(t *testing.T) {
Time: midnight(), Method: long, Host: long, Path: long, Status: 403, UserAgent: long,
}
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Request: request})
ban := ledger.BanForLimit(netblock, midnight(), bans.Notes{Request: request})
cut := long[:256]
want := bans.Request{
@@ -454,45 +352,6 @@ func TestRequestTextsAreCutTo256Bytes(t *testing.T) {
}
}
func TestLookupFillsTheNotesOfTheNetblocksBansWithoutOne(t *testing.T) {
t.Parallel()
ledger := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
other := netip.MustParsePrefix("198.51.100.7/32")
// A ban made with the client's lookup, one made before it came, after
// the first ended, and one on another netblock.
ledger.BanForLimit(netblock, midnight(), bans.Notes{
ASN: "AS64497", ASName: "Other Net", Country: "FR",
})
ledger.BanForLimit(netblock, midnight().Add(time.Hour), bans.Notes{})
ledger.BanForLimit(other, midnight(), bans.Notes{})
ledger.AddLookup(netblock, "AS64496", "Example Net", "DE")
held := ledger.Bans(netblock)
if len(held) != 2 {
t.Fatalf("%s has %d bans, want 2", netblock, len(held))
}
for i, want := range []bans.Notes{
{ASN: "AS64497", ASName: "Other Net", Country: "FR"},
{ASN: "AS64496", ASName: "Example Net", Country: "DE"},
} {
got := held[i].Notes
if got.ASN != want.ASN || got.ASName != want.ASName || got.Country != want.Country {
t.Errorf("ban %d's notes give %q, %q and %q, want %q, %q and %q", i+1,
got.ASN, got.ASName, got.Country, want.ASN, want.ASName, want.Country)
}
}
if notes := ledger.Bans(other)[0].Notes; notes.ASN != "" || notes.Country != "" {
t.Errorf("the ban on %s has %q and %q, want neither",
other, notes.ASN, notes.Country)
}
}
// defaultRules are the rules at the settings' defaults.
func defaultRules() bans.Rules {
return bans.Rules{
internal/bans/snapshot_test.go
+22 -39
View File
@@ -42,7 +42,7 @@ func TestSnapshotListsEveryBanByNetblock(t *testing.T) {
high := netip.MustParsePrefix("203.0.113.10/32")
low := netip.MustParsePrefix("203.0.113.9/32")
first, _ := ledger.BanForLimit(v6, midnight(), bans.Notes{})
first := ledger.BanForLimit(v6, midnight(), bans.Notes{})
ledger.BanForLimit(high, midnight(), bans.Notes{})
ledger.BanForLimit(low, midnight(), bans.Notes{})
ledger.BanForLimit(v6, first.Expires, bans.Notes{})
@@ -68,7 +68,7 @@ func TestLoadedBansCarryOn(t *testing.T) {
before := bans.New(defaultRules())
netblock := netip.MustParsePrefix("203.0.113.9/32")
ban, _ := before.BanForLimit(netblock, midnight(), bans.Notes{Limit: 1})
ban := before.BanForLimit(netblock, midnight(), bans.Notes{Limit: 1})
// Loaded into a new ledger, as across a restart, the ban still refuses
// while it lasts, and once it has ended a broken limit bans for three
@@ -76,15 +76,14 @@ func TestLoadedBansCarryOn(t *testing.T) {
after := bans.New(defaultRules())
after.Load(before.Snapshot())
_, banned, _ := after.Check(netblock.Addr(), ban.Expires.Add(-time.Second))
_, banned := after.Check(netblock.Addr(), ban.Expires.Add(-time.Second))
if !banned {
t.Error("the loaded ban does not refuse")
}
again, _ := after.BanForLimit(netblock, ban.Expires, bans.Notes{})
if again.Expires.Sub(again.Start) != 3*time.Hour ||
again.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, want 3h and 1 for a limit",
again := after.BanForLimit(netblock, ban.Expires, bans.Notes{})
if again.Expires.Sub(again.Start) != 3*time.Hour || again.Notes.EarlierBans != 1 {
t.Errorf("the next ban lasts %s with %d earlier bans, want 3h and 1",
again.Expires.Sub(again.Start), again.Notes.EarlierBans)
}
}
@@ -111,7 +110,7 @@ func TestLoadedBanRefusesEveryClientInItsNetblock(t *testing.T) {
"198.51.100.7": true,
"198.51.100.8": false,
} {
_, banned, _ := ledger.Check(netip.MustParseAddr(client), midnight())
_, banned := ledger.Check(netip.MustParseAddr(client), midnight())
if banned != want {
t.Errorf("%s is refused: %t, want %t", client, banned, want)
}
@@ -150,19 +149,19 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
now := midnight().Add(2 * time.Hour)
client := netip.MustParseAddr("203.0.113.9")
ban, banned, _ := ledger.Find(client, now)
ban, banned := ledger.Find(client, now)
if !banned || !ban.Permanent() {
t.Errorf("find gives %+v and %t, want the permanent ban", ban, banned)
}
ban, banned, _ = ledger.Check(client, now)
ban, banned = ledger.Check(client, now)
if !banned || !ban.Permanent() {
t.Errorf("the client is refused: %t, under %+v, want under the permanent ban",
banned, ban)
}
// A limit broken now makes no shorter ban over the permanent one.
ban, _ = ledger.BanForLimit(netblock, now, bans.Notes{})
ban = ledger.BanForLimit(netblock, now, bans.Notes{})
if !ban.Permanent() || len(ledger.Bans(netblock)) != 2 {
t.Errorf("a broken limit returned %+v and left the netblock %d bans, "+
"want the permanent ban and 2", ban, len(ledger.Bans(netblock)))
@@ -173,14 +172,13 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
t.Parallel()
// A 9-hour ban smallwebwaf made, the third in a row, and an admin's
// 1-hour ban added to bans.json over it, with no cause and no notes.
// 1-hour ban added to bans.json over it, with no notes.
netblock := netip.MustParsePrefix("203.0.113.9/32")
nineHours := bans.Ban{
Netblock: netblock,
Start: midnight(),
Expires: midnight().Add(9 * time.Hour),
Cause: bans.CauseLimit,
Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 2}},
Notes: bans.Notes{EarlierBans: 2},
}
admins := bans.Ban{
Netblock: netblock,
@@ -193,12 +191,10 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
// Once both have ended, a limit broken within the repeat window bans
// for three times the 9 hours, and the notes count the two bans
// before the 9-hour one and it, for a limit, and the admin's.
ban, _ := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
t.Errorf("the next ban lasts %s with earlier bans %+v, "+
"want 27h, 3 for a limit and 1 an admin's",
// before the 9-hour one, it, and the admin's.
ban := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
if ban.Expires.Sub(ban.Start) != 27*time.Hour || ban.Notes.EarlierBans != 4 {
t.Errorf("the next ban lasts %s with %d earlier bans, want 27h and 4",
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
}
}
@@ -207,15 +203,10 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
t.Parallel()
// bans.json lists the bans by netblock, not in the order they began.
later := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.1/32"),
Start: midnight(),
Cause: bans.CauseLimit,
}
later := bans.Ban{Netblock: netip.MustParsePrefix("203.0.113.1/32"), Start: midnight()}
earlier := bans.Ban{
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
Start: midnight().Add(-time.Hour),
Cause: bans.CauseLimit,
}
rules := defaultRules()
@@ -237,17 +228,9 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
rules := defaultRules()
rules.MaxBans = 3
ledger := bans.New(rules)
kept := bans.Ban{
Netblock: netip.MustParsePrefix("2001:db8::/64"),
Start: midnight(),
Cause: bans.CauseLimit,
}
kept := bans.Ban{Netblock: netip.MustParsePrefix("2001:db8::/64"), Start: midnight()}
ledger.Load([]bans.Ban{
{
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
Start: midnight(),
Cause: bans.CauseLimit,
},
{Netblock: netip.MustParsePrefix("203.0.113.0/24"), Start: midnight()},
kept,
})
@@ -255,15 +238,15 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
// bans.json is taken in, that ban is lifted.
ledger.Load([]bans.Ban{kept})
_, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.9"), midnight())
_, banned := ledger.Check(netip.MustParseAddr("203.0.113.9"), midnight())
if banned {
t.Error("a ban left out of the second load still refuses")
}
// The ledger holds one ban, so it makes two more without dropping any.
first, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(),
first := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(),
bans.Notes{})
second, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.8/32"), midnight(),
second := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.8/32"), midnight(),
bans.Notes{})
want := []bans.Ban{first, second, kept}
internal/config/config.go
+31 -946
View File
File diff suppressed because it is too large Load Diff
internal/config/config_test.go
+30 -1261
View File
File diff suppressed because it is too large Load Diff
internal/lookup/file.go
-233
View File
@@ -1,233 +0,0 @@
package lookup
import (
"context"
"fmt"
"log/slog"
"net/netip"
"os"
"path/filepath"
"sync"
"time"
"github.com/fsnotify/fsnotify"
"github.com/oschwald/maxminddb-golang/v2"
"sneak.berlin/go/smallwebwaf/internal/alerts"
)
// quietTime is how long the lookup database must go without a change
// before it is read again, so that a file still being copied in is read
// only once whole.
const quietTime = 2 * time.Second
// FileParams are what OpenFile needs.
type FileParams struct {
// Path is the lookup database, the IPinfo Lite file in its .mmdb form
// (SWWAF_LOOKUP_DB_PATH).
Path string
// Now tells the time, normally time.Now.
Now func() time.Time
// ProcessLog receives each reading of the file, and why a replacement
// of it cannot be read.
ProcessLog *slog.Logger
// Alerts receive a file_error alert for each replacement that cannot
// be read.
Alerts *alerts.Queue
}
// File looks up clients' AS numbers and countries in the lookup database,
// held in memory, and reads it again when it is replaced. It is safe for
// concurrent use.
type File struct {
params FileParams
mu sync.Mutex
// reader is the database in use, and lastRead when it was read.
// readFailures are the replacements that could not be read.
reader *maxminddb.Reader
lastRead time.Time
readFailures int
}
// record is what the lookup database holds about a network, of the fields
// smallwebwaf reads.
type record struct {
ASN string `maxminddb:"asn"`
ASName string `maxminddb:"as_name"`
CountryCode string `maxminddb:"country_code"`
}
// OpenFile reads the lookup database. A file that cannot be read, or that
// is not a .mmdb file, is an error.
func OpenFile(params FileParams) (*File, error) {
reader, err := read(params.Path)
if err != nil {
return nil, err
}
f := &File{params: params}
f.use(reader)
return f, nil
}
// LookUp returns what the lookup database says about client: its AS
// number, such as AS64496, the AS's name, and its country, such as DE,
// each "" when the database does not give it, as for an address missing
// from it. The database is asked about the client's first address, as
// GeoJS is.
func (f *File) LookUp(client netip.Prefix) Answer {
f.mu.Lock()
reader := f.reader
f.mu.Unlock()
var found record
// A record that cannot be decoded places the client nowhere, as a
// missing one does.
err := reader.Lookup(client.Addr()).Decode(&found)
if err != nil {
found = record{}
}
return Answer{
Client: client,
ASN: found.ASN,
ASName: found.ASName,
Country: found.CountryCode,
Answered: f.params.Now(),
}
}
// LastRead returns when the lookup database in use was read.
func (f *File) LastRead() time.Time {
f.mu.Lock()
defer f.mu.Unlock()
return f.lastRead
}
// ReadFailures returns how many replacements of the lookup database could
// not be read.
func (f *File) ReadFailures() int {
f.mu.Lock()
defer f.mu.Unlock()
return f.readFailures
}
// Watch watches the directory of the lookup database until ctx is done,
// and reads the file again once it has gone without a change for
// quietTime, after it is replaced, written or removed, and after Watch
// starts watching. If the directory cannot be watched, that is logged, and
// the database read at start stays in use.
func (f *File) Watch(ctx context.Context) {
watcher, err := fsnotify.NewWatcher()
if err == nil {
defer func() {
_ = watcher.Close()
}()
err = watcher.Add(filepath.Dir(f.params.Path))
}
if err != nil {
f.params.ProcessLog.Error("cannot watch the lookup database for replacements",
"error", err.Error())
return
}
f.params.ProcessLog.Info("watching the lookup database for replacements",
"file", f.params.Path)
f.readAfterChanges(ctx, watcher.Events, watcher.Errors)
}
// readAfterChanges reads the lookup database again once quietTime has
// passed without a change to it from events, until ctx is done, and logs
// the errors from errs. A change to another file in its directory does not
// count. The wait starts at once, as if for a change, so that a file
// replaced after OpenFile read it, and before its directory was watched,
// is read too.
func (f *File) readAfterChanges(
ctx context.Context, events <-chan fsnotify.Event, errs <-chan error,
) {
path := filepath.Clean(f.params.Path)
quiet := time.NewTimer(quietTime)
defer quiet.Stop()
for {
select {
case <-ctx.Done():
return
case event := <-events:
if filepath.Clean(event.Name) == path {
quiet.Reset(quietTime)
}
case <-quiet.C:
f.readAgain()
case err := <-errs:
f.params.ProcessLog.Warn("watching the lookup database failed",
"error", err.Error())
}
}
}
// readAgain reads the lookup database again, in place of the one in use,
// or, if it cannot be read, counts that, raises a file_error alert for it
// and logs it, and the one in use stays in use.
func (f *File) readAgain() {
reader, err := read(f.params.Path)
if err != nil {
const kept = "the lookup database cannot be read, " +
"and the one read before stays in use"
f.mu.Lock()
f.readFailures++
f.mu.Unlock()
// Raised before it is logged, so that the alert is there once the
// log line is.
f.params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError,
Reason: kept,
Detail: map[string]any{"file": f.params.Path, "error": err.Error()},
})
f.params.ProcessLog.Error(kept, "error", err.Error())
return
}
f.use(reader)
}
// use puts reader in use, in place of the database read before, and logs
// that the file was read.
func (f *File) use(reader *maxminddb.Reader) {
f.mu.Lock()
f.reader = reader
f.lastRead = f.params.Now()
f.mu.Unlock()
f.params.ProcessLog.Info("read the lookup database", "file", f.params.Path)
}
// read reads the lookup database at path. The whole file is read into
// memory, rather than mapped into it as the reader can, so that a file
// overwritten in place cannot change, or end, under a lookup.
func read(path string) (*maxminddb.Reader, error) {
data, err := os.ReadFile(path) //nolint:gosec // the file the admin names
if err != nil {
return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH cannot be read: %w", err)
}
reader, err := maxminddb.OpenBytes(data)
if err != nil {
return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w", path, err)
}
return reader, nil
}
internal/lookup/file_internal_test.go
-379
View File
@@ -1,379 +0,0 @@
package lookup
import (
"context"
"log/slog"
"net/netip"
"net/url"
"os"
"path/filepath"
"reflect"
"testing"
"testing/synctest"
"time"
"github.com/fsnotify/fsnotify"
"github.com/maxmind/mmdbwriter/mmdbtype"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
)
// testNetblock is the netblock the tests' lookup databases place, and
// testClient a client in it.
const (
testNetblock = "203.0.113.0/24"
testClient = "203.0.113.9/32"
)
func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(t *testing.T) {
t.Parallel()
germany := lookuptest.Network{ASN: "AS64496", ASName: "Example Net", Country: "DE"}
northKorea := lookuptest.Network{ASN: "AS64511", ASName: "Other Net", Country: "KP"}
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
lookuptest.Write(t, path, map[string]lookuptest.Network{
testNetblock: germany,
"2001:db8::/32": northKorea,
})
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
f, err := OpenFile(FileParams{
Path: path,
Now: func() time.Time { return now },
ProcessLog: slog.New(slog.DiscardHandler),
Alerts: newQueue(),
})
if err != nil {
t.Fatalf("open %s: %v", path, err)
}
for client, want := range map[string]lookuptest.Network{
testClient: germany,
// An IPv6 client is its /64.
"2001:db8:1:2::/64": northKorea,
"198.51.100.7/32": {},
} {
prefix := netip.MustParsePrefix(client)
got := f.LookUp(prefix)
if got != (Answer{
Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country,
Answered: now,
}) {
t.Errorf("%s has the answer %+v, want %+v, answered %s", client, got, want, now)
}
}
}
func TestRecordThatCannotBeReadPlacesTheClientNowhere(t *testing.T) {
t.Parallel()
// The AS number is a number, where a string belongs. The writer writes
// a record's fields in the order of their names, so as_name is read
// before the AS number fails.
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
lookuptest.WriteRecords(t, path, map[string]mmdbtype.Map{
testNetblock: {
"asn": mmdbtype.Uint32(64496),
"as_name": mmdbtype.String("Example Net"),
"country_code": mmdbtype.String("DE"),
},
})
f := openFile(t, path, newQueue())
answer := f.LookUp(netip.MustParsePrefix(testClient))
if answer.ASN != "" || answer.ASName != "" || answer.Country != "" {
t.Errorf("%s is placed %+v, want nowhere", testClient, answer)
}
}
func TestFileThatCannotBeReadIsAnError(t *testing.T) {
t.Parallel()
dir := t.TempDir()
missing := filepath.Join(dir, "missing.mmdb")
notDatabase := filepath.Join(dir, "not.mmdb")
writeFile(t, notDatabase, "not a lookup database\n")
for path, want := range map[string]string{
missing: "SWWAF_LOOKUP_DB_PATH cannot be read: open " + missing +
": no such file or directory",
notDatabase: "SWWAF_LOOKUP_DB_PATH " + notDatabase +
" is not a .mmdb file: error opening database: invalid MaxMind DB file",
} {
_, err := OpenFile(FileParams{
Path: path,
Now: time.Now,
ProcessLog: slog.New(slog.DiscardHandler),
Alerts: newQueue(),
})
if err == nil || err.Error() != want {
t.Errorf("opening %s failed with %v, want %s", path, err, want)
}
}
}
// The tests below run readAfterChanges in a synctest bubble, where time is
// a clock of the test's own: time.Sleep moves it on at once, and
// synctest.Wait returns once readAfterChanges waits again, so that every
// reading due by then is done. The test sends the changes itself, as the
// watch of a directory cannot run in a bubble.
func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "ipinfo_lite.mmdb")
writeDatabase(t, path, "DE")
queue := newQueue()
f := openFile(t, path, queue)
changes := watch(t, f)
other := filepath.Join(dir, "replacement.mmdb")
writeDatabase(t, other, "KP")
replacement, err := os.ReadFile(other) //nolint:gosec // a file the test wrote
if err != nil {
t.Fatalf("read %s: %v", other, err)
}
// The file in use is overwritten in place, and keeps giving what
// it gave. Its first part alone is not a .mmdb file.
file, err := os.Create(path) //nolint:gosec // a file the test wrote
if err != nil {
t.Fatalf("create %s: %v", path, err)
}
defer func() {
_ = file.Close()
}()
half := len(replacement) / 2
write(t, file, replacement[:half])
changes <- fsnotify.Event{Name: path, Op: fsnotify.Write}
time.Sleep(quietTime - time.Nanosecond)
synctest.Wait()
wantCountry(t, f, "DE")
// The second part starts the wait again.
write(t, file, replacement[half:])
changes <- fsnotify.Event{Name: path, Op: fsnotify.Write}
time.Sleep(quietTime - time.Nanosecond)
synctest.Wait()
wantCountry(t, f, "DE")
time.Sleep(time.Nanosecond)
synctest.Wait()
wantCountry(t, f, "KP")
if !f.LastRead().Equal(time.Now()) || f.ReadFailures() != 0 {
t.Errorf("read at %s, with %d failures; want read now, with none",
f.LastRead(), f.ReadFailures())
}
wantAlerts(t, queue)
})
}
func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
writeDatabase(t, path, "DE")
queue := newQueue()
f := openFile(t, path, queue)
read := f.LastRead()
changes := watch(t, f)
writeFile(t, path, "not a lookup database\n")
changes <- fsnotify.Event{Name: path, Op: fsnotify.Write}
// Long after, the replacement has been read once.
time.Sleep(time.Hour)
synctest.Wait()
wantCountry(t, f, "DE")
if !f.LastRead().Equal(read) || f.ReadFailures() != 1 {
t.Errorf("read at %s, with %d failures; want read at %s, with one",
f.LastRead(), f.ReadFailures(), read)
}
wantAlerts(t, queue, alerts.Alert{
Time: read.Add(quietTime),
Event: alerts.EventFileError,
Reason: "the lookup database cannot be read, and the one read before stays in use",
Detail: map[string]any{
"file": path,
"error": "SWWAF_LOOKUP_DB_PATH " + path + " is not a .mmdb file: " +
"error opening database: invalid MaxMind DB file",
},
})
})
}
func TestChangeOfAnotherFileInTheDirectoryIsNoReplacement(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "ipinfo_lite.mmdb")
writeDatabase(t, path, "DE")
f := openFile(t, path, newQueue())
changes := watch(t, f)
// The wait that starts with the watch ends with a reading.
time.Sleep(quietTime)
synctest.Wait()
writeDatabase(t, path, "KP")
changes <- fsnotify.Event{Name: filepath.Join(dir, "other.mmdb"), Op: fsnotify.Create}
time.Sleep(quietTime)
synctest.Wait()
wantCountry(t, f, "DE")
changes <- fsnotify.Event{Name: path, Op: fsnotify.Write}
time.Sleep(quietTime)
synctest.Wait()
wantCountry(t, f, "KP")
})
}
func TestReplacementSavedBeforeTheWatchStartsIsRead(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
writeDatabase(t, path, "DE")
f := openFile(t, path, newQueue())
// Saved after OpenFile read the file, and before its directory was
// watched, so that no change is seen for it.
writeDatabase(t, path, "KP")
watch(t, f)
time.Sleep(quietTime)
synctest.Wait()
wantCountry(t, f, "KP")
})
}
// newQueue returns a queue of alerts for a webhook that is never sent
// them, so that they wait in it for the test to look at.
func newQueue() *alerts.Queue {
return alerts.New(alerts.Params{
WebhookURL: &url.URL{Scheme: "https", Host: "alerts.example"},
Events: alerts.Events(),
Cooldown: 15 * time.Minute,
Now: time.Now,
})
}
// writeDatabase writes a lookup database at path that places testNetblock
// in country, and no other address.
func writeDatabase(t *testing.T, path, country string) {
t.Helper()
lookuptest.Write(t, path, map[string]lookuptest.Network{
testNetblock: {ASN: "AS64496", ASName: "Example Net", Country: country},
})
}
// openFile opens the lookup database at path, which raises its alerts to
// queue.
func openFile(t *testing.T, path string, queue *alerts.Queue) *File {
t.Helper()
f, err := OpenFile(FileParams{
Path: path,
Now: time.Now,
ProcessLog: slog.New(slog.DiscardHandler),
Alerts: queue,
})
if err != nil {
t.Fatalf("open %s: %v", path, err)
}
return f
}
// watch runs f's readAfterChanges until the test ends, and returns the
// channel that sends it changes.
func watch(t *testing.T, f *File) chan<- fsnotify.Event {
t.Helper()
changes := make(chan fsnotify.Event)
ctx, stop := context.WithCancel(t.Context())
stopped := make(chan struct{})
go func() {
f.readAfterChanges(ctx, changes, nil)
close(stopped)
}()
t.Cleanup(func() {
stop()
<-stopped
})
return changes
}
// wantCountry checks the country f gives testClient.
func wantCountry(t *testing.T, f *File, want string) {
t.Helper()
got := f.LookUp(netip.MustParsePrefix(testClient)).Country
if got != want {
t.Errorf("%s is in %q, want %q", testClient, got, want)
}
}
// wantAlerts checks the alerts waiting in queue, and that it held none
// back.
func wantAlerts(t *testing.T, queue *alerts.Queue, want ...alerts.Alert) {
t.Helper()
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != len(want) || (len(want) > 0 && !reflect.DeepEqual(waiting, want)) {
t.Errorf("alerts waiting %+v, want %+v", waiting, want)
}
if queue.Suppressed() != 0 {
t.Errorf("%d alerts held back, want none", queue.Suppressed())
}
}
// writeFile writes content to the file at path.
func writeFile(t *testing.T, path, content string) {
t.Helper()
err := os.WriteFile(path, []byte(content), 0o600)
if err != nil {
t.Fatalf("write %s: %v", path, err)
}
}
// write writes data to the end of file.
func write(t *testing.T, file *os.File, data []byte) {
t.Helper()
_, err := file.Write(data)
if err != nil {
t.Fatalf("write: %v", err)
}
}
internal/lookup/lookup.go
+65 -140
View File
@@ -1,8 +1,7 @@
// Package lookup looks up each client's AS number and country, through
// the GeoJS web service or in the lookup database, the IPinfo Lite file
// SWWAF_LOOKUP_DB_PATH names. GeoJS's answers are kept in memory, for at
// most 100,000 clients and for 7 days each, and are written to
// lookups.json and read from it by the state package.
// Package lookup looks up each client's country through the GeoJS web
// service, and keeps the answers in memory, for at most 100,000 clients
// and for 7 days each. The answers are written to lookups.json and read
// from it by the state package.
package lookup
import (
@@ -15,20 +14,17 @@ import (
"net/http"
"net/netip"
"slices"
"strconv"
"strings"
"sync"
"time"
"github.com/hashicorp/golang-lru/v2/simplelru"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/metrics"
)
// URL is GeoJS's endpoint for an address's place and network. Asked about
// several addresses at once, comma separated in its ip parameter, it
// answers with a list.
const URL = "https://get.geojs.io/v1/ip/geo.json"
// URL is GeoJS's country endpoint. Asked about several addresses at once,
// comma separated in its ip parameter, it answers with a list.
const URL = "https://get.geojs.io/v1/ip/country.json"
const (
// keepFor is how long an answer is used instead of asking GeoJS again.
@@ -43,8 +39,9 @@ const (
maxWaiting = 10000
// maxPerRequest is how many addresses one request to GeoJS asks about.
maxPerRequest = 200
// unknownASN is the AS number GeoJS gives when it knows none.
unknownASN = 64512
// timeout is how long a new client waits for its answer, and how long
// a request to GeoJS may take before it is abandoned.
timeout = time.Second
// After a failure GeoJS is not asked again for a second, and for
// retryDelayFactor times as long after each further failure in a row,
// up to five minutes.
@@ -64,16 +61,6 @@ var (
type Params struct {
// URL is where GeoJS is asked, normally URL.
URL string
// Timeout is how long a request waits for its client's first answer,
// and how long a request to GeoJS may take before it is abandoned
// (SWWAF_LOOKUP_TIMEOUT).
Timeout time.Duration
// Wait is true when a setting needs each request's answer before the
// request goes on. Otherwise no request waits for one.
Wait bool
// Answered, unless nil, is given each answer GeoJS gives, once it is
// kept.
Answered func(Answer)
// Now tells the time, normally time.Now.
Now func() time.Time
// ProcessLog receives GeoJS's failures.
@@ -81,22 +68,16 @@ type Params struct {
// Metrics count the requests to GeoJS, those that failed, and the
// clients that go without an answer.
Metrics *metrics.Metrics
// Alerts receive a source_failure alert each time GeoJS fails.
Alerts *alerts.Queue
}
// GeoJS looks up clients' AS numbers and countries through GeoJS. At most
// one request to GeoJS is under way at a time, and it asks about every
// client waiting, up to maxPerRequest. It is safe for concurrent use.
// GeoJS looks up clients' countries through GeoJS. At most one request
// to GeoJS is under way at a time, and it asks about every client waiting,
// up to maxPerRequest. It is safe for concurrent use.
type GeoJS struct {
url string
timeout time.Duration
wait bool
answered func(Answer)
now func() time.Time
processLog *slog.Logger
metrics *metrics.Metrics
alerts *alerts.Queue
// httpClient follows no redirect, so that visitors' addresses go to
// GeoJS alone: a redirect is a failure.
httpClient *http.Client
@@ -114,18 +95,11 @@ type GeoJS struct {
retryAt time.Time
}
// Answer is what GeoJS or the lookup database said about a client: its AS
// number, such as AS64496, and the AS's name, both "" when the source knows
// no AS number for it; its country, "" when the source cannot place it;
// when the source said so; and, for GeoJS's answers, which lookups.json
// holds, when the answer was last used. The zero Answer is that of a
// client with no answer.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
// Answer is what GeoJS said about a client, as lookups.json holds it: its
// country, "" when GeoJS cannot place it, when GeoJS said so, and when
// the answer was last used.
type Answer struct {
Client netip.Prefix `json:"client"`
ASN string `json:"asn"`
ASName string `json:"as_name"`
Country string `json:"country"`
Answered time.Time `json:"answered"`
Used time.Time `json:"used"`
@@ -150,13 +124,9 @@ func New(params Params) *GeoJS {
return &GeoJS{
url: params.URL,
timeout: params.Timeout,
wait: params.Wait,
answered: params.Answered,
now: params.Now,
processLog: params.ProcessLog,
metrics: params.Metrics,
alerts: params.Alerts,
httpClient: &http.Client{
CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
@@ -167,23 +137,23 @@ func New(params Params) *GeoJS {
}
}
// LookUp returns the answer GeoJS gave about client, with its country as
// a two-letter code in capitals, or the zero Answer when there is none
// yet. An answer is kept for 7 days. Without one, the client is asked
// about in the background, and, while Wait is set, the request waits up
// to Timeout for the answer, unless the client has gone without one
// before. ctx is the context of the client's request, and ends the wait
// when it ends.
// Country returns the country GeoJS places client in, as a two-letter
// code in capitals, or "" when the country cannot be found: GeoJS cannot
// place the client, or has not answered in time. An answer is kept for 7
// days. Without one, a client waits up to timeout for it, unless it has
// gone without one before; until GeoJS answers, the client is asked about
// again in the background. ctx is the context of the client's request,
// and ends the wait when it ends.
//
// GeoJS is asked about the client's first address, which is the client's
// own address for IPv4, and an address in the same place for an IPv6 /64.
func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
answer, asked := g.answerOrWait(ctx, client)
func (g *GeoJS) Country(ctx context.Context, client netip.Prefix) string {
country, asked := g.answerOrWait(ctx, client)
if asked == nil {
return answer
return country
}
timer := time.NewTimer(g.timeout)
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
@@ -195,7 +165,7 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
g.mu.Lock()
defer g.mu.Unlock()
answer, found := g.kept(client)
country, found := g.kept(client)
if !found {
g.metrics.GeoJSUnanswered.Inc()
}
@@ -205,15 +175,7 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
w.late = true
}
return answer
}
// Kept returns client's answer, if one is kept, without asking GeoJS.
func (g *GeoJS) Kept(client netip.Prefix) (Answer, bool) {
g.mu.Lock()
defer g.mu.Unlock()
return g.kept(client)
return country
}
// Snapshot returns every answer kept, sorted by client, as lookups.json
@@ -265,13 +227,13 @@ func (g *GeoJS) Load(answers []Answer) {
// nil when there is nothing to wait for.
func (g *GeoJS) answerOrWait(
ctx context.Context, client netip.Prefix,
) (Answer, <-chan struct{}) {
) (string, <-chan struct{}) {
g.mu.Lock()
defer g.mu.Unlock()
answer, found := g.kept(client)
country, found := g.kept(client)
if found {
return answer, nil
return country, nil
}
w, waiting := g.waiting[client]
@@ -282,14 +244,10 @@ func (g *GeoJS) answerOrWait(
g.ask(ctx)
if !g.wait {
return Answer{}, nil // the answer is not needed before the request goes on
}
if w == nil {
g.metrics.GeoJSUnanswered.Inc()
return Answer{}, nil // too many clients wait already
return "", nil // too many clients wait already
}
if !g.asking {
@@ -300,25 +258,25 @@ func (g *GeoJS) answerOrWait(
if w.late {
g.metrics.GeoJSUnanswered.Inc()
return Answer{}, nil
return "", nil
}
return Answer{}, w.asked
return "", w.asked
}
// kept returns client's answer, if GeoJS gave it less than keepFor ago,
// and notes that it was used.
func (g *GeoJS) kept(client netip.Prefix) (Answer, bool) {
func (g *GeoJS) kept(client netip.Prefix) (string, bool) {
now := g.now()
kept, found := g.answers.Get(client)
if !found || now.Sub(kept.Answered) >= keepFor {
return Answer{}, false
return "", false
}
kept.Used = now
return *kept, true
return kept.Country, true
}
// ask starts asking GeoJS about the waiting clients, unless a request to
@@ -336,8 +294,7 @@ func (g *GeoJS) ask(ctx context.Context) {
}
// askAboutWaiting asks GeoJS about the waiting clients, one request at a
// time, until none is left or GeoJS fails. Each answer kept is given to
// Answered, outside the lock, since Answered takes locks of its own.
// time, until none is left or GeoJS fails.
func (g *GeoJS) askAboutWaiting(ctx context.Context) {
for {
clients := g.nextClients()
@@ -345,16 +302,8 @@ func (g *GeoJS) askAboutWaiting(ctx context.Context) {
return
}
given, err := g.request(ctx, clients)
kept, answered := g.keep(clients, given, err)
if g.answered != nil {
for _, answer := range kept {
g.answered(answer)
}
}
if !answered {
countries, err := g.request(ctx, clients)
if !g.keep(clients, countries, err) {
return
}
}
@@ -386,35 +335,32 @@ func (g *GeoJS) nextClients() []netip.Prefix {
return clients
}
// keep notes how a request to GeoJS about clients ended, given being the
// answer for each address GeoJS's answer names. It returns the answers it
// kept, and reports whether GeoJS answered about all of the clients. Each
// client whose address GeoJS's answer names gets its answer. An answer
// that leaves an address out is a failure. After a failure GeoJS is left
// alone for a while, and every client still waiting stops waiting and is
// asked about once GeoJS is asked again.
// keep notes how a request to GeoJS about clients ended, and reports
// whether GeoJS answered about all of them. Each client whose address
// GeoJS's answer names gets its answer, with no country when GeoJS gave
// none. An answer that leaves an address out is a failure. After a
// failure GeoJS is left alone for a while, and every client still waiting
// stops waiting and is asked about once GeoJS is asked again.
func (g *GeoJS) keep(
clients []netip.Prefix, given map[netip.Addr]Answer, err error,
) ([]Answer, bool) {
clients []netip.Prefix, countries map[netip.Addr]string, err error,
) bool {
g.mu.Lock()
defer g.mu.Unlock()
now := g.now()
kept := make([]Answer, 0, len(clients))
leftOut := 0
for _, client := range clients {
answer, named := given[client.Addr()]
country, named := countries[client.Addr()]
if !named {
leftOut++
continue
}
answer.Client, answer.Answered, answer.Used = client, now, now
g.answers.Add(client, &answer)
kept = append(kept, answer)
g.answers.Add(client, &Answer{
Client: client, Country: country, Answered: now, Used: now,
})
close(g.waiting[client].asked)
delete(g.waiting, client)
}
@@ -440,37 +386,27 @@ func (g *GeoJS) keep(
g.processLog.Warn("asking GeoJS failed",
"error", err.Error(), "asking_again_in", g.retryDelay.String())
g.alerts.Raise(alerts.Alert{
Event: alerts.EventSourceFailure,
Reason: "asking GeoJS failed",
Detail: map[string]any{
"source": "geojs", "error": err.Error(),
"asking_again_in": g.retryDelay.String(),
},
})
return kept, false
return false
}
g.retryDelay = 0
return kept, true
return true
}
// request asks GeoJS about clients in one request, and returns the answer
// for each address GeoJS's answer names: its AS number and the AS's name,
// both "" for the AS number 64512, which GeoJS gives when it knows none,
// and its country, in capitals.
// request asks GeoJS about clients in one request, and returns the
// country it gave, in capitals, for each address its answer names.
func (g *GeoJS) request(
ctx context.Context, clients []netip.Prefix,
) (map[netip.Addr]Answer, error) {
) (map[netip.Addr]string, error) {
addrs := make([]string, 0, len(clients))
for _, client := range clients {
addrs = append(addrs, client.Addr().String())
}
ctx, cancel := context.WithTimeout(ctx, g.timeout)
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, g.url, http.NoBody)
@@ -497,12 +433,9 @@ func (g *GeoJS) request(
return nil, fmt.Errorf("%w %s", errStatus, res.Status)
}
//nolint:tagliatelle // GeoJS's own names
var answers []struct {
IP string `json:"ip"`
ASN int64 `json:"asn"`
ASName string `json:"organization_name"`
CountryCode string `json:"country_code"`
IP string `json:"ip"`
Country string `json:"country"`
}
err = json.NewDecoder(io.LimitReader(res.Body, maxResponseBytes)).Decode(&answers)
@@ -510,22 +443,14 @@ func (g *GeoJS) request(
return nil, fmt.Errorf("read GeoJS's answer: %w", err)
}
given := make(map[netip.Addr]Answer, len(answers))
countries := make(map[netip.Addr]string, len(answers))
for _, item := range answers {
addr, err := netip.ParseAddr(item.IP)
if err != nil {
continue
if err == nil {
countries[addr] = strings.ToUpper(item.Country)
}
answer := Answer{Country: strings.ToUpper(item.CountryCode)}
if item.ASN != 0 && item.ASN != unknownASN {
answer.ASN = "AS" + strconv.FormatInt(item.ASN, 10)
answer.ASName = item.ASName
}
given[addr] = answer
}
return given, nil
return countries, nil
}
internal/lookup/lookup_test.go
+16 -247
View File
@@ -6,8 +6,6 @@ import (
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"reflect"
"slices"
"strings"
"sync"
@@ -16,20 +14,15 @@ import (
"time"
"github.com/prometheus/client_golang/prometheus/testutil"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
)
const (
// germany is where the stand-in for GeoJS places every address but
// unplaced, and asNumber, kept as asn, and asName the AS it gives them.
germany = "DE"
asNumber = 64496
asn = "AS64496"
asName = "Example Net"
// unplaced is the address it cannot place, for which it gives the AS
// number 64512 and the AS name Unknown, as GeoJS does.
// unplaced.
germany = "DE"
// unplaced is the address it cannot place.
unplaced = "192.0.2.1"
// leftOut is the address it leaves out of its answer when
// answeringWithoutLeftOut.
@@ -87,7 +80,7 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
var earlier sync.WaitGroup
earlier.Go(func() { g.LookUp(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) })
earlier.Go(func() { g.Country(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) })
defer earlier.Wait()
waitForRequests(t, geojs, 1)
@@ -119,58 +112,6 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
})
}
func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
// A timeout longer than the default second, and a GeoJS that does
// not answer.
const longerTimeout = 3 * time.Second
m := metrics.New(1, "app")
g := lookup.New(lookup.Params{
URL: lookup.URL,
Timeout: longerTimeout,
Wait: true,
Now: time.Now,
ProcessLog: slog.New(slog.DiscardHandler),
Metrics: m,
Alerts: alerts.New(alerts.Params{}),
})
g.SetTransport(&standIn{answers: hanging})
var (
request sync.WaitGroup
waited time.Duration
)
request.Go(func() {
began := time.Now()
wantCountry(t, g, netip.MustParsePrefix("203.0.113.9/32"), "")
waited = time.Since(began)
})
// A moment before the timeout runs out, GeoJS is still being asked:
// the request to it has not failed.
time.Sleep(longerTimeout - time.Millisecond)
synctest.Wait()
wantFailures(t, m, 0)
// As it runs out, the client's request goes on, and the request to
// GeoJS is abandoned, which counts as a failure.
request.Wait()
synctest.Wait()
if waited != longerTimeout {
t.Errorf("waited %s for the answer, want %s", waited, longerTimeout)
}
wantFailures(t, m, 1)
})
}
func TestAddressLeftOutOfAnAnswerIsAskedAboutAgain(t *testing.T) {
t.Parallel()
@@ -243,96 +184,6 @@ func TestCountryIsKeptInCapitals(t *testing.T) {
})
}
func TestAnswerHoldsTheASNumberTheASNameAndTheCountry(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
_, clock, g := start()
placed := netip.MustParsePrefix("203.0.113.9/32")
notPlaced := netip.MustParsePrefix(unplaced + "/32")
now := clock.Now()
// For the client it cannot place, GeoJS gives the AS number 64512
// and the AS name Unknown, which count as unknown.
for client, want := range map[netip.Prefix]lookup.Answer{
placed: {
Client: placed, ASN: asn, ASName: asName, Country: germany,
Answered: now, Used: now,
},
notPlaced: {Client: notPlaced, Answered: now, Used: now},
} {
got := g.LookUp(t.Context(), client)
if got != want {
t.Errorf("answer for %s\n%+v\nwant\n%+v", client, got, want)
}
}
})
}
func TestWithoutWaitTheRequestGoesOnAtOnceAndTheAnswerIsGivenWhenItComes(
t *testing.T,
) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
var (
mu sync.Mutex
given []lookup.Answer
)
geojs := &standIn{answers: answeringSlowly}
clock := newClock()
m := metrics.New(1, "app")
g := lookup.New(lookup.Params{
URL: lookup.URL,
Timeout: timeout,
Answered: func(answer lookup.Answer) {
mu.Lock()
defer mu.Unlock()
given = append(given, answer)
},
Now: clock.Now,
ProcessLog: slog.New(slog.DiscardHandler),
Metrics: m,
Alerts: alerts.New(alerts.Params{}),
})
g.SetTransport(geojs)
client := netip.MustParsePrefix("203.0.113.9/32")
// The request goes on at once, without an answer, and GeoJS is asked
// about the client, which it answers most of a second later.
began := time.Now()
got := g.LookUp(t.Context(), client)
if took := time.Since(began); took != 0 || got != (lookup.Answer{}) {
t.Errorf("waited %s for %+v, want no wait and no answer", took, got)
}
waitForRequests(t, geojs, 1)
wantAsked(t, geojs, 0, "203.0.113.9")
time.Sleep(timeout)
synctest.Wait()
now := clock.Now()
want := lookup.Answer{
Client: client, ASN: asn, ASName: asName, Country: germany,
Answered: now, Used: now,
}
mu.Lock()
if !slices.Equal(given, []lookup.Answer{want}) {
t.Errorf("answers given %+v, want only %+v", given, want)
}
mu.Unlock()
wantCountry(t, g, client, germany)
wantUnanswered(t, m, 0)
})
}
func TestFailureIsLoggedWithoutTheAddressesAskedAbout(t *testing.T) {
t.Parallel()
@@ -343,12 +194,9 @@ func TestFailureIsLoggedWithoutTheAddressesAskedAbout(t *testing.T) {
geojs := &standIn{answers: hanging}
g := lookup.New(lookup.Params{
URL: lookup.URL,
Timeout: timeout,
Wait: true,
Now: time.Now,
ProcessLog: slog.New(slog.NewTextHandler(&log, nil)),
Metrics: metrics.New(1, "app"),
Alerts: alerts.New(alerts.Params{}),
Metrics: metrics.New(1),
})
g.SetTransport(geojs)
@@ -363,45 +211,6 @@ func TestFailureIsLoggedWithoutTheAddressesAskedAbout(t *testing.T) {
})
}
func TestFailureRaisesASourceFailureAlertOncePerCooldown(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
geojs, clock, g, queue := startWithAlerts()
clients := newClients()
geojs.set(failing)
wantCountry(t, g, clients(), "")
want := alerts.Alert{
Time: clock.Now(),
Event: alerts.EventSourceFailure,
Reason: "asking GeoJS failed",
Detail: map[string]any{
"source": "geojs",
"error": "GeoJS answered 503 Service Unavailable",
"asking_again_in": "1s",
},
}
// The next failure, a second later, is a repeat within the
// cooldown.
clock.advance(time.Second)
wantCountry(t, g, clients(), "")
wantRequests(t, geojs, 2)
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || !reflect.DeepEqual(waiting[0], want) {
t.Errorf("alerts waiting %+v, want only %+v", waiting, want)
}
if queue.Suppressed() != 1 {
t.Errorf("%d alerts held back, want the repeat", queue.Suppressed())
}
})
}
func TestWaitingClientsAreAskedAboutInOneRequest(t *testing.T) {
t.Parallel()
@@ -546,15 +355,12 @@ func TestClientsWithoutAnAnswerAreCounted(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
m := metrics.New(1, "app")
m := metrics.New(1)
g := lookup.New(lookup.Params{
URL: lookup.URL,
Timeout: timeout,
Wait: true,
Now: time.Now,
ProcessLog: slog.New(slog.DiscardHandler),
Metrics: m,
Alerts: alerts.New(alerts.Params{}),
})
g.SetTransport(&standIn{answers: failing})
@@ -644,24 +450,21 @@ func (s *standIn) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
}
list := make([]map[string]any, 0, len(addrs))
list := make([]map[string]string, 0, len(addrs))
for _, addr := range addrs {
item := map[string]any{
"ip": addr, "asn": asNumber, "organization_name": asName,
"country_code": germany,
}
country := germany
switch {
case addr == unplaced:
item = map[string]any{"ip": addr, "asn": 64512, "organization_name": "Unknown"}
country = ""
case addr == leftOut && answers == answeringWithoutLeftOut:
continue
case answers == answeringInLowerCase:
item["country_code"] = strings.ToLower(germany)
country = strings.ToLower(germany)
}
list = append(list, item)
list = append(list, map[string]string{"ip": addr, "country": country})
}
var answer any = list
@@ -719,43 +522,19 @@ func (c *testClock) advance(d time.Duration) {
}
// start returns a stand-in for GeoJS that answers, a clock, and a GeoJS
// asking the stand-in by that clock, for which a request waits for its
// client's first answer.
// asking the stand-in by that clock.
func start() (*standIn, *testClock, *lookup.GeoJS) {
geojs, clock, g, _ := startWithAlerts()
return geojs, clock, g
}
// startWithAlerts is start, and returns the queue of the alerts GeoJS
// raises as well, for a webhook that is never sent them, with the default
// cooldown, by the same clock.
func startWithAlerts() (*standIn, *testClock, *lookup.GeoJS, *alerts.Queue) {
geojs := &standIn{}
clock := newClock()
queue := alerts.New(alerts.Params{
WebhookURL: &url.URL{Scheme: "https", Host: "alerts.example"},
Events: alerts.Events(),
Cooldown: 15 * time.Minute,
Now: clock.Now,
})
clock := &testClock{now: time.Date(2026, 10, 4, 0, 0, 0, 0, time.UTC)}
g := lookup.New(lookup.Params{
URL: lookup.URL,
Timeout: timeout,
Wait: true,
Now: clock.Now,
ProcessLog: slog.New(slog.DiscardHandler),
Metrics: metrics.New(1, "app"),
Alerts: queue,
Metrics: metrics.New(1),
})
g.SetTransport(geojs)
return geojs, clock, g, queue
}
// newClock returns a clock set to the start of a day.
func newClock() *testClock {
return &testClock{now: time.Date(2026, 10, 4, 0, 0, 0, 0, time.UTC)}
return geojs, clock, g
}
// newClients returns what returns a new IPv4 client each time it is
@@ -774,7 +553,7 @@ func newClients() func() netip.Prefix {
func wantCountry(t *testing.T, g *lookup.GeoJS, client netip.Prefix, want string) {
t.Helper()
got := g.LookUp(t.Context(), client).Country
got := g.Country(t.Context(), client)
if got != want {
t.Errorf("%s is in %q, want %q", client, got, want)
}
@@ -819,16 +598,6 @@ func wantUnanswered(t *testing.T, m *metrics.Metrics, want float64) {
}
}
// wantFailures checks how many requests to GeoJS m counts as failed.
func wantFailures(t *testing.T, m *metrics.Metrics, want float64) {
t.Helper()
got := testutil.ToFloat64(m.GeoJSFailures)
if got != want {
t.Errorf("%v requests to GeoJS failed, want %v", got, want)
}
}
// waitForRequests waits until g has done all it can before time passes,
// checks that GeoJS has had count requests, and returns the addresses each
// asked about.
internal/lookup/lookuptest/lookuptest.go
-83
View File
@@ -1,83 +0,0 @@
// Package lookuptest writes lookup databases, IPinfo Lite files in their
// .mmdb form, for the tests of the packages that read them.
package lookuptest
import (
"bytes"
"net"
"os"
"testing"
"github.com/maxmind/mmdbwriter"
"github.com/maxmind/mmdbwriter/mmdbtype"
)
// fileMode is the mode of the files written: read and written by their
// owner alone.
const fileMode = 0o600
// Network is what a lookup database holds about a netblock, of the fields
// smallwebwaf reads: its AS number, such as AS64496, the AS's name, and
// its country, such as DE.
type Network struct {
ASN string
ASName string
Country string
}
// Write writes a lookup database at path that places each netblock in
// networks, such as 203.0.113.0/24, as its Network says, and no other
// address.
func Write(tb testing.TB, path string, networks map[string]Network) {
tb.Helper()
records := make(map[string]mmdbtype.Map, len(networks))
for netblock, network := range networks {
records[netblock] = mmdbtype.Map{
"asn": mmdbtype.String(network.ASN),
"as_name": mmdbtype.String(network.ASName),
"country_code": mmdbtype.String(network.Country),
}
}
WriteRecords(tb, path, records)
}
// WriteRecords writes a lookup database at path that holds each record in
// records for its netblock, and nothing for any other address.
func WriteRecords(tb testing.TB, path string, records map[string]mmdbtype.Map) {
tb.Helper()
tree, err := mmdbwriter.New(mmdbwriter.Options{
DatabaseType: "ipinfo_lite",
// The tests' clients are in the netblocks kept for documentation.
IncludeReservedNetworks: true,
})
if err != nil {
tb.Fatalf("new lookup database: %v", err)
}
for netblock, record := range records {
_, network, err := net.ParseCIDR(netblock)
if err != nil {
tb.Fatalf("netblock %q: %v", netblock, err)
}
err = tree.Insert(network, record)
if err != nil {
tb.Fatalf("insert %s: %v", netblock, err)
}
}
var database bytes.Buffer
_, err = tree.WriteTo(&database)
if err != nil {
tb.Fatalf("write the lookup database: %v", err)
}
err = os.WriteFile(path, database.Bytes(), fileMode)
if err != nil {
tb.Fatalf("write %s: %v", path, err)
}
}
internal/lookup/snapshot_test.go
+2 -5
View File
@@ -26,11 +26,8 @@ func TestSnapshotHoldsEachAnswerAndWhenItWasLastUsed(t *testing.T) {
wantCountry(t, g, placed, germany)
want := []lookup.Answer{
{Client: notPlaced, Answered: asked, Used: asked},
{
Client: placed, ASN: asn, ASName: asName, Country: germany,
Answered: asked, Used: asked.Add(time.Hour),
},
{Client: notPlaced, Country: "", Answered: asked, Used: asked},
{Client: placed, Country: germany, Answered: asked, Used: asked.Add(time.Hour)},
}
if got := g.Snapshot(); !slices.Equal(got, want) {
t.Errorf("snapshot\n%+v\nwant\n%+v", got, want)
internal/metrics/busiest.go
-155
View File
@@ -1,155 +0,0 @@
package metrics
import (
"sync"
"github.com/prometheus/client_golang/prometheus"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// other is the label value under which the countries or AS numbers
// outside the busiest are counted.
const other = "other"
// busiest are the metrics by one thing the lookup finds of the client,
// its country or its AS number, for requests whose client's is known.
// The topN busiest countries or AS numbers, by their requests since the
// start, have series of their own, and the others are counted under
// other, so that there are never more than topN + 1 series. One that
// drops out of the busiest loses its series, and its next requests are
// counted under other; one that becomes one of them gets a series that
// counts from then on. Each series therefore only ever goes up.
type busiest struct {
topN int
requests *prometheus.CounterVec
requestBytes *prometheus.CounterVec
responseBytes *prometheus.CounterVec
// refused are, by country, the requests the country lists refused; nil
// by AS number.
refused *prometheus.CounterVec
mu sync.Mutex
// seen is each country's or AS number's requests since the start, by
// which they are ranked.
seen map[string]int64
// top are those with series of their own.
top map[string]bool
}
// newCountries returns the metrics by country, with series of their own
// for the topN busiest countries.
func newCountries(topN int) *busiest {
countries := newBusiest(topN, "country", "the client's country")
countries.refused = counterVec("smallwebwaf_country_list_refusals_total",
"Requests the country lists refused, by the client's country.",
[]string{"country"})
return countries
}
// newASNs returns the metrics by AS number, with series of their own for
// the topN busiest AS numbers.
func newASNs(topN int) *busiest {
return newBusiest(topN, "asn", "the client's AS number")
}
// newBusiest returns the metrics by label, which is described as
// description, with series of their own for the topN busiest values.
func newBusiest(topN int, label, description string) *busiest {
by := []string{label}
return &busiest{
topN: topN,
requests: counterVec("smallwebwaf_"+label+"_requests_total",
"Requests, by "+description+".", by),
requestBytes: counterVec("smallwebwaf_"+label+"_request_bytes_total",
"Request body bytes, by "+description+".", by),
responseBytes: counterVec("smallwebwaf_"+label+"_response_bytes_total",
"Response body bytes, by "+description+".", by),
seen: map[string]int64{},
top: map[string]bool{},
}
}
// Describe and Collect make the metrics a prometheus.Collector, so that
// they are registered together.
func (b *busiest) Describe(ch chan<- *prometheus.Desc) {
for _, vec := range b.vecs() {
vec.Describe(ch)
}
}
// Collect is the other half of prometheus.Collector, with Describe.
func (b *busiest) Collect(ch chan<- prometheus.Metric) {
for _, vec := range b.vecs() {
vec.Collect(ch)
}
}
// vecs returns the metrics: by AS number, those of requests and bytes; by
// country, the refusals by the country lists as well.
func (b *busiest) vecs() []*prometheus.CounterVec {
vecs := []*prometheus.CounterVec{b.requests, b.requestBytes, b.responseBytes}
if b.refused != nil {
vecs = append(vecs, b.refused)
}
return vecs
}
// add counts a request from its log line, whose client's country or AS
// number, value, is known.
func (b *busiest) add(value string, line *requestlog.Line) {
b.mu.Lock()
defer b.mu.Unlock()
b.seen[value]++
label := b.label(value)
b.requests.WithLabelValues(label).Inc()
b.requestBytes.WithLabelValues(label).Add(float64(line.RequestBytes))
b.responseBytes.WithLabelValues(label).Add(float64(line.ResponseBytes))
if b.refused != nil && line.Action == requestlog.ActionCountryDenied {
b.refused.WithLabelValues(label).Inc()
}
}
// label returns the label a request from value is counted under: value
// while it is one of the busiest, other while it is not. A value busier
// than the least busy of them takes its place, and that one's series are
// dropped.
func (b *busiest) label(value string) string {
if b.top[value] {
return value
}
if len(b.top) < b.topN {
b.top[value] = true
return value
}
least := ""
for top := range b.top {
if least == "" || b.seen[top] < b.seen[least] {
least = top
}
}
if b.seen[value] <= b.seen[least] {
return other
}
delete(b.top, least)
for _, vec := range b.vecs() {
vec.DeleteLabelValues(least)
}
b.top[value] = true
return value
}
internal/metrics/countries.go
+116
View File
@@ -0,0 +1,116 @@
package metrics
import (
"sync"
"github.com/prometheus/client_golang/prometheus"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// other is the label under which the countries outside the busiest are
// counted.
const other = "other"
// countries are the metrics by the client's country, for requests whose
// client's country is known. The topN busiest countries, by their requests
// since the start, have series of their own, and the others are counted
// under other, so that there are never more than topN + 1 series. A
// country that drops out of the busiest loses its series, and its next
// requests are counted under other; one that becomes one of them gets a
// series that counts from then on. Each series therefore only ever goes
// up.
type countries struct {
topN int
requests *prometheus.CounterVec
requestBytes *prometheus.CounterVec
responseBytes *prometheus.CounterVec
// refused are the requests the country lists refused.
refused *prometheus.CounterVec
mu sync.Mutex
// seen is each country's requests since the start, by which the
// countries are ranked. GeoJS gives two-letter codes, so it holds at
// most a few hundred.
seen map[string]int64
// top are the countries with series of their own.
top map[string]bool
}
// newCountries returns the metrics by country, with series of their own
// for the topN busiest countries.
func newCountries(topN int) *countries {
byCountry := []string{"country"}
return &countries{
topN: topN,
requests: counterVec("smallwebwaf_country_requests_total",
"Requests, by the client's country.", byCountry),
requestBytes: counterVec("smallwebwaf_country_request_bytes_total",
"Request body bytes, by the client's country.", byCountry),
responseBytes: counterVec("smallwebwaf_country_response_bytes_total",
"Response body bytes, by the client's country.", byCountry),
refused: counterVec("smallwebwaf_country_list_refusals_total",
"Requests the country lists refused, by the client's country.",
byCountry),
seen: map[string]int64{},
top: map[string]bool{},
}
}
// add counts a request from its log line, whose country is known.
func (c *countries) add(line *requestlog.Line) {
c.mu.Lock()
defer c.mu.Unlock()
c.seen[line.Country]++
label := c.label(line.Country)
c.requests.WithLabelValues(label).Inc()
c.requestBytes.WithLabelValues(label).Add(float64(line.RequestBytes))
c.responseBytes.WithLabelValues(label).Add(float64(line.ResponseBytes))
if line.Action == requestlog.ActionCountryDenied {
c.refused.WithLabelValues(label).Inc()
}
}
// label returns the label a request from country is counted under: the
// country while it is one of the busiest, other while it is not. A
// country busier than the least busy of them takes its place, and that
// country's series are dropped.
func (c *countries) label(country string) string {
if c.top[country] {
return country
}
if len(c.top) < c.topN {
c.top[country] = true
return country
}
least := ""
for top := range c.top {
if least == "" || c.seen[top] < c.seen[least] {
least = top
}
}
if c.seen[country] <= c.seen[least] {
return other
}
delete(c.top, least)
for _, vec := range []*prometheus.CounterVec{
c.requests, c.requestBytes, c.responseBytes, c.refused,
} {
vec.DeleteLabelValues(least)
}
c.top[country] = true
return country
}
internal/metrics/metrics.go
+22 -134
View File
@@ -6,24 +6,20 @@ package metrics
import (
"net/http"
"strconv"
"strings"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/collectors"
"github.com/prometheus/client_golang/prometheus/promhttp"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/remotelog"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// Metrics are smallwebwaf's metrics. They are safe for concurrent use.
type Metrics struct {
// registry gives every metric registered with it the label instance.
registry prometheus.Registerer
registry *prometheus.Registry
handler http.Handler
inFlight prometheus.Gauge
@@ -37,13 +33,12 @@ type Metrics struct {
offences *prometheus.CounterVec
// ruleMatches are made by AddRules.
ruleMatches *prometheus.CounterVec
countries *busiest
asns *busiest
countries *countries
// GeoJSRequests are the requests to GeoJS, and GeoJSFailures those
// that failed. GeoJSUnanswered are the requests that needed their
// client's answer, for a setting that acts on it, and went on without
// it because GeoJS had not given it in time.
// that failed. GeoJSUnanswered are the requests whose client counted
// as coming from an unknown country because GeoJS had not answered
// about it in time.
GeoJSRequests prometheus.Counter
GeoJSFailures prometheus.Counter
GeoJSUnanswered prometheus.Counter
@@ -57,18 +52,14 @@ type Metrics struct {
}
// New returns the metrics, with the Go runtime's and the process's own.
// topN is how many countries and how many AS numbers get series of their
// own (SWWAF_METRICS_TOP_N). Every metric carries instanceName
// (SWWAF_INSTANCE_NAME) as its label instance.
func New(topN int, instanceName string) *Metrics {
// topN is how many countries get series of their own
// (SWWAF_METRICS_TOP_N).
func New(topN int) *Metrics {
byStatus := []string{"status_class", "action"}
byFile := []string{"file"}
registry := prometheus.NewRegistry()
m := &Metrics{
registry: prometheus.WrapRegistererWith(
prometheus.Labels{"instance": instanceName}, registry),
handler: promhttp.HandlerFor(registry, promhttp.HandlerOpts{}),
registry: prometheus.NewRegistry(),
inFlight: prometheus.NewGauge(prometheus.GaugeOpts{
Name: "smallwebwaf_requests_in_flight",
Help: "Requests under way.",
@@ -90,16 +81,14 @@ func New(topN int, instanceName string) *Metrics {
Help: "How long requests passed to the app took, from then to their end.",
}),
rateLimitHits: counterVec("smallwebwaf_rate_limit_hits_total",
"Requests that broke a rate limit or a byte limit, by its window and "+
"its kind, requests or bytes.",
[]string{"window", "kind"}),
"Requests that broke a rate limit, by its window.",
[]string{"window"}),
sizeAndTimeLimitHits: counterVec("smallwebwaf_size_and_time_limit_hits_total",
"Requests that passed a size or time limit, by its setting.",
[]string{"limit"}),
offences: counterVec("smallwebwaf_offences_total",
"Offences, by kind.", []string{"kind"}),
countries: newCountries(topN),
asns: newASNs(topN),
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
Name: "smallwebwaf_geojs_requests_total",
Help: "Requests to GeoJS.",
@@ -110,8 +99,8 @@ func New(topN int, instanceName string) *Metrics {
}),
GeoJSUnanswered: prometheus.NewCounter(prometheus.CounterOpts{
Name: "smallwebwaf_geojs_unanswered_total",
Help: "Requests that needed their client's answer from GeoJS and " +
"went on without it, because GeoJS had not given it in time.",
Help: "Requests whose client counted as coming from an unknown " +
"country because GeoJS had not answered about it in time.",
}),
stateFileWrites: counterVec("smallwebwaf_state_file_writes_total",
"Writes of each state file.", byFile),
@@ -128,12 +117,16 @@ func New(topN int, instanceName string) *Metrics {
byFile),
}
m.handler = promhttp.HandlerFor(m.registry, promhttp.HandlerOpts{})
m.registry.MustRegister(
collectors.NewGoCollector(),
collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}),
m.inFlight, m.requests, m.requestBytes, m.responseBytes,
m.requestDuration, m.upstreamDuration,
m.rateLimitHits, m.sizeAndTimeLimitHits, m.offences, m.countries, m.asns,
m.rateLimitHits, m.sizeAndTimeLimitHits, m.offences,
m.countries.requests, m.countries.requestBytes, m.countries.responseBytes,
m.countries.refused,
m.GeoJSRequests, m.GeoJSFailures, m.GeoJSUnanswered,
m.stateFileWrites, m.stateFileWriteFailures,
m.stateFileLastWrite, m.stateFileSize,
@@ -150,7 +143,7 @@ func New(topN int, instanceName string) *Metrics {
func (m *Metrics) AddBansAndClients(
ledger *bans.Ledger, limiter *ratelimit.Limiter, now func() time.Time,
) {
for _, cause := range []string{bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin} {
for _, cause := range []string{bans.CauseLimit, bans.CauseAttack} {
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_bans_made_total",
Help: "Bans made, by cause.",
@@ -171,7 +164,7 @@ func (m *Metrics) AddBansAndClients(
}),
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "smallwebwaf_permanent_bans",
Help: "Permanent bans not lifted.",
Help: "Permanent bans.",
}, func() float64 {
_, permanent := ledger.Count(now())
@@ -204,99 +197,6 @@ func (m *Metrics) AddRules(ruleFiles *rules.Files) {
}))
}
// AddRemoteLog adds the metrics of sending the log lines to
// SWWAF_LOG_REMOTE_URL, read from remote as the metrics are asked for: the
// lines sent, those dropped, and those waiting in the buffer.
func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
m.registry.MustRegister(
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_remote_log_lines_sent_total",
Help: "Log lines sent to SWWAF_LOG_REMOTE_URL.",
}, func() float64 {
return float64(remote.Sent())
}),
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_remote_log_lines_dropped_total",
Help: "Log lines dropped: the oldest in a full buffer, and those " +
"whose sending failed.",
}, func() float64 {
return float64(remote.Dropped())
}),
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "smallwebwaf_remote_log_buffer_depth",
Help: "Log lines in the buffer, waiting to be sent.",
}, func() float64 {
return float64(remote.Depth())
}),
)
}
// AddLookupFile adds the metrics of the lookup database, read as the
// metrics are asked for: when the file in use was read, which lastRead
// returns, and the replacements of it that could not be read, which
// readFailures returns. The lookup package's File, which has both, cannot
// be named here: that package counts GeoJS's requests in these metrics.
func (m *Metrics) AddLookupFile(lastRead func() time.Time, readFailures func() int) {
m.registry.MustRegister(
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "smallwebwaf_lookup_database_last_read_timestamp_seconds",
Help: "When the lookup database in use was read, in seconds since 1970.",
}, func() float64 {
return float64(lastRead().Unix())
}),
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_lookup_database_read_failures_total",
Help: "Replacements of the lookup database that could not be read.",
}, func() float64 {
return float64(readFailures())
}),
)
}
// AddAlerts adds the metrics of the alerts sent to each destination set,
// read from queue as the metrics are asked for, by destination: the
// alerts sent, the requests to the destination that failed, the alerts
// held back, which are the same for every destination, and those
// dropped. With no destination set, it adds none.
func (m *Metrics) AddAlerts(queue *alerts.Queue) {
for _, name := range queue.DestinationsSet() {
destination := prometheus.Labels{"destination": name}
m.registry.MustRegister(
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_alerts_sent_total",
Help: "Alerts the destination took.",
ConstLabels: destination,
}, func() float64 {
return float64(queue.Counts(name).Sent)
}),
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_alerts_failed_total",
Help: "Requests to the destination that failed.",
ConstLabels: destination,
}, func() float64 {
return float64(queue.Counts(name).Failed)
}),
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_alerts_suppressed_total",
Help: "Alerts held back: repeats within SWWAF_ALERT_COOLDOWN, and " +
"alerts past SWWAF_ALERT_MAX_PER_HOUR, for the hour's summary.",
ConstLabels: destination,
}, func() float64 {
return float64(queue.Suppressed())
}),
prometheus.NewCounterFunc(prometheus.CounterOpts{
Name: "smallwebwaf_alerts_dropped_total",
Help: "Alerts dropped, the oldest first, from a full queue, and " +
"alerts given up as the destination refused them.",
ConstLabels: destination,
}, func() float64 {
return float64(queue.Counts(name).Dropped)
}),
)
}
}
// ServeHTTP answers with the metrics in the Prometheus text format.
func (m *Metrics) ServeHTTP(w http.ResponseWriter, r *http.Request) {
m.handler.ServeHTTP(w, r)
@@ -327,15 +227,7 @@ func (m *Metrics) RequestEnded(
}
if line.LimitHit != "" {
// The log line names a byte limit's window with _bytes after it.
window, isBytes := strings.CutSuffix(line.LimitHit, "_bytes")
kind := ratelimit.KindRequests
if isBytes {
kind = ratelimit.KindBytes
}
m.rateLimitHits.WithLabelValues(window, kind).Inc()
m.rateLimitHits.WithLabelValues(line.LimitHit).Inc()
}
if limit != "" {
@@ -347,11 +239,7 @@ func (m *Metrics) RequestEnded(
}
if line.Country != "" {
m.countries.add(line.Country, line)
}
if line.ASN != "" {
m.asns.add(line.ASN, line)
m.countries.add(line)
}
}
internal/proxy/admin.go
+7 -289
View File
@@ -1,60 +1,26 @@
package proxy
import (
"bytes"
"crypto/subtle"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/netip"
"os"
"strings"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/state"
)
// banBodyMaxBytes is the most of the body of a request to add a ban that
// is read; its three fields need far less.
const banBodyMaxBytes = 4 << 10
// permanent is how the log line and the ban endpoint name a ban that
// never ends.
const permanent = "permanent"
var (
errNotBanToAdd = errors.New(
"the body is not a JSON object of netblock, duration and reason")
errNotNetblock = errors.New(
"is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24")
errMappedNetblock = errors.New(
"is IPv4-mapped: give the IPv4 netblock, such as 203.0.113.0/24")
errZone = errors.New("has a zone, which a netblock cannot have")
errNotDuration = errors.New(
"is not a duration above zero, such as 1h or 7d, or permanent")
errNotAddress = errors.New("is not an address, such as 203.0.113.9")
)
// answerAdmin answers a request for smallwebwaf itself, under
// /_smallwebwaf/, once it has passed the checks. Each endpoint needs a
// token, sent as Authorization: Bearer <token>: the metrics
// SWWAF_METRICS_TOKEN, the others SWWAF_ADMIN_TOKEN. A request without
// it is refused with 401. An endpoint whose token is unset answers 404,
// as any other request under /_smallwebwaf/ does.
// /_smallwebwaf/, once it has passed the checks: GET MetricsPath with
// SWWAF_METRICS_TOKEN gets the metrics, and without it is refused with
// 401. Any other request gets 404, as the metrics do while
// SWWAF_METRICS_TOKEN is unset.
func (rq *request) answerAdmin() {
rq.line.Action = requestlog.ActionAdmin
rq.startClientResponseTimeout()
token, answer := rq.endpoint()
token := rq.h.config.MetricsToken
switch {
case token == "":
case token == "" || rq.in.Method != http.MethodGet || rq.in.URL.Path != MetricsPath:
http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
case !hasToken(rq.in, token):
rq.out.Header().Set("WWW-Authenticate", "Bearer")
@@ -63,29 +29,7 @@ func (rq *request) answerAdmin() {
action: requestlog.ActionAdmin,
})
default:
answer()
}
}
// endpoint returns the token the request's endpoint needs, and what
// answers the request there; "" when there is no such endpoint.
func (rq *request) endpoint() (string, func()) {
cfg := rq.h.config
method, path := rq.in.Method, rq.in.URL.Path
switch {
case method == http.MethodGet && path == MetricsPath:
return cfg.MetricsToken, func() { rq.h.metrics.ServeHTTP(rq.out, rq.in) }
case method == http.MethodGet && path == BansPath:
return cfg.AdminToken, rq.listBans
case method == http.MethodPost && path == BansPath:
return cfg.AdminToken, rq.addBan
case method == http.MethodDelete && strings.HasPrefix(path, BansPath+"/"):
return cfg.AdminToken, rq.liftBans
case method == http.MethodGet && strings.HasPrefix(path, ClientsPath):
return cfg.AdminToken, rq.showClient
default:
return "", nil
rq.h.metrics.ServeHTTP(rq.out, rq.in)
}
}
@@ -97,229 +41,3 @@ func hasToken(r *http.Request, token string) bool {
return strings.EqualFold(scheme, "Bearer") &&
subtle.ConstantTimeCompare([]byte(sent), []byte(token)) == 1
}
// listBans answers GET BansPath with every ban held.
func (rq *request) listBans() {
rq.answerBans(rq.h.ledger.Snapshot())
}
// banToAdd is the body of POST BansPath.
type banToAdd struct {
// Netblock is a netblock, or a client's address, which stands for the
// netblock a ban on that client covers.
Netblock string `json:"netblock"`
// Duration is how long the ban lasts, as a setting gives a duration,
// or permanent.
Duration string `json:"duration"`
Reason string `json:"reason"`
}
// addBan answers POST BansPath: it bans the netblock the body names, as
// an admin, from now for the duration the body gives, with its reason,
// and answers with that ban.
func (rq *request) addBan() {
// The body must arrive within SWWAF_CLIENT_REQUEST_TIMEOUT, as any
// other request's must.
rq.stopReadingBody(rq.clientRequestDeadline())
toAdd, err := rq.readBanToAdd()
if refused := rq.refused.Load(); refused != nil {
rq.answer(*refused) // the body is over SWWAF_REQUEST_MAX_BYTES
return
}
if errors.Is(err, os.ErrDeadlineExceeded) {
rq.answer(refusal{
status: http.StatusRequestTimeout,
action: requestlog.ActionTimedOut,
limit: "SWWAF_CLIENT_REQUEST_TIMEOUT",
})
return
}
var (
netblock netip.Prefix
expires time.Time
now = rq.h.now()
)
if err == nil {
netblock, err = rq.h.banNetblock(toAdd.Netblock)
}
if err == nil {
expires, err = expiry(toAdd.Duration, now)
}
if err != nil {
http.Error(rq.out, err.Error(), http.StatusBadRequest)
return
}
ban := rq.h.ledger.BanForAdmin(netblock, now, expires, toAdd.Reason)
rq.answerBans([]bans.Ban{ban})
}
// readBanToAdd reads the body of POST BansPath: a JSON object with
// nothing but whitespace after it, in at most banBodyMaxBytes.
func (rq *request) readBanToAdd() (banToAdd, error) {
var body io.ReadCloser = http.NoBody
if rq.body != nil {
body = rq.body
}
data, err := io.ReadAll(http.MaxBytesReader(nil, body, banBodyMaxBytes))
if err != nil {
return banToAdd{}, fmt.Errorf("%w: %w", errNotBanToAdd, err)
}
var toAdd banToAdd
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.DisallowUnknownFields()
err = decoder.Decode(&toAdd)
if err != nil {
return banToAdd{}, fmt.Errorf("%w: %w", errNotBanToAdd, err)
}
// Token returns io.EOF only when nothing but whitespace is left.
_, err = decoder.Token()
if !errors.Is(err, io.EOF) {
return banToAdd{}, fmt.Errorf("%w: more follows the object", errNotBanToAdd)
}
return toAdd, nil
}
// banNetblock reads value, a netblock such as 203.0.113.0/24, or a
// client's address, which stands for the netblock a ban on that client
// covers. An IPv4-mapped netblock, such as ::ffff:203.0.113.0/120, is
// refused, since a client's address is looked up as IPv4 and a ban on it
// would refuse nothing, and so is a value with a zone.
func (h *handler) banNetblock(value string) (netip.Prefix, error) {
netblock, err := netip.ParsePrefix(value)
if err == nil {
if netblock.Addr().Is4In6() {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errMappedNetblock)
}
return netblock, nil
}
// ParsePrefix refuses a zone, but ParseAddr reads the /48 of
// 2001:db8::1%x/48 as part of the zone.
addr, err := netip.ParseAddr(value)
if err != nil {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errNotNetblock)
}
if addr.Zone() != "" {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errZone)
}
return h.netblock(addr), nil
}
// expiry returns when a ban made at now for duration ends: duration
// later, for a duration as a setting gives one, or zero for permanent.
func expiry(duration string, now time.Time) (time.Time, error) {
if duration == permanent {
return time.Time{}, nil
}
length, err := config.ParseDurationNotOff(duration)
if err != nil {
return time.Time{}, fmt.Errorf("duration %q %w", duration, errNotDuration)
}
return now.Add(length), nil
}
// liftBans answers DELETE BansPath/<client>: it lifts every ban active on
// a netblock the client's address is in, and answers with those bans, or
// with 404 when none is active.
func (rq *request) liftBans() {
client, err := pathAddress(rq.in.URL.Path, BansPath+"/")
if err != nil {
http.Error(rq.out, err.Error(), http.StatusBadRequest)
return
}
lifted := rq.h.ledger.Lift(client, rq.h.now())
if len(lifted) == 0 {
http.Error(rq.out, "no ban is active on "+client.String(), http.StatusNotFound)
return
}
rq.answerBans(lifted)
}
// clientAnswer is the answer to GET ClientsPath<ip>: the client the
// address is, as clients.json holds it, or null when the table of
// clients does not hold it, and the bans on each netblock the address is
// in, as bans.json lists them.
type clientAnswer struct {
Client *ratelimit.Client `json:"client"`
Bans []state.BanEntry `json:"bans"`
}
// showClient answers GET ClientsPath<ip> with what smallwebwaf knows of
// the client: its counters, its history, which holds its country as last
// looked up and its offences, and its bans with their notes.
func (rq *request) showClient() {
addr, err := pathAddress(rq.in.URL.Path, ClientsPath)
if err != nil {
http.Error(rq.out, err.Error(), http.StatusBadRequest)
return
}
answer := clientAnswer{Bans: state.BanEntries(rq.h.ledger.Covering(addr))}
client, seen := rq.h.limiter.Client(clientGroup(addr))
if seen {
answer.Client = &client
}
rq.answerJSON(answer)
}
// pathAddress reads the client's address that follows prefix in path.
func pathAddress(path, prefix string) (netip.Addr, error) {
value := strings.TrimPrefix(path, prefix)
addr, err := netip.ParseAddr(value)
if err != nil {
return netip.Addr{}, fmt.Errorf("%q %w", value, errNotAddress)
}
return addr.Unmap(), nil
}
// answerBans answers with held under bans, as bans.json lists them.
func (rq *request) answerBans(held []bans.Ban) {
rq.answerJSON(struct {
Bans []state.BanEntry `json:"bans"`
}{state.BanEntries(held)})
}
// answerJSON answers with value as indented JSON.
func (rq *request) answerJSON(value any) {
body, err := json.MarshalIndent(value, "", " ")
if err != nil {
rq.h.processLog.Error("encoding an answer failed", "error", err.Error())
http.Error(rq.out, http.StatusText(http.StatusInternalServerError),
http.StatusInternalServerError)
return
}
rq.out.Header().Set("Content-Type", "application/json")
_, _ = rq.out.Write(append(body, '\n'))
}
internal/proxy/admin_test.go
-535
View File
@@ -1,535 +0,0 @@
package proxy_test
import (
"encoding/json"
"net/http"
"net/netip"
"slices"
"strconv"
"strings"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/state"
)
const (
adminToken = "SWWAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
// adminSecret is the SWWAF_ADMIN_TOKEN the tests set, and adminBearer
// how a request carries it.
adminSecret = "fedcba9876543210fedcba9876543210"
adminBearer = "Bearer " + adminSecret
// adminClient is the client the tests' admin sends its requests from.
adminClient = "192.0.2.10"
// banOtherClient is the body of a request to ban otherClient for an
// hour.
banOtherClient = `{"netblock": "` + otherClient + `", "duration": "1h", ` +
`"reason": "probes for logins"}`
)
func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
t.Parallel()
// The metrics token is set, and opens none of them.
s, clk, server := startWithClock(t, "", map[string]string{metricsToken: token})
server.Ledger.BanForLimit(netip.MustParsePrefix(otherClient+"/32"), clk.Now(),
bans.Notes{})
before := server.Ledger.Snapshot()
// An empty token does not match the unset one either.
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
for _, e := range adminEndpoints() {
s.adminRequest(adminClient, authorization, e.method, e.path, e.body,
http.StatusNotFound, requestlog.ActionAdmin)
}
}
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) {
t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
}
}
func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
metricsToken: token,
})
// Listing the bans, banning otherClient, lifting that ban, and asking
// about otherClient, in that order. Without the admin token, with the
// metrics token, or with one that differs, each is refused, and
// changes nothing; with the admin token, it is answered.
for _, e := range adminEndpoints() {
before := server.Ledger.Snapshot()
for _, authorization := range []string{
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
} {
got := s.adminRequest(adminClient, authorization, e.method, e.path, e.body,
http.StatusUnauthorized, requestlog.ActionAdmin)
if got.header.Get("WWW-Authenticate") != "Bearer" {
t.Errorf("%s %s with %q was answered without WWW-Authenticate: Bearer",
e.method, e.path, authorization)
}
}
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) {
t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
e.method, e.path, after, before)
}
got := s.admin(e.method, e.path, e.body, http.StatusOK)
if got.header.Get("Content-Type") != "application/json" {
t.Errorf("%s %s answered %q", e.method, e.path, got.header.Get("Content-Type"))
}
}
// Any other request under /_smallwebwaf/ is not found.
for _, e := range []adminEndpoint{
{http.MethodPut, proxy.BansPath, banOtherClient},
{http.MethodDelete, proxy.BansPath, ""},
{http.MethodGet, proxy.BansPath + "/" + otherClient, ""},
{http.MethodPost, proxy.ClientsPath + otherClient, ""},
{http.MethodGet, strings.TrimSuffix(proxy.ClientsPath, "/"), ""},
} {
s.admin(e.method, e.path, e.body, http.StatusNotFound)
}
}
func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
t.Parallel()
s, clk, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
banScopeV4Prefix: "24",
})
// A ban on otherClient bans the /24 a ban on that client covers, so it
// refuses client too, for an hour.
start := clk.Now()
expires := start.Add(time.Hour)
want := state.BanEntry{
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
Start: start,
Expires: &expires,
Cause: bans.CauseAdmin,
Reason: "probes for logins",
}
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK),
want)
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
if line.BanExpires != requestlog.FormatTime(expires) {
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
}
// Its notes count the request it refused.
want.Notes.Requests, want.Notes.Refused = 1, 1
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want)
// Ten minutes on, lifting the bans on client lifts that one, which is
// kept, marked lifted.
clk.advance(10 * time.Minute)
lifted := clk.Now()
want.Lifted = &lifted
wantBans(t, s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "",
http.StatusOK), want)
s.get(client, http.StatusOK, requestlog.ActionForward)
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want)
// No ban on it is active any more.
s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "", http.StatusNotFound)
}
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
t.Parallel()
s, clk, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
banScopeV4Prefix: "24",
})
start := clk.Now()
for _, tc := range []struct {
netblock, duration string
want string
length time.Duration // 0 for a permanent ban
}{
// An address stands for the netblock a ban on that client covers.
{client, "7d", "203.0.113.0/24", 7 * 24 * time.Hour},
{"::ffff:198.51.100.7", "90m", "198.51.100.0/24", 90 * time.Minute},
{"2001:db8:5::1", "permanent", "2001:db8:5::/64", 0},
// A netblock stands for itself, its bits past its length cleared.
{"198.51.100.7/16", "1h", "198.51.0.0/16", time.Hour},
{"2001:db8:6::/48", "1h", "2001:db8:6::/48", time.Hour},
} {
// Whitespace may follow the object.
body := `{"netblock": "` + tc.netblock + `", "duration": "` + tc.duration + `"}` +
"\r\n"
want := state.BanEntry{
Netblock: netip.MustParsePrefix(tc.want), Start: start, Cause: bans.CauseAdmin,
}
if tc.length != 0 {
expires := start.Add(tc.length)
want.Expires = &expires
}
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, body, http.StatusOK), want)
}
}
func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{adminToken: adminSecret})
for _, tc := range []struct{ body, want string }{
{"", "the body is not a JSON object of netblock, duration and reason: EOF"},
{"netblock=203.0.113.9", "the body is not a JSON object"},
{
`{"netblock": "203.0.113.9", "duration": "1h", "until": "2027"}`,
`unknown field "until"`,
},
{
`{"netblock": "203.0.113", "duration": "1h"}`,
`netblock "203.0.113" is not an address or a netblock`,
},
// A client's address is looked up as IPv4, so a ban on an
// IPv4-mapped netblock would refuse nothing.
{
`{"netblock": "::ffff:203.0.113.0/120", "duration": "1h"}`,
`netblock "::ffff:203.0.113.0/120" is IPv4-mapped`,
},
// Read as an address, its zone would be "x/48", and its ban on the
// /64 around it.
{
`{"netblock": "2001:db8::1%x/48", "duration": "1h"}`,
`netblock "2001:db8::1%x/48" has a zone`,
},
{
`{"netblock": "fe80::1%eth0", "duration": "1h"}`,
`netblock "fe80::1%eth0" has a zone`,
},
// Anything but whitespace after the object.
{
`{"netblock": "203.0.113.9", "duration": "1h"}` +
`{"netblock": "198.51.100.0/24", "duration": "1h"}`,
"more follows the object",
},
{`{"netblock": "203.0.113.9", "duration": "1h"} x`, "more follows the object"},
{`{"duration": "1h"}`, `netblock "" is not an address or a netblock`},
{`{"netblock": "203.0.113.9"}`, `duration "" is not a duration above zero`},
{
`{"netblock": "203.0.113.9", "duration": "off"}`,
`duration "off" is not a duration above zero`,
},
{
`{"netblock": "203.0.113.9", "duration": "0s"}`,
`duration "0s" is not a duration above zero`,
},
{
`{"netblock": "203.0.113.9", "duration": "forever"}`,
`duration "forever" is not a duration above zero, such as 1h or 7d, ` +
`or permanent`,
},
// Over the 4 KiB read of a body, even when the object comes first.
{
`{"netblock": "203.0.113.9", "duration": "1h", "reason": "` +
strings.Repeat("x", 4<<10) + `"}`,
"request body too large",
},
{
`{"netblock": "203.0.113.9", "duration": "1h"}` + strings.Repeat(" ", 4<<10),
"request body too large",
},
} {
got := s.admin(http.MethodPost, proxy.BansPath, tc.body, http.StatusBadRequest)
if !strings.Contains(string(got.body), tc.want) {
t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
}
}
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
func TestBanToAddOverTheRequestSizeLimitIsRefused(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
requestMaxBytes: "16",
})
// Sent in a chunk, its length is not announced, so that it is found
// over SWWAF_REQUEST_MAX_BYTES only as it is read.
chunk := `{"netblock": "203.0.113.9", "duration": "1h"}`
s.adminRequest(adminClient, adminBearer+"\r\nTransfer-Encoding: chunked",
http.MethodPost, proxy.BansPath,
strconv.FormatInt(int64(len(chunk)), 16)+"\r\n"+chunk+"\r\n0\r\n\r\n",
http.StatusRequestEntityTooLarge, requestlog.ActionTooLarge)
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
func TestBanToAddSlowerThanTheClientRequestTimeoutIsRefused(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
metricsToken: token,
clientRequestTimeout: shortTimeoutSetting,
})
// The chunk announces 256 bytes and the rest of it never comes, so only
// the timeout ends the wait. A hold-up of the test process can only
// make the answer later, so the time is checked only for not being
// shorter than the timeout.
start := time.Now()
s.adminRequest(adminClient, adminBearer+"\r\nTransfer-Encoding: chunked",
http.MethodPost, proxy.BansPath, "100\r\n"+`{"netblock": "203.0.113.9", `,
http.StatusRequestTimeout, requestlog.ActionTimedOut)
if took := time.Since(start); took < shortTimeout {
t.Errorf("answered after %s, before the timeout of %s ran out", took, shortTimeout)
}
wantLimitHits(t, s.addr, clientRequestTimeout, 1)
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
t.Parallel()
s, clk, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
rateLimitPerMinute: "2",
rateLimitExemptNets: adminClient,
})
start := clk.Now()
// Two of otherClient's requests are let through; the third breaks the
// limit of two a minute, and bans it.
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusForbidden, requestlog.ActionRateLimited)
// Asked about by its address in IPv6 form too.
for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
var got struct {
Client *ratelimit.Client `json:"client"`
Bans []state.BanEntry `json:"bans"`
}
decode(t, s.admin(http.MethodGet, proxy.ClientsPath+addr, "", http.StatusOK), &got)
if got.Client == nil {
t.Fatalf("%s: no client", addr)
}
history := got.Client.History
if got.Client.Client != netip.MustParsePrefix(otherClient+"/32") ||
history.Requests != 3 || history.Forwarded != 2 || history.Refused != 1 ||
history.Offences.Limit != 1 || !history.FirstSeen.Equal(start) {
t.Errorf("%s: client %+v", addr, got.Client)
}
if len(got.Bans) != 1 || got.Bans[0].Cause != bans.CauseLimit ||
got.Bans[0].Reason != "requests per minute over the limit of 2" ||
got.Bans[0].Notes.Count != 3 {
t.Errorf("%s: bans %+v, want the one for the broken limit", addr, got.Bans)
}
}
// Of an address no request came from and no ban covers, nothing is
// known.
got := s.admin(http.MethodGet, proxy.ClientsPath+"198.51.100.99", "", http.StatusOK)
if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
t.Errorf("an unknown client is answered\n%s", got.body)
}
s.admin(http.MethodGet, proxy.ClientsPath+"203.0.113", "", http.StatusBadRequest)
s.admin(http.MethodDelete, proxy.BansPath+"/203.0.113.0/24", "",
http.StatusBadRequest)
}
func TestBannedClientIsRefusedAtTheEndpointsEvenWithTheToken(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{adminToken: adminSecret})
s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK)
// otherClient cannot lift its own ban either.
for _, e := range adminEndpoints() {
s.adminRequest(otherClient, adminBearer, e.method, e.path, e.body,
http.StatusForbidden, requestlog.ActionBanned)
}
}
func TestAdminRequestsCountTowardTheLimits(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
rateLimitPerMinute: "2",
})
// A request refused for a missing token and one answered count toward
// the limit of two a minute, so the next breaks it.
s.adminRequest(client, "", http.MethodGet, proxy.BansPath, "",
http.StatusUnauthorized, requestlog.ActionAdmin)
s.adminRequest(client, adminBearer, http.MethodGet, proxy.BansPath, "",
http.StatusOK, requestlog.ActionAdmin)
s.adminRequest(client, adminBearer, http.MethodGet, proxy.BansPath, "",
http.StatusForbidden, requestlog.ActionRateLimited)
}
func TestClientInAllowNetsSkipsTheChecksButNeedsTheToken(t *testing.T) {
t.Parallel()
const allowed = "192.0.2.60" // in SWWAF_ALLOW_NETS
s, clk, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
allowNets: allowed,
rateLimitPerMinute: "1",
})
// A ban on it refuses nothing, and its requests are not counted.
server.Ledger.BanForAdmin(netip.MustParsePrefix(allowed+"/32"), clk.Now(),
time.Time{}, "")
for range 2 {
s.adminRequest(allowed, "", http.MethodGet, proxy.BansPath, "",
http.StatusUnauthorized, requestlog.ActionAdmin)
s.adminRequest(allowed, adminBearer, http.MethodGet, proxy.BansPath, "",
http.StatusOK, requestlog.ActionAdmin)
}
}
func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
mode: observe,
})
for _, e := range adminEndpoints() {
s.adminRequest(adminClient, "", e.method, e.path, e.body,
http.StatusUnauthorized, requestlog.ActionAdmin)
}
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
// adminEndpoint is a request to an endpoint SWWAF_ADMIN_TOKEN opens.
type adminEndpoint struct {
method, path, body string
}
// adminEndpoints returns a request to each endpoint SWWAF_ADMIN_TOKEN
// opens: listing the bans, banning otherClient for an hour, lifting the
// bans on otherClient, and asking about otherClient.
func adminEndpoints() []adminEndpoint {
return []adminEndpoint{
{http.MethodGet, proxy.BansPath, ""},
{http.MethodPost, proxy.BansPath, banOtherClient},
{http.MethodDelete, proxy.BansPath + "/" + otherClient, ""},
{http.MethodGet, proxy.ClientsPath + otherClient, ""},
}
}
// admin sends a request with method for path, with body, from
// adminClient, with the admin token, and checks that it is answered with
// status, its log line's action admin. It returns the answer.
func (s *sender) admin(method, path, body string, status int) answer {
s.t.Helper()
return s.adminRequest(adminClient, adminBearer, method, path, body, status,
requestlog.ActionAdmin)
}
// adminRequest sends a request with method for path, with body, from the
// client at from, with authorization as its Authorization header unless
// it is "", and checks its answer's status and its log line's action, as
// request does. authorization may end in more header lines. A body that
// is not "" has its length announced, unless authorization names
// Transfer-Encoding. It returns the answer.
func (s *sender) adminRequest(
from, authorization, method, path, body string, status int, action string,
) answer {
s.t.Helper()
var header []string
if authorization != "" {
header = append(header, "Authorization: "+authorization)
}
if body != "" && !strings.Contains(authorization, "Transfer-Encoding") {
header = append(header, "Content-Length: "+strconv.Itoa(len(body)))
}
_, got := s.requestWithBody(method, from, path, strings.Join(header, "\r\n"),
body, status, action)
return got
}
// wantBans checks that a ban endpoint answered with want, and no other
// ban.
func wantBans(t *testing.T, got answer, want ...state.BanEntry) {
t.Helper()
var decoded struct {
Bans []state.BanEntry `json:"bans"`
}
decode(t, got, &decoded)
gotJSON, err := json.Marshal(decoded.Bans)
if err != nil {
t.Fatalf("encode %+v: %v", decoded.Bans, err)
}
wantJSON, err := json.Marshal(want)
if err != nil {
t.Fatalf("encode %+v: %v", want, err)
}
if string(gotJSON) != string(wantJSON) {
t.Errorf("bans\n%s\nwant\n%s", gotJSON, wantJSON)
}
}
// decode reads the JSON answer of an endpoint into value.
func decode(t *testing.T, got answer, value any) {
t.Helper()
err := json.Unmarshal(got.body, value)
if err != nil {
t.Fatalf("decode %s: %v", got.body, err)
}
}
internal/proxy/alerts_test.go
-273
View File
@@ -1,273 +0,0 @@
package proxy_test
import (
"maps"
"net/http"
"net/netip"
"reflect"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
const (
alertWebhookURL = "SWWAF_ALERT_WEBHOOK_URL"
alertMaxPerHour = "SWWAF_ALERT_MAX_PER_HOUR"
// alertInstance is the instance every alert of these tests gives.
alertInstance = "fsn1app1/gitea"
)
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
t.Parallel()
s, clk, server, queue := startWithAlerts(t, map[string]string{
rateLimitPerMinute: "1",
banScopeV4Prefix: "24",
})
start := clk.Now()
s.get(client, http.StatusOK, requestlog.ActionForward)
s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
netblock := netip.MustParsePrefix("203.0.113.0/24")
ban := server.Ledger.Bans(netblock)[0]
// A request refused under the ban raises no other alert.
clk.advance(time.Minute)
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
wantAlerts(t, queue, banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netblock, Cause: bans.CauseLimit,
Reason: "requests per minute over the limit of 1", Notes: ban.Notes,
}, requestlog.FormatTime(start.Add(time.Hour))))
if ban.Notes.Limit != 1 || ban.Notes.Request.Path != "/" {
t.Errorf("the alert's notes are %+v, want those of the broken limit", ban.Notes)
}
}
func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
t.Parallel()
s, clk, server, queue := startWithAlerts(t, map[string]string{
rulesDir: writeRules(t, testRules),
})
start := clk.Now()
netblock := netip.MustParsePrefix(client + "/32")
other := netip.MustParsePrefix(otherClient + "/32")
// The probe bans the client for seven days, and its next request makes
// the ban permanent. The request after that changes nothing.
s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
attackBan := server.Ledger.Bans(netblock)[0]
clk.advance(time.Minute)
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
permanentBan := server.Ledger.Bans(netblock)[0]
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
// Another client's probe after its first ban has run out without a
// request makes a permanent ban at once.
s.request(otherClient, "/.env", http.StatusForbidden, requestlog.ActionBanned)
clk.advance(7 * 24 * time.Hour)
s.request(otherClient, "/.env", http.StatusForbidden, requestlog.ActionBanned)
otherBans := server.Ledger.Bans(other)
wantAlerts(t, queue,
attackAlert(alerts.EventBan, start, client, attackBan,
requestlog.FormatTime(start.Add(7*24*time.Hour))),
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute), client,
permanentBan, "permanent"),
attackAlert(alerts.EventBan, start.Add(time.Minute), otherClient, otherBans[0],
requestlog.FormatTime(start.Add(time.Minute+7*24*time.Hour))),
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute+7*24*time.Hour),
otherClient, otherBans[1], "permanent"),
)
}
func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
t.Parallel()
s, clk, server, queue := startWithAlerts(t, map[string]string{
mode: observe,
rateLimitPerMinute: "2",
rulesDir: writeRules(t, testRules),
})
start := clk.Now()
// A ban for a clear sign of attack, which a request under it would make
// permanent.
group := netip.MustParsePrefix(ipv6Group)
attackBan, _ := server.Ledger.BanForAttack(group, start, bans.Notes{RuleID: "probe"})
// The third request breaks the limit, and so does the fourth, within the
// cooldown, which raises nothing. The probe is a clear sign of attack.
for range 4 {
s.get(client, http.StatusOK, requestlog.ActionForward)
}
s.request(otherClient, "/.env", http.StatusOK, requestlog.ActionForward)
line := s.get(ipv6Client, http.StatusOK, requestlog.ActionForward)
// No ban is made, and none made permanent.
if held := server.Ledger.Snapshot(); len(held) != 1 || held[0] != attackBan ||
line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+
"for the attack alone, as it was", held, line.BanExpires)
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 3 || queue.Suppressed() != 0 {
t.Fatalf("%d alerts wait and %d are held back, want 3 and 0: %+v",
len(waiting), queue.Suppressed(), waiting)
}
limitNotes, _ := waiting[0].Detail["notes"].(bans.Notes)
attackNotes, _ := waiting[1].Detail["notes"].(bans.Notes)
if limitNotes.Limit != 2 || limitNotes.Request.Path != "/" ||
attackNotes.Request.Path != "/.env" {
t.Errorf("the notes are %+v and %+v, want those of the broken limit and "+
"of the probe", limitNotes, attackNotes)
}
// Each alert is the one enforce mode would have raised, with mode
// observe in its detail.
want := []alerts.Alert{
banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
Reason: "requests per minute over the limit of 2", Notes: limitNotes,
}, requestlog.FormatTime(start.Add(time.Hour))),
attackAlert(alerts.EventBan, start, otherClient, bans.Ban{
Netblock: netip.MustParsePrefix(otherClient + "/32"), Notes: attackNotes,
}, requestlog.FormatTime(start.Add(7*24*time.Hour))),
attackAlert(alerts.EventPermanentBan, start, ipv6Client, attackBan, permanent),
}
for _, alert := range want {
alert.Detail["mode"] = observe
}
wantAlerts(t, queue, want...)
}
func TestObserveModeWorksOutABanOnlyWhenItsAlertWouldBeSent(t *testing.T) {
t.Parallel()
s, _, _, queue := startWithAlerts(t, map[string]string{
mode: observe,
rateLimitPerMinute: "2",
rulesDir: writeRules(t, testRules),
alertMaxPerHour: "2",
})
// The client's third request breaks the limit, and raises the first
// alert of the hour. Its fourth is within the cooldown.
for range 4 {
s.get(client, http.StatusOK, requestlog.ActionForward)
}
// The other client's first probe raises the second. Its second probe is
// within the cooldown.
for range 2 {
s.request(otherClient, "/.env", http.StatusOK, requestlog.ActionForward)
}
// The IPv6 client's third request breaks the limit past the two alerts
// an hour.
for range 3 {
s.get(ipv6Client, http.StatusOK, requestlog.ActionForward)
}
// Had the ban been worked out for any of the requests within the
// cooldown or past the two an hour, its alert would have been raised,
// held back and counted.
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 2 || queue.Suppressed() != 0 {
t.Errorf("%d alerts wait and %d are held back, want 2 and 0: %+v",
len(waiting), queue.Suppressed(), waiting)
}
}
// startWithAlerts is startWithClock with alerts to a webhook, which is
// never sent them, and returns the queue they wait in as well.
func startWithAlerts(
t *testing.T, env map[string]string,
) (*sender, *clock, *proxy.Server, *alerts.Queue) {
t.Helper()
return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env)
}
// startAppWithAlerts is startWithAlerts in front of the app handler.
func startAppWithAlerts(
t *testing.T, handler http.HandlerFunc, env map[string]string,
) (*sender, *clock, *proxy.Server, *alerts.Queue) {
t.Helper()
app := startApp(t, handler)
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
settings := map[string]string{
trustedProxies: trustLocalhost,
alertWebhookURL: "https://alerts.example/smallwebwaf",
instanceName: alertInstance,
}
maps.Copy(settings, env)
addr, out, server, queue := startProxyWithAlerts(t, app.URL, "", clk.Now, settings)
return &sender{t: t, addr: addr, out: out}, clk, server, queue
}
// banAlert returns the alert for event, raised by a request from client at
// the time raised, for ban, with its netblock, cause, reason and notes,
// which ends at expires, as the log line gives it.
func banAlert(
event string, raised time.Time, client string, ban bans.Ban, expires string,
) alerts.Alert {
return alerts.Alert{
Instance: alertInstance,
Time: raised,
Event: event,
Client: netip.MustParseAddr(client),
Netblock: ban.Netblock,
Reason: ban.Reason,
Detail: map[string]any{
"cause": ban.Cause, "ban_expires": expires, "notes": ban.Notes,
},
}
}
// attackAlert is banAlert for a ban for the probe rule of testRules, with
// the netblock and the notes of ban.
func attackAlert(
event string, raised time.Time, client string, ban bans.Ban, expires string,
) alerts.Alert {
return banAlert(event, raised, client, bans.Ban{
Netblock: ban.Netblock, Cause: bans.CauseAttack, Reason: "matched the rule probe",
Notes: ban.Notes,
}, expires)
}
// wantAlerts checks the alerts waiting in queue, in order.
func wantAlerts(t *testing.T, queue *alerts.Queue, want ...alerts.Alert) {
t.Helper()
got := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(got) != len(want) {
t.Fatalf("%d alerts wait, want %d: %+v", len(got), len(want), got)
}
for i := range want {
if !reflect.DeepEqual(got[i], want[i]) {
t.Errorf("alert %d is\n%+v\nwant\n%+v", i, got[i], want[i])
}
}
}
internal/proxy/anomaly_internal_test.go
-30
View File
@@ -1,30 +0,0 @@
package proxy
import (
"net/netip"
"testing"
"sneak.berlin/go/smallwebwaf/internal/config"
)
func TestWithEveryAnomalyThresholdOffARequestIsNotCounted(t *testing.T) {
t.Parallel()
// A request from a client looked up through GeoJS, with every anomaly
// threshold off. Its handler has neither GeoJS's answers nor the
// anomaly counters, nor a clock, and the request no response: reading
// any of them to count the request panics.
rq := &request{
h: &handler{config: &config.Config{LookupSource: "geojs"}},
client: netip.MustParseAddr("203.0.113.9"),
lookedUp: true,
}
defer func() {
if r := recover(); r != nil {
t.Errorf("counting the request did work, with every threshold off: %v", r)
}
}()
rq.countAnomalies()
}
internal/proxy/anomaly_test.go
-377
View File
@@ -1,377 +0,0 @@
package proxy_test
import (
"maps"
"net/http"
"net/netip"
"reflect"
"slices"
"strconv"
"sync/atomic"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// The anomaly thresholds: the prefix of a scope followed by the end of a
// count.
const (
anomalyClient = "SWWAF_ANOMALY_CLIENT_"
anomalyNet = "SWWAF_ANOMALY_NET_"
anomalyASN = "SWWAF_ANOMALY_ASN_"
anomalyTotal = "SWWAF_ANOMALY_TOTAL_"
anomalyWatch = "SWWAF_WATCH_"
requestsPerMinute = "REQUESTS_PER_MINUTE"
requestsPerHour = "REQUESTS_PER_HOUR"
bytesPerMinute = "BYTES_PER_MINUTE"
bytesPerHour = "BYTES_PER_HOUR"
)
// The other anomaly settings.
const (
anomalyNetV4Prefix = "SWWAF_ANOMALY_NET_V4_PREFIX"
anomalyNetV6Prefix = "SWWAF_ANOMALY_NET_V6_PREFIX"
watchNets = "SWWAF_WATCH_NETS"
)
const (
// clientsNet is the netblock around client at the default length, and
// office a named netblock of the same.
clientsNet = "203.0.113.0/24"
office = "office=" + clientsNet
// aLot is a threshold no test reaches.
aLot = "1000"
// hour is the window an alert names for a threshold per hour.
hour = "hour"
)
func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(t *testing.T) {
t.Parallel()
for _, scope := range []struct {
prefix, scope string
// netblock is the alert's, and counted what its reason names. extra
// is what its detail gives besides what every anomaly alert's does.
netblock netip.Prefix
counted string
extra map[string]any
}{
{
anomalyClient, anomaly.ScopeClient, netip.MustParsePrefix(client + "/32"),
"the client " + client + "/32", nil,
},
{
anomalyNet, anomaly.ScopeNet, netip.MustParsePrefix(clientsNet),
"the netblock " + clientsNet, nil,
},
{anomalyASN, anomaly.ScopeASN, netip.Prefix{}, asnDE, map[string]any{"asn": asnDE}},
{anomalyTotal, anomaly.ScopeTotal, netip.Prefix{}, "the whole service", nil},
{
anomalyWatch, anomaly.ScopeWatch, netip.MustParsePrefix(clientsNet),
"the named netblock office, " + clientsNet, map[string]any{"name": "office"},
},
} {
for _, threshold := range []struct {
end, kind, window string
// value is the threshold, which the third upload of 100 bytes
// takes the count over, to count.
value int64
count float64
}{
{requestsPerMinute, ratelimit.KindRequests, minute, 2, 3},
{requestsPerHour, ratelimit.KindRequests, hour, 2, 3},
{bytesPerMinute, ratelimit.KindBytes, minute, 250, 300},
{bytesPerHour, ratelimit.KindBytes, hour, 250, 300},
} {
setting := scope.prefix + threshold.end
value := strconv.FormatInt(threshold.value, 10)
t.Run(setting, func(t *testing.T) {
t.Parallel()
s, clk, server, queue := startWithLookupsAndClock(t, map[string]string{
setting: value, watchNets: office,
})
start := clk.Now()
// The third upload takes the count over the threshold, and the
// fourth, within the cooldown, is held back. Each is passed to
// the app.
for range 4 {
s.uploadFrom(client)
}
detail := map[string]any{
"scope": scope.scope, "window": threshold.window, "kind": threshold.kind,
"count": threshold.count, "threshold": threshold.value,
}
maps.Copy(detail, scope.extra)
wantAlerts(t, queue, alerts.Alert{
Instance: alertInstance,
Time: start,
Event: alerts.EventAnomaly,
Client: netip.MustParseAddr(client),
Netblock: scope.netblock,
ASN: asnDE,
ASName: asNameDE,
Country: "DE",
Reason: threshold.kind + " per " + threshold.window + " of " +
scope.counted + " over the threshold of " + value,
Detail: detail,
})
wantAlertedAgainOnceTheCooldownHasRunOut(t, s, clk, queue)
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
})
}
}
}
// wantAlertedAgainOnceTheCooldownHasRunOut checks that, once the cooldown
// has run out after a first alert, which held back one repeat, the next
// count over the threshold, at the latest three uploads from client on,
// raises another alert, giving that repeat.
func wantAlertedAgainOnceTheCooldownHasRunOut(
t *testing.T, s *sender, clk *clock, queue *alerts.Queue,
) {
t.Helper()
clk.advance(15 * time.Minute)
for range 3 {
s.uploadFrom(client)
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 2 || !waiting[1].Time.Equal(clk.Now()) ||
waiting[1].SuppressedRepeats != 1 {
t.Errorf("alerts wait %+v, want the first and another, with 1 repeat", waiting)
}
}
func TestEveryRequestIsCountedWhateverIsDoneWithIt(t *testing.T) {
t.Parallel()
const (
allowed = "192.0.2.7" // in SWWAF_ALLOW_NETS
exempt = "192.0.2.10" // in SWWAF_RATE_LIMIT_EXEMPT_NETS
denied = "192.0.2.20" // in SWWAF_DENY_NETS
)
s, _, _, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
anomalyClient + requestsPerMinute: "2",
allowNets: allowed,
rateLimitExemptNets: exempt,
rateLimitExemptPaths: "/static/",
denyNets: denied,
})
// The third request of each takes its client's count over the threshold
// of 2.
for _, sent := range []struct {
from, path string
status int
action string
}{
{allowed, "/", http.StatusOK, requestlog.ActionForward},
{exempt, "/", http.StatusOK, requestlog.ActionForward},
{client, "/static/app.js", http.StatusOK, requestlog.ActionForward},
{denied, "/", http.StatusForbidden, requestlog.ActionDenied},
} {
for range 3 {
s.request(sent.from, sent.path, sent.status, sent.action)
}
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
got := make([]string, 0, len(waiting))
for _, alert := range waiting {
got = append(got, alert.Client.String())
}
if want := []string{allowed, exempt, client, denied}; !slices.Equal(got, want) {
t.Errorf("alerts for the clients %v, want %v", got, want)
}
}
func TestThresholdsOffCountNothingAndAlertNothing(t *testing.T) {
t.Parallel()
// With every threshold off, nothing is counted.
s, server, queue := startWithLookups(t, map[string]string{watchNets: office})
for range 5 {
s.uploadFrom(client)
}
if counters := server.Anomalies.Snapshot(); len(counters) != 0 {
t.Errorf("counters %+v, want none", counters)
}
wantAlerts(t, queue)
// With one set, its count alone is counted, in its scope alone.
s, clk, server, queue := startWithLookupsAndClock(t, map[string]string{
anomalyNet + requestsPerMinute: aLot, watchNets: office,
})
for range 5 {
s.uploadFrom(client)
}
want := []anomaly.Counter{{
Scope: anomaly.ScopeNet,
Netblock: netip.MustParsePrefix(clientsNet),
Minute: ratelimit.Buckets{Start: clk.Now(), Current: 5},
}}
if got := server.Anomalies.Snapshot(); !reflect.DeepEqual(got, want) {
t.Errorf("counters\n%+v\nwant\n%+v", got, want)
}
wantAlerts(t, queue)
}
func TestNetblockAroundAClientIsAsLongAsTheSettingsSay(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
env map[string]string
// Each client of sent sends one request, and want gives the
// netblocks they are counted in, each with its requests.
sent []string
want map[string]int64
}{
{
"by default", nil,
[]string{client, "203.0.113.200", "192.0.2.7", ipv6Client, "2001:db8:0:ffff::1"},
map[string]int64{clientsNet: 2, "192.0.2.0/24": 1, "2001:db8::/48": 2},
},
{
"as set", map[string]string{anomalyNetV4Prefix: "16", anomalyNetV6Prefix: "32"},
[]string{client, "203.0.200.1", ipv6Client, "2001:db8:ffff::1"},
map[string]int64{"203.0.0.0/16": 2, "2001:db8::/32": 2},
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
env := map[string]string{anomalyNet + requestsPerMinute: aLot}
maps.Copy(env, tc.env)
s, _, server, _ := startAppWithAlerts(t, readAndAnswer, env)
for _, from := range tc.sent {
s.get(from, http.StatusOK, requestlog.ActionForward)
}
got := map[string]int64{}
for _, counter := range server.Anomalies.Snapshot() {
got[counter.Netblock.String()] = counter.Minute.Current
}
if !maps.Equal(got, tc.want) {
t.Errorf("requests by netblock %v, want %v", got, tc.want)
}
})
}
}
func TestClientIsCountedForItsASNumberOnceTheLookupGivesOne(t *testing.T) {
t.Parallel()
s, server, _ := startWithLookups(t, map[string]string{
anomalyASN + requestsPerMinute: aLot,
})
// The lookup database does not hold unplaced.
for _, from := range []string{fromDE, fromDE, fromKP, noCountry, unplaced} {
s.uploadFrom(from)
}
got := map[string]int64{}
for _, counter := range server.Anomalies.Snapshot() {
got[counter.ASN] = counter.Minute.Current
}
if want := map[string]int64{asnDE: 2, asnKP: 1, "AS64500": 1}; !maps.Equal(got, want) {
t.Errorf("requests by AS number %v, want %v", got, want)
}
}
func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
t.Parallel()
// The stand-in for GeoJS answers only once released, which the app
// does as it answers the request, and then waits until the answer is
// kept.
geojsURL, _, release := startHeldGeoJS(t)
var server atomic.Pointer[proxy.Server]
app := startApp(t, func(http.ResponseWriter, *http.Request) {
release()
waitUntil(func() bool {
_, kept := server.Load().GeoJS.Kept(netip.MustParsePrefix(fromDE + "/32"))
return kept
})
})
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
addr, out, started := startProxyWithClock(t, app.URL, geojsURL, clk.Now,
map[string]string{
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
anomalyASN + requestsPerMinute: aLot,
})
server.Store(started)
// The request went on without the answer, and is counted for the AS
// number it gives.
s := &sender{t: t, addr: addr, out: out}
if line := s.get(fromDE, http.StatusOK, requestlog.ActionForward); line.ASN != "" {
t.Errorf("log line has AS number %q, want none: the request waited", line.ASN)
}
want := []anomaly.Counter{{
Scope: anomaly.ScopeASN, ASN: asnDE,
Minute: ratelimit.Buckets{Start: clk.Now(), Current: 1},
}}
if got := started.Anomalies.Snapshot(); !reflect.DeepEqual(got, want) {
t.Errorf("counters\n%+v\nwant\n%+v", got, want)
}
}
func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
t.Parallel()
s, _, server, _ := startAppWithAlerts(t, readAndAnswer, map[string]string{
anomalyWatch + requestsPerMinute: aLot,
watchNets: office + ",wide=203.0.0.0/16,other=198.51.100.0/25",
})
// client is in office and in wide.
for _, from := range []string{client, "203.0.200.1", "192.0.2.7"} {
s.get(from, http.StatusOK, requestlog.ActionForward)
}
got := map[string]int64{}
for _, counter := range server.Anomalies.Snapshot() {
got[counter.Name] = counter.Minute.Current
}
if want := map[string]int64{"office": 1, "wide": 2}; !maps.Equal(got, want) {
t.Errorf("requests by named netblock %v, want %v", got, want)
}
}
internal/proxy/bans.go
+34 -202
View File
@@ -4,9 +4,7 @@ import (
"net/netip"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
)
@@ -18,244 +16,78 @@ func (rq *request) banResponse(action string) *refusal {
}
// banned reports whether a ban on a netblock the client is in covers the
// request at now, and notes for the log line when that ban ends. A
// request that makes the ban permanent, or in observe mode would have,
// raises the alert for it.
// request at now, and notes for the log line when that ban ends.
func (rq *request) banned(now time.Time) bool {
check := rq.h.ledger.Check
if rq.h.config.Observe {
check = rq.h.ledger.Find // the ban refuses nothing, and stays as it is
check = rq.h.ledger.Find // in observe mode the ban refuses nothing
}
ban, banned, madePermanent := check(rq.client, now)
ban, banned := check(rq.client, now)
if banned {
rq.line.BanExpires = banExpires(ban)
}
if madePermanent {
ban.Expires = time.Time{} // the ban made permanent, which Find leaves as it is
rq.alertBan(ban)
}
return banned
}
// limitBroken counts the request for the rate limits at now, notes the
// client's counts for the log line, and reports whether the request takes
// the client over a rate limit, as its limit percentage lowers it, which
// breaks it.
// limitBroken counts the request for the rate limits at now, and reports
// whether it takes the client over one. In enforce mode such a request
// bans the client's netblock, and sets the client's counters back to
// zero; in observe mode it does neither.
func (rq *request) limitBroken(now time.Time) bool {
counts, hit, over := rq.h.limiter.Count(clientGroup(rq.client), now,
rq.limitPercent.percent)
rq.line.Counts = counts
group := clientGroup(rq.client)
if over {
rq.banForLimit(now, hit, rq.h.config.BanResponse)
hit, over := rq.h.limiter.Count(group, now)
if !over {
return false
}
return over
}
// countBytes counts the request's bytes, as countedBytes gives them, for
// the byte limits, once its response has ended, and notes the client's
// byte totals for the log line; its requests stay there as the rate limits
// counted them. Only a request passed to the app has them counted, and
// only one the rate limits counted; in observe mode, not one that enforce
// mode would have refused. Bytes that take the client over a byte limit,
// as its limit percentage for the byte limits lowers it, break it; the
// response was passed on whole.
func (rq *request) countBytes() {
if !rq.counted || rq.line.WouldAction != "" {
return
}
now := rq.h.now()
counts, hit, over := rq.h.limiter.CountBytes(clientGroup(rq.client), now,
rq.countedBytes(), rq.bytesPercent.percent)
rq.line.Counts.MinuteBytes = counts.MinuteBytes
rq.line.Counts.HourBytes = counts.HourBytes
rq.line.Counts.DayBytes = counts.DayBytes
if over {
rq.banForLimit(now, hit, rq.out.status)
}
}
// countedBytes returns the request's bytes, once it has ended, as the
// byte limits and the anomaly thresholds count them: the response's body
// bytes, the request's, or both, as SWWAF_BYTES_COUNT says. For an
// upgraded connection, such as a WebSocket, which has closed by then, what
// it carried from the app counts with the response's and what it carried
// from the client with the request's.
func (rq *request) countedBytes() int64 {
response, request := rq.out.bytes, rq.requestBytes()
if rq.upgraded != nil {
response += rq.upgraded.fromApp.Load()
request += rq.upgraded.toApp.Load()
}
switch rq.h.config.BytesCount {
case "response":
return response
case "request":
return request
default: // both
return response + request
}
}
// banForLimit bans the client's netblock at now for a broken limit, the
// one hit names, and notes the offence for the log line. status is what
// the client was sent, or is sent: SWWAF_BAN_RESPONSE for a request over
// a rate limit, the app's answer for one whose bytes broke a byte limit.
// The ban's notes give the client's limit percentage for that kind of
// limit. The ban sets the client's counters back to zero. In observe mode
// it makes no ban and sets nothing back, and raises the alert for the ban
// it would have made, if that alert would be sent.
func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
rq.line.LimitHit = hit.Window
if hit.Kind == ratelimit.KindBytes {
rq.line.LimitHit += "_bytes" // as counts names the byte totals
}
rq.line.Offence = requestlog.OffenceLimit
netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseLimit) {
return
if rq.h.config.Observe {
return true
}
notes := bans.Notes{
ASN: rq.line.ASN,
ASName: rq.line.ASName,
netblock := rq.netblock()
ban := rq.h.ledger.BanForLimit(netblock, now, bans.Notes{
Country: rq.line.Country,
Kind: hit.Kind,
Limit: hit.Limit,
Window: hit.Window,
Count: hit.Count,
Request: rq.noted(now, status),
Count: hit.Requests,
Request: rq.noted(now),
Requests: rq.netblockRequests(netblock),
}
percent := rq.limitPercent
if hit.Kind == ratelimit.KindBytes {
percent = rq.bytesPercent
}
notes.LimitPercent, notes.LimitPercentSetting = percent.logged()
if rq.h.config.Observe {
ban, wouldBan := rq.h.ledger.WouldBanForLimit(netblock, now, notes)
if wouldBan {
rq.alertBan(ban)
}
return
}
ban, made := rq.h.ledger.BanForLimit(netblock, now, notes)
rq.h.limiter.Reset(clientGroup(rq.client))
})
rq.h.limiter.Reset(group)
rq.line.BanExpires = banExpires(ban)
if made {
rq.alertBan(ban)
}
return true
}
// banForAttack bans the client's netblock at now for a clear sign of
// attack, the match of rule, a ban rule. In observe mode it makes no ban,
// and raises the alert for the ban it would have made, if that alert
// would be sent.
// attack, the match of rule, a ban rule.
func (rq *request) banForAttack(now time.Time, rule rules.Rule) {
netblock := rq.h.netblock(rq.client)
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
return
}
notes := bans.Notes{
ASN: rq.line.ASN,
ASName: rq.line.ASName,
netblock := rq.netblock()
ban := rq.h.ledger.BanForAttack(netblock, now, bans.Notes{
Country: rq.line.Country,
RuleID: rule.ID,
Target: rule.Target,
Request: rq.noted(now, rq.h.config.BanResponse),
Request: rq.noted(now),
Requests: rq.netblockRequests(netblock),
}
if rq.h.config.Observe {
ban, wouldBan := rq.h.ledger.WouldBanForAttack(netblock, now, notes)
if wouldBan {
rq.alertBan(ban)
}
return
}
ban, made := rq.h.ledger.BanForAttack(netblock, now, notes)
rq.line.BanExpires = banExpires(ban)
if made {
rq.alertBan(ban)
}
}
// wouldAlertBan reports whether the alert for a ban on netblock for cause
// made at now would be sent. In observe mode the ban the request would
// have made is worked out only then, at most once per
// SWWAF_ALERT_COOLDOWN and never with no webhook set: its notes count the
// netblock's requests, which can mean going through every client.
func (rq *request) wouldAlertBan(
netblock netip.Prefix, now time.Time, cause string,
) bool {
event := alerts.EventBan
if rq.h.ledger.WouldBePermanent(netblock, now, cause) {
event = alerts.EventPermanentBan
}
return rq.h.alerts.WouldSend(event, netblock)
}
// alertBan raises the alert for ban, which the request made, or made
// permanent: permanent_ban for a permanent ban, ban for another. Its
// detail gives the ban's cause, when it ends, and its notes, and in
// observe mode, where ban is the ban that would have been made, or made
// permanent, mode, observe.
func (rq *request) alertBan(ban bans.Ban) {
event := alerts.EventBan
if ban.Permanent() {
event = alerts.EventPermanentBan
}
detail := map[string]any{
"cause": ban.Cause, "ban_expires": banExpires(ban), "notes": ban.Notes,
}
if rq.h.config.Observe {
detail["mode"] = "observe"
}
rq.h.alerts.Raise(alerts.Alert{
Event: event,
Client: rq.client,
Netblock: ban.Netblock,
ASN: ban.Notes.ASN,
ASName: ban.Notes.ASName,
Country: ban.Notes.Country,
Reason: ban.Reason,
Detail: detail,
})
rq.line.BanExpires = banExpires(ban)
}
// noted is the request, at now, with status, what the client was sent, or
// in observe mode would have been, as the notes of the ban it makes keep
// it.
func (rq *request) noted(now time.Time, status int) bans.Request {
// noted is the request, refused at now with SWWAF_BAN_RESPONSE, as the
// notes of the ban it makes keep it.
func (rq *request) noted(now time.Time) bans.Request {
return bans.Request{
Time: now,
Method: rq.in.Method,
Host: rq.in.Host,
Path: rq.in.URL.RequestURI(),
Status: status,
Status: rq.h.config.BanResponse,
UserAgent: rq.in.UserAgent(),
}
}
@@ -267,13 +99,13 @@ func (rq *request) netblockRequests(netblock netip.Prefix) int64 {
return rq.h.limiter.Requests(netblock) + 1
}
// netblock is the netblock a ban on client covers: its IPv4 address,
// netblock is the netblock a ban on the client covers: its IPv4 address,
// widened to SWWAF_BAN_SCOPE_V4_PREFIX, or the IPv6 group clientGroup
// counts it in.
func (h *handler) netblock(client netip.Addr) netip.Prefix {
addr := client.Unmap()
func (rq *request) netblock() netip.Prefix {
addr := rq.client.Unmap()
if addr.Is4() {
return netip.PrefixFrom(addr, h.config.BanScopeV4Prefix).Masked()
return netip.PrefixFrom(addr, rq.h.config.BanScopeV4Prefix).Masked()
}
return clientGroup(addr)
@@ -283,7 +115,7 @@ func (h *handler) netblock(client netip.Addr) netip.Prefix {
// permanent.
func banExpires(ban bans.Ban) string {
if ban.Permanent() {
return permanent
return "permanent"
}
return requestlog.FormatTime(ban.Expires)
internal/proxy/bans_test.go
+8 -27
View File
@@ -279,12 +279,8 @@ func TestBanNotes(t *testing.T) {
Start: start,
Expires: start.Add(time.Hour),
Cause: bans.CauseLimit,
Reason: "requests per minute over the limit of 1",
Notes: bans.Notes{
ASN: asnDE,
ASName: asNameDE,
Country: "DE",
Kind: "requests",
Limit: 1,
Window: minute,
Count: 2,
@@ -300,7 +296,7 @@ func TestBanNotes(t *testing.T) {
// refused under the ban.
Requests: 4,
Refused: 2,
EarlierBans: bans.EarlierBans{},
EarlierBans: 0,
},
}
@@ -317,8 +313,8 @@ func TestBanNotes(t *testing.T) {
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
got = ledger.Bans(netblock)
if len(got) != 2 || got[1].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
t.Errorf("bans %+v, want two, the second with one earlier ban for a limit", got)
if len(got) != 2 || got[1].Notes.EarlierBans != 1 {
t.Errorf("bans %+v, want two, the second with one earlier ban", got)
}
}
@@ -365,9 +361,8 @@ func (c *clock) advance(d time.Duration) {
// startWithClock starts smallwebwaf in front of an app that answers 200,
// with the settings in env on top of trusting localhost's
// X-Forwarded-For, clients' AS numbers and countries looked up at
// geojsURL, and a clock set to midnight, the start of a bucket in every
// window.
// X-Forwarded-For, clients' countries looked up at geojsURL, and a clock
// set to midnight, the start of a bucket in every window.
func startWithClock(
t *testing.T, geojsURL string, env map[string]string,
) (*sender, *clock, *proxy.Server) {
@@ -421,28 +416,14 @@ func (s *sender) requestWithHeader(
) (logLine, string) {
s.t.Helper()
line, got := s.requestWithBody(http.MethodGet, from, path, header, "", status, action)
return line, string(got.body)
}
// requestWithBody is requestWithHeader for a request with method, whose
// body is sent as it is after the headers, header holding its
// Content-Length or Transfer-Encoding. header may hold several lines,
// separated by "\r\n". It returns the whole answer.
func (s *sender) requestWithBody(
method, from, path, header, body string, status int, action string,
) (logLine, answer) {
s.t.Helper()
if header != "" {
header += "\r\n"
}
conn := dial(s.t, s.addr)
send(s.t, conn, method+" "+path+" HTTP/1.1\r\nHost: "+appHost+
send(s.t, conn, "GET "+path+" HTTP/1.1\r\nHost: "+appHost+
"\r\nUser-Agent: "+userAgent+"\r\n"+forwardedFor+": "+from+"\r\n"+
header+"\r\n"+body)
header+"\r\n")
err := conn.SetReadDeadline(time.Now().Add(waitLimit))
if err != nil {
@@ -471,5 +452,5 @@ func (s *sender) requestWithBody(
s.sent++
wantLine(s.t, line, status, action)
return line, got
return line, string(got.body)
}
internal/proxy/biased.go
-96
View File
@@ -1,96 +0,0 @@
package proxy
import (
"sneak.berlin/go/smallwebwaf/internal/config"
)
// whole is the percentage of each limit a client gets when no biased
// threshold lowers its limits.
const whole = 100
// percentage is a client's limit percentage for the rate limits or for
// the byte limits, as the biased thresholds give it, and the setting that
// gave it: "" with whole when none lowers that kind of limit.
type percentage struct {
percent int64
setting string
}
// biasedThresholdsSet reports whether a biased threshold can lower a
// client's limits: one of its lists is not empty, or
// SWWAF_UNKNOWN_LIMIT_PERCENT is below 100. The client's lookup is then
// needed before its request goes on.
func biasedThresholdsSet(cfg *config.Config) bool {
return len(cfg.ASNLimitPercent) > 0 || len(cfg.CountryLimitPercent) > 0 ||
len(cfg.ASNBytesPercent) > 0 || len(cfg.CountryBytesPercent) > 0 ||
cfg.UnknownLimitPercent < whole
}
// limitPercentages returns a client's limit percentages, for the rate
// limits and for the byte limits, by its AS number and country as looked
// up, each "" when unknown. Each is the lowest of those the settings give
// it, the first of them in the order below when several are lowest: the
// percentage SWWAF_ASN_LIMIT_PERCENT gives its AS number, the one
// SWWAF_COUNTRY_LIMIT_PERCENT gives its country, and, for a client
// without a country, SWWAF_UNKNOWN_LIMIT_PERCENT. For the byte limits,
// SWWAF_ASN_BYTES_PERCENT and SWWAF_COUNTRY_BYTES_PERCENT take the place
// of the first two for an AS number or a country they list.
func limitPercentages(
cfg *config.Config, asn, country string,
) (percentage, percentage) {
unknown := percentage{percent: whole}
if country == "" {
unknown = percentage{cfg.UnknownLimitPercent, "SWWAF_UNKNOWN_LIMIT_PERCENT"}
}
asnRequests := given(cfg.ASNLimitPercent, asn, "SWWAF_ASN_LIMIT_PERCENT")
countryRequests := given(cfg.CountryLimitPercent, country,
"SWWAF_COUNTRY_LIMIT_PERCENT")
asnBytes, countryBytes := asnRequests, countryRequests
if _, listed := cfg.ASNBytesPercent[asn]; listed {
asnBytes = given(cfg.ASNBytesPercent, asn, "SWWAF_ASN_BYTES_PERCENT")
}
if _, listed := cfg.CountryBytesPercent[country]; listed {
countryBytes = given(cfg.CountryBytesPercent, country, "SWWAF_COUNTRY_BYTES_PERCENT")
}
return lowest(asnRequests, countryRequests, unknown),
lowest(asnBytes, countryBytes, unknown)
}
// given returns the percentage percents, the setting named setting, gives
// code, an AS number or a country, or whole when it does not list code.
func given(percents map[string]int64, code, setting string) percentage {
percent, listed := percents[code]
if !listed {
return percentage{percent: whole}
}
return percentage{percent, setting}
}
// lowest returns the lowest of percentages below whole, the first of them
// when several are lowest, or whole when none is below it.
func lowest(percentages ...percentage) percentage {
low := percentage{percent: whole}
for _, p := range percentages {
if p.percent < low.percent {
low = p
}
}
return low
}
// logged returns p as the log line and the notes of a ban give it: its
// percent and setting, or nil and "" for whole, which they leave out.
func (p percentage) logged() (*int64, string) {
if p.percent == whole {
return nil, ""
}
return &p.percent, p.setting
}
internal/proxy/biased_test.go
-503
View File
@@ -1,503 +0,0 @@
package proxy_test
import (
"fmt"
"io"
"maps"
"net"
"net/http"
"net/http/httptest"
"net/netip"
"path/filepath"
"strings"
"testing"
"testing/synctest"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// The biased thresholds.
const (
asnLimitPercent = "SWWAF_ASN_LIMIT_PERCENT"
countryLimitPercent = "SWWAF_COUNTRY_LIMIT_PERCENT"
asnBytesPercent = "SWWAF_ASN_BYTES_PERCENT"
countryBytesPercent = "SWWAF_COUNTRY_BYTES_PERCENT"
unknownLimitPercent = "SWWAF_UNKNOWN_LIMIT_PERCENT"
)
const (
// asnDEHalf and countryDEHalf give fromDE's AS number and its country
// half of every limit, and asnDEQuarter gives its AS number a quarter.
asnDEHalf = asnDE + ":50"
asnDEQuarter = asnDE + ":25"
countryDEHalf = "de:50"
// noCountry is in an AS of its own, AS64500, and in no country.
noCountry = "192.0.2.80"
// fourAMinute is the rate limit these tests set: half of it is 2
// requests a minute, a quarter of it 1.
fourAMinute = "4"
// twoUploads is the byte limit these tests set: 199 bytes, which an
// upload, a request with a body and its answer, 100 bytes, is within,
// and half of which, 99 bytes, it is over.
twoUploads = "199"
// none is how percentText gives a percentage left out.
none = "none"
)
func TestEachBiasedThresholdLowersTheRateLimits(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
setting, value, from string
}{
{asnLimitPercent, asnDEHalf, fromDE},
{countryLimitPercent, countryDEHalf, fromDE},
{unknownLimitPercent, "50", unplaced},
} {
t.Run(tc.setting, func(t *testing.T) {
t.Parallel()
s, _, _ := startWithLookups(t, map[string]string{
rateLimitPerMinute: fourAMinute, tc.setting: tc.value,
})
// Half of 4 requests a minute: the third breaks the limit.
for _, sent := range []struct {
status int
action string
}{
{http.StatusOK, requestlog.ActionForward},
{http.StatusOK, requestlog.ActionForward},
{http.StatusForbidden, requestlog.ActionRateLimited},
} {
line := s.get(tc.from, sent.status, sent.action)
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
"50 from "+tc.setting)
}
// fromKP, which no setting lists, has the whole limit.
for range 3 {
line := s.get(fromKP, http.StatusOK, requestlog.ActionForward)
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
none)
}
})
}
}
func TestEachBiasedThresholdLowersTheByteLimits(t *testing.T) {
t.Parallel()
// The AS numbers and countries are given in either case.
for _, tc := range []struct {
setting, value, from string
}{
{asnLimitPercent, asnDEHalf, fromDE},
{countryLimitPercent, "DE:50", fromDE},
{unknownLimitPercent, "50", unplaced},
{asnBytesPercent, "as64496:50", fromDE},
{countryBytesPercent, countryDEHalf, fromDE},
} {
t.Run(tc.setting, func(t *testing.T) {
t.Parallel()
s, _, _ := startWithLookups(t, map[string]string{
bytesLimitPerMinute: twoUploads, tc.setting: tc.value,
})
// The upload's 100 bytes are over half of 199, 99.
line := s.uploadFrom(tc.from)
if line.LimitHit != minuteBytes {
t.Errorf("log line has limit_hit %q, want %s", line.LimitHit, minuteBytes)
}
wantPercent(t, "bytes_percent", line.BytesPercent, line.BytesPercentSetting,
"50 from "+tc.setting)
// fromKP, which no setting lists, has the whole limit.
line = s.uploadFrom(fromKP)
if line.LimitHit != "" {
t.Errorf("log line for %s has limit_hit %q, want none", fromKP, line.LimitHit)
}
wantPercent(t, "bytes_percent", line.BytesPercent, line.BytesPercentSetting, none)
})
}
}
func TestBytesPercentSettingsTakeThePlaceOfTheOthersForByteLimits(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
env map[string]string
// limitPercent and bytesPercent are the log line's, as percentText
// gives them, and limitHit is its limit_hit.
limitPercent, bytesPercent, limitHit string
}{
{
"lowering the byte limits alone",
map[string]string{asnBytesPercent: asnDEHalf},
none, "50 from " + asnBytesPercent, minuteBytes,
},
{
"lowering the byte limits alone, by country",
map[string]string{countryBytesPercent: countryDEHalf},
none, "50 from " + countryBytesPercent, minuteBytes,
},
{
"raising the byte limits back",
map[string]string{asnLimitPercent: asnDEHalf, asnBytesPercent: asnDE + ":100"},
"50 from " + asnLimitPercent, none, "",
},
{
"raising the byte limits back, by country",
map[string]string{countryLimitPercent: countryDEHalf, countryBytesPercent: "de:100"},
"50 from " + countryLimitPercent, none, "",
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
env := map[string]string{bytesLimitPerMinute: twoUploads}
maps.Copy(env, tc.env)
s, _, _ := startWithLookups(t, env)
// The upload's 100 bytes are over 99, half of 199, and within 199.
line := s.uploadFrom(fromDE)
if line.LimitHit != tc.limitHit {
t.Errorf("log line has limit_hit %q, want %q", line.LimitHit, tc.limitHit)
}
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
tc.limitPercent)
wantPercent(t, "bytes_percent", line.BytesPercent, line.BytesPercentSetting,
tc.bytesPercent)
})
}
}
func TestZeroPercentIsAZeroAllowance(t *testing.T) {
t.Parallel()
s, _, _ := startWithLookups(t, map[string]string{asnLimitPercent: asnDE + ":0"})
// The first request breaks the limit, and bans the client; the log line
// gives the 0.
line := s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
if line.fields["limit_percent"] != float64(0) ||
line.fields["limit_percent_setting"] != asnLimitPercent {
t.Errorf("log line has limit_percent %v from %v, want 0 from %s",
line.fields["limit_percent"], line.fields["limit_percent_setting"],
asnLimitPercent)
}
s.get(fromDE, http.StatusForbidden, requestlog.ActionBanned)
}
func TestLowestPercentageApplies(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
env map[string]string
from string
// want is the log line's limit_percent, as percentText gives it.
want string
}{
{
"the country's",
map[string]string{asnLimitPercent: asnDEHalf, countryLimitPercent: "de:25"},
fromDE, "25 from " + countryLimitPercent,
},
{
"the AS number's",
map[string]string{asnLimitPercent: asnDEQuarter, countryLimitPercent: countryDEHalf},
fromDE, "25 from " + asnLimitPercent,
},
{
"the AS number's, the first of two alike",
map[string]string{asnLimitPercent: asnDEQuarter, countryLimitPercent: "de:25"},
fromDE, "25 from " + asnLimitPercent,
},
{
"that for a client without a country",
map[string]string{asnLimitPercent: "AS64500:50", unknownLimitPercent: "25"},
noCountry, "25 from " + unknownLimitPercent,
},
{
// SWWAF_UNKNOWN_LIMIT_PERCENT is left at its default, 100.
"the AS number's, for a client without a country",
map[string]string{asnLimitPercent: "AS64500:25"},
noCountry, "25 from " + asnLimitPercent,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
env := map[string]string{rateLimitPerMinute: fourAMinute}
maps.Copy(env, tc.env)
s, _, _ := startWithLookups(t, env)
// A quarter of 4 requests a minute: the second breaks the limit.
s.get(tc.from, http.StatusOK, requestlog.ActionForward)
line := s.get(tc.from, http.StatusForbidden, requestlog.ActionRateLimited)
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
tc.want)
})
}
}
func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(t *testing.T) {
t.Parallel()
s, _, _ := startWithLookups(t, map[string]string{
rateLimitPerMinute: fourAMinute, unknownLimitPercent: "50",
})
// One the lookup database does not hold, and one on a private address,
// which is never looked up: the third request of each breaks half of 4.
for _, from := range []string{unplaced, "10.0.0.8"} {
s.get(from, http.StatusOK, requestlog.ActionForward)
s.get(from, http.StatusOK, requestlog.ActionForward)
s.get(from, http.StatusForbidden, requestlog.ActionRateLimited)
}
// One in a country has the whole limit.
for range 3 {
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
}
}
func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
t.Parallel()
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
// says, with a GeoJS that never answers.
synctest.Test(t, func(t *testing.T) {
server, out, _ := newProxy(t, "http://app.invalid", unansweredGeoJSURL,
time.Now, map[string]string{unknownLimitPercent: "0"})
// Once the second the request waits for its answer is up, the client
// counts as without a country, and its zero allowance refuses the
// request before it reaches the app.
serveFromDE(t, server, http.MethodGet, http.NoBody)
line := out.requestLine(t)
wantLine(t, line, http.StatusForbidden, requestlog.ActionRateLimited)
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
"0 from "+unknownLimitPercent)
})
}
func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
t.Parallel()
const timeout = 3 * time.Second
for _, tc := range []struct {
setting, value string
waits bool
}{
{asnLimitPercent, asnDEHalf, true},
{countryLimitPercent, countryDEHalf, true},
{asnBytesPercent, asnDEHalf, true},
{countryBytesPercent, countryDEHalf, true},
{unknownLimitPercent, "99", true},
// At 100, its default, it lowers no limit.
{unknownLimitPercent, "100", false},
} {
t.Run(tc.setting+"="+tc.value, func(t *testing.T) {
t.Parallel()
// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
// says, with a GeoJS that never answers.
synctest.Test(t, func(t *testing.T) {
// The request's body is over SWWAF_REQUEST_MAX_BYTES, so that it
// is refused after the checks, and never reaches the app.
server, out, _ := newProxy(t, "http://app.invalid", unansweredGeoJSURL,
time.Now, map[string]string{
lookupTimeout: timeout.String(), requestMaxBytes: "1",
tc.setting: tc.value,
})
began := time.Now()
serveFromDE(t, server, http.MethodPost, strings.NewReader("ab"))
want := time.Duration(0)
if tc.waits {
want = timeout
}
if waited := time.Since(began); waited != want {
t.Errorf("the request waited %s for its answer, want %s", waited, want)
}
wantLine(t, out.requestLine(t), http.StatusRequestEntityTooLarge,
requestlog.ActionTooLarge)
// The bubble's clock stops once this function returns, so the
// request to GeoJS, which a request that did not wait leaves
// under way, has to be abandoned before then.
time.Sleep(timeout)
})
})
}
}
func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
env map[string]string
// before is how many uploads come before the one that breaks a
// limit, which is answered with status and logged with action.
before int
status int
action string
// reason and want are the ban's reason, and its notes' limit
// percentage, as percentText gives it.
reason, want string
}{
{
// A quarter of 12 requests a minute is 3: the fourth breaks it.
"a rate limit",
map[string]string{rateLimitPerMinute: "12", asnLimitPercent: asnDEQuarter},
3, http.StatusForbidden, requestlog.ActionRateLimited,
"requests per minute over the limit of 3", "25 from " + asnLimitPercent,
},
{
// The byte limits' percentage, not the rate limits'.
"a byte limit",
map[string]string{
bytesLimitPerMinute: twoUploads, asnLimitPercent: asnDEQuarter,
asnBytesPercent: asnDEHalf,
},
0, http.StatusOK, requestlog.ActionForward,
"bytes per minute over the limit of 99", "50 from " + asnBytesPercent,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
s, server, queue := startWithLookups(t, tc.env)
for range tc.before {
s.uploadFrom(fromDE)
}
s.requestWithBody(http.MethodPost, fromDE, "/", uploadHeader, uploadBody,
tc.status, tc.action)
held := server.Ledger.Bans(netip.MustParsePrefix(fromDE + "/32"))
if len(held) != 1 {
t.Fatalf("bans %+v, want one", held)
}
notes := held[0].Notes
if held[0].Reason != tc.reason {
t.Errorf("the ban's reason is %q, want %q", held[0].Reason, tc.reason)
}
wantPercent(t, "the notes' limit_percent", notes.LimitPercent,
notes.LimitPercentSetting, tc.want)
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 {
t.Fatalf("%d alerts wait, want the ban's alone: %+v", len(waiting), waiting)
}
alerted, _ := waiting[0].Detail["notes"].(bans.Notes)
wantPercent(t, "the alert's notes' limit_percent", alerted.LimitPercent,
alerted.LimitPercentSetting, tc.want)
})
}
}
// startWithLookups is startWithLookupsAndClock for a test that needs no
// clock.
func startWithLookups(
t *testing.T, env map[string]string,
) (*sender, *proxy.Server, *alerts.Queue) {
t.Helper()
s, _, server, queue := startWithLookupsAndClock(t, env)
return s, server, queue
}
// startWithLookupsAndClock is startAppWithAlerts in front of
// readAndAnswer, with the settings in env on top of clients looked up in a
// lookup database, which places fromDE and fromKP in the AS numbers and
// countries the stand-in for GeoJS gives them, noCountry in AS64500 and no
// country, and no other address.
func startWithLookupsAndClock(
t *testing.T, env map[string]string,
) (*sender, *clock, *proxy.Server, *alerts.Queue) {
t.Helper()
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
lookuptest.Write(t, path, map[string]lookuptest.Network{
fromDE + "/32": {ASN: asnDE, ASName: asNameDE, Country: "DE"},
fromKP + "/32": {ASN: asnKP, ASName: asNameKP, Country: "KP"},
noCountry + "/32": {ASN: "AS64500", ASName: "Nowhere Net"},
})
settings := map[string]string{lookupSource: fileSource, lookupDBPath: path}
maps.Copy(settings, env)
return startAppWithAlerts(t, readAndAnswer, settings)
}
// uploadFrom is upload from the client at from.
func (s *sender) uploadFrom(from string) logLine {
s.t.Helper()
line, _ := s.requestWithBody(http.MethodPost, from, "/", uploadHeader, uploadBody,
http.StatusOK, requestlog.ActionForward)
return line
}
// serveFromDE hands a request from fromDE with method and body straight to
// server's handler, without the network, and returns once it is answered.
func serveFromDE(t *testing.T, server *proxy.Server, method string, body io.Reader) {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), method, "/", body)
req.RemoteAddr = net.JoinHostPort(fromDE, "1234")
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
}
// wantPercent checks a limit percentage that a log line or a ban's notes
// give, what, and the setting that gave it, against want, as percentText
// gives them.
func wantPercent(t *testing.T, what string, percent *int64, setting, want string) {
t.Helper()
if got := percentText(percent, setting); got != want {
t.Errorf("%s is %s, want %s", what, got, want)
}
}
// percentText gives a limit percentage and the setting that gave it as
// text, such as "50 from SWWAF_ASN_LIMIT_PERCENT", or none when both are
// left out.
func percentText(percent *int64, setting string) string {
switch {
case percent == nil && setting == "":
return none
case percent == nil:
return "none from " + setting
default:
return fmt.Sprintf("%d from %s", *percent, setting)
}
}
internal/proxy/bodies.go
-42
View File
@@ -103,48 +103,6 @@ func (b *responseBody) Close() error {
return b.body.Close()
}
// upgradedConn is the connection to the app once the app has switched
// protocols, as for a WebSocket. ReverseProxy writes to it what the client
// sends and reads from it what the app sends, on goroutines of its own,
// until the connection closes; it counts the bytes each way, for the byte
// limits.
type upgradedConn struct {
io.ReadWriteCloser
// fromApp is how many bytes the app has sent, and toApp how many the
// client has.
fromApp atomic.Int64
toApp atomic.Int64
}
// Read reads what the app sends.
func (c *upgradedConn) Read(p []byte) (int, error) {
n, err := c.ReadWriteCloser.Read(p)
c.fromApp.Add(int64(n))
return n, err
}
// Write sends the app what the client sent.
func (c *upgradedConn) Write(p []byte) (int, error) {
n, err := c.ReadWriteCloser.Write(p)
c.toApp.Add(int64(n))
return n, err
}
// CloseWrite tells the app that the client sends no more, while what the
// app sends still passes. ReverseProxy calls it once the client has
// stopped sending, and closes the connection there if it is not supported.
func (c *upgradedConn) CloseWrite() error {
conn, ok := c.ReadWriteCloser.(interface{ CloseWrite() error })
if !ok {
return http.ErrNotSupported
}
return conn.CloseWrite()
}
// limitBody returns body, cut off with an *http.MaxBytesError after
// maxBytes, or unchanged if maxBytes is zero, which is off.
func limitBody(body io.ReadCloser, maxBytes int64) io.ReadCloser {
internal/proxy/bytelimits_test.go
-583
View File
@@ -1,583 +0,0 @@
package proxy_test
import (
"bufio"
"io"
"net"
"net/http"
"net/netip"
"strconv"
"strings"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// The byte limit settings.
const (
bytesLimitPerMinute = "SWWAF_BYTES_LIMIT_PER_MINUTE"
bytesLimitPerHour = "SWWAF_BYTES_LIMIT_PER_HOUR"
bytesLimitPerDay = "SWWAF_BYTES_LIMIT_PER_DAY"
bytesCount = "SWWAF_BYTES_COUNT"
)
// The values of SWWAF_BYTES_COUNT.
const (
countResponse = "response"
countRequest = "request"
countBoth = "both"
)
const (
// bodyBytes is the size of the body of each request these tests send
// with one, and answerBytes that of each answer of the app.
bodyBytes = 30
answerBytes = 70
// byteLimit is the byte limit these tests set, as a setting: a request
// with a body and its answer, 100 bytes, go over it.
byteLimit = "99"
// minuteBytes is limit_hit for SWWAF_BYTES_LIMIT_PER_MINUTE.
minuteBytes = "minute_bytes"
)
func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
t.Parallel()
const scraper = "192.0.2.200"
for _, tc := range []struct {
setting, window string
// apart is the time between the two requests, which the window
// still covers.
apart time.Duration
}{
{bytesLimitPerMinute, minute, 0},
{bytesLimitPerHour, "hour", 2 * time.Minute},
{bytesLimitPerDay, "day", 2 * time.Hour},
} {
t.Run(tc.setting, func(t *testing.T) {
t.Parallel()
s, clk := startWithAnswers(t, map[string]string{
tc.setting: byteLimit, metricsToken: token,
})
// 70 bytes are within the limit of 99.
line, _ := s.download()
if line.LimitHit != "" || line.Offence != "" {
t.Errorf("log line has limit_hit %q and offence %q, want neither",
line.LimitHit, line.Offence)
}
// 140 bytes are over it. The response is passed on whole, and
// then bans the client for an hour.
clk.advance(tc.apart)
expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
line, got := s.download()
if got.err != nil || len(got.body) != answerBytes ||
line.ResponseBytes != answerBytes {
t.Errorf("got %d bytes (%v), and the log line has response_bytes %d, "+
"want %d", len(got.body), got.err, line.ResponseBytes, answerBytes)
}
if line.LimitHit != tc.window+"_bytes" || line.Offence != requestlog.OffenceLimit ||
line.BanExpires != expires {
t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"want %s_bytes, limit and %s", line.LimitHit, line.Offence,
line.BanExpires, tc.window, expires)
}
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
wantMetric(t, s.scrape(scraper), `smallwebwaf_rate_limit_hits_total{`+
`instance="`+alertInstance+`",kind="bytes",window="`+tc.window+`"}`, 1)
})
}
}
func TestResponseOverAByteLimitByItselfIsPassedOnWhole(t *testing.T) {
t.Parallel()
s, _ := startWithAnswers(t, map[string]string{bytesLimitPerMinute: "50"})
// The answer's 70 bytes are over the limit of 50 on their own.
line, got := s.download()
if got.err != nil || len(got.body) != answerBytes || line.LimitHit != minuteBytes {
t.Errorf("got %d bytes (%v), and the log line has limit_hit %q, want %d and %s",
len(got.body), got.err, line.LimitHit, answerBytes, minuteBytes)
}
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
}
func TestBytesOfAnAnswerThatBreaksOffAreCounted(t *testing.T) {
t.Parallel()
s, clk, _, _ := startAppWithAlerts(t, breakOff, map[string]string{
bytesLimitPerMinute: "50",
})
expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
// The 70 bytes passed on before the app broke off are over the limit of
// 50, and ban the client for an hour.
line, got := s.requestWithBody(http.MethodGet, client, "/", "", "", http.StatusOK,
requestlog.ActionUpstreamError)
if len(got.body) != answerBytes || line.LimitHit != minuteBytes ||
line.BanExpires != expires {
t.Errorf("got %d bytes, and the log line has limit_hit %q and ban_expires %q, "+
"want %d, %s and %s", len(got.body), line.LimitHit, line.BanExpires,
answerBytes, minuteBytes, expires)
}
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
}
func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
setting string
counted float64
}{
{countResponse, answerBytes},
{countRequest, bodyBytes},
{countBoth, bodyBytes + answerBytes},
} {
t.Run(tc.setting, func(t *testing.T) {
t.Parallel()
s, _, _, _ := startAppWithAlerts(t, answerAfterUpgrade, map[string]string{
bytesLimitPerMinute: "29", bytesCount: tc.setting,
})
// The client sends 30 bytes and the app 70, each over the limit
// of 29, which bans the client once the WebSocket has closed.
line := s.webSocket()
if line.LimitHit != minuteBytes || line.Counts.MinuteBytes != tc.counted {
t.Errorf("log line has limit_hit %q and minute_bytes %v, want %s and %v",
line.LimitHit, line.Counts.MinuteBytes, minuteBytes, tc.counted)
}
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
})
}
}
func TestWebSocketPassesTheAnswerAfterTheClientStopsSending(t *testing.T) {
t.Parallel()
app := startApp(t, echoOnceTheClientStops)
addr, out := startProxy(t, app.URL,
map[string]string{trustedProxies: trustLocalhost})
s := &sender{t: t, addr: addr, out: out}
conn, reader := s.openWebSocket()
send(t, conn, uploadBody)
// The client closes its sending side and waits for the answer, which the
// app sends only once it has seen the client stop. smallwebwaf passes the
// close on to the app through CloseWrite on upgradedConn; without that,
// it closes both connections, and the answer is lost.
tcp, ok := conn.(*net.TCPConn)
if !ok {
t.Fatalf("connection is a %T, want a *net.TCPConn", conn)
}
err := tcp.CloseWrite()
if err != nil {
t.Fatalf("close the sending side: %v", err)
}
got, err := io.ReadAll(reader)
if err != nil || string(got) != uploadBody {
t.Errorf("got %q (%v), want %q", got, err, uploadBody)
}
s.closeWebSocket(conn)
}
func TestBytesCountSaysWhichBytesCount(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
setting string
// each is the bytes each request counts, and breaking the request
// that goes over the limit of 99.
each float64
breaking int
}{
{countResponse, answerBytes, 2},
{countRequest, bodyBytes, 4},
{countBoth, bodyBytes + answerBytes, 1},
} {
t.Run(tc.setting, func(t *testing.T) {
t.Parallel()
s, _ := startWithAnswers(t, map[string]string{
bytesLimitPerMinute: byteLimit, bytesCount: tc.setting,
})
for i := 1; i <= tc.breaking; i++ {
line := s.upload()
want := ""
if i == tc.breaking {
want = minuteBytes
}
counted := float64(i) * tc.each
if line.LimitHit != want || line.Counts.MinuteBytes != counted {
t.Errorf("request %d: log line has limit_hit %q and minute_bytes %v, "+
"want %q and %v", i, line.LimitHit, line.Counts.MinuteBytes,
want, counted)
}
}
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
})
}
}
func TestByteLimitsLeaveOutWhatTheRateLimitsLeaveOut(t *testing.T) {
t.Parallel()
const (
allowed = "192.0.2.7" // in SWWAF_ALLOW_NETS
exempt = "192.0.2.10" // in SWWAF_RATE_LIMIT_EXEMPT_NETS
)
s, _ := startWithAnswers(t, map[string]string{
bytesLimitPerMinute: byteLimit,
allowNets: allowed,
rateLimitExemptNets: exempt,
rateLimitExemptPaths: "/assets/",
})
// Each sends 200 bytes, none of which is counted.
for _, sent := range []struct{ from, path string }{
{allowed, "/"}, {exempt, "/"}, {client, "/assets/app.js"},
} {
for range 2 {
line, _ := s.requestWithBody(http.MethodPost, sent.from, sent.path,
uploadHeader, uploadBody, http.StatusOK, requestlog.ActionForward)
if _, counted := line.fields["counts"]; counted || line.LimitHit != "" {
t.Errorf("%s %s: log line has counts %v and limit_hit %q, want neither",
sent.from, sent.path, line.fields["counts"], line.LimitHit)
}
}
}
// A path that is not exempt is counted, and breaks the limit.
line := s.upload()
if line.LimitHit != minuteBytes {
t.Errorf("log line has limit_hit %q, want %s", line.LimitHit, minuteBytes)
}
}
func TestByteLimitsOffCountTheBytesAndBanNoOne(t *testing.T) {
t.Parallel()
const off = "off"
s, _ := startWithAnswers(t, map[string]string{
bytesLimitPerMinute: off, bytesLimitPerHour: off, bytesLimitPerDay: off,
})
for i := 1; i <= 3; i++ {
line := s.upload()
counted := float64(i * (bodyBytes + answerBytes))
if line.LimitHit != "" || line.Counts.MinuteBytes != counted ||
line.Counts.HourBytes != counted || line.Counts.DayBytes != counted {
t.Errorf("request %d: log line has limit_hit %q and counts %+v, "+
"want none and %v bytes in each window", i, line.LimitHit,
line.Counts, counted)
}
}
}
func TestBanForABrokenByteLimitHasItsNotesAndItsAlert(t *testing.T) {
t.Parallel()
s, clk, server, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
bytesLimitPerMinute: byteLimit,
})
start := clk.Now()
s.requestWithBody(http.MethodPost, client, "/upload?part=1", uploadHeader,
uploadBody, http.StatusOK, requestlog.ActionForward)
netblock := netip.MustParsePrefix(client + "/32")
want := bans.Ban{
Netblock: netblock,
Start: start,
Expires: start.Add(time.Hour),
Cause: bans.CauseLimit,
Reason: "bytes per minute over the limit of " + byteLimit,
Notes: bans.Notes{
Kind: "bytes",
Limit: 99,
Window: minute,
Count: bodyBytes + answerBytes,
// The request as it was answered, by the app.
Request: bans.Request{
Time: start,
Method: http.MethodPost,
Host: appHost,
Path: "/upload?part=1",
Status: http.StatusOK,
UserAgent: userAgent,
},
Requests: 1,
},
}
got := server.Ledger.Bans(netblock)
if len(got) != 1 || got[0] != want {
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
}
wantAlerts(t, queue, banAlert(alerts.EventBan, start, client, want,
requestlog.FormatTime(want.Expires)))
if offences := historyOf(t, server, client).Offences.Limit; offences != 1 {
t.Errorf("history counts %d offences for a limit, want 1", offences)
}
}
func TestObserveModeLogsAndAlertsAByteLimitAndBansNoOne(t *testing.T) {
t.Parallel()
s, clk, server, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
mode: observe,
bytesLimitPerMinute: byteLimit,
})
start := clk.Now()
// No ban sets the client's counters back to zero, so each request
// breaks the limit again. The answer is the app's either way, and the
// alert for the ban is not sent twice within the cooldown.
for range 2 {
line := s.upload()
wantWouldAction(t, line, "")
if line.LimitHit != minuteBytes || line.Offence != requestlog.OffenceLimit ||
line.BanExpires != "" {
t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
"want %s, limit and none", line.LimitHit, line.Offence, line.BanExpires,
minuteBytes)
}
}
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 {
t.Fatalf("%d alerts wait, want 1: %+v", len(waiting), waiting)
}
notes, _ := waiting[0].Detail["notes"].(bans.Notes)
alert := banAlert(alerts.EventBan, start, client, bans.Ban{
Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
Reason: "bytes per minute over the limit of " + byteLimit, Notes: notes,
}, requestlog.FormatTime(start.Add(time.Hour)))
alert.Detail["mode"] = observe
wantAlerts(t, queue, alert)
}
func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses(t *testing.T) {
t.Parallel()
s, _ := startWithAnswers(t, map[string]string{
mode: observe,
rateLimitPerMinute: "1",
bytesLimitPerMinute: "150",
})
s.upload()
// The second request breaks the rate limit, which in enforce mode would
// refuse it before the app sent anything, so its 100 bytes are not
// counted, and the byte limit is not broken. Its line gives the bytes
// counted before it.
line := s.upload()
wantWouldAction(t, line, requestlog.ActionRateLimited)
if line.LimitHit != minute || line.Counts.MinuteBytes != bodyBytes+answerBytes {
t.Errorf("log line has limit_hit %q and minute_bytes %v, want minute and %d",
line.LimitHit, line.Counts.MinuteBytes, bodyBytes+answerBytes)
}
}
// uploadHeader and uploadBody are the header and the body of a request
// with a body of bodyBytes.
//
//nolint:gochecknoglobals // a constant cannot call strings.Repeat
var (
uploadHeader = "Content-Length: " + strconv.Itoa(bodyBytes)
uploadBody = strings.Repeat("u", bodyBytes)
)
// readAndAnswer is the app of these tests: it reads each request's whole
// body and answers with answerBytes bytes.
func readAndAnswer(w http.ResponseWriter, r *http.Request) {
_, _ = io.Copy(io.Discard, r.Body)
_, _ = io.WriteString(w, strings.Repeat("a", answerBytes))
}
// breakOff is an app that announces an answer of twice answerBytes, and
// breaks off after answerBytes.
func breakOff(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Length", strconv.Itoa(2*answerBytes))
_, _ = io.WriteString(w, strings.Repeat("a", answerBytes))
}
// answerAfterUpgrade is an app that switches protocols, as for a
// WebSocket, and then answers each line it receives with a line of
// answerBytes.
func answerAfterUpgrade(w http.ResponseWriter, _ *http.Request) {
conn, buffered, err := http.NewResponseController(w).Hijack()
if err != nil {
return
}
defer func() {
_ = conn.Close()
}()
_, _ = buffered.WriteString("HTTP/1.1 101 Switching Protocols\r\n" +
"Connection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
_ = buffered.Flush()
for {
_, err := buffered.ReadString('\n')
if err != nil {
return
}
_, _ = buffered.WriteString(strings.Repeat("a", answerBytes-1) + "\n")
_ = buffered.Flush()
}
}
// echoOnceTheClientStops is an app that switches protocols, as for a
// WebSocket, reads what the client sends until the client stops sending,
// and then sends it all back.
func echoOnceTheClientStops(w http.ResponseWriter, _ *http.Request) {
conn, buffered, err := http.NewResponseController(w).Hijack()
if err != nil {
return
}
defer func() {
_ = conn.Close()
}()
_, _ = buffered.WriteString("HTTP/1.1 101 Switching Protocols\r\n" +
"Connection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
_ = buffered.Flush()
received, _ := io.ReadAll(buffered)
_, _ = buffered.Write(received)
_ = buffered.Flush()
}
// webSocket opens a WebSocket from client to answerAfterUpgrade, sends a
// line of bodyBytes on it, reads the answer, and closes it. It checks the
// answer, and the log line as request does, and returns the log line.
func (s *sender) webSocket() logLine {
s.t.Helper()
conn, reader := s.openWebSocket()
send(s.t, conn, strings.Repeat("u", bodyBytes-1)+"\n")
got, err := reader.ReadString('\n')
if err != nil || len(got) != answerBytes {
s.t.Errorf("got %d bytes (%v), want %d", len(got), err, answerBytes)
}
return s.closeWebSocket(conn)
}
// openWebSocket sends a request from client to switch protocols, as for a
// WebSocket, and checks that the app switches. It returns the connection,
// on which reading fails once waitLimit has passed, and a reader of what
// the app sends on it.
func (s *sender) openWebSocket() (net.Conn, *bufio.Reader) {
s.t.Helper()
conn := dial(s.t, s.addr)
send(s.t, conn, "GET /socket HTTP/1.1\r\nHost: "+appHost+"\r\n"+forwardedFor+
": "+client+"\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
err := conn.SetReadDeadline(time.Now().Add(waitLimit))
if err != nil {
s.t.Fatalf("set read deadline: %v", err)
}
reader := bufio.NewReader(conn)
res, err := http.ReadResponse(reader, nil)
if err != nil {
s.t.Fatalf("read the answer to the upgrade: %v", err)
}
_ = res.Body.Close()
if res.StatusCode != http.StatusSwitchingProtocols {
s.t.Fatalf("status %d, want %d", res.StatusCode, http.StatusSwitchingProtocols)
}
return conn, reader
}
// closeWebSocket closes conn, a WebSocket openWebSocket opened, checks its
// log line as request does, and returns it.
func (s *sender) closeWebSocket(conn net.Conn) logLine {
s.t.Helper()
_ = conn.Close()
line := s.out.requestLines(s.t, s.sent+1)[s.sent]
s.sent++
wantLine(s.t, line, http.StatusSwitchingProtocols, requestlog.ActionForward)
return line
}
// startWithAnswers is startAppWithAlerts in front of readAndAnswer, for a
// test that looks at neither the server nor the alerts.
func startWithAnswers(t *testing.T, env map[string]string) (*sender, *clock) {
t.Helper()
s, clk, _, _ := startAppWithAlerts(t, readAndAnswer, env)
return s, clk
}
// download sends a GET request for / from client, and checks that the
// app's answer is passed on, as request does. It returns the log line and
// the answer.
func (s *sender) download() (logLine, answer) {
s.t.Helper()
return s.requestWithBody(http.MethodGet, client, "/", "", "", http.StatusOK,
requestlog.ActionForward)
}
// upload is download for a POST request with a body of bodyBytes, and
// returns the log line.
func (s *sender) upload() logLine {
s.t.Helper()
line, _ := s.requestWithBody(http.MethodPost, client, "/", uploadHeader,
uploadBody, http.StatusOK, requestlog.ActionForward)
return line
}
internal/proxy/client.go
-28
View File
@@ -1,7 +1,6 @@
package proxy
import (
"crypto/rand"
"net/http"
"net/netip"
"slices"
@@ -49,33 +48,6 @@ func clientAddress(
return client
}
// requestIDHeader carries the request's id, from traefik and to the app.
const requestIDHeader = "X-Request-ID"
// requestID is the request's id: the one a trusted proxy sent, or a new
// random one. A peer outside the trusted proxies did not come through
// traefik, so the id it sends is its own claim, and is replaced.
func requestID(r *http.Request, peerTrusted bool) string {
id := r.Header.Get(requestIDHeader)
if !peerTrusted || id == "" {
id = rand.Text()
}
return id
}
// scheme is how the client reached traefik, as a trusted proxy says in
// X-Forwarded-Proto, or otherwise http, the only scheme smallwebwaf
// serves.
func scheme(r *http.Request, peerTrusted bool) string {
proto := r.Header.Get("X-Forwarded-Proto")
if !peerTrusted || proto == "" {
return "http"
}
return proto
}
// ipv6GroupPrefix is the length of the IPv6 netblock that is one client.
const ipv6GroupPrefix = 64
internal/proxy/client_test.go
+13 -17
View File
@@ -14,14 +14,10 @@ const (
appHost = "app.example"
// client is the client's address, as a proxy names it.
client = "203.0.113.9"
// forwardedFor is the header that lists the client and its proxies,
// and forwardedProto the one that gives the scheme the client used.
forwardedFor = "X-Forwarded-For"
forwardedProto = "X-Forwarded-Proto"
// secure is the scheme a client reached traefik with, and plain the
// one smallwebwaf serves.
// forwardedFor is the header that lists the client and its proxies.
forwardedFor = "X-Forwarded-For"
// secure is the scheme a client reached traefik with.
secure = "https"
plain = "http"
)
// appHeaders is what the app tells about the headers it received.
@@ -69,13 +65,13 @@ func TestClientAddressAndForwardedHeaders(t *testing.T) {
func clientAddressCases() []clientAddressCase {
trusted := map[string]string{trustedProxies: trustLocalhost}
forged := http.Header{
forwardedFor: {client},
"X-Forwarded-Host": {"forged.example"},
forwardedProto: {secure},
"X-Real-Ip": {client},
forwardedFor: {client},
"X-Forwarded-Host": {"forged.example"},
"X-Forwarded-Proto": {secure},
"X-Real-Ip": {client},
}
replaced := appHeaders{
ForwardedFor: localhost, ForwardedHost: appHost, ForwardedProto: plain,
ForwardedFor: localhost, ForwardedHost: appHost, ForwardedProto: "http",
}
return []clientAddressCase{{
@@ -91,10 +87,10 @@ func clientAddressCases() []clientAddressCase {
"outside the trusted proxies from the right",
env: trusted,
header: http.Header{
forwardedFor: {"198.51.100.7, " + client + ", 10.0.0.2"},
"X-Forwarded-Host": {appHost},
forwardedProto: {secure},
"X-Real-Ip": {client},
forwardedFor: {"198.51.100.7, " + client + ", 10.0.0.2"},
"X-Forwarded-Host": {appHost},
"X-Forwarded-Proto": {secure},
"X-Real-Ip": {client},
},
wantClient: client,
wantApp: appHeaders{
@@ -142,7 +138,7 @@ func requestWithHeaders(
Host: r.Host,
ForwardedFor: r.Header.Get(forwardedFor),
ForwardedHost: r.Header.Get("X-Forwarded-Host"),
ForwardedProto: r.Header.Get(forwardedProto),
ForwardedProto: r.Header.Get("X-Forwarded-Proto"),
RealIP: r.Header.Get("X-Real-IP"),
})
})
internal/proxy/countries.go
+26 -6
View File
@@ -1,17 +1,31 @@
package proxy
import (
"context"
"net/netip"
"slices"
)
// countryDenied reports whether the country lists refuse the request, by
// the client's country as it was looked up. A client without a country,
// or whose country cannot be found, is refused only by
// SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
func (rq *request) countryDenied() bool {
// countryDenied reports whether the country lists refuse the request.
// The client's country is looked up only while a list is set, and never
// for a client on a private, loopback or link-local address, which has
// no country. A client without a country, or whose country cannot be
// found, is refused only by SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES. ctx is
// the request's own context.
func (rq *request) countryDenied(ctx context.Context) bool {
denied := rq.h.config.DeniedCountries
allowed := rq.h.config.ExclusivelyAllowedCountries
country := rq.line.Country
if len(denied) == 0 && len(allowed) == 0 {
return false
}
var country string
if hasCountry(rq.client) {
country = rq.h.geojs.Country(ctx, clientGroup(rq.client))
}
rq.line.Country = country
if slices.Contains(denied, country) {
return true
@@ -19,3 +33,9 @@ func (rq *request) countryDenied() bool {
return len(allowed) > 0 && !slices.Contains(allowed, country)
}
// hasCountry reports whether addr can be placed in a country: private,
// loopback and link-local addresses cannot.
func hasCountry(addr netip.Addr) bool {
return !addr.IsPrivate() && !addr.IsLoopback() && !addr.IsLinkLocalUnicast()
}
internal/proxy/countries_test.go
+34 -99
View File
@@ -56,21 +56,16 @@ func TestCountryLists(t *testing.T) {
maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
// The AS number GeoJS gives unplaced, 64512, counts as unknown.
for i, sent := range []struct{ client, asn, asName, country string }{
{fromDE, asnDE, asNameDE, "DE"}, {fromKP, asnKP, asNameKP, "KP"},
{unplaced, "", "", ""},
for i, sent := range []struct{ client, country string }{
{fromDE, "DE"}, {fromKP, "KP"}, {unplaced, ""},
} {
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedFor, sent.client)
got := do(t, req)
line := out.requestLines(t, i+1)[i]
if line.ASN != sent.asn || line.ASName != sent.asName ||
line.Country != sent.country {
t.Errorf("log line has %q, %q and %q, want %q, %q and %q",
line.ASN, line.ASName, line.Country,
sent.asn, sent.asName, sent.country)
if line.Country != sent.country {
t.Errorf("log line has country %q, want %q", line.Country, sent.country)
}
if slices.Contains(tc.refused, sent.client) {
@@ -135,7 +130,7 @@ func TestRequestRefusedByCountryIsNotCounted(t *testing.T) {
return
}
answer := []geojsAnswer{{IP: r.URL.Query().Get("ip"), CountryCode: "DE"}}
answer := []map[string]string{{"ip": r.URL.Query().Get("ip"), "country": "DE"}}
err := json.NewEncoder(w).Encode(answer)
if err != nil {
@@ -179,15 +174,20 @@ func TestRequestRefusedByCountryIsNotCounted(t *testing.T) {
wantStatus(t, got, http.StatusOK)
}
func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
func TestCountryNotLookedUpWithoutAListOrForAPrivateAddress(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
env map[string]string
name string
env map[string]string
clients []string // "" sends no X-Forwarded-For: the client is 127.0.0.1
}{
{"no setting needs the lookup", nil},
{"a country list is set", map[string]string{deniedCountries: "kp"}},
{"no country list is set", nil, []string{fromKP, fromDE}},
{
"private, loopback and link-local addresses",
map[string]string{deniedCountries: "kp"},
[]string{"10.0.0.5", "192.168.1.9", "fd00::5", "", "169.254.0.9", "fe80::9"},
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
@@ -198,10 +198,7 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
maps.Copy(env, tc.env)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, env)
// "" sends no X-Forwarded-For: the client is 127.0.0.1.
for i, sent := range []string{
"10.0.0.5", "192.168.1.9", "fd00::5", "", "169.254.0.9", "fe80::9",
} {
for i, sent := range tc.clients {
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
if sent != "" {
req.Header.Set(forwardedFor, sent)
@@ -212,26 +209,15 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
line := out.requestLines(t, i+1)[i]
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
for _, field := range []string{"asn", "as_name", "country"} {
value, present := line.fields[field]
if !present || value != "" {
t.Errorf("log line for %q has %s %v, want an empty one",
line.ClientIP, field, value)
}
country, present := line.fields["country"]
if !present || country != "" {
t.Errorf("log line for %q has country %v, want an empty one",
line.ClientIP, country)
}
}
// GeoJS is asked about up to 200 waiting clients at once, so once it
// has been asked about fromDE, which comes last, it has been asked
// about every client before it that waited for an answer.
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedFor, fromDE)
wantStatus(t, do(t, req), http.StatusOK)
waitUntil(func() bool { return slices.Contains(asked(), fromDE) })
if got := asked(); !slices.Equal(got, []string{fromDE}) {
t.Errorf("GeoJS was asked about %v, want %s alone", got, fromDE)
if len(asked()) != 0 {
t.Errorf("GeoJS was asked about %v, want nothing", asked())
}
})
}
@@ -278,31 +264,18 @@ func TestExclusiveListRefusesAPrivateAddressUnlessAllowed(t *testing.T) {
}
}
// startGeoJS starts a stand-in for GeoJS, which places fromDE and fromKP,
// each in an AS of its own, and no other address. It returns its URL, and
// what returns the addresses it has been asked about.
// startGeoJS starts a stand-in for GeoJS, which places fromDE and fromKP
// and no other address. It returns its URL, and what returns the
// addresses it has been asked about.
func startGeoJS(t *testing.T) (string, func() []string) {
t.Helper()
geojsURL, asked, release := startHeldGeoJS(t)
release()
places := map[string]string{fromDE: "DE", fromKP: "KP"}
return geojsURL, asked
}
// startHeldGeoJS is startGeoJS for a stand-in that answers nothing until
// release is called. Each request to it waits until then.
func startHeldGeoJS(t *testing.T) (string, func() []string, func()) {
t.Helper()
var (
asked struct {
mu sync.Mutex
addrs []string
}
released = make(chan struct{})
once sync.Once
)
var asked struct {
mu sync.Mutex
addrs []string
}
geojs := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
@@ -312,11 +285,11 @@ func startHeldGeoJS(t *testing.T) (string, func() []string, func()) {
asked.addrs = append(asked.addrs, addrs...)
asked.mu.Unlock()
<-released
answers := make([]geojsAnswer, 0, len(addrs))
answers := make([]map[string]string, 0, len(addrs))
for _, addr := range addrs {
answers = append(answers, answerAbout(addr))
answers = append(answers, map[string]string{
"ip": addr, "country": places[addr],
})
}
err := json.NewEncoder(w).Encode(answers)
@@ -326,48 +299,10 @@ func startHeldGeoJS(t *testing.T) (string, func() []string, func()) {
}))
t.Cleanup(geojs.Close)
release := func() { once.Do(func() { close(released) }) }
// Run before geojs.Close, which waits for every request to be answered.
t.Cleanup(release)
return geojs.URL, func() []string {
asked.mu.Lock()
defer asked.mu.Unlock()
return slices.Clone(asked.addrs)
}, release
}
// The AS numbers and names the stand-in for GeoJS gives fromDE and
// fromKP, as they are logged.
const (
asnDE = "AS64496"
asNameDE = "Example Net"
asnKP = "AS64511"
asNameKP = "Other Net"
)
// geojsAnswer is an answer of GeoJS about one address, with the fields
// smallwebwaf reads.
//
//nolint:tagliatelle // GeoJS's own names
type geojsAnswer struct {
IP string `json:"ip"`
ASN int `json:"asn"`
ASName string `json:"organization_name"`
CountryCode string `json:"country_code,omitempty"`
}
// answerAbout is what the stand-in for GeoJS answers about addr: for an
// address it cannot place, the AS number 64512 and the AS name Unknown
// with no country, as GeoJS does.
func answerAbout(addr string) geojsAnswer {
switch addr {
case fromDE:
return geojsAnswer{IP: addr, ASN: 64496, ASName: asNameDE, CountryCode: "DE"}
case fromKP:
return geojsAnswer{IP: addr, ASN: 64511, ASName: asNameKP, CountryCode: "KP"}
}
return geojsAnswer{IP: addr, ASN: 64512, ASName: "Unknown"}
}
internal/proxy/health_test.go
+3 -12
View File
@@ -21,18 +21,14 @@ func TestHealthEndpointIsAnsweredBeforeAnyCheck(t *testing.T) {
// the last one would have it refused.
addr, out := startProxy(t, app.URL, map[string]string{rateLimitPerMinute: "1"})
const (
healthChecks = 3
contentType = "text/plain; charset=utf-8"
)
const healthChecks = 3
for range healthChecks {
got := get(t, addr, proxy.HealthPath)
wantStatus(t, got, http.StatusOK)
if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType {
t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+
"with %q", got.body, got.header.Get("Content-Type"), contentType)
if string(got.body) != "ok\n" {
t.Errorf("health endpoint answered %q, want ok", got.body)
}
}
@@ -41,11 +37,6 @@ func TestHealthEndpointIsAnsweredBeforeAnyCheck(t *testing.T) {
lines := out.requestLines(t, healthChecks+1)
for _, line := range lines[:healthChecks] {
wantLine(t, line, http.StatusOK, requestlog.ActionAdmin)
if line.ResponseContentType != contentType {
t.Errorf("health check's log line has response_content_type %q, "+
"want %q", line.ResponseContentType, contentType)
}
}
wantLine(t, lines[healthChecks], http.StatusOK, requestlog.ActionForward)
internal/proxy/history_test.go
+2 -6
View File
@@ -24,9 +24,7 @@ func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
start := clk.Now()
// Two let through, one over the limit, which bans the client, and one
// refused under that ban, for which the client is not looked up. GeoJS
// answers about the client at its first request, and its later ones
// use that answer.
// refused under that ban, for which the country is not looked up.
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
clk.advance(time.Second)
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
@@ -37,10 +35,8 @@ func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
want := ratelimit.History{
FirstSeen: start,
LastSeen: start.Add(2 * time.Second),
ASN: asnDE,
ASName: asNameDE,
Country: "DE",
LookedUp: start,
LookedUp: start.Add(time.Second),
Requests: 4,
Forwarded: 2,
Refused: 2,
internal/proxy/lookup.go
-71
View File
@@ -1,71 +0,0 @@
package proxy
import (
"context"
"net/http"
"net/netip"
"sneak.berlin/go/smallwebwaf/internal/lookup"
)
// The headers in which the app is passed the client's AS number and
// country while SWWAF_ADD_LOOKUP_HEADERS is set. Go writes every header
// name in this form, as it sends it and as it receives it, so X-Client-ASN
// arrives as X-Client-Asn, and Del removes a client's own whatever their
// case; header names are not case-sensitive.
const (
asnHeader = "X-Client-Asn"
countryHeader = "X-Client-Country"
)
// lookUp looks up the client's AS number and country, in the lookup
// database or through GeoJS, and notes them for the log line, unless
// SWWAF_LOOKUP_SOURCE is off or the client is on a private, loopback or
// link-local address, which no lookup can place. The lookup database
// answers at once. With GeoJS, while a setting needs the answer, such as a
// country list or a biased threshold, a new client's request waits for it.
// ctx is the request's own context.
func (rq *request) lookUp(ctx context.Context) {
if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
return
}
if rq.h.config.LookupSource == "file" {
rq.lookupAnswer = rq.h.lookupFile.LookUp(clientGroup(rq.client))
} else {
rq.lookupAnswer = rq.h.geojs.LookUp(ctx, clientGroup(rq.client))
}
rq.lookedUp = true
rq.line.ASN = rq.lookupAnswer.ASN
rq.line.ASName = rq.lookupAnswer.ASName
rq.line.Country = rq.lookupAnswer.Country
}
// addLookup adds answer, an answer about a client from the lookup
// database or GeoJS, to the client's history, and to the notes of the bans
// on its netblock that have no AS number, AS name or country yet.
func (h *handler) addLookup(answer lookup.Answer) {
h.limiter.AddLookup(answer.Client, answer.Answered,
answer.ASN, answer.ASName, answer.Country)
h.ledger.AddLookup(h.netblock(answer.Client.Addr()),
answer.ASN, answer.ASName, answer.Country)
}
// setLookupHeaders sets the headers in which the app is passed the
// client's AS number and country, leaving out one that is unknown.
func setLookupHeaders(header http.Header, asn, country string) {
if asn != "" {
header.Set(asnHeader, asn)
}
if country != "" {
header.Set(countryHeader, country)
}
}
// canBePlaced reports whether a lookup can place addr: private, loopback
// and link-local addresses have no AS number or country.
func canBePlaced(addr netip.Addr) bool {
return !addr.IsPrivate() && !addr.IsLoopback() && !addr.IsLinkLocalUnicast()
}
internal/proxy/lookup_test.go
-377
View File
@@ -1,377 +0,0 @@
package proxy_test
import (
"net"
"net/http"
"net/http/httptest"
"net/netip"
"path/filepath"
"slices"
"sync"
"testing"
"testing/synctest"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// asnAndCountry is what a lookup gives a client: its AS number, AS name
// and country.
type asnAndCountry struct{ asn, asName, country string }
// fileSource is the SWWAF_LOOKUP_SOURCE that looks clients up in the
// lookup database.
const fileSource = "file"
func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(t *testing.T) {
t.Parallel()
// The stand-in for GeoJS answers only once released. A request that
// waited for it would wait an hour, and get no answer within
// waitLimit.
geojsURL, asked, release := startHeldGeoJS(t)
s, _, server := startWithClock(t, geojsURL, map[string]string{
lookupTimeout: "1h",
rateLimitPerMinute: "1",
})
// fromDE's second request breaks the limit and bans it, and fromKP
// comes too. None waits for GeoJS.
for _, line := range []logLine{
s.get(fromDE, http.StatusOK, requestlog.ActionForward),
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited),
s.get(fromKP, http.StatusOK, requestlog.ActionForward),
} {
got := asnAndCountry{line.ASN, line.ASName, line.Country}
if got != (asnAndCountry{}) {
t.Errorf("log line has %+v before GeoJS answered, want nothing", got)
}
}
// Once GeoJS answers, each answer reaches the client's history, and
// fromDE's reaches the notes of its ban.
release()
netblock := netip.MustParsePrefix(fromDE + "/32")
waitUntil(func() bool {
return historyOf(t, server, fromDE).ASN != "" &&
historyOf(t, server, fromKP).ASN != "" &&
server.Ledger.Bans(netblock)[0].Notes.ASN != ""
})
de := asnAndCountry{asnDE, asNameDE, "DE"}
for addr, want := range map[string]asnAndCountry{
fromDE: de, fromKP: {asnKP, asNameKP, "KP"},
} {
h := historyOf(t, server, addr)
if got := (asnAndCountry{h.ASN, h.ASName, h.Country}); got != want {
t.Errorf("%s's history has %+v, want %+v", addr, got, want)
}
}
notes := server.Ledger.Bans(netblock)[0].Notes
if got := (asnAndCountry{notes.ASN, notes.ASName, notes.Country}); got != de {
t.Errorf("the ban's notes have %+v, want %+v", got, de)
}
// GeoJS was asked about each client once, fromKP after fromDE, whose
// request was under way when fromKP came.
if got := asked(); !slices.Equal(got, []string{fromDE, fromKP}) {
t.Errorf("GeoJS was asked about %v, want %s and %s", got, fromDE, fromKP)
}
}
func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(t *testing.T) {
t.Parallel()
geojsURL, _ := startGeoJS(t)
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)}
addr, out, server, queue := startProxyWithAlerts(t, app.URL, geojsURL, clk.Now,
map[string]string{
trustedProxies: trustLocalhost,
alertWebhookURL: "https://alerts.example/smallwebwaf",
rateLimitPerMinute: "1",
})
s := &sender{t: t, addr: addr, out: out}
// The answer is kept before the requests, so GeoJS is not asked, and
// gives no answer of its own.
netblock := netip.MustParsePrefix(fromDE + "/32")
server.GeoJS.Load([]lookup.Answer{{
Client: netblock, ASN: asnDE, ASName: asNameDE, Country: "DE",
Answered: clk.Now(), Used: clk.Now(),
}})
want := asnAndCountry{asnDE, asNameDE, "DE"}
for _, line := range []logLine{
s.get(fromDE, http.StatusOK, requestlog.ActionForward),
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited),
} {
if got := (asnAndCountry{line.ASN, line.ASName, line.Country}); got != want {
t.Errorf("log line has %+v, want %+v", got, want)
}
}
h := historyOf(t, server, fromDE)
if got := (asnAndCountry{h.ASN, h.ASName, h.Country}); got != want ||
!h.LookedUp.Equal(clk.Now()) {
t.Errorf("history has %+v, looked up at %s; want %+v, at %s",
got, h.LookedUp, want, clk.Now())
}
notes := server.Ledger.Bans(netblock)[0].Notes
if got := (asnAndCountry{notes.ASN, notes.ASName, notes.Country}); got != want {
t.Errorf("the ban's notes have %+v, want %+v", got, want)
}
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 {
t.Fatalf("alerts waiting %+v, want the ban's alone", waiting)
}
alert := waiting[0]
if got := (asnAndCountry{alert.ASN, alert.ASName, alert.Country}); got != want {
t.Errorf("the ban's alert has %+v, want %+v", got, want)
}
}
func TestLookupSourceOffLooksNoClientUp(t *testing.T) {
t.Parallel()
geojsURL, asked := startGeoJS(t)
s, clk, server := startWithClock(t, geojsURL, map[string]string{lookupSource: "off"})
// Even an answer kept from before is not used.
server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix(fromDE + "/32"), ASN: asnDE, ASName: asNameDE,
Country: "DE", Answered: clk.Now(), Used: clk.Now(),
}})
for _, from := range []string{fromDE, fromKP} {
line := s.get(from, http.StatusOK, requestlog.ActionForward)
got := asnAndCountry{line.ASN, line.ASName, line.Country}
if got != (asnAndCountry{}) {
t.Errorf("log line has %+v, want nothing", got)
}
}
if h := historyOf(t, server, fromDE); h.ASN != "" || !h.LookedUp.IsZero() {
t.Errorf("history has %q, looked up at %s, want no lookup", h.ASN, h.LookedUp)
}
if len(asked()) != 0 {
t.Errorf("GeoJS was asked about %v, want nothing", asked())
}
}
func TestClientsAreLookedUpInTheLookupDatabaseAndGeoJSIsNotAsked(t *testing.T) {
t.Parallel()
geojsURL, asked := startGeoJS(t)
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
lookuptest.Write(t, path, map[string]lookuptest.Network{
fromDE + "/32": {ASN: asnDE, ASName: asNameDE, Country: "DE"},
fromKP + "/32": {ASN: asnKP, ASName: asNameKP, Country: "KP"},
})
s, clk, server := startWithClock(t, geojsURL, map[string]string{
lookupSource: fileSource,
lookupDBPath: path,
allowedCountries: "DE",
rateLimitPerMinute: "1",
})
// fromDE's second request breaks the limit and bans it. The list
// refuses fromKP, and unplaced, which the file does not hold.
de := asnAndCountry{asnDE, asNameDE, "DE"}
for _, tc := range []struct {
line logLine
want asnAndCountry
}{
{s.get(fromDE, http.StatusOK, requestlog.ActionForward), de},
{s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited), de},
{
s.get(fromKP, http.StatusForbidden, requestlog.ActionCountryDenied),
asnAndCountry{asnKP, asNameKP, "KP"},
},
{
s.get(unplaced, http.StatusForbidden, requestlog.ActionCountryDenied),
asnAndCountry{},
},
} {
got := asnAndCountry{tc.line.ASN, tc.line.ASName, tc.line.Country}
if got != tc.want {
t.Errorf("log line has %+v, want %+v", got, tc.want)
}
}
h := historyOf(t, server, fromDE)
if got := (asnAndCountry{h.ASN, h.ASName, h.Country}); got != de ||
!h.LookedUp.Equal(clk.Now()) {
t.Errorf("history has %+v, looked up at %s; want %+v, at %s",
got, h.LookedUp, de, clk.Now())
}
notes := server.Ledger.Bans(netip.MustParsePrefix(fromDE + "/32"))[0].Notes
if got := (asnAndCountry{notes.ASN, notes.ASName, notes.Country}); got != de {
t.Errorf("the ban's notes have %+v, want %+v", got, de)
}
if len(asked()) != 0 {
t.Errorf("GeoJS was asked about %v, want nothing", asked())
}
}
func TestLookupHeadersArePassedToTheAppAndTheClientsOwnRemoved(t *testing.T) {
t.Parallel()
var (
mu sync.Mutex
got [][2][]string // each request's X-Client-ASN and X-Client-Country
)
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
got = append(got, [2][]string{
r.Header.Values("X-Client-Asn"), r.Header.Values("X-Client-Country"),
})
})
geojsURL, _ := startGeoJS(t)
addr, out, _ := startProxyWithClock(t, app.URL, geojsURL, time.Now, map[string]string{
trustedProxies: trustLocalhost,
addLookupHeaders: "true",
})
s := &sender{t: t, addr: addr, out: out}
// Each client sends headers of its own. fromDE's first request waits
// for its answer, which the app is passed; unplaced has none to pass,
// and a client on a private address is not looked up.
for _, from := range []string{fromDE, unplaced, "10.0.0.8"} {
s.requestWithHeader(from, "/", clientsOwnLookupHeaders,
http.StatusOK, requestlog.ActionForward)
}
mu.Lock()
defer mu.Unlock()
want := [][2][]string{{{asnDE}, {"DE"}}, {nil, nil}, {nil, nil}}
if !slices.EqualFunc(got, want, func(a, b [2][]string) bool {
return slices.Equal(a[0], b[0]) && slices.Equal(a[1], b[1])
}) {
t.Errorf("the app was passed %v, want %v", got, want)
}
}
func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(t *testing.T) {
t.Parallel()
var (
mu sync.Mutex
asn, country []string
)
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
asn, country = r.Header.Values("X-Client-Asn"), r.Header.Values("X-Client-Country")
})
geojsURL, _ := startGeoJS(t)
addr, out, _ := startProxyWithClock(t, app.URL, geojsURL, time.Now, map[string]string{
trustedProxies: trustLocalhost,
})
s := &sender{t: t, addr: addr, out: out}
s.requestWithHeader(fromDE, "/", clientsOwnLookupHeaders,
http.StatusOK, requestlog.ActionForward)
mu.Lock()
defer mu.Unlock()
if asn != nil || country != nil {
t.Errorf("the app was passed X-Client-ASN %v and X-Client-Country %v, want neither",
asn, country)
}
}
func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
t.Parallel()
// The test runs in a synctest bubble, where the time package runs on a
// clock of the test's own: the wait lasts exactly as long as it should,
// however slowly the test process runs. Nothing in it may wait on the
// network, which would keep that clock from moving on: the request is
// handed to the proxy's handler, and GeoJS is one that never answers.
synctest.Test(t, func(t *testing.T) {
// Not the default second. The exclusive list needs the answer, and
// the app is never reached.
const timeout = 3 * time.Second
server, out, _ := newProxy(t, "http://app.invalid", unansweredGeoJSURL,
time.Now, map[string]string{
lookupTimeout: timeout.String(),
allowedCountries: "DE",
})
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/",
http.NoBody)
req.RemoteAddr = net.JoinHostPort(fromDE, "1234")
began := time.Now()
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
if waited := time.Since(began); waited != timeout {
t.Errorf("the request waited %s for its answer, want %s", waited, timeout)
}
// Without an answer, the client is in no country the list allows.
wantLine(t, out.requestLine(t), http.StatusForbidden,
requestlog.ActionCountryDenied)
})
}
// unansweredGeoJSURL is where a GeoJS that never answers is asked: a
// request to it waits, without the network, until it is abandoned.
// TestMain registers it with Go's default transport, through which GeoJS
// is asked.
const unansweredGeoJSURL = "unanswered://geojs/v1/ip/geo.json"
func TestMain(m *testing.M) {
transport, _ := http.DefaultTransport.(*http.Transport)
transport.RegisterProtocol("unanswered", unansweredGeoJS{})
m.Run()
}
// unansweredGeoJS is the GeoJS at unansweredGeoJSURL.
type unansweredGeoJS struct{}
// RoundTrip waits until req is abandoned.
func (unansweredGeoJS) RoundTrip(req *http.Request) (*http.Response, error) {
<-req.Context().Done()
return nil, req.Context().Err()
}
// clientsOwnLookupHeaders are the X-Client-ASN and X-Client-Country a
// client sends of its own, each twice, in two cases.
const clientsOwnLookupHeaders = "X-Client-ASN: AS1\r\nx-client-asn: AS2\r\n" +
"X-CLIENT-COUNTRY: KP\r\nx-client-country: CN"
// waitUntil waits until done reports true, for at most waitLimit.
func waitUntil(done func() bool) {
deadline := time.Now().Add(waitLimit)
for !done() && time.Now().Before(deadline) {
time.Sleep(pollInterval)
}
}
internal/proxy/metrics_test.go
+44 -134
View File
@@ -12,7 +12,6 @@ import (
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
@@ -148,8 +147,8 @@ func TestMetricsCountTheTraffic(t *testing.T) {
wantStatus(t, get(t, addr, "/_smallwebwaf/nothing"), http.StatusNotFound)
out.requestLines(t, 2)
forward := `{action="forward",instance="app",status_class="2xx"}`
notFound := `{action="admin",instance="app",status_class="4xx"}`
forward := `{action="forward",status_class="2xx"}`
notFound := `{action="admin",status_class="4xx"}`
// The request for the metrics is itself under way.
metrics := scrape(t, addr)
@@ -159,13 +158,11 @@ func TestMetricsCountTheTraffic(t *testing.T) {
wantMetric(t, metrics, "smallwebwaf_response_bytes_total"+forward, 5)
wantMetric(t, metrics, "smallwebwaf_response_bytes_total"+notFound,
float64(len("Not Found\n")))
wantMetric(t, metrics,
`smallwebwaf_request_duration_seconds_count{instance="app"}`, 2)
wantMetric(t, metrics,
`smallwebwaf_upstream_duration_seconds_count{instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_requests_in_flight{instance="app"}`, 1)
metric(t, metrics, `go_goroutines{instance="app"}`)
metric(t, metrics, `process_start_time_seconds{instance="app"}`)
wantMetric(t, metrics, "smallwebwaf_request_duration_seconds_count", 2)
wantMetric(t, metrics, "smallwebwaf_upstream_duration_seconds_count", 1)
wantMetric(t, metrics, "smallwebwaf_requests_in_flight", 1)
metric(t, metrics, "go_goroutines")
metric(t, metrics, "process_start_time_seconds")
// A request the app holds is under way until it ends.
httpClient := newClient(t)
@@ -182,7 +179,7 @@ func TestMetricsCountTheTraffic(t *testing.T) {
}()
<-arrived
wantMetric(t, scrape(t, addr), `smallwebwaf_requests_in_flight{instance="app"}`, 2)
wantMetric(t, scrape(t, addr), "smallwebwaf_requests_in_flight", 2)
releaseApp()
err := <-ended
@@ -191,7 +188,7 @@ func TestMetricsCountTheTraffic(t *testing.T) {
}
out.requestLines(t, 5)
wantMetric(t, scrape(t, addr), `smallwebwaf_requests_in_flight{instance="app"}`, 1)
wantMetric(t, scrape(t, addr), "smallwebwaf_requests_in_flight", 1)
}
func TestMetricsCountLimitsAndBans(t *testing.T) {
@@ -221,16 +218,15 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
metrics := s.scrape(scraper)
wantMetric(t, metrics,
`smallwebwaf_requests_total{action="denied",instance="app",status_class="none"}`, 1)
wantMetric(t, metrics, `smallwebwaf_rate_limit_hits_total{instance="app",`+
`kind="requests",window="minute"}`, 1)
wantMetric(t, metrics, `smallwebwaf_offences_total{instance="app",kind="limit"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_active_bans{instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_permanent_bans{instance="app"}`, 0)
`smallwebwaf_requests_total{action="denied",status_class="none"}`, 1)
wantMetric(t, metrics, `smallwebwaf_rate_limit_hits_total{window="minute"}`, 1)
wantMetric(t, metrics, `smallwebwaf_offences_total{kind="limit"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit"}`, 1)
wantMetric(t, metrics, "smallwebwaf_active_bans", 1)
wantMetric(t, metrics, "smallwebwaf_permanent_bans", 0)
clk.advance(time.Hour)
wantMetric(t, s.scrape(scraper), `smallwebwaf_active_bans{instance="app"}`, 0)
wantMetric(t, s.scrape(scraper), "smallwebwaf_active_bans", 0)
// A limit broken again right after would ban for three hours, longer
// than SWWAF_MAX_BAN_DURATION, so the ban is permanent.
@@ -238,42 +234,13 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
s.get(client, 0, requestlog.ActionRateLimited)
metrics = s.scrape(scraper)
wantMetric(t, metrics, `smallwebwaf_rate_limit_hits_total{instance="app",`+
`kind="requests",window="minute"}`, 2)
wantMetric(t, metrics, `smallwebwaf_offences_total{instance="app",kind="limit"}`, 2)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 2)
wantMetric(t, metrics, `smallwebwaf_active_bans{instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_permanent_bans{instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_rate_limit_hits_total{window="minute"}`, 2)
wantMetric(t, metrics, `smallwebwaf_offences_total{kind="limit"}`, 2)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit"}`, 2)
wantMetric(t, metrics, "smallwebwaf_active_bans", 1)
wantMetric(t, metrics, "smallwebwaf_permanent_bans", 1)
// denied, client, and the scraper as of its earlier requests.
wantMetric(t, metrics, `smallwebwaf_tracked_clients{instance="app"}`, 3)
}
func TestMetricsCountTheBansAnAdminMakes(t *testing.T) {
t.Parallel()
const scraper = "192.0.2.200" // in SWWAF_RATE_LIMIT_EXEMPT_NETS
s, clk, server := startWithClock(t, "", map[string]string{
metricsToken: token,
adminToken: adminSecret,
rateLimitExemptNets: scraper,
})
const admins = `smallwebwaf_bans_made_total{cause="admin",instance="app"}`
wantMetric(t, s.scrape(scraper), admins, 0)
// As an admin's edit of bans.json that adds a ban is taken in.
server.Ledger.LoadEdit([]bans.Ban{{
Netblock: netip.MustParsePrefix(client + "/32"),
Start: clk.Now(),
}})
wantMetric(t, s.scrape(scraper), admins, 1)
// And a ban made through the endpoint.
s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK)
wantMetric(t, s.scrape(scraper), admins, 2)
wantMetric(t, metrics, "smallwebwaf_tracked_clients", 3)
}
func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
@@ -291,8 +258,7 @@ func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
metricsTopN: "2",
deniedCountries: "kp",
}
geojsURL, _ := startGeoJS(t)
addr, out, server := startProxyWithClock(t, app.URL, geojsURL, time.Now, env)
addr, out, server := startProxyWithClock(t, app.URL, "", time.Now, env)
// The answers are kept before the requests, so that none waits for
// GeoJS.
@@ -324,82 +290,28 @@ func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
metrics := scrape(t, addr)
lines++
wantMetric(t, metrics,
`smallwebwaf_country_requests_total{country="KP",instance="app"}`, 3)
wantMetric(t, metrics,
`smallwebwaf_country_requests_total{country="DE",instance="app"}`, 2)
wantMetric(t, metrics,
`smallwebwaf_country_requests_total{country="other",instance="app"}`, 1)
wantMetric(t, metrics,
`smallwebwaf_country_list_refusals_total{country="KP",instance="app"}`, 3)
wantMetric(t, metrics,
`smallwebwaf_country_request_bytes_total{country="KP",instance="app"}`, 0)
wantMetric(t, metrics,
`smallwebwaf_country_request_bytes_total{country="DE",instance="app"}`, 6)
wantMetric(t, metrics,
`smallwebwaf_country_response_bytes_total{country="KP",instance="app"}`,
wantMetric(t, metrics, `smallwebwaf_country_requests_total{country="KP"}`, 3)
wantMetric(t, metrics, `smallwebwaf_country_requests_total{country="DE"}`, 2)
wantMetric(t, metrics, `smallwebwaf_country_requests_total{country="other"}`, 1)
wantMetric(t, metrics, `smallwebwaf_country_list_refusals_total{country="KP"}`, 3)
wantMetric(t, metrics, `smallwebwaf_country_request_bytes_total{country="KP"}`, 0)
wantMetric(t, metrics, `smallwebwaf_country_request_bytes_total{country="DE"}`, 6)
wantMetric(t, metrics, `smallwebwaf_country_response_bytes_total{country="KP"}`,
float64(3*len("Forbidden\n")))
wantMetric(t, metrics,
`smallwebwaf_country_response_bytes_total{country="other",instance="app"}`,
wantMetric(t, metrics, `smallwebwaf_country_response_bytes_total{country="other"}`,
float64(len("hello")))
wantNoSeries(t, metrics,
`smallwebwaf_country_requests_total{country="FR",instance="app"}`)
wantNoSeries(t, metrics, `smallwebwaf_country_requests_total{country="FR"}`)
// Once FR is busier than DE, it takes DE's place: its series counts
// from then on, and DE's is gone.
send(fromFR, 3, http.StatusOK)
metrics = scrape(t, addr)
wantMetric(t, metrics,
`smallwebwaf_country_requests_total{country="KP",instance="app"}`, 3)
wantMetric(t, metrics,
`smallwebwaf_country_requests_total{country="FR",instance="app"}`, 2)
wantMetric(t, metrics,
`smallwebwaf_country_requests_total{country="other",instance="app"}`, 2)
wantNoSeries(t, metrics,
`smallwebwaf_country_requests_total{country="DE",instance="app"}`)
wantNoSeries(t, metrics,
`smallwebwaf_country_request_bytes_total{country="DE",instance="app"}`)
}
func TestMetricsByASNumberKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
t.Parallel()
geojsURL, _ := startGeoJS(t)
s, clk, server := startWithClock(t, geojsURL, map[string]string{
metricsToken: token,
metricsTopN: "1",
})
// The answers are kept before the requests, so that GeoJS gives none
// of its own. Each client is in an AS of its own.
answer := func(addr, asn string) lookup.Answer {
return lookup.Answer{
Client: netip.MustParsePrefix(addr + "/32"), ASN: asn,
Answered: clk.Now(), Used: clk.Now(),
}
}
server.GeoJS.Load([]lookup.Answer{
answer(fromDE, "AS64501"), answer(fromKP, "AS64502"),
})
// With one AS number of its own, the other is counted as other. The
// metrics are asked for from a private address, which has no AS number.
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
s.get(fromKP, http.StatusOK, requestlog.ActionForward)
metrics := s.scrape("10.0.0.9")
wantMetric(t, metrics,
`smallwebwaf_asn_requests_total{asn="AS64501",instance="app"}`, 2)
wantMetric(t, metrics,
`smallwebwaf_asn_requests_total{asn="other",instance="app"}`, 1)
wantMetric(t, metrics,
`smallwebwaf_asn_request_bytes_total{asn="AS64501",instance="app"}`, 0)
wantMetric(t, metrics,
`smallwebwaf_asn_response_bytes_total{asn="other",instance="app"}`, 0)
wantNoSeries(t, metrics,
`smallwebwaf_asn_requests_total{asn="AS64502",instance="app"}`)
wantMetric(t, metrics, `smallwebwaf_country_requests_total{country="KP"}`, 3)
wantMetric(t, metrics, `smallwebwaf_country_requests_total{country="FR"}`, 2)
wantMetric(t, metrics, `smallwebwaf_country_requests_total{country="other"}`, 2)
wantNoSeries(t, metrics, `smallwebwaf_country_requests_total{country="DE"}`)
wantNoSeries(t, metrics, `smallwebwaf_country_request_bytes_total{country="DE"}`)
}
func TestMetricsCountGeoJSRequestsAndFailures(t *testing.T) {
@@ -429,16 +341,16 @@ func TestMetricsCountGeoJSRequestsAndFailures(t *testing.T) {
deadline := time.Now().Add(waitLimit)
metrics := scrape(t, addr)
for metric(t, metrics, `smallwebwaf_geojs_failures_total{instance="app"}`) == 0 &&
for metric(t, metrics, "smallwebwaf_geojs_failures_total") == 0 &&
time.Now().Before(deadline) {
time.Sleep(pollInterval)
metrics = scrape(t, addr)
}
wantMetric(t, metrics, `smallwebwaf_geojs_requests_total{instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_geojs_failures_total{instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_geojs_unanswered_total{instance="app"}`, 1)
wantMetric(t, metrics, "smallwebwaf_geojs_requests_total", 1)
wantMetric(t, metrics, "smallwebwaf_geojs_failures_total", 1)
wantMetric(t, metrics, "smallwebwaf_geojs_unanswered_total", 1)
}
// keptAnswer returns GeoJS's answer that the client at addr is in
@@ -481,9 +393,8 @@ func (s *sender) scrape(from string) string {
// metric returns the value of series in metrics, which are in the
// Prometheus text format. series is a name and its labels in the order of
// their names, such as
// smallwebwaf_offences_total{instance="app",kind="limit"}. It fails the
// test if there is no such series.
// their names, such as smallwebwaf_offences_total{kind="limit"}. It fails
// the test if there is no such series.
func metric(t *testing.T, metrics, series string) float64 {
t.Helper()
@@ -531,8 +442,7 @@ func wantNoSeries(t *testing.T, metrics, series string) {
func wantLimitHits(t *testing.T, addr, limit string, hits int) {
t.Helper()
series := `smallwebwaf_size_and_time_limit_hits_total{instance="app",limit="` +
limit + `"}`
series := `smallwebwaf_size_and_time_limit_hits_total{limit="` + limit + `"}`
metrics := scrape(t, addr)
if hits == 0 {
internal/proxy/observe_test.go
-1
View File
@@ -95,7 +95,6 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
Netblock: netip.MustParsePrefix(otherClient + "/32"),
Start: clk.Now(),
Expires: clk.Now().Add(time.Hour),
Cause: bans.CauseAdmin,
}
server.Ledger.Load([]bans.Ban{kept})
internal/proxy/passthrough_test.go
+15 -31
View File
@@ -15,7 +15,6 @@ import (
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
@@ -117,37 +116,27 @@ func wantAnswer(t *testing.T, got answer, body []byte) {
}
}
// wantRequestFields checks the log line's fields about the request. Its
// time, its id and its timings are checked only for being there.
// wantRequestFields checks the log line's fields about the request.
func wantRequestFields(t *testing.T, line logLine, host string, sent, received int) {
t.Helper()
bytes := float64(sent + received)
want := withTimings(line, requestlog.Line{
Type: requestType, Time: line.Time, Instance: "app",
ClientIP: localhost, Method: http.MethodPatch, Scheme: plain, Host: host,
Path: rawPath, Query: rawQuery, Protocol: protocol,
Status: http.StatusTeapot, RequestBytes: int64(sent),
want := requestlog.Line{
Type: "request", Time: line.Time, ClientIP: localhost, PeerIP: localhost,
Method: http.MethodPatch, Host: host, Path: rawPath, Query: rawQuery,
Protocol: "HTTP/1.1", Status: http.StatusTeapot,
UpstreamStatus: http.StatusTeapot, RequestBytes: int64(sent),
ResponseBytes: int64(received), UserAgent: "test-agent",
RequestID: line.RequestID, PeerIP: localhost, ClientGroup: localhost + "/32",
ContentLength: int64(sent), ResponseContentType: "text/plain; charset=utf-8",
UpstreamStatus: http.StatusTeapot, Action: requestlog.ActionForward,
Counts: ratelimit.Counts{
Minute: 1, Hour: 1, Day: 1,
MinuteBytes: bytes, HourBytes: bytes, DayBytes: bytes,
},
})
Action: requestlog.ActionForward, DurationTotal: line.DurationTotal,
DurationUpstreamTotal: line.DurationUpstreamTotal,
}
if !reflect.DeepEqual(line.Line, want) {
t.Errorf("log line\n%+v\nwant\n%+v", line.Line, want)
}
_, err := time.Parse(time.RFC3339, line.Time)
if err != nil || line.RequestID == "" || line.DurationTotal <= 0 ||
line.DurationUpstreamTotal == nil || *line.DurationUpstreamTotal <= 0 {
t.Errorf("log line has time %q, request_id %q and durations %v and %v",
line.Time, line.RequestID, line.DurationTotal,
line.fields["duration_upstream_total"])
if err != nil || line.DurationTotal <= 0 || line.DurationUpstreamTotal <= 0 {
t.Errorf("log line has time %q and durations %v and %v",
line.Time, line.DurationTotal, line.DurationUpstreamTotal)
}
}
@@ -320,7 +309,7 @@ func TestServerHasTheDefaultLimits(t *testing.T) {
server := proxy.New(proxy.Params{
Config: cfg,
RequestLog: io.Discard,
ProcessLog: requestlog.NewProcessLogger(io.Discard, cfg.InstanceName),
ProcessLog: requestlog.NewProcessLogger(io.Discard),
})
if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 ||
@@ -383,13 +372,8 @@ func TestAnswers502WhenTheAppCannotBeReached(t *testing.T) {
addr, out := startProxy(t, "http://"+localhost+":1", nil)
wantStatus(t, get(t, addr, "/"), http.StatusBadGateway)
line := out.requestLine(t)
wantLine(t, line, http.StatusBadGateway, requestlog.ActionUpstreamError)
// There never was a connection to the app, nor an answer from it.
wantTimings(t, line, "duration_total", "duration_checks",
"duration_upstream_total")
wantLine(t, out.requestLine(t), http.StatusBadGateway,
requestlog.ActionUpstreamError)
logged := slices.ContainsFunc(out.lines(t), func(line map[string]any) bool {
return line["type"] == "process" && line["msg"] == "request to the app failed"
internal/proxy/proxy.go
+21 -81
View File
@@ -11,8 +11,6 @@ import (
"strings"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup"
@@ -40,14 +38,6 @@ const HealthPath = "/_smallwebwaf/healthz"
// SWWAF_METRICS_TOKEN.
const MetricsPath = "/_smallwebwaf/metrics"
// BansPath is where an admin lists and adds bans, and, followed by / and
// a client's address, lifts them, with SWWAF_ADMIN_TOKEN.
const BansPath = "/_smallwebwaf/bans"
// ClientsPath is where an admin asks what smallwebwaf knows of a client,
// by the client's address after it, with SWWAF_ADMIN_TOKEN.
const ClientsPath = "/_smallwebwaf/clients/"
// Params are what New needs.
type Params struct {
Config *config.Config
@@ -55,12 +45,9 @@ type Params struct {
RequestLog io.Writer
// ProcessLog receives the process's own messages.
ProcessLog *slog.Logger
// GeoJSURL is where clients' AS numbers and countries are looked up
// while SWWAF_LOOKUP_SOURCE is geojs, normally lookup.URL.
// GeoJSURL is where clients' countries are looked up, normally
// lookup.URL. GeoJS is asked only while a country list is set.
GeoJSURL string
// LookupFile is the lookup database they are looked up in while
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
LookupFile *lookup.File
// Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give.
@@ -68,24 +55,17 @@ type Params struct {
// Rules are the rule files' rules, which each request is checked
// against.
Rules *rules.Files
// Alerts receive the alert for each ban the proxy makes or makes
// permanent, for each count over an anomaly threshold, and for GeoJS
// failing.
Alerts *alerts.Queue
}
// Server is the server smallwebwaf runs, with the parts of the proxy
// whose state the state files keep, the lookup database, nil unless
// SWWAF_LOOKUP_SOURCE is file, and the metrics.
// whose state the state files keep, and the metrics.
type Server struct {
*http.Server
Ledger *bans.Ledger
Limiter *ratelimit.Limiter
GeoJS *lookup.GeoJS
Anomalies *anomaly.Counters
LookupFile *lookup.File
Metrics *metrics.Metrics
Ledger *bans.Ledger
Limiter *ratelimit.Limiter
GeoJS *lookup.GeoJS
Metrics *metrics.Metrics
}
// New returns the server smallwebwaf runs: each request it reads passes
@@ -96,7 +76,7 @@ type Server struct {
// applies the timeouts and size limits from then on.
func New(params Params) *Server {
errorLog := slog.NewLogLogger(params.ProcessLog.Handler(), slog.LevelWarn)
m := metrics.New(params.Config.MetricsTopN, params.Config.InstanceName)
m := metrics.New(params.Config.MetricsTopN)
h := &handler{
config: params.Config,
requestLog: params.RequestLog,
@@ -106,12 +86,9 @@ func New(params Params) *Server {
now: params.Now,
metrics: m,
limiter: ratelimit.New(ratelimit.Limits{
PerMinute: params.Config.RateLimitPerMinute,
PerHour: params.Config.RateLimitPerHour,
PerDay: params.Config.RateLimitPerDay,
BytesPerMinute: params.Config.BytesLimitPerMinute,
BytesPerHour: params.Config.BytesLimitPerHour,
BytesPerDay: params.Config.BytesLimitPerDay,
PerMinute: params.Config.RateLimitPerMinute,
PerHour: params.Config.RateLimitPerHour,
PerDay: params.Config.RateLimitPerDay,
}),
ledger: bans.New(bans.Rules{
LimitBanDuration: params.Config.LimitBanDuration,
@@ -120,35 +97,14 @@ func New(params Params) *Server {
AttackBanDuration: params.Config.AttackBanDuration,
MaxBans: params.Config.MaxBans,
}),
anomalies: anomaly.New(anomaly.Params{
Client: params.Config.AnomalyClient,
Net: params.Config.AnomalyNet,
ASN: params.Config.AnomalyASN,
Total: params.Config.AnomalyTotal,
Watch: params.Config.AnomalyWatch,
NetV4Prefix: params.Config.AnomalyNetV4Prefix,
NetV6Prefix: params.Config.AnomalyNetV6Prefix,
NamedNetblocks: params.Config.WatchNets,
Alerts: params.Alerts,
geojs: lookup.New(lookup.Params{
URL: params.GeoJSURL,
Now: params.Now,
ProcessLog: params.ProcessLog,
Metrics: m,
}),
lookupFile: params.LookupFile,
rules: params.Rules,
alerts: params.Alerts,
rules: params.Rules,
}
h.geojs = lookup.New(lookup.Params{
URL: params.GeoJSURL,
Timeout: params.Config.LookupTimeout,
// The country lists, the headers and the biased thresholds act on
// the answer before the request goes on.
Wait: len(params.Config.DeniedCountries) > 0 ||
len(params.Config.ExclusivelyAllowedCountries) > 0 ||
params.Config.AddLookupHeaders || biasedThresholdsSet(params.Config),
Answered: h.addLookup,
Now: params.Now,
ProcessLog: params.ProcessLog,
Metrics: m,
Alerts: params.Alerts,
})
m.AddBansAndClients(h.ledger, h.limiter, params.Now)
m.AddRules(params.Rules)
@@ -165,12 +121,10 @@ func New(params Params) *Server {
MaxHeaderBytes: int(params.Config.ClientRequestHeaderMaxBytes - 4<<10),
ErrorLog: errorLog,
},
Ledger: h.ledger,
Limiter: h.limiter,
GeoJS: h.geojs,
Anomalies: h.anomalies,
LookupFile: h.lookupFile,
Metrics: m,
Ledger: h.ledger,
Limiter: h.limiter,
GeoJS: h.geojs,
Metrics: m,
}
}
@@ -187,10 +141,7 @@ type handler struct {
limiter *ratelimit.Limiter
ledger *bans.Ledger
geojs *lookup.GeoJS
anomalies *anomaly.Counters
lookupFile *lookup.File
rules *rules.Files
alerts *alerts.Queue
}
// newTransport returns what carries requests to the app. It never goes
@@ -217,9 +168,6 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// a health checker is never refused. It does not ask the app.
if r.Method == http.MethodGet && r.URL.Path == HealthPath {
rq.line.Action = requestlog.ActionAdmin
// Set here rather than left to Go's server, which would set it only
// after the log line has taken the response's headers.
rq.out.Header().Set("Content-Type", "text/plain; charset=utf-8")
_, _ = io.WriteString(rq.out, "ok\n")
return
@@ -227,11 +175,8 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Once the request has ended, before its log line is written.
defer rq.addToHistory()
defer rq.countAnomalies()
refused := rq.check(r.Context())
rq.checked = time.Now()
if refused != nil {
rq.answer(*refused)
@@ -246,10 +191,5 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
return
}
// Once the response has ended, before the request is added to its
// client's history. Deferred, since ReverseProxy panics to end a
// response it cannot finish.
defer rq.countBytes()
rq.forward(r.Context())
}
internal/proxy/proxy_test.go
+32 -115
View File
@@ -14,9 +14,7 @@ import (
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
@@ -38,10 +36,6 @@ const (
// localhost is where every test server listens, and so the address
// smallwebwaf sees each test's requests come from.
localhost = "127.0.0.1"
// requestType is the type that marks a request log line.
requestType = "request"
// protocol is the protocol of every test's requests.
protocol = "HTTP/1.1"
)
// shortTimeoutSetting is shortTimeout as a setting's value.
@@ -66,11 +60,6 @@ const (
denyNets = "SWWAF_DENY_NETS"
rateLimitPerMinute = "SWWAF_RATE_LIMIT_PER_MINUTE"
rateLimitPerDay = "SWWAF_RATE_LIMIT_PER_DAY"
rateLimitExemptPaths = "SWWAF_RATE_LIMIT_EXEMPT_PATHS"
lookupSource = "SWWAF_LOOKUP_SOURCE"
lookupDBPath = "SWWAF_LOOKUP_DB_PATH"
lookupTimeout = "SWWAF_LOOKUP_TIMEOUT"
addLookupHeaders = "SWWAF_ADD_LOOKUP_HEADERS"
deniedCountries = "SWWAF_DENIED_COUNTRIES"
allowedCountries = "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES"
banResponse = "SWWAF_BAN_RESPONSE"
@@ -79,8 +68,6 @@ const (
maxBanDuration = "SWWAF_MAX_BAN_DURATION"
maxBans = "SWWAF_MAX_BANS"
banScopeV4Prefix = "SWWAF_BAN_SCOPE_V4_PREFIX"
instanceName = "SWWAF_INSTANCE_NAME"
logRequestHeaders = "SWWAF_LOG_REQUEST_HEADERS"
attackBanDuration = "SWWAF_ATTACK_BAN_DURATION"
rulesDir = "SWWAF_RULES_DIR"
)
@@ -99,14 +86,6 @@ func (o *output) Write(p []byte) (int, error) {
return o.buf.Write(p)
}
// text returns everything written so far.
func (o *output) text() string {
o.mu.Lock()
defer o.mu.Unlock()
return o.buf.String()
}
// lines returns every line written so far, decoded.
func (o *output) lines(t *testing.T) []map[string]any {
t.Helper()
@@ -146,7 +125,7 @@ func (o *output) requestLines(t *testing.T, count int) []logLine {
var found []logLine
for _, fields := range o.lines(t) {
if fields["type"] == requestType {
if fields["type"] == "request" {
found = append(found, decodeLine(t, fields))
}
}
@@ -208,8 +187,8 @@ func startProxy(t *testing.T, appURL string, env map[string]string) (string, *ou
return startProxyWithGeoJS(t, appURL, "", env)
}
// startProxyWithGeoJS is startProxy with clients' AS numbers and
// countries looked up at geojsURL.
// startProxyWithGeoJS is startProxy with clients' countries looked up at
// geojsURL.
func startProxyWithGeoJS(
t *testing.T, appURL, geojsURL string, env map[string]string,
) (string, *output) {
@@ -222,29 +201,43 @@ func startProxyWithGeoJS(
// startProxyWithClock is startProxyWithGeoJS with requests counted and
// bans made by the time now tells, and returns the server as well. Unless
// env sets SWWAF_RULES_DIR, it is an empty directory, of no rules, and
// unless it sets SWWAF_INSTANCE_NAME, that is app, the label instance of
// every metric.
// env sets SWWAF_RULES_DIR, it is an empty directory, of no rules.
func startProxyWithClock(
t *testing.T, appURL, geojsURL string, now func() time.Time,
env map[string]string,
) (string, *output, *proxy.Server) {
t.Helper()
addr, out, server, _ := startProxyWithAlerts(t, appURL, geojsURL, now, env)
settings := map[string]string{"SWWAF_UPSTREAM_URL": appURL, rulesDir: t.TempDir()}
maps.Copy(settings, env)
return addr, out, server
}
cfg, err := config.FromEnvironment(func(name string) (string, bool) {
value, ok := settings[name]
// startProxyWithAlerts is startProxyWithClock, and returns the queue of
// the alerts the proxy raises as well, as newProxy makes them.
func startProxyWithAlerts(
t *testing.T, appURL, geojsURL string, now func() time.Time,
env map[string]string,
) (string, *output, *proxy.Server, *alerts.Queue) {
t.Helper()
return value, ok
})
if err != nil {
t.Fatalf("settings %v: %v", settings, err)
}
server, out, alertQueue := newProxy(t, appURL, geojsURL, now, env)
out := &output{}
processLog := requestlog.NewProcessLogger(out)
ruleFiles, err := rules.Load(rules.Params{
Dir: cfg.RulesDir, Enabled: cfg.RulesEnabled, ProcessLog: processLog,
})
if err != nil {
t.Fatalf("rule files: %v", err)
}
server := proxy.New(proxy.Params{
Config: cfg,
RequestLog: out,
ProcessLog: processLog,
GeoJSURL: geojsURL,
Now: now,
Rules: ruleFiles,
})
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
if err != nil {
@@ -259,83 +252,7 @@ func startProxyWithAlerts(
_ = server.Close()
})
return listener.Addr().String(), out, server, alertQueue
}
// newProxy makes the server startProxyWithClock starts, without starting
// it, and returns it, what it writes, and the queue of the alerts the
// proxy raises, as the settings in env make it. No alert is sent from the
// queue: they wait in it, for the test to look at. With no geojsURL, there
// is no stand-in for GeoJS to look clients up at, and SWWAF_LOOKUP_SOURCE
// is off unless env sets it. While it is file, the lookup database
// SWWAF_LOOKUP_DB_PATH names is read.
func newProxy(
t *testing.T, appURL, geojsURL string, now func() time.Time,
env map[string]string,
) (*proxy.Server, *output, *alerts.Queue) {
t.Helper()
settings := map[string]string{
"SWWAF_UPSTREAM_URL": appURL, rulesDir: t.TempDir(), instanceName: "app",
}
if geojsURL == "" {
settings[lookupSource] = "off"
}
maps.Copy(settings, env)
cfg, err := config.FromEnvironment(func(name string) (string, bool) {
value, ok := settings[name]
return value, ok
})
if err != nil {
t.Fatalf("settings %v: %v", settings, err)
}
out := &output{}
processLog := requestlog.NewProcessLogger(out, cfg.InstanceName)
ruleFiles, err := rules.Load(rules.Params{
Dir: cfg.RulesDir, Enabled: cfg.RulesEnabled, ProcessLog: processLog,
})
if err != nil {
t.Fatalf("rule files: %v", err)
}
alertQueue := alerts.New(alerts.Params{
WebhookURL: cfg.AlertWebhookURL,
Events: cfg.AlertEvents,
Cooldown: cfg.AlertCooldown,
MaxPerHour: cfg.AlertMaxPerHour,
Instance: cfg.InstanceName,
Now: now,
ProcessLog: processLog,
})
var lookupFile *lookup.File
if cfg.LookupSource == fileSource {
lookupFile, err = lookup.OpenFile(lookup.FileParams{
Path: cfg.LookupDBPath, Now: now, ProcessLog: processLog, Alerts: alertQueue,
})
if err != nil {
t.Fatalf("lookup database: %v", err)
}
}
server := proxy.New(proxy.Params{
Config: cfg,
RequestLog: out,
ProcessLog: processLog,
GeoJSURL: geojsURL,
LookupFile: lookupFile,
Now: now,
Rules: ruleFiles,
Alerts: alertQueue,
})
return server, out, alertQueue
return listener.Addr().String(), out, server
}
// newClient returns an HTTP client that sends requests as they are made,
internal/proxy/ratelimits_test.go
-89
View File
@@ -5,8 +5,6 @@ import (
"sync/atomic"
"testing"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
@@ -71,90 +69,3 @@ func TestRateLimitRefusesBeforeTheApp(t *testing.T) {
t.Errorf("the app was called %d times, want 4", calls.Load())
}
}
func TestRateLimitExemptPathsAreNeitherCountedNorRefused(t *testing.T) {
t.Parallel()
const denied = "192.0.2.50" // in SWWAF_DENY_NETS
geojsURL, _ := startGeoJS(t)
s, _, server := startWithClock(t, geojsURL, map[string]string{
rateLimitPerMinute: "1",
rateLimitExemptPaths: "/assets/,/favicon.ico",
denyNets: denied,
deniedCountries: "kp",
})
// The answers are kept before the requests, so that none waits for
// GeoJS.
server.GeoJS.Load([]lookup.Answer{
keptAnswer(client, "DE"), keptAnswer(fromKP, "KP"),
})
// With a limit of one request a minute, the requests for paths under a
// prefix are not counted, so client's first request for / is within
// the limit; and once client has reached it, they are not refused.
s.request(client, "/assets/app.js", http.StatusOK, requestlog.ActionForward)
s.request(client, "/favicon.ico?v=2", http.StatusOK, requestlog.ActionForward)
s.get(client, http.StatusOK, requestlog.ActionForward)
line := s.request(client, "/assets/app.js", http.StatusOK, requestlog.ActionForward)
if line.LimitHit != "" || line.Counts != (ratelimit.Counts{}) {
t.Errorf("log line has limit_hit %q and counts %+v, want neither",
line.LimitHit, line.Counts)
}
// A path outside every prefix is counted: /assets is not under
// /assets/, and breaks the limit.
s.request(client, "/assets", http.StatusForbidden, requestlog.ActionRateLimited)
// A ban, SWWAF_DENY_NETS and the country lists still refuse a path
// under a prefix.
s.request(client, "/assets/app.js", http.StatusForbidden, requestlog.ActionBanned)
s.request(denied, "/assets/app.js", http.StatusForbidden, requestlog.ActionDenied)
s.request(fromKP, "/assets/app.js",
http.StatusForbidden, requestlog.ActionCountryDenied)
}
func TestRateLimitCountsPathsThatAreNotExempt(t *testing.T) {
t.Parallel()
for _, sent := range []string{
// A prefix matches only at the start of the path.
"/static/assets/app.js",
// A prefix matches the path as sent: a router that matches the
// path as received does not take /%61ssets/x for a path under
// /assets/.
"/%61ssets/x",
// .. once percent-decoded: an app may act on these as /login, the
// last as a path under /sneak/app/ or as /assets/x.
"/assets/../login",
"/assets/%2e%2e/login",
"/assets/..%2Flogin",
"/assets/..;/login",
"/sneak/app/src/branch/main/..%2F..%2F..%2F..%2F..%2F..%2Fassets/x",
// Not under /assets/ as sent: Go's router takes /assets%2Fx for one
// path segment, not a path under /assets/.
"/assets%2Fx",
"/assets%2fx",
// Under /assets/ as sent, but holding an encoded slash, in either
// case, or a backslash: never exempt, whatever the prefix.
"/assets/x%2Fy",
"/assets/x%2fy",
`/assets/x\y`,
} {
t.Run(sent, func(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{
rateLimitPerMinute: "1",
rateLimitExemptPaths: "/assets/",
})
// Counted, the second request breaks the limit of one request
// a minute.
s.request(client, sent, http.StatusOK, requestlog.ActionForward)
s.request(client, sent, http.StatusForbidden, requestlog.ActionRateLimited)
})
}
}
internal/proxy/request.go
+47 -271
View File
@@ -3,22 +3,15 @@ package proxy
import (
"context"
"errors"
"io"
"net/http"
"net/http/httptrace"
"net/http/httputil"
"net/netip"
"net/url"
"os"
"slices"
"strings"
"sync"
"sync/atomic"
"time"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
@@ -52,21 +45,8 @@ type request struct {
client netip.Addr
peer netip.Addr
peerTrusted bool
// lookedUp is true once the client's AS number and country have been
// looked up, whether or not an answer was there, and lookupAnswer is
// what the lookup gave then, the zero Answer while GeoJS had given none.
lookedUp bool
lookupAnswer lookup.Answer
// counted is true for a request the rate limits counted, whose bytes
// the byte limits count once it has ended. limitPercent and
// bytesPercent are then its client's limit percentages for the rate
// limits and for the byte limits.
counted bool
limitPercent, bytesPercent percentage
start time.Time
// checked is when the checks were done, and upstreamStart when the
// request was handed to the app.
checked time.Time
start time.Time
// upstreamStart is when the request was handed to the app.
upstreamStart time.Time
// cancel ends the request to the app.
cancel context.CancelFunc
@@ -74,39 +54,28 @@ type request struct {
refused atomic.Pointer[refusal]
// complete is true once the app's whole answer has been passed on.
complete bool
// upgraded is the connection to the app once the app has switched
// protocols, as for a WebSocket, and nil otherwise.
upgraded *upgradedConn
// mu guards what follows. The timeouts run on goroutines of their
// own, and the transport starts and stops them, and notes the times
// below, from its own; once timersStopped is set, none of the timeouts
// acts any more.
// own, and the transport starts and stops them from its own; once
// timersStopped is set, none of them acts any more.
mu sync.Mutex
timersStopped bool
clientRequestTimer *time.Timer
upstreamRequestTimer *time.Timer
upstreamResponseTimer *time.Timer
// connected is when there was a connection to the app, requestSent
// when the app had been sent the whole request, and answerStarted
// when the first byte of its answer arrived.
connected time.Time
requestSent time.Time
answerStarted time.Time
// requestSent is when the app had been sent the whole request.
requestSent time.Time
}
// newRequest starts handling r: it notes the time, counts the request as
// under way, works out the client, and starts the log line with what is
// known of the request.
// under way, and works out the client.
func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
h.metrics.RequestStarted()
start := time.Now()
peer := peerAddress(r)
trusted := h.config.TrustedProxies
peerTrusted := isInside(peer, trusted)
forwardedFor := r.Header.Values("X-Forwarded-For")
client := clientAddress(peer, forwardedFor, trusted)
client := clientAddress(peer, r.Header.Values("X-Forwarded-For"), trusted)
rq := &request{
h: h,
@@ -115,37 +84,22 @@ func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
out: &responseWriter{ResponseWriter: w},
client: client,
peer: peer,
peerTrusted: peerTrusted,
peerTrusted: isInside(peer, trusted),
start: start,
line: requestlog.Line{
Time: requestlog.FormatTime(start),
Instance: h.config.InstanceName,
ClientIP: client.String(),
Method: r.Method,
Scheme: scheme(r, peerTrusted),
Host: r.Host,
Path: r.URL.EscapedPath(),
Query: r.URL.RawQuery,
Protocol: r.Proto,
Referer: r.Referer(),
UserAgent: r.UserAgent(),
RequestID: requestID(r, peerTrusted),
PeerIP: peer.String(),
ForwardedFor: strings.Join(forwardedFor, ", "),
ClientGroup: clientGroup(client).String(),
ContentType: r.Header.Get("Content-Type"),
RequestHeaders: requestHeaders(r, h.config.LogRequestHeaders),
HasAuthorization: len(r.Header.Values("Authorization")) > 0,
HasCookie: len(r.Header.Values("Cookie")) > 0,
Action: requestlog.ActionForward,
Time: requestlog.FormatTime(start),
ClientIP: client.String(),
PeerIP: peer.String(),
Method: r.Method,
Host: r.Host,
Path: r.URL.EscapedPath(),
Query: r.URL.RawQuery,
Protocol: r.Proto,
Referer: r.Referer(),
UserAgent: r.UserAgent(),
Action: requestlog.ActionForward,
},
}
// A length of -1 is a body whose length was not announced.
if r.ContentLength > 0 {
rq.line.ContentLength = r.ContentLength
}
if r.Body != http.NoBody {
rq.body = &requestBody{body: limitBody(r.Body, h.config.RequestMaxBytes), rq: rq}
}
@@ -153,27 +107,6 @@ func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
return rq
}
// requestHeaders returns the headers of r that names lists, by name in
// lower case, each with its values joined by ", ". Authorization, Cookie
// and Set-Cookie are never among them, whatever names says.
func requestHeaders(r *http.Request, names []string) map[string]string {
headers := map[string]string{}
for _, name := range names {
switch name {
case "authorization", "cookie", "set-cookie":
continue
}
values := r.Header.Values(name)
if len(values) > 0 {
headers[name] = strings.Join(values, ", ")
}
}
return headers
}
// check is the one place where a request can be refused once its client
// is known, before its body is read or anything reaches the app. It
// returns nil to let the request through. The checks of checkClient come
@@ -210,17 +143,13 @@ func (rq *request) check(ctx context.Context) *refusal {
// checkClient runs the checks on the request's client, and returns the
// action of the first that refuses the request, or "" when none does. A
// client in SWWAF_ALLOW_NETS skips them, and is not looked up. For any
// other client, SWWAF_DENY_NETS comes first, then a ban on its netblock,
// so that a client either refuses is not looked up, then the lookup of
// its AS number and country, and then the country lists; a request any of
// them refuses is not counted for the rate limits. Then come the rate
// limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the
// request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that
// every other request is counted, each of them by the client's limit
// percentages, and last the rule files. A request exempt from the rate
// limits is exempt from the byte limits too. ctx is the request's own
// context.
// client in SWWAF_ALLOW_NETS skips them. For any other client,
// SWWAF_DENY_NETS comes first, then a ban on its netblock, so that a
// client either refuses is not looked up, and then the country lists; a
// request any of them refuses is not counted for the rate limits. Then
// come the rate limits, unless the client is in
// SWWAF_RATE_LIMIT_EXEMPT_NETS, so that every other request is counted,
// and last the rule files. ctx is the request's own context.
func (rq *request) checkClient(ctx context.Context) string {
cfg := rq.h.config
if isInside(rq.client, cfg.AllowNets) {
@@ -237,51 +166,17 @@ func (rq *request) checkClient(ctx context.Context) string {
return requestlog.ActionBanned
}
rq.lookUp(ctx)
if rq.countryDenied() {
if rq.countryDenied(ctx) {
return requestlog.ActionCountryDenied
}
rq.counted = !isInside(rq.client, cfg.RateLimitExemptNets) &&
!pathExempt(rq.in.URL, cfg.RateLimitExemptPaths)
if rq.counted {
rq.limitPercent, rq.bytesPercent = limitPercentages(cfg, rq.line.ASN, rq.line.Country)
rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
rq.line.BytesPercent, rq.line.BytesPercentSetting = rq.bytesPercent.logged()
}
if rq.counted && rq.limitBroken(now) {
if !isInside(rq.client, cfg.RateLimitExemptNets) && rq.limitBroken(now) {
return requestlog.ActionRateLimited
}
return rq.checkRules(now)
}
// pathExempt reports whether the rate limits leave out a request for u
// because of SWWAF_RATE_LIMIT_EXEMPT_PATHS: whether its path as sent, the
// path the app receives, not percent-decoded, starts with one of
// prefixes, so that /%61ssets/x is not under /assets/ for an app whose
// router matches the path as received. A request whose decoded path
// contains .. anywhere or a backslash, or whose path as sent holds an
// encoded slash (%2F or %2f), never is, since an app may act on it as a
// path outside every prefix: /assets/..%2Flogin as /login, or /assets%2Fx
// as one path segment, as Go's router does.
func pathExempt(u *url.URL, prefixes []string) bool {
decoded := u.Path
// EscapedPath is the path as the app receives it, not decoded.
sent := u.EscapedPath()
if strings.Contains(decoded, "..") || strings.Contains(decoded, `\`) ||
strings.Contains(strings.ToLower(sent), "%2f") {
return false
}
return slices.ContainsFunc(prefixes, func(prefix string) bool {
return strings.HasPrefix(sent, prefix)
})
}
// forward passes the request to the app and the app's answer back. ctx
// is the request's own context.
func (rq *request) forward(ctx context.Context) {
@@ -290,9 +185,7 @@ func (rq *request) forward(ctx context.Context) {
rq.cancel = cancel
ctx = httptrace.WithClientTrace(ctx, &httptrace.ClientTrace{
GotConn: rq.gotConn,
WroteRequest: rq.wroteRequest,
GotFirstResponseByte: rq.gotFirstResponseByte,
WroteRequest: rq.wroteRequest,
})
out := rq.in.WithContext(ctx)
@@ -315,10 +208,7 @@ func (rq *request) forward(ctx context.Context) {
}
// rewrite makes the request the app receives: the client's request,
// unchanged, sent to SWWAF_UPSTREAM_URL, with the forwarded headers and
// the request's id set, without any X-Client-ASN or X-Client-Country the
// client sent, whatever SWWAF_ADD_LOOKUP_HEADERS says, and, while it is
// set, with the client's AS number and country in them.
// unchanged, sent to SWWAF_UPSTREAM_URL, with the forwarded headers set.
func (rq *request) rewrite(pr *httputil.ProxyRequest) {
upstream := rq.h.config.UpstreamURL
pr.Out.URL.Scheme = upstream.Scheme
@@ -327,13 +217,6 @@ func (rq *request) rewrite(pr *httputil.ProxyRequest) {
// the query as the client sent it.
pr.Out.URL.RawQuery = pr.In.URL.RawQuery
setForwardedHeaders(pr.In, pr.Out, rq.peer, rq.peerTrusted)
pr.Out.Header.Set(requestIDHeader, rq.line.RequestID)
pr.Out.Header.Del(asnHeader)
pr.Out.Header.Del(countryHeader)
if rq.h.config.AddLookupHeaders {
setLookupHeaders(pr.Out.Header, rq.line.ASN, rq.line.Country)
}
}
// modifyResponse looks at the app's answer before ReverseProxy passes it
@@ -344,17 +227,9 @@ func (rq *request) modifyResponse(res *http.Response) error {
if res.StatusCode == http.StatusSwitchingProtocols {
// An upgraded connection, such as a WebSocket, is not cut by the
// timeouts. ReverseProxy writes this answer straight to the
// connection it takes over, not through rq.out, and then copies
// what passes each way through res.Body, the connection to the app.
// connection it takes over, not through rq.out.
rq.stopTimers()
rq.out.status = res.StatusCode
rq.line.Websocket = true
conn, ok := res.Body.(io.ReadWriteCloser)
if ok {
rq.upgraded = &upgradedConn{ReadWriteCloser: conn}
res.Body = rq.upgraded
}
return nil
}
@@ -431,14 +306,10 @@ func (rq *request) answer(r refusal) {
}
// refuse records r, unless an earlier refusal was, and ends the request
// to the app, if one was made: smallwebwaf reads the body of a request
// it answers itself too.
// to the app.
func (rq *request) refuse(r refusal) {
rq.refused.CompareAndSwap(nil, &r)
if rq.cancel != nil {
rq.cancel()
}
rq.cancel()
}
// finish ends the request's timeouts, counts it in the metrics and writes
@@ -454,11 +325,10 @@ func (rq *request) finish() {
line := &rq.line
line.Status = rq.out.status
line.ResponseBytes = rq.out.bytes
header := rq.out.Header()
line.ResponseContentType = header.Get("Content-Type")
line.CacheControl = header.Get("Cache-Control")
line.Location = header.Get("Location")
line.RequestBytes = rq.requestBytes()
if rq.body != nil {
line.RequestBytes = rq.body.bytes.Load()
}
// limit is the setting whose size or time limit the request passed.
var limit string
@@ -479,18 +349,12 @@ func (rq *request) finish() {
now := time.Now()
duration := now.Sub(rq.start)
line.DurationTotal = requestlog.Milliseconds(duration)
line.DurationChecks = timing(rq.start, rq.checked)
var upstreamDuration time.Duration
if !rq.upstreamStart.IsZero() {
upstreamDuration = now.Sub(rq.upstreamStart)
line.DurationUpstreamTotal = new(requestlog.Milliseconds(upstreamDuration))
rq.mu.Lock()
line.DurationUpstreamConnect = timing(rq.upstreamStart, rq.connected)
line.DurationUpstreamFirstByte = timing(rq.upstreamStart, rq.answerStarted)
rq.mu.Unlock()
line.DurationUpstreamTotal = requestlog.Milliseconds(upstreamDuration)
}
// Counted before the log line is written, so that the metrics count
@@ -503,95 +367,25 @@ func (rq *request) finish() {
}
}
// timing is the time from start to end in milliseconds, for one of the
// log line's timings, or nil when end is zero: what it times never
// happened.
func timing(start, end time.Time) *float64 {
if end.IsZero() {
return nil
// addToHistory adds the request, which has ended, to its client's
// history.
func (rq *request) addToHistory() {
var requestBytes int64
if rq.body != nil {
requestBytes = rq.body.bytes.Load()
}
return new(requestlog.Milliseconds(end.Sub(start)))
}
// addToHistory adds the request, which has ended, to its client's
// history, and then the lookup's answer about the client, as
// answerAtTheEnd gives it, to that history and to the notes of the bans
// on its netblock: an answer may have come before either was there, and
// one from GeoJS that comes later is added when it comes.
func (rq *request) addToHistory() {
forwarded := !rq.upstreamStart.IsZero()
rq.h.limiter.AddToHistory(clientGroup(rq.client), rq.h.now(), ratelimit.Request{
Country: rq.line.Country,
Forwarded: forwarded,
Refused: !forwarded && rq.refused.Load() != nil,
Status: rq.out.status,
RequestBytes: rq.requestBytes(),
RequestBytes: requestBytes,
ResponseBytes: rq.out.bytes,
BrokeLimit: rq.line.Offence == requestlog.OffenceLimit,
})
answer, found := rq.answerAtTheEnd()
if found {
rq.h.addLookup(answer)
}
}
// countAnomalies counts the request, which has ended, and its bytes, as
// countedBytes gives them, for the anomaly thresholds, whatever was done
// with it: a request refused, one from a client in SWWAF_ALLOW_NETS or
// SWWAF_RATE_LIMIT_EXEMPT_NETS, and one for a path in
// SWWAF_RATE_LIMIT_EXEMPT_PATHS are counted too. It is counted for its
// client's AS number when answerAtTheEnd gives one. With every anomaly
// threshold off, the default, it does nothing.
func (rq *request) countAnomalies() {
if !anomalyThresholdsSet(rq.h.config) {
return
}
answer, _ := rq.answerAtTheEnd()
rq.h.anomalies.Count(rq.h.now(), anomaly.Request{
Client: rq.client,
ClientGroup: clientGroup(rq.client),
ASN: answer.ASN,
ASName: answer.ASName,
Country: answer.Country,
Bytes: rq.countedBytes(),
})
}
// anomalyThresholdsSet reports whether any anomaly threshold is set.
func anomalyThresholdsSet(cfg *config.Config) bool {
off := anomaly.Thresholds{}
return cfg.AnomalyClient != off || cfg.AnomalyNet != off || cfg.AnomalyASN != off ||
cfg.AnomalyTotal != off || cfg.AnomalyWatch != off
}
// answerAtTheEnd returns, for a client that was looked up, the lookup's
// answer about it as the request ends, and whether there is one: the
// lookup database's, which was there at once, or the one GeoJS has given
// by then, which a request does not wait for unless a setting needs it.
func (rq *request) answerAtTheEnd() (lookup.Answer, bool) {
if !rq.lookedUp {
return lookup.Answer{}, false
}
if rq.h.config.LookupSource == "file" {
return rq.lookupAnswer, true
}
return rq.h.geojs.Kept(clientGroup(rq.client))
}
// requestBytes is how many bytes of the request's body have been read.
func (rq *request) requestBytes() int64 {
if rq.body == nil {
return 0
}
return rq.body.bytes.Load()
}
// clientRequestDeadline is when the client must have sent its whole
@@ -682,24 +476,6 @@ func (rq *request) bodyReceived() {
stopTimer(rq.clientRequestTimer)
}
// gotConn is called once there is a connection to the app, a new one or
// one kept open from an earlier request.
func (rq *request) gotConn(httptrace.GotConnInfo) {
rq.mu.Lock()
defer rq.mu.Unlock()
rq.connected = time.Now()
}
// gotFirstResponseByte is called once the first byte of the app's answer
// has arrived.
func (rq *request) gotFirstResponseByte() {
rq.mu.Lock()
defer rq.mu.Unlock()
rq.answerStarted = time.Now()
}
// wroteRequest is called once the app has been sent the whole request:
// the request timeouts end and SWWAF_UPSTREAM_RESPONSE_TIMEOUT starts.
func (rq *request) wroteRequest(info httptrace.WroteRequestInfo) {
internal/proxy/requestlog_test.go
-371
View File
@@ -1,371 +0,0 @@
package proxy_test
import (
"io"
"maps"
"math"
"net/http"
"reflect"
"slices"
"strings"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
const (
// requestIDHeader carries the request's id.
requestIDHeader = "X-Request-ID"
// instance is the SWWAF_INSTANCE_NAME a test sets.
instance = "fsn1app1/gitea"
// ipv6Client is a client on IPv6, and ipv6Group the netblock the rate
// limits count it as.
ipv6Client = "2001:db8::7"
ipv6Group = "2001:db8::/64"
)
func TestLogLineHasEachFieldWhereItApplies(t *testing.T) {
t.Parallel()
received := make(chan string, 2) // the request ids the app received
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
received <- r.Header.Get(requestIDHeader)
_, _ = io.Copy(io.Discard, r.Body)
if r.URL.Path != "/full" {
w.WriteHeader(http.StatusNoContent)
return
}
w.Header().Set("Content-Type", "text/html")
w.Header().Set("Cache-Control", "no-store")
w.Header().Set("Location", "/elsewhere")
w.WriteHeader(http.StatusFound)
_, _ = io.WriteString(w, "moved")
})
addr, out := startProxy(t, app.URL, map[string]string{
trustedProxies: trustLocalhost,
rateLimitExemptNets: localhost,
instanceName: instance,
logRequestHeaders: "Accept,x-custom,Authorization,cookie,SET-COOKIE",
})
// This request comes from ipv6Client through a trusted proxy, with a
// body and each header the log line looks at, and is answered with a
// redirect.
conn := dial(t, addr)
send(t, conn, "POST /full HTTP/1.1\r\nHost: "+appHost+"\r\n"+
forwardedFor+": 198.51.100.7, "+ipv6Client+"\r\n"+
forwardedProto+": "+secure+"\r\n"+requestIDHeader+": from-traefik\r\n"+
"Content-Type: application/x-www-form-urlencoded\r\nContent-Length: 3\r\n"+
"Accept: text/html\r\nX-Custom: one\r\nX-Custom: two\r\n"+
"Authorization: Bearer secret-token\r\nCookie: session=secret-cookie\r\n"+
"Set-Cookie: secret-set-cookie\r\n\r\na=b")
wantStatus(t, readResponse(t, conn), http.StatusFound)
// A request's log line can come after its answer: each is waited for
// before the next request, so that the lines are in order.
full := out.requestLines(t, 1)[0]
// This one comes from 127.0.0.1, which the rate limits do not count,
// with a body of 4 bytes whose length it does not announce, so that its
// request_bytes is not its content_length, and no header the log line
// looks at, and is answered with 204 and no header.
conn = dial(t, addr)
send(t, conn, "POST /bare HTTP/1.1\r\nHost: "+appHost+"\r\n"+
"Transfer-Encoding: chunked\r\n\r\n4\r\nbody\r\n0\r\n\r\n")
wantStatus(t, readResponse(t, conn), http.StatusNoContent)
bare := out.requestLines(t, 2)[1]
wantFullLine(t, full)
wantBareLine(t, bare)
for _, line := range []logLine{full, bare} {
got := <-received
if got != line.RequestID {
t.Errorf("the app received request id %q, the log line has %q",
got, line.RequestID)
}
}
if strings.Contains(out.text(), "secret") {
t.Errorf("a value of Authorization, Cookie or Set-Cookie is logged:\n%s",
out.text())
}
}
// wantFullLine checks the log line of the request with every header the
// line looks at. Its timings are checked by TestTimingsAreInOrder.
func wantFullLine(t *testing.T, line logLine) {
t.Helper()
headers := map[string]string{"accept": "text/html", "x-custom": "one, two"}
want := withTimings(line, requestlog.Line{
Type: requestType, Time: line.Time, Instance: instance,
ClientIP: ipv6Client, Method: http.MethodPost, Scheme: secure,
Host: appHost, Path: "/full", Protocol: protocol,
Status: http.StatusFound, RequestBytes: 3, ResponseBytes: 5,
RequestID: "from-traefik", PeerIP: localhost,
ForwardedFor: "198.51.100.7, " + ipv6Client, ClientGroup: ipv6Group,
ContentType: "application/x-www-form-urlencoded", ContentLength: 3,
RequestHeaders: headers, HasAuthorization: true, HasCookie: true,
ResponseContentType: "text/html", UpstreamStatus: http.StatusFound,
CacheControl: "no-store", Location: "/elsewhere",
Action: requestlog.ActionForward,
// Its 3 bytes in and 5 out, each way counted by default.
Counts: ratelimit.Counts{
Minute: 1, Hour: 1, Day: 1, MinuteBytes: 8, HourBytes: 8, DayBytes: 8,
},
})
if !reflect.DeepEqual(line.Line, want) {
t.Errorf("log line\n%+v\nwant\n%+v", line.Line, want)
}
}
// wantBareLine checks the log line of the request with none of them, and
// that the fields that do not apply to it are left out.
func wantBareLine(t *testing.T, line logLine) {
t.Helper()
want := withTimings(line, requestlog.Line{
Type: requestType, Time: line.Time, Instance: instance,
ClientIP: localhost, Method: http.MethodPost, Scheme: plain,
Host: appHost, Path: "/bare", Protocol: protocol,
Status: http.StatusNoContent, RequestBytes: 4, RequestID: line.RequestID,
PeerIP: localhost, ClientGroup: localhost + "/32",
UpstreamStatus: http.StatusNoContent, Action: requestlog.ActionForward,
})
if !reflect.DeepEqual(line.Line, want) || line.RequestID == "" {
t.Errorf("log line\n%+v\nwant\n%+v, with a request id", line.Line, want)
}
for _, name := range []string{
"forwarded_for", "content_type", "content_length", "request_headers",
"has_authorization", "has_cookie", "websocket", "response_content_type",
"cache_control", "location", "counts",
} {
_, present := line.fields[name]
if present {
t.Errorf("log line has %s, which does not apply", name)
}
}
}
// withTimings returns want with the timings of line.
func withTimings(line logLine, want requestlog.Line) requestlog.Line {
want.DurationTotal = line.DurationTotal
want.DurationChecks = line.DurationChecks
want.DurationUpstreamConnect = line.DurationUpstreamConnect
want.DurationUpstreamFirstByte = line.DurationUpstreamFirstByte
want.DurationUpstreamTotal = line.DurationUpstreamTotal
return want
}
func TestHasAuthorizationAndHasCookieEachComeFromTheirOwnHeader(t *testing.T) {
t.Parallel()
const hasAuthorization, hasCookie = "has_authorization", "has_cookie"
for _, tc := range []struct{ header, field, other string }{
{"Authorization", hasAuthorization, hasCookie},
{"Cookie", hasCookie, hasAuthorization},
} {
t.Run("only "+tc.header, func(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out := startProxy(t, app.URL, nil)
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(tc.header, "secret")
wantStatus(t, do(t, req), http.StatusOK)
line := out.requestLine(t)
_, otherPresent := line.fields[tc.other]
if line.fields[tc.field] != true || otherPresent {
t.Errorf("log line has %s %v and %s %v, want true and none",
tc.field, line.fields[tc.field], tc.other, line.fields[tc.other])
}
})
}
}
func TestRequestIDAndSchemeComeOnlyFromATrustedProxy(t *testing.T) {
t.Parallel()
const sentID = "from-traefik"
sent := http.Header{requestIDHeader: {sentID}, forwardedProto: {secure}}
trusted := map[string]string{trustedProxies: trustLocalhost}
for _, tc := range []struct {
name string
env map[string]string
header http.Header
// wantID is the request id logged, "" for a new one.
wantID, wantScheme string
}{
{"a trusted proxy's are kept", trusted, sent, sentID, secure},
{"without them, the id is new and the scheme http", trusted, nil, "", plain},
{"another peer's are replaced", nil, sent, "", plain},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
received := make(chan string, 2)
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
received <- r.Header.Get(requestIDHeader)
})
addr, out := startProxy(t, app.URL, tc.env)
// Two requests, so that two new ids can be told apart.
ids := make([]string, 0, 2)
for i := range 2 {
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
maps.Copy(req.Header, tc.header)
wantStatus(t, do(t, req), http.StatusOK)
line := out.requestLines(t, i+1)[i]
ids = append(ids, line.RequestID)
got := <-received
if line.RequestID != got || line.Scheme != tc.wantScheme {
t.Errorf("log line has request_id %q and scheme %q, and the "+
"app received id %q; want the same id and scheme %q",
line.RequestID, line.Scheme, got, tc.wantScheme)
}
}
switch {
case tc.wantID != "" && (ids[0] != tc.wantID || ids[1] != tc.wantID):
t.Errorf("request ids %q, want %q", ids, tc.wantID)
case tc.wantID == "" && (slices.Contains(ids, sentID) ||
slices.Contains(ids, "") || ids[0] == ids[1]):
t.Errorf("request ids %q, want two new ones", ids)
}
})
}
}
func TestTimingsAreInOrder(t *testing.T) {
t.Parallel()
const denied = "192.0.2.50" // in SWWAF_DENY_NETS
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
// The pauses set the times apart; a hold-up of the test only
// lengthens them.
time.Sleep(time.Millisecond)
w.WriteHeader(http.StatusOK)
_ = http.NewResponseController(w).Flush()
time.Sleep(time.Millisecond)
_, _ = io.WriteString(w, "done")
})
addr, out := startProxy(t, app.URL, map[string]string{
trustedProxies: trustLocalhost,
denyNets: denied,
})
// Each log line is waited for before the next request, so that the
// lines are in order.
wantStatus(t, get(t, addr, "/"), http.StatusOK)
forwarded := out.requestLines(t, 1)[0]
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedFor, denied)
wantStatus(t, do(t, req), http.StatusForbidden)
refused := out.requestLines(t, 2)[1]
wantStatus(t, get(t, addr, proxy.HealthPath), http.StatusOK)
health := out.requestLines(t, 3)[2]
// A request passed to the app has every timing; one refused, none of
// the app's; the health check, which runs no check, only the total.
wantTimings(t, forwarded, "duration_total", "duration_checks",
"duration_upstream_connect", "duration_upstream_first_byte",
"duration_upstream_total")
wantTimings(t, refused, "duration_total", "duration_checks")
wantTimings(t, health, "duration_total")
if t.Failed() {
return
}
// In whole microseconds, as they are logged, so that the sum below is
// exact.
total := microseconds(forwarded.DurationTotal)
checks := microseconds(*forwarded.DurationChecks)
connect := microseconds(*forwarded.DurationUpstreamConnect)
firstByte := microseconds(*forwarded.DurationUpstreamFirstByte)
upstream := microseconds(*forwarded.DurationUpstreamTotal)
// The checks end before the request is handed to the app, and the
// connection comes before the answer, which the app ends after a
// pause.
if checks+upstream > total || connect >= firstByte || firstByte >= upstream {
t.Errorf("timings in microseconds: total %d, checks %d, connect %d, "+
"first byte %d, upstream total %d", total, checks, connect, firstByte,
upstream)
}
if *refused.DurationChecks > refused.DurationTotal {
t.Errorf("refused request's checks took %v of %v milliseconds",
*refused.DurationChecks, refused.DurationTotal)
}
}
// wantTimings checks that the timings named are the only ones line has.
func wantTimings(t *testing.T, line logLine, want ...string) {
t.Helper()
var got []string
for name := range line.fields {
if strings.HasPrefix(name, "duration_") {
got = append(got, name)
}
}
slices.Sort(got)
slices.Sort(want)
if !slices.Equal(got, want) {
t.Errorf("log line of %s has timings %v, want %v", line.Path, got, want)
}
}
// microseconds is a timing in whole microseconds.
func microseconds(milliseconds float64) int64 {
return int64(math.Round(milliseconds * 1000))
}
func TestLogsAnUpgradedConnection(t *testing.T) {
t.Parallel()
app := startApp(t, echoAfterUpgrade)
addr, out := startProxy(t, app.URL, nil)
conn := dial(t, addr)
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
"Connection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
wantStatus(t, readResponse(t, conn), http.StatusSwitchingProtocols)
_ = conn.Close()
line := out.requestLine(t)
if line.fields["websocket"] != true {
t.Errorf("log line has websocket %v, want true", line.fields["websocket"])
}
}
internal/proxy/rulefiles_test.go
+8 -9
View File
@@ -55,7 +55,6 @@ func TestEachRuleAction(t *testing.T) {
Start: start,
Expires: start.Add(7 * 24 * time.Hour),
Cause: bans.CauseAttack,
Reason: "matched the rule probe",
Notes: bans.Notes{
RuleID: "probe",
Target: "path",
@@ -197,15 +196,15 @@ func TestMetricsCountRuleMatchesAndBansForAnAttack(t *testing.T) {
metrics := s.scrape(scraper)
wantMetric(t, metrics,
`smallwebwaf_rule_matches_total{action="block",instance="app",rule_id="blocked"}`, 1)
`smallwebwaf_rule_matches_total{action="block",rule_id="blocked"}`, 1)
wantMetric(t, metrics,
`smallwebwaf_rule_matches_total{action="ban",instance="app",rule_id="probe"}`, 1)
wantMetric(t, metrics, `smallwebwaf_rules_loaded{instance="app"}`, 2)
wantMetric(t, metrics, `smallwebwaf_requests_total{action="rule_blocked",`+
`instance="app",status_class="4xx"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="attack",instance="app"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit",instance="app"}`, 0)
wantMetric(t, metrics, `smallwebwaf_permanent_bans{instance="app"}`, 1)
`smallwebwaf_rule_matches_total{action="ban",rule_id="probe"}`, 1)
wantMetric(t, metrics, "smallwebwaf_rules_loaded", 2)
wantMetric(t, metrics,
`smallwebwaf_requests_total{action="rule_blocked",status_class="4xx"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="attack"}`, 1)
wantMetric(t, metrics, `smallwebwaf_bans_made_total{cause="limit"}`, 0)
wantMetric(t, metrics, "smallwebwaf_permanent_bans", 1)
}
// writeRules writes content as a rule file into a new directory, and
internal/proxy/rules.go
+5 -4
View File
@@ -10,9 +10,8 @@ import (
// checkRules checks the request against the rules of the rule files at
// now, notes the ids of those it matches in the log line, and returns the
// action of the rule that refuses it, ActionRuleBlocked for a block rule
// and ActionBanned for a ban rule, or "" when none does. A ban rule bans
// the client's netblock for a clear sign of attack, or in observe mode
// raises the alert for the ban it would have made.
// and ActionBanned for a ban rule, or "" when none does. In enforce mode
// a ban rule bans the client's netblock for a clear sign of attack.
func (rq *request) checkRules(now time.Time) string {
matched := rq.h.rules.Match(rq.in)
@@ -30,7 +29,9 @@ func (rq *request) checkRules(now time.Time) string {
case rules.ActionBlock:
return requestlog.ActionRuleBlocked
case rules.ActionBan:
rq.banForAttack(now, last)
if !rq.h.config.Observe {
rq.banForAttack(now, last)
}
return requestlog.ActionBanned
default:
internal/ratelimit/history_test.go
+4 -39
View File
@@ -16,10 +16,10 @@ func TestHistoryKeepsEveryRequest(t *testing.T) {
start := midnight()
for i, r := range []ratelimit.Request{
{Forwarded: true, Status: 200, RequestBytes: 10, ResponseBytes: 100},
{Country: "DE", Forwarded: true, Status: 200, RequestBytes: 10, ResponseBytes: 100},
{Forwarded: true, Status: 101},
{Forwarded: true, Status: 304, RequestBytes: 5},
{Refused: true, Status: 403, ResponseBytes: 10, BrokeLimit: true},
{Country: "FR", Refused: true, Status: 403, ResponseBytes: 10, BrokeLimit: true},
{Forwarded: true, Status: 502, ResponseBytes: 12},
// Closed without an answer: refused, and no response.
{Refused: true, Status: 0},
@@ -33,6 +33,8 @@ func TestHistoryKeepsEveryRequest(t *testing.T) {
want := ratelimit.History{
FirstSeen: start,
LastSeen: start.Add(6 * time.Minute),
Country: "FR",
LookedUp: start.Add(3 * time.Minute),
Requests: 7,
Forwarded: 4,
Refused: 2,
@@ -50,43 +52,6 @@ func TestHistoryKeepsEveryRequest(t *testing.T) {
}
}
func TestLookupReachesTheHistoryOfAClientInTheTable(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{})
client := netip.MustParsePrefix("203.0.113.9/32")
other := netip.MustParsePrefix("198.51.100.7/32")
start := midnight()
limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
limiter.AddLookup(client, start, "AS64496", "Example Net", "DE")
// A later answer replaces it, and one for a client the table does not
// hold adds no client.
limiter.AddLookup(client, start.Add(time.Hour), "AS64497", "Other Net", "FR")
limiter.AddLookup(other, start, "AS64496", "Example Net", "DE")
want := ratelimit.History{
FirstSeen: start,
LastSeen: start,
ASN: "AS64497",
ASName: "Other Net",
Country: "FR",
LookedUp: start.Add(time.Hour),
Requests: 1,
Forwarded: 1,
}
got := historyOf(t, limiter, client)
if got != want {
t.Errorf("history\n%+v\nwant\n%+v", got, want)
}
if clients := limiter.Snapshot(); len(clients) != 1 {
t.Errorf("the table holds %+v, want %s alone", clients, client)
}
}
func TestResetKeepsTheHistory(t *testing.T) {
t.Parallel()
internal/ratelimit/ratelimit.go
+78 -222
View File
@@ -1,9 +1,9 @@
// Package ratelimit keeps the table of clients: each client's requests
// and bytes counted over a minute, an hour and a day, as the "Counting
// method" section of SPEC.md describes, which tell when a request takes
// the client over a rate limit or a byte limit, and each client's history
// since it was first seen. At most 20,000 clients are kept, in memory, and
// written to clients.json and read from it by the state package.
// counted over a minute, an hour and a day, as the "Counting method"
// section of SPEC.md describes, which tell when a request takes the client
// over a rate limit, and each client's history since it was first seen.
// At most 20,000 clients are kept, in memory, and written to clients.json
// and read from it by the state package.
package ratelimit
import (
@@ -23,30 +23,19 @@ const maxClients = 20000
const day = 24 * time.Hour
// The kinds of limits, as the metrics name them.
const (
// KindRequests is a rate limit, on a client's requests.
KindRequests = "requests"
// KindBytes is a byte limit, on a client's bytes.
KindBytes = "bytes"
)
// Limits are the most requests a client may make in a minute, an hour and
// a day, and the most bytes. Zero is no limit.
// a day. Zero is no limit.
type Limits struct {
PerMinute int64
PerHour int64
PerDay int64
BytesPerMinute int64
BytesPerHour int64
BytesPerDay int64
PerMinute int64
PerHour int64
PerDay int64
}
// Limiter counts each client's requests and bytes against the limits, and
// keeps its history. It is safe for concurrent use.
// Limiter counts each client's requests against the limits, and keeps
// its history. It is safe for concurrent use.
type Limiter struct {
// windows are the minute, the hour and the day, in the order of
// Client.buckets and Client.byteBuckets.
// Client.buckets.
windows [3]window
mu sync.Mutex
@@ -54,23 +43,17 @@ type Limiter struct {
}
// Client is a client in the table, as clients.json holds it: its buckets
// of requests and of bytes in each window, and its history.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
// in each window, and its history.
type Client struct {
Client netip.Prefix `json:"client"`
Minute Buckets `json:"minute"`
Hour Buckets `json:"hour"`
Day Buckets `json:"day"`
MinuteBytes Buckets `json:"minute_bytes"`
HourBytes Buckets `json:"hour_bytes"`
DayBytes Buckets `json:"day_bytes"`
History History `json:"history"`
Client netip.Prefix `json:"client"`
Minute Buckets `json:"minute"`
Hour Buckets `json:"hour"`
Day Buckets `json:"day"`
History History `json:"history"`
}
// Buckets are a client's two buckets in one window: the requests, or the
// bytes, in the bucket under way, which began at Start, and in the bucket
// before it.
// Buckets are a client's two buckets in one window: the requests in the
// bucket under way, which began at Start, and in the bucket before it.
type Buckets struct {
Start time.Time `json:"start"`
Current int64 `json:"current"`
@@ -83,12 +66,8 @@ type Buckets struct {
type History struct {
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
// ASN, ASName and Country are the client's AS number, AS name and
// country as last looked up, each empty when the lookup could not
// find it, and LookedUp is when the lookup gave that answer; all are
// empty while the client never was looked up.
ASN string `json:"asn,omitempty"`
ASName string `json:"as_name,omitempty"`
// Country is the client's country as it was last looked up, and
// LookedUp when that was; both are empty while it never was.
Country string `json:"country,omitempty"`
LookedUp time.Time `json:"looked_up,omitzero"`
// Requests are all the client's requests: Forwarded those passed to
@@ -118,12 +97,14 @@ type Responses struct {
// Offences are a client's offences, by kind.
type Offences struct {
// Limit is its requests that broke a rate limit or a byte limit.
// Limit is its requests that broke a rate limit.
Limit int64 `json:"limit"`
}
// Request is what a client's history keeps of one of its requests.
type Request struct {
// Country is the client's country, when the request looked it up.
Country string
// Forwarded is true for a request passed to the app, Refused for one
// refused before anything reached it, a 401 at smallwebwaf's own
// endpoints included. Both are false for any other request smallwebwaf
@@ -136,8 +117,7 @@ type Request struct {
// and of its response.
RequestBytes int64
ResponseBytes int64
// BrokeLimit is true for a request that broke a rate limit or a byte
// limit.
// BrokeLimit is true for a request that broke a rate limit.
BrokeLimit bool
}
@@ -150,74 +130,49 @@ func New(limits Limits) *Limiter {
return &Limiter{
windows: [3]window{
{
name: "minute", length: time.Minute,
limit: limits.PerMinute, byteLimit: limits.BytesPerMinute,
},
{
name: "hour", length: time.Hour,
limit: limits.PerHour, byteLimit: limits.BytesPerHour,
},
{
name: "day", length: day,
limit: limits.PerDay, byteLimit: limits.BytesPerDay,
},
{name: "minute", length: time.Minute, limit: limits.PerMinute},
{name: "hour", length: time.Hour, limit: limits.PerHour},
{name: "day", length: day, limit: limits.PerDay},
},
clients: clients,
}
}
// Hit is a request that takes a client over a rate limit, or whose bytes
// take it over a byte limit.
// Hit is a request that takes a client over a rate limit.
type Hit struct {
// Kind is KindRequests for a rate limit, KindBytes for a byte limit.
Kind string
// Window is "minute", "hour" or "day".
Window string
// Limit is the window's limit, as the client's percentage of it.
// Limit is the window's limit.
Limit int64
// Count is the client's requests, or bytes, counted in the window,
// this request's included.
Count float64
}
// Counts are a client's requests and bytes in the minute, the hour and
// the day that end at a request, that request's included.
//
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
type Counts struct {
Minute float64 `json:"minute"`
Hour float64 `json:"hour"`
Day float64 `json:"day"`
MinuteBytes float64 `json:"minute_bytes"`
HourBytes float64 `json:"hour_bytes"`
DayBytes float64 `json:"day_bytes"`
// Requests is the client's requests counted in the window, this one
// included.
Requests float64
}
// Count counts a request from client at now, in every window, whether or
// not it is refused, and returns the client's counts in each window. It
// reports whether the request takes the client over a rate limit, of
// which the client gets the percentage percent, rounded down, and the hit:
// the window whose limit it goes over, the shortest if it is over
// several. A limit that is off stays off.
func (l *Limiter) Count(
client netip.Prefix, now time.Time, percent int64,
) (Counts, Hit, bool) {
return l.count(client, now, 1, 0, percent)
// not it is refused. It reports whether the request takes the client over
// a limit, and the window whose limit it goes over, the shortest if it is
// over several.
func (l *Limiter) Count(client netip.Prefix, now time.Time) (Hit, bool) {
l.mu.Lock()
defer l.mu.Unlock()
var hit Hit
for i, b := range l.get(client).buckets() {
w := l.windows[i]
requests := b.add(now, w.length)
if hit.Window == "" && w.limit > 0 && requests > float64(w.limit) {
hit = Hit{Window: w.name, Limit: w.limit, Requests: requests}
}
}
return hit, hit.Window != ""
}
// CountBytes counts bytes, those of a request from client that has ended,
// at now, in every window, and returns the client's counts in each window.
// It reports whether the bytes take the client over a byte limit, of which
// the client gets the percentage percent, and the hit, as Count does.
func (l *Limiter) CountBytes(
client netip.Prefix, now time.Time, bytes, percent int64,
) (Counts, Hit, bool) {
return l.count(client, now, 0, bytes, percent)
}
// Reset sets client's counts of requests and of bytes in every window
// back to zero. Its history keeps its totals.
// Reset sets client's counts in every window back to zero. Its history
// keeps its totals.
func (l *Limiter) Reset(client netip.Prefix) {
l.mu.Lock()
defer l.mu.Unlock()
@@ -225,7 +180,6 @@ func (l *Limiter) Reset(client netip.Prefix) {
c, seen := l.clients.Peek(client)
if seen {
c.Minute, c.Hour, c.Day = Buckets{}, Buckets{}, Buckets{}
c.MinuteBytes, c.HourBytes, c.DayBytes = Buckets{}, Buckets{}, Buckets{}
}
}
@@ -242,6 +196,11 @@ func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
h.LastSeen = now
if r.Country != "" {
h.Country = r.Country
h.LookedUp = now
}
h.Requests++
if r.Forwarded {
h.Forwarded++
@@ -260,25 +219,6 @@ func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
}
}
// AddLookup gives client's history its AS number, AS name and country, as
// the lookup gave them at lookedUp, if the table of clients holds the
// client.
// It does not make the client the most recently seen.
func (l *Limiter) AddLookup(
client netip.Prefix, lookedUp time.Time, asn, asName, country string,
) {
l.mu.Lock()
defer l.mu.Unlock()
c, held := l.clients.Peek(client)
if !held {
return
}
h := &c.History
h.ASN, h.ASName, h.Country, h.LookedUp = asn, asName, country, lookedUp
}
// Requests returns how many requests the clients inside netblock have
// sent, as their histories count them.
func (l *Limiter) Requests(netblock netip.Prefix) int64 {
@@ -302,20 +242,6 @@ func (l *Limiter) Requests(netblock netip.Prefix) int64 {
return requests
}
// Client returns client as the table holds it, and whether it does. It is
// not a request from client, and leaves when it was last seen unchanged.
func (l *Limiter) Client(client netip.Prefix) (Client, bool) {
l.mu.Lock()
defer l.mu.Unlock()
c, seen := l.clients.Peek(client)
if !seen {
return Client{}, false
}
return *c, true
}
// Len returns how many clients are in the table.
func (l *Limiter) Len() int {
l.mu.Lock()
@@ -359,11 +285,12 @@ func (l *Limiter) Load(clients []Client, now time.Time) {
l.clients.Purge()
for _, c := range clients {
for i, w := range l.windows {
for _, b := range []*Buckets{c.buckets()[i], c.byteBuckets()[i]} {
if b.Passed(now, w.length) {
*b = Buckets{}
}
for i, b := range c.buckets() {
// The window that ends at now covers neither bucket once it
// begins after the bucket under way has ended.
length := l.windows[i].length
if !now.Add(-length).Before(b.Start.Add(length)) {
*b = Buckets{}
}
}
@@ -371,51 +298,6 @@ func (l *Limiter) Load(clients []Client, now time.Time) {
}
}
// count adds requests and bytes from client at now to its buckets in
// every window, and returns its counts. A limit is broken only by what is
// added to it, so that a request whose bytes are counted after another of
// the client's requests broke a rate limit does not break it too. The
// client gets the percentage percent of each limit.
func (l *Limiter) count(
client netip.Prefix, now time.Time, requests, bytes, percent int64,
) (Counts, Hit, bool) {
l.mu.Lock()
defer l.mu.Unlock()
c := l.get(client)
requestBuckets, byteBuckets := c.buckets(), c.byteBuckets()
var (
requestCounts, byteCounts [3]float64
hit Hit
)
for i, w := range l.windows {
requestCounts[i] = requestBuckets[i].Add(now, w.length, requests)
byteCounts[i] = byteBuckets[i].Add(now, w.length, bytes)
limit, byteLimit := percentOf(w.limit, percent), percentOf(w.byteLimit, percent)
switch {
case hit.Window != "":
case requests > 0 && w.limit > 0 && requestCounts[i] > float64(limit):
hit = Hit{
Kind: KindRequests, Window: w.name, Limit: limit, Count: requestCounts[i],
}
case bytes > 0 && w.byteLimit > 0 && byteCounts[i] > float64(byteLimit):
hit = Hit{
Kind: KindBytes, Window: w.name, Limit: byteLimit, Count: byteCounts[i],
}
}
}
counts := Counts{
Minute: requestCounts[0], Hour: requestCounts[1], Day: requestCounts[2],
MinuteBytes: byteCounts[0], HourBytes: byteCounts[1], DayBytes: byteCounts[2],
}
return counts, hit, hit.Window != ""
}
// get returns client's entry in the table, a new one if it has none, and
// makes it the most recently seen.
func (l *Limiter) get(client netip.Prefix) *Client {
@@ -428,48 +310,30 @@ func (l *Limiter) get(client netip.Prefix) *Client {
return c
}
// buckets returns c's buckets of requests in the minute, the hour and the
// day.
// buckets returns c's buckets in the minute, the hour and the day.
func (c *Client) buckets() [3]*Buckets {
return [3]*Buckets{&c.Minute, &c.Hour, &c.Day}
}
// byteBuckets returns c's buckets of bytes in the minute, the hour and the
// day.
func (c *Client) byteBuckets() [3]*Buckets {
return [3]*Buckets{&c.MinuteBytes, &c.HourBytes, &c.DayBytes}
}
// window is a length of time over which requests and bytes are counted,
// and the most requests and the most bytes a client may have in it.
// window is a length of time over which requests are counted, and the
// most requests a client may make in it.
type window struct {
name string
length time.Duration
limit int64
byteLimit int64
name string
length time.Duration
limit int64
}
// percentOf returns the percentage percent of limit, rounded down. It is
// written as limit's hundreds times percent, plus the rest's share, since
// limit*percent can overflow for a byte limit.
func percentOf(limit, percent int64) int64 {
const hundred = 100
return limit/hundred*percent + limit%hundred*percent/hundred
}
// Add counts n requests, or n bytes, at now in a window of length, and
// returns the count in the window that ends at now: what is in the bucket
// under way, and what is in the bucket before it weighted by how much of
// that bucket the window still covers. With n zero it counts nothing, and
// returns the count. The anomaly counters count in Buckets too.
// add counts a request at now in a window of length, and returns the
// client's requests in the window that ends at now: those in the bucket
// under way, and those in the bucket before it weighted by how much of
// that bucket the window still covers.
//
// Concurrent requests can be counted out of order, so now can be a moment
// before the bucket under way began; such a request is counted in that
// bucket. A request dated more than a second before it means the clock
// was set back, and the buckets start afresh: otherwise the bucket before
// would keep its full weight until the clock caught up.
func (b *Buckets) Add(now time.Time, length time.Duration, n int64) float64 {
func (b *Buckets) add(now time.Time, length time.Duration) float64 {
if now.Before(b.Start.Add(-time.Second)) {
*b = Buckets{}
}
@@ -486,7 +350,7 @@ func (b *Buckets) Add(now time.Time, length time.Duration, n int64) float64 {
b.Current = 0
}
b.Current += n
b.Current++
elapsed := max(now.Sub(b.Start), 0)
covered := 1 - float64(elapsed)/float64(length)
@@ -494,14 +358,6 @@ func (b *Buckets) Add(now time.Time, length time.Duration, n int64) float64 {
return float64(b.Previous)*covered + float64(b.Current)
}
// Passed reports whether the window of length that ends at now covers
// neither of b's buckets: it begins after the bucket under way has ended.
// What they hold then counts no more, and a state file read at now drops
// it.
func (b *Buckets) Passed(now time.Time, length time.Duration) bool {
return !now.Add(-length).Before(b.Start.Add(length))
}
// add counts a response with status in its class. A status of 0, for
// nothing sent, is not a response.
func (r *Responses) add(status int) {
internal/ratelimit/ratelimit_test.go
+4 -205
View File
@@ -1,7 +1,6 @@
package ratelimit_test
import (
"math"
"net/netip"
"testing"
"time"
@@ -12,10 +11,6 @@ import (
// limit is the limit the tests set.
const limit = 3
// whole is the percentage of each limit a client gets when nothing lowers
// its limits.
const whole = 100
// The windows, as Count names them.
const (
minute = "minute"
@@ -67,203 +62,22 @@ func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
start := midnight()
for range limit {
_, _, over := limiter.Count(client, start, whole)
_, over := limiter.Count(client, start)
if over {
t.Fatal("a request within the limit is over it")
}
}
// Over both limits; the minute's is named, with the four requests.
_, hit, over := limiter.Count(client, start, whole)
hit, over := limiter.Count(client, start)
want := ratelimit.Hit{
Kind: ratelimit.KindRequests, Window: minute, Limit: limit, Count: limit + 1,
}
want := ratelimit.Hit{Window: minute, Limit: limit, Requests: limit + 1}
if !over || hit != want {
t.Errorf("request over the limit gives %+v and %t, want %+v and true",
hit, over, want)
}
}
func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: 5, BytesPerDay: math.MaxInt64})
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
// Half of 5 requests is 2.5, rounded down to 2: the third is over.
for range 2 {
_, _, over := limiter.Count(client, start, 50)
if over {
t.Fatal("a request within half the limit is over it")
}
}
_, hit, over := limiter.Count(client, start, 50)
want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: minute, Limit: 2, Count: 3}
if !over || hit != want {
t.Errorf("the third request gives %+v and %t, want %+v and true", hit, over, want)
}
// Half of the largest byte limit is still far above a TiB: working it
// out does not overflow.
_, hit, over = limiter.CountBytes(client, start, 1<<40, 50)
if over {
t.Errorf("a TiB is over half the largest byte limit: %+v", hit)
}
}
func TestZeroPercentIsAZeroAllowanceAndALimitOffStaysOff(t *testing.T) {
t.Parallel()
// Only the hour has limits: the minute's and the day's are off.
limiter := ratelimit.New(ratelimit.Limits{PerHour: limit, BytesPerHour: 1000})
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
// At 0 percent, the first request and the first byte are over the
// hour's limits, which are 0; the minute's, which are off, stay off.
_, hit, _ := limiter.Count(client, start, 0)
want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: hour, Limit: 0, Count: 1}
if hit != want {
t.Errorf("the first request gives %+v, want %+v", hit, want)
}
_, hit, _ = limiter.CountBytes(client, start, 1, 0)
want = ratelimit.Hit{Kind: ratelimit.KindBytes, Window: hour, Limit: 0, Count: 1}
if hit != want {
t.Errorf("the first byte gives %+v, want %+v", hit, want)
}
}
func TestEachByteLimitIsBrokenByTheBytesCounted(t *testing.T) {
t.Parallel()
const byteLimit = 1000
for _, tc := range []struct {
window string
limits ratelimit.Limits
}{
{minute, ratelimit.Limits{BytesPerMinute: byteLimit}},
{hour, ratelimit.Limits{BytesPerHour: byteLimit}},
{"day", ratelimit.Limits{BytesPerDay: byteLimit}},
} {
t.Run(tc.window, func(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(tc.limits)
client := netip.MustParsePrefix("203.0.113.9/32")
// 600 bytes are within the limit, 600 more over it.
_, _, over := limiter.CountBytes(client, midnight(), 600, whole)
if over {
t.Fatal("600 bytes are over the limit of 1000")
}
_, hit, over := limiter.CountBytes(client, midnight(), 600, whole)
want := ratelimit.Hit{
Kind: ratelimit.KindBytes, Window: tc.window, Limit: byteLimit, Count: 1200,
}
if !over || hit != want {
t.Errorf("1200 bytes give %+v and %t, want %+v and true", hit, over, want)
}
})
}
}
func TestALimitIsBrokenOnlyByWhatIsAddedToIt(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{PerMinute: 2, BytesPerMinute: 1000})
client := netip.MustParsePrefix("203.0.113.9/32")
other := netip.MustParsePrefix("203.0.113.10/32")
start := midnight()
// The third request breaks the rate limit. The bytes of a request
// counted after it, within the byte limit, do not break it again.
for range 2 {
wantCount(t, limiter, client, start, "")
}
wantCount(t, limiter, client, start, minute)
wantBytesCount(t, limiter, client, start, 500, "")
wantBytesCount(t, limiter, client, start, 600, ratelimit.KindBytes)
// Bytes over the byte limit do not have the next request break it, nor
// the rate limit, which that request is within.
wantBytesCount(t, limiter, other, start, 1200, ratelimit.KindBytes)
wantCount(t, limiter, other, start, "")
}
func TestCountGivesTheBytesInEachWindow(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{})
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
limiter.CountBytes(client, start, 300, whole)
// A quarter into the next hour, the minute has only these 100 bytes.
// The hour still covers three quarters of the bucket before, whose 300
// bytes count 225, and these: 325. The day covers all 400.
later := start.Add(time.Hour + time.Hour/4)
limiter.CountBytes(client, later, 100, whole)
// A request's counts give the bytes counted so far too.
counts, _, _ := limiter.Count(client, later, whole)
want := ratelimit.Counts{
Minute: 1, Hour: 1, Day: 1, MinuteBytes: 100, HourBytes: 325, DayBytes: 400,
}
if counts != want {
t.Errorf("counts %+v, want %+v", counts, want)
}
}
func TestResetSetsTheBytesBackToZero(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{BytesPerDay: 1000})
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
wantBytesCount(t, limiter, client, start, 1200, ratelimit.KindBytes)
limiter.Reset(client)
// The client has its whole allowance of bytes again.
wantBytesCount(t, limiter, client, start, 1000, "")
}
func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
t.Parallel()
limiter := ratelimit.New(ratelimit.Limits{})
client := netip.MustParsePrefix("203.0.113.9/32")
start := midnight()
for range 3 {
limiter.Count(client, start, whole)
}
// A quarter into the next hour, the minute has only this request. The
// hour still covers three quarters of the bucket before, with its three
// requests, which count 2.25, and this one: 3.25. The day covers all
// four.
counts, _, _ := limiter.Count(client, start.Add(time.Hour+time.Hour/4), whole)
want := ratelimit.Counts{Minute: 1, Hour: 3.25, Day: 4}
if counts != want {
t.Errorf("counts %+v, want %+v", counts, want)
}
}
func TestResetSetsTheCountsBackToZero(t *testing.T) {
t.Parallel()
@@ -424,24 +238,9 @@ func wantCount(
) {
t.Helper()
_, hit, _ := limiter.Count(client, now, whole)
hit, _ := limiter.Count(client, now)
if hit.Window != want {
t.Errorf("request from %s at %s is over %q, want %q",
client, now.Format(time.RFC3339), hit.Window, want)
}
}
// wantBytesCount counts bytes from client at now, and checks the kind of
// the limit they break, "" for none.
func wantBytesCount(
t *testing.T, limiter *ratelimit.Limiter, client netip.Prefix, now time.Time,
bytes int64, want string,
) {
t.Helper()
_, hit, _ := limiter.CountBytes(client, now, bytes, whole)
if hit.Kind != want {
t.Errorf("%d bytes from %s at %s break a limit on %q, want %q",
bytes, client, now.Format(time.RFC3339), hit.Kind, want)
}
}
internal/ratelimit/snapshot_test.go
+7 -14
View File
@@ -16,7 +16,7 @@ func TestSnapshotListsTheClientsByAddress(t *testing.T) {
limiter := ratelimit.New(ratelimit.Limits{})
for _, i := range []int{2, 3, 0, 1} {
limiter.Count(netip.MustParsePrefix(want[i]), midnight(), whole)
limiter.Count(netip.MustParsePrefix(want[i]), midnight())
}
snapshot := limiter.Snapshot()
@@ -63,8 +63,7 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
start := midnight()
limiter := ratelimit.New(ratelimit.Limits{})
limiter.Count(client, start, whole)
limiter.CountBytes(client, start, 5, whole)
limiter.Count(client, start)
limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
loaded := func(now time.Time) ratelimit.Client {
@@ -77,25 +76,19 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
}
// Two minutes on, the window that ends then covers neither of the
// minute's buckets, of requests and of bytes, which are emptied; the
// hour's and the day's stay, and so does the history.
// minute's buckets, which are emptied; the hour's and the day's stay,
// and so does the history.
got := loaded(start.Add(2 * time.Minute))
if got.Minute != (ratelimit.Buckets{}) || got.Hour.Current != 1 ||
got.Day.Current != 1 || got.History.Requests != 1 {
t.Errorf("loaded two minutes on as %+v", got)
}
if got.MinuteBytes != (ratelimit.Buckets{}) || got.HourBytes.Current != 5 ||
got.DayBytes.Current != 5 {
t.Errorf("loaded two minutes on with buckets of bytes %+v, %+v and %+v",
got.MinuteBytes, got.HourBytes, got.DayBytes)
}
// A moment before, the window still covers some of the earlier one.
got = loaded(start.Add(2*time.Minute - time.Nanosecond))
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 {
t.Errorf("loaded just under two minutes on with minute buckets %+v and %+v",
got.Minute, got.MinuteBytes)
if got.Minute.Current != 1 {
t.Errorf("loaded just under two minutes on with minute buckets %+v",
got.Minute)
}
}
internal/remotelog/remotelog.go
-310
View File
@@ -1,310 +0,0 @@
// Package remotelog sends the lines smallwebwaf writes on stdout to the
// remote log endpoint, SWWAF_LOG_REMOTE_URL, as the "Request log" section
// of SPEC.md describes: each line as the message of an RFC 5424 syslog
// record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so a slow
// or unreachable endpoint never holds up a request or stdout.
package remotelog
import (
"bytes"
"context"
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"log/slog"
"net"
"net/url"
"os"
"strconv"
"sync/atomic"
"syscall"
"time"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// The forms of SWWAF_LOG_REMOTE_URL, by its scheme.
const (
SchemeUDP = "syslog+udp"
SchemeTCP = "syslog+tcp"
SchemeTLS = "syslog+tls"
)
// A record's priority is the number of its facility times the number of
// severities there are, plus the number of its severity. Every record's
// severity is informational.
const (
severities = 8
informational = 6
)
const (
// dialTimeout bounds connecting to the endpoint, the TLS handshake
// included.
dialTimeout = 10 * time.Second
// After a failed attempt to connect, or a connection on which a record
// fails, the next attempt to connect is made a second later, and
// retryDelayFactor times as long after each further failure in a row,
// up to a minute. A connection that fails after it has stayed up for
// resetRetryDelayAfter ends the row.
firstRetryDelay = time.Second
retryDelayFactor = 2
maxRetryDelay = time.Minute
resetRetryDelayAfter = time.Minute
)
// Params are what New needs.
type Params struct {
// URL is the endpoint (SWWAF_LOG_REMOTE_URL): SchemeUDP, SchemeTCP or
// SchemeTLS, a host and a port.
URL *url.URL
// RootCAs are the certificates a SchemeTLS endpoint's certificate
// must chain to (SWWAF_LOG_REMOTE_TLS_CA_FILE), nil for the host's.
RootCAs *x509.CertPool
// Buffer is the most lines held while they wait to be sent
// (SWWAF_LOG_REMOTE_BUFFER).
Buffer int
// Facility is the number of the records' syslog facility
// (SWWAF_LOG_REMOTE_FACILITY), and AppName their APP-NAME
// (SWWAF_LOG_REMOTE_APP_NAME).
Facility int
AppName string
}
// Sender sends lines to the endpoint. Write puts them in its buffer, and
// Run sends them from there.
type Sender struct {
url *url.URL
tlsConfig *tls.Config
// beforeTime and afterTime are the parts of every record's header
// before and after its time, as RFC 5424 lays the header out.
beforeTime string
afterTime string
// records is the buffer: each line's record, framed to be sent.
records chan []byte
sent atomic.Int64
dropped atomic.Int64
}
// New returns a Sender for the endpoint params.URL.
func New(params Params) *Sender {
hostname, err := os.Hostname()
if err != nil || hostname == "" {
hostname = "-" // RFC 5424's value for a field that has none
}
priority := params.Facility*severities + informational
return &Sender{
url: params.URL,
tlsConfig: &tls.Config{
RootCAs: params.RootCAs,
MinVersion: tls.VersionTLS12,
},
// The 1 is the version of the format. The process id, the message
// id and the structured data have no value.
beforeTime: "<" + strconv.Itoa(priority) + ">1 ",
afterTime: " " + hostname + " " + params.AppName + " - - - ",
records: make(chan []byte, params.Buffer),
}
}
// Write puts each line in p in the buffer, as the message of a record of
// its own, and never waits: when the buffer is full, the oldest record in
// it is dropped to make room. It is safe for concurrent use.
func (s *Sender) Write(p []byte) (int, error) {
at := requestlog.FormatTime(time.Now())
for line := range bytes.Lines(p) {
line = bytes.TrimSuffix(line, []byte("\n"))
if len(line) > 0 {
s.put(s.record(at, line))
}
}
return len(p), nil
}
// Sent is how many records have been sent.
func (s *Sender) Sent() int64 {
return s.sent.Load()
}
// Dropped is how many records were dropped: the oldest in a full buffer,
// and those whose sending failed.
func (s *Sender) Dropped() int64 {
return s.dropped.Load()
}
// Depth is how many records are in the buffer.
func (s *Sender) Depth() int {
return len(s.records)
}
// Run connects to the endpoint and sends each record as it comes into the
// buffer, until ctx is done. Then it sends the records still in the buffer,
// on the connection open at that time or, if there is none, on a new one,
// until none is left or one fails, and returns. How long it may take over
// that is for the caller to bound.
//
// A connection on which a record fails is closed and the record dropped.
// That failure, like a failed attempt to connect, is logged to processLog
// and followed by the next attempt after firstRetryDelay, retryDelayFactor
// times as long after each further failure in a row up to maxRetryDelay,
// and firstRetryDelay again after a connection that stayed up for
// resetRetryDelayAfter. Meanwhile the records wait in the buffer.
func (s *Sender) Run(ctx context.Context, processLog *slog.Logger) {
conn := s.send(ctx, processLog)
if conn == nil && len(s.records) > 0 {
conn, _ = s.dial(context.WithoutCancel(ctx))
}
if conn == nil {
return
}
defer func() {
_ = conn.Close()
}()
for {
select {
case record := <-s.records:
if s.write(conn, record) != nil {
return
}
default:
return
}
}
}
// record returns line as an RFC 5424 record made at the time at, framed
// for the endpoint: on its own over UDP, since each datagram holds one,
// and over TCP and TLS after its length in bytes and a space, the
// octet-counted framing of RFC 6587 and RFC 5425.
func (s *Sender) record(at string, line []byte) []byte {
record := make([]byte, 0, len(s.beforeTime)+len(at)+len(s.afterTime)+len(line))
record = append(record, s.beforeTime...)
record = append(record, at...)
record = append(record, s.afterTime...)
record = append(record, line...)
if s.url.Scheme == SchemeUDP {
return record
}
return append([]byte(strconv.Itoa(len(record))+" "), record...)
}
// put adds record to the buffer, first dropping the oldest record in it
// while it is full.
func (s *Sender) put(record []byte) {
for {
select {
case s.records <- record:
return
default:
}
select {
case <-s.records:
s.dropped.Add(1)
default:
}
}
}
// send connects to the endpoint and sends each record as it comes into
// the buffer, until ctx is done, and returns the connection then open, or
// nil.
func (s *Sender) send(ctx context.Context, processLog *slog.Logger) net.Conn {
delay := firstRetryDelay
for {
conn, err := s.dial(ctx)
if ctx.Err() != nil {
return conn
}
if err == nil {
connected := time.Now()
err = s.sendOn(ctx, conn)
if err == nil {
return conn
}
_ = conn.Close()
if time.Since(connected) >= resetRetryDelayAfter {
delay = firstRetryDelay
}
}
processLog.Warn("sending to SWWAF_LOG_REMOTE_URL failed",
"error", err.Error(), "connecting_again_in", delay.String())
select {
case <-time.After(delay):
case <-ctx.Done():
return nil
}
delay = min(retryDelayFactor*delay, maxRetryDelay)
}
}
// sendOn sends each record on conn as it comes into the buffer, until one
// fails, whose error it returns, or ctx is done.
func (s *Sender) sendOn(ctx context.Context, conn net.Conn) error {
for {
select {
case record := <-s.records:
err := s.write(conn, record)
if err != nil {
return err
}
case <-ctx.Done():
return nil
}
}
}
// write sends record on conn, and counts it as sent or, if that fails,
// as dropped. A record too long for one UDP datagram is dropped without
// an error, since the connection has not failed: a long request must not
// hold up the lines after it.
func (s *Sender) write(conn net.Conn, record []byte) error {
_, err := conn.Write(record)
if err != nil {
s.dropped.Add(1)
if errors.Is(err, syscall.EMSGSIZE) {
return nil
}
return fmt.Errorf("send a record: %w", err)
}
s.sent.Add(1)
return nil
}
// dial connects to the endpoint.
func (s *Sender) dial(ctx context.Context) (net.Conn, error) {
dialer := &net.Dialer{Timeout: dialTimeout}
switch s.url.Scheme {
case SchemeUDP:
return dialer.DialContext(ctx, "udp", s.url.Host)
case SchemeTLS:
tlsDialer := &tls.Dialer{NetDialer: dialer, Config: s.tlsConfig}
return tlsDialer.DialContext(ctx, "tcp", s.url.Host)
default:
return dialer.DialContext(ctx, "tcp", s.url.Host)
}
}
internal/remotelog/remotelog_test.go
-640
View File
@@ -1,640 +0,0 @@
package remotelog_test
import (
"bufio"
"bytes"
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/json"
"fmt"
"io"
"log/slog"
"math/big"
"net"
"net/url"
"os"
"slices"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing"
"testing/synctest"
"time"
"sneak.berlin/go/smallwebwaf/internal/remotelog"
)
// The tests run in a synctest bubble, where the time package runs on a
// clock of the test's own, which starts at 2000-01-01T00:00:00Z: a wait
// lasts exactly as long as it should, however slowly the test process
// runs, and synctest.Wait returns once the sender has done all it can
// before time passes. The endpoint is a listener on the loopback address.
// A test reads from it only once the records are on their way, and checks
// the sender's counts first, since a goroutine of the bubble that waits on
// the network keeps that clock from moving on. For the same reason the
// endpoint that refuses connections, a tlsEndpoint, runs outside the
// bubble: a sender connecting over TLS waits on the endpoint's answer.
const (
// started is the time a record made as a test starts gives.
started = "2000-01-01T00:00:00.000Z"
appName = "fsn1app1/gitea"
// local0 is the number of the default facility, and local0Info the
// priority of its records.
local0 = 16
local0Info = "<134>"
// loopback is where the endpoints listen.
loopback = "127.0.0.1:0"
)
func TestRecordsOverUDPGoOnePerDatagram(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
if err != nil {
t.Fatalf("listen: %v", err)
}
t.Cleanup(func() { _ = endpoint.Close() })
sender, _, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
_, _ = sender.Write([]byte(`{"type":"request"}` + "\n" + `{"type":"process"}` + "\n"))
synctest.Wait()
wantCounts(t, sender, 2, 0, 0)
for _, line := range []string{`{"type":"request"}`, `{"type":"process"}`} {
datagram := make([]byte, 1024)
n, _, err := endpoint.ReadFrom(datagram)
if err != nil {
t.Fatalf("read: %v", err)
}
want := record(t, local0Info, appName, line)
if string(datagram[:n]) != want {
t.Errorf("datagram %q, want %q", datagram[:n], want)
}
}
})
}
func TestRecordsOverTCPAreOctetCountedWithTheirFacility(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint := listen(t)
endpointParams := params(remotelog.SchemeTCP, endpoint.Addr())
endpointParams.Facility = 19 // local3
endpointParams.AppName = "gitea"
sender, _, _ := run(t, endpointParams)
_, _ = sender.Write([]byte("first\nsecond\n"))
synctest.Wait()
wantCounts(t, sender, 2, 0, 0)
frames := bufio.NewReader(accept(t, endpoint))
wantFrame(t, frames, record(t, "<158>", "gitea", "first"))
wantFrame(t, frames, record(t, "<158>", "gitea", "second"))
})
}
func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
certificate, roots := testCertificate(t)
endpoint := listen(t)
endpointParams := params(remotelog.SchemeTLS, endpoint.Addr())
endpointParams.RootCAs = roots
endpointParams.Buffer = 3
sender, _, _ := run(t, endpointParams)
// The sender connects, and its TLS handshake waits for an answer
// the endpoint does not give yet.
conn := accept(t, endpoint)
var stdout bytes.Buffer
out := io.MultiWriter(&stdout, sender)
for i := range 5 {
_, _ = fmt.Fprintf(out, "line %d\n", i+1)
}
if stdout.String() != "line 1\nline 2\nline 3\nline 4\nline 5\n" {
t.Errorf("stdout has %q", stdout.String())
}
wantCounts(t, sender, 0, 2, 3)
// Once the endpoint answers, the three newest records are sent.
server := tls.Server(conn, &tls.Config{
Certificates: []tls.Certificate{certificate},
MinVersion: tls.VersionTLS12,
})
err := server.HandshakeContext(t.Context())
if err != nil {
t.Fatalf("handshake: %v", err)
}
synctest.Wait()
wantCounts(t, sender, 3, 2, 0)
frames := bufio.NewReader(server)
for _, line := range []string{"line 3", "line 4", "line 5"} {
wantFrame(t, frames, record(t, local0Info, appName, line))
}
})
}
func TestReconnectsWithBackoffAfterTheEndpointGoesAway(t *testing.T) {
t.Parallel()
certificate, roots := testCertificate(t)
endpoint := startTLSEndpoint(t, certificate)
synctest.Test(t, func(t *testing.T) {
endpointParams := params(remotelog.SchemeTLS, endpoint.addr)
endpointParams.RootCAs = roots
sender, logged, _ := run(t, endpointParams)
_, _ = sender.Write([]byte("one\n"))
synctest.Wait()
wantCounts(t, sender, 1, 0, 0)
conn := endpoint.next(t)
wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "one"))
// The endpoint goes away: it closes the connection, and refuses the
// next ones. The sender notices when a record fails, and tries to
// connect again a second later, then two seconds after that.
endpoint.refusing.Store(true)
_ = conn.Close()
writeUntilDropped(t, sender, 1)
sent := sender.Sent()
_, _ = sender.Write([]byte("two\n"))
time.Sleep(time.Second)
synctest.Wait()
endpoint.refusing.Store(false)
time.Sleep(2*time.Second - time.Nanosecond)
synctest.Wait()
wantCounts(t, sender, sent, 1, 1)
// The endpoint is back, and the record waiting is sent.
time.Sleep(time.Nanosecond)
synctest.Wait()
wantCounts(t, sender, sent+1, 1, 0)
conn = endpoint.next(t)
wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "two"))
wantRetries(t, logged, "1s", "2s")
})
}
func TestAConnectionClosedAtOnceIsMadeAgainAfterAGrowingDelay(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint := listen(t)
sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
// The endpoint closes each connection as soon as it takes it. The
// sender notices when a record fails, and connects again a second
// later, then two seconds after that, then four.
delays := []time.Duration{time.Second, 2 * time.Second, 4 * time.Second}
for i, delay := range delays {
_ = accept(t, endpoint).Close()
writeUntilDropped(t, sender, int64(i+1))
wantConnectedAgainAfter(t, sender, delay)
}
wantRetries(t, logged, "1s", "2s", "4s")
})
}
func TestTheDelayStartsAgainAfterAConnectionThatStayedUpAMinute(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint := listen(t)
sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
_ = accept(t, endpoint).Close()
writeUntilDropped(t, sender, 1)
wantConnectedAgainAfter(t, sender, time.Second)
// A connection that fails just short of a minute after it was made
// leaves the delay growing.
conn := accept(t, endpoint)
time.Sleep(time.Minute - time.Nanosecond)
_ = conn.Close()
writeUntilDropped(t, sender, 2)
wantConnectedAgainAfter(t, sender, 2*time.Second)
// One that fails a minute after it was made starts it again from a
// second.
conn = accept(t, endpoint)
time.Sleep(time.Minute)
_ = conn.Close()
writeUntilDropped(t, sender, 3)
wantConnectedAgainAfter(t, sender, time.Second)
wantRetries(t, logged, "1s", "2s", "1s")
})
}
func TestALineTooLongForADatagramIsDroppedAlone(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
if err != nil {
t.Fatalf("listen: %v", err)
}
t.Cleanup(func() { _ = endpoint.Close() })
sender, logged, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
// With its header, the first line's record is longer than the 65507
// bytes a UDP datagram over IPv4 holds. It is dropped, nothing is
// logged, and the next line is sent at once.
_, _ = sender.Write([]byte(strings.Repeat("x", 65507) + "\nnext\n"))
synctest.Wait()
wantCounts(t, sender, 1, 1, 0)
wantRetries(t, logged)
datagram := make([]byte, 1024)
n, _, err := endpoint.ReadFrom(datagram)
if err != nil {
t.Fatalf("read: %v", err)
}
want := record(t, local0Info, appName, "next")
if string(datagram[:n]) != want {
t.Errorf("datagram %q, want %q", datagram[:n], want)
}
})
}
func TestRecordsWaitingAtTheStopAreSent(t *testing.T) {
t.Parallel()
certificate, roots := testCertificate(t)
endpoint := startTLSEndpoint(t, certificate)
synctest.Test(t, func(t *testing.T) {
// The endpoint refuses the sender's first connection: it fails to
// connect, and waits a second to try again.
endpoint.refusing.Store(true)
endpointParams := params(remotelog.SchemeTLS, endpoint.addr)
endpointParams.RootCAs = roots
sender, logged, stop := run(t, endpointParams)
synctest.Wait()
wantRetries(t, logged, "1s")
_, _ = sender.Write([]byte("one\ntwo\n"))
endpoint.refusing.Store(false)
// Stopped before that second is over, it connects to send them.
stop()
wantCounts(t, sender, 2, 0, 0)
frames := bufio.NewReader(endpoint.next(t))
wantFrame(t, frames, record(t, local0Info, appName, "one"))
wantFrame(t, frames, record(t, local0Info, appName, "two"))
})
}
// output collects what the sender logs.
type output struct {
mu sync.Mutex
buf bytes.Buffer
}
// Write adds lines the sender logs.
func (o *output) Write(p []byte) (int, error) {
o.mu.Lock()
defer o.mu.Unlock()
return o.buf.Write(p)
}
// text returns everything logged so far.
func (o *output) text() string {
o.mu.Lock()
defer o.mu.Unlock()
return o.buf.String()
}
// params returns the settings of a Sender for the endpoint at addr, in
// the form scheme names: room for ten lines, the default facility, and
// appName.
func params(scheme string, addr net.Addr) remotelog.Params {
return remotelog.Params{
URL: &url.URL{Scheme: scheme, Host: addr.String()},
Buffer: 10,
Facility: local0,
AppName: appName,
}
}
// run runs a Sender with settings until the test ends or the function
// it returns is called, which waits for Run to return. It returns the
// Sender, and what it logs.
func run(t *testing.T, settings remotelog.Params) (*remotelog.Sender, *output, func()) {
t.Helper()
sender := remotelog.New(settings)
logged := &output{}
ctx, cancel := context.WithCancel(t.Context())
ran := make(chan struct{})
go func() {
sender.Run(ctx, slog.New(slog.NewJSONHandler(logged, nil)))
close(ran)
}()
stop := func() {
cancel()
<-ran
}
t.Cleanup(stop)
return sender, logged, stop
}
// listen returns a TCP listener on the loopback address, closed when the
// test ends.
func listen(t *testing.T) net.Listener {
t.Helper()
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", loopback)
if err != nil {
t.Fatalf("listen: %v", err)
}
t.Cleanup(func() { _ = listener.Close() })
return listener
}
// accept returns the next connection to listener, closed when the test
// ends.
func accept(t *testing.T, listener net.Listener) net.Conn {
t.Helper()
conn, err := listener.Accept()
if err != nil {
t.Fatalf("accept: %v", err)
}
t.Cleanup(func() { _ = conn.Close() })
return conn
}
// tlsEndpoint is a syslog+tls endpoint on the loopback address, which a
// test starts outside its bubble. It keeps its listener until the test
// ends, and either takes each connection or refuses it.
type tlsEndpoint struct {
addr net.Addr
// refusing is set while the endpoint closes each connection before the
// TLS handshake, which fails the sender's attempt to connect.
refusing atomic.Bool
// conns are the connections it has taken, after the handshake.
conns chan net.Conn
}
// startTLSEndpoint starts a tlsEndpoint with certificate, which takes
// connections until it is told to refuse them.
func startTLSEndpoint(t *testing.T, certificate tls.Certificate) *tlsEndpoint {
t.Helper()
listener := listen(t)
endpoint := &tlsEndpoint{addr: listener.Addr(), conns: make(chan net.Conn, 10)}
config := &tls.Config{
Certificates: []tls.Certificate{certificate},
MinVersion: tls.VersionTLS12,
}
go func() {
for {
conn, err := listener.Accept()
if err != nil {
return
}
server := tls.Server(conn, config)
if endpoint.refusing.Load() || server.HandshakeContext(t.Context()) != nil {
_ = conn.Close()
continue
}
endpoint.conns <- server
}
}()
return endpoint
}
// next returns the next connection the endpoint has taken, closed when
// the test ends.
func (e *tlsEndpoint) next(t *testing.T) net.Conn {
t.Helper()
conn := <-e.conns
t.Cleanup(func() { _ = conn.Close() })
return conn
}
// record returns the record of line made as the test started, with the
// priority and the app name given.
func record(t *testing.T, priority, app, line string) string {
t.Helper()
hostname, err := os.Hostname()
if err != nil || hostname == "" {
hostname = "-"
}
return priority + "1 " + started + " " + hostname + " " + app + " - - - " + line
}
// wantFrame reads the next octet-counted frame from frames, and checks
// that it holds want.
func wantFrame(t *testing.T, frames *bufio.Reader, want string) {
t.Helper()
count, err := frames.ReadString(' ')
if err != nil {
t.Fatalf("read a frame's length: %v", err)
}
length, err := strconv.Atoi(strings.TrimSuffix(count, " "))
if err != nil {
t.Fatalf("frame starts %q, not with its length", count)
}
got := make([]byte, length)
_, err = io.ReadFull(frames, got)
if err != nil {
t.Fatalf("read a frame: %v", err)
}
if string(got) != want {
t.Errorf("frame %q, want %q", got, want)
}
}
// wantCounts checks the records sender has sent, dropped and holds in
// its buffer.
func wantCounts(
t *testing.T, sender *remotelog.Sender, sent, dropped int64, depth int,
) {
t.Helper()
if sender.Sent() != sent || sender.Dropped() != dropped || sender.Depth() != depth {
t.Fatalf("sent %d, dropped %d, %d in the buffer; want %d, %d and %d",
sender.Sent(), sender.Dropped(), sender.Depth(), sent, dropped, depth)
}
}
// writeUntilDropped writes a line at a time until the count of records
// sender has dropped reaches dropped. The records it sends on a
// connection the endpoint has closed are lost before one fails; how many
// depends on when the endpoint's host answers that the connection is
// gone.
func writeUntilDropped(t *testing.T, sender *remotelog.Sender, dropped int64) {
t.Helper()
for sender.Dropped() < dropped {
_, _ = sender.Write([]byte("lost\n"))
synctest.Wait()
}
}
// wantConnectedAgainAfter writes a line while the sender waits to connect
// again, and checks that it connects, and takes the line from the buffer,
// only once delay is over.
func wantConnectedAgainAfter(
t *testing.T, sender *remotelog.Sender, delay time.Duration,
) {
t.Helper()
_, _ = sender.Write([]byte("waiting\n"))
time.Sleep(delay - time.Nanosecond)
synctest.Wait()
if sender.Depth() != 1 {
t.Fatalf("connected again before %v", delay)
}
time.Sleep(time.Nanosecond)
synctest.Wait()
if sender.Depth() != 0 {
t.Fatalf("not connected again after %v", delay)
}
}
// wantRetries checks that the sender logged a failure, of an attempt to
// connect or of a connection, for each of delays, the time until the next
// attempt, in order, and logged nothing else.
func wantRetries(t *testing.T, logged *output, delays ...string) {
t.Helper()
var got []string
for line := range strings.Lines(logged.text()) {
var fields map[string]any
err := json.Unmarshal([]byte(line), &fields)
if err != nil || fields["msg"] != "sending to SWWAF_LOG_REMOTE_URL failed" {
t.Fatalf("logged %q", line)
}
delay, _ := fields["connecting_again_in"].(string)
got = append(got, delay)
}
if !slices.Equal(got, delays) {
t.Errorf("logged failures to connect again in %v, want %v", got, delays)
}
}
// testCertificate returns a certificate for 127.0.0.1 that is its own
// CA, and a pool that holds it. It is valid on the bubble's clock, which
// starts at 2000-01-01T00:00:00Z.
func testCertificate(t *testing.T) (tls.Certificate, *x509.CertPool) {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatalf("generate a key: %v", err)
}
template := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "smallwebwaf test CA"},
NotBefore: time.Date(1999, 12, 31, 0, 0, 0, 0, time.UTC),
NotAfter: time.Date(2000, 1, 2, 0, 0, 0, 0, time.UTC),
IsCA: true,
BasicConstraintsValid: true,
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
}
der, err := x509.CreateCertificate(rand.Reader, template, template,
&key.PublicKey, key)
if err != nil {
t.Fatalf("create a certificate: %v", err)
}
certificate, err := x509.ParseCertificate(der)
if err != nil {
t.Fatalf("parse the certificate: %v", err)
}
roots := x509.NewCertPool()
roots.AddCert(certificate)
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}, roots
}
internal/requestlog/requestlog.go
+30 -93
View File
@@ -9,8 +9,6 @@ import (
"io"
"log/slog"
"time"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
)
// The action a request line names: what smallwebwaf did with the
@@ -45,115 +43,54 @@ const (
)
// OffenceLimit is the offence a request line names for a request that
// broke a rate limit, or whose bytes broke a byte limit.
// broke a rate limit.
const OffenceLimit = "limit"
// timeLayout is RFC 3339 with milliseconds.
const timeLayout = "2006-01-02T15:04:05.000Z07:00"
// Line is one request's line in the request log. The field names, and
// their order, are those of the "Request log" section of SPEC.md. A field
// that may not apply to a request is left out of its line when it does
// not.
// Line is one request's line in the request log. The field names are
// those of the "Request log" section of SPEC.md.
//
//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
type Line struct {
Type string `json:"type"`
// The standard web log fields. Scheme is how the client reached
// smallwebwaf, or the trusted proxy in front of it.
Time string `json:"time"`
Instance string `json:"instance"`
ClientIP string `json:"client_ip"`
Method string `json:"method"`
Scheme string `json:"scheme"`
Host string `json:"host"`
Path string `json:"path"`
Query string `json:"query"`
Protocol string `json:"protocol"`
Status int `json:"status"`
RequestBytes int64 `json:"request_bytes"`
ResponseBytes int64 `json:"response_bytes"`
Referer string `json:"referer"`
UserAgent string `json:"user_agent"`
// Request detail. RequestID is the X-Request-ID a trusted proxy sent,
// or a new one, and is sent on to the app. ForwardedFor is the
// X-Forwarded-For header as received. ClientGroup is the netblock the
// client is counted as. ASN, ASName and Country are the client's AS
// number, AS name and country, as looked up.
RequestID string `json:"request_id"`
PeerIP string `json:"peer_ip"`
ForwardedFor string `json:"forwarded_for,omitempty"`
ClientGroup string `json:"client_group"`
ASN string `json:"asn"`
ASName string `json:"as_name"`
Country string `json:"country"`
ContentType string `json:"content_type,omitempty"`
// ContentLength is the length of its body the request announced.
ContentLength int64 `json:"content_length,omitempty"`
// RequestHeaders are the headers SWWAF_LOG_REQUEST_HEADERS names that
// the request carried, by name in lower case.
RequestHeaders map[string]string `json:"request_headers,omitempty"`
HasAuthorization bool `json:"has_authorization,omitempty"`
HasCookie bool `json:"has_cookie,omitempty"`
// Websocket is true when the connection was upgraded, as for a
// WebSocket.
Websocket bool `json:"websocket,omitempty"`
// Response detail, from the headers of the answer: the app's, as
// passed on, or those of smallwebwaf's own. Aborted is true when the
// client went away early.
ResponseContentType string `json:"response_content_type,omitempty"`
UpstreamStatus int `json:"upstream_status,omitempty"`
CacheControl string `json:"cache_control,omitempty"`
Location string `json:"location,omitempty"`
Aborted bool `json:"aborted,omitempty"`
// The decision.
Action string `json:"action"`
Type string `json:"type"`
Time string `json:"time"`
ClientIP string `json:"client_ip"`
PeerIP string `json:"peer_ip"`
Country string `json:"country"`
Method string `json:"method"`
Host string `json:"host"`
Path string `json:"path"`
Query string `json:"query"`
Protocol string `json:"protocol"`
Status int `json:"status"`
UpstreamStatus int `json:"upstream_status,omitempty"`
RequestBytes int64 `json:"request_bytes"`
ResponseBytes int64 `json:"response_bytes"`
Referer string `json:"referer"`
UserAgent string `json:"user_agent"`
Action string `json:"action"`
// WouldAction is, in observe mode, the action enforce mode would have
// taken with a request it would have refused: ActionDenied,
// ActionBanned, ActionCountryDenied, ActionRateLimited or
// ActionRuleBlocked.
WouldAction string `json:"would_action,omitempty"`
// LimitPercent and LimitPercentSetting are, for a request the rate
// limits counted whose client a biased threshold gives a percentage of
// the rate limits below 100, that percentage and the setting that gave
// it. BytesPercent and BytesPercentSetting are the same for the byte
// limits.
LimitPercent *int64 `json:"limit_percent,omitempty"`
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
BytesPercent *int64 `json:"bytes_percent,omitempty"`
BytesPercentSetting string `json:"bytes_percent_setting,omitempty"`
// Counts are, for a request the rate limits counted, the client's
// requests as they counted them with this one, and its bytes as the
// byte limits counted them, with this request's once it has ended if
// they count them.
Counts ratelimit.Counts `json:"counts,omitzero"`
// RuleIDs are the ids of the rule file rules the request matched.
RuleIDs []string `json:"rule_ids,omitempty"`
// LimitHit is the window whose limit the request went over, named as
// Counts names its count: minute, hour or day for a rate limit, and
// minute_bytes, hour_bytes or day_bytes for a byte limit.
// LimitHit is the window whose rate limit the request went over:
// minute, hour or day.
LimitHit string `json:"limit_hit,omitempty"`
// Offence is the offence the request was held as, OffenceLimit.
Offence string `json:"offence,omitempty"`
// BanExpires is when the ban the request made, or was refused under,
// ends: a time, or "permanent".
BanExpires string `json:"ban_expires,omitempty"`
// The timings, in milliseconds. DurationChecks is the time until the
// checks were done. DurationUpstreamConnect, DurationUpstreamFirstByte
// and DurationUpstreamTotal run from when the request was handed to the
// app: until there was a connection to it, until the first byte of its
// answer arrived, and until the end. Each but DurationTotal is nil for
// a request that did not get that far.
DurationTotal float64 `json:"duration_total"`
DurationChecks *float64 `json:"duration_checks,omitempty"`
DurationUpstreamConnect *float64 `json:"duration_upstream_connect,omitempty"`
DurationUpstreamFirstByte *float64 `json:"duration_upstream_first_byte,omitempty"`
DurationUpstreamTotal *float64 `json:"duration_upstream_total,omitempty"`
// Aborted is true when the client went away early.
Aborted bool `json:"aborted,omitempty"`
// DurationTotal and DurationUpstreamTotal are in milliseconds.
DurationTotal float64 `json:"duration_total"`
DurationUpstreamTotal float64 `json:"duration_upstream_total,omitempty"`
}
// Write writes line to w as one JSON line marked "type":"request".
@@ -186,8 +123,8 @@ func Milliseconds(d time.Duration) float64 {
// NewProcessLogger returns the logger for the process's own messages:
// JSON lines on w, marked "type":"process", with the time in the same form
// as a request line's, and instanceName, SWWAF_INSTANCE_NAME, as instance.
func NewProcessLogger(w io.Writer, instanceName string) *slog.Logger {
// as a request line's.
func NewProcessLogger(w io.Writer) *slog.Logger {
handler := slog.NewJSONHandler(w, &slog.HandlerOptions{
ReplaceAttr: func(groups []string, attr slog.Attr) slog.Attr {
if attr.Key == slog.TimeKey && len(groups) == 0 {
@@ -198,5 +135,5 @@ func NewProcessLogger(w io.Writer, instanceName string) *slog.Logger {
},
})
return slog.New(handler).With("type", "process", "instance", instanceName)
return slog.New(handler).With("type", "process")
}
internal/requestlog/requestlog_test.go
+5 -10
View File
@@ -50,11 +50,7 @@ func TestWriteWritesOneJSONLineMarkedRequest(t *testing.T) {
}
unset := []string{
"forwarded_for", "content_type", "content_length", "request_headers",
"has_authorization", "has_cookie", "websocket", "response_content_type",
"upstream_status", "cache_control", "location", "aborted", "counts",
"limit_hit", "offence", "ban_expires", "duration_checks",
"duration_upstream_connect", "duration_upstream_first_byte",
"upstream_status", "limit_hit", "offence", "ban_expires", "aborted",
"duration_upstream_total",
}
for _, name := range unset {
@@ -65,12 +61,12 @@ func TestWriteWritesOneJSONLineMarkedRequest(t *testing.T) {
}
}
func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
func TestProcessLinesAreMarkedProcess(t *testing.T) {
t.Parallel()
var out bytes.Buffer
requestlog.NewProcessLogger(&out, "fsn1app1/gitea").Info("starting", "version", "v1")
requestlog.NewProcessLogger(&out).Info("starting", "version", "v1")
var fields map[string]any
@@ -79,9 +75,8 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
t.Fatalf("decode %q: %v", out.String(), err)
}
if fields["type"] != "process" || fields["instance"] != "fsn1app1/gitea" ||
fields["msg"] != "starting" || fields["level"] != "INFO" ||
fields["version"] != "v1" {
if fields["type"] != "process" || fields["msg"] != "starting" ||
fields["level"] != "INFO" || fields["version"] != "v1" {
t.Errorf("process line %v", fields)
}
internal/rules/rules.go
+32 -125
View File
@@ -1,8 +1,7 @@
// Package rules reads the rule files: the plain text files in
// SWWAF_RULES_DIR, one rule to a line, that each request is checked
// against, as the "Rule files" section of SPEC.md describes. They are read
// at start, and again once the directory has had no change for a short
// time after one is edited, added or removed.
// at start, and again whenever one is edited, added or removed.
package rules
import (
@@ -18,12 +17,8 @@ import (
"slices"
"strings"
"sync/atomic"
"time"
"github.com/fsnotify/fsnotify"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config"
)
// The actions a rule takes when it matches.
@@ -40,12 +35,6 @@ const (
// extension ends the name of every rule file.
const extension = ".rules"
// quietTime is how long SWWAF_RULES_DIR must go without a change before
// the rule files are read again, so that a file still being written, such
// as one saved in place, appended to or copied in with scp, is read only
// once whole.
const quietTime = 2 * time.Second
// headerTarget starts the target that is one request header,
// header:<Name>.
const headerTarget = "header:"
@@ -68,11 +57,6 @@ var (
errNotID = errors.New("is not an id of letters, digits, - and _")
errNotTarget = errors.New(
"is not path, query, uri, method, host, user_agent, referer or header:<Name>")
errNotHeaderName = errors.New(
"has a character after header: that no header name can have")
errHeaderTakenOut = errors.New(
"names a header that Go's HTTP server takes out of every request, " +
"so a rule never sees it")
errNotAction = errors.New("is not log, block or ban")
errNotRegex = errors.New("does not compile")
errUsedTwice = errors.New("is already the id of the rule at")
@@ -101,8 +85,6 @@ type Params struct {
// ProcessLog receives how many rules were read, and the error in a
// rule file edited while smallwebwaf runs.
ProcessLog *slog.Logger
// Alerts receive a file_error alert for that error.
Alerts *alerts.Queue
}
// Files are the rule files of a running smallwebwaf, and the rules read
@@ -116,11 +98,8 @@ type Files struct {
// Load reads the rules of every *.rules file in Dir, in the order of the
// files' names, unless Enabled is false. A Dir that cannot be read is an
// error, and so is a line that is not a rule, a header name with a
// character no header name can have, a rule for the Host or the
// Transfer-Encoding header, which Go's HTTP server takes out of every
// request, a regex that does not compile and an id used twice, each named
// with its file and line.
// error, and so is a line that is not a rule, a regex that does not
// compile and an id used twice, each named with its file and line.
func Load(params Params) (*Files, error) {
f := &Files{params: params}
f.rules.Store(&[]Rule{})
@@ -129,7 +108,7 @@ func Load(params Params) (*Files, error) {
return f, nil
}
rules, _, err := read(params.Dir)
rules, err := read(params.Dir)
if err != nil {
return nil, err
}
@@ -166,12 +145,11 @@ func (f *Files) Len() int {
}
// Watch watches Dir until ctx is done, and reads the rule files again
// once Dir has had no change for quietTime, after one is edited, added or
// removed, and after Watch starts watching. If they then hold an error,
// whenever one is edited, added or removed. If they then hold an error,
// the rules stay as they were, the error is logged with its file and
// line, and the files are read again after the next change. If Dir cannot
// be watched, that is logged, and the rules stay as they were loaded.
// While Enabled is false, Watch returns at once.
// line, and the files are read again at the next change. If Dir cannot be
// watched, that is logged, and the rules stay as they were loaded. While
// Enabled is false, Watch returns at once.
func (f *Files) Watch(ctx context.Context) {
if !f.params.Enabled {
return
@@ -196,29 +174,15 @@ func (f *Files) Watch(ctx context.Context) {
f.params.ProcessLog.Info("watching the rule files for edits",
"directory", f.params.Dir)
f.readAfterChanges(ctx, watcher.Events, watcher.Errors)
}
// readAfterChanges reads the rule files again once quietTime has passed
// without a change from events, until ctx is done, and logs the errors
// from errs. The wait starts at once, as if for a change, so that an edit
// saved after Load read the files, and before Dir was watched, is taken
// in too.
func (f *Files) readAfterChanges(
ctx context.Context, events <-chan fsnotify.Event, errs <-chan error,
) {
quiet := time.NewTimer(quietTime)
defer quiet.Stop()
for {
select {
case <-ctx.Done():
return
case <-events:
quiet.Reset(quietTime)
case <-quiet.C:
f.readAgain()
case err := <-errs:
case event := <-watcher.Events:
if filepath.Ext(event.Name) == extension {
f.readAgain()
}
case err = <-watcher.Errors:
f.params.ProcessLog.Warn("watching the rule files failed",
"error", err.Error())
}
@@ -226,21 +190,13 @@ func (f *Files) readAfterChanges(
}
// readAgain reads the rule files again, in place of the rules loaded, or
// logs the error that keeps the rules as they were, and raises a
// file_error alert for it, for the file it is in.
// logs the error that keeps the rules as they were.
func (f *Files) readAgain() {
rules, path, err := read(f.params.Dir)
rules, err := read(f.params.Dir)
if err != nil {
const kept = "a rule file has an error, and the rules stay as they were"
// Raised before it is logged, so that the alert is there once the
// log line is.
f.params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError,
Reason: kept,
Detail: map[string]any{"file": path, "error": err.Error()},
})
f.params.ProcessLog.Error(kept, "error", err.Error())
f.params.ProcessLog.Error(
"a rule file has an error, and the rules stay as they were",
"error", err.Error())
return
}
@@ -257,14 +213,11 @@ func (f *Files) logRead(count int) {
}
// read returns the rules of every rule file in dir, in the order of the
// files' names, and then of their lines, or an error, with the path of the
// rule file it is in, or dir. A file whose name starts with a dot, such as
// an editor's lock file .#50-app.rules, is not a rule file, as a shell's
// *.rules would not match it.
func read(dir string) ([]Rule, string, error) {
// files' names, and then of their lines.
func read(dir string) ([]Rule, error) {
entries, err := os.ReadDir(dir)
if err != nil {
return nil, dir, fmt.Errorf("SWWAF_RULES_DIR cannot be read: %w", err)
return nil, fmt.Errorf("SWWAF_RULES_DIR cannot be read: %w", err)
}
var rules []Rule
@@ -273,20 +226,17 @@ func read(dir string) ([]Rule, string, error) {
places := map[string]string{}
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || strings.HasPrefix(name, ".") || filepath.Ext(name) != extension {
if entry.IsDir() || filepath.Ext(entry.Name()) != extension {
continue
}
path := filepath.Join(dir, name)
rules, err = readFile(path, rules, places)
rules, err = readFile(filepath.Join(dir, entry.Name()), rules, places)
if err != nil {
return nil, path, err
return nil, err
}
}
return rules, "", nil
return rules, nil
}
// readFile appends the rules of the rule file at path to rules. places
@@ -327,11 +277,9 @@ func readFile(path string, rules []Rule, places map[string]string) ([]Rule, erro
}
// parse reads a line of a rule file. It returns false for a blank line
// and for a comment, a line that starts with #. Spaces and tabs at the
// end of the line are not part of its regex, so a line with only those
// after its action has no regex, and is not a rule.
// and for a comment, a line that starts with #.
func parse(line string) (Rule, bool, error) {
line = strings.Trim(line, " \t")
line = strings.TrimLeft(line, " \t")
if line == "" || strings.HasPrefix(line, "#") {
return Rule{}, false, nil
}
@@ -342,21 +290,12 @@ func parse(line string) (Rule, bool, error) {
}
rule := Rule{ID: fields[1], Target: fields[2], Action: fields[3]}
headerName, isHeader := strings.CutPrefix(rule.Target, headerTarget)
switch {
case !idChars.MatchString(rule.ID):
return Rule{}, false, fmt.Errorf("the id %q %w", rule.ID, errNotID)
case !isTarget(rule.Target):
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotTarget)
case isHeader && !config.IsHeaderName(headerName):
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotHeaderName)
case strings.EqualFold(rule.Target, headerTarget+"Host"):
return Rule{}, false, fmt.Errorf(
"the target %q %w; the request's host is the target host",
rule.Target, errHeaderTakenOut)
case strings.EqualFold(rule.Target, headerTarget+"Transfer-Encoding"):
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errHeaderTakenOut)
case !slices.Contains([]string{ActionLog, ActionBlock, ActionBan}, rule.Action):
return Rule{}, false, fmt.Errorf("the action %q %w", rule.Action, errNotAction)
}
@@ -388,7 +327,7 @@ func isTarget(target string) bool {
// them once percent-decoded, so that an encoded probe cannot slip past.
func (rule Rule) matches(r *http.Request) bool {
if rule.Target == "uri" {
uri := pathAndQuery(r)
uri := r.URL.RequestURI()
return rule.regex.MatchString(uri) || rule.regex.MatchString(decodeOnce(uri))
}
@@ -398,18 +337,14 @@ func (rule Rule) matches(r *http.Request) bool {
// value returns what a rule with target, other than uri, is matched
// against in r: the path and the query as the client sent them, before
// any decoding or re-encoding, split at the first ?, and a header's values
// joined by ", ", as HTTP joins those of a header sent more than once.
// any decoding, and a header's values joined by ", ", as HTTP joins those
// of a header sent more than once.
func value(target string, r *http.Request) string {
switch target {
case "path":
path, _, _ := strings.Cut(pathAndQuery(r), "?")
return path
return r.URL.EscapedPath()
case "query":
_, query, _ := strings.Cut(pathAndQuery(r), "?")
return query
return r.URL.RawQuery
case "method":
return r.Method
case "host":
@@ -423,34 +358,6 @@ func value(target string, r *http.Request) string {
}
}
// pathAndQuery returns the target of r's request line, r.RequestURI, as
// the client sent it, less any scheme and host: a target with a scheme
// gives what follows the scheme and its :, and the host when // follows.
// So http://host/path, as a client sends it to a proxy, gives /path, and
// so does http:/path, which Go reads as a target with a scheme and no
// host. r.URL is not used: when the path holds a character it escapes,
// such as \ or a non-ASCII byte, it decodes the whole path and escapes it
// again, so that \ becomes %5C and %2e a dot.
func pathAndQuery(r *http.Request) string {
if !r.URL.IsAbs() {
return r.RequestURI
}
_, afterScheme, _ := strings.Cut(r.RequestURI, ":")
hostAndRest, hasHost := strings.CutPrefix(afterScheme, "//")
if !hasHost {
return afterScheme
}
start := strings.IndexAny(hostAndRest, "/?")
if start < 0 {
return ""
}
return hostAndRest[start:]
}
// header returns the values of r's header name joined by ", ", or "" if
// r has no such header.
func header(r *http.Request, name string) string {
internal/rules/rules_test.go
+9 -126
View File
@@ -7,15 +7,12 @@ import (
"maps"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"slices"
"strconv"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/rules"
)
@@ -98,35 +95,6 @@ func TestEachTargetMatchesWhatItNames(t *testing.T) {
}
}
func TestPathMatchedAsTheClientSentIt(t *testing.T) {
t.Parallel()
// Each path holds a character Go's URL type would escape again, \ or
// a non-ASCII byte, and each rule is written for the path as sent.
for _, tc := range []struct {
rule string // its target, action and regex
sent string // the path and query the client sent
}{
{`path log ^/\.\.\\\.\.\\windows\\win\.ini$`, `/..\..\windows\win.ini`},
{`path log ^/%2e%2e\\%2e%2e\\windows\\win\.ini$`, `/%2e%2e\%2e%2e\windows\win.ini`},
{`path log ^/café$`, "/café?x=1"},
{`uri log ^/%2e%2e\\%2e%2e\\boot\.ini\?x=1$`, `/%2e%2e\%2e%2e\boot.ini?x=1`},
} {
files := load(t, ruleFiles{testFile: "as-sent " + tc.rule + "\n"})
// The target in origin form, as traefik sends it, in absolute form,
// as a client sends it to a proxy, and with a scheme but no host,
// which Go reads as absolute form with no host, sending the app
// the path.
for _, target := range []string{
tc.sent, "http://app.example" + tc.sent, "http:" + tc.sent, "foo:" + tc.sent,
} {
r := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
wantMatched(t, files, r, "as-sent")
}
}
}
func TestMatchingStopsAtTheFirstRuleThatRefuses(t *testing.T) {
t.Parallel()
@@ -148,14 +116,6 @@ after path log ^/
wantMatched(t, files, get(t, "/"), "ban")
}
func TestSpacesAndTabsEndingALineAreNotPartOfItsRegex(t *testing.T) {
t.Parallel()
files := load(t, ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"})
wantMatched(t, files, get(t, "/.env"), "env-file")
}
func TestFilesReadInNameOrderThenLineOrder(t *testing.T) {
t.Parallel()
@@ -175,28 +135,6 @@ func TestFilesReadInNameOrderThenLineOrder(t *testing.T) {
}
}
func TestFileWhoseNameStartsWithADotIsNotARuleFile(t *testing.T) {
t.Parallel()
dir := writeFiles(t, ruleFiles{firstFile: "probe path block ^/probe\n"})
// The lock file Emacs makes beside a file while it is edited: a link to
// nothing, which cannot be read.
err := os.Symlink("user@host.1234:1700000000", filepath.Join(dir, ".#"+firstFile))
if err != nil {
t.Fatalf("symlink: %v", err)
}
params, _ := newParams(dir)
files, err := rules.Load(params)
if err != nil {
t.Fatalf("load: %v", err)
}
wantMatched(t, files, get(t, "/probe"), "probe")
}
func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
t.Parallel()
@@ -211,12 +149,6 @@ func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
"is not a rule: an id, a target, an action and a regex, " +
"separated by spaces or tabs",
},
{
// Else its regex would be a space, found in nearly every user agent.
"a regex of only spaces and tabs", "scanner user_agent ban\t \n", 1,
"is not a rule: an id, a target, an action and a regex, " +
"separated by spaces or tabs",
},
{
"an id of other characters", "# ids\n\nenv.file path ban ^/\n", 3,
`the id "env.file" is not an id of letters, digits, - and _`,
@@ -231,33 +163,6 @@ func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
`the target "header:" is not path, query, uri, method, host, ` +
"user_agent, referer or header:<Name>",
},
{
"a header name written with its colon", "sqlmap header:User-Agent: ban sqlmap\n", 1,
`the target "header:User-Agent:" has a character after header: ` +
"that no header name can have",
},
{
"a header name with a semicolon", "accept header:Accept;q log ^$\n", 1,
`the target "header:Accept;q" has a character after header: ` +
"that no header name can have",
},
{
"a header name with brackets", "x-header header:X(y) log ^$\n", 1,
`the target "header:X(y)" has a character after header: ` +
"that no header name can have",
},
{
"the Host header", "host-header header:host block ^$\n", 1,
`the target "header:host" names a header that Go's HTTP server ` +
"takes out of every request, so a rule never sees it; " +
"the request's host is the target host",
},
{
"the Transfer-Encoding header",
"# bodies sent in chunks\nchunked header:Transfer-Encoding block ^chunked$\n", 2,
`the target "header:Transfer-Encoding" names a header that Go's ` +
"HTTP server takes out of every request, so a rule never sees it",
},
{
"an unknown action", "env-file path deny ^/\n", 1,
`the action "deny" is not log, block or ban`,
@@ -340,7 +245,7 @@ func TestEditsTakenInWhileRunning(t *testing.T) {
t.Parallel()
dir := writeFiles(t, ruleFiles{firstFile: "first path block ^/first\n"})
files, lines, _ := watch(t, dir)
files, lines := watch(t, dir)
// matches reports whether path matches a rule.
matches := func(path string) bool { return len(files.Match(get(t, path))) == 1 }
@@ -369,7 +274,7 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
t.Parallel()
dir := writeFiles(t, ruleFiles{firstFile: "first path block ^/first\n"})
files, lines, queue := watch(t, dir)
files, lines := watch(t, dir)
// The edit's second line has an unknown action, so the rules stay as
// they were, the first line's earlier version included.
@@ -383,27 +288,13 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
t.Errorf("logged %v, want an error %q", line, want)
}
// The error is raised as a file_error alert too, for the file.
wantFileError := func() {
t.Helper()
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError ||
waiting[0].Reason != hasError || waiting[0].Detail["error"] != want ||
waiting[0].Detail["file"] != filepath.Join(dir, firstFile) {
t.Errorf("alerts waiting %+v, want one file_error alert for %q", waiting, want)
}
}
wantFileError()
wantMatched(t, files, get(t, "/first"), "first")
wantMatched(t, files, get(t, "/second"))
// Once mended, the file is read again, and raises no alert.
// Once mended, the file is read again.
save(t, dir, firstFile, "first path block ^/edited\nsecond path ban ^/second\n")
lines.waitUntil(t, func() bool { return len(files.Match(get(t, "/second"))) == 1 })
wantMatched(t, files, get(t, "/edited"), "first")
wantFileError()
}
func TestDefaultFileBansProbesAtTheSiteRootAlone(t *testing.T) {
@@ -521,8 +412,7 @@ func save(t *testing.T, dir, name, content string) {
}
// newParams returns Params for the rule files in dir, switched on, with
// the process log in the processLog returned, and the alerts waiting in a
// queue for a webhook that is never sent them.
// the process log in the processLog returned.
func newParams(dir string) (rules.Params, processLog) {
lines := make(processLog, maxLogLines)
@@ -530,12 +420,6 @@ func newParams(dir string) (rules.Params, processLog) {
Dir: dir,
Enabled: true,
ProcessLog: slog.New(slog.NewJSONHandler(lines, nil)),
Alerts: alerts.New(alerts.Params{
WebhookURL: &url.URL{Scheme: "https", Host: "alerts.example"},
Events: alerts.Events(),
Cooldown: 15 * time.Minute,
Now: time.Now,
}),
}, lines
}
@@ -555,9 +439,8 @@ func load(t *testing.T, files ruleFiles) *rules.Files {
}
// watch loads the rules in dir, runs their Watch until the test ends, and
// waits until it watches the directory. It returns the alerts' queue as
// well.
func watch(t *testing.T, dir string) (*rules.Files, processLog, *alerts.Queue) {
// waits until it watches the directory.
func watch(t *testing.T, dir string) (*rules.Files, processLog) {
t.Helper()
params, lines := newParams(dir)
@@ -582,7 +465,7 @@ func watch(t *testing.T, dir string) (*rules.Files, processLog, *alerts.Queue) {
lines.waitFor(t, watching)
return files, lines, params.Alerts
return files, lines
}
// wantRefused checks that loading the rule files in dir fails with the
@@ -676,8 +559,8 @@ func (l processLog) waitFor(t *testing.T, msg string) map[string]any {
}
// waitUntil waits for the rule files to be read until done reports true,
// as it does once they have been read after the test's last change. They
// can be read before then too, as they are once Watch starts watching.
// as it does once they have been read after the test's last change. One
// change can be seen more than once, and so read more than once.
func (l processLog) waitUntil(t *testing.T, done func() bool) {
t.Helper()
internal/rules/watch_internal_test.go
-165
View File
@@ -1,165 +0,0 @@
package rules
import (
"context"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"slices"
"testing"
"testing/synctest"
"time"
"github.com/fsnotify/fsnotify"
"sneak.berlin/go/smallwebwaf/internal/alerts"
)
// The tests below run readAfterChanges in a synctest bubble, where time is
// a clock of the test's own: time.Sleep moves it on at once, and
// synctest.Wait returns once readAfterChanges waits again, so that every
// reading due by then is done. The test sends the changes itself, as the
// watch of a directory cannot run in a bubble.
func TestFileWrittenInTwoPartsTakenInOnlyWhole(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "50-app.rules")
writeFile(t, path, "first path block ^/first\n")
files := load(t, dir)
changes := run(t, files)
file, err := os.Create(path) //nolint:gosec // a file the test wrote
if err != nil {
t.Fatalf("create: %v", err)
}
defer func() {
_ = file.Close()
}()
// The first part ends in the middle of a ban rule's regex, which,
// read then, would ban every request.
write(t, file, "first path block ^/first\nprobe path ban ^/")
changes <- fsnotify.Event{Name: path, Op: fsnotify.Write}
time.Sleep(quietTime - time.Nanosecond)
synctest.Wait()
wantMatched(t, files, "/anything")
// The second part starts the wait again.
write(t, file, `\.env$`+"\n")
changes <- fsnotify.Event{Name: path, Op: fsnotify.Write}
time.Sleep(quietTime - time.Nanosecond)
synctest.Wait()
wantMatched(t, files, "/.env")
time.Sleep(time.Nanosecond)
synctest.Wait()
wantMatched(t, files, "/.env", "probe")
wantMatched(t, files, "/anything")
})
}
func TestEditSavedBeforeTheWatchStartsTakenIn(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "50-app.rules")
writeFile(t, path, "first path block ^/first\n")
files := load(t, dir)
// Saved after Load read the files, and before the directory was
// watched, so that no change is seen for it.
writeFile(t, path, "first path block ^/edited\n")
run(t, files)
time.Sleep(quietTime)
synctest.Wait()
wantMatched(t, files, "/edited", "first")
})
}
// load loads the rules in dir.
func load(t *testing.T, dir string) *Files {
t.Helper()
files, err := Load(Params{
Dir: dir, Enabled: true, ProcessLog: slog.New(slog.DiscardHandler),
Alerts: alerts.New(alerts.Params{}),
})
if err != nil {
t.Fatalf("load: %v", err)
}
return files
}
// run runs files' readAfterChanges until the test ends, and returns the
// channel that sends it changes.
func run(t *testing.T, files *Files) chan<- fsnotify.Event {
t.Helper()
changes := make(chan fsnotify.Event)
ctx, stop := context.WithCancel(t.Context())
stopped := make(chan struct{})
go func() {
files.readAfterChanges(ctx, changes, nil)
close(stopped)
}()
t.Cleanup(func() {
stop()
<-stopped
})
return changes
}
// writeFile writes content to the file at path.
func writeFile(t *testing.T, path, content string) {
t.Helper()
err := os.WriteFile(path, []byte(content), 0o600)
if err != nil {
t.Fatalf("write %s: %v", path, err)
}
}
// write writes text to the end of file.
func write(t *testing.T, file *os.File, text string) {
t.Helper()
_, err := file.WriteString(text)
if err != nil {
t.Fatalf("write: %v", err)
}
}
// wantMatched checks the ids of the rules that a GET request for path
// matches, in order.
func wantMatched(t *testing.T, files *Files, path string, want ...string) {
t.Helper()
r := httptest.NewRequestWithContext(t.Context(), http.MethodGet,
"http://app.example"+path, nil)
matched := files.Match(r)
got := make([]string, 0, len(matched))
for _, rule := range matched {
got = append(got, rule.ID)
}
if !slices.Equal(got, want) {
t.Errorf("%s matched %v, want %v", path, got, want)
}
}
internal/smallwebwaf/healthcheck.go
+3 -5
View File
@@ -23,8 +23,6 @@ var errHealthEndpoint = errors.New("smallwebwaf's health endpoint answered")
// smallwebwaf answers its health endpoint on 127.0.0.1, at the port in
// SWWAF_LISTEN_ADDR, and the app accepts connections at the address in
// SWWAF_UPSTREAM_URL. Otherwise it writes why to stderr and returns 1.
// It reads no other setting, nor a file that another names, so neither
// can fail it.
// args are the arguments after `healthcheck`; it takes none, and given
// one it names it on stderr and returns 1 without checking anything.
func HealthCheck(
@@ -52,13 +50,13 @@ func healthCheck(ctx context.Context, lookupEnv func(string) (string, bool)) err
ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout)
defer cancel()
listenAddr, upstreamURL, err := config.ListenAddrAndUpstreamURL(lookupEnv)
cfg, err := config.FromEnvironment(lookupEnv)
if err != nil {
return fmt.Errorf("invalid setting: %w", err)
}
// The settings have checked that the address has a port.
_, port, _ := net.SplitHostPort(listenAddr)
_, port, _ := net.SplitHostPort(cfg.ListenAddr)
health := "http://" + net.JoinHostPort("127.0.0.1", port) + proxy.HealthPath
req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody)
@@ -77,7 +75,7 @@ func healthCheck(ctx context.Context, lookupEnv func(string) (string, bool)) err
return fmt.Errorf("%w %s", errHealthEndpoint, res.Status)
}
conn, err := (&net.Dialer{}).DialContext(ctx, "tcp", appAddress(upstreamURL))
conn, err := (&net.Dialer{}).DialContext(ctx, "tcp", appAddress(cfg.UpstreamURL))
if err != nil {
return fmt.Errorf("connect to the app: %w", err)
}
internal/smallwebwaf/healthcheck_test.go
-32
View File
@@ -6,8 +6,6 @@ import (
"net"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
@@ -45,21 +43,6 @@ func TestHealthCheck(t *testing.T) {
wantHealthCheck(t, env, 0, "")
// The health check reads those two settings alone, here given as
// files: a removed or invalid token file, or an invalid value of
// another setting, does not fail it.
for _, other := range []struct{ name, value string }{
{"SWWAF_METRICS_TOKEN_FILE", filepath.Join(t.TempDir(), "removed")},
{"SWWAF_METRICS_TOKEN_FILE", writeFile(t, "too short\n")},
{"SWWAF_MODE", "neither"},
} {
wantHealthCheck(t, map[string]string{
listenAddr + "_FILE": writeFile(t, ":"+port+"\n"),
upstreamURL + "_FILE": writeFile(t, app.URL+"\n"),
other.name: other.value,
}, 0, "")
}
app.Close()
wantHealthCheck(t, env, 1, "unhealthy: connect to the app: ")
@@ -115,18 +98,3 @@ func wantHealthCheck(t *testing.T, env map[string]string, status int, message st
got, wrote, status, message)
}
}
// writeFile writes contents to a file in a directory of its own, removed
// when the test ends, and returns the file's path.
func writeFile(t *testing.T, contents string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "setting")
err := os.WriteFile(path, []byte(contents), 0o600)
if err != nil {
t.Fatalf("write %s: %v", path, err)
}
return path
}
internal/smallwebwaf/smallwebwaf.go
+52 -201
View File
@@ -1,7 +1,6 @@
// Package smallwebwaf runs the smallwebwaf process: it reads the settings,
// the rule files, the lookup database and the state files, serves requests
// until it is told to stop, and then stops in an orderly way, writing the
// state files.
// the rule files and the state files, serves requests until it is told to
// stop, and then stops in an orderly way, writing the state files.
package smallwebwaf
import (
@@ -16,11 +15,9 @@ import (
"syscall"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/remotelog"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/state"
@@ -31,11 +28,6 @@ import (
// runit and docker wait a little longer before they kill the process.
const shutdownTimeout = 5 * time.Second
// remoteLogStopTimeout is how long, as smallwebwaf stops, the log lines
// still waiting are sent to SWWAF_LOG_REMOTE_URL before they are given
// up. stdout has carried them.
const remoteLogStopTimeout = 2 * time.Second
// Params are what Run needs from the process.
type Params struct {
// Version is the version of the binary, set when it is built.
@@ -65,12 +57,11 @@ func Main(version string) int {
})
}
// Run reads the settings, the rule files, the lookup database and the
// state files, then serves requests until ctx is done. It returns the
// process's exit status, 1 when smallwebwaf cannot start.
// Run reads the settings, the rule files and the state files, then serves
// requests until ctx is done. It returns the process's exit status, 1
// when smallwebwaf cannot start.
func Run(ctx context.Context, params Params) int {
processLog := requestlog.NewProcessLogger(params.Stdout,
config.InstanceName(params.LookupEnv))
processLog := requestlog.NewProcessLogger(params.Stdout)
cfg, err := config.FromEnvironment(params.LookupEnv)
if err != nil {
@@ -79,31 +70,10 @@ func Run(ctx context.Context, params Params) int {
return 1
}
// While SWWAF_LOG_REMOTE_URL is set, every line on stdout from here on
// is sent there too.
stdout := params.Stdout
var remote *remotelog.Sender
if cfg.LogRemoteURL != nil {
remote = newRemoteLogSender(cfg)
stdout = io.MultiWriter(params.Stdout, remote)
processLog = requestlog.NewProcessLogger(stdout, cfg.InstanceName)
stopSending := startSending(ctx, remote, processLog)
defer stopSending()
}
// The state files and the alerts give times in UTC.
now := func() time.Time { return time.Now().UTC() }
alertQueue := newAlertQueue(cfg, now, processLog)
ruleFiles, err := rules.Load(rules.Params{
Dir: cfg.RulesDir,
Enabled: cfg.RulesEnabled,
ProcessLog: processLog,
Alerts: alertQueue,
})
if err != nil {
processLog.Error("cannot use the rule files", "error", err.Error())
@@ -111,18 +81,29 @@ func Run(ctx context.Context, params Params) int {
return 1
}
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue)
if err != nil {
processLog.Error("cannot use the lookup database", "error", err.Error())
// The state files give times in UTC.
now := func() time.Time { return time.Now().UTC() }
return 1
}
server := proxy.New(proxy.Params{
Config: cfg,
RequestLog: params.Stdout,
ProcessLog: processLog,
GeoJSURL: lookup.URL,
Now: now,
Rules: ruleFiles,
})
if remote != nil {
server.Metrics.AddRemoteLog(remote)
}
files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
files, err := state.Load(state.Params{
Dir: cfg.StateDir,
WriteDelay: cfg.StateWriteDelay,
CounterInterval: cfg.StateCounterInterval,
Ledger: server.Ledger,
Limiter: server.Limiter,
GeoJS: server.GeoJS,
Now: now,
ProcessLog: processLog,
Metrics: server.Metrics,
})
if err != nil {
processLog.Error("cannot use the state files", "error", err.Error())
@@ -142,139 +123,16 @@ func Run(ctx context.Context, params Params) int {
"address", listener.Addr().String(),
"settings", cfg)
return serve(ctx, server, listener, files, ruleFiles, alertQueue, processLog)
}
// newServer returns the server smallwebwaf runs, with the metrics of the
// alerts, after reading the lookup database while SWWAF_LOOKUP_SOURCE is
// file. A lookup database that cannot be read is an error.
func newServer(
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
) (*proxy.Server, error) {
var lookupFile *lookup.File
if cfg.LookupSource == "file" {
var err error
lookupFile, err = lookup.OpenFile(lookup.FileParams{
Path: cfg.LookupDBPath,
Now: now,
ProcessLog: processLog,
Alerts: alertQueue,
})
if err != nil {
return nil, err
}
}
server := proxy.New(proxy.Params{
Config: cfg,
RequestLog: stdout,
ProcessLog: processLog,
GeoJSURL: lookup.URL,
LookupFile: lookupFile,
Now: now,
Rules: ruleFiles,
Alerts: alertQueue,
})
server.Metrics.AddAlerts(alertQueue)
if lookupFile != nil {
server.Metrics.AddLookupFile(lookupFile.LastRead, lookupFile.ReadFailures)
}
return server, nil
}
// loadStateFiles reads the state files into the parts of server and into
// alertQueue, as state.Load does.
func loadStateFiles(
cfg *config.Config, server *proxy.Server, alertQueue *alerts.Queue,
now func() time.Time, processLog *slog.Logger,
) (*state.Files, error) {
return state.Load(state.Params{
Dir: cfg.StateDir,
WriteDelay: cfg.StateWriteDelay,
CounterInterval: cfg.StateCounterInterval,
Ledger: server.Ledger,
Limiter: server.Limiter,
GeoJS: server.GeoJS,
Alerts: alertQueue,
Anomalies: server.Anomalies,
Now: now,
ProcessLog: processLog,
Metrics: server.Metrics,
})
}
// newAlertQueue returns the queue of the alerts to the webhook, Slack and
// ntfy, with the settings for them.
func newAlertQueue(
cfg *config.Config, now func() time.Time, processLog *slog.Logger,
) *alerts.Queue {
return alerts.New(alerts.Params{
WebhookURL: cfg.AlertWebhookURL,
WebhookHeaders: cfg.AlertWebhookHeaders,
SlackURL: cfg.AlertSlackWebhookURL,
NtfyURL: cfg.AlertNtfyURL,
NtfyToken: cfg.AlertNtfyToken,
Events: cfg.AlertEvents,
Cooldown: cfg.AlertCooldown,
MaxPerHour: cfg.AlertMaxPerHour,
Instance: cfg.InstanceName,
Now: now,
ProcessLog: processLog,
})
}
// newRemoteLogSender returns a sender of the log lines to
// SWWAF_LOG_REMOTE_URL, with the settings for it.
func newRemoteLogSender(cfg *config.Config) *remotelog.Sender {
return remotelog.New(remotelog.Params{
URL: cfg.LogRemoteURL,
RootCAs: cfg.LogRemoteTLSCAs,
Buffer: cfg.LogRemoteBuffer,
Facility: cfg.LogRemoteFacility,
AppName: cfg.LogRemoteAppName,
})
}
// startSending runs remote until the function it returns is called, which
// then waits at most remoteLogStopTimeout for the lines still waiting to
// be sent. Sending goes on after ctx is done, so that the lines written
// while smallwebwaf stops are sent too.
func startSending(
ctx context.Context, remote *remotelog.Sender, processLog *slog.Logger,
) func() {
sending, stop := context.WithCancel(context.WithoutCancel(ctx))
sent := make(chan struct{})
go func() {
remote.Run(sending, processLog)
close(sent)
}()
return func() {
stop()
select {
case <-sent:
case <-time.After(remoteLogStopTimeout):
}
}
return serve(ctx, server.Server, listener, files, ruleFiles, processLog)
}
// serve serves requests on listener, writes the state files as they are
// due, takes in an admin's edits of them, reads the rule files again as
// they change, and the lookup database when it is replaced, and sends the
// alerts, until ctx is done. Then it gives the requests in progress
// shutdownTimeout to finish, and writes every state file, alerts.json with
// the alerts still waiting.
// due, takes in an admin's edits of them, and reads the rule files again
// as they change, until ctx is done. Then it gives the requests in
// progress shutdownTimeout to finish, and writes every state file.
func serve(
ctx context.Context, server *proxy.Server, listener net.Listener,
files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue,
processLog *slog.Logger,
ctx context.Context, server *http.Server, listener net.Listener,
files *state.Files, ruleFiles *rules.Files, processLog *slog.Logger,
) int {
served := make(chan error, 1)
@@ -285,15 +143,24 @@ func serve(
writing, stopWriting := context.WithCancel(ctx)
defer stopWriting()
written := inBackground(func() { files.Run(writing) })
watched := inBackground(func() { files.Watch(writing) })
rulesWatched := inBackground(func() { ruleFiles.Watch(writing) })
lookupFileWatched := inBackground(func() {
if server.LookupFile != nil {
server.LookupFile.Watch(writing)
}
})
alertsSent := inBackground(func() { alertQueue.Run(writing) })
written := make(chan struct{})
watched := make(chan struct{})
rulesWatched := make(chan struct{})
go func() {
files.Run(writing)
close(written)
}()
go func() {
files.Watch(writing)
close(watched)
}()
go func() {
ruleFiles.Watch(writing)
close(rulesWatched)
}()
select {
case err := <-served:
@@ -325,8 +192,7 @@ func serve(
}
// Run and Watch have ended, so nothing else reads or writes the
// files, and no alert is being sent, so that alerts.json keeps every
// alert not yet sent. Every request has ended too, but for two kinds
// files. Every request has ended too, but for two kinds
// that Go's server does not wait for: one cut off because Shutdown
// timed out, and one whose connection switched protocols, such as a
// WebSocket. Such a request adds to its client's history only as it
@@ -335,8 +201,6 @@ func serve(
<-written
<-watched
<-rulesWatched
<-lookupFileWatched
<-alertsSent
err = files.WriteAll()
if err != nil {
@@ -349,16 +213,3 @@ func serve(
return 0
}
// inBackground runs task on a goroutine of its own, and returns a channel
// that is closed once task has returned.
func inBackground(task func()) <-chan struct{} {
done := make(chan struct{})
go func() {
task()
close(done)
}()
return done
}
internal/smallwebwaf/smallwebwaf_internal_test.go
-82
View File
@@ -1,82 +0,0 @@
package smallwebwaf
import (
"log/slog"
"net/url"
"testing"
"testing/synctest"
"time"
"sneak.berlin/go/smallwebwaf/internal/remotelog"
)
// The stop's tests run in a synctest bubble, where the time package runs
// on a clock of the test's own, so that how long the stop takes can be
// told exactly. The sender is held up by its process log, not by the
// network: a goroutine of the bubble that waits on the network keeps that
// clock from moving on.
func TestStopWaitsForTheSenderToFinish(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
took := stopHeldSender(t, time.Second)
if took != time.Second {
t.Errorf("the stop took %s, want the second the sender took", took)
}
})
}
func TestStopWaitsForTheSenderAtMostTwoSeconds(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
took := stopHeldSender(t, time.Minute)
if took != 2*time.Second {
t.Errorf("the stop took %s, want 2s", took)
}
})
}
// heldLog holds each line written to it until it is closed.
type heldLog chan struct{}
// Write waits until the log is closed.
func (l heldLog) Write(p []byte) (int, error) {
<-l
return len(p), nil
}
// stopHeldSender starts sending to an endpoint the sender cannot connect
// to, holds the sender as it logs that failure until release has passed,
// stops the sending, and returns how long the stop took. It returns once
// the sender has ended, as a bubble must.
func stopHeldSender(t *testing.T, release time.Duration) time.Duration {
t.Helper()
log := make(heldLog)
sender := remotelog.New(remotelog.Params{
// No port is 65536, so each attempt to connect fails at once,
// before it reaches the network.
URL: &url.URL{Scheme: remotelog.SchemeTCP, Host: "127.0.0.1:65536"},
Buffer: 1,
})
stopSending := startSending(t.Context(), sender,
slog.New(slog.NewJSONHandler(log, nil)))
synctest.Wait()
time.AfterFunc(release, func() { close(log) })
stopped := time.Now()
stopSending()
took := time.Since(stopped)
time.Sleep(release)
synctest.Wait()
return took
}
internal/smallwebwaf/smallwebwaf_test.go
+17 -831
View File
@@ -10,15 +10,11 @@ import (
"net/http/httptest"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
)
@@ -39,20 +35,8 @@ const (
stateCounterInterval = "SWWAF_STATE_COUNTER_INTERVAL"
rateLimitPerDay = "SWWAF_RATE_LIMIT_PER_DAY"
rulesDir = "SWWAF_RULES_DIR"
lookupSource = "SWWAF_LOOKUP_SOURCE"
lookupDBPath = "SWWAF_LOOKUP_DB_PATH"
instanceName = "SWWAF_INSTANCE_NAME"
adminToken = "SWWAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
metricsToken = "SWWAF_METRICS_TOKEN" //nolint:gosec // the setting's name
// adminSecret is the SWWAF_ADMIN_TOKEN the tests set.
adminSecret = "fedcba9876543210fedcba9876543210"
// instance is the SWWAF_INSTANCE_NAME the tests set where they look at
// it.
instance = "fsn1app1/gitea"
// greeting is what the tests' app answers.
greeting = "hello from the app"
// placed is the client the tests' lookup databases place.
placed = "203.0.113.9"
)
// output collects what smallwebwaf writes on stdout.
@@ -109,16 +93,12 @@ func (o *output) text() string {
}
// run runs smallwebwaf with the settings in env until ctx is done, and
// returns its exit status. SWWAF_LOOKUP_SOURCE is off unless env sets it,
// so that no test sends GeoJS its clients' addresses.
// returns its exit status.
func run(ctx context.Context, env map[string]string, out *output) int {
return smallwebwaf.Run(ctx, smallwebwaf.Params{
Version: testVersion,
LookupEnv: func(name string) (string, bool) {
value, ok := env[name]
if !ok && name == "SWWAF_LOOKUP_SOURCE" {
return "off", true
}
return value, ok
},
@@ -131,10 +111,7 @@ func TestInvalidSettingStopsTheStart(t *testing.T) {
out := &output{}
status := run(t.Context(), map[string]string{
"SWWAF_REQUEST_MAX_BYTES": "lots",
instanceName: instance,
}, out)
status := run(t.Context(), map[string]string{"SWWAF_REQUEST_MAX_BYTES": "lots"}, out)
if status != 1 {
t.Errorf("exit status %d, want 1", status)
}
@@ -142,38 +119,31 @@ func TestInvalidSettingStopsTheStart(t *testing.T) {
line := out.line(t, "msg", "invalid setting")
message, _ := line["error"].(string)
if line["type"] != "process" || line["instance"] != instance ||
line["level"] != "ERROR" ||
if line["type"] != "process" || line["level"] != "ERROR" ||
!strings.HasPrefix(message, "SWWAF_REQUEST_MAX_BYTES: ") {
t.Errorf("start refused with %v", line)
}
}
func TestShortTokenStopsTheStartUnshown(t *testing.T) {
func TestShortMetricsTokenStopsTheStartUnshown(t *testing.T) {
t.Parallel()
const token = "a-token-of-31-characters-at-all" //nolint:gosec // too short to use
for _, name := range []string{adminToken, metricsToken} {
t.Run(name, func(t *testing.T) {
t.Parallel()
out := &output{}
out := &output{}
status := run(t.Context(), map[string]string{"SWWAF_METRICS_TOKEN": token}, out)
if status != 1 {
t.Errorf("exit status %d, want 1", status)
}
status := run(t.Context(), map[string]string{name: token}, out)
if status != 1 {
t.Errorf("exit status %d, want 1", status)
}
line := out.line(t, "msg", "invalid setting")
if line["error"] != "SWWAF_METRICS_TOKEN: is shorter than 32 characters" {
t.Errorf("start refused with %v", line)
}
line := out.line(t, "msg", "invalid setting")
if line["error"] != name+": is shorter than 32 characters" {
t.Errorf("start refused with %v", line)
}
if strings.Contains(out.text(), token) {
t.Errorf("the output shows the token:\n%s", out.text())
}
})
if strings.Contains(out.text(), token) {
t.Errorf("the output shows the token:\n%s", out.text())
}
}
@@ -248,46 +218,6 @@ func TestServesUntilToldToStop(t *testing.T) {
out.line(t, "msg", "stopped")
}
func TestEveryLogLineAndMetricCarriesTheInstanceName(t *testing.T) {
t.Parallel()
const token = "0123456789abcdef0123456789abcdef"
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: t.TempDir(),
rulesDir: t.TempDir(),
metricsToken: token,
instanceName: instance,
}
var metrics string
out := runUntilStopped(t, env, func(url string) {
metrics = metricsText(t, url+"_smallwebwaf/metrics", token)
})
// The process's lines from its start to its stop, and the request's.
for line := range strings.Lines(out.text()) {
var fields map[string]any
err := json.Unmarshal([]byte(line), &fields)
if err != nil || fields["instance"] != instance {
t.Errorf("line %q (%v), want instance %s", line, err, instance)
}
}
// Each series, Go's and the process's included; the other lines are
// the comments.
for line := range strings.Lines(metrics) {
if !strings.HasPrefix(line, "#") &&
!strings.Contains(line, `instance="fsn1app1/gitea"`) {
t.Errorf("series %q, without instance=\"fsn1app1/gitea\"", line)
}
}
}
func TestStateKeptAcrossRestarts(t *testing.T) {
t.Parallel()
@@ -397,51 +327,6 @@ func TestBanAddedAndLiftedByEditingBansJSON(t *testing.T) {
})
}
func TestBanAddedAndLiftedThroughTheEndpointsKeptInBansJSON(t *testing.T) {
t.Parallel()
dir := t.TempDir()
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: dir,
rulesDir: t.TempDir(),
trustedProxies: localhost + "/32",
adminToken: adminSecret,
// Neither comes due in the test: the files are written as
// smallwebwaf stops.
stateWriteDelay: "1h",
stateCounterInterval: "1h",
}
runUntilStopped(t, env, func(url string) {
askAsAdmin(t, http.MethodPost, url+"_smallwebwaf/bans",
`{"netblock": "203.0.113.0/24", "duration": "permanent", `+
`"reason": "probes for logins"}`)
wantStatus(t, url, "203.0.113.9", http.StatusForbidden)
})
ban := onlyBan(t, dir)
if ban["netblock"] != "203.0.113.0/24" || ban["cause"] != "admin" ||
ban["reason"] != "probes for logins" || ban["expires"] != nil ||
ban["lifted"] != nil {
t.Errorf("bans.json holds %v, want the admin's permanent ban", ban)
}
// After a restart the ban still refuses; once lifted, it refuses no
// more, and bans.json keeps it, marked lifted.
runUntilStopped(t, env, func(url string) {
wantStatus(t, url, "203.0.113.9", http.StatusForbidden)
askAsAdmin(t, http.MethodDelete, url+"_smallwebwaf/bans/203.0.113.9", "")
wantStatus(t, url, "203.0.113.9", http.StatusOK)
})
ban = onlyBan(t, dir)
if ban["netblock"] != "203.0.113.0/24" || ban["lifted"] == nil {
t.Errorf("bans.json holds %v, want the admin's ban, lifted", ban)
}
}
func TestRuleFileAddedWhileRunningTakesEffect(t *testing.T) {
t.Parallel()
@@ -458,21 +343,8 @@ func TestRuleFileAddedWhileRunningTakesEffect(t *testing.T) {
out := runUntilStopped(t, env, func(url string) {
wantGreeting(t, url)
// Written once: each change would start the rule files' wait
// again. A file written before smallwebwaf watches the directory is
// read once it does.
err := os.WriteFile(filepath.Join(dir, "50-app.rules"),
[]byte("everything path block ^/\n"), 0o600)
if err != nil {
t.Fatalf("write the rule file: %v", err)
}
// As long as that takes, so that a slow test process cannot fail
// the test.
for statusFrom(t, url, "203.0.113.9") != http.StatusForbidden {
time.Sleep(pollInterval)
}
saveUntilAnswered(t, filepath.Join(dir, "50-app.rules"),
"everything path block ^/\n", url, "203.0.113.9", http.StatusForbidden)
})
out.line(t, "action", "rule_blocked")
}
@@ -500,379 +372,6 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
": no such file or directory")
}
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
t.Parallel()
const token = "0123456789abcdef0123456789abcdef"
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
writeLookupDatabase(t, path, "DE")
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: t.TempDir(),
rulesDir: t.TempDir(),
trustedProxies: localhost + "/32",
metricsToken: token,
instanceName: instance,
lookupSource: "file",
lookupDBPath: path,
"SWWAF_DENIED_COUNTRIES": "kp",
// The requests sent until a replacement takes effect, and those
// for the metrics, must not break a rate limit, whose ban would
// refuse them too.
"SWWAF_RATE_LIMIT_EXEMPT_NETS": placed + "," + localhost,
}
began := time.Now()
// Each replacement is written beside the file and renamed over it, as
// a refresh is.
replacement := path + ".new"
var lastRead float64
runUntilStopped(t, env, func(url string) {
wantStatus(t, url, placed, http.StatusOK)
// smallwebwaf reads the file again once it watches its directory,
// which may be after the first replacement. Once that has taken
// effect, only the watch can show the next. Each takes as long as
// it takes, so that a slow test process cannot fail the test.
writeLookupDatabase(t, replacement, "KP")
rename(t, replacement, path)
for statusFrom(t, url, placed) != http.StatusForbidden {
time.Sleep(pollInterval)
}
writeLookupDatabase(t, replacement, "DE")
rename(t, replacement, path)
for statusFrom(t, url, placed) != http.StatusOK {
time.Sleep(pollInterval)
}
// One that cannot be read leaves the file read before in use.
err := os.WriteFile(replacement, []byte("not a lookup database\n"), 0o600)
if err != nil {
t.Fatalf("write %s: %v", replacement, err)
}
rename(t, replacement, path)
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token,
`smallwebwaf_lookup_database_read_failures_total{instance="fsn1app1/gitea"} 1`)
lastRead = seriesValue(t, metrics,
`smallwebwaf_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/gitea"}`)
wantStatus(t, url, placed, http.StatusOK)
})
if lastRead < float64(began.Unix()) || lastRead > float64(time.Now().Unix()) {
t.Errorf("the lookup database was last read at %v, want a time in the test", lastRead)
}
}
func TestLookupDatabaseThatCannotBeReadStopsTheStart(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
// If it starts instead, it is stopped after waitLimit.
ctx, stop := context.WithTimeout(t.Context(), waitLimit)
defer stop()
out := &output{}
status := run(ctx, map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
lookupSource: "file", lookupDBPath: path,
}, out)
if status != 1 {
t.Fatalf("exit status %d, want 1", status)
}
want := "SWWAF_LOOKUP_DB_PATH cannot be read: open " + path +
": no such file or directory"
line := out.line(t, "msg", "cannot use the lookup database")
if line["error"] != want {
t.Errorf("start refused with %v, want the error %q", line, want)
}
}
func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) {
t.Parallel()
endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
if err != nil {
t.Fatalf("listen: %v", err)
}
defer func() {
_ = endpoint.Close()
}()
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: t.TempDir(),
rulesDir: t.TempDir(),
"SWWAF_LOG_REMOTE_URL": "syslog+tcp://" + endpoint.Addr().String(),
}
out := runUntilStopped(t, env, func(url string) {
wantGreeting(t, url)
})
out.line(t, "type", "request")
// smallwebwaf connected as it started, and closes the connection once
// it has sent the lines written as it stopped.
conn, err := endpoint.Accept()
if err != nil {
t.Fatalf("accept: %v", err)
}
received, err := io.ReadAll(conn)
_ = conn.Close()
if err != nil {
t.Fatalf("read: %v", err)
}
// Lines written at once by several goroutines may reach stdout and
// the endpoint in different orders.
sent := messages(t, string(received))
written := slices.Collect(strings.Lines(out.text()))
slices.Sort(sent)
slices.Sort(written)
if !slices.Equal(sent, written) {
t.Errorf("sent\n%v\nwrote\n%v", sent, written)
}
}
func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
t.Parallel()
const token = "0123456789abcdef0123456789abcdef"
// The endpoint takes connections and never answers, so the TLS
// handshake of each waits on it, and no line is ever sent.
endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
if err != nil {
t.Fatalf("listen: %v", err)
}
defer func() {
_ = endpoint.Close()
}()
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: t.TempDir(),
rulesDir: t.TempDir(),
"SWWAF_LOG_REMOTE_URL": "syslog+tls://" + endpoint.Addr().String(),
"SWWAF_LOG_REMOTE_BUFFER": "1",
metricsToken: token,
instanceName: instance,
}
out := runUntilStopped(t, env, func(url string) {
wantGreeting(t, url)
// More than one line has been written, and the buffer holds the
// last.
metrics := metricsText(t, url+"_smallwebwaf/metrics", token)
for _, series := range []string{
`smallwebwaf_remote_log_lines_sent_total{instance="fsn1app1/gitea"} 0`,
`smallwebwaf_remote_log_buffer_depth{instance="fsn1app1/gitea"} 1`,
} {
if !strings.Contains(metrics, "\n"+series+"\n") {
t.Errorf("no %q in the metrics:\n%s", series, metrics)
}
}
const dropped = "\nsmallwebwaf_remote_log_lines_dropped_total" +
`{instance="fsn1app1/gitea"} `
if strings.Contains(metrics, dropped+"0\n") || !strings.Contains(metrics, dropped) {
t.Errorf("no line dropped in the metrics:\n%s", metrics)
}
// Closed, the endpoint refuses the connection made to send the
// lines still waiting at the stop, which then does not wait.
_ = endpoint.Close()
})
out.line(t, "type", "request")
// While the lines are sent, the process's lines give the instance name
// too.
line := out.line(t, "msg", "starting")
if line["instance"] != instance {
t.Errorf("start logged with instance %v, want %s", line["instance"], instance)
}
}
func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
t.Parallel()
webhook := startWebhook(t)
rules := t.TempDir()
err := os.WriteFile(filepath.Join(rules, "50-app.rules"),
[]byte(`probe path ban ^/\.env$`+"\n"), 0o600)
if err != nil {
t.Fatalf("write the rule file: %v", err)
}
dir := t.TempDir()
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: dir,
rulesDir: rules,
"SWWAF_ALERT_WEBHOOK_URL": webhook.url,
"SWWAF_ALERT_WEBHOOK_HEADERS": "Authorization:Bearer " + adminSecret,
instanceName: instance,
}
runUntilStopped(t, env, func(url string) {
// The probe bans the client, and the webhook is sent the alert.
wantRefused(t, url+".env")
post := webhook.waitFor(t, "ban", true)
if post.alert["client"] != localhost || post.alert["netblock"] != localhost+"/32" ||
post.authorization != "Bearer "+adminSecret {
t.Errorf("the webhook was sent %v, with Authorization %q", post.alert,
post.authorization)
}
// The webhook fails, so the alert for the ban made permanent by the
// client's next request waits.
webhook.failing.Store(true)
wantRefused(t, url)
webhook.waitFor(t, "permanent_ban", false)
})
// alerts.json keeps it as smallwebwaf stops, and once started again,
// smallwebwaf sends it.
var file struct {
Waiting map[string][]struct {
Event string `json:"event"`
} `json:"waiting"`
}
path := filepath.Join(dir, "alerts.json")
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
if err == nil {
err = json.Unmarshal(data, &file)
}
waiting := file.Waiting["webhook"]
if err != nil || len(waiting) != 1 || waiting[0].Event != "permanent_ban" {
t.Fatalf("alerts.json holds %s (%v), want the permanent_ban alert waiting for "+
"the webhook", data, err)
}
// It counts the alert sent in the metrics, read here from a client the
// ban does not cover.
const token = "0123456789abcdef0123456789abcdef"
webhook.failing.Store(false)
env["SWWAF_ALLOW_NETS"] = localhost
env[metricsToken] = token
runUntilStopped(t, env, func(url string) {
webhook.waitFor(t, "permanent_ban", true)
const ofWebhook = `{destination="webhook",instance="fsn1app1/gitea"}`
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token,
"\nsmallwebwaf_alerts_sent_total"+ofWebhook+" 1\n")
for _, series := range []string{"failed", "suppressed", "dropped"} {
zero := "\nsmallwebwaf_alerts_" + series + "_total" + ofWebhook + " 0\n"
if !strings.Contains(metrics, zero) {
t.Errorf("no %q in the metrics:\n%s", zero, metrics)
}
}
})
}
func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
t.Parallel()
const (
ntfyToken = "tk_0123456789abcdefghijklmnopq"
token = "abcdef0123456789abcdef0123456789"
client = "203.0.113.9"
)
slack, ntfy := startDestination(t), startDestination(t)
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: t.TempDir(),
rulesDir: t.TempDir(),
trustedProxies: localhost + "/32",
rateLimitPerDay: "1",
// The metrics are read from 127.0.0.1, which no limit counts.
"SWWAF_ALLOW_NETS": localhost + "/32",
metricsToken: token,
instanceName: instance,
"SWWAF_ALERT_SLACK_WEBHOOK_URL": slack.url,
"SWWAF_ALERT_NTFY_URL": ntfy.url,
"SWWAF_ALERT_NTFY_TOKEN": ntfyToken,
}
runUntilStopped(t, env, func(url string) {
// The client's second request breaks the day limit, and bans it;
// Slack and ntfy are each sent the alert.
wantStatus(t, url, client, http.StatusOK)
wantStatus(t, url, client, http.StatusForbidden)
var message struct {
Text string `json:"text"`
}
slackPost := slack.firstPost(t)
err := json.Unmarshal([]byte(slackPost.body), &message)
if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/gitea: ban*\n") ||
!strings.Contains(message.Text, "\nclient: "+client+"\n") {
t.Errorf("Slack was sent %s", slackPost.body)
}
ntfyPost := ntfy.firstPost(t)
if ntfyPost.header.Get("Title") != "fsn1app1/gitea: ban" ||
ntfyPost.header.Get("Authorization") != "Bearer "+ntfyToken ||
!strings.Contains(ntfyPost.body, "\nclient: "+client+"\n") {
t.Errorf("ntfy was sent %s, with the headers %v", ntfyPost.body,
ntfyPost.header)
}
// The metrics count it for each, and give no series for the
// webhook, which is not set. As long as that takes, so that a slow
// test process cannot fail the test.
sent := []string{
"\nsmallwebwaf_alerts_sent_total{destination=\"slack\"," +
"instance=\"fsn1app1/gitea\"} 1\n",
"\nsmallwebwaf_alerts_sent_total{destination=\"ntfy\"," +
"instance=\"fsn1app1/gitea\"} 1\n",
}
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token, sent...)
if strings.Contains(metrics, `destination="webhook"`) {
t.Errorf("the metrics give the webhook:\n%s", metrics)
}
})
}
func TestStateFileThatDoesNotParseStopsTheStart(t *testing.T) {
t.Parallel()
@@ -1019,7 +518,6 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
"SWWAF_RATE_LIMIT_PER_MINUTE": "1000",
"SWWAF_RATE_LIMIT_PER_HOUR": "10000",
rateLimitPerDay: "50000",
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",
"SWWAF_DENIED_COUNTRIES": "",
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
"SWWAF_BAN_RESPONSE": "403",
@@ -1069,184 +567,6 @@ func wantGreeting(t *testing.T, url string) {
}
}
// messages returns the message of each record in received, octet-counted
// frames of RFC 5424 records with the default facility and app name, each
// with the newline that ends a line on stdout.
func messages(t *testing.T, received string) []string {
t.Helper()
hostname, _ := os.Hostname()
header := " " + hostname + " " + hostname + " - - - "
var found []string
for received != "" {
count, rest, _ := strings.Cut(received, " ")
length, err := strconv.Atoi(count)
if err != nil || length > len(rest) {
t.Fatalf("no frame at %q", received)
}
record := rest[:length]
received = rest[length:]
_, message, ok := strings.Cut(record, header)
if !ok || !strings.HasPrefix(record, "<134>1 ") {
t.Fatalf("record %q, want priority <134> and header %q", record, header)
}
found = append(found, message+"\n")
}
return found
}
// metricsText asks for the metrics at url with token, and returns them.
func metricsText(t *testing.T, url, token string) string {
t.Helper()
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, url,
http.NoBody)
if err != nil {
t.Fatalf("new request: %v", err)
}
req.Header.Set("Authorization", "Bearer "+token)
transport := &http.Transport{}
defer transport.CloseIdleConnections()
res, err := (&http.Client{Transport: transport}).Do(req)
if err != nil {
t.Fatalf("request: %v", err)
}
body, err := io.ReadAll(res.Body)
_ = res.Body.Close()
if err != nil || res.StatusCode != http.StatusOK {
t.Fatalf("metrics answered %d (%v)", res.StatusCode, err)
}
return string(body)
}
// metricsWith asks for the metrics at url with token until they hold each
// of series, as long as that takes, so that a slow test process cannot
// fail the test, and returns them.
func metricsWith(t *testing.T, url, token string, series ...string) string {
t.Helper()
for {
metrics := metricsText(t, url, token)
missing := slices.ContainsFunc(series, func(one string) bool {
return !strings.Contains(metrics, one)
})
if !missing {
return metrics
}
time.Sleep(pollInterval)
}
}
// seriesValue returns the value of series, such as
// name{instance="app"}, in metrics.
func seriesValue(t *testing.T, metrics, series string) float64 {
t.Helper()
for line := range strings.Lines(metrics) {
value, found := strings.CutPrefix(strings.TrimSpace(line), series+" ")
if !found {
continue
}
number, err := strconv.ParseFloat(value, 64)
if err != nil {
t.Fatalf("%s has the value %q: %v", series, value, err)
}
return number
}
t.Fatalf("no %s in the metrics:\n%s", series, metrics)
return 0
}
// rename renames the file at from to, replacing any file there.
func rename(t *testing.T, from, to string) {
t.Helper()
err := os.Rename(from, to)
if err != nil {
t.Fatalf("rename %s to %s: %v", from, to, err)
}
}
// writeLookupDatabase writes a lookup database at path that places the
// client placed in country, and no other address.
func writeLookupDatabase(t *testing.T, path, country string) {
t.Helper()
lookuptest.Write(t, path, map[string]lookuptest.Network{
placed + "/32": {ASN: "AS64496", ASName: "Example Net", Country: country},
})
}
// askAsAdmin sends a request with method to url, with body and
// adminSecret, and checks that it is answered 200.
func askAsAdmin(t *testing.T, method, url, body string) {
t.Helper()
req, err := http.NewRequestWithContext(t.Context(), method, url,
strings.NewReader(body))
if err != nil {
t.Fatalf("new request: %v", err)
}
req.Header.Set("Authorization", "Bearer "+adminSecret)
transport := &http.Transport{}
defer transport.CloseIdleConnections()
res, err := (&http.Client{Transport: transport}).Do(req)
if err != nil {
t.Fatalf("request: %v", err)
}
_ = res.Body.Close()
if res.StatusCode != http.StatusOK {
t.Fatalf("%s %s answered %d", method, url, res.StatusCode)
}
}
// onlyBan returns the one ban bans.json in dir holds.
func onlyBan(t *testing.T, dir string) map[string]any {
t.Helper()
path := filepath.Join(dir, "bans.json")
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
if err != nil {
t.Fatalf("read bans.json: %v", err)
}
var file struct {
Bans []map[string]any `json:"bans"`
}
err = json.Unmarshal(data, &file)
if err != nil || len(file.Bans) != 1 {
t.Fatalf("bans.json holds\n%s\nwant one ban (%v)", data, err)
}
return file.Bans[0]
}
// wantRefused checks that a request to url is refused with 403, the
// default SWWAF_BAN_RESPONSE.
func wantRefused(t *testing.T, url string) {
@@ -1307,140 +627,6 @@ func saveUntilAnswered(t *testing.T, path, content, url, from string, status int
}
}
// destination is a stand-in for SWWAF_ALERT_SLACK_WEBHOOK_URL or
// SWWAF_ALERT_NTFY_URL. It notes each request it is sent, and answers
// 200.
type destination struct {
url string
mu sync.Mutex
posts []destinationPost
}
// destinationPost is a request a destination was sent: its headers and
// its body.
type destinationPost struct {
header http.Header
body string
}
// startDestination starts a destination that takes every alert.
func startDestination(t *testing.T) *destination {
t.Helper()
d := &destination{}
server := httptest.NewServer(http.HandlerFunc(
func(_ http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
d.mu.Lock()
d.posts = append(d.posts, destinationPost{
header: r.Header.Clone(), body: string(body),
})
d.mu.Unlock()
}))
t.Cleanup(server.Close)
d.url = server.URL + "/alerts"
return d
}
// firstPost waits until the destination has been sent a request, and
// returns the first. It waits as long as that takes, so that a slow test
// process cannot fail the test.
func (d *destination) firstPost(t *testing.T) destinationPost {
t.Helper()
for {
d.mu.Lock()
if len(d.posts) > 0 {
post := d.posts[0]
d.mu.Unlock()
return post
}
d.mu.Unlock()
time.Sleep(pollInterval)
}
}
// webhook is a stand-in for SWWAF_ALERT_WEBHOOK_URL. It notes each alert
// it is sent, and answers 204, or 503 while failing.
type webhook struct {
url string
failing atomic.Bool
mu sync.Mutex
posts []webhookPost
}
// webhookPost is an alert the webhook was sent, with the Authorization
// header sent with it, and whether the webhook took it.
type webhookPost struct {
alert map[string]any
authorization string
answered bool
}
// startWebhook starts a webhook that takes every alert.
func startWebhook(t *testing.T) *webhook {
t.Helper()
w := &webhook{}
server := httptest.NewServer(http.HandlerFunc(
func(rw http.ResponseWriter, r *http.Request) {
var alert map[string]any
_ = json.NewDecoder(r.Body).Decode(&alert)
failing := w.failing.Load()
w.mu.Lock()
w.posts = append(w.posts, webhookPost{
alert: alert, authorization: r.Header.Get("Authorization"),
answered: !failing,
})
w.mu.Unlock()
if failing {
rw.WriteHeader(http.StatusServiceUnavailable)
return
}
rw.WriteHeader(http.StatusNoContent)
}))
t.Cleanup(server.Close)
w.url = server.URL + "/alerts"
return w
}
// waitFor waits until the webhook has been sent an alert for event that
// it took, or, unless answered, failed, and returns it. It waits as long
// as that takes, so that a slow test process cannot fail the test.
func (w *webhook) waitFor(t *testing.T, event string, answered bool) webhookPost {
t.Helper()
for {
w.mu.Lock()
for _, post := range w.posts {
if post.alert["event"] == event && post.answered == answered {
w.mu.Unlock()
return post
}
}
w.mu.Unlock()
time.Sleep(pollInterval)
}
}
// statusFrom returns the status a request to url from the client at
// from, as X-Forwarded-For names it, is answered with.
func statusFrom(t *testing.T, url, from string) int {
internal/state/state.go
+58 -267
View File
@@ -1,14 +1,11 @@
// Package state keeps smallwebwaf's state in JSON files in
// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md describes:
// bans.json holds the bans, clients.json each client's counters and
// history, lookups.json GeoJS's answers, and alerts.json the cooldowns,
// the hour under way, the alerts waiting for each destination and the
// anomaly counters. Load
// reads them at start, Watch takes in an admin's edit of one while
// smallwebwaf runs, and Run and WriteAll write them. The disk is read and
// written outside the parts' locks, which are held only to take a
// snapshot or to put in what a file holds, so that no request waits on
// the disk.
// history, and lookups.json GeoJS's answers. Load reads them at start,
// Watch takes in an admin's edit of one while smallwebwaf runs, and Run
// and WriteAll write them. The disk is read and written outside the
// parts' locks, which are held only to take a snapshot or to put in what
// a file holds, so that no request waits on the disk.
package state
import (
@@ -20,17 +17,13 @@ import (
"fmt"
"io/fs"
"log/slog"
"maps"
"net/netip"
"os"
"path/filepath"
"slices"
"sync"
"time"
"github.com/fsnotify/fsnotify"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
@@ -49,19 +42,13 @@ const (
bansJSON = "bans.json"
clientsJSON = "clients.json"
lookupsJSON = "lookups.json"
alertsJSON = "alerts.json"
)
var (
errVersion = errors.New("unknown version")
// errMissing is for an entry without a field it needs.
errMissing = errors.New("has no")
errCause = errors.New("is not limit, attack or admin")
errDestination = errors.New("is not webhook, slack or ntfy")
errScope = errors.New("is not client, net, asn, total or watch")
errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
`or remove the file`)
errMissing = errors.New("has no")
errCause = errors.New("is not limit or attack")
)
// Params are what Load needs.
@@ -73,14 +60,10 @@ type Params struct {
// is (SWWAF_STATE_COUNTER_INTERVAL).
WriteDelay time.Duration
CounterInterval time.Duration
// Ledger, Limiter, GeoJS, Alerts and Anomalies hold the state. Alerts
// also receive a file_error alert for an edit set aside, and for a
// write that fails while smallwebwaf runs.
Ledger *bans.Ledger
Limiter *ratelimit.Limiter
GeoJS *lookup.GeoJS
Alerts *alerts.Queue
Anomalies *anomaly.Counters
// Ledger, Limiter and GeoJS hold the state.
Ledger *bans.Ledger
Limiter *ratelimit.Limiter
GeoJS *lookup.GeoJS
// Now tells the time by which the counters' buckets run out, normally
// time.Now in UTC.
Now func() time.Time
@@ -109,20 +92,16 @@ type Files struct {
// bansFile is bans.json, indented for an admin to read and edit.
type bansFile struct {
Version int `json:"version"`
Bans []BanEntry `json:"bans"`
Bans []banEntry `json:"bans"`
}
// BanEntry is a ban as bans.json holds it: a permanent ban's expires is
// null, a ban an admin added may have no cause, which makes it an
// admin's, and lifted is left out until an admin lifts the ban. The ban
// endpoints answer with bans in this form too.
type BanEntry struct {
// banEntry is a ban as bans.json holds it: a permanent ban's expires is
// null, and a ban an admin added may have no cause.
type banEntry struct {
Netblock netip.Prefix `json:"netblock"`
Start time.Time `json:"start"`
Expires *time.Time `json:"expires"`
Cause string `json:"cause"`
Reason string `json:"reason,omitempty"`
Lifted *time.Time `json:"lifted,omitempty"`
Cause string `json:"cause,omitempty"`
Notes bans.Notes `json:"notes"`
}
@@ -138,17 +117,6 @@ type lookupsFile struct {
Lookups []lookup.Answer `json:"lookups"`
}
// alertsFile is alerts.json, indented for an admin to read and edit.
//
//nolint:tagliatelle // the state files use snake_case, as the request log does
type alertsFile struct {
Version int `json:"version"`
Cooldowns []alerts.Cooldown `json:"cooldowns"`
Hour alerts.Hour `json:"hour"`
Waiting map[string][]alerts.Alert `json:"waiting"`
AnomalyCounters []anomaly.Counter `json:"anomaly_counters"`
}
// stateFile is the struct of a state file. Once the file is decoded, its
// check refuses the first entry without a field it needs, which would
// otherwise be read as something the entry does not say. data is the
@@ -175,25 +143,23 @@ func Load(params Params) (*Files, error) {
bansRead, bansErr := f.read(bansJSON)
clientsRead, clientsErr := f.read(clientsJSON)
lookupsRead, lookupsErr := f.read(lookupsJSON)
alertsRead, alertsErr := f.read(alertsJSON)
err = errors.Join(bansErr, clientsErr, lookupsErr, alertsErr)
err = errors.Join(bansErr, clientsErr, lookupsErr)
if err != nil {
return nil, err
}
params.ProcessLog.Info("read the state files", "directory", params.Dir,
"bans", bansRead, "clients", clientsRead, "lookups", lookupsRead,
"alerts_waiting", alertsRead)
"bans", bansRead, "clients", clientsRead, "lookups", lookupsRead)
return f, nil
}
// Run writes bans.json WriteDelay after a ban is made, with every ban
// made in between, and every file every CounterInterval, until ctx is
// done. A write that fails is logged, raised as a file_error alert, and
// the file is written again at its next write. Each write takes in an
// admin's edit of its file first, as writeFile describes.
// done. A write that fails is logged, and the file is written again at
// its next write. Each write takes in an admin's edit of its file first,
// as writeFile describes.
func (f *Files) Run(ctx context.Context) {
interval := time.NewTicker(f.params.CounterInterval)
defer interval.Stop()
@@ -211,11 +177,9 @@ func (f *Files) Run(ctx context.Context) {
case <-bansDue:
bansDue = nil
f.logFailure(bansJSON, f.writeFile(bansJSON))
f.logFailure(f.writeFile(bansJSON))
case <-interval.C:
for _, name := range []string{bansJSON, clientsJSON, lookupsJSON, alertsJSON} {
f.logFailure(name, f.writeFile(name))
}
f.logFailure(f.WriteAll())
}
}
}
@@ -224,7 +188,7 @@ func (f *Files) Run(ctx context.Context) {
// fails does not keep the others from being written.
func (f *Files) WriteAll() error {
return errors.Join(f.writeFile(bansJSON), f.writeFile(clientsJSON),
f.writeFile(lookupsJSON), f.writeFile(alertsJSON))
f.writeFile(lookupsJSON))
}
// Watch watches Dir until ctx is done, and takes in an admin's edit of a
@@ -259,7 +223,7 @@ func (f *Files) Watch(ctx context.Context) {
return
case event := <-watcher.Events:
switch name := filepath.Base(event.Name); name {
case bansJSON, clientsJSON, lookupsJSON, alertsJSON:
case bansJSON, clientsJSON, lookupsJSON:
f.fileChanged(name)
}
case err = <-watcher.Errors:
@@ -269,22 +233,11 @@ func (f *Files) Watch(ctx context.Context) {
}
}
// logFailure logs a write of the state file name that failed, and raises
// a file_error alert for it.
func (f *Files) logFailure(name string, err error) {
// logFailure logs a write that failed.
func (f *Files) logFailure(err error) {
if err != nil {
const failed = "writing the state files failed"
// Raised before it is logged, so that the alert is there once the
// log line is.
f.params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError,
Reason: failed,
Detail: map[string]any{
"file": filepath.Join(f.params.Dir, name), "error": err.Error(),
},
})
f.params.ProcessLog.Error(failed, "error", err.Error())
f.params.ProcessLog.Error("writing the state files failed",
"error", err.Error())
}
}
@@ -309,7 +262,7 @@ func (f *Files) fileChanged(name string) {
// runs, by Watch or by a write, is taken in here. An edit that does not
// parse is neither counted nor logged, and takeIn's error returned.
func (f *Files) takeInEdit(name string, data []byte) error {
_, err := f.takeIn(name, data, true)
_, err := f.takeIn(name, data)
if err != nil {
return err
}
@@ -331,7 +284,7 @@ func (f *Files) read(name string) (int, error) {
return 0, err
}
return f.takeIn(name, data, false)
return f.takeIn(name, data)
}
// readChanged returns what the state file name holds, and whether that
@@ -355,11 +308,9 @@ func (f *Files) readChanged(name string) ([]byte, bool, error) {
// takeIn parses data, what the state file name holds, puts it into the
// part that keeps that state, in place of what the part held, and returns
// how many entries the file holds. edit is whether data is an admin's
// edit taken in while smallwebwaf runs, rather than the file read at the
// start. An error names the file and, where the JSON decoder tells it,
// the line and column, or else the entry.
func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
// how many entries the file holds. An error names the file and, where the
// JSON decoder tells it, the line and column, or else the entry.
func (f *Files) takeIn(name string, data []byte) (int, error) {
path := filepath.Join(f.params.Dir, name)
var entries int
@@ -378,12 +329,7 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
held = append(held, entry.ban())
}
if edit {
f.params.Ledger.LoadEdit(held)
} else {
f.params.Ledger.Load(held)
}
f.params.Ledger.Load(held)
entries = len(held)
case clientsJSON:
var file clientsFile
@@ -405,33 +351,6 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
f.params.GeoJS.Load(file.Lookups)
entries = len(file.Lookups)
case alertsJSON:
// waiting was a list, of the alerts waiting for the webhook, before
// alerts went to Slack and ntfy too.
var written struct {
Waiting json.RawMessage `json:"waiting"`
}
if json.Unmarshal(data, &written) == nil &&
bytes.HasPrefix(written.Waiting, []byte("[")) {
return 0, fmt.Errorf("%s: %w", path, errWaitingList)
}
var file alertsFile
err := parse(path, data, &file)
if err != nil {
return 0, err
}
f.params.Alerts.Load(alerts.State{
Cooldowns: file.Cooldowns, Hour: file.Hour, Waiting: file.Waiting,
})
f.params.Anomalies.Load(file.AnomalyCounters, f.params.Now())
for _, waiting := range file.Waiting {
entries += len(waiting)
}
}
f.sums[name] = sha256.Sum256(data)
@@ -483,9 +402,8 @@ func (f *Files) writeFile(name string) error {
// setAside renames the state file name, an edit that does not parse with
// parseErr, to name.bad, for the admin to mend, and logs it with where in
// the file the error is, and raises a file_error alert for it. If the
// rename fails, the edit is left as it is, and the error returned is
// parseErr joined with the rename's.
// the file the error is. If the rename fails, the edit is left as it is,
// and the error returned is parseErr joined with the rename's.
func (f *Files) setAside(name string, parseErr error) error {
path := filepath.Join(f.params.Dir, name)
@@ -494,16 +412,8 @@ func (f *Files) setAside(name string, parseErr error) error {
return errors.Join(parseErr, err)
}
const setAside = "set aside an edit of a state file that does not parse"
// Raised before it is logged, so that the alert is there once the log
// line is.
f.params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError,
Reason: setAside,
Detail: map[string]any{"file": path + ".bad", "error": parseErr.Error()},
})
f.params.ProcessLog.Error(setAside, "file", path+".bad", "error", parseErr.Error())
f.params.ProcessLog.Error("set aside an edit of a state file that does not parse",
"file", path+".bad", "error", parseErr.Error())
f.params.Metrics.StateFileEditSetAside(name)
return nil
@@ -514,7 +424,12 @@ func (f *Files) setAside(name string, parseErr error) error {
func (f *Files) encode(name string) ([]byte, error) {
switch name {
case bansJSON:
file := bansFile{Version: version, Bans: BanEntries(f.params.Ledger.Snapshot())}
held := f.params.Ledger.Snapshot()
file := bansFile{Version: version, Bans: make([]banEntry, 0, len(held))}
for _, ban := range held {
file.Bans = append(file.Bans, newBanEntry(ban))
}
data, err := json.MarshalIndent(file, "", " ")
if err != nil {
@@ -524,66 +439,30 @@ func (f *Files) encode(name string) ([]byte, error) {
return append(data, '\n'), nil
case clientsJSON:
return encodeOnePerLine("clients", f.params.Limiter.Snapshot())
case lookupsJSON:
default: // lookups.json
return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot())
default: // alerts.json
held := f.params.Alerts.Snapshot()
file := alertsFile{
Version: version, Cooldowns: held.Cooldowns, Hour: held.Hour,
Waiting: held.Waiting, AnomalyCounters: f.params.Anomalies.Snapshot(),
}
data, err := json.MarshalIndent(file, "", " ")
if err != nil {
return nil, err
}
return append(data, '\n'), nil
}
}
// BanEntries returns held as bans.json lists them, an empty list for
// none.
func BanEntries(held []bans.Ban) []BanEntry {
entries := make([]BanEntry, 0, len(held))
for _, ban := range held {
entries = append(entries, newBanEntry(ban))
}
return entries
}
// newBanEntry returns ban as bans.json holds it.
func newBanEntry(ban bans.Ban) BanEntry {
entry := BanEntry{
Netblock: ban.Netblock, Start: ban.Start, Cause: ban.Cause, Reason: ban.Reason,
Notes: ban.Notes,
func newBanEntry(ban bans.Ban) banEntry {
entry := banEntry{
Netblock: ban.Netblock, Start: ban.Start, Cause: ban.Cause, Notes: ban.Notes,
}
if !ban.Permanent() {
entry.Expires = &ban.Expires
}
if !ban.Lifted.IsZero() {
entry.Lifted = &ban.Lifted
}
return entry
}
// ban returns the ban an entry of bans.json holds.
func (e BanEntry) ban() bans.Ban {
ban := bans.Ban{
Netblock: e.Netblock, Start: e.Start, Cause: e.Cause, Reason: e.Reason,
Notes: e.Notes,
}
func (e banEntry) ban() bans.Ban {
ban := bans.Ban{Netblock: e.Netblock, Start: e.Start, Cause: e.Cause, Notes: e.Notes}
if e.Expires != nil {
ban.Expires = *e.Expires
}
if e.Lifted != nil {
ban.Lifted = *e.Lifted
}
return ban
}
@@ -591,8 +470,8 @@ func (e BanEntry) ban() bans.Ban {
// client, a start, from which the length of the netblock's next ban is
// worked out, or an expires, which would make it permanent. A permanent
// ban's expires is null, which Bans cannot tell from a missing one, so
// each expires is read again as written. A cause other than limit,
// attack or admin, most likely misspelt, is refused too.
// each expires is read again as written. A cause other than limit or
// attack, most likely misspelt, is refused too.
func (f *bansFile) check(data []byte) error {
var written struct {
Bans []struct {
@@ -614,7 +493,7 @@ func (f *bansFile) check(data []byte) error {
case written.Bans[i].Expires == nil:
return missing(i, "expires")
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin:
entry.Cause != bans.CauseAttack:
return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause)
}
}
@@ -623,8 +502,8 @@ func (f *bansFile) check(data []byte) error {
}
// check refuses a client without its address, which would count nobody's
// requests, or with requests or bytes in a window but no start, which
// would drop them and give the client a fresh allowance.
// requests, or with requests in a window but no start, which would drop
// them and give the client a fresh allowance.
func (f *clientsFile) check([]byte) error {
for i, client := range f.Clients {
switch {
@@ -636,12 +515,6 @@ func (f *clientsFile) check([]byte) error {
return missing(i, "hour.start")
case countsWithoutStart(client.Day):
return missing(i, "day.start")
case countsWithoutStart(client.MinuteBytes):
return missing(i, "minute_bytes.start")
case countsWithoutStart(client.HourBytes):
return missing(i, "hour_bytes.start")
case countsWithoutStart(client.DayBytes):
return missing(i, "day_bytes.start")
}
}
@@ -679,90 +552,8 @@ func (f *lookupsFile) check(data []byte) error {
return nil
}
// check refuses a cooldown without its event or when its alert was sent,
// which would hold back no repeat, alerts waiting for a destination with
// another name than webhook, slack or ntfy, most likely misspelt, an
// alert waiting without its event or its time, and an anomaly counter as
// checkAnomalyCounters does.
func (f *alertsFile) check([]byte) error {
for i, cooldown := range f.Cooldowns {
switch {
case cooldown.Event == "":
return fmt.Errorf("cooldowns %w", missing(i, "event"))
case cooldown.Sent.IsZero():
return fmt.Errorf("cooldowns %w", missing(i, "sent"))
}
}
for _, destination := range slices.Sorted(maps.Keys(f.Waiting)) {
if !slices.Contains(alerts.Destinations(), destination) {
return fmt.Errorf("waiting %q %w", destination, errDestination)
}
for i, alert := range f.Waiting[destination] {
switch {
case alert.Event == "":
return fmt.Errorf("waiting %s %w", destination, missing(i, "event"))
case alert.Time.IsZero():
return fmt.Errorf("waiting %s %w", destination, missing(i, "time"))
}
}
}
return checkAnomalyCounters(f.AnomalyCounters)
}
// checkAnomalyCounters refuses an anomaly counter whose scope is not
// client, net, asn, total or watch, most likely misspelt, and one without
// a field it needs, as missingFromCounter tells.
func checkAnomalyCounters(counters []anomaly.Counter) error {
for i, counter := range counters {
if !slices.Contains(anomaly.Scopes(), counter.Scope) {
return fmt.Errorf("anomaly_counters entry %d's scope %q %w", i+1,
counter.Scope, errScope)
}
field := missingFromCounter(counter)
if field != "" {
return fmt.Errorf("anomaly_counters %w", missing(i, field))
}
}
return nil
}
// missingFromCounter returns the first field counter, an anomaly counter,
// needs and has not, or "" when it has them all: what tells it from the
// others in its scope, without which it would never be counted again, the
// netblock of a client, net or watch counter, the AS number of an asn one
// and the name of a watch one; and the start of a window in which it has
// requests or bytes, without which they would be dropped.
func missingFromCounter(counter anomaly.Counter) string {
scope := counter.Scope
switch {
case scope != anomaly.ScopeASN && scope != anomaly.ScopeTotal &&
!counter.Netblock.IsValid():
return "netblock"
case scope == anomaly.ScopeASN && counter.ASN == "":
return "asn"
case scope == anomaly.ScopeWatch && counter.Name == "":
return "name"
case countsWithoutStart(counter.Minute):
return "minute.start"
case countsWithoutStart(counter.Hour):
return "hour.start"
case countsWithoutStart(counter.MinuteBytes):
return "minute_bytes.start"
case countsWithoutStart(counter.HourBytes):
return "hour_bytes.start"
default:
return ""
}
}
// countsWithoutStart reports whether b holds requests, or bytes, but no
// start, which places them in time.
// countsWithoutStart reports whether b holds requests but no start, which
// places them in time.
func countsWithoutStart(b ratelimit.Buckets) bool {
return b.Start.IsZero() && (b.Current != 0 || b.Previous != 0)
}
internal/state/state_test.go
+35 -671
View File
@@ -10,10 +10,8 @@ import (
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"os"
"path/filepath"
"reflect"
"slices"
"strconv"
"strings"
@@ -21,8 +19,6 @@ import (
"testing/synctest"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
@@ -35,10 +31,6 @@ const (
bansJSON = "bans.json"
clientsJSON = "clients.json"
lookupsJSON = "lookups.json"
alertsJSON = "alerts.json"
// The AS number and AS name the tests' clients are looked up in.
asn = "AS64496"
asName = "Example Net"
// What the process log says once Watch watches the directory, and as
// it takes in an edit.
watching = "watching the state files for edits"
@@ -46,9 +38,6 @@ const (
// maxLogLines is how many lines of the process log wait for a test to
// read them.
maxLogLines = 64
// whole is the percentage of each limit a client gets when nothing
// lowers its limits.
whole = 100
)
// permanentBansJSON is bans.json holding permanentBan.
@@ -59,11 +48,7 @@ const permanentBansJSON = `{
"netblock": "2001:db8::/64",
"start": "2026-10-06T00:00:00Z",
"expires": null,
"cause": "admin",
"reason": "scrapes every commit",
"notes": {
"asn": "AS64496",
"as_name": "Example Net",
"country": "DE",
"limit": 1000,
"window": "minute",
@@ -78,126 +63,7 @@ const permanentBansJSON = `{
},
"requests": 1500,
"refused": 3,
"earlier_bans": {
"limit": 3,
"attack": 1,
"admin": 1
}
}
}
]
}
`
// liftedClient is the client whose ban liftedBansJSON holds.
const liftedClient = "203.0.113.9"
// liftedBansJSON is bans.json holding an hour's ban for a broken limit on
// liftedClient, from midnight, that an admin lifted ten minutes in.
const liftedBansJSON = `{"version": 1, "bans": [{"netblock": "203.0.113.9/32", ` +
`"start": "2026-10-06T00:00:00Z", "expires": "2026-10-06T01:00:00Z", ` +
`"cause": "limit", "lifted": "2026-10-06T00:10:00Z"}]}`
// filledAlertsJSON is alerts.json holding the alerts of fill.
const filledAlertsJSON = `{
"version": 1,
"cooldowns": [
{
"event": "file_error",
"netblock": "",
"file": "/var/lib/smallwebwaf/bans.json",
"sent": "2026-10-06T00:00:00Z",
"suppressed_repeats": 0
},
{
"event": "ban",
"netblock": "203.0.113.9/32",
"sent": "2026-10-06T00:00:00Z",
"suppressed_repeats": 1
}
],
"hour": {
"start": "2026-10-06T00:00:00Z",
"sent": 2,
"held_back": {
"source_failure": 1
}
},
"waiting": {
"webhook": [
{
"instance": "fsn1app1/gitea",
"time": "2026-10-06T00:00:00Z",
"event": "ban",
"client": "203.0.113.9",
"netblock": "203.0.113.9/32",
"asn": "",
"as_name": "",
"country": "DE",
"reason": "requests per minute over the limit of 1",
"detail": {
"cause": "limit"
},
"suppressed_repeats": 0
},
{
"instance": "fsn1app1/gitea",
"time": "2026-10-06T00:00:00Z",
"event": "file_error",
"client": "",
"netblock": "",
"asn": "",
"as_name": "",
"country": "",
"reason": "writing the state files failed",
"detail": {
"error": "no space left on device",
"file": "/var/lib/smallwebwaf/bans.json"
},
"suppressed_repeats": 0
}
]
},
"anomaly_counters": [
{
"scope": "asn",
"asn": "AS64496",
"hour_bytes": {
"start": "2026-10-06T00:00:00Z",
"current": 8,
"previous": 0
}
},
{
"scope": "net",
"netblock": "203.0.113.0/24",
"minute": {
"start": "2026-10-06T00:00:00Z",
"current": 1,
"previous": 0
}
},
{
"scope": "total",
"minute": {
"start": "2026-10-06T00:00:00Z",
"current": 1,
"previous": 0
},
"minute_bytes": {
"start": "2026-10-06T00:00:00Z",
"current": 8,
"previous": 0
}
},
{
"scope": "watch",
"netblock": "203.0.113.0/24",
"name": "office",
"hour": {
"start": "2026-10-06T00:00:00Z",
"current": 1,
"previous": 0
"earlier_bans": 5
}
}
]
@@ -230,114 +96,13 @@ func TestFilesWrittenAndReadBack(t *testing.T) {
wantEqual(t, clientsJSON, after.Limiter.Snapshot(), before.Limiter.Snapshot())
wantEqual(t, lookupsJSON, after.GeoJS.Snapshot(), before.GeoJS.Snapshot())
if got, want := after.Alerts.Snapshot(), before.Alerts.Snapshot(); !reflect.DeepEqual(
got, want) {
t.Errorf("%s read back\n%+v\nwant\n%+v", alertsJSON, got, want)
}
wantEqual(t, alertsJSON, after.Anomalies.Snapshot(), before.Anomalies.Snapshot())
// Each one-per-line file lists its entries by client, and nothing
// but the four files is left in the directory.
// but the three files is left in the directory.
wantEntries(t, filepath.Join(dir, clientsJSON), "clients",
"192.0.2.1/32", "203.0.113.9/32", "2001:db8::/64")
wantEntries(t, filepath.Join(dir, lookupsJSON), "lookups",
"192.0.2.1/32", "203.0.113.9/32")
wantFiles(t, dir, alertsJSON, bansJSON, clientsJSON, lookupsJSON)
}
func TestAlertsJSONIsIndentedWithTheCooldownsTheHourAndTheAlertsWaiting(t *testing.T) {
t.Parallel()
dir := t.TempDir()
params := newParams(dir)
fill(params)
files := load(t, params)
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
got := readFile(t, filepath.Join(dir, alertsJSON))
if got != filledAlertsJSON {
t.Errorf("alerts.json\n%s\nwant\n%s", got, filledAlertsJSON)
}
}
func TestSourceFailureCooldownKeptInAlertsJSONAcrossARestart(t *testing.T) {
t.Parallel()
dir := t.TempDir()
before := newParams(dir)
files := load(t, before)
failure := alerts.Alert{
Event: alerts.EventSourceFailure, Reason: "asking GeoJS failed",
Detail: map[string]any{"source": "geojs"},
}
before.Alerts.Raise(failure)
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
// After the restart, the cooldown read back holds back a repeat for the
// same source.
after := newParams(dir)
load(t, after)
after.Alerts.Raise(failure)
waiting := after.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || after.Alerts.Suppressed() != 1 {
t.Errorf("%d alerts wait and %d are held back, want the one read back and 1",
len(waiting), after.Alerts.Suppressed())
}
}
func TestAnomalyCountersKeptInAlertsJSONAcrossARestart(t *testing.T) {
t.Parallel()
dir := t.TempDir()
request := anomaly.Request{
Client: netip.MustParseAddr("203.0.113.9"),
ClientGroup: netip.MustParsePrefix("203.0.113.9/32"),
}
// The whole service may have two requests a minute.
withThreshold := func() state.Params {
params := newParams(dir)
params.Anomalies = anomaly.New(anomaly.Params{
Total: anomaly.Thresholds{RequestsPerMinute: 2}, Alerts: params.Alerts,
})
return params
}
before := withThreshold()
files := load(t, before)
for range 2 {
before.Anomalies.Count(midnight(), request)
}
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
// After the restart, the third request in the minute is over it.
after := withThreshold()
load(t, after)
after.Anomalies.Count(midnight(), request)
waiting := after.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Event != alerts.EventAnomaly ||
waiting[0].Detail["count"] != float64(3) {
t.Errorf("alerts wait %+v, want one for 3 requests", waiting)
}
wantFiles(t, dir, bansJSON, clientsJSON, lookupsJSON)
}
func TestBansJSONIsIndentedWithNullForAPermanentBan(t *testing.T) {
@@ -366,10 +131,8 @@ func TestMissingFilesAreEmptyState(t *testing.T) {
params := newParams(t.TempDir())
load(t, params)
held := params.Alerts.Snapshot()
if len(params.Ledger.Snapshot()) != 0 || len(params.Limiter.Snapshot()) != 0 ||
len(params.GeoJS.Snapshot()) != 0 || len(held.Cooldowns) != 0 ||
len(held.Waiting[alerts.DestinationWebhook]) != 0 || held.Hour.Sent != 0 {
len(params.GeoJS.Snapshot()) != 0 {
t.Error("state from no files")
}
}
@@ -411,23 +174,6 @@ func TestFileThatDoesNotParseStopsTheStart(t *testing.T) {
`{"version": 1, "bans": [{"netblock": "203.0.113.300/32"}]}`,
`: netip.ParsePrefix("203.0.113.300/32")`,
},
{
"an unknown field of an alert waiting", alertsJSON,
`{"version": 1, "waiting": {"webhook": [{"event": "ban", "evnet": "ban"}]}}`,
`: json: unknown field "evnet"`,
},
{
"alerts waiting for an unknown destination", alertsJSON,
`{"version": 1, "waiting": {"webhook": [], "slak": []}}`,
`: waiting "slak" is not webhook, slack or ntfy`,
},
{
"an anomaly counter of an unknown scope", alertsJSON,
`{"version": 1, "anomaly_counters": [{"scope": "total"}, ` +
`{"scope": "nett", "netblock": "203.0.113.0/24"}]}`,
`: anomaly_counters entry 2's scope "nett" is not client, net, asn, total ` +
`or watch`,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
@@ -491,12 +237,6 @@ func TestEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
`"hour": {"current": 3}}]}`,
`: entry 1 has no "hour.start"`,
},
{
"a client with bytes in a window without its start", clientsJSON,
`{"version": 1, "clients": [{"client": "203.0.113.9/32", ` +
`"minute_bytes": {"previous": 5120}}]}`,
`: entry 1 has no "minute_bytes.start"`,
},
{
"an answer without a client", lookupsJSON,
`{"version": 1, "lookups": [{"country": "DE", ` + answer + `}]}`,
@@ -523,110 +263,21 @@ func TestEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
}
}
func TestAlertsJSONEntryWithoutAFieldItNeedsStopsTheStart(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name, content string
// want is what the error says after the file's path.
want string
}{
{
"a cooldown without its event",
`{"version": 1, "cooldowns": [{"sent": "2026-10-06T00:00:00Z"}]}`,
`: cooldowns entry 1 has no "event"`,
},
{
"a cooldown without when it was sent",
`{"version": 1, "cooldowns": [{"event": "ban"}]}`,
`: cooldowns entry 1 has no "sent"`,
},
{
"an alert waiting without its event",
`{"version": 1, "waiting": {"ntfy": [{"time": "2026-10-06T00:00:00Z"}]}}`,
`: waiting ntfy entry 1 has no "event"`,
},
{
"an alert waiting without its time",
`{"version": 1, "waiting": {"slack": [{"event": "ban"}]}}`,
`: waiting slack entry 1 has no "time"`,
},
{
// The whole service's counter needs nothing to tell it apart.
"an anomaly counter of a netblock without it",
`{"version": 1, "anomaly_counters": [{"scope": "total"}, {"scope": "net"}]}`,
`: anomaly_counters entry 2 has no "netblock"`,
},
{
"an anomaly counter of a client without its netblock",
`{"version": 1, "anomaly_counters": [{"scope": "client"}]}`,
`: anomaly_counters entry 1 has no "netblock"`,
},
{
"an anomaly counter of an AS number without it",
`{"version": 1, "anomaly_counters": [{"scope": "asn"}]}`,
`: anomaly_counters entry 1 has no "asn"`,
},
{
"an anomaly counter of a named netblock without its name",
`{"version": 1, "anomaly_counters": [` +
`{"scope": "watch", "netblock": "203.0.113.0/24"}]}`,
`: anomaly_counters entry 1 has no "name"`,
},
{
"an anomaly counter of a named netblock without its netblock",
`{"version": 1, "anomaly_counters": [{"scope": "watch", "name": "office"}]}`,
`: anomaly_counters entry 1 has no "netblock"`,
},
{
"an anomaly counter with bytes in a window without its start",
`{"version": 1, "anomaly_counters": [` +
`{"scope": "total", "hour_bytes": {"current": 5}}]}`,
`: anomaly_counters entry 1 has no "hour_bytes.start"`,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
wantRefused(t, alertsJSON, tc.content, tc.want)
})
}
}
func TestAlertsJSONWithWaitingAsAListStopsTheStartSayingWhatToChange(t *testing.T) {
t.Parallel()
// alerts.json as it was written before alerts went to Slack and ntfy too,
// with no alert waiting, or one.
for _, waiting := range []string{
`[]`,
`[{"event": "ban", "time": "2026-10-06T00:00:00Z"}]`,
} {
wantRefused(t, alertsJSON, `{"version": 1, "cooldowns": [], `+
`"hour": {"start": "2026-10-06T00:00:00Z", "sent": 0, "held_back": {}}, `+
`"waiting": `+waiting+`}`,
`: waiting is a list, but now lists the alerts by destination: put the `+
`list under "webhook", as "waiting": {"webhook": [...]}, or remove the file`)
}
}
func TestBanWithAnotherCauseStopsTheStart(t *testing.T) {
func TestBanWithACauseSmallwebwafDoesNotGiveStopsTheStart(t *testing.T) {
t.Parallel()
wantRefused(t, bansJSON, `{"version": 1, "bans": [`+
`{"netblock": "203.0.113.9/32", "start": "2026-10-06T00:00:00Z", `+
`"expires": null, "cause": "attack"}, `+
`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
`"expires": null, "cause": "admin"}, `+
`{"netblock": "203.0.113.11/32", "start": "2026-10-06T00:00:00Z", `+
`"expires": null, "cause": "atack"}]}`,
`: entry 3's cause "atack" is not limit, attack or admin`)
`: entry 2's cause "atack" is not limit or attack`)
}
func TestUnknownVersionStopsTheStart(t *testing.T) {
t.Parallel()
for _, file := range []string{bansJSON, clientsJSON, lookupsJSON, alertsJSON} {
for _, file := range []string{bansJSON, clientsJSON, lookupsJSON} {
for _, content := range []string{`{"version": 2}`, `{}`} {
t.Run(file+" "+content, func(t *testing.T) {
t.Parallel()
@@ -676,12 +327,12 @@ func TestBansWrittenOnceWriteDelayAfterABan(t *testing.T) {
// A second ban, made while the first waits to be written, puts the
// write off no further, and is written with it.
first, _ := params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"),
first := params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"),
midnight(), bans.Notes{})
time.Sleep(5 * time.Second)
second, _ := params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.10/32"),
second := params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.10/32"),
midnight(), bans.Notes{})
time.Sleep(5*time.Second - time.Nanosecond)
@@ -728,8 +379,8 @@ func TestEveryFileWrittenEveryCounterInterval(t *testing.T) {
time.Sleep(time.Nanosecond)
synctest.Wait()
wantFiles(t, dir, alertsJSON, bansJSON, clientsJSON, lookupsJSON)
removeFiles(t, dir, alertsJSON, bansJSON, clientsJSON, lookupsJSON)
wantFiles(t, dir, bansJSON, clientsJSON, lookupsJSON)
removeFiles(t, dir, bansJSON, clientsJSON, lookupsJSON)
}
})
}
@@ -772,57 +423,6 @@ func TestEditJustBeforeAScheduledWriteSurvivesIt(t *testing.T) {
})
}
func TestWriteThatFailsWhileRunningRaisesAFileErrorAlertOncePerCooldown(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
dir := t.TempDir()
params := newParams(dir)
params.CounterInterval = time.Minute
run(t, load(t, params).Run)
// A directory in the way of clients.json's temporary file fails each
// of its writes, while the other files are written. It holds a file,
// so that the write cannot remove it.
err := os.Mkdir(filepath.Join(dir, clientsJSON+".tmp"), 0o700)
if err == nil {
err = os.WriteFile(filepath.Join(dir, clientsJSON+".tmp", "kept"), nil, 0o600)
}
if err != nil {
t.Fatalf("put a directory in the way: %v", err)
}
time.Sleep(time.Minute)
synctest.Wait()
waiting := params.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 {
t.Fatalf("%d alerts wait, want 1", len(waiting))
}
message, _ := waiting[0].Detail["error"].(string)
if waiting[0].Event != alerts.EventFileError ||
waiting[0].Reason != "writing the state files failed" ||
waiting[0].Detail["file"] != filepath.Join(dir, clientsJSON) ||
!strings.Contains(message, clientsJSON+".tmp") {
t.Fatalf("alerts waiting %+v, want a file_error alert for clients.json, "+
"naming its temporary file", waiting)
}
// The next write fails too, within the cooldown, which holds it back.
time.Sleep(time.Minute)
synctest.Wait()
waiting = params.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || params.Alerts.Suppressed() != 1 {
t.Errorf("%d alerts wait and %d are held back, want 1 and 1",
len(waiting), params.Alerts.Suppressed())
}
})
}
func TestFailedWriteLeavesTheFileAsItWas(t *testing.T) {
t.Parallel()
@@ -845,7 +445,7 @@ func TestFailedWriteLeavesTheFileAsItWas(t *testing.T) {
params.Ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(),
bans.Notes{})
params.Limiter.Count(netip.MustParsePrefix("203.0.113.9/32"), midnight(), whole)
params.Limiter.Count(netip.MustParsePrefix("203.0.113.9/32"), midnight())
err = files.WriteAll()
if err == nil || !strings.Contains(err.Error(), bansJSON+".tmp") {
@@ -879,8 +479,8 @@ func TestWritesAreCountedInTheMetrics(t *testing.T) {
}
const (
ofBans = `{file="bans.json",instance="app"}`
ofClients = `{file="clients.json",instance="app"}`
ofBans = `{file="bans.json"}`
ofClients = `{file="clients.json"}`
)
got := scrape(t, params)
@@ -948,7 +548,7 @@ func TestFileThatCannotBeReadIsNotWrittenOver(t *testing.T) {
t.Errorf("bans.json is now %v (%v), want the socket", info, err)
}
wantFiles(t, dir, alertsJSON, bansJSON, clientsJSON, lookupsJSON)
wantFiles(t, dir, bansJSON, clientsJSON, lookupsJSON)
wantWriteFailed(t, params, bansJSON)
}
@@ -1007,7 +607,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
`"start": "2026-10-06T00:00:00Z", "expires": null}]}`)
wantTakenIn(t, lines, dir, bansJSON)
wantEqual(t, bansJSON, params.Ledger.Snapshot(),
[]bans.Ban{{Netblock: client, Start: midnight(), Cause: bans.CauseAdmin}})
[]bans.Ban{{Netblock: client, Start: midnight()}})
edit(t, dir, clientsJSON, `{"version": 1, "clients": [`+
`{"client": "198.51.100.7/32", "history": {"requests": 7}}]}`)
@@ -1020,27 +620,6 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
wantTakenIn(t, lines, dir, lookupsJSON)
wantEqual(t, lookupsJSON, params.GeoJS.Snapshot(),
[]lookup.Answer{{Client: client, Country: "FR", Answered: midnight()}})
// A netblock with bits past its length is read as the netblock it is
// in.
edit(t, dir, alertsJSON, `{"version": 1, "cooldowns": [{"event": "ban", `+
`"netblock": "198.51.100.9/24", "sent": "2026-10-06T00:00:00Z"}], `+
`"waiting": {"webhook": [{"event": "file_error", "time": "2026-10-06T00:00:00Z"}]}}`)
wantTakenIn(t, lines, dir, alertsJSON)
want := alerts.State{
Cooldowns: []alerts.Cooldown{{
Event: alerts.EventBan, Netblock: netip.MustParsePrefix("198.51.100.0/24"),
Sent: midnight(),
}},
Hour: alerts.Hour{HeldBack: map[string]int{}},
Waiting: map[string][]alerts.Alert{
alerts.DestinationWebhook: {{Event: alerts.EventFileError, Time: midnight()}},
},
}
if got := params.Alerts.Snapshot(); !reflect.DeepEqual(got, want) {
t.Errorf("%s taken in as\n%+v\nwant\n%+v", alertsJSON, got, want)
}
}
func TestOwnWritesAreNotTakenIn(t *testing.T) {
@@ -1112,7 +691,7 @@ func TestBanAddedAndLiftedThroughBansJSON(t *testing.T) {
`"start": "2026-10-06T00:00:00Z", "expires": null}]}`)
wantTakenIn(t, lines, dir, bansJSON)
_, banned, _ := params.Ledger.Check(client, midnight())
_, banned := params.Ledger.Check(client, midnight())
if !banned {
t.Error("the ban added to bans.json does not refuse")
}
@@ -1121,106 +700,12 @@ func TestBanAddedAndLiftedThroughBansJSON(t *testing.T) {
edit(t, dir, bansJSON, `{"version": 1, "bans": []}`)
wantTakenIn(t, lines, dir, bansJSON)
_, banned, _ = params.Ledger.Check(client, midnight())
_, banned = params.Ledger.Check(client, midnight())
if banned {
t.Error("the ban removed from bans.json still refuses")
}
}
func TestBanWithoutACauseTakenInAsAnAdmins(t *testing.T) {
t.Parallel()
const reason = "probes for logins"
// adminsBansJSON is bans.json as an admin writes it, with a ban on
// netblock without a cause, and adminsBans the bans it holds.
adminsBansJSON := func(netblock string) string {
return `{"version": 1, "bans": [{"netblock": "` + netblock + `", ` +
`"start": "2026-10-06T00:00:00Z", "expires": null, "reason": "` +
reason + `"}]}`
}
adminsBans := func(netblock string) []bans.Ban {
return []bans.Ban{{
Netblock: netip.MustParsePrefix(netblock),
Start: midnight(),
Cause: bans.CauseAdmin,
Reason: reason,
}}
}
// Read at the start, the ban is taken in as an admin's, though not
// counted among the bans made since the start, and written back with
// that cause and the admin's reason.
dir := t.TempDir()
edit(t, dir, bansJSON, adminsBansJSON("203.0.113.0/24"))
params := newParams(dir)
lines := logInto(&params)
files := load(t, params)
wantEqual(t, bansJSON, params.Ledger.Snapshot(), adminsBans("203.0.113.0/24"))
wantMadeByAnAdmin(t, params.Ledger, 0)
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
var written struct {
Bans []struct {
Cause string `json:"cause"`
Reason string `json:"reason"`
} `json:"bans"`
}
err = json.Unmarshal([]byte(readFile(t, filepath.Join(dir, bansJSON))), &written)
if err != nil || len(written.Bans) != 1 || written.Bans[0].Cause != bans.CauseAdmin ||
written.Bans[0].Reason != reason {
t.Errorf("bans.json holds %+v (%v), want the ban with the cause admin "+
"and the reason %q", written, err, reason)
}
// Taken in while smallwebwaf runs, a ban on another netblock is an
// admin's too, and one made since the start.
watch(t, files, lines)
edit(t, dir, bansJSON, adminsBansJSON("198.51.100.0/24"))
wantTakenIn(t, lines, dir, bansJSON)
wantEqual(t, bansJSON, params.Ledger.Snapshot(), adminsBans("198.51.100.0/24"))
wantMadeByAnAdmin(t, params.Ledger, 1)
}
func TestLiftedBanReadAtTheStart(t *testing.T) {
t.Parallel()
dir := t.TempDir()
edit(t, dir, bansJSON, liftedBansJSON)
params := newParams(dir)
wantLiftedBanKept(t, load(t, params), dir, params.Ledger)
}
func TestBanLiftedByAnEditWhileRunning(t *testing.T) {
t.Parallel()
dir := t.TempDir()
params := newParams(dir)
lines := logInto(&params)
files := load(t, params)
watch(t, files, lines)
// The ban that liftedBansJSON lifts, before it is lifted.
netblock := netip.MustParsePrefix(liftedClient + "/32")
params.Ledger.BanForLimit(netblock, midnight(), bans.Notes{})
_, banned, _ := params.Ledger.Find(netblock.Addr(), afterLifting())
if !banned {
t.Fatal("the ban does not refuse before it is lifted")
}
edit(t, dir, bansJSON, liftedBansJSON)
wantTakenIn(t, lines, dir, bansJSON)
wantLiftedBanKept(t, files, dir, params.Ledger)
}
func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) {
t.Parallel()
@@ -1248,7 +733,7 @@ func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) {
edit(t, dir, bansJSON, broken)
edit(t, dir, clientsJSON, `{"version": 1, "clients": []}`)
wantTakenIn(t, lines, dir, clientsJSON)
wantFiles(t, dir, alertsJSON, bansJSON, clientsJSON, lookupsJSON)
wantFiles(t, dir, bansJSON, clientsJSON, lookupsJSON)
// The next write sets it aside, logged with where the error is, and
// writes bans.json again from what smallwebwaf still holds.
@@ -1265,14 +750,7 @@ func TestBrokenEditSetAsideAtTheNextWrite(t *testing.T) {
t.Errorf("set aside with %v", line)
}
// It is raised as a file_error alert, with the same file and error.
waiting := params.Alerts.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError ||
waiting[0].Detail["file"] != path+".bad" || waiting[0].Detail["error"] != message {
t.Errorf("alerts waiting %+v, want a file_error alert for %s", waiting, path+".bad")
}
wantFiles(t, dir, alertsJSON, bansJSON, bansJSON+".bad", clientsJSON, lookupsJSON)
wantFiles(t, dir, bansJSON, bansJSON+".bad", clientsJSON, lookupsJSON)
if got := readFile(t, path+".bad"); got != broken {
t.Errorf("bans.json.bad holds\n%s\nwant the edit", got)
@@ -1305,7 +783,7 @@ func TestEditsTakenInAreCountedInTheMetrics(t *testing.T) {
wantTakenIn(t, lines, dir, bansJSON)
wantMetric(t, scrape(t, params),
`smallwebwaf_state_file_edits_taken_in_total{file="bans.json",instance="app"}`, 2)
`smallwebwaf_state_file_edits_taken_in_total{file="bans.json"}`, 2)
}
func TestEditTakenInByAWriteIsLoggedAsWatchLogsIt(t *testing.T) {
@@ -1370,7 +848,7 @@ func TestEditsSetAsideAreCountedInTheMetrics(t *testing.T) {
}
wantMetric(t, scrape(t, params),
`smallwebwaf_state_file_edits_set_aside_total{file="bans.json",instance="app"}`, 1)
`smallwebwaf_state_file_edits_set_aside_total{file="bans.json"}`, 1)
}
func TestDirectoryThatCannotBeWatchedIsLogged(t *testing.T) {
@@ -1401,20 +879,10 @@ func midnight() time.Time {
}
// newParams returns Params for the state files in dir, with parts that
// hold nothing yet. GeoJS is never asked, and the alerts, at most two an
// hour, are never sent. The anomaly counters count the scopes fill
// counts, with thresholds fill does not reach.
// hold nothing yet. GeoJS is never asked.
func newParams(dir string) state.Params {
discard := slog.New(slog.DiscardHandler)
m := metrics.New(1, "app")
queue := alerts.New(alerts.Params{
WebhookURL: &url.URL{Scheme: "https", Host: "alerts.example"},
Events: alerts.Events(),
Cooldown: 15 * time.Minute,
MaxPerHour: 2,
Instance: "fsn1app1/gitea",
Now: midnight,
})
m := metrics.New(1)
return state.Params{
Dir: dir,
@@ -1431,86 +899,39 @@ func newParams(dir string) state.Params {
GeoJS: lookup.New(lookup.Params{
Now: midnight, ProcessLog: discard, Metrics: m,
}),
Alerts: queue,
Anomalies: anomaly.New(anomaly.Params{
Net: anomaly.Thresholds{RequestsPerMinute: 1000},
ASN: anomaly.Thresholds{BytesPerHour: 1 << 30},
Total: anomaly.Thresholds{RequestsPerMinute: 1000, BytesPerMinute: 1 << 30},
Watch: anomaly.Thresholds{RequestsPerHour: 1000},
NetV4Prefix: 24,
NetV6Prefix: 48,
NamedNetblocks: []anomaly.NamedNetblock{{Name: "office", Netblock: office()}},
Alerts: queue,
}),
Now: midnight,
ProcessLog: discard,
Metrics: m,
}
}
// office is the named netblock of the anomaly counters of newParams.
func office() netip.Prefix {
return netip.MustParsePrefix("203.0.113.0/24")
}
// fill puts a permanent ban an admin made, a ban for a broken limit and
// one for a clear sign of attack, clients with counts and histories,
// GeoJS answers, and alerts and anomaly counters, as filledAlertsJSON
// holds them, into the parts of params.
// fill puts a permanent ban without a cause, as an admin adds one, a ban
// for a broken limit and one for a clear sign of attack, clients with
// counts and histories, and GeoJS answers into the parts of params.
func fill(params state.Params) {
now := midnight()
client := netip.MustParsePrefix("203.0.113.9/32")
params.Ledger.Load([]bans.Ban{permanentBan()})
params.Ledger.BanForLimit(client, now, bans.Notes{
ASN: asn, ASName: asName, Country: "DE", Limit: 1,
})
params.Ledger.BanForLimit(client, now, bans.Notes{Country: "DE", Limit: 1})
params.Ledger.BanForAttack(netip.MustParsePrefix("192.0.2.1/32"), now,
bans.Notes{RuleID: "env-file", Target: "path"})
for _, c := range []string{"2001:db8::/64", "203.0.113.9/32", "192.0.2.1/32"} {
params.Limiter.Count(netip.MustParsePrefix(c), now, whole)
params.Limiter.Count(netip.MustParsePrefix(c), now)
}
params.Limiter.CountBytes(client, now, 8, whole)
params.Limiter.AddToHistory(client, now, ratelimit.Request{
Forwarded: true, Status: 200, RequestBytes: 3, ResponseBytes: 5,
Country: "DE", Forwarded: true, Status: 200, RequestBytes: 3, ResponseBytes: 5,
})
params.Limiter.AddLookup(client, now.Add(-time.Hour), asn, asName, "DE")
params.GeoJS.Load([]lookup.Answer{
{
Client: client, ASN: asn, ASName: asName, Country: "DE",
Answered: now.Add(-time.Hour), Used: now,
},
{Client: client, Country: "DE", Answered: now.Add(-time.Hour), Used: now},
{
Client: netip.MustParsePrefix("192.0.2.1/32"),
Answered: now.Add(-time.Hour), Used: now.Add(-time.Minute),
},
})
// An alert waiting, a repeat of it the cooldown holds back, another
// alert waiting, and one past the two an hour, for the hour's summary.
ban := alerts.Alert{
Event: alerts.EventBan, Client: client.Addr(), Netblock: client, Country: "DE",
Reason: "requests per minute over the limit of 1",
Detail: map[string]any{"cause": "limit"},
}
params.Alerts.Raise(ban)
params.Alerts.Raise(ban)
params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError, Reason: "writing the state files failed",
Detail: map[string]any{
"file": "/var/lib/smallwebwaf/bans.json", "error": "no space left on device",
},
})
params.Alerts.Raise(alerts.Alert{
Event: alerts.EventSourceFailure, Reason: "asking GeoJS failed",
})
params.Anomalies.Count(now, anomaly.Request{
Client: client.Addr(), ClientGroup: client, ASN: asn, Bytes: 8,
})
}
// permanentBan is the ban permanentBansJSON holds.
@@ -1518,11 +939,7 @@ func permanentBan() bans.Ban {
return bans.Ban{
Netblock: netip.MustParsePrefix("2001:db8::/64"),
Start: midnight(),
Cause: bans.CauseAdmin,
Reason: "scrapes every commit",
Notes: bans.Notes{
ASN: asn,
ASName: asName,
Country: "DE",
Limit: 1000,
Window: "minute",
@@ -1537,64 +954,11 @@ func permanentBan() bans.Ban {
},
Requests: 1500,
Refused: 3,
EarlierBans: bans.EarlierBans{Limit: 3, Attack: 1, Admin: 1},
EarlierBans: 5,
},
}
}
// afterLifting is a time after the ban liftedBansJSON holds was lifted,
// while it would still last.
func afterLifting() time.Time {
return midnight().Add(30 * time.Minute)
}
// wantLiftedBanKept checks that ledger holds the ban liftedBansJSON holds,
// which refuses nothing and does not make the next ban for a broken limit
// longer, and that files write it to bans.json, in dir, still lifted.
func wantLiftedBanKept(
t *testing.T, files *state.Files, dir string, ledger *bans.Ledger,
) {
t.Helper()
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
const lifted = `"lifted": "2026-10-06T00:10:00Z"`
if got := readFile(t, filepath.Join(dir, bansJSON)); !strings.Contains(got, lifted) {
t.Errorf("bans.json holds\n%s\nwant the ban with %s", got, lifted)
}
netblock := netip.MustParsePrefix(liftedClient + "/32")
_, banned, _ := ledger.Check(netblock.Addr(), afterLifting())
if banned {
t.Error("the lifted ban refuses")
}
// Were the lifted ban counted, the next would last three hours.
ban, _ := ledger.BanForLimit(netblock, afterLifting(), bans.Notes{})
if ban.Expires.Sub(ban.Start) != time.Hour {
t.Errorf("the next ban lasts %s, want 1h", ban.Expires.Sub(ban.Start))
}
held := ledger.Bans(netblock)
if len(held) != 2 || !held[0].Lifted.Equal(midnight().Add(10*time.Minute)) {
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
}
}
// wantMadeByAnAdmin checks how many bans ledger counts as made by an
// admin since the start.
func wantMadeByAnAdmin(t *testing.T, ledger *bans.Ledger, want int) {
t.Helper()
if got := ledger.Made(bans.CauseAdmin); got != want {
t.Errorf("%d bans made by an admin, want %d", got, want)
}
}
// load reads the state files into the parts of params.
func load(t *testing.T, params state.Params) *state.Files {
t.Helper()
@@ -1820,8 +1184,8 @@ func scrape(t *testing.T, params state.Params) string {
}
// metric returns the value of series in text, the metrics, such as
// smallwebwaf_state_file_writes_total{file="bans.json",instance="app"}, or
// fails the test if there is no such series.
// smallwebwaf_state_file_writes_total{file="bans.json"}, or fails the test
// if there is no such series.
func metric(t *testing.T, text, series string) float64 {
t.Helper()
@@ -1850,7 +1214,7 @@ func wantWriteFailed(t *testing.T, params state.Params, name string) {
t.Helper()
got := scrape(t, params)
file := `{file="` + name + `",instance="app"}`
file := `{file="` + name + `"}`
wantMetric(t, got, "smallwebwaf_state_file_writes_total"+file, 1)
wantMetric(t, got, "smallwebwaf_state_file_write_failures_total"+file, 1)
script/example-app
+8 -14
View File
@@ -7,10 +7,9 @@
# request bans for good, that `sv stop` stops smallwebwaf in order, that
# `docker stop` stops the container without having to kill it, and that
# a new container on the same volume still refuses the banned client. The
# containers run with SWWAF_LOOKUP_SOURCE=off, so that no address is sent
# to GeoJS. The containers, the volume and both images are removed however
# the script ends. Building the app needs network access, for nixpkgs'
# binary cache. script/check does not run this.
# containers, the volume and both images are removed however the script
# ends. Building the app needs network access, for nixpkgs' binary cache.
# script/check does not run this.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -54,23 +53,18 @@ healthy() {
[ "$status" = healthy ]
}
# logged <text>...: a line of the container's output holds every text,
# in any order.
# logged <text>: the container's output holds text.
logged() {
lines="$(docker logs "$CONTAINER" 2>&1)"
for text in "$@"; do
lines="$(printf '%s\n' "$lines" | grep -F "$text")" || return 1
done
docker logs "$CONTAINER" 2>&1 | grep -qF "$1"
}
# start_container: run the app's container, with the state files on the
# volume, a rate limit of one request a minute and no client looked up,
# and wait until it is healthy.
# volume and a rate limit of one request a minute, and wait until it is
# healthy.
start_container() {
docker run --detach --name "$CONTAINER" --publish 127.0.0.1::8080 \
--volume "$VOLUME:/var/lib/smallwebwaf" \
--env SWWAF_RATE_LIMIT_PER_MINUTE=1 \
--env SWWAF_LOOKUP_SOURCE=off \
"$APP_IMAGE" >/dev/null
wait_for "the health check did not pass" healthy
address="$(docker port "$CONTAINER" 8080/tcp)"
@@ -118,7 +112,7 @@ main() {
refused_from 203.0.113.9 /.env || fail "a probe for /.env was not refused"
wait_for "smallwebwaf logged no ban for the probe" \
logged '"action":"banned"' '"rule_ids":["env-file"]'
logged '"action":"banned","rule_ids":["env-file"]'
refused_from 203.0.113.9 / || fail "the client of the probe was let through"
wait_for "the client's next request did not make its ban permanent" \
logged '"ban_expires":"permanent"'
script/tidy
-19
View File
@@ -1,19 +0,0 @@
#!/bin/sh
# script/tidy: write go.mod and go.sum as `go mod tidy` writes them, which
# the test phase of the Dockerfile checks. This builds the Dockerfile's
# tidy-files stage, which holds the two files alone, and --output writes
# them into the working tree. The build makes no image, so it has no tag.
# --no-cache because `go mod tidy` asks the module proxy, whose answers a
# cached layer would repeat.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
docker build --no-cache \
--target tidy-files \
--output "type=local,dest=$ROOT" .
}
main "$@"