1 Commits
Author SHA1 Message Date
clawbot 5bf7802404 Admin endpoints for bans and clients on the single listener (closes #27)
check / check (push) Successful in 4m1s
SWWAF_ADMIN_TOKEN, or its _FILE form, opens GET and POST
/_smallwebwaf/bans, DELETE /_smallwebwaf/bans/<client> and GET
/_smallwebwaf/clients/<ip>. Unset, they answer 404; a missing or wrong
token gets 401, in observe mode too. They go through every check, as
the metrics do. POST takes a netblock, not IPv4-mapped and without a
zone, or a client's address, a duration or permanent, and a reason, and
makes an admin ban even while another lasts. DELETE lifts every active
ban covering the address, kept and marked lifted. Bans come back as
bans.json entries; a client as clients.json holds it, with its bans.

Judgement call: answers leave out bans.json's version field.
Judgement call: DELETE takes an address, not a netblock.
Rule suppressed: gosec G304 on a test reading bans.json.

Model: opus-5-5
2026-10-06 22:57:12 +00:00
15 changed files with 1373 additions and 104 deletions
+114 -34
View File
@@ -19,24 +19,26 @@ ledger with the bans you make, keep and lift, the JSON state files with your
edits taken in while it runs and the paths the rate limits do not count, which
come next in the build order, `observe` mode and the rest of the request log's
fields, which come a little later, and the metrics endpoint and the header size
and the idle time as settings, which come last in it. So are two parts of the
and the idle time as settings, which come last in it. So are three parts of the
stage after it: the rule files, the first part, with the bans for a clear sign
of attack, and remote log sending. `smallwebwaf` passes each request to the app
and the app's answer back, unchanged, within its timeouts and size limits, works
out each client's address, bans a client that sends too many requests, not
counting those for the paths you choose, refuses a client that comes from a
country you refuse or from a network you refuse, lets the networks you choose
through, checks each request against the rule files and bans a client whose
request is a clear sign of attack, keeps its bans, each client's counters and
history, and GeoJS's answers in JSON files across restarts, takes in your edits
of those files, such as a ban you make, keep or lift, and of the rule files
while it runs, writes a JSON log line for every request, sends its log lines to
a syslog server too if you name one, serves Prometheus metrics to a scraper that
holds the metrics token, and in `observe` mode passes on the requests it would
refuse, logging what it would have done with them. It comes as the image the
app's own image is built on. The rest of the design comes after that, in the
order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
that led to the design is in [`EVALUATION.md`](EVALUATION.md).
of attack, the other admin endpoints, the second, and remote log sending.
`smallwebwaf` passes each request to the app and the app's answer back,
unchanged, within its timeouts and size limits, works out each client's address,
bans a client that sends too many requests, not counting those for the paths you
choose, refuses a client that comes from a country you refuse or from a network
you refuse, lets the networks you choose through, checks each request against
the rule files and bans a client whose request is a clear sign of attack, keeps
its bans, each client's counters and history, and GeoJS's answers in JSON files
across restarts, takes in your edits of those files, such as a ban you make,
keep or lift, and of the rule files while it runs, writes a JSON log line for
every request, sends its log lines to a syslog server too if you name one,
serves Prometheus metrics to a scraper that holds the metrics token, lets an
admin who holds the admin token list, add and lift bans and ask what it knows of
a client, and in `observe` mode passes on the requests it would refuse, logging
what it would have done with them. It comes as the image the app's own image is
built on. The rest of the design comes after that, in the order of the build
order in [`SPEC.md`](SPEC.md). The survey of existing tools that led to the
design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started
@@ -162,18 +164,23 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
not make it permanent. The bans in `bans.json` are kept, and refuse requests
again when `smallwebwaf` next runs in `enforce` mode, as long as they last.
The timeouts and size limits still apply, since they protect `smallwebwaf` and
the app themselves, and a request for the metrics without the token is still
answered `401`. It is for trying a configuration before enforcing it.
the app themselves, and a request for one of `smallwebwaf`'s own endpoints
without its token is still answered `401`. It is for trying a configuration
before enforcing it.
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
check and without asking the app, for the image's health check.
- Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
a request that carries `SWWAF_METRICS_TOKEN` as
`Authorization: Bearer <token>`, and with `401` for one that does not. While
the token is unset the metrics answer `404`, as does any other request under
`/_smallwebwaf/`. Unlike the health check, such a request goes through every
check any other request goes through, and is answered where another would be
passed to the app: a banned client stays refused, and each counts toward the
client's rate limits. None of them reaches the app.
the token is unset the metrics answer `404`, as does any request under
`/_smallwebwaf/` that is not for one of its endpoints. Unlike the health
check, such a request goes through every check any other request goes through,
and is answered where another would be passed to the app: a banned client
stays refused, and each counts toward the client's rate limits. None of them
reaches the app.
- Lets an admin list, add and lift bans, and ask what it knows of a client,
through the endpoints `SWWAF_ADMIN_TOKEN` opens, which go through the checks
as the metrics do (see "Admin endpoints" below).
- Writes a line in the request log for each request (see "Request log" below).
- Sends every line it writes on stdout to a syslog server as well, while
`SWWAF_LOG_REMOTE_URL` names one (see "Sending the log to a syslog server"
@@ -286,6 +293,12 @@ effective settings are logged at start.
(see "Request log" below). An entry naming `Host` or `Transfer-Encoding` stops
the start, since Go's HTTP server takes both out of the request; the request's
host is the field `host`.
- `SWWAF_ADMIN_TOKEN` (default unset): the token an admin sends for the ban
endpoints and `/_smallwebwaf/clients/<ip>` (see "Admin endpoints" below), a
long random value. While it is unset they are off; one shorter than 32
characters stops the start. The settings logged at start show `********` in
its place. Given as a file, with `SWWAF_ADMIN_TOKEN_FILE`, it can be kept out
of the app's reach (see "Settings given as files" below).
- `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for the
metrics, a long random value. While it is unset the metrics are off; one
shorter than 32 characters stops the start. The settings logged at start show
@@ -513,13 +526,13 @@ entries by client address, with times in UTC.
- `lookups.json`: GeoJS's answers, one to a line, with when GeoJS gave each and
when it was last used.
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made or made
permanent, with every such change in between, and every file every
`SWWAF_STATE_COUNTER_INTERVAL` and when `smallwebwaf` stops. Each write goes to
a temporary file in the same directory, which then replaces the file, so a crash
leaves the old file or the new one, whole. A write that fails is logged, and
tried again at the next write. A hard kill loses what changed since the last
write.
`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted
through `DELETE /_smallwebwaf/bans/<client>`, or made permanent, with every such
change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when
`smallwebwaf` stops. Each write goes to a temporary file in the same directory,
which then replaces the file, so a crash leaves the old file or the new one,
whole. A write that fails is logged, and tried again at the next write. A hard
kill loses what changed since the last write.
At start the files are read back: each client keeps its counts, so a restart
gives it no fresh allowance, and each ban keeps refusing every client in its
@@ -584,6 +597,9 @@ next ban longer; it is kept in `bans.json` with its notes, as any other ban is.
To forget a ban altogether, delete its entry: it then refuses nothing either,
and does not make the netblock's next ban longer.
The ban endpoints add and lift bans without an edit of the file (see "Admin
endpoints" below).
## Rule files
`smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`,
@@ -677,9 +693,10 @@ other request. No metric carries a client's address.
`smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
limit was passed, `smallwebwaf_offences_total` by `kind`, and
`smallwebwaf_bans_made_total` by `cause`, `limit`, `attack` or `admin`, the
last for the bans whose `cause` is `admin` that you add to `bans.json` while
`smallwebwaf` runs; `smallwebwaf_active_bans` and
`smallwebwaf_permanent_bans`, neither of which counts a lifted ban.
last for the bans you add through `POST /_smallwebwaf/bans`, and those whose
`cause` is `admin` that you add to `bans.json` while `smallwebwaf` runs;
`smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`, neither of which
counts a lifted ban.
- `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
`rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
rules read from the rule files.
@@ -716,6 +733,69 @@ The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request
log") are not counted. The metrics of the features still to come, such as the
Core Rule Set, come with them.
## Admin endpoints
While `SWWAF_ADMIN_TOKEN` is set, `smallwebwaf` answers these requests itself,
on the app's own address and through traefik like any other request, for a
request that carries the token as `Authorization: Bearer <token>`:
- `GET /_smallwebwaf/bans`: every ban held, past, active and permanent.
- `POST /_smallwebwaf/bans`: bans a netblock, as adding an entry to `bans.json`
does. The body is a JSON object of `netblock`, `duration` and, if you like,
`reason`. `netblock` is a netblock such as `203.0.113.0/24`, or a client's
address, which bans the netblock a ban on that client covers: its IPv4
address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or
its IPv6 /64. `duration` is a duration such as `1h` or `7d`, or `permanent`.
The ban starts at once, its `cause` is `admin`, and it is made even while
another ban on the netblock lasts. A body that is not such an object, has
another field, has anything but whitespace after the object, or is longer than
4 KiB is answered `400`, saying what is wrong, and so is an IPv4-mapped
netblock, such as `::ffff:203.0.113.0/120`, or a value with a zone, such as
`fe80::1%eth0`.
- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban on a netblock
that `<client>`, an address, is in, as adding `lifted` to its entry in
`bans.json` does, and answers `404` when no ban on it is active.
- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client at
the address `<ip>`: under `client`, the client as `clients.json` holds it,
with its counters and its history, which holds its country as last looked up
and its offences, or `null` when the table of clients does not hold it; and
under `bans`, every ban on a netblock the address is in, with its notes.
The ban endpoints answer with the bans listed, made or lifted, under `bans`,
each as an entry of `bans.json` (see "State files" above), and a ban they make
or lift is written to `bans.json` `SWWAF_STATE_WRITE_DELAY` later. Refusals, and
the answers to requests that cannot be read, are plain text.
A request without the token, or with another, such as the metrics token, is
answered `401`, in `observe` mode too. While the token is unset, each of these
answers `404`, as does any request under `/_smallwebwaf/` that is not for one of
its endpoints. Like the metrics, these requests go through every check any other
request goes through, and are answered where another would be passed to the app:
a banned client stays refused, so an admin whose own address is banned lifts
that ban by editing `bans.json`, and each request counts toward the client's
rate limits. A client in `SWWAF_ALLOW_NETS` skips the checks, and still needs
the token.
With the token in `$TOKEN`, for an app at `https://app.example`:
```sh
# Every ban.
curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans
# Ban 203.0.113.0/24 for seven days.
curl -H "Authorization: Bearer $TOKEN" \
--json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \
https://app.example/_smallwebwaf/bans
# Lift the bans on 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" -X DELETE \
https://app.example/_smallwebwaf/bans/203.0.113.9
# What smallwebwaf knows of 203.0.113.9.
curl -H "Authorization: Bearer $TOKEN" \
https://app.example/_smallwebwaf/clients/203.0.113.9
```
## Why
Small self-hosted sites now receive a great deal of traffic nobody asked for:
+100
View File
@@ -184,3 +184,103 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
ledger.Made(bans.CauseAdmin), ledger.Made(bans.CauseLimit))
}
}
func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
t.Parallel()
netblock := netip.MustParsePrefix("203.0.113.0/24")
ledger := bans.New(defaultRules())
// An hour's ban for a broken limit.
ledger.BanForLimit(netblock, midnight(), bans.Notes{})
wantChanged(t, ledger, true)
// A minute later an admin bans the netblock for good, named by an
// address in it: that ban is made, and counts the other among the
// earlier bans.
now := midnight().Add(time.Minute)
want := bans.Ban{
Netblock: netblock,
Start: now,
Cause: bans.CauseAdmin,
Reason: "probes for logins",
Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 1}},
}
got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
"probes for logins")
if got != want {
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
}
wantChanged(t, ledger, true)
if made := ledger.Made(bans.CauseAdmin); made != 1 {
t.Errorf("%d bans made by an admin, want 1", made)
}
// It refuses once the ban for the limit has ended.
ban, banned := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
if !banned || ban != want {
t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
ban, banned, want)
}
}
func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
t.Parallel()
client := netip.MustParseAddr("203.0.113.9")
own := netip.MustParsePrefix("203.0.113.9/32")
wide := netip.MustParsePrefix("203.0.113.0/24")
other := netip.MustParsePrefix("203.0.113.10/32")
ledger := bans.New(defaultRules())
ledger.Load([]bans.Ban{
// Ended an hour ago.
{
Netblock: own, Start: midnight().Add(-2 * time.Hour),
Expires: midnight().Add(-time.Hour), Cause: bans.CauseLimit,
},
// Active, on the client's address and on its /24.
{
Netblock: own, Start: midnight(), Expires: midnight().Add(time.Hour),
Cause: bans.CauseLimit,
},
{Netblock: wide, Start: midnight(), Cause: bans.CauseAdmin},
// Another client's.
{Netblock: other, Start: midnight(), Cause: bans.CauseAdmin},
})
now := midnight().Add(time.Minute)
lifted := ledger.Lift(client, now)
if len(lifted) != 2 || lifted[0].Lifted != now || lifted[1].Lifted != now {
t.Errorf("lifted %+v, want the two active bans covering the client", lifted)
}
wantChanged(t, ledger, true)
if _, banned := ledger.Check(client, now); banned {
t.Error("the client is still banned")
}
if _, banned := ledger.Check(other.Addr(), now); !banned {
t.Error("the other client's ban was lifted")
}
// The lifted bans are kept, and the one that had ended is not lifted.
covering := ledger.Covering(client)
if len(covering) != 3 || covering[0].Netblock != wide ||
!covering[1].Lifted.IsZero() || covering[2].Lifted != now {
t.Errorf("the bans covering the client are %+v, want the /24's and both "+
"of its own, the earlier not lifted", covering)
}
// With none active, nothing is lifted or changed.
if lifted = ledger.Lift(client, now); len(lifted) != 0 {
t.Errorf("lifted %+v again", lifted)
}
wantChanged(t, ledger, false)
}
+102 -6
View File
@@ -154,7 +154,8 @@ type Ledger struct {
// at most rules.MaxBans.
held int
// made is how many bans have been made since the start, by cause: by
// the ledger, and by an admin in an edit of bans.json.
// the ledger, and by an admin, through BanForAdmin or in an edit of
// bans.json.
made map[string]int
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
// netblocks that have been banned. Check looks for a ban at each of
@@ -182,9 +183,9 @@ func New(rules Rules) *Ledger {
}
}
// Changed receives a value after a ban is made or made permanent, so that
// bans.json can be written. Several changes before it is read leave one
// value.
// Changed receives a value after a ban is made, lifted or made permanent,
// so that bans.json can be written. Several changes before it is read
// leave one value.
func (l *Ledger) Changed() <-chan struct{} {
return l.changed
}
@@ -267,6 +268,81 @@ func (l *Ledger) BanForAttack(netblock netip.Prefix, now time.Time, notes Notes)
return l.ban(netblock, now, CauseAttack, "matched the rule "+notes.RuleID, notes)
}
// BanForAdmin bans netblock at now for an admin, with reason, until
// expires, or for good when expires is zero, and returns the ban, whose
// cause is CauseAdmin. Unlike BanForLimit and BanForAttack, it makes the
// ban even while another on netblock is active, since the admin asked
// for this one. The ledger fills in the notes' EarlierBans, and counts
// the ban among those made.
func (l *Ledger) BanForAdmin(
netblock netip.Prefix, now, expires time.Time, reason string,
) Ban {
l.mu.Lock()
defer l.mu.Unlock()
ban := Ban{
Netblock: netblock.Masked(), Start: now, Expires: expires, Cause: CauseAdmin,
Reason: reason,
}
held, found := l.netblocks.Get(ban.Netblock)
if found {
ban.Notes.EarlierBans = earlierBans(*held)
}
l.add(ban)
l.made[CauseAdmin]++
l.markChanged()
return ban
}
// Lift lifts, at now, every ban active then on a netblock client is in,
// as an admin does, and returns those bans. A lifted ban is kept, refuses
// nothing, and does not make the netblock's next ban longer.
func (l *Ledger) Lift(client netip.Addr, now time.Time) []Ban {
l.mu.Lock()
defer l.mu.Unlock()
var lifted []Ban
for _, bans := range l.covering(client) {
for i := range *bans {
ban := &(*bans)[i]
if ban.ActiveAt(now) {
ban.Lifted = now
lifted = append(lifted, *ban)
}
}
}
if len(lifted) > 0 {
l.markChanged()
}
return lifted
}
// Covering returns every ban held on a netblock client is in, active or
// not, sorted by netblock, and each netblock's bans oldest first. It is
// not a request from client, and leaves when the netblocks were last seen
// unchanged.
func (l *Ledger) Covering(client netip.Addr) []Ban {
l.mu.Lock()
defer l.mu.Unlock()
var held []Ban
for _, bans := range l.covering(client) {
held = append(held, *bans...)
}
slices.SortStableFunc(held, func(a, b Ban) int {
return a.Netblock.Compare(b.Netblock)
})
return held
}
// Bans returns the bans held on netblock, oldest first. It is not a
// request from netblock, and leaves when it was last seen unchanged.
func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
@@ -283,8 +359,8 @@ func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
// Made returns how many bans for cause have been made since the start:
// for CauseLimit and CauseAttack, by the ledger; for CauseAdmin, by an
// admin in an edit of bans.json, as LoadEdit counts them. The bans read
// from bans.json at the start are not among them.
// admin, with BanForAdmin or in an edit of bans.json, as LoadEdit counts
// them. The bans read from bans.json at the start are not among them.
func (l *Ledger) Made(cause string) int {
l.mu.Lock()
defer l.mu.Unlock()
@@ -496,6 +572,26 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
return nil
}
// covering returns the bans of each netblock held that client is in,
// leaving when the netblocks were last seen unchanged.
func (l *Ledger) covering(client netip.Addr) []*[]Ban {
lengths := l.v6Lengths
if client.Is4() {
lengths = l.v4Lengths
}
var found []*[]Ban
for _, length := range lengths {
bans, ok := l.netblocks.Peek(netip.PrefixFrom(client, length).Masked())
if ok {
found = append(found, bans)
}
}
return found
}
// add adds ban to its netblock's bans, after the last, and makes its
// netblock the most recently seen. With MaxBans held, it drops one first,
// unless ban's cause is CauseAdmin, which does not count toward MaxBans.
+9 -4
View File
@@ -129,6 +129,10 @@ type Config struct {
// LogRequestHeaders are the request headers whose values the request
// log gives, in lower case (SWWAF_LOG_REQUEST_HEADERS).
LogRequestHeaders []string
// AdminToken is the bearer token an admin sends for the ban endpoints
// and /_smallwebwaf/clients/<ip> (SWWAF_ADMIN_TOKEN), "" while it is
// unset and they are off.
AdminToken string
// MetricsToken is the bearer token a scraper sends for the metrics
// (SWWAF_METRICS_TOKEN), "" while it is unset and the metrics are off.
// MetricsTopN is how many countries get series of their own in the
@@ -274,6 +278,7 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
StateCounterInterval: env.durationNotOff("SWWAF_STATE_COUNTER_INTERVAL", "15m"),
LogRequestHeaders: env.headerNames("SWWAF_LOG_REQUEST_HEADERS",
"accept,accept-language,accept-encoding,content-type,origin,range"),
AdminToken: env.token("SWWAF_ADMIN_TOKEN"),
MetricsToken: env.token("SWWAF_METRICS_TOKEN"),
MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
RulesDir: env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"),
@@ -496,7 +501,7 @@ func (e *environment) headerNames(name, defaultValue string) []string {
// durationNotOff reads a setting that is a duration and, unlike a
// timeout, cannot be off.
func (e *environment) durationNotOff(name, defaultValue string) time.Duration {
duration, err := parseDurationNotOff(e.value(name, defaultValue))
duration, err := ParseDurationNotOff(e.value(name, defaultValue))
e.check(name, err)
return duration
@@ -732,9 +737,9 @@ func parseCount(value string) (int64, error) {
return n, nil
}
// parseDurationNotOff reads a duration above zero, as parseDuration does,
// but not off.
func parseDurationNotOff(value string) (time.Duration, error) {
// ParseDurationNotOff reads a duration above zero, as parseDuration does,
// but not off. The ban endpoint reads the duration of a ban with it too.
func ParseDurationNotOff(value string) (time.Duration, error) {
duration, err := parseDuration(value)
if err != nil || duration == 0 {
return 0, fmt.Errorf("%q %w", value, errNotDurationAboveZero)
+30 -16
View File
@@ -50,6 +50,7 @@ const (
stateDir = "SWWAF_STATE_DIR"
stateWriteDelay = "SWWAF_STATE_WRITE_DELAY"
stateCounterInterval = "SWWAF_STATE_COUNTER_INTERVAL"
adminToken = "SWWAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
metricsToken = "SWWAF_METRICS_TOKEN" //nolint:gosec // the setting's name
metricsTopN = "SWWAF_METRICS_TOP_N"
instanceName = "SWWAF_INSTANCE_NAME"
@@ -81,8 +82,12 @@ rlG9y/jrJb6ORy3kTLWo2EA0BA67vuI=
-----END CERTIFICATE-----
`
// token is a token of 32 characters, the shortest allowed.
const token = "0123456789abcdef0123456789abcdef"
// token is a token of 32 characters, the shortest allowed, and
// otherToken another.
const (
token = "0123456789abcdef0123456789abcdef"
otherToken = "fedcba9876543210fedcba9876543210"
)
// instance is an SWWAF_INSTANCE_NAME that is a valid app name too, and
// remoteURL an SWWAF_LOG_REMOTE_URL, for the tests that send the lines.
@@ -693,32 +698,40 @@ func TestShortTokenStopsTheStartWithoutShowingIt(t *testing.T) {
t.Parallel()
// Characters are counted, not bytes: each é takes two.
for _, value := range []string{"", token[1:], strings.Repeat("é", 31)} {
t.Run(value, func(t *testing.T) {
t.Parallel()
for _, name := range []string{adminToken, metricsToken} {
for _, value := range []string{"", token[1:], strings.Repeat("é", 31)} {
t.Run(name+"="+value, func(t *testing.T) {
t.Parallel()
_, err := config.FromEnvironment(environment{metricsToken: value}.lookupEnv)
_, err := config.FromEnvironment(environment{name: value}.lookupEnv)
want := metricsToken + ": is shorter than 32 characters"
if err == nil || err.Error() != want {
t.Errorf("error %v, want %s", err, want)
}
})
want := name + ": is shorter than 32 characters"
if err == nil || err.Error() != want {
t.Errorf("error %v, want %s", err, want)
}
})
}
}
}
func TestTokenIsLoggedMasked(t *testing.T) {
func TestTokensAreReadAndLoggedMasked(t *testing.T) {
t.Parallel()
cfg := fromEnvironment(t, environment{metricsToken: token})
cfg := fromEnvironment(t, environment{adminToken: otherToken, metricsToken: token})
if cfg.AdminToken != otherToken || cfg.MetricsToken != token {
t.Errorf("admin token %q and metrics token %q, want %q and %q",
cfg.AdminToken, cfg.MetricsToken, otherToken, token)
}
var out bytes.Buffer
slog.New(slog.NewJSONHandler(&out, nil)).Info("starting", "settings", cfg)
if strings.Contains(out.String(), token) ||
!strings.Contains(out.String(), `"`+metricsToken+`":"********"`) {
t.Errorf("the token is not logged masked: %s", out.String())
logged := out.String()
if strings.Contains(logged, token) || strings.Contains(logged, otherToken) ||
!strings.Contains(logged, `"`+adminToken+`":"********"`) ||
!strings.Contains(logged, `"`+metricsToken+`":"********"`) {
t.Errorf("the tokens are not logged masked: %s", logged)
}
}
@@ -901,6 +914,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
stateDir: "/var/lib/smallwebwaf",
stateWriteDelay: "10s",
stateCounterInterval: "15m",
adminToken: "",
metricsToken: "",
metricsTopN: "50",
instanceName: hostname,
+289 -7
View File
@@ -1,26 +1,60 @@
package proxy
import (
"bytes"
"crypto/subtle"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/netip"
"os"
"strings"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/state"
)
// banBodyMaxBytes is the most of the body of a request to add a ban that
// is read; its three fields need far less.
const banBodyMaxBytes = 4 << 10
// permanent is how the log line and the ban endpoint name a ban that
// never ends.
const permanent = "permanent"
var (
errNotBanToAdd = errors.New(
"the body is not a JSON object of netblock, duration and reason")
errNotNetblock = errors.New(
"is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24")
errMappedNetblock = errors.New(
"is IPv4-mapped: give the IPv4 netblock, such as 203.0.113.0/24")
errZone = errors.New("has a zone, which a netblock cannot have")
errNotDuration = errors.New(
"is not a duration above zero, such as 1h or 7d, or permanent")
errNotAddress = errors.New("is not an address, such as 203.0.113.9")
)
// answerAdmin answers a request for smallwebwaf itself, under
// /_smallwebwaf/, once it has passed the checks: GET MetricsPath with
// SWWAF_METRICS_TOKEN gets the metrics, and without it is refused with
// 401. Any other request gets 404, as the metrics do while
// SWWAF_METRICS_TOKEN is unset.
// /_smallwebwaf/, once it has passed the checks. Each endpoint needs a
// token, sent as Authorization: Bearer <token>: the metrics
// SWWAF_METRICS_TOKEN, the others SWWAF_ADMIN_TOKEN. A request without
// it is refused with 401. An endpoint whose token is unset answers 404,
// as any other request under /_smallwebwaf/ does.
func (rq *request) answerAdmin() {
rq.line.Action = requestlog.ActionAdmin
rq.startClientResponseTimeout()
token := rq.h.config.MetricsToken
token, answer := rq.endpoint()
switch {
case token == "" || rq.in.Method != http.MethodGet || rq.in.URL.Path != MetricsPath:
case token == "":
http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
case !hasToken(rq.in, token):
rq.out.Header().Set("WWW-Authenticate", "Bearer")
@@ -29,7 +63,29 @@ func (rq *request) answerAdmin() {
action: requestlog.ActionAdmin,
})
default:
rq.h.metrics.ServeHTTP(rq.out, rq.in)
answer()
}
}
// endpoint returns the token the request's endpoint needs, and what
// answers the request there; "" when there is no such endpoint.
func (rq *request) endpoint() (string, func()) {
cfg := rq.h.config
method, path := rq.in.Method, rq.in.URL.Path
switch {
case method == http.MethodGet && path == MetricsPath:
return cfg.MetricsToken, func() { rq.h.metrics.ServeHTTP(rq.out, rq.in) }
case method == http.MethodGet && path == BansPath:
return cfg.AdminToken, rq.listBans
case method == http.MethodPost && path == BansPath:
return cfg.AdminToken, rq.addBan
case method == http.MethodDelete && strings.HasPrefix(path, BansPath+"/"):
return cfg.AdminToken, rq.liftBans
case method == http.MethodGet && strings.HasPrefix(path, ClientsPath):
return cfg.AdminToken, rq.showClient
default:
return "", nil
}
}
@@ -41,3 +97,229 @@ func hasToken(r *http.Request, token string) bool {
return strings.EqualFold(scheme, "Bearer") &&
subtle.ConstantTimeCompare([]byte(sent), []byte(token)) == 1
}
// listBans answers GET BansPath with every ban held.
func (rq *request) listBans() {
rq.answerBans(rq.h.ledger.Snapshot())
}
// banToAdd is the body of POST BansPath.
type banToAdd struct {
// Netblock is a netblock, or a client's address, which stands for the
// netblock a ban on that client covers.
Netblock string `json:"netblock"`
// Duration is how long the ban lasts, as a setting gives a duration,
// or permanent.
Duration string `json:"duration"`
Reason string `json:"reason"`
}
// addBan answers POST BansPath: it bans the netblock the body names, as
// an admin, from now for the duration the body gives, with its reason,
// and answers with that ban.
func (rq *request) addBan() {
// The body must arrive within SWWAF_CLIENT_REQUEST_TIMEOUT, as any
// other request's must.
rq.stopReadingBody(rq.clientRequestDeadline())
toAdd, err := rq.readBanToAdd()
if refused := rq.refused.Load(); refused != nil {
rq.answer(*refused) // the body is over SWWAF_REQUEST_MAX_BYTES
return
}
if errors.Is(err, os.ErrDeadlineExceeded) {
rq.answer(refusal{
status: http.StatusRequestTimeout,
action: requestlog.ActionTimedOut,
limit: "SWWAF_CLIENT_REQUEST_TIMEOUT",
})
return
}
var (
netblock netip.Prefix
expires time.Time
now = rq.h.now()
)
if err == nil {
netblock, err = rq.h.banNetblock(toAdd.Netblock)
}
if err == nil {
expires, err = expiry(toAdd.Duration, now)
}
if err != nil {
http.Error(rq.out, err.Error(), http.StatusBadRequest)
return
}
ban := rq.h.ledger.BanForAdmin(netblock, now, expires, toAdd.Reason)
rq.answerBans([]bans.Ban{ban})
}
// readBanToAdd reads the body of POST BansPath: a JSON object with
// nothing but whitespace after it, in at most banBodyMaxBytes.
func (rq *request) readBanToAdd() (banToAdd, error) {
var body io.ReadCloser = http.NoBody
if rq.body != nil {
body = rq.body
}
data, err := io.ReadAll(http.MaxBytesReader(nil, body, banBodyMaxBytes))
if err != nil {
return banToAdd{}, fmt.Errorf("%w: %w", errNotBanToAdd, err)
}
var toAdd banToAdd
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.DisallowUnknownFields()
err = decoder.Decode(&toAdd)
if err != nil {
return banToAdd{}, fmt.Errorf("%w: %w", errNotBanToAdd, err)
}
// Token returns io.EOF only when nothing but whitespace is left.
_, err = decoder.Token()
if !errors.Is(err, io.EOF) {
return banToAdd{}, fmt.Errorf("%w: more follows the object", errNotBanToAdd)
}
return toAdd, nil
}
// banNetblock reads value, a netblock such as 203.0.113.0/24, or a
// client's address, which stands for the netblock a ban on that client
// covers. An IPv4-mapped netblock, such as ::ffff:203.0.113.0/120, is
// refused, since a client's address is looked up as IPv4 and a ban on it
// would refuse nothing, and so is a value with a zone.
func (h *handler) banNetblock(value string) (netip.Prefix, error) {
netblock, err := netip.ParsePrefix(value)
if err == nil {
if netblock.Addr().Is4In6() {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errMappedNetblock)
}
return netblock, nil
}
// ParsePrefix refuses a zone, but ParseAddr reads the /48 of
// 2001:db8::1%x/48 as part of the zone.
addr, err := netip.ParseAddr(value)
if err != nil {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errNotNetblock)
}
if addr.Zone() != "" {
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errZone)
}
return h.netblock(addr), nil
}
// expiry returns when a ban made at now for duration ends: duration
// later, for a duration as a setting gives one, or zero for permanent.
func expiry(duration string, now time.Time) (time.Time, error) {
if duration == permanent {
return time.Time{}, nil
}
length, err := config.ParseDurationNotOff(duration)
if err != nil {
return time.Time{}, fmt.Errorf("duration %q %w", duration, errNotDuration)
}
return now.Add(length), nil
}
// liftBans answers DELETE BansPath/<client>: it lifts every ban active on
// a netblock the client's address is in, and answers with those bans, or
// with 404 when none is active.
func (rq *request) liftBans() {
client, err := pathAddress(rq.in.URL.Path, BansPath+"/")
if err != nil {
http.Error(rq.out, err.Error(), http.StatusBadRequest)
return
}
lifted := rq.h.ledger.Lift(client, rq.h.now())
if len(lifted) == 0 {
http.Error(rq.out, "no ban is active on "+client.String(), http.StatusNotFound)
return
}
rq.answerBans(lifted)
}
// clientAnswer is the answer to GET ClientsPath<ip>: the client the
// address is, as clients.json holds it, or null when the table of
// clients does not hold it, and the bans on each netblock the address is
// in, as bans.json lists them.
type clientAnswer struct {
Client *ratelimit.Client `json:"client"`
Bans []state.BanEntry `json:"bans"`
}
// showClient answers GET ClientsPath<ip> with what smallwebwaf knows of
// the client: its counters, its history, which holds its country as last
// looked up and its offences, and its bans with their notes.
func (rq *request) showClient() {
addr, err := pathAddress(rq.in.URL.Path, ClientsPath)
if err != nil {
http.Error(rq.out, err.Error(), http.StatusBadRequest)
return
}
answer := clientAnswer{Bans: state.BanEntries(rq.h.ledger.Covering(addr))}
client, seen := rq.h.limiter.Client(clientGroup(addr))
if seen {
answer.Client = &client
}
rq.answerJSON(answer)
}
// pathAddress reads the client's address that follows prefix in path.
func pathAddress(path, prefix string) (netip.Addr, error) {
value := strings.TrimPrefix(path, prefix)
addr, err := netip.ParseAddr(value)
if err != nil {
return netip.Addr{}, fmt.Errorf("%q %w", value, errNotAddress)
}
return addr.Unmap(), nil
}
// answerBans answers with held under bans, as bans.json lists them.
func (rq *request) answerBans(held []bans.Ban) {
rq.answerJSON(struct {
Bans []state.BanEntry `json:"bans"`
}{state.BanEntries(held)})
}
// answerJSON answers with value as indented JSON.
func (rq *request) answerJSON(value any) {
body, err := json.MarshalIndent(value, "", " ")
if err != nil {
rq.h.processLog.Error("encoding an answer failed", "error", err.Error())
http.Error(rq.out, http.StatusText(http.StatusInternalServerError),
http.StatusInternalServerError)
return
}
rq.out.Header().Set("Content-Type", "application/json")
_, _ = rq.out.Write(append(body, '\n'))
}
+535
View File
@@ -0,0 +1,535 @@
package proxy_test
import (
"encoding/json"
"net/http"
"net/netip"
"slices"
"strconv"
"strings"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/state"
)
const (
adminToken = "SWWAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
// adminSecret is the SWWAF_ADMIN_TOKEN the tests set, and adminBearer
// how a request carries it.
adminSecret = "fedcba9876543210fedcba9876543210"
adminBearer = "Bearer " + adminSecret
// adminClient is the client the tests' admin sends its requests from.
adminClient = "192.0.2.10"
// banOtherClient is the body of a request to ban otherClient for an
// hour.
banOtherClient = `{"netblock": "` + otherClient + `", "duration": "1h", ` +
`"reason": "probes for logins"}`
)
func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
t.Parallel()
// The metrics token is set, and opens none of them.
s, clk, server := startWithClock(t, "", map[string]string{metricsToken: token})
server.Ledger.BanForLimit(netip.MustParsePrefix(otherClient+"/32"), clk.Now(),
bans.Notes{})
before := server.Ledger.Snapshot()
// An empty token does not match the unset one either.
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
for _, e := range adminEndpoints() {
s.adminRequest(adminClient, authorization, e.method, e.path, e.body,
http.StatusNotFound, requestlog.ActionAdmin)
}
}
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) {
t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
}
}
func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
metricsToken: token,
})
// Listing the bans, banning otherClient, lifting that ban, and asking
// about otherClient, in that order. Without the admin token, with the
// metrics token, or with one that differs, each is refused, and
// changes nothing; with the admin token, it is answered.
for _, e := range adminEndpoints() {
before := server.Ledger.Snapshot()
for _, authorization := range []string{
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
} {
got := s.adminRequest(adminClient, authorization, e.method, e.path, e.body,
http.StatusUnauthorized, requestlog.ActionAdmin)
if got.header.Get("WWW-Authenticate") != "Bearer" {
t.Errorf("%s %s with %q was answered without WWW-Authenticate: Bearer",
e.method, e.path, authorization)
}
}
if after := server.Ledger.Snapshot(); !slices.Equal(after, before) {
t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
e.method, e.path, after, before)
}
got := s.admin(e.method, e.path, e.body, http.StatusOK)
if got.header.Get("Content-Type") != "application/json" {
t.Errorf("%s %s answered %q", e.method, e.path, got.header.Get("Content-Type"))
}
}
// Any other request under /_smallwebwaf/ is not found.
for _, e := range []adminEndpoint{
{http.MethodPut, proxy.BansPath, banOtherClient},
{http.MethodDelete, proxy.BansPath, ""},
{http.MethodGet, proxy.BansPath + "/" + otherClient, ""},
{http.MethodPost, proxy.ClientsPath + otherClient, ""},
{http.MethodGet, strings.TrimSuffix(proxy.ClientsPath, "/"), ""},
} {
s.admin(e.method, e.path, e.body, http.StatusNotFound)
}
}
func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
t.Parallel()
s, clk, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
banScopeV4Prefix: "24",
})
// A ban on otherClient bans the /24 a ban on that client covers, so it
// refuses client too, for an hour.
start := clk.Now()
expires := start.Add(time.Hour)
want := state.BanEntry{
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
Start: start,
Expires: &expires,
Cause: bans.CauseAdmin,
Reason: "probes for logins",
}
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK),
want)
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
if line.BanExpires != requestlog.FormatTime(expires) {
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
}
// Its notes count the request it refused.
want.Notes.Requests, want.Notes.Refused = 1, 1
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want)
// Ten minutes on, lifting the bans on client lifts that one, which is
// kept, marked lifted.
clk.advance(10 * time.Minute)
lifted := clk.Now()
want.Lifted = &lifted
wantBans(t, s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "",
http.StatusOK), want)
s.get(client, http.StatusOK, requestlog.ActionForward)
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want)
// No ban on it is active any more.
s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "", http.StatusNotFound)
}
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
t.Parallel()
s, clk, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
banScopeV4Prefix: "24",
})
start := clk.Now()
for _, tc := range []struct {
netblock, duration string
want string
length time.Duration // 0 for a permanent ban
}{
// An address stands for the netblock a ban on that client covers.
{client, "7d", "203.0.113.0/24", 7 * 24 * time.Hour},
{"::ffff:198.51.100.7", "90m", "198.51.100.0/24", 90 * time.Minute},
{"2001:db8:5::1", "permanent", "2001:db8:5::/64", 0},
// A netblock stands for itself, its bits past its length cleared.
{"198.51.100.7/16", "1h", "198.51.0.0/16", time.Hour},
{"2001:db8:6::/48", "1h", "2001:db8:6::/48", time.Hour},
} {
// Whitespace may follow the object.
body := `{"netblock": "` + tc.netblock + `", "duration": "` + tc.duration + `"}` +
"\r\n"
want := state.BanEntry{
Netblock: netip.MustParsePrefix(tc.want), Start: start, Cause: bans.CauseAdmin,
}
if tc.length != 0 {
expires := start.Add(tc.length)
want.Expires = &expires
}
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, body, http.StatusOK), want)
}
}
func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{adminToken: adminSecret})
for _, tc := range []struct{ body, want string }{
{"", "the body is not a JSON object of netblock, duration and reason: EOF"},
{"netblock=203.0.113.9", "the body is not a JSON object"},
{
`{"netblock": "203.0.113.9", "duration": "1h", "until": "2027"}`,
`unknown field "until"`,
},
{
`{"netblock": "203.0.113", "duration": "1h"}`,
`netblock "203.0.113" is not an address or a netblock`,
},
// A client's address is looked up as IPv4, so a ban on an
// IPv4-mapped netblock would refuse nothing.
{
`{"netblock": "::ffff:203.0.113.0/120", "duration": "1h"}`,
`netblock "::ffff:203.0.113.0/120" is IPv4-mapped`,
},
// Read as an address, its zone would be "x/48", and its ban on the
// /64 around it.
{
`{"netblock": "2001:db8::1%x/48", "duration": "1h"}`,
`netblock "2001:db8::1%x/48" has a zone`,
},
{
`{"netblock": "fe80::1%eth0", "duration": "1h"}`,
`netblock "fe80::1%eth0" has a zone`,
},
// Anything but whitespace after the object.
{
`{"netblock": "203.0.113.9", "duration": "1h"}` +
`{"netblock": "198.51.100.0/24", "duration": "1h"}`,
"more follows the object",
},
{`{"netblock": "203.0.113.9", "duration": "1h"} x`, "more follows the object"},
{`{"duration": "1h"}`, `netblock "" is not an address or a netblock`},
{`{"netblock": "203.0.113.9"}`, `duration "" is not a duration above zero`},
{
`{"netblock": "203.0.113.9", "duration": "off"}`,
`duration "off" is not a duration above zero`,
},
{
`{"netblock": "203.0.113.9", "duration": "0s"}`,
`duration "0s" is not a duration above zero`,
},
{
`{"netblock": "203.0.113.9", "duration": "forever"}`,
`duration "forever" is not a duration above zero, such as 1h or 7d, ` +
`or permanent`,
},
// Over the 4 KiB read of a body, even when the object comes first.
{
`{"netblock": "203.0.113.9", "duration": "1h", "reason": "` +
strings.Repeat("x", 4<<10) + `"}`,
"request body too large",
},
{
`{"netblock": "203.0.113.9", "duration": "1h"}` + strings.Repeat(" ", 4<<10),
"request body too large",
},
} {
got := s.admin(http.MethodPost, proxy.BansPath, tc.body, http.StatusBadRequest)
if !strings.Contains(string(got.body), tc.want) {
t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
}
}
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
func TestBanToAddOverTheRequestSizeLimitIsRefused(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
requestMaxBytes: "16",
})
// Sent in a chunk, its length is not announced, so that it is found
// over SWWAF_REQUEST_MAX_BYTES only as it is read.
chunk := `{"netblock": "203.0.113.9", "duration": "1h"}`
s.adminRequest(adminClient, adminBearer+"\r\nTransfer-Encoding: chunked",
http.MethodPost, proxy.BansPath,
strconv.FormatInt(int64(len(chunk)), 16)+"\r\n"+chunk+"\r\n0\r\n\r\n",
http.StatusRequestEntityTooLarge, requestlog.ActionTooLarge)
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
func TestBanToAddSlowerThanTheClientRequestTimeoutIsRefused(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
metricsToken: token,
clientRequestTimeout: shortTimeoutSetting,
})
// The chunk announces 256 bytes and the rest of it never comes, so only
// the timeout ends the wait. A hold-up of the test process can only
// make the answer later, so the time is checked only for not being
// shorter than the timeout.
start := time.Now()
s.adminRequest(adminClient, adminBearer+"\r\nTransfer-Encoding: chunked",
http.MethodPost, proxy.BansPath, "100\r\n"+`{"netblock": "203.0.113.9", `,
http.StatusRequestTimeout, requestlog.ActionTimedOut)
if took := time.Since(start); took < shortTimeout {
t.Errorf("answered after %s, before the timeout of %s ran out", took, shortTimeout)
}
wantLimitHits(t, s.addr, clientRequestTimeout, 1)
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
t.Parallel()
s, clk, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
rateLimitPerMinute: "2",
rateLimitExemptNets: adminClient,
})
start := clk.Now()
// Two of otherClient's requests are let through; the third breaks the
// limit of two a minute, and bans it.
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
s.get(otherClient, http.StatusForbidden, requestlog.ActionRateLimited)
// Asked about by its address in IPv6 form too.
for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
var got struct {
Client *ratelimit.Client `json:"client"`
Bans []state.BanEntry `json:"bans"`
}
decode(t, s.admin(http.MethodGet, proxy.ClientsPath+addr, "", http.StatusOK), &got)
if got.Client == nil {
t.Fatalf("%s: no client", addr)
}
history := got.Client.History
if got.Client.Client != netip.MustParsePrefix(otherClient+"/32") ||
history.Requests != 3 || history.Forwarded != 2 || history.Refused != 1 ||
history.Offences.Limit != 1 || !history.FirstSeen.Equal(start) {
t.Errorf("%s: client %+v", addr, got.Client)
}
if len(got.Bans) != 1 || got.Bans[0].Cause != bans.CauseLimit ||
got.Bans[0].Reason != "requests per minute over the limit of 2" ||
got.Bans[0].Notes.Count != 3 {
t.Errorf("%s: bans %+v, want the one for the broken limit", addr, got.Bans)
}
}
// Of an address no request came from and no ban covers, nothing is
// known.
got := s.admin(http.MethodGet, proxy.ClientsPath+"198.51.100.99", "", http.StatusOK)
if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
t.Errorf("an unknown client is answered\n%s", got.body)
}
s.admin(http.MethodGet, proxy.ClientsPath+"203.0.113", "", http.StatusBadRequest)
s.admin(http.MethodDelete, proxy.BansPath+"/203.0.113.0/24", "",
http.StatusBadRequest)
}
func TestBannedClientIsRefusedAtTheEndpointsEvenWithTheToken(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{adminToken: adminSecret})
s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK)
// otherClient cannot lift its own ban either.
for _, e := range adminEndpoints() {
s.adminRequest(otherClient, adminBearer, e.method, e.path, e.body,
http.StatusForbidden, requestlog.ActionBanned)
}
}
func TestAdminRequestsCountTowardTheLimits(t *testing.T) {
t.Parallel()
s, _, _ := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
rateLimitPerMinute: "2",
})
// A request refused for a missing token and one answered count toward
// the limit of two a minute, so the next breaks it.
s.adminRequest(client, "", http.MethodGet, proxy.BansPath, "",
http.StatusUnauthorized, requestlog.ActionAdmin)
s.adminRequest(client, adminBearer, http.MethodGet, proxy.BansPath, "",
http.StatusOK, requestlog.ActionAdmin)
s.adminRequest(client, adminBearer, http.MethodGet, proxy.BansPath, "",
http.StatusForbidden, requestlog.ActionRateLimited)
}
func TestClientInAllowNetsSkipsTheChecksButNeedsTheToken(t *testing.T) {
t.Parallel()
const allowed = "192.0.2.60" // in SWWAF_ALLOW_NETS
s, clk, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
allowNets: allowed,
rateLimitPerMinute: "1",
})
// A ban on it refuses nothing, and its requests are not counted.
server.Ledger.BanForAdmin(netip.MustParsePrefix(allowed+"/32"), clk.Now(),
time.Time{}, "")
for range 2 {
s.adminRequest(allowed, "", http.MethodGet, proxy.BansPath, "",
http.StatusUnauthorized, requestlog.ActionAdmin)
s.adminRequest(allowed, adminBearer, http.MethodGet, proxy.BansPath, "",
http.StatusOK, requestlog.ActionAdmin)
}
}
func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{
adminToken: adminSecret,
mode: observe,
})
for _, e := range adminEndpoints() {
s.adminRequest(adminClient, "", e.method, e.path, e.body,
http.StatusUnauthorized, requestlog.ActionAdmin)
}
if held := server.Ledger.Snapshot(); len(held) != 0 {
t.Errorf("the ledger holds %+v, want no ban", held)
}
}
// adminEndpoint is a request to an endpoint SWWAF_ADMIN_TOKEN opens.
type adminEndpoint struct {
method, path, body string
}
// adminEndpoints returns a request to each endpoint SWWAF_ADMIN_TOKEN
// opens: listing the bans, banning otherClient for an hour, lifting the
// bans on otherClient, and asking about otherClient.
func adminEndpoints() []adminEndpoint {
return []adminEndpoint{
{http.MethodGet, proxy.BansPath, ""},
{http.MethodPost, proxy.BansPath, banOtherClient},
{http.MethodDelete, proxy.BansPath + "/" + otherClient, ""},
{http.MethodGet, proxy.ClientsPath + otherClient, ""},
}
}
// admin sends a request with method for path, with body, from
// adminClient, with the admin token, and checks that it is answered with
// status, its log line's action admin. It returns the answer.
func (s *sender) admin(method, path, body string, status int) answer {
s.t.Helper()
return s.adminRequest(adminClient, adminBearer, method, path, body, status,
requestlog.ActionAdmin)
}
// adminRequest sends a request with method for path, with body, from the
// client at from, with authorization as its Authorization header unless
// it is "", and checks its answer's status and its log line's action, as
// request does. authorization may end in more header lines. A body that
// is not "" has its length announced, unless authorization names
// Transfer-Encoding. It returns the answer.
func (s *sender) adminRequest(
from, authorization, method, path, body string, status int, action string,
) answer {
s.t.Helper()
var header []string
if authorization != "" {
header = append(header, "Authorization: "+authorization)
}
if body != "" && !strings.Contains(authorization, "Transfer-Encoding") {
header = append(header, "Content-Length: "+strconv.Itoa(len(body)))
}
_, got := s.requestWithBody(method, from, path, strings.Join(header, "\r\n"),
body, status, action)
return got
}
// wantBans checks that a ban endpoint answered with want, and no other
// ban.
func wantBans(t *testing.T, got answer, want ...state.BanEntry) {
t.Helper()
var decoded struct {
Bans []state.BanEntry `json:"bans"`
}
decode(t, got, &decoded)
gotJSON, err := json.Marshal(decoded.Bans)
if err != nil {
t.Fatalf("encode %+v: %v", decoded.Bans, err)
}
wantJSON, err := json.Marshal(want)
if err != nil {
t.Fatalf("encode %+v: %v", want, err)
}
if string(gotJSON) != string(wantJSON) {
t.Errorf("bans\n%s\nwant\n%s", gotJSON, wantJSON)
}
}
// decode reads the JSON answer of an endpoint into value.
func decode(t *testing.T, got answer, value any) {
t.Helper()
err := json.Unmarshal(got.body, value)
if err != nil {
t.Fatalf("decode %s: %v", got.body, err)
}
}
+7 -7
View File
@@ -53,7 +53,7 @@ func (rq *request) limitBroken(now time.Time) bool {
return true
}
netblock := rq.netblock()
netblock := rq.h.netblock(rq.client)
ban := rq.h.ledger.BanForLimit(netblock, now, bans.Notes{
Country: rq.line.Country,
Limit: hit.Limit,
@@ -71,7 +71,7 @@ func (rq *request) limitBroken(now time.Time) bool {
// banForAttack bans the client's netblock at now for a clear sign of
// attack, the match of rule, a ban rule.
func (rq *request) banForAttack(now time.Time, rule rules.Rule) {
netblock := rq.netblock()
netblock := rq.h.netblock(rq.client)
ban := rq.h.ledger.BanForAttack(netblock, now, bans.Notes{
Country: rq.line.Country,
RuleID: rule.ID,
@@ -102,13 +102,13 @@ func (rq *request) netblockRequests(netblock netip.Prefix) int64 {
return rq.h.limiter.Requests(netblock) + 1
}
// netblock is the netblock a ban on the client covers: its IPv4 address,
// netblock is the netblock a ban on client covers: its IPv4 address,
// widened to SWWAF_BAN_SCOPE_V4_PREFIX, or the IPv6 group clientGroup
// counts it in.
func (rq *request) netblock() netip.Prefix {
addr := rq.client.Unmap()
func (h *handler) netblock(client netip.Addr) netip.Prefix {
addr := client.Unmap()
if addr.Is4() {
return netip.PrefixFrom(addr, rq.h.config.BanScopeV4Prefix).Masked()
return netip.PrefixFrom(addr, h.config.BanScopeV4Prefix).Masked()
}
return clientGroup(addr)
@@ -118,7 +118,7 @@ func (rq *request) netblock() netip.Prefix {
// permanent.
func banExpires(ban bans.Ban) string {
if ban.Permanent() {
return "permanent"
return permanent
}
return requestlog.FormatTime(ban.Expires)
+17 -3
View File
@@ -417,14 +417,28 @@ func (s *sender) requestWithHeader(
) (logLine, string) {
s.t.Helper()
line, got := s.requestWithBody(http.MethodGet, from, path, header, "", status, action)
return line, string(got.body)
}
// requestWithBody is requestWithHeader for a request with method, whose
// body is sent as it is after the headers, header holding its
// Content-Length or Transfer-Encoding. header may hold several lines,
// separated by "\r\n". It returns the whole answer.
func (s *sender) requestWithBody(
method, from, path, header, body string, status int, action string,
) (logLine, answer) {
s.t.Helper()
if header != "" {
header += "\r\n"
}
conn := dial(s.t, s.addr)
send(s.t, conn, "GET "+path+" HTTP/1.1\r\nHost: "+appHost+
send(s.t, conn, method+" "+path+" HTTP/1.1\r\nHost: "+appHost+
"\r\nUser-Agent: "+userAgent+"\r\n"+forwardedFor+": "+from+"\r\n"+
header+"\r\n")
header+"\r\n"+body)
err := conn.SetReadDeadline(time.Now().Add(waitLimit))
if err != nil {
@@ -453,5 +467,5 @@ func (s *sender) requestWithHeader(
s.sent++
wantLine(s.t, line, status, action)
return line, string(got.body)
return line, got
}
+5
View File
@@ -251,6 +251,7 @@ func TestMetricsCountTheBansAnAdminMakes(t *testing.T) {
s, clk, server := startWithClock(t, "", map[string]string{
metricsToken: token,
adminToken: adminSecret,
rateLimitExemptNets: scraper,
})
@@ -265,6 +266,10 @@ func TestMetricsCountTheBansAnAdminMakes(t *testing.T) {
}})
wantMetric(t, s.scrape(scraper), admins, 1)
// And a ban made through the endpoint.
s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK)
wantMetric(t, s.scrape(scraper), admins, 2)
}
func TestMetricsByCountryKeepTheBusiestAndCountTheRestAsOther(t *testing.T) {
+8
View File
@@ -38,6 +38,14 @@ const HealthPath = "/_smallwebwaf/healthz"
// SWWAF_METRICS_TOKEN.
const MetricsPath = "/_smallwebwaf/metrics"
// BansPath is where an admin lists and adds bans, and, followed by / and
// a client's address, lifts them, with SWWAF_ADMIN_TOKEN.
const BansPath = "/_smallwebwaf/bans"
// ClientsPath is where an admin asks what smallwebwaf knows of a client,
// by the client's address after it, with SWWAF_ADMIN_TOKEN.
const ClientsPath = "/_smallwebwaf/clients/"
// Params are what New needs.
type Params struct {
Config *config.Config
+6 -2
View File
@@ -387,10 +387,14 @@ func (rq *request) answer(r refusal) {
}
// refuse records r, unless an earlier refusal was, and ends the request
// to the app.
// to the app, if one was made: smallwebwaf reads the body of a request
// it answers itself too.
func (rq *request) refuse(r refusal) {
rq.refused.CompareAndSwap(nil, &r)
rq.cancel()
if rq.cancel != nil {
rq.cancel()
}
}
// finish ends the request's timeouts, counts it in the metrics and writes
+14
View File
@@ -255,6 +255,20 @@ func (l *Limiter) Requests(netblock netip.Prefix) int64 {
return requests
}
// Client returns client as the table holds it, and whether it does. It is
// not a request from client, and leaves when it was last seen unchanged.
func (l *Limiter) Client(client netip.Prefix) (Client, bool) {
l.mu.Lock()
defer l.mu.Unlock()
c, seen := l.clients.Peek(client)
if !seen {
return Client{}, false
}
return *c, true
}
// Len returns how many clients are in the table.
func (l *Limiter) Len() int {
l.mu.Lock()
+117 -12
View File
@@ -37,6 +37,9 @@ const (
stateCounterInterval = "SWWAF_STATE_COUNTER_INTERVAL"
rateLimitPerDay = "SWWAF_RATE_LIMIT_PER_DAY"
rulesDir = "SWWAF_RULES_DIR"
adminToken = "SWWAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
// adminSecret is the SWWAF_ADMIN_TOKEN the tests set.
adminSecret = "fedcba9876543210fedcba9876543210"
// greeting is what the tests' app answers.
greeting = "hello from the app"
)
@@ -127,25 +130,31 @@ func TestInvalidSettingStopsTheStart(t *testing.T) {
}
}
func TestShortMetricsTokenStopsTheStartUnshown(t *testing.T) {
func TestShortTokenStopsTheStartUnshown(t *testing.T) {
t.Parallel()
const token = "a-token-of-31-characters-at-all" //nolint:gosec // too short to use
out := &output{}
for _, name := range []string{adminToken, "SWWAF_METRICS_TOKEN"} {
t.Run(name, func(t *testing.T) {
t.Parallel()
status := run(t.Context(), map[string]string{"SWWAF_METRICS_TOKEN": token}, out)
if status != 1 {
t.Errorf("exit status %d, want 1", status)
}
out := &output{}
line := out.line(t, "msg", "invalid setting")
if line["error"] != "SWWAF_METRICS_TOKEN: is shorter than 32 characters" {
t.Errorf("start refused with %v", line)
}
status := run(t.Context(), map[string]string{name: token}, out)
if status != 1 {
t.Errorf("exit status %d, want 1", status)
}
if strings.Contains(out.text(), token) {
t.Errorf("the output shows the token:\n%s", out.text())
line := out.line(t, "msg", "invalid setting")
if line["error"] != name+": is shorter than 32 characters" {
t.Errorf("start refused with %v", line)
}
if strings.Contains(out.text(), token) {
t.Errorf("the output shows the token:\n%s", out.text())
}
})
}
}
@@ -329,6 +338,51 @@ func TestBanAddedAndLiftedByEditingBansJSON(t *testing.T) {
})
}
func TestBanAddedAndLiftedThroughTheEndpointsKeptInBansJSON(t *testing.T) {
t.Parallel()
dir := t.TempDir()
env := map[string]string{
listenAddr: localhost + ":0",
upstreamURL: startApp(t),
stateDir: dir,
rulesDir: t.TempDir(),
trustedProxies: localhost + "/32",
adminToken: adminSecret,
// Neither comes due in the test: the files are written as
// smallwebwaf stops.
stateWriteDelay: "1h",
stateCounterInterval: "1h",
}
runUntilStopped(t, env, func(url string) {
askAsAdmin(t, http.MethodPost, url+"_smallwebwaf/bans",
`{"netblock": "203.0.113.0/24", "duration": "permanent", `+
`"reason": "probes for logins"}`)
wantStatus(t, url, "203.0.113.9", http.StatusForbidden)
})
ban := onlyBan(t, dir)
if ban["netblock"] != "203.0.113.0/24" || ban["cause"] != "admin" ||
ban["reason"] != "probes for logins" || ban["expires"] != nil ||
ban["lifted"] != nil {
t.Errorf("bans.json holds %v, want the admin's permanent ban", ban)
}
// After a restart the ban still refuses; once lifted, it refuses no
// more, and bans.json keeps it, marked lifted.
runUntilStopped(t, env, func(url string) {
wantStatus(t, url, "203.0.113.9", http.StatusForbidden)
askAsAdmin(t, http.MethodDelete, url+"_smallwebwaf/bans/203.0.113.9", "")
wantStatus(t, url, "203.0.113.9", http.StatusOK)
})
ban = onlyBan(t, dir)
if ban["netblock"] != "203.0.113.0/24" || ban["lifted"] == nil {
t.Errorf("bans.json holds %v, want the admin's ban, lifted", ban)
}
}
func TestRuleFileAddedWhileRunningTakesEffect(t *testing.T) {
t.Parallel()
@@ -752,6 +806,57 @@ func metricsText(t *testing.T, url, token string) string {
return string(body)
}
// askAsAdmin sends a request with method to url, with body and
// adminSecret, and checks that it is answered 200.
func askAsAdmin(t *testing.T, method, url, body string) {
t.Helper()
req, err := http.NewRequestWithContext(t.Context(), method, url,
strings.NewReader(body))
if err != nil {
t.Fatalf("new request: %v", err)
}
req.Header.Set("Authorization", "Bearer "+adminSecret)
transport := &http.Transport{}
defer transport.CloseIdleConnections()
res, err := (&http.Client{Transport: transport}).Do(req)
if err != nil {
t.Fatalf("request: %v", err)
}
_ = res.Body.Close()
if res.StatusCode != http.StatusOK {
t.Fatalf("%s %s answered %d", method, url, res.StatusCode)
}
}
// onlyBan returns the one ban bans.json in dir holds.
func onlyBan(t *testing.T, dir string) map[string]any {
t.Helper()
path := filepath.Join(dir, "bans.json")
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
if err != nil {
t.Fatalf("read bans.json: %v", err)
}
var file struct {
Bans []map[string]any `json:"bans"`
}
err = json.Unmarshal(data, &file)
if err != nil || len(file.Bans) != 1 {
t.Fatalf("bans.json holds\n%s\nwant one ban (%v)", data, err)
}
return file.Bans[0]
}
// wantRefused checks that a request to url is refused with 403, the
// default SWWAF_BAN_RESPONSE.
func wantRefused(t *testing.T, url string) {
+20 -13
View File
@@ -92,13 +92,14 @@ type Files struct {
// bansFile is bans.json, indented for an admin to read and edit.
type bansFile struct {
Version int `json:"version"`
Bans []banEntry `json:"bans"`
Bans []BanEntry `json:"bans"`
}
// banEntry is a ban as bans.json holds it: a permanent ban's expires is
// BanEntry is a ban as bans.json holds it: a permanent ban's expires is
// null, a ban an admin added may have no cause, which makes it an
// admin's, and lifted is left out until an admin lifts the ban.
type banEntry struct {
// admin's, and lifted is left out until an admin lifts the ban. The ban
// endpoints answer with bans in this form too.
type BanEntry struct {
Netblock netip.Prefix `json:"netblock"`
Start time.Time `json:"start"`
Expires *time.Time `json:"expires"`
@@ -434,12 +435,7 @@ func (f *Files) setAside(name string, parseErr error) error {
func (f *Files) encode(name string) ([]byte, error) {
switch name {
case bansJSON:
held := f.params.Ledger.Snapshot()
file := bansFile{Version: version, Bans: make([]banEntry, 0, len(held))}
for _, ban := range held {
file.Bans = append(file.Bans, newBanEntry(ban))
}
file := bansFile{Version: version, Bans: BanEntries(f.params.Ledger.Snapshot())}
data, err := json.MarshalIndent(file, "", " ")
if err != nil {
@@ -454,9 +450,20 @@ func (f *Files) encode(name string) ([]byte, error) {
}
}
// BanEntries returns held as bans.json lists them, an empty list for
// none.
func BanEntries(held []bans.Ban) []BanEntry {
entries := make([]BanEntry, 0, len(held))
for _, ban := range held {
entries = append(entries, newBanEntry(ban))
}
return entries
}
// newBanEntry returns ban as bans.json holds it.
func newBanEntry(ban bans.Ban) banEntry {
entry := banEntry{
func newBanEntry(ban bans.Ban) BanEntry {
entry := BanEntry{
Netblock: ban.Netblock, Start: ban.Start, Cause: ban.Cause, Reason: ban.Reason,
Notes: ban.Notes,
}
@@ -472,7 +479,7 @@ func newBanEntry(ban bans.Ban) banEntry {
}
// ban returns the ban an entry of bans.json holds.
func (e banEntry) ban() bans.Ban {
func (e BanEntry) ban() bans.Ban {
ban := bans.Ban{
Netblock: e.Netblock, Start: e.Start, Cause: e.Cause, Reason: e.Reason,
Notes: e.Notes,