Compare commits

..
1 Commits
Author SHA1 Message Date
clawbot 638f355080 Re-vendor the canonical files from sneak/prompts at dd4027b (closes #65)
check / check (push) Successful in 4m10s
The vendored files are fetched from sneak/prompts commit dd4027b, with
this repository's /bin carried forward in .dockerignore; the
test-support deny list has no entries of its own. The lint phase moves
to golangci-lint v2.14.0. The build stage now takes the version from
git describe on the .git the build context carries, unless VERSION is
passed, and fails when .git is present but no version comes out. The
test phase drops -count=1, which the policy says it does not need, and
keeps its tmpfs build cache. One test calls Header.Get with X-Real-IP,
as canonicalheader asks.

Deviation: the Go lines of .gitignore and .editorconfig are dropped;
they are not exempt from byte-identity.

Model: opus-5-5
2026-10-05 23:59:29 +00:00
12 changed files with 55 additions and 359 deletions
-3
View File
@@ -10,6 +10,3 @@ insert_final_newline = true
[Makefile] [Makefile]
indent_style = tab indent_style = tab
[*.go]
indent_style = tab
-6
View File
@@ -45,9 +45,3 @@ node_modules/
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK] [iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
[iI][dD]_[eE][dD]25519 [iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK] [iI][dD]_[eE][dD]25519_[sS][kK]
# Go: the binary `make build` writes, test binaries, profiles and logs
/bin/
*.test
*.out
*.log
+36 -53
View File
@@ -13,15 +13,14 @@ JSON log line for every request.
Status: the first two milestones are built Status: the first two milestones are built
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and (https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are the static lists, https://git.eeqj.de/sneak/smallwebwaf/issues/14). `smallwebwaf` passes each
which come next in the build order. `smallwebwaf` passes each request to the app request to the app and the app's answer back, unchanged, within its timeouts and
and the app's answer back, unchanged, within its timeouts and size limits, works size limits, works out each client's address, refuses a client that sends too
out each client's address, refuses a client that sends too many requests, comes many requests or comes from a country you refuse, and writes a JSON log line for
from a country you refuse or from a network you refuse, lets the networks you every request. It comes as the image the app's own image is built on. The rest
choose through, and writes a JSON log line for every request. It comes as the of the design comes after that, in the order of the build order in
image the app's own image is built on. The rest of the design comes after that, [`SPEC.md`](SPEC.md). The survey of existing tools that led to the design is in
in the order of the build order in [`SPEC.md`](SPEC.md). The survey of existing [`EVALUATION.md`](EVALUATION.md).
tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started ## Getting started
@@ -83,17 +82,8 @@ and `make run` builds and runs it, listening on port 8080 in front of an app at
counted for the rate limits. While one of the country lists below is set, each counted for the rate limits. While one of the country lists below is set, each
client's country is looked up through GeoJS (see "Country and AS number client's country is looked up through GeoJS (see "Country and AS number
lookup" below); with neither set, no visitor's address leaves the host. A lookup" below); with neither set, no visitor's address leaves the host. A
client on a private, loopback or link-local address has no country and is client on a private, loopback or link-local address has no country, and
never looked up: `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless it is neither list checks it.
in `SWWAF_ALLOW_NETS`, and `SWWAF_DENIED_COUNTRIES` does not refuse it.
- Checks the client's own address against the static lists, the three netblock
settings below, before anything else, its country included. A client in
`SWWAF_ALLOW_NETS` skips the country lists and the rate limits, and is not
looked up; the timeouts and size limits still apply. A client in
`SWWAF_DENY_NETS` is refused with `403` before its body is read, and the
request is not counted for the rate limits; an address in `SWWAF_ALLOW_NETS`
too is let through. A client in `SWWAF_RATE_LIMIT_EXEMPT_NETS` is neither
counted nor refused by the rate limits; the country lists still apply to it.
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any - Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
check and without asking the app, for the image's health check. check and without asking the app, for the image's health check.
- Writes a line in the request log for each request (see "Request log" below). - Writes a line in the request log for each request (see "Request log" below).
@@ -121,11 +111,6 @@ it, and the effective settings are logged at start.
to send its whole answer, from the end of the request to the last byte. to send its whole answer, from the end of the request to the last byte.
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. - `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body.
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. - `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
- `SWWAF_ALLOW_NETS` (default empty): netblocks whose clients skip the country
lists and the rate limits, such as your monitoring or your own networks.
- `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients the
rate limits do not apply to, such as a machine that talks to the app all day.
- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR` - `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
(default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
requests a client may make in a minute, an hour and a day. The defaults are requests a client may make in a minute, an hour and a day. The defaults are
@@ -168,19 +153,18 @@ refused ones included:
- `time` is when the request arrived, in UTC. `peer_ip` is the TCP peer, - `time` is when the request arrived, in UTC. `peer_ip` is the TCP peer,
normally traefik. `path` and `query` are as the client sent them. normally traefik. `path` and `query` are as the client sent them.
- `country` is the client's country as GeoJS places it, and empty when it is not - `country` is the client's country as GeoJS places it, and empty when it is not
known: with neither country list set, for a client in `SWWAF_ALLOW_NETS` or known: with neither country list set, for a client on a private, loopback or
`SWWAF_DENY_NETS`, for a client on a private, loopback or link-local address, link-local address, and when GeoJS cannot place the client or has not answered
and when GeoJS cannot place the client or has not answered in time. in time.
- `status` is what the client was sent, `0` if nothing was; `upstream_status` is - `status` is what the client was sent, `0` if nothing was; `upstream_status` is
what the app answered, and is left out when the app did not answer. what the app answered, and is left out when the app did not answer.
- `request_bytes` and `response_bytes` count body bytes. - `request_bytes` and `response_bytes` count body bytes.
- `action` is `forward` for a request passed to the app, `denied` for one - `action` is `forward` for a request passed to the app, `country_denied` for
refused because its client is in `SWWAF_DENY_NETS`, `country_denied` for one one refused for its client's country, `rate_limited` for one refused for a
refused for its client's country, `rate_limited` for one refused for a rate rate limit, `too_large` for a request or response over its size limit,
limit, `too_large` for a request or response over its size limit, `timed_out` `timed_out` for one that ran out of time, `upstream_error` when the app could
for one that ran out of time, `upstream_error` when the app could not be not be reached or its answer broke off, and `admin` for one `smallwebwaf`
reached or its answer broke off, and `admin` for one `smallwebwaf` answered at answered at its own endpoint.
its own endpoint.
- `limit_hit` is there for a request refused for a rate limit, and names the - `limit_hit` is there for a request refused for a rate limit, and names the
window whose limit it went over: `minute`, `hour` or `day`, the shortest if it window whose limit it went over: `minute`, `hour` or `day`, the shortest if it
went over several. went over several.
@@ -418,17 +402,16 @@ the metrics, failure behaviour and the build order.
So far `smallwebwaf` looks up only the country, only through GeoJS, and only So far `smallwebwaf` looks up only the country, only through GeoJS, and only
while `SWWAF_DENIED_COUNTRIES` or `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set: while `SWWAF_DENIED_COUNTRIES` or `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set:
then the address of every new visitor outside `SWWAF_ALLOW_NETS` and then the address of every new visitor is sent to GeoJS, and with neither set,
`SWWAF_DENY_NETS` is sent to GeoJS, and with neither set, none is. An IPv6 none is. An IPv6 visitor is asked about by the first address of its /64. A new
visitor is asked about by the first address of its /64. A new visitor waits at visitor waits at most a second for its answer, and without one counts as coming
most a second for its answer, and without one counts as coming from an unknown from an unknown country until the answer arrives. The addresses waiting are
country until the answer arrives. The addresses waiting are asked about asked about together, up to 200 in one request, one request at a time; at most
together, up to 200 in one request, one request at a time; at most 10,000 10,000 visitors wait, and one more counts as coming from an unknown country
visitors wait, and one more counts as coming from an unknown country until there until there is room. While GeoJS fails, visitors with a kept answer are
is room. While GeoJS fails, visitors with a kept answer are unaffected and new unaffected and new ones count as coming from an unknown country. GeoJS is then
ones count as coming from an unknown country. GeoJS is then left alone for a left alone for a second, twice as long after each further failure up to five
second, twice as long after each further failure up to five minutes, and asked minutes, and asked again by the next request that needs it.
again by the next request that needs it.
In the full design, `smallwebwaf` looks up the AS number and country of every In the full design, `smallwebwaf` looks up the AS number and country of every
client, for the request log, the metrics and the ban notes, and for the country client, for the request log, the metrics and the ban notes, and for the country
@@ -464,8 +447,9 @@ data is powered by IPinfo". A service that uses the database through
Neither source can place a private address, so a client on one, such as a Neither source can place a private address, so a client on one, such as a
visitor on your local network, another container or your monitoring, has no visitor on your local network, another container or your monitoring, has no
country: `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless you list it in country: `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless you list it in
`SWWAF_ALLOW_NETS`, and `SWWAF_DENIED_COUNTRIES` does not refuse it. Such `SWWAF_ALLOW_NETS`. Such addresses are never sent to GeoJS. In milestone 2,
addresses are never sent to GeoJS. which has no `SWWAF_ALLOW_NETS`, neither country list checks such a client; the
refusal comes with `SWWAF_ALLOW_NETS` in milestone 3 or later.
## How the code is laid out ## How the code is laid out
@@ -478,9 +462,8 @@ addresses are never sent to GeoJS.
the checks, passes the request to the app and the answer back with the the checks, passes the request to the app and the answer back with the
standard library's `httputil.ReverseProxy` within the timeouts and size standard library's `httputil.ReverseProxy` within the timeouts and size
limits, and writes the request's log line. Its `check` method is where a limits, and writes the request's log line. Its `check` method is where a
request is refused before anything reaches the app: for `SWWAF_DENY_NETS`, for request is refused before anything reaches the app: for the country lists, for
the country lists, for a rate limit, and for an announced body over the size a rate limit, and for an announced body over the size limit.
limit.
- `internal/lookup`: looks up each client's country through GeoJS, and keeps the - `internal/lookup`: looks up each client's country through GeoJS, and keeps the
answers. answers.
- `internal/ratelimit`: counts each client's requests and tells when one takes - `internal/ratelimit`: counts each client's requests and tells when one takes
@@ -535,8 +518,8 @@ so that they run in minimal containers.
## TODO ## TODO
- The rest of milestone 3, after the static lists, and the rest of the design, - Milestone 3 and the rest of the design, in the order of the build order in
in the order of the build order in [`SPEC.md`](SPEC.md). [`SPEC.md`](SPEC.md).
## Documents ## Documents
-11
View File
@@ -45,14 +45,6 @@ type Config struct {
// ResponseMaxBytes is the largest response body // ResponseMaxBytes is the largest response body
// (SWWAF_RESPONSE_MAX_BYTES). // (SWWAF_RESPONSE_MAX_BYTES).
ResponseMaxBytes int64 ResponseMaxBytes int64
// AllowNets are the netblocks whose clients skip every check
// (SWWAF_ALLOW_NETS). RateLimitExemptNets are those whose clients the
// rate limits neither count nor refuse (SWWAF_RATE_LIMIT_EXEMPT_NETS).
// DenyNets are those whose clients are always refused
// (SWWAF_DENY_NETS).
AllowNets []netip.Prefix
RateLimitExemptNets []netip.Prefix
DenyNets []netip.Prefix
// RateLimitPerMinute, RateLimitPerHour and RateLimitPerDay are the // RateLimitPerMinute, RateLimitPerHour and RateLimitPerDay are the
// most requests a client may make in a minute, an hour and a day // most requests a client may make in a minute, an hour and a day
// (SWWAF_RATE_LIMIT_PER_MINUTE, SWWAF_RATE_LIMIT_PER_HOUR and // (SWWAF_RATE_LIMIT_PER_MINUTE, SWWAF_RATE_LIMIT_PER_HOUR and
@@ -120,9 +112,6 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
UpstreamResponseTimeout: env.duration("SWWAF_UPSTREAM_RESPONSE_TIMEOUT", "30m"), UpstreamResponseTimeout: env.duration("SWWAF_UPSTREAM_RESPONSE_TIMEOUT", "30m"),
RequestMaxBytes: env.size("SWWAF_REQUEST_MAX_BYTES", "100M"), RequestMaxBytes: env.size("SWWAF_REQUEST_MAX_BYTES", "100M"),
ResponseMaxBytes: env.size("SWWAF_RESPONSE_MAX_BYTES", "5G"), ResponseMaxBytes: env.size("SWWAF_RESPONSE_MAX_BYTES", "5G"),
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "1000"), RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "1000"),
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "10000"), RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "10000"),
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "50000"), RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "50000"),
-18
View File
@@ -25,9 +25,6 @@ const (
upstreamResponseTimeout = "SWWAF_UPSTREAM_RESPONSE_TIMEOUT" upstreamResponseTimeout = "SWWAF_UPSTREAM_RESPONSE_TIMEOUT"
requestMaxBytes = "SWWAF_REQUEST_MAX_BYTES" requestMaxBytes = "SWWAF_REQUEST_MAX_BYTES"
responseMaxBytes = "SWWAF_RESPONSE_MAX_BYTES" responseMaxBytes = "SWWAF_RESPONSE_MAX_BYTES"
allowNets = "SWWAF_ALLOW_NETS"
rateLimitExemptNets = "SWWAF_RATE_LIMIT_EXEMPT_NETS"
denyNets = "SWWAF_DENY_NETS"
rateLimitPerMinute = "SWWAF_RATE_LIMIT_PER_MINUTE" rateLimitPerMinute = "SWWAF_RATE_LIMIT_PER_MINUTE"
rateLimitPerHour = "SWWAF_RATE_LIMIT_PER_HOUR" rateLimitPerHour = "SWWAF_RATE_LIMIT_PER_HOUR"
rateLimitPerDay = "SWWAF_RATE_LIMIT_PER_DAY" rateLimitPerDay = "SWWAF_RATE_LIMIT_PER_DAY"
@@ -84,9 +81,6 @@ func TestDefaults(t *testing.T) {
wantNetblocks(t, cfg.TrustedProxies, wantNetblocks(t, cfg.TrustedProxies,
"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16") "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16")
wantNetblocks(t, cfg.AllowNets)
wantNetblocks(t, cfg.RateLimitExemptNets)
wantNetblocks(t, cfg.DenyNets)
wantCountries(t, deniedCountries, cfg.DeniedCountries) wantCountries(t, deniedCountries, cfg.DeniedCountries)
wantCountries(t, allowedCountries, cfg.ExclusivelyAllowedCountries) wantCountries(t, allowedCountries, cfg.ExclusivelyAllowedCountries)
} }
@@ -104,9 +98,6 @@ func TestValuesAsSet(t *testing.T) {
upstreamResponseTimeout: off, upstreamResponseTimeout: off,
requestMaxBytes: "512K", requestMaxBytes: "512K",
responseMaxBytes: "1234", responseMaxBytes: "1234",
allowNets: "192.0.2.7",
rateLimitExemptNets: "2001:db8::/48, 10.9.8.7",
denyNets: "198.51.100.0/24",
rateLimitPerMinute: "60", rateLimitPerMinute: "60",
rateLimitPerHour: "600", rateLimitPerHour: "600",
rateLimitPerDay: "6000", rateLimitPerDay: "6000",
@@ -132,9 +123,6 @@ func TestValuesAsSet(t *testing.T) {
} }
wantNetblocks(t, cfg.TrustedProxies, "192.0.2.1/32", "10.0.0.0/8", "2001:db8::/32") wantNetblocks(t, cfg.TrustedProxies, "192.0.2.1/32", "10.0.0.0/8", "2001:db8::/32")
wantNetblocks(t, cfg.AllowNets, "192.0.2.7/32")
wantNetblocks(t, cfg.RateLimitExemptNets, "2001:db8::/48", "10.9.8.7/32")
wantNetblocks(t, cfg.DenyNets, "198.51.100.0/24")
wantCountries(t, deniedCountries, cfg.DeniedCountries, "CN", "RU", "KP", "XK") wantCountries(t, deniedCountries, cfg.DeniedCountries, "CN", "RU", "KP", "XK")
wantCountries(t, allowedCountries, cfg.ExclusivelyAllowedCountries, "DE") wantCountries(t, allowedCountries, cfg.ExclusivelyAllowedCountries, "DE")
} }
@@ -217,9 +205,6 @@ func TestInvalidValueStopsTheStart(t *testing.T) {
{trustedProxies, "traefik"}, {trustedProxies, "traefik"},
{trustedProxies, "10.0.0.0/8,,192.168.0.0/16"}, {trustedProxies, "10.0.0.0/8,,192.168.0.0/16"},
{trustedProxies, "fe80::1%eth0"}, {trustedProxies, "fe80::1%eth0"},
{allowNets, "192.0.2.0/24,monitoring"},
{rateLimitExemptNets, "2001:db8::/129"},
{denyNets, "198.51.100.0/24,"},
{clientRequestTimeout, "60"}, {clientRequestTimeout, "60"},
{clientRequestTimeout, ""}, {clientRequestTimeout, ""},
{clientResponseTimeout, "1y"}, {clientResponseTimeout, "1y"},
@@ -294,9 +279,6 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
upstreamResponseTimeout: "30m", upstreamResponseTimeout: "30m",
requestMaxBytes: "100M", requestMaxBytes: "100M",
responseMaxBytes: "5G", responseMaxBytes: "5G",
allowNets: "",
rateLimitExemptNets: "",
denyNets: "",
rateLimitPerMinute: "1000", rateLimitPerMinute: "1000",
rateLimitPerHour: "10000", rateLimitPerHour: "10000",
rateLimitPerDay: "50000", rateLimitPerDay: "50000",
+6 -6
View File
@@ -9,9 +9,9 @@ import (
// countryDenied reports whether the country lists refuse the request. // countryDenied reports whether the country lists refuse the request.
// The client's country is looked up only while a list is set, and never // The client's country is looked up only while a list is set, and never
// for a client on a private, loopback or link-local address, which has // for a client on a private, loopback or link-local address, which has
// no country. A client without a country, or whose country cannot be // no country and which neither list checks. A client whose country
// found, is refused only by SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES. ctx is // cannot be found is refused only by SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
// the request's own context. // ctx is the request's own context.
func (rq *request) countryDenied(ctx context.Context) bool { func (rq *request) countryDenied(ctx context.Context) bool {
denied := rq.h.config.DeniedCountries denied := rq.h.config.DeniedCountries
allowed := rq.h.config.ExclusivelyAllowedCountries allowed := rq.h.config.ExclusivelyAllowedCountries
@@ -20,11 +20,11 @@ func (rq *request) countryDenied(ctx context.Context) bool {
return false return false
} }
var country string if !hasCountry(rq.client) {
if hasCountry(rq.client) { return false
country = rq.h.geojs.Country(ctx, clientGroup(rq.client))
} }
country := rq.h.geojs.Country(ctx, clientGroup(rq.client))
rq.line.Country = country rq.line.Country = country
if slices.Contains(denied, country) { if slices.Contains(denied, country) {
+1 -42
View File
@@ -185,7 +185,7 @@ func TestCountryNotLookedUpWithoutAListOrForAPrivateAddress(t *testing.T) {
{"no country list is set", nil, []string{fromKP, fromDE}}, {"no country list is set", nil, []string{fromKP, fromDE}},
{ {
"private, loopback and link-local addresses", "private, loopback and link-local addresses",
map[string]string{deniedCountries: "kp"}, map[string]string{allowedCountries: "de"},
[]string{"10.0.0.5", "192.168.1.9", "fd00::5", "", "169.254.0.9", "fe80::9"}, []string{"10.0.0.5", "192.168.1.9", "fd00::5", "", "169.254.0.9", "fe80::9"},
}, },
} { } {
@@ -223,47 +223,6 @@ func TestCountryNotLookedUpWithoutAListOrForAPrivateAddress(t *testing.T) {
} }
} }
func TestExclusiveListRefusesAPrivateAddressUnlessAllowed(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name string
allowNets string
status int
action string
}{
{
"not in SWWAF_ALLOW_NETS", "",
http.StatusForbidden, requestlog.ActionCountryDenied,
},
{
"in SWWAF_ALLOW_NETS", "10.0.0.7,fd00::/8",
http.StatusOK, requestlog.ActionForward,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
geojsURL, asked := startGeoJS(t)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
trustedProxies: trustLocalhost,
allowedCountries: "de",
allowNets: tc.allowNets,
})
wantAnswers(t, addr, out, []sentRequest{
{"10.0.0.7", tc.status, tc.action},
{"fd00::5", tc.status, tc.action},
})
if len(asked()) != 0 {
t.Errorf("GeoJS was asked about %v, want nothing", asked())
}
})
}
}
// startGeoJS starts a stand-in for GeoJS, which places fromDE and fromKP // startGeoJS starts a stand-in for GeoJS, which places fromDE and fromKP
// and no other address. It returns its URL, and what returns the // and no other address. It returns its URL, and what returns the
// addresses it has been asked about. // addresses it has been asked about.
-3
View File
@@ -51,9 +51,6 @@ const (
requestMaxBytes = "SWWAF_REQUEST_MAX_BYTES" requestMaxBytes = "SWWAF_REQUEST_MAX_BYTES"
responseMaxBytes = "SWWAF_RESPONSE_MAX_BYTES" responseMaxBytes = "SWWAF_RESPONSE_MAX_BYTES"
trustedProxies = "SWWAF_TRUSTED_PROXIES" trustedProxies = "SWWAF_TRUSTED_PROXIES"
allowNets = "SWWAF_ALLOW_NETS"
rateLimitExemptNets = "SWWAF_RATE_LIMIT_EXEMPT_NETS"
denyNets = "SWWAF_DENY_NETS"
rateLimitPerMinute = "SWWAF_RATE_LIMIT_PER_MINUTE" rateLimitPerMinute = "SWWAF_RATE_LIMIT_PER_MINUTE"
deniedCountries = "SWWAF_DENIED_COUNTRIES" deniedCountries = "SWWAF_DENIED_COUNTRIES"
allowedCountries = "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES" allowedCountries = "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES"
+12 -27
View File
@@ -103,44 +103,29 @@ func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
// check is the one place where a request can be refused once its client // check is the one place where a request can be refused once its client
// is known, before its body is read or anything reaches the app. It // is known, before its body is read or anything reaches the app. It
// returns nil to let the request through. A client in SWWAF_ALLOW_NETS // returns nil to let the request through. The country lists come first,
// skips every check but the size limit. For any other client, // and a request they refuse is not counted for the rate limits; then the
// SWWAF_DENY_NETS comes first, so that a client it refuses is not looked // rate limits, so that every other request is counted, one refused for
// up, and then the country lists; a request either refuses is not counted // its size too. ctx is the request's own context.
// for the rate limits. Then come the rate limits, unless the client is in
// SWWAF_RATE_LIMIT_EXEMPT_NETS, so that every other request is counted,
// one refused for its size too. ctx is the request's own context.
func (rq *request) check(ctx context.Context) *refusal { func (rq *request) check(ctx context.Context) *refusal {
cfg := rq.h.config if rq.countryDenied(ctx) {
allowed := isInside(rq.client, cfg.AllowNets)
if !allowed && isInside(rq.client, cfg.DenyNets) {
return &refusal{
status: http.StatusForbidden,
action: requestlog.ActionDenied,
}
}
if !allowed && rq.countryDenied(ctx) {
return &refusal{ return &refusal{
status: http.StatusForbidden, status: http.StatusForbidden,
action: requestlog.ActionCountryDenied, action: requestlog.ActionCountryDenied,
} }
} }
if !allowed && !isInside(rq.client, cfg.RateLimitExemptNets) { limitHit := rq.h.limiter.Count(clientGroup(rq.client), rq.start)
limitHit := rq.h.limiter.Count(clientGroup(rq.client), rq.start) if limitHit != "" {
if limitHit != "" { rq.line.LimitHit = limitHit
rq.line.LimitHit = limitHit
return &refusal{ return &refusal{
status: http.StatusTooManyRequests, status: http.StatusTooManyRequests,
action: requestlog.ActionRateLimited, action: requestlog.ActionRateLimited,
}
} }
} }
maxBytes := cfg.RequestMaxBytes maxBytes := rq.h.config.RequestMaxBytes
if maxBytes > 0 && rq.in.ContentLength > maxBytes { if maxBytes > 0 && rq.in.ContentLength > maxBytes {
return &refusal{ return &refusal{
status: http.StatusRequestEntityTooLarge, status: http.StatusRequestEntityTooLarge,
-184
View File
@@ -1,184 +0,0 @@
package proxy_test
import (
"net/http"
"strings"
"sync/atomic"
"testing"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
)
// The rate limits count an IPv6 client by its /64, so these two addresses
// are one client for them. The static lists match each address on its own,
// and the tests list listedAddr alone.
const (
listedAddr = "2001:db8::1"
unlistedAddr = "2001:db8::2"
)
func TestAllowNetsSkipEveryCheckButTheSizeLimit(t *testing.T) {
t.Parallel()
var calls atomic.Int32
app := startApp(t, func(http.ResponseWriter, *http.Request) {
calls.Add(1)
})
geojsURL, asked := startGeoJS(t)
// fromKP is in SWWAF_ALLOW_NETS, and in SWWAF_DENY_NETS too, which
// comes after it.
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
trustedProxies: trustLocalhost,
allowNets: "198.51.100.0/24",
denyNets: fromKP,
deniedCountries: "kp",
rateLimitPerMinute: "1",
requestMaxBytes: "1K",
})
// Neither SWWAF_DENY_NETS, the country lists nor the limit of one
// request a minute refuses the client, and its country is not looked
// up.
wantAnswers(t, addr, out, []sentRequest{
{fromKP, http.StatusOK, requestlog.ActionForward},
{fromKP, http.StatusOK, requestlog.ActionForward},
})
if len(asked()) != 0 {
t.Errorf("GeoJS was asked about %v, want nothing", asked())
}
// The size limit still applies.
body := strings.NewReader(strings.Repeat("a", 2<<10))
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set(forwardedFor, fromKP)
wantStatus(t, do(t, req), http.StatusRequestEntityTooLarge)
wantLine(t, out.requestLines(t, 3)[2],
http.StatusRequestEntityTooLarge, requestlog.ActionTooLarge)
if calls.Load() != 2 {
t.Errorf("the app was called %d times, want 2", calls.Load())
}
}
func TestRequestFromAllowNetsIsNotCounted(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out := startProxy(t, app.URL, map[string]string{
trustedProxies: trustLocalhost,
allowNets: listedAddr,
rateLimitPerMinute: "1",
})
// listedAddr's requests are not counted, so the first request from
// unlistedAddr is within the limit of one a minute.
wantAnswers(t, addr, out, []sentRequest{
{listedAddr, http.StatusOK, requestlog.ActionForward},
{listedAddr, http.StatusOK, requestlog.ActionForward},
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
{unlistedAddr, http.StatusTooManyRequests, requestlog.ActionRateLimited},
})
}
func TestDenyNetsRefuseBeforeTheLookupAndTheBody(t *testing.T) {
t.Parallel()
var calls atomic.Int32
app := startApp(t, func(http.ResponseWriter, *http.Request) {
calls.Add(1)
})
geojsURL, asked := startGeoJS(t)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
trustedProxies: trustLocalhost,
denyNets: "203.0.113.0/24",
deniedCountries: "kp",
})
req := newRequest(t, http.MethodPost, addr, "/", strings.NewReader("a body"))
req.Header.Set(forwardedFor, fromDE)
wantStatus(t, do(t, req), http.StatusForbidden)
line := out.requestLine(t)
wantLine(t, line, http.StatusForbidden, requestlog.ActionDenied)
if line.RequestBytes != 0 {
t.Errorf("log line has request_bytes %d, want 0", line.RequestBytes)
}
if len(asked()) != 0 {
t.Errorf("GeoJS was asked about %v, want nothing", asked())
}
if calls.Load() != 0 {
t.Errorf("the app was called %d times, want none", calls.Load())
}
}
func TestRequestRefusedByDenyNetsIsNotCounted(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out := startProxy(t, app.URL, map[string]string{
trustedProxies: trustLocalhost,
denyNets: listedAddr,
rateLimitPerMinute: "1",
})
// listedAddr's refused requests are not counted, so the first request
// from unlistedAddr is within the limit of one a minute.
wantAnswers(t, addr, out, []sentRequest{
{listedAddr, http.StatusForbidden, requestlog.ActionDenied},
{listedAddr, http.StatusForbidden, requestlog.ActionDenied},
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
{unlistedAddr, http.StatusTooManyRequests, requestlog.ActionRateLimited},
})
}
func TestRateLimitExemptNetsAreNeitherCountedNorRefused(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
geojsURL, _ := startGeoJS(t)
addr, out := startProxyWithGeoJS(t, app.URL, geojsURL, map[string]string{
trustedProxies: trustLocalhost,
rateLimitExemptNets: listedAddr + "," + fromKP,
deniedCountries: "kp",
rateLimitPerMinute: "1",
})
// listedAddr's requests are neither refused nor counted, so the first
// request from unlistedAddr is within the limit of one a minute. The
// country lists still refuse an exempt client.
wantAnswers(t, addr, out, []sentRequest{
{listedAddr, http.StatusOK, requestlog.ActionForward},
{listedAddr, http.StatusOK, requestlog.ActionForward},
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
{unlistedAddr, http.StatusTooManyRequests, requestlog.ActionRateLimited},
{fromKP, http.StatusForbidden, requestlog.ActionCountryDenied},
})
}
// sentRequest is a GET request from client, as X-Forwarded-For names it,
// and the status and log line action it should get.
type sentRequest struct {
client string
status int
action string
}
// wantAnswers sends requests to smallwebwaf at addr one after another and
// checks each one's answer and log line. They must be the first requests
// smallwebwaf is sent, since the log lines are matched to them in order.
func wantAnswers(t *testing.T, addr string, out *output, requests []sentRequest) {
t.Helper()
for i, sent := range requests {
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedFor, sent.client)
wantStatus(t, do(t, req), sent.status)
wantLine(t, out.requestLines(t, i+1)[i], sent.status, sent.action)
}
}
-3
View File
@@ -26,9 +26,6 @@ const (
// ActionRateLimited is a request refused because it took its client // ActionRateLimited is a request refused because it took its client
// over a rate limit, or came while the client was over one. // over a rate limit, or came while the client was over one.
ActionRateLimited = "rate_limited" ActionRateLimited = "rate_limited"
// ActionDenied is a request refused because its client is in
// SWWAF_DENY_NETS.
ActionDenied = "denied"
// ActionCountryDenied is a request refused for its client's country. // ActionCountryDenied is a request refused for its client's country.
ActionCountryDenied = "country_denied" ActionCountryDenied = "country_denied"
// ActionAdmin is a request smallwebwaf answered at one of its own // ActionAdmin is a request smallwebwaf answered at one of its own
-3
View File
@@ -186,9 +186,6 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL string) {
"SWWAF_UPSTREAM_RESPONSE_TIMEOUT": "30m", "SWWAF_UPSTREAM_RESPONSE_TIMEOUT": "30m",
"SWWAF_REQUEST_MAX_BYTES": "100M", "SWWAF_REQUEST_MAX_BYTES": "100M",
"SWWAF_RESPONSE_MAX_BYTES": "5G", "SWWAF_RESPONSE_MAX_BYTES": "5G",
"SWWAF_ALLOW_NETS": "",
"SWWAF_RATE_LIMIT_EXEMPT_NETS": "",
"SWWAF_DENY_NETS": "",
"SWWAF_RATE_LIMIT_PER_MINUTE": "1000", "SWWAF_RATE_LIMIT_PER_MINUTE": "1000",
"SWWAF_RATE_LIMIT_PER_HOUR": "10000", "SWWAF_RATE_LIMIT_PER_HOUR": "10000",
"SWWAF_RATE_LIMIT_PER_DAY": "50000", "SWWAF_RATE_LIMIT_PER_DAY": "50000",