From 4a2d57dca2b16a489d411cf76d9c8a03a02819ac Mon Sep 17 00:00:00 2001 From: sneak Date: Wed, 4 Nov 2026 19:41:12 +0000 Subject: [PATCH] servewaf answers rproxy's question about each request and takes its report (closes #170) `smallwebwaf servewaf` runs today's checks on the visitor's request each question carries, answers rproxy, and passes nothing to an app. rproxy reports every request servewaf answered. A report counts an allowed request's bytes and writes servewaf's second log line. The table holds a copy of what a report needs, never the request, so a report cannot reach a request being handled. Judgement call: a question's body is read before any check; over 64 KiB gets 400. Judgement call: a report's bytes and durations have ceilings, so they cannot overflow. Judgement call: a `close` disposition for SWWAF_BAN_RESPONSE=close. Model: opus-5-5 --- README.md | 5 + SPEC.md | 210 ++- docs/admin-endpoints.md | 3 +- docs/alerts.md | 5 +- docs/metrics.md | 9 + docs/request-log.md | 62 + docs/settings.md | 20 +- internal/alerts/alerts.go | 15 +- internal/alerts/alerts_test.go | 4 +- internal/anomaly/anomaly.go | 3 + internal/metrics/metrics.go | 60 +- internal/proxy/admin.go | 9 + internal/proxy/anomaly_internal_test.go | 8 +- internal/proxy/bans.go | 9 +- internal/proxy/bans_test.go | 4 +- internal/proxy/coreruleset.go | 1 + internal/proxy/metrics_test.go | 2 +- internal/proxy/proxy.go | 85 +- internal/proxy/proxy_test.go | 12 +- internal/proxy/reputation.go | 1 + internal/proxy/request.go | 152 ++- internal/proxy/servewaf.go | 464 +++++++ internal/proxy/servewaf_test.go | 1511 ++++++++++++++++++++++ internal/requestlog/requestlog.go | 17 +- internal/smallwebwaf/servewaf_test.go | 210 +++ internal/smallwebwaf/smallwebwaf.go | 29 +- internal/smallwebwaf/smallwebwaf_test.go | 10 +- 27 files changed, 2779 insertions(+), 141 deletions(-) create mode 100644 internal/proxy/servewaf.go create mode 100644 internal/proxy/servewaf_test.go create mode 100644 internal/smallwebwaf/servewaf_test.go diff --git a/README.md b/README.md index 712ad5a..c37a8c8 100644 --- a/README.md +++ b/README.md @@ -339,6 +339,9 @@ pinned. [`docs/settings.md`](docs/settings.md)). - The state files live in `/var/lib/smallwebwaf`. A volume mounted there keeps them across a deploy. +- `smallwebwaf servewaf` runs the same checks for the sites that ask it about + each request, and passes nothing to an app (see "`servewaf`" in + [`SPEC.md`](SPEC.md)). - `docker stop` stops both processes at once. `smallwebwaf` gives the requests in progress five seconds to finish, so the app must keep answering for five seconds after `SIGTERM`. @@ -351,6 +354,8 @@ what each does. ## TODO - Planned, as "Planned, not built" in [`SPEC.md`](SPEC.md) describes: + - `smallwebwaf rproxy`, the reverse proxy in front of each app that asks + `smallwebwaf servewaf` about each request; - sharing bans between the instances of several apps; - sending the log over RELP. diff --git a/SPEC.md b/SPEC.md index 0b3d7b6..bd78b5a 100644 --- a/SPEC.md +++ b/SPEC.md @@ -47,14 +47,16 @@ to them. - `internal/smallwebwaf`: the process. It reads the settings, the rule files, the lookup database and the state files, listens, and serves until `SIGTERM` or `SIGINT`. Then it stops, writing the state files. Run as - `smallwebwaf healthcheck`, it is the image's health check instead. + `smallwebwaf servewaf`, it answers `rproxy` instead of passing requests to + an app (see "`servewaf`"). Run as `smallwebwaf healthcheck`, it is the + image's health check instead. - `internal/config`: reads the settings, the one place they are read. - `internal/proxy`: what happens to each request. It works out the client and runs the checks in `check`. It passes the request to the app with the standard library's `httputil.ReverseProxy`, within the timeouts and size limits, and answers its own endpoints in `answerAdmin`. Once the request has ended, it counts bytes, refusals and anomalies, and writes the log - line. + line. It also holds `servewaf`'s question, answer and report. - `internal/ratelimit`: the table of clients. It counts requests, bytes and refusals, and keeps each client's history. - `internal/bans`: the bans. It holds each netblock's bans with their notes, @@ -1006,6 +1008,205 @@ answers. costs). - A ban made through the endpoints starts at once. +## `servewaf` + +`smallwebwaf servewaf` runs the checks for `smallwebwaf rproxy`, a small reverse +proxy in front of each app. `rproxy` asks `servewaf` about each visitor's +request. `servewaf` answers `rproxy` and passes nothing to an app itself. One +`servewaf` serves many sites. + +This section is the agreement between the two. The header names, the fields and +the statuses below are exact. Each `SWWAF_` setting and check keeps the meaning +this document gives it, except as this section says. + +`servewaf` starts as `smallwebwaf` with no command does. It reads the settings, +the rule files, the lookup database and the state files, and listens at +`SWWAF_LISTEN_ADDR`. It stops in the same orderly way, writing the state files. +`docs/settings.md` lists the settings it ignores. + +### The question + +A request carrying `X-Smallwebwaf-Client` is `rproxy` asking about one visitor's +request. It is the visitor's request as the visitor sent it: + +- the method; +- the path and query as sent; +- `Host`; +- every header the visitor sent, including its own `X-Forwarded-For`; +- the first 64 KiB of the body, or all of it when shorter. + +A body whose length the visitor did not announce comes chunked. Any other body +comes with a `Content-Length` of the part it carries. + +`rproxy` removes these headers if the visitor sent them, then sets them. It adds +no other header: + +- `X-Smallwebwaf-Client`: the visitor's address, which `rproxy` has worked out. + `servewaf` takes it as the client, and groups an IPv6 address as "What happens + to a request" says. +- `X-Smallwebwaf-Peer`: the address of `rproxy`'s own TCP peer, for the log + line's `peer_ip`. +- `X-Smallwebwaf-Content-Length`: the body length the visitor announced. It is + absent when the visitor announced none. +- `X-Request-ID`: the request's id, which `rproxy` makes fresh for every request + when it arrives. `servewaf` takes it as given and never makes one. +- `X-Forwarded-Proto`: the scheme. + +Before any check, `servewaf` removes the three `X-Smallwebwaf-` headers. It also +sets `Content-Length` to the length the visitor announced, or removes it. So the +rule files and the Core Rule Set see the headers the visitor sent, with +`rproxy`'s `X-Request-ID` and `X-Forwarded-Proto`. + +`servewaf` refuses a question with `400` when: + +- `X-Smallwebwaf-Client` is not an address, or is an address with a zone, such + as `2001:db8::1%eth0`; +- `X-Request-ID` is absent, empty, over 128 bytes or not valid UTF-8; +- `X-Request-ID` is the id of a question `servewaf` is still handling, or of a + request it holds (see "The report"); +- the body is over 64 KiB. + +For a refused question, `servewaf` writes a process warning. It runs no check, +writes no request log line and counts nothing in the metrics. + +These do not refuse a question: + +- `X-Smallwebwaf-Peer` absent or not an address: the log line's `peer_ip` is + `invalid IP`. +- `X-Smallwebwaf-Content-Length` not a whole number, or negative: it counts as + absent. +- `X-Forwarded-Proto` absent: the log line's `scheme` is `http`. +- One of the five headers above sent more than once: `servewaf` uses its first + value. + +`servewaf` reads the whole body before any check. A question whose body stops +arriving gets no answer. `servewaf` closes the connection once the 60 seconds +below run out, or as soon as `rproxy` closes its side. It writes a process +warning, runs no check, writes no request log line and counts nothing in the +metrics. + +### The answer + +`servewaf` runs steps 3 to 12 of "What happens to a request", in that order. +Every answer but the last below carries `X-Smallwebwaf-Disposition`. + +- `X-Smallwebwaf-Disposition: allow`, status `200`, no body. `rproxy` passes the + request to the app. With `SWWAF_ADD_LOOKUP_HEADERS` the answer also carries + `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the request it + passes on. A header whose value is unknown is left out. +- `X-Smallwebwaf-Disposition: deny`, with the status, headers and body the + visitor is to get. `rproxy` gives them to the visitor, less this header. The + status is one of: + - `SWWAF_BAN_RESPONSE`, for a refusal; + - `403`, for a `block` rule or the Core Rule Set; + - `404`, for a path under `/_smallwebwaf/` that passes the checks, so that + the app never sees that prefix. +- `X-Smallwebwaf-Disposition: close`, status `200`, no body. This takes the + place of a refusal while `SWWAF_BAN_RESPONSE` is `close`. `rproxy` closes the + visitor's connection without an answer. +- No such header, status `400`, for a question `servewaf` refuses. `rproxy` + treats this answer as it treats no answer at all. + +In `observe` mode a request that `enforce` mode would refuse is answered +`allow`, and its log line has `would_action`. A path under `/_smallwebwaf/` is +still answered `404`. + +The Core Rule Set inspects the body a question carries only while +`SWWAF_WAF_BODY_LIMIT` is set. It inspects at most 64 KiB less one byte. So it +does not inspect a JSON or XML body of 64 KiB or more, which may have been cut +short. + +### The report + +Once the visitor's answer has ended, `rproxy` sends `POST /_smallwebwaf/report` +for each request `servewaf` answered with `allow`, `deny` or `close`. It sends +none after a `400` or no answer. The report carries no `X-Smallwebwaf-Client`. +Its body is one JSON object with all of these fields, named as the request log's +fields of the same meaning: + +- `request_id`: the question's `X-Request-ID`; +- `status`: what the visitor got, `0` if nothing, otherwise from `100` to `599`; +- `request_bytes` and `response_bytes`: the body bytes each way, with what a + WebSocket carried each way, each a whole number from `0` to `1125899906842624` + (1 PiB); +- `duration_total`: milliseconds from the request's headers being read to the + end of the answer, from `0` to `1000000000000` (about 31 years); +- `duration_upstream_total`: milliseconds from the hand-over to the app to the + end of the answer, in the same range, and `0` for a request not passed to the + app. + +`servewaf` answers `400` and writes a process warning when the body: + +- is not one JSON object, such as `null`, or an object followed by other data; +- lacks one of these fields, or gives it as `null`; +- gives a value of the wrong type, or out of its range; +- is over 1 MiB. + +It answers `204` to any other report, also for a request it does not hold, which +it ignores. It ignores a field not listed here. `rproxy` does not act on the +answer. + +For a request it holds, `servewaf` writes the request's last log line (see "The +log"). For a request it allowed, it first: + +- counts the bytes as `SWWAF_BYTES_COUNT` says, toward the client's byte limits + at the percentages it gave that request; +- bans a client that passes a byte limit; +- adds the request with its status to the client's history, the anomaly counters + and the metrics. + +A denied request's counting is finished at its decision, so its report counts +nothing. + +`servewaf` holds each answered request until its report comes, up to 10,000 +requests. Past that, the request answered longest ago is dropped. A request +whose report never comes gets no last log line. If it was allowed, it is in +neither the client's history nor the metrics, and its bytes are not counted. + +`servewaf` holds only the decision's log line and what counting needs. It never +holds the request or response objects, and no `Authorization` or `Cookie`. A +report reaches only what is held, so it can never reach a question still being +handled. `servewaf` ignores a report naming such a question. + +### The log + +Each request gets three request log lines, joined by `request_id`: + +- `rproxy`'s short line; +- `servewaf`'s line at its decision; +- `servewaf`'s line when the report comes, with everything known: the decision + line's fields and the report's together. + +The repetition is intended. The field `servewaf_line` says which of `servewaf`'s +two lines a line is. `docs/request-log.md` gives the fields. + +### `servewaf`'s own endpoints + +A request without `X-Smallwebwaf-Client` is for `servewaf` itself, at its own +address on the private network. Nothing routes the internet there. These are: + +- the health check, the metrics, and the ban and client endpoints, as + `docs/admin-endpoints.md` lists them, with the same tokens; +- the report. + +Anything else gets `404`. No check runs on these requests, since their peers are +`rproxy` and the admin's tools. The report needs no token, so anything on the +private network can send one or ask a question. + +`servewaf` gives each request 60 seconds to arrive and its answer 60 seconds to +leave, and closes a connection idle for 120 seconds. It takes a request line and +headers up to 1 MiB, Go's default, so that it never refuses a question for the +size of a visitor's headers. + +### Sites + +One `servewaf` keeps one table of clients, counters and bans for every site that +asks it. A client banned through one site is refused on all of them, and its +requests on all of them count toward one set of limits. Log lines, alerts raised +by a request and ban notes name the site by its host. + +`docs/metrics.md` says how its metrics count. + ## Deployment `smallwebwaf` runs inside the app's own container: the app's Dockerfile builds @@ -1171,9 +1372,10 @@ and `script/gitea-requests` checks those notes. ## Planned, not built -These two points are specified as later work, and nothing in the code does them -yet: +These points are specified as later work, and nothing in the code does them yet: +- Planned: `smallwebwaf rproxy`, the reverse proxy in front of each app that + asks `servewaf` about each request (see "`servewaf`"). - Planned: sharing bans between the instances of several apps, by running one CrowdSec engine per host that every instance reports its bans to. It is reconsidered once real ban volumes are known. Until then each instance keeps diff --git a/docs/admin-endpoints.md b/docs/admin-endpoints.md index dde1d67..72ef830 100644 --- a/docs/admin-endpoints.md +++ b/docs/admin-endpoints.md @@ -2,7 +2,8 @@ The endpoints under `/_smallwebwaf/`, what each takes and what each answers. "Admin endpoints" in [`SPEC.md`](../SPEC.md) says which checks they go through -and how their tokens work. +and how their tokens work. `smallwebwaf servewaf` answers them at its own +address, without those checks (see "`servewaf`" in [`SPEC.md`](../SPEC.md)). - `GET /_smallwebwaf/healthz`: `200` and `ok`, to anyone. - `GET /_smallwebwaf/metrics`: the metrics, for `SWWAF_METRICS_TOKEN` (see diff --git a/docs/alerts.md b/docs/alerts.md index 916912d..7c53686 100644 --- a/docs/alerts.md +++ b/docs/alerts.md @@ -78,6 +78,9 @@ on one line: - `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`: one of the events above. +- `host`: the site of the request that raised the alert. Only `servewaf` sends + this field, since it serves many sites, and not for `source_failure`, + `file_error` and `summary`. - `client`: the address whose request raised the alert. Empty for `source_failure`, `file_error` and `summary`. - `netblock`: empty for `source_failure`, `file_error` and `summary`; otherwise @@ -110,7 +113,7 @@ Slack and ntfy get a message with a title and a text. The title is the instance and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then one line for each of these that the alert has: -- the client, the netblock and the country; +- the host, the client, the netblock and the country; - the file, the source, the error and the mode from `detail`; - the suppressed repeats, if any. diff --git a/docs/metrics.md b/docs/metrics.md index 87877e7..9e10ec2 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -4,6 +4,15 @@ What `GET /_smallwebwaf/metrics` answers with. "Metrics endpoint" in [`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus keeps the `instance` label as `exported_instance` unless the scrape sets `honor_labels: true`. +`smallwebwaf servewaf` counts a denied request at its decision. It counts an +allowed request when its report comes, with the status, bytes and times the +report gives. These are not counted: + +- a report itself; +- an allowed request whose report never comes; +- a question refused with `400`; +- a question whose body stopped arriving. + ## Traffic - `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and diff --git a/docs/request-log.md b/docs/request-log.md index 10c67f3..c2adbc7 100644 --- a/docs/request-log.md +++ b/docs/request-log.md @@ -117,3 +117,65 @@ All in milliseconds: connection; - `duration_upstream_first_byte`: until the first byte of the answer; - `duration_upstream_total`: until the end of the answer. + +## `servewaf`'s lines + +`smallwebwaf servewaf` writes two lines for each question it answers, joined by +`request_id`. The field `servewaf_line` says which of the two a line is: + +- `decision`: the line it writes once it has answered the question; +- `report`: the line it writes when `rproxy`'s report comes, with everything + known. The repetition is intended. + +A request to its own endpoints writes one line, without `servewaf_line`. A +report writes no line but the `report` line of the request it names. + +These questions write no line: + +- a question refused with `400` (see "The question" under "`servewaf`" in + [`SPEC.md`](../SPEC.md)); +- a question whose body stopped arriving, which gets no answer. + +### The `decision` line + +It has the fields above, with these meanings where they differ: + +- `client_ip`: `X-Smallwebwaf-Client`. +- `peer_ip`: `X-Smallwebwaf-Peer`. +- `host`: the site the visitor asked for. +- `protocol`: the question's own, `HTTP/1.1`. +- `content_length`: `X-Smallwebwaf-Content-Length`. +- `request_id`: the question's `X-Request-ID`. +- `status`: what `servewaf` answered: `200` for `allow`, the status of `deny`, + or `0` for `close`, as the visitor is then sent nothing. +- `request_bytes`: how much of the question's body the Core Rule Set read. +- `response_bytes`: the body of `servewaf`'s answer. +- `action`: `forward` for `allow`, and `admin` for the `404` for a path under + `/_smallwebwaf/`. +- `aborted`: true when `rproxy` went away before the answer. +- `duration_total`: from the question's headers being read to the line. + +The line leaves out `upstream_status`, `websocket` and the +`duration_upstream_...` fields, since `servewaf` passes nothing to the app. + +The bytes of an allowed request are counted when its report comes. So `counts` +holds the client's bytes without this request's. + +### The `report` line + +It has every field of the `decision` line, and `time` stays when the request +arrived. These fields say what the report says: + +- `status`: what the visitor got, `0` if nothing; +- `request_bytes` and `response_bytes`: the body bytes each way, with what a + WebSocket carried each way; +- `duration_total`: from the request's headers being read by `rproxy` to the end + of the answer; +- `duration_upstream_total`: from the hand-over to the app to the end of the + answer, for an allowed request. + +For an allowed request, the report's counting also changes these fields: + +- `counts`: the byte totals now include this request's; +- `limit_hit`, `offence` and `ban_expires`: set when the report's bytes broke a + byte limit. diff --git a/docs/settings.md b/docs/settings.md index ad41aee..73b99f7 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -4,6 +4,23 @@ Every setting, its default and the values it takes, in the groups of "Settings" in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings that take `off`, the `_FILE` form, and what each group does and why. +## `servewaf` + +`smallwebwaf servewaf` uses every setting here but these, which it ignores: + +- `SWWAF_UPSTREAM_URL` and `SWWAF_TRUSTED_PROXIES`; +- the five timeouts under "Size and time limits"; +- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, `SWWAF_REQUEST_MAX_BYTES` and + `SWWAF_RESPONSE_MAX_BYTES`. + +An invalid value of one of these still stops the start, and the line of settings +at start still shows them. + +`SWWAF_ADD_LOOKUP_HEADERS` puts the headers on the answer to `rproxy`, not on a +request to an app. `SWWAF_WAF_BODY_LIMIT` has an effect up to 64 KiB less one +byte, whatever its value. A question carries no more of a body than 64 KiB (see +"`servewaf`" in [`SPEC.md`](../SPEC.md)). + ## Forms of values - A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. Zero @@ -327,4 +344,5 @@ A few limits are fixed rather than settings: - at most 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept; - at most 1,000 DNS blocklist queries are under way at once; - at most 20,000 anomaly counters are kept; -- each alert destination's queue holds at most 1,000 alerts. +- each alert destination's queue holds at most 1,000 alerts; +- `servewaf` holds at most 10,000 answered requests awaiting their reports. diff --git a/internal/alerts/alerts.go b/internal/alerts/alerts.go index 7864637..422433c 100644 --- a/internal/alerts/alerts.go +++ b/internal/alerts/alerts.go @@ -150,13 +150,16 @@ type Params struct { // Alert is one alert, as the webhook is sent it and alerts.json holds it, // with the fields of "The JSON object" in docs/alerts.md. ASN, ASName and -// Country are, for a ban, the client's as the ban's notes give them. +// Country are, for a ban, the client's as the ban's notes give them. Host is +// the site of the request that raised the alert, while one servewaf serves +// many sites, and "" otherwise. // //nolint:tagliatelle // docs/alerts.md names the alert's fields in snake_case type Alert struct { Instance string `json:"instance"` Time time.Time `json:"time"` Event string `json:"event"` + Host string `json:"host,omitempty"` Client netip.Addr `json:"client"` Netblock netip.Prefix `json:"netblock"` ASN string `json:"asn"` @@ -899,12 +902,16 @@ func title(alert *Alert) string { } // text returns the text of alert in Slack and ntfy: its reason, then a -// line for each of its client, netblock and country, the file, source, -// error and mode its detail gives, and its suppressed repeats, that it -// has. +// line for each of its host, client, netblock and country, the file, +// source, error and mode its detail gives, and its suppressed repeats, that +// it has. func text(alert *Alert) string { lines := []string{alert.Reason} + if alert.Host != "" { + lines = append(lines, "host: "+alert.Host) + } + if alert.Client.IsValid() { lines = append(lines, "client: "+alert.Client.String()) } diff --git a/internal/alerts/alerts_test.go b/internal/alerts/alerts_test.go index 81d8b95..51afbb9 100644 --- a/internal/alerts/alerts_test.go +++ b/internal/alerts/alerts_test.go @@ -1284,9 +1284,11 @@ func anAlertForEachEvent() []eventMessage { Event: alerts.EventWAFBlock, Client: netip.MustParseAddr("192.0.2.1"), Netblock: netip.MustParsePrefix("192.0.2.1/32"), Reason: "refused by the Core Rule Set", + Host: "app.example", }, "fsn1app1/gitea: waf_block", "default shield", - "refused by the Core Rule Set\nclient: 192.0.2.1\nnetblock: 192.0.2.1/32", + "refused by the Core Rule Set\nhost: app.example\nclient: 192.0.2.1\n" + + "netblock: 192.0.2.1/32", }, { alerts.Alert{ diff --git a/internal/anomaly/anomaly.go b/internal/anomaly/anomaly.go index d30ae3e..3c4734d 100644 --- a/internal/anomaly/anomaly.go +++ b/internal/anomaly/anomaly.go @@ -103,6 +103,8 @@ type Counter struct { // Request is a request that has ended, as the counters count it. type Request struct { + // Host is the site the alert names, "" for none. + Host string // Client is the client's address, and ClientGroup the client it is // counted as: its IPv4 address, or the IPv6 netblock of // SWWAF_IPV6_GROUP_PREFIX its address is in. @@ -396,6 +398,7 @@ func alertFor(r Request, k key, o overThreshold) alerts.Alert { return alerts.Alert{ Event: alerts.EventAnomaly, + Host: r.Host, Client: r.Client, Netblock: k.netblock, ASN: r.ASN, diff --git a/internal/metrics/metrics.go b/internal/metrics/metrics.go index c7f7d96..d47dc95 100644 --- a/internal/metrics/metrics.go +++ b/internal/metrics/metrics.go @@ -403,28 +403,21 @@ func (m *Metrics) RequestEnded( line *requestlog.Line, limit string, duration, upstreamDuration time.Duration, ) { m.inFlight.Dec() + m.count(line, limit, duration, upstreamDuration) +} - class := statusClass(line.Status) - m.requests.WithLabelValues(class, line.Action).Inc() - m.requestBytes.WithLabelValues(class, line.Action).Add(float64(line.RequestBytes)) - m.responseBytes.WithLabelValues(class, line.Action).Add(float64(line.ResponseBytes)) - m.requestDuration.Observe(duration.Seconds()) +// RequestAllowed counts a question of rproxy's, answered allow, as no longer +// under way. RequestReported counts the request it was about. +func (m *Metrics) RequestAllowed() { + m.inFlight.Dec() +} - if upstreamDuration > 0 { - m.upstreamDuration.Observe(upstreamDuration.Seconds()) - } - - if limit != "" { - m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc() - } - - if line.Country != "" { - m.countries.add(line.Country, line) - } - - if line.ASN != "" { - m.asns.add(line.ASN, line) - } +// RequestReported counts a request rproxy passed to the app, as RequestEnded +// does, from what its report says, which line holds. +func (m *Metrics) RequestReported( + line *requestlog.Line, duration, upstreamDuration time.Duration, +) { + m.count(line, "", duration, upstreamDuration) } // LimitHit counts a request that broke a rate limit, a byte limit or the @@ -493,6 +486,33 @@ func (m *Metrics) StateFileEditSetAside(name string) { m.stateFileEditsSetAside.WithLabelValues(name).Inc() } +// count counts a request that has ended, as RequestEnded describes. +func (m *Metrics) count( + line *requestlog.Line, limit string, duration, upstreamDuration time.Duration, +) { + class := statusClass(line.Status) + m.requests.WithLabelValues(class, line.Action).Inc() + m.requestBytes.WithLabelValues(class, line.Action).Add(float64(line.RequestBytes)) + m.responseBytes.WithLabelValues(class, line.Action).Add(float64(line.ResponseBytes)) + m.requestDuration.Observe(duration.Seconds()) + + if upstreamDuration > 0 { + m.upstreamDuration.Observe(upstreamDuration.Seconds()) + } + + if limit != "" { + m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc() + } + + if line.Country != "" { + m.countries.add(line.Country, line) + } + + if line.ASN != "" { + m.asns.add(line.ASN, line) + } +} + // addReputationQueries adds the counter of the queries to source, a DNSBL // zone, or of the checks of clients with AbuseIPDB, which count tells. func (m *Metrics) addReputationQueries(source prometheus.Labels, count func() int) { diff --git a/internal/proxy/admin.go b/internal/proxy/admin.go index cbf86d6..f5f2a9e 100644 --- a/internal/proxy/admin.go +++ b/internal/proxy/admin.go @@ -41,6 +41,15 @@ var ( errNotAddress = errors.New("is not an address, such as 203.0.113.9") ) +// answerHealth answers a request for the health endpoint. +func (rq *request) answerHealth() { + rq.line.Action = requestlog.ActionAdmin + // Set here rather than left to Go's server, which would set it only + // after the log line has taken the response's headers. + rq.out.Header().Set("Content-Type", "text/plain; charset=utf-8") + _, _ = io.WriteString(rq.out, "ok\n") +} + // answerAdmin answers a request for smallwebwaf itself, under // /_smallwebwaf/, once it has passed the checks. Each endpoint needs a // token, sent as Authorization: Bearer : the metrics diff --git a/internal/proxy/anomaly_internal_test.go b/internal/proxy/anomaly_internal_test.go index af8034d..88fc46f 100644 --- a/internal/proxy/anomaly_internal_test.go +++ b/internal/proxy/anomaly_internal_test.go @@ -15,9 +15,11 @@ func TestWithEveryAnomalyThresholdOffARequestIsNotCounted(t *testing.T) { // anomaly counters, nor a clock, and the request no response: reading // any of them to count the request panics. rq := &request{ - h: &handler{config: &config.Config{LookupSource: "geojs"}}, - client: netip.MustParseAddr("203.0.113.9"), - lookedUp: true, + decision: decision{ + client: netip.MustParseAddr("203.0.113.9"), + lookedUp: true, + }, + h: &handler{config: &config.Config{LookupSource: "geojs"}}, } defer func() { diff --git a/internal/proxy/bans.go b/internal/proxy/bans.go index 5b1eedb..10f7fc2 100644 --- a/internal/proxy/bans.go +++ b/internal/proxy/bans.go @@ -323,6 +323,7 @@ func (rq *request) alertBan(ban bans.Ban) { rq.h.alerts.Raise(alerts.Alert{ Event: event, + Host: rq.site, Client: rq.client, Netblock: ban.Netblock, ASN: ban.Notes.ASN, @@ -339,11 +340,11 @@ func (rq *request) alertBan(ban bans.Ban) { func (rq *request) noted(now time.Time, status int) bans.Request { return bans.Request{ Time: now, - Method: rq.in.Method, - Host: rq.in.Host, - Path: rq.in.URL.RequestURI(), + Method: rq.line.Method, + Host: rq.line.Host, + Path: rq.requestURI, Status: status, - UserAgent: rq.in.UserAgent(), + UserAgent: rq.line.UserAgent, } } diff --git a/internal/proxy/bans_test.go b/internal/proxy/bans_test.go index d6379d9..aae674d 100644 --- a/internal/proxy/bans_test.go +++ b/internal/proxy/bans_test.go @@ -25,6 +25,8 @@ const ( userAgent = "ban-test/1.0" // permanent is the log line's ban_expires for a permanent ban. permanent = "permanent" + // banResponseClose is the SWWAF_BAN_RESPONSE that closes the connection. + banResponseClose = "close" ) func TestBrokenLimitBansTheClient(t *testing.T) { @@ -237,7 +239,7 @@ func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) { {"", http.StatusForbidden}, {"403", http.StatusForbidden}, {"429", http.StatusTooManyRequests}, - {"close", 0}, + {banResponseClose, 0}, } { t.Run(banResponse+"="+tc.setting, func(t *testing.T) { t.Parallel() diff --git a/internal/proxy/coreruleset.go b/internal/proxy/coreruleset.go index 223c9f3..e5c4bf9 100644 --- a/internal/proxy/coreruleset.go +++ b/internal/proxy/coreruleset.go @@ -113,6 +113,7 @@ func (rq *request) alertWAFBlock(result waf.Result) { rq.h.alerts.Raise(alerts.Alert{ Event: alerts.EventWAFBlock, + Host: rq.site, Client: rq.client, Netblock: rq.h.clientGroup(rq.client), ASN: rq.line.ASN, diff --git a/internal/proxy/metrics_test.go b/internal/proxy/metrics_test.go index c291829..401c102 100644 --- a/internal/proxy/metrics_test.go +++ b/internal/proxy/metrics_test.go @@ -207,7 +207,7 @@ func TestMetricsCountLimitsAndBans(t *testing.T) { rateLimitPerMinute: "1", rateLimitExemptNets: scraper, denyNets: denied, - banResponse: "close", + banResponse: banResponseClose, limitBanDuration: "1h", maxBanDuration: "2h", }) diff --git a/internal/proxy/proxy.go b/internal/proxy/proxy.go index b575464..5eef404 100644 --- a/internal/proxy/proxy.go +++ b/internal/proxy/proxy.go @@ -1,6 +1,8 @@ -// Package proxy passes each request to the app and the app's answer back, -// unchanged, within the size and time limits, and writes one request log -// line for each request. +// Package proxy runs the checks on each request and writes its request log +// line. New passes each request it lets through to the app and the app's +// answer back, unchanged, within the size and time limits. NewServeWAF instead +// answers rproxy's question about each request, and takes rproxy's report once +// the visitor's answer has ended, which writes the request a second log line. package proxy import ( @@ -11,6 +13,8 @@ import ( "strings" "time" + lru "github.com/hashicorp/golang-lru/v2" + "sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/bans" @@ -19,7 +23,6 @@ import ( "sneak.berlin/go/smallwebwaf/internal/metrics" "sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/reputation" - "sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/waf" ) @@ -53,7 +56,8 @@ const ClientsPath = "/_smallwebwaf/clients/" // Params are what New needs. type Params struct { Config *config.Config - // RequestLog receives one JSON line per request. + // RequestLog receives one JSON line per request, and from servewaf a second + // when the request's report comes. RequestLog io.Writer // ProcessLog receives the process's own messages. ProcessLog *slog.Logger @@ -109,6 +113,23 @@ type Server struct { // SWWAF_CLIENT_REQUEST_TIMEOUT while the headers arrive; the proxy // applies the timeouts and size limits from then on. func New(params Params) *Server { + h := newHandler(params) + + return h.server(&http.Server{ + Handler: h, + ReadHeaderTimeout: params.Config.ClientRequestTimeout, + // Off is an IdleTimeout of 0, which Go's server replaces with + // ReadTimeout: no limit, as long as ReadTimeout stays unset. + IdleTimeout: params.Config.ClientIdleTimeout, + // Go's server reads 4 KiB past MaxHeaderBytes before it + // refuses, so the limit a client meets is the setting. + MaxHeaderBytes: int(params.Config.ClientRequestHeaderMaxBytes - 4<<10), + }) +} + +// newHandler returns the handler with what every request shares, as the +// settings in params make it. +func newHandler(params Params) *handler { errorLog := slog.NewLogLogger(params.ProcessLog.Handler(), slog.LevelWarn) m := metrics.New(params.Config.MetricsTopN, params.Config.InstanceName) lists, dnsbl, abuseIPDB := newReputation(params, m) @@ -171,29 +192,7 @@ func New(params Params) *Server { m.AddBansAndClients(h.ledger, h.limiter, params.Now) m.AddRules(params.Rules) - return &Server{ - Server: &http.Server{ - Addr: params.Config.ListenAddr, - Handler: h, - ReadHeaderTimeout: params.Config.ClientRequestTimeout, - // Off is an IdleTimeout of 0, which Go's server replaces with - // ReadTimeout: no limit, as long as ReadTimeout stays unset. - IdleTimeout: params.Config.ClientIdleTimeout, - // Go's server reads 4 KiB past MaxHeaderBytes before it - // refuses, so the limit a client meets is the setting. - MaxHeaderBytes: int(params.Config.ClientRequestHeaderMaxBytes - 4<<10), - ErrorLog: errorLog, - }, - Ledger: h.ledger, - Limiter: h.limiter, - GeoJS: h.geojs, - Anomalies: h.anomalies, - LookupFile: h.lookupFile, - Lists: h.lists, - DNSBL: h.dnsbl, - AbuseIPDB: h.abuseIPDB, - Metrics: m, - } + return h } // newReputation returns the lists fetched from URLs, the DNSBL zones' @@ -274,6 +273,10 @@ type handler struct { rules *rules.Files coreRuleSet *waf.CoreRuleSet alerts *alerts.Queue + // awaitingReport holds, in servewaf, the decision on each answered + // question by its id until its report comes, and nil for a question still + // being handled. + awaitingReport *lru.Cache[string, *decision] } // newTransport returns what carries requests to the app. It never goes @@ -299,11 +302,7 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) { // The health endpoint is answered at once, before any check, so that // a health checker is never refused. It does not ask the app. if r.Method == http.MethodGet && r.URL.Path == HealthPath { - rq.line.Action = requestlog.ActionAdmin - // Set here rather than left to Go's server, which would set it only - // after the log line has taken the response's headers. - rq.out.Header().Set("Content-Type", "text/plain; charset=utf-8") - _, _ = io.WriteString(rq.out, "ok\n") + rq.answerHealth() return } @@ -339,3 +338,23 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) { rq.forward(r.Context()) } + +// server returns the Server of h, which listens at SWWAF_LISTEN_ADDR and +// logs through h, with what httpServer sets. +func (h *handler) server(httpServer *http.Server) *Server { + httpServer.Addr = h.config.ListenAddr + httpServer.ErrorLog = h.errorLog + + return &Server{ + Server: httpServer, + Ledger: h.ledger, + Limiter: h.limiter, + GeoJS: h.geojs, + Anomalies: h.anomalies, + LookupFile: h.lookupFile, + Lists: h.lists, + DNSBL: h.dnsbl, + AbuseIPDB: h.abuseIPDB, + Metrics: h.metrics, + } +} diff --git a/internal/proxy/proxy_test.go b/internal/proxy/proxy_test.go index 8507de6..6af3e76 100644 --- a/internal/proxy/proxy_test.go +++ b/internal/proxy/proxy_test.go @@ -284,6 +284,16 @@ func newProxy( ) (*proxy.Server, *output, *alerts.Queue) { t.Helper() + return newServerWith(t, proxy.New, appURL, geojsURL, now, env) +} + +// newServerWith is newProxy for the server build makes. +func newServerWith( + t *testing.T, build func(proxy.Params) *proxy.Server, + appURL, geojsURL string, now func() time.Time, env map[string]string, +) (*proxy.Server, *output, *alerts.Queue) { + t.Helper() + settings := map[string]string{ "SWWAF_UPSTREAM_URL": appURL, rulesDir: t.TempDir(), instanceName: "app", wafMode: off, @@ -334,7 +344,7 @@ func newProxy( } } - server := proxy.New(proxy.Params{ + server := build(proxy.Params{ Config: cfg, RequestLog: out, ProcessLog: processLog, diff --git a/internal/proxy/reputation.go b/internal/proxy/reputation.go index 0020c60..8ecaf5f 100644 --- a/internal/proxy/reputation.go +++ b/internal/proxy/reputation.go @@ -110,6 +110,7 @@ func (rq *request) noteHit(hit bans.ReputationHit, reason string) { rq.h.metrics.ReputationHit(hit.Source) rq.h.alerts.Raise(alerts.Alert{ Event: alerts.EventReputationHit, + Host: rq.site, Client: rq.client, Netblock: rq.h.clientGroup(rq.client), ASN: rq.line.ASN, diff --git a/internal/proxy/request.go b/internal/proxy/request.go index e9620d9..99c38d0 100644 --- a/internal/proxy/request.go +++ b/internal/proxy/request.go @@ -39,20 +39,20 @@ type refusal struct { limit string } -// request is one request on its way through smallwebwaf, from the moment -// its headers have been read to its log line. -type request struct { - h *handler - in *http.Request - // rc sets the deadlines of the connection to the client. - rc *http.ResponseController - out *responseWriter - body *requestBody // nil for a request without a body - line requestlog.Line - - client netip.Addr - peer netip.Addr - peerTrusted bool +// decision is what the checks note of a request: its log line, and what +// counting the request needs once it has ended. servewaf keeps a copy of it +// until the request's report comes. It holds no credentials, since the log line +// never does, and neither the request's objects nor their bodies. +type decision struct { + line requestlog.Line + client netip.Addr + // requestURI is the path with the query, as a ban's notes keep it. + requestURI string + // site is, in servewaf, the host the visitor asked for, which the + // alerts name, since one servewaf serves many sites. It is "" otherwise. + site string + // allowed is true for a question answered with DispositionAllow. + allowed bool // lookedUp is true once the client's AS number and country have been // looked up, whether or not an answer was there, and lookupAnswer is // what the lookup gave then, the zero Answer while GeoJS had given none. @@ -65,18 +65,37 @@ type request struct { counted bool limitPercent, bytesPercent percentage // attack is true for a request that matched a ban rule or asked for a - // trap path, ruleBlocked for one a block rule refused, wafBlocked for - // one the Core Rule Set refused, and tokenRefused for one refused for a - // missing or wrong token, each an offence its client's history counts. - attack, ruleBlocked, wafBlocked, tokenRefused bool + // trap path, ruleBlocked for one a block rule refused, and wafBlocked for + // one the Core Rule Set refused, each an offence its client's history + // counts. + attack, ruleBlocked, wafBlocked bool + // reputation is the reputation sources that list the client, for the + // notes of a ban the request makes. + reputation []bans.ReputationHit +} + +// request is one request on its way through smallwebwaf, from the moment +// its headers have been read to its log line. +type request struct { + decision + + h *handler + in *http.Request + // rc sets the deadlines of the connection to the client. + rc *http.ResponseController + out *responseWriter + body *requestBody // nil for a request without a body + + peer netip.Addr + peerTrusted bool + // tokenRefused is true for a request refused for a missing or wrong + // token, an offence its client's history counts. + tokenRefused bool // blocklisted is true once a blocklist is found to list the client, // dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once // AbuseIPDB's score of it is a hit. blocklisted, dnsblListed, abuseIPDBHit bool - // reputation is the reputation sources that list the client, for the - // notes of a ban the request makes. - reputation []bans.ReputationHit - start time.Time + start time.Time // checked is when the checks were done, and upstreamStart when the // request was handed to the app. checked time.Time @@ -108,50 +127,62 @@ type request struct { answerStarted time.Time } -// newRequest starts handling r: it notes the time, counts the request as -// under way, works out the client, and starts the log line with what is -// known of the request. +// newRequest starts handling r: it works out the client from the TCP peer +// and X-Forwarded-For, and goes on as startRequest does. func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request { + peer := peerAddress(r) + trusted := h.config.TrustedProxies + client := clientAddress(peer, r.Header.Values("X-Forwarded-For"), trusted) + + return h.startRequest(w, r, client, peer, isInside(peer, trusted)) +} + +// startRequest starts handling r from client, which came through peer: it +// notes the time, counts the request as under way, and starts the log line +// with what is known of the request. peerTrusted says whether to believe +// the request's id and scheme, which r's headers give. +func (h *handler) startRequest( + w http.ResponseWriter, r *http.Request, client, peer netip.Addr, peerTrusted bool, +) *request { h.metrics.RequestStarted() start := time.Now() - peer := peerAddress(r) - trusted := h.config.TrustedProxies - peerTrusted := isInside(peer, trusted) forwardedFor := r.Header.Values("X-Forwarded-For") - client := clientAddress(peer, forwardedFor, trusted) rq := &request{ + decision: decision{ + client: client, + requestURI: r.URL.RequestURI(), + line: requestlog.Line{ + Time: requestlog.FormatTime(start), + Instance: h.config.InstanceName, + ClientIP: client.String(), + Method: r.Method, + Scheme: scheme(r, peerTrusted), + Host: r.Host, + Path: rules.Path(r), + Query: r.URL.RawQuery, + Protocol: r.Proto, + Referer: r.Referer(), + UserAgent: r.UserAgent(), + RequestID: requestID(r, peerTrusted), + PeerIP: peer.String(), + ForwardedFor: strings.Join(forwardedFor, ", "), + ClientGroup: h.clientGroup(client).String(), + ContentType: r.Header.Get("Content-Type"), + RequestHeaders: requestHeaders(r, h.config.LogRequestHeaders), + HasAuthorization: len(r.Header.Values("Authorization")) > 0, + HasCookie: len(r.Header.Values("Cookie")) > 0, + Action: requestlog.ActionForward, + }, + }, h: h, in: r, rc: http.NewResponseController(w), out: &responseWriter{ResponseWriter: w}, - client: client, peer: peer, peerTrusted: peerTrusted, start: start, - line: requestlog.Line{ - Time: requestlog.FormatTime(start), - Instance: h.config.InstanceName, - ClientIP: client.String(), - Method: r.Method, - Scheme: scheme(r, peerTrusted), - Host: r.Host, - Path: rules.Path(r), - Query: r.URL.RawQuery, - Protocol: r.Proto, - Referer: r.Referer(), - UserAgent: r.UserAgent(), - RequestID: requestID(r, peerTrusted), - PeerIP: peer.String(), - ForwardedFor: strings.Join(forwardedFor, ", "), - ClientGroup: h.clientGroup(client).String(), - ContentType: r.Header.Get("Content-Type"), - RequestHeaders: requestHeaders(r, h.config.LogRequestHeaders), - HasAuthorization: len(r.Header.Values("Authorization")) > 0, - HasCookie: len(r.Header.Values("Cookie")) > 0, - Action: requestlog.ActionForward, - }, } // A length of -1 is a body whose length was not announced. @@ -569,10 +600,20 @@ func (rq *request) finish() { } // Counted before the log line is written, so that the metrics count - // every request whose line is out. - rq.h.metrics.RequestEnded(line, limit, duration, upstreamDuration) + // every request whose line is out. An allowed request is the exception: + // its report gives the status, bytes and times the metrics count. + if rq.allowed { + rq.h.metrics.RequestAllowed() + } else { + rq.h.metrics.RequestEnded(line, limit, duration, upstreamDuration) + } - err := requestlog.Write(rq.h.requestLog, line) + rq.writeLine() +} + +// writeLine writes the request's log line. +func (rq *request) writeLine() { + err := requestlog.Write(rq.h.requestLog, &rq.line) if err != nil { rq.h.processLog.Error("writing the request log failed", "error", err.Error()) } @@ -634,6 +675,7 @@ func (rq *request) countAnomalies() { answer, _ := rq.answerAtTheEnd() rq.h.anomalies.Count(rq.h.now(), anomaly.Request{ + Host: rq.site, Client: rq.client, ClientGroup: rq.h.clientGroup(rq.client), ASN: answer.ASN, diff --git a/internal/proxy/servewaf.go b/internal/proxy/servewaf.go new file mode 100644 index 0000000..9fe3923 --- /dev/null +++ b/internal/proxy/servewaf.go @@ -0,0 +1,464 @@ +package proxy + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/netip" + "strconv" + "strings" + "time" + "unicode/utf8" + + lru "github.com/hashicorp/golang-lru/v2" + + "sneak.berlin/go/smallwebwaf/internal/requestlog" +) + +// The headers of rproxy's question and of servewaf's answer, as +// "servewaf" in SPEC.md gives them. +const ( + // ClientHeader marks a request as rproxy's question, and gives the + // visitor's address. + ClientHeader = "X-Smallwebwaf-Client" + // PeerHeader gives the address of rproxy's own TCP peer. + PeerHeader = "X-Smallwebwaf-Peer" + // ContentLengthHeader gives the body length the visitor announced. + ContentLengthHeader = "X-Smallwebwaf-Content-Length" + // DispositionHeader gives servewaf's answer. + DispositionHeader = "X-Smallwebwaf-Disposition" +) + +// The values of DispositionHeader. +const ( + // DispositionAllow tells rproxy to pass the request to the app. + DispositionAllow = "allow" + // DispositionDeny tells rproxy to give the visitor the answer's status, + // headers and body. + DispositionDeny = "deny" + // DispositionClose tells rproxy to close the visitor's connection + // without an answer, which SWWAF_BAN_RESPONSE asks for when it is close. + // servewaf closing the question's connection instead would read as + // servewaf being down, and rproxy would then let the request through. + DispositionClose = "close" +) + +// ReportPath is where rproxy sends the report of each request servewaf +// answered. +const ReportPath = "/_smallwebwaf/report" + +// QuestionBodyMaxBytes is the most of a visitor's body a question carries. +const QuestionBodyMaxBytes = 64 << 10 + +const ( + // maxAwaitingReport is the most answered requests held until their + // reports come. Past it, the request answered longest ago is dropped. + maxAwaitingReport = 10000 + // maxRequestIDBytes is the longest id a question may give. + maxRequestIDBytes = 128 + // reportBodyMaxBytes is the most of a report's body that is read. It is + // far above any report's. + reportBodyMaxBytes = 1 << 20 + // maxHTTPStatus is the highest status a report may give. + maxHTTPStatus = 599 + // maxReportedBytes is the most bytes a report may give for each way, 1 + // PiB, so that adding them up cannot overflow. + maxReportedBytes = 1 << 50 + // maxReportedMilliseconds is the longest duration a report may give, + // about 31 years, so that it fits a time.Duration. + maxReportedMilliseconds = 1e12 + // serveWAFTimeout bounds how long a request to servewaf takes to arrive, + // and its answer to leave, and serveWAFIdleTimeout how long a kept-open + // connection waits for its next request. + serveWAFTimeout = 60 * time.Second + serveWAFIdleTimeout = 120 * time.Second +) + +var ( + errReportFieldMissing = errors.New("a field is missing") + errReportValue = errors.New("a value is out of range") +) + +// Report is the JSON object rproxy posts to ReportPath once the visitor's +// answer has ended, its fields named as the request log's of the same +// meaning. RequestID is the question's id. Status is what the visitor got, 0 +// if nothing. The durations are in milliseconds. +// +//nolint:tagliatelle // docs/request-log.md names the fields in snake_case +type Report struct { + RequestID string `json:"request_id"` + Status int `json:"status"` + RequestBytes int64 `json:"request_bytes"` + ResponseBytes int64 `json:"response_bytes"` + DurationTotal float64 `json:"duration_total"` + DurationUpstreamTotal float64 `json:"duration_upstream_total"` +} + +// NewServeWAF returns the server `smallwebwaf servewaf` runs. It answers +// rproxy's questions with the checks New runs, takes rproxy's reports and +// answers its own endpoints. It passes nothing to an app. rproxy applies +// the size and time limits to the visitor, so their settings do not apply +// here. Go's default limit on a request's line and headers, 1 MiB, stays: +// it is far above rproxy's, and a question refused for its size would have +// rproxy let the request through. +func NewServeWAF(params Params) *Server { + cfg := *params.Config + cfg.RequestMaxBytes = 0 + cfg.ClientRequestTimeout = serveWAFTimeout + // The server's WriteTimeout does this. It sets a new deadline for each + // request, where a deadline a handler sets stays on a kept-open + // connection, and would run out under a later request's answer. + cfg.ClientResponseTimeout = 0 + // A question's body of QuestionBodyMaxBytes may be cut short, and the + // Core Rule Set does not read a JSON or XML body over its limit. + cfg.WAFBodyLimit = min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1) + params.Config = &cfg + + h := newHandler(params) + + awaitingReport, err := lru.New[string, *decision](maxAwaitingReport) + if err != nil { + panic(err) // New fails only for a size below one + } + + h.awaitingReport = awaitingReport + + return h.server(&http.Server{ + Handler: http.HandlerFunc(h.serveWAF), + ReadTimeout: serveWAFTimeout, + WriteTimeout: serveWAFTimeout, + IdleTimeout: serveWAFIdleTimeout, + }) +} + +// serveWAF handles one request to servewaf: a request carrying +// ClientHeader is rproxy's question, and any other is for servewaf itself. +func (h *handler) serveWAF(w http.ResponseWriter, r *http.Request) { + if len(r.Header.Values(ClientHeader)) > 0 { + h.answerQuestion(w, r) + + return + } + + h.answerOwnEndpoint(w, r) +} + +// answerQuestion runs the checks on the visitor's request a question +// carries, and answers rproxy what to do with it. A denied request's counting +// is finished here, and an allowed one's by its report. +func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) { + client, body, ok := h.readQuestion(w, r) + if !ok { + return + } + + // One that cannot be read leaves the log line's peer_ip invalid. + peer, _ := netip.ParseAddr(r.Header.Get(PeerHeader)) + + rq := h.startRequest(w, asTheVisitorSentIt(r, body), client.Unmap(), peer.Unmap(), + true) + // The last deferred runs first, so that what is held has the finished log + // line. Go's server sends the answer only once this function has + // returned, so the report cannot come before it is held. + defer rq.holdForReport() + defer rq.finish() + + rq.site = r.Host + rq.line.ServeWAFLine = requestlog.ServeWAFLineDecision + refused := rq.check(r.Context()) + rq.checked = time.Now() + underOwnPrefix := strings.HasPrefix(r.URL.Path, adminPrefix) + + if refused == nil && !underOwnPrefix { + // In observe mode, what enforce mode would have refused counts here. + defer rq.countRefusal() + + rq.allow() + + return + } + + // The last deferred runs first, as in ServeHTTP. + defer rq.addToHistory() + defer rq.countAnomalies() + defer rq.countRefusal() + + if refused != nil { + rq.deny(*refused) + + return + } + + // The app never sees this prefix. servewaf's own endpoints are at its + // own address, so this answers as a path with no endpoint does. + rq.line.Action = requestlog.ActionAdmin + rq.out.Header().Set(DispositionHeader, DispositionDeny) + http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound) +} + +// readQuestion returns the visitor's address and the body a question carries, +// and whether servewaf takes the question. It answers a question it does not +// take with 400 and no disposition, which rproxy treats as it treats no +// answer. The id of a question it takes is held from here on, without a +// decision yet, so that another question with that id is refused while this +// one is being handled. +// +// The body is read before any check, since Go's server does not send an answer +// until it has read the body. A question whose body stops arriving gets no +// answer, and is not looked at. +func (h *handler) readQuestion( + w http.ResponseWriter, r *http.Request, +) (netip.Addr, []byte, bool) { + client, err := netip.ParseAddr(r.Header.Get(ClientHeader)) + if err != nil || client.Zone() != "" { + h.refuseQuestion(w, ClientHeader+" is not an address without a zone") + + return netip.Addr{}, nil, false + } + + id := r.Header.Get(requestIDHeader) + if id == "" || len(id) > maxRequestIDBytes || !utf8.ValidString(id) { + h.refuseQuestion(w, requestIDHeader+" is not a valid id") + + return netip.Addr{}, nil, false + } + + body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, QuestionBodyMaxBytes)) + + if _, tooLarge := errors.AsType[*http.MaxBytesError](err); tooLarge { + h.refuseQuestion(w, "the body is over 64 KiB") + + return netip.Addr{}, nil, false + } + + if err != nil { + h.processLog.Warn("a question's body stopped arriving", "error", err.Error()) + // This panic has Go's server close the connection without an answer, + // and log nothing. + panic(http.ErrAbortHandler) + } + + held, _ := h.awaitingReport.ContainsOrAdd(id, nil) + if held { + h.refuseQuestion(w, requestIDHeader+" is the id of a request servewaf holds") + + return netip.Addr{}, nil, false + } + + return client, body, true +} + +// refuseQuestion answers a question servewaf does not take, for reason. +func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) { + h.processLog.Warn("a question is refused", "reason", reason) + http.Error(w, reason, http.StatusBadRequest) +} + +// asTheVisitorSentIt returns a copy of r, a question, as the checks and the +// log line are to see it: without the headers rproxy adds for servewaf, with +// body, which has been read of r, and with the length the visitor announced, +// as -1 when it announced none. The question's own length covers only the part +// of the body it carries. +func asTheVisitorSentIt(r *http.Request, body []byte) *http.Request { + sent := r.Clone(r.Context()) + announced := sent.Header.Get(ContentLengthHeader) + + sent.Header.Del(ClientHeader) + sent.Header.Del(PeerHeader) + sent.Header.Del(ContentLengthHeader) + sent.Header.Del("Content-Length") + + sent.ContentLength = -1 + + length, err := strconv.ParseInt(announced, 10, 64) + if err == nil && length >= 0 { + sent.ContentLength = length + sent.Header.Set("Content-Length", announced) + } + + sent.Body = http.NoBody + if len(body) > 0 { + sent.Body = io.NopCloser(bytes.NewReader(body)) + } + + return sent +} + +// allow answers a question with DispositionAllow and, while +// SWWAF_ADD_LOOKUP_HEADERS is set, the client's AS number and country. +func (rq *request) allow() { + rq.allowed = true + + header := rq.out.Header() + header.Set(DispositionHeader, DispositionAllow) + + if rq.h.config.AddLookupHeaders { + setLookupHeaders(header, rq.line.ASN, rq.line.Country) + } + + rq.out.WriteHeader(http.StatusOK) +} + +// holdForReport holds a copy of what the report's steps need, by the request's +// id. A report only ever reaches that copy, never the request being handled. +func (rq *request) holdForReport() { + held := rq.decision + rq.h.awaitingReport.Add(rq.line.RequestID, &held) +} + +// deny answers a question with what the visitor is to get for r. +func (rq *request) deny(r refusal) { + if r.status == 0 { + // SWWAF_BAN_RESPONSE is close. Written past rq.out, since the log line's + // status stays 0: the visitor is sent nothing. + rq.refused.CompareAndSwap(nil, &r) + rq.out.Header().Set(DispositionHeader, DispositionClose) + rq.out.ResponseWriter.WriteHeader(http.StatusOK) + + return + } + + rq.out.Header().Set(DispositionHeader, DispositionDeny) + rq.answer(r) +} + +// answerOwnEndpoint answers a request to servewaf's own address. No check +// runs on it: the address is on the private network, where the peers are +// rproxy and the admin's tools, not visitors. +func (h *handler) answerOwnEndpoint(w http.ResponseWriter, r *http.Request) { + if r.Method == http.MethodPost && r.URL.Path == ReportPath { + h.takeReport(w, r) + + return + } + + peer := peerAddress(r) + + rq := h.startRequest(w, r, peer, peer, false) + defer rq.finish() + + if r.Method == http.MethodGet && r.URL.Path == HealthPath { + rq.answerHealth() + + return + } + + rq.answerAdmin() +} + +// takeReport answers POST ReportPath: it finishes the request the report +// names, if it holds that request's decision, and ignores the report +// otherwise. +func (h *handler) takeReport(w http.ResponseWriter, r *http.Request) { + report, err := readReport(w, r) + if err != nil { + h.processLog.Warn("a report is refused", "error", err.Error()) + http.Error(w, "the body is not a report", http.StatusBadRequest) + + return + } + + held, found := h.awaitingReport.Peek(report.RequestID) + // Remove reports true to one report only, of several that name the id. + if found && held != nil && h.awaitingReport.Remove(report.RequestID) { + h.finishWith(held, report) + } + + w.WriteHeader(http.StatusNoContent) +} + +// readReport reads a report from r's body, which must be one JSON object with +// all of Report's fields, in the ranges SPEC.md gives. +func readReport(w http.ResponseWriter, r *http.Request) (Report, error) { + var report Report + + body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, reportBodyMaxBytes)) + if err != nil { + return report, fmt.Errorf("read the body: %w", err) + } + + var fields map[string]json.RawMessage + + err = json.Unmarshal(body, &fields) + if err != nil { + return report, fmt.Errorf("decode the body: %w", err) + } + + for _, name := range []string{ + "request_id", "status", "request_bytes", "response_bytes", + "duration_total", "duration_upstream_total", + } { + value, there := fields[name] + if !there || string(value) == "null" { + return report, fmt.Errorf("%w: %s", errReportFieldMissing, name) + } + } + + err = json.Unmarshal(body, &report) + if err != nil { + return report, fmt.Errorf("decode the body: %w", err) + } + + return report, report.checkRanges() +} + +// checkRanges returns an error for a value of the report that cannot be true. +func (report *Report) checkRanges() error { + switch { + case report.Status != 0 && + (report.Status < http.StatusContinue || report.Status > maxHTTPStatus): + return fmt.Errorf("%w: status", errReportValue) + case report.RequestBytes < 0 || report.RequestBytes > maxReportedBytes: + return fmt.Errorf("%w: request_bytes", errReportValue) + case report.ResponseBytes < 0 || report.ResponseBytes > maxReportedBytes: + return fmt.Errorf("%w: response_bytes", errReportValue) + case report.DurationTotal < 0 || report.DurationTotal > maxReportedMilliseconds: + return fmt.Errorf("%w: duration_total", errReportValue) + case report.DurationUpstreamTotal < 0 || + report.DurationUpstreamTotal > maxReportedMilliseconds: + return fmt.Errorf("%w: duration_upstream_total", errReportValue) + } + + return nil +} + +// finishWith finishes a request with what its report says, and writes the +// request's last log line, which holds its decision's fields and the report's. +// For an allowed request it first takes the steps ServeHTTP takes once a +// request passed to the app has ended, with the status and bytes of the report +// in place of those it would have seen. A denied request's counting was +// finished at its decision. +func (h *handler) finishWith(held *decision, report Report) { + rq := &request{ + decision: *held, + h: h, + out: &responseWriter{status: report.Status, bytes: report.ResponseBytes}, + body: &requestBody{}, + } + rq.body.bytes.Store(report.RequestBytes) + + rq.line.ServeWAFLine = requestlog.ServeWAFLineReport + rq.line.Status = report.Status + rq.line.RequestBytes = report.RequestBytes + rq.line.ResponseBytes = report.ResponseBytes + rq.line.DurationTotal = report.DurationTotal + + if rq.allowed { + rq.line.DurationUpstreamTotal = &report.DurationUpstreamTotal + // addToHistory goes by this for whether the request reached the app. + rq.upstreamStart = time.Now() + + rq.countBytes() + rq.countAnomalies() + rq.addToHistory() + + rq.h.metrics.RequestReported(&rq.line, + time.Duration(report.DurationTotal*float64(time.Millisecond)), + time.Duration(report.DurationUpstreamTotal*float64(time.Millisecond))) + } + + rq.writeLine() +} diff --git a/internal/proxy/servewaf_test.go b/internal/proxy/servewaf_test.go new file mode 100644 index 0000000..21dbcfb --- /dev/null +++ b/internal/proxy/servewaf_test.go @@ -0,0 +1,1511 @@ +package proxy_test + +import ( + "context" + "crypto/rand" + "encoding/json" + "fmt" + "io" + "maps" + "math" + "net" + "net/http" + "net/netip" + "reflect" + "strconv" + "strings" + "testing" + "time" + + "sneak.berlin/go/smallwebwaf/internal/alerts" + "sneak.berlin/go/smallwebwaf/internal/bans" + "sneak.berlin/go/smallwebwaf/internal/proxy" + "sneak.berlin/go/smallwebwaf/internal/ratelimit" + "sneak.berlin/go/smallwebwaf/internal/requestlog" +) + +const ( + // siteA and siteB are two sites behind one servewaf. + siteA = "a.example" + siteB = "b.example" + // rproxyPeer is what the tests' questions give as rproxy's TCP peer. + rproxyPeer = "172.18.0.2" + // heldLimit is how many answered requests "servewaf" in SPEC.md says it + // holds. + heldLimit = 10000 + // reportedBytesLimit is the most bytes for each way SPEC.md lets a report + // give. + reportedBytesLimit = 1 << 50 + // anomalyClientPerMinute is the anomaly threshold these tests set. + anomalyClientPerMinute = "SWWAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE" +) + +// The fields of a report, as "servewaf" in SPEC.md names them. +const ( + reportID = "request_id" + reportStatus = "status" + reportRequestBytes = "request_bytes" + reportResponseBytes = "response_bytes" + reportDuration = "duration_total" + reportUpstreamDuration = "duration_upstream_total" +) + +// serveWAF is a servewaf running for one test, on a clock set to midnight. +type serveWAF struct { + t *testing.T + addr string + out *output + server *proxy.Server + clock *clock + queue *alerts.Queue +} + +// startServeWAF starts servewaf with the settings in env, clients' AS numbers +// and countries looked up at geojsURL. +func startServeWAF(t *testing.T, geojsURL string, env map[string]string) *serveWAF { + t.Helper() + + return startServeWAFWith(t, geojsURL, env, func(*proxy.Server) {}) +} + +// startServeWAFWith is startServeWAF with change applied to the server before +// it starts. +func startServeWAFWith( + t *testing.T, geojsURL string, env map[string]string, change func(*proxy.Server), +) *serveWAF { + t.Helper() + + clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)} + settings := map[string]string{ + alertWebhookURL: "https://alerts.example/servewaf", + instanceName: alertInstance, + } + maps.Copy(settings, env) + + server, out, queue := newServerWith(t, proxy.NewServeWAF, + "http://127.0.0.1:8081", geojsURL, clk.Now, settings) + change(server) + + listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0") + if err != nil { + t.Fatalf("listen: %v", err) + } + + go func() { + _ = server.Serve(listener) + }() + + t.Cleanup(func() { + _ = server.Close() + }) + + return &serveWAF{ + t: t, addr: listener.Addr().String(), out: out, server: server, + clock: clk, queue: queue, + } +} + +// questionWith makes rproxy's question about a request for path on host, from +// the visitor at from, carrying body and a new id, as rproxy makes for every +// request. +func (s *serveWAF) questionWith( + method, host, from, path string, body io.Reader, +) *http.Request { + s.t.Helper() + + req := newRequest(s.t, method, s.addr, path, body) + req.Host = host + req.Header.Set(proxy.ClientHeader, from) + req.Header.Set(proxy.PeerHeader, rproxyPeer) + req.Header.Set(requestIDHeader, rand.Text()) + + return req +} + +// question makes rproxy's question about a request for path without a body. +func (s *serveWAF) question(host, from, path string) *http.Request { + s.t.Helper() + + return s.questionWith(http.MethodGet, host, from, path, http.NoBody) +} + +// ask sends the question about a request for path on siteA from the visitor at +// from, and returns the answer. +func (s *serveWAF) ask(from, path string) answer { + s.t.Helper() + + return do(s.t, s.question(siteA, from, path)) +} + +// allowed asks about a request for / on siteA, checks that the answer is allow, +// and returns the request's id. +func (s *serveWAF) allowed(from string) string { + s.t.Helper() + + req := s.question(siteA, from, "/") + wantDisposition(s.t, do(s.t, req), http.StatusOK, proxy.DispositionAllow) + + return req.Header.Get(requestIDHeader) +} + +// report sends rproxy's report and returns the answer. +func (s *serveWAF) report(report proxy.Report) answer { + s.t.Helper() + + body, err := json.Marshal(report) + if err != nil { + s.t.Fatalf("encode the report: %v", err) + } + + return s.reportBody(string(body)) +} + +// reportBody sends body as rproxy's report and returns the answer. +func (s *serveWAF) reportBody(body string) answer { + s.t.Helper() + + return do(s.t, newRequest(s.t, http.MethodPost, s.addr, proxy.ReportPath, + strings.NewReader(body))) +} + +// passedOn asks about a request from the visitor at from on siteA, and reports +// it as answered 200 with bodyBytes and answerBytes. +func (s *serveWAF) passedOn(from string) { + s.t.Helper() + + got := s.report(proxy.Report{ + RequestID: s.allowed(from), + Status: http.StatusOK, + RequestBytes: bodyBytes, + ResponseBytes: answerBytes, + }) + wantStatus(s.t, got, http.StatusNoContent) +} + +// history returns what the table of clients holds of client's history. +func (s *serveWAF) history() ratelimit.History { + held, _ := s.server.Limiter.Client(netip.MustParsePrefix(client + "/32")) + + return held.History +} + +// metrics returns what the metrics endpoint answers, for a server started with +// the token as SWWAF_METRICS_TOKEN. +func (s *serveWAF) metrics() string { + s.t.Helper() + + req := newRequest(s.t, http.MethodGet, s.addr, proxy.MetricsPath, http.NoBody) + req.Header.Set("Authorization", bearer) + + got := do(s.t, req) + wantStatus(s.t, got, http.StatusOK) + + return string(got.body) +} + +// wantDisposition checks an answer's status and disposition. +func wantDisposition(t *testing.T, got answer, status int, disposition string) { + t.Helper() + wantStatus(t, got, status) + + if answered := got.header.Get(proxy.DispositionHeader); answered != disposition { + t.Errorf("answered with disposition %q, want %q", answered, disposition) + } +} + +// wantBody checks an answer's body. +func wantBody(t *testing.T, got answer, want string) { + t.Helper() + + if string(got.body) != want { + t.Errorf("answered with the body %q, want %q", got.body, want) + } +} + +// wantProcessMessages checks that msg is among the process's messages count +// times, each from the server's log at the level warnings are. +func wantProcessMessages(t *testing.T, s *serveWAF, msg string, count int) { + t.Helper() + + found := 0 + + for _, fields := range s.out.lines(t) { + if fields["msg"] == msg && fields["level"] == "WARN" { + found++ + } + } + + if found != count { + t.Errorf("%d process warnings %q, want %d; output:\n%s", found, msg, count, + s.out.text()) + } +} + +// wantNoRequestLines checks that no request log line has been written. +func wantNoRequestLines(t *testing.T, s *serveWAF) { + t.Helper() + + for _, fields := range s.out.lines(t) { + if fields["type"] == requestType { + t.Errorf("a request log line was written: %v", fields) + } + } +} + +func TestQuestionAllowedIsAnsweredAndLogged(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + req := s.question(siteA, client, "/page?q=1&token=visible") + req.Header.Set(requestIDHeader, "the-id") + req.Header.Set("X-Forwarded-Proto", "https") + req.Header.Set(proxy.ContentLengthHeader, "1234") + + got := do(t, req) + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) + wantBody(t, got, "") + + if id := got.header.Get(requestIDHeader); id != "" { + t.Errorf("answered with the id %q, want none: rproxy makes the id", id) + } + + line := s.out.requestLine(t) + wantLine(t, line, http.StatusOK, requestlog.ActionForward) + + if line.ClientIP != client || line.PeerIP != rproxyPeer || line.Host != siteA || + line.Path != "/page" || line.Query != "q=1&token=visible" || + line.RequestID != "the-id" || line.Scheme != "https" || + line.ContentLength != 1234 || + line.ServeWAFLine != requestlog.ServeWAFLineDecision { + t.Errorf("log line is %+v", line.Line) + } +} + +func TestIPv6VisitorIsCountedInItsGroup(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{rateLimitPerMinute: "1"}) + + s.allowed("2001:db8::1") + + got := s.ask("2001:db8::2", "/") + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) +} + +func TestIPv4VisitorInIPv6FormIsTheIPv4Client(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{denyNets: client}) + + req := s.question(siteA, "::ffff:"+client, "/") + req.Header.Set(proxy.PeerHeader, "::ffff:"+rproxyPeer) + + wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny) + + line := s.out.requestLine(t) + if line.ClientIP != client || line.PeerIP != rproxyPeer { + t.Errorf("log line has client_ip %q and peer_ip %q", line.ClientIP, line.PeerIP) + } +} + +func TestMalformedQuestionGets400AndAWarningAndNoLine(t *testing.T) { + t.Parallel() + + for _, tc := range []struct { + name string + change func(*http.Request) + }{ + {"client not an address", func(req *http.Request) { + req.Header.Set(proxy.ClientHeader, "not an address") + }}, + {"client empty", func(req *http.Request) { + req.Header.Set(proxy.ClientHeader, "") + }}, + {"client address with a zone", func(req *http.Request) { + req.Header.Set(proxy.ClientHeader, "2001:db8::1%eth0") + }}, + {"id absent", func(req *http.Request) { + req.Header.Del(requestIDHeader) + }}, + {"id empty", func(req *http.Request) { + req.Header.Set(requestIDHeader, "") + }}, + {"id not UTF-8", func(req *http.Request) { + req.Header.Set(requestIDHeader, "not-utf-8-\xff") + }}, + {"id too long", func(req *http.Request) { + req.Header.Set(requestIDHeader, strings.Repeat("x", 129)) + }}, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{metricsToken: token}) + + req := s.question(siteA, client, "/") + tc.change(req) + wantDisposition(t, do(t, req), http.StatusBadRequest, "") + + wantProcessMessages(t, s, "a question is refused", 1) + wantNoRequestLines(t, s) + + metrics := s.metrics() + if strings.Contains(metrics, `status_class="4xx"`) { + t.Errorf("a refused question is counted in the metrics:\n%s", metrics) + } + }) + } +} + +func TestLongestValidIDIsTaken(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + req := s.question(siteA, client, "/") + req.Header.Set(requestIDHeader, strings.Repeat("x", 128)) + wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) +} + +func TestQuestionWithABodyOver64KiBGets400(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + over := strings.NewReader(strings.Repeat("x", proxy.QuestionBodyMaxBytes+1)) + got := do(t, s.questionWith(http.MethodPost, siteA, client, "/", over)) + wantDisposition(t, got, http.StatusBadRequest, "") + wantProcessMessages(t, s, "a question is refused", 1) + wantNoRequestLines(t, s) + + whole := strings.NewReader(strings.Repeat("x", proxy.QuestionBodyMaxBytes)) + got = do(t, s.questionWith(http.MethodPost, siteA, client, "/", whole)) + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) +} + +func TestQuestionWithAHeldIDIsRefused(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + id := s.allowed(client) + + again := s.question(siteA, client, "/") + again.Header.Set(requestIDHeader, id) + wantDisposition(t, do(t, again), http.StatusBadRequest, "") + wantProcessMessages(t, s, "a question is refused", 1) + + // A denied request's id is held as well. + denied := s.question(siteA, otherClient, proxy.HealthPath) + wantDisposition(t, do(t, denied), http.StatusNotFound, proxy.DispositionDeny) + + again = s.question(siteA, client, "/") + again.Header.Set(requestIDHeader, denied.Header.Get(requestIDHeader)) + wantDisposition(t, do(t, again), http.StatusBadRequest, "") + + // The report ends the holding, and the refused questions changed nothing. + got := s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes}) + wantStatus(t, got, http.StatusNoContent) + + if history := s.history(); history.ResponseBytes != answerBytes { + t.Errorf("history is %+v, want the first question's report", history) + } + + again = s.question(siteA, client, "/") + again.Header.Set(requestIDHeader, id) + wantDisposition(t, do(t, again), http.StatusOK, proxy.DispositionAllow) +} + +func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) { + t.Parallel() + + geojsURL, asked, release := startHeldGeoJS(t) + s := startServeWAF(t, geojsURL, map[string]string{ + lookupTimeout: "1h", deniedCountries: "kp", bytesLimitPerMinute: byteLimit, + }) + + first := s.question(siteA, client, "/") + id := first.Header.Get(requestIDHeader) + shared := newClient(t) + answered := make(chan answer, 1) + + go func() { + res, err := shared.Do(first) + if err != nil { + t.Errorf("question: %v", err) + + answered <- answer{} + + return + } + + answered <- readAnswer(res) + }() + + // The lookup holds the question up. + waitUntil(func() bool { return len(asked()) > 0 }) + + // A report for a question not yet answered is ignored, and that question's + // id is taken. + got := s.report(proxy.Report{RequestID: id, ResponseBytes: 1000}) + wantStatus(t, got, http.StatusNoContent) + + // This would wait on the same lookup were the id not taken. + ctx, cancel := context.WithTimeout(t.Context(), waitLimit) + defer cancel() + + again := s.question(siteA, client, "/").WithContext(ctx) + again.Header.Set(requestIDHeader, id) + wantDisposition(t, do(t, again), http.StatusBadRequest, "") + + release() + + select { + case got := <-answered: + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) + case <-time.After(waitLimit): + t.Fatal("the question got no answer") + } + + line := s.out.requestLine(t) + if line.ResponseBytes != 0 || line.ServeWAFLine != requestlog.ServeWAFLineDecision { + t.Errorf("the report reached the question being handled: %+v", line.Line) + } + + if held := s.server.Ledger.Snapshot(); len(held) != 0 { + t.Errorf("the ignored report made bans: %v", held) + } + + // The true report comes after, and is the one that counts. + got = s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes}) + wantStatus(t, got, http.StatusNoContent) + + if history := s.history(); history.ResponseBytes != answerBytes { + t.Errorf("history is %+v, want the true report's bytes", history) + } +} + +func TestPeerAbsentOrUnreadableIsAnsweredAndLoggedInvalid(t *testing.T) { + t.Parallel() + + for _, peer := range []string{"", "not an address"} { + t.Run(peer, func(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + req := s.question(siteA, client, "/") + req.Header.Del(proxy.PeerHeader) + + if peer != "" { + req.Header.Set(proxy.PeerHeader, peer) + } + + wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) + + if line := s.out.requestLine(t); line.PeerIP != "invalid IP" { + t.Errorf("log line has peer_ip %q, want invalid IP", line.PeerIP) + } + }) + } +} + +func TestContentLengthUnreadableOrNegativeCountsAsAbsent(t *testing.T) { + t.Parallel() + + for _, announced := range []string{"many", "-1", "1.5"} { + t.Run(announced, func(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + rulesDir: writeRules(t, "any-length header:Content-Length block .\n"), + }) + + req := s.question(siteA, client, "/") + req.Header.Set(proxy.ContentLengthHeader, announced) + wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) + + if line := s.out.requestLine(t); line.ContentLength != 0 { + t.Errorf("log line has content_length %d, want none", line.ContentLength) + } + }) + } +} + +func TestHeaderSentTwiceUsesItsFirstValue(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + req := s.question(siteA, client, "/") + req.Header.Add(proxy.ClientHeader, otherClient) + req.Header.Add(proxy.PeerHeader, "172.18.0.3") + req.Header.Set(requestIDHeader, "first") + req.Header.Add(requestIDHeader, "second") + req.Header.Set(proxy.ContentLengthHeader, "1234") + req.Header.Add(proxy.ContentLengthHeader, "5678") + wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) + + line := s.out.requestLine(t) + if line.ClientIP != client || line.PeerIP != rproxyPeer || + line.RequestID != "first" || line.ContentLength != 1234 { + t.Errorf("log line is %+v", line.Line) + } +} + +func TestSchemeAbsentIsLoggedAsHTTP(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + s.allowed(client) + + if line := s.out.requestLine(t); line.Scheme != "http" { + t.Errorf("log line has scheme %q, want http", line.Scheme) + } +} + +// sendQuestionWithPartOfItsBody sends a question over conn whose Content-Length +// announces more than it sends. +func sendQuestionWithPartOfItsBody(t *testing.T, conn net.Conn) { + t.Helper() + + _, err := fmt.Fprintf(conn, + "POST / HTTP/1.1\r\nHost: %s\r\n%s: %s\r\n%s: an-id\r\n"+ + "Content-Length: 100\r\n\r\npart", + siteA, proxy.ClientHeader, client, requestIDHeader) + if err != nil { + t.Fatalf("send the question: %v", err) + } +} + +func TestQuestionWhoseBodyStopsArrivingGetsNoAnswer(t *testing.T) { + t.Parallel() + + // Stalling would take the server's 60 seconds otherwise. + const readTimeout = time.Second + + for _, tc := range []struct { + name string + stop func(*testing.T, net.Conn) + }{ + {"it stalls", func(*testing.T, net.Conn) {}}, + {"rproxy stops sending", func(t *testing.T, conn net.Conn) { + t.Helper() + + err := conn.(*net.TCPConn).CloseWrite() //nolint:forcetypeassert // dial's + if err != nil { + t.Fatalf("close the sending side: %v", err) + } + }}, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + + s := startServeWAFWith(t, "", map[string]string{metricsToken: token}, + func(server *proxy.Server) { server.ReadTimeout = readTimeout }) + + conn := dial(t, s.addr) + sendQuestionWithPartOfItsBody(t, conn) + tc.stop(t, conn) + + err := conn.SetReadDeadline(time.Now().Add(waitLimit)) + if err != nil { + t.Fatalf("set the deadline: %v", err) + } + + sent, err := io.ReadAll(conn) + if err != nil || len(sent) != 0 { + t.Errorf("rproxy read %q (%v), want the connection closed unanswered", + sent, err) + } + + wantProcessMessages(t, s, "a question's body stopped arriving", 1) + wantNoRequestLines(t, s) + + // The id was not taken, and the metrics count nothing for it. + again := s.question(siteA, client, "/") + again.Header.Set(requestIDHeader, "an-id") + wantDisposition(t, do(t, again), http.StatusOK, proxy.DispositionAllow) + wantNoSeries(t, s.metrics(), + `smallwebwaf_size_and_time_limit_hits_total{instance="`+alertInstance+ + `",limit="SWWAF_CLIENT_REQUEST_TIMEOUT"}`) + }) + } +} + +func TestQuestionDeniedGetsTheBanResponse(t *testing.T) { + t.Parallel() + + for _, status := range []int{http.StatusForbidden, http.StatusTooManyRequests} { + t.Run(strconv.Itoa(status), func(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + denyNets: client, banResponse: strconv.Itoa(status), + }) + + got := s.ask(client, "/") + wantDisposition(t, got, status, proxy.DispositionDeny) + wantBody(t, got, http.StatusText(status)+"\n") + wantLine(t, s.out.requestLine(t), status, requestlog.ActionDenied) + }) + } +} + +func TestBanResponseCloseIsAnAnswerOfItsOwn(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + denyNets: client, banResponse: banResponseClose, metricsToken: token, + }) + + got := s.ask(client, "/") + wantDisposition(t, got, http.StatusOK, proxy.DispositionClose) + wantBody(t, got, "") + // The visitor is sent nothing. + wantLine(t, s.out.requestLine(t), 0, requestlog.ActionDenied) + wantMetric(t, s.metrics(), `smallwebwaf_requests_total{action="denied",instance="`+ + alertInstance+`",status_class="none"}`, 1) +} + +func TestBlockRuleIsDeniedWith403AndABanRuleWithTheBanResponse(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + rulesDir: writeRules(t, testRules), + banResponse: "429", + }) + + got := s.ask(client, "/blocked") + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) + wantLine(t, s.out.requestLine(t), http.StatusForbidden, + requestlog.ActionRuleBlocked) + + got = s.ask(client, "/.env") + wantDisposition(t, got, http.StatusTooManyRequests, proxy.DispositionDeny) + wantLine(t, s.out.requestLines(t, 2)[1], http.StatusTooManyRequests, + requestlog.ActionBanned) +} + +func TestQuestionUnderTheOwnPrefixIsDeniedWith404AfterTheChecks(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + metricsToken: token, denyNets: otherClient, + }) + + for _, path := range []string{proxy.HealthPath, proxy.MetricsPath, proxy.ReportPath} { + got := s.ask(client, path) + wantDisposition(t, got, http.StatusNotFound, proxy.DispositionDeny) + } + + wantLine(t, s.out.requestLine(t), http.StatusNotFound, requestlog.ActionAdmin) + + got := s.ask(otherClient, proxy.MetricsPath) + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) +} + +func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{mode: observe, denyNets: client}) + + got := s.ask(client, "/") + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) + + line := s.out.requestLine(t) + wantLine(t, line, http.StatusOK, requestlog.ActionForward) + wantWouldAction(t, line, requestlog.ActionDenied) +} + +func TestObserveModeRefusalAnsweredAllowCountsForTheErrorBurst(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + mode: observe, rulesDir: writeRules(t, testRules), errorBurstThreshold: "1", + }) + + for range 2 { + got := s.ask(client, "/blocked") + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) + } + + lines := s.out.requestLines(t, 2) + if lines[0].LimitHit != "" || lines[1].LimitHit != requestlog.LimitHitErrorBurst { + t.Errorf("log lines have limit_hit %q and %q, want none and %q", + lines[0].LimitHit, lines[1].LimitHit, requestlog.LimitHitErrorBurst) + } +} + +func TestAllowCarriesTheLookupHeadersWhileTheSettingIsSet(t *testing.T) { + t.Parallel() + + for _, tc := range []struct { + setting string + wantASN, wantCountry string + }{ + {"true", "AS64496", "DE"}, + {"false", "", ""}, + } { + t.Run(tc.setting, func(t *testing.T) { + t.Parallel() + + geojsURL, _ := startGeoJS(t) + s := startServeWAF(t, geojsURL, map[string]string{ + addLookupHeaders: tc.setting, lookupTimeout: "1h", deniedCountries: "kp", + }) + + got := s.ask(fromDE, "/") + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) + + asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country") + if asn != tc.wantASN || country != tc.wantCountry { + t.Errorf("answered with the AS number %q and the country %q, want %q and %q", + asn, country, tc.wantASN, tc.wantCountry) + } + }) + } +} + +func TestReportCountsTowardAByteLimitWhichThenBans(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit}) + + // The request's own question counts no bytes: it is the report that does. + id := s.allowed(client) + s.allowed(client) + + if held := s.server.Ledger.Snapshot(); len(held) != 0 { + t.Fatalf("banned before any report: %v", held) + } + + // 100 bytes are over the limit of 99. + got := s.report(proxy.Report{ + RequestID: id, Status: http.StatusBadGateway, + RequestBytes: bodyBytes, ResponseBytes: answerBytes, + }) + wantStatus(t, got, http.StatusNoContent) + + held := s.server.Ledger.Snapshot() + if len(held) != 1 || held[0].Cause != bans.CauseLimit || + held[0].Notes.Kind != ratelimit.KindBytes || + held[0].Notes.Request.Status != http.StatusBadGateway || + held[0].Notes.Request.Host != siteA || + held[0].Notes.Request.Path != "/" || + held[0].Notes.Request.Method != http.MethodGet { + t.Errorf("bans held are %+v", held) + } + + third := s.out.requestLines(t, 3)[2] + if third.ServeWAFLine != requestlog.ServeWAFLineReport || + third.LimitHit != minuteBytes || third.Offence != requestlog.OffenceLimit || + third.BanExpires == "" { + t.Errorf("the report's log line is %+v", third.Line) + } + + got = s.ask(client, "/") + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) +} + +func TestReportCountsAtThePercentageItsQuestionWasGiven(t *testing.T) { + t.Parallel() + + geojsURL, _ := startGeoJS(t) + // 100 bytes pass half of a limit of 99 and are within a limit of 199. + s := startServeWAF(t, geojsURL, map[string]string{ + bytesLimitPerMinute: "199", countryBytesPercent: countryDEHalf, + lookupTimeout: "1h", + }) + + s.passedOn(fromKP) + wantDisposition(t, s.ask(fromKP, "/"), http.StatusOK, proxy.DispositionAllow) + + s.passedOn(fromDE) + wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, proxy.DispositionDeny) +} + +func TestReportAddsTheRequestToItsClientsHistoryAndTheAnomalyCounters(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{anomalyClientPerMinute: "1000"}) + + id := s.allowed(client) + + if requests := s.history().Requests; requests != 0 { + t.Errorf("history holds %d requests before the report, want none", requests) + } + + if counters := s.server.Anomalies.Snapshot(); len(counters) != 0 { + t.Errorf("anomaly counters %v before the report, want none", counters) + } + + got := s.report(proxy.Report{ + RequestID: id, Status: http.StatusBadGateway, + RequestBytes: bodyBytes, ResponseBytes: answerBytes, + }) + wantStatus(t, got, http.StatusNoContent) + + history := s.history() + if history.Requests != 1 || history.Forwarded != 1 || + history.RequestBytes != bodyBytes || history.ResponseBytes != answerBytes || + history.Responses.Status5xx != 1 { + t.Errorf("history is %+v", history) + } + + if counters := s.server.Anomalies.Snapshot(); len(counters) != 1 { + t.Errorf("anomaly counters %v, want the client's", counters) + } +} + +func TestDeniedRequestIsFinishedAtItsDecision(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + rateLimitPerMinute: "1", metricsToken: token, anomalyClientPerMinute: "1000", + }) + + s.allowed(client) + + denied := s.question(siteA, client, "/") + wantDisposition(t, do(t, denied), http.StatusForbidden, proxy.DispositionDeny) + + counters := s.server.Anomalies.Snapshot() + if len(counters) != 1 || counters[0].Minute.Current != 1 { + t.Errorf("anomaly counters %+v, want the denied request's", counters) + } + + history := s.history() + if history.Requests != 1 || history.Refused != 1 { + t.Errorf("history is %+v, want the refused request alone", history) + } + + const series = `smallwebwaf_requests_total{action="rate_limited",instance="` + + alertInstance + `",status_class="4xx"}` + + wantMetric(t, s.metrics(), series, 1) + + got := s.report(proxy.Report{ + RequestID: denied.Header.Get(requestIDHeader), Status: http.StatusForbidden, + ResponseBytes: 1000, DurationTotal: 2, + }) + wantStatus(t, got, http.StatusNoContent) + + if after := s.history(); after != history { + t.Errorf("the report changed the history to %+v, from %+v", after, history) + } + + counters = s.server.Anomalies.Snapshot() + if len(counters) != 1 || counters[0].Minute.Current != 1 || + counters[0].MinuteBytes.Current != 0 { + t.Errorf("the report changed the anomaly counters: %+v", counters) + } + + wantMetric(t, s.metrics(), series, 1) +} + +func TestMetricsCountAnAllowedRequestFromItsReport(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{metricsToken: token}) + + id := s.allowed(client) + + const series = `{action="forward",instance="` + alertInstance + `",status_class="5xx"}` + + metrics := s.metrics() + wantNoSeries(t, metrics, "smallwebwaf_requests_total"+series) + wantMetric(t, metrics, `smallwebwaf_requests_in_flight{instance="`+alertInstance+`"}`, + 1) // the scrape itself + + got := s.report(proxy.Report{ + RequestID: id, Status: http.StatusBadGateway, + RequestBytes: bodyBytes, ResponseBytes: answerBytes, + DurationTotal: 5, DurationUpstreamTotal: 4, + }) + wantStatus(t, got, http.StatusNoContent) + + metrics = s.metrics() + wantMetric(t, metrics, "smallwebwaf_requests_total"+series, 1) + wantMetric(t, metrics, "smallwebwaf_request_bytes_total"+series, bodyBytes) + wantMetric(t, metrics, "smallwebwaf_response_bytes_total"+series, answerBytes) + wantMetric(t, metrics, `smallwebwaf_upstream_duration_seconds_sum{instance="`+ + alertInstance+`"}`, 0.004) +} + +func TestReportForARequestNotHeldIsIgnored(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit}) + + got := s.report(proxy.Report{RequestID: "no-such-request", ResponseBytes: 1000}) + wantStatus(t, got, http.StatusNoContent) + + if clients := s.server.Limiter.Snapshot(); len(clients) != 0 { + t.Errorf("the table of clients holds %v", clients) + } + + if held := s.server.Ledger.Snapshot(); len(held) != 0 { + t.Errorf("bans held are %v", held) + } + + wantNoRequestLines(t, s) + wantProcessMessages(t, s, "a report is refused", 0) + + // A second report of the same request does not count it twice. + id := s.allowed(client) + for range 2 { + got = s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes}) + wantStatus(t, got, http.StatusNoContent) + } + + if counted := s.history().ResponseBytes; counted != answerBytes { + t.Errorf("history holds %d response bytes, want %d", counted, answerBytes) + } + + // The question's line and one for the first report, none for the second. + if lines := s.out.requestLines(t, 2); len(lines) != 2 { + t.Errorf("request log lines are %+v, want two", lines) + } +} + +// reportFields is the JSON of a report of the request id with a true value for +// every field, for a test to change and leave out. +func reportFields(id string) map[string]any { + return map[string]any{ + reportID: id, reportStatus: http.StatusOK, + reportRequestBytes: bodyBytes, reportResponseBytes: answerBytes, + reportDuration: 5, reportUpstreamDuration: 4, + } +} + +// encodedReport returns a report of id, with the value of each field in changed +// in place of its true one, and without the fields named in missing. +func encodedReport( + t *testing.T, id string, changed map[string]any, missing ...string, +) string { + t.Helper() + + fields := reportFields(id) + maps.Copy(fields, changed) + + for _, name := range missing { + delete(fields, name) + } + + encoded, err := json.Marshal(fields) + if err != nil { + t.Fatalf("encode the report: %v", err) + } + + return string(encoded) +} + +func TestReportThatIsNotOneObjectWithAllItsFieldsIsRefused(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + id := s.allowed(client) + whole := encodedReport(t, id, nil) + fields := reportFields(id) + bodies := make([]string, 0, 10+3*len(fields)) + bodies = append(bodies, + "", "not JSON", "null", "[]", `"text"`, "{}", + whole+" {}", + whole+whole, + `{"request_id":"`+id+`"}`, + `{"request_id":"`+id+`","padding":"`+strings.Repeat("x", 1<<20)+`"}`) + + for name := range fields { + bodies = append(bodies, + encodedReport(t, id, nil, name), + encodedReport(t, id, map[string]any{name: nil}), + encodedReport(t, id, map[string]any{name: []int{1}})) + } + + for _, body := range bodies { + wantStatus(t, s.reportBody(body), http.StatusBadRequest) + } + + wantProcessMessages(t, s, "a report is refused", len(bodies)) + + // None of those finished the request, so its true report still counts. + wantStatus(t, s.reportBody(whole), http.StatusNoContent) + + if counted := s.history().ResponseBytes; counted != answerBytes { + t.Errorf("history holds %d response bytes, want %d", counted, answerBytes) + } +} + +func TestReportWithAValueOutOfRangeIsRefused(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + id := s.allowed(client) + cases := []struct { + field string + value any + }{ + {reportStatus, -5}, + {reportStatus, 99}, + {reportStatus, 600}, + {reportStatus, 99999}, + {reportStatus, 200.5}, + {reportStatus, "200"}, + {reportRequestBytes, -1}, + {reportRequestBytes, reportedBytesLimit + 1}, + {reportRequestBytes, int64(math.MaxInt64)}, + {reportRequestBytes, 1.5}, + {reportResponseBytes, -1}, + {reportResponseBytes, reportedBytesLimit + 1}, + {reportResponseBytes, int64(math.MaxInt64)}, + {reportDuration, -1}, + {reportDuration, 1e13}, + {reportDuration, "5"}, + {reportUpstreamDuration, -0.5}, + {reportUpstreamDuration, 1e13}, + } + + for _, tc := range cases { + body := encodedReport(t, id, map[string]any{tc.field: tc.value}) + wantStatus(t, s.reportBody(body), http.StatusBadRequest) + } + + wantProcessMessages(t, s, "a report is refused", len(cases)) + + if held := s.history(); held.Requests != 0 { + t.Errorf("a refused report counted the request: %+v", held) + } +} + +func TestReportValuesAtTheEndsOfTheirRangesAreTaken(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + bytesLimitPerMinute: off, bytesLimitPerHour: off, bytesLimitPerDay: off, + }) + + for _, tc := range []struct { + field string + value any + }{ + {reportStatus, 0}, + {reportStatus, 100}, + {reportStatus, 599}, + {reportRequestBytes, 0}, + {reportRequestBytes, reportedBytesLimit}, + {reportResponseBytes, reportedBytesLimit}, + {reportDuration, 0}, + {reportDuration, 1e12}, + {reportUpstreamDuration, 1e12}, + } { + id := s.allowed(client) + body := encodedReport(t, id, map[string]any{tc.field: tc.value}) + wantStatus(t, s.reportBody(body), http.StatusNoContent) + } + + // A field of no meaning here is read as if it were not there. + id := s.allowed(client) + body := encodedReport(t, id, map[string]any{"some_later_field": 1}) + wantStatus(t, s.reportBody(body), http.StatusNoContent) + wantProcessMessages(t, s, "a report is refused", 0) +} + +func TestBytesAtTheirMostEachWayBanRatherThanOverflow(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit}) + + got := s.report(proxy.Report{ + RequestID: s.allowed(client), Status: http.StatusOK, + RequestBytes: reportedBytesLimit, ResponseBytes: reportedBytesLimit, + }) + wantStatus(t, got, http.StatusNoContent) + + if held := s.server.Ledger.Snapshot(); len(held) != 1 { + t.Errorf("bans held are %v, want the client's", held) + } +} + +// wantReportsFields checks that line has what the report in +// TestReportWritesTheLastLineWithTheDecisionsAndTheReportsFields says. +func wantReportsFields(t *testing.T, line logLine) { + t.Helper() + + if line.ServeWAFLine != requestlog.ServeWAFLineReport || + line.Status != http.StatusCreated || + line.RequestBytes != bodyBytes || line.ResponseBytes != answerBytes || + line.DurationTotal != 12.5 || line.DurationUpstreamTotal == nil || + *line.DurationUpstreamTotal != 10.25 { + t.Errorf("the report's line lacks the report's fields: %+v", line.Line) + } +} + +// wantSameDecision checks that the report's line is the decision's, but for the +// fields a report says and the byte totals its counting changes. +func wantSameDecision(t *testing.T, reported, decided logLine) { + t.Helper() + + same := reported.Line + same.ServeWAFLine = decided.ServeWAFLine + same.Status = decided.Status + same.RequestBytes = decided.RequestBytes + same.ResponseBytes = decided.ResponseBytes + same.DurationTotal = decided.DurationTotal + same.DurationUpstreamTotal = decided.DurationUpstreamTotal + same.Counts.MinuteBytes = decided.Counts.MinuteBytes + same.Counts.HourBytes = decided.Counts.HourBytes + same.Counts.DayBytes = decided.Counts.DayBytes + + if !reflect.DeepEqual(same, decided.Line) { + t.Errorf("the report's line lacks the decision's fields:\n%+v\n%+v", + reported.Line, decided.Line) + } +} + +// wantDecisionsFields checks what servewaf answered and noted on the decision's +// line, of a request with these fields. +func wantDecisionsFields(t *testing.T, line logLine, id, query, accept string) { + t.Helper() + + if line.ServeWAFLine != requestlog.ServeWAFLineDecision || + line.Status != http.StatusOK || line.ResponseBytes != 0 || + line.DurationUpstreamTotal != nil || line.DurationChecks == nil || + line.RequestID != id || line.Query != query || + !line.HasAuthorization || !line.HasCookie || + line.RequestHeaders["accept"] != accept || len(line.RuleIDs) != 1 || + line.Counts.Minute != 1 { + t.Errorf("the decision's line is %+v", line.Line) + } +} + +func TestReportWritesTheLastLineWithTheDecisionsAndTheReportsFields(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + logRequestHeaders: "accept", + rulesDir: writeRules(t, "seen path log ^/page$\n"), + }) + + const ( + secret = "Bearer the-visitors-secret" + cookie = "session=the-visitors-cookie" + verbatim = "q=1&token=visible" + accept = "text/css" + ) + + req := s.question(siteA, client, "/page?"+verbatim) + req.Header.Set("Accept", accept) + req.Header.Set("Authorization", secret) + req.Header.Set("Cookie", cookie) + req.Header.Set("X-Forwarded-Proto", "https") + id := req.Header.Get(requestIDHeader) + + wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) + + got := s.report(proxy.Report{ + RequestID: id, Status: http.StatusCreated, + RequestBytes: bodyBytes, ResponseBytes: answerBytes, + DurationTotal: 12.5, DurationUpstreamTotal: 10.25, + }) + wantStatus(t, got, http.StatusNoContent) + + lines := s.out.requestLines(t, 2) + decided, reported := lines[0], lines[1] + + wantDecisionsFields(t, decided, id, verbatim, accept) + wantReportsFields(t, reported) + wantSameDecision(t, reported, decided) + + // The report's line has the request's bytes in the totals, which the + // decision could not know. + if decided.Counts.DayBytes != 0 || reported.Counts.DayBytes != 100 { + t.Errorf("the lines' day_bytes are %v and %v, want 0 and 100", + decided.Counts.DayBytes, reported.Counts.DayBytes) + } + + // Neither line holds a credential. + text := s.out.text() + if strings.Contains(text, "the-visitors-secret") || + strings.Contains(text, "the-visitors-cookie") { + t.Errorf("a line holds a credential:\n%s", text) + } +} + +func TestEveryDeniedRequestsReportGetsItsLastLine(t *testing.T) { + t.Parallel() + + for _, tc := range []struct { + name string + env map[string]string + path string + status int + action string + }{ + { + "deny", map[string]string{denyNets: client}, "/", http.StatusForbidden, + requestlog.ActionDenied, + }, + { + "close", map[string]string{denyNets: client, banResponse: banResponseClose}, "/", + 0, requestlog.ActionDenied, + }, + {"own prefix", nil, proxy.HealthPath, http.StatusNotFound, requestlog.ActionAdmin}, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", tc.env) + + req := s.question(siteA, client, tc.path) + id := req.Header.Get(requestIDHeader) + do(t, req) + + got := s.report(proxy.Report{ + RequestID: id, Status: tc.status, ResponseBytes: 8, DurationTotal: 3, + }) + wantStatus(t, got, http.StatusNoContent) + + line := s.out.requestLines(t, 2)[1] + wantLine(t, line, tc.status, tc.action) + + if line.ServeWAFLine != requestlog.ServeWAFLineReport || + line.RequestID != id || line.ResponseBytes != 8 || line.DurationTotal != 3 || + line.DurationUpstreamTotal != nil { + t.Errorf("the report's line is %+v", line.Line) + } + }) + } +} + +func TestOldestAnsweredRequestIsDroppedPastTheBound(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + rateLimitExemptNets: otherClient, + }) + + oldest := s.allowed(client) + next := s.allowed(client) + + // One connection for them all, as rproxy keeps one open. + shared := newClient(t) + + for range heldLimit - 1 { + res, err := shared.Do(s.question(siteA, otherClient, "/")) + if err != nil { + t.Fatalf("question: %v", err) + } + + wantDisposition(t, readAnswer(res), http.StatusOK, proxy.DispositionAllow) + } + + // The table is full again with next and those, without oldest. + got := s.report(proxy.Report{RequestID: oldest, ResponseBytes: answerBytes}) + wantStatus(t, got, http.StatusNoContent) + + if counted := s.history().Requests; counted != 0 { + t.Errorf("history holds %d requests, want the dropped one not counted", counted) + } + + got = s.report(proxy.Report{RequestID: next, ResponseBytes: answerBytes}) + wantStatus(t, got, http.StatusNoContent) + + if counted := s.history().Requests; counted != 1 { + t.Errorf("history holds %d requests, want the one still held", counted) + } +} + +func TestBanMadeThroughOneHostRefusesTheClientOnAnother(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{rulesDir: writeRules(t, testRules)}) + + got := do(t, s.question(siteA, client, "/.env")) + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) + + got = do(t, s.question(siteB, client, "/")) + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) + + lines := s.out.requestLines(t, 2) + if lines[0].Host != siteA || lines[1].Host != siteB { + t.Errorf("log lines have the hosts %q and %q", lines[0].Host, lines[1].Host) + } + + held := s.server.Ledger.Snapshot() + if len(held) != 1 || held[0].Notes.Request.Host != siteA { + t.Errorf("bans held are %+v, want one whose notes name %s", held, siteA) + } + + waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook] + if len(waiting) == 0 || waiting[0].Host != siteA { + t.Errorf("alerts waiting are %+v, want the ban's, naming %s", waiting, siteA) + } +} + +func TestRequestsOnEverySiteCountTowardOneSetOfLimits(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{rateLimitPerMinute: "1"}) + + s.allowed(client) + + got := do(t, s.question(siteB, client, "/")) + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) +} + +func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) { + t.Parallel() + + // Were a check run on its requests, localhost's would be refused. + s := startServeWAF(t, "", map[string]string{ + metricsToken: token, adminToken: adminSecret, denyNets: localhost, + }) + + for _, tc := range []struct { + path, authorization string + want int + }{ + {proxy.HealthPath, "", http.StatusOK}, + {proxy.MetricsPath, bearer, http.StatusOK}, + {proxy.MetricsPath, "", http.StatusUnauthorized}, + {proxy.BansPath, adminBearer, http.StatusOK}, + {proxy.BansPath, "", http.StatusUnauthorized}, + {proxy.ClientsPath + client, adminBearer, http.StatusOK}, + {proxy.ReportPath, "", http.StatusNotFound}, + {"/_smallwebwaf/other", "", http.StatusNotFound}, + {"/", "", http.StatusNotFound}, + } { + req := newRequest(t, http.MethodGet, s.addr, tc.path, http.NoBody) + req.Header.Set("Authorization", tc.authorization) + wantDisposition(t, do(t, req), tc.want, "") + + // The same request carrying the header is a question about it. + req = s.question(siteA, client, tc.path) + req.Header.Set("Authorization", tc.authorization) + + if strings.HasPrefix(tc.path, "/_smallwebwaf/") { + wantDisposition(t, do(t, req), http.StatusNotFound, proxy.DispositionDeny) + } else { + wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) + } + } +} + +func TestOwnEndpointsLinesDoNotSayWhichOfTheTwoTheyAre(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", nil) + + wantStatus(t, get(t, s.addr, proxy.HealthPath), http.StatusOK) + + line := s.out.requestLine(t) + if _, there := line.fields["servewaf_line"]; there || line.Path != proxy.HealthPath { + t.Errorf("the health check's log line is %+v", line.fields) + } +} + +func TestSettingsOfTheLimitsAndTheProxiesAreIgnored(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + requestMaxBytes: "1", + clientHeaderMaxBytes: "5K", + clientRequestTimeout: off, + clientResponseTimeout: off, + trustedProxies: localhost, + }) + + req := s.questionWith(http.MethodPost, siteA, client, "/", + strings.NewReader("a body")) + req.Header.Set(proxy.ContentLengthHeader, "1073741824") + req.Header.Set(forwardedFor, otherClient) + req.Header.Set("X-Long", strings.Repeat("x", 40<<10)) + + got := do(t, req) + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) + + line := s.out.requestLine(t) + if line.ClientIP != client || line.ContentLength != 1<<30 { + t.Errorf("log line has client_ip %q and content_length %d", + line.ClientIP, line.ContentLength) + } +} + +func TestChecksSeeTheHeadersTheVisitorSent(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + rulesDir: writeRules(t, ` +own-header header:X-Smallwebwaf-Client block . +announced header:Content-Length block ^70000$ +carried header:Content-Length block ^6$ +`), + }) + + // What the question's own Content-Length says, 6, is not what the visitor + // announced. + for _, tc := range []struct { + announced string + want int + }{ + {"70000", http.StatusForbidden}, + {"", http.StatusOK}, + {"6000", http.StatusOK}, + } { + req := s.questionWith(http.MethodPost, siteA, client, "/", + strings.NewReader("a body")) + + if tc.announced != "" { + req.Header.Set(proxy.ContentLengthHeader, tc.announced) + } + + wantStatus(t, do(t, req), tc.want) + } +} + +func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{ + wafMode: block, wafBodyLimit: "1M", + }) + + ask := func(contentType, body string) answer { + req := s.questionWith(http.MethodPost, siteA, client, "/", + strings.NewReader(body)) + req.Header.Set("Content-Type", contentType) + req.Header.Set(proxy.ContentLengthHeader, "100000") + + return do(t, req) + } + + got := ask(formData, "id=1'%20OR%20'1'='1") + wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) + + // The first 64 KiB of a longer JSON body cannot be parsed, which is no + // attack, so the Core Rule Set does not read it. + cut := `{"text":"` + strings.Repeat("x", proxy.QuestionBodyMaxBytes) + got = ask("application/json", cut[:proxy.QuestionBodyMaxBytes]) + wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) +} + +func TestAnomalyAlertNamesTheSite(t *testing.T) { + t.Parallel() + + s := startServeWAF(t, "", map[string]string{anomalyClientPerMinute: "1"}) + + for range 2 { + s.passedOn(client) + } + + waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook] + if len(waiting) != 1 || waiting[0].Event != alerts.EventAnomaly || + waiting[0].Host != siteA { + t.Errorf("alerts waiting are %+v, want an anomaly naming %s", waiting, siteA) + } +} diff --git a/internal/requestlog/requestlog.go b/internal/requestlog/requestlog.go index 829f8a0..c05c0c2 100644 --- a/internal/requestlog/requestlog.go +++ b/internal/requestlog/requestlog.go @@ -1,6 +1,6 @@ // Package requestlog writes the lines smallwebwaf prints on stdout: one -// JSON object per request, marked "type":"request", and the process's own -// messages as JSON lines marked "type":"process". +// JSON object per request, or two from servewaf, marked "type":"request", and +// the process's own messages as JSON lines marked "type":"process". package requestlog import ( @@ -60,6 +60,16 @@ const OffenceLimit = "limit" // that broke the error burst. const LimitHitErrorBurst = "error_burst" +// The servewaf_line a request line names, which says which of servewaf's two +// lines about a question it is. +const ( + // ServeWAFLineDecision is servewaf's line at its decision. + ServeWAFLineDecision = "decision" + // ServeWAFLineReport is servewaf's line when rproxy's report comes, with + // the decision's fields and the report's together. + ServeWAFLineReport = "report" +) + // timeLayout is RFC 3339 with milliseconds. const timeLayout = "2006-01-02T15:04:05.000Z07:00" @@ -70,6 +80,9 @@ const timeLayout = "2006-01-02T15:04:05.000Z07:00" //nolint:tagliatelle // docs/request-log.md names the fields in snake_case type Line struct { Type string `json:"type"` + // ServeWAFLine is ServeWAFLineDecision or ServeWAFLineReport on servewaf's + // two lines about a question. + ServeWAFLine string `json:"servewaf_line,omitempty"` // The standard web log fields. Scheme is how the client reached // smallwebwaf, or the trusted proxy in front of it. diff --git a/internal/smallwebwaf/servewaf_test.go b/internal/smallwebwaf/servewaf_test.go new file mode 100644 index 0000000..6f6f5da --- /dev/null +++ b/internal/smallwebwaf/servewaf_test.go @@ -0,0 +1,210 @@ +package smallwebwaf_test + +import ( + "context" + "crypto/rand" + "io" + "net/http" + "os" + "os/exec" + "path/filepath" + "strings" + "syscall" + "testing" + "time" + + "sneak.berlin/go/smallwebwaf/internal/proxy" + "sneak.berlin/go/smallwebwaf/internal/smallwebwaf" +) + +// commandForMain, while set in the environment, has this test binary run Main +// with the command it gives, in place of the tests, so that a test can run +// Main in a process of its own. +const commandForMain = "SMALLWEBWAF_TEST_COMMAND" + +func TestMain(m *testing.M) { + command, set := os.LookupEnv(commandForMain) + if !set { + os.Exit(m.Run()) + } + + os.Args = strings.Fields("smallwebwaf " + command) + + os.Exit(smallwebwaf.Main(testVersion)) +} + +func TestMainAnswersQuestionsOnlyAsServeWAF(t *testing.T) { + t.Parallel() + + for _, tc := range []struct { + command, disposition string + }{ + {"servewaf", proxy.DispositionAllow}, + {"", ""}, + } { + t.Run("command "+tc.command, func(t *testing.T) { + t.Parallel() + + binary, err := os.Executable() + if err != nil { + t.Fatalf("find the test binary: %v", err) + } + + ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit) + defer cancel() + + out := &output{} + cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary + cmd.Stdout = out + + cmd.Env = append(os.Environ(), + commandForMain+"="+tc.command, + listenAddr+"="+localhost+":0", + upstreamURL+"="+startApp(t), + stateDir+"="+t.TempDir(), + rulesDir+"="+t.TempDir(), + lookupSource+"=off") + + err = cmd.Start() + if err != nil { + t.Fatalf("start: %v", err) + } + + t.Cleanup(func() { + _ = cmd.Process.Kill() + _ = cmd.Wait() + }) + + addr, _ := out.line(t, "msg", "starting")["address"].(string) + ask(t, addr, "/", http.StatusOK, tc.disposition) + + err = cmd.Process.Signal(syscall.SIGTERM) + if err != nil { + t.Fatalf("signal: %v", err) + } + + err = cmd.Wait() + if err != nil { + t.Errorf("exit: %v; output:\n%s", err, out.text()) + } + }) + } +} + +// ask sends servewaf at addr rproxy's question about a request for path from +// placed, checks the answer's status and disposition, and closes it. +func ask(t *testing.T, addr, path string, status int, disposition string) { + t.Helper() + + req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, + "http://"+addr+path, http.NoBody) + if err != nil { + t.Fatalf("new request: %v", err) + } + + req.Header.Set(proxy.ClientHeader, placed) + req.Header.Set("X-Request-ID", rand.Text()) + + transport := &http.Transport{} + defer transport.CloseIdleConnections() + + res, err := (&http.Client{Transport: transport}).Do(req) + if err != nil { + t.Fatalf("question: %v", err) + } + + _, _ = io.Copy(io.Discard, res.Body) + _ = res.Body.Close() + + if res.StatusCode != status || + res.Header.Get(proxy.DispositionHeader) != disposition { + t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode, + res.Header.Get(proxy.DispositionHeader), status, disposition) + } +} + +// runServeWAF runs `smallwebwaf servewaf` with the settings in env, has use +// send it requests at the address it listens on, stops it as SIGTERM does, and +// checks that it stopped in order. +func runServeWAF(t *testing.T, env map[string]string, use func(addr string)) { + t.Helper() + + ctx, stop := context.WithCancel(t.Context()) + out := &output{} + exited := make(chan int, 1) + + go func() { + exited <- runAs(ctx, true, env, out) + }() + + addr, _ := out.line(t, "msg", "starting")["address"].(string) + use(addr) + stop() + + select { + case status := <-exited: + if status != 0 { + t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text()) + } + case <-time.After(waitLimit): + t.Fatal("still running after being told to stop") + } + + out.line(t, "msg", "stopped") +} + +func TestServeWAFKeepsItsBansAcrossARestartAndNeedsNoApp(t *testing.T) { + t.Parallel() + + env := map[string]string{ + listenAddr: localhost + ":0", + stateDir: t.TempDir(), + rulesDir: filepath.Join("..", "..", "share", "rules.d"), + } + + runServeWAF(t, env, func(addr string) { + ask(t, addr, "/", http.StatusOK, proxy.DispositionAllow) + // The default rule file bans a probe for this. + ask(t, addr, "/.env", http.StatusForbidden, proxy.DispositionDeny) + }) + + runServeWAF(t, env, func(addr string) { + ask(t, addr, "/", http.StatusForbidden, proxy.DispositionDeny) + }) +} + +func TestWithoutACommandTheClientHeaderIsNoQuestion(t *testing.T) { + t.Parallel() + + runUntilStopped(t, map[string]string{ + listenAddr: localhost + ":0", + upstreamURL: startApp(t), + stateDir: t.TempDir(), + rulesDir: t.TempDir(), + }, func(url string) { + req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, url, + http.NoBody) + if err != nil { + t.Fatalf("new request: %v", err) + } + + req.Header.Set(proxy.ClientHeader, placed) + + transport := &http.Transport{} + defer transport.CloseIdleConnections() + + res, err := (&http.Client{Transport: transport}).Do(req) + if err != nil { + t.Fatalf("request: %v", err) + } + + body, err := io.ReadAll(res.Body) + _ = res.Body.Close() + + if err != nil || string(body) != greeting || + res.Header.Get(proxy.DispositionHeader) != "" { + t.Errorf("answered %q (%v) with disposition %q, want the app's greeting", + body, err, res.Header.Get(proxy.DispositionHeader)) + } + }) +} diff --git a/internal/smallwebwaf/smallwebwaf.go b/internal/smallwebwaf/smallwebwaf.go index ab43736..3545ef4 100644 --- a/internal/smallwebwaf/smallwebwaf.go +++ b/internal/smallwebwaf/smallwebwaf.go @@ -1,7 +1,8 @@ // Package smallwebwaf runs the smallwebwaf process: it reads the settings, // the rule files, the lookup database and the state files, serves requests // until it is told to stop, and then stops in an orderly way, writing the -// state files. +// state files. It serves them in front of an app, or, as `smallwebwaf +// servewaf`, by answering rproxy's questions about them. package smallwebwaf import ( @@ -45,11 +46,15 @@ type Params struct { LookupEnv func(string) (string, bool) // Stdout receives the request log and the process's own messages. Stdout io.Writer + // ServeWAF makes it `smallwebwaf servewaf`, which answers rproxy's + // questions, rather than the proxy in front of an app. + ServeWAF bool } // Main runs smallwebwaf until SIGTERM or SIGINT, and returns the -// process's exit status. Run as `smallwebwaf healthcheck`, it is the -// container's health check instead. +// process's exit status. Run as `smallwebwaf servewaf`, it answers +// rproxy's questions rather than pass requests to an app. Run as +// `smallwebwaf healthcheck`, it is the container's health check instead. func Main(version string) int { if len(os.Args) > 1 && os.Args[1] == "healthcheck" { return HealthCheck(context.Background(), os.Args[2:], os.LookupEnv, os.Stderr) @@ -63,6 +68,7 @@ func Main(version string) int { Version: version, LookupEnv: os.LookupEnv, Stdout: os.Stdout, + ServeWAF: len(os.Args) > 1 && os.Args[1] == "servewaf", }) } @@ -117,7 +123,12 @@ func Run(ctx context.Context, params Params) int { return 1 } - server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue) + newProxy := proxy.New + if params.ServeWAF { + newProxy = proxy.NewServeWAF + } + + server, err := newServer(newProxy, cfg, stdout, processLog, now, ruleFiles, alertQueue) if err != nil { processLog.Error("cannot use the lookup database", "error", err.Error()) @@ -151,10 +162,12 @@ func Run(ctx context.Context, params Params) int { return serve(ctx, server, listener, files, ruleFiles, alertQueue, processLog) } -// newServer returns the server smallwebwaf runs, with the metrics of the -// alerts, after reading the lookup database while SWWAF_LOOKUP_SOURCE is -// file. A lookup database that cannot be read is an error. +// newServer returns the server smallwebwaf runs, which newProxy makes, with +// the metrics of the alerts, after reading the lookup database while +// SWWAF_LOOKUP_SOURCE is file. A lookup database that cannot be read is an +// error. func newServer( + newProxy func(proxy.Params) *proxy.Server, cfg *config.Config, stdout io.Writer, processLog *slog.Logger, now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue, ) (*proxy.Server, error) { @@ -174,7 +187,7 @@ func newServer( } } - server := proxy.New(proxy.Params{ + server := newProxy(proxy.Params{ Config: cfg, RequestLog: stdout, ProcessLog: processLog, diff --git a/internal/smallwebwaf/smallwebwaf_test.go b/internal/smallwebwaf/smallwebwaf_test.go index 0aee935..690af0f 100644 --- a/internal/smallwebwaf/smallwebwaf_test.go +++ b/internal/smallwebwaf/smallwebwaf_test.go @@ -113,6 +113,13 @@ func (o *output) text() string { // returns its exit status. SWWAF_LOOKUP_SOURCE is off unless env sets it, // so that no test sends GeoJS its clients' addresses. func run(ctx context.Context, env map[string]string, out *output) int { + return runAs(ctx, false, env, out) +} + +// runAs is run as `smallwebwaf servewaf` while serveWAF is true. +func runAs( + ctx context.Context, serveWAF bool, env map[string]string, out *output, +) int { return smallwebwaf.Run(ctx, smallwebwaf.Params{ Version: testVersion, LookupEnv: func(name string) (string, bool) { @@ -123,7 +130,8 @@ func run(ctx context.Context, env map[string]string, out *output) int { return value, ok }, - Stdout: out, + Stdout: out, + ServeWAF: serveWAF, }) }