README rewritten from scratch so a newcomer can follow it (closes #154, closes #155)

The old README opened with project history and one list of a hundred
settings. The new one leads with what smallwebwaf is, what it protects
against, how to build and deploy it, what happens to a request, the
settings most deployments touch, how to check it works, the entrypoints,
the rationale and the design, as REPO_POLICIES.md requires. The reference
lists, every setting, the rule file format, the endpoints, the request
log's fields, the alert events, the metrics, the state files, the image
and the gitea notes, move to files under docs/, each linked from the
README, with SPEC.md as the one place for behaviour. Every statement was
checked against the code on next.

Model: fable-5-1

README.md

@@ -8,7 +8,8 @@ scanners and other abusers before they reach the app.
 A web application firewall reads each request on its way to a web app and
 refuses the ones that look like abuse. `smallwebwaf` is also a reverse proxy: it
 accepts requests on the app's behalf, passes them to the app, and passes the
-app's answers back.
+app's answers back. A request it does not refuse reaches the app as it came,
+apart from a few headers (see "Design" below).
 
 It needs no setting. Every setting has a default chosen for a service on the
 open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and why. The
@@ -17,20 +18,11 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
 ## What it protects against
 
 - Scrapers that fetch every page of a site, all day. Rate limiting gives each
-  client 500 requests a minute, 20,000 an hour and 300,000 a day. That leaves
+  client 1000 requests a minute, 10,000 an hour and 50,000 a day. Byte limits
-  room for programmatic use of an API, such as an automation fleet on another
+  give it 10 GiB a minute, 20 GiB an hour and 50 GiB a day. A client over a
-  host. A fleet of about 20 agents makes about 120 requests a minute at its
+  limit is refused and banned. The first ban lasts an hour. Each repeat within a
-  busiest. A gitea page opened with an empty cache loads about 40 files, so a
+  day makes the next ban three times as long. A ban that would last more than
-  few such first visits fit in the minute limit too. The minute limit catches,
+  seven days is permanent.
-  within seconds, a client firing requests as fast as it can. The hour limit
-  catches a client that keeps up more than about 5.5 a second for an hour. The
-  day limit catches a client that keeps up 4 a second around the clock. Counting
-  cannot catch a client that stays under all three limits, such as one that
-  keeps up 3 a second; only the protections below can. Byte limits give each
-  client 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over a limit
-  is refused and banned. The first ban lasts an hour. Each repeat within a day
-  makes the next ban three times as long. A ban that would last more than seven
-  days is permanent.
 - Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
   default rule file bans such a client for seven days at its first probe. Its
   next request during the ban makes the ban permanent. The default rule file
@@ -38,8 +30,8 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
   of your own, or an entry in `SWWAF_TRAP_PATHS`, bans those the same way.
 - Attacks carried in a request's URL or headers, such as SQL injection and path
   traversal. The OWASP Core Rule Set is a published set of rules that recognise
-  common web attacks. Coraza, a Go engine, runs those rules (see "Attack
+  common web attacks. Coraza, a Go engine, runs those rules (step 11 under "What
-  detection" in [`SPEC.md`](SPEC.md)). A request the rules take for an attack is
+  happens to a request" below). A request the rules take for an attack is
   refused.
 - Abuse from networks and countries you do not want. You can:
     - refuse whole netblocks and countries;
@@ -150,32 +142,89 @@ while working on it, see "Entrypoints" below.
 
 ## What happens to a request
 
-`smallwebwaf` handles each request in fifteen steps, in this order. Once a step
+Two terms recur below. A clear sign of attack is a request that matches a rule
-refuses a request, the steps after it are skipped, apart from step 15, and
+whose action is `ban`, or asks for a trap path (steps 9 and 10). An offence is a
-nothing of the request reaches the app. "What happens to a request" in
+request that counts against its client in its history. It is one of:
-[`SPEC.md`](SPEC.md) gives each step in full, under the same numbers.
 
-1. The health check, `GET /_smallwebwaf/healthz`, is answered at once.
+- a clear sign of attack;
-2. The client is identified.
+- a broken limit;
-3. The static lists, `SWWAF_ALLOW_NETS` and `SWWAF_DENY_NETS`.
+- a refusal by a rule, by the Core Rule Set or for a missing or wrong token.
-4. The bans.
-5. The lookup of the client's AS number and country.
-6. The country lists.
-7. The reputation sources.
-8. The client's limit percentage.
-9. The rate limits.
-10. The trap paths.
-11. The rule files.
-12. The Core Rule Set.
-13. `smallwebwaf`'s own endpoints, under `/_smallwebwaf/`.
-14. The request is passed to the app.
-15. After the answer: the byte limits, the error burst and the client's history.
 
-A refused visitor is answered with `SWWAF_BAN_RESPONSE`, `403` by default (see
+`smallwebwaf` handles each request in this order. A request one step refuses
-"Ban lengths, response and scope" in [`SPEC.md`](SPEC.md)). A `block` rule or a
+skips the steps after it, apart from step 13. Nothing of a refused request
-Core Rule Set match answers `403`. A missing or wrong token gets `401`. A
+reaches the app. "What happens to a request" in [`SPEC.md`](SPEC.md) gives each
-request over a size or time limit gets the status for that limit, such as `413`
+step in full.
-for a body too large (see "Size and time limits" in [`SPEC.md`](SPEC.md)).
+
+1. `GET /_smallwebwaf/healthz` is answered `200` with `ok` at once, before any
+   check and without asking the app.
+2. The client is worked out. A netblock is a range of addresses, written as an
+   address and a prefix length: `203.0.113.0/24`. `SWWAF_TRUSTED_PROXIES` lists
+   the netblocks traefik comes from, by default the private address ranges. For
+   a TCP peer inside them, `X-Forwarded-For` is read from the right, and the
+   first address outside those netblocks is the client. A peer outside them is
+   the client itself. An IPv6 client is its group: the whole netblock around its
+   address, a /64 by default; `SWWAF_IPV6_GROUP_PREFIX` sets its length.
+3. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 11, the byte limits and the
+   error burst, but still needs a token at step 12. A client in
+   `SWWAF_DENY_NETS` is refused.
+4. A client under a ban is refused. If the ban was made for a clear sign of
+   attack, this request makes it permanent.
+5. The client's AS number and country are looked up, unless
+   `SWWAF_LOOKUP_SOURCE` is `off`. An AS number names the network an address
+   belongs to, such as `AS64496`.
+6. The country lists are checked. A client from a country in
+   `SWWAF_DENIED_COUNTRIES` is refused. When
+   `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` is set, a client from any other country
+   is refused, and so is a client without a country. Such a refusal is not an
+   offence.
+7. The reputation sources are checked: the blocklists, the CrowdSec decision
+   list, the DNS blocklists and AbuseIPDB. A client on the CrowdSec decision
+   list is banned until the decision ends. Any other source refuses the client,
+   lowers its limits or only notes it, as `SWWAF_BLOCKLIST_ACTION` and
+   `SWWAF_REPUTATION_ACTION` say. No request waits for a query.
+8. Rate limiting counts the request, unless the client or the path is exempt. A
+   request that takes the client over a limit is refused and bans the client.
+9. A request for a path in `SWWAF_TRAP_PATHS` is refused and bans the client, as
+   a clear sign of attack.
+10. The rule files are checked. A `log` rule notes the match. A `block` rule
+    refuses the request. A `ban` rule refuses it and bans the client, as a clear
+    sign of attack.
+11. The Core Rule Set inspects the request's method, URL and headers, and, once
+    `SWWAF_WAF_BODY_LIMIT` is set, form, JSON and XML bodies. A request it takes
+    for an attack is refused while `SWWAF_WAF_MODE` is `block`, the default, and
+    let through while it is `detect`. It bans no one by itself.
+12. A request under `/_smallwebwaf/` is answered here by `smallwebwaf`, after
+    its token is checked. Any other request is passed to the app. The app's
+    answer is streamed back within the size and time limits.
+13. Once the answer has ended, a request passed to the app has its bytes counted
+    for the byte limits. If a rule, a trap path, the Core Rule Set or a token
+    check refused the request, the refusal is counted for the error burst. The
+    client's history and the anomaly counters are updated. More than
+    `SWWAF_ERROR_BURST_THRESHOLD` refusals of one client in a minute ban it.
+
+In `observe` mode (`SWWAF_MODE=observe`), `smallwebwaf` makes no ban of its own.
+It refuses a request only for a size or time limit, or for a missing or wrong
+token. A request it would otherwise have refused is passed to the app, and its
+log line says what `enforce` mode would have done. Observe mode is for trying a
+configuration before enforcing it.
+
+What a refused visitor sees:
+
+- `SWWAF_BAN_RESPONSE`, by default `403` with the plain text `Forbidden`. It can
+  be `429` instead, or `close`, which closes the connection, so that traefik
+  answers `502`. It answers:
+    - a ban or a broken rate limit;
+    - a `ban` rule or a trap path;
+    - `SWWAF_DENY_NETS` and the country lists;
+    - a reputation source whose action is `deny`.
+- `403` for a `block` rule and the Core Rule Set.
+- For a size or time limit:
+    - `431` for headers over their limit;
+    - `413` for a body over its limit;
+    - `408` for a client too slow to send its body;
+    - `502` or `504` when the app passes a limit or cannot be reached.
+- `401` from an endpoint of `smallwebwaf`'s own, for a request without its
+  token.
 
 ## Settings you are likely to touch
 
@@ -184,11 +233,8 @@ Each setting is an environment variable of the app's container, such as
 [`docs/settings.md`](docs/settings.md) lists every setting, its default and the
 values it takes. The ones most deployments touch:
 
-- `SWWAF_MODE=observe`: `smallwebwaf` makes no ban of its own, and refuses a
+- `SWWAF_MODE=observe`: log what would have been refused, refuse almost nothing,
-  request only for a size or time limit or a missing or wrong token. A request
+  ban no one. For trying a configuration.
-  it would have refused is passed to the app, and its log line says what
-  `enforce` mode would have done. For trying a configuration before enforcing
-  it.
 - `SWWAF_ALLOW_NETS`: netblocks you trust, such as your monitoring. Their
   clients are refused only for a size or time limit or a missing or wrong token.
 - `SWWAF_RATE_LIMIT_EXEMPT_NETS`: netblocks exempt from rate limiting and the
@@ -330,41 +376,54 @@ nixpkgs fixed at one release. An app's `Dockerfile` installs a package with
 [`docs/image.md`](docs/image.md) lists what the image holds and how each part is
 pinned.
 
-- runit, a service supervisor, runs `smallwebwaf` and the app side by side, each
+- The container's first process is `runsvinit`, which starts runit, a service
-  as its own user, and starts either again when it exits. The app cannot touch
+  supervisor. runit runs the `run` script of each directory under
-  the state files or the process of `smallwebwaf`.
+  `/etc/service`, and runs it again a second after its program exits.
-- `smallwebwaf` listens on port 8080 and passes requests to the app on
+  `smallwebwaf` runs as the `smallwebwaf` user (uid 65532) and the app as a user
-  `127.0.0.1:8081`. The app listens on `127.0.0.1` only, so nothing reaches it
+  of its own. So the app cannot touch the state files or the process of
-  except through `smallwebwaf`. `smallwebwaf`'s own endpoints are under
+  `smallwebwaf`.
-  `/_smallwebwaf/`, on the same port.
+- `smallwebwaf` listens on port 8080, on every address. traefik sends the app's
-- The app sees the headers traefik sent, with traefik's address added to
+  requests there. `smallwebwaf` passes them to the app on `127.0.0.1:8081`. The
-  `X-Forwarded-For`, from a TCP peer on `127.0.0.1`. So the app must trust
+  app listens on `127.0.0.1` only, so nothing reaches it except through
-  `127.0.0.1` and `::1` for forwarded headers, besides the private address
+  `smallwebwaf`. The health check, the metrics and the ban management are on the
-  ranges.
+  same port, under `/_smallwebwaf/`. A request for that prefix never reaches the
-- Both processes start with the container's environment variables, so every
+  app.
-  setting of `smallwebwaf` starts with `SWWAF_`, and the app can read a token
+- The app sees what it would see from traefik directly: the same `Host`, the
-  given as one. A token given as a file that only the `smallwebwaf` user can
+  same `X-Forwarded-Proto`, and `X-Forwarded-For` with traefik's address added
-  read is out of its reach (see "Settings given as files" in
+  at the end. The app's TCP peer is `smallwebwaf` on `127.0.0.1`. So the app
+  must trust `127.0.0.1` and `::1` for forwarded headers, besides the private
+  address ranges. An app that keeps its usual trusted proxies sees every visitor
+  as `127.0.0.1`. Each request also carries its id in `X-Request-ID`. Any
+  `X-Client-ASN` or `X-Client-Country` header a client sent is removed.
+- Both processes start with the container's environment variables. That is why
+  every setting of `smallwebwaf` starts with `SWWAF_`. A token given as an
+  environment variable is readable by the app. A token given as a file that only
+  the `smallwebwaf` user can read is not (see "Settings given as files" in
   [`docs/settings.md`](docs/settings.md)).
-- The state files live in `/var/lib/smallwebwaf`. A volume mounted there keeps
+- The state files live in `/var/lib/smallwebwaf`. Without a volume there, they
-  them across a deploy.
+  survive a restart of the container and are lost when a deploy replaces it. The
-- `smallwebwaf servewaf` runs the same checks for the sites that ask it about
+  `run` script gives that directory and every file in it to the `smallwebwaf`
-  each request, and passes nothing to an app (see "`servewaf`" in
+  user at each start, so a host directory mounted there needs no change of
-  [`SPEC.md`](SPEC.md)).
+  owner.
-- `docker stop` stops both processes at once. `smallwebwaf` gives the requests
+- The image's health check passes while `smallwebwaf` answers
-  in progress five seconds to finish, so the app must keep answering for five
+  `GET /_smallwebwaf/healthz` and the app accepts connections. traefik sends a
+  container no requests until it is healthy.
+- When `smallwebwaf` or the app exits, runit starts it again a second later. The
+  other process keeps running. While either is down, the client gets `502`. A
+  `smallwebwaf` that stops at start, as on an invalid setting, is started again
+  every second until the cause is fixed, and logs its message each time.
+- `docker stop` has runit send `SIGTERM` to both processes at once.
+  `smallwebwaf` gives the requests in progress five seconds to finish and writes
+  its state files. To let them finish, the app must keep answering for five
   seconds after `SIGTERM`.
 
-"Deployment" in [`SPEC.md`](SPEC.md) has the rest: the health check, what
+"Deployment" in [`SPEC.md`](SPEC.md) has the rest: moving `smallwebwaf` to
-happens when a process stops, moving `smallwebwaf` to another port, and the
+another port, the health check's timing, and the forwarded headers.
-forwarded headers. "Architecture" in [`SPEC.md`](SPEC.md) lists the packages and
+"Architecture" in [`SPEC.md`](SPEC.md) lists the packages and what each does.
-what each does.
 
 ## TODO
 
 - Planned, as "Planned, not built" in [`SPEC.md`](SPEC.md) describes:
-    - `smallwebwaf rproxy`, the reverse proxy in front of each app that asks
-      `smallwebwaf servewaf` about each request;
     - sharing bans between the instances of several apps;
     - sending the log over RELP.
 

cmd/smallwebwaf/main.go

@@ -1,7 +1,6 @@
 // Command smallwebwaf is a web application firewall for one app: it runs
-// between traefik and the app and passes requests through within its limits,
+// between traefik and the app, and passes requests through within its
-// or, as `smallwebwaf servewaf`, answers rproxy's questions about them.
+// limits. Everything it does is in internal/smallwebwaf.
-// Everything it does is in internal/smallwebwaf.
 package main
 
 import (

docs/admin-endpoints.md

@@ -46,8 +46,3 @@ curl -H "Authorization: Bearer $TOKEN" -X DELETE \
 curl -H "Authorization: Bearer $TOKEN" \
     https://app.example/_smallwebwaf/clients/203.0.113.9
 ```
-
-## `servewaf`
-
-`smallwebwaf servewaf` answers these endpoints at its own address. No check runs
-on those requests (see "`servewaf`" in [`SPEC.md`](../SPEC.md)).

docs/alerts.md

@@ -35,34 +35,34 @@ on one line:
 ```json
 {
     "instance": "fsn1app1/gitea",
-    "time": "2026-10-06T12:00:50.123461Z",
+    "time": "2026-10-06T12:00:00.123461Z",
     "event": "ban",
     "client": "203.0.113.9",
     "netblock": "203.0.113.9/32",
     "asn": "",
     "as_name": "",
     "country": "",
-    "reason": "requests per minute over the limit of 500",
+    "reason": "requests per minute over the limit of 1000",
     "detail": {
-        "ban_expires": "2026-10-06T13:00:50.123Z",
+        "ban_expires": "2026-10-06T13:00:00.123Z",
         "cause": "limit",
         "notes": {
             "asn": "",
             "as_name": "",
             "country": "",
             "kind": "requests",
-            "limit": 500,
+            "limit": 1000,
             "window": "minute",
-            "count": 501,
+            "count": 1001,
             "request": {
-                "time": "2026-10-06T12:00:50.123456789Z",
+                "time": "2026-10-06T12:00:00.123456789Z",
                 "method": "GET",
                 "host": "app.example",
-                "path": "/owner/repo/commits/branch/main?page=501",
+                "path": "/owner/repo/commits/branch/main?page=812",
                 "status": 403,
                 "user_agent": "scraper/1.0"
             },
-            "requests": 501,
+            "requests": 5210,
             "refused": 0,
             "earlier_bans": {
                 "limit": 0,
@@ -78,21 +78,17 @@ on one line:
 
 - `instance`: `SWWAF_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
   one of the events above.
-- `host`: the site of the request that raised the alert. Only `servewaf` sends
-  this field, since it serves many sites, and not for `source_failure`,
-  `file_error` and `summary`.
 - `client`: the address whose request raised the alert. Empty for
-  `source_failure`, `file_error` and `summary`.
+  `source_failure` and `file_error`.
-- `netblock`: empty for `source_failure`, `file_error` and `summary`; otherwise
+- `netblock`: empty for `source_failure` and `file_error`; otherwise one of:
-  one of:
     - the ban's netblock;
-    - the netblock an anomaly counted, empty in the `asn` and `total` scopes;
+    - the netblock an anomaly counted;
     - for the other events, the client as rate limiting counts it: an IPv4
       address as a /32, or an IPv6 group.
 - `asn`, `as_name` and `country`: the client's, as known when the alert was
   raised. They are empty when GeoJS had not answered yet.
 - `reason`: a short sentence, such as
-  `requests per minute over the limit of 500` or `listed by a blocklist`.
+  `requests per minute over the limit of 1000` or `listed by a blocklist`.
 - `detail`: what is particular to the event:
     - for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed
       under `bans.json` in [`state-files.md`](state-files.md));
@@ -101,8 +97,6 @@ on one line:
     - for a Core Rule Set match, the `rule_ids`, `score`, `method` and `path`;
     - for a source failure, the `source` and `error`;
     - for a file error, the `file` and `error`;
-    - for a summary, the `hour` it covers, the `count` of alerts held back, and
-      that count by event in `events`;
     - `mode`, when the request was not refused: `detect` or `observe`.
 - `suppressed_repeats`: how many repeats the cooldown held back before this
   alert.
@@ -113,7 +107,7 @@ Slack and ntfy get a message with a title and a text. The title is the instance
 and the event, such as `fsn1app1/gitea: ban`. The text is the `reason`, then one
 line for each of these that the alert has:
 
-- the host, the client, the netblock and the country;
+- the client, the netblock and the country;
 - the file, the source, the error and the mode from `detail`;
 - the suppressed repeats, if any.
 

docs/gitea.md

@@ -11,10 +11,8 @@ checked again when gitea or the Core Rule Set changes version.
 
 - SSH access to gitea does not pass through `smallwebwaf` and is not protected
   by it.
-- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
+- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all
-  repository is large or the link slow. A repository whose clone is larger than
+  but the largest repositories and slowest links.
-  1 GiB needs `SWWAF_RESPONSE_MAX_BYTES` raised, and the byte limits with it
-  ("Byte limits, per client" in `SPEC.md`).
 - A push is a request, and so is every other upload:
     - a git push;
     - an LFS object;
@@ -32,15 +30,12 @@ checked again when gitea or the Core Rule Set changes version.
   request limits do most of the work there.
 - An Actions runner that reaches gitea through `smallwebwaf` sends requests all
   day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
-  43,200 requests a day. While it runs a job, it reports the job's log and state
+  43,200 requests a day, and reports a running job's log and state every second.
-  every second instead, two requests a second. So one runner busy all day sends
+  So it passes the day limit of 50,000 after about an hour and a quarter of
-  120 requests a minute and 172,800 a day, under the request limits. Two such
+  jobs. It is then banned, the job it is running fails, and each repeat within a
-  runners on one address, each busy all day, make 345,600 and pass the day limit
+  day makes the ban longer. Newer runners ask less often, but a busy one, or
-  of 300,000. Three busy at once make 21,600 an hour and pass the hour limit of
+  several on one address, can still pass it. Put the runners' addresses in
-  20,000. Five busy at once make 600 a minute and pass the minute limit of 500.
+  `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
-  Their address is then banned, their running jobs fail, and each repeat within
-  a day makes the ban longer. Newer runners ask for work less often. Put the
-  runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
 
 ## What the Core Rule Set lets through
 

docs/metrics.md

@@ -1,8 +1,8 @@
 # The metrics
 
-What `GET /_smallwebwaf/metrics` answers with. "Metrics endpoint" in
+The metrics `GET /_smallwebwaf/metrics` answers with, as "Metrics endpoint" in
-[`SPEC.md`](../SPEC.md) describes the endpoint. Prometheus keeps the `instance`
+[`SPEC.md`](../SPEC.md) describes it. Prometheus keeps the `instance` label as
-label as `exported_instance` unless the scrape sets `honor_labels: true`.
+`exported_instance` unless the scrape sets `honor_labels: true`.
 
 ## Traffic
 
@@ -17,8 +17,7 @@ label as `exported_instance` unless the scrape sets `honor_labels: true`.
 
 - `smallwebwaf_rate_limit_hits_total`, by `window` and by `kind` (`requests`,
   `bytes` or `refusals`);
-- `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting passed.
+- `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting passed;
-  `servewaf`'s header limits have no setting, so it leaves them out;
 - `smallwebwaf_offences_total`, by `kind`;
 - `smallwebwaf_bans_made_total`, by `cause`;
 - `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
@@ -80,14 +79,3 @@ By `file`:
   `smallwebwaf_alerts_failed_total`, `smallwebwaf_alerts_suppressed_total` and
   `smallwebwaf_alerts_dropped_total`;
 - Go's own `go_` metrics and the process's `process_` metrics.
-
-## `servewaf`
-
-`smallwebwaf servewaf` counts a denied request at its decision. It counts an
-allowed request when its report comes, with the status, bytes and times the
-report gives. These are not counted:
-
-- a report itself;
-- an allowed request whose report never comes;
-- a question refused with `400`;
-- a question whose body stopped arriving.

docs/request-log.md

@@ -67,8 +67,7 @@ Every line has:
       CrowdSec decision, each of which bans the client;
     - `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
       refused by that check;
-    - `too_large`: a size limit was passed, or a header limit in `servewaf`;
+    - `too_large` and `timed_out`: a size or time limit was passed;
-    - `timed_out`: a time limit was passed;
     - `upstream_error`: the app could not be reached, or its answer broke off;
     - `admin`: answered at one of `smallwebwaf`'s own endpoints.
 - `would_action`: in `observe` mode, what `enforce` mode would have done with a
@@ -118,66 +117,3 @@ All in milliseconds:
   connection;
 - `duration_upstream_first_byte`: until the first byte of the answer;
 - `duration_upstream_total`: until the end of the answer.
-
-## `servewaf`'s lines
-
-`smallwebwaf servewaf` writes two lines for each question it answers, joined by
-`request_id`. The field `servewaf_line` says which of the two a line is:
-
-- `decision`: the line it writes once it has answered the question;
-- `report`: the line it writes when `rproxy`'s report comes, with everything
-  known. The repetition is intended.
-
-A request to its own endpoints writes one line, without `servewaf_line`. A
-report writes no line but the `report` line of the request it names.
-
-These questions write no line:
-
-- a question refused with `400` (see "The question" under "`servewaf`" in
-  [`SPEC.md`](../SPEC.md));
-- a question whose body stopped arriving, which gets no answer.
-
-### The `decision` line
-
-It has the fields above, with these meanings where they differ:
-
-- `client_ip`: `X-Smallwebwaf-Client`.
-- `peer_ip`: `X-Smallwebwaf-Peer`.
-- `host`: the site the visitor asked for.
-- `protocol`: the question's own, `HTTP/1.1`.
-- `content_length`: `X-Smallwebwaf-Content-Length`.
-- `request_id`: the question's `X-Request-ID`.
-- `status`: what `servewaf` answered: `200` for `allow`, the status of `deny`,
-  or `0` for `close`, as the visitor is then sent nothing.
-- `request_bytes`: how much of the question's body the Core Rule Set read.
-- `response_bytes`: the body of `servewaf`'s answer.
-- `action`: `forward` for `allow`, and `admin` for the `404` for a path under
-  `/_smallwebwaf/`.
-- `aborted`: true when `rproxy` went away before the answer.
-- `duration_total`: from `servewaf` having read the question's body to the line.
-- `time`: when `servewaf` had read the question's body.
-
-The line leaves out `upstream_status`, `websocket` and the
-`duration_upstream_...` fields, since `servewaf` passes nothing to the app.
-
-The bytes of an allowed request are counted when its report comes. So `counts`
-holds the client's bytes without this request's.
-
-### The `report` line
-
-It has every field of the `decision` line, and `time` stays as the `decision`
-line gives it. These fields say what the report says:
-
-- `status`: what the visitor got, `0` if nothing;
-- `request_bytes` and `response_bytes`: the body bytes each way, with what a
-  WebSocket carried each way;
-- `duration_total`: from the request's headers being read by `rproxy` to the end
-  of the answer;
-- `duration_upstream_total`: from the hand-over to the app to the end of the
-  answer, for an allowed request.
-
-For an allowed request, the report's counting also changes these fields:
-
-- `counts`: the byte totals now include this request's;
-- `limit_hit`, `offence` and `ban_expires`: set when the report's bytes broke a
-  byte limit.

docs/settings.md

@@ -91,36 +91,24 @@ that take `off`, the `_FILE` form, and what each group does and why.
 ## Static lists
 
 - `SWWAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
-  monitoring (see step 3 in [`SPEC.md`](../SPEC.md)).
+  monitoring. Their clients are refused only for a size or time limit or a
+  missing or wrong token.
 - `SWWAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
   exempt from rate limiting and the byte limits.
-- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are refused (see
+- `SWWAF_DENY_NETS` (default empty): netblocks whose clients are always refused.
-  step 3 in [`SPEC.md`](../SPEC.md)).
 
 ## Request rate limits, per client
 
-- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `500`, about 8 a second): the most
+- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
-  requests a client may make in a minute. A count or `off`. The default catches,
+  (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
-  within seconds, a client firing requests as fast as it can. It leaves room for
+  requests a client may make in each window. A count or `off`.
-  a fleet of about 20 agents at its busiest, about 120 requests a minute. It
-  also leaves room for a few quick first visits to gitea with an empty cache,
-  about 40 files each.
-- `SWWAF_RATE_LIMIT_PER_HOUR` (default `20000`, about 5.5 a second): the most
-  requests a client may make in an hour. A count or `off`. The default leaves
-  room for an automation fleet at work. It catches a client that keeps up more
-  than about 5.5 a second for an hour.
-- `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most
-  requests a client may make in a day. A count or `off`. The default catches a
-  scraper that keeps up 4 a second around the clock.
-- Counting cannot catch a client that stays under all three limits (see "Request
-  rate limits, per client" in [`SPEC.md`](../SPEC.md)).
 - `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
   rate limiting and the byte limits, such as `/assets/`.
 
 ## Byte limits, per client
 
-- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `4G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
+- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `10G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
-  (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `200G`): the most
+  (default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `50G`): the most
   bytes counted for a client in each window. A size or `off`.
 - `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
   `response`, `request` or `both`.
@@ -143,21 +131,15 @@ that take `off`, the `_FILE` form, and what each group does and why.
   to send its whole answer. A duration or `off`.
 - `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body. A size
   or `off`.
-- `SWWAF_RESPONSE_MAX_BYTES` (default `1G`): the largest response body. A size
+- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body. A size
   or `off`.
 
 ## Lookup of AS number and country
 
-- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops
+- `SWWAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`.
-  the start while a setting that needs a lookup is set:
-    - the two country lists;
-    - `SWWAF_ADD_LOOKUP_HEADERS`;
-    - the four biased threshold lists and `SWWAF_ASN_LIMIT_PERCENT_URL`;
-    - `SWWAF_UNKNOWN_LIMIT_PERCENT` below `100`;
-    - an anomaly threshold per AS number.
 - `SWWAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
   `.mmdb` form, for `SWWAF_LOOKUP_SOURCE=file`.
-- `SWWAF_LOOKUP_TIMEOUT` (default `800ms`): how long a request waits for its
+- `SWWAF_LOOKUP_TIMEOUT` (default `1s`): how long a request waits for its
   client's first answer from GeoJS, and how long a request to GeoJS may take. A
   duration; never `off`.
 - `SWWAF_ADD_LOOKUP_HEADERS` (default `false`): `true` passes the app the
@@ -208,7 +190,8 @@ that take `off`, the `_FILE` form, and what each group does and why.
 - `SWWAF_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body
   the Core Rule Set reads, a size of at most `1G`.
 - `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
-  in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
+  in a minute for a rule match, a trap path, a Core Rule Set match or a failed
+  token check. A count or `off`.
 
 ## Ban lengths, response and scope
 
@@ -303,7 +286,7 @@ as `REQUESTS_PER_MINUTE`:
       client;
     - `SWWAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
       client.
-- `SWWAF_ANOMALY_ASN_...`: per AS number.
+- `SWWAF_ANOMALY_ASN_...`: per AS number. Needs a lookup source.
 - `SWWAF_ANOMALY_TOTAL_...`: the whole service.
 - `SWWAF_WATCH_...`: each netblock in `SWWAF_WATCH_NETS`, as a whole.
   `SWWAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
@@ -339,31 +322,4 @@ A few limits are fixed rather than settings:
 - at most 100,000 DNS blocklist verdicts and 100,000 AbuseIPDB scores are kept;
 - at most 1,000 DNS blocklist queries are under way at once;
 - at most 20,000 anomaly counters are kept;
-- each alert destination's queue holds at most 1,000 alerts;
+- each alert destination's queue holds at most 1,000 alerts.
-- `servewaf` holds at most 10,000 answered requests awaiting their reports;
-- `servewaf` refuses a request with more than 50 cookies or more than 100 header
-  fields (see "Answering in time" under "`servewaf`" in
-  [`SPEC.md`](../SPEC.md)).
-
-## `servewaf`
-
-`smallwebwaf servewaf` uses every setting here but these, which it ignores:
-
-- `SWWAF_UPSTREAM_URL` and `SWWAF_TRUSTED_PROXIES`;
-- the five timeouts under "Size and time limits";
-- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, `SWWAF_REQUEST_MAX_BYTES` and
-  `SWWAF_RESPONSE_MAX_BYTES`.
-
-An invalid value of one of these still stops the start, and the line of settings
-at start still shows them.
-
-`SWWAF_ADD_LOOKUP_HEADERS` puts the headers on the answer to `rproxy`, not on a
-request to an app. `SWWAF_WAF_BODY_LIMIT` has an effect up to 64 KiB less one
-byte, whatever its value. A question carries no more of a body than 64 KiB (see
-"`servewaf`" in [`SPEC.md`](../SPEC.md)).
-
-`SWWAF_LOOKUP_TIMEOUT` is the longest a question waits on an outside source.
-Keep it well under `rproxy`'s `SWRP_TIMEOUT_MS`, as the default is. No question
-waits for `SWWAF_REPUTATION_TIMEOUT`, since the DNS blocklists and AbuseIPDB are
-asked in the background (see "Answering in time" under "`servewaf`" in
-[`SPEC.md`](../SPEC.md)).

internal/alerts/alerts.go

@@ -1,8 +1,8 @@
 // Package alerts sends alerts on bans, on traffic over an anomaly
 // threshold, on a source that fails and on a file with an error to each
 // destination set: to the webhook
-// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
+// SWWAF_ALERT_WEBHOOK_URL names, each as one JSON object, as the "Alert
-// object" in docs/alerts.md describes, to the Slack incoming
+// webhook schema" section of SPEC.md describes, to the Slack incoming
 // webhook SWWAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
 // ntfy topic SWWAF_ALERT_NTFY_URL names. A repeat within
 // SWWAF_ALERT_COOLDOWN is held back, and so is an alert past
@@ -149,17 +149,15 @@ type Params struct {
 }
 
 // Alert is one alert, as the webhook is sent it and alerts.json holds it,
-// with the fields of "The JSON object" in docs/alerts.md. ASN, ASName and
+// with the fields of the "Alert webhook schema" section of SPEC.md. ASN,
-// Country are, for a ban, the client's as the ban's notes give them. Host is
+// ASName and Country are, for a ban, the client's as the ban's notes give
-// the site of the request that raised the alert, while one servewaf serves
+// them.
-// many sites, and "" otherwise.
 //
-//nolint:tagliatelle // docs/alerts.md names the alert's fields in snake_case
+//nolint:tagliatelle // SPEC.md's alert webhook schema names its fields in snake_case
 type Alert struct {
 	Instance string       `json:"instance"`
 	Time     time.Time    `json:"time"`
 	Event    string       `json:"event"`
-	Host     string       `json:"host,omitempty"`
 	Client   netip.Addr   `json:"client"`
 	Netblock netip.Prefix `json:"netblock"`
 	ASN      string       `json:"asn"`
@@ -902,16 +900,12 @@ func title(alert *Alert) string {
 }
 
 // text returns the text of alert in Slack and ntfy: its reason, then a
-// line for each of its host, client, netblock and country, the file,
+// line for each of its client, netblock and country, the file, source,
-// source, error and mode its detail gives, and its suppressed repeats, that
+// error and mode its detail gives, and its suppressed repeats, that it
-// it has.
+// has.
 func text(alert *Alert) string {
 	lines := []string{alert.Reason}
 
-	if alert.Host != "" {
-		lines = append(lines, "host: "+alert.Host)
-	}
-
 	if alert.Client.IsValid() {
 		lines = append(lines, "client: "+alert.Client.String())
 	}

internal/alerts/alerts_test.go

@@ -1284,11 +1284,9 @@ func anAlertForEachEvent() []eventMessage {
 				Event: alerts.EventWAFBlock, Client: netip.MustParseAddr("192.0.2.1"),
 				Netblock: netip.MustParsePrefix("192.0.2.1/32"),
 				Reason:   "refused by the Core Rule Set",
-				Host:     "app.example",
 			},
 			"fsn1app1/gitea: waf_block", "default shield",
-			"refused by the Core Rule Set\nhost: app.example\nclient: 192.0.2.1\n" +
+			"refused by the Core Rule Set\nclient: 192.0.2.1\nnetblock: 192.0.2.1/32",
-				"netblock: 192.0.2.1/32",
 		},
 		{
 			alerts.Alert{

internal/anomaly/anomaly.go

@@ -1,8 +1,8 @@
 // Package anomaly counts requests and bytes over a minute and an hour, per
 // client, per surrounding netblock, per AS number, for the whole service
 // and per named netblock, and raises an anomaly alert for a count over its
-// threshold, as "Anomaly thresholds" under "Settings" in SPEC.md
+// threshold, as "Anomaly thresholds" under "Configuration surface" in
-// describes. It refuses and bans nothing. At most 20,000 counters
+// SPEC.md describes. It refuses and bans nothing. At most 20,000 counters
 // are kept, in memory, and written to alerts.json and read from it by the
 // state package.
 package anomaly
@@ -103,8 +103,6 @@ type Counter struct {
 
 // Request is a request that has ended, as the counters count it.
 type Request struct {
-	// Host is the site the alert names, "" for none.
-	Host string
 	// Client is the client's address, and ClientGroup the client it is
 	// counted as: its IPv4 address, or the IPv6 netblock of
 	// SWWAF_IPV6_GROUP_PREFIX its address is in.
@@ -398,7 +396,6 @@ func alertFor(r Request, k key, o overThreshold) alerts.Alert {
 
 	return alerts.Alert{
 		Event:    alerts.EventAnomaly,
-		Host:     r.Host,
 		Client:   r.Client,
 		Netblock: k.netblock,
 		ASN:      r.ASN,

internal/config/config.go

@@ -502,21 +502,21 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
 		UpstreamRequestTimeout:  env.duration("SWWAF_UPSTREAM_REQUEST_TIMEOUT", "60s"),
 		UpstreamResponseTimeout: env.duration("SWWAF_UPSTREAM_RESPONSE_TIMEOUT", "30m"),
 		RequestMaxBytes:         env.size("SWWAF_REQUEST_MAX_BYTES", "100M"),
-		ResponseMaxBytes:        env.size("SWWAF_RESPONSE_MAX_BYTES", "1G"),
+		ResponseMaxBytes:        env.size("SWWAF_RESPONSE_MAX_BYTES", "5G"),
 		AllowNets:               env.netblocks("SWWAF_ALLOW_NETS", ""),
 		RateLimitExemptNets:     env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
 		DenyNets:                env.netblocks("SWWAF_DENY_NETS", ""),
-		RateLimitPerMinute:      env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "500"),
+		RateLimitPerMinute:      env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "1000"),
-		RateLimitPerHour:        env.count("SWWAF_RATE_LIMIT_PER_HOUR", "20000"),
+		RateLimitPerHour:        env.count("SWWAF_RATE_LIMIT_PER_HOUR", "10000"),
-		RateLimitPerDay:         env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"),
+		RateLimitPerDay:         env.count("SWWAF_RATE_LIMIT_PER_DAY", "50000"),
 		RateLimitExemptPaths:    env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
-		BytesLimitPerMinute:     env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"),
+		BytesLimitPerMinute:     env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "10G"),
 		BytesLimitPerHour:       env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "20G"),
-		BytesLimitPerDay:        env.size("SWWAF_BYTES_LIMIT_PER_DAY", "200G"),
+		BytesLimitPerDay:        env.size("SWWAF_BYTES_LIMIT_PER_DAY", "50G"),
 		BytesCount:              env.bytesCount("SWWAF_BYTES_COUNT", "both"),
 		LookupSource:            env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
 		LookupDBPath:            env.value("SWWAF_LOOKUP_DB_PATH", ""),
-		LookupTimeout:           env.durationNotOff("SWWAF_LOOKUP_TIMEOUT", "800ms"),
+		LookupTimeout:           env.durationNotOff("SWWAF_LOOKUP_TIMEOUT", "1s"),
 		AddLookupHeaders:        env.boolean("SWWAF_ADD_LOOKUP_HEADERS", "false"),
 		DeniedCountries:         env.countries("SWWAF_DENIED_COUNTRIES", ""),
 		ExclusivelyAllowedCountries: env.countries(

internal/config/config_test.go

@@ -261,10 +261,10 @@ func TestDefaults(t *testing.T) {
 		UpstreamRequestTimeout:      time.Minute,
 		UpstreamResponseTimeout:     30 * time.Minute,
 		RequestMaxBytes:             100 << 20,
-		ResponseMaxBytes:            1 << 30,
+		ResponseMaxBytes:            5 << 30,
-		RateLimitPerMinute:          500,
+		RateLimitPerMinute:          1000,
-		RateLimitPerHour:            20000,
+		RateLimitPerHour:            10000,
-		RateLimitPerDay:             300000,
+		RateLimitPerDay:             50000,
 		BanResponse:                 403,
 		LimitBanDuration:            time.Hour,
 		LimitBanRepeatWindow:        24 * time.Hour,
@@ -281,11 +281,11 @@ func TestDefaults(t *testing.T) {
 		RulesEnabled:                true,
 	})
 	wantLookupSettings(t, cfg, config.Config{
-		LookupSource: defaultLookupSource, LookupTimeout: 800 * time.Millisecond,
+		LookupSource: defaultLookupSource, LookupTimeout: time.Second,
 	})
 	wantByteLimitSettings(t, cfg, config.Config{
-		BytesLimitPerMinute: 4 << 30, BytesLimitPerHour: 20 << 30,
+		BytesLimitPerMinute: 10 << 30, BytesLimitPerHour: 20 << 30,
-		BytesLimitPerDay: 200 << 30, BytesCount: "both",
+		BytesLimitPerDay: 50 << 30, BytesCount: "both",
 	})
 
 	if cfg.UpstreamURL.String() != "http://127.0.0.1:8081" {
@@ -1165,7 +1165,7 @@ func TestLookupSettingsAsSet(t *testing.T) {
 
 	cfg = fromEnvironment(t, environment{lookupSource: off})
 	wantLookupSettings(t, cfg, config.Config{
-		LookupSource: off, LookupTimeout: 800 * time.Millisecond,
+		LookupSource: off, LookupTimeout: time.Second,
 	})
 }
 
@@ -2371,21 +2371,21 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
 		upstreamRequestTimeout:  "60s",
 		upstreamResponseTimeout: "30m",
 		requestMaxBytes:         "100M",
-		responseMaxBytes:        "1G",
+		responseMaxBytes:        "5G",
 		allowNets:               "",
 		rateLimitExemptNets:     "",
 		denyNets:                "",
-		rateLimitPerMinute:      "500",
+		rateLimitPerMinute:      "1000",
-		rateLimitPerHour:        "20000",
+		rateLimitPerHour:        "10000",
-		rateLimitPerDay:         "300000",
+		rateLimitPerDay:         "50000",
 		rateLimitExemptPaths:    "",
-		bytesLimitPerMinute:     "4G",
+		bytesLimitPerMinute:     "10G",
 		bytesLimitPerHour:       "20G",
-		bytesLimitPerDay:        "200G",
+		bytesLimitPerDay:        "50G",
 		bytesCount:              "both",
 		lookupSource:            defaultLookupSource,
 		lookupDBPath:            "",
-		lookupTimeout:           "800ms",
+		lookupTimeout:           "1s",
 		addLookupHeaders:        "false",
 		deniedCountries:         "",
 		allowedCountries:        "",

internal/lookup/lookup.go

@@ -179,7 +179,7 @@ func New(params Params) *GeoJS {
 // own address for IPv4, and an address in the same place for an IPv6
 // netblock.
 func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
-	answer, asked := g.answerOrWait(ctx, client, g.wait)
+	answer, asked := g.answerOrWait(ctx, client)
 	if asked == nil {
 		return answer
 	}
@@ -209,14 +209,6 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
 	return answer
 }
 
-// LookUpWithoutWaiting is LookUp for a request that does not need the answer,
-// whatever Wait says: it returns the zero Answer at once when none is kept.
-func (g *GeoJS) LookUpWithoutWaiting(ctx context.Context, client netip.Prefix) Answer {
-	answer, _ := g.answerOrWait(ctx, client, false)
-
-	return answer
-}
-
 // Kept returns client's answer, if one is kept, without asking GeoJS.
 func (g *GeoJS) Kept(client netip.Prefix) (Answer, bool) {
 	g.mu.Lock()
@@ -271,9 +263,9 @@ func (g *GeoJS) Load(answers []Answer) {
 // answerOrWait returns client's kept answer if it has one. Otherwise it
 // puts the client among those waiting if there is room, has GeoJS asked
 // about them if it can be, and returns what to wait on for the answer, or
-// nil when there is nothing to wait for or mustWait is false.
+// nil when there is nothing to wait for.
 func (g *GeoJS) answerOrWait(
-	ctx context.Context, client netip.Prefix, mustWait bool,
+	ctx context.Context, client netip.Prefix,
 ) (Answer, <-chan struct{}) {
 	g.mu.Lock()
 	defer g.mu.Unlock()
@@ -291,7 +283,7 @@ func (g *GeoJS) answerOrWait(
 
 	g.ask(ctx)
 
-	if !mustWait {
+	if !g.wait {
 		return Answer{}, nil // the answer is not needed before the request goes on
 	}
 

internal/lookup/lookup_test.go

@@ -72,39 +72,6 @@ func TestKeptAnswerIsUsedFor7DaysThenAskedAgain(t *testing.T) {
 	})
 }
 
-func TestLookUpWithoutWaitingAsksGeoJSAndGoesOnAtOnce(t *testing.T) {
-	t.Parallel()
-
-	synctest.Test(t, func(t *testing.T) {
-		geojs, clock, g := start()
-		client := netip.MustParsePrefix("203.0.113.9/32")
-
-		// Wait is set, and GeoJS does not answer.
-		geojs.set(hanging)
-
-		began := time.Now()
-		answer := g.LookUpWithoutWaiting(t.Context(), client)
-
-		if waited := time.Since(began); waited != 0 || answer != (lookup.Answer{}) {
-			t.Errorf("got %+v after %s, want no answer at once", answer, waited)
-		}
-
-		waitForRequests(t, geojs, 1)
-		wantAsked(t, geojs, 0, "203.0.113.9")
-
-		// That request is abandoned, and has failed once this returns. GeoJS
-		// answers when it is asked again, and the answer is kept.
-		time.Sleep(timeout)
-		synctest.Wait()
-		geojs.set(answering)
-		waitForCountry(t, g, clock, client, germany)
-
-		if got := g.LookUpWithoutWaiting(t.Context(), client).Country; got != germany {
-			t.Errorf("the kept answer gives the country %q, want %q", got, germany)
-		}
-	})
-}
-
 func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
 	t.Parallel()
 
@@ -156,8 +123,8 @@ func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(t *testing
 	t.Parallel()
 
 	synctest.Test(t, func(t *testing.T) {
-		// A timeout longer than the default, and a GeoJS that does not
+		// A timeout longer than the default second, and a GeoJS that does
-		// answer.
+		// not answer.
 		const longerTimeout = 3 * time.Second
 
 		m := metrics.New(1, "app")

internal/metrics/metrics.go

@@ -1,7 +1,6 @@
-// Package metrics keeps smallwebwaf's Prometheus metrics, as
+// Package metrics keeps smallwebwaf's Prometheus metrics, as the "Metrics
-// docs/metrics.md lists them, and serves them in the Prometheus text
+// endpoint" section of SPEC.md lists them, and serves them in the
-// format, as the "Metrics endpoint" section of SPEC.md requires. No
+// Prometheus text format. No metric carries a client's address.
-// metric carries a client's address.
 package metrics
 
 import (
@@ -403,21 +402,28 @@ func (m *Metrics) RequestEnded(
 	line *requestlog.Line, limit string, duration, upstreamDuration time.Duration,
 ) {
 	m.inFlight.Dec()
-	m.count(line, limit, duration, upstreamDuration)
-}
 
-// RequestAllowed counts a question of rproxy's, answered allow, as no longer
+	class := statusClass(line.Status)
-// under way. RequestReported counts the request it was about.
+	m.requests.WithLabelValues(class, line.Action).Inc()
-func (m *Metrics) RequestAllowed() {
+	m.requestBytes.WithLabelValues(class, line.Action).Add(float64(line.RequestBytes))
-	m.inFlight.Dec()
+	m.responseBytes.WithLabelValues(class, line.Action).Add(float64(line.ResponseBytes))
-}
+	m.requestDuration.Observe(duration.Seconds())
 
-// RequestReported counts a request rproxy passed to the app, as RequestEnded
+	if upstreamDuration > 0 {
-// does, from what its report says, which line holds.
+		m.upstreamDuration.Observe(upstreamDuration.Seconds())
-func (m *Metrics) RequestReported(
+	}
-	line *requestlog.Line, duration, upstreamDuration time.Duration,
+
-) {
+	if limit != "" {
-	m.count(line, "", duration, upstreamDuration)
+		m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc()
+	}
+
+	if line.Country != "" {
+		m.countries.add(line.Country, line)
+	}
+
+	if line.ASN != "" {
+		m.asns.add(line.ASN, line)
+	}
 }
 
 // LimitHit counts a request that broke a rate limit, a byte limit or the
@@ -486,33 +492,6 @@ func (m *Metrics) StateFileEditSetAside(name string) {
 	m.stateFileEditsSetAside.WithLabelValues(name).Inc()
 }
 
-// count counts a request that has ended, as RequestEnded describes.
-func (m *Metrics) count(
-	line *requestlog.Line, limit string, duration, upstreamDuration time.Duration,
-) {
-	class := statusClass(line.Status)
-	m.requests.WithLabelValues(class, line.Action).Inc()
-	m.requestBytes.WithLabelValues(class, line.Action).Add(float64(line.RequestBytes))
-	m.responseBytes.WithLabelValues(class, line.Action).Add(float64(line.ResponseBytes))
-	m.requestDuration.Observe(duration.Seconds())
-
-	if upstreamDuration > 0 {
-		m.upstreamDuration.Observe(upstreamDuration.Seconds())
-	}
-
-	if limit != "" {
-		m.sizeAndTimeLimitHits.WithLabelValues(limit).Inc()
-	}
-
-	if line.Country != "" {
-		m.countries.add(line.Country, line)
-	}
-
-	if line.ASN != "" {
-		m.asns.add(line.ASN, line)
-	}
-}
-
 // addReputationQueries adds the counter of the queries to source, a DNSBL
 // zone, or of the checks of clients with AbuseIPDB, which count tells.
 func (m *Metrics) addReputationQueries(source prometheus.Labels, count func() int) {

internal/proxy/admin.go

@@ -41,15 +41,6 @@ var (
 	errNotAddress = errors.New("is not an address, such as 203.0.113.9")
 )
 
-// answerHealth answers a request for the health endpoint.
-func (rq *request) answerHealth() {
-	rq.line.Action = requestlog.ActionAdmin
-	// Set here rather than left to Go's server, which would set it only
-	// after the log line has taken the response's headers.
-	rq.out.Header().Set("Content-Type", "text/plain; charset=utf-8")
-	_, _ = io.WriteString(rq.out, "ok\n")
-}
-
 // answerAdmin answers a request for smallwebwaf itself, under
 // /_smallwebwaf/, once it has passed the checks. Each endpoint needs a
 // token, sent as Authorization: Bearer <token>: the metrics

internal/proxy/anomaly_internal_test.go

@@ -15,11 +15,9 @@ func TestWithEveryAnomalyThresholdOffARequestIsNotCounted(t *testing.T) {
 	// anomaly counters, nor a clock, and the request no response: reading
 	// any of them to count the request panics.
 	rq := &request{
-		decision: decision{
+		h:        &handler{config: &config.Config{LookupSource: "geojs"}},
-			client:   netip.MustParseAddr("203.0.113.9"),
+		client:   netip.MustParseAddr("203.0.113.9"),
-			lookedUp: true,
+		lookedUp: true,
-		},
-		h: &handler{config: &config.Config{LookupSource: "geojs"}},
 	}
 
 	defer func() {

internal/proxy/answertime_test.go

@@ -1,600 +0,0 @@
-package proxy_test
-
-import (
-	"fmt"
-	"maps"
-	"net"
-	"net/http"
-	"net/http/httptest"
-	"net/netip"
-	"slices"
-	"strconv"
-	"strings"
-	"testing"
-	"testing/synctest"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-	"sneak.berlin/go/smallwebwaf/internal/lookup"
-	"sneak.berlin/go/smallwebwaf/internal/proxy"
-	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
-	"sneak.berlin/go/smallwebwaf/internal/reputation"
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-const (
-	// answerTimeLimit is the default of rproxy's SWRP_TIMEOUT_MS, the time
-	// servewaf has to answer.
-	answerTimeLimit = 2 * time.Second
-	// defaultLookupWait is the default of SWWAF_LOOKUP_TIMEOUT.
-	defaultLookupWait = 800 * time.Millisecond
-	// The header limits SPEC.md gives.
-	cookieLimit      = 50
-	headerFieldLimit = 100
-	// cookieHeader is the header the cookie limit counts.
-	cookieHeader = "Cookie"
-	// tooLarge is how servewaf refuses a request over the header limits.
-	tooLarge = http.StatusRequestHeaderFieldsTooLarge
-	// silentAbuseIPDBURL is where AbuseIPDB never answers: at the stand-in
-	// that unansweredGeoJSURL names too.
-	silentAbuseIPDBURL = "unanswered://abuseipdb/api/v2/check"
-	// wholeLimit is the percentage of a rate limit that lowers it not at all.
-	wholeLimit = 100
-	// clientsDecision is the CrowdSec decision list holding client alone.
-	clientsDecision = `[{"duration": "4h0m0s", "scenario": "crowdsecurity/ssh-bf", ` +
-		`"scope": "Ip", "type": "ban", "value": "` + client + `"}]`
-)
-
-// cookies returns a Cookie header's value with count cookies.
-func cookies(count int) string {
-	parts := make([]string, 0, count)
-	for i := range count {
-		parts = append(parts, "c"+strconv.Itoa(i)+"=1")
-	}
-
-	return strings.Join(parts, "; ")
-}
-
-// fillHeaders adds headers to a question until the checks see total header
-// fields: all but ClientHeader and PeerHeader, which servewaf removes. Go's
-// client sends nothing else but Host, which the limit does not count, once
-// User-Agent is set.
-func fillHeaders(req *http.Request, total int) {
-	req.Header.Set("User-Agent", "test")
-
-	for i := len(req.Header) - 2; i < total; i++ {
-		req.Header.Set("X-Filler-"+strconv.Itoa(i), "1")
-	}
-}
-
-func TestSixteenThousandCookiesAreRefusedWellWithinTheTimeLimit(t *testing.T) {
-	t.Parallel()
-
-	s := startServeWAF(t, "", map[string]string{wafMode: block})
-	req := s.question(siteA, client, "/")
-	req.Header.Set(cookieHeader, strings.Repeat("a;", 16000))
-
-	began := time.Now()
-	got := do(t, req)
-
-	if took := time.Since(began); took >= answerTimeLimit/2 {
-		t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
-	}
-
-	wantDisposition(t, got, tooLarge, proxy.DispositionDeny)
-
-	line := s.out.requestLine(t)
-	wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
-
-	if line.DurationWAF != nil {
-		t.Errorf("the Core Rule Set read the request, for %f ms", *line.DurationWAF)
-	}
-}
-
-func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
-	t.Parallel()
-
-	repeated := make([]string, headerFieldLimit)
-	for i := range repeated {
-		repeated[i] = "1"
-	}
-
-	for _, tc := range []struct {
-		name string
-		make func(*http.Request)
-		// refused is true for a request over a limit, and false for one at it.
-		refused bool
-	}{
-		{"cookies at the limit", func(req *http.Request) {
-			req.Header.Set(cookieHeader, cookies(cookieLimit))
-		}, false},
-		{"cookies over the limit", func(req *http.Request) {
-			req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-		}, true},
-		{"parts empty or only spaces and tabs are no cookies", func(req *http.Request) {
-			req.Header.Set(cookieHeader, cookies(cookieLimit)+"; \t;;")
-		}, false},
-		{"parts of no-break spaces count", func(req *http.Request) {
-			req.Header.Set(cookieHeader, strings.Repeat(" ;", cookieLimit+1))
-		}, true},
-		{"cookies over the limit across two headers", func(req *http.Request) {
-			req.Header[cookieHeader] = []string{cookies(cookieLimit), "last=1"}
-		}, true},
-		{"cookies without a value count", func(req *http.Request) {
-			req.Header.Set(cookieHeader, strings.Repeat("a;", cookieLimit+1))
-		}, true},
-		{"header fields at the limit", func(req *http.Request) {
-			fillHeaders(req, headerFieldLimit)
-		}, false},
-		{"header fields over the limit", func(req *http.Request) {
-			fillHeaders(req, headerFieldLimit+1)
-		}, true},
-		{"a header sent many times counts each time", func(req *http.Request) {
-			req.Header["X-Filler"] = repeated
-		}, true},
-	} {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			s := startServeWAF(t, "", map[string]string{wafMode: block})
-			req := s.question(siteA, client, "/")
-			tc.make(req)
-
-			got := do(t, req)
-			line := s.out.requestLine(t)
-
-			if tc.refused {
-				wantDisposition(t, got, tooLarge, proxy.DispositionDeny)
-				wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
-
-				return
-			}
-
-			wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
-			wantLine(t, line, http.StatusOK, requestlog.ActionForward)
-
-			if line.DurationWAF == nil {
-				t.Error("the Core Rule Set did not read a request within the limits")
-			}
-		})
-	}
-}
-
-func TestHeaderLimitsHoldWhateverTheCoreRuleSetSettingsSay(t *testing.T) {
-	t.Parallel()
-
-	for _, env := range []map[string]string{
-		{wafMode: off},
-		{wafMode: "detect"},
-		{wafMode: block, "SWWAF_WAF_EXEMPT_PATHS": "/"},
-	} {
-		t.Run(fmt.Sprint(env), func(t *testing.T) {
-			t.Parallel()
-
-			s := startServeWAF(t, "", env)
-			req := s.question(siteA, client, "/")
-			req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-			wantDisposition(t, do(t, req), tooLarge, proxy.DispositionDeny)
-		})
-	}
-}
-
-func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) {
-	t.Parallel()
-
-	s := startServeWAF(t, "", map[string]string{wafMode: block, allowNets: client})
-	req := s.question(siteA, client, "/")
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-	wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
-}
-
-func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
-	t.Parallel()
-
-	s := startServeWAF(t, "", map[string]string{mode: observe, wafMode: block})
-	req := s.question(siteA, client, "/")
-	req.Header.Set(cookieHeader, strings.Repeat("a;", 16000))
-
-	began := time.Now()
-	got := do(t, req)
-
-	if took := time.Since(began); took >= answerTimeLimit/2 {
-		t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
-	}
-
-	wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
-
-	line := s.out.requestLine(t)
-	if line.WouldAction != requestlog.ActionTooLarge || line.DurationWAF != nil {
-		t.Errorf("would_action %q and duration_waf %v, want %q and none",
-			line.WouldAction, line.DurationWAF, requestlog.ActionTooLarge)
-	}
-}
-
-func TestRequestsOverTheHeaderLimitsCountTowardTheRateLimits(t *testing.T) {
-	t.Parallel()
-
-	s := startServeWAF(t, "", map[string]string{rateLimitPerMinute: "1"})
-	ask := func() answer {
-		req := s.question(siteA, client, "/")
-		req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-		return do(t, req)
-	}
-
-	wantDisposition(t, ask(), tooLarge, proxy.DispositionDeny)
-	wantDisposition(t, ask(), http.StatusForbidden, proxy.DispositionDeny)
-
-	lines := s.out.requestLines(t, 2)
-	wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionRateLimited)
-}
-
-func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
-	t.Parallel()
-
-	for _, tc := range []struct {
-		name, path, action string
-	}{
-		{"a trap path", "/wp-login.php", requestlog.ActionBanned},
-		{"a block rule", "/blocked", requestlog.ActionRuleBlocked},
-	} {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			s := startServeWAF(t, "", map[string]string{
-				trapPaths: trapPathList, rulesDir: writeRules(t, testRules),
-			})
-			req := s.question(siteA, client, tc.path)
-			req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-			wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny)
-			wantLine(t, s.out.requestLine(t), http.StatusForbidden, tc.action)
-		})
-	}
-}
-
-func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
-	t.Parallel()
-
-	geojsURL, _ := startGeoJS(t)
-	s := startServeWAF(t, geojsURL, map[string]string{
-		unknownLimitPercent: "0", lookupTimeout: "1h",
-	})
-	req := s.question(siteA, fromDE, "/")
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-	wantDisposition(t, do(t, req), tooLarge, proxy.DispositionDeny)
-
-	line := s.out.requestLine(t)
-	wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
-
-	if line.LimitPercent != nil {
-		t.Errorf("limit percentage %d, want none", *line.LimitPercent)
-	}
-
-	// The client is under no ban, and GeoJS places it in a country.
-	wantDisposition(t, s.ask(fromDE, "/"), http.StatusOK, proxy.DispositionAllow)
-}
-
-func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
-	t.Parallel()
-
-	geojsURL, _ := startGeoJS(t)
-	s := startServeWAF(t, geojsURL, map[string]string{unknownLimitPercent: "0"})
-	s.server.GeoJS.Load([]lookup.Answer{{
-		Client: netip.MustParsePrefix(client + "/32"), ASN: "AS64512",
-		Answered: s.clock.Now(), Used: s.clock.Now(),
-	}})
-
-	req := s.question(siteA, client, "/")
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-	wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny)
-	wantLine(t, s.out.requestLine(t), http.StatusForbidden, requestlog.ActionRateLimited)
-}
-
-func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
-	t.Parallel()
-
-	geojsURL, _ := startGeoJS(t)
-	s := startServeWAF(t, geojsURL, map[string]string{
-		mode: observe, addLookupHeaders: on, lookupTimeout: "1h",
-	})
-	req := s.question(siteA, fromDE, "/")
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-
-	got := do(t, req)
-	wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
-
-	asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
-	if asn != asnDE || country != "DE" {
-		t.Errorf("answered with the AS number %q and the country %q, want %q and DE",
-			asn, country, asnDE)
-	}
-}
-
-func TestWithoutACommandTheHeaderLimitsDoNotApply(t *testing.T) {
-	t.Parallel()
-
-	app := startApp(t, func(http.ResponseWriter, *http.Request) {})
-	addr, out := startProxy(t, app.URL, nil)
-
-	req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
-	req.Header.Set(cookieHeader, cookies(cookieLimit+1))
-	fillHeaders(req, headerFieldLimit+1)
-
-	wantStatus(t, do(t, req), http.StatusOK)
-	wantLine(t, out.requestLine(t), http.StatusOK, requestlog.ActionForward)
-}
-
-// newServeWAFInBubble makes a servewaf for a synctest bubble: GeoJS never
-// answers, a setting needs its answer, and everything is at its default but
-// for env. No request to it may use the network.
-func newServeWAFInBubble(t *testing.T, env map[string]string) (*proxy.Server, *output) {
-	t.Helper()
-
-	settings := map[string]string{deniedCountries: "KP"}
-	maps.Copy(settings, env)
-
-	server, out, _ := newServerWith(t, proxy.NewServeWAF, "http://app.invalid",
-		unansweredGeoJSURL, time.Now, settings)
-
-	return server, out
-}
-
-// askDirectly hands a question about a request from client straight to server,
-// as change leaves it, and returns the answer's status and how long it took.
-func askDirectly(
-	t *testing.T, server *proxy.Server, change func(*http.Request),
-) (int, time.Duration) {
-	t.Helper()
-
-	req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
-	req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234")
-	req.Header.Set(proxy.ClientHeader, client)
-	req.Header.Set(requestIDHeader, "the-id")
-	change(req)
-
-	recorder := httptest.NewRecorder()
-	began := time.Now()
-
-	server.Handler.ServeHTTP(recorder, req)
-
-	return recorder.Code, time.Since(began)
-}
-
-// banClient bans client at now.
-func banClient(_ *testing.T, server *proxy.Server, now time.Time) {
-	server.Ledger.BanForLimit(netip.MustParsePrefix(client+"/32"), now, bans.Notes{})
-}
-
-// countClient counts one request from client at now for the rate limits.
-func countClient(_ *testing.T, server *proxy.Server, now time.Time) {
-	server.Limiter.Count(netip.MustParsePrefix(client+"/32"), now, wholeLimit)
-}
-
-// listClient puts client on the blocklist at dropURL.
-func listClient(t *testing.T, server *proxy.Server, now time.Time) {
-	t.Helper()
-
-	err := server.Lists.Load([]reputation.List{{
-		URL: dropURL, Tried: now, Fetched: now, Lines: []string{client},
-	}})
-	if err != nil {
-		t.Fatalf("load the blocklist: %v", err)
-	}
-}
-
-// decideOnClient puts client on the CrowdSec decision list.
-func decideOnClient(t *testing.T, server *proxy.Server, now time.Time) {
-	t.Helper()
-	loadDecisions(t, server, now, clientsDecision)
-}
-
-// leaveAsItIs prepares nothing.
-func leaveAsItIs(*testing.T, *proxy.Server, time.Time) {}
-
-func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
-	t.Parallel()
-
-	const forbidden = http.StatusForbidden
-
-	for _, tc := range []struct {
-		name string
-		env  map[string]string
-		// prepare gets the server ready, at now, before the question.
-		prepare func(t *testing.T, server *proxy.Server, now time.Time)
-		// cookie is the request's Cookie header.
-		cookie string
-		status int
-		action string
-	}{
-		{
-			name: "SWWAF_DENY_NETS", env: map[string]string{denyNets: client},
-			prepare: leaveAsItIs, status: forbidden, action: requestlog.ActionDenied,
-		},
-		{
-			name:    "a ban",
-			prepare: banClient, status: forbidden, action: requestlog.ActionBanned,
-		},
-		{
-			name: "the rate limits", env: map[string]string{rateLimitPerMinute: "1"},
-			prepare: countClient, status: forbidden, action: requestlog.ActionRateLimited,
-		},
-		{
-			name:    "the rate limits, with a biased threshold",
-			env:     map[string]string{rateLimitPerMinute: "1", unknownLimitPercent: "50"},
-			prepare: countClient, status: forbidden, action: requestlog.ActionRateLimited,
-		},
-		{
-			name: "a blocklist", env: map[string]string{blocklistURLs: dropURL},
-			prepare: listClient, status: forbidden, action: requestlog.ActionDenied,
-		},
-		{
-			name:    "the CrowdSec decision list",
-			env:     map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey},
-			prepare: decideOnClient, status: forbidden, action: requestlog.ActionBanned,
-		},
-		{
-			name: "the header limits", cookie: cookies(cookieLimit + 1),
-			prepare: leaveAsItIs, status: tooLarge, action: requestlog.ActionTooLarge,
-		},
-	} {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			// In a synctest bubble, as TestRequestWaitsAsLongAsTheLookupTimeoutSays
-			// says, so that a request that does not wait takes no time at all.
-			synctest.Test(t, func(t *testing.T) {
-				server, out := newServeWAFInBubble(t, tc.env)
-				tc.prepare(t, server, time.Now())
-
-				status, took := askDirectly(t, server, func(req *http.Request) {
-					req.Header.Set(cookieHeader, tc.cookie)
-				})
-
-				if status != tc.status || took != 0 {
-					t.Errorf("answered %d after %s, want %d at once; output:\n%s",
-						status, took, tc.status, out.text())
-				}
-
-				wantLine(t, out.requestLine(t), tc.status, tc.action)
-
-				// GeoJS's request is abandoned before the bubble ends.
-				time.Sleep(defaultLookupWait)
-			})
-		})
-	}
-}
-
-func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(t *testing.T) {
-	t.Parallel()
-
-	for _, tc := range []struct {
-		name string
-		env  map[string]string
-		// prepare gets the server ready, at now, before the question.
-		prepare func(t *testing.T, server *proxy.Server, now time.Time)
-		status  int
-		action  string
-	}{
-		{"nothing refuses it", nil, leaveAsItIs, http.StatusOK, requestlog.ActionForward},
-		{
-			"a country list needs the answer", map[string]string{allowedCountries: "DE"},
-			leaveAsItIs, http.StatusForbidden, requestlog.ActionCountryDenied,
-		},
-		{
-			"a blocklist that only logs",
-			map[string]string{blocklistURLs: dropURL, blocklistAction: actionLog},
-			listClient, http.StatusOK, requestlog.ActionForward,
-		},
-		{
-			"a blocklist that only limits",
-			map[string]string{blocklistURLs: dropURL, blocklistAction: limitQuarter},
-			listClient, http.StatusOK, requestlog.ActionForward,
-		},
-		{
-			"over a rate limit on an exempt path",
-			map[string]string{rateLimitPerMinute: "1", rateLimitExemptPaths: "/"},
-			countClient, http.StatusOK, requestlog.ActionForward,
-		},
-	} {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			synctest.Test(t, func(t *testing.T) {
-				server, out := newServeWAFInBubble(t, tc.env)
-				tc.prepare(t, server, time.Now())
-
-				status, took := askDirectly(t, server, func(*http.Request) {})
-
-				if status != tc.status || took != defaultLookupWait {
-					t.Errorf("answered %d after %s, want %d after %s; output:\n%s",
-						status, took, tc.status, defaultLookupWait, out.text())
-				}
-
-				wantLine(t, out.requestLine(t), tc.status, tc.action)
-			})
-		})
-	}
-}
-
-func TestWithoutACommandARefusedRequestStillWaitsForGeoJS(t *testing.T) {
-	t.Parallel()
-
-	synctest.Test(t, func(t *testing.T) {
-		server, out, _ := newProxy(t, "http://app.invalid", unansweredGeoJSURL,
-			time.Now, map[string]string{deniedCountries: "KP", blocklistURLs: dropURL})
-		listClient(t, server, time.Now())
-
-		status, took := askDirectly(t, server, func(req *http.Request) {
-			req.RemoteAddr = net.JoinHostPort(client, "1234")
-		})
-
-		if status != http.StatusForbidden || took != defaultLookupWait {
-			t.Errorf("answered %d after %s, want %d after %s; output:\n%s",
-				status, took, http.StatusForbidden, defaultLookupWait, out.text())
-		}
-
-		wantLine(t, out.requestLine(t), http.StatusForbidden, requestlog.ActionDenied)
-	})
-}
-
-func TestServeWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(t *testing.T) {
-	t.Parallel()
-
-	held, askedGeoJS, _ := startHeldGeoJS(t)
-
-	// A resolver that takes every query and answers none.
-	resolver, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", localhost+":0")
-	if err != nil {
-		t.Fatalf("listen: %v", err)
-	}
-
-	t.Cleanup(func() { _ = resolver.Close() })
-
-	server, out, _ := newServerWith(t, func(params proxy.Params) *proxy.Server {
-		params.AbuseIPDBURL = silentAbuseIPDBURL
-
-		return proxy.NewServeWAF(params)
-	}, "http://app.invalid", held, time.Now, map[string]string{
-		deniedCountries: "KP", wafMode: block,
-		dnsblZones: dnsblZone, dnsblResolver: resolver.LocalAddr().String(),
-		abuseIPDBKey:  accountKey,
-		blocklistURLs: held + "/drop.txt", asnLimitPercentURL: held + "/asn.txt",
-		crowdSecURL: held, crowdSecKey: bouncerKey,
-	})
-
-	// AbuseIPDB is asked only about a client with an offence.
-	group := netip.MustParsePrefix(client + "/32")
-	server.Limiter.AddToHistory(group, time.Now(), ratelimit.Request{Attack: true})
-
-	go server.Lists.Run(t.Context())
-
-	status, took := askDirectly(t, server, func(req *http.Request) {
-		req.Header.Set("User-Agent", "test")
-		req.Header.Set("Accept", "*/*")
-		req.Host = siteA
-	})
-
-	if status != http.StatusOK || took >= answerTimeLimit {
-		t.Errorf("answered %d after %s, want %d within %s; output:\n%s",
-			status, took, http.StatusOK, answerTimeLimit, out.text())
-	}
-
-	// Each outside source was asked, and none has answered. The stand-in for
-	// GeoJS is the lists' too, which name no address.
-	if !slices.Contains(askedGeoJS(), client) {
-		t.Errorf("GeoJS was asked about %q, not %s", askedGeoJS(), client)
-	}
-
-	if queries := server.DNSBL.Queries(dnsblZone); queries != 1 {
-		t.Errorf("%d queries to the DNS blocklist, want 1", queries)
-	}
-
-	if checked := server.AbuseIPDB.Checked(); checked != 1 {
-		t.Errorf("%d checks with AbuseIPDB, want 1", checked)
-	}
-}

internal/proxy/bans.go

@@ -323,7 +323,6 @@ func (rq *request) alertBan(ban bans.Ban) {
 
 	rq.h.alerts.Raise(alerts.Alert{
 		Event:    event,
-		Host:     rq.site,
 		Client:   rq.client,
 		Netblock: ban.Netblock,
 		ASN:      ban.Notes.ASN,
@@ -340,11 +339,11 @@ func (rq *request) alertBan(ban bans.Ban) {
 func (rq *request) noted(now time.Time, status int) bans.Request {
 	return bans.Request{
 		Time:      now,
-		Method:    rq.line.Method,
+		Method:    rq.in.Method,
-		Host:      rq.line.Host,
+		Host:      rq.in.Host,
-		Path:      rq.requestURI,
+		Path:      rq.in.URL.RequestURI(),
 		Status:    status,
-		UserAgent: rq.line.UserAgent,
+		UserAgent: rq.in.UserAgent(),
 	}
 }
 

internal/proxy/bans_test.go

@@ -25,8 +25,6 @@ const (
 	userAgent = "ban-test/1.0"
 	// permanent is the log line's ban_expires for a permanent ban.
 	permanent = "permanent"
-	// banResponseClose is the SWWAF_BAN_RESPONSE that closes the connection.
-	banResponseClose = "close"
 )
 
 func TestBrokenLimitBansTheClient(t *testing.T) {
@@ -239,7 +237,7 @@ func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
 		{"", http.StatusForbidden},
 		{"403", http.StatusForbidden},
 		{"429", http.StatusTooManyRequests},
-		{banResponseClose, 0},
+		{"close", 0},
 	} {
 		t.Run(banResponse+"="+tc.setting, func(t *testing.T) {
 			t.Parallel()

internal/proxy/biased.go

@@ -34,19 +34,19 @@ func biasedThresholdsSet(cfg *config.Config) bool {
 // percentage SWWAF_ASN_LIMIT_PERCENT gives its AS number, the one the file
 // SWWAF_ASN_LIMIT_PERCENT_URL names gives it, the one
 // SWWAF_COUNTRY_LIMIT_PERCENT gives its country, for a client without a
-// country, unless its request skipped the wait for GeoJS,
+// country, SWWAF_UNKNOWN_LIMIT_PERCENT, for a client a blocklist lists,
-// SWWAF_UNKNOWN_LIMIT_PERCENT, for a client a blocklist lists, the
+// the percentage of SWWAF_BLOCKLIST_ACTION while it is limit, and for a
-// percentage of SWWAF_BLOCKLIST_ACTION while it is limit, and for a client
+// client a DNSBL zone's verdict lists, or whose AbuseIPDB score is a hit,
-// a DNSBL zone's verdict lists, or whose AbuseIPDB score is a hit, the
+// the percentage of SWWAF_REPUTATION_ACTION while it is limit. For the
-// percentage of SWWAF_REPUTATION_ACTION while it is limit. For the byte
+// byte limits, SWWAF_ASN_BYTES_PERCENT and SWWAF_COUNTRY_BYTES_PERCENT
-// limits, SWWAF_ASN_BYTES_PERCENT and SWWAF_COUNTRY_BYTES_PERCENT take the
+// take the place of the first three for an AS number or a country they
-// place of the first three for an AS number or a country they list.
+// list.
 func (rq *request) limitPercentages() (percentage, percentage) {
 	cfg := rq.h.config
 	asn, country := rq.line.ASN, rq.line.Country
 
 	unknown := percentage{percent: whole}
-	if country == "" && !rq.skippedLookupWait {
+	if country == "" {
 		unknown = percentage{cfg.UnknownLimitPercent, "SWWAF_UNKNOWN_LIMIT_PERCENT"}
 	}
 

internal/proxy/biased_test.go

@@ -284,7 +284,7 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
 		server, out, _ := newProxy(t, "http://app.invalid", unansweredGeoJSURL,
 			time.Now, map[string]string{unknownLimitPercent: "0"})
 
-		// Once the time the request waits for its answer is up, the client
+		// Once the second the request waits for its answer is up, the client
 		// counts as without a country, and its zero allowance refuses the
 		// request before it reaches the app.
 		serveFromDE(t, server, http.MethodGet, http.NoBody)

internal/proxy/coreruleset.go

@@ -113,7 +113,6 @@ func (rq *request) alertWAFBlock(result waf.Result) {
 
 	rq.h.alerts.Raise(alerts.Alert{
 		Event:    alerts.EventWAFBlock,
-		Host:     rq.site,
 		Client:   rq.client,
 		Netblock: rq.h.clientGroup(rq.client),
 		ASN:      rq.line.ASN,

internal/proxy/countries_test.go

@@ -284,9 +284,9 @@ func TestExclusiveListRefusesAPrivateAddressUnlessAllowed(t *testing.T) {
 // each in an AS of its own, and no other address. It returns its URL, and
 // what returns the addresses it has been asked about. A test that needs
 // the stand-in to be asked or to answer sets SWWAF_LOOKUP_TIMEOUT to an
-// hour, whether or not a request waits for the answer: on the default, a
+// hour, whether or not a request waits for the answer: on the default
-// hold-up of the test process can abandon the request to the stand-in, and
+// second, a hold-up of the test process can abandon the request to the
-// leave the client unknown.
+// stand-in, and leave the client unknown.
 func startGeoJS(t *testing.T) (string, func() []string) {
 	t.Helper()
 

internal/proxy/lookup.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"net/http"
 	"net/netip"
-	"time"
 
 	"sneak.berlin/go/smallwebwaf/internal/lookup"
 )
@@ -24,24 +23,17 @@ const (
 // SWWAF_LOOKUP_SOURCE is off or the client is on a private, loopback or
 // link-local address, which no lookup can place. The lookup database
 // answers at once. With GeoJS, while a setting needs the answer, such as a
-// country list or a biased threshold, a new client's request waits for it,
+// country list or a biased threshold, a new client's request waits for it.
-// unless refusedWhateverTheAnswer reports true. ctx is the request's own
+// ctx is the request's own context.
-// context.
+func (rq *request) lookUp(ctx context.Context) {
-func (rq *request) lookUp(ctx context.Context, now time.Time) {
 	if rq.h.config.LookupSource == "off" || !canBePlaced(rq.client) {
 		return
 	}
 
-	client := rq.h.clientGroup(rq.client)
+	if rq.h.config.LookupSource == "file" {
-
+		rq.lookupAnswer = rq.h.lookupFile.LookUp(rq.h.clientGroup(rq.client))
-	switch {
+	} else {
-	case rq.h.config.LookupSource == "file":
+		rq.lookupAnswer = rq.h.geojs.LookUp(ctx, rq.h.clientGroup(rq.client))
-		rq.lookupAnswer = rq.h.lookupFile.LookUp(client)
-	case rq.h.answersQuestions() && rq.refusedWhateverTheAnswer(now):
-		rq.lookupAnswer = rq.h.geojs.LookUpWithoutWaiting(ctx, client)
-		rq.skippedLookupWait = rq.lookupAnswer.Answered.IsZero()
-	default:
-		rq.lookupAnswer = rq.h.geojs.LookUp(ctx, client)
 	}
 
 	rq.lookedUp = true

internal/proxy/lookup_test.go

@@ -319,7 +319,7 @@ func TestLookupHeadersArePassedToTheAppAndTheClientsOwnRemoved(t *testing.T) {
 	addr, out, _ := startProxyWithClock(t, app.URL, geojsURL, time.Now, map[string]string{
 		trustedProxies:   trustLocalhost,
 		lookupTimeout:    "1h",
-		addLookupHeaders: on,
+		addLookupHeaders: "true",
 	})
 	s := &sender{t: t, addr: addr, out: out}
 
@@ -383,8 +383,8 @@ func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
 	// network, which would keep that clock from moving on: the request is
 	// handed to the proxy's handler, and GeoJS is one that never answers.
 	synctest.Test(t, func(t *testing.T) {
-		// Not the default. The exclusive list needs the answer, and the app
+		// Not the default second. The exclusive list needs the answer, and
-		// is never reached.
+		// the app is never reached.
 		const timeout = 3 * time.Second
 
 		server, out, _ := newProxy(t, "http://app.invalid", unansweredGeoJSURL,

internal/proxy/metrics_test.go

@@ -207,7 +207,7 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
 		rateLimitPerMinute:  "1",
 		rateLimitExemptNets: scraper,
 		denyNets:            denied,
-		banResponse:         banResponseClose,
+		banResponse:         "close",
 		limitBanDuration:    "1h",
 		maxBanDuration:      "2h",
 	})
@@ -424,8 +424,8 @@ func TestMetricsCountGeoJSRequestsAndFailures(t *testing.T) {
 	req.Header.Set(forwardedFor, fromDE)
 	wantStatus(t, do(t, req), http.StatusOK)
 
-	// The client stops waiting for GeoJS once SWWAF_LOOKUP_TIMEOUT is up, so
+	// The client stops waiting for GeoJS after a second, so GeoJS's
-	// GeoJS's failure can come after its request has ended.
+	// failure can come after its request has ended.
 	deadline := time.Now().Add(waitLimit)
 	metrics := scrape(t, addr)
 

internal/proxy/proxy.go

@@ -1,8 +1,6 @@
-// Package proxy runs the checks on each request and writes its request log
+// Package proxy passes each request to the app and the app's answer back,
-// line. New passes each request it lets through to the app and the app's
+// unchanged, within the size and time limits, and writes one request log
-// answer back, unchanged, within the size and time limits. NewServeWAF instead
+// line for each request.
-// answers rproxy's question about each request, and takes rproxy's report once
-// the visitor's answer has ended, which writes the request a second log line.
 package proxy
 
 import (
@@ -13,8 +11,6 @@ import (
 	"strings"
 	"time"
 
-	lru "github.com/hashicorp/golang-lru/v2"
-
 	"sneak.berlin/go/smallwebwaf/internal/alerts"
 	"sneak.berlin/go/smallwebwaf/internal/anomaly"
 	"sneak.berlin/go/smallwebwaf/internal/bans"
@@ -23,6 +19,7 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/metrics"
 	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
 	"sneak.berlin/go/smallwebwaf/internal/reputation"
+	"sneak.berlin/go/smallwebwaf/internal/requestlog"
 	"sneak.berlin/go/smallwebwaf/internal/rules"
 	"sneak.berlin/go/smallwebwaf/internal/waf"
 )
@@ -56,8 +53,7 @@ const ClientsPath = "/_smallwebwaf/clients/"
 // Params are what New needs.
 type Params struct {
 	Config *config.Config
-	// RequestLog receives one JSON line per request, and from servewaf a second
+	// RequestLog receives one JSON line per request.
-	// when the request's report comes.
 	RequestLog io.Writer
 	// ProcessLog receives the process's own messages.
 	ProcessLog *slog.Logger
@@ -113,23 +109,6 @@ type Server struct {
 // SWWAF_CLIENT_REQUEST_TIMEOUT while the headers arrive; the proxy
 // applies the timeouts and size limits from then on.
 func New(params Params) *Server {
-	h := newHandler(params)
-
-	return h.server(&http.Server{
-		Handler:           h,
-		ReadHeaderTimeout: params.Config.ClientRequestTimeout,
-		// Off is an IdleTimeout of 0, which Go's server replaces with
-		// ReadTimeout: no limit, as long as ReadTimeout stays unset.
-		IdleTimeout: params.Config.ClientIdleTimeout,
-		// Go's server reads 4 KiB past MaxHeaderBytes before it
-		// refuses, so the limit a client meets is the setting.
-		MaxHeaderBytes: int(params.Config.ClientRequestHeaderMaxBytes - 4<<10),
-	})
-}
-
-// newHandler returns the handler with what every request shares, as the
-// settings in params make it.
-func newHandler(params Params) *handler {
 	errorLog := slog.NewLogLogger(params.ProcessLog.Handler(), slog.LevelWarn)
 	m := metrics.New(params.Config.MetricsTopN, params.Config.InstanceName)
 	lists, dnsbl, abuseIPDB := newReputation(params, m)
@@ -192,7 +171,29 @@ func newHandler(params Params) *handler {
 	m.AddBansAndClients(h.ledger, h.limiter, params.Now)
 	m.AddRules(params.Rules)
 
-	return h
+	return &Server{
+		Server: &http.Server{
+			Addr:              params.Config.ListenAddr,
+			Handler:           h,
+			ReadHeaderTimeout: params.Config.ClientRequestTimeout,
+			// Off is an IdleTimeout of 0, which Go's server replaces with
+			// ReadTimeout: no limit, as long as ReadTimeout stays unset.
+			IdleTimeout: params.Config.ClientIdleTimeout,
+			// Go's server reads 4 KiB past MaxHeaderBytes before it
+			// refuses, so the limit a client meets is the setting.
+			MaxHeaderBytes: int(params.Config.ClientRequestHeaderMaxBytes - 4<<10),
+			ErrorLog:       errorLog,
+		},
+		Ledger:     h.ledger,
+		Limiter:    h.limiter,
+		GeoJS:      h.geojs,
+		Anomalies:  h.anomalies,
+		LookupFile: h.lookupFile,
+		Lists:      h.lists,
+		DNSBL:      h.dnsbl,
+		AbuseIPDB:  h.abuseIPDB,
+		Metrics:    m,
+	}
 }
 
 // newReputation returns the lists fetched from URLs, the DNSBL zones'
@@ -273,10 +274,6 @@ type handler struct {
 	rules       *rules.Files
 	coreRuleSet *waf.CoreRuleSet
 	alerts      *alerts.Queue
-	// awaitingReport holds, in servewaf, the decision on each answered
-	// question by its id until its report comes, and nil for a question still
-	// being handled.
-	awaitingReport *lru.Cache[string, *decision]
 }
 
 // newTransport returns what carries requests to the app. It never goes
@@ -302,7 +299,11 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	// The health endpoint is answered at once, before any check, so that
 	// a health checker is never refused. It does not ask the app.
 	if r.Method == http.MethodGet && r.URL.Path == HealthPath {
-		rq.answerHealth()
+		rq.line.Action = requestlog.ActionAdmin
+		// Set here rather than left to Go's server, which would set it only
+		// after the log line has taken the response's headers.
+		rq.out.Header().Set("Content-Type", "text/plain; charset=utf-8")
+		_, _ = io.WriteString(rq.out, "ok\n")
 
 		return
 	}
@@ -338,23 +339,3 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 	rq.forward(r.Context())
 }
-
-// server returns the Server of h, which listens at SWWAF_LISTEN_ADDR and
-// logs through h, with what httpServer sets.
-func (h *handler) server(httpServer *http.Server) *Server {
-	httpServer.Addr = h.config.ListenAddr
-	httpServer.ErrorLog = h.errorLog
-
-	return &Server{
-		Server:     httpServer,
-		Ledger:     h.ledger,
-		Limiter:    h.limiter,
-		GeoJS:      h.geojs,
-		Anomalies:  h.anomalies,
-		LookupFile: h.lookupFile,
-		Lists:      h.lists,
-		DNSBL:      h.dnsbl,
-		AbuseIPDB:  h.abuseIPDB,
-		Metrics:    h.metrics,
-	}
-}

internal/proxy/proxy_test.go

@@ -91,9 +91,6 @@ const (
 // off is the value that switches a setting off.
 const off = "off"
 
-// on is the value that switches a setting that takes true or false on.
-const on = "true"
-
 // output collects what smallwebwaf writes on stdout.
 type output struct {
 	mu  sync.Mutex
@@ -287,16 +284,6 @@ func newProxy(
 ) (*proxy.Server, *output, *alerts.Queue) {
 	t.Helper()
 
-	return newServerWith(t, proxy.New, appURL, geojsURL, now, env)
-}
-
-// newServerWith is newProxy for the server build makes.
-func newServerWith(
-	t *testing.T, build func(proxy.Params) *proxy.Server,
-	appURL, geojsURL string, now func() time.Time, env map[string]string,
-) (*proxy.Server, *output, *alerts.Queue) {
-	t.Helper()
-
 	settings := map[string]string{
 		"SWWAF_UPSTREAM_URL": appURL, rulesDir: t.TempDir(), instanceName: "app",
 		wafMode: off,
@@ -347,7 +334,7 @@ func newServerWith(
 		}
 	}
 
-	server := build(proxy.Params{
+	server := proxy.New(proxy.Params{
 		Config:       cfg,
 		RequestLog:   out,
 		ProcessLog:   processLog,

internal/proxy/reputation.go

@@ -110,7 +110,6 @@ func (rq *request) noteHit(hit bans.ReputationHit, reason string) {
 	rq.h.metrics.ReputationHit(hit.Source)
 	rq.h.alerts.Raise(alerts.Alert{
 		Event:    alerts.EventReputationHit,
-		Host:     rq.site,
 		Client:   rq.client,
 		Netblock: rq.h.clientGroup(rq.client),
 		ASN:      rq.line.ASN,

internal/proxy/request.go

@@ -39,29 +39,25 @@ type refusal struct {
 	limit  string
 }
 
-// decision is what the checks note of a request: its log line, and what
+// request is one request on its way through smallwebwaf, from the moment
-// counting the request needs once it has ended. servewaf keeps a copy of it
+// its headers have been read to its log line.
-// until the request's report comes. It holds no credentials, since the log line
+type request struct {
-// never does, and neither the request's objects nor their bodies.
+	h  *handler
-type decision struct {
+	in *http.Request
-	line   requestlog.Line
+	// rc sets the deadlines of the connection to the client.
-	client netip.Addr
+	rc   *http.ResponseController
-	// requestURI is the path with the query, as a ban's notes keep it.
+	out  *responseWriter
-	requestURI string
+	body *requestBody // nil for a request without a body
-	// site is, in servewaf, the host the visitor asked for, which the
+	line requestlog.Line
-	// alerts name, since one servewaf serves many sites. It is "" otherwise.
+
-	site string
+	client      netip.Addr
-	// allowed is true for a question answered with DispositionAllow.
+	peer        netip.Addr
-	allowed bool
+	peerTrusted bool
 	// lookedUp is true once the client's AS number and country have been
 	// looked up, whether or not an answer was there, and lookupAnswer is
 	// what the lookup gave then, the zero Answer while GeoJS had given none.
 	lookedUp     bool
 	lookupAnswer lookup.Answer
-	// skippedLookupWait is true for a request that did not wait for GeoJS and
-	// found no answer kept. SWWAF_UNKNOWN_LIMIT_PERCENT does not apply to it,
-	// so that not waiting never lowers its client's limits.
-	skippedLookupWait bool
 	// counted is true for a request the rate limits counted, whose bytes
 	// the byte limits count once it has ended. limitPercent and
 	// bytesPercent are then its client's limit percentages for the rate
@@ -69,37 +65,18 @@ type decision struct {
 	counted                    bool
 	limitPercent, bytesPercent percentage
 	// attack is true for a request that matched a ban rule or asked for a
-	// trap path, ruleBlocked for one a block rule refused, and wafBlocked for
+	// trap path, ruleBlocked for one a block rule refused, wafBlocked for
-	// one the Core Rule Set refused, each an offence its client's history
+	// one the Core Rule Set refused, and tokenRefused for one refused for a
-	// counts.
+	// missing or wrong token, each an offence its client's history counts.
-	attack, ruleBlocked, wafBlocked bool
+	attack, ruleBlocked, wafBlocked, tokenRefused bool
-	// reputation is the reputation sources that list the client, for the
-	// notes of a ban the request makes.
-	reputation []bans.ReputationHit
-}
-
-// request is one request on its way through smallwebwaf, from the moment
-// its headers have been read to its log line.
-type request struct {
-	decision
-
-	h  *handler
-	in *http.Request
-	// rc sets the deadlines of the connection to the client.
-	rc   *http.ResponseController
-	out  *responseWriter
-	body *requestBody // nil for a request without a body
-
-	peer        netip.Addr
-	peerTrusted bool
-	// tokenRefused is true for a request refused for a missing or wrong
-	// token, an offence its client's history counts.
-	tokenRefused bool
 	// blocklisted is true once a blocklist is found to list the client,
 	// dnsblListed once a DNSBL zone's verdict is, and abuseIPDBHit once
 	// AbuseIPDB's score of it is a hit.
 	blocklisted, dnsblListed, abuseIPDBHit bool
-	start                                  time.Time
+	// reputation is the reputation sources that list the client, for the
+	// notes of a ban the request makes.
+	reputation []bans.ReputationHit
+	start      time.Time
 	// checked is when the checks were done, and upstreamStart when the
 	// request was handed to the app.
 	checked       time.Time
@@ -131,62 +108,50 @@ type request struct {
 	answerStarted time.Time
 }
 
-// newRequest starts handling r: it works out the client from the TCP peer
+// newRequest starts handling r: it notes the time, counts the request as
-// and X-Forwarded-For, and goes on as startRequest does.
+// under way, works out the client, and starts the log line with what is
+// known of the request.
 func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
-	peer := peerAddress(r)
-	trusted := h.config.TrustedProxies
-	client := clientAddress(peer, r.Header.Values("X-Forwarded-For"), trusted)
-
-	return h.startRequest(w, r, client, peer, isInside(peer, trusted))
-}
-
-// startRequest starts handling r from client, which came through peer: it
-// notes the time, counts the request as under way, and starts the log line
-// with what is known of the request. peerTrusted says whether to believe
-// the request's id and scheme, which r's headers give.
-func (h *handler) startRequest(
-	w http.ResponseWriter, r *http.Request, client, peer netip.Addr, peerTrusted bool,
-) *request {
 	h.metrics.RequestStarted()
 
 	start := time.Now()
+	peer := peerAddress(r)
+	trusted := h.config.TrustedProxies
+	peerTrusted := isInside(peer, trusted)
 	forwardedFor := r.Header.Values("X-Forwarded-For")
+	client := clientAddress(peer, forwardedFor, trusted)
 
 	rq := &request{
-		decision: decision{
-			client:     client,
-			requestURI: r.URL.RequestURI(),
-			line: requestlog.Line{
-				Time:             requestlog.FormatTime(start),
-				Instance:         h.config.InstanceName,
-				ClientIP:         client.String(),
-				Method:           r.Method,
-				Scheme:           scheme(r, peerTrusted),
-				Host:             r.Host,
-				Path:             rules.Path(r),
-				Query:            r.URL.RawQuery,
-				Protocol:         r.Proto,
-				Referer:          r.Referer(),
-				UserAgent:        r.UserAgent(),
-				RequestID:        requestID(r, peerTrusted),
-				PeerIP:           peer.String(),
-				ForwardedFor:     strings.Join(forwardedFor, ", "),
-				ClientGroup:      h.clientGroup(client).String(),
-				ContentType:      r.Header.Get("Content-Type"),
-				RequestHeaders:   requestHeaders(r, h.config.LogRequestHeaders),
-				HasAuthorization: len(r.Header.Values("Authorization")) > 0,
-				HasCookie:        len(r.Header.Values("Cookie")) > 0,
-				Action:           requestlog.ActionForward,
-			},
-		},
 		h:           h,
 		in:          r,
 		rc:          http.NewResponseController(w),
 		out:         &responseWriter{ResponseWriter: w},
+		client:      client,
 		peer:        peer,
 		peerTrusted: peerTrusted,
 		start:       start,
+		line: requestlog.Line{
+			Time:             requestlog.FormatTime(start),
+			Instance:         h.config.InstanceName,
+			ClientIP:         client.String(),
+			Method:           r.Method,
+			Scheme:           scheme(r, peerTrusted),
+			Host:             r.Host,
+			Path:             rules.Path(r),
+			Query:            r.URL.RawQuery,
+			Protocol:         r.Proto,
+			Referer:          r.Referer(),
+			UserAgent:        r.UserAgent(),
+			RequestID:        requestID(r, peerTrusted),
+			PeerIP:           peer.String(),
+			ForwardedFor:     strings.Join(forwardedFor, ", "),
+			ClientGroup:      h.clientGroup(client).String(),
+			ContentType:      r.Header.Get("Content-Type"),
+			RequestHeaders:   requestHeaders(r, h.config.LogRequestHeaders),
+			HasAuthorization: len(r.Header.Values("Authorization")) > 0,
+			HasCookie:        len(r.Header.Values("Cookie")) > 0,
+			Action:           requestlog.ActionForward,
+		},
 	}
 
 	// A length of -1 is a body whose length was not announced.
@@ -226,12 +191,12 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
 // is known, before anything reaches the app, and before its body is read,
 // but for the part the Core Rule Set reads. It returns nil to let the
 // request through. The checks of checkClient come first, answered with
-// SWWAF_BAN_RESPONSE, 403 for a block rule or the Core Rule Set, or 431 for
+// SWWAF_BAN_RESPONSE, or 403 for a block rule or the Core Rule Set, and
-// servewaf's header limits, and then the size limit, so that a request the
+// then the size limit, so that a request the rate limits count is counted
-// rate limits count is counted even when it is refused for its size. In
+// even when it is refused for its size. In observe mode a request
-// observe mode a request checkClient refuses goes on to the size limit like
+// checkClient refuses goes on to the size limit like any other. A size or
-// any other. A size or time limit the Core Rule Set's reading of the body
+// time limit the Core Rule Set's reading of the body meets ends the
-// meets ends the request in either mode. ctx is the request's own context.
+// request in either mode. ctx is the request's own context.
 func (rq *request) check(ctx context.Context) *refusal {
 	action := rq.checkClient(ctx)
 
@@ -247,8 +212,6 @@ func (rq *request) check(ctx context.Context) *refusal {
 		rq.line.WouldAction = action
 	case action == requestlog.ActionRuleBlocked || action == requestlog.ActionWAFBlocked:
 		return &refusal{status: http.StatusForbidden, action: action}
-	case action == requestlog.ActionTooLarge:
-		return &refusal{status: http.StatusRequestHeaderFieldsTooLarge, action: action}
 	default:
 		return rq.banResponse(action)
 	}
@@ -277,10 +240,9 @@ func (rq *request) check(ctx context.Context) *refusal {
 // rate limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the
 // request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that
 // every other request is counted, each of them by the client's limit
-// percentages, then SWWAF_TRAP_PATHS, then the rule files, then, in
+// percentages, then SWWAF_TRAP_PATHS, then the rule files, and last the
-// servewaf, the header limits, and last the Core Rule Set. A request exempt
+// Core Rule Set. A request exempt from the rate limits is exempt from the
-// from the rate limits is exempt from the byte limits too. ctx is the
+// byte limits too. ctx is the request's own context.
-// request's own context.
 func (rq *request) checkClient(ctx context.Context) string {
 	cfg := rq.h.config
 	if isInside(rq.client, cfg.AllowNets) {
@@ -297,7 +259,7 @@ func (rq *request) checkClient(ctx context.Context) string {
 		return requestlog.ActionBanned
 	}
 
-	rq.lookUp(ctx, now)
+	rq.lookUp(ctx)
 
 	// The lookup can wait for GeoJS, so the checks after it read the time
 	// again: a request counted at the time it arrived could be counted
@@ -321,7 +283,8 @@ func (rq *request) checkClient(ctx context.Context) string {
 		return requestlog.ActionDenied
 	}
 
-	rq.counted = rq.countedByRateLimits()
+	rq.counted = !isInside(rq.client, cfg.RateLimitExemptNets) &&
+		!pathExempt(rq.in, cfg.RateLimitExemptPaths)
 	if rq.counted {
 		rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
 		rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
@@ -341,23 +304,9 @@ func (rq *request) checkClient(ctx context.Context) string {
 		return action
 	}
 
-	if rq.overHeaderLimits() {
-		return requestlog.ActionTooLarge
-	}
-
 	return rq.checkCoreRuleSet()
 }
 
-// countedByRateLimits reports whether the rate limits count the request: its
-// client is not in SWWAF_RATE_LIMIT_EXEMPT_NETS, and its path is not exempt
-// under SWWAF_RATE_LIMIT_EXEMPT_PATHS.
-func (rq *request) countedByRateLimits() bool {
-	cfg := rq.h.config
-
-	return !isInside(rq.client, cfg.RateLimitExemptNets) &&
-		!pathExempt(rq.in, cfg.RateLimitExemptPaths)
-}
-
 // pathExempt reports whether r is exempt under prefixes,
 // SWWAF_RATE_LIMIT_EXEMPT_PATHS or SWWAF_WAF_EXEMPT_PATHS: whether its
 // path as sent, the path the app receives, not percent-decoded, starts
@@ -620,20 +569,10 @@ func (rq *request) finish() {
 	}
 
 	// Counted before the log line is written, so that the metrics count
-	// every request whose line is out. An allowed request is the exception:
+	// every request whose line is out.
-	// its report gives the status, bytes and times the metrics count.
+	rq.h.metrics.RequestEnded(line, limit, duration, upstreamDuration)
-	if rq.allowed {
-		rq.h.metrics.RequestAllowed()
-	} else {
-		rq.h.metrics.RequestEnded(line, limit, duration, upstreamDuration)
-	}
 
-	rq.writeLine()
+	err := requestlog.Write(rq.h.requestLog, line)
-}
-
-// writeLine writes the request's log line.
-func (rq *request) writeLine() {
-	err := requestlog.Write(rq.h.requestLog, &rq.line)
 	if err != nil {
 		rq.h.processLog.Error("writing the request log failed", "error", err.Error())
 	}
@@ -695,7 +634,6 @@ func (rq *request) countAnomalies() {
 	answer, _ := rq.answerAtTheEnd()
 
 	rq.h.anomalies.Count(rq.h.now(), anomaly.Request{
-		Host:        rq.site,
 		Client:      rq.client,
 		ClientGroup: rq.h.clientGroup(rq.client),
 		ASN:         answer.ASN,

internal/proxy/servewaf.go

@@ -1,527 +0,0 @@
-package proxy
-
-import (
-	"bytes"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"net/http"
-	"net/netip"
-	"strconv"
-	"strings"
-	"time"
-	"unicode/utf8"
-
-	lru "github.com/hashicorp/golang-lru/v2"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// The headers of rproxy's question and of servewaf's answer, as
-// "servewaf" in SPEC.md gives them.
-const (
-	// ClientHeader marks a request as rproxy's question, and gives the
-	// visitor's address.
-	ClientHeader = "X-Smallwebwaf-Client"
-	// PeerHeader gives the address of rproxy's own TCP peer.
-	PeerHeader = "X-Smallwebwaf-Peer"
-	// ContentLengthHeader gives the body length the visitor announced.
-	ContentLengthHeader = "X-Smallwebwaf-Content-Length"
-	// DispositionHeader gives servewaf's answer.
-	DispositionHeader = "X-Smallwebwaf-Disposition"
-)
-
-// The values of DispositionHeader.
-const (
-	// DispositionAllow tells rproxy to pass the request to the app.
-	DispositionAllow = "allow"
-	// DispositionDeny tells rproxy to give the visitor the answer's status,
-	// headers and body.
-	DispositionDeny = "deny"
-	// DispositionClose tells rproxy to close the visitor's connection
-	// without an answer, which SWWAF_BAN_RESPONSE asks for when it is close.
-	// servewaf closing the question's connection instead would read as
-	// servewaf being down, and rproxy would then let the request through.
-	DispositionClose = "close"
-)
-
-// ReportPath is where rproxy sends the report of each request servewaf
-// answered.
-const ReportPath = "/_smallwebwaf/report"
-
-// QuestionBodyMaxBytes is the most of a visitor's body a question carries.
-const QuestionBodyMaxBytes = 64 << 10
-
-// The header limits: the most cookies and header fields a request may have
-// when the checks reach the Core Rule Set. Real clients never reach them.
-const (
-	maxCookies      = 50
-	maxHeaderFields = 100
-)
-
-const (
-	// maxAwaitingReport is the most answered requests held until their
-	// reports come. Past it, the request answered longest ago is dropped.
-	maxAwaitingReport = 10000
-	// maxRequestIDBytes is the longest id a question may give.
-	maxRequestIDBytes = 128
-	// reportBodyMaxBytes is the most of a report's body that is read. It is
-	// far above any report's.
-	reportBodyMaxBytes = 1 << 20
-	// maxHTTPStatus is the highest status a report may give.
-	maxHTTPStatus = 599
-	// maxReportedBytes is the most bytes a report may give for each way, 1
-	// PiB, so that adding them up cannot overflow.
-	maxReportedBytes = 1 << 50
-	// maxReportedMilliseconds is the longest duration a report may give,
-	// about 31 years, so that it fits a time.Duration.
-	maxReportedMilliseconds = 1e12
-	// serveWAFTimeout bounds how long a request to servewaf takes to arrive,
-	// and its answer to leave, and serveWAFIdleTimeout how long a kept-open
-	// connection waits for its next request.
-	serveWAFTimeout     = 60 * time.Second
-	serveWAFIdleTimeout = 120 * time.Second
-)
-
-var (
-	errReportFieldMissing = errors.New("a field is missing")
-	errReportValue        = errors.New("a value is out of range")
-)
-
-// Report is the JSON object rproxy posts to ReportPath once the visitor's
-// answer has ended, its fields named as the request log's of the same
-// meaning. RequestID is the question's id. Status is what the visitor got, 0
-// if nothing. The durations are in milliseconds.
-//
-//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
-type Report struct {
-	RequestID             string  `json:"request_id"`
-	Status                int     `json:"status"`
-	RequestBytes          int64   `json:"request_bytes"`
-	ResponseBytes         int64   `json:"response_bytes"`
-	DurationTotal         float64 `json:"duration_total"`
-	DurationUpstreamTotal float64 `json:"duration_upstream_total"`
-}
-
-// NewServeWAF returns the server `smallwebwaf servewaf` runs. It answers
-// rproxy's questions with the checks New runs, takes rproxy's reports and
-// answers its own endpoints. It passes nothing to an app. rproxy applies
-// the size and time limits to the visitor, so their settings do not apply
-// here. Go's default limit on a request's line and headers, 1 MiB, stays:
-// it is far above rproxy's, and a question refused for its size would have
-// rproxy let the request through.
-func NewServeWAF(params Params) *Server {
-	cfg := *params.Config
-	cfg.RequestMaxBytes = 0
-	cfg.ClientRequestTimeout = serveWAFTimeout
-	// The server's WriteTimeout does this. It sets a new deadline for each
-	// request, where a deadline a handler sets stays on a kept-open
-	// connection, and would run out under a later request's answer.
-	cfg.ClientResponseTimeout = 0
-	// A question's body of QuestionBodyMaxBytes may be cut short, and the
-	// Core Rule Set does not read a JSON or XML body over its limit.
-	cfg.WAFBodyLimit = min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1)
-	params.Config = &cfg
-
-	h := newHandler(params)
-
-	awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
-	if err != nil {
-		panic(err) // New fails only for a size below one
-	}
-
-	h.awaitingReport = awaitingReport
-
-	return h.server(&http.Server{
-		Handler:      http.HandlerFunc(h.serveWAF),
-		ReadTimeout:  serveWAFTimeout,
-		WriteTimeout: serveWAFTimeout,
-		IdleTimeout:  serveWAFIdleTimeout,
-	})
-}
-
-// serveWAF handles one request to servewaf: a request carrying
-// ClientHeader is rproxy's question, and any other is for servewaf itself.
-func (h *handler) serveWAF(w http.ResponseWriter, r *http.Request) {
-	if len(r.Header.Values(ClientHeader)) > 0 {
-		h.answerQuestion(w, r)
-
-		return
-	}
-
-	h.answerOwnEndpoint(w, r)
-}
-
-// answersQuestions reports whether h is servewaf, which alone holds requests
-// for their reports.
-func (h *handler) answersQuestions() bool {
-	return h.awaitingReport != nil
-}
-
-// refusedWhateverTheAnswer reports whether the request is one that does not wait
-// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
-// its client: it is over the header limits, a blocklist lists its client while
-// SWWAF_BLOCKLIST_ACTION is deny, the CrowdSec decision list lists it, or it is
-// over a whole rate limit. In observe mode, which refuses nothing, none is.
-// rproxy lets a request through when servewaf answers too late.
-func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
-	if rq.h.config.Observe {
-		return false
-	}
-
-	lists := rq.h.lists
-	_, decided := lists.CrowdSecDecision(rq.client, now)
-
-	return decided || rq.overHeaderLimits() ||
-		(rq.h.config.BlocklistAction == deny && len(lists.ListedBy(rq.client)) > 0) ||
-		(rq.countedByRateLimits() &&
-			rq.h.limiter.OverARateLimit(rq.h.clientGroup(rq.client), now))
-}
-
-// overHeaderLimits reports whether servewaf refuses the request for its
-// headers: it has more than maxHeaderFields fields, each value of a header sent
-// twice being one, or more than maxCookies cookies, each part of a Cookie header
-// between semicolons being one unless it is empty or only spaces and tabs. The
-// Core Rule Set runs each of its rules on each cookie, so that thousands of small
-// cookies keep it busy for over a second. It trims only spaces and control
-// characters, of which a header value can hold only tabs, so a part of Unicode
-// spaces is a cookie to it.
-func (rq *request) overHeaderLimits() bool {
-	if !rq.h.answersQuestions() {
-		return false
-	}
-
-	fields, cookies := 0, 0
-
-	for _, values := range rq.in.Header {
-		fields += len(values)
-	}
-
-	for _, value := range rq.in.Header.Values("Cookie") {
-		for part := range strings.SplitSeq(value, ";") {
-			if strings.Trim(part, " \t") != "" {
-				cookies++
-			}
-		}
-	}
-
-	return fields > maxHeaderFields || cookies > maxCookies
-}
-
-// answerQuestion runs the checks on the visitor's request a question
-// carries, and answers rproxy what to do with it. A denied request's counting
-// is finished here, and an allowed one's by its report.
-func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
-	client, body, ok := h.readQuestion(w, r)
-	if !ok {
-		return
-	}
-
-	// One that cannot be read leaves the log line's peer_ip invalid.
-	peer, _ := netip.ParseAddr(r.Header.Get(PeerHeader))
-
-	rq := h.startRequest(w, asTheVisitorSentIt(r, body), client.Unmap(), peer.Unmap(),
-		true)
-	// The last deferred runs first, so that what is held has the finished log
-	// line. Go's server sends the answer only once this function has
-	// returned, so the report cannot come before it is held.
-	defer rq.holdForReport()
-	defer rq.finish()
-
-	rq.site = r.Host
-	rq.line.ServeWAFLine = requestlog.ServeWAFLineDecision
-	refused := rq.check(r.Context())
-	rq.checked = time.Now()
-	underOwnPrefix := strings.HasPrefix(r.URL.Path, adminPrefix)
-
-	if refused == nil && !underOwnPrefix {
-		// In observe mode, what enforce mode would have refused counts here.
-		defer rq.countRefusal()
-
-		rq.allow()
-
-		return
-	}
-
-	// The last deferred runs first, as in ServeHTTP.
-	defer rq.addToHistory()
-	defer rq.countAnomalies()
-	defer rq.countRefusal()
-
-	if refused != nil {
-		rq.deny(*refused)
-
-		return
-	}
-
-	// The app never sees this prefix. servewaf's own endpoints are at its
-	// own address, so this answers as a path with no endpoint does.
-	rq.line.Action = requestlog.ActionAdmin
-	rq.out.Header().Set(DispositionHeader, DispositionDeny)
-	http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
-}
-
-// readQuestion returns the visitor's address and the body a question carries,
-// and whether servewaf takes the question. It answers a question it does not
-// take with 400 and no disposition, which rproxy treats as it treats no
-// answer. The id of a question it takes is held from here on, without a
-// decision yet, so that another question with that id is refused while this
-// one is being handled.
-//
-// The body is read before any check, since Go's server does not send an answer
-// until it has read the body. A question whose body stops arriving gets no
-// answer, and is not looked at.
-func (h *handler) readQuestion(
-	w http.ResponseWriter, r *http.Request,
-) (netip.Addr, []byte, bool) {
-	client, err := netip.ParseAddr(r.Header.Get(ClientHeader))
-	if err != nil || client.Zone() != "" {
-		h.refuseQuestion(w, ClientHeader+" is not an address without a zone")
-
-		return netip.Addr{}, nil, false
-	}
-
-	id := r.Header.Get(requestIDHeader)
-	if id == "" || len(id) > maxRequestIDBytes || !utf8.ValidString(id) {
-		h.refuseQuestion(w, requestIDHeader+" is not a valid id")
-
-		return netip.Addr{}, nil, false
-	}
-
-	body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, QuestionBodyMaxBytes))
-
-	if _, tooLarge := errors.AsType[*http.MaxBytesError](err); tooLarge {
-		h.refuseQuestion(w, "the body is over 64 KiB")
-
-		return netip.Addr{}, nil, false
-	}
-
-	if err != nil {
-		h.processLog.Warn("a question's body stopped arriving", "error", err.Error())
-		// This panic has Go's server close the connection without an answer,
-		// and log nothing.
-		panic(http.ErrAbortHandler)
-	}
-
-	held, _ := h.awaitingReport.ContainsOrAdd(id, nil)
-	if held {
-		h.refuseQuestion(w, requestIDHeader+" is the id of a request servewaf holds")
-
-		return netip.Addr{}, nil, false
-	}
-
-	return client, body, true
-}
-
-// refuseQuestion answers a question servewaf does not take, for reason.
-func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) {
-	h.processLog.Warn("a question is refused", "reason", reason)
-	http.Error(w, reason, http.StatusBadRequest)
-}
-
-// asTheVisitorSentIt returns a copy of r, a question, as the checks and the
-// log line are to see it: without the headers rproxy adds for servewaf, with
-// body, which has been read of r, and with the length the visitor announced,
-// as -1 when it announced none. The question's own length covers only the part
-// of the body it carries.
-func asTheVisitorSentIt(r *http.Request, body []byte) *http.Request {
-	sent := r.Clone(r.Context())
-	announced := sent.Header.Get(ContentLengthHeader)
-
-	sent.Header.Del(ClientHeader)
-	sent.Header.Del(PeerHeader)
-	sent.Header.Del(ContentLengthHeader)
-	sent.Header.Del("Content-Length")
-
-	sent.ContentLength = -1
-
-	length, err := strconv.ParseInt(announced, 10, 64)
-	if err == nil && length >= 0 {
-		sent.ContentLength = length
-		sent.Header.Set("Content-Length", announced)
-	}
-
-	sent.Body = http.NoBody
-	if len(body) > 0 {
-		sent.Body = io.NopCloser(bytes.NewReader(body))
-	}
-
-	return sent
-}
-
-// allow answers a question with DispositionAllow and, while
-// SWWAF_ADD_LOOKUP_HEADERS is set, the client's AS number and country.
-func (rq *request) allow() {
-	rq.allowed = true
-
-	header := rq.out.Header()
-	header.Set(DispositionHeader, DispositionAllow)
-
-	if rq.h.config.AddLookupHeaders {
-		setLookupHeaders(header, rq.line.ASN, rq.line.Country)
-	}
-
-	rq.out.WriteHeader(http.StatusOK)
-}
-
-// holdForReport holds a copy of what the report's steps need, by the request's
-// id. A report only ever reaches that copy, never the request being handled.
-func (rq *request) holdForReport() {
-	held := rq.decision
-	rq.h.awaitingReport.Add(rq.line.RequestID, &held)
-}
-
-// deny answers a question with what the visitor is to get for r.
-func (rq *request) deny(r refusal) {
-	if r.status == 0 {
-		// SWWAF_BAN_RESPONSE is close. Written past rq.out, since the log line's
-		// status stays 0: the visitor is sent nothing.
-		rq.refused.CompareAndSwap(nil, &r)
-		rq.out.Header().Set(DispositionHeader, DispositionClose)
-		rq.out.ResponseWriter.WriteHeader(http.StatusOK)
-
-		return
-	}
-
-	rq.out.Header().Set(DispositionHeader, DispositionDeny)
-	rq.answer(r)
-}
-
-// answerOwnEndpoint answers a request to servewaf's own address. No check
-// runs on it: the address is on the private network, where the peers are
-// rproxy and the admin's tools, not visitors.
-func (h *handler) answerOwnEndpoint(w http.ResponseWriter, r *http.Request) {
-	if r.Method == http.MethodPost && r.URL.Path == ReportPath {
-		h.takeReport(w, r)
-
-		return
-	}
-
-	peer := peerAddress(r)
-
-	rq := h.startRequest(w, r, peer, peer, false)
-	defer rq.finish()
-
-	if r.Method == http.MethodGet && r.URL.Path == HealthPath {
-		rq.answerHealth()
-
-		return
-	}
-
-	rq.answerAdmin()
-}
-
-// takeReport answers POST ReportPath: it finishes the request the report
-// names, if it holds that request's decision, and ignores the report
-// otherwise.
-func (h *handler) takeReport(w http.ResponseWriter, r *http.Request) {
-	report, err := readReport(w, r)
-	if err != nil {
-		h.processLog.Warn("a report is refused", "error", err.Error())
-		http.Error(w, "the body is not a report", http.StatusBadRequest)
-
-		return
-	}
-
-	held, found := h.awaitingReport.Peek(report.RequestID)
-	// Remove reports true to one report only, of several that name the id.
-	if found && held != nil && h.awaitingReport.Remove(report.RequestID) {
-		h.finishWith(held, report)
-	}
-
-	w.WriteHeader(http.StatusNoContent)
-}
-
-// readReport reads a report from r's body, which must be one JSON object with
-// all of Report's fields, in the ranges SPEC.md gives.
-func readReport(w http.ResponseWriter, r *http.Request) (Report, error) {
-	var report Report
-
-	body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, reportBodyMaxBytes))
-	if err != nil {
-		return report, fmt.Errorf("read the body: %w", err)
-	}
-
-	var fields map[string]json.RawMessage
-
-	err = json.Unmarshal(body, &fields)
-	if err != nil {
-		return report, fmt.Errorf("decode the body: %w", err)
-	}
-
-	for _, name := range []string{
-		"request_id", "status", "request_bytes", "response_bytes",
-		"duration_total", "duration_upstream_total",
-	} {
-		value, there := fields[name]
-		if !there || string(value) == "null" {
-			return report, fmt.Errorf("%w: %s", errReportFieldMissing, name)
-		}
-	}
-
-	err = json.Unmarshal(body, &report)
-	if err != nil {
-		return report, fmt.Errorf("decode the body: %w", err)
-	}
-
-	return report, report.checkRanges()
-}
-
-// checkRanges returns an error for a value of the report that cannot be true.
-func (report *Report) checkRanges() error {
-	switch {
-	case report.Status != 0 &&
-		(report.Status < http.StatusContinue || report.Status > maxHTTPStatus):
-		return fmt.Errorf("%w: status", errReportValue)
-	case report.RequestBytes < 0 || report.RequestBytes > maxReportedBytes:
-		return fmt.Errorf("%w: request_bytes", errReportValue)
-	case report.ResponseBytes < 0 || report.ResponseBytes > maxReportedBytes:
-		return fmt.Errorf("%w: response_bytes", errReportValue)
-	case report.DurationTotal < 0 || report.DurationTotal > maxReportedMilliseconds:
-		return fmt.Errorf("%w: duration_total", errReportValue)
-	case report.DurationUpstreamTotal < 0 ||
-		report.DurationUpstreamTotal > maxReportedMilliseconds:
-		return fmt.Errorf("%w: duration_upstream_total", errReportValue)
-	}
-
-	return nil
-}
-
-// finishWith finishes a request with what its report says, and writes the
-// request's last log line, which holds its decision's fields and the report's.
-// For an allowed request it first takes the steps ServeHTTP takes once a
-// request passed to the app has ended, with the status and bytes of the report
-// in place of those it would have seen. A denied request's counting was
-// finished at its decision.
-func (h *handler) finishWith(held *decision, report Report) {
-	rq := &request{
-		decision: *held,
-		h:        h,
-		out:      &responseWriter{status: report.Status, bytes: report.ResponseBytes},
-		body:     &requestBody{},
-	}
-	rq.body.bytes.Store(report.RequestBytes)
-
-	rq.line.ServeWAFLine = requestlog.ServeWAFLineReport
-	rq.line.Status = report.Status
-	rq.line.RequestBytes = report.RequestBytes
-	rq.line.ResponseBytes = report.ResponseBytes
-	rq.line.DurationTotal = report.DurationTotal
-
-	if rq.allowed {
-		rq.line.DurationUpstreamTotal = &report.DurationUpstreamTotal
-		// addToHistory goes by this for whether the request reached the app.
-		rq.upstreamStart = time.Now()
-
-		rq.countBytes()
-		rq.countAnomalies()
-		rq.addToHistory()
-
-		rq.h.metrics.RequestReported(&rq.line,
-			time.Duration(report.DurationTotal*float64(time.Millisecond)),
-			time.Duration(report.DurationUpstreamTotal*float64(time.Millisecond)))
-	}
-
-	rq.writeLine()
-}

internal/ratelimit/ratelimit.go

@@ -206,7 +206,7 @@ type Hit struct {
 // Counts are a client's requests and bytes in the minute, the hour and
 // the day that end at a request, that request's included.
 //
-//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
+//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
 type Counts struct {
 	Minute      float64 `json:"minute"`
 	Hour        float64 `json:"hour"`
@@ -228,30 +228,6 @@ func (l *Limiter) Count(
 	return l.count(client, now, 1, 0, percent)
 }
 
-// OverARateLimit reports whether one more request from client at now would
-// take it over a whole rate limit. A client's percentage can only lower a
-// limit, so that request is over the limit at any percentage. It counts
-// nothing, and does not make the client the most recently seen.
-func (l *Limiter) OverARateLimit(client netip.Prefix, now time.Time) bool {
-	l.mu.Lock()
-	defer l.mu.Unlock()
-
-	c, seen := l.clients.Peek(client)
-	if !seen {
-		return false
-	}
-
-	buckets := c.buckets()
-
-	for i, w := range l.windows {
-		if w.limit > 0 && buckets[i].Add(now, w.length, 0)+1 > float64(w.limit) {
-			return true
-		}
-	}
-
-	return false
-}
-
 // CountBytes counts bytes, those of a request from client that has ended,
 // at now, in every window, and returns the client's counts in each window.
 // It reports whether the bytes take the client over a byte limit, of which

internal/ratelimit/ratelimit_test.go

@@ -26,50 +26,6 @@ const (
 	hour   = "hour"
 )
 
-func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.T) {
-	t.Parallel()
-
-	client := netip.MustParsePrefix("203.0.113.9/32")
-	now := midnight()
-	limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerDay: limit + 1},
-		tableSize)
-
-	if limiter.OverARateLimit(client, now) {
-		t.Error("a client never seen is over a rate limit")
-	}
-
-	if limiter.Len() != 0 {
-		t.Error("asking about a client never seen added it to the table")
-	}
-
-	for range limit {
-		if limiter.OverARateLimit(client, now) {
-			t.Error("a client with room for another request is over a rate limit")
-		}
-
-		wantCount(t, limiter, client, now, "")
-	}
-
-	if !limiter.OverARateLimit(client, now) {
-		t.Error("a client at its limit is not over a rate limit")
-	}
-
-	held, _ := limiter.Client(client)
-	if held.Minute.Current != limit {
-		t.Errorf("the client's minute holds %d requests, want %d", held.Minute.Current, limit)
-	}
-
-	wantCount(t, limiter, client, now, minute)
-
-	// A limit that is off is never gone over.
-	off := ratelimit.New(ratelimit.Limits{}, tableSize)
-	off.Count(client, now, whole)
-
-	if off.OverARateLimit(client, now) {
-		t.Error("a client is over a rate limit that is off")
-	}
-}
-
 func TestEachWindowRefusesAtItsLimitAndLetsTheClientBack(t *testing.T) {
 	t.Parallel()
 

internal/requestlog/requestlog.go

@@ -1,6 +1,6 @@
 // Package requestlog writes the lines smallwebwaf prints on stdout: one
-// JSON object per request, or two from servewaf, marked "type":"request", and
+// JSON object per request, marked "type":"request", and the process's own
-// the process's own messages as JSON lines marked "type":"process".
+// messages as JSON lines marked "type":"process".
 package requestlog
 
 import (
@@ -18,8 +18,7 @@ import (
 const (
 	// ActionForward is a request passed to the app.
 	ActionForward = "forward"
-	// ActionTooLarge is a request or response over its size limit, or, in
+	// ActionTooLarge is a request or response over its size limit.
-	// servewaf, a request over the header limits.
 	ActionTooLarge = "too_large"
 	// ActionTimedOut is a request or response that ran out of time.
 	ActionTimedOut = "timed_out"
@@ -61,29 +60,17 @@ const OffenceLimit = "limit"
 // that broke the error burst.
 const LimitHitErrorBurst = "error_burst"
 
-// The servewaf_line a request line names, which says which of servewaf's two
-// lines about a question it is.
-const (
-	// ServeWAFLineDecision is servewaf's line at its decision.
-	ServeWAFLineDecision = "decision"
-	// ServeWAFLineReport is servewaf's line when rproxy's report comes, with
-	// the decision's fields and the report's together.
-	ServeWAFLineReport = "report"
-)
-
 // timeLayout is RFC 3339 with milliseconds.
 const timeLayout = "2006-01-02T15:04:05.000Z07:00"
 
 // Line is one request's line in the request log. The field names, and
-// their order, are those of docs/request-log.md. A field that may not
+// their order, are those of the "Request log" section of SPEC.md. A field
-// apply to a request is left out of its line when it does not.
+// that may not apply to a request is left out of its line when it does
+// not.
 //
-//nolint:tagliatelle // docs/request-log.md names the fields in snake_case
+//nolint:tagliatelle // SPEC.md's request log names its fields in snake_case
 type Line struct {
 	Type string `json:"type"`
-	// ServeWAFLine is ServeWAFLineDecision or ServeWAFLineReport on servewaf's
-	// two lines about a question.
-	ServeWAFLine string `json:"servewaf_line,omitempty"`
 
 	// The standard web log fields. Scheme is how the client reached
 	// smallwebwaf, or the trusted proxy in front of it.
@@ -140,8 +127,7 @@ type Line struct {
 	// WouldAction is, in observe mode, the action enforce mode would have
 	// taken with a request it would have refused: ActionDenied,
 	// ActionBanned, ActionCountryDenied, ActionRateLimited,
-	// ActionRuleBlocked, ActionWAFBlocked, or, in servewaf, ActionTooLarge
+	// ActionRuleBlocked or ActionWAFBlocked.
-	// for the header limits.
 	WouldAction string `json:"would_action,omitempty"`
 	// LimitPercent and LimitPercentSetting are, for a request the rate
 	// limits counted whose client a biased threshold gives a percentage of

internal/smallwebwaf/healthcheck_test.go

@@ -60,19 +60,8 @@ func TestHealthCheck(t *testing.T) {
 		}, 0, "")
 	}
 
-	// Another test can take a closed server's port at once, so the app's
+	app.Close()
-	// address here is a multicast address, to which TCP cannot connect.
+	wantHealthCheck(t, env, 1, "unhealthy: connect to the app: ")
-	wantHealthCheck(t, map[string]string{
-		listenAddr:  ":" + port,
-		upstreamURL: "http://224.0.0.1",
-	}, 1, "unhealthy: connect to the app: ")
-
-	// Listening on port 0 takes any free port, so nothing listens on port 0
-	// itself.
-	wantHealthCheck(t, map[string]string{listenAddr: ":0"}, 1,
-		"unhealthy: ask smallwebwaf: ")
-	wantHealthCheck(t, map[string]string{listenAddr: "8080"}, 1,
-		"unhealthy: invalid setting: SWWAF_LISTEN_ADDR: ")
 
 	stop()
 
@@ -81,6 +70,10 @@ func TestHealthCheck(t *testing.T) {
 	case <-time.After(waitLimit):
 		t.Fatal("still running after being told to stop")
 	}
+
+	wantHealthCheck(t, env, 1, "unhealthy: ask smallwebwaf: ")
+	wantHealthCheck(t, map[string]string{listenAddr: "8080"}, 1,
+		"unhealthy: invalid setting: SWWAF_LISTEN_ADDR: ")
 }
 
 func TestHealthCheckRefusesAnArgument(t *testing.T) {

internal/smallwebwaf/servewaf_test.go

@@ -1,210 +0,0 @@
-package smallwebwaf_test
-
-import (
-	"context"
-	"crypto/rand"
-	"io"
-	"net/http"
-	"os"
-	"os/exec"
-	"path/filepath"
-	"strings"
-	"syscall"
-	"testing"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/proxy"
-	"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
-)
-
-// commandForMain, while set in the environment, has this test binary run Main
-// with the command it gives, in place of the tests, so that a test can run
-// Main in a process of its own.
-const commandForMain = "SMALLWEBWAF_TEST_COMMAND"
-
-func TestMain(m *testing.M) {
-	command, set := os.LookupEnv(commandForMain)
-	if !set {
-		os.Exit(m.Run())
-	}
-
-	os.Args = strings.Fields("smallwebwaf " + command)
-
-	os.Exit(smallwebwaf.Main(testVersion))
-}
-
-func TestMainAnswersQuestionsOnlyAsServeWAF(t *testing.T) {
-	t.Parallel()
-
-	for _, tc := range []struct {
-		command, disposition string
-	}{
-		{"servewaf", proxy.DispositionAllow},
-		{"", ""},
-	} {
-		t.Run("command "+tc.command, func(t *testing.T) {
-			t.Parallel()
-
-			binary, err := os.Executable()
-			if err != nil {
-				t.Fatalf("find the test binary: %v", err)
-			}
-
-			ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit)
-			defer cancel()
-
-			out := &output{}
-			cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary
-			cmd.Stdout = out
-
-			cmd.Env = append(os.Environ(),
-				commandForMain+"="+tc.command,
-				listenAddr+"="+localhost+":0",
-				upstreamURL+"="+startApp(t),
-				stateDir+"="+t.TempDir(),
-				rulesDir+"="+t.TempDir(),
-				lookupSource+"=off")
-
-			err = cmd.Start()
-			if err != nil {
-				t.Fatalf("start: %v", err)
-			}
-
-			t.Cleanup(func() {
-				_ = cmd.Process.Kill()
-				_ = cmd.Wait()
-			})
-
-			addr, _ := out.line(t, "msg", "starting")["address"].(string)
-			ask(t, addr, "/", http.StatusOK, tc.disposition)
-
-			err = cmd.Process.Signal(syscall.SIGTERM)
-			if err != nil {
-				t.Fatalf("signal: %v", err)
-			}
-
-			err = cmd.Wait()
-			if err != nil {
-				t.Errorf("exit: %v; output:\n%s", err, out.text())
-			}
-		})
-	}
-}
-
-// ask sends servewaf at addr rproxy's question about a request for path from
-// placed, checks the answer's status and disposition, and closes it.
-func ask(t *testing.T, addr, path string, status int, disposition string) {
-	t.Helper()
-
-	req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
-		"http://"+addr+path, http.NoBody)
-	if err != nil {
-		t.Fatalf("new request: %v", err)
-	}
-
-	req.Header.Set(proxy.ClientHeader, placed)
-	req.Header.Set("X-Request-ID", rand.Text())
-
-	transport := &http.Transport{}
-	defer transport.CloseIdleConnections()
-
-	res, err := (&http.Client{Transport: transport}).Do(req)
-	if err != nil {
-		t.Fatalf("question: %v", err)
-	}
-
-	_, _ = io.Copy(io.Discard, res.Body)
-	_ = res.Body.Close()
-
-	if res.StatusCode != status ||
-		res.Header.Get(proxy.DispositionHeader) != disposition {
-		t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode,
-			res.Header.Get(proxy.DispositionHeader), status, disposition)
-	}
-}
-
-// runServeWAF runs `smallwebwaf servewaf` with the settings in env, has use
-// send it requests at the address it listens on, stops it as SIGTERM does, and
-// checks that it stopped in order.
-func runServeWAF(t *testing.T, env map[string]string, use func(addr string)) {
-	t.Helper()
-
-	ctx, stop := context.WithCancel(t.Context())
-	out := &output{}
-	exited := make(chan int, 1)
-
-	go func() {
-		exited <- runAs(ctx, true, env, out)
-	}()
-
-	addr, _ := out.line(t, "msg", "starting")["address"].(string)
-	use(addr)
-	stop()
-
-	select {
-	case status := <-exited:
-		if status != 0 {
-			t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
-		}
-	case <-time.After(waitLimit):
-		t.Fatal("still running after being told to stop")
-	}
-
-	out.line(t, "msg", "stopped")
-}
-
-func TestServeWAFKeepsItsBansAcrossARestartAndNeedsNoApp(t *testing.T) {
-	t.Parallel()
-
-	env := map[string]string{
-		listenAddr: localhost + ":0",
-		stateDir:   t.TempDir(),
-		rulesDir:   filepath.Join("..", "..", "share", "rules.d"),
-	}
-
-	runServeWAF(t, env, func(addr string) {
-		ask(t, addr, "/", http.StatusOK, proxy.DispositionAllow)
-		// The default rule file bans a probe for this.
-		ask(t, addr, "/.env", http.StatusForbidden, proxy.DispositionDeny)
-	})
-
-	runServeWAF(t, env, func(addr string) {
-		ask(t, addr, "/", http.StatusForbidden, proxy.DispositionDeny)
-	})
-}
-
-func TestWithoutACommandTheClientHeaderIsNoQuestion(t *testing.T) {
-	t.Parallel()
-
-	runUntilStopped(t, map[string]string{
-		listenAddr:  localhost + ":0",
-		upstreamURL: startApp(t),
-		stateDir:    t.TempDir(),
-		rulesDir:    t.TempDir(),
-	}, func(url string) {
-		req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, url,
-			http.NoBody)
-		if err != nil {
-			t.Fatalf("new request: %v", err)
-		}
-
-		req.Header.Set(proxy.ClientHeader, placed)
-
-		transport := &http.Transport{}
-		defer transport.CloseIdleConnections()
-
-		res, err := (&http.Client{Transport: transport}).Do(req)
-		if err != nil {
-			t.Fatalf("request: %v", err)
-		}
-
-		body, err := io.ReadAll(res.Body)
-		_ = res.Body.Close()
-
-		if err != nil || string(body) != greeting ||
-			res.Header.Get(proxy.DispositionHeader) != "" {
-			t.Errorf("answered %q (%v) with disposition %q, want the app's greeting",
-				body, err, res.Header.Get(proxy.DispositionHeader))
-		}
-	})
-}

internal/smallwebwaf/smallwebwaf.go

@@ -1,8 +1,7 @@
 // Package smallwebwaf runs the smallwebwaf process: it reads the settings,
 // the rule files, the lookup database and the state files, serves requests
 // until it is told to stop, and then stops in an orderly way, writing the
-// state files. It serves them in front of an app, or, as `smallwebwaf
+// state files.
-// servewaf`, by answering rproxy's questions about them.
 package smallwebwaf
 
 import (
@@ -46,15 +45,11 @@ type Params struct {
 	LookupEnv func(string) (string, bool)
 	// Stdout receives the request log and the process's own messages.
 	Stdout io.Writer
-	// ServeWAF makes it `smallwebwaf servewaf`, which answers rproxy's
-	// questions, rather than the proxy in front of an app.
-	ServeWAF bool
 }
 
 // Main runs smallwebwaf until SIGTERM or SIGINT, and returns the
-// process's exit status. Run as `smallwebwaf servewaf`, it answers
+// process's exit status. Run as `smallwebwaf healthcheck`, it is the
-// rproxy's questions rather than pass requests to an app. Run as
+// container's health check instead.
-// `smallwebwaf healthcheck`, it is the container's health check instead.
 func Main(version string) int {
 	if len(os.Args) > 1 && os.Args[1] == "healthcheck" {
 		return HealthCheck(context.Background(), os.Args[2:], os.LookupEnv, os.Stderr)
@@ -68,7 +63,6 @@ func Main(version string) int {
 		Version:   version,
 		LookupEnv: os.LookupEnv,
 		Stdout:    os.Stdout,
-		ServeWAF:  len(os.Args) > 1 && os.Args[1] == "servewaf",
 	})
 }
 
@@ -123,12 +117,7 @@ func Run(ctx context.Context, params Params) int {
 		return 1
 	}
 
-	newProxy := proxy.New
+	server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue)
-	if params.ServeWAF {
-		newProxy = proxy.NewServeWAF
-	}
-
-	server, err := newServer(newProxy, cfg, stdout, processLog, now, ruleFiles, alertQueue)
 	if err != nil {
 		processLog.Error("cannot use the lookup database", "error", err.Error())
 
@@ -162,12 +151,10 @@ func Run(ctx context.Context, params Params) int {
 	return serve(ctx, server, listener, files, ruleFiles, alertQueue, processLog)
 }
 
-// newServer returns the server smallwebwaf runs, which newProxy makes, with
+// newServer returns the server smallwebwaf runs, with the metrics of the
-// the metrics of the alerts, after reading the lookup database while
+// alerts, after reading the lookup database while SWWAF_LOOKUP_SOURCE is
-// SWWAF_LOOKUP_SOURCE is file. A lookup database that cannot be read is an
+// file. A lookup database that cannot be read is an error.
-// error.
 func newServer(
-	newProxy func(proxy.Params) *proxy.Server,
 	cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
 	now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
 ) (*proxy.Server, error) {
@@ -187,7 +174,7 @@ func newServer(
 		}
 	}
 
-	server := newProxy(proxy.Params{
+	server := proxy.New(proxy.Params{
 		Config:       cfg,
 		RequestLog:   stdout,
 		ProcessLog:   processLog,

internal/smallwebwaf/smallwebwaf_test.go

@@ -113,13 +113,6 @@ func (o *output) text() string {
 // returns its exit status. SWWAF_LOOKUP_SOURCE is off unless env sets it,
 // so that no test sends GeoJS its clients' addresses.
 func run(ctx context.Context, env map[string]string, out *output) int {
-	return runAs(ctx, false, env, out)
-}
-
-// runAs is run as `smallwebwaf servewaf` while serveWAF is true.
-func runAs(
-	ctx context.Context, serveWAF bool, env map[string]string, out *output,
-) int {
 	return smallwebwaf.Run(ctx, smallwebwaf.Params{
 		Version: testVersion,
 		LookupEnv: func(name string) (string, bool) {
@@ -130,8 +123,7 @@ func runAs(
 
 			return value, ok
 		},
-		Stdout:   out,
+		Stdout: out,
-		ServeWAF: serveWAF,
 	})
 }
 
@@ -1248,13 +1240,13 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
 		"SWWAF_UPSTREAM_REQUEST_TIMEOUT":        "60s",
 		"SWWAF_UPSTREAM_RESPONSE_TIMEOUT":       "30m",
 		"SWWAF_REQUEST_MAX_BYTES":               "100M",
-		"SWWAF_RESPONSE_MAX_BYTES":              "1G",
+		"SWWAF_RESPONSE_MAX_BYTES":              "5G",
 		"SWWAF_ALLOW_NETS":                      "",
 		rateLimitExemptNets:                     "",
 		"SWWAF_DENY_NETS":                       "",
-		"SWWAF_RATE_LIMIT_PER_MINUTE":           "500",
+		"SWWAF_RATE_LIMIT_PER_MINUTE":           "1000",
-		"SWWAF_RATE_LIMIT_PER_HOUR":             "20000",
+		"SWWAF_RATE_LIMIT_PER_HOUR":             "10000",
-		rateLimitPerDay:                         "300000",
+		rateLimitPerDay:                         "50000",
 		"SWWAF_RATE_LIMIT_EXEMPT_PATHS":         "",
 		"SWWAF_DENIED_COUNTRIES":                "",
 		"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES":   "",

internal/state/state.go

@@ -1,6 +1,5 @@
 // Package state keeps smallwebwaf's state in JSON files in
-// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md requires
+// SWWAF_STATE_DIR, as the "Persistent state" section of SPEC.md describes:
-// and docs/state-files.md lists them:
 // bans.json holds the bans, clients.json each client's counters and
 // history, lookups.json GeoJS's answers, reputation.json the last try and
 // last good copy of each list fetched from a URL, the DNSBL zones'

internal/state/state_test.go

@@ -1019,8 +1019,8 @@ func TestEditJustBeforeAScheduledWriteSurvivesIt(t *testing.T) {
 		edit(t, dir, bansJSON, permanentBansJSON)
 
 		// The write takes the edit in first, and writes it back. The second
-		// ban, made after the admin opened the file, is lost, as the
+		// ban, made after the admin opened the file, is lost, as "Edits
-		// "Persistent state" section of SPEC.md says.
+		// while running" in SPEC.md says.
 		time.Sleep(time.Nanosecond)
 		synctest.Wait()
 

internal/waf/waf.go

@@ -1,8 +1,8 @@
 // Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
 // method, the URL with its query and the headers of a request, and on its
 // body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
-// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
+// makes to it, as "Attack detection" under "Configuration surface" in
-// them. It reads no response.
+// SPEC.md describes them. It reads no response.
 //
 // smallwebwaf writes only to its state directory, so Coraza is built with
 // its no_fs_access tag, as the Dockerfile and script/build build it: of a

script/gitea-requests

@@ -2,12 +2,11 @@
 # script/gitea-requests: build the image and run gitea 1.27.3 behind it,
 # with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
 # which names gitea's container. Then send through it the gitea requests
-# that reading gitea's source found the Core Rule Set refuses, which
+# that reading gitea's source found the Core Rule Set refuses, and a short
-# docs/gitea.md records, and a short sample of gitea's ordinary use:
+# sample of gitea's ordinary use: signing in, pages of files and commits,
-# signing in, pages of files and commits, a diff, the API, a push, clone
+# a diff, the API, a push, clone and fetch over HTTP, and two container
-# and fetch over HTTP, and two container image pushes. Last, print each
+# image pushes. Last, print each request smallwebwaf logged, with what it
-# request smallwebwaf logged, with what it did with it and the ids of the
+# did with it and the ids of the Core Rule Set's rules that matched it.
-# Core Rule Set's rules that matched it.
 # Every request comes from one client, docker's gateway, so more than
 # SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
 # the requests after would show as banned. The containers, the network,