A client that stops sending with little of its body left after the app has answered is timed out (closes #136)

When less than 256 KiB of a client's body is left, Go's server reads the
rest before it sends the app's answer, and the transport reads it as it
closes the body after a failed write to the app. Neither read goes
through requestBody.Read, so when SWWAF_CLIENT_REQUEST_TIMEOUT ran out
smallwebwaf took the 504 branch, which set no read deadline, and the
request held its connection for as long as the client kept it open.
That branch now ends reading the client's body at the client's
deadline too.

Model: opus-5-5

Makefile

@@ -1,12 +1,11 @@
 .PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
-	run example-app gitea-requests
+	run example-app
 
 # Makefile targets are thin shims; the implementations live in script/
 # per the scripts-to-rule-them-all pattern (see the Entrypoints section
 # of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
 # which test checks. build and run are for working on the code by hand;
-# example-app checks the image with an app built on it; gitea-requests
+# example-app checks the image with an app built on it.
-# reports the gitea requests the Core Rule Set refuses at the defaults.
 
 bootstrap:
 	@script/bootstrap
@@ -46,6 +45,3 @@ run:
 
 example-app:
 	@script/example-app
-
-gitea-requests:
-	@script/gitea-requests

README.md

@@ -96,14 +96,11 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
 ## What it does so far
 
 - Passes each request to the app and the app's answer back unchanged: method,
-  path, query, headers, body and status. The one exception is a path starting
+  path, query, headers, body and status. Bodies stream through in both
-  with `//` that holds a character Go escapes, such as `|`: Go cannot send it as
+  directions and are never held whole in memory, but for the part of a request
-  it is, and the app receives it decoded and escaped again, `%2F` as `/`. Bodies
+  body the Core Rule Set reads, at most `SWWAF_WAF_BODY_LIMIT` and one byte
-  stream through in both directions and are never held whole in memory, but for
+  more, which is held until the app is sent it. A WebSocket, or any other
-  the part of a request body the Core Rule Set reads, at most
+  upgraded connection, passes through, and the timeouts do not cut it.
-  `SWWAF_WAF_BODY_LIMIT` and one byte more, which is held until the app is sent
-  it. A WebSocket, or any other upgraded connection, passes through, and the
-  timeouts do not cut it.
 - Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is
   the client, and the forwarded headers it sends are replaced, not passed on.
   For a peer inside it, `X-Forwarded-For` is read from the right, and the first
@@ -113,16 +110,16 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
   `X-Forwarded-Proto`, and `X-Forwarded-For` with the peer added at the end. It
   also gets the request's id in `X-Request-ID`, the same id as in the request's
   log line (see `request_id` in "Request log" below).
-- Enforces the timeouts and the size limits below. Once a request's line and
+- Enforces the timeouts and the size limits below. A limit passed before the
-  headers have been read, a limit passed before the response has started gets
+  response has started gets `smallwebwaf`'s own answer: `408` for a client too
-  `smallwebwaf`'s own answer: `408` for a client too slow to send its body,
+  slow to send its request, `413` for a request body that is too large, `504`
-  `413` for a request body that is too large, `504` for an app too slow to
+  for an app too slow to answer, and `502` for a response that is too large or
-  answer, and `502` for a response that is too large or an app that cannot be
+  an app that cannot be reached. A request that announces a body over the limit
-  reached. A request that announces a body over the limit is refused before
+  is refused before anything reaches the app. While a request body is still on
-  anything reaches the app. While a request body is still on its way, a request
+  its way, a request timeout that runs out answers `408` if `smallwebwaf` was
-  timeout that runs out answers `408` if `smallwebwaf` was waiting for the
+  waiting for the client to send more, and `504` if it was waiting for the app
-  client to send more, and `504` if it was waiting for the app to take what it
+  to take what it had. Once the response has started, a limit can only cut the
-  had. Once the response has started, a limit can only cut the connection.
+  connection.
 - Counts each client's requests over a minute, an hour and a day. A request that
   takes the client over one of the rate limits below is refused with
   `SWWAF_BAN_RESPONSE`, `403` by default, before anything reaches the app, and
@@ -1045,11 +1042,9 @@ less severe than `SWWAF_LOG_LEVEL` are not written.
 Go's HTTP server, on which `smallwebwaf` is built, reads a request's line and
 headers before `smallwebwaf` sees the request, and some requests end there,
 without a line in the log: headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
-which it answers `431`, and requests it cannot read at all, which it answers
+which it answers `431`, headers slower than `SWWAF_CLIENT_REQUEST_TIMEOUT`,
-itself, mostly with `400`. A client too slow to send its request line and
+whose connection it closes without an answer, and requests it cannot read at
-headers within `SWWAF_CLIENT_REQUEST_TIMEOUT` gets either no answer or an error
+all, which it answers itself, mostly with `400`.
-answer from the server, mostly `400 Bad Request`; either way the server closes
-its connection, nothing reaches the app, and no line is written in the log.
 
 ### Sending the log to a syslog server
 
@@ -1449,12 +1444,11 @@ whole service, its `asn`, for an AS number, its `name`, for a named netblock, or
 the `start` of a window in which it has requests or bytes; a list's `url`,
 `fetched` or `lines`, which is `[]` for an empty list; a verdict's `zone`,
 `client`, `listed`, which is `false` for a client the zone does not list, or
-`fetched`. So does a ban whose `expires` is not after its `start` or whose
+`fetched`. So does a ban whose `cause` is not `limit`, `attack`, `admin` or
-`cause` is not `limit`, `attack`, `admin` or `crowdsec`, alerts waiting for a
+`crowdsec`, alerts waiting for a destination that is not `webhook`, `slack` or
-destination that is not `webhook`, `slack` or `ntfy`, an anomaly counter whose
+`ntfy`, an anomaly counter whose `scope` is not `client`, `net`, `asn`, `total`
-`scope` is not `client`, `net`, `asn`, `total` or `watch`, and a copy of a list
+or `watch`, and a copy of a list with a line that would make its fetch fail. An
-with a line that would make its fetch fail. An answer's `asn` or `as_name` left
+answer's `asn` or `as_name` left out reads as empty.
-out reads as empty.
 
 While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
 a state file as soon as you save it: what the file then holds replaces what
@@ -1464,15 +1458,14 @@ writes a file it takes in any edit made since, so your edit is not overwritten;
 a change `smallwebwaf` made after you opened the file, such as a new ban, is
 lost when you save over it. An edit that would stop the start, because it does
 not parse, has another `version`, leaves out a field an entry needs, gives a ban
-an `expires` not after its `start` or another `cause`, names another
+another `cause`, names another destination, gives an anomaly counter another
-destination, gives an anomaly counter another `scope` or gives a list's copy a
+`scope` or gives a list's copy a line that would make its fetch fail, does not
-line that would make its fetch fail, does not stop the running `smallwebwaf`: it
+stop the running `smallwebwaf`: it keeps what it holds, and at the file's next
-keeps what it holds, and at the file's next write renames your file to
+write renames your file to `<name>.bad`, such as `bans.json.bad`, writes the
-`<name>.bad`, such as `bans.json.bad`, writes the file again from memory, logs
+file again from memory, logs the file and where the error is, and raises a
-the file and where the error is, and raises a `file_error` alert for it. It
+`file_error` alert for it. It waits for that write because an editor's file can
-waits for that write because an editor's file can be read before the editor has
+be read before the editor has finished writing it. Mend the `.bad` file and move
-finished writing it. Mend the `.bad` file and move it back. A file you remove is
+it back. A file you remove is written again at its next write.
-written again at its next write.
 
 To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
 and its `expires`, `null` for a ban that never ends; its `reason` and its
@@ -1961,14 +1954,8 @@ main "$@"
   without one, it still starts. At each start the `run` script of `smallwebwaf`
   gives that directory and every file in it to the `smallwebwaf` user, so a host
   directory mounted there needs no change of owner.
-- `docker stop` has runit stop both processes at once, each with SIGTERM.
+- `docker stop` has runit stop both processes. `smallwebwaf` then stops taking
-  `smallwebwaf` then stops taking requests and gives those in progress five
+  requests and gives those in progress five seconds to finish.
-  seconds to finish, which they can do only while the app still answers. An app
-  that exits at once on SIGTERM ends them: the client gets `502`, or a response
-  cut short if the app had already begun to answer. An app that should let them
-  finish keeps taking and answering requests, not only the ones it already has,
-  for the whole five seconds after SIGTERM, since `smallwebwaf` can still pass
-  it requests it took before then.
 
 A rule file is one rule per line: a name, what to match against, what to do, and
 a regex.
@@ -2160,14 +2147,13 @@ addresses sends, so that one client costs at most one check every
 `SWWAF_REPUTATION_CACHE_TTL`.
 
 Only a client whose history counts an offence is checked, so that the checks are
-spent on suspects: so far, one that has broken a rate limit, a byte limit or the
+spent on suspects: so far, one that has broken a rate limit or a byte limit,
-error burst, matched a ban rule, asked for a trap path, or had a request refused
+matched a ban rule, or had a request refused by a block rule or the Core Rule
-by a block rule, by the Core Rule Set or for a missing or wrong token. A client
+Set. A client dropped from the table of clients loses its history, and with it
-dropped from the table of clients loses its history, and with it its offences. A
+its offences. A client is checked in the background, at its first request after
-client is checked in the background, at its first request after its offence that
+its offence that reaches the check: no request waits, a request refused under
-reaches the check: no request waits, a request refused under its ban is not
+its ban is not checked, and the request that has it checked, and any other from
-checked, and the request that has it checked, and any other from it before the
+it before the answer comes, goes on as from a client without a score.
-answer comes, goes on as from a client without a score.
 
 A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and
 `SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL
@@ -2384,34 +2370,10 @@ so that they run in minimal containers.
   that the health check passes, that a request reaches the app through
   `smallwebwaf`, that a second request in a minute bans the client, that a probe
   for `/.env` bans another client, whose next request makes the ban permanent,
-  that `sv stop` and `docker stop` stop it in order, that a new container on the
+  that `sv stop` and `docker stop` stop it in order, and that a new container on
-  same volume still refuses the banned client, that a container with its state
+  the same volume still refuses the banned client; then removes the containers,
-  files on a new volume mounted at the directory `SWWAF_STATE_DIR_FILE` names
+  the volume and both images. It needs network access, for nixpkgs' binary
-  becomes healthy, and that one whose `SWWAF_STATE_DIR` does not exist logs
+  cache, and `script/check` does not run it; `make example-app` does.
-  `smallwebwaf`'s message about it; then removes the containers, the volumes and
-  both images. It needs network access, for nixpkgs' binary cache, and
-  `script/check` does not run it; `make example-app` does.
-- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it,
-  with every setting of `smallwebwaf` at its default but `SWWAF_UPSTREAM_URL`,
-  which names gitea's container. Of the requests that three of the notes
-  specific to gitea in [`SPEC.md`](SPEC.md) say are refused, it sends through
-  it: for each file the note on a file name in a query parameter names, the
-  API's list of its commits; for `Runtime.java` alone, the commit graph of that
-  file, the hidden lines of its diff and the API's call for the contents of
-  files that names it; the sign-in page gitea sends a visitor to on the way to
-  the edit page of `document.write.js`; the pages of `java.lang.String.md`,
-  `META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
-  and `.../notes.txt`; and one container image push that reuses a layer. It does
-  not send the raw file, history, blame or edit page of those four files, the
-  pages of the directories `META-INF/services` and `...`, or the push again. It
-  also sends a short sample of gitea's ordinary use: signing in, pages of files
-  and commits, a diff, the API, a push, clone and fetch over HTTP, and two
-  container image pushes. It prints a line for each request `smallwebwaf`
-  logged, with what it did with it and the ids of the Core Rule Set's rules that
-  matched it, then removes the containers, the network and the images it made,
-  but gitea's own image. It needs network access, and a docker daemon that
-  pushes to a registry on `localhost` over plain HTTP, as docker does by
-  default; `script/check` does not run it, and `make gitea-requests` does.
 
 ## TODO
 

SPEC.md

@@ -225,9 +225,9 @@ address or one /64, so a permanent ban does not reach neighbours who did
 nothing.
 
 An offence is a request `smallwebwaf` holds against the client: one that carries
-a clear sign of attack, breaks a limit, or is refused by a rule file, by the
+a clear sign of attack, breaks a limit, or is refused by a rule file or the Core
-Core Rule Set or for a missing or wrong token. Offences are counted by kind in
+Rule Set. Offences are counted by kind in the client's history. Two kinds lead
-the client's history. Two kinds lead to a ban, each by its own rule:
+to a ban, each by its own rule:
 
 - A clear sign of attack: a request that matches a rule file rule whose action
   is `ban`, or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a
@@ -414,10 +414,10 @@ The settings, by group:
       to send its request line and headers, and then, from the end of the
       headers, its body.
     - `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest
-      request line and headers a client may send. Over it, Go's HTTP server
+      request line and headers a client may send. Over it, `smallwebwaf` answers
-      answers `431` and closes the connection, and nothing reaches the app. It
+      `431` and closes the connection, and nothing reaches the app. It must be
-      must be more than `4K`, and cannot be `off`: Go's HTTP server always has
+      more than `4K`, and cannot be `off`: Go's HTTP server always has such a
-      such a limit, and reads 4 KiB past the one it is given before it refuses.
+      limit, and reads 4 KiB past the one it is given before it refuses.
     - `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open
       connection may wait for its next request before `smallwebwaf` closes it.
       It is longer than the 90 seconds after which traefik, by default, closes a
@@ -435,13 +435,12 @@ The settings, by group:
       the client sends it and the app receives it.
     - `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body, as
       the app sends it and the client receives it.
-    - When a limit is passed after a request's line and headers have been read,
+    - When a limit is passed before the response has started, `smallwebwaf`
-      but before the response has started, `smallwebwaf` answers itself: `413`
+      answers itself: `413` for a request body that is too large, `408` for a
-      for a request body that is too large, `408` for a client too slow to send
+      client that is too slow, `502` for a response that is too large, `504` for
-      its body, `502` for a response that is too large, `504` for an app that is
+      an app that is too slow. A request that announces a body larger than its
-      too slow. A request that announces a body larger than its limit is refused
+      limit is refused before anything reaches the app. Once the response has
-      before anything reaches the app. Once the response has started it can only
+      started it can only be cut off, and the connection is closed.
-      be cut off, and the connection is closed.
     - Since a request body streams through, each side can hold up the other: a
       slow client slows the send to the app, and an app slow to take the body
       slows the client's send. So while a request body is still on its way, a
@@ -451,11 +450,10 @@ The settings, by group:
       waiting for the app to take what it had.
     - Go's HTTP server, on which `smallwebwaf` is built, reads a request's line
       and headers before `smallwebwaf` sees the request. A client that takes
-      longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to send them gets either no
+      longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to send them gets no answer:
-      answer or an error answer from the server, mostly `400 Bad Request`;
+      the server closes its connection. Headers over
-      either way the server closes its connection and nothing reaches the app.
+      `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by the server
-      Headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by
+      itself. Neither request gets a line in the request log.
-      the server itself. Neither request gets a line in the request log.
     - A WebSocket connection leaves these limits behind once it is upgraded: it
       stays open until either side closes it.
 - Lookup of AS number and country (R7). On by default through GeoJS, which needs
@@ -1326,16 +1324,9 @@ The two processes:
 - A `smallwebwaf` that stops at start, for example on invalid configuration, is
   started again every second and stops again, logging its message each time,
   until the cause is fixed. The health check fails meanwhile.
-- `docker stop` sends SIGTERM to `runsvinit`, which has runit stop both services
+- `docker stop` sends SIGTERM to `runsvinit`, which has runit stop both
-  at once, each with SIGTERM, and exits once they have stopped. `smallwebwaf`
+  services, each with SIGTERM, and exits once they have stopped. `smallwebwaf`
-  stops taking requests, gives those in progress five seconds to finish, and
+  writes its state files as it stops (see "Persistent state").
-  writes its state files as it stops (see "Persistent state"). The requests in
-  progress can finish only while the app still answers. An app that exits at
-  once on SIGTERM ends them: the client gets `502`, or a response cut short if
-  the app had already begun to answer. An app that should let them finish keeps
-  taking and answering requests, not only the ones it already has, for the whole
-  five seconds after SIGTERM, since `smallwebwaf` can still pass it requests it
-  took before then.
 - The container's root filesystem stays writable: runit writes each service's
   status into its directory under `/etc/service`.
 
@@ -1486,47 +1477,8 @@ holds any token file.
           Saving such a rule stores it, but the branch settings page gitea then
           returns to gets the 403 answer of `smallwebwaf`, so the save looks as
           if it failed.
-        - A file name in a query parameter, those the fourth change leaves out
-          of 930120, 932160 and 932260 included, in which the Core Rule Set's
-          rules for Java, script, PowerShell and SQL find what they look for:
-          `runtime` in OpenJDK's
-          `src/java.base/share/classes/java/lang/Runtime.java` (944110), a Java
-          class, as in `java.lang.String.md` (944130), `document.write.js`
-          (941180), `process.env.d.ts` (934100), `__proto__` (934130), the
-          PowerShell command `Install-Script`, as in `install-script.sh`
-          (932120), or a database's name, as in PostgreSQL's
-          `information_schema.sql` (942140). For such a file, the API's list of
-          its commits (`path`) is refused, and so are the API's call for the
-          contents of files that names it in its query (`body`), the commit
-          graph of that file alone (`file`), and the hidden lines of its diff
-          (`path`), which stay hidden. A visitor whom gitea sends to sign in on
-          the way to a page of such a file that needs signing in, such as its
-          edit page, gets the 403 answer of `smallwebwaf` in place of the
-          sign-in page (`redirect_to`), and can still sign in by opening the
-          sign-in page itself.
         - A path: a file name ending in `~`, or an `.xhtml` file whose path
           holds a space.
-        - A path that holds what the Core Rule Set takes for a Java class
-          (944130), as `java.lang.String.md` and
-          `META-INF/services/org.apache.commons.logging.LogFactory` do, or for
-          Node.js code (934100), as `process.env.d.ts` does, or that has a
-          directory named `...` (930100, 930110), such as `.../notes.txt`. The
-          page of such a file, its raw file, its history, its blame and its edit
-          page are refused; the directory that holds it lists it, and a clone
-          carries it.
-        - A container image push that reuses a layer of an image docker has
-          pushed to that gitea before, when that image's owner, or its name
-          after the owner, starts with an entry on the list of commands
-          (932260), such as `owner/python3-app`, `owner/docker-base` or any
-          image of an owner named `ansible` or `docker-team`. docker asks gitea
-          to take the layer from that image, naming it in the query parameter
-          `from`, which 932260 checks, and gets the 403 answer of `smallwebwaf`.
-          A docker that keeps its images in containerd's image store, as a new
-          install of Docker Engine 29 does, then fails the push, and fails each
-          new try the same way. A docker that keeps them in its own older image
-          store uploads the layer instead, so the push succeeds after one
-          refusal for each layer it reuses, and each of those refusals counts
-          toward the error burst.
     - Artifact uploads from `actions/upload-artifact@v3` send the header
       `Content-Range`, which the Core Rule Set refuses (920450). The action
       sends two files at a time, does not retry a 403 and sends no further file
@@ -1614,19 +1566,10 @@ holds any token file.
 - Core Rule Set false positives against real apps (a search for code, and any
   body once `SWWAF_WAF_BODY_LIMIT` is set): a match refuses only that request
   and bans no one by itself; the request log names the rule, and exclusions by
-  rule id and path fix it. `script/gitea-requests` runs gitea behind the image
+  rule id and path fix it. Further gitea requests the Core Rule Set may refuse
-  at the defaults and sends through it a short sample of gitea's ordinary use
+  at the defaults, found by reading gitea's source rather than a running gitea,
-  and some of the requests that three of the notes specific to gitea say are
+  are collected in https://git.eeqj.de/sneak/smallwebwaf/issues/30 and checked
-  refused: for each file the note on a file name in a query parameter names, the
+  when milestone 1 runs in front of a real gitea.
-  API's list of its commits; for `Runtime.java` alone, the commit graph of that
-  file, the hidden lines of its diff and the API's call for the contents of
-  files that names it; the sign-in page gitea sends a visitor to on the way to
-  the edit page of `document.write.js`; the pages of `java.lang.String.md`,
-  `META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
-  and `.../notes.txt`; and one container image push that reuses a layer, which
-  it does not try again. It reports each request the Core Rule Set refuses, so
-  that those requests can be checked again when gitea or the Core Rule Set
-  changes version.
 - Attacks carried in request bodies: not refused by default, since on a code
   forge bodies are full of code the Core Rule Set takes for attacks. Most of
   what shows in URLs and headers is still refused, the client that sends an

internal/bans/admin_test.go

@@ -61,59 +61,6 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
 	wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1})
 }
 
-func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
-	t.Parallel()
-
-	rules := defaultRules()
-	rules.MaxBans = 2
-	ledger := bans.New(rules)
-	netblock := netip.MustParsePrefix("203.0.113.1/32")
-	other := netip.MustParsePrefix("203.0.113.2/32")
-
-	// An admin bans the netblock for a minute, and it then breaks a limit
-	// twice, each time after its last ban has ended.
-	ledger.BanForAdmin(netblock, midnight(), midnight().Add(time.Minute),
-		"probes for logins")
-	first, _ := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
-	second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
-
-	// A ban on another netblock fills the ledger, and the netblock's first
-	// ban for a limit, its earliest that is not an admin's, goes.
-	ledger.BanForLimit(other, second.Start, bans.Notes{})
-	wantBans(t, ledger, map[netip.Prefix]int{netblock: 2, other: 1})
-
-	// The netblock's next ban still counts it.
-	third, _ := ledger.BanForLimit(netblock, second.Expires, bans.Notes{})
-	if third.Notes.EarlierBans != (bans.EarlierBans{Limit: 2, Admin: 1}) {
-		t.Errorf("the third ban for a limit has earlier bans %+v, "+
-			"want 2 for a limit and 1 an admin's", third.Notes.EarlierBans)
-	}
-}
-
-func TestEarlierBansCountABanAnAdminKeepsOnce(t *testing.T) {
-	t.Parallel()
-
-	ledger := bans.New(defaultRules())
-	netblock := netip.MustParsePrefix("203.0.113.1/32")
-
-	// The netblock breaks a limit twice, the second time after its first
-	// ban has ended, and the second ban's notes count the first. An admin
-	// then keeps the first ban.
-	first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
-	second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
-
-	kept := first
-	kept.Cause = bans.CauseAdmin
-	ledger.LoadEdit([]bans.Ban{kept, second})
-
-	// The netblock's next ban counts the kept ban once, as an admin's.
-	third, _ := ledger.BanForLimit(netblock, second.Expires, bans.Notes{})
-	if third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Admin: 1}) {
-		t.Errorf("the third ban for a limit has earlier bans %+v, "+
-			"want 1 for a limit and 1 an admin's", third.Notes.EarlierBans)
-	}
-}
-
 func TestReasonOfTheBansSmallwebwafMakes(t *testing.T) {
 	t.Parallel()
 

internal/bans/bans.go

@@ -687,51 +687,26 @@ func (l *Ledger) ban(
 }
 
 // earlierBans returns how many bans a netblock with the bans held, oldest
-// first, has had, by cause. Each ban held gives a count: the bans its
+// first, has had, by cause: the first ban held counts the bans the
-// notes count before it, and one for it and each ban held after it. A
+// netblock had before that one, since dropped to make room, and each ban
-// ban's count misses a ban dropped after it to make room, and that of an
+// held adds one.
-// admin's ban added to bans.json without notes misses the bans before it,
-// so the count that names the most bans is taken, whole: a ban an admin
-// keeps by giving it CauseAdmin is still counted under its old cause in
-// later bans' notes, and taking each cause from another count would count
-// it twice.
 func earlierBans(held []Ban) EarlierBans {
-	// fromHere counts ban and the bans held after it.
+	earlier := held[0].Notes.EarlierBans
-	var most, fromHere EarlierBans
 
-	for _, ban := range slices.Backward(held) {
+	for _, ban := range held {
 		switch ban.Cause {
 		case CauseLimit:
-			fromHere.Limit++
+			earlier.Limit++
 		case CauseAttack:
-			fromHere.Attack++
+			earlier.Attack++
 		case CauseAdmin:
-			fromHere.Admin++
+			earlier.Admin++
 		case CauseCrowdSec:
-			fromHere.CrowdSec++
+			earlier.CrowdSec++
-		}
-
-		notes := ban.Notes.EarlierBans
-		count := EarlierBans{
-			Limit:    notes.Limit + fromHere.Limit,
-			Attack:   notes.Attack + fromHere.Attack,
-			Admin:    notes.Admin + fromHere.Admin,
-			CrowdSec: notes.CrowdSec + fromHere.CrowdSec,
-		}
-
-		// Of counts that name as many bans, the earlier ban's is taken: it
-		// counts more of the bans held by the cause they have now.
-		if count.total() >= most.total() {
-			most = count
 		}
 	}
 
-	return most
+	return earlier
-}
-
-// total returns how many bans e counts, of all causes.
-func (e EarlierBans) total() int {
-	return e.Limit + e.Attack + e.Admin + e.CrowdSec
 }
 
 // markChanged has Changed receive a value, unless one is waiting already.

internal/proxy/bodies.go

@@ -1,10 +1,8 @@
 package proxy
 
 import (
-	"bufio"
 	"errors"
 	"io"
-	"net"
 	"net/http"
 	"sync/atomic"
 
@@ -213,20 +211,9 @@ func (w *responseWriter) FlushError() error {
 	return err
 }
 
-// Hijack takes over the connection to the client, which ReverseProxy does
-// only to pass on the app's 101 Switching Protocols: that is then the
-// status sent.
-func (w *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	conn, buffered, err := http.NewResponseController(w.ResponseWriter).Hijack()
-	if err == nil {
-		w.status = http.StatusSwitchingProtocols
-	}
-
-	return conn, buffered, err
-}
-
 // Unwrap lets http.ResponseController reach net/http's own
-// ResponseWriter for what responseWriter does not do itself.
+// ResponseWriter, which is how ReverseProxy takes over the connection of
+// an upgraded request.
 func (w *responseWriter) Unwrap() http.ResponseWriter {
 	return w.ResponseWriter
 }

internal/proxy/coreruleset.go

@@ -23,7 +23,7 @@ import (
 // limit while the Core Rule Set reads it, which it notes nothing of.
 func (rq *request) checkCoreRuleSet() string {
 	cfg := rq.h.config
-	if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in, cfg.WAFExemptPaths) {
+	if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in.URL, cfg.WAFExemptPaths) {
 		return ""
 	}
 

internal/proxy/passthrough_test.go

@@ -74,47 +74,6 @@ func TestPassesRequestAndAnswerUnchanged(t *testing.T) {
 	wantRequestFields(t, line, addr, len(requestBody), len(answerBody))
 }
 
-func TestPassesThePathAsSent(t *testing.T) {
-	t.Parallel()
-
-	for _, path := range []string{
-		// An encoded slash and a character Go escapes: |, ^, and é sent
-		// as its UTF-8 bytes.
-		"/a%2Fb|c", "/feat%2Fx/file^1", "/x%2Fy/café",
-		// A path starting with //, which Go would send as a scheme and
-		// host if it were put in URL.Opaque.
-		"//a%2Fb",
-	} {
-		t.Run(path, func(t *testing.T) {
-			t.Parallel()
-
-			saw := make(chan string, 1)
-			app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
-				saw <- r.RequestURI
-			})
-			addr, out := startProxy(t, app.URL, nil)
-
-			conn := dial(t, addr)
-			send(t, conn, "GET "+path+"?q=1 HTTP/1.1\r\nHost: "+appHost+"\r\n\r\n")
-
-			got := readResponse(t, conn)
-			if got.status != http.StatusOK {
-				t.Fatalf("status %d, want %d", got.status, http.StatusOK)
-			}
-
-			target := <-saw
-			if target != path+"?q=1" {
-				t.Errorf("app saw %s, want %s", target, path+"?q=1")
-			}
-
-			line := out.requestLine(t)
-			if line.Path != path {
-				t.Errorf("log line has path %s, want %s", line.Path, path)
-			}
-		})
-	}
-}
-
 // wantAppSaw checks that the app received the test's request unchanged.
 func wantAppSaw(t *testing.T, saw appSaw, body []byte) {
 	t.Helper()
@@ -192,36 +151,6 @@ func wantRequestFields(t *testing.T, line logLine, host string, sent, received i
 	}
 }
 
-func TestAddsNoContentTypeTheAppDidNotSend(t *testing.T) {
-	t.Parallel()
-
-	const typed = "text/plain"
-
-	app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
-		if r.URL.Path == "/typed" {
-			w.Header().Set("Content-Type", typed)
-		} else {
-			// The key with no value keeps Go's server from adding one.
-			w.Header()["Content-Type"] = nil
-		}
-
-		_, _ = io.WriteString(w, "<!doctype html><title>page</title>")
-	})
-	addr, _ := startProxy(t, app.URL, nil)
-
-	got := get(t, addr, "/")
-	if got.header.Values("Content-Type") != nil {
-		t.Errorf("client got Content-Type %q, want none",
-			got.header.Get("Content-Type"))
-	}
-
-	got = get(t, addr, "/typed")
-	if got.header.Get("Content-Type") != typed {
-		t.Errorf("client got Content-Type %q, want %q",
-			got.header.Get("Content-Type"), typed)
-	}
-}
-
 func TestStreamsTheRequestBody(t *testing.T) {
 	t.Parallel()
 
@@ -380,23 +309,6 @@ func echoAfterUpgrade(w http.ResponseWriter, r *http.Request) {
 	}
 }
 
-func TestAnswers502WhenTheAppSwitchesToAnotherProtocol(t *testing.T) {
-	t.Parallel()
-
-	// answerAfterUpgrade switches to websocket whatever the client asked
-	// for, and ReverseProxy passes on a switch only to the protocol the
-	// client asked for, here "websocket, foo".
-	app := startApp(t, answerAfterUpgrade)
-	addr, out := startProxy(t, app.URL, nil)
-
-	conn := dial(t, addr)
-	send(t, conn, "GET /ws HTTP/1.1\r\nHost: app\r\n"+
-		"Connection: Upgrade\r\nUpgrade: websocket, foo\r\n\r\n")
-
-	wantStatus(t, readResponse(t, conn), http.StatusBadGateway)
-	wantLine(t, out.requestLine(t), http.StatusBadGateway, requestlog.ActionUpstreamError)
-}
-
 func TestServerHasTheDefaultLimits(t *testing.T) {
 	t.Parallel()
 

internal/proxy/ratelimits_test.go

@@ -185,9 +185,6 @@ func TestRateLimitCountsPathsThatAreNotExempt(t *testing.T) {
 		"/assets/x%2Fy",
 		"/assets/x%2fy",
 		`/assets/x\y`,
-		// The same with a character Go escapes, |, for which Go's
-		// URL.EscapedPath gives /assets/x/y%7Cz, with no encoded slash.
-		"/assets/x%2Fy|z",
 	} {
 		t.Run(sent, func(t *testing.T) {
 			t.Parallel()

internal/proxy/request.go

@@ -8,6 +8,7 @@ import (
 	"net/http/httptrace"
 	"net/http/httputil"
 	"net/netip"
+	"net/url"
 	"os"
 	"slices"
 	"strings"
@@ -21,7 +22,6 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/lookup"
 	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
 	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-	"sneak.berlin/go/smallwebwaf/internal/rules"
 )
 
 // flushAfterEachWrite has ReverseProxy pass on each part of the app's
@@ -137,7 +137,7 @@ func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
 			Method:           r.Method,
 			Scheme:           scheme(r, peerTrusted),
 			Host:             r.Host,
-			Path:             rules.Path(r),
+			Path:             r.URL.EscapedPath(),
 			Query:            r.URL.RawQuery,
 			Protocol:         r.Proto,
 			Referer:          r.Referer(),
@@ -284,7 +284,7 @@ func (rq *request) checkClient(ctx context.Context) string {
 	}
 
 	rq.counted = !isInside(rq.client, cfg.RateLimitExemptNets) &&
-		!pathExempt(rq.in, cfg.RateLimitExemptPaths)
+		!pathExempt(rq.in.URL, cfg.RateLimitExemptPaths)
 	if rq.counted {
 		rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
 		rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
@@ -307,7 +307,7 @@ func (rq *request) checkClient(ctx context.Context) string {
 	return rq.checkCoreRuleSet()
 }
 
-// pathExempt reports whether r is exempt under prefixes,
+// pathExempt reports whether a request for u is exempt under prefixes,
 // SWWAF_RATE_LIMIT_EXEMPT_PATHS or SWWAF_WAF_EXEMPT_PATHS: whether its
 // path as sent, the path the app receives, not percent-decoded, starts
 // with one of prefixes, so that /%61ssets/x is not under /assets/ for an
@@ -316,9 +316,10 @@ func (rq *request) checkClient(ctx context.Context) string {
 // encoded slash (%2F or %2f), never is, since an app may act on it as a
 // path outside every prefix: /assets/..%2Flogin as /login, or /assets%2Fx
 // as one path segment, as Go's router does.
-func pathExempt(r *http.Request, prefixes []string) bool {
+func pathExempt(u *url.URL, prefixes []string) bool {
-	decoded := r.URL.Path
+	decoded := u.Path
-	sent := rules.Path(r)
+	// EscapedPath is the path as the app receives it, not decoded.
+	sent := u.EscapedPath()
 
 	if strings.Contains(decoded, "..") || strings.Contains(decoded, `\`) ||
 		strings.Contains(strings.ToLower(sent), "%2f") {
@@ -374,17 +375,6 @@ func (rq *request) rewrite(pr *httputil.ProxyRequest) {
 	// ReverseProxy drops query parameters it cannot parse; the app gets
 	// the query as the client sent it.
 	pr.Out.URL.RawQuery = pr.In.URL.RawQuery
-	// ReverseProxy sends the path URL.EscapedPath gives, which, for a path
-	// holding a character Go escapes, such as |, is the whole path decoded
-	// and escaped again, %2F turned into /. Opaque is sent as it is, so the
-	// app gets the path as the client sent it. Left out are a target that
-	// is no path, such as *, and a path starting with //, which Go would
-	// send in Opaque as a scheme and host.
-	path := rules.Path(pr.In)
-	if strings.HasPrefix(path, "/") && !strings.HasPrefix(path, "//") {
-		pr.Out.URL.Opaque = path
-	}
-
 	setForwardedHeaders(pr.In, pr.Out, rq.peer, rq.peerTrusted)
 	pr.Out.Header.Set(requestIDHeader, rq.line.RequestID)
 	pr.Out.Header.Del(asnHeader)
@@ -405,10 +395,8 @@ func (rq *request) modifyResponse(res *http.Response) error {
 		// timeouts. ReverseProxy writes this answer straight to the
 		// connection it takes over, not through rq.out, and then copies
 		// what passes each way through res.Body, the connection to the app.
-		// It may still refuse the switch, as when the app's Upgrade is not
-		// the client's, so the status is noted only once it has taken over
-		// the connection, by rq.out's Hijack.
 		rq.stopTimers()
+		rq.out.status = res.StatusCode
 		rq.line.Websocket = true
 
 		conn, ok := res.Body.(io.ReadWriteCloser)
@@ -431,13 +419,6 @@ func (rq *request) modifyResponse(res *http.Response) error {
 		return errResponseTooLarge
 	}
 
-	// Go's server adds a Content-Type it guesses from the body to a
-	// response whose headers have no Content-Type key. The key with no
-	// value sends none, as the app sent none.
-	if res.Header.Get("Content-Type") == "" {
-		rq.out.Header()["Content-Type"] = nil
-	}
-
 	res.Body = &responseBody{body: limitBody(res.Body, maxBytes), rq: rq}
 	rq.startClientResponseTimeout()
 
@@ -447,13 +428,6 @@ func (rq *request) modifyResponse(res *http.Response) error {
 // answerError is ReverseProxy's ErrorHandler: the request could not be
 // passed to the app, or the app's answer cannot be passed on.
 func (rq *request) answerError(_ http.ResponseWriter, _ *http.Request, err error) {
-	if rq.upgraded != nil {
-		// The app switched protocols, but the switch was not passed on,
-		// and ReverseProxy leaves the app's connection open when it
-		// refuses it.
-		_ = rq.upgraded.Close()
-	}
-
 	refused := rq.refused.Load()
 	if refused == nil {
 		if rq.in.Context().Err() != nil {
@@ -736,21 +710,17 @@ func (rq *request) requestTimedOut(limit string) {
 	}
 
 	if rq.body == nil || !rq.body.waiting.Load() {
-		// Once the app has answered, the transport, ReverseProxy and Go's
-		// server each close the client's body, and closing it reads what
-		// is left of a body under 256 KiB without requestBody.Read, so
-		// waiting is false while that read waits for the client. That
-		// read ends now, before refuse: should smallwebwaf still answer
-		// itself, answer then sets the deadline it gives its own answers.
-		if !rq.answerStarted.IsZero() {
-			rq.stopReadingBody(time.Now())
-		}
-
 		rq.refuse(refusal{
 			status: http.StatusGatewayTimeout,
 			action: requestlog.ActionTimedOut,
 			limit:  limit,
 		})
+		// Go's server, before it sends the app's answer, and the
+		// transport, as it closes the body, read what is left of a body
+		// under 256 KiB without requestBody.Read, so waiting is false
+		// while they wait for the client. Those reads end when the client
+		// is out of time.
+		rq.stopReadingBody(rq.clientRequestDeadline())
 
 		return
 	}

internal/proxy/request_internal_test.go

@@ -1,55 +0,0 @@
-package proxy
-
-import (
-	"io"
-	"net/http"
-	"net/http/httptest"
-	"testing"
-
-	"sneak.berlin/go/smallwebwaf/internal/config"
-)
-
-func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
-	t.Parallel()
-
-	srv := httptest.NewServer(http.HandlerFunc(
-		func(w http.ResponseWriter, _ *http.Request) {
-			rq := &request{
-				h:   &handler{config: &config.Config{}},
-				out: &responseWriter{ResponseWriter: w},
-			}
-			app := &http.Response{
-				StatusCode: http.StatusOK,
-				Header:     http.Header{},
-				Body:       http.NoBody,
-			}
-
-			err := rq.modifyResponse(app)
-			if err != nil {
-				t.Errorf("modifyResponse: %v", err)
-			}
-
-			// The headers go out with this, the first part of the body,
-			// which Go's server would guess a Content-Type from.
-			_, _ = io.WriteString(w, "<!doctype html><title>page</title>")
-		}))
-	t.Cleanup(srv.Close)
-
-	req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, srv.URL,
-		http.NoBody)
-	if err != nil {
-		t.Fatalf("new request: %v", err)
-	}
-
-	res, err := srv.Client().Do(req)
-	if err != nil {
-		t.Fatalf("get: %v", err)
-	}
-
-	_ = res.Body.Close()
-
-	if res.Header.Values("Content-Type") != nil {
-		t.Errorf("client got Content-Type %q, want none",
-			res.Header.Get("Content-Type"))
-	}
-}

internal/proxy/timeouts_test.go

@@ -252,92 +252,54 @@ func TestClientStopsSendingAfterTheAppHasAnswered(t *testing.T) {
 func TestClientStopsSendingWithLittleOfItsBodyLeft(t *testing.T) {
 	t.Parallel()
 
-	for _, tc := range []struct {
+	// appClosed is closed once smallwebwaf has closed its connection to
-		name string
+	// the app, which it does once it has the app's answer.
-		// limit is the setting set to shortTimeout, which runs out, and
+	appClosed := make(chan struct{})
-		// clientTimeout what SWWAF_CLIENT_REQUEST_TIMEOUT is set to when it
-		// is not limit.
-		limit, clientTimeout string
-	}{
-		{
-			name:  "client request timeout",
-			limit: clientRequestTimeout,
-		},
-		{
-			name:          "upstream request timeout, client request timeout off",
-			limit:         upstreamRequestTimeout,
-			clientTimeout: off,
-		},
-		{
-			name:          "upstream request timeout, client request timeout longer",
-			limit:         upstreamRequestTimeout,
-			clientTimeout: longTimeoutSetting,
-		},
-	} {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
 
-			// appClosed is closed once smallwebwaf has closed its
+	// The app answers as soon as it has the headers, without reading the
-			// connection to the app, which it does once it has the app's
+	// body, and says it closes the connection.
-			// answer.
+	app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
-			appClosed := make(chan struct{})
+		conn, buf, err := http.NewResponseController(w).Hijack()
+		if err != nil {
+			return
+		}
 
-			// The app answers without reading the rest of the body, and
+		_, _ = buf.WriteString("HTTP/1.1 404 Not Found\r\n" +
-			// says it closes the connection, only once smallwebwaf waits
+			"Content-Length: 0\r\nConnection: close\r\n\r\n")
-			// for the client to send more: it takes the bytes the client
+		_ = buf.Flush()
-			// sent, and then waits half the timeout, far longer than
+		_, _ = io.Copy(io.Discard, buf.Reader)
-			// smallwebwaf takes to go on to wait.
+		_ = conn.Close()
-			app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
-				conn, buf, err := http.NewResponseController(w).Hijack()
-				if err != nil {
-					return
-				}
 
-				_, _ = io.ReadFull(buf.Reader, make([]byte, len("the first bytes")))
+		close(appClosed)
+	})
+	addr, out := startProxy(t, app.URL, map[string]string{
+		clientRequestTimeout: shortTimeoutSetting,
+		metricsToken:         token,
+	})
 
-				time.Sleep(shortTimeout / 2)
+	// Less than 256 KiB of the body is left after the first bytes, little
+	// enough for Go's server in smallwebwaf to read the rest itself before
+	// it sends the app's answer.
+	start := time.Now()
+	conn := dial(t, addr)
+	send(t, conn, "POST /upload HTTP/1.1\r\nHost: app\r\nContent-Length: 200000\r\n\r\n"+
+		"the first bytes")
 
-				_, _ = buf.WriteString("HTTP/1.1 404 Not Found\r\n" +
+	select {
-					"Content-Length: 0\r\nConnection: close\r\n\r\n")
+	case <-appClosed:
-				_ = buf.Flush()
+	case <-time.After(waitLimit):
-				_, _ = io.Copy(io.Discard, buf.Reader)
+		t.Fatal("smallwebwaf did not close its connection to the app")
-				_ = conn.Close()
-
-				close(appClosed)
-			})
-
-			env := map[string]string{tc.limit: shortTimeoutSetting, metricsToken: token}
-			if tc.clientTimeout != "" {
-				env[clientRequestTimeout] = tc.clientTimeout
-			}
-
-			addr, out := startProxy(t, app.URL, env)
-
-			// Less than 256 KiB of the body is left after the first bytes,
-			// little enough for Go's server in smallwebwaf to read the rest
-			// itself before it sends the app's answer.
-			start := time.Now()
-			conn := dial(t, addr)
-			send(t, conn, "POST /upload HTTP/1.1\r\nHost: app\r\nContent-Length: 200000\r\n\r\n"+
-				"the first bytes")
-
-			select {
-			case <-appClosed:
-			case <-time.After(waitLimit):
-				t.Fatal("smallwebwaf did not close its connection to the app")
-			}
-
-			// More of the body, which can no longer go to the app, and then
-			// nothing.
-			send(t, conn, strings.Repeat("x", 8<<10))
-
-			got := readResponse(t, conn)
-			wantTimedOut(t, start)
-			wantStatus(t, got, http.StatusNotFound)
-			wantLine(t, out.requestLine(t), http.StatusNotFound, requestlog.ActionTimedOut)
-			wantLimitHits(t, addr, tc.limit, 1)
-		})
 	}
+
+	// More of the body, which can no longer go to the app, and then
+	// nothing.
+	send(t, conn, strings.Repeat("x", 8<<10))
+
+	got := readResponse(t, conn)
+	wantTimedOut(t, start)
+	wantStatus(t, got, http.StatusNotFound)
+	wantLine(t, out.requestLine(t), http.StatusNotFound, requestlog.ActionTimedOut)
+	wantLimitHits(t, addr, clientRequestTimeout, 1)
 }
 
 func TestAppTooSlowToAnswer(t *testing.T) {

internal/state/state.go

@@ -61,7 +61,6 @@ var (
 	// errMissing is for an entry without a field it needs.
 	errMissing     = errors.New("has no")
 	errCause       = errors.New("is not limit, attack, admin or crowdsec")
-	errExpires     = errors.New("is not after its start")
 	errDestination = errors.New("is not webhook, slack or ntfy")
 	errScope       = errors.New("is not client, net, asn, total or watch")
 	errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
@@ -647,10 +646,8 @@ func (e BanEntry) ban() bans.Ban {
 // client, a start, from which the length of the netblock's next ban is
 // worked out, or an expires, which would make it permanent. A permanent
 // ban's expires is null, which Bans cannot tell from a missing one, so
-// each expires is read again as written. An expires not after the start
+// each expires is read again as written. A cause other than limit,
-// is refused too, since the netblock's next ban for a broken limit, three
+// attack, admin or crowdsec, most likely misspelt, is refused too.
-// times as long as this one, would then refuse nothing; and so is a cause
-// other than limit, attack, admin or crowdsec, most likely misspelt.
 func (f *bansFile) check(data []byte) error {
 	var written struct {
 		Bans []struct {
@@ -671,8 +668,6 @@ func (f *bansFile) check(data []byte) error {
 			return missing(i, "start")
 		case written.Bans[i].Expires == nil:
 			return missing(i, "expires")
-		case entry.Expires != nil && !entry.Expires.After(entry.Start):
-			return fmt.Errorf("entry %d's expires %w", i+1, errExpires)
 		case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
 			entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
 			entry.Cause != bans.CauseCrowdSec:

internal/state/state_test.go

@@ -865,21 +865,6 @@ func TestBanWithAnotherCauseStopsTheStart(t *testing.T) {
 		`: entry 4's cause "atack" is not limit, attack, admin or crowdsec`)
 }
 
-func TestBanThatDoesNotEndAfterItsStartStopsTheStart(t *testing.T) {
-	t.Parallel()
-
-	// The first ban ends a second after its start; the second ends as it
-	// starts, or an hour before.
-	for _, expires := range []string{`"2026-10-06T00:00:00Z"`, `"2026-10-05T23:00:00Z"`} {
-		wantRefused(t, bansJSON, `{"version": 1, "bans": [`+
-			`{"netblock": "203.0.113.9/32", "start": "2026-10-06T00:00:00Z", `+
-			`"expires": "2026-10-06T00:00:01Z"}, `+
-			`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
-			`"expires": `+expires+`}]}`,
-			`: entry 2's expires is not after its start`)
-	}
-}
-
 func TestUnknownVersionStopsTheStart(t *testing.T) {
 	t.Parallel()
 

script/example-app

@@ -5,15 +5,12 @@
 # served through smallwebwaf, that a second one in a minute bans the
 # client, that a probe for /.env bans another client, which its next
 # request bans for good, that `sv stop` stops smallwebwaf in order, that
-# `docker stop` stops the container without having to kill it, that a
+# `docker stop` stops the container without having to kill it, and that
-# new container on the same volume still refuses the banned client, that
+# a new container on the same volume still refuses the banned client. The
-# a container with its state files on a new volume mounted at the
-# directory SWWAF_STATE_DIR_FILE names becomes healthy, and that one whose
-# SWWAF_STATE_DIR does not exist logs smallwebwaf's message about it. The
 # containers run with SWWAF_LOOKUP_SOURCE=off, so that no address is sent
-# to GeoJS. The containers, the volumes and both images are removed
+# to GeoJS. The containers, the volume and both images are removed however
-# however the script ends. Building the app needs network access, for
+# the script ends. Building the app needs network access, for nixpkgs'
-# nixpkgs' binary cache. script/check does not run this.
+# binary cache. script/check does not run this.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -26,15 +23,10 @@ IMAGE="$NAME-base"
 APP_IMAGE="$NAME-app"
 CONTAINER="$NAME"
 VOLUME="$NAME-state"
-# The volumes of the container whose state directory SWWAF_STATE_DIR_FILE
-# names: its state files, and the file that names their directory.
-MOVED_VOLUME="$NAME-moved-state"
-SETTINGS_VOLUME="$NAME-settings"
 
 cleanup() {
     docker rm --force "$CONTAINER" >/dev/null 2>&1 || true
-    docker volume rm --force "$VOLUME" "$MOVED_VOLUME" "$SETTINGS_VOLUME" \
+    docker volume rm --force "$VOLUME" >/dev/null 2>&1 || true
-        >/dev/null 2>&1 || true
     docker rmi --force "$APP_IMAGE" "$IMAGE" >/dev/null 2>&1 || true
 }
 
@@ -84,27 +76,6 @@ start_container() {
     address="$(docker port "$CONTAINER" 8080/tcp)"
 }
 
-# start_moved_container: run the app's container with the state files on
-# a new volume at /srv/state, which SWWAF_STATE_DIR_FILE names in a file
-# only the smallwebwaf user can read, as under "Settings given as files"
-# in README.md, and wait until it is healthy. smallwebwaf can write the
-# new volume only once the run script has given it to that user.
-start_moved_container() {
-    docker volume create "$MOVED_VOLUME" >/dev/null
-    docker volume create "$SETTINGS_VOLUME" >/dev/null
-    docker run --rm --volume "$SETTINGS_VOLUME:/settings" \
-        --entrypoint bash "$IMAGE" -c 'echo /srv/state > /settings/state-dir &&
-            chown smallwebwaf:smallwebwaf /settings/state-dir &&
-            chmod 0400 /settings/state-dir'
-    docker run --detach --name "$CONTAINER" \
-        --volume "$MOVED_VOLUME:/srv/state" \
-        --volume "$SETTINGS_VOLUME:/etc/smallwebwaf/settings:ro" \
-        --env SWWAF_STATE_DIR_FILE=/etc/smallwebwaf/settings/state-dir \
-        --env SWWAF_LOOKUP_SOURCE=off \
-        "$APP_IMAGE" >/dev/null
-    wait_for "the health check did not pass" healthy
-}
-
 # refused: a request to the container gets 403, SWWAF_BAN_RESPONSE's
 # default.
 refused() {
@@ -169,20 +140,6 @@ main() {
     wait_for "smallwebwaf logged no request refused under the ban" \
         logged '"action":"banned"'
     echo "example-app: a new container on the same volume keeps the ban"
-
-    docker rm --force "$CONTAINER" >/dev/null
-    start_moved_container
-    echo "example-app: the run script gives the state files' directory named" \
-        "by SWWAF_STATE_DIR_FILE to the smallwebwaf user"
-
-    docker rm --force "$CONTAINER" >/dev/null
-    docker run --detach --name "$CONTAINER" \
-        --env SWWAF_STATE_DIR=/srv/missing \
-        --env SWWAF_LOOKUP_SOURCE=off \
-        "$APP_IMAGE" >/dev/null
-    wait_for "smallwebwaf did not report the missing state directory" \
-        logged "SWWAF_STATE_DIR cannot be written"
-    echo "example-app: smallwebwaf reports a state directory that does not exist"
 }
 
 main "$@"

script/gitea-requests

@@ -1,335 +0,0 @@
-#!/bin/sh
-# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
-# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
-# which names gitea's container. Then send through it the gitea requests
-# that reading gitea's source found the Core Rule Set refuses, and a short
-# sample of gitea's ordinary use: signing in, pages of files and commits,
-# a diff, the API, a push, clone and fetch over HTTP, and two container
-# image pushes. Last, print each request smallwebwaf logged, with what it
-# did with it and the ids of the Core Rule Set's rules that matched it.
-# Every request comes from one client, docker's gateway, so more than
-# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
-# the requests after would show as banned. The containers, the network,
-# the images and the files the script makes are removed however it ends;
-# gitea's own image stays, as the base images of a build do. It needs
-# network access, and a docker daemon that pushes to a registry on
-# localhost over plain HTTP, as docker does by default. script/check does
-# not run this.
-set -eu
-
-SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
-ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
-
-# gitea 1.27.3, 2026-08-29
-GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
-
-# Named after this run, so that runs in other clones on the same host
-# never touch each other's.
-NAME="$("$SCRIPT_DIR/projectname")-gitea-$$"
-IMAGE="$NAME-image"
-NETWORK="$NAME"
-SMALLWEBWAF="$NAME-smallwebwaf"
-GITEA="$NAME-gitea"
-
-# gitea's one user, who owns the repository and the container images.
-PASSWORD="owner-password"
-
-BROWSER="Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
-
-# The files the script makes, and smallwebwaf's address once it runs,
-# localhost and the port docker publishes it on: a host name, since the
-# Core Rule Set takes a Host that is an IP address for a warning (920350).
-WORK=""
-address=""
-
-cleanup() {
-    # So that a signal cannot stop it partway through.
-    trap '' HUP INT TERM
-    docker rm --force --volumes "$SMALLWEBWAF" "$GITEA" >/dev/null 2>&1 || true
-    docker network rm "$NETWORK" >/dev/null 2>&1 || true
-    docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
-    if [ -n "$address" ]; then
-        docker rmi --force "$address/owner/python3-app:1" "$address/owner/app:1" \
-            >/dev/null 2>&1 || true
-    fi
-    rm -rf "$WORK"
-}
-
-fail() {
-    echo "gitea-requests: $*" >&2
-    report
-    exit 1
-}
-
-# wait_for <what fails> <command>...: run the command every second until
-# it succeeds, for at most a minute.
-wait_for() {
-    failure="$1"
-    shift
-    tries=0
-    until "$@"; do
-        tries=$((tries + 1))
-        [ "$tries" -lt 60 ] || fail "$failure"
-        sleep 1
-    done
-}
-
-healthy() {
-    status="$(docker inspect --format '{{.State.Health.Status}}' "$SMALLWEBWAF")"
-    [ "$status" = healthy ]
-}
-
-# logged <text>...: a line of smallwebwaf's output holds every text, in
-# any order.
-logged() {
-    lines="$(docker logs "$SMALLWEBWAF" 2>&1)"
-    for text in "$@"; do
-        lines="$(printf '%s\n' "$lines" | grep -F "$text")" || return 1
-    done
-}
-
-# send <label> <path> <curl option>...: send a request for path to
-# smallwebwaf, with label as its X-Request-ID, which the report gives,
-# and keep the body of the answer in $WORK/answer. The Core Rule Set
-# reads that header too, so a label names no file the cases name.
-send() {
-    label="$1"
-    path="$2"
-    shift 2
-    curl --silent --show-error --max-time 30 --output "$WORK/answer" \
-        --header "X-Request-ID: $label" "$@" "http://$address$path"
-}
-
-# page <label> <path> <curl option>...: send the request as a browser
-# does, with the cookies gitea has set. Its Referer is the repository's
-# page, which links to the others, and never a page the Core Rule Set
-# refused: its address would have the next request refused too.
-page() {
-    label="$1"
-    path="$2"
-    shift 2
-    send "$label" "$path" --compressed --user-agent "$BROWSER" \
-        --header "Accept: text/html,application/xhtml+xml,*/*;q=0.8" \
-        --header "Accept-Language: en-US,en;q=0.5" \
-        --cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
-        --referer "http://$address/owner/repo" "$@"
-}
-
-# api <label> <path> <curl option>...: send the request as a client of
-# gitea's API does, as the owner.
-api() {
-    label="$1"
-    path="$2"
-    shift 2
-    send "$label" "$path" --user "owner:$PASSWORD" \
-        --header "Accept: application/json" "$@"
-}
-
-# repo_git <git argument>...: run git as the owner, without the host's
-# own git settings, so that nothing it does is kept outside $WORK.
-repo_git() {
-    GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
-        -c user.name=owner -c user.email=owner@example.invalid "$@"
-}
-
-# lines: forty lines of text, enough for a diff to hide some of them.
-lines() {
-    i=1
-    while [ "$i" -le 40 ]; do
-        echo "line $i"
-        i=$((i + 1))
-    done
-}
-
-# make_repository: create the repository owner/repo, holding a file of
-# each name the cases name, and push it, which is the push over HTTP of
-# the sample. The second commit adds a line at the end of Runtime.java,
-# so that its diff hides the lines above.
-make_repository() {
-    status="$(api create-repository /api/v1/user/repos \
-        --header "Content-Type: application/json" --data '{"name":"repo"}' \
-        --write-out '%{http_code}')"
-    [ "$status" = 201 ] || fail "creating the repository was answered $status"
-
-    dir="$WORK/repo"
-    repo_git init --quiet --initial-branch=main "$dir"
-    for file in \
-        src/java.base/share/classes/java/lang/Runtime.java \
-        java.lang.String.md \
-        META-INF/services/org.apache.commons.logging.LogFactory \
-        document.write.js \
-        process.env.d.ts \
-        test/fixtures/__proto__.json \
-        install-script.sh \
-        src/backend/catalog/information_schema.sql \
-        .../notes.txt \
-        README.md; do
-        mkdir -p "$dir/$(dirname "$file")"
-        lines >"$dir/$file"
-        repo_git -C "$dir" add "$file"
-    done
-    repo_git -C "$dir" commit --quiet --message "Add the files"
-    echo "line 41" >>"$dir/src/java.base/share/classes/java/lang/Runtime.java"
-    repo_git -C "$dir" commit --quiet --all --message "Add a line"
-
-    repo_git -C "$dir" -c "http.extraHeader=X-Request-ID: git-push" push --quiet \
-        "http://owner:$PASSWORD@$address/owner/repo.git" main ||
-        fail "the push of the repository failed"
-}
-
-# browse: sign in, then open pages as a person does. The cases are those
-# of https://git.eeqj.de/sneak/smallwebwaf/issues/30.
-browse() {
-    # Case: gitea sends a visitor who is not signed in to the sign-in
-    # page, with the page asked for in redirect_to, when that page needs
-    # signing in, as a file's edit page does.
-    page issue-30-case /owner/repo/_edit/main/document.write.js --location
-
-    page sign-in /user/login
-    status="$(page sign-in /user/login --header "Origin: http://$address" \
-        --data-urlencode user_name=owner --data-urlencode "password=$PASSWORD" \
-        --write-out '%{http_code}')"
-    [ "$status" = 303 ] || fail "signing in was answered $status"
-
-    page browse /
-    page browse /owner/repo
-    page browse /owner/repo/src/branch/main/README.md
-    page browse /owner/repo/commits/branch/main
-    page browse "/owner/repo/commit/$(repo_git -C "$WORK/repo" rev-parse HEAD)"
-
-    # Case: expanding the lines the diff hides, by the link the diff's
-    # page gives for it.
-    excerpt="$(sed -n 's/.*data-fetch-url="\([^"]*blob_excerpt[^"]*\)".*/\1/p' \
-        "$WORK/answer" | head -n 1 | sed 's/&amp;/\&/g')"
-    [ -n "$excerpt" ] || fail "the diff's page has no link to its hidden lines"
-    page issue-30-case "$excerpt"
-
-    # Cases: the commit graph of one file, and the pages of files whose
-    # path the Core Rule Set takes for an attack.
-    page issue-30-case "/owner/repo/graph?file=src/java.base/share/classes/java/lang/Runtime.java"
-    page issue-30-case /owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
-    page issue-30-case /owner/repo/src/branch/main/java.lang.String.md
-    page issue-30-case /owner/repo/src/branch/main/process.env.d.ts
-    page issue-30-case /owner/repo/src/branch/main/.../notes.txt
-}
-
-# call_api: the API's calls the cases name: the commits of a file, for a
-# file of each name the Core Rule Set takes for an attack in the query
-# parameter path, and the file-contents call, which takes its list of
-# files as JSON in the query parameter body.
-call_api() {
-    for file in \
-        src/java.base/share/classes/java/lang/Runtime.java \
-        java.lang.String.md \
-        document.write.js \
-        process.env.d.ts \
-        test/fixtures/__proto__.json \
-        install-script.sh \
-        src/backend/catalog/information_schema.sql; do
-        api issue-30-case /api/v1/repos/owner/repo/commits --get --data-urlencode "path=$file"
-    done
-    api issue-30-case /api/v1/repos/owner/repo/file-contents --get \
-        --data-urlencode 'body={"files":["src/java.base/share/classes/java/lang/Runtime.java"]}'
-}
-
-# push_images: push a container image to owner/python3-app, then the same
-# image to owner/app. For the second, the case, docker asks gitea to take
-# its layer from owner/python3-app, naming it in the query parameter
-# from. docker sends no X-Request-ID, so its requests have the ids
-# smallwebwaf gives them. The image is one small file, imported rather
-# than built.
-push_images() {
-    echo "$NAME" >"$WORK/layer.txt"
-    tar -cf - -C "$WORK" layer.txt |
-        docker import - "$address/owner/python3-app:1" >/dev/null
-    docker tag "$address/owner/python3-app:1" "$address/owner/app:1"
-
-    # The login is written here rather than made with docker login, which
-    # would keep it in the host's credential helper when it has one.
-    mkdir "$WORK/docker"
-    printf '{"auths":{"%s":{"auth":"%s"}}}\n' "$address" \
-        "$(printf 'owner:%s' "$PASSWORD" | base64)" >"$WORK/docker/config.json"
-    for image in "$address/owner/python3-app:1" "$address/owner/app:1"; do
-        DOCKER_CONFIG="$WORK/docker" docker push --quiet "$image" >/dev/null ||
-            echo "gitea-requests: the push of $image failed" >&2
-    done
-}
-
-# clone_and_fetch: the clone and fetch over HTTP of the sample.
-clone_and_fetch() {
-    repo_git -c "http.extraHeader=X-Request-ID: git-clone" clone --quiet \
-        "http://owner:$PASSWORD@$address/owner/repo.git" "$WORK/clone" ||
-        fail "the clone failed"
-    repo_git -C "$WORK/clone" -c "http.extraHeader=X-Request-ID: git-fetch" fetch --quiet ||
-        fail "the fetch failed"
-}
-
-# field <name> <line>: the text of the field name in line, a line of the
-# request log, or nothing when it has none.
-field() {
-    printf '%s\n' "$2" | sed -n "s/.*\"$1\":\"\([^\"]*\)\".*/\1/p"
-}
-
-# report: print a line for each request smallwebwaf logged, but its
-# health checks: what it did with the request, the ids of the Core Rule
-# Set's rules that matched it, its X-Request-ID, and the request.
-# waf_blocked is a refusal by the Core Rule Set, and forward a request
-# passed on to gitea, whatever gitea answered.
-report() {
-    docker logs "$SMALLWEBWAF" 2>/dev/null | grep '"type":"request"' |
-        grep -v '"action":"admin"' | sed 's/\\u0026/\&/g' >"$WORK/requests"
-    while IFS= read -r line; do
-        ids="$(printf '%s\n' "$line" | sed -n 's/.*"waf_rule_ids":\[\([0-9,]*\)\].*/\1/p')"
-        query="$(field query "$line")"
-        printf '%-11s %-13s %-17s %s %s%s\n' "$(field action "$line")" "${ids:--}" \
-            "$(field request_id "$line")" "$(field method "$line")" \
-            "$(field path "$line")" "${query:+?$query}"
-    done <"$WORK/requests"
-    echo "gitea-requests: $(grep -c . "$WORK/requests") requests," \
-        "$(grep -c '"action":"waf_blocked"' "$WORK/requests") refused by the Core Rule Set"
-}
-
-main() {
-    cd "$ROOT"
-    trap cleanup EXIT
-    trap 'exit 1' HUP INT TERM
-    WORK="$(mktemp -d)"
-
-    docker build --no-cache -t "$IMAGE" .
-
-    # smallwebwaf first, so that gitea can be given the address it is
-    # reached at, and gitea's image pulled before, so that gitea starts
-    # within the minute smallwebwaf's health check waits for it.
-    docker pull --quiet "$GITEA_IMAGE" >/dev/null
-    docker network create "$NETWORK" >/dev/null
-    docker run --detach --name "$SMALLWEBWAF" --network "$NETWORK" \
-        --publish 127.0.0.1::8080 \
-        --env SWWAF_UPSTREAM_URL="http://$GITEA:3000" \
-        "$IMAGE" >/dev/null
-    published="$(docker port "$SMALLWEBWAF" 8080/tcp)"
-    address="localhost:${published##*:}"
-
-    docker run --detach --name "$GITEA" --network "$NETWORK" \
-        --env GITEA__security__INSTALL_LOCK=true \
-        --env GITEA__server__ROOT_URL="http://$address/" \
-        "$GITEA_IMAGE" >/dev/null
-    # smallwebwaf's health check passes once gitea takes connections.
-    wait_for "the health check did not pass" healthy
-    docker exec --user git "$GITEA" gitea admin user create --admin \
-        --username owner --password "$PASSWORD" --email owner@example.invalid \
-        --must-change-password=false >/dev/null
-
-    make_repository
-    browse
-    call_api
-    push_images
-    clone_and_fetch
-
-    # The fetch's last request is the last of all; once it is logged, so
-    # is every other.
-    wait_for "smallwebwaf logged no line for the fetch" \
-        logged '"request_id":"git-fetch"' '"method":"POST"'
-    report
-}
-
-main "$@"

share/smallwebwaf.run

@@ -5,32 +5,12 @@ set -euo pipefail
 # exits; the wait spaces out the restarts. The state directory and every
 # file in it are given to the smallwebwaf user, so that a volume mounted
 # there needs no change of owner; chown -R changes a symbolic link itself,
-# never what it points to. When SWWAF_STATE_DIR, in either form, cannot be
+# never what it points to. exec, so that the signal `sv stop` sends
-# read or is not the absolute path of a directory, no owner is changed,
+# reaches smallwebwaf itself.
-# and smallwebwaf starts and stops with a message naming the setting. A
-# change of owner that fails, as on a volume mounted read-only, does not
-# end the script either: smallwebwaf reports a directory it cannot write.
-# exec, so that the signal `sv stop` sends reaches smallwebwaf itself.
 main() {
     sleep 1
-    local dir
+    chown -R smallwebwaf:smallwebwaf "${SWWAF_STATE_DIR:-/var/lib/smallwebwaf}"
-    if dir="$(state_dir)" && [[ "$dir" == /* && -d "$dir" ]]; then
-        chown -R smallwebwaf:smallwebwaf "$dir" || true
-    fi
     exec chpst -u smallwebwaf:smallwebwaf /usr/local/bin/smallwebwaf
 }
 
-# state_dir prints SWWAF_STATE_DIR, /var/lib/smallwebwaf when it is unset,
-# or what is in the file SWWAF_STATE_DIR_FILE names, less every newline at
-# its end. It fails when both are set or the file cannot be read.
-state_dir() {
-    if [[ ! -v SWWAF_STATE_DIR_FILE ]]; then
-        printf '%s\n' "${SWWAF_STATE_DIR-/var/lib/smallwebwaf}"
-    elif [[ ! -v SWWAF_STATE_DIR ]]; then
-        cat -- "$SWWAF_STATE_DIR_FILE" 2>/dev/null
-    else
-        return 1
-    fi
-}
-
 main "$@"