Per-client request rate limits over a minute, an hour and a day (closes #43)

Each client, one IPv4 address or one IPv6 /64, is counted in two buckets
per window, the earlier weighted by how much of it the window covers; at
most 20,000 clients are kept, least recently seen dropped first. A
request over SWWAF_RATE_LIMIT_PER_MINUTE, _HOUR or _DAY (1000, 10000,
50000, or off) gets 429 before reaching the app. Refused requests count,
413s included. A clock set back over a second behind a bucket's start
restarts that window. The log line gains limit_hit and the action
rate_limited.

Deviation from SPEC.md, per the issue: the 20,000 bound and /64 are fixed.
Judgement call: golang-lru/v2 holds the table; httprate does not count refused requests.
Deviation: go.mod and go.sum hand-written; no make target tidies them.

Model: opus-5-5

README.md

@@ -12,14 +12,15 @@ state in memory and in JSON files you can read and edit, and writes a detailed
 JSON log line for every request.
 
 Status: the first milestone is built
-(https://git.eeqj.de/sneak/smallwebwaf/issues/13). `smallwebwaf` passes each
-request to the app and the app's answer back, unchanged, within its timeouts and
-size limits, works out each client's address, and writes a JSON log line for
-every request. Rate limits per client, the country lists and the image an app
-builds on come with milestone 2
-(https://git.eeqj.de/sneak/smallwebwaf/issues/14), and the rest of the design
-after that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey
-of existing tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
+(https://git.eeqj.de/sneak/smallwebwaf/issues/13), and the rate limits of the
+second (https://git.eeqj.de/sneak/smallwebwaf/issues/14). `smallwebwaf` passes
+each request to the app and the app's answer back, unchanged, within its
+timeouts and size limits, works out each client's address, refuses a client that
+sends too many requests, and writes a JSON log line for every request. The
+country lists and the image an app builds on come with the rest of milestone 2,
+and the rest of the design after that, in the order of the build order in
+[`SPEC.md`](SPEC.md). The survey of existing tools that led to the design is in
+[`EVALUATION.md`](EVALUATION.md).
 
 ## Getting started
 
@@ -38,7 +39,7 @@ It then listens on port 8080 and passes every request to the app at
 app on `127.0.0.1:8081`. On `SIGTERM` or `SIGINT` it stops taking requests and
 gives those in progress five seconds to finish.
 
-## What milestone 1 does
+## What it does so far
 
 - Passes each request to the app and the app's answer back unchanged: method,
   path, query, headers, body and status. Bodies stream through in both
@@ -61,6 +62,16 @@ gives those in progress five seconds to finish.
   `smallwebwaf` was waiting for the client to send more, and `504` if it was
   waiting for the app to take what it had. Once the response has started, a
   limit can only cut the connection.
+- Counts each client's requests over a minute, an hour and a day. A request that
+  takes the client over one of the rate limits below is refused with `429`
+  before anything reaches the app, and so is each request after it until the
+  client is back under every limit. A client is one IPv4 address, or one IPv6
+  /64, since one abuser usually holds a whole /64. Refused requests count too,
+  so a client that keeps sending too fast stays refused until it slows down.
+  Each window is counted in two fixed buckets, the earlier one weighted by how
+  much of it the window still covers. At most 20,000 clients are kept, the least
+  recently seen dropped first, and only in memory: a restart starts every client
+  afresh.
 - Writes a line in the request log for each request (see "Request log" below).
 
 ## Settings
@@ -86,18 +97,25 @@ it, and the effective settings are logged at start.
   to send its whole answer, from the end of the request to the last byte.
 - `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body.
 - `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
+- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `1000`), `SWWAF_RATE_LIMIT_PER_HOUR`
+  (default `10000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `50000`): the most
+  requests a client may make in a minute, an hour and a day. The defaults are
+  several times what one busy person produces, since a browser loading a heavy
+  page makes a few hundred requests and several people often share one address.
 
 Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are
 bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024
-bytes). Netblocks are in CIDR form, and a bare address stands for itself alone.
-`off` switches a timeout or a size limit off.
+bytes). Rate limits are whole numbers of requests. Netblocks are in CIDR form,
+and a bare address stands for itself alone. `off` switches a timeout, a size
+limit or a rate limit off.
 
-Two limits are fixed rather than settings: the request line and headers may take
-up to 32 KiB, above which the answer is `431` and nothing reaches the app, and a
-kept-open connection that sends nothing for 120 seconds is closed. That is
+Four limits are fixed rather than settings. The request line and headers may
+take up to 32 KiB, above which the answer is `431` and nothing reaches the app.
+A kept-open connection that sends nothing for 120 seconds is closed. That is
 longer than the 90 seconds after which traefik closes a connection it is not
 using, so traefik never sends a request on a connection `smallwebwaf` is
-closing.
+closing. At most 20,000 clients are kept for the rate limits, and an IPv6 client
+is counted by its /64.
 
 ## Request log
 
@@ -113,10 +131,13 @@ refused ones included:
 - `status` is what the client was sent, `0` if nothing was; `upstream_status` is
   what the app answered, and is left out when the app did not answer.
 - `request_bytes` and `response_bytes` count body bytes.
-- `action` is `forward` for a request passed to the app, `too_large` for a
-  request or response over its size limit, `timed_out` for one that ran out of
-  time, and `upstream_error` when the app could not be reached or its answer
-  broke off.
+- `action` is `forward` for a request passed to the app, `rate_limited` for one
+  refused for a rate limit, `too_large` for a request or response over its size
+  limit, `timed_out` for one that ran out of time, and `upstream_error` when the
+  app could not be reached or its answer broke off.
+- `limit_hit` is there for a request refused for a rate limit, and names the
+  window whose limit it went over: `minute`, `hour` or `day`, the shortest if it
+  went over several.
 - `aborted` is there, and true, when the client went away early.
 - `duration_total` and `duration_upstream_total` are in milliseconds.
 
@@ -391,12 +412,17 @@ refusal comes with `SWWAF_ALLOW_NETS` in milestone 3 or later.
 - `internal/proxy`: what happens to each request: it works out the client, runs
   the checks, passes the request to the app and the answer back with the
   standard library's `httputil.ReverseProxy` within the timeouts and size
-  limits, and writes the request's log line. Its `check` method is where
-  milestone 2's rate limits and country lists refuse a request.
+  limits, and writes the request's log line. Its `check` method is where a
+  request is refused before anything reaches the app: for a rate limit, for an
+  announced body over the size limit, and, with the rest of milestone 2, for the
+  country lists.
+- `internal/ratelimit`: counts each client's requests and tells when one takes
+  it over a rate limit.
 - `internal/requestlog`: the lines on stdout: the request log line and the
   process's own messages.
 
-Only the Go standard library is used.
+Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
+table of clients to 20,000, dropping the least recently seen.
 
 ## Entrypoints
 
@@ -430,8 +456,8 @@ so that they run in minimal containers.
 
 ## TODO
 
-- Milestone 2: rate limits per client, the country lists and the image an app
-  builds on (https://git.eeqj.de/sneak/smallwebwaf/issues/14).
+- Milestone 2: the country lists and the image an app builds on
+  (https://git.eeqj.de/sneak/smallwebwaf/issues/14); its rate limits are built.
 - The rest of the design, in the order of the build order in
   [`SPEC.md`](SPEC.md).