Leave SWWAF_RATE_LIMIT_EXEMPT_PATHS out of the request rate limits (closes #77)

A request is neither counted nor refused by the request rate limits
when its path as sent, the path the app receives, not percent-decoded,
starts with one of the comma-separated prefixes in
SWWAF_RATE_LIMIT_EXEMPT_PATHS, so /%61ssets/x is not under /assets/. A
request whose decoded path contains .. or a backslash, or whose path as
sent holds an encoded slash, is never exempt, since an app may act on
it as a path outside every prefix, such as /assets/..%2Flogin as
/login. The static lists, bans and the country lists still apply. The
setting is empty by default, and a prefix that does not start with /
stops the start. README.md documents it.

Model: opus-5-5

README.md

@@ -13,23 +13,24 @@ JSON log line for every request.
 
 Status: the first two milestones are built
 (https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
-https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are six parts of
-milestone 3: the static lists, the bans that broken rate limits lead to and the
-JSON state files with your edits taken in while it runs, which come next in the
-build order, `observe` mode, which comes a little later, and the metrics
-endpoint and the header size and the idle time as settings, which come last in
-it. `smallwebwaf` passes each request to the app and the app's answer back,
-unchanged, within its timeouts and size limits, works out each client's address,
-bans a client that sends too many requests, refuses a client that comes from a
-country you refuse or from a network you refuse, lets the networks you choose
-through, keeps its bans, each client's counters and history, and GeoJS's answers
-in JSON files across restarts, takes in your edits of those files while it runs,
-writes a JSON log line for every request, serves Prometheus metrics to a scraper
-that holds the metrics token, and in `observe` mode passes on the requests it
-would refuse, logging what it would have done with them. It comes as the image
-the app's own image is built on. The rest of the design comes after that, in the
-order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
-that led to the design is in [`EVALUATION.md`](EVALUATION.md).
+https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are seven parts of
+milestone 3: the static lists, the bans that broken rate limits lead to, the
+JSON state files with your edits taken in while it runs and the paths the rate
+limits do not count, which come next in the build order, `observe` mode, which
+comes a little later, and the metrics endpoint and the header size and the idle
+time as settings, which come last in it. `smallwebwaf` passes each request to
+the app and the app's answer back, unchanged, within its timeouts and size
+limits, works out each client's address, bans a client that sends too many
+requests, not counting those for the paths you choose, refuses a client that
+comes from a country you refuse or from a network you refuse, lets the networks
+you choose through, keeps its bans, each client's counters and history, and
+GeoJS's answers in JSON files across restarts, takes in your edits of those
+files while it runs, writes a JSON log line for every request, serves Prometheus
+metrics to a scraper that holds the metrics token, and in `observe` mode passes
+on the requests it would refuse, logging what it would have done with them. It
+comes as the image the app's own image is built on. The rest of the design comes
+after that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey
+of existing tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
 
 ## Getting started
 
@@ -80,12 +81,15 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set.
 - Counts each client's requests over a minute, an hour and a day. A request that
   takes the client over one of the rate limits below is refused with
   `SWWAF_BAN_RESPONSE`, `403` by default, before anything reaches the app, and
-  bans the client. A client is one IPv4 address, or one IPv6 /64, since one
-  abuser usually holds a whole /64. Each window is counted in two fixed buckets,
-  the earlier one weighted by how much of it the window still covers. At most
-  20,000 clients are kept, the least recently seen dropped first, with their
-  history, and a restart gives no client a fresh allowance (see "State files"
-  below).
+  bans the client. A request whose path starts with one of
+  `SWWAF_RATE_LIMIT_EXEMPT_PATHS`, as that setting below describes, is neither
+  counted nor refused by the rate limits; the static lists, bans and the country
+  lists still apply to it. A client is one IPv4 address, or one IPv6 /64, since
+  one abuser usually holds a whole /64. Each window is counted in two fixed
+  buckets, the earlier one weighted by how much of it the window still covers.
+  At most 20,000 clients are kept, the least recently seen dropped first, with
+  their history, and a restart gives no client a fresh allowance (see "State
+  files" below).
 - Bans a client that breaks a rate limit, as "Bans" in [`SPEC.md`](SPEC.md)
   describes: the first ban lasts an hour, and a limit broken again within a day
   of a ban ending bans for three times as long as that ban, so 1, 3, 9, 27 and
@@ -192,6 +196,20 @@ it, and the effective settings are logged at start.
   requests a client may make in a minute, an hour and a day. The defaults are
   several times what one busy person produces, since a browser loading a heavy
   page makes a few hundred requests and several people often share one address.
+- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes whose requests
+  the rate limits neither count nor refuse, such as `/assets/` for static
+  assets; each starts with `/`. A request whose path, percent-decoded, contains
+  `..` anywhere or a backslash, or whose path as sent holds an encoded slash
+  (`%2F` or `%2f`), is never exempt, since the app may act on it as a path
+  outside every prefix: `/assets/..%2Flogin` as `/login`. Any other request is
+  exempt when its path as sent, the path the app receives, before any query
+  string and not percent-decoded, starts with a prefix, character for character.
+  `/assets/` matches `/assets/app.js` and `/assets/`, but not `/assets`,
+  `/Assets/app.js`, `/%61ssets/app.js`, `/static/assets/app.js`,
+  `/static/../assets/app.js` or `/assets%2Fapp.js`. A character the client sends
+  percent-encoded, such as a space, is written percent-encoded in a prefix, as
+  in `/my%20files/`, and there are no wildcards: `*` is a character like any
+  other.
 - `SWWAF_DENIED_COUNTRIES` (default empty): countries whose clients are refused,
   for example `cn,ru,kp`.
 - `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
@@ -773,9 +791,8 @@ so that they run in minimal containers.
 
 ## TODO
 
-- The rest of milestone 3: exemptions and the rest of the request log's fields;
-  then the rest of the design, in the order of the build order in
-  [`SPEC.md`](SPEC.md).
+- The rest of milestone 3: the rest of the request log's fields; then the rest
+  of the design, in the order of the build order in [`SPEC.md`](SPEC.md).
 
 ## Documents