rproxy: circuit breaker for asks to waf (closes #392)

Five asks that get no answer in time within five seconds open the
breaker. While open, a request takes
SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION without asking waf, except
one ask a second; the first answer closes it. Slow answers count as
failures. SPEC.md describes it.

The sww test that stops and restarts waf now waits for the breaker to
close before it expects the ban again.

Model: opus-5-5

SPEC.md

@@ -2583,9 +2583,10 @@ request and ban notes name the site by its host.
 ## `rproxy`
 
 `sww rproxy` is the reverse proxy in front of one app. It asks `waf` about each
-request, and passes on what `waf` allows. It keeps nothing between requests and
-writes nothing to disk. When `waf` cannot be connected to, does not answer at
-all, or does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`,
+request, and passes on what `waf` allows. It writes nothing to disk. The only
+thing it keeps between requests is its circuit breaker for asks to `waf`. When
+`waf` cannot be connected to, does not answer at all, or does not answer within
+`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`,
 `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides. Its default, `allow`,
 keeps the site answering.
 
@@ -2679,23 +2680,24 @@ log line has status `0`.
 ### Without an answer from `waf`
 
 `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides what is done with the
-request only when `waf`:
+request when `waf`:
 
 - cannot be connected to;
 - does not answer at all;
 - does not answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, which covers
   connecting, sending the question and reading the whole answer.
 
-An answer whose body breaks off counts as no answer at all.
+An answer whose body breaks off counts as no answer at all. The setting also
+decides, without asking `waf`, while the circuit breaker is open.
 
 `allow`, the default, passes the request to the app. `deny` answers `503`.
 Either way `rproxy` writes a process warning, its log line has
 `waf_backend_no_answer`, and it sends no report.
 
-A visitor can make `waf` slow. A request that keeps `waf` busy past
-`SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`, like any request while `waf` is overloaded,
-gets `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`. With `deny`, such a request
-gets `503` instead of reaching the app.
+`waf` taking longer than `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` to decide a request
+is a bug in `waf`. Such a request gets
+`SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`, like any request while `waf` is
+overloaded. With `deny`, it gets `503` instead of reaching the app.
 
 A path under `/_sww/` that `allow` would pass on is answered `404` instead, so
 that the app never sees that prefix.
@@ -2710,6 +2712,27 @@ A visitor that goes away while `waf` is being asked is no failure of `waf`.
 `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` does not apply. Its log line has
 status `0` and no `disposition`.
 
+### The circuit breaker
+
+`rproxy` stops asking `waf` for a while when `waf` keeps failing to answer. An
+ask fails in each case that "Without an answer from `waf`" lists. Any answer in
+time succeeds, whatever it decides, and so does an answer `rproxy` cannot pass
+on. A visitor that goes away while `waf` is being asked counts as neither.
+
+The breaker works like this:
+
+- It opens when 5 asks fail within 5 seconds. Fewer failures never open it.
+- While it is open, a request gets
+  `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` at once, without asking `waf`.
+  It gets the same process warning and `waf_backend_no_answer` as a request
+  whose ask failed.
+- While it is open, one request a second still asks `waf`.
+- The first ask that succeeds closes it, and the failures before it no longer
+  count.
+
+The breaker lives in memory, and a restart of `rproxy` closes it. It has no
+settings.
+
 ### Passing a request to the app
 
 `rproxy` sends the app at `SWW_RPROXY_UPSTREAM_URL` the visitor's request: the
@@ -2983,8 +3006,8 @@ host directory, keeps them across deploys. A host directory mounted there needs
 no change of owner, and files an earlier owner left in it can be read and
 replaced. The volume holds a few tens of MiB at most with the defaults, and
 needs no backup beyond the host's own. The image declares no volume, since every
-app image built on it would inherit it. `rproxy` keeps nothing, and needs no
-volume.
+app image built on it would inherit it. `rproxy` writes nothing to disk, and
+needs no volume.
 
 `waf` keeps its cache files in `SWW_WAF_CACHE_DIR`, `/var/cache/sww` by default,
 which the `run` script makes. The cache directory needs no volume and can be