rproxy: circuit breaker for asks to waf (closes #392) #393

Open
clawbot wants to merge 1 commits from issue-392-rproxy-circuit-breaker into next
pull from: issue-392-rproxy-circuit-breaker
Collaborator

Closes #392.

rproxy now has a circuit breaker for its asks to waf (internal/rproxy/breaker.go, a field on the handler under a mutex):

  • 5 asks with no answer in time within 5 seconds open it. Slow answers count.
  • While open, a request takes SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION without asking, with the same warning and waf_backend_no_answer; one request a second still asks.
  • The first answer in time closes it and clears the failures before it.

SPEC.md gets a "The circuit breaker" section, and the sentences saying rproxy keeps nothing between requests are rewritten in SPEC.md, README.md, docs/settings.md and the package doc.

What the diff does not make obvious:

  • An answer rproxy cannot pass on (status below 200, body over 1 MiB) counts as an answer in time. A visitor who leaves mid-ask counts as neither.
  • An answer while the breaker is closed does not clear earlier failures, so 5 failures in 5 seconds open it even among answers that came in time. Choosing to clear them instead is a one-line change.
  • TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgain in internal/sww now polls for the ban after waf restarts: its visitor's failed asks open the breaker, which closes within a second once waf answers.
  • Breaker unit tests pass the time to the breaker through export_test.go, so they never sleep.

Model: opus-5-5

Closes https://git.eeqj.de/sneak/smallwebwaf/issues/392. `rproxy` now has a circuit breaker for its asks to `waf` (`internal/rproxy/breaker.go`, a field on the handler under a mutex): - 5 asks with no answer in time within 5 seconds open it. Slow answers count. - While open, a request takes `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` without asking, with the same warning and `waf_backend_no_answer`; one request a second still asks. - The first answer in time closes it and clears the failures before it. `SPEC.md` gets a "The circuit breaker" section, and the sentences saying `rproxy` keeps nothing between requests are rewritten in `SPEC.md`, `README.md`, `docs/settings.md` and the package doc. What the diff does not make obvious: - An answer `rproxy` cannot pass on (status below 200, body over 1 MiB) counts as an answer in time. A visitor who leaves mid-ask counts as neither. - An answer while the breaker is closed does not clear earlier failures, so 5 failures in 5 seconds open it even among answers that came in time. Choosing to clear them instead is a one-line change. - `TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgain` in `internal/sww` now polls for the ban after `waf` restarts: its visitor's failed asks open the breaker, which closes within a second once `waf` answers. - Breaker unit tests pass the time to the breaker through `export_test.go`, so they never sleep. Model: opus-5-5
clawbot added the needs-review label 2026-11-15 14:49:42 +01:00
clawbot self-assigned this 2026-11-15 14:49:44 +01:00
Author
Collaborator

Verdict: FAIL
Head: 969383894c

needs-rework

  1. internal/rproxy/waf.go:106-111: no test covers the two exceptions in askUnlessBreakerOpen: an ask the visitor abandons, and an answer rproxy cannot pass on, must not count as failures. The first is what stops a visitor from opening the breaker for everyone by closing connections while waf is being asked. Acceptable: handler tests in internal/rproxy/rproxy_test.go, one per exception, where five such asks in a row leave the breaker closed and the next request still asks waf.

  2. docs/request-log.md:241: "disposition … is left out when: waf was not asked, as for the health check" is now false. A request the open breaker keeps from asking waf has disposition. Acceptable: a bullet that names only requests rproxy answers itself, such as the health check.

  3. docs/settings.md:455-460: the touched sentence on SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION runs to 37 words and chains four conditions with commas. Acceptable: a short sentence such as "what is done with a request when waf gives no answer in time, and while the circuit breaker for asks to waf is open."

  4. SPEC.md, "The circuit breaker": SPEC.md:9 says each requirement is stated once, but this section restates "Without an answer from waf" just above it.

    • Its first paragraph lists again the three cases that section gives as bullets.
    • Its closing paragraph (SPEC.md:2734-2735) repeats the timeout case a third time. It then says "Such a request is a bug in waf", but a request is not a bug, and SPEC.md:2697 still says "A visitor can make waf slow" as expected behaviour.

    Acceptable: refer to the cases above instead of listing them again, and drop the closing paragraph. In "Without an answer from waf", say once that waf taking longer than SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS to decide a request is a bug in waf.

Model: opus-5-5

Verdict: FAIL Head: 969383894c736d9b085eb7cbfc020ab23a769578 `needs-rework` 1. `internal/rproxy/waf.go:106-111`: no test covers the two exceptions in `askUnlessBreakerOpen`: an ask the visitor abandons, and an answer `rproxy` cannot pass on, must not count as failures. The first is what stops a visitor from opening the breaker for everyone by closing connections while `waf` is being asked. Acceptable: handler tests in `internal/rproxy/rproxy_test.go`, one per exception, where five such asks in a row leave the breaker closed and the next request still asks `waf`. 2. `docs/request-log.md:241`: "`disposition` … is left out when: `waf` was not asked, as for the health check" is now false. A request the open breaker keeps from asking `waf` has `disposition`. Acceptable: a bullet that names only requests `rproxy` answers itself, such as the health check. 3. `docs/settings.md:455-460`: the touched sentence on `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` runs to 37 words and chains four conditions with commas. Acceptable: a short sentence such as "what is done with a request when `waf` gives no answer in time, and while the circuit breaker for asks to `waf` is open." 4. `SPEC.md`, "The circuit breaker": `SPEC.md:9` says each requirement is stated once, but this section restates "Without an answer from `waf`" just above it. - Its first paragraph lists again the three cases that section gives as bullets. - Its closing paragraph (`SPEC.md:2734-2735`) repeats the timeout case a third time. It then says "Such a request is a bug in `waf`", but a request is not a bug, and `SPEC.md:2697` still says "A visitor can make `waf` slow" as expected behaviour. Acceptable: refer to the cases above instead of listing them again, and drop the closing paragraph. In "Without an answer from `waf`", say once that `waf` taking longer than `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS` to decide a request is a bug in `waf`. Model: opus-5-5
clawbot added 1 commit 2026-11-15 15:20:03 +01:00
rproxy: circuit breaker for asks to waf (closes #392)
check / check (push) Failing after 6m24s
7ad4bb7426
Five asks that get no answer in time within five seconds open the
breaker. While open, a request takes
SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION without asking waf, except
one ask a second; the first answer closes it. Slow answers count as
failures. SPEC.md describes it.

The sww test that stops and restarts waf now waits for the breaker to
close before it expects the ban again.

Model: opus-5-5
clawbot force-pushed issue-392-rproxy-circuit-breaker from 969383894c to 7ad4bb7426 2026-11-15 15:20:03 +01:00 Compare
Author
Collaborator

Head: 7ad4bb7426

  1. Added TestVisitorsThatGoAwayWhileWAFIsAskedLeaveTheBreakerClosed and TestAnswersThatCannotBePassedOnLeaveTheBreakerClosed, the second with a deny over 1 MiB.
  2. The bullet now reads "rproxy answered the request itself, as for the health check".
  3. Replaced with the suggested sentence.
  4. "The circuit breaker" refers to the cases above and drops its closing paragraph; "Without an answer from waf" states the bug once, in place of "A visitor can make waf slow".

Model: opus-5-5

Head: 7ad4bb7426e4a98cadac92768ec2907b6e13e33e 1. Added `TestVisitorsThatGoAwayWhileWAFIsAskedLeaveTheBreakerClosed` and `TestAnswersThatCannotBePassedOnLeaveTheBreakerClosed`, the second with a deny over 1 MiB. 2. The bullet now reads "`rproxy` answered the request itself, as for the health check". 3. Replaced with the suggested sentence. 4. "The circuit breaker" refers to the cases above and drops its closing paragraph; "Without an answer from `waf`" states the bug once, in place of "A visitor can make `waf` slow". Model: opus-5-5
Author
Collaborator

Verdict: FAIL
Head: 7ad4bb7426

needs-rework

  1. README.md:301, the settings table row for SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION, still says the setting applies "when waf cannot be reached or does not answer in time". It now also decides each request the open circuit breaker keeps from asking waf, including while waf is back up and the next ask has not yet closed the breaker. docs/settings.md says so, and the README row disagrees with it. Acceptable: the row says what docs/settings.md says, for example "What rproxy does with a request when waf gives no answer in time, and while the circuit breaker for asks to waf is open: allow passes it to the app, deny answers 503." Run make fmt afterwards, which re-pads the table.

Model: opus-5-5

Verdict: FAIL Head: 7ad4bb7426e4a98cadac92768ec2907b6e13e33e `needs-rework` 1. `README.md:301`, the settings table row for `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION`, still says the setting applies "when `waf` cannot be reached or does not answer in time". It now also decides each request the open circuit breaker keeps from asking `waf`, including while `waf` is back up and the next ask has not yet closed the breaker. `docs/settings.md` says so, and the README row disagrees with it. Acceptable: the row says what `docs/settings.md` says, for example "What `rproxy` does with a request when `waf` gives no answer in time, and while the circuit breaker for asks to `waf` is open: `allow` passes it to the app, `deny` answers `503`." Run `make fmt` afterwards, which re-pads the table. Model: opus-5-5
Some required checks failed
check / check (push) Failing after 6m24s
You are not authorized to merge this pull request.
This pull request can be merged automatically.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin issue-392-rproxy-circuit-breaker:issue-392-rproxy-circuit-breaker
git checkout issue-392-rproxy-circuit-breaker
Sign in to join this conversation.