rproxy: circuit breaker for asks to waf #392

Open
opened 2026-11-15 14:23:49 +01:00 by clawbot · 1 comment
Collaborator

sneak's ruling on #385 (comments 181327 and 181334), reworded:

  • rproxy gets a circuit breaker for its asks to waf.
  • A failed ask is one that gets no answer in time: refused, unreachable, or no complete answer within SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS. Any answer in time is a success, whatever it decides.
  • One failure never opens the breaker; several within about five seconds do.
  • Once open, it stays open until an ask succeeds; one success closes it.
  • Slow answers count as failures. A request waf cannot decide within the timeout is a bug in waf, not a reason to exempt slow answers from the breaker.

Design

Constants that were not in his words (reversible; chosen as the simplest fit):

  • Opens after 5 failed asks within 5 seconds.
  • While open, a request takes SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION at once, without asking waf, with the same warning and waf_backend_no_answer: true log field as an ask that failed. At most one request a second still asks waf; its success closes the breaker, its failure keeps it open.
  • No new settings: the three numbers are constants in internal/rproxy.
  • State lives on the handler (internal/rproxy/rproxy.go) under a mutex. Update the sentence in SPEC.md that says rproxy keeps nothing between requests, and document the breaker there.

Definition of done

  • The breaker behaves as above, in internal/rproxy.
  • Tests: 4 failures in 5 s leave it closed; 5 open it; while open, requests take the disposition without asking, except one ask a second; one success closes it; a slow answer past the timeout counts as a failure. Tests use a fake clock or short constants injected in tests, not real 5 s sleeps.
  • SPEC.md describes the breaker; make check passes.

Model: opus-5-5

sneak's ruling on https://git.eeqj.de/sneak/smallwebwaf/issues/385 (comments 181327 and 181334), reworded: - `rproxy` gets a circuit breaker for its asks to `waf`. - A failed ask is one that gets no answer in time: refused, unreachable, or no complete answer within `SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS`. Any answer in time is a success, whatever it decides. - One failure never opens the breaker; several within about five seconds do. - Once open, it stays open until an ask succeeds; one success closes it. - Slow answers count as failures. A request `waf` cannot decide within the timeout is a bug in `waf`, not a reason to exempt slow answers from the breaker. ## Design Constants that were not in his words (reversible; chosen as the simplest fit): - Opens after 5 failed asks within 5 seconds. - While open, a request takes `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` at once, without asking `waf`, with the same warning and `waf_backend_no_answer: true` log field as an ask that failed. At most one request a second still asks `waf`; its success closes the breaker, its failure keeps it open. - No new settings: the three numbers are constants in `internal/rproxy`. - State lives on the handler (`internal/rproxy/rproxy.go`) under a mutex. Update the sentence in `SPEC.md` that says `rproxy` keeps nothing between requests, and document the breaker there. ## Definition of done - The breaker behaves as above, in `internal/rproxy`. - Tests: 4 failures in 5 s leave it closed; 5 open it; while open, requests take the disposition without asking, except one ask a second; one success closes it; a slow answer past the timeout counts as a failure. Tests use a fake clock or short constants injected in tests, not real 5 s sleeps. - `SPEC.md` describes the breaker; `make check` passes. Model: opus-5-5
Author
Collaborator

Built as #393, branch issue-392-rproxy-circuit-breaker, with the constants from the issue. Judgement call: an answer while the breaker is closed does not clear earlier failures; only the answer that closes it does.

Model: opus-5-5

Built as https://git.eeqj.de/sneak/smallwebwaf/pulls/393, branch `issue-392-rproxy-circuit-breaker`, with the constants from the issue. Judgement call: an answer while the breaker is closed does not clear earlier failures; only the answer that closes it does. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/smallwebwaf#392