rproxy and sww tests: the circuit breaker test and the five-second stop test fail under load #474

Open
opened 2026-11-19 19:10:23 +01:00 by clawbot · 1 comment
Collaborator

Two tests failed in the race pass of make check on 2026-11-19, on a review clone of #468, which changes only test files in internal/waf. Both passed on the -v rerun in the same step, and on the next two full runs. The host's load average was between about 60 and 200 on 48 cores.

  • TestAnswersThatCannotBePassedOnOpenTheBreaker in internal/rproxy/rproxy_test.go failed after 16.22 s: rproxy_test.go:1549: rproxy did not write "the circuit breaker is open". waitFor waits 10 seconds (waitLimit) for that line.
  • TestRProxyStoppingCutsOffARequestAfterFiveSeconds in internal/sww/rproxy_test.go failed after 11.61 s: read "" (... i/o timeout), want the connection cut off, then cut off after 4.462139816s, want 5s. sendAndKeepOpen sets a 10-second deadline when the request is sent, and the wait for the app used up more than 5 seconds of it.

The test phase exits 1 even when the rerun passes, so either test can turn a PR's check, or next, red on its own.

Definition of done

  • Neither test fails because other work slows the host.
  • Each still fails when the behaviour it checks is broken.
  • make check passes.

Model: opus-5-5

Two tests failed in the race pass of `make check` on 2026-11-19, on a review clone of https://git.eeqj.de/sneak/smallwebwaf/pulls/468, which changes only test files in `internal/waf`. Both passed on the `-v` rerun in the same step, and on the next two full runs. The host's load average was between about 60 and 200 on 48 cores. - `TestAnswersThatCannotBePassedOnOpenTheBreaker` in `internal/rproxy/rproxy_test.go` failed after 16.22 s: `rproxy_test.go:1549: rproxy did not write "the circuit breaker is open"`. `waitFor` waits 10 seconds (`waitLimit`) for that line. - `TestRProxyStoppingCutsOffARequestAfterFiveSeconds` in `internal/sww/rproxy_test.go` failed after 11.61 s: `read "" (... i/o timeout), want the connection cut off`, then `cut off after 4.462139816s, want 5s`. `sendAndKeepOpen` sets a 10-second deadline when the request is sent, and the wait for the app used up more than 5 seconds of it. The test phase exits 1 even when the rerun passes, so either test can turn a PR's check, or `next`, red on its own. ## Definition of done - Neither test fails because other work slows the host. - Each still fails when the behaviour it checks is broken. - `make check` passes. Model: opus-5-5
Author
Collaborator

PR: #475, branch issue-474-breaker-and-stop-tests-under-load.

  • The circuit breaker reads the time through a clock the handler holds, time.Now in production. The breaker test stops that clock, so host speed no longer decides whether the breaker opens or turns the next request away.
  • The stop test counts its read deadline from the stop, not from the send.

Disclosure: TestFiveFailedAsksKeepTheNextRequestFromAskingWAF still reads the real clock and can fail the same way on a slow enough host; it is not named here, so it is unchanged.

Model: opus-5-5

PR: https://git.eeqj.de/sneak/smallwebwaf/pulls/475, branch `issue-474-breaker-and-stop-tests-under-load`. - The circuit breaker reads the time through a clock the handler holds, `time.Now` in production. The breaker test stops that clock, so host speed no longer decides whether the breaker opens or turns the next request away. - The stop test counts its read deadline from the stop, not from the send. Disclosure: `TestFiveFailedAsksKeepTheNextRequestFromAskingWAF` still reads the real clock and can fail the same way on a slow enough host; it is not named here, so it is unchanged. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/smallwebwaf#474