rproxy: circuit breaker for asks to waf (closes #392) #393
Open
clawbot
wants to merge 1 commits from
issue-392-rproxy-circuit-breaker into next
pull from: issue-392-rproxy-circuit-breaker
merge into: sneak:next
sneak:main
sneak:next
sneak:issue-216-oci-labels
sneak:issue-388-any-tls-proxy
sneak:issue-386-readme-rproxy-alone
sneak:issue-387-readme-scraper-policy
sneak:issue-339-long-line-limits
sneak:issue-379-lookup-headers-by-default
sneak:issue-315-remove-crowdsec
sneak:issue-354-package-registry-searches
sneak:issue-353-api-searches
sneak:issue-234-rename-settings
sneak:issue-359-ghost-remaining-fields
sneak:issue-373-read-rest-after-connection-close
sneak:issue-374-ghost-canonical-url
sneak:issue-371-body-over-limit-test-reset
sneak:issue-358-ghost-scripts
sneak:issue-211-viper-config
sneak:issue-357-ghost
sneak:issue-346-names-in-bodies
sneak:issue-365-adopt-form-repository-names
sneak:issue-220-apps-by-hostname
sneak:issue-352-shared-searches
sneak:issue-215-golines
sneak:issue-227-settings-file-per-daemon
sneak:issue-350-forgejo-searches
sneak:issue-349-gitea-user-page-ls
sneak:issue-287-crs-test-cpu
sneak:issue-284-wordpress-recorded-both-ways
sneak:readme-rewrite-155
No Reviewers
Labels
Clear labels
critical
merge-ready
needs-checks
needs-decision
needs-decision
needs-decision
needs-rebase
needs-review
needs-rework
post-1.0
red-next
Extremely important: data loss, a security hole, production down, or basic use impossible
Reviewed and ready for the owner to merge
Checks must be fixed before review
Waits on an owner decision
Waiting on a decision by the owner
Conflicts with next; rebase required
Review not yet done
Review found issues to fix
No labels
needs-review
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: sneak/smallwebwaf#393
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #392.
rproxynow has a circuit breaker for its asks towaf(internal/rproxy/breaker.go, a field on the handler under a mutex):SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITIONwithout asking, with the same warning andwaf_backend_no_answer; one request a second still asks.SPEC.mdgets a "The circuit breaker" section, and the sentences sayingrproxykeeps nothing between requests are rewritten inSPEC.md,README.md,docs/settings.mdand the package doc.What the diff does not make obvious:
rproxycannot pass on (status below 200, body over 1 MiB) counts as an answer in time. A visitor who leaves mid-ask counts as neither.TestTheSiteAnswersWhileWAFIsStoppedAndStartedAgainininternal/swwnow polls for the ban afterwafrestarts: its visitor's failed asks open the breaker, which closes within a second oncewafanswers.export_test.go, so they never sleep.Model: opus-5-5
Verdict: FAIL
Head:
969383894cneeds-reworkinternal/rproxy/waf.go:106-111: no test covers the two exceptions inaskUnlessBreakerOpen: an ask the visitor abandons, and an answerrproxycannot pass on, must not count as failures. The first is what stops a visitor from opening the breaker for everyone by closing connections whilewafis being asked. Acceptable: handler tests ininternal/rproxy/rproxy_test.go, one per exception, where five such asks in a row leave the breaker closed and the next request still askswaf.docs/request-log.md:241: "disposition… is left out when:wafwas not asked, as for the health check" is now false. A request the open breaker keeps from askingwafhasdisposition. Acceptable: a bullet that names only requestsrproxyanswers itself, such as the health check.docs/settings.md:455-460: the touched sentence onSWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITIONruns to 37 words and chains four conditions with commas. Acceptable: a short sentence such as "what is done with a request whenwafgives no answer in time, and while the circuit breaker for asks towafis open."SPEC.md, "The circuit breaker":SPEC.md:9says each requirement is stated once, but this section restates "Without an answer fromwaf" just above it.SPEC.md:2734-2735) repeats the timeout case a third time. It then says "Such a request is a bug inwaf", but a request is not a bug, andSPEC.md:2697still says "A visitor can makewafslow" as expected behaviour.Acceptable: refer to the cases above instead of listing them again, and drop the closing paragraph. In "Without an answer from
waf", say once thatwaftaking longer thanSWW_RPROXY_WAF_BACKEND_TIMEOUT_MSto decide a request is a bug inwaf.Model: opus-5-5
969383894cto7ad4bb7426Head:
7ad4bb7426TestVisitorsThatGoAwayWhileWAFIsAskedLeaveTheBreakerClosedandTestAnswersThatCannotBePassedOnLeaveTheBreakerClosed, the second with a deny over 1 MiB.rproxyanswered the request itself, as for the health check".waf" states the bug once, in place of "A visitor can makewafslow".Model: opus-5-5
Verdict: FAIL
Head:
7ad4bb7426needs-reworkREADME.md:301, the settings table row forSWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION, still says the setting applies "whenwafcannot be reached or does not answer in time". It now also decides each request the open circuit breaker keeps from askingwaf, including whilewafis back up and the next ask has not yet closed the breaker.docs/settings.mdsays so, and the README row disagrees with it. Acceptable: the row says whatdocs/settings.mdsays, for example "Whatrproxydoes with a request whenwafgives no answer in time, and while the circuit breaker for asks towafis open:allowpasses it to the app,denyanswers503." Runmake fmtafterwards, which re-pads the table.Model: opus-5-5
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.