sneak/smallwebwaf
1
0 Fork 0
Code Issues 24 Pull Requests 6 Actions 70 Packages Projects Releases Wiki Activity

Is sww rproxy ready for production traffic? Test it end to end #321

New Issue
Open
opened 2026-11-11 02:51:15 +01:00 by clawbot · 1 comment
clawbot commented 2026-11-11 02:51:15 +01:00
Collaborator
Copy link
Copy source

sneak, chat 11 November 2026, about 02:00 UTC: "i expect to deploy it in the next couple days. is it fit for a beta yet? specifically, is the rproxy production ready? we can redeploy the waf repeatedly in production because the rproxy fails open by design for waf upgrades/downtime."

What to do

Answer whether sww rproxy (the reverse proxy; the waf mode is its decider) is ready to sit in front of real traffic, by using it, not by reading reviews. Build next fresh with the repo's own entrypoints, run sww rproxy in front of a small local web app in containers, with and without sww waf running, and check:

  • Fails open: with the waf stopped, crashed, restarting, slow or returning garbage, every request still reaches the app with no error and no long stall; measure the added delay while the waf is down.
  • Passes traffic correctly: methods, bodies (large uploads and downloads, streaming, chunked), query strings, headers (Host, cookies, X-Forwarded-* set correctly and spoofed ones from untrusted peers ignored, with the RFC 1918 trusted-proxy default), WebSockets, HTTP/1.1 keep-alive and HTTP/2 if supported, redirects, error codes passed through unchanged.
  • Survives: restarts with connections open, a backend that is down or slow, many concurrent clients (small, light load), memory and file descriptors steady over a run of some minutes.
  • Operates: startup fails loudly on a set but invalid config value; logs are useful and contain no secrets; the README's deploy instructions work as written.

Definition of done

  • Every defect filed as its own smallwebwaf issue, marked blocking (must fix before real traffic) or not.
  • One comment on this issue: a plain verdict (ready for a beta in front of real traffic, or not, and what must be fixed first), the issues filed, and anything in the README that turned out wrong.
  • No code changes in this issue.

Model: opus-5-5

sneak, chat 11 November 2026, about 02:00 UTC: "i expect to deploy it in the next couple days. is it fit for a beta yet? specifically, is the rproxy production ready? we can redeploy the waf repeatedly in production because the rproxy fails open by design for waf upgrades/downtime." ## What to do Answer whether `sww rproxy` (the reverse proxy; the `waf` mode is its decider) is ready to sit in front of real traffic, by using it, not by reading reviews. Build `next` fresh with the repo's own entrypoints, run `sww rproxy` in front of a small local web app in containers, with and without `sww waf` running, and check: - **Fails open:** with the waf stopped, crashed, restarting, slow or returning garbage, every request still reaches the app with no error and no long stall; measure the added delay while the waf is down. - **Passes traffic correctly:** methods, bodies (large uploads and downloads, streaming, chunked), query strings, headers (Host, cookies, `X-Forwarded-*` set correctly and spoofed ones from untrusted peers ignored, with the RFC 1918 trusted-proxy default), WebSockets, HTTP/1.1 keep-alive and HTTP/2 if supported, redirects, error codes passed through unchanged. - **Survives:** restarts with connections open, a backend that is down or slow, many concurrent clients (small, light load), memory and file descriptors steady over a run of some minutes. - **Operates:** startup fails loudly on a set but invalid config value; logs are useful and contain no secrets; the README's deploy instructions work as written. ## Definition of done - Every defect filed as its own smallwebwaf issue, marked blocking (must fix before real traffic) or not. - One comment on this issue: a plain verdict (ready for a beta in front of real traffic, or not, and what must be fixed first), the issues filed, and anything in the README that turned out wrong. - No code changes in this issue. Model: opus-5-5
clawbot commented 2026-11-11 04:11:06 +01:00
Author
Collaborator
Copy link
Copy source

Verdict: sww rproxy is fit for a beta in front of real traffic. Nothing must be fixed first.

Tested end to end on next at f7e3190, in containers, in front of a small web app, with and without sww waf. All four groups of checks in this issue hold, within the limits below.

The premise holds: waf can be redeployed under traffic. No request is lost; at each redeploy one request waits about 2 s.

Added delay while waf is down

  • waf stopped, killed, removed or restarting: none. The request reaches the app within a few milliseconds, as with waf up.
  • waf answering garbage, or breaking off its answer: none.
  • waf hung or slower than SWRP_TIMEOUT_MS, or its host unreachable: the full SWRP_TIMEOUT_MS, 2 s by default, on every request for as long as it lasts. rproxy keeps no memory that waf is down. Every request still reaches the app.
  • The request in progress at the moment waf stops waits up to SWRP_TIMEOUT_MS, once.

What failing open does not cover

These are the design as written, not defects. They are what to know before relying on it.

  • An answer from waf is obeyed, wrong or not. A new waf that denies too much blocks visitors until it is rolled back.
  • The same holds for anything else that answers at SWRP_WAF_BACKEND: its 404 or 5xx goes to the visitor.
  • rproxy itself. At a restart of rproxy, new connections are refused for about 2 to 7 s. Requests in progress get 5 s and are then cut, WebSockets, streams and long transfers included.

Before deploying

  • The defaults cut large or slow transfers. Raise them for a site with large files, such as Gitea or Nextcloud:
    • an upload over 100 MiB gets 413 (SWRP_MAX_REQUEST_SIZE);
    • an upload that takes over 60 s to send is cut (SWRP_MAX_REQUEST_TIME_MS);
    • a download over 1 GiB is cut (SWRP_MAX_RESPONSE_SIZE);
    • an answer that takes over 30 minutes is cut (SWRP_MAX_RESPONSE_TIME_MS).
  • If 2 s on every request is too long while waf hangs, lower SWRP_TIMEOUT_MS. Lower waf's SWW_LOOKUP_TIMEOUT (800 ms) with it: docs/settings.md says to keep it well under.
  • A misspelt setting name, such as SWRP_TIMEOUT, is ignored and the default applies. The starting line lists the settings in effect: read it after the first deploy.
  • rproxy takes HTTP/1.1 only, so HTTP/2 has to end at traefik.

Issues filed

None blocks real traffic.

  • Spoofed X-Forwarded-Scheme and X-Forwarded-Ssl from an untrusted peer reach the app: #325
  • The request log writes an ampersand in a query string as an escape sequence, so a text search misses it: #327
  • A comment in waf wrongly says a refused question still reaches the app: #326
  • An rproxy test fails on a busy host: #323
  • An sww test fails on a busy host: #324

Already known and not filed again: a clean stop of the container sometimes writes /usr/bin/runsvdir exited with error: waitid: no child processes. It is noise, left in place by the closed issue on image build warnings (#58).

README

Nothing turned out wrong. Followed as written: "Smallest setup", "Checking that it works", "rproxy in the app's container", "rproxy beside the app" and make example-app.

Disclosures

  • The host was under heavy load from other work, so the times are approximate.
  • Deviation: the README examples were followed with SWW_LOOKUP_SOURCE=off, and without the lookup database and ntfy, so that no address left the host.
  • Judgement call: the 2 s wait while waf hangs and the outage at a restart of rproxy are reported as design, not filed as defects.
  • Not checked, since the issue asked for a small local app and light load:
    • a real traefik in front, and TLS;
    • a real app such as Gitea behind it;
    • more than 6 minutes of steady load, or heavy load.

Model: fable-5-2

**Verdict: `sww rproxy` is fit for a beta in front of real traffic. Nothing must be fixed first.** Tested end to end on `next` at `f7e3190`, in containers, in front of a small web app, with and without `sww waf`. All four groups of checks in this issue hold, within the limits below. The premise holds: `waf` can be redeployed under traffic. No request is lost; at each redeploy one request waits about 2 s. ## Added delay while `waf` is down - `waf` stopped, killed, removed or restarting: none. The request reaches the app within a few milliseconds, as with `waf` up. - `waf` answering garbage, or breaking off its answer: none. - `waf` hung or slower than `SWRP_TIMEOUT_MS`, or its host unreachable: the full `SWRP_TIMEOUT_MS`, 2 s by default, on every request for as long as it lasts. `rproxy` keeps no memory that `waf` is down. Every request still reaches the app. - The request in progress at the moment `waf` stops waits up to `SWRP_TIMEOUT_MS`, once. ## What failing open does not cover These are the design as written, not defects. They are what to know before relying on it. - An answer from `waf` is obeyed, wrong or not. A new `waf` that denies too much blocks visitors until it is rolled back. - The same holds for anything else that answers at `SWRP_WAF_BACKEND`: its `404` or `5xx` goes to the visitor. - `rproxy` itself. At a restart of `rproxy`, new connections are refused for about 2 to 7 s. Requests in progress get 5 s and are then cut, WebSockets, streams and long transfers included. ## Before deploying - The defaults cut large or slow transfers. Raise them for a site with large files, such as Gitea or Nextcloud: - an upload over 100 MiB gets `413` (`SWRP_MAX_REQUEST_SIZE`); - an upload that takes over 60 s to send is cut (`SWRP_MAX_REQUEST_TIME_MS`); - a download over 1 GiB is cut (`SWRP_MAX_RESPONSE_SIZE`); - an answer that takes over 30 minutes is cut (`SWRP_MAX_RESPONSE_TIME_MS`). - If 2 s on every request is too long while `waf` hangs, lower `SWRP_TIMEOUT_MS`. Lower `waf`'s `SWW_LOOKUP_TIMEOUT` (800 ms) with it: `docs/settings.md` says to keep it well under. - A misspelt setting name, such as `SWRP_TIMEOUT`, is ignored and the default applies. The `starting` line lists the settings in effect: read it after the first deploy. - `rproxy` takes HTTP/1.1 only, so HTTP/2 has to end at traefik. ## Issues filed None blocks real traffic. - Spoofed `X-Forwarded-Scheme` and `X-Forwarded-Ssl` from an untrusted peer reach the app: https://git.eeqj.de/sneak/smallwebwaf/issues/325 - The request log writes an ampersand in a query string as an escape sequence, so a text search misses it: https://git.eeqj.de/sneak/smallwebwaf/issues/327 - A comment in `waf` wrongly says a refused question still reaches the app: https://git.eeqj.de/sneak/smallwebwaf/issues/326 - An `rproxy` test fails on a busy host: https://git.eeqj.de/sneak/smallwebwaf/issues/323 - An `sww` test fails on a busy host: https://git.eeqj.de/sneak/smallwebwaf/issues/324 Already known and not filed again: a clean stop of the container sometimes writes `/usr/bin/runsvdir exited with error: waitid: no child processes`. It is noise, left in place by the closed issue on image build warnings (https://git.eeqj.de/sneak/smallwebwaf/issues/58). ## README Nothing turned out wrong. Followed as written: "Smallest setup", "Checking that it works", "`rproxy` in the app's container", "`rproxy` beside the app" and `make example-app`. ## Disclosures - The host was under heavy load from other work, so the times are approximate. - Deviation: the README examples were followed with `SWW_LOOKUP_SOURCE=off`, and without the lookup database and ntfy, so that no address left the host. - Judgement call: the 2 s wait while `waf` hangs and the outage at a restart of `rproxy` are reported as design, not filed as defects. - Not checked, since the issue asked for a small local app and light load: - a real traefik in front, and TLS; - a real app such as Gitea behind it; - more than 6 minutes of steady load, or heavy load. Model: fable-5-2
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
critical
Extremely important: data loss, a security hole, production down, or basic use impossible
 merge-ready
Reviewed and ready for the owner to merge
 needs-checks
Checks must be fixed before review
 needs-rebase
Conflicts with next; rebase required
 needs-review
Review not yet done
 needs-rework
Review found issues to fix
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/smallwebwaf#321