"i was under the impression that rproxy sends the waf the request from the client nearly as-is (perhaps body truncated at max 1MB or something, as more than that isn't needed for deciding), with just a few headers added for the actual connecting ip and potentially a tls fingerprint later. rproxy should drop connections when headers are >32KB, unconfigurable. nuanced decisions are for the waf, rproxy just needs basic sanity hardcoded in."
What to build:
rproxy drops the connection when a request's headers total more than 32 KB. The limit is a constant, not a setting; remove any setting that configures it today.
Check what rproxy sends waf against the model above: the client's request nearly as-is, the body cut at about 1 MB, plus only the headers for the connecting address (and later the TLS fingerprint). Where the code differs, post the difference on this issue before changing anything beyond the header limit; sneak decides those.
Definition of done: the 32 KB limit is in, with a test of a request just under and just over it; the comparison with sneak's model is posted here; make check passes.
Model: opus-5-5
sneak on https://git.eeqj.de/sneak/smallwebwaf/issues/283#issuecomment-183070:
"i was under the impression that rproxy sends the waf the request from the client nearly as-is (perhaps body truncated at max 1MB or something, as more than that isn't needed for deciding), with just a few headers added for the actual connecting ip and potentially a tls fingerprint later. rproxy should drop connections when headers are >32KB, unconfigurable. nuanced decisions are for the waf, rproxy just needs basic sanity hardcoded in."
What to build:
- `rproxy` drops the connection when a request's headers total more than 32 KB. The limit is a constant, not a setting; remove any setting that configures it today.
- Check what `rproxy` sends `waf` against the model above: the client's request nearly as-is, the body cut at about 1 MB, plus only the headers for the connecting address (and later the TLS fingerprint). Where the code differs, post the difference on this issue before changing anything beyond the header limit; sneak decides those.
Definition of done: the 32 KB limit is in, with a test of a request just under and just over it; the comparison with sneak's model is posted here; `make check` passes.
Model: opus-5-5
How rproxy differs today from sneak's model: the client's request nearly as-is, the body cut at about 1 MB, plus only the connecting address (and later a TLS fingerprint). Nothing below is changed; each is sneak's to decide.
Body:rproxy sends waf the first 64 KiB of the body, not about 1 MB.
Added headers: besides X-Sww-Client (the connecting address, worked out through SWW_RPROXY_TRUSTED_PROXIES), rproxy sets four more, replacing any the visitor sent:
X-Sww-Peer: its own TCP peer's address, for the log line's peer_ip;
X-Sww-Content-Length: the body length the visitor announced, since the question's own Content-Length is that of the part sent;
X-Request-ID: a fresh id for each request;
X-Forwarded-Proto: the scheme.
A second request: after each answered request, rproxy posts waf a report (POST /_sww/report) with the status the visitor got, the bytes each way and the times.
Refusals without asking waf:rproxy answers these itself:
405 to a method outside its list of 18 (TRACE, for one);
400 to a request Go's client cannot resend as sent: a target not starting with one /, an empty Host, a Host with an IPv6 zone, or more than one User-Agent;
its own health check.
Header changes by Go: an empty User-Agent is not sent, and Pragma: no-cache without Cache-Control gains Cache-Control: no-cache.
Settings, not hard-coded:rproxy's request and response size limits and time limits are settings (SWW_RPROXY_MAX_REQUEST_SIZE, SWW_RPROXY_MAX_RESPONSE_SIZE, SWW_RPROXY_MAX_REQUEST_TIME_MS, SWW_RPROXY_MAX_RESPONSE_TIME_MS), as are the wait for waf and what happens without its answer.
Over 32 KiB: Go's server answers 431 and then closes the connection; it does not close it silently. The PR keeps that.
Model: opus-5-5
How `rproxy` differs today from sneak's model: the client's request nearly as-is, the body cut at about 1 MB, plus only the connecting address (and later a TLS fingerprint). Nothing below is changed; each is sneak's to decide.
- **Body:** `rproxy` sends `waf` the first 64 KiB of the body, not about 1 MB.
- **Added headers:** besides `X-Sww-Client` (the connecting address, worked out through `SWW_RPROXY_TRUSTED_PROXIES`), `rproxy` sets four more, replacing any the visitor sent:
- `X-Sww-Peer`: its own TCP peer's address, for the log line's `peer_ip`;
- `X-Sww-Content-Length`: the body length the visitor announced, since the question's own `Content-Length` is that of the part sent;
- `X-Request-ID`: a fresh id for each request;
- `X-Forwarded-Proto`: the scheme.
- **A second request:** after each answered request, `rproxy` posts `waf` a report (`POST /_sww/report`) with the status the visitor got, the bytes each way and the times.
- **Refusals without asking `waf`:** `rproxy` answers these itself:
- `405` to a method outside its list of 18 (`TRACE`, for one);
- `400` to a request Go's client cannot resend as sent: a target not starting with one `/`, an empty `Host`, a `Host` with an IPv6 zone, or more than one `User-Agent`;
- its own health check.
- **Header changes by Go:** an empty `User-Agent` is not sent, and `Pragma: no-cache` without `Cache-Control` gains `Cache-Control: no-cache`.
- **Settings, not hard-coded:** `rproxy`'s request and response size limits and time limits are settings (`SWW_RPROXY_MAX_REQUEST_SIZE`, `SWW_RPROXY_MAX_RESPONSE_SIZE`, `SWW_RPROXY_MAX_REQUEST_TIME_MS`, `SWW_RPROXY_MAX_RESPONSE_TIME_MS`), as are the wait for `waf` and what happens without its answer.
- **Over 32 KiB:** Go's server answers `431` and then closes the connection; it does not close it silently. The PR keeps that.
Model: opus-5-5
PR: #419, branch issue-415-header-limit-test. The 32 KiB limit was already a constant with no setting; the PR adds the just-under and just-over tests.
Judgement call: Go's server answers 431 and then closes the connection, rather than closing it with no answer; the PR keeps that.
Model: opus-5-5
PR: https://git.eeqj.de/sneak/smallwebwaf/pulls/419, branch `issue-415-header-limit-test`. The 32 KiB limit was already a constant with no setting; the PR adds the just-under and just-over tests.
Judgement call: Go's server answers `431` and then closes the connection, rather than closing it with no answer; the PR keeps that.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
sneak on #283 (comment):
"i was under the impression that rproxy sends the waf the request from the client nearly as-is (perhaps body truncated at max 1MB or something, as more than that isn't needed for deciding), with just a few headers added for the actual connecting ip and potentially a tls fingerprint later. rproxy should drop connections when headers are >32KB, unconfigurable. nuanced decisions are for the waf, rproxy just needs basic sanity hardcoded in."
What to build:
rproxydrops the connection when a request's headers total more than 32 KB. The limit is a constant, not a setting; remove any setting that configures it today.rproxysendswafagainst the model above: the client's request nearly as-is, the body cut at about 1 MB, plus only the headers for the connecting address (and later the TLS fingerprint). Where the code differs, post the difference on this issue before changing anything beyond the header limit; sneak decides those.Definition of done: the 32 KB limit is in, with a test of a request just under and just over it; the comparison with sneak's model is posted here;
make checkpasses.Model: opus-5-5
How
rproxydiffers today from sneak's model: the client's request nearly as-is, the body cut at about 1 MB, plus only the connecting address (and later a TLS fingerprint). Nothing below is changed; each is sneak's to decide.rproxysendswafthe first 64 KiB of the body, not about 1 MB.X-Sww-Client(the connecting address, worked out throughSWW_RPROXY_TRUSTED_PROXIES),rproxysets four more, replacing any the visitor sent:X-Sww-Peer: its own TCP peer's address, for the log line'speer_ip;X-Sww-Content-Length: the body length the visitor announced, since the question's ownContent-Lengthis that of the part sent;X-Request-ID: a fresh id for each request;X-Forwarded-Proto: the scheme.rproxypostswafa report (POST /_sww/report) with the status the visitor got, the bytes each way and the times.waf:rproxyanswers these itself:405to a method outside its list of 18 (TRACE, for one);400to a request Go's client cannot resend as sent: a target not starting with one/, an emptyHost, aHostwith an IPv6 zone, or more than oneUser-Agent;User-Agentis not sent, andPragma: no-cachewithoutCache-ControlgainsCache-Control: no-cache.rproxy's request and response size limits and time limits are settings (SWW_RPROXY_MAX_REQUEST_SIZE,SWW_RPROXY_MAX_RESPONSE_SIZE,SWW_RPROXY_MAX_REQUEST_TIME_MS,SWW_RPROXY_MAX_RESPONSE_TIME_MS), as are the wait forwafand what happens without its answer.431and then closes the connection; it does not close it silently. The PR keeps that.Model: opus-5-5
PR: #419, branch
issue-415-header-limit-test. The 32 KiB limit was already a constant with no setting; the PR adds the just-under and just-over tests.Judgement call: Go's server answers
431and then closes the connection, rather than closing it with no answer; the PR keeps that.Model: opus-5-5
64k body is fine, 1mb was just chosen out of thin air, if 64kb is sufficient then 64kb is sufficient
Recorded:
rproxykeeps sendingwafthe first 64 KB of the body, as it does today; the 1 MB in the issue was not a requirement.Model: opus-5-5