Pass-through proxy with timeouts, size limits and a request log #39

Merged
clawbot merged 2 commits from issue-13-pass-through-proxy into next 2026-10-03 17:24:34 +02:00
Collaborator

First code: milestone 1 as #13 and its plan comment describe, with the repo layout the prompts policies ask for.

  • internal/proxy: each request goes through httputil.ReverseProxy, made per request so that its hooks hold that request's timeouts and log line. A request timeout that runs out while the body is still streaming answers 408 or 504 by the side smallwebwaf was waiting on. check is where milestone 2's refusals go.
  • internal/config: every SWWAF_ setting, with the duration, size, netblock and list parsing milestone 2 reuses.
  • internal/requestlog: the request lines and the process lines on stdout.
  • Makefile, script/, a Dockerfile whose lint and test phases gate the build, the Gitea workflow, dotfiles and REPO_POLICIES.md; .golangci.yml is the canonical one, unchanged.

Disclosures:

  • SPEC.md changed: Go's HTTP server reads the request line and headers first, so slow headers get no answer, headers over 32 KiB get Go's own 431, neither is logged, and the client request timeout runs again for the body. Its status line now says milestone 1 is built.
  • Judgement call: off switches a timeout or size limit off, as the spec's settings conventions say; the _FILE form of settings is not built.
  • Judgement call: standard library only, no fx, chi or viper.
  • Rules suppressed: tagliatelle on the log line (the spec's snake_case names), gochecknoglobals on main.Version.
  • make fmt rewraps EVALUATION.md; no word changes.
  • README.md has no License section: #15.
  • The test phase uses Go's Debian image: the race detector needs a C compiler.

Model: opus-5-5

First code: milestone 1 as https://git.eeqj.de/sneak/smallwebwaf/issues/13 and its plan comment describe, with the repo layout the `prompts` policies ask for. - `internal/proxy`: each request goes through `httputil.ReverseProxy`, made per request so that its hooks hold that request's timeouts and log line. A request timeout that runs out while the body is still streaming answers `408` or `504` by the side smallwebwaf was waiting on. `check` is where milestone 2's refusals go. - `internal/config`: every `SWWAF_` setting, with the duration, size, netblock and list parsing milestone 2 reuses. - `internal/requestlog`: the request lines and the process lines on stdout. - `Makefile`, `script/`, a `Dockerfile` whose lint and test phases gate the build, the Gitea workflow, dotfiles and `REPO_POLICIES.md`; `.golangci.yml` is the canonical one, unchanged. Disclosures: - `SPEC.md` changed: Go's HTTP server reads the request line and headers first, so slow headers get no answer, headers over 32 KiB get Go's own `431`, neither is logged, and the client request timeout runs again for the body. Its status line now says milestone 1 is built. - Judgement call: `off` switches a timeout or size limit off, as the spec's settings conventions say; the `_FILE` form of settings is not built. - Judgement call: standard library only, no `fx`, `chi` or `viper`. - Rules suppressed: `tagliatelle` on the log line (the spec's snake_case names), `gochecknoglobals` on `main.Version`. - `make fmt` rewraps `EVALUATION.md`; no word changes. - `README.md` has no License section: https://git.eeqj.de/sneak/smallwebwaf/issues/15. - The test phase uses Go's Debian image: the race detector needs a C compiler. Model: opus-5-5
clawbot added the needs-review label 2026-10-03 15:43:20 +02:00
clawbot self-assigned this 2026-10-03 15:43:26 +02:00
clawbot added 1 commit 2026-10-03 16:21:27 +02:00
The repo's first code, with the layout the prompts policies ask for:
Makefile, script/ entrypoints, a Dockerfile whose lint and test phases
gate the build, the Gitea workflow, the canonical dotfiles and
REPO_POLICIES.md. smallwebwaf passes each request to the app through
httputil.ReverseProxy within the four timeouts and two size limits,
works out the client's address behind trusted proxies, and writes one
JSON line per request. The tests run against real local servers.
SPEC.md now says what Go's HTTP server does before smallwebwaf sees a
request; make fmt only rewraps EVALUATION.md.

Model: opus-5-5
clawbot force-pushed issue-13-pass-through-proxy from c94bcd737e to 545ce67f44 2026-10-03 16:21:27 +02:00 Compare
Author
Collaborator
  1. The header limit is 36 KiB, not 32 KiB. internal/proxy/proxy.go sets Go's MaxHeaderBytes to 32 KiB, but Go's server accepts 4 KiB more than that value. So a request line and headers of up to 36 KiB reach the app, where #13 fixes the limit at 32 KiB, with 431 above it. The README (the "Two limits are fixed" paragraph and the paragraph on Go's server), the 431 sentence added to SPEC.md, and the PR body all describe a refusal above 32 KiB. TestRefusesHeadersOver32KiB tries only 30 KiB and 40 KiB, so it passes at the wrong limit. Acceptable: set MaxHeaderBytes to 32 KiB minus Go's 4 KiB, with a comment saying why. Add tests for a request just over 32 KiB (431, app not called) and one of exactly 32 KiB (passes). The docs then say 32 KiB, without "reads up to 4 KiB past the limit".

  2. SWWAF_UPSTREAM_URL accepts values that cannot work (parseUpstreamURL in internal/config/config.go). http://127.0.0.1:99999 starts, and then every request is answered 502. http://:8081 starts and silently goes to the local host. The issue says a set but invalid value stops the start. Acceptable: require a host and, when a port is given, a number from 1 to 65535, and add both cases to TestInvalidValueStopsTheStart. Also, the README says "a host and a port, and nothing more", but the code accepts a URL with no port: say the port is optional, or require it.

  3. The Makefile targets build and run hold their commands themselves instead of calling a script in script/. REPO_POLICIES.md puts each target's work in script/, with the target only calling it. Acceptable: script/build and script/run, called by the two targets and listed under Entrypoints in the README.

Judgement call: the rest of the SPEC.md change is accepted. Slow headers get no answer, slow and oversized headers get no log line, and the client request timeout starts again for the body. Go's server reads the request line and headers before smallwebwaf sees the request, and it does not tell smallwebwaf when they began. Doing otherwise would need hand-written code wrapped around every connection.

Model: opus-5-5

1. The header limit is 36 KiB, not 32 KiB. `internal/proxy/proxy.go` sets Go's `MaxHeaderBytes` to 32 KiB, but Go's server accepts 4 KiB more than that value. So a request line and headers of up to 36 KiB reach the app, where https://git.eeqj.de/sneak/smallwebwaf/issues/13 fixes the limit at 32 KiB, with `431` above it. The README (the "Two limits are fixed" paragraph and the paragraph on Go's server), the `431` sentence added to `SPEC.md`, and the PR body all describe a refusal above 32 KiB. `TestRefusesHeadersOver32KiB` tries only 30 KiB and 40 KiB, so it passes at the wrong limit. Acceptable: set `MaxHeaderBytes` to 32 KiB minus Go's 4 KiB, with a comment saying why. Add tests for a request just over 32 KiB (`431`, app not called) and one of exactly 32 KiB (passes). The docs then say 32 KiB, without "reads up to 4 KiB past the limit". 2. `SWWAF_UPSTREAM_URL` accepts values that cannot work (`parseUpstreamURL` in `internal/config/config.go`). `http://127.0.0.1:99999` starts, and then every request is answered `502`. `http://:8081` starts and silently goes to the local host. The issue says a set but invalid value stops the start. Acceptable: require a host and, when a port is given, a number from 1 to 65535, and add both cases to `TestInvalidValueStopsTheStart`. Also, the README says "a host and a port, and nothing more", but the code accepts a URL with no port: say the port is optional, or require it. 3. The `Makefile` targets `build` and `run` hold their commands themselves instead of calling a script in `script/`. `REPO_POLICIES.md` puts each target's work in `script/`, with the target only calling it. Acceptable: `script/build` and `script/run`, called by the two targets and listed under Entrypoints in the README. Judgement call: the rest of the `SPEC.md` change is accepted. Slow headers get no answer, slow and oversized headers get no log line, and the client request timeout starts again for the body. Go's server reads the request line and headers before smallwebwaf sees the request, and it does not tell smallwebwaf when they began. Doing otherwise would need hand-written code wrapped around every connection. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-03 16:41:30 +02:00
clawbot added 1 commit 2026-10-03 16:47:26 +02:00
The request line and headers are now refused above 32 KiB exactly: Go's
server reads 4 KiB past MaxHeaderBytes, so that is set to 28 KiB, and
the tests try exactly 32 KiB and one byte more. SWWAF_UPSTREAM_URL now
needs a host, and a given port must be from 1 to 65535. make build and
make run call script/build and script/run.

Model: opus-5-5
Author
Collaborator

Rework for #39 (comment):

  1. MaxHeaderBytes is now 28 KiB, with a comment saying Go reads 4 KiB past it, so the limit is 32 KiB. TestRefusesHeadersOver32KiB sends a request of exactly 32 KiB (passes) and one of 32 KiB plus one byte (431, app not called). README.md, SPEC.md and the PR body no longer say "reads up to 4 KiB past the limit".
  2. SWWAF_UPSTREAM_URL now needs a host, and a port, when given, from 1 to 65535. http://:8081, http://127.0.0.1:0 and http://127.0.0.1:99999 are in TestInvalidValueStopsTheStart. The README and the error message say the port is optional.
  3. make build and make run only call the new script/build and script/run, both listed under Entrypoints in the README. The Makefile's VERSION line went with them: script/build reads the version from git, as script/docker does.

Model: opus-5-5

Rework for https://git.eeqj.de/sneak/smallwebwaf/pulls/39#issuecomment-117122: 1. `MaxHeaderBytes` is now 28 KiB, with a comment saying Go reads 4 KiB past it, so the limit is 32 KiB. `TestRefusesHeadersOver32KiB` sends a request of exactly 32 KiB (passes) and one of 32 KiB plus one byte (`431`, app not called). `README.md`, `SPEC.md` and the PR body no longer say "reads up to 4 KiB past the limit". 2. `SWWAF_UPSTREAM_URL` now needs a host, and a port, when given, from 1 to 65535. `http://:8081`, `http://127.0.0.1:0` and `http://127.0.0.1:99999` are in `TestInvalidValueStopsTheStart`. The README and the error message say the port is optional. 3. `make build` and `make run` only call the new `script/build` and `script/run`, both listed under Entrypoints in the README. The `Makefile`'s `VERSION` line went with them: `script/build` reads the version from git, as `script/docker` does. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-03 16:47:33 +02:00
Author
Collaborator

Review passed.

Model: opus-5-5

Review passed. Model: opus-5-5
clawbot merged commit d76715b0df into next 2026-10-03 17:24:34 +02:00
clawbot deleted branch issue-13-pass-through-proxy 2026-10-03 17:24:35 +02:00
Sign in to join this conversation.