Accept a chunked upstream body of exactly the maximum size (closes #248) #267

Merged
clawbot merged 2 commits from issue-248-chunked-body-at-limit into next 2026-10-15 07:44:09 +02:00
Collaborator

Fixes #248.

Once MaxResponseSize bytes were read, limitedReader.Read in internal/httpfetcher refused the next read without looking for the end of the body. Go's HTTP client returns io.EOF with the last bytes of a body sent with a Content-Length, so only a chunked body of exactly upstream_max_response_size bytes was refused; the image routes answered 500 after downloading it whole.

Once the limit is used up, Read now reads one more byte from the upstream body. If there is none, it returns the upstream body's error, io.EOF at its end, and the body is accepted. If there is one, it returns ErrResponseTooLarge.

Not visible in the diff: Go's client also returns the end of a chunked body together with its last data when that end has already arrived, so a chunked test would pass by luck. The new chunked test's upstream sends the end of the body only after the test has read MaxResponseSize bytes, so its first commit fails as the issue describes. The Content-Length test passes before and after the fix.

Judgement call: once a body is found over the limit, remaining goes below 0 and every later Read returns ErrResponseTooLarge without reading on, as the old code did once the limit was reached.

Model: opus-5-5

Fixes https://git.eeqj.de/sneak/pixa/issues/248. Once `MaxResponseSize` bytes were read, `limitedReader.Read` in `internal/httpfetcher` refused the next read without looking for the end of the body. Go's HTTP client returns `io.EOF` with the last bytes of a body sent with a `Content-Length`, so only a chunked body of exactly `upstream_max_response_size` bytes was refused; the image routes answered 500 after downloading it whole. Once the limit is used up, `Read` now reads one more byte from the upstream body. If there is none, it returns the upstream body's error, `io.EOF` at its end, and the body is accepted. If there is one, it returns `ErrResponseTooLarge`. Not visible in the diff: Go's client also returns the end of a chunked body together with its last data when that end has already arrived, so a chunked test would pass by luck. The new chunked test's upstream sends the end of the body only after the test has read `MaxResponseSize` bytes, so its first commit fails as the issue describes. The `Content-Length` test passes before and after the fix. Judgement call: once a body is found over the limit, `remaining` goes below 0 and every later `Read` returns `ErrResponseTooLarge` without reading on, as the old code did once the limit was reached. Model: opus-5-5
clawbot added the needs-review label 2026-10-15 05:32:43 +02:00
clawbot self-assigned this 2026-10-15 05:32:43 +02:00
Author
Collaborator

Review passed.

Model: opus-5-5

Review passed. Model: opus-5-5
clawbot added 2 commits 2026-10-15 07:09:34 +02:00
A chunked body of exactly MaxResponseSize bytes is refused with
ErrResponseTooLarge; this test fails until the fix. The upstream sends
the end of the chunked body only after the test has read that many
bytes, so the read that finds the limit used up comes first. A body
sent with a Content-Length, and a body one byte over the limit under
either framing, already behave as intended.

Model: opus-5-5
Once MaxResponseSize bytes were read, limitedReader.Read refused the
next read without looking for the end of the body. A body sent with a
Content-Length gets io.EOF with its last bytes, but the end of a
chunked body can come in a later read, so a chunked body of exactly
the limit was refused. Once the limit is used up, Read now reads one
more byte: at the end of the body it returns io.EOF, and with more
data it returns ErrResponseTooLarge, as does every later Read.

Model: opus-5-5
clawbot force-pushed issue-248-chunked-body-at-limit from 8ac33d47e0 to 3afc70999b 2026-10-15 07:09:34 +02:00 Compare
clawbot merged commit 761ade2e36 into next 2026-10-15 07:44:09 +02:00
clawbot deleted branch issue-248-chunked-body-at-limit 2026-10-15 07:44:09 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/pixa#267