The event-log page renders stored bodies untruncated, so buffered rendering can hold ~25 MB per request #135

New Issue

clawbot · 2026-08-12T11:56:52+02:00

clawbot commented

2026-08-12 11:56:52 +02:00

Found in the independent review of #131. Not milestoned 1.0.0: the page is authenticated and the size is bounded and paginated, so by the tag's blocking rule it belongs to the next cycle. It is filed because #131 changed the memory profile of this page and that change should not go unrecorded.

templates/source_logs.html:39 renders {{.Body}} untruncated. With paginationPerPage = 25 (internal/handlers/handlers.go:31) and the 1 MB ingest cap (internal/handlers/webhook.go:17), one page can carry ~25 MB of raw bodies — before HTML escaping, which inflates adversarial payloads several-fold (html/template escapes + to &#43; and similar).

Before #131 this streamed to the socket, so the peak was small. That PR buffers the whole page in memory to fix partial-page renders (#123) — the right trade, but it converts this page's size from a bandwidth cost into a resident-memory cost of up to tens of MB per concurrent viewer.

REPO_POLICIES.md:297 asks for a maximum response size where applicable, which this page arguably is.

Definition of done

The event-log page truncates each rendered body to a fixed cap (8 KB is a reasonable starting point) with a visible "truncated" marker, so page size is bounded by the cap rather than by stored payload size.
The full body remains retrievable — truncation is a rendering concern, not a data-loss one. Say in the PR where the untruncated value is still reachable, or file the follow-up if it is not.
A test asserting the cap holds for a stored body far above it.

Implementation requirements

Branch from next, PR based on next, single commit, title ending (closes #N).
Do not modify TODO.md (see #112).
Gate on make check plus the Docker lint path with the cache defeated (#119).

Found in the independent review of https://git.eeqj.de/sneak/webhooker/pulls/131. Not milestoned 1.0.0: the page is authenticated and the size is bounded and paginated, so by the tag's blocking rule it belongs to the next cycle. It is filed because https://git.eeqj.de/sneak/webhooker/pulls/131 changed the memory profile of this page and that change should not go unrecorded. `templates/source_logs.html:39` renders `{{.Body}}` untruncated. With `paginationPerPage = 25` (`internal/handlers/handlers.go:31`) and the 1 MB ingest cap (`internal/handlers/webhook.go:17`), one page can carry ~25 MB of raw bodies — before HTML escaping, which inflates adversarial payloads several-fold (`html/template` escapes `+` to `&#43;` and similar). Before https://git.eeqj.de/sneak/webhooker/pulls/131 this streamed to the socket, so the peak was small. That PR buffers the whole page in memory to fix partial-page renders (https://git.eeqj.de/sneak/webhooker/issues/123) — the right trade, but it converts this page's size from a bandwidth cost into a resident-memory cost of up to tens of MB per concurrent viewer. `REPO_POLICIES.md:297` asks for a maximum response size where applicable, which this page arguably is. ## Definition of done - The event-log page truncates each rendered body to a fixed cap (8 KB is a reasonable starting point) with a visible "truncated" marker, so page size is bounded by the cap rather than by stored payload size. - The full body remains retrievable — truncation is a rendering concern, not a data-loss one. Say in the PR where the untruncated value is still reachable, or file the follow-up if it is not. - A test asserting the cap holds for a stored body far above it. ## Implementation requirements - Branch from `next`, PR based on `next`, single commit, title ending ` (closes #N)`. - Do not modify `TODO.md` (see https://git.eeqj.de/sneak/webhooker/issues/112). - Gate on `make check` plus the Docker lint path with the cache defeated (https://git.eeqj.de/sneak/webhooker/issues/119).

clawbot self-assigned this 2026-08-12 11:56:52 +02:00

clawbot referenced this issue

2026-08-14 06:17:38 +02:00

Render templates via a buffer, not the ResponseWriter (closes #123) #131

clawbot commented

2026-08-14 06:22:00 +02:00

Plan: bound the body in the query layer, not the template.

loadEventsWithDeliveries will stop loading []database.Event and instead select a projection whose body column is substr(cast(body as blob), 1, 8192), plus length(cast(body as blob)) for the real size. The cast makes both byte-wise rather than character-wise, so the bound is bytes. A stored body over the cap therefore never becomes a Go string at all — the buffered render is bounded by cap x page size regardless of what was ingested, which template-side truncation would not achieve (it would still materialise the full string, and truncating post-escape would be worse still).

SQLite cuts at an arbitrary byte, so the tail can be a partial UTF-8 sequence. A helper walks back at most utf8.UTFMax bytes to the last utf8.RuneStart byte and drops that sequence if utf8.FullRune says it is incomplete; bytes that are merely invalid UTF-8 (a binary payload) are left alone.

Events become EventLogView alongside the existing DeliveryView/TargetView projections, carrying BodyTruncated and BodyBytes so templates/source_logs.html can show a visible marker with the real byte count.

Also corrects the now-false "these pages are small" comment at internal/handlers/handlers.go:232.

Plan: bound the body in the query layer, not the template. `loadEventsWithDeliveries` will stop loading `[]database.Event` and instead select a projection whose body column is `substr(cast(body as blob), 1, 8192)`, plus `length(cast(body as blob))` for the real size. The cast makes both byte-wise rather than character-wise, so the bound is bytes. A stored body over the cap therefore never becomes a Go string at all — the buffered render is bounded by cap x page size regardless of what was ingested, which template-side truncation would not achieve (it would still materialise the full string, and truncating post-escape would be worse still). SQLite cuts at an arbitrary byte, so the tail can be a partial UTF-8 sequence. A helper walks back at most `utf8.UTFMax` bytes to the last `utf8.RuneStart` byte and drops that sequence if `utf8.FullRune` says it is incomplete; bytes that are merely invalid UTF-8 (a binary payload) are left alone. Events become `EventLogView` alongside the existing `DeliveryView`/`TargetView` projections, carrying `BodyTruncated` and `BodyBytes` so `templates/source_logs.html` can show a visible marker with the real byte count. Also corrects the now-false "these pages are small" comment at `internal/handlers/handlers.go:232`.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sneak/webhooker#135