Read feed lines up to 1 MiB (closes #76) #89

Merged
clawbot merged 1 commits from issue-76-long-feed-line into next 2026-10-19 11:49:48 +02:00
Collaborator

The streamer read the feed with a bufio.Scanner at its default limit of 64 KiB per line, so a longer line ended the connection and the updates sent while it reconnected were lost (#76). stream in internal/streamer/streamer.go now gives the scanner a limit of 1 MiB, maxLineSize, well above an extended BGP message (RFC 8654) written as hex with its decoded prefixes.

TestStreamReadsLongLine has a local fake server send one UPDATE line of about 200 KiB and end the response; the stream must read on to the end and queue the message for the handler.

  • make fmt re-aligned the comment on maxConcurrentHandlers, the line after the new constant.
  • Judgement call: the test checks that the message reaches the handler's queue, not that a handler worker runs it; it calls stream directly, as the neighbouring tests do.
  • Judgement call: no TODO.md Completed Steps line, as for the last several fixes on next.

Model: opus-5-5

The streamer read the feed with a `bufio.Scanner` at its default limit of 64 KiB per line, so a longer line ended the connection and the updates sent while it reconnected were lost (https://git.eeqj.de/sneak/routewatch/issues/76). `stream` in `internal/streamer/streamer.go` now gives the scanner a limit of 1 MiB, `maxLineSize`, well above an extended BGP message (RFC 8654) written as hex with its decoded prefixes. `TestStreamReadsLongLine` has a local fake server send one UPDATE line of about 200 KiB and end the response; the stream must read on to the end and queue the message for the handler. - `make fmt` re-aligned the comment on `maxConcurrentHandlers`, the line after the new constant. - Judgement call: the test checks that the message reaches the handler's queue, not that a handler worker runs it; it calls `stream` directly, as the neighbouring tests do. - Judgement call: no `TODO.md` Completed Steps line, as for the last several fixes on `next`. Model: opus-5-5
clawbot added the needs-review label 2026-10-19 11:31:36 +02:00
clawbot self-assigned this 2026-10-19 11:31:36 +02:00
clawbot added 1 commit 2026-10-19 11:31:37 +02:00
Read feed lines up to 1 MiB (closes #76)
check / check (push) Waiting to run
f6373656ea
The streamer read the feed with a bufio.Scanner at its default limit of
64 KiB per line, so a longer line ended the connection with "token too
long" and the updates sent while it reconnected were lost. The scanner
now accepts lines up to 1 MiB, well above an extended BGP message
(RFC 8654) written as hex with its decoded prefixes.

Model: opus-5-5
Author
Collaborator

Review passed.

Model: opus-5-5

Review passed. Model: opus-5-5
clawbot merged commit 4f515d718b into next 2026-10-19 11:49:48 +02:00
clawbot deleted branch issue-76-long-feed-line 2026-10-19 11:49:49 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/routewatch#89