Compare commits

..
1 Commits
Author SHA1 Message Date
sneak 49d2f4ede8 Drop an update only when its handler's queue is full (closes #229)
check / check (push) Successful in 4m0s
The streamer dropped a random share of a handler's updates once its
queue was half full. When the feed paused and then sent its backlog
while the routing table's Overview walked every prefix, the prefix
handler lost about 10,000 updates with its queue never full.

The streamer now drops an update only when the queue is full. The
prefix handler's queue holds 100,000 messages, about 100 MiB, so it
can hold such a backlog. The README's Memory and Feed handling
sections give the new sizes and the rule.

Model: opus-5-5
2026-11-03 17:02:20 +00:00
+9 -9
View File
@@ -120,8 +120,8 @@ type handlerInfo struct {
// Streamer manages a connection to the RIPE RIS Live streaming API for receiving // Streamer manages a connection to the RIPE RIS Live streaming API for receiving
// real-time BGP UPDATE messages. It handles automatic reconnection with exponential // real-time BGP UPDATE messages. It handles automatic reconnection with exponential
// backoff, and dispatches messages to registered handlers via per-handler queues. // backoff and dispatches messages to registered handlers via per-handler queues.
// It never waits for room on a queue (see queueMessage). // Its read loop never waits for room on a queue (see queueMessage).
type Streamer struct { type Streamer struct {
logger *logger.Logger logger *logger.Logger
client *http.Client client *http.Client
@@ -882,13 +882,13 @@ func (s *Streamer) dispatchMessage(
// msg instead. It never waits for room, so a full queue does not hold up the // msg instead. It never waits for room, so a full queue does not hold up the
// read loop. // read loop.
// //
// A message that finds the queue full is dropped, except a STATE message: one // A message that finds the queue full is dropped, except a STATE message. A
// whose state is down is the only notice that all of a peer's routes are gone. // STATE message whose state is down is the only notice that all of a peer's
// A STATE message that finds the queue full is held in info.heldState. Held // routes are gone. A STATE message that finds the queue full is held in
// messages go into the queue, oldest first, before anything else for the // info.heldState instead. Held messages go into the queue, oldest first, when a
// handler, as soon as there is room, or when Stop is called; while any is // later message for the handler finds room, or when Stop is called. While any
// still held, the handler's other messages are dropped, so none of them // message is held, the handler's other messages are dropped, so none of them
// overtakes a held down. // overtakes a held STATE message.
// //
// The held list stays small: a STATE message is a few hundred bytes and the // The held list stays small: a STATE message is a few hundred bytes and the
// feed sends about ten a second, so a queue full for a minute holds about 600. // feed sends about ten a second, so a queue full for a minute holds about 600.