Routes of a peer whose BGP session goes down are never removed #56

Closed
opened 2026-10-14 14:33:44 +02:00 by clawbot · 2 comments
Collaborator

When a RIS collector loses its BGP session with a peer, every route learned from that peer is gone, and RIS sends no withdrawals for them. routewatch keeps them.

Where: internal/streamer/streamer.go, stream: messages of type STATE (and RIS_PEER_STATE) are skipped, and NOTIFICATION only changes the peer count. ristypes.RISMessage does not decode the state field. The feed sends, for example (docs/message-examples.json):

{"type":"ris_message","data":{"peer":"196.60.9.141","peer_asn":"328214","host":"rrc19.ripe.net","type":"STATE","state":"down",...}}

Failure scenario: a peer at an exchange point goes down for an hour. Its few hundred thousand routes stay in the live tables; IP lookups and prefix pages keep listing routes through a peer that has none, and when the session comes back the peer only re-sends what it announces then, so routes it no longer has are never cleaned up.

Severity: high

Definition of done

  • A STATE message with state down from a peer removes all of that peer's live routes, IPv4 and IPv6, with the in-memory counts and prefix distribution kept exact. The removal goes through the prefix handler's batch, in feed order with that peer's updates, so an update the peer sends after the session comes back is not lost.
  • Routes are kept per peer IP, as now; if the same peer IP has sessions on two collectors, a down on one removes routes the other still carries until they are re-announced. Say so in one line in the README.
  • Tests that fail before the fix: a peer's routes are gone after its down; another peer's routes for the same prefixes stay.
  • make check passes.

Model: opus-5-5

When a RIS collector loses its BGP session with a peer, every route learned from that peer is gone, and RIS sends no withdrawals for them. routewatch keeps them. Where: `internal/streamer/streamer.go`, `stream`: messages of type `STATE` (and `RIS_PEER_STATE`) are skipped, and `NOTIFICATION` only changes the peer count. `ristypes.RISMessage` does not decode the `state` field. The feed sends, for example (`docs/message-examples.json`): `{"type":"ris_message","data":{"peer":"196.60.9.141","peer_asn":"328214","host":"rrc19.ripe.net","type":"STATE","state":"down",...}}` Failure scenario: a peer at an exchange point goes down for an hour. Its few hundred thousand routes stay in the live tables; IP lookups and prefix pages keep listing routes through a peer that has none, and when the session comes back the peer only re-sends what it announces then, so routes it no longer has are never cleaned up. Severity: high ## Definition of done - A `STATE` message with state `down` from a peer removes all of that peer's live routes, IPv4 and IPv6, with the in-memory counts and prefix distribution kept exact. The removal goes through the prefix handler's batch, in feed order with that peer's updates, so an update the peer sends after the session comes back is not lost. - Routes are kept per peer IP, as now; if the same peer IP has sessions on two collectors, a down on one removes routes the other still carries until they are re-announced. Say so in one line in the README. - Tests that fail before the fix: a peer's routes are gone after its `down`; another peer's routes for the same prefixes stay. - `make check` passes. Model: opus-5-5
clawbot self-assigned this 2026-10-14 14:33:44 +02:00
clawbot added the critical label 2026-10-14 15:01:07 +02:00
Author
Collaborator

Plan, built on the per-prefix-and-peer routes of #55 and the flush of #65, both now on next:

  • ristypes.RISMessage decodes the feed's state field.
  • The streamer hands STATE messages to the handlers that ask for them instead of skipping them; the prefix handler asks for UPDATE and STATE.
  • When the prefix handler gets a STATE message whose state is down, it drops that peer's updates still waiting in its batch (the session ending replaces them) and notes the peer as gone down in the batch. When the batch is written, every live route of a peer noted as gone down is deleted, IPv4 and IPv6, before the batch's other writes, so updates the peer sends after its session comes back are written after the removal.
  • A new database.Store method deletes all live routes of one peer IP in both tables and keeps the in-memory route counts and prefix distribution exact, as the existing batch delete does.
  • README: one line saying routes are kept per peer IP, so a peer IP with sessions on two collectors loses, on a down from one, routes the other still carries until they are announced again.
  • Tests that fail before the fix: a peer's routes are gone after its down; another peer's routes for the same prefixes stay; an update from the peer after its down in the same batch is kept.

Model: opus-5-5

Plan, built on the per-prefix-and-peer routes of https://git.eeqj.de/sneak/routewatch/issues/55 and the flush of https://git.eeqj.de/sneak/routewatch/issues/65, both now on `next`: - `ristypes.RISMessage` decodes the feed's `state` field. - The streamer hands `STATE` messages to the handlers that ask for them instead of skipping them; the prefix handler asks for `UPDATE` and `STATE`. - When the prefix handler gets a `STATE` message whose state is `down`, it drops that peer's updates still waiting in its batch (the session ending replaces them) and notes the peer as gone down in the batch. When the batch is written, every live route of a peer noted as gone down is deleted, IPv4 and IPv6, before the batch's other writes, so updates the peer sends after its session comes back are written after the removal. - A new `database.Store` method deletes all live routes of one peer IP in both tables and keeps the in-memory route counts and prefix distribution exact, as the existing batch delete does. - README: one line saying routes are kept per peer IP, so a peer IP with sessions on two collectors loses, on a `down` from one, routes the other still carries until they are announced again. - Tests that fail before the fix: a peer's routes are gone after its `down`; another peer's routes for the same prefixes stay; an update from the peer after its `down` in the same batch is kept. Model: opus-5-5
Author
Collaborator

Built in #88, following the plan above. One addition to it: an index on peer_ip in both live route tables, so deleting a peer's routes does not read the whole table.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/routewatch/pulls/88, following the plan above. One addition to it: an index on `peer_ip` in both live route tables, so deleting a peer's routes does not read the whole table. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/routewatch#56