Question for sneak: which BGP peers should routewatch keep routes from?
routewatch keeps one route per prefix for every peer of every RIS collector, so its route tables never stop growing on the live feed. Measured on next (d84dd56) running as README "Running under upaas" describes:
After 6.5 hours: 30.1 million routes in a 10.8 GB database, still growing 4 to 5 million an hour; memory steady at 1.3 to 1.6 GB.
582 IPv4 peers have routes so far, only 4 of them a full table: routes are learned as the feed updates them, so each peer's table fills over days.
RIPEstat lists 1,427 RIS peers, 330 with full IPv4 tables; together they carry about 460 million routes, roughly 165 GB at today's size per route.
Writes keep up below about 15 million routes (no updates dropped in the first 3 hours); past 20 million the prefix handler drops again, 6 to 7 percent of its updates in the last half hour, and lost withdrawals leave routes in the table.
Options:
Chosen peers: a setting lists the peer addresses whose routes are kept, by default about ten full-table peers spread over several collectors. The tables stop at roughly 1.2 million routes per peer (about 12 million, under 5 GB). The AS and peering pages keep using every peer. IP lookups and prefix pages show those peers' view.
Chosen collectors: a setting lists RIS collectors instead. rrc00 alone has 109 peers carrying about 56 million routes (about 20 GB), beyond what writes keep up with today.
All peers: as now. Needs about 165 GB of disk and a different way of storing routes before writes keep up; a lot of work.
Recommendation: chosen peers. Until you answer, the other fixes continue.
Model: opus-5-5
**Question for sneak:** which BGP peers should routewatch keep routes from?
routewatch keeps one route per prefix for every peer of every RIS collector, so its route tables never stop growing on the live feed. Measured on `next` (`d84dd56`) running as README "Running under upaas" describes:
- After 6.5 hours: 30.1 million routes in a 10.8 GB database, still growing 4 to 5 million an hour; memory steady at 1.3 to 1.6 GB.
- 582 IPv4 peers have routes so far, only 4 of them a full table: routes are learned as the feed updates them, so each peer's table fills over days.
- RIPEstat lists 1,427 RIS peers, 330 with full IPv4 tables; together they carry about 460 million routes, roughly 165 GB at today's size per route.
- Writes keep up below about 15 million routes (no updates dropped in the first 3 hours); past 20 million the prefix handler drops again, 6 to 7 percent of its updates in the last half hour, and lost withdrawals leave routes in the table.
Options:
- **Chosen peers**: a setting lists the peer addresses whose routes are kept, by default about ten full-table peers spread over several collectors. The tables stop at roughly 1.2 million routes per peer (about 12 million, under 5 GB). The AS and peering pages keep using every peer. IP lookups and prefix pages show those peers' view.
- **Chosen collectors**: a setting lists RIS collectors instead. `rrc00` alone has 109 peers carrying about 56 million routes (about 20 GB), beyond what writes keep up with today.
- **All peers**: as now. Needs about 165 GB of disk and a different way of storing routes before writes keep up; a lot of work.
Recommendation: chosen peers. Until you answer, the other fixes continue.
Model: opus-5-5
sneak
was assigned by clawbot2026-10-19 22:40:42 +02:00
The same run after 12 hours: 50.4 million routes in 18.0 GB, still growing about 2 million an hour overnight (4 to 5 million an hour in the evening). The prefix handler lost 43 million of its 298 million updates, almost all of them after the tables passed 20 million routes; memory stayed between 1.3 and 1.9 GB.
Model: opus-5-5
The same run after 12 hours: 50.4 million routes in 18.0 GB, still growing about 2 million an hour overnight (4 to 5 million an hour in the evening). The prefix handler lost 43 million of its 298 million updates, almost all of them after the tables passed 20 million routes; memory stayed between 1.3 and 1.9 GB.
Model: opus-5-5
One more fact for the decision: the feed sends no starting table and routewatch loads none, so the tables hold only routes announced since it first started. An address whose route has not changed since then gets 404 from the IP lookup, whichever option you pick; with chosen peers, a full table for each would take days to arrive this way. Loading those peers' tables from RIS's published table dumps at start would fill them at once, but that is a new feature and not part of any option above.
Model: opus-5-5
One more fact for the decision: the feed sends no starting table and routewatch loads none, so the tables hold only routes announced since it first started. An address whose route has not changed since then gets 404 from the IP lookup, whichever option you pick; with chosen peers, a full table for each would take days to arrive this way. Loading those peers' tables from RIS's published table dumps at start would fill them at once, but that is a new feature and not part of any option above.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Question for sneak: which BGP peers should routewatch keep routes from?
routewatch keeps one route per prefix for every peer of every RIS collector, so its route tables never stop growing on the live feed. Measured on
next(d84dd56) running as README "Running under upaas" describes:Options:
rrc00alone has 109 peers carrying about 56 million routes (about 20 GB), beyond what writes keep up with today.Recommendation: chosen peers. Until you answer, the other fixes continue.
Model: opus-5-5
The same run after 12 hours: 50.4 million routes in 18.0 GB, still growing about 2 million an hour overnight (4 to 5 million an hour in the evening). The prefix handler lost 43 million of its 298 million updates, almost all of them after the tables passed 20 million routes; memory stayed between 1.3 and 1.9 GB.
Model: opus-5-5
One more fact for the decision: the feed sends no starting table and routewatch loads none, so the tables hold only routes announced since it first started. An address whose route has not changed since then gets 404 from the IP lookup, whichever option you pick; with chosen peers, a full table for each would take days to arrive this way. Loading those peers' tables from RIS's published table dumps at start would fill them at once, but that is a new feature and not part of any option above.
Model: opus-5-5