Record origin changes and show them on the prefix page (closes #161)
check / check (push) Successful in 1m3s

The routing table records a row each time a prefix and origin AS gets its first peer (announced) or loses its last (withdrawn), with the time and how many peers carry the prefix just after. The prefix handler writes the rows to a new origin_changes table, apart from routes. Every hour the database maintainer keeps each prefix's rows from the last 72 hours or its newest 500, whichever keeps more. The prefix page lists them newest first, at most 5,000.

At most 100,000 rows wait in memory to be written; past that the oldest are dropped and logged. A stop loses the rows past its last write.

Model: deepseek-v4-pro (implementation); opus-5-5 (commit message)
This commit was merged in pull request #231.
This commit is contained in:
2026-11-05 06:57:41 +01:00
parent 15bf20a894
commit e58c6635e7
20 changed files with 1651 additions and 88 deletions
+31 -8
View File
@@ -142,7 +142,16 @@ and origin AS can have two kept routes, one or none.
- `/prefix/{prefix}/{length}`: a prefix, such as `/prefix/192.0.2.0/24`, with
its origin ASes and every peer that carries it. Only the kept routes show an
AS path, a next hop and a time. The next hop is the router the peer sends the
prefix's traffic to. RouteWatch records the peer itself as the next hop.
prefix's traffic to. RouteWatch records the peer itself as the next hop. The
page also lists the prefix's origin changes, newest first. An origin change is
an AS starting to announce the prefix, when its first peer carries it, or
stopping, when its last peer no longer does. The page shows at most the newest
5,000. Each row shows:
- the time
- announced or withdrawn
- the origin AS
- the peer count: how many peers carry the prefix just after the change,
with any origin AS
- `/prefixlength/{length}`: up to 500 IPv4 prefixes with that mask length,
picked at random. Each shows its origin AS and the age of its newest kept
route. When peers give a prefix different origin ASes, the page shows the one
@@ -516,7 +525,8 @@ The ceilings add up to 3.7 GiB:
pairs, so it takes about 1 GB. The rest of the heap takes up to about 0.5 GiB,
most of it the handlers' batches and queues (see below). The remaining 1 GiB
is room for the garbage collector, which frees memory the program no longer
uses. With much less, it would run almost all the time.
uses. With much less, it would run almost all the time. The routing table also
keeps at most 100,000 origin changes waiting to be written, about 8 MB.
- SQLite: 1 GiB, set in the code. The connection that does every write has a 512
MiB page cache: the database pages it keeps in memory from one write to the
next. The `routes` table of a full routing table takes about 450 MB, so its
@@ -567,6 +577,9 @@ What happens at each limit:
and after the held-back messages of every peer whose session went down before
it. It holds at most 150,000 such messages, about 150 MiB. At that number it
takes nothing more off its queue until it has applied one of them.
- Waiting origin changes: the routing table keeps at most 100,000 origin changes
waiting to be written. Past that, it drops the oldest, and the prefix handler
logs how many. The prefix page never shows a dropped origin change.
To replace the image's Go soft limit, set `GOMEMLIMIT` in the environment, for
example `-e GOMEMLIMIT=3GiB`. A limit close to the routing table's size makes
@@ -620,8 +633,9 @@ The parts:
messages. It gives each message to the handlers that want it, through one
queue per handler.
- The prefix handler keeps the routing table in memory current. Every second it
writes the table's changes to the `routes` table of the database. It also
stores each new prefix with the time it was first seen.
writes the table's changes to the `routes` table and the table's origin
changes to the `origin_changes` table of the database. It also stores each new
prefix with the time it was first seen.
- The AS handler records every AS seen in an AS path in the `asns` table, with
the times it was first and last seen.
- The peer handler records every peer in the `bgp_peers` table.
@@ -638,11 +652,14 @@ The parts:
- The database maintainer checkpoints the database every 5 seconds: it folds the
write-ahead log, where SQLite first writes changes, into the database file. It
reclaims free space every 10 minutes. Every hour it refreshes the statistics
SQLite uses to plan its queries.
SQLite uses to plan its queries, and prunes the `origin_changes` table. A
prune keeps each prefix's rows from the last 72 hours, or its newest 500,
whichever keeps more.
- The HTTP server serves the pages and the API. Every route it shows comes from
the routing table in memory. WHOIS details and first-seen times come from the
database. AS names come from the database or from the AS list built into
RouteWatch. The AS name search reads the AS list built into RouteWatch.
the routing table in memory. WHOIS details, first-seen times and origin
changes come from the database. AS names come from the database or from the AS
list built into RouteWatch. The AS name search reads the AS list built into
RouteWatch.
### The routing table
@@ -753,6 +770,8 @@ What a stop can lose:
changed pairs. A full-table peer is a peer that carries the whole routing
table. Its session going down changes about a million pairs, which take about
a minute to write. A stop in that minute leaves the rest unwritten.
- The prefix handler's last write of origin changes takes at most 20,000. The
rest are lost.
- A withdrawal, or a session going down, that is lost this way leaves its routes
in the `routes` table. The next start reads them back. They stay until the
peer sends another update for the prefix, or its session goes down again.
@@ -809,6 +828,10 @@ routes(prefix, origin_asn, peers,
-- The peers of the routes table, each with its number in the peer sets
route_peers(number PRIMARY KEY, address UNIQUE)
-- The origin changes of the routing table: each time a prefix and origin AS
-- gained its first peer or lost its last
origin_changes(id PRIMARY KEY, prefix, origin_asn, announced, peer_count, time)
peerings(id PRIMARY KEY, as_a, as_b, first_seen, last_seen,
UNIQUE(as_a, as_b))
bgp_peers(id PRIMARY KEY, peer_ip UNIQUE, peer_asn, first_seen, last_seen,