Record origin changes and show them on the prefix page (closes #161)
check / check (push) Successful in 1m3s
check / check (push) Successful in 1m3s
The routing table records a row each time a prefix and origin AS gets its first peer (announced) or loses its last (withdrawn), with the time and how many peers carry the prefix just after. The prefix handler writes the rows to a new origin_changes table, apart from routes. Every hour the database maintainer keeps each prefix's rows from the last 72 hours or its newest 500, whichever keeps more. The prefix page lists them newest first, at most 5,000. At most 100,000 rows wait in memory to be written; past that the oldest are dropped and logged. A stop loses the rows past its last write. Model: deepseek-v4-pro (implementation); opus-5-5 (commit message)
This commit was merged in pull request #231.
This commit is contained in:
@@ -142,7 +142,16 @@ and origin AS can have two kept routes, one or none.
|
||||
- `/prefix/{prefix}/{length}`: a prefix, such as `/prefix/192.0.2.0/24`, with
|
||||
its origin ASes and every peer that carries it. Only the kept routes show an
|
||||
AS path, a next hop and a time. The next hop is the router the peer sends the
|
||||
prefix's traffic to. RouteWatch records the peer itself as the next hop.
|
||||
prefix's traffic to. RouteWatch records the peer itself as the next hop. The
|
||||
page also lists the prefix's origin changes, newest first. An origin change is
|
||||
an AS starting to announce the prefix, when its first peer carries it, or
|
||||
stopping, when its last peer no longer does. The page shows at most the newest
|
||||
5,000. Each row shows:
|
||||
- the time
|
||||
- announced or withdrawn
|
||||
- the origin AS
|
||||
- the peer count: how many peers carry the prefix just after the change,
|
||||
with any origin AS
|
||||
- `/prefixlength/{length}`: up to 500 IPv4 prefixes with that mask length,
|
||||
picked at random. Each shows its origin AS and the age of its newest kept
|
||||
route. When peers give a prefix different origin ASes, the page shows the one
|
||||
@@ -516,7 +525,8 @@ The ceilings add up to 3.7 GiB:
|
||||
pairs, so it takes about 1 GB. The rest of the heap takes up to about 0.5 GiB,
|
||||
most of it the handlers' batches and queues (see below). The remaining 1 GiB
|
||||
is room for the garbage collector, which frees memory the program no longer
|
||||
uses. With much less, it would run almost all the time.
|
||||
uses. With much less, it would run almost all the time. The routing table also
|
||||
keeps at most 100,000 origin changes waiting to be written, about 8 MB.
|
||||
- SQLite: 1 GiB, set in the code. The connection that does every write has a 512
|
||||
MiB page cache: the database pages it keeps in memory from one write to the
|
||||
next. The `routes` table of a full routing table takes about 450 MB, so its
|
||||
@@ -567,6 +577,9 @@ What happens at each limit:
|
||||
and after the held-back messages of every peer whose session went down before
|
||||
it. It holds at most 150,000 such messages, about 150 MiB. At that number it
|
||||
takes nothing more off its queue until it has applied one of them.
|
||||
- Waiting origin changes: the routing table keeps at most 100,000 origin changes
|
||||
waiting to be written. Past that, it drops the oldest, and the prefix handler
|
||||
logs how many. The prefix page never shows a dropped origin change.
|
||||
|
||||
To replace the image's Go soft limit, set `GOMEMLIMIT` in the environment, for
|
||||
example `-e GOMEMLIMIT=3GiB`. A limit close to the routing table's size makes
|
||||
@@ -620,8 +633,9 @@ The parts:
|
||||
messages. It gives each message to the handlers that want it, through one
|
||||
queue per handler.
|
||||
- The prefix handler keeps the routing table in memory current. Every second it
|
||||
writes the table's changes to the `routes` table of the database. It also
|
||||
stores each new prefix with the time it was first seen.
|
||||
writes the table's changes to the `routes` table and the table's origin
|
||||
changes to the `origin_changes` table of the database. It also stores each new
|
||||
prefix with the time it was first seen.
|
||||
- The AS handler records every AS seen in an AS path in the `asns` table, with
|
||||
the times it was first and last seen.
|
||||
- The peer handler records every peer in the `bgp_peers` table.
|
||||
@@ -638,11 +652,14 @@ The parts:
|
||||
- The database maintainer checkpoints the database every 5 seconds: it folds the
|
||||
write-ahead log, where SQLite first writes changes, into the database file. It
|
||||
reclaims free space every 10 minutes. Every hour it refreshes the statistics
|
||||
SQLite uses to plan its queries.
|
||||
SQLite uses to plan its queries, and prunes the `origin_changes` table. A
|
||||
prune keeps each prefix's rows from the last 72 hours, or its newest 500,
|
||||
whichever keeps more.
|
||||
- The HTTP server serves the pages and the API. Every route it shows comes from
|
||||
the routing table in memory. WHOIS details and first-seen times come from the
|
||||
database. AS names come from the database or from the AS list built into
|
||||
RouteWatch. The AS name search reads the AS list built into RouteWatch.
|
||||
the routing table in memory. WHOIS details, first-seen times and origin
|
||||
changes come from the database. AS names come from the database or from the AS
|
||||
list built into RouteWatch. The AS name search reads the AS list built into
|
||||
RouteWatch.
|
||||
|
||||
### The routing table
|
||||
|
||||
@@ -753,6 +770,8 @@ What a stop can lose:
|
||||
changed pairs. A full-table peer is a peer that carries the whole routing
|
||||
table. Its session going down changes about a million pairs, which take about
|
||||
a minute to write. A stop in that minute leaves the rest unwritten.
|
||||
- The prefix handler's last write of origin changes takes at most 20,000. The
|
||||
rest are lost.
|
||||
- A withdrawal, or a session going down, that is lost this way leaves its routes
|
||||
in the `routes` table. The next start reads them back. They stay until the
|
||||
peer sends another update for the prefix, or its session goes down again.
|
||||
@@ -809,6 +828,10 @@ routes(prefix, origin_asn, peers,
|
||||
-- The peers of the routes table, each with its number in the peer sets
|
||||
route_peers(number PRIMARY KEY, address UNIQUE)
|
||||
|
||||
-- The origin changes of the routing table: each time a prefix and origin AS
|
||||
-- gained its first peer or lost its last
|
||||
origin_changes(id PRIMARY KEY, prefix, origin_asn, announced, peer_count, time)
|
||||
|
||||
peerings(id PRIMARY KEY, as_a, as_b, first_seen, last_seen,
|
||||
UNIQUE(as_a, as_b))
|
||||
bgp_peers(id PRIMARY KEY, peer_ip UNIQUE, peer_asn, first_seen, last_seen,
|
||||
|
||||
Reference in New Issue
Block a user