Commit Graph
12 Commits
Author SHA1 Message Date
sneak 4581bc2cc4 Fix the third review of the origin changes
check / check (push) Canceled after 57s
The prune test gains a third prefix with more than 500 rows newer than 72
hours, and checks every newer row survives and every older one is deleted, in
a helper shared by the three prefixes. The prefix page test uses an origin AS
the routing table does not carry. README's limit list says the routing table
drops the oldest waiting origin changes past 100,000, and the prefix page
entry says the page shows at most the newest 5,000. The tests read 5,000 and
100,000 from named limits instead of bare numbers, so the routing table's
waiting cap is now exported for them.

Model: deepseek-v4-pro
2026-11-05 05:36:57 +00:00
sneak 22783eb425 Fix the second review of the origin changes
check / check (push) Failing after 15m27s
Stop now counts the origin changes inside a running write, in a new
writingOriginChanges field, and adds them to the count it logs. The prune test
checks which rows survive, including a prefix with rows on both sides of 72
hours. The prefix page tests cover 5,000 and 5,001 rows and a row's time and
peer count. New tests cover the dropped-rows warning and the dropped count's
reset. The prune floor is named minOriginChangesPerPrefix. The README, TODO,
package doc and doc comments are corrected to match.

Model: deepseek-v4-pro
2026-11-05 05:08:27 +00:00
sneak 56cfc7e554 Fix the origin-changes review findings
check / check (push) Failing after 2m14s
Hold the 100,000-row cap after a failed write's rows are put back, dropping and counting the oldest rows past it. Stop counts the unwritten origin changes. The prefix handler's per-write origin change limit is a field, so a test checks that one write leaves the rest. The server and README now say where origin changes come from. README, TODO and schema say 72 hours or 500 rows, whichever keeps more. The prefix page and TODO list the origin change columns and say what the peer count counts. The page takes its 5,000-row note from the constant.

Model: deepseek-v4-pro
2026-11-05 04:33:51 +00:00
sneak 347e1456a2 Record origin changes and show them on the prefix page (closes #161)
check / check (push) Successful in 4m6s
The routing table records each prefix and origin AS pair's origin changes:
when it gains its first peer and when it loses its last. Each row holds the
prefix, the origin AS, announced or withdrawn, the peer count and the time.
The prefix handler writes them to a new origin_changes table, and the database
maintainer prunes each prefix to its newest 500 rows and 72 hours. The prefix
page lists them newest first, at most 5,000.

Model: deepseek-v4-pro
2026-11-05 04:07:39 +00:00
clawbot e5afd94a39 Take every down peer out of the table in one walk (closes #225)
check / check (push) Canceled after 0s
The prefix handler ran one PeerDown per peer, one after another. When
the sessions of several peers went down together, their walks queued up.
A returning full-table peer's messages stayed held back while the walks
ahead of its own ran, and could reach the limit of 20,000.

The table's PeersDown now takes several peers out of every prefix in one
walk. One walk runs at a time. The next one takes every peer whose
session went down since the previous one began. The doc of
maxHeldBackMessages says how long a peer's messages are held back. The
limit is now 150,000 messages, about 150 MiB. The status page shows how
many messages each handler holds back.

Model: opus-5-5
2026-11-02 23:10:09 +01:00
clawbot 9bd0e91a3b Let the lock go for as long as it was held during Overview's walk (closes #226)
check / check (push) Canceled after 0s
The home page's cards are worked out once a minute by Overview, which
walks every prefix of the routing table under the table's read lock.
After each batch of 10,000 prefixes, the walk let the lock go and took
it back at once. The prefix handler, waiting for the write lock, then got
in only once per batch. During a walk, its queue could fill, and the
streamer then dropped updates for it.

The walk now lets the lock go for as long as it held the lock for the
batch, as PeerDown does. A test applies another peer's 1,000
announcements while Overview walks a table with more prefixes than five
of the walk's batches.

Model: opus-5-5
2026-11-02 08:00:00 +01:00
clawbot 26d939ad72 Keep taking messages while a peer's session goes down (closes #216)
check / check (push) Canceled after 0s
The table's PeerDown visits every prefix under the write lock, which
takes over a second on a full table. The prefix handler called it
inline, so its queue filled and the streamer dropped updates.

PeerDown now holds the lock for 10,000 prefixes at a time, then lets
the lock go for as long again. The handler runs PeerDown on a separate
goroutine, one peer at a time. The handler holds back the down peer's
later messages until that PeerDown returns. Then the handler applies
them in order, one at a time, and takes new messages in between. It
holds back at most 20,000 messages. Stop waits for all of this before
its last flush.

Model: opus-5-5
2026-11-02 03:27:10 +01:00
clawbot a671b8a0b0 Give every IP lookup result the same fields (closes #200)
check / check (push) Canceled after 0s
Every result of /ip/, /ip/{addr} and /api/v1/ip/{ip} now has every
field. A field without a value is null instead of left out, so an
address with no live route has all its route and AS fields set to null.
ptr is an empty list when there are no PTR names.

Two new fields, num_origins and origin_asns, give how many origin ASes
peers carry the netblock with, and which. The routing table's prefix
summary now lists them. asn still names the origin that most peers
carry.

first_seen is null, no longer the zero time, when the database has not
stored the netblock yet.

Model: opus-5-5
2026-10-27 19:54:02 +01:00
clawbot 281d162e27 Show figures of the routing table on the home page (closes #154)
check / check (push) Canceled after 0s
The home page gets cards for the newest and oldest kept route, the newest
and oldest updated AS, the AS with the most prefixes, the ASes with a
single prefix and fifteen well-known networks. Two more cards count the
bogon announcements and the prefixes with several origin ASes. They link
to the new pages /bogons and /multiple-origins, which list up to 1,000
entries each.

The new package internal/bogon tells reserved prefixes and AS numbers,
from the IANA special-purpose and address space registries. Table.Overview
walks the table and lets the read lock go every 10,000 prefixes. The
server works the cards out once a minute and serves that copy. A new index
on asns(last_seen) finds the newest and oldest updated AS.

Model: opus-5-5
2026-10-27 18:27:01 +01:00
clawbot bac531c76d Write the routing table every second and read it at startup (part of #122)
check / check (push) Canceled after 0s
The prefix handler applies each update to the routing table in memory as
it handles it, sorted into announcements and withdrawals as for the live
route tables. Each flush first writes at most 20,000 of the table's
changed pairs, with every new peer; the rest follow in the next flushes.
A failed write puts its changes and new peers back in the table. Stop
makes one last flush, and logs the changed pairs left.

Startup reads the table before the streamer connects, and logs its pairs,
peers and time. The per-peer counts are a slice, so the read takes about
a third less time. The health check's start period is 60 seconds, so a
full table's read fits before upaas looks.

Model: opus-5-5
2026-10-27 10:27:11 +01:00
clawbot eda37da369 Store the routing table in the database (part of #122)
check / check (push) Canceled after 0s
A new routes table holds one row per prefix and origin AS: the peer set
as the bitset's bytes, and the two kept routes, each with its peer's
number, AS path and time in seconds. A new route_peers table holds each
peer's number and address.

WriteRouteChanges writes one TakeChanges result in a transaction.
ReadRouteTable reads both tables into a new routing table, through the
new Table.Load, which records no change. It fails on a row that names a
peer number missing from route_peers. Nothing uses them yet.

Model: opus-5-5
2026-10-27 08:54:57 +01:00
clawbot 7a34a46c4d Add internal/routetable, the routing table in memory (part of #122)
check / check (push) Canceled after 0s
A new package holds every prefix some peer carries. For each prefix and
origin AS it keeps the set of peers that carry it, one bit per peer,
and the two newest of their routes in full. It applies announcements,
withdrawals and peers' sessions going down by the rule confirmed on the
issue, and records which pairs changed for the database writes to come.
It answers the IP lookup, an AS's prefixes, a prefix's origins, a sample
of one mask length, and the counts the stats show. Nothing uses it yet.

The peer sets use github.com/bits-and-blooms/bitset. The kept routes'
times are held in order in github.com/google/btree, so the oldest and
newest need no scan.

Model: opus-5-5
2026-10-27 07:51:55 +01:00