sneak/routewatch
1
0
Fork 0
Code Issues 4 Pull Requests 2 Actions 6 Packages Projects Releases Wiki Activity

Compare commits

base: sneak/routewatch:main
Branches Tags
sneak/routewatch:main sneak/routewatch:next sneak/routewatch:issue-157-readme sneak/routewatch:prod sneak/routewatch:issue-27-cached-stats
..
compare: sneak/routewatch:555f01920d1fe3d4fdf019f4cf41eb25518bd207
Branches Tags
sneak/routewatch:next sneak/routewatch:issue-157-readme sneak/routewatch:prod sneak/routewatch:main sneak/routewatch:issue-27-cached-stats
1 Commits
main .. 555f01920d
Author SHA1 Message Date
sneak 555f01920d Remove the per-peer route tables (closes #122)
check / check (push) Canceled after 0s
Details 
The routing table in memory now holds every route, so the two per-peer route
tables go, with every write, delete, count and read that served them. The
prefix handler applies each update to the table, writes the table's changes
every second, and stores each new prefix with its first-seen time. The peer
delete, its retry on the next flush and the updates held for it go too.

The image's memory settings fit a full table of about 1 GB: GOMEMLIMIT of
2.5 GiB, a 512 MiB page cache for the writing connection, and SQLite heap
limits of 768 MiB and 1 GiB. The README describes the new table, its
database copy and the memory budget.

Model: opus-5-5
 2026-10-27 15:48:18 +00:00
8 changed files with 182 additions and 203 deletions
Expand all files Collapse all files
README.md
+10 -12
View File
@@ -314,9 +314,7 @@ the routing table, and everything else from the database.
3. **Message Processing Pipeline** 3. **Message Processing Pipeline**
``` ```
RIS Live Stream → JSON Parser → Message Dispatcher → Handler Queues RIS Live Stream → JSON Parser → Message Dispatcher → Handler Queues → Batch Writers → SQLite
├→ Batch Writers → SQLite
└→ PrefixHandler → Routing Table in Memory → SQLite, every second
``` ```
- Streamer connects to `ris-live.ripe.net` via HTTP - Streamer connects to `ris-live.ripe.net` via HTTP
@@ -338,11 +336,11 @@ the routing table, and everything else from the database.
AS path is empty or holds AS 0 anywhere. AS path is empty or holds AS 0 anywhere.
- A peer whose BGP session goes down leaves every prefix. - A peer whose BGP session goes down leaves every prefix.
Every second it writes the changed prefix and origin AS pairs to the Every second it writes the prefix and origin AS pairs that changed to
`routes` table. A write takes at most 20,000 pairs, and the rest follow the `routes` table. A write takes at most 20,000 pairs, and the rest
in the next writes. It also stores each new prefix with the time it was follow in the next writes. It also stores each new prefix with the time
first seen. Peers are known by their IP address. A peer IP can have it was first seen. Peers are known by their IP address. A peer IP can
sessions on two collectors. A session going down on one then also have sessions on two collectors. A session going down on one then also
removes the routes the other still carries, until they are announced removes the routes the other still carries, until they are announced
again. again.
@@ -520,12 +518,12 @@ What happens at each limit:
at ten times the batch size: at ten times the batch size:
- 300,000 updates for the AS handler (`asnBatchMaxSize`, about 10 MiB); - 300,000 updates for the AS handler (`asnBatchMaxSize`, about 10 MiB);
- 100,000 for the peer handler (`peerBatchMaxSize`, about 10 MiB); - 100,000 for the peer handler (`peerBatchMaxSize`, about 10 MiB);
- 250,000 for the prefix handler (`prefixBatchMaxSize`, about 15 MiB). - 250,000 for the prefix handler (`prefixBatchMaxSize`, about 60 MiB).
At that size the handler takes nothing more off its queue until the write At that size the handler takes nothing more off its queue until the write
ends. The queue then fills, and the streamer drops messages as above. With ends. The queue then fills, and the streamer drops messages as above. With
the batches being written, the three handlers' batches take at most about the batches being written, the three handlers' batches take at most about
0.1 GiB of the Go heap. 0.25 GiB of the Go heap.
With `DEBUG=routewatch` the daemon logs a `System stats` line every 60 seconds With `DEBUG=routewatch` the daemon logs a `System stats` line every 60 seconds
with the goroutine count and Go memory figures. with the goroutine count and Go memory figures.
@@ -564,8 +562,8 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app needs:
- A lost withdrawal or peer down leaves its routes in the `routes` table, - A lost withdrawal or peer down leaves its routes in the `routes` table,
and the next start reads them back. They stay until the peer sends another and the next start reads them back. They stay until the peer sends another
update for the prefix, or its session goes down. update for the prefix, or its session goes down.
- When the 8 seconds run out the database is not closed. When the database - When the 8 seconds run out the database is not closed. SQLite rolls back a
next opens, SQLite rolls back any write the exit cut off. write the exit cut off when the database next opens.
## Development ## Development
internal/database/database.go
+6
View File
@@ -96,6 +96,12 @@ const (
// Common errors // Common errors
var ( var (
// ErrInvalidIP says that an IP address is malformed. The server's IP
// lookup returns it.
ErrInvalidIP = errors.New("invalid IP address")
// ErrNoRoute says that the routing table has no route for an address, a
// prefix or an AS. The server's lookups return it.
ErrNoRoute = errors.New("no route found")
// ErrNoStaleASN is returned when no ASN needs WHOIS refresh // ErrNoStaleASN is returned when no ASN needs WHOIS refresh
ErrNoStaleASN = errors.New("no stale ASN found") ErrNoStaleASN = errors.New("no stale ASN found")
// ErrCheckpointIncomplete is returned by Checkpoint when it left part of // ErrCheckpointIncomplete is returned by Checkpoint when it left part of
internal/database/models.go
+52
View File
@@ -66,6 +66,58 @@ type ASNPeering struct {
LastSeen time.Time LastSeen time.Time
} }
// LiveRoute represents a route in the live routing table. JSON leaves out an
// empty AS path, next hop or time: the routing table in memory keeps them only
// for the two newest routes of each prefix and origin AS.
//
//nolint:tagliatelle // field names are the existing JSON API
type LiveRoute struct {
Prefix string `json:"prefix"`
MaskLength int `json:"mask_length"`
IPVersion int `json:"ip_version"`
OriginASN int `json:"origin_asn"`
PeerIP string `json:"peer_ip"`
ASPath []int `json:"as_path,omitempty"`
NextHop string `json:"next_hop,omitempty"`
LastUpdated time.Time `json:"last_updated,omitzero"`
}
// PrefixDistribution represents the distribution of prefixes by mask length
//
//nolint:tagliatelle // field names are the existing JSON API
type PrefixDistribution struct {
MaskLength int `json:"mask_length"`
Count int `json:"count"`
}
// IPInfo represents comprehensive IP information for the /ip endpoint
//
//nolint:tagliatelle // field names are the existing JSON API
type IPInfo struct {
IP string `json:"ip"`
Netblock string `json:"netblock"`
MaskLength int `json:"mask_length"`
IPVersion int `json:"ip_version"`
NumPeers int `json:"num_peers"`
// AS information
ASN int `json:"asn"`
ASName string `json:"as_name,omitempty"`
Handle string `json:"handle,omitempty"`
Description string `json:"description,omitempty"`
OrgName string `json:"org_name,omitempty"`
OrgID string `json:"org_id,omitempty"`
Address string `json:"address,omitempty"`
CountryCode string `json:"country_code,omitempty"`
AbuseEmail string `json:"abuse_email,omitempty"`
RIR string `json:"rir,omitempty"`
// Timestamps. LastSeen is zero, and left out of JSON, when the prefix
// keeps no route.
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen,omitzero"`
// Indicates if WHOIS data needs refresh (not serialized)
NeedsWHOISRefresh bool `json:"-"`
}
// SeenTimes is the earliest and the latest time one batch of updates saw an AS, // SeenTimes is the earliest and the latest time one batch of updates saw an AS,
// a peer or a peering. A row the batch adds is stored first seen at First and // a peer or a peering. A row the batch adds is stored first seen at First and
// last seen at Last. A row already stored takes First as its first_seen only // last seen at Last. A row already stored takes First as its first_seen only
internal/routewatch/prefixhandler.go
+7 -16
View File
@@ -26,7 +26,7 @@ const (
prefixBatchSize = 25000 prefixBatchSize = 25000
// prefixBatchMaxSize is the most prefix updates the batch holds while the // prefixBatchMaxSize is the most prefix updates the batch holds while the
// flush goroutine writes the previous one: ten batches, about 15 MiB // flush goroutine writes the previous one: ten batches, about 60 MiB
// (README "Memory"). At this size HandleMessage waits until the flush // (README "Memory"). At this size HandleMessage waits until the flush
// goroutine takes the batch. // goroutine takes the batch.
prefixBatchMaxSize = 250000 prefixBatchMaxSize = 250000
@@ -65,7 +65,7 @@ type PrefixHandler struct {
// mu, so HandleMessage waits for a database write only while batch is at // mu, so HandleMessage waits for a database write only while batch is at
// prefixBatchMaxSize. // prefixBatchMaxSize.
mu sync.Mutex mu sync.Mutex
batch []seenPrefix batch []prefixUpdate
// writing is how many updates the flush goroutine took and has not // writing is how many updates the flush goroutine took and has not
// finished writing. // finished writing.
writing int writing int
@@ -82,15 +82,6 @@ type PrefixHandler struct {
done chan struct{} // closed when the flush goroutine returns done chan struct{} // closed when the flush goroutine returns
} }
// seenPrefix is what the batch keeps of an update: the prefix it names and its
// time. A flush needs only these to store each new prefix's first-seen time.
type seenPrefix struct {
prefix string
timestamp time.Time
}
// prefixUpdate is one prefix of an update message, as applyToTable applies it
// to the routing table.
type prefixUpdate struct { type prefixUpdate struct {
prefix string prefix string
originASN int originASN int
@@ -133,7 +124,7 @@ func newPrefixHandler(
db: db, db: db,
logger: logger, logger: logger,
table: table, table: table,
batch: make([]seenPrefix, 0, prefixBatchSize), batch: make([]prefixUpdate, 0, prefixBatchSize),
batchFullCh: make(chan struct{}, 1), batchFullCh: make(chan struct{}, 1),
stopCh: make(chan context.Context, 1), stopCh: make(chan context.Context, 1),
done: make(chan struct{}), done: make(chan struct{}),
@@ -207,7 +198,7 @@ func (h *PrefixHandler) HandleMessage(msg *ristypes.RISMessage) {
messageType: "withdrawal", messageType: "withdrawal",
timestamp: timestamp, timestamp: timestamp,
} }
h.batch = append(h.batch, seenPrefix{update.prefix, update.timestamp}) h.batch = append(h.batch, update)
h.applyToTable(update) h.applyToTable(update)
// Record withdrawal in metrics // Record withdrawal in metrics
@@ -227,7 +218,7 @@ func (h *PrefixHandler) HandleMessage(msg *ristypes.RISMessage) {
path: path, path: path,
endsInSet: len(msg.Path.Set) > 0, endsInSet: len(msg.Path.Set) > 0,
} }
h.batch = append(h.batch, seenPrefix{update.prefix, update.timestamp}) h.batch = append(h.batch, update)
h.applyToTable(update) h.applyToTable(update)
// Record announcement in metrics // Record announcement in metrics
@@ -286,7 +277,7 @@ func (h *PrefixHandler) Stop(ctx context.Context) {
// takeBatch returns the pending updates and leaves an empty batch in their // takeBatch returns the pending updates and leaves an empty batch in their
// place for HandleMessage to fill. The updates count in writing until // place for HandleMessage to fill. The updates count in writing until
// flushBatch has written them. // flushBatch has written them.
func (h *PrefixHandler) takeBatch() []seenPrefix { func (h *PrefixHandler) takeBatch() []prefixUpdate {
h.mu.Lock() h.mu.Lock()
defer h.mu.Unlock() defer h.mu.Unlock()
@@ -296,7 +287,7 @@ func (h *PrefixHandler) takeBatch() []seenPrefix {
batch := h.batch batch := h.batch
h.writing = len(batch) h.writing = len(batch)
h.batch = make([]seenPrefix, 0, prefixBatchSize) h.batch = make([]prefixUpdate, 0, prefixBatchSize)
h.batchTaken.Broadcast() h.batchTaken.Broadcast()
return batch return batch
internal/server/handlers.go
+94 -94
View File
@@ -83,7 +83,7 @@ const (
maxASSearchResults = 50 maxASSearchResults = 50
) )
// IP versions as LiveRoute.IPVersion records them. // IP versions as database.LiveRoute.IPVersion records them.
const ( const (
ipVersionV4 = 4 ipVersionV4 = 4
ipVersionV6 = 6 ipVersionV6 = 6
@@ -233,34 +233,34 @@ type WHOISStatsInfo struct {
// //
//nolint:tagliatelle // field names are the existing JSON API //nolint:tagliatelle // field names are the existing JSON API
type Stats struct { type Stats struct {
Uptime string `json:"uptime"` Uptime string `json:"uptime"`
TotalMessages uint64 `json:"total_messages"` TotalMessages uint64 `json:"total_messages"`
TotalBytes uint64 `json:"total_bytes"` TotalBytes uint64 `json:"total_bytes"`
TotalWireBytes uint64 `json:"total_wire_bytes"` TotalWireBytes uint64 `json:"total_wire_bytes"`
MessagesPerSec float64 `json:"messages_per_sec"` MessagesPerSec float64 `json:"messages_per_sec"`
MbitsPerSec float64 `json:"mbits_per_sec"` MbitsPerSec float64 `json:"mbits_per_sec"`
WireMbitsPerSec float64 `json:"wire_mbits_per_sec"` WireMbitsPerSec float64 `json:"wire_mbits_per_sec"`
Connected bool `json:"connected"` Connected bool `json:"connected"`
GoVersion string `json:"go_version"` GoVersion string `json:"go_version"`
Goroutines int `json:"goroutines"` Goroutines int `json:"goroutines"`
MemoryUsage string `json:"memory_usage"` MemoryUsage string `json:"memory_usage"`
ASNs int `json:"asns"` ASNs int `json:"asns"`
Prefixes int `json:"prefixes"` Prefixes int `json:"prefixes"`
IPv4Prefixes int `json:"ipv4_prefixes"` IPv4Prefixes int `json:"ipv4_prefixes"`
IPv6Prefixes int `json:"ipv6_prefixes"` IPv6Prefixes int `json:"ipv6_prefixes"`
Peerings int `json:"peerings"` Peerings int `json:"peerings"`
Peers int `json:"peers"` Peers int `json:"peers"`
DatabaseSizeBytes int64 `json:"database_size_bytes"` DatabaseSizeBytes int64 `json:"database_size_bytes"`
LiveRoutes int `json:"live_routes"` LiveRoutes int `json:"live_routes"`
IPv4Routes int `json:"ipv4_routes"` IPv4Routes int `json:"ipv4_routes"`
IPv6Routes int `json:"ipv6_routes"` IPv6Routes int `json:"ipv6_routes"`
OldestRoute *time.Time `json:"oldest_route,omitempty"` OldestRoute *time.Time `json:"oldest_route,omitempty"`
NewestRoute *time.Time `json:"newest_route,omitempty"` NewestRoute *time.Time `json:"newest_route,omitempty"`
IPv4UpdatesPerSec float64 `json:"ipv4_updates_per_sec"` IPv4UpdatesPerSec float64 `json:"ipv4_updates_per_sec"`
IPv6UpdatesPerSec float64 `json:"ipv6_updates_per_sec"` IPv6UpdatesPerSec float64 `json:"ipv6_updates_per_sec"`
IPv4PrefixDistribution []PrefixDistribution `json:"ipv4_prefix_distribution"` IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6PrefixDistribution []PrefixDistribution `json:"ipv6_prefix_distribution"` IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"` WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
// OriginsWithoutKeptRoute counts the prefix and origin AS pairs that peers // OriginsWithoutKeptRoute counts the prefix and origin AS pairs that peers
// carry but whose kept routes were all withdrawn. // carry but whose kept routes were all withdrawn.
@@ -394,12 +394,12 @@ func (s *Server) statusStats(
// prefixDistribution lists the mask lengths that have at least one prefix, in // prefixDistribution lists the mask lengths that have at least one prefix, in
// ascending order, with their prefix counts. countsByLength[n] is the number // ascending order, with their prefix counts. countsByLength[n] is the number
// of prefixes of mask length n. // of prefixes of mask length n.
func prefixDistribution(countsByLength []int) []PrefixDistribution { func prefixDistribution(countsByLength []int) []database.PrefixDistribution {
var distribution []PrefixDistribution var distribution []database.PrefixDistribution
for maskLength, count := range countsByLength { for maskLength, count := range countsByLength {
if count > 0 { if count > 0 {
distribution = append(distribution, PrefixDistribution{ distribution = append(distribution, database.PrefixDistribution{
MaskLength: maskLength, Count: count, MaskLength: maskLength, Count: count,
}) })
} }
@@ -508,39 +508,39 @@ type StreamStats struct {
// //
//nolint:tagliatelle // field names are the existing JSON API //nolint:tagliatelle // field names are the existing JSON API
type StatsResponse struct { type StatsResponse struct {
Uptime string `json:"uptime"` Uptime string `json:"uptime"`
TotalMessages uint64 `json:"total_messages"` TotalMessages uint64 `json:"total_messages"`
TotalBytes uint64 `json:"total_bytes"` TotalBytes uint64 `json:"total_bytes"`
TotalWireBytes uint64 `json:"total_wire_bytes"` TotalWireBytes uint64 `json:"total_wire_bytes"`
MessagesPerSec float64 `json:"messages_per_sec"` MessagesPerSec float64 `json:"messages_per_sec"`
MbitsPerSec float64 `json:"mbits_per_sec"` MbitsPerSec float64 `json:"mbits_per_sec"`
WireMbitsPerSec float64 `json:"wire_mbits_per_sec"` WireMbitsPerSec float64 `json:"wire_mbits_per_sec"`
Connected bool `json:"connected"` Connected bool `json:"connected"`
ConnectionDuration string `json:"connection_duration"` ConnectionDuration string `json:"connection_duration"`
ReconnectCount uint64 `json:"reconnect_count"` ReconnectCount uint64 `json:"reconnect_count"`
GoVersion string `json:"go_version"` GoVersion string `json:"go_version"`
Goroutines int `json:"goroutines"` Goroutines int `json:"goroutines"`
MemoryUsage string `json:"memory_usage"` MemoryUsage string `json:"memory_usage"`
GC GCStats `json:"gc"` GC GCStats `json:"gc"`
Stream StreamStats `json:"stream"` Stream StreamStats `json:"stream"`
ASNs int `json:"asns"` ASNs int `json:"asns"`
Prefixes int `json:"prefixes"` Prefixes int `json:"prefixes"`
IPv4Prefixes int `json:"ipv4_prefixes"` IPv4Prefixes int `json:"ipv4_prefixes"`
IPv6Prefixes int `json:"ipv6_prefixes"` IPv6Prefixes int `json:"ipv6_prefixes"`
Peerings int `json:"peerings"` Peerings int `json:"peerings"`
Peers int `json:"peers"` Peers int `json:"peers"`
DatabaseSizeBytes int64 `json:"database_size_bytes"` DatabaseSizeBytes int64 `json:"database_size_bytes"`
LiveRoutes int `json:"live_routes"` LiveRoutes int `json:"live_routes"`
IPv4Routes int `json:"ipv4_routes"` IPv4Routes int `json:"ipv4_routes"`
IPv6Routes int `json:"ipv6_routes"` IPv6Routes int `json:"ipv6_routes"`
OldestRoute *time.Time `json:"oldest_route,omitempty"` OldestRoute *time.Time `json:"oldest_route,omitempty"`
NewestRoute *time.Time `json:"newest_route,omitempty"` NewestRoute *time.Time `json:"newest_route,omitempty"`
IPv4UpdatesPerSec float64 `json:"ipv4_updates_per_sec"` IPv4UpdatesPerSec float64 `json:"ipv4_updates_per_sec"`
IPv6UpdatesPerSec float64 `json:"ipv6_updates_per_sec"` IPv6UpdatesPerSec float64 `json:"ipv6_updates_per_sec"`
HandlerStats []HandlerStatsInfo `json:"handler_stats"` HandlerStats []HandlerStatsInfo `json:"handler_stats"`
IPv4PrefixDistribution []PrefixDistribution `json:"ipv4_prefix_distribution"` IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6PrefixDistribution []PrefixDistribution `json:"ipv6_prefix_distribution"` IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"` WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
OriginsWithoutKeptRoute int `json:"origins_without_kept_route"` OriginsWithoutKeptRoute int `json:"origins_without_kept_route"`
} }
@@ -753,7 +753,7 @@ type IPLookupResponse struct {
// fields. IP hides the IPInfo field of the same name, so that an address with // fields. IP hides the IPInfo field of the same name, so that an address with
// no route has it too. // no route has it too.
type IPLookupResult struct { type IPLookupResult struct {
*IPInfo *database.IPInfo
IP string `json:"ip"` IP string `json:"ip"`
PTR []string `json:"ptr,omitempty"` PTR []string `json:"ptr,omitempty"`
@@ -813,10 +813,10 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
} }
switch { switch {
case errors.Is(err, ErrNoRoute): case errors.Is(err, database.ErrNoRoute):
// The address still gets a result, without route fields. // The address still gets a result, without route fields.
ipInfo = nil ipInfo = nil
case errors.Is(err, ErrInvalidIP): case errors.Is(err, database.ErrInvalidIP):
response.Errors = append(response.Errors, ip+": "+err.Error()) response.Errors = append(response.Errors, ip+": "+err.Error())
continue continue
@@ -860,15 +860,15 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
// route. The database gives the prefix's first-seen time and the AS's WHOIS // route. The database gives the prefix's first-seen time and the AS's WHOIS
// details. It returns ErrNoRoute when no prefix holds the address, and // details. It returns ErrNoRoute when no prefix holds the address, and
// ErrInvalidIP when ip does not parse. // ErrInvalidIP when ip does not parse.
func (s *Server) ipInfo(ctx context.Context, ip string) (*IPInfo, error) { func (s *Server) ipInfo(ctx context.Context, ip string) (*database.IPInfo, error) {
addr, err := netip.ParseAddr(ip) addr, err := netip.ParseAddr(ip)
if err != nil { if err != nil {
return nil, fmt.Errorf("%w: %s", ErrInvalidIP, ip) return nil, fmt.Errorf("%w: %s", database.ErrInvalidIP, ip)
} }
summary, found := s.table.Lookup(addr) summary, found := s.table.Lookup(addr)
if !found { if !found {
return nil, fmt.Errorf("%w for IP %s", ErrNoRoute, ip) return nil, fmt.Errorf("%w for IP %s", database.ErrNoRoute, ip)
} }
firstSeen, err := s.db.GetPrefixFirstSeen(ctx, summary.Prefix) firstSeen, err := s.db.GetPrefixFirstSeen(ctx, summary.Prefix)
@@ -881,7 +881,7 @@ func (s *Server) ipInfo(ctx context.Context, ip string) (*IPInfo, error) {
return nil, err return nil, err
} }
info := &IPInfo{ info := &database.IPInfo{
IP: ip, IP: ip,
Netblock: summary.Prefix.String(), Netblock: summary.Prefix.String(),
MaskLength: summary.Prefix.Bits(), MaskLength: summary.Prefix.Bits(),
@@ -1165,7 +1165,7 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
} }
if err != nil { if err != nil {
if errors.Is(err, ErrNoRoute) { if errors.Is(err, database.ErrNoRoute) {
s.writeJSONError(w, http.StatusNotFound, err.Error()) s.writeJSONError(w, http.StatusNotFound, err.Error())
} else { } else {
s.logger.Error("Failed to get AS details", "asn", asn, "error", err) s.logger.Error("Failed to get AS details", "asn", asn, "error", err)
@@ -1201,9 +1201,9 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
// routing table all lack. // routing table all lack.
func (s *Server) asDetails( func (s *Server) asDetails(
ctx context.Context, asn int, ctx context.Context, asn int,
) (*database.ASN, []LiveRoute, error) { ) (*database.ASN, []database.LiveRoute, error) {
if asn == ristypes.NoOrigin { if asn == ristypes.NoOrigin {
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn) return nil, nil, fmt.Errorf("%w: AS%d", database.ErrNoRoute, asn)
} }
asInfo, known, err := s.db.GetAS(ctx, asn) asInfo, known, err := s.db.GetAS(ctx, asn)
@@ -1213,13 +1213,13 @@ func (s *Server) asDetails(
originated := s.table.OriginatedPrefixes(asn) originated := s.table.OriginatedPrefixes(asn)
if !known && len(originated) == 0 { if !known && len(originated) == 0 {
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn) return nil, nil, fmt.Errorf("%w: AS%d", database.ErrNoRoute, asn)
} }
prefixes := make([]LiveRoute, 0, len(originated)) prefixes := make([]database.LiveRoute, 0, len(originated))
for _, p := range originated { for _, p := range originated {
route := LiveRoute{ route := database.LiveRoute{
Prefix: p.Prefix.String(), Prefix: p.Prefix.String(),
MaskLength: p.Prefix.Bits(), MaskLength: p.Prefix.Bits(),
IPVersion: ipVersionOf(p.Prefix), IPVersion: ipVersionOf(p.Prefix),
@@ -1239,9 +1239,9 @@ func (s *Server) asDetails(
// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones. Neither // splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones. Neither
// group is nil, so that a JSON answer holds [] for an empty one and not null. // group is nil, so that a JSON answer holds [] for an empty one and not null.
func splitRoutesByIPVersion( func splitRoutesByIPVersion(
routes []LiveRoute, routes []database.LiveRoute,
) ([]LiveRoute, []LiveRoute) { ) ([]database.LiveRoute, []database.LiveRoute) {
ipv4Routes, ipv6Routes := []LiveRoute{}, []LiveRoute{} ipv4Routes, ipv6Routes := []database.LiveRoute{}, []database.LiveRoute{}
for _, route := range routes { for _, route := range routes {
if route.IPVersion == ipVersionV4 { if route.IPVersion == ipVersionV4 {
@@ -1256,8 +1256,8 @@ func splitRoutesByIPVersion(
// sortRoutesByMaskLength sorts routes by mask length, shortest first, and // sortRoutesByMaskLength sorts routes by mask length, shortest first, and
// routes with the same mask length by network address. // routes with the same mask length by network address.
func sortRoutesByMaskLength(routes []LiveRoute) { func sortRoutesByMaskLength(routes []database.LiveRoute) {
slices.SortFunc(routes, func(a, b LiveRoute) int { slices.SortFunc(routes, func(a, b database.LiveRoute) int {
prefixA, _ := netip.ParsePrefix(a.Prefix) prefixA, _ := netip.ParsePrefix(a.Prefix)
prefixB, _ := netip.ParsePrefix(b.Prefix) prefixB, _ := netip.ParsePrefix(b.Prefix)
@@ -1294,7 +1294,7 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
origins := s.table.Origins(parsed) origins := s.table.Origins(parsed)
if len(origins) == 0 { if len(origins) == 0 {
s.writeJSONError(w, http.StatusNotFound, s.writeJSONError(w, http.StatusNotFound,
ErrNoRoute.Error()+": "+prefix) database.ErrNoRoute.Error()+": "+prefix)
return return
} }
@@ -1302,7 +1302,7 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
routes := prefixRoutes(parsed, origins) routes := prefixRoutes(parsed, origins)
// Group by origin AS // Group by origin AS
originMap := make(map[int][]LiveRoute) originMap := make(map[int][]database.LiveRoute)
for _, route := range routes { for _, route := range routes {
originMap[route.OriginASN] = append(originMap[route.OriginASN], route) originMap[route.OriginASN] = append(originMap[route.OriginASN], route)
} }
@@ -1392,7 +1392,7 @@ func (s *Server) handleASDetail() http.HandlerFunc {
} }
if err != nil { if err != nil {
if errors.Is(err, ErrNoRoute) { if errors.Is(err, database.ErrNoRoute) {
http.Error(w, "AS not found", http.StatusNotFound) http.Error(w, "AS not found", http.StatusNotFound)
} else { } else {
s.logger.Error("Failed to get AS details", "error", err) s.logger.Error("Failed to get AS details", "error", err)
@@ -1425,8 +1425,8 @@ func (s *Server) handleASDetail() http.HandlerFunc {
// Prepare template data // Prepare template data
data := struct { data := struct {
ASN *database.ASN ASN *database.ASN
IPv4Prefixes []LiveRoute IPv4Prefixes []database.LiveRoute
IPv6Prefixes []LiveRoute IPv6Prefixes []database.LiveRoute
TotalCount int TotalCount int
IPv4Count int IPv4Count int
IPv6Count int IPv6Count int
@@ -1585,11 +1585,11 @@ func (s *Server) originASNs(
// them, since the routing table keeps only their addresses. // them, since the routing table keeps only their addresses.
func prefixRoutes( func prefixRoutes(
prefix netip.Prefix, origins []routetable.Origin, prefix netip.Prefix, origins []routetable.Origin,
) []LiveRoute { ) []database.LiveRoute {
var routes []LiveRoute var routes []database.LiveRoute
for _, origin := range origins { for _, origin := range origins {
base := LiveRoute{ base := database.LiveRoute{
Prefix: prefix.String(), Prefix: prefix.String(),
MaskLength: prefix.Bits(), MaskLength: prefix.Bits(),
IPVersion: ipVersionOf(prefix), IPVersion: ipVersionOf(prefix),
@@ -1630,14 +1630,14 @@ type ASPathEntry struct {
// EnhancedRoute is a live route with the handle of each AS in its path. // EnhancedRoute is a live route with the handle of each AS in its path.
type EnhancedRoute struct { type EnhancedRoute struct {
LiveRoute database.LiveRoute
ASPathWithHandle []ASPathEntry ASPathWithHandle []ASPathEntry
} }
// routesWithASPathHandles looks up the handle of each AS in the path of each // routesWithASPathHandles looks up the handle of each AS in the path of each
// route. // route.
func routesWithASPathHandles(routes []LiveRoute) []EnhancedRoute { func routesWithASPathHandles(routes []database.LiveRoute) []EnhancedRoute {
enhancedRoutes := make([]EnhancedRoute, len(routes)) enhancedRoutes := make([]EnhancedRoute, len(routes))
for i, route := range routes { for i, route := range routes {
@@ -1736,7 +1736,7 @@ func (s *Server) prefixLengthHandler(ipVersion, maxMaskLength int) http.HandlerF
// EnhancedPrefix is a prefix on the prefix length page, with the origin AS // EnhancedPrefix is a prefix on the prefix length page, with the origin AS
// that most peers carry it with, that AS's description, and its age. // that most peers carry it with, that AS's description, and its age.
type EnhancedPrefix struct { type EnhancedPrefix struct {
LiveRoute database.LiveRoute
OriginASDescription string OriginASDescription string
Age string Age string
internal/server/handlers_internal_test.go
+9 -9
View File
@@ -158,8 +158,8 @@ type tableStats struct {
OldestRoute time.Time `json:"oldest_route"` OldestRoute time.Time `json:"oldest_route"`
NewestRoute time.Time `json:"newest_route"` NewestRoute time.Time `json:"newest_route"`
IPv4Distribution []PrefixDistribution `json:"ipv4_prefix_distribution"` IPv4Distribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6Distribution []PrefixDistribution `json:"ipv6_prefix_distribution"` IPv6Distribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
} }
// TestStatsHandlersAnswerFromMemory checks that both stats handlers answer 200 // TestStatsHandlersAnswerFromMemory checks that both stats handlers answer 200
@@ -211,8 +211,8 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
OriginsWithoutKeptRoute: 1, OriginsWithoutKeptRoute: 1,
OldestRoute: testRouteTime(0), OldestRoute: testRouteTime(0),
NewestRoute: testRouteTime(4), NewestRoute: testRouteTime(4),
IPv4Distribution: []PrefixDistribution{{MaskLength: 24, Count: 2}}, IPv4Distribution: []database.PrefixDistribution{{MaskLength: 24, Count: 2}},
IPv6Distribution: []PrefixDistribution{{MaskLength: 32, Count: 1}}, IPv6Distribution: []database.PrefixDistribution{{MaskLength: 32, Count: 1}},
} }
for name, handler := range handlers { for name, handler := range handlers {
@@ -1698,9 +1698,9 @@ func TestASDetailJSONAnswersBuiltInAS(t *testing.T) {
//nolint:tagliatelle // field names are the existing JSON API //nolint:tagliatelle // field names are the existing JSON API
var body struct { var body struct {
Data struct { Data struct {
ASN database.ASN `json:"asn"` ASN database.ASN `json:"asn"`
IPv4Prefixes []LiveRoute `json:"ipv4_prefixes"` IPv4Prefixes []database.LiveRoute `json:"ipv4_prefixes"`
IPv6Prefixes []LiveRoute `json:"ipv6_prefixes"` IPv6Prefixes []database.LiveRoute `json:"ipv6_prefixes"`
} `json:"data"` } `json:"data"`
} }
@@ -1845,9 +1845,9 @@ func TestSortRoutesByMaskLength(t *testing.T) {
} }
for _, tt := range tests { for _, tt := range tests {
routes := make([]LiveRoute, 0, len(tt.prefixes)) routes := make([]database.LiveRoute, 0, len(tt.prefixes))
for _, prefix := range tt.prefixes { for _, prefix := range tt.prefixes {
routes = append(routes, LiveRoute{Prefix: prefix}) routes = append(routes, database.LiveRoute{Prefix: prefix})
} }
sortRoutesByMaskLength(routes) sortRoutesByMaskLength(routes)
internal/server/models.go
-67
View File
@@ -1,67 +0,0 @@
package server
import (
"errors"
"time"
)
var (
// ErrInvalidIP says that an IP address is malformed. The IP lookup returns
// it.
ErrInvalidIP = errors.New("invalid IP address")
// ErrNoRoute says that the routing table has no route for an address, a
// prefix or an AS. The lookups return it.
ErrNoRoute = errors.New("no route found")
)
// LiveRoute represents a route in the live routing table. JSON leaves out an
// empty AS path, next hop or time: the routing table in memory keeps them only
// for the two newest routes of each prefix and origin AS.
//
//nolint:tagliatelle // field names are the existing JSON API
type LiveRoute struct {
Prefix string `json:"prefix"`
MaskLength int `json:"mask_length"`
IPVersion int `json:"ip_version"`
OriginASN int `json:"origin_asn"`
PeerIP string `json:"peer_ip"`
ASPath []int `json:"as_path,omitempty"`
NextHop string `json:"next_hop,omitempty"`
LastUpdated time.Time `json:"last_updated,omitzero"`
}
// PrefixDistribution represents the distribution of prefixes by mask length
//
//nolint:tagliatelle // field names are the existing JSON API
type PrefixDistribution struct {
MaskLength int `json:"mask_length"`
Count int `json:"count"`
}
// IPInfo represents comprehensive IP information for the /ip endpoint
//
//nolint:tagliatelle // field names are the existing JSON API
type IPInfo struct {
IP string `json:"ip"`
Netblock string `json:"netblock"`
MaskLength int `json:"mask_length"`
IPVersion int `json:"ip_version"`
NumPeers int `json:"num_peers"`
// AS information
ASN int `json:"asn"`
ASName string `json:"as_name,omitempty"`
Handle string `json:"handle,omitempty"`
Description string `json:"description,omitempty"`
OrgName string `json:"org_name,omitempty"`
OrgID string `json:"org_id,omitempty"`
Address string `json:"address,omitempty"`
CountryCode string `json:"country_code,omitempty"`
AbuseEmail string `json:"abuse_email,omitempty"`
RIR string `json:"rir,omitempty"`
// Timestamps. LastSeen is zero, and left out of JSON, when the prefix
// keeps no route.
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen,omitzero"`
// Indicates if WHOIS data needs refresh (not serialized)
NeedsWHOISRefresh bool `json:"-"`
}
internal/templates/templates_test.go
+4 -5
View File
@@ -7,7 +7,6 @@ import (
"testing" "testing"
"git.eeqj.de/sneak/routewatch/internal/database" "git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/server"
"git.eeqj.de/sneak/routewatch/internal/templates" "git.eeqj.de/sneak/routewatch/internal/templates"
) )
@@ -115,14 +114,14 @@ func TestASDetailPrefixListsFold(t *testing.T) {
prefixV6 = "2001:db8::/32" prefixV6 = "2001:db8::/32"
) )
ipv4 := []server.LiveRoute{{Prefix: prefixV4}} ipv4 := []database.LiveRoute{{Prefix: prefixV4}}
ipv6 := []server.LiveRoute{{Prefix: prefixV6}} ipv6 := []database.LiveRoute{{Prefix: prefixV6}}
// The fields the AS page handler gives the template. // The fields the AS page handler gives the template.
data := struct { data := struct {
ASN *database.ASN ASN *database.ASN
IPv4Prefixes []server.LiveRoute IPv4Prefixes []database.LiveRoute
IPv6Prefixes []server.LiveRoute IPv6Prefixes []database.LiveRoute
TotalCount int TotalCount int
IPv4Count int IPv4Count int
IPv6Count int IPv6Count int