sneak/routewatch
1
0
Fork 0
Code Issues 4 Pull Requests 2 Actions 19 Packages Projects Releases Wiki Activity

deploy #197

Merged
sneak merged 26 commits from main into prod 2026-10-27 18:16:15 +01:00
Conversation 0 Commits 26 Files Changed 67
+1335 -656
18 changed files with 1335 additions and 656 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 506e289a63 - Show all commits
README.md
+49 -34
View File
@@ -12,16 +12,16 @@ table in real time. It works like this:
RIPE RIS Live streams the BGP updates that RIPE's route collectors receive, as
they happen, from the moment a client connects. It reports changes, not the
routing table they add up to, so it cannot say which AS an IP address is routed
to now or which routes a prefix has. RouteWatch applies the feed's updates to
live routing tables in its own SQLite database and answers IP-to-AS lookups and
prefix pages from them.
to now or which routes a prefix has. RouteWatch applies the feed's updates to a
routing table that it keeps in memory, with a copy in its own SQLite database.
It answers IP-to-AS lookups and prefix pages from that table.
The feed sends no table when a client connects, so these tables hold only the
routes the feed has announced since RouteWatch first started: an IP-to-AS lookup
finds nothing for an address whose routes have not been announced since then.
The tables also miss the updates RouteWatch drops when it falls behind under
load and those sent while it is stopped, so a lookup or a prefix page can show a
route the feed has withdrawn.
The feed sends no table when a client connects. So the routing table holds only
the routes announced since RouteWatch first started. An IP-to-AS lookup finds
nothing for an address whose routes have not been announced since then. The
routing table also misses updates RouteWatch drops when it falls behind, and
updates sent while it is stopped. So a lookup or a prefix page can show a route
the feed has withdrawn.
## Features
@@ -58,13 +58,24 @@ DEBUG=routewatch ./routewatch
Every route answers `HEAD` as it answers `GET`, without the body.
The pages and the API read routes from the routing table that RouteWatch keeps
in memory. For each prefix and origin AS, the table keeps the two newest routes
in full, with their AS paths and times. These are the kept routes. The table
keeps only the address of every other peer that carries the prefix. When both
kept routes of an origin AS are withdrawn, that origin has no kept route until
one of its peers sends an update.
### Web Interface
- `GET /` - Home page (HTML): headline figures, lookups by AS number and by IP
address, and a search of AS names. A card shows the `curl` command that looks
up the visitor's own address at `/ip/{addr}`, and that command's live answer.
- `GET /status` - HTML status dashboard, which reads `/api/v1/stats`
- `GET /status.json` - JSON statistics: feed, memory, database and route figures
- `GET /status.json` - JSON statistics: feed, memory, database and route
figures. The route figures come from the routing table. `oldest_route` and
`newest_route` are the times of the oldest and the newest kept routes.
`origins_without_kept_route` counts, across all prefixes, the origin ASes that
have no kept route.
- `GET /.well-known/healthcheck.json` - Health check (JSON): 200 when the
database answers and the RIS Live feed is connected, otherwise 503;
`data.status` is `ok` or `error` and `data.checks` says which check failed
@@ -73,10 +84,11 @@ Every route answers `HEAD` as it answers `GET`, without the body.
description come from the AS list built into RouteWatch. The page then shows
no first-seen date.
- `GET /prefix/{prefix}/{length}` - Prefix detail page (HTML), such as
`/prefix/192.0.2.0/24`: every live route for the prefix, one per peer, and its
origin ASes
`/prefix/192.0.2.0/24`: its origin ASes and every peer that carries it. Only
the kept routes show an AS path, a next hop and a time.
- `GET /prefixlength/{length}` - Up to 500 IPv4 prefixes with that mask length,
picked at random, each with its newest route (HTML)
picked at random (HTML). Each shows the origin AS that most peers carry it
with, and the age of its newest kept route.
- `GET /prefixlength6/{length}` - The same for IPv6
- `GET /ip/{addr}` - The same JSON answer as `/api/v1/ip/{ip}`
- `GET /ip/` - The same, for the caller's own address: the address the request
@@ -93,8 +105,10 @@ Every route answers `HEAD` as it answers `GET`, without the body.
name, or a comma-separated list of at most 10 of these. `results` has one
entry for each address, with:
- `ip` and `ptr`: the address and its PTR names
- `netblock` and the fields beside it: the address's most specific live
route, and that route's origin AS with its WHOIS details
- `netblock` and the fields beside it: the most specific prefix that holds
the address, and its origin AS with that AS's WHOIS details. For a prefix
with several origin ASes, `asn` is the one that most peers carry.
`last_seen` is the time of the prefix's newest kept route.
- `blocklists`: the public blocklists the address is on (see Blocklists)
An address with no live route has a result without the route and AS fields.
@@ -110,11 +124,13 @@ Every route answers `HEAD` as it answers `GET`, without the body.
answered without PTR names.
- `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6
apart, each with when its newest route was updated. When the database has no
record of the AS, its handle and description come from the AS list built into
apart, each with the time of its newest kept route. A prefix that keeps no
route with the AS has no `last_updated`. When the database has no record of
the AS, its handle and description come from the AS list built into
RouteWatch. The answer then has no `first_seen` or `last_seen`.
- `GET /api/v1/prefix/{prefix}/{length}` - Every live route for the prefix, one
per peer, and the same routes grouped by origin AS in `origins`
- `GET /api/v1/prefix/{prefix}/{length}` - One route for each peer that carries
the prefix, and the same routes grouped by origin AS in `origins`. Only the
kept routes have `as_path`, `next_hop` and `last_updated`.
- `GET /api/v1/search/as?q={text}` - Finds the ASes whose handle or description
contains `{text}`, ignoring case. `data` holds:
- `query`: `{text}` without its leading and trailing spaces
@@ -137,16 +153,16 @@ took. The health check is the exception: its 503 answer is also
A request that runs past its time limit answers 504: the limit is 30 seconds,
and 4 seconds for the database reads behind `/status.json` and `/api/v1/stats`.
An AS number, prefix or mask length that does not parse answers 400, as does a
mask length out of range for its IP version. An AS answers 404 when it is not in
the AS list built into RouteWatch and the database has neither a record of it
nor a live route it originates. AS 0 always answers 404, because it means no
known origin. The AS name search never lists it. A prefix with no live route
answers 404 too. The HTML pages give these errors as plain text. Under `/api/v1`
and `/ip/`, a path no route matches answers 404 with the message `Not found`,
and `POST`, `PUT`, `PATCH`, `DELETE`, `OPTIONS`, `TRACE` or `CONNECT` answers
405 with `Method not allowed`. A 500 answer says only `Internal server error`,
and a host name that does not resolve only `could not resolve host name`; the
detail goes to the log.
mask length out of range for its IP version. An AS answers 404 when it
originates no prefix and has no record in the database or in the AS list built
into RouteWatch. AS 0 always answers 404, because it means no known origin. The
AS name search never lists it. A prefix with no live route answers 404 too. The
HTML pages give these errors as plain text. Under `/api/v1` and `/ip/`, a path
no route matches answers 404 with the message `Not found`, and `POST`, `PUT`,
`PATCH`, `DELETE`, `OPTIONS`, `TRACE` or `CONNECT` answers 405 with
`Method not allowed`. A 500 answer says only `Internal server error`, and a host
name that does not resolve only `could not resolve host name`; the detail goes
to the log.
Every answer carries `Strict-Transport-Security`, `Content-Security-Policy`,
`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` and
@@ -340,11 +356,10 @@ routewatch/
with exponential backoff (5s-320s). The backoff resets after a stream that
lasted more than 30s. A stream with no line for 60s is dropped and
reconnected.
- **IP Lookup**: the most specific live route is found by looking up the
address's prefix at each mask length, longest first, in the prefix index (at
most 25 lookups for IPv4, 113 for IPv6); prefixes are stored in the text form
Go's `net/netip` prints. Routes shorter than /8 for IPv4 or /16 for IPv6, such
as a default route, are not used.
- **IP Lookup**: the routing table in memory finds the most specific prefix that
holds the address. It looks up the address's prefix at each mask length,
longest first: at most 25 lookups for IPv4 and 113 for IPv6. Prefixes shorter
than /8 for IPv4 or /16 for IPv6, such as a default route, are not used.
### Database Schema
TODO.md
+8 -3
View File
@@ -33,6 +33,12 @@ tested.
# Completed Steps
- 2026-10-27: the pages, the API and the stats read every route from the routing
table in memory. For a prefix with several origin ASes, the IP lookup gives
the one most peers carry. Its `last_seen` is the newest kept route's time. The
prefix page and API list every peer that carries the prefix, with an AS path,
next hop and time only for the kept routes. The stats count the origins whose
kept routes were all withdrawn, in `origins_without_kept_route` (part of #122)
- 2026-10-27: the prefix, AS and peer handlers make a timed write on every tick
of their timers, every 1, 2 and 2 seconds. They used to skip every other tick.
A full batch's write restarts the timer, so the next timed write comes one
@@ -271,6 +277,5 @@ tested.
# Future Steps
- the rest of https://git.eeqj.de/sneak/routewatch/issues/122, one PR each: the
pages and the API reading from the routing table, then removing the per-peer
route tables
- the rest of https://git.eeqj.de/sneak/routewatch/issues/122: removing the
per-peer route tables
internal/database/counts_internal_test.go
-54
View File
@@ -657,60 +657,6 @@ func checkOneLiveRoutePerIPVersion(t *testing.T, db *Database) {
assertDistribution(t, "IPv6 distribution query", queryV6, wantV6)
}
// TestStatsRouteTimestamps checks the oldest/newest route timestamps are read
// from the right rows across both tables and parse into time.Time. The old
// MIN/MAX union query read its result into *time.Time, which the driver could
// not parse, so it logged a warning every call and left both timestamps nil.
func TestStatsRouteTimestamps(t *testing.T) {
t.Parallel()
cfg := &config.Config{StateDir: t.TempDir()}
db, err := New(cfg, logger.New())
if err != nil {
t.Fatalf("failed to create database: %v", err)
}
defer func() { _ = db.Close() }()
ctx := context.Background()
empty, err := db.GetStatsContext(ctx)
if err != nil {
t.Fatalf("GetStatsContext on empty database: %v", err)
}
if empty.OldestRoute != nil || empty.NewestRoute != nil {
t.Fatalf("empty database timestamps = (%v, %v), want (nil, nil)",
empty.OldestRoute, empty.NewestRoute)
}
base := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
oldest := base
newest := base.Add(2 * time.Minute)
err = db.UpsertLiveRouteBatch([]*LiveRoute{
mkV4Route(t, "198.51.100.0/24", 64500, base.Add(time.Minute)),
mkV4Route(t, "203.0.113.0/24", 64501, oldest),
mkV6Route(testV6Prefix, 64502, newest),
})
if err != nil {
t.Fatalf("UpsertLiveRouteBatch: %v", err)
}
stats, err := db.GetStatsContext(ctx)
if err != nil {
t.Fatalf("GetStatsContext: %v", err)
}
if stats.OldestRoute == nil || !stats.OldestRoute.Equal(oldest) {
t.Errorf("OldestRoute = %v, want %v", stats.OldestRoute, oldest)
}
if stats.NewestRoute == nil || !stats.NewestRoute.Equal(newest) {
t.Errorf("NewestRoute = %v, want %v", stats.NewestRoute, newest)
}
}
// TestLiveCountsConcurrentReadWrite runs writers and stats readers at once so
// the race detector proves the counters are safe under concurrent use.
func TestLiveCountsConcurrentReadWrite(t *testing.T) {
internal/database/database.go
+113 -108
View File
@@ -1452,10 +1452,9 @@ func (d *Database) GetStats() (Stats, error) {
//
// The row counts (ASNs, prefixes, peerings, peers, live routes) and the prefix
// distribution come from the in-memory counters, seeded at startup and kept
// current on every write. The oldest/newest route timestamps are read from the
// ends of the last_updated index, and the file size from a stat(). No part of
// the read scans a table or a whole index.
func (d *Database) GetStatsContext(ctx context.Context) (Stats, error) {
// current on every write, and the file size from a stat(). The read touches
// no table.
func (d *Database) GetStatsContext(_ context.Context) (Stats, error) {
var stats Stats
// Row counts and prefix distribution from memory, as a single consistent
@@ -1470,20 +1469,6 @@ func (d *Database) GetStatsContext(ctx context.Context) (Stats, error) {
stats.FileSizeBytes = fileInfo.Size()
}
// Oldest and newest route timestamps read one row from each end of the
// last_updated index (a log-time lookup, not a scan). Selecting the column
// directly lets the driver parse the DATETIME into time.Time; the old
// MIN/MAX union scan returned an untyped string that failed to scan and
// logged a warning on every call.
oldest, newest, err := d.routeTimestampRange(ctx)
if err != nil {
// Display-only fields; log but keep the rest of the stats.
d.logger.Warn("Failed to get route timestamps", "error", err)
} else {
stats.OldestRoute = oldest
stats.NewestRoute = newest
}
return stats, nil
}
@@ -1778,7 +1763,35 @@ func (d *Database) GetASDetailsContext(
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
}
// Get AS information
asnInfo, known, err := d.GetAS(ctx, asn)
if err != nil {
return nil, nil, err
}
allPrefixes, err := d.originPrefixes(ctx, asn, ipVersionV4)
if err != nil {
return asnInfo, nil, err
}
prefixesV6, err := d.originPrefixes(ctx, asn, ipVersionV6)
if err != nil {
return asnInfo, allPrefixes, err
}
allPrefixes = append(allPrefixes, prefixesV6...)
if !known && len(allPrefixes) == 0 {
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
}
return asnInfo, allPrefixes, nil
}
// GetAS returns the AS's handle, description and first and last seen times
// from the asns table. When the table has no record of the AS, its handle and
// description come from the AS info lookup, and it has no seen times. The bool
// says whether the asns table or the AS info lookup holds the AS.
func (d *Database) GetAS(ctx context.Context, asn int) (*ASN, bool, error) {
var (
asnInfo ASN
handle, description sql.NullString
@@ -1791,9 +1804,6 @@ func (d *Database) GetASDetailsContext(
).Scan(&asnInfo.ASN, &handle, &description, &asnInfo.FirstSeen,
&asnInfo.LastSeen)
// known is whether the asns table or the AS info lookup holds the AS.
known := true
switch {
case errors.Is(err, sql.ErrNoRows):
asnInfo.ASN = asn
@@ -1804,31 +1814,96 @@ func (d *Database) GetASDetailsContext(
asnInfo.Description = info.Description
}
known = ok
return &asnInfo, ok, nil
case err != nil:
return nil, nil, fmt.Errorf("failed to query AS: %w", err)
default:
asnInfo.Handle = handle.String
asnInfo.Description = description.String
return nil, false, fmt.Errorf("failed to query AS: %w", err)
}
asnInfo.Handle = handle.String
asnInfo.Description = description.String
return &asnInfo, true, nil
}
// asWHOISQuery reads the handle, description and WHOIS details of one AS, and
// when the WHOIS details were fetched.
const asWHOISQuery = `
SELECT handle, description, as_name, org_name, org_id, address,
country_code, abuse_email, rir, whois_updated_at
FROM asns WHERE asn = ?`
// GetASWHOIS returns the handle, description and WHOIS details that the asns
// table holds for the AS, and when the WHOIS details were fetched. When the
// table has no record of the AS, it returns an ASN with only its number.
func (d *Database) GetASWHOIS(ctx context.Context, asn int) (*ASN, error) {
var (
handle, description, asName, orgName, orgID sql.NullString
address, countryCode, abuseEmail, rir sql.NullString
whoisUpdatedAt sql.NullTime
)
err := d.db.QueryRowContext(ctx, asWHOISQuery, asn).Scan(
&handle, &description, &asName, &orgName, &orgID, &address,
&countryCode, &abuseEmail, &rir, &whoisUpdatedAt,
)
if errors.Is(err, sql.ErrNoRows) {
return &ASN{ASN: asn}, nil
}
allPrefixes, err := d.originPrefixes(ctx, asn, ipVersionV4)
if err != nil {
return &asnInfo, nil, err
return nil, fmt.Errorf("failed to query AS: %w", err)
}
info := &ASN{
ASN: asn,
Handle: handle.String,
Description: description.String,
ASName: asName.String,
OrgName: orgName.String,
OrgID: orgID.String,
Address: address.String,
CountryCode: countryCode.String,
AbuseEmail: abuseEmail.String,
RIR: rir.String,
}
if whoisUpdatedAt.Valid {
info.WHOISUpdatedAt = &whoisUpdatedAt.Time
}
return info, nil
}
// The queries GetPrefixFirstSeen reads an IPv4 and an IPv6 prefix's first-seen
// time with.
const (
prefixFirstSeenV4SQL = `SELECT first_seen FROM prefixes_v4 WHERE prefix = ?`
prefixFirstSeenV6SQL = `SELECT first_seen FROM prefixes_v6 WHERE prefix = ?`
)
// GetPrefixFirstSeen returns when the prefix tables first saw prefix, or the
// zero time when they have no record of it.
func (d *Database) GetPrefixFirstSeen(
ctx context.Context, prefix netip.Prefix,
) (time.Time, error) {
query := prefixFirstSeenV6SQL
if prefix.Addr().Is4() {
query = prefixFirstSeenV4SQL
}
var firstSeen time.Time
err := d.db.QueryRowContext(ctx, query, prefix.Masked().String()).
Scan(&firstSeen)
if errors.Is(err, sql.ErrNoRows) {
return time.Time{}, nil
}
prefixesV6, err := d.originPrefixes(ctx, asn, ipVersionV6)
if err != nil {
return &asnInfo, allPrefixes, err
return time.Time{}, fmt.Errorf("failed to query prefix: %w", err)
}
allPrefixes = append(allPrefixes, prefixesV6...)
if !known && len(allPrefixes) == 0 {
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
}
return &asnInfo, allPrefixes, nil
return firstSeen, nil
}
// ASPeer represents a peering relationship with another AS including handle,
@@ -2564,76 +2639,6 @@ func (d *Database) deletePeerRouteChunk(peerIP string) (int, int, error) {
return int(deletedV4), int(deletedV6), nil
}
// routeTimestampRange returns the earliest and latest last_updated across both
// live route tables, or nil values when both tables are empty. Each query reads
// one row from an end of the last_updated index rather than scanning the tables.
func (d *Database) routeTimestampRange(
ctx context.Context,
) (*time.Time, *time.Time, error) {
var oldest, newest *time.Time
oldestV4, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
"FROM live_routes_v4 ORDER BY last_updated ASC LIMIT 1")
if err != nil {
return nil, nil, err
}
if ok {
oldest = &oldestV4
}
oldestV6, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
"FROM live_routes_v6 ORDER BY last_updated ASC LIMIT 1")
if err != nil {
return nil, nil, err
}
if ok && (oldest == nil || oldestV6.Before(*oldest)) {
oldest = &oldestV6
}
newestV4, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
"FROM live_routes_v4 ORDER BY last_updated DESC LIMIT 1")
if err != nil {
return nil, nil, err
}
if ok {
newest = &newestV4
}
newestV6, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
"FROM live_routes_v6 ORDER BY last_updated DESC LIMIT 1")
if err != nil {
return nil, nil, err
}
if ok && (newest == nil || newestV6.After(*newest)) {
newest = &newestV6
}
return oldest, newest, nil
}
// scanRouteTimestamp runs a single-row timestamp query. The bool is false when
// the table is empty. The query selects the last_updated column directly so the
// driver parses the DATETIME value into a time.Time.
func (d *Database) scanRouteTimestamp(
ctx context.Context, query string,
) (time.Time, bool, error) {
var ts time.Time
err := d.db.QueryRowContext(ctx, query).Scan(&ts)
switch {
case errors.Is(err, sql.ErrNoRows):
return time.Time{}, false, nil
case err != nil:
return time.Time{}, false, err
default:
return ts, true, nil
}
}
// The AS page lists the prefixes an AS originates one at a time, each in two
// index searches that read no table row: one of the origin_asn and prefix
// index for the AS's first prefix after the previous one, and one of the
internal/database/database_internal_test.go
+159
View File
@@ -1780,6 +1780,165 @@ func TestGetASDetailsOfASNotInDatabase(t *testing.T) {
}
}
// TestGetASReadsTheRecordOrTheBuiltInList asks GetAS for an AS the asns table
// holds, one only the AS info lookup holds, and one neither holds. The first
// gets its record's seen times, the second its handle and description from the
// lookup, and only the third is not known.
func TestGetASReadsTheRecordOrTheBuiltInList(t *testing.T) {
t.Parallel()
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
if err != nil {
t.Fatalf("failed to create database: %v", err)
}
defer func() { _ = db.Close() }()
const (
storedASN = 64500 // a documentation AS, not in the lookup
listedASN = 15169 // in the lookup, not stored
unknownASN = 4200000001 // in neither
)
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
_, err = db.GetOrCreateASN(storedASN, ts)
if err != nil {
t.Fatalf("GetOrCreateASN: %v", err)
}
listed, ok := asinfo.Get(listedASN)
if !ok {
t.Fatalf("AS%d is not in the AS info data", listedASN)
}
tests := []struct {
asn int
want ASN
wantKnown bool
}{
{storedASN, ASN{ASN: storedASN, FirstSeen: ts, LastSeen: ts}, true},
{listedASN, ASN{
ASN: listedASN, Handle: listed.Handle, Description: listed.Description,
}, true},
{unknownASN, ASN{ASN: unknownASN}, false},
}
for _, tt := range tests {
got, known, err := db.GetAS(t.Context(), tt.asn)
if err != nil {
t.Errorf("GetAS(%d): %v", tt.asn, err)
continue
}
if !got.FirstSeen.Equal(tt.want.FirstSeen) ||
!got.LastSeen.Equal(tt.want.LastSeen) {
t.Errorf("GetAS(%d) seen %v to %v, want %v to %v", tt.asn,
got.FirstSeen, got.LastSeen, tt.want.FirstSeen, tt.want.LastSeen)
}
got.FirstSeen, got.LastSeen = tt.want.FirstSeen, tt.want.LastSeen
if *got != tt.want || known != tt.wantKnown {
t.Errorf("GetAS(%d) = %+v, known %t; want %+v, known %t",
tt.asn, *got, known, tt.want, tt.wantKnown)
}
}
}
// TestGetASWHOISReadsTheStoredDetails stores an AS with WHOIS details and asks
// GetASWHOIS for it and for an AS the asns table does not hold. The first gets
// its stored handle, description and details, and the time they were fetched.
// The second gets only its number, although the AS info lookup holds it.
func TestGetASWHOISReadsTheStoredDetails(t *testing.T) {
t.Parallel()
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
if err != nil {
t.Fatalf("failed to create database: %v", err)
}
defer func() { _ = db.Close() }()
const storedASN, unstoredASN = 15169, 13335
_, err = db.GetOrCreateASN(storedASN, time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC))
if err != nil {
t.Fatalf("GetOrCreateASN: %v", err)
}
update := &ASNWHOISUpdate{
ASN: storedASN, ASName: "TEST-AS", OrgName: "Test Org", OrgID: "ORG-1",
Address: "1 Test Street", CountryCode: "DE", AbuseEmail: "abuse@test",
RIR: "RIPE",
}
err = db.UpdateASNWHOIS(t.Context(), update)
if err != nil {
t.Fatalf("UpdateASNWHOIS: %v", err)
}
got, err := db.GetASWHOIS(t.Context(), storedASN)
if err != nil {
t.Fatalf("GetASWHOIS(%d): %v", storedASN, err)
}
want := ASN{
ASN: storedASN, Handle: asinfo.GetHandle(storedASN),
Description: asinfo.GetDescription(storedASN), ASName: update.ASName,
OrgName: update.OrgName, OrgID: update.OrgID, Address: update.Address,
CountryCode: update.CountryCode, AbuseEmail: update.AbuseEmail,
RIR: update.RIR,
}
if got.WHOISUpdatedAt == nil {
t.Errorf("GetASWHOIS(%d) has no time the details were fetched", storedASN)
}
got.WHOISUpdatedAt = nil
if *got != want {
t.Errorf("GetASWHOIS(%d) = %+v, want %+v", storedASN, *got, want)
}
got, err = db.GetASWHOIS(t.Context(), unstoredASN)
if err != nil || *got != (ASN{ASN: unstoredASN}) {
t.Errorf("GetASWHOIS(%d) = %+v, %v; want only the number",
unstoredASN, got, err)
}
}
// TestGetPrefixFirstSeenReadsThePrefixTables stores an IPv4 and an IPv6
// prefix and expects GetPrefixFirstSeen to give each its first-seen time, and
// the zero time for a prefix that is not stored.
func TestGetPrefixFirstSeenReadsThePrefixTables(t *testing.T) {
t.Parallel()
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
if err != nil {
t.Fatalf("failed to create database: %v", err)
}
defer func() { _ = db.Close() }()
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
stored := map[string]time.Time{
"192.0.2.0/24": ts,
testV6Prefix: ts.Add(time.Hour),
}
err = db.UpdatePrefixesBatch(stored)
if err != nil {
t.Fatalf("UpdatePrefixesBatch: %v", err)
}
stored["198.51.100.0/24"] = time.Time{}
for prefix, want := range stored {
got, err := db.GetPrefixFirstSeen(t.Context(), netip.MustParsePrefix(prefix))
if err != nil || !got.Equal(want) {
t.Errorf("GetPrefixFirstSeen(%s) = %v, %v; want %v",
prefix, got, err, want)
}
}
}
// TestASPrefixQueriesUseCoveringIndexes checks the plan of each query that
// lists the prefixes an AS originates: a search of a covering index and no
// temporary B-tree, so the list reads one index entry per prefix, not one
internal/database/interface.go
+4 -2
View File
@@ -6,6 +6,7 @@ package database
import (
"context"
"net/netip"
"time"
"git.eeqj.de/sneak/routewatch/internal/routetable"
@@ -24,8 +25,6 @@ type Stats struct {
LiveRoutes int
IPv4Routes int
IPv6Routes int
OldestRoute *time.Time
NewestRoute *time.Time
IPv4PrefixDistribution []PrefixDistribution
IPv6PrefixDistribution []PrefixDistribution
}
@@ -85,6 +84,9 @@ type Store interface {
GetWHOISStats(ctx context.Context, staleThreshold time.Duration) (*WHOISStats, error)
// AS and prefix detail operations
GetAS(ctx context.Context, asn int) (*ASN, bool, error)
GetASWHOIS(ctx context.Context, asn int) (*ASN, error)
GetPrefixFirstSeen(ctx context.Context, prefix netip.Prefix) (time.Time, error)
GetASDetails(asn int) (*ASN, []LiveRoute, error)
GetASDetailsContext(ctx context.Context, asn int) (*ASN, []LiveRoute, error)
GetASPeers(asn int) ([]ASPeer, error)
internal/database/models.go
+9 -6
View File
@@ -66,7 +66,9 @@ type ASNPeering struct {
LastSeen time.Time
}
// LiveRoute represents a route in the live routing table
// LiveRoute represents a route in the live routing table. JSON leaves out an
// empty AS path, next hop or time: the routing table in memory keeps them only
// for the two newest routes of each prefix and origin AS.
//
//nolint:tagliatelle // field names are the existing JSON API
type LiveRoute struct {
@@ -75,9 +77,9 @@ type LiveRoute struct {
IPVersion int `json:"ip_version"`
OriginASN int `json:"origin_asn"`
PeerIP string `json:"peer_ip"`
ASPath []int `json:"as_path"`
NextHop string `json:"next_hop"`
LastUpdated time.Time `json:"last_updated"`
ASPath []int `json:"as_path,omitempty"`
NextHop string `json:"next_hop,omitempty"`
LastUpdated time.Time `json:"last_updated,omitzero"`
}
// PrefixDistribution represents the distribution of prefixes by mask length
@@ -108,9 +110,10 @@ type IPInfo struct {
CountryCode string `json:"country_code,omitempty"`
AbuseEmail string `json:"abuse_email,omitempty"`
RIR string `json:"rir,omitempty"`
// Timestamps
// Timestamps. LastSeen is zero, and left out of JSON, when the prefix
// keeps no route.
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
LastSeen time.Time `json:"last_seen,omitzero"`
// Indicates if WHOIS data needs refresh (not serialized)
NeedsWHOISRefresh bool `json:"-"`
}
internal/routewatch/app.go
+14 -15
View File
@@ -31,11 +31,13 @@ import (
// are disabled, since only the batched handlers exist.
var errNonBatchedNotImplemented = errors.New("non-batched handlers not implemented")
// Dependencies contains all dependencies for RouteWatch
// Dependencies contains all dependencies for RouteWatch. Table is the routing
// table read from the database at startup, which the server reads too.
type Dependencies struct {
fx.In
DB database.Store
Table *routetable.Table
Streamer *streamer.Streamer
Server *server.Server
Logger *logger.Logger
@@ -45,6 +47,7 @@ type Dependencies struct {
// RouteWatch represents the main application instance
type RouteWatch struct {
db database.Store
table *routetable.Table
streamer *streamer.Streamer
server *server.Server
logger *logger.Logger
@@ -69,6 +72,7 @@ const blocklistsDirName = "blocklists"
func New(deps Dependencies) *RouteWatch {
rw := &RouteWatch{
db: deps.DB,
table: deps.Table,
streamer: deps.Streamer,
server: deps.Server,
logger: deps.Logger,
@@ -97,13 +101,6 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
if rw.config.EnableBatchedDatabaseWrites {
rw.logger.Info("Using batched database handlers for improved performance")
// The routing table is read before the streamer connects, so that
// the prefix handler applies the feed's updates to all of it.
table, err := rw.readRouteTable(ctx)
if err != nil {
return err
}
// ASHandler maintains the asns table
rw.dbHandler = NewASHandler(rw.db, rw.logger)
rw.streamer.RegisterHandler(rw.dbHandler)
@@ -114,7 +111,7 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
// PrefixHandler maintains the prefixes and live_routes tables, and
// the routing table in memory and its copy in the routes table
rw.prefixHandler = NewPrefixHandler(rw.db, rw.logger, table)
rw.prefixHandler = NewPrefixHandler(rw.db, rw.logger, rw.table)
rw.prefixHandler.SetMetricsTracker(rw.streamer.GetMetricsTracker())
rw.streamer.RegisterHandler(rw.prefixHandler)
@@ -269,20 +266,21 @@ func (rw *RouteWatch) closeDatabase(ctx context.Context) {
// readRouteTable reads the routing table from the database and logs how many
// prefix and origin AS pairs and peers it holds and how long the read took.
// ctx is fx's start context, which ends 15 seconds after the start. Reading a
// full table can take longer, so the read does not stop with ctx.
func (rw *RouteWatch) readRouteTable(
ctx context.Context,
// fx calls it once, to build the server and RouteWatch, before the streamer
// connects, so the prefix handler applies the feed's updates to all of it. No
// start timeout limits the read: fx builds them before it runs a start hook.
func readRouteTable(
db database.Store, logger *logger.Logger,
) (*routetable.Table, error) {
start := time.Now()
table, err := rw.db.ReadRouteTable(context.WithoutCancel(ctx))
table, err := db.ReadRouteTable(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to read the routing table: %w", err)
}
counts := table.Counts()
rw.logger.Info("Read the routing table", "pairs", counts.Pairs,
logger.Info("Read the routing table", "pairs", counts.Pairs,
"peers", counts.Peers, "duration_ms", time.Since(start).Milliseconds())
return table, nil
@@ -299,6 +297,7 @@ func getModule() []fx.Option {
database.New,
fx.As(new(database.Store)),
),
readRouteTable,
streamer.New,
server.New,
New,
internal/routewatch/app_integration_internal_test.go
+34 -4
View File
@@ -7,6 +7,7 @@ import (
"net"
"net/http"
"net/http/httptest"
"net/netip"
"slices"
"strconv"
"strings"
@@ -265,6 +266,32 @@ func (m *mockStore) GetASDetailsContext(
return m.GetASDetails(asn)
}
// GetAS returns the stored AS, or only its number when none is stored.
func (m *mockStore) GetAS(_ context.Context, asn int) (*database.ASN, bool, error) {
m.mu.Lock()
defer m.mu.Unlock()
if asnInfo, exists := m.ASNs[asn]; exists {
return asnInfo, true, nil
}
return &database.ASN{ASN: asn}, false, nil
}
// GetASWHOIS returns the stored AS, or only its number when none is stored.
func (m *mockStore) GetASWHOIS(ctx context.Context, asn int) (*database.ASN, error) {
asnInfo, _, err := m.GetAS(ctx, asn)
return asnInfo, err
}
// GetPrefixFirstSeen returns the zero time: the mock keeps no prefix times.
func (m *mockStore) GetPrefixFirstSeen(
_ context.Context, _ netip.Prefix,
) (time.Time, error) {
return time.Time{}, nil
}
// GetPrefixDetails mock implementation
func (m *mockStore) GetPrefixDetails(
_ string,
@@ -502,12 +529,15 @@ func TestRouteWatchLiveFeed(t *testing.T) {
EnableBatchedDatabaseWrites: true,
}
table := routetable.New(routetable.Params{})
// Create server
srv := server.New(mockDB, s, logger, cfg)
srv := server.New(mockDB, table, s, logger, cfg)
// Create RouteWatch with 5 second limit
deps := Dependencies{
DB: mockDB,
Table: table,
Streamer: s,
Server: srv,
Logger: logger,
@@ -626,7 +656,7 @@ func TestShutdownWritesTheUpdatesQueuedInTheStreamer(t *testing.T) {
rw := New(Dependencies{
DB: store,
Streamer: s,
Server: server.New(store, s, log, cfg),
Server: server.New(store, nil, s, log, cfg),
Logger: log,
Config: cfg,
})
@@ -728,7 +758,7 @@ func TestShutdownReturnsByItsDeadlineWhileAWriteIsBlocked(t *testing.T) {
rw := New(Dependencies{
DB: store,
Streamer: s,
Server: server.New(store, s, log, cfg),
Server: server.New(store, nil, s, log, cfg),
Logger: log,
Config: cfg,
})
@@ -842,7 +872,7 @@ func TestShutdownCheckpointsAndClosesTheDatabaseLast(t *testing.T) {
rw := New(Dependencies{
DB: store,
Streamer: s,
Server: server.New(store, s, log, cfg),
Server: server.New(store, nil, s, log, cfg),
Logger: log,
Config: cfg,
})
internal/routewatch/prefixhandler_table_internal_test.go
+2 -6
View File
@@ -416,12 +416,8 @@ func TestARestartReadsTheRoutingTableBack(t *testing.T) {
var logs bytes.Buffer
rw := &RouteWatch{
db: db,
logger: &logger.Logger{Logger: slog.New(slog.NewJSONHandler(&logs, nil))},
}
table, err := rw.readRouteTable(t.Context())
table, err := readRouteTable(db,
&logger.Logger{Logger: slog.New(slog.NewJSONHandler(&logs, nil))})
if err != nil {
t.Fatalf("readRouteTable: %v", err)
}
internal/server/handlers.go
+322 -170
View File
@@ -5,12 +5,12 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"net"
"net/http"
"net/netip"
"runtime"
"slices"
"sort"
"strconv"
"strings"
"time"
@@ -19,6 +19,7 @@ import (
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/metrics"
"git.eeqj.de/sneak/routewatch/internal/ristypes"
"git.eeqj.de/sneak/routewatch/internal/routetable"
"git.eeqj.de/sneak/routewatch/internal/streamer"
"git.eeqj.de/sneak/routewatch/internal/templates"
asinfo "git.eeqj.de/sneak/routewatch/pkg/asinfo"
@@ -38,7 +39,7 @@ const (
const (
// maxLookupTargets is the most addresses and host names an IP lookup may
// list, and the most addresses it may have once the names are resolved:
// each address costs up to 113 database reads and a PTR lookup.
// each address costs two database reads and a PTR lookup.
maxLookupTargets = 10
// ptrLookupTimeout is each PTR lookup's own time limit, so that ten
@@ -227,7 +228,8 @@ type WHOISStatsInfo struct {
}
// Stats is the /status.json answer: uptime, message counts, database stats,
// and route information.
// and the routing table's figures. OldestRoute and NewestRoute are the times
// of the oldest and the newest kept routes.
//
//nolint:tagliatelle // field names are the existing JSON API
type Stats struct {
@@ -259,6 +261,10 @@ type Stats struct {
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
// OriginsWithoutKeptRoute counts the prefix and origin AS pairs that peers
// carry but whose kept routes were all withdrawn.
OriginsWithoutKeptRoute int `json:"origins_without_kept_route"`
}
// databaseStats gets the database stats, giving up when ctx is done; it then
@@ -317,15 +323,6 @@ func (s *Server) handleStatusJSON() http.HandlerFunc {
return
}
const bitsPerMegabit = 1000000.0
// Get route update metrics
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
// Get memory stats
var memStats runtime.MemStats
runtime.ReadMemStats(&memStats)
// Get WHOIS stats if fetcher is available
var whoisStats *WHOISStatsInfo
if s.asnFetcher != nil {
@@ -337,44 +334,80 @@ func (s *Server) handleStatusJSON() http.HandlerFunc {
}
}
stats := Stats{
Uptime: streamUptime(metrics.ConnectedSince),
TotalMessages: metrics.TotalMessages,
TotalBytes: metrics.TotalBytes,
TotalWireBytes: metrics.TotalWireBytes,
MessagesPerSec: metrics.MessagesPerSec,
MbitsPerSec: metrics.BitsPerSec / bitsPerMegabit,
WireMbitsPerSec: metrics.WireBitsPerSec / bitsPerMegabit,
Connected: metrics.Connected,
GoVersion: runtime.Version(),
Goroutines: runtime.NumGoroutine(),
MemoryUsage: humanize.Bytes(memStats.Alloc),
ASNs: dbStats.ASNs,
Prefixes: dbStats.Prefixes,
IPv4Prefixes: dbStats.IPv4Prefixes,
IPv6Prefixes: dbStats.IPv6Prefixes,
Peerings: dbStats.Peerings,
Peers: dbStats.Peers,
DatabaseSizeBytes: dbStats.FileSizeBytes,
LiveRoutes: dbStats.LiveRoutes,
IPv4Routes: dbStats.IPv4Routes,
IPv6Routes: dbStats.IPv6Routes,
OldestRoute: dbStats.OldestRoute,
NewestRoute: dbStats.NewestRoute,
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
IPv4PrefixDistribution: dbStats.IPv4PrefixDistribution,
IPv6PrefixDistribution: dbStats.IPv6PrefixDistribution,
WHOISStats: whoisStats,
}
err = writeJSONSuccess(w, stats)
err = writeJSONSuccess(w, s.statusStats(metrics, dbStats, whoisStats))
if err != nil {
s.logger.Error("Failed to encode stats", "error", err)
}
}
}
// statusStats builds the /status.json answer from the stream metrics, the
// database stats, the routing table's figures and the WHOIS stats, with the
// route update rates and memory use as they are now.
func (s *Server) statusStats(
streamMetrics metrics.StreamMetrics,
dbStats database.Stats,
whoisStats *WHOISStatsInfo,
) Stats {
const bitsPerMegabit = 1000000.0
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
counts := s.table.Counts()
var memStats runtime.MemStats
runtime.ReadMemStats(&memStats)
return Stats{
Uptime: streamUptime(streamMetrics.ConnectedSince),
TotalMessages: streamMetrics.TotalMessages,
TotalBytes: streamMetrics.TotalBytes,
TotalWireBytes: streamMetrics.TotalWireBytes,
MessagesPerSec: streamMetrics.MessagesPerSec,
MbitsPerSec: streamMetrics.BitsPerSec / bitsPerMegabit,
WireMbitsPerSec: streamMetrics.WireBitsPerSec / bitsPerMegabit,
Connected: streamMetrics.Connected,
GoVersion: runtime.Version(),
Goroutines: runtime.NumGoroutine(),
MemoryUsage: humanize.Bytes(memStats.Alloc),
ASNs: dbStats.ASNs,
Prefixes: counts.IPv4Prefixes + counts.IPv6Prefixes,
IPv4Prefixes: counts.IPv4Prefixes,
IPv6Prefixes: counts.IPv6Prefixes,
Peerings: dbStats.Peerings,
Peers: dbStats.Peers,
DatabaseSizeBytes: dbStats.FileSizeBytes,
LiveRoutes: counts.IPv4Routes + counts.IPv6Routes,
IPv4Routes: counts.IPv4Routes,
IPv6Routes: counts.IPv6Routes,
OldestRoute: counts.OldestKeptRoute,
NewestRoute: counts.NewestKeptRoute,
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
IPv4PrefixDistribution: prefixDistribution(counts.IPv4PrefixesByLength),
IPv6PrefixDistribution: prefixDistribution(counts.IPv6PrefixesByLength),
WHOISStats: whoisStats,
OriginsWithoutKeptRoute: counts.OriginsWithoutKeptRoute,
}
}
// prefixDistribution lists the mask lengths that have at least one prefix, in
// ascending order, with their prefix counts. countsByLength[n] is the number
// of prefixes of mask length n.
func prefixDistribution(countsByLength []int) []database.PrefixDistribution {
var distribution []database.PrefixDistribution
for maskLength, count := range countsByLength {
if count > 0 {
distribution = append(distribution, database.PrefixDistribution{
MaskLength: maskLength, Count: count,
})
}
}
return distribution
}
// getWHOISStats builds WHOIS statistics from database and fetcher.
func (s *Server) getWHOISStats(ctx context.Context) *WHOISStatsInfo {
// Get database WHOIS stats
@@ -470,7 +503,8 @@ type StreamStats struct {
BGPPeerCount int `json:"bgp_peer_count"`
}
// StatsResponse represents the API statistics response
// StatsResponse represents the API statistics response. The fields it shares
// with Stats mean what they mean there.
//
//nolint:tagliatelle // field names are the existing JSON API
type StatsResponse struct {
@@ -507,6 +541,8 @@ type StatsResponse struct {
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
OriginsWithoutKeptRoute int `json:"origins_without_kept_route"`
}
// handlerStatsInfo converts the streamer's handler statistics for the API
@@ -626,8 +662,8 @@ func (s *Server) handleStats() http.HandlerFunc {
}
// statsResponse builds the /api/v1/stats answer from the stream metrics, the
// database stats and the WHOIS stats, with the route update rates, memory use
// and handler queues as they are now.
// database stats, the routing table's figures and the WHOIS stats, with the
// route update rates, memory use and handler queues as they are now.
func (s *Server) statsResponse(
streamMetrics metrics.StreamMetrics,
dbStats database.Stats,
@@ -637,6 +673,7 @@ func (s *Server) statsResponse(
// Get route update metrics
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
counts := s.table.Counts()
// Get memory stats
var memStats runtime.MemStats
@@ -660,23 +697,25 @@ func (s *Server) statsResponse(
GC: gcStats(&memStats),
Stream: s.streamStats(),
ASNs: dbStats.ASNs,
Prefixes: dbStats.Prefixes,
IPv4Prefixes: dbStats.IPv4Prefixes,
IPv6Prefixes: dbStats.IPv6Prefixes,
Prefixes: counts.IPv4Prefixes + counts.IPv6Prefixes,
IPv4Prefixes: counts.IPv4Prefixes,
IPv6Prefixes: counts.IPv6Prefixes,
Peerings: dbStats.Peerings,
Peers: dbStats.Peers,
DatabaseSizeBytes: dbStats.FileSizeBytes,
LiveRoutes: dbStats.LiveRoutes,
IPv4Routes: dbStats.IPv4Routes,
IPv6Routes: dbStats.IPv6Routes,
OldestRoute: dbStats.OldestRoute,
NewestRoute: dbStats.NewestRoute,
LiveRoutes: counts.IPv4Routes + counts.IPv6Routes,
IPv4Routes: counts.IPv4Routes,
IPv6Routes: counts.IPv6Routes,
OldestRoute: counts.OldestKeptRoute,
NewestRoute: counts.NewestKeptRoute,
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
HandlerStats: handlerStatsInfo(s.streamer.GetHandlerStats()),
IPv4PrefixDistribution: dbStats.IPv4PrefixDistribution,
IPv6PrefixDistribution: dbStats.IPv6PrefixDistribution,
IPv4PrefixDistribution: prefixDistribution(counts.IPv4PrefixesByLength),
IPv6PrefixDistribution: prefixDistribution(counts.IPv6PrefixesByLength),
WHOISStats: whoisStats,
OriginsWithoutKeptRoute: counts.OriginsWithoutKeptRoute,
}
}
@@ -766,7 +805,7 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
// Look up each IP
for _, ip := range ipsToLookup {
ipInfo, err := s.db.GetIPInfoContext(ctx, ip)
ipInfo, err := s.ipInfo(ctx, ip)
if deadlinePassed(ctx) {
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
@@ -815,6 +854,81 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
}
}
// ipInfo returns the IP lookup's answer for ip. The routing table gives the
// most specific prefix that holds the address, the number of peers that carry
// it, the origin AS that most of them carry, and the time of its newest kept
// route. The database gives the prefix's first-seen time and the AS's WHOIS
// details. It returns ErrNoRoute when no prefix holds the address, and
// ErrInvalidIP when ip does not parse.
func (s *Server) ipInfo(ctx context.Context, ip string) (*database.IPInfo, error) {
addr, err := netip.ParseAddr(ip)
if err != nil {
return nil, fmt.Errorf("%w: %s", database.ErrInvalidIP, ip)
}
summary, found := s.table.Lookup(addr)
if !found {
return nil, fmt.Errorf("%w for IP %s", database.ErrNoRoute, ip)
}
firstSeen, err := s.db.GetPrefixFirstSeen(ctx, summary.Prefix)
if err != nil {
return nil, err
}
asInfo, err := s.db.GetASWHOIS(ctx, summary.OriginASN)
if err != nil {
return nil, err
}
info := &database.IPInfo{
IP: ip,
Netblock: summary.Prefix.String(),
MaskLength: summary.Prefix.Bits(),
IPVersion: ipVersionOf(summary.Prefix),
NumPeers: summary.Peers,
ASN: summary.OriginASN,
ASName: asInfo.ASName,
Handle: asInfo.Handle,
Description: asInfo.Description,
OrgName: asInfo.OrgName,
OrgID: asInfo.OrgID,
Address: asInfo.Address,
CountryCode: asInfo.CountryCode,
AbuseEmail: asInfo.AbuseEmail,
RIR: asInfo.RIR,
FirstSeen: firstSeen,
NeedsWHOISRefresh: needsWHOISRefresh(asInfo),
}
if summary.Newest != nil {
info.LastSeen = *summary.Newest
}
return info, nil
}
// needsWHOISRefresh says whether the WHOIS details of asInfo were never
// fetched or are older than whoisStaleThreshold. AS 0, no known origin, has
// none to fetch.
func needsWHOISRefresh(asInfo *database.ASN) bool {
if asInfo.ASN == ristypes.NoOrigin {
return false
}
return asInfo.WHOISUpdatedAt == nil ||
time.Since(*asInfo.WHOISUpdatedAt) > whoisStaleThreshold
}
// ipVersionOf returns 4 or 6, the IP version of prefix.
func ipVersionOf(prefix netip.Prefix) int {
if prefix.Addr().Is4() {
return ipVersionV4
}
return ipVersionV6
}
// blocklistListings returns the blocklists that list ip. It returns an empty
// list, never nil, so that an answer's blocklists is [] rather than null.
func (s *Server) blocklistListings(ip string) []blocklist.Listing {
@@ -1043,7 +1157,7 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
return
}
asInfo, prefixes, err := s.db.GetASDetailsContext(r.Context(), asn)
asInfo, prefixes, err := s.asDetails(r.Context(), asn)
if deadlinePassed(r.Context()) {
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
@@ -1079,6 +1193,49 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
}
}
// asDetails returns the AS's handle, description and seen times, from the
// database, and the prefixes it originates, from the routing table. Each
// prefix has the time of its newest kept route, if it keeps one. It returns
// ErrNoRoute for AS 0, which means no known origin. It also returns ErrNoRoute
// for an AS that the database, the AS list built into RouteWatch and the
// routing table all lack.
func (s *Server) asDetails(
ctx context.Context, asn int,
) (*database.ASN, []database.LiveRoute, error) {
if asn == ristypes.NoOrigin {
return nil, nil, fmt.Errorf("%w: AS%d", database.ErrNoRoute, asn)
}
asInfo, known, err := s.db.GetAS(ctx, asn)
if err != nil {
return nil, nil, err
}
originated := s.table.OriginatedPrefixes(asn)
if !known && len(originated) == 0 {
return nil, nil, fmt.Errorf("%w: AS%d", database.ErrNoRoute, asn)
}
prefixes := make([]database.LiveRoute, 0, len(originated))
for _, p := range originated {
route := database.LiveRoute{
Prefix: p.Prefix.String(),
MaskLength: p.Prefix.Bits(),
IPVersion: ipVersionOf(p.Prefix),
OriginASN: asn,
}
if p.Newest != nil {
route.LastUpdated = *p.Newest
}
prefixes = append(prefixes, route)
}
return asInfo, prefixes, nil
}
// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones. Neither
// group is nil, so that a JSON answer holds [] for an empty one and not null.
func splitRoutesByIPVersion(
@@ -1123,8 +1280,7 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
return
}
// Look the prefix up in the text form net/netip prints, the form the
// tables store, so that any valid way of writing it finds the route.
// Masking the prefix lets any valid way of writing it find its routes.
parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
if err != nil {
s.writeJSONError(w, http.StatusBadRequest, invalidPrefixMessage)
@@ -1132,27 +1288,18 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
return
}
prefix := parsed.Masked().String()
parsed = parsed.Masked()
prefix := parsed.String()
routes, _, err := s.db.GetPrefixDetailsContext(r.Context(), prefix)
if deadlinePassed(r.Context()) {
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
origins := s.table.Origins(parsed)
if len(origins) == 0 {
s.writeJSONError(w, http.StatusNotFound,
database.ErrNoRoute.Error()+": "+prefix)
return
}
if err != nil {
if errors.Is(err, database.ErrNoRoute) {
s.writeJSONError(w, http.StatusNotFound, err.Error())
} else {
s.logger.Error("Failed to get prefix details",
"prefix", prefix, "error", err)
s.writeJSONError(w, http.StatusInternalServerError,
internalErrorMessage)
}
return
}
routes := prefixRoutes(parsed, origins)
// Group by origin AS
originMap := make(map[int][]database.LiveRoute)
@@ -1237,7 +1384,7 @@ func (s *Server) handleASDetail() http.HandlerFunc {
return
}
asInfo, prefixes, err := s.db.GetASDetailsContext(r.Context(), asn)
asInfo, prefixes, err := s.asDetails(r.Context(), asn)
if deadlinePassed(r.Context()) {
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
@@ -1327,8 +1474,7 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
return
}
// Look the prefix up in the text form net/netip prints, the form the
// tables store, so that any valid way of writing it finds the route.
// Masking the prefix lets any valid way of writing it find its routes.
parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
if err != nil {
http.Error(w, invalidPrefixMessage, http.StatusBadRequest)
@@ -1336,9 +1482,16 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
return
}
prefix := parsed.Masked().String()
parsed = parsed.Masked()
routes, originNames, err := s.db.GetPrefixDetailsContext(r.Context(), prefix)
origins := s.table.Origins(parsed)
if len(origins) == 0 {
http.Error(w, "Prefix not found", http.StatusNotFound)
return
}
originInfos, err := s.originASNs(r.Context(), origins)
if deadlinePassed(r.Context()) {
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
@@ -1346,27 +1499,13 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
}
if err != nil {
if errors.Is(err, database.ErrNoRoute) {
http.Error(w, "Prefix not found", http.StatusNotFound)
} else {
s.logger.Error("Failed to get prefix details", "error", err)
http.Error(w, internalErrorMessage, http.StatusInternalServerError)
}
s.logger.Error("Failed to get the origin ASes' names", "error", err)
http.Error(w, internalErrorMessage, http.StatusInternalServerError)
return
}
// Group by origin AS and collect unique AS info
origins := originASNs(routes, originNames)
// Get the first route to extract some common info
var maskLength, ipVersion int
if len(routes) > 0 {
maskLength, ipVersion = prefixMaskLengthAndIPVersion(prefix)
}
// Create enhanced routes with AS path handles
enhancedRoutes := routesWithASPathHandles(routes)
routes := prefixRoutes(parsed, origins)
// Prepare template data
data := struct {
@@ -1379,13 +1518,13 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
OriginCount int
CSPNonce string
}{
Prefix: prefix,
MaskLength: maskLength,
IPVersion: ipVersion,
Routes: enhancedRoutes,
Origins: origins,
Prefix: parsed.String(),
MaskLength: parsed.Bits(),
IPVersion: ipVersionOf(parsed),
Routes: routesWithASPathHandles(routes),
Origins: originInfos,
PeerCount: len(routes),
OriginCount: len(origins),
OriginCount: len(originInfos),
CSPNonce: cspNonce(r.Context()),
}
@@ -1415,34 +1554,72 @@ type ASNInfo struct {
PeerCount int
}
// originASNs groups routes by origin AS, with each AS's handle and description
// from names, as the prefix query returned them, and the number of routes it
// originates. The origin ASes come in no particular order.
func originASNs(
routes []database.LiveRoute,
names map[int]database.ASN,
) []*ASNInfo {
originMap := make(map[int]*ASNInfo)
// originASNs returns each of origins, a prefix's origin ASes, with its handle
// and description from the database and the number of peers that carry the
// prefix with it.
func (s *Server) originASNs(
ctx context.Context, origins []routetable.Origin,
) ([]*ASNInfo, error) {
infos := make([]*ASNInfo, 0, len(origins))
for _, route := range routes {
if _, exists := originMap[route.OriginASN]; !exists {
originMap[route.OriginASN] = &ASNInfo{
ASN: route.OriginASN,
Handle: names[route.OriginASN].Handle,
Description: names[route.OriginASN].Description,
PeerCount: 0,
}
for _, origin := range origins {
asInfo, _, err := s.db.GetAS(ctx, origin.ASN)
if err != nil {
return nil, err
}
originMap[route.OriginASN].PeerCount++
infos = append(infos, &ASNInfo{
ASN: origin.ASN,
Handle: asInfo.Handle,
Description: asInfo.Description,
PeerCount: len(origin.Peers),
})
}
origins := make([]*ASNInfo, 0, len(originMap))
for _, origin := range originMap {
origins = append(origins, origin)
return infos, nil
}
// prefixRoutes returns one route for each peer that carries prefix, by origin
// AS in the order of origins. Each origin's kept routes come first, newest
// first, with their AS path, next hop and time. Its other peers follow without
// them, since the routing table keeps only their addresses.
func prefixRoutes(
prefix netip.Prefix, origins []routetable.Origin,
) []database.LiveRoute {
var routes []database.LiveRoute
for _, origin := range origins {
base := database.LiveRoute{
Prefix: prefix.String(),
MaskLength: prefix.Bits(),
IPVersion: ipVersionOf(prefix),
OriginASN: origin.ASN,
}
kept := make(map[string]bool, len(origin.Routes))
for _, r := range origin.Routes {
kept[r.Peer] = true
route := base
route.PeerIP = r.Peer
route.ASPath = r.ASPath
// The next hop is the peer, as the prefix handler records it.
route.NextHop = r.Peer
route.LastUpdated = r.Time
routes = append(routes, route)
}
for _, peer := range origin.Peers {
if !kept[peer] {
route := base
route.PeerIP = peer
routes = append(routes, route)
}
}
}
return origins
return routes
}
// ASPathEntry is one AS of a route's AS path, with its handle.
@@ -1484,23 +1661,6 @@ func routesWithASPathHandles(routes []database.LiveRoute) []EnhancedRoute {
return enhancedRoutes
}
// prefixMaskLengthAndIPVersion returns the mask length and IP version of a
// prefix in CIDR notation, or zeros if it does not parse.
func prefixMaskLengthAndIPVersion(prefix string) (int, int) {
_, ipNet, err := net.ParseCIDR(prefix)
if err != nil {
return 0, 0
}
maskLength, _ := ipNet.Mask.Size()
if ipNet.IP.To4() != nil {
return maskLength, ipVersionV4
}
return maskLength, ipVersionV6
}
// handlePrefixLength shows a random sample of IPv4 prefixes with the
// specified mask length
func (s *Server) handlePrefixLength() http.HandlerFunc {
@@ -1539,29 +1699,11 @@ func (s *Server) prefixLengthHandler(ipVersion, maxMaskLength int) http.HandlerF
// Get random sample of prefixes
const maxPrefixes = 500
prefixes, err := s.db.GetRandomPrefixesByLengthContext(
r.Context(), maskLength, ipVersion, maxPrefixes)
if deadlinePassed(r.Context()) {
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
return
}
if err != nil {
s.logger.Error("Failed to get prefixes by length", "error", err)
http.Error(w, internalErrorMessage, http.StatusInternalServerError)
return
}
prefixes := s.table.SamplePrefixes(ipVersion, maskLength, maxPrefixes)
// Sort prefixes for display
sort.Slice(prefixes, func(i, j int) bool {
// First compare by IP version
if prefixes[i].IPVersion != prefixes[j].IPVersion {
return prefixes[i].IPVersion < prefixes[j].IPVersion
}
// Then by prefix
return prefixes[i].Prefix < prefixes[j].Prefix
slices.SortFunc(prefixes, func(a, b routetable.PrefixSummary) int {
return a.Prefix.Compare(b.Prefix)
})
// Render template
@@ -1591,8 +1733,8 @@ func (s *Server) prefixLengthHandler(ipVersion, maxMaskLength int) http.HandlerF
}
}
// EnhancedPrefix is a live route on the prefix length page, with the
// description of its origin AS and its age.
// EnhancedPrefix is a prefix on the prefix length page, with the origin AS
// that most peers carry it with, that AS's description, and its age.
type EnhancedPrefix struct {
database.LiveRoute
@@ -1601,13 +1743,23 @@ type EnhancedPrefix struct {
}
// prefixesWithASDescriptions looks up the description of the origin AS of
// each prefix and gives each one's age.
func prefixesWithASDescriptions(prefixes []database.LiveRoute) []EnhancedPrefix {
// each prefix. It gives each prefix the age of its newest kept route, or no
// age when it keeps none.
func prefixesWithASDescriptions(
prefixes []routetable.PrefixSummary,
) []EnhancedPrefix {
enhancedPrefixes := make([]EnhancedPrefix, len(prefixes))
for i, prefix := range prefixes {
enhancedPrefixes[i] = EnhancedPrefix{
LiveRoute: prefix,
Age: formatAge(prefix.LastUpdated),
Prefix: prefix.Prefix.String(),
MaskLength: prefix.Prefix.Bits(),
IPVersion: ipVersionOf(prefix.Prefix),
OriginASN: prefix.OriginASN,
}
if prefix.Newest != nil {
enhancedPrefixes[i].LastUpdated = *prefix.Newest
enhancedPrefixes[i].Age = formatAge(*prefix.Newest)
}
// Get AS description
internal/server/handlers_internal_test.go
+230 -246
View File
@@ -9,6 +9,7 @@ import (
"net/http"
"net/http/httptest"
"net/netip"
"reflect"
"runtime"
"slices"
"strconv"
@@ -23,6 +24,7 @@ import (
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/metrics"
"git.eeqj.de/sneak/routewatch/internal/ristypes"
"git.eeqj.de/sneak/routewatch/internal/routetable"
"git.eeqj.de/sneak/routewatch/internal/streamer"
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
)
@@ -35,6 +37,38 @@ const (
testPeerV6 = "2001:db8::1"
)
// testPeer returns the address of the nth of the IPv4 peers that tests
// announce routes from: 198.51.100.n.
func testPeer(n int) string {
return "198.51.100." + strconv.Itoa(n)
}
// testRouteTime is the time the tests' routes are announced at, or a number of
// minutes after it.
func testRouteTime(minutes int) time.Time {
return time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC).
Add(time.Duration(minutes) * time.Minute)
}
// announce has peer announce prefix in table at time at, with an AS path
// that holds only the origin AS asn.
func announce(
table *routetable.Table, prefix string, asn int, peer string, at time.Time,
) {
table.Announce(netip.MustParsePrefix(prefix), asn,
routetable.Route{Peer: peer, ASPath: []int{asn}, Time: at})
}
// withdraw has peer withdraw prefix in table.
func withdraw(table *routetable.Table, prefix, peer string) {
table.Withdraw(netip.MustParsePrefix(prefix), peer)
}
// emptyTable returns a routing table that holds no route.
func emptyTable() *routetable.Table {
return routetable.New(routetable.Params{})
}
// Paths several tests request.
const (
asPagePath = "/as/64500"
@@ -76,7 +110,8 @@ func (d blockingStatsDB) GetStatsContext(_ context.Context) (database.Stats, err
func TestStatsHandlersDoNotLeakOnTimeout(t *testing.T) {
release := make(chan struct{})
db := blockingStatsDB{release: release}
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
s := New(db, emptyTable(), streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
handlers := map[string]http.HandlerFunc{
statusJSONHandlerName: s.handleStatusJSON(),
@@ -108,51 +143,78 @@ func TestStatsHandlersDoNotLeakOnTimeout(t *testing.T) {
}
}
// tableStats holds the figures of a stats answer that come from the routing
// table.
//
//nolint:tagliatelle // field names are the existing JSON API
type tableStats struct {
Prefixes int `json:"prefixes"`
IPv4Prefixes int `json:"ipv4_prefixes"`
IPv6Prefixes int `json:"ipv6_prefixes"`
LiveRoutes int `json:"live_routes"`
IPv4Routes int `json:"ipv4_routes"`
IPv6Routes int `json:"ipv6_routes"`
OriginsWithoutKeptRoute int `json:"origins_without_kept_route"`
OldestRoute time.Time `json:"oldest_route"`
NewestRoute time.Time `json:"newest_route"`
IPv4Distribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6Distribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
}
// TestStatsHandlersAnswerFromMemory checks that both stats handlers answer 200
// with the live route counts and the prefix distribution while the database is
// closed, so that any query would fail: the request path reads them from
// memory. The prefix distribution query it used to run read every live route
// and, on a large database, took the whole 4-second deadline, so
// with the routing table's figures while the database is closed, so that any
// query would fail. The prefix distribution query they used to run read every
// live route and, on a large database, took the whole 4-second deadline, so
// /api/v1/stats answered 500 (https://git.eeqj.de/sneak/routewatch/issues/30).
// The oldest and newest route times still come from one-row lookups at the ends
// of an index; with the database closed they are left out of the answer.
//
// The table holds two IPv4 prefixes and an IPv6 one. Of the three peers that
// carried 203.0.113.0/24, the two with kept routes withdrew it, so its origin
// keeps no route. The oldest and newest routes are those of the other two
// prefixes.
func TestStatsHandlersAnswerFromMemory(t *testing.T) {
t.Parallel()
const withdrawnPrefix = "203.0.113.0/24"
table := emptyTable()
announce(table, testPrefixV4, 64502, testPeer(1), testRouteTime(0))
for n := 1; n <= 3; n++ {
announce(table, withdrawnPrefix, 64500, testPeer(n), testRouteTime(n))
}
withdraw(table, withdrawnPrefix, testPeer(2))
withdraw(table, withdrawnPrefix, testPeer(3))
announce(table, testPrefixV6, 64501, testPeerV6, testRouteTime(4))
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
if err != nil {
t.Fatalf("database.New: %v", err)
}
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{
{
Prefix: "198.51.100.0/24", MaskLength: 24, IPVersion: 4,
OriginASN: 64500, PeerIP: "192.0.2.1", ASPath: []int{64500},
NextHop: "192.0.2.1", LastUpdated: ts,
},
{
Prefix: "2001:db8::/32", MaskLength: 32, IPVersion: 6,
OriginASN: 64501, PeerIP: testPeerV6, ASPath: []int{64501},
NextHop: testPeerV6, LastUpdated: ts,
},
})
if err != nil {
t.Fatalf("UpsertLiveRouteBatch: %v", err)
}
err = db.Close()
if err != nil {
t.Fatalf("Close: %v", err)
}
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
s := New(db, table, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{})
handlers := map[string]http.HandlerFunc{
statusJSONHandlerName: s.handleStatusJSON(),
statsHandlerName: s.handleStats(),
}
want := tableStats{
Prefixes: 3, IPv4Prefixes: 2, IPv6Prefixes: 1,
LiveRoutes: 3, IPv4Routes: 2, IPv6Routes: 1,
OriginsWithoutKeptRoute: 1,
OldestRoute: testRouteTime(0),
NewestRoute: testRouteTime(4),
IPv4Distribution: []database.PrefixDistribution{{MaskLength: 24, Count: 2}},
IPv6Distribution: []database.PrefixDistribution{{MaskLength: 32, Count: 1}},
}
for name, handler := range handlers {
rec := httptest.NewRecorder()
handler(rec, httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil))
@@ -164,14 +226,8 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
continue
}
//nolint:tagliatelle // field names are the existing JSON API
var body struct {
Data struct {
IPv4Routes int `json:"ipv4_routes"`
IPv6Routes int `json:"ipv6_routes"`
IPv4Distribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6Distribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
} `json:"data"`
Data tableStats `json:"data"`
}
err := json.Unmarshal(rec.Body.Bytes(), &body)
@@ -179,21 +235,16 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
t.Fatalf("%s: decoding the answer: %v", name, err)
}
if body.Data.IPv4Routes != 1 || body.Data.IPv6Routes != 1 {
t.Errorf("%s: routes = (v4 %d, v6 %d), want (1, 1)",
name, body.Data.IPv4Routes, body.Data.IPv6Routes)
got := body.Data
if !got.OldestRoute.Equal(want.OldestRoute) ||
!got.NewestRoute.Equal(want.NewestRoute) {
t.Errorf("%s: routes from %v to %v, want from %v to %v", name,
got.OldestRoute, got.NewestRoute, want.OldestRoute, want.NewestRoute)
}
wantV4 := []database.PrefixDistribution{{MaskLength: 24, Count: 1}}
if !slices.Equal(body.Data.IPv4Distribution, wantV4) {
t.Errorf("%s: IPv4 distribution = %v, want %v",
name, body.Data.IPv4Distribution, wantV4)
}
wantV6 := []database.PrefixDistribution{{MaskLength: 32, Count: 1}}
if !slices.Equal(body.Data.IPv6Distribution, wantV6) {
t.Errorf("%s: IPv6 distribution = %v, want %v",
name, body.Data.IPv6Distribution, wantV6)
got.OldestRoute, got.NewestRoute = want.OldestRoute, want.NewestRoute
if !reflect.DeepEqual(got, want) {
t.Errorf("%s: figures %+v, want %+v", name, got, want)
}
}
}
@@ -243,11 +294,29 @@ func TestStatsAnswerCarriesChurnPerMinute(t *testing.T) {
}
}
// newServerWithTestRoutes returns a server on a new database that holds a
// route for testPrefixV4 and one for testPrefixV6.
// newTestTable returns a routing table that holds a route for testPrefixV4,
// with origin AS 64500, and one for testPrefixV6, with origin AS 64501.
func newTestTable() *routetable.Table {
table := emptyTable()
announce(table, testPrefixV4, 64500, testPeerV4, testRouteTime(0))
announce(table, testPrefixV6, 64501, testPeerV6, testRouteTime(0))
return table
}
// newServerWithTestRoutes returns a server on a new database that reads the
// routes of newTestTable.
func newServerWithTestRoutes(t *testing.T) *Server {
t.Helper()
return newServerWithTable(t, newTestTable())
}
// newServerWithTable returns a server on a new database that reads routes from
// table.
func newServerWithTable(t *testing.T, table *routetable.Table) *Server {
t.Helper()
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
if err != nil {
t.Fatalf("database.New: %v", err)
@@ -255,25 +324,7 @@ func newServerWithTestRoutes(t *testing.T) *Server {
t.Cleanup(func() { _ = db.Close() })
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{
{
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
OriginASN: 64500, PeerIP: testPeerV4, ASPath: []int{64500},
NextHop: testPeerV4, LastUpdated: ts,
},
{
Prefix: testPrefixV6, MaskLength: 32, IPVersion: 6,
OriginASN: 64501, PeerIP: testPeerV6, ASPath: []int{64501},
NextHop: testPeerV6, LastUpdated: ts,
},
})
if err != nil {
t.Fatalf("UpsertLiveRouteBatch: %v", err)
}
return New(db, streamer.New(logger.New(), metrics.New()), logger.New(),
return New(db, table, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{})
}
@@ -402,34 +453,20 @@ func TestPrefixDetailShowsOriginNames(t *testing.T) {
t.Cleanup(func() { _ = db.Close() })
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
origins := []int{15169, 13335}
table := emptyTable()
for _, asn := range origins {
_, err := db.GetOrCreateASN(asn, ts)
for i, asn := range origins {
_, err := db.GetOrCreateASN(asn, testRouteTime(0))
if err != nil {
t.Fatalf("GetOrCreateASN(%d): %v", asn, err)
}
announce(table, testPrefixV4, asn, testPeer(i+1), testRouteTime(0))
}
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{
{
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
OriginASN: origins[0], PeerIP: testPeerV4,
ASPath: []int{origins[0]}, NextHop: testPeerV4, LastUpdated: ts,
},
{
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
OriginASN: origins[1], PeerIP: "198.51.100.2",
ASPath: []int{origins[1]}, NextHop: "198.51.100.2", LastUpdated: ts,
},
})
if err != nil {
t.Fatalf("UpsertLiveRouteBatch: %v", err)
}
s := New(failingASDetailsDB{db}, streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
s := New(failingASDetailsDB{db}, table,
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
rec := httptest.NewRecorder()
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/prefix/"+testPrefixV4, nil))
@@ -466,23 +503,14 @@ func TestPrefixDetailShowsOriginNames(t *testing.T) {
func TestPagesSayOriginZeroIsNoKnownOrigin(t *testing.T) {
t.Parallel()
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
if err != nil {
t.Fatalf("database.New: %v", err)
}
table := emptyTable()
table.Announce(netip.MustParsePrefix(testPrefixV4), ristypes.NoOrigin,
routetable.Route{
Peer: testPeerV4, ASPath: []int{64500, 64501, 64502},
Time: testRouteTime(0),
})
t.Cleanup(func() { _ = db.Close() })
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{{
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
OriginASN: 0, PeerIP: testPeerV4, ASPath: []int{64500, 64501, 64502},
NextHop: testPeerV4, LastUpdated: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC),
}})
if err != nil {
t.Fatalf("UpsertLiveRouteBatch: %v", err)
}
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
s := newServerWithTable(t, table)
for _, path := range []string{"/prefix/" + testPrefixV4, prefixLengthPagePath} {
rec := httptest.NewRecorder()
@@ -533,8 +561,9 @@ func TestIndexShowsTheClientAddress(t *testing.T) {
}
for _, tt := range tests {
// The home page reads nothing from the database, so the server has none.
s := New(nil, streamer.New(logger.New(), metrics.New()), logger.New(),
// The home page reads no route and nothing from the database, so the
// server has neither a routing table nor a database.
s := New(nil, nil, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{TrustedProxies: tt.trustedProxies})
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
@@ -571,13 +600,6 @@ func TestIndexShowsTheClientAddress(t *testing.T) {
func TestPagesShowAgesInWords(t *testing.T) {
t.Parallel()
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
if err != nil {
t.Fatalf("database.New: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
// Each age is well past a whole number of its unit, so the time the test
// takes to run does not change the words.
ages := []struct {
@@ -591,22 +613,13 @@ func TestPagesShowAgesInWords(t *testing.T) {
}
now := time.Now().UTC()
routes := make([]*database.LiveRoute, 0, len(ages))
table := emptyTable()
for _, a := range ages {
routes = append(routes, &database.LiveRoute{
Prefix: a.prefix, MaskLength: 24, IPVersion: 4,
OriginASN: 64500, PeerIP: testPeerV4, ASPath: []int{64500},
NextHop: testPeerV4, LastUpdated: now.Add(-a.age),
})
announce(table, a.prefix, 64500, testPeerV4, now.Add(-a.age))
}
err = db.UpsertLiveRouteBatch(routes)
if err != nil {
t.Fatalf("UpsertLiveRouteBatch: %v", err)
}
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
s := newServerWithTable(t, table)
for _, a := range ages {
cell := `<td class="age">` + a.want + `</td>`
@@ -643,7 +656,7 @@ const shortRequestDeadline = 50 * time.Millisecond
func TestRequestPastTimeLimitAnswers504(t *testing.T) {
t.Parallel()
s := New(nil, streamer.New(logger.New(), metrics.New()), logger.New(),
s := New(nil, nil, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{})
waitForTimeLimit := func(_ http.ResponseWriter, r *http.Request) {
@@ -693,8 +706,8 @@ func TestStatsHandlersAnswer504WhenDatabaseReadRunsOut(t *testing.T) {
defer close(release)
db := blockingStatsDB{release: release}
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{})
s := New(db, emptyTable(), streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
handlers := map[string]http.HandlerFunc{
statusJSONHandlerName: s.handleStatusJSON(),
@@ -720,10 +733,9 @@ func TestStatsHandlersAnswer504WhenDatabaseReadRunsOut(t *testing.T) {
// slowDB embeds database.Store (left nil) and overrides the reads the routes
// in TestRoutesAnswer504WhenDatabaseReadIsCutOff make. Each read waits until
// its context ends and then answers as the real store does when the time
// limit cuts it off: the IP read with the context's error, the prefix read
// that there is no route, and the others with what they had read, here
// nothing, and no error. Every other Store method is unused and would panic
// if called.
// limit cuts it off: the stats read with what it had read, here nothing, and
// no error, and the others with the context's error. Every other Store method
// is unused and would panic if called.
type slowDB struct {
database.Store
}
@@ -734,66 +746,44 @@ func (slowDB) GetStatsContext(ctx context.Context) (database.Stats, error) {
return database.Stats{}, nil
}
func (slowDB) GetIPInfoContext(
ctx context.Context, _ string,
) (*database.IPInfo, error) {
func (slowDB) GetAS(ctx context.Context, _ int) (*database.ASN, bool, error) {
<-ctx.Done()
return nil, ctx.Err()
return nil, false, ctx.Err()
}
func (slowDB) GetASDetailsContext(
ctx context.Context, asn int,
) (*database.ASN, []database.LiveRoute, error) {
func (slowDB) GetPrefixFirstSeen(
ctx context.Context, _ netip.Prefix,
) (time.Time, error) {
<-ctx.Done()
return &database.ASN{ASN: asn}, nil, nil
}
func (slowDB) GetPrefixDetailsContext(
ctx context.Context, _ string,
) ([]database.LiveRoute, map[int]database.ASN, error) {
<-ctx.Done()
return nil, nil, database.ErrNoRoute
}
func (slowDB) GetRandomPrefixesByLengthContext(
ctx context.Context, _, _, _ int,
) ([]database.LiveRoute, error) {
<-ctx.Done()
return nil, nil
return time.Time{}, ctx.Err()
}
// TestRoutesAnswer504WhenDatabaseReadIsCutOff sends a request through the
// server's router for each route that reads the database, with a store whose
// reads are cut off when the request's time limit passes. Each route answers
// 504 with the time limit message. Without its check after the read, a route
// answers with what the cut-off read returned: 500, 404, 200, or, on a page
// that stops before rendering once the request has ended, 504 with an empty
// body. The health check is left out: it answers 503 when its own 2-second
// ping runs out.
// answers with what the cut-off read returned: 500 or 200. The health check is
// left out: it answers 503 when its own 2-second ping runs out. The prefix
// length pages and the prefix API read only the routing table.
func TestRoutesAnswer504WhenDatabaseReadIsCutOff(t *testing.T) {
t.Parallel()
s := New(slowDB{}, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{})
s := New(slowDB{}, newTestTable(), streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
tests := []struct {
path string
wantJSONBody bool
}{
{path: asPagePath},
{path: "/prefix/192.0.2.0/24"},
{path: prefixLengthPagePath},
{path: "/prefixlength6/48"},
{path: "/prefix/" + testPrefixV4},
{path: statusJSONPath, wantJSONBody: true},
{path: "/ip/192.0.2.1", wantJSONBody: true},
{path: "/ip/" + addrInTestPrefixV4, wantJSONBody: true},
{path: statsPath, wantJSONBody: true},
{path: "/api/v1/ip/192.0.2.1", wantJSONBody: true},
{path: "/api/v1/ip/" + addrInTestPrefixV4, wantJSONBody: true},
{path: "/api/v1/as/64500", wantJSONBody: true},
{path: "/api/v1/prefix/192.0.2.0/24", wantJSONBody: true},
}
for _, tt := range tests {
@@ -810,10 +800,8 @@ type slowSecondReadDB struct {
database.Store
}
func (slowSecondReadDB) GetASDetailsContext(
_ context.Context, asn int,
) (*database.ASN, []database.LiveRoute, error) {
return &database.ASN{ASN: asn}, nil, nil
func (slowSecondReadDB) GetAS(_ context.Context, asn int) (*database.ASN, bool, error) {
return &database.ASN{ASN: asn}, true, nil
}
func (slowSecondReadDB) GetASPeersContext(
@@ -855,8 +843,8 @@ func (idleASNFetcher) GetStats() ASNFetcherStats {
func TestRoutesAnswer504WhenSecondDatabaseReadIsCutOff(t *testing.T) {
t.Parallel()
s := New(slowSecondReadDB{}, streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
s := New(slowSecondReadDB{}, emptyTable(),
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
s.SetASNFetcher(idleASNFetcher{})
tests := []struct {
@@ -950,34 +938,27 @@ func (failingDB) GetStatsContext(_ context.Context) (database.Stats, error) {
return database.Stats{}, errDiskIO
}
func (failingDB) GetIPInfoContext(
_ context.Context, _ string,
) (*database.IPInfo, error) {
return nil, errDiskIO
func (failingDB) GetAS(_ context.Context, _ int) (*database.ASN, bool, error) {
return nil, false, errDiskIO
}
func (failingDB) GetASDetailsContext(
_ context.Context, _ int,
) (*database.ASN, []database.LiveRoute, error) {
return nil, nil, errDiskIO
func (failingDB) GetPrefixFirstSeen(
_ context.Context, _ netip.Prefix,
) (time.Time, error) {
return time.Time{}, errDiskIO
}
func (failingDB) GetPrefixDetailsContext(
_ context.Context, _ string,
) ([]database.LiveRoute, map[int]database.ASN, error) {
return nil, nil, errDiskIO
}
// TestJSONRoutesDoNotSendErrorDetails sends a request to each JSON route and
// the health check with a store whose reads fail with the driver's error text,
// and a prefix that does not parse, and checks that no answer carries the
// driver's or the prefix parser's text. Before the fix the answers sent it to
// the client (https://git.eeqj.de/sneak/routewatch/issues/108).
// TestJSONRoutesDoNotSendErrorDetails sends a request to each JSON route that
// reads the database, and to the health check, with a store whose reads fail
// with the driver's error text. It also sends a prefix that does not parse.
// No answer may carry the driver's or the prefix parser's text. Before the fix
// the answers sent it to the client
// (https://git.eeqj.de/sneak/routewatch/issues/108).
func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
t.Parallel()
s := New(failingDB{}, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{})
s := New(failingDB{}, newTestTable(), streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
tests := []struct {
path string
@@ -987,8 +968,7 @@ func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
{statusJSONPath, http.StatusInternalServerError},
{statsPath, http.StatusInternalServerError},
{"/api/v1/as/64500", http.StatusInternalServerError},
{"/api/v1/prefix/192.0.2.0/24", http.StatusInternalServerError},
{"/api/v1/ip/192.0.2.1", http.StatusInternalServerError},
{"/api/v1/ip/" + addrInTestPrefixV4, http.StatusInternalServerError},
{"/api/v1/prefix/foo/24", http.StatusBadRequest},
}
@@ -1011,45 +991,47 @@ func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
}
}
// The addresses failingIPLookupDB fails the read of and finds a route for.
// The address whose read failingIPLookupDB fails, in testPrefixV4, and one it
// reads, in testPrefixV6.
const (
failingLookupIP = "192.0.2.1"
routedLookupIP = "192.0.2.2"
failingLookupIP = addrInTestPrefixV4
routedLookupIP = addrInTestPrefixV6
)
// failingIPLookupDB embeds database.Store (left nil) and answers the IP lookup
// with errDiskIO for failingLookupIP, a route for routedLookupIP, and no route
// for any other address. Every other Store method is unused and would panic if
// called.
// failingIPLookupDB embeds database.Store (left nil). It fails the read of
// testPrefixV4's first-seen time with errDiskIO, and answers the IP lookup's
// other reads with nothing stored. Every other Store method is unused and
// would panic if called.
type failingIPLookupDB struct {
database.Store
}
func (failingIPLookupDB) GetIPInfoContext(
_ context.Context, ip string,
) (*database.IPInfo, error) {
switch ip {
case failingLookupIP:
return nil, errDiskIO
case routedLookupIP:
return &database.IPInfo{IP: ip}, nil
default:
return nil, database.ErrNoRoute
func (failingIPLookupDB) GetPrefixFirstSeen(
_ context.Context, prefix netip.Prefix,
) (time.Time, error) {
if prefix == netip.MustParsePrefix(testPrefixV4) {
return time.Time{}, errDiskIO
}
return time.Time{}, nil
}
func (failingIPLookupDB) GetASWHOIS(_ context.Context, asn int) (*database.ASN, error) {
return &database.ASN{ASN: asn}, nil
}
// TestIPLookupAnswers500WhenDatabaseReadFails sends IP lookups through the
// server's router with a store whose read of one address fails with the
// server's router with a store whose read for one address fails with the
// driver's error. A lookup that asks for that address answers 500 with the
// JSON error body, as the AS and prefix JSON routes do; an address with no
// route answers 200. Before the fix the failed read was reported as if the
// address had no route: 404 for that address alone, and 200 beside a routed
// address (https://git.eeqj.de/sneak/routewatch/issues/109).
// JSON error body, as the AS JSON route does; an address with no route answers
// 200. Before the fix the failed read was reported as if the address had no
// route: 404 for that address alone, and 200 beside a routed address
// (https://git.eeqj.de/sneak/routewatch/issues/109).
func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
t.Parallel()
s := New(failingIPLookupDB{}, streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
s := New(failingIPLookupDB{}, newTestTable(),
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
s.resolver = &fakeResolver{}
tests := []struct {
@@ -1122,30 +1104,37 @@ func (r *fakeResolver) LookupAddr(ctx context.Context, _ string) ([]string, erro
return nil, errNoSuchHost
}
// countingIPLookupDB embeds database.Store (left nil) and counts its IP
// lookups, answering each that the address has no route. Every other Store
// method is unused and would panic if called.
// countingIPLookupDB embeds database.Store (left nil) and answers the IP
// lookup's reads with nothing stored. It counts the lookups, as the reads of a
// prefix's first-seen time. Every other Store method is unused and would panic
// if called.
type countingIPLookupDB struct {
database.Store
lookups atomic.Int64
}
func (d *countingIPLookupDB) GetIPInfoContext(
_ context.Context, _ string,
) (*database.IPInfo, error) {
func (d *countingIPLookupDB) GetPrefixFirstSeen(
_ context.Context, _ netip.Prefix,
) (time.Time, error) {
d.lookups.Add(1)
return nil, database.ErrNoRoute
return time.Time{}, nil
}
func (d *countingIPLookupDB) GetASWHOIS(
_ context.Context, asn int,
) (*database.ASN, error) {
return &database.ASN{ASN: asn}, nil
}
// TestIPLookupRefusesTooManyAddresses sends IP lookups of maxLookupTargets
// addresses and of one more, listed or as a host name that resolves to them,
// and a lookup listing maxLookupTargets+1 host names. The longer lookups answer
// 400 without reading the database, and the list of host names without
// resolving any of them; the others read the database once per address. Before
// the fix every address was read and every name resolved, however many there
// were (https://git.eeqj.de/sneak/routewatch/issues/110).
// addresses in testPrefixV4 and of one more, listed or as a host name that
// resolves to them, and a lookup listing maxLookupTargets+1 host names. The
// longer lookups answer 400 without reading the database, and the list of host
// names without resolving any of them; the others read the database for each
// address. Before the fix every address was read and every name resolved,
// however many there were (https://git.eeqj.de/sneak/routewatch/issues/110).
func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
t.Parallel()
@@ -1180,8 +1169,8 @@ func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
for _, tt := range tests {
db := &countingIPLookupDB{}
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(),
&config.Config{})
s := New(db, newTestTable(), streamer.New(logger.New(), metrics.New()),
logger.New(), &config.Config{})
resolver := &fakeResolver{hosts: hosts}
s.resolver = resolver
@@ -1901,16 +1890,11 @@ func TestNoOriginHasNoASAnswers(t *testing.T) {
const noOriginPrefix = "203.0.113.0/24"
s := newServerWithTestRoutes(t)
err := s.db.UpsertLiveRouteBatch([]*database.LiveRoute{{
Prefix: noOriginPrefix, MaskLength: 24, IPVersion: 4,
OriginASN: ristypes.NoOrigin, PeerIP: testPeerV4,
ASPath: []int{64500, 64501, 64502}, NextHop: testPeerV4,
LastUpdated: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC),
}})
if err != nil {
t.Fatalf("UpsertLiveRouteBatch: %v", err)
}
s.table.Announce(netip.MustParsePrefix(noOriginPrefix), ristypes.NoOrigin,
routetable.Route{
Peer: testPeerV4, ASPath: []int{64500, 64501, 64502},
Time: testRouteTime(0),
})
asn := strconv.Itoa(ristypes.NoOrigin)
internal/server/handlers_table_internal_test.go
+368
View File
@@ -0,0 +1,368 @@
package server
import (
"encoding/json"
"net/http"
"net/http/httptest"
"slices"
"strconv"
"strings"
"testing"
"time"
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/routetable"
)
// The origin ASes in the routing table that newTwoOriginTable fills.
const (
mostPeersASN = 64500
fewerPeersASN = 64501
v6OriginASN = 64502
)
// noKeptRoutePrefix is the prefix of newTwoOriginTable whose origin keeps no
// route.
const noKeptRoutePrefix = "198.51.100.0/24"
// routeTimeFormat is how the pages write a route's time.
const routeTimeFormat = "2006-01-02 15:04:05"
// newTwoOriginTable returns a routing table that holds:
// - testPrefixV4 with two origin ASes. Peers 1, 2 and 3 carry it with
// mostPeersASN, which keeps the routes of peers 3 and 2. Peer 4 carries
// it with fewerPeersASN, in the prefix's newest route.
// - noKeptRoutePrefix with mostPeersASN. Only peer 7 carries it, and its
// route is not kept: peers 5 and 6, whose routes were, withdrew it.
// - testPrefixV6 with v6OriginASN.
func newTwoOriginTable() *routetable.Table {
table := emptyTable()
for n := 1; n <= 3; n++ {
announce(table, testPrefixV4, mostPeersASN, testPeer(n), testRouteTime(n))
}
announce(table, testPrefixV4, fewerPeersASN, testPeer(4), testRouteTime(5))
announce(table, noKeptRoutePrefix, mostPeersASN, testPeer(5), testRouteTime(4))
announce(table, noKeptRoutePrefix, mostPeersASN, testPeer(6), testRouteTime(6))
announce(table, noKeptRoutePrefix, mostPeersASN, testPeer(7), testRouteTime(0))
withdraw(table, noKeptRoutePrefix, testPeer(5))
withdraw(table, noKeptRoutePrefix, testPeer(6))
announce(table, testPrefixV6, v6OriginASN, testPeerV6, testRouteTime(0))
return table
}
// get sends a GET for path through s's router and fails the test unless it
// answers 200.
func get(t *testing.T, s *Server, path string) *httptest.ResponseRecorder {
t.Helper()
rec := httptest.NewRecorder()
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, path, nil))
if rec.Code != http.StatusOK {
t.Fatalf("%s: status %d, want %d; body %s",
path, rec.Code, http.StatusOK, rec.Body)
}
return rec
}
// TestIPLookupGivesTheOriginMostPeersCarry looks up an address in a prefix
// with two origin ASes, through each IP lookup route. The answer names the
// origin AS that more peers carry, counts every peer, and gives the time of
// the prefix's newest kept route. The prefix's first-seen time and the AS's
// WHOIS details come from the database.
func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
t.Parallel()
s := newServerWithTable(t, newTwoOriginTable())
s.resolver = &fakeResolver{}
firstSeen := testRouteTime(-1)
err := s.db.UpdatePrefixesBatch(map[string]time.Time{testPrefixV4: firstSeen})
if err != nil {
t.Fatalf("UpdatePrefixesBatch: %v", err)
}
_, err = s.db.GetOrCreateASN(mostPeersASN, firstSeen)
if err != nil {
t.Fatalf("GetOrCreateASN: %v", err)
}
whois := &database.ASNWHOISUpdate{
ASN: mostPeersASN, ASName: "TEST-AS", CountryCode: "DE",
}
err = s.db.UpdateASNWHOIS(t.Context(), whois)
if err != nil {
t.Fatalf("UpdateASNWHOIS: %v", err)
}
// ownAddressPath looks up the request's own address, which httptest
// gives as addrInTestPrefixV4.
for _, path := range []string{
"/api/v1/ip/" + addrInTestPrefixV4, "/ip/" + addrInTestPrefixV4,
ownAddressPath,
} {
var body struct {
Data IPLookupResponse `json:"data"`
}
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
if err != nil || len(body.Data.Results) != 1 ||
body.Data.Results[0].IPInfo == nil {
t.Fatalf("%s: answer %+v (%v), want one result with a route",
path, body.Data, err)
}
got := body.Data.Results[0].IPInfo
if got.Netblock != testPrefixV4 || got.ASN != mostPeersASN ||
got.NumPeers != 4 || !got.LastSeen.Equal(testRouteTime(5)) ||
!got.FirstSeen.Equal(firstSeen) || got.ASName != whois.ASName ||
got.CountryCode != whois.CountryCode {
t.Errorf("%s: route %+v, want %s from AS%d, 4 peers, last seen %v, "+
"first seen %v, %s in %s", path, *got, testPrefixV4,
mostPeersASN, testRouteTime(5), firstSeen, whois.ASName,
whois.CountryCode)
}
}
}
// prefixRoute is one route of the prefix API's answer. A field the answer
// leaves out stays nil.
//
//nolint:tagliatelle // field names are the existing JSON API
type prefixRoute struct {
OriginASN int `json:"origin_asn"`
PeerIP string `json:"peer_ip"`
ASPath []int `json:"as_path"`
NextHop *string `json:"next_hop"`
LastUpdated *time.Time `json:"last_updated"`
}
// TestPrefixAnswersListEveryPeer asks the prefix API and page for a prefix
// with two origin ASes, and for one whose origin keeps no route. Each answer
// lists every peer that carries the prefix, by origin AS. Only the kept routes
// have an AS path, a next hop and a time, newest first. The other peers follow
// without them.
func TestPrefixAnswersListEveryPeer(t *testing.T) {
t.Parallel()
s := newServerWithTable(t, newTwoOriginTable())
// keptAt[peer] is the time of the peer's kept route; a peer not in it is
// listed without a route.
keptAt := map[string]time.Time{
testPeer(3): testRouteTime(3),
testPeer(2): testRouteTime(2),
testPeer(4): testRouteTime(5),
}
tests := []struct {
prefix string
wantPeers []string
wantOrigins []int
}{
{
testPrefixV4,
[]string{testPeer(3), testPeer(2), testPeer(1), testPeer(4)},
[]int{mostPeersASN, mostPeersASN, mostPeersASN, fewerPeersASN},
},
{noKeptRoutePrefix, []string{testPeer(7)}, []int{mostPeersASN}},
}
for _, tt := range tests {
//nolint:tagliatelle // field names are the existing JSON API
var body struct {
Data struct {
Routes []prefixRoute `json:"routes"`
PeerCount int `json:"peer_count"`
OriginCount int `json:"origin_count"`
} `json:"data"`
}
path := "/api/v1/prefix/" + tt.prefix
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
if err != nil {
t.Fatalf("%s: decoding the answer: %v", path, err)
}
wantOriginCount := len(slices.Compact(slices.Clone(tt.wantOrigins)))
if body.Data.PeerCount != len(tt.wantPeers) ||
body.Data.OriginCount != wantOriginCount {
t.Errorf("%s: %d peers and %d origins, want %d and %d", path,
body.Data.PeerCount, body.Data.OriginCount, len(tt.wantPeers),
wantOriginCount)
}
checkPrefixRoutes(t, path, body.Data.Routes, tt.wantPeers, tt.wantOrigins,
keptAt)
checkPrefixPage(t, s, tt.prefix, tt.wantPeers, keptAt)
}
}
// checkPrefixRoutes checks that routes come from wantPeers, in that order,
// with wantOrigins as their origin ASes. The route of a peer in keptAt must
// have an AS path, the peer as next hop, and the time keptAt gives. Any other
// route must have none of the three.
func checkPrefixRoutes(
t *testing.T, path string, routes []prefixRoute, wantPeers []string,
wantOrigins []int, keptAt map[string]time.Time,
) {
t.Helper()
if len(routes) != len(wantPeers) {
t.Fatalf("%s: routes %+v, want one each from %v", path, routes, wantPeers)
}
for i, route := range routes {
if route.PeerIP != wantPeers[i] || route.OriginASN != wantOrigins[i] {
t.Errorf("%s: route %d from %s with AS%d, want from %s with AS%d",
path, i, route.PeerIP, route.OriginASN, wantPeers[i], wantOrigins[i])
}
at, kept := keptAt[route.PeerIP]
if !kept {
if route.ASPath != nil || route.NextHop != nil || route.LastUpdated != nil {
t.Errorf("%s: route from %s is not kept but has %+v",
path, route.PeerIP, route)
}
continue
}
if route.ASPath == nil || route.NextHop == nil ||
*route.NextHop != route.PeerIP || route.LastUpdated == nil ||
!route.LastUpdated.Equal(at) {
t.Errorf("%s: kept route from %s is %+v, want its AS path, the peer "+
"as next hop and the time %v", path, route.PeerIP, route, at)
}
}
}
// checkPrefixPage checks that the page of prefix shows each of wantPeers, the
// time of each kept route among them, and no time that was never set.
func checkPrefixPage(
t *testing.T, s *Server, prefix string, wantPeers []string,
keptAt map[string]time.Time,
) {
t.Helper()
path := "/prefix/" + prefix
page := get(t, s, path).Body.String()
for _, peer := range wantPeers {
want := `<td class="peer-ip">` + peer + `</td>`
if !strings.Contains(page, want) {
t.Errorf("%s does not list the peer %s", path, peer)
}
at, kept := keptAt[peer]
if kept && !strings.Contains(page, at.Format(routeTimeFormat)) {
t.Errorf("%s does not show the time %v of the route from %s",
path, at, peer)
}
}
if strings.Contains(page, zeroDate) {
t.Errorf("%s shows the date %s", path, zeroDate)
}
}
// TestASAnswersGiveEachPrefixItsNewestKeptRoute asks the AS API and page for
// mostPeersASN. Its prefixes are testPrefixV4, whose newest kept route with
// that origin AS is peer 3's, and noKeptRoutePrefix, which keeps no route. The
// first has that route's time. The second is listed without a time.
func TestASAnswersGiveEachPrefixItsNewestKeptRoute(t *testing.T) {
t.Parallel()
s := newServerWithTable(t, newTwoOriginTable())
path := "/api/v1/as/" + strconv.Itoa(mostPeersASN)
//nolint:tagliatelle // field names are the existing JSON API
var body struct {
Data struct {
IPv4Prefixes []struct {
Prefix string `json:"prefix"`
LastUpdated *time.Time `json:"last_updated"`
} `json:"ipv4_prefixes"`
} `json:"data"`
}
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
if err != nil {
t.Fatalf("%s: decoding the answer: %v", path, err)
}
prefixes := body.Data.IPv4Prefixes
if len(prefixes) != 2 || prefixes[0].Prefix != testPrefixV4 ||
prefixes[0].LastUpdated == nil ||
!prefixes[0].LastUpdated.Equal(testRouteTime(3)) ||
prefixes[1].Prefix != noKeptRoutePrefix || prefixes[1].LastUpdated != nil {
t.Errorf("%s: prefixes %+v, want %s updated %v and %s without a time",
path, prefixes, testPrefixV4, testRouteTime(3), noKeptRoutePrefix)
}
path = "/as/" + strconv.Itoa(mostPeersASN)
page := get(t, s, path).Body.String()
for _, want := range []string{
testPrefixV4, noKeptRoutePrefix, testRouteTime(3).Format(routeTimeFormat),
} {
if !strings.Contains(page, want) {
t.Errorf("%s does not show %s", path, want)
}
}
if strings.Contains(page, zeroDate) {
t.Errorf("%s shows the date %s", path, zeroDate)
}
}
// TestPrefixLengthPagesShowTheOriginMostPeersCarry asks for the IPv4 /24
// page and the IPv6 /32 page. The first lists both /24 prefixes with
// mostPeersASN, the origin AS more peers carry testPrefixV4 with, and gives
// noKeptRoutePrefix no age. The second lists testPrefixV6.
func TestPrefixLengthPagesShowTheOriginMostPeersCarry(t *testing.T) {
t.Parallel()
s := newServerWithTable(t, newTwoOriginTable())
page := get(t, s, prefixLengthPagePath).Body.String()
for _, want := range []string{
">" + testPrefixV4 + "<", ">" + noKeptRoutePrefix + "<",
`href="/as/` + strconv.Itoa(mostPeersASN) + `"`,
} {
if !strings.Contains(page, want) {
t.Errorf("%s does not show %s", prefixLengthPagePath, want)
}
}
if strings.Contains(page, `href="/as/`+strconv.Itoa(fewerPeersASN)+`"`) {
t.Errorf("%s links to AS%d, which fewer peers carry",
prefixLengthPagePath, fewerPeersASN)
}
const noAge = `<td class="age"></td>`
if strings.Count(page, noAge) != 1 {
t.Errorf("%s gives %d prefixes no age, want 1 (%s)",
prefixLengthPagePath, strings.Count(page, noAge), noKeptRoutePrefix)
}
const v6PagePath = "/prefixlength6/32"
page = get(t, s, v6PagePath).Body.String()
if !strings.Contains(page, ">"+testPrefixV6+"<") ||
!strings.Contains(page, `href="/as/`+strconv.Itoa(v6OriginASN)+`"`) {
t.Errorf("%s does not show %s with AS%d", v6PagePath, testPrefixV6,
v6OriginASN)
}
}
internal/server/server.go
+10 -1
View File
@@ -3,6 +3,10 @@
// - the AS, prefix and prefix length pages, and the JSON answers behind them
// - the IP lookups
// - the AS name search
//
// Every route it shows comes from the routing table in memory. AS names,
// WHOIS details and the times a prefix and an AS were first seen come from
// the database.
package server
import (
@@ -19,6 +23,7 @@ import (
"git.eeqj.de/sneak/routewatch/internal/config"
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/routetable"
"git.eeqj.de/sneak/routewatch/internal/streamer"
"github.com/go-chi/chi/v5"
)
@@ -55,6 +60,7 @@ type dnsResolver interface {
type Server struct {
router *chi.Mux
db database.Store
table *routetable.Table
streamer *streamer.Streamer
logger *logger.Logger
port int
@@ -71,15 +77,18 @@ type Server struct {
resolver dnsResolver
}
// New creates a new HTTP server
// New creates a new HTTP server that reads routes from table, the routing
// table that the prefix handler keeps current.
func New(
db database.Store,
table *routetable.Table,
streamer *streamer.Streamer,
logger *logger.Logger,
cfg *config.Config,
) *Server {
s := &Server{
db: db,
table: table,
streamer: streamer,
logger: logger,
port: cfg.Port,
internal/server/server_test.go
+1 -1
View File
@@ -34,7 +34,7 @@ func TestStartFailsWhenPortIsHeld(t *testing.T) {
port := addr.Port
log := logger.New()
s := server.New(nil, streamer.New(log, metrics.New()), log,
s := server.New(nil, nil, streamer.New(log, metrics.New()), log,
&config.Config{Port: port})
defer func() { _ = s.Stop(context.Background()) }()
internal/templates/as_detail.html
+4 -4
View File
@@ -281,8 +281,8 @@
<tr>
<td><a href="{{.Prefix | prefixURL}}" class="prefix-link">{{.Prefix}}</a></td>
<td>/{{.MaskLength}}</td>
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
<td class="age">{{.LastUpdated | timeSince}}</td>
<td>{{if not .LastUpdated.IsZero}}{{.LastUpdated.Format "2006-01-02 15:04:05"}}{{end}}</td>
<td class="age">{{if not .LastUpdated.IsZero}}{{.LastUpdated | timeSince}}{{end}}</td>
</tr>
{{end}}
</tbody>
@@ -310,8 +310,8 @@
<tr>
<td><a href="{{.Prefix | prefixURL}}" class="prefix-link">{{.Prefix}}</a></td>
<td>/{{.MaskLength}}</td>
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
<td class="age">{{.LastUpdated | timeSince}}</td>
<td>{{if not .LastUpdated.IsZero}}{{.LastUpdated.Format "2006-01-02 15:04:05"}}{{end}}</td>
<td class="age">{{if not .LastUpdated.IsZero}}{{.LastUpdated | timeSince}}{{end}}</td>
</tr>
{{end}}
</tbody>
internal/templates/prefix_detail.html
+2 -2
View File
@@ -329,8 +329,8 @@
<td class="peer-ip">{{.PeerIP}}</td>
<td class="as-path">{{range $i, $as := .ASPathWithHandle}}{{if $i}} → {{end}}<a href="/as/{{$as.ASN}}" class="as-link">{{if $as.Handle}}{{$as.Handle}}{{else}}AS{{$as.ASN}}{{end}}</a>{{end}}</td>
<td class="peer-ip">{{.NextHop}}</td>
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
<td class="age">{{.LastUpdated | timeSince}}</td>
<td>{{if not .LastUpdated.IsZero}}{{.LastUpdated.Format "2006-01-02 15:04:05"}}{{end}}</td>
<td class="age">{{if not .LastUpdated.IsZero}}{{.LastUpdated | timeSince}}{{end}}</td>
</tr>
{{end}}
</tbody>
internal/templates/status.html
+6
View File
@@ -318,6 +318,10 @@
<span class="metric-label">IPv6 Routes</span>
<span class="metric-value" id="ipv6_routes">-</span>
</div>
<div class="metric">
<span class="metric-label">Origins Without a Kept Route</span>
<span class="metric-value" id="origins_without_kept_route">-</span>
</div>
<div class="metric">
<span class="metric-label">IPv4 Updates/sec</span>
<span class="metric-value" id="ipv4_updates_per_sec">-</span>
@@ -571,6 +575,7 @@
document.getElementById('live_routes').textContent = '-';
document.getElementById('ipv4_routes').textContent = '-';
document.getElementById('ipv6_routes').textContent = '-';
document.getElementById('origins_without_kept_route').textContent = '-';
document.getElementById('ipv4_updates_per_sec').textContent = '-';
document.getElementById('ipv6_updates_per_sec').textContent = '-';
document.getElementById('oldest_route').textContent = '-';
@@ -634,6 +639,7 @@
document.getElementById('live_routes').textContent = formatNumber(data.live_routes);
document.getElementById('ipv4_routes').textContent = formatNumber(data.ipv4_routes);
document.getElementById('ipv6_routes').textContent = formatNumber(data.ipv6_routes);
document.getElementById('origins_without_kept_route').textContent = formatNumber(data.origins_without_kept_route);
document.getElementById('ipv4_updates_per_sec').textContent = formatNumber(Math.round(data.ipv4_updates_per_sec));
document.getElementById('ipv6_updates_per_sec').textContent = formatNumber(Math.round(data.ipv6_updates_per_sec));
document.getElementById('oldest_route').textContent = formatRelativeTime(data.oldest_route);