Compare commits
1
Commits
prod
...
16db3250bf
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
16db3250bf |
@@ -12,16 +12,16 @@ table in real time. It works like this:
|
|||||||
RIPE RIS Live streams the BGP updates that RIPE's route collectors receive, as
|
RIPE RIS Live streams the BGP updates that RIPE's route collectors receive, as
|
||||||
they happen, from the moment a client connects. It reports changes, not the
|
they happen, from the moment a client connects. It reports changes, not the
|
||||||
routing table they add up to, so it cannot say which AS an IP address is routed
|
routing table they add up to, so it cannot say which AS an IP address is routed
|
||||||
to now or which routes a prefix has. RouteWatch applies the feed's updates to
|
to now or which routes a prefix has. RouteWatch applies the feed's updates to a
|
||||||
live routing tables in its own SQLite database and answers IP-to-AS lookups and
|
routing table that it keeps in memory, with a copy in its own SQLite database.
|
||||||
prefix pages from them.
|
It answers IP-to-AS lookups and prefix pages from that table.
|
||||||
|
|
||||||
The feed sends no table when a client connects, so these tables hold only the
|
The feed sends no table when a client connects. So the routing table holds only
|
||||||
routes the feed has announced since RouteWatch first started: an IP-to-AS lookup
|
the routes announced since RouteWatch first started. An IP-to-AS lookup finds
|
||||||
finds nothing for an address whose routes have not been announced since then.
|
nothing for an address whose routes have not been announced since then. The
|
||||||
The tables also miss the updates RouteWatch drops when it falls behind under
|
routing table also misses updates RouteWatch drops when it falls behind, and
|
||||||
load and those sent while it is stopped, so a lookup or a prefix page can show a
|
updates sent while it is stopped. So a lookup or a prefix page can show a route
|
||||||
route the feed has withdrawn.
|
the feed has withdrawn.
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
@@ -58,13 +58,24 @@ DEBUG=routewatch ./routewatch
|
|||||||
|
|
||||||
Every route answers `HEAD` as it answers `GET`, without the body.
|
Every route answers `HEAD` as it answers `GET`, without the body.
|
||||||
|
|
||||||
|
The pages and the API read routes from the routing table that RouteWatch keeps
|
||||||
|
in memory. For each prefix and origin AS, the table keeps the two newest routes
|
||||||
|
in full, with their AS paths and times. These are the kept routes. The table
|
||||||
|
keeps only the address of every other peer that carries the prefix. When both
|
||||||
|
kept routes of an origin AS are withdrawn, that origin has no kept route until
|
||||||
|
one of its peers sends an update.
|
||||||
|
|
||||||
### Web Interface
|
### Web Interface
|
||||||
|
|
||||||
- `GET /` - Home page (HTML): headline figures, lookups by AS number and by IP
|
- `GET /` - Home page (HTML): headline figures, lookups by AS number and by IP
|
||||||
address, and a search of AS names. A card shows the `curl` command that looks
|
address, and a search of AS names. A card shows the `curl` command that looks
|
||||||
up the visitor's own address at `/ip/{addr}`, and that command's live answer.
|
up the visitor's own address at `/ip/{addr}`, and that command's live answer.
|
||||||
- `GET /status` - HTML status dashboard, which reads `/api/v1/stats`
|
- `GET /status` - HTML status dashboard, which reads `/api/v1/stats`
|
||||||
- `GET /status.json` - JSON statistics: feed, memory, database and route figures
|
- `GET /status.json` - JSON statistics: feed, memory, database and route
|
||||||
|
figures. The route figures come from the routing table. `oldest_route` and
|
||||||
|
`newest_route` are the times of the oldest and the newest kept routes.
|
||||||
|
`origins_without_kept_route` counts, across all prefixes, the origin ASes that
|
||||||
|
have no kept route.
|
||||||
- `GET /.well-known/healthcheck.json` - Health check (JSON): 200 when the
|
- `GET /.well-known/healthcheck.json` - Health check (JSON): 200 when the
|
||||||
database answers and the RIS Live feed is connected, otherwise 503;
|
database answers and the RIS Live feed is connected, otherwise 503;
|
||||||
`data.status` is `ok` or `error` and `data.checks` says which check failed
|
`data.status` is `ok` or `error` and `data.checks` says which check failed
|
||||||
@@ -73,10 +84,11 @@ Every route answers `HEAD` as it answers `GET`, without the body.
|
|||||||
description come from the AS list built into RouteWatch. The page then shows
|
description come from the AS list built into RouteWatch. The page then shows
|
||||||
no first-seen date.
|
no first-seen date.
|
||||||
- `GET /prefix/{prefix}/{length}` - Prefix detail page (HTML), such as
|
- `GET /prefix/{prefix}/{length}` - Prefix detail page (HTML), such as
|
||||||
`/prefix/192.0.2.0/24`: every live route for the prefix, one per peer, and its
|
`/prefix/192.0.2.0/24`: its origin ASes and every peer that carries it. Only
|
||||||
origin ASes
|
the kept routes show an AS path, a next hop and a time.
|
||||||
- `GET /prefixlength/{length}` - Up to 500 IPv4 prefixes with that mask length,
|
- `GET /prefixlength/{length}` - Up to 500 IPv4 prefixes with that mask length,
|
||||||
picked at random, each with its newest route (HTML)
|
picked at random (HTML). Each shows the origin AS that most peers carry it
|
||||||
|
with, and the age of its newest kept route.
|
||||||
- `GET /prefixlength6/{length}` - The same for IPv6
|
- `GET /prefixlength6/{length}` - The same for IPv6
|
||||||
- `GET /ip/{addr}` - The same JSON answer as `/api/v1/ip/{ip}`
|
- `GET /ip/{addr}` - The same JSON answer as `/api/v1/ip/{ip}`
|
||||||
- `GET /ip/` - The same, for the caller's own address: the address the request
|
- `GET /ip/` - The same, for the caller's own address: the address the request
|
||||||
@@ -93,8 +105,10 @@ Every route answers `HEAD` as it answers `GET`, without the body.
|
|||||||
name, or a comma-separated list of at most 10 of these. `results` has one
|
name, or a comma-separated list of at most 10 of these. `results` has one
|
||||||
entry for each address, with:
|
entry for each address, with:
|
||||||
- `ip` and `ptr`: the address and its PTR names
|
- `ip` and `ptr`: the address and its PTR names
|
||||||
- `netblock` and the fields beside it: the address's most specific live
|
- `netblock` and the fields beside it: the most specific prefix that holds
|
||||||
route, and that route's origin AS with its WHOIS details
|
the address, and its origin AS with that AS's WHOIS details. For a prefix
|
||||||
|
with several origin ASes, `asn` is the one that most peers carry.
|
||||||
|
`last_seen` is the time of the prefix's newest kept route.
|
||||||
- `blocklists`: the public blocklists the address is on (see Blocklists)
|
- `blocklists`: the public blocklists the address is on (see Blocklists)
|
||||||
|
|
||||||
An address with no live route has a result without the route and AS fields.
|
An address with no live route has a result without the route and AS fields.
|
||||||
@@ -110,11 +124,13 @@ Every route answers `HEAD` as it answers `GET`, without the body.
|
|||||||
answered without PTR names.
|
answered without PTR names.
|
||||||
|
|
||||||
- `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6
|
- `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6
|
||||||
apart, each with when its newest route was updated. When the database has no
|
apart, each with the time of its newest kept route. A prefix that keeps no
|
||||||
record of the AS, its handle and description come from the AS list built into
|
route with the AS has no `last_updated`. When the database has no record of
|
||||||
|
the AS, its handle and description come from the AS list built into
|
||||||
RouteWatch. The answer then has no `first_seen` or `last_seen`.
|
RouteWatch. The answer then has no `first_seen` or `last_seen`.
|
||||||
- `GET /api/v1/prefix/{prefix}/{length}` - Every live route for the prefix, one
|
- `GET /api/v1/prefix/{prefix}/{length}` - One route for each peer that carries
|
||||||
per peer, and the same routes grouped by origin AS in `origins`
|
the prefix, and the same routes grouped by origin AS in `origins`. Only the
|
||||||
|
kept routes have `as_path`, `next_hop` and `last_updated`.
|
||||||
- `GET /api/v1/search/as?q={text}` - Finds the ASes whose handle or description
|
- `GET /api/v1/search/as?q={text}` - Finds the ASes whose handle or description
|
||||||
contains `{text}`, ignoring case. `data` holds:
|
contains `{text}`, ignoring case. `data` holds:
|
||||||
- `query`: `{text}` without its leading and trailing spaces
|
- `query`: `{text}` without its leading and trailing spaces
|
||||||
@@ -137,16 +153,16 @@ took. The health check is the exception: its 503 answer is also
|
|||||||
A request that runs past its time limit answers 504: the limit is 30 seconds,
|
A request that runs past its time limit answers 504: the limit is 30 seconds,
|
||||||
and 4 seconds for the database reads behind `/status.json` and `/api/v1/stats`.
|
and 4 seconds for the database reads behind `/status.json` and `/api/v1/stats`.
|
||||||
An AS number, prefix or mask length that does not parse answers 400, as does a
|
An AS number, prefix or mask length that does not parse answers 400, as does a
|
||||||
mask length out of range for its IP version. An AS answers 404 when it is not in
|
mask length out of range for its IP version. An AS answers 404 when it
|
||||||
the AS list built into RouteWatch and the database has neither a record of it
|
originates no prefix and has no record in the database or in the AS list built
|
||||||
nor a live route it originates. AS 0 always answers 404, because it means no
|
into RouteWatch. AS 0 always answers 404, because it means no known origin. The
|
||||||
known origin. The AS name search never lists it. A prefix with no live route
|
AS name search never lists it. A prefix with no live route answers 404 too. The
|
||||||
answers 404 too. The HTML pages give these errors as plain text. Under `/api/v1`
|
HTML pages give these errors as plain text. Under `/api/v1` and `/ip/`, a path
|
||||||
and `/ip/`, a path no route matches answers 404 with the message `Not found`,
|
no route matches answers 404 with the message `Not found`, and `POST`, `PUT`,
|
||||||
and `POST`, `PUT`, `PATCH`, `DELETE`, `OPTIONS`, `TRACE` or `CONNECT` answers
|
`PATCH`, `DELETE`, `OPTIONS`, `TRACE` or `CONNECT` answers 405 with
|
||||||
405 with `Method not allowed`. A 500 answer says only `Internal server error`,
|
`Method not allowed`. A 500 answer says only `Internal server error`, and a host
|
||||||
and a host name that does not resolve only `could not resolve host name`; the
|
name that does not resolve only `could not resolve host name`; the detail goes
|
||||||
detail goes to the log.
|
to the log.
|
||||||
|
|
||||||
Every answer carries `Strict-Transport-Security`, `Content-Security-Policy`,
|
Every answer carries `Strict-Transport-Security`, `Content-Security-Policy`,
|
||||||
`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` and
|
`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` and
|
||||||
@@ -340,11 +356,10 @@ routewatch/
|
|||||||
with exponential backoff (5s-320s). The backoff resets after a stream that
|
with exponential backoff (5s-320s). The backoff resets after a stream that
|
||||||
lasted more than 30s. A stream with no line for 60s is dropped and
|
lasted more than 30s. A stream with no line for 60s is dropped and
|
||||||
reconnected.
|
reconnected.
|
||||||
- **IP Lookup**: the most specific live route is found by looking up the
|
- **IP Lookup**: the routing table in memory finds the most specific prefix that
|
||||||
address's prefix at each mask length, longest first, in the prefix index (at
|
holds the address. It looks up the address's prefix at each mask length,
|
||||||
most 25 lookups for IPv4, 113 for IPv6); prefixes are stored in the text form
|
longest first: at most 25 lookups for IPv4 and 113 for IPv6. Prefixes shorter
|
||||||
Go's `net/netip` prints. Routes shorter than /8 for IPv4 or /16 for IPv6, such
|
than /8 for IPv4 or /16 for IPv6, such as a default route, are not used.
|
||||||
as a default route, are not used.
|
|
||||||
|
|
||||||
### Database Schema
|
### Database Schema
|
||||||
|
|
||||||
|
|||||||
@@ -33,6 +33,12 @@ tested.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-27: the pages, the API and the stats read every route from the routing
|
||||||
|
table in memory. For a prefix with several origin ASes, the IP lookup gives
|
||||||
|
the one most peers carry. Its `last_seen` is the newest kept route's time. The
|
||||||
|
prefix page and API list every peer that carries the prefix, with an AS path,
|
||||||
|
next hop and time only for the kept routes. The stats count the origins whose
|
||||||
|
kept routes were all withdrawn, in `origins_without_kept_route` (part of #122)
|
||||||
- 2026-10-27: the prefix, AS and peer handlers make a timed write on every tick
|
- 2026-10-27: the prefix, AS and peer handlers make a timed write on every tick
|
||||||
of their timers, every 1, 2 and 2 seconds. They used to skip every other tick.
|
of their timers, every 1, 2 and 2 seconds. They used to skip every other tick.
|
||||||
A full batch's write restarts the timer, so the next timed write comes one
|
A full batch's write restarts the timer, so the next timed write comes one
|
||||||
@@ -271,6 +277,5 @@ tested.
|
|||||||
|
|
||||||
# Future Steps
|
# Future Steps
|
||||||
|
|
||||||
- the rest of https://git.eeqj.de/sneak/routewatch/issues/122, one PR each: the
|
- the rest of https://git.eeqj.de/sneak/routewatch/issues/122: removing the
|
||||||
pages and the API reading from the routing table, then removing the per-peer
|
per-peer route tables
|
||||||
route tables
|
|
||||||
|
|||||||
@@ -657,60 +657,6 @@ func checkOneLiveRoutePerIPVersion(t *testing.T, db *Database) {
|
|||||||
assertDistribution(t, "IPv6 distribution query", queryV6, wantV6)
|
assertDistribution(t, "IPv6 distribution query", queryV6, wantV6)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestStatsRouteTimestamps checks the oldest/newest route timestamps are read
|
|
||||||
// from the right rows across both tables and parse into time.Time. The old
|
|
||||||
// MIN/MAX union query read its result into *time.Time, which the driver could
|
|
||||||
// not parse, so it logged a warning every call and left both timestamps nil.
|
|
||||||
func TestStatsRouteTimestamps(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
cfg := &config.Config{StateDir: t.TempDir()}
|
|
||||||
|
|
||||||
db, err := New(cfg, logger.New())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create database: %v", err)
|
|
||||||
}
|
|
||||||
defer func() { _ = db.Close() }()
|
|
||||||
|
|
||||||
ctx := context.Background()
|
|
||||||
|
|
||||||
empty, err := db.GetStatsContext(ctx)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GetStatsContext on empty database: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if empty.OldestRoute != nil || empty.NewestRoute != nil {
|
|
||||||
t.Fatalf("empty database timestamps = (%v, %v), want (nil, nil)",
|
|
||||||
empty.OldestRoute, empty.NewestRoute)
|
|
||||||
}
|
|
||||||
|
|
||||||
base := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
|
||||||
oldest := base
|
|
||||||
newest := base.Add(2 * time.Minute)
|
|
||||||
|
|
||||||
err = db.UpsertLiveRouteBatch([]*LiveRoute{
|
|
||||||
mkV4Route(t, "198.51.100.0/24", 64500, base.Add(time.Minute)),
|
|
||||||
mkV4Route(t, "203.0.113.0/24", 64501, oldest),
|
|
||||||
mkV6Route(testV6Prefix, 64502, newest),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
stats, err := db.GetStatsContext(ctx)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GetStatsContext: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if stats.OldestRoute == nil || !stats.OldestRoute.Equal(oldest) {
|
|
||||||
t.Errorf("OldestRoute = %v, want %v", stats.OldestRoute, oldest)
|
|
||||||
}
|
|
||||||
|
|
||||||
if stats.NewestRoute == nil || !stats.NewestRoute.Equal(newest) {
|
|
||||||
t.Errorf("NewestRoute = %v, want %v", stats.NewestRoute, newest)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestLiveCountsConcurrentReadWrite runs writers and stats readers at once so
|
// TestLiveCountsConcurrentReadWrite runs writers and stats readers at once so
|
||||||
// the race detector proves the counters are safe under concurrent use.
|
// the race detector proves the counters are safe under concurrent use.
|
||||||
func TestLiveCountsConcurrentReadWrite(t *testing.T) {
|
func TestLiveCountsConcurrentReadWrite(t *testing.T) {
|
||||||
|
|||||||
+113
-108
@@ -1452,10 +1452,9 @@ func (d *Database) GetStats() (Stats, error) {
|
|||||||
//
|
//
|
||||||
// The row counts (ASNs, prefixes, peerings, peers, live routes) and the prefix
|
// The row counts (ASNs, prefixes, peerings, peers, live routes) and the prefix
|
||||||
// distribution come from the in-memory counters, seeded at startup and kept
|
// distribution come from the in-memory counters, seeded at startup and kept
|
||||||
// current on every write. The oldest/newest route timestamps are read from the
|
// current on every write, and the file size from a stat(). The read touches
|
||||||
// ends of the last_updated index, and the file size from a stat(). No part of
|
// no table.
|
||||||
// the read scans a table or a whole index.
|
func (d *Database) GetStatsContext(_ context.Context) (Stats, error) {
|
||||||
func (d *Database) GetStatsContext(ctx context.Context) (Stats, error) {
|
|
||||||
var stats Stats
|
var stats Stats
|
||||||
|
|
||||||
// Row counts and prefix distribution from memory, as a single consistent
|
// Row counts and prefix distribution from memory, as a single consistent
|
||||||
@@ -1470,20 +1469,6 @@ func (d *Database) GetStatsContext(ctx context.Context) (Stats, error) {
|
|||||||
stats.FileSizeBytes = fileInfo.Size()
|
stats.FileSizeBytes = fileInfo.Size()
|
||||||
}
|
}
|
||||||
|
|
||||||
// Oldest and newest route timestamps read one row from each end of the
|
|
||||||
// last_updated index (a log-time lookup, not a scan). Selecting the column
|
|
||||||
// directly lets the driver parse the DATETIME into time.Time; the old
|
|
||||||
// MIN/MAX union scan returned an untyped string that failed to scan and
|
|
||||||
// logged a warning on every call.
|
|
||||||
oldest, newest, err := d.routeTimestampRange(ctx)
|
|
||||||
if err != nil {
|
|
||||||
// Display-only fields; log but keep the rest of the stats.
|
|
||||||
d.logger.Warn("Failed to get route timestamps", "error", err)
|
|
||||||
} else {
|
|
||||||
stats.OldestRoute = oldest
|
|
||||||
stats.NewestRoute = newest
|
|
||||||
}
|
|
||||||
|
|
||||||
return stats, nil
|
return stats, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1778,7 +1763,35 @@ func (d *Database) GetASDetailsContext(
|
|||||||
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
|
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get AS information
|
asnInfo, known, err := d.GetAS(ctx, asn)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
allPrefixes, err := d.originPrefixes(ctx, asn, ipVersionV4)
|
||||||
|
if err != nil {
|
||||||
|
return asnInfo, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
prefixesV6, err := d.originPrefixes(ctx, asn, ipVersionV6)
|
||||||
|
if err != nil {
|
||||||
|
return asnInfo, allPrefixes, err
|
||||||
|
}
|
||||||
|
|
||||||
|
allPrefixes = append(allPrefixes, prefixesV6...)
|
||||||
|
|
||||||
|
if !known && len(allPrefixes) == 0 {
|
||||||
|
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
|
||||||
|
}
|
||||||
|
|
||||||
|
return asnInfo, allPrefixes, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetAS returns the AS's handle, description and first and last seen times
|
||||||
|
// from the asns table. When the table has no record of the AS, its handle and
|
||||||
|
// description come from the AS info lookup, and it has no seen times. The bool
|
||||||
|
// says whether the asns table or the AS info lookup holds the AS.
|
||||||
|
func (d *Database) GetAS(ctx context.Context, asn int) (*ASN, bool, error) {
|
||||||
var (
|
var (
|
||||||
asnInfo ASN
|
asnInfo ASN
|
||||||
handle, description sql.NullString
|
handle, description sql.NullString
|
||||||
@@ -1791,9 +1804,6 @@ func (d *Database) GetASDetailsContext(
|
|||||||
).Scan(&asnInfo.ASN, &handle, &description, &asnInfo.FirstSeen,
|
).Scan(&asnInfo.ASN, &handle, &description, &asnInfo.FirstSeen,
|
||||||
&asnInfo.LastSeen)
|
&asnInfo.LastSeen)
|
||||||
|
|
||||||
// known is whether the asns table or the AS info lookup holds the AS.
|
|
||||||
known := true
|
|
||||||
|
|
||||||
switch {
|
switch {
|
||||||
case errors.Is(err, sql.ErrNoRows):
|
case errors.Is(err, sql.ErrNoRows):
|
||||||
asnInfo.ASN = asn
|
asnInfo.ASN = asn
|
||||||
@@ -1804,31 +1814,96 @@ func (d *Database) GetASDetailsContext(
|
|||||||
asnInfo.Description = info.Description
|
asnInfo.Description = info.Description
|
||||||
}
|
}
|
||||||
|
|
||||||
known = ok
|
return &asnInfo, ok, nil
|
||||||
case err != nil:
|
case err != nil:
|
||||||
return nil, nil, fmt.Errorf("failed to query AS: %w", err)
|
return nil, false, fmt.Errorf("failed to query AS: %w", err)
|
||||||
default:
|
}
|
||||||
asnInfo.Handle = handle.String
|
|
||||||
asnInfo.Description = description.String
|
asnInfo.Handle = handle.String
|
||||||
|
asnInfo.Description = description.String
|
||||||
|
|
||||||
|
return &asnInfo, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// asWHOISQuery reads the handle, description and WHOIS details of one AS, and
|
||||||
|
// when the WHOIS details were fetched.
|
||||||
|
const asWHOISQuery = `
|
||||||
|
SELECT handle, description, as_name, org_name, org_id, address,
|
||||||
|
country_code, abuse_email, rir, whois_updated_at
|
||||||
|
FROM asns WHERE asn = ?`
|
||||||
|
|
||||||
|
// GetASWHOIS returns the handle, description and WHOIS details that the asns
|
||||||
|
// table holds for the AS, and when the WHOIS details were fetched. When the
|
||||||
|
// table has no record of the AS, it returns an ASN with only its number.
|
||||||
|
func (d *Database) GetASWHOIS(ctx context.Context, asn int) (*ASN, error) {
|
||||||
|
var (
|
||||||
|
handle, description, asName, orgName, orgID sql.NullString
|
||||||
|
address, countryCode, abuseEmail, rir sql.NullString
|
||||||
|
whoisUpdatedAt sql.NullTime
|
||||||
|
)
|
||||||
|
|
||||||
|
err := d.db.QueryRowContext(ctx, asWHOISQuery, asn).Scan(
|
||||||
|
&handle, &description, &asName, &orgName, &orgID, &address,
|
||||||
|
&countryCode, &abuseEmail, &rir, &whoisUpdatedAt,
|
||||||
|
)
|
||||||
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
|
return &ASN{ASN: asn}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
allPrefixes, err := d.originPrefixes(ctx, asn, ipVersionV4)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return &asnInfo, nil, err
|
return nil, fmt.Errorf("failed to query AS: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
info := &ASN{
|
||||||
|
ASN: asn,
|
||||||
|
Handle: handle.String,
|
||||||
|
Description: description.String,
|
||||||
|
ASName: asName.String,
|
||||||
|
OrgName: orgName.String,
|
||||||
|
OrgID: orgID.String,
|
||||||
|
Address: address.String,
|
||||||
|
CountryCode: countryCode.String,
|
||||||
|
AbuseEmail: abuseEmail.String,
|
||||||
|
RIR: rir.String,
|
||||||
|
}
|
||||||
|
|
||||||
|
if whoisUpdatedAt.Valid {
|
||||||
|
info.WHOISUpdatedAt = &whoisUpdatedAt.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
return info, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// The queries GetPrefixFirstSeen reads an IPv4 and an IPv6 prefix's first-seen
|
||||||
|
// time with.
|
||||||
|
const (
|
||||||
|
prefixFirstSeenV4SQL = `SELECT first_seen FROM prefixes_v4 WHERE prefix = ?`
|
||||||
|
prefixFirstSeenV6SQL = `SELECT first_seen FROM prefixes_v6 WHERE prefix = ?`
|
||||||
|
)
|
||||||
|
|
||||||
|
// GetPrefixFirstSeen returns when the prefix tables first saw prefix, or the
|
||||||
|
// zero time when they have no record of it.
|
||||||
|
func (d *Database) GetPrefixFirstSeen(
|
||||||
|
ctx context.Context, prefix netip.Prefix,
|
||||||
|
) (time.Time, error) {
|
||||||
|
query := prefixFirstSeenV6SQL
|
||||||
|
if prefix.Addr().Is4() {
|
||||||
|
query = prefixFirstSeenV4SQL
|
||||||
|
}
|
||||||
|
|
||||||
|
var firstSeen time.Time
|
||||||
|
|
||||||
|
err := d.db.QueryRowContext(ctx, query, prefix.Masked().String()).
|
||||||
|
Scan(&firstSeen)
|
||||||
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
|
return time.Time{}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
prefixesV6, err := d.originPrefixes(ctx, asn, ipVersionV6)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return &asnInfo, allPrefixes, err
|
return time.Time{}, fmt.Errorf("failed to query prefix: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
allPrefixes = append(allPrefixes, prefixesV6...)
|
return firstSeen, nil
|
||||||
|
|
||||||
if !known && len(allPrefixes) == 0 {
|
|
||||||
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
|
|
||||||
}
|
|
||||||
|
|
||||||
return &asnInfo, allPrefixes, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ASPeer represents a peering relationship with another AS including handle,
|
// ASPeer represents a peering relationship with another AS including handle,
|
||||||
@@ -2564,76 +2639,6 @@ func (d *Database) deletePeerRouteChunk(peerIP string) (int, int, error) {
|
|||||||
return int(deletedV4), int(deletedV6), nil
|
return int(deletedV4), int(deletedV6), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// routeTimestampRange returns the earliest and latest last_updated across both
|
|
||||||
// live route tables, or nil values when both tables are empty. Each query reads
|
|
||||||
// one row from an end of the last_updated index rather than scanning the tables.
|
|
||||||
func (d *Database) routeTimestampRange(
|
|
||||||
ctx context.Context,
|
|
||||||
) (*time.Time, *time.Time, error) {
|
|
||||||
var oldest, newest *time.Time
|
|
||||||
|
|
||||||
oldestV4, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
|
|
||||||
"FROM live_routes_v4 ORDER BY last_updated ASC LIMIT 1")
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if ok {
|
|
||||||
oldest = &oldestV4
|
|
||||||
}
|
|
||||||
|
|
||||||
oldestV6, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
|
|
||||||
"FROM live_routes_v6 ORDER BY last_updated ASC LIMIT 1")
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if ok && (oldest == nil || oldestV6.Before(*oldest)) {
|
|
||||||
oldest = &oldestV6
|
|
||||||
}
|
|
||||||
|
|
||||||
newestV4, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
|
|
||||||
"FROM live_routes_v4 ORDER BY last_updated DESC LIMIT 1")
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if ok {
|
|
||||||
newest = &newestV4
|
|
||||||
}
|
|
||||||
|
|
||||||
newestV6, ok, err := d.scanRouteTimestamp(ctx, "SELECT last_updated "+
|
|
||||||
"FROM live_routes_v6 ORDER BY last_updated DESC LIMIT 1")
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if ok && (newest == nil || newestV6.After(*newest)) {
|
|
||||||
newest = &newestV6
|
|
||||||
}
|
|
||||||
|
|
||||||
return oldest, newest, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// scanRouteTimestamp runs a single-row timestamp query. The bool is false when
|
|
||||||
// the table is empty. The query selects the last_updated column directly so the
|
|
||||||
// driver parses the DATETIME value into a time.Time.
|
|
||||||
func (d *Database) scanRouteTimestamp(
|
|
||||||
ctx context.Context, query string,
|
|
||||||
) (time.Time, bool, error) {
|
|
||||||
var ts time.Time
|
|
||||||
|
|
||||||
err := d.db.QueryRowContext(ctx, query).Scan(&ts)
|
|
||||||
switch {
|
|
||||||
case errors.Is(err, sql.ErrNoRows):
|
|
||||||
return time.Time{}, false, nil
|
|
||||||
case err != nil:
|
|
||||||
return time.Time{}, false, err
|
|
||||||
default:
|
|
||||||
return ts, true, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// The AS page lists the prefixes an AS originates one at a time, each in two
|
// The AS page lists the prefixes an AS originates one at a time, each in two
|
||||||
// index searches that read no table row: one of the origin_asn and prefix
|
// index searches that read no table row: one of the origin_asn and prefix
|
||||||
// index for the AS's first prefix after the previous one, and one of the
|
// index for the AS's first prefix after the previous one, and one of the
|
||||||
|
|||||||
@@ -1780,6 +1780,165 @@ func TestGetASDetailsOfASNotInDatabase(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestGetASReadsTheRecordOrTheBuiltInList asks GetAS for an AS the asns table
|
||||||
|
// holds, one only the AS info lookup holds, and one neither holds. The first
|
||||||
|
// gets its record's seen times, the second its handle and description from the
|
||||||
|
// lookup, and only the third is not known.
|
||||||
|
func TestGetASReadsTheRecordOrTheBuiltInList(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create database: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = db.Close() }()
|
||||||
|
|
||||||
|
const (
|
||||||
|
storedASN = 64500 // a documentation AS, not in the lookup
|
||||||
|
listedASN = 15169 // in the lookup, not stored
|
||||||
|
unknownASN = 4200000001 // in neither
|
||||||
|
)
|
||||||
|
|
||||||
|
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
||||||
|
|
||||||
|
_, err = db.GetOrCreateASN(storedASN, ts)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetOrCreateASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
listed, ok := asinfo.Get(listedASN)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("AS%d is not in the AS info data", listedASN)
|
||||||
|
}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
asn int
|
||||||
|
want ASN
|
||||||
|
wantKnown bool
|
||||||
|
}{
|
||||||
|
{storedASN, ASN{ASN: storedASN, FirstSeen: ts, LastSeen: ts}, true},
|
||||||
|
{listedASN, ASN{
|
||||||
|
ASN: listedASN, Handle: listed.Handle, Description: listed.Description,
|
||||||
|
}, true},
|
||||||
|
{unknownASN, ASN{ASN: unknownASN}, false},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
got, known, err := db.GetAS(t.Context(), tt.asn)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("GetAS(%d): %v", tt.asn, err)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if !got.FirstSeen.Equal(tt.want.FirstSeen) ||
|
||||||
|
!got.LastSeen.Equal(tt.want.LastSeen) {
|
||||||
|
t.Errorf("GetAS(%d) seen %v to %v, want %v to %v", tt.asn,
|
||||||
|
got.FirstSeen, got.LastSeen, tt.want.FirstSeen, tt.want.LastSeen)
|
||||||
|
}
|
||||||
|
|
||||||
|
got.FirstSeen, got.LastSeen = tt.want.FirstSeen, tt.want.LastSeen
|
||||||
|
if *got != tt.want || known != tt.wantKnown {
|
||||||
|
t.Errorf("GetAS(%d) = %+v, known %t; want %+v, known %t",
|
||||||
|
tt.asn, *got, known, tt.want, tt.wantKnown)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGetASWHOISReadsTheStoredDetails stores an AS with WHOIS details and asks
|
||||||
|
// GetASWHOIS for it and for an AS the asns table does not hold. The first gets
|
||||||
|
// its stored handle, description and details, and the time they were fetched.
|
||||||
|
// The second gets only its number, although the AS info lookup holds it.
|
||||||
|
func TestGetASWHOISReadsTheStoredDetails(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create database: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = db.Close() }()
|
||||||
|
|
||||||
|
const storedASN, unstoredASN = 15169, 13335
|
||||||
|
|
||||||
|
_, err = db.GetOrCreateASN(storedASN, time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetOrCreateASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
update := &ASNWHOISUpdate{
|
||||||
|
ASN: storedASN, ASName: "TEST-AS", OrgName: "Test Org", OrgID: "ORG-1",
|
||||||
|
Address: "1 Test Street", CountryCode: "DE", AbuseEmail: "abuse@test",
|
||||||
|
RIR: "RIPE",
|
||||||
|
}
|
||||||
|
|
||||||
|
err = db.UpdateASNWHOIS(t.Context(), update)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpdateASNWHOIS: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err := db.GetASWHOIS(t.Context(), storedASN)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetASWHOIS(%d): %v", storedASN, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
want := ASN{
|
||||||
|
ASN: storedASN, Handle: asinfo.GetHandle(storedASN),
|
||||||
|
Description: asinfo.GetDescription(storedASN), ASName: update.ASName,
|
||||||
|
OrgName: update.OrgName, OrgID: update.OrgID, Address: update.Address,
|
||||||
|
CountryCode: update.CountryCode, AbuseEmail: update.AbuseEmail,
|
||||||
|
RIR: update.RIR,
|
||||||
|
}
|
||||||
|
|
||||||
|
if got.WHOISUpdatedAt == nil {
|
||||||
|
t.Errorf("GetASWHOIS(%d) has no time the details were fetched", storedASN)
|
||||||
|
}
|
||||||
|
|
||||||
|
got.WHOISUpdatedAt = nil
|
||||||
|
if *got != want {
|
||||||
|
t.Errorf("GetASWHOIS(%d) = %+v, want %+v", storedASN, *got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err = db.GetASWHOIS(t.Context(), unstoredASN)
|
||||||
|
if err != nil || *got != (ASN{ASN: unstoredASN}) {
|
||||||
|
t.Errorf("GetASWHOIS(%d) = %+v, %v; want only the number",
|
||||||
|
unstoredASN, got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGetPrefixFirstSeenReadsThePrefixTables stores an IPv4 and an IPv6
|
||||||
|
// prefix and expects GetPrefixFirstSeen to give each its first-seen time, and
|
||||||
|
// the zero time for a prefix that is not stored.
|
||||||
|
func TestGetPrefixFirstSeenReadsThePrefixTables(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create database: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = db.Close() }()
|
||||||
|
|
||||||
|
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
||||||
|
stored := map[string]time.Time{
|
||||||
|
"192.0.2.0/24": ts,
|
||||||
|
testV6Prefix: ts.Add(time.Hour),
|
||||||
|
}
|
||||||
|
|
||||||
|
err = db.UpdatePrefixesBatch(stored)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpdatePrefixesBatch: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
stored["198.51.100.0/24"] = time.Time{}
|
||||||
|
|
||||||
|
for prefix, want := range stored {
|
||||||
|
got, err := db.GetPrefixFirstSeen(t.Context(), netip.MustParsePrefix(prefix))
|
||||||
|
if err != nil || !got.Equal(want) {
|
||||||
|
t.Errorf("GetPrefixFirstSeen(%s) = %v, %v; want %v",
|
||||||
|
prefix, got, err, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestASPrefixQueriesUseCoveringIndexes checks the plan of each query that
|
// TestASPrefixQueriesUseCoveringIndexes checks the plan of each query that
|
||||||
// lists the prefixes an AS originates: a search of a covering index and no
|
// lists the prefixes an AS originates: a search of a covering index and no
|
||||||
// temporary B-tree, so the list reads one index entry per prefix, not one
|
// temporary B-tree, so the list reads one index entry per prefix, not one
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ package database
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"net/netip"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
@@ -24,8 +25,6 @@ type Stats struct {
|
|||||||
LiveRoutes int
|
LiveRoutes int
|
||||||
IPv4Routes int
|
IPv4Routes int
|
||||||
IPv6Routes int
|
IPv6Routes int
|
||||||
OldestRoute *time.Time
|
|
||||||
NewestRoute *time.Time
|
|
||||||
IPv4PrefixDistribution []PrefixDistribution
|
IPv4PrefixDistribution []PrefixDistribution
|
||||||
IPv6PrefixDistribution []PrefixDistribution
|
IPv6PrefixDistribution []PrefixDistribution
|
||||||
}
|
}
|
||||||
@@ -85,6 +84,9 @@ type Store interface {
|
|||||||
GetWHOISStats(ctx context.Context, staleThreshold time.Duration) (*WHOISStats, error)
|
GetWHOISStats(ctx context.Context, staleThreshold time.Duration) (*WHOISStats, error)
|
||||||
|
|
||||||
// AS and prefix detail operations
|
// AS and prefix detail operations
|
||||||
|
GetAS(ctx context.Context, asn int) (*ASN, bool, error)
|
||||||
|
GetASWHOIS(ctx context.Context, asn int) (*ASN, error)
|
||||||
|
GetPrefixFirstSeen(ctx context.Context, prefix netip.Prefix) (time.Time, error)
|
||||||
GetASDetails(asn int) (*ASN, []LiveRoute, error)
|
GetASDetails(asn int) (*ASN, []LiveRoute, error)
|
||||||
GetASDetailsContext(ctx context.Context, asn int) (*ASN, []LiveRoute, error)
|
GetASDetailsContext(ctx context.Context, asn int) (*ASN, []LiveRoute, error)
|
||||||
GetASPeers(asn int) ([]ASPeer, error)
|
GetASPeers(asn int) ([]ASPeer, error)
|
||||||
|
|||||||
@@ -66,7 +66,9 @@ type ASNPeering struct {
|
|||||||
LastSeen time.Time
|
LastSeen time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
// LiveRoute represents a route in the live routing table
|
// LiveRoute represents a route in the live routing table. JSON leaves out an
|
||||||
|
// empty AS path, next hop or time: the routing table in memory keeps them only
|
||||||
|
// for the two newest routes of each prefix and origin AS.
|
||||||
//
|
//
|
||||||
//nolint:tagliatelle // field names are the existing JSON API
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
type LiveRoute struct {
|
type LiveRoute struct {
|
||||||
@@ -75,9 +77,9 @@ type LiveRoute struct {
|
|||||||
IPVersion int `json:"ip_version"`
|
IPVersion int `json:"ip_version"`
|
||||||
OriginASN int `json:"origin_asn"`
|
OriginASN int `json:"origin_asn"`
|
||||||
PeerIP string `json:"peer_ip"`
|
PeerIP string `json:"peer_ip"`
|
||||||
ASPath []int `json:"as_path"`
|
ASPath []int `json:"as_path,omitempty"`
|
||||||
NextHop string `json:"next_hop"`
|
NextHop string `json:"next_hop,omitempty"`
|
||||||
LastUpdated time.Time `json:"last_updated"`
|
LastUpdated time.Time `json:"last_updated,omitzero"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// PrefixDistribution represents the distribution of prefixes by mask length
|
// PrefixDistribution represents the distribution of prefixes by mask length
|
||||||
@@ -108,9 +110,10 @@ type IPInfo struct {
|
|||||||
CountryCode string `json:"country_code,omitempty"`
|
CountryCode string `json:"country_code,omitempty"`
|
||||||
AbuseEmail string `json:"abuse_email,omitempty"`
|
AbuseEmail string `json:"abuse_email,omitempty"`
|
||||||
RIR string `json:"rir,omitempty"`
|
RIR string `json:"rir,omitempty"`
|
||||||
// Timestamps
|
// Timestamps. LastSeen is zero, and left out of JSON, when the prefix
|
||||||
|
// keeps no route.
|
||||||
FirstSeen time.Time `json:"first_seen"`
|
FirstSeen time.Time `json:"first_seen"`
|
||||||
LastSeen time.Time `json:"last_seen"`
|
LastSeen time.Time `json:"last_seen,omitzero"`
|
||||||
// Indicates if WHOIS data needs refresh (not serialized)
|
// Indicates if WHOIS data needs refresh (not serialized)
|
||||||
NeedsWHOISRefresh bool `json:"-"`
|
NeedsWHOISRefresh bool `json:"-"`
|
||||||
}
|
}
|
||||||
|
|||||||
+14
-15
@@ -31,11 +31,13 @@ import (
|
|||||||
// are disabled, since only the batched handlers exist.
|
// are disabled, since only the batched handlers exist.
|
||||||
var errNonBatchedNotImplemented = errors.New("non-batched handlers not implemented")
|
var errNonBatchedNotImplemented = errors.New("non-batched handlers not implemented")
|
||||||
|
|
||||||
// Dependencies contains all dependencies for RouteWatch
|
// Dependencies contains all dependencies for RouteWatch. Table is the routing
|
||||||
|
// table read from the database at startup, which the server reads too.
|
||||||
type Dependencies struct {
|
type Dependencies struct {
|
||||||
fx.In
|
fx.In
|
||||||
|
|
||||||
DB database.Store
|
DB database.Store
|
||||||
|
Table *routetable.Table
|
||||||
Streamer *streamer.Streamer
|
Streamer *streamer.Streamer
|
||||||
Server *server.Server
|
Server *server.Server
|
||||||
Logger *logger.Logger
|
Logger *logger.Logger
|
||||||
@@ -45,6 +47,7 @@ type Dependencies struct {
|
|||||||
// RouteWatch represents the main application instance
|
// RouteWatch represents the main application instance
|
||||||
type RouteWatch struct {
|
type RouteWatch struct {
|
||||||
db database.Store
|
db database.Store
|
||||||
|
table *routetable.Table
|
||||||
streamer *streamer.Streamer
|
streamer *streamer.Streamer
|
||||||
server *server.Server
|
server *server.Server
|
||||||
logger *logger.Logger
|
logger *logger.Logger
|
||||||
@@ -69,6 +72,7 @@ const blocklistsDirName = "blocklists"
|
|||||||
func New(deps Dependencies) *RouteWatch {
|
func New(deps Dependencies) *RouteWatch {
|
||||||
rw := &RouteWatch{
|
rw := &RouteWatch{
|
||||||
db: deps.DB,
|
db: deps.DB,
|
||||||
|
table: deps.Table,
|
||||||
streamer: deps.Streamer,
|
streamer: deps.Streamer,
|
||||||
server: deps.Server,
|
server: deps.Server,
|
||||||
logger: deps.Logger,
|
logger: deps.Logger,
|
||||||
@@ -97,13 +101,6 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
|
|||||||
if rw.config.EnableBatchedDatabaseWrites {
|
if rw.config.EnableBatchedDatabaseWrites {
|
||||||
rw.logger.Info("Using batched database handlers for improved performance")
|
rw.logger.Info("Using batched database handlers for improved performance")
|
||||||
|
|
||||||
// The routing table is read before the streamer connects, so that
|
|
||||||
// the prefix handler applies the feed's updates to all of it.
|
|
||||||
table, err := rw.readRouteTable(ctx)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// ASHandler maintains the asns table
|
// ASHandler maintains the asns table
|
||||||
rw.dbHandler = NewASHandler(rw.db, rw.logger)
|
rw.dbHandler = NewASHandler(rw.db, rw.logger)
|
||||||
rw.streamer.RegisterHandler(rw.dbHandler)
|
rw.streamer.RegisterHandler(rw.dbHandler)
|
||||||
@@ -114,7 +111,7 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
|
|||||||
|
|
||||||
// PrefixHandler maintains the prefixes and live_routes tables, and
|
// PrefixHandler maintains the prefixes and live_routes tables, and
|
||||||
// the routing table in memory and its copy in the routes table
|
// the routing table in memory and its copy in the routes table
|
||||||
rw.prefixHandler = NewPrefixHandler(rw.db, rw.logger, table)
|
rw.prefixHandler = NewPrefixHandler(rw.db, rw.logger, rw.table)
|
||||||
rw.prefixHandler.SetMetricsTracker(rw.streamer.GetMetricsTracker())
|
rw.prefixHandler.SetMetricsTracker(rw.streamer.GetMetricsTracker())
|
||||||
rw.streamer.RegisterHandler(rw.prefixHandler)
|
rw.streamer.RegisterHandler(rw.prefixHandler)
|
||||||
|
|
||||||
@@ -269,20 +266,21 @@ func (rw *RouteWatch) closeDatabase(ctx context.Context) {
|
|||||||
|
|
||||||
// readRouteTable reads the routing table from the database and logs how many
|
// readRouteTable reads the routing table from the database and logs how many
|
||||||
// prefix and origin AS pairs and peers it holds and how long the read took.
|
// prefix and origin AS pairs and peers it holds and how long the read took.
|
||||||
// ctx is fx's start context, which ends 15 seconds after the start. Reading a
|
// fx calls it once, to build the server and RouteWatch, before the streamer
|
||||||
// full table can take longer, so the read does not stop with ctx.
|
// connects, so the prefix handler applies the feed's updates to all of it. No
|
||||||
func (rw *RouteWatch) readRouteTable(
|
// start timeout limits the read: fx builds them before it runs a start hook.
|
||||||
ctx context.Context,
|
func readRouteTable(
|
||||||
|
db database.Store, logger *logger.Logger,
|
||||||
) (*routetable.Table, error) {
|
) (*routetable.Table, error) {
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
|
|
||||||
table, err := rw.db.ReadRouteTable(context.WithoutCancel(ctx))
|
table, err := db.ReadRouteTable(context.Background())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("failed to read the routing table: %w", err)
|
return nil, fmt.Errorf("failed to read the routing table: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
counts := table.Counts()
|
counts := table.Counts()
|
||||||
rw.logger.Info("Read the routing table", "pairs", counts.Pairs,
|
logger.Info("Read the routing table", "pairs", counts.Pairs,
|
||||||
"peers", counts.Peers, "duration_ms", time.Since(start).Milliseconds())
|
"peers", counts.Peers, "duration_ms", time.Since(start).Milliseconds())
|
||||||
|
|
||||||
return table, nil
|
return table, nil
|
||||||
@@ -299,6 +297,7 @@ func getModule() []fx.Option {
|
|||||||
database.New,
|
database.New,
|
||||||
fx.As(new(database.Store)),
|
fx.As(new(database.Store)),
|
||||||
),
|
),
|
||||||
|
readRouteTable,
|
||||||
streamer.New,
|
streamer.New,
|
||||||
server.New,
|
server.New,
|
||||||
New,
|
New,
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
|
"net/netip"
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -265,6 +266,32 @@ func (m *mockStore) GetASDetailsContext(
|
|||||||
return m.GetASDetails(asn)
|
return m.GetASDetails(asn)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GetAS returns the stored AS, or only its number when none is stored.
|
||||||
|
func (m *mockStore) GetAS(_ context.Context, asn int) (*database.ASN, bool, error) {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
|
||||||
|
if asnInfo, exists := m.ASNs[asn]; exists {
|
||||||
|
return asnInfo, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &database.ASN{ASN: asn}, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetASWHOIS returns the stored AS, or only its number when none is stored.
|
||||||
|
func (m *mockStore) GetASWHOIS(ctx context.Context, asn int) (*database.ASN, error) {
|
||||||
|
asnInfo, _, err := m.GetAS(ctx, asn)
|
||||||
|
|
||||||
|
return asnInfo, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetPrefixFirstSeen returns the zero time: the mock keeps no prefix times.
|
||||||
|
func (m *mockStore) GetPrefixFirstSeen(
|
||||||
|
_ context.Context, _ netip.Prefix,
|
||||||
|
) (time.Time, error) {
|
||||||
|
return time.Time{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
// GetPrefixDetails mock implementation
|
// GetPrefixDetails mock implementation
|
||||||
func (m *mockStore) GetPrefixDetails(
|
func (m *mockStore) GetPrefixDetails(
|
||||||
_ string,
|
_ string,
|
||||||
@@ -502,12 +529,15 @@ func TestRouteWatchLiveFeed(t *testing.T) {
|
|||||||
EnableBatchedDatabaseWrites: true,
|
EnableBatchedDatabaseWrites: true,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
table := routetable.New(routetable.Params{})
|
||||||
|
|
||||||
// Create server
|
// Create server
|
||||||
srv := server.New(mockDB, s, logger, cfg)
|
srv := server.New(mockDB, table, s, logger, cfg)
|
||||||
|
|
||||||
// Create RouteWatch with 5 second limit
|
// Create RouteWatch with 5 second limit
|
||||||
deps := Dependencies{
|
deps := Dependencies{
|
||||||
DB: mockDB,
|
DB: mockDB,
|
||||||
|
Table: table,
|
||||||
Streamer: s,
|
Streamer: s,
|
||||||
Server: srv,
|
Server: srv,
|
||||||
Logger: logger,
|
Logger: logger,
|
||||||
@@ -626,7 +656,7 @@ func TestShutdownWritesTheUpdatesQueuedInTheStreamer(t *testing.T) {
|
|||||||
rw := New(Dependencies{
|
rw := New(Dependencies{
|
||||||
DB: store,
|
DB: store,
|
||||||
Streamer: s,
|
Streamer: s,
|
||||||
Server: server.New(store, s, log, cfg),
|
Server: server.New(store, nil, s, log, cfg),
|
||||||
Logger: log,
|
Logger: log,
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
})
|
})
|
||||||
@@ -728,7 +758,7 @@ func TestShutdownReturnsByItsDeadlineWhileAWriteIsBlocked(t *testing.T) {
|
|||||||
rw := New(Dependencies{
|
rw := New(Dependencies{
|
||||||
DB: store,
|
DB: store,
|
||||||
Streamer: s,
|
Streamer: s,
|
||||||
Server: server.New(store, s, log, cfg),
|
Server: server.New(store, nil, s, log, cfg),
|
||||||
Logger: log,
|
Logger: log,
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
})
|
})
|
||||||
@@ -842,7 +872,7 @@ func TestShutdownCheckpointsAndClosesTheDatabaseLast(t *testing.T) {
|
|||||||
rw := New(Dependencies{
|
rw := New(Dependencies{
|
||||||
DB: store,
|
DB: store,
|
||||||
Streamer: s,
|
Streamer: s,
|
||||||
Server: server.New(store, s, log, cfg),
|
Server: server.New(store, nil, s, log, cfg),
|
||||||
Logger: log,
|
Logger: log,
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -416,12 +416,8 @@ func TestARestartReadsTheRoutingTableBack(t *testing.T) {
|
|||||||
|
|
||||||
var logs bytes.Buffer
|
var logs bytes.Buffer
|
||||||
|
|
||||||
rw := &RouteWatch{
|
table, err := readRouteTable(db,
|
||||||
db: db,
|
&logger.Logger{Logger: slog.New(slog.NewJSONHandler(&logs, nil))})
|
||||||
logger: &logger.Logger{Logger: slog.New(slog.NewJSONHandler(&logs, nil))},
|
|
||||||
}
|
|
||||||
|
|
||||||
table, err := rw.readRouteTable(t.Context())
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("readRouteTable: %v", err)
|
t.Fatalf("readRouteTable: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
+322
-170
@@ -5,12 +5,12 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
|
"fmt"
|
||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"runtime"
|
"runtime"
|
||||||
"slices"
|
"slices"
|
||||||
"sort"
|
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
@@ -19,6 +19,7 @@ import (
|
|||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/templates"
|
"git.eeqj.de/sneak/routewatch/internal/templates"
|
||||||
asinfo "git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
asinfo "git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
||||||
@@ -38,7 +39,7 @@ const (
|
|||||||
const (
|
const (
|
||||||
// maxLookupTargets is the most addresses and host names an IP lookup may
|
// maxLookupTargets is the most addresses and host names an IP lookup may
|
||||||
// list, and the most addresses it may have once the names are resolved:
|
// list, and the most addresses it may have once the names are resolved:
|
||||||
// each address costs up to 113 database reads and a PTR lookup.
|
// each address costs two database reads and a PTR lookup.
|
||||||
maxLookupTargets = 10
|
maxLookupTargets = 10
|
||||||
|
|
||||||
// ptrLookupTimeout is each PTR lookup's own time limit, so that ten
|
// ptrLookupTimeout is each PTR lookup's own time limit, so that ten
|
||||||
@@ -227,7 +228,8 @@ type WHOISStatsInfo struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Stats is the /status.json answer: uptime, message counts, database stats,
|
// Stats is the /status.json answer: uptime, message counts, database stats,
|
||||||
// and route information.
|
// and the routing table's figures. OldestRoute and NewestRoute are the times
|
||||||
|
// of the oldest and the newest kept routes.
|
||||||
//
|
//
|
||||||
//nolint:tagliatelle // field names are the existing JSON API
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
type Stats struct {
|
type Stats struct {
|
||||||
@@ -259,6 +261,10 @@ type Stats struct {
|
|||||||
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
|
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
|
||||||
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
|
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
|
||||||
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
|
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
|
||||||
|
|
||||||
|
// OriginsWithoutKeptRoute counts the prefix and origin AS pairs that peers
|
||||||
|
// carry but whose kept routes were all withdrawn.
|
||||||
|
OriginsWithoutKeptRoute int `json:"origins_without_kept_route"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// databaseStats gets the database stats, giving up when ctx is done; it then
|
// databaseStats gets the database stats, giving up when ctx is done; it then
|
||||||
@@ -317,15 +323,6 @@ func (s *Server) handleStatusJSON() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
const bitsPerMegabit = 1000000.0
|
|
||||||
|
|
||||||
// Get route update metrics
|
|
||||||
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
|
|
||||||
|
|
||||||
// Get memory stats
|
|
||||||
var memStats runtime.MemStats
|
|
||||||
runtime.ReadMemStats(&memStats)
|
|
||||||
|
|
||||||
// Get WHOIS stats if fetcher is available
|
// Get WHOIS stats if fetcher is available
|
||||||
var whoisStats *WHOISStatsInfo
|
var whoisStats *WHOISStatsInfo
|
||||||
if s.asnFetcher != nil {
|
if s.asnFetcher != nil {
|
||||||
@@ -337,44 +334,80 @@ func (s *Server) handleStatusJSON() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
stats := Stats{
|
err = writeJSONSuccess(w, s.statusStats(metrics, dbStats, whoisStats))
|
||||||
Uptime: streamUptime(metrics.ConnectedSince),
|
|
||||||
TotalMessages: metrics.TotalMessages,
|
|
||||||
TotalBytes: metrics.TotalBytes,
|
|
||||||
TotalWireBytes: metrics.TotalWireBytes,
|
|
||||||
MessagesPerSec: metrics.MessagesPerSec,
|
|
||||||
MbitsPerSec: metrics.BitsPerSec / bitsPerMegabit,
|
|
||||||
WireMbitsPerSec: metrics.WireBitsPerSec / bitsPerMegabit,
|
|
||||||
Connected: metrics.Connected,
|
|
||||||
GoVersion: runtime.Version(),
|
|
||||||
Goroutines: runtime.NumGoroutine(),
|
|
||||||
MemoryUsage: humanize.Bytes(memStats.Alloc),
|
|
||||||
ASNs: dbStats.ASNs,
|
|
||||||
Prefixes: dbStats.Prefixes,
|
|
||||||
IPv4Prefixes: dbStats.IPv4Prefixes,
|
|
||||||
IPv6Prefixes: dbStats.IPv6Prefixes,
|
|
||||||
Peerings: dbStats.Peerings,
|
|
||||||
Peers: dbStats.Peers,
|
|
||||||
DatabaseSizeBytes: dbStats.FileSizeBytes,
|
|
||||||
LiveRoutes: dbStats.LiveRoutes,
|
|
||||||
IPv4Routes: dbStats.IPv4Routes,
|
|
||||||
IPv6Routes: dbStats.IPv6Routes,
|
|
||||||
OldestRoute: dbStats.OldestRoute,
|
|
||||||
NewestRoute: dbStats.NewestRoute,
|
|
||||||
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
|
|
||||||
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
|
|
||||||
IPv4PrefixDistribution: dbStats.IPv4PrefixDistribution,
|
|
||||||
IPv6PrefixDistribution: dbStats.IPv6PrefixDistribution,
|
|
||||||
WHOISStats: whoisStats,
|
|
||||||
}
|
|
||||||
|
|
||||||
err = writeJSONSuccess(w, stats)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.logger.Error("Failed to encode stats", "error", err)
|
s.logger.Error("Failed to encode stats", "error", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// statusStats builds the /status.json answer from the stream metrics, the
|
||||||
|
// database stats, the routing table's figures and the WHOIS stats, with the
|
||||||
|
// route update rates and memory use as they are now.
|
||||||
|
func (s *Server) statusStats(
|
||||||
|
streamMetrics metrics.StreamMetrics,
|
||||||
|
dbStats database.Stats,
|
||||||
|
whoisStats *WHOISStatsInfo,
|
||||||
|
) Stats {
|
||||||
|
const bitsPerMegabit = 1000000.0
|
||||||
|
|
||||||
|
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
|
||||||
|
counts := s.table.Counts()
|
||||||
|
|
||||||
|
var memStats runtime.MemStats
|
||||||
|
runtime.ReadMemStats(&memStats)
|
||||||
|
|
||||||
|
return Stats{
|
||||||
|
Uptime: streamUptime(streamMetrics.ConnectedSince),
|
||||||
|
TotalMessages: streamMetrics.TotalMessages,
|
||||||
|
TotalBytes: streamMetrics.TotalBytes,
|
||||||
|
TotalWireBytes: streamMetrics.TotalWireBytes,
|
||||||
|
MessagesPerSec: streamMetrics.MessagesPerSec,
|
||||||
|
MbitsPerSec: streamMetrics.BitsPerSec / bitsPerMegabit,
|
||||||
|
WireMbitsPerSec: streamMetrics.WireBitsPerSec / bitsPerMegabit,
|
||||||
|
Connected: streamMetrics.Connected,
|
||||||
|
GoVersion: runtime.Version(),
|
||||||
|
Goroutines: runtime.NumGoroutine(),
|
||||||
|
MemoryUsage: humanize.Bytes(memStats.Alloc),
|
||||||
|
ASNs: dbStats.ASNs,
|
||||||
|
Prefixes: counts.IPv4Prefixes + counts.IPv6Prefixes,
|
||||||
|
IPv4Prefixes: counts.IPv4Prefixes,
|
||||||
|
IPv6Prefixes: counts.IPv6Prefixes,
|
||||||
|
Peerings: dbStats.Peerings,
|
||||||
|
Peers: dbStats.Peers,
|
||||||
|
DatabaseSizeBytes: dbStats.FileSizeBytes,
|
||||||
|
LiveRoutes: counts.IPv4Routes + counts.IPv6Routes,
|
||||||
|
IPv4Routes: counts.IPv4Routes,
|
||||||
|
IPv6Routes: counts.IPv6Routes,
|
||||||
|
OldestRoute: counts.OldestKeptRoute,
|
||||||
|
NewestRoute: counts.NewestKeptRoute,
|
||||||
|
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
|
||||||
|
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
|
||||||
|
IPv4PrefixDistribution: prefixDistribution(counts.IPv4PrefixesByLength),
|
||||||
|
IPv6PrefixDistribution: prefixDistribution(counts.IPv6PrefixesByLength),
|
||||||
|
WHOISStats: whoisStats,
|
||||||
|
|
||||||
|
OriginsWithoutKeptRoute: counts.OriginsWithoutKeptRoute,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// prefixDistribution lists the mask lengths that have at least one prefix, in
|
||||||
|
// ascending order, with their prefix counts. countsByLength[n] is the number
|
||||||
|
// of prefixes of mask length n.
|
||||||
|
func prefixDistribution(countsByLength []int) []database.PrefixDistribution {
|
||||||
|
var distribution []database.PrefixDistribution
|
||||||
|
|
||||||
|
for maskLength, count := range countsByLength {
|
||||||
|
if count > 0 {
|
||||||
|
distribution = append(distribution, database.PrefixDistribution{
|
||||||
|
MaskLength: maskLength, Count: count,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return distribution
|
||||||
|
}
|
||||||
|
|
||||||
// getWHOISStats builds WHOIS statistics from database and fetcher.
|
// getWHOISStats builds WHOIS statistics from database and fetcher.
|
||||||
func (s *Server) getWHOISStats(ctx context.Context) *WHOISStatsInfo {
|
func (s *Server) getWHOISStats(ctx context.Context) *WHOISStatsInfo {
|
||||||
// Get database WHOIS stats
|
// Get database WHOIS stats
|
||||||
@@ -470,7 +503,8 @@ type StreamStats struct {
|
|||||||
BGPPeerCount int `json:"bgp_peer_count"`
|
BGPPeerCount int `json:"bgp_peer_count"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// StatsResponse represents the API statistics response
|
// StatsResponse represents the API statistics response. The fields it shares
|
||||||
|
// with Stats mean what they mean there.
|
||||||
//
|
//
|
||||||
//nolint:tagliatelle // field names are the existing JSON API
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
type StatsResponse struct {
|
type StatsResponse struct {
|
||||||
@@ -507,6 +541,8 @@ type StatsResponse struct {
|
|||||||
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
|
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
|
||||||
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
|
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
|
||||||
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
|
WHOISStats *WHOISStatsInfo `json:"whois_stats,omitempty"`
|
||||||
|
|
||||||
|
OriginsWithoutKeptRoute int `json:"origins_without_kept_route"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// handlerStatsInfo converts the streamer's handler statistics for the API
|
// handlerStatsInfo converts the streamer's handler statistics for the API
|
||||||
@@ -626,8 +662,8 @@ func (s *Server) handleStats() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// statsResponse builds the /api/v1/stats answer from the stream metrics, the
|
// statsResponse builds the /api/v1/stats answer from the stream metrics, the
|
||||||
// database stats and the WHOIS stats, with the route update rates, memory use
|
// database stats, the routing table's figures and the WHOIS stats, with the
|
||||||
// and handler queues as they are now.
|
// route update rates, memory use and handler queues as they are now.
|
||||||
func (s *Server) statsResponse(
|
func (s *Server) statsResponse(
|
||||||
streamMetrics metrics.StreamMetrics,
|
streamMetrics metrics.StreamMetrics,
|
||||||
dbStats database.Stats,
|
dbStats database.Stats,
|
||||||
@@ -637,6 +673,7 @@ func (s *Server) statsResponse(
|
|||||||
|
|
||||||
// Get route update metrics
|
// Get route update metrics
|
||||||
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
|
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
|
||||||
|
counts := s.table.Counts()
|
||||||
|
|
||||||
// Get memory stats
|
// Get memory stats
|
||||||
var memStats runtime.MemStats
|
var memStats runtime.MemStats
|
||||||
@@ -660,23 +697,25 @@ func (s *Server) statsResponse(
|
|||||||
GC: gcStats(&memStats),
|
GC: gcStats(&memStats),
|
||||||
Stream: s.streamStats(),
|
Stream: s.streamStats(),
|
||||||
ASNs: dbStats.ASNs,
|
ASNs: dbStats.ASNs,
|
||||||
Prefixes: dbStats.Prefixes,
|
Prefixes: counts.IPv4Prefixes + counts.IPv6Prefixes,
|
||||||
IPv4Prefixes: dbStats.IPv4Prefixes,
|
IPv4Prefixes: counts.IPv4Prefixes,
|
||||||
IPv6Prefixes: dbStats.IPv6Prefixes,
|
IPv6Prefixes: counts.IPv6Prefixes,
|
||||||
Peerings: dbStats.Peerings,
|
Peerings: dbStats.Peerings,
|
||||||
Peers: dbStats.Peers,
|
Peers: dbStats.Peers,
|
||||||
DatabaseSizeBytes: dbStats.FileSizeBytes,
|
DatabaseSizeBytes: dbStats.FileSizeBytes,
|
||||||
LiveRoutes: dbStats.LiveRoutes,
|
LiveRoutes: counts.IPv4Routes + counts.IPv6Routes,
|
||||||
IPv4Routes: dbStats.IPv4Routes,
|
IPv4Routes: counts.IPv4Routes,
|
||||||
IPv6Routes: dbStats.IPv6Routes,
|
IPv6Routes: counts.IPv6Routes,
|
||||||
OldestRoute: dbStats.OldestRoute,
|
OldestRoute: counts.OldestKeptRoute,
|
||||||
NewestRoute: dbStats.NewestRoute,
|
NewestRoute: counts.NewestKeptRoute,
|
||||||
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
|
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
|
||||||
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
|
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
|
||||||
HandlerStats: handlerStatsInfo(s.streamer.GetHandlerStats()),
|
HandlerStats: handlerStatsInfo(s.streamer.GetHandlerStats()),
|
||||||
IPv4PrefixDistribution: dbStats.IPv4PrefixDistribution,
|
IPv4PrefixDistribution: prefixDistribution(counts.IPv4PrefixesByLength),
|
||||||
IPv6PrefixDistribution: dbStats.IPv6PrefixDistribution,
|
IPv6PrefixDistribution: prefixDistribution(counts.IPv6PrefixesByLength),
|
||||||
WHOISStats: whoisStats,
|
WHOISStats: whoisStats,
|
||||||
|
|
||||||
|
OriginsWithoutKeptRoute: counts.OriginsWithoutKeptRoute,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -766,7 +805,7 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
|||||||
|
|
||||||
// Look up each IP
|
// Look up each IP
|
||||||
for _, ip := range ipsToLookup {
|
for _, ip := range ipsToLookup {
|
||||||
ipInfo, err := s.db.GetIPInfoContext(ctx, ip)
|
ipInfo, err := s.ipInfo(ctx, ip)
|
||||||
if deadlinePassed(ctx) {
|
if deadlinePassed(ctx) {
|
||||||
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
|
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
|
||||||
|
|
||||||
@@ -815,6 +854,81 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ipInfo returns the IP lookup's answer for ip. The routing table gives the
|
||||||
|
// most specific prefix that holds the address, the number of peers that carry
|
||||||
|
// it, the origin AS that most of them carry, and the time of its newest kept
|
||||||
|
// route. The database gives the prefix's first-seen time and the AS's WHOIS
|
||||||
|
// details. It returns ErrNoRoute when no prefix holds the address, and
|
||||||
|
// ErrInvalidIP when ip does not parse.
|
||||||
|
func (s *Server) ipInfo(ctx context.Context, ip string) (*database.IPInfo, error) {
|
||||||
|
addr, err := netip.ParseAddr(ip)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%w: %s", database.ErrInvalidIP, ip)
|
||||||
|
}
|
||||||
|
|
||||||
|
summary, found := s.table.Lookup(addr)
|
||||||
|
if !found {
|
||||||
|
return nil, fmt.Errorf("%w for IP %s", database.ErrNoRoute, ip)
|
||||||
|
}
|
||||||
|
|
||||||
|
firstSeen, err := s.db.GetPrefixFirstSeen(ctx, summary.Prefix)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
asInfo, err := s.db.GetASWHOIS(ctx, summary.OriginASN)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
info := &database.IPInfo{
|
||||||
|
IP: ip,
|
||||||
|
Netblock: summary.Prefix.String(),
|
||||||
|
MaskLength: summary.Prefix.Bits(),
|
||||||
|
IPVersion: ipVersionOf(summary.Prefix),
|
||||||
|
NumPeers: summary.Peers,
|
||||||
|
ASN: summary.OriginASN,
|
||||||
|
ASName: asInfo.ASName,
|
||||||
|
Handle: asInfo.Handle,
|
||||||
|
Description: asInfo.Description,
|
||||||
|
OrgName: asInfo.OrgName,
|
||||||
|
OrgID: asInfo.OrgID,
|
||||||
|
Address: asInfo.Address,
|
||||||
|
CountryCode: asInfo.CountryCode,
|
||||||
|
AbuseEmail: asInfo.AbuseEmail,
|
||||||
|
RIR: asInfo.RIR,
|
||||||
|
FirstSeen: firstSeen,
|
||||||
|
NeedsWHOISRefresh: needsWHOISRefresh(asInfo),
|
||||||
|
}
|
||||||
|
|
||||||
|
if summary.Newest != nil {
|
||||||
|
info.LastSeen = *summary.Newest
|
||||||
|
}
|
||||||
|
|
||||||
|
return info, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// needsWHOISRefresh says whether the WHOIS details of asInfo were never
|
||||||
|
// fetched or are older than whoisStaleThreshold. AS 0, no known origin, has
|
||||||
|
// none to fetch.
|
||||||
|
func needsWHOISRefresh(asInfo *database.ASN) bool {
|
||||||
|
if asInfo.ASN == ristypes.NoOrigin {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
return asInfo.WHOISUpdatedAt == nil ||
|
||||||
|
time.Since(*asInfo.WHOISUpdatedAt) > whoisStaleThreshold
|
||||||
|
}
|
||||||
|
|
||||||
|
// ipVersionOf returns 4 or 6, the IP version of prefix.
|
||||||
|
func ipVersionOf(prefix netip.Prefix) int {
|
||||||
|
if prefix.Addr().Is4() {
|
||||||
|
return ipVersionV4
|
||||||
|
}
|
||||||
|
|
||||||
|
return ipVersionV6
|
||||||
|
}
|
||||||
|
|
||||||
// blocklistListings returns the blocklists that list ip. It returns an empty
|
// blocklistListings returns the blocklists that list ip. It returns an empty
|
||||||
// list, never nil, so that an answer's blocklists is [] rather than null.
|
// list, never nil, so that an answer's blocklists is [] rather than null.
|
||||||
func (s *Server) blocklistListings(ip string) []blocklist.Listing {
|
func (s *Server) blocklistListings(ip string) []blocklist.Listing {
|
||||||
@@ -1043,7 +1157,7 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
asInfo, prefixes, err := s.db.GetASDetailsContext(r.Context(), asn)
|
asInfo, prefixes, err := s.asDetails(r.Context(), asn)
|
||||||
if deadlinePassed(r.Context()) {
|
if deadlinePassed(r.Context()) {
|
||||||
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
|
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
|
||||||
|
|
||||||
@@ -1079,6 +1193,49 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// asDetails returns the AS's handle, description and seen times, from the
|
||||||
|
// database, and the prefixes it originates, from the routing table. Each
|
||||||
|
// prefix has the time of its newest kept route, if it keeps one. It returns
|
||||||
|
// ErrNoRoute for AS 0, which means no known origin. It also returns ErrNoRoute
|
||||||
|
// for an AS that the database, the AS list built into RouteWatch and the
|
||||||
|
// routing table all lack.
|
||||||
|
func (s *Server) asDetails(
|
||||||
|
ctx context.Context, asn int,
|
||||||
|
) (*database.ASN, []database.LiveRoute, error) {
|
||||||
|
if asn == ristypes.NoOrigin {
|
||||||
|
return nil, nil, fmt.Errorf("%w: AS%d", database.ErrNoRoute, asn)
|
||||||
|
}
|
||||||
|
|
||||||
|
asInfo, known, err := s.db.GetAS(ctx, asn)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
originated := s.table.OriginatedPrefixes(asn)
|
||||||
|
if !known && len(originated) == 0 {
|
||||||
|
return nil, nil, fmt.Errorf("%w: AS%d", database.ErrNoRoute, asn)
|
||||||
|
}
|
||||||
|
|
||||||
|
prefixes := make([]database.LiveRoute, 0, len(originated))
|
||||||
|
|
||||||
|
for _, p := range originated {
|
||||||
|
route := database.LiveRoute{
|
||||||
|
Prefix: p.Prefix.String(),
|
||||||
|
MaskLength: p.Prefix.Bits(),
|
||||||
|
IPVersion: ipVersionOf(p.Prefix),
|
||||||
|
OriginASN: asn,
|
||||||
|
}
|
||||||
|
|
||||||
|
if p.Newest != nil {
|
||||||
|
route.LastUpdated = *p.Newest
|
||||||
|
}
|
||||||
|
|
||||||
|
prefixes = append(prefixes, route)
|
||||||
|
}
|
||||||
|
|
||||||
|
return asInfo, prefixes, nil
|
||||||
|
}
|
||||||
|
|
||||||
// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones. Neither
|
// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones. Neither
|
||||||
// group is nil, so that a JSON answer holds [] for an empty one and not null.
|
// group is nil, so that a JSON answer holds [] for an empty one and not null.
|
||||||
func splitRoutesByIPVersion(
|
func splitRoutesByIPVersion(
|
||||||
@@ -1123,8 +1280,7 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Look the prefix up in the text form net/netip prints, the form the
|
// Masking the prefix lets any valid way of writing it find its routes.
|
||||||
// tables store, so that any valid way of writing it finds the route.
|
|
||||||
parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
|
parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.writeJSONError(w, http.StatusBadRequest, invalidPrefixMessage)
|
s.writeJSONError(w, http.StatusBadRequest, invalidPrefixMessage)
|
||||||
@@ -1132,27 +1288,18 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
prefix := parsed.Masked().String()
|
parsed = parsed.Masked()
|
||||||
|
prefix := parsed.String()
|
||||||
|
|
||||||
routes, _, err := s.db.GetPrefixDetailsContext(r.Context(), prefix)
|
origins := s.table.Origins(parsed)
|
||||||
if deadlinePassed(r.Context()) {
|
if len(origins) == 0 {
|
||||||
s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
|
s.writeJSONError(w, http.StatusNotFound,
|
||||||
|
database.ErrNoRoute.Error()+": "+prefix)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
if err != nil {
|
routes := prefixRoutes(parsed, origins)
|
||||||
if errors.Is(err, database.ErrNoRoute) {
|
|
||||||
s.writeJSONError(w, http.StatusNotFound, err.Error())
|
|
||||||
} else {
|
|
||||||
s.logger.Error("Failed to get prefix details",
|
|
||||||
"prefix", prefix, "error", err)
|
|
||||||
s.writeJSONError(w, http.StatusInternalServerError,
|
|
||||||
internalErrorMessage)
|
|
||||||
}
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Group by origin AS
|
// Group by origin AS
|
||||||
originMap := make(map[int][]database.LiveRoute)
|
originMap := make(map[int][]database.LiveRoute)
|
||||||
@@ -1237,7 +1384,7 @@ func (s *Server) handleASDetail() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
asInfo, prefixes, err := s.db.GetASDetailsContext(r.Context(), asn)
|
asInfo, prefixes, err := s.asDetails(r.Context(), asn)
|
||||||
if deadlinePassed(r.Context()) {
|
if deadlinePassed(r.Context()) {
|
||||||
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
|
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
|
||||||
|
|
||||||
@@ -1327,8 +1474,7 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Look the prefix up in the text form net/netip prints, the form the
|
// Masking the prefix lets any valid way of writing it find its routes.
|
||||||
// tables store, so that any valid way of writing it finds the route.
|
|
||||||
parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
|
parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
http.Error(w, invalidPrefixMessage, http.StatusBadRequest)
|
http.Error(w, invalidPrefixMessage, http.StatusBadRequest)
|
||||||
@@ -1336,9 +1482,16 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
prefix := parsed.Masked().String()
|
parsed = parsed.Masked()
|
||||||
|
|
||||||
routes, originNames, err := s.db.GetPrefixDetailsContext(r.Context(), prefix)
|
origins := s.table.Origins(parsed)
|
||||||
|
if len(origins) == 0 {
|
||||||
|
http.Error(w, "Prefix not found", http.StatusNotFound)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
originInfos, err := s.originASNs(r.Context(), origins)
|
||||||
if deadlinePassed(r.Context()) {
|
if deadlinePassed(r.Context()) {
|
||||||
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
|
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
|
||||||
|
|
||||||
@@ -1346,27 +1499,13 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, database.ErrNoRoute) {
|
s.logger.Error("Failed to get the origin ASes' names", "error", err)
|
||||||
http.Error(w, "Prefix not found", http.StatusNotFound)
|
http.Error(w, internalErrorMessage, http.StatusInternalServerError)
|
||||||
} else {
|
|
||||||
s.logger.Error("Failed to get prefix details", "error", err)
|
|
||||||
http.Error(w, internalErrorMessage, http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Group by origin AS and collect unique AS info
|
routes := prefixRoutes(parsed, origins)
|
||||||
origins := originASNs(routes, originNames)
|
|
||||||
|
|
||||||
// Get the first route to extract some common info
|
|
||||||
var maskLength, ipVersion int
|
|
||||||
if len(routes) > 0 {
|
|
||||||
maskLength, ipVersion = prefixMaskLengthAndIPVersion(prefix)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create enhanced routes with AS path handles
|
|
||||||
enhancedRoutes := routesWithASPathHandles(routes)
|
|
||||||
|
|
||||||
// Prepare template data
|
// Prepare template data
|
||||||
data := struct {
|
data := struct {
|
||||||
@@ -1379,13 +1518,13 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
|
|||||||
OriginCount int
|
OriginCount int
|
||||||
CSPNonce string
|
CSPNonce string
|
||||||
}{
|
}{
|
||||||
Prefix: prefix,
|
Prefix: parsed.String(),
|
||||||
MaskLength: maskLength,
|
MaskLength: parsed.Bits(),
|
||||||
IPVersion: ipVersion,
|
IPVersion: ipVersionOf(parsed),
|
||||||
Routes: enhancedRoutes,
|
Routes: routesWithASPathHandles(routes),
|
||||||
Origins: origins,
|
Origins: originInfos,
|
||||||
PeerCount: len(routes),
|
PeerCount: len(routes),
|
||||||
OriginCount: len(origins),
|
OriginCount: len(originInfos),
|
||||||
CSPNonce: cspNonce(r.Context()),
|
CSPNonce: cspNonce(r.Context()),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1415,34 +1554,72 @@ type ASNInfo struct {
|
|||||||
PeerCount int
|
PeerCount int
|
||||||
}
|
}
|
||||||
|
|
||||||
// originASNs groups routes by origin AS, with each AS's handle and description
|
// originASNs returns each of origins, a prefix's origin ASes, with its handle
|
||||||
// from names, as the prefix query returned them, and the number of routes it
|
// and description from the database and the number of peers that carry the
|
||||||
// originates. The origin ASes come in no particular order.
|
// prefix with it.
|
||||||
func originASNs(
|
func (s *Server) originASNs(
|
||||||
routes []database.LiveRoute,
|
ctx context.Context, origins []routetable.Origin,
|
||||||
names map[int]database.ASN,
|
) ([]*ASNInfo, error) {
|
||||||
) []*ASNInfo {
|
infos := make([]*ASNInfo, 0, len(origins))
|
||||||
originMap := make(map[int]*ASNInfo)
|
|
||||||
|
|
||||||
for _, route := range routes {
|
for _, origin := range origins {
|
||||||
if _, exists := originMap[route.OriginASN]; !exists {
|
asInfo, _, err := s.db.GetAS(ctx, origin.ASN)
|
||||||
originMap[route.OriginASN] = &ASNInfo{
|
if err != nil {
|
||||||
ASN: route.OriginASN,
|
return nil, err
|
||||||
Handle: names[route.OriginASN].Handle,
|
|
||||||
Description: names[route.OriginASN].Description,
|
|
||||||
PeerCount: 0,
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
originMap[route.OriginASN].PeerCount++
|
infos = append(infos, &ASNInfo{
|
||||||
|
ASN: origin.ASN,
|
||||||
|
Handle: asInfo.Handle,
|
||||||
|
Description: asInfo.Description,
|
||||||
|
PeerCount: len(origin.Peers),
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
origins := make([]*ASNInfo, 0, len(originMap))
|
return infos, nil
|
||||||
for _, origin := range originMap {
|
}
|
||||||
origins = append(origins, origin)
|
|
||||||
|
// prefixRoutes returns one route for each peer that carries prefix, by origin
|
||||||
|
// AS in the order of origins. Each origin's kept routes come first, newest
|
||||||
|
// first, with their AS path, next hop and time. Its other peers follow without
|
||||||
|
// them, since the routing table keeps only their addresses.
|
||||||
|
func prefixRoutes(
|
||||||
|
prefix netip.Prefix, origins []routetable.Origin,
|
||||||
|
) []database.LiveRoute {
|
||||||
|
var routes []database.LiveRoute
|
||||||
|
|
||||||
|
for _, origin := range origins {
|
||||||
|
base := database.LiveRoute{
|
||||||
|
Prefix: prefix.String(),
|
||||||
|
MaskLength: prefix.Bits(),
|
||||||
|
IPVersion: ipVersionOf(prefix),
|
||||||
|
OriginASN: origin.ASN,
|
||||||
|
}
|
||||||
|
|
||||||
|
kept := make(map[string]bool, len(origin.Routes))
|
||||||
|
|
||||||
|
for _, r := range origin.Routes {
|
||||||
|
kept[r.Peer] = true
|
||||||
|
|
||||||
|
route := base
|
||||||
|
route.PeerIP = r.Peer
|
||||||
|
route.ASPath = r.ASPath
|
||||||
|
// The next hop is the peer, as the prefix handler records it.
|
||||||
|
route.NextHop = r.Peer
|
||||||
|
route.LastUpdated = r.Time
|
||||||
|
routes = append(routes, route)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, peer := range origin.Peers {
|
||||||
|
if !kept[peer] {
|
||||||
|
route := base
|
||||||
|
route.PeerIP = peer
|
||||||
|
routes = append(routes, route)
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return origins
|
return routes
|
||||||
}
|
}
|
||||||
|
|
||||||
// ASPathEntry is one AS of a route's AS path, with its handle.
|
// ASPathEntry is one AS of a route's AS path, with its handle.
|
||||||
@@ -1484,23 +1661,6 @@ func routesWithASPathHandles(routes []database.LiveRoute) []EnhancedRoute {
|
|||||||
return enhancedRoutes
|
return enhancedRoutes
|
||||||
}
|
}
|
||||||
|
|
||||||
// prefixMaskLengthAndIPVersion returns the mask length and IP version of a
|
|
||||||
// prefix in CIDR notation, or zeros if it does not parse.
|
|
||||||
func prefixMaskLengthAndIPVersion(prefix string) (int, int) {
|
|
||||||
_, ipNet, err := net.ParseCIDR(prefix)
|
|
||||||
if err != nil {
|
|
||||||
return 0, 0
|
|
||||||
}
|
|
||||||
|
|
||||||
maskLength, _ := ipNet.Mask.Size()
|
|
||||||
|
|
||||||
if ipNet.IP.To4() != nil {
|
|
||||||
return maskLength, ipVersionV4
|
|
||||||
}
|
|
||||||
|
|
||||||
return maskLength, ipVersionV6
|
|
||||||
}
|
|
||||||
|
|
||||||
// handlePrefixLength shows a random sample of IPv4 prefixes with the
|
// handlePrefixLength shows a random sample of IPv4 prefixes with the
|
||||||
// specified mask length
|
// specified mask length
|
||||||
func (s *Server) handlePrefixLength() http.HandlerFunc {
|
func (s *Server) handlePrefixLength() http.HandlerFunc {
|
||||||
@@ -1539,29 +1699,11 @@ func (s *Server) prefixLengthHandler(ipVersion, maxMaskLength int) http.HandlerF
|
|||||||
// Get random sample of prefixes
|
// Get random sample of prefixes
|
||||||
const maxPrefixes = 500
|
const maxPrefixes = 500
|
||||||
|
|
||||||
prefixes, err := s.db.GetRandomPrefixesByLengthContext(
|
prefixes := s.table.SamplePrefixes(ipVersion, maskLength, maxPrefixes)
|
||||||
r.Context(), maskLength, ipVersion, maxPrefixes)
|
|
||||||
if deadlinePassed(r.Context()) {
|
|
||||||
http.Error(w, requestTimeoutMessage, http.StatusGatewayTimeout)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if err != nil {
|
|
||||||
s.logger.Error("Failed to get prefixes by length", "error", err)
|
|
||||||
http.Error(w, internalErrorMessage, http.StatusInternalServerError)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sort prefixes for display
|
// Sort prefixes for display
|
||||||
sort.Slice(prefixes, func(i, j int) bool {
|
slices.SortFunc(prefixes, func(a, b routetable.PrefixSummary) int {
|
||||||
// First compare by IP version
|
return a.Prefix.Compare(b.Prefix)
|
||||||
if prefixes[i].IPVersion != prefixes[j].IPVersion {
|
|
||||||
return prefixes[i].IPVersion < prefixes[j].IPVersion
|
|
||||||
}
|
|
||||||
// Then by prefix
|
|
||||||
return prefixes[i].Prefix < prefixes[j].Prefix
|
|
||||||
})
|
})
|
||||||
|
|
||||||
// Render template
|
// Render template
|
||||||
@@ -1591,8 +1733,8 @@ func (s *Server) prefixLengthHandler(ipVersion, maxMaskLength int) http.HandlerF
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// EnhancedPrefix is a live route on the prefix length page, with the
|
// EnhancedPrefix is a prefix on the prefix length page, with the origin AS
|
||||||
// description of its origin AS and its age.
|
// that most peers carry it with, that AS's description, and its age.
|
||||||
type EnhancedPrefix struct {
|
type EnhancedPrefix struct {
|
||||||
database.LiveRoute
|
database.LiveRoute
|
||||||
|
|
||||||
@@ -1601,13 +1743,23 @@ type EnhancedPrefix struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// prefixesWithASDescriptions looks up the description of the origin AS of
|
// prefixesWithASDescriptions looks up the description of the origin AS of
|
||||||
// each prefix and gives each one's age.
|
// each prefix. It gives each prefix the age of its newest kept route, or no
|
||||||
func prefixesWithASDescriptions(prefixes []database.LiveRoute) []EnhancedPrefix {
|
// age when it keeps none.
|
||||||
|
func prefixesWithASDescriptions(
|
||||||
|
prefixes []routetable.PrefixSummary,
|
||||||
|
) []EnhancedPrefix {
|
||||||
enhancedPrefixes := make([]EnhancedPrefix, len(prefixes))
|
enhancedPrefixes := make([]EnhancedPrefix, len(prefixes))
|
||||||
for i, prefix := range prefixes {
|
for i, prefix := range prefixes {
|
||||||
enhancedPrefixes[i] = EnhancedPrefix{
|
enhancedPrefixes[i] = EnhancedPrefix{
|
||||||
LiveRoute: prefix,
|
Prefix: prefix.Prefix.String(),
|
||||||
Age: formatAge(prefix.LastUpdated),
|
MaskLength: prefix.Prefix.Bits(),
|
||||||
|
IPVersion: ipVersionOf(prefix.Prefix),
|
||||||
|
OriginASN: prefix.OriginASN,
|
||||||
|
}
|
||||||
|
|
||||||
|
if prefix.Newest != nil {
|
||||||
|
enhancedPrefixes[i].LastUpdated = *prefix.Newest
|
||||||
|
enhancedPrefixes[i].Age = formatAge(*prefix.Newest)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get AS description
|
// Get AS description
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"runtime"
|
"runtime"
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
@@ -23,6 +24,7 @@ import (
|
|||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
||||||
)
|
)
|
||||||
@@ -35,6 +37,38 @@ const (
|
|||||||
testPeerV6 = "2001:db8::1"
|
testPeerV6 = "2001:db8::1"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// testPeer returns the address of the nth of the IPv4 peers that tests
|
||||||
|
// announce routes from: 198.51.100.n.
|
||||||
|
func testPeer(n int) string {
|
||||||
|
return "198.51.100." + strconv.Itoa(n)
|
||||||
|
}
|
||||||
|
|
||||||
|
// testRouteTime is the time the tests' routes are announced at, or a number of
|
||||||
|
// minutes after it.
|
||||||
|
func testRouteTime(minutes int) time.Time {
|
||||||
|
return time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC).
|
||||||
|
Add(time.Duration(minutes) * time.Minute)
|
||||||
|
}
|
||||||
|
|
||||||
|
// announce has peer announce prefix in table at time at, with an AS path
|
||||||
|
// that holds only the origin AS asn.
|
||||||
|
func announce(
|
||||||
|
table *routetable.Table, prefix string, asn int, peer string, at time.Time,
|
||||||
|
) {
|
||||||
|
table.Announce(netip.MustParsePrefix(prefix), asn,
|
||||||
|
routetable.Route{Peer: peer, ASPath: []int{asn}, Time: at})
|
||||||
|
}
|
||||||
|
|
||||||
|
// withdraw has peer withdraw prefix in table.
|
||||||
|
func withdraw(table *routetable.Table, prefix, peer string) {
|
||||||
|
table.Withdraw(netip.MustParsePrefix(prefix), peer)
|
||||||
|
}
|
||||||
|
|
||||||
|
// emptyTable returns a routing table that holds no route.
|
||||||
|
func emptyTable() *routetable.Table {
|
||||||
|
return routetable.New(routetable.Params{})
|
||||||
|
}
|
||||||
|
|
||||||
// Paths several tests request.
|
// Paths several tests request.
|
||||||
const (
|
const (
|
||||||
asPagePath = "/as/64500"
|
asPagePath = "/as/64500"
|
||||||
@@ -76,7 +110,8 @@ func (d blockingStatsDB) GetStatsContext(_ context.Context) (database.Stats, err
|
|||||||
func TestStatsHandlersDoNotLeakOnTimeout(t *testing.T) {
|
func TestStatsHandlersDoNotLeakOnTimeout(t *testing.T) {
|
||||||
release := make(chan struct{})
|
release := make(chan struct{})
|
||||||
db := blockingStatsDB{release: release}
|
db := blockingStatsDB{release: release}
|
||||||
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
s := New(db, emptyTable(), streamer.New(logger.New(), metrics.New()),
|
||||||
|
logger.New(), &config.Config{})
|
||||||
|
|
||||||
handlers := map[string]http.HandlerFunc{
|
handlers := map[string]http.HandlerFunc{
|
||||||
statusJSONHandlerName: s.handleStatusJSON(),
|
statusJSONHandlerName: s.handleStatusJSON(),
|
||||||
@@ -108,51 +143,78 @@ func TestStatsHandlersDoNotLeakOnTimeout(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// tableStats holds the figures of a stats answer that come from the routing
|
||||||
|
// table.
|
||||||
|
//
|
||||||
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
|
type tableStats struct {
|
||||||
|
Prefixes int `json:"prefixes"`
|
||||||
|
IPv4Prefixes int `json:"ipv4_prefixes"`
|
||||||
|
IPv6Prefixes int `json:"ipv6_prefixes"`
|
||||||
|
LiveRoutes int `json:"live_routes"`
|
||||||
|
IPv4Routes int `json:"ipv4_routes"`
|
||||||
|
IPv6Routes int `json:"ipv6_routes"`
|
||||||
|
OriginsWithoutKeptRoute int `json:"origins_without_kept_route"`
|
||||||
|
OldestRoute time.Time `json:"oldest_route"`
|
||||||
|
NewestRoute time.Time `json:"newest_route"`
|
||||||
|
|
||||||
|
IPv4Distribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
|
||||||
|
IPv6Distribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
|
||||||
|
}
|
||||||
|
|
||||||
// TestStatsHandlersAnswerFromMemory checks that both stats handlers answer 200
|
// TestStatsHandlersAnswerFromMemory checks that both stats handlers answer 200
|
||||||
// with the live route counts and the prefix distribution while the database is
|
// with the routing table's figures while the database is closed, so that any
|
||||||
// closed, so that any query would fail: the request path reads them from
|
// query would fail. The prefix distribution query they used to run read every
|
||||||
// memory. The prefix distribution query it used to run read every live route
|
// live route and, on a large database, took the whole 4-second deadline, so
|
||||||
// and, on a large database, took the whole 4-second deadline, so
|
|
||||||
// /api/v1/stats answered 500 (https://git.eeqj.de/sneak/routewatch/issues/30).
|
// /api/v1/stats answered 500 (https://git.eeqj.de/sneak/routewatch/issues/30).
|
||||||
// The oldest and newest route times still come from one-row lookups at the ends
|
//
|
||||||
// of an index; with the database closed they are left out of the answer.
|
// The table holds two IPv4 prefixes and an IPv6 one. Of the three peers that
|
||||||
|
// carried 203.0.113.0/24, the two with kept routes withdrew it, so its origin
|
||||||
|
// keeps no route. The oldest and newest routes are those of the other two
|
||||||
|
// prefixes.
|
||||||
func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
const withdrawnPrefix = "203.0.113.0/24"
|
||||||
|
|
||||||
|
table := emptyTable()
|
||||||
|
announce(table, testPrefixV4, 64502, testPeer(1), testRouteTime(0))
|
||||||
|
|
||||||
|
for n := 1; n <= 3; n++ {
|
||||||
|
announce(table, withdrawnPrefix, 64500, testPeer(n), testRouteTime(n))
|
||||||
|
}
|
||||||
|
|
||||||
|
withdraw(table, withdrawnPrefix, testPeer(2))
|
||||||
|
withdraw(table, withdrawnPrefix, testPeer(3))
|
||||||
|
announce(table, testPrefixV6, 64501, testPeerV6, testRouteTime(4))
|
||||||
|
|
||||||
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("database.New: %v", err)
|
t.Fatalf("database.New: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
|
||||||
|
|
||||||
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{
|
|
||||||
{
|
|
||||||
Prefix: "198.51.100.0/24", MaskLength: 24, IPVersion: 4,
|
|
||||||
OriginASN: 64500, PeerIP: "192.0.2.1", ASPath: []int{64500},
|
|
||||||
NextHop: "192.0.2.1", LastUpdated: ts,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
Prefix: "2001:db8::/32", MaskLength: 32, IPVersion: 6,
|
|
||||||
OriginASN: 64501, PeerIP: testPeerV6, ASPath: []int{64501},
|
|
||||||
NextHop: testPeerV6, LastUpdated: ts,
|
|
||||||
},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = db.Close()
|
err = db.Close()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Close: %v", err)
|
t.Fatalf("Close: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
s := New(db, table, streamer.New(logger.New(), metrics.New()), logger.New(),
|
||||||
|
&config.Config{})
|
||||||
handlers := map[string]http.HandlerFunc{
|
handlers := map[string]http.HandlerFunc{
|
||||||
statusJSONHandlerName: s.handleStatusJSON(),
|
statusJSONHandlerName: s.handleStatusJSON(),
|
||||||
statsHandlerName: s.handleStats(),
|
statsHandlerName: s.handleStats(),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
want := tableStats{
|
||||||
|
Prefixes: 3, IPv4Prefixes: 2, IPv6Prefixes: 1,
|
||||||
|
LiveRoutes: 3, IPv4Routes: 2, IPv6Routes: 1,
|
||||||
|
OriginsWithoutKeptRoute: 1,
|
||||||
|
OldestRoute: testRouteTime(0),
|
||||||
|
NewestRoute: testRouteTime(4),
|
||||||
|
IPv4Distribution: []database.PrefixDistribution{{MaskLength: 24, Count: 2}},
|
||||||
|
IPv6Distribution: []database.PrefixDistribution{{MaskLength: 32, Count: 1}},
|
||||||
|
}
|
||||||
|
|
||||||
for name, handler := range handlers {
|
for name, handler := range handlers {
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
handler(rec, httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil))
|
handler(rec, httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil))
|
||||||
@@ -164,14 +226,8 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
//nolint:tagliatelle // field names are the existing JSON API
|
|
||||||
var body struct {
|
var body struct {
|
||||||
Data struct {
|
Data tableStats `json:"data"`
|
||||||
IPv4Routes int `json:"ipv4_routes"`
|
|
||||||
IPv6Routes int `json:"ipv6_routes"`
|
|
||||||
IPv4Distribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
|
|
||||||
IPv6Distribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
|
|
||||||
} `json:"data"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
err := json.Unmarshal(rec.Body.Bytes(), &body)
|
err := json.Unmarshal(rec.Body.Bytes(), &body)
|
||||||
@@ -179,21 +235,16 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
|||||||
t.Fatalf("%s: decoding the answer: %v", name, err)
|
t.Fatalf("%s: decoding the answer: %v", name, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if body.Data.IPv4Routes != 1 || body.Data.IPv6Routes != 1 {
|
got := body.Data
|
||||||
t.Errorf("%s: routes = (v4 %d, v6 %d), want (1, 1)",
|
if !got.OldestRoute.Equal(want.OldestRoute) ||
|
||||||
name, body.Data.IPv4Routes, body.Data.IPv6Routes)
|
!got.NewestRoute.Equal(want.NewestRoute) {
|
||||||
|
t.Errorf("%s: routes from %v to %v, want from %v to %v", name,
|
||||||
|
got.OldestRoute, got.NewestRoute, want.OldestRoute, want.NewestRoute)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantV4 := []database.PrefixDistribution{{MaskLength: 24, Count: 1}}
|
got.OldestRoute, got.NewestRoute = want.OldestRoute, want.NewestRoute
|
||||||
if !slices.Equal(body.Data.IPv4Distribution, wantV4) {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Errorf("%s: IPv4 distribution = %v, want %v",
|
t.Errorf("%s: figures %+v, want %+v", name, got, want)
|
||||||
name, body.Data.IPv4Distribution, wantV4)
|
|
||||||
}
|
|
||||||
|
|
||||||
wantV6 := []database.PrefixDistribution{{MaskLength: 32, Count: 1}}
|
|
||||||
if !slices.Equal(body.Data.IPv6Distribution, wantV6) {
|
|
||||||
t.Errorf("%s: IPv6 distribution = %v, want %v",
|
|
||||||
name, body.Data.IPv6Distribution, wantV6)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -243,11 +294,29 @@ func TestStatsAnswerCarriesChurnPerMinute(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// newServerWithTestRoutes returns a server on a new database that holds a
|
// newTestTable returns a routing table that holds a route for testPrefixV4,
|
||||||
// route for testPrefixV4 and one for testPrefixV6.
|
// with origin AS 64500, and one for testPrefixV6, with origin AS 64501.
|
||||||
|
func newTestTable() *routetable.Table {
|
||||||
|
table := emptyTable()
|
||||||
|
announce(table, testPrefixV4, 64500, testPeerV4, testRouteTime(0))
|
||||||
|
announce(table, testPrefixV6, 64501, testPeerV6, testRouteTime(0))
|
||||||
|
|
||||||
|
return table
|
||||||
|
}
|
||||||
|
|
||||||
|
// newServerWithTestRoutes returns a server on a new database that reads the
|
||||||
|
// routes of newTestTable.
|
||||||
func newServerWithTestRoutes(t *testing.T) *Server {
|
func newServerWithTestRoutes(t *testing.T) *Server {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
|
return newServerWithTable(t, newTestTable())
|
||||||
|
}
|
||||||
|
|
||||||
|
// newServerWithTable returns a server on a new database that reads routes from
|
||||||
|
// table.
|
||||||
|
func newServerWithTable(t *testing.T, table *routetable.Table) *Server {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("database.New: %v", err)
|
t.Fatalf("database.New: %v", err)
|
||||||
@@ -255,25 +324,7 @@ func newServerWithTestRoutes(t *testing.T) *Server {
|
|||||||
|
|
||||||
t.Cleanup(func() { _ = db.Close() })
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
return New(db, table, streamer.New(logger.New(), metrics.New()), logger.New(),
|
||||||
|
|
||||||
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{
|
|
||||||
{
|
|
||||||
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
|
|
||||||
OriginASN: 64500, PeerIP: testPeerV4, ASPath: []int{64500},
|
|
||||||
NextHop: testPeerV4, LastUpdated: ts,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
Prefix: testPrefixV6, MaskLength: 32, IPVersion: 6,
|
|
||||||
OriginASN: 64501, PeerIP: testPeerV6, ASPath: []int{64501},
|
|
||||||
NextHop: testPeerV6, LastUpdated: ts,
|
|
||||||
},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return New(db, streamer.New(logger.New(), metrics.New()), logger.New(),
|
|
||||||
&config.Config{})
|
&config.Config{})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -402,34 +453,20 @@ func TestPrefixDetailShowsOriginNames(t *testing.T) {
|
|||||||
|
|
||||||
t.Cleanup(func() { _ = db.Close() })
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
|
||||||
origins := []int{15169, 13335}
|
origins := []int{15169, 13335}
|
||||||
|
table := emptyTable()
|
||||||
|
|
||||||
for _, asn := range origins {
|
for i, asn := range origins {
|
||||||
_, err := db.GetOrCreateASN(asn, ts)
|
_, err := db.GetOrCreateASN(asn, testRouteTime(0))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("GetOrCreateASN(%d): %v", asn, err)
|
t.Fatalf("GetOrCreateASN(%d): %v", asn, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
announce(table, testPrefixV4, asn, testPeer(i+1), testRouteTime(0))
|
||||||
}
|
}
|
||||||
|
|
||||||
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{
|
s := New(failingASDetailsDB{db}, table,
|
||||||
{
|
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
|
|
||||||
OriginASN: origins[0], PeerIP: testPeerV4,
|
|
||||||
ASPath: []int{origins[0]}, NextHop: testPeerV4, LastUpdated: ts,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
|
|
||||||
OriginASN: origins[1], PeerIP: "198.51.100.2",
|
|
||||||
ASPath: []int{origins[1]}, NextHop: "198.51.100.2", LastUpdated: ts,
|
|
||||||
},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
s := New(failingASDetailsDB{db}, streamer.New(logger.New(), metrics.New()),
|
|
||||||
logger.New(), &config.Config{})
|
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
t.Context(), http.MethodGet, "/prefix/"+testPrefixV4, nil))
|
t.Context(), http.MethodGet, "/prefix/"+testPrefixV4, nil))
|
||||||
@@ -466,23 +503,14 @@ func TestPrefixDetailShowsOriginNames(t *testing.T) {
|
|||||||
func TestPagesSayOriginZeroIsNoKnownOrigin(t *testing.T) {
|
func TestPagesSayOriginZeroIsNoKnownOrigin(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
table := emptyTable()
|
||||||
if err != nil {
|
table.Announce(netip.MustParsePrefix(testPrefixV4), ristypes.NoOrigin,
|
||||||
t.Fatalf("database.New: %v", err)
|
routetable.Route{
|
||||||
}
|
Peer: testPeerV4, ASPath: []int{64500, 64501, 64502},
|
||||||
|
Time: testRouteTime(0),
|
||||||
|
})
|
||||||
|
|
||||||
t.Cleanup(func() { _ = db.Close() })
|
s := newServerWithTable(t, table)
|
||||||
|
|
||||||
err = db.UpsertLiveRouteBatch([]*database.LiveRoute{{
|
|
||||||
Prefix: testPrefixV4, MaskLength: 24, IPVersion: 4,
|
|
||||||
OriginASN: 0, PeerIP: testPeerV4, ASPath: []int{64500, 64501, 64502},
|
|
||||||
NextHop: testPeerV4, LastUpdated: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC),
|
|
||||||
}})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
|
||||||
|
|
||||||
for _, path := range []string{"/prefix/" + testPrefixV4, prefixLengthPagePath} {
|
for _, path := range []string{"/prefix/" + testPrefixV4, prefixLengthPagePath} {
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
@@ -533,8 +561,9 @@ func TestIndexShowsTheClientAddress(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
// The home page reads nothing from the database, so the server has none.
|
// The home page reads no route and nothing from the database, so the
|
||||||
s := New(nil, streamer.New(logger.New(), metrics.New()), logger.New(),
|
// server has neither a routing table nor a database.
|
||||||
|
s := New(nil, nil, streamer.New(logger.New(), metrics.New()), logger.New(),
|
||||||
&config.Config{TrustedProxies: tt.trustedProxies})
|
&config.Config{TrustedProxies: tt.trustedProxies})
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
|
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
|
||||||
@@ -571,13 +600,6 @@ func TestIndexShowsTheClientAddress(t *testing.T) {
|
|||||||
func TestPagesShowAgesInWords(t *testing.T) {
|
func TestPagesShowAgesInWords(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("database.New: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Cleanup(func() { _ = db.Close() })
|
|
||||||
|
|
||||||
// Each age is well past a whole number of its unit, so the time the test
|
// Each age is well past a whole number of its unit, so the time the test
|
||||||
// takes to run does not change the words.
|
// takes to run does not change the words.
|
||||||
ages := []struct {
|
ages := []struct {
|
||||||
@@ -591,22 +613,13 @@ func TestPagesShowAgesInWords(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
now := time.Now().UTC()
|
now := time.Now().UTC()
|
||||||
routes := make([]*database.LiveRoute, 0, len(ages))
|
table := emptyTable()
|
||||||
|
|
||||||
for _, a := range ages {
|
for _, a := range ages {
|
||||||
routes = append(routes, &database.LiveRoute{
|
announce(table, a.prefix, 64500, testPeerV4, now.Add(-a.age))
|
||||||
Prefix: a.prefix, MaskLength: 24, IPVersion: 4,
|
|
||||||
OriginASN: 64500, PeerIP: testPeerV4, ASPath: []int{64500},
|
|
||||||
NextHop: testPeerV4, LastUpdated: now.Add(-a.age),
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
|
|
||||||
err = db.UpsertLiveRouteBatch(routes)
|
s := newServerWithTable(t, table)
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
|
||||||
|
|
||||||
for _, a := range ages {
|
for _, a := range ages {
|
||||||
cell := `<td class="age">` + a.want + `</td>`
|
cell := `<td class="age">` + a.want + `</td>`
|
||||||
@@ -643,7 +656,7 @@ const shortRequestDeadline = 50 * time.Millisecond
|
|||||||
func TestRequestPastTimeLimitAnswers504(t *testing.T) {
|
func TestRequestPastTimeLimitAnswers504(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := New(nil, streamer.New(logger.New(), metrics.New()), logger.New(),
|
s := New(nil, nil, streamer.New(logger.New(), metrics.New()), logger.New(),
|
||||||
&config.Config{})
|
&config.Config{})
|
||||||
|
|
||||||
waitForTimeLimit := func(_ http.ResponseWriter, r *http.Request) {
|
waitForTimeLimit := func(_ http.ResponseWriter, r *http.Request) {
|
||||||
@@ -693,8 +706,8 @@ func TestStatsHandlersAnswer504WhenDatabaseReadRunsOut(t *testing.T) {
|
|||||||
defer close(release)
|
defer close(release)
|
||||||
|
|
||||||
db := blockingStatsDB{release: release}
|
db := blockingStatsDB{release: release}
|
||||||
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(),
|
s := New(db, emptyTable(), streamer.New(logger.New(), metrics.New()),
|
||||||
&config.Config{})
|
logger.New(), &config.Config{})
|
||||||
|
|
||||||
handlers := map[string]http.HandlerFunc{
|
handlers := map[string]http.HandlerFunc{
|
||||||
statusJSONHandlerName: s.handleStatusJSON(),
|
statusJSONHandlerName: s.handleStatusJSON(),
|
||||||
@@ -720,10 +733,9 @@ func TestStatsHandlersAnswer504WhenDatabaseReadRunsOut(t *testing.T) {
|
|||||||
// slowDB embeds database.Store (left nil) and overrides the reads the routes
|
// slowDB embeds database.Store (left nil) and overrides the reads the routes
|
||||||
// in TestRoutesAnswer504WhenDatabaseReadIsCutOff make. Each read waits until
|
// in TestRoutesAnswer504WhenDatabaseReadIsCutOff make. Each read waits until
|
||||||
// its context ends and then answers as the real store does when the time
|
// its context ends and then answers as the real store does when the time
|
||||||
// limit cuts it off: the IP read with the context's error, the prefix read
|
// limit cuts it off: the stats read with what it had read, here nothing, and
|
||||||
// that there is no route, and the others with what they had read, here
|
// no error, and the others with the context's error. Every other Store method
|
||||||
// nothing, and no error. Every other Store method is unused and would panic
|
// is unused and would panic if called.
|
||||||
// if called.
|
|
||||||
type slowDB struct {
|
type slowDB struct {
|
||||||
database.Store
|
database.Store
|
||||||
}
|
}
|
||||||
@@ -734,66 +746,44 @@ func (slowDB) GetStatsContext(ctx context.Context) (database.Stats, error) {
|
|||||||
return database.Stats{}, nil
|
return database.Stats{}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (slowDB) GetIPInfoContext(
|
func (slowDB) GetAS(ctx context.Context, _ int) (*database.ASN, bool, error) {
|
||||||
ctx context.Context, _ string,
|
|
||||||
) (*database.IPInfo, error) {
|
|
||||||
<-ctx.Done()
|
<-ctx.Done()
|
||||||
|
|
||||||
return nil, ctx.Err()
|
return nil, false, ctx.Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
func (slowDB) GetASDetailsContext(
|
func (slowDB) GetPrefixFirstSeen(
|
||||||
ctx context.Context, asn int,
|
ctx context.Context, _ netip.Prefix,
|
||||||
) (*database.ASN, []database.LiveRoute, error) {
|
) (time.Time, error) {
|
||||||
<-ctx.Done()
|
<-ctx.Done()
|
||||||
|
|
||||||
return &database.ASN{ASN: asn}, nil, nil
|
return time.Time{}, ctx.Err()
|
||||||
}
|
|
||||||
|
|
||||||
func (slowDB) GetPrefixDetailsContext(
|
|
||||||
ctx context.Context, _ string,
|
|
||||||
) ([]database.LiveRoute, map[int]database.ASN, error) {
|
|
||||||
<-ctx.Done()
|
|
||||||
|
|
||||||
return nil, nil, database.ErrNoRoute
|
|
||||||
}
|
|
||||||
|
|
||||||
func (slowDB) GetRandomPrefixesByLengthContext(
|
|
||||||
ctx context.Context, _, _, _ int,
|
|
||||||
) ([]database.LiveRoute, error) {
|
|
||||||
<-ctx.Done()
|
|
||||||
|
|
||||||
return nil, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestRoutesAnswer504WhenDatabaseReadIsCutOff sends a request through the
|
// TestRoutesAnswer504WhenDatabaseReadIsCutOff sends a request through the
|
||||||
// server's router for each route that reads the database, with a store whose
|
// server's router for each route that reads the database, with a store whose
|
||||||
// reads are cut off when the request's time limit passes. Each route answers
|
// reads are cut off when the request's time limit passes. Each route answers
|
||||||
// 504 with the time limit message. Without its check after the read, a route
|
// 504 with the time limit message. Without its check after the read, a route
|
||||||
// answers with what the cut-off read returned: 500, 404, 200, or, on a page
|
// answers with what the cut-off read returned: 500 or 200. The health check is
|
||||||
// that stops before rendering once the request has ended, 504 with an empty
|
// left out: it answers 503 when its own 2-second ping runs out. The prefix
|
||||||
// body. The health check is left out: it answers 503 when its own 2-second
|
// length pages and the prefix API read only the routing table.
|
||||||
// ping runs out.
|
|
||||||
func TestRoutesAnswer504WhenDatabaseReadIsCutOff(t *testing.T) {
|
func TestRoutesAnswer504WhenDatabaseReadIsCutOff(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := New(slowDB{}, streamer.New(logger.New(), metrics.New()), logger.New(),
|
s := New(slowDB{}, newTestTable(), streamer.New(logger.New(), metrics.New()),
|
||||||
&config.Config{})
|
logger.New(), &config.Config{})
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
path string
|
path string
|
||||||
wantJSONBody bool
|
wantJSONBody bool
|
||||||
}{
|
}{
|
||||||
{path: asPagePath},
|
{path: asPagePath},
|
||||||
{path: "/prefix/192.0.2.0/24"},
|
{path: "/prefix/" + testPrefixV4},
|
||||||
{path: prefixLengthPagePath},
|
|
||||||
{path: "/prefixlength6/48"},
|
|
||||||
{path: statusJSONPath, wantJSONBody: true},
|
{path: statusJSONPath, wantJSONBody: true},
|
||||||
{path: "/ip/192.0.2.1", wantJSONBody: true},
|
{path: "/ip/" + addrInTestPrefixV4, wantJSONBody: true},
|
||||||
{path: statsPath, wantJSONBody: true},
|
{path: statsPath, wantJSONBody: true},
|
||||||
{path: "/api/v1/ip/192.0.2.1", wantJSONBody: true},
|
{path: "/api/v1/ip/" + addrInTestPrefixV4, wantJSONBody: true},
|
||||||
{path: "/api/v1/as/64500", wantJSONBody: true},
|
{path: "/api/v1/as/64500", wantJSONBody: true},
|
||||||
{path: "/api/v1/prefix/192.0.2.0/24", wantJSONBody: true},
|
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
@@ -810,10 +800,8 @@ type slowSecondReadDB struct {
|
|||||||
database.Store
|
database.Store
|
||||||
}
|
}
|
||||||
|
|
||||||
func (slowSecondReadDB) GetASDetailsContext(
|
func (slowSecondReadDB) GetAS(_ context.Context, asn int) (*database.ASN, bool, error) {
|
||||||
_ context.Context, asn int,
|
return &database.ASN{ASN: asn}, true, nil
|
||||||
) (*database.ASN, []database.LiveRoute, error) {
|
|
||||||
return &database.ASN{ASN: asn}, nil, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (slowSecondReadDB) GetASPeersContext(
|
func (slowSecondReadDB) GetASPeersContext(
|
||||||
@@ -855,8 +843,8 @@ func (idleASNFetcher) GetStats() ASNFetcherStats {
|
|||||||
func TestRoutesAnswer504WhenSecondDatabaseReadIsCutOff(t *testing.T) {
|
func TestRoutesAnswer504WhenSecondDatabaseReadIsCutOff(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := New(slowSecondReadDB{}, streamer.New(logger.New(), metrics.New()),
|
s := New(slowSecondReadDB{}, emptyTable(),
|
||||||
logger.New(), &config.Config{})
|
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
s.SetASNFetcher(idleASNFetcher{})
|
s.SetASNFetcher(idleASNFetcher{})
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
@@ -950,34 +938,27 @@ func (failingDB) GetStatsContext(_ context.Context) (database.Stats, error) {
|
|||||||
return database.Stats{}, errDiskIO
|
return database.Stats{}, errDiskIO
|
||||||
}
|
}
|
||||||
|
|
||||||
func (failingDB) GetIPInfoContext(
|
func (failingDB) GetAS(_ context.Context, _ int) (*database.ASN, bool, error) {
|
||||||
_ context.Context, _ string,
|
return nil, false, errDiskIO
|
||||||
) (*database.IPInfo, error) {
|
|
||||||
return nil, errDiskIO
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (failingDB) GetASDetailsContext(
|
func (failingDB) GetPrefixFirstSeen(
|
||||||
_ context.Context, _ int,
|
_ context.Context, _ netip.Prefix,
|
||||||
) (*database.ASN, []database.LiveRoute, error) {
|
) (time.Time, error) {
|
||||||
return nil, nil, errDiskIO
|
return time.Time{}, errDiskIO
|
||||||
}
|
}
|
||||||
|
|
||||||
func (failingDB) GetPrefixDetailsContext(
|
// TestJSONRoutesDoNotSendErrorDetails sends a request to each JSON route that
|
||||||
_ context.Context, _ string,
|
// reads the database, and to the health check, with a store whose reads fail
|
||||||
) ([]database.LiveRoute, map[int]database.ASN, error) {
|
// with the driver's error text. It also sends a prefix that does not parse.
|
||||||
return nil, nil, errDiskIO
|
// No answer may carry the driver's or the prefix parser's text. Before the fix
|
||||||
}
|
// the answers sent it to the client
|
||||||
|
// (https://git.eeqj.de/sneak/routewatch/issues/108).
|
||||||
// TestJSONRoutesDoNotSendErrorDetails sends a request to each JSON route and
|
|
||||||
// the health check with a store whose reads fail with the driver's error text,
|
|
||||||
// and a prefix that does not parse, and checks that no answer carries the
|
|
||||||
// driver's or the prefix parser's text. Before the fix the answers sent it to
|
|
||||||
// the client (https://git.eeqj.de/sneak/routewatch/issues/108).
|
|
||||||
func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
|
func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := New(failingDB{}, streamer.New(logger.New(), metrics.New()), logger.New(),
|
s := New(failingDB{}, newTestTable(), streamer.New(logger.New(), metrics.New()),
|
||||||
&config.Config{})
|
logger.New(), &config.Config{})
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
path string
|
path string
|
||||||
@@ -987,8 +968,7 @@ func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
|
|||||||
{statusJSONPath, http.StatusInternalServerError},
|
{statusJSONPath, http.StatusInternalServerError},
|
||||||
{statsPath, http.StatusInternalServerError},
|
{statsPath, http.StatusInternalServerError},
|
||||||
{"/api/v1/as/64500", http.StatusInternalServerError},
|
{"/api/v1/as/64500", http.StatusInternalServerError},
|
||||||
{"/api/v1/prefix/192.0.2.0/24", http.StatusInternalServerError},
|
{"/api/v1/ip/" + addrInTestPrefixV4, http.StatusInternalServerError},
|
||||||
{"/api/v1/ip/192.0.2.1", http.StatusInternalServerError},
|
|
||||||
{"/api/v1/prefix/foo/24", http.StatusBadRequest},
|
{"/api/v1/prefix/foo/24", http.StatusBadRequest},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1011,45 +991,47 @@ func TestJSONRoutesDoNotSendErrorDetails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// The addresses failingIPLookupDB fails the read of and finds a route for.
|
// The address whose read failingIPLookupDB fails, in testPrefixV4, and one it
|
||||||
|
// reads, in testPrefixV6.
|
||||||
const (
|
const (
|
||||||
failingLookupIP = "192.0.2.1"
|
failingLookupIP = addrInTestPrefixV4
|
||||||
routedLookupIP = "192.0.2.2"
|
routedLookupIP = addrInTestPrefixV6
|
||||||
)
|
)
|
||||||
|
|
||||||
// failingIPLookupDB embeds database.Store (left nil) and answers the IP lookup
|
// failingIPLookupDB embeds database.Store (left nil). It fails the read of
|
||||||
// with errDiskIO for failingLookupIP, a route for routedLookupIP, and no route
|
// testPrefixV4's first-seen time with errDiskIO, and answers the IP lookup's
|
||||||
// for any other address. Every other Store method is unused and would panic if
|
// other reads with nothing stored. Every other Store method is unused and
|
||||||
// called.
|
// would panic if called.
|
||||||
type failingIPLookupDB struct {
|
type failingIPLookupDB struct {
|
||||||
database.Store
|
database.Store
|
||||||
}
|
}
|
||||||
|
|
||||||
func (failingIPLookupDB) GetIPInfoContext(
|
func (failingIPLookupDB) GetPrefixFirstSeen(
|
||||||
_ context.Context, ip string,
|
_ context.Context, prefix netip.Prefix,
|
||||||
) (*database.IPInfo, error) {
|
) (time.Time, error) {
|
||||||
switch ip {
|
if prefix == netip.MustParsePrefix(testPrefixV4) {
|
||||||
case failingLookupIP:
|
return time.Time{}, errDiskIO
|
||||||
return nil, errDiskIO
|
|
||||||
case routedLookupIP:
|
|
||||||
return &database.IPInfo{IP: ip}, nil
|
|
||||||
default:
|
|
||||||
return nil, database.ErrNoRoute
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return time.Time{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (failingIPLookupDB) GetASWHOIS(_ context.Context, asn int) (*database.ASN, error) {
|
||||||
|
return &database.ASN{ASN: asn}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestIPLookupAnswers500WhenDatabaseReadFails sends IP lookups through the
|
// TestIPLookupAnswers500WhenDatabaseReadFails sends IP lookups through the
|
||||||
// server's router with a store whose read of one address fails with the
|
// server's router with a store whose read for one address fails with the
|
||||||
// driver's error. A lookup that asks for that address answers 500 with the
|
// driver's error. A lookup that asks for that address answers 500 with the
|
||||||
// JSON error body, as the AS and prefix JSON routes do; an address with no
|
// JSON error body, as the AS JSON route does; an address with no route answers
|
||||||
// route answers 200. Before the fix the failed read was reported as if the
|
// 200. Before the fix the failed read was reported as if the address had no
|
||||||
// address had no route: 404 for that address alone, and 200 beside a routed
|
// route: 404 for that address alone, and 200 beside a routed address
|
||||||
// address (https://git.eeqj.de/sneak/routewatch/issues/109).
|
// (https://git.eeqj.de/sneak/routewatch/issues/109).
|
||||||
func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
|
func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := New(failingIPLookupDB{}, streamer.New(logger.New(), metrics.New()),
|
s := New(failingIPLookupDB{}, newTestTable(),
|
||||||
logger.New(), &config.Config{})
|
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
s.resolver = &fakeResolver{}
|
s.resolver = &fakeResolver{}
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
@@ -1122,30 +1104,37 @@ func (r *fakeResolver) LookupAddr(ctx context.Context, _ string) ([]string, erro
|
|||||||
return nil, errNoSuchHost
|
return nil, errNoSuchHost
|
||||||
}
|
}
|
||||||
|
|
||||||
// countingIPLookupDB embeds database.Store (left nil) and counts its IP
|
// countingIPLookupDB embeds database.Store (left nil) and answers the IP
|
||||||
// lookups, answering each that the address has no route. Every other Store
|
// lookup's reads with nothing stored. It counts the lookups, as the reads of a
|
||||||
// method is unused and would panic if called.
|
// prefix's first-seen time. Every other Store method is unused and would panic
|
||||||
|
// if called.
|
||||||
type countingIPLookupDB struct {
|
type countingIPLookupDB struct {
|
||||||
database.Store
|
database.Store
|
||||||
|
|
||||||
lookups atomic.Int64
|
lookups atomic.Int64
|
||||||
}
|
}
|
||||||
|
|
||||||
func (d *countingIPLookupDB) GetIPInfoContext(
|
func (d *countingIPLookupDB) GetPrefixFirstSeen(
|
||||||
_ context.Context, _ string,
|
_ context.Context, _ netip.Prefix,
|
||||||
) (*database.IPInfo, error) {
|
) (time.Time, error) {
|
||||||
d.lookups.Add(1)
|
d.lookups.Add(1)
|
||||||
|
|
||||||
return nil, database.ErrNoRoute
|
return time.Time{}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *countingIPLookupDB) GetASWHOIS(
|
||||||
|
_ context.Context, asn int,
|
||||||
|
) (*database.ASN, error) {
|
||||||
|
return &database.ASN{ASN: asn}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestIPLookupRefusesTooManyAddresses sends IP lookups of maxLookupTargets
|
// TestIPLookupRefusesTooManyAddresses sends IP lookups of maxLookupTargets
|
||||||
// addresses and of one more, listed or as a host name that resolves to them,
|
// addresses in testPrefixV4 and of one more, listed or as a host name that
|
||||||
// and a lookup listing maxLookupTargets+1 host names. The longer lookups answer
|
// resolves to them, and a lookup listing maxLookupTargets+1 host names. The
|
||||||
// 400 without reading the database, and the list of host names without
|
// longer lookups answer 400 without reading the database, and the list of host
|
||||||
// resolving any of them; the others read the database once per address. Before
|
// names without resolving any of them; the others read the database for each
|
||||||
// the fix every address was read and every name resolved, however many there
|
// address. Before the fix every address was read and every name resolved,
|
||||||
// were (https://git.eeqj.de/sneak/routewatch/issues/110).
|
// however many there were (https://git.eeqj.de/sneak/routewatch/issues/110).
|
||||||
func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
|
func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1180,8 +1169,8 @@ func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
|
|||||||
|
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
db := &countingIPLookupDB{}
|
db := &countingIPLookupDB{}
|
||||||
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(),
|
s := New(db, newTestTable(), streamer.New(logger.New(), metrics.New()),
|
||||||
&config.Config{})
|
logger.New(), &config.Config{})
|
||||||
resolver := &fakeResolver{hosts: hosts}
|
resolver := &fakeResolver{hosts: hosts}
|
||||||
s.resolver = resolver
|
s.resolver = resolver
|
||||||
|
|
||||||
@@ -1901,16 +1890,11 @@ func TestNoOriginHasNoASAnswers(t *testing.T) {
|
|||||||
const noOriginPrefix = "203.0.113.0/24"
|
const noOriginPrefix = "203.0.113.0/24"
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
|
s.table.Announce(netip.MustParsePrefix(noOriginPrefix), ristypes.NoOrigin,
|
||||||
err := s.db.UpsertLiveRouteBatch([]*database.LiveRoute{{
|
routetable.Route{
|
||||||
Prefix: noOriginPrefix, MaskLength: 24, IPVersion: 4,
|
Peer: testPeerV4, ASPath: []int{64500, 64501, 64502},
|
||||||
OriginASN: ristypes.NoOrigin, PeerIP: testPeerV4,
|
Time: testRouteTime(0),
|
||||||
ASPath: []int{64500, 64501, 64502}, NextHop: testPeerV4,
|
})
|
||||||
LastUpdated: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC),
|
|
||||||
}})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
asn := strconv.Itoa(ristypes.NoOrigin)
|
asn := strconv.Itoa(ristypes.NoOrigin)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,368 @@
|
|||||||
|
package server
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"slices"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The origin ASes in the routing table that newTwoOriginTable fills.
|
||||||
|
const (
|
||||||
|
mostPeersASN = 64500
|
||||||
|
fewerPeersASN = 64501
|
||||||
|
v6OriginASN = 64502
|
||||||
|
)
|
||||||
|
|
||||||
|
// noKeptRoutePrefix is the prefix of newTwoOriginTable whose origin keeps no
|
||||||
|
// route.
|
||||||
|
const noKeptRoutePrefix = "198.51.100.0/24"
|
||||||
|
|
||||||
|
// routeTimeFormat is how the pages write a route's time.
|
||||||
|
const routeTimeFormat = "2006-01-02 15:04:05"
|
||||||
|
|
||||||
|
// newTwoOriginTable returns a routing table that holds:
|
||||||
|
// - testPrefixV4 with two origin ASes. Peers 1, 2 and 3 carry it with
|
||||||
|
// mostPeersASN, which keeps the routes of peers 3 and 2. Peer 4 carries
|
||||||
|
// it with fewerPeersASN, in the prefix's newest route.
|
||||||
|
// - noKeptRoutePrefix with mostPeersASN. Only peer 7 carries it, and its
|
||||||
|
// route is not kept: peers 5 and 6, whose routes were, withdrew it.
|
||||||
|
// - testPrefixV6 with v6OriginASN.
|
||||||
|
func newTwoOriginTable() *routetable.Table {
|
||||||
|
table := emptyTable()
|
||||||
|
|
||||||
|
for n := 1; n <= 3; n++ {
|
||||||
|
announce(table, testPrefixV4, mostPeersASN, testPeer(n), testRouteTime(n))
|
||||||
|
}
|
||||||
|
|
||||||
|
announce(table, testPrefixV4, fewerPeersASN, testPeer(4), testRouteTime(5))
|
||||||
|
|
||||||
|
announce(table, noKeptRoutePrefix, mostPeersASN, testPeer(5), testRouteTime(4))
|
||||||
|
announce(table, noKeptRoutePrefix, mostPeersASN, testPeer(6), testRouteTime(6))
|
||||||
|
announce(table, noKeptRoutePrefix, mostPeersASN, testPeer(7), testRouteTime(0))
|
||||||
|
withdraw(table, noKeptRoutePrefix, testPeer(5))
|
||||||
|
withdraw(table, noKeptRoutePrefix, testPeer(6))
|
||||||
|
|
||||||
|
announce(table, testPrefixV6, v6OriginASN, testPeerV6, testRouteTime(0))
|
||||||
|
|
||||||
|
return table
|
||||||
|
}
|
||||||
|
|
||||||
|
// get sends a GET for path through s's router and fails the test unless it
|
||||||
|
// answers 200.
|
||||||
|
func get(t *testing.T, s *Server, path string) *httptest.ResponseRecorder {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, path, nil))
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("%s: status %d, want %d; body %s",
|
||||||
|
path, rec.Code, http.StatusOK, rec.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
return rec
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestIPLookupGivesTheOriginMostPeersCarry looks up an address in a prefix
|
||||||
|
// with two origin ASes, through each IP lookup route. The answer names the
|
||||||
|
// origin AS that more peers carry, counts every peer, and gives the time of
|
||||||
|
// the prefix's newest kept route. The prefix's first-seen time and the AS's
|
||||||
|
// WHOIS details come from the database.
|
||||||
|
func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTable(t, newTwoOriginTable())
|
||||||
|
s.resolver = &fakeResolver{}
|
||||||
|
|
||||||
|
firstSeen := testRouteTime(-1)
|
||||||
|
|
||||||
|
err := s.db.UpdatePrefixesBatch(map[string]time.Time{testPrefixV4: firstSeen})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpdatePrefixesBatch: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = s.db.GetOrCreateASN(mostPeersASN, firstSeen)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetOrCreateASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
whois := &database.ASNWHOISUpdate{
|
||||||
|
ASN: mostPeersASN, ASName: "TEST-AS", CountryCode: "DE",
|
||||||
|
}
|
||||||
|
|
||||||
|
err = s.db.UpdateASNWHOIS(t.Context(), whois)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpdateASNWHOIS: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ownAddressPath looks up the request's own address, which httptest
|
||||||
|
// gives as addrInTestPrefixV4.
|
||||||
|
for _, path := range []string{
|
||||||
|
"/api/v1/ip/" + addrInTestPrefixV4, "/ip/" + addrInTestPrefixV4,
|
||||||
|
ownAddressPath,
|
||||||
|
} {
|
||||||
|
var body struct {
|
||||||
|
Data IPLookupResponse `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
|
||||||
|
if err != nil || len(body.Data.Results) != 1 ||
|
||||||
|
body.Data.Results[0].IPInfo == nil {
|
||||||
|
t.Fatalf("%s: answer %+v (%v), want one result with a route",
|
||||||
|
path, body.Data, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got := body.Data.Results[0].IPInfo
|
||||||
|
if got.Netblock != testPrefixV4 || got.ASN != mostPeersASN ||
|
||||||
|
got.NumPeers != 4 || !got.LastSeen.Equal(testRouteTime(5)) ||
|
||||||
|
!got.FirstSeen.Equal(firstSeen) || got.ASName != whois.ASName ||
|
||||||
|
got.CountryCode != whois.CountryCode {
|
||||||
|
t.Errorf("%s: route %+v, want %s from AS%d, 4 peers, last seen %v, "+
|
||||||
|
"first seen %v, %s in %s", path, *got, testPrefixV4,
|
||||||
|
mostPeersASN, testRouteTime(5), firstSeen, whois.ASName,
|
||||||
|
whois.CountryCode)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// prefixRoute is one route of the prefix API's answer. A field the answer
|
||||||
|
// leaves out stays nil.
|
||||||
|
//
|
||||||
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
|
type prefixRoute struct {
|
||||||
|
OriginASN int `json:"origin_asn"`
|
||||||
|
PeerIP string `json:"peer_ip"`
|
||||||
|
ASPath []int `json:"as_path"`
|
||||||
|
NextHop *string `json:"next_hop"`
|
||||||
|
LastUpdated *time.Time `json:"last_updated"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestPrefixAnswersListEveryPeer asks the prefix API and page for a prefix
|
||||||
|
// with two origin ASes, and for one whose origin keeps no route. Each answer
|
||||||
|
// lists every peer that carries the prefix, by origin AS. Only the kept routes
|
||||||
|
// have an AS path, a next hop and a time, newest first. The other peers follow
|
||||||
|
// without them.
|
||||||
|
func TestPrefixAnswersListEveryPeer(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTable(t, newTwoOriginTable())
|
||||||
|
|
||||||
|
// keptAt[peer] is the time of the peer's kept route; a peer not in it is
|
||||||
|
// listed without a route.
|
||||||
|
keptAt := map[string]time.Time{
|
||||||
|
testPeer(3): testRouteTime(3),
|
||||||
|
testPeer(2): testRouteTime(2),
|
||||||
|
testPeer(4): testRouteTime(5),
|
||||||
|
}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
prefix string
|
||||||
|
wantPeers []string
|
||||||
|
wantOrigins []int
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
testPrefixV4,
|
||||||
|
[]string{testPeer(3), testPeer(2), testPeer(1), testPeer(4)},
|
||||||
|
[]int{mostPeersASN, mostPeersASN, mostPeersASN, fewerPeersASN},
|
||||||
|
},
|
||||||
|
{noKeptRoutePrefix, []string{testPeer(7)}, []int{mostPeersASN}},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
|
var body struct {
|
||||||
|
Data struct {
|
||||||
|
Routes []prefixRoute `json:"routes"`
|
||||||
|
PeerCount int `json:"peer_count"`
|
||||||
|
OriginCount int `json:"origin_count"`
|
||||||
|
} `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
path := "/api/v1/prefix/" + tt.prefix
|
||||||
|
|
||||||
|
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: decoding the answer: %v", path, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
wantOriginCount := len(slices.Compact(slices.Clone(tt.wantOrigins)))
|
||||||
|
if body.Data.PeerCount != len(tt.wantPeers) ||
|
||||||
|
body.Data.OriginCount != wantOriginCount {
|
||||||
|
t.Errorf("%s: %d peers and %d origins, want %d and %d", path,
|
||||||
|
body.Data.PeerCount, body.Data.OriginCount, len(tt.wantPeers),
|
||||||
|
wantOriginCount)
|
||||||
|
}
|
||||||
|
|
||||||
|
checkPrefixRoutes(t, path, body.Data.Routes, tt.wantPeers, tt.wantOrigins,
|
||||||
|
keptAt)
|
||||||
|
checkPrefixPage(t, s, tt.prefix, tt.wantPeers, keptAt)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// checkPrefixRoutes checks that routes come from wantPeers, in that order,
|
||||||
|
// with wantOrigins as their origin ASes. The route of a peer in keptAt must
|
||||||
|
// have an AS path, the peer as next hop, and the time keptAt gives. Any other
|
||||||
|
// route must have none of the three.
|
||||||
|
func checkPrefixRoutes(
|
||||||
|
t *testing.T, path string, routes []prefixRoute, wantPeers []string,
|
||||||
|
wantOrigins []int, keptAt map[string]time.Time,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if len(routes) != len(wantPeers) {
|
||||||
|
t.Fatalf("%s: routes %+v, want one each from %v", path, routes, wantPeers)
|
||||||
|
}
|
||||||
|
|
||||||
|
for i, route := range routes {
|
||||||
|
if route.PeerIP != wantPeers[i] || route.OriginASN != wantOrigins[i] {
|
||||||
|
t.Errorf("%s: route %d from %s with AS%d, want from %s with AS%d",
|
||||||
|
path, i, route.PeerIP, route.OriginASN, wantPeers[i], wantOrigins[i])
|
||||||
|
}
|
||||||
|
|
||||||
|
at, kept := keptAt[route.PeerIP]
|
||||||
|
if !kept {
|
||||||
|
if route.ASPath != nil || route.NextHop != nil || route.LastUpdated != nil {
|
||||||
|
t.Errorf("%s: route from %s is not kept but has %+v",
|
||||||
|
path, route.PeerIP, route)
|
||||||
|
}
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if route.ASPath == nil || route.NextHop == nil ||
|
||||||
|
*route.NextHop != route.PeerIP || route.LastUpdated == nil ||
|
||||||
|
!route.LastUpdated.Equal(at) {
|
||||||
|
t.Errorf("%s: kept route from %s is %+v, want its AS path, the peer "+
|
||||||
|
"as next hop and the time %v", path, route.PeerIP, route, at)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// checkPrefixPage checks that the page of prefix shows each of wantPeers, the
|
||||||
|
// time of each kept route among them, and no time that was never set.
|
||||||
|
func checkPrefixPage(
|
||||||
|
t *testing.T, s *Server, prefix string, wantPeers []string,
|
||||||
|
keptAt map[string]time.Time,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
path := "/prefix/" + prefix
|
||||||
|
page := get(t, s, path).Body.String()
|
||||||
|
|
||||||
|
for _, peer := range wantPeers {
|
||||||
|
want := `<td class="peer-ip">` + peer + `</td>`
|
||||||
|
if !strings.Contains(page, want) {
|
||||||
|
t.Errorf("%s does not list the peer %s", path, peer)
|
||||||
|
}
|
||||||
|
|
||||||
|
at, kept := keptAt[peer]
|
||||||
|
if kept && !strings.Contains(page, at.Format(routeTimeFormat)) {
|
||||||
|
t.Errorf("%s does not show the time %v of the route from %s",
|
||||||
|
path, at, peer)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.Contains(page, zeroDate) {
|
||||||
|
t.Errorf("%s shows the date %s", path, zeroDate)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASAnswersGiveEachPrefixItsNewestKeptRoute asks the AS API and page for
|
||||||
|
// mostPeersASN. Its prefixes are testPrefixV4, whose newest kept route with
|
||||||
|
// that origin AS is peer 3's, and noKeptRoutePrefix, which keeps no route. The
|
||||||
|
// first has that route's time. The second is listed without a time.
|
||||||
|
func TestASAnswersGiveEachPrefixItsNewestKeptRoute(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTable(t, newTwoOriginTable())
|
||||||
|
path := "/api/v1/as/" + strconv.Itoa(mostPeersASN)
|
||||||
|
|
||||||
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
|
var body struct {
|
||||||
|
Data struct {
|
||||||
|
IPv4Prefixes []struct {
|
||||||
|
Prefix string `json:"prefix"`
|
||||||
|
LastUpdated *time.Time `json:"last_updated"`
|
||||||
|
} `json:"ipv4_prefixes"`
|
||||||
|
} `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: decoding the answer: %v", path, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
prefixes := body.Data.IPv4Prefixes
|
||||||
|
if len(prefixes) != 2 || prefixes[0].Prefix != testPrefixV4 ||
|
||||||
|
prefixes[0].LastUpdated == nil ||
|
||||||
|
!prefixes[0].LastUpdated.Equal(testRouteTime(3)) ||
|
||||||
|
prefixes[1].Prefix != noKeptRoutePrefix || prefixes[1].LastUpdated != nil {
|
||||||
|
t.Errorf("%s: prefixes %+v, want %s updated %v and %s without a time",
|
||||||
|
path, prefixes, testPrefixV4, testRouteTime(3), noKeptRoutePrefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
path = "/as/" + strconv.Itoa(mostPeersASN)
|
||||||
|
page := get(t, s, path).Body.String()
|
||||||
|
|
||||||
|
for _, want := range []string{
|
||||||
|
testPrefixV4, noKeptRoutePrefix, testRouteTime(3).Format(routeTimeFormat),
|
||||||
|
} {
|
||||||
|
if !strings.Contains(page, want) {
|
||||||
|
t.Errorf("%s does not show %s", path, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.Contains(page, zeroDate) {
|
||||||
|
t.Errorf("%s shows the date %s", path, zeroDate)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestPrefixLengthPagesShowTheOriginMostPeersCarry asks for the IPv4 /24
|
||||||
|
// page and the IPv6 /32 page. The first lists both /24 prefixes with
|
||||||
|
// mostPeersASN, the origin AS more peers carry testPrefixV4 with, and gives
|
||||||
|
// noKeptRoutePrefix no age. The second lists testPrefixV6.
|
||||||
|
func TestPrefixLengthPagesShowTheOriginMostPeersCarry(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTable(t, newTwoOriginTable())
|
||||||
|
|
||||||
|
page := get(t, s, prefixLengthPagePath).Body.String()
|
||||||
|
|
||||||
|
for _, want := range []string{
|
||||||
|
">" + testPrefixV4 + "<", ">" + noKeptRoutePrefix + "<",
|
||||||
|
`href="/as/` + strconv.Itoa(mostPeersASN) + `"`,
|
||||||
|
} {
|
||||||
|
if !strings.Contains(page, want) {
|
||||||
|
t.Errorf("%s does not show %s", prefixLengthPagePath, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.Contains(page, `href="/as/`+strconv.Itoa(fewerPeersASN)+`"`) {
|
||||||
|
t.Errorf("%s links to AS%d, which fewer peers carry",
|
||||||
|
prefixLengthPagePath, fewerPeersASN)
|
||||||
|
}
|
||||||
|
|
||||||
|
const noAge = `<td class="age"></td>`
|
||||||
|
if strings.Count(page, noAge) != 1 {
|
||||||
|
t.Errorf("%s gives %d prefixes no age, want 1 (%s)",
|
||||||
|
prefixLengthPagePath, strings.Count(page, noAge), noKeptRoutePrefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
const v6PagePath = "/prefixlength6/32"
|
||||||
|
|
||||||
|
page = get(t, s, v6PagePath).Body.String()
|
||||||
|
if !strings.Contains(page, ">"+testPrefixV6+"<") ||
|
||||||
|
!strings.Contains(page, `href="/as/`+strconv.Itoa(v6OriginASN)+`"`) {
|
||||||
|
t.Errorf("%s does not show %s with AS%d", v6PagePath, testPrefixV6,
|
||||||
|
v6OriginASN)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,6 +3,10 @@
|
|||||||
// - the AS, prefix and prefix length pages, and the JSON answers behind them
|
// - the AS, prefix and prefix length pages, and the JSON answers behind them
|
||||||
// - the IP lookups
|
// - the IP lookups
|
||||||
// - the AS name search
|
// - the AS name search
|
||||||
|
//
|
||||||
|
// Every route it shows comes from the routing table in memory. AS names,
|
||||||
|
// WHOIS details and the times a prefix and an AS were first seen come from
|
||||||
|
// the database.
|
||||||
package server
|
package server
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -19,6 +23,7 @@ import (
|
|||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
"github.com/go-chi/chi/v5"
|
"github.com/go-chi/chi/v5"
|
||||||
)
|
)
|
||||||
@@ -55,6 +60,7 @@ type dnsResolver interface {
|
|||||||
type Server struct {
|
type Server struct {
|
||||||
router *chi.Mux
|
router *chi.Mux
|
||||||
db database.Store
|
db database.Store
|
||||||
|
table *routetable.Table
|
||||||
streamer *streamer.Streamer
|
streamer *streamer.Streamer
|
||||||
logger *logger.Logger
|
logger *logger.Logger
|
||||||
port int
|
port int
|
||||||
@@ -71,15 +77,18 @@ type Server struct {
|
|||||||
resolver dnsResolver
|
resolver dnsResolver
|
||||||
}
|
}
|
||||||
|
|
||||||
// New creates a new HTTP server
|
// New creates a new HTTP server that reads routes from table, the routing
|
||||||
|
// table that the prefix handler keeps current.
|
||||||
func New(
|
func New(
|
||||||
db database.Store,
|
db database.Store,
|
||||||
|
table *routetable.Table,
|
||||||
streamer *streamer.Streamer,
|
streamer *streamer.Streamer,
|
||||||
logger *logger.Logger,
|
logger *logger.Logger,
|
||||||
cfg *config.Config,
|
cfg *config.Config,
|
||||||
) *Server {
|
) *Server {
|
||||||
s := &Server{
|
s := &Server{
|
||||||
db: db,
|
db: db,
|
||||||
|
table: table,
|
||||||
streamer: streamer,
|
streamer: streamer,
|
||||||
logger: logger,
|
logger: logger,
|
||||||
port: cfg.Port,
|
port: cfg.Port,
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ func TestStartFailsWhenPortIsHeld(t *testing.T) {
|
|||||||
|
|
||||||
port := addr.Port
|
port := addr.Port
|
||||||
log := logger.New()
|
log := logger.New()
|
||||||
s := server.New(nil, streamer.New(log, metrics.New()), log,
|
s := server.New(nil, nil, streamer.New(log, metrics.New()), log,
|
||||||
&config.Config{Port: port})
|
&config.Config{Port: port})
|
||||||
|
|
||||||
defer func() { _ = s.Stop(context.Background()) }()
|
defer func() { _ = s.Stop(context.Background()) }()
|
||||||
|
|||||||
@@ -281,8 +281,8 @@
|
|||||||
<tr>
|
<tr>
|
||||||
<td><a href="{{.Prefix | prefixURL}}" class="prefix-link">{{.Prefix}}</a></td>
|
<td><a href="{{.Prefix | prefixURL}}" class="prefix-link">{{.Prefix}}</a></td>
|
||||||
<td>/{{.MaskLength}}</td>
|
<td>/{{.MaskLength}}</td>
|
||||||
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
|
<td>{{if not .LastUpdated.IsZero}}{{.LastUpdated.Format "2006-01-02 15:04:05"}}{{end}}</td>
|
||||||
<td class="age">{{.LastUpdated | timeSince}}</td>
|
<td class="age">{{if not .LastUpdated.IsZero}}{{.LastUpdated | timeSince}}{{end}}</td>
|
||||||
</tr>
|
</tr>
|
||||||
{{end}}
|
{{end}}
|
||||||
</tbody>
|
</tbody>
|
||||||
@@ -310,8 +310,8 @@
|
|||||||
<tr>
|
<tr>
|
||||||
<td><a href="{{.Prefix | prefixURL}}" class="prefix-link">{{.Prefix}}</a></td>
|
<td><a href="{{.Prefix | prefixURL}}" class="prefix-link">{{.Prefix}}</a></td>
|
||||||
<td>/{{.MaskLength}}</td>
|
<td>/{{.MaskLength}}</td>
|
||||||
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
|
<td>{{if not .LastUpdated.IsZero}}{{.LastUpdated.Format "2006-01-02 15:04:05"}}{{end}}</td>
|
||||||
<td class="age">{{.LastUpdated | timeSince}}</td>
|
<td class="age">{{if not .LastUpdated.IsZero}}{{.LastUpdated | timeSince}}{{end}}</td>
|
||||||
</tr>
|
</tr>
|
||||||
{{end}}
|
{{end}}
|
||||||
</tbody>
|
</tbody>
|
||||||
|
|||||||
@@ -329,8 +329,8 @@
|
|||||||
<td class="peer-ip">{{.PeerIP}}</td>
|
<td class="peer-ip">{{.PeerIP}}</td>
|
||||||
<td class="as-path">{{range $i, $as := .ASPathWithHandle}}{{if $i}} → {{end}}<a href="/as/{{$as.ASN}}" class="as-link">{{if $as.Handle}}{{$as.Handle}}{{else}}AS{{$as.ASN}}{{end}}</a>{{end}}</td>
|
<td class="as-path">{{range $i, $as := .ASPathWithHandle}}{{if $i}} → {{end}}<a href="/as/{{$as.ASN}}" class="as-link">{{if $as.Handle}}{{$as.Handle}}{{else}}AS{{$as.ASN}}{{end}}</a>{{end}}</td>
|
||||||
<td class="peer-ip">{{.NextHop}}</td>
|
<td class="peer-ip">{{.NextHop}}</td>
|
||||||
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
|
<td>{{if not .LastUpdated.IsZero}}{{.LastUpdated.Format "2006-01-02 15:04:05"}}{{end}}</td>
|
||||||
<td class="age">{{.LastUpdated | timeSince}}</td>
|
<td class="age">{{if not .LastUpdated.IsZero}}{{.LastUpdated | timeSince}}{{end}}</td>
|
||||||
</tr>
|
</tr>
|
||||||
{{end}}
|
{{end}}
|
||||||
</tbody>
|
</tbody>
|
||||||
|
|||||||
@@ -318,6 +318,10 @@
|
|||||||
<span class="metric-label">IPv6 Routes</span>
|
<span class="metric-label">IPv6 Routes</span>
|
||||||
<span class="metric-value" id="ipv6_routes">-</span>
|
<span class="metric-value" id="ipv6_routes">-</span>
|
||||||
</div>
|
</div>
|
||||||
|
<div class="metric">
|
||||||
|
<span class="metric-label">Origins Without a Kept Route</span>
|
||||||
|
<span class="metric-value" id="origins_without_kept_route">-</span>
|
||||||
|
</div>
|
||||||
<div class="metric">
|
<div class="metric">
|
||||||
<span class="metric-label">IPv4 Updates/sec</span>
|
<span class="metric-label">IPv4 Updates/sec</span>
|
||||||
<span class="metric-value" id="ipv4_updates_per_sec">-</span>
|
<span class="metric-value" id="ipv4_updates_per_sec">-</span>
|
||||||
@@ -571,6 +575,7 @@
|
|||||||
document.getElementById('live_routes').textContent = '-';
|
document.getElementById('live_routes').textContent = '-';
|
||||||
document.getElementById('ipv4_routes').textContent = '-';
|
document.getElementById('ipv4_routes').textContent = '-';
|
||||||
document.getElementById('ipv6_routes').textContent = '-';
|
document.getElementById('ipv6_routes').textContent = '-';
|
||||||
|
document.getElementById('origins_without_kept_route').textContent = '-';
|
||||||
document.getElementById('ipv4_updates_per_sec').textContent = '-';
|
document.getElementById('ipv4_updates_per_sec').textContent = '-';
|
||||||
document.getElementById('ipv6_updates_per_sec').textContent = '-';
|
document.getElementById('ipv6_updates_per_sec').textContent = '-';
|
||||||
document.getElementById('oldest_route').textContent = '-';
|
document.getElementById('oldest_route').textContent = '-';
|
||||||
@@ -634,6 +639,7 @@
|
|||||||
document.getElementById('live_routes').textContent = formatNumber(data.live_routes);
|
document.getElementById('live_routes').textContent = formatNumber(data.live_routes);
|
||||||
document.getElementById('ipv4_routes').textContent = formatNumber(data.ipv4_routes);
|
document.getElementById('ipv4_routes').textContent = formatNumber(data.ipv4_routes);
|
||||||
document.getElementById('ipv6_routes').textContent = formatNumber(data.ipv6_routes);
|
document.getElementById('ipv6_routes').textContent = formatNumber(data.ipv6_routes);
|
||||||
|
document.getElementById('origins_without_kept_route').textContent = formatNumber(data.origins_without_kept_route);
|
||||||
document.getElementById('ipv4_updates_per_sec').textContent = formatNumber(Math.round(data.ipv4_updates_per_sec));
|
document.getElementById('ipv4_updates_per_sec').textContent = formatNumber(Math.round(data.ipv4_updates_per_sec));
|
||||||
document.getElementById('ipv6_updates_per_sec').textContent = formatNumber(Math.round(data.ipv6_updates_per_sec));
|
document.getElementById('ipv6_updates_per_sec').textContent = formatNumber(Math.round(data.ipv6_updates_per_sec));
|
||||||
document.getElementById('oldest_route').textContent = formatRelativeTime(data.oldest_route);
|
document.getElementById('oldest_route').textContent = formatRelativeTime(data.oldest_route);
|
||||||
|
|||||||
Reference in New Issue
Block a user