Compare commits
15
Commits
46c1c89568
...
next
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
203a4e13a5 | ||
|
|
2375480e2e | ||
|
|
a5bff4e13c | ||
|
|
73f53c8220 | ||
|
|
cd1b9b94a4 | ||
|
|
023c3f9b41 | ||
|
|
57469b6370 | ||
|
|
6d55d39737 | ||
|
|
86e1f20e11 | ||
|
|
a1de67ebe1 | ||
|
|
355cd37496 | ||
|
|
087b8de9ae | ||
|
|
99c7efc156 | ||
|
|
71ee2aeda7 | ||
|
|
8d16bf9ea6 |
@@ -52,7 +52,6 @@ clean:
|
|||||||
run: build
|
run: build
|
||||||
DEBUG=routewatch ./bin/routewatch 2>&1 | tee log.txt
|
DEBUG=routewatch ./bin/routewatch 2>&1 | tee log.txt
|
||||||
|
|
||||||
|
# Rebuild the AS list built into RouteWatch from as.csv (needs the network).
|
||||||
asupdate:
|
asupdate:
|
||||||
@echo "Updating AS info data..."
|
@script/asupdate
|
||||||
@go run cmd/asinfo-gen/main.go | gzip > pkg/asinfo/asdata.json.gz.tmp && \
|
|
||||||
mv pkg/asinfo/asdata.json.gz.tmp pkg/asinfo/asdata.json.gz
|
|
||||||
|
|||||||
@@ -5,6 +5,22 @@ table in real time: it streams BGP UPDATE messages from the RIPE RIS Live
|
|||||||
service, maintains a live routing table in SQLite, and provides HTTP APIs for
|
service, maintains a live routing table in SQLite, and provides HTTP APIs for
|
||||||
querying routing information.
|
querying routing information.
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
RIPE RIS Live streams the BGP updates that RIPE's route collectors receive, as
|
||||||
|
they happen, from the moment a client connects. It reports changes, not the
|
||||||
|
routing table they add up to, so it cannot say which AS an IP address is routed
|
||||||
|
to now or which routes a prefix has. RouteWatch applies the feed's updates to
|
||||||
|
live routing tables in its own SQLite database and answers IP-to-AS lookups and
|
||||||
|
prefix pages from them.
|
||||||
|
|
||||||
|
The feed sends no table when a client connects, so these tables hold only the
|
||||||
|
routes the feed has announced since RouteWatch first started: an IP-to-AS lookup
|
||||||
|
finds nothing for an address whose routes have not been announced since then.
|
||||||
|
The tables also miss the updates RouteWatch drops when it falls behind under
|
||||||
|
load and those sent while it is stopped, so a lookup or a prefix page can show a
|
||||||
|
route the feed has withdrawn.
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Real-time streaming of BGP updates from RIPE RIS Live
|
- Real-time streaming of BGP updates from RIPE RIS Live
|
||||||
@@ -40,15 +56,17 @@ Every route answers `HEAD` as it answers `GET`, without the body.
|
|||||||
|
|
||||||
### Web Interface
|
### Web Interface
|
||||||
|
|
||||||
- `GET /` - Home page (HTML): headline figures, and lookups by AS number and by
|
- `GET /` - Home page (HTML): headline figures, lookups by AS number and by IP
|
||||||
IP address
|
address, and a search of AS names
|
||||||
- `GET /status` - HTML status dashboard, which reads `/api/v1/stats`
|
- `GET /status` - HTML status dashboard, which reads `/api/v1/stats`
|
||||||
- `GET /status.json` - JSON statistics: feed, memory, database and route figures
|
- `GET /status.json` - JSON statistics: feed, memory, database and route figures
|
||||||
- `GET /.well-known/healthcheck.json` - Health check (JSON): 200 when the
|
- `GET /.well-known/healthcheck.json` - Health check (JSON): 200 when the
|
||||||
database answers and the RIS Live feed is connected, otherwise 503;
|
database answers and the RIS Live feed is connected, otherwise 503;
|
||||||
`data.status` is `ok` or `error` and `data.checks` says which check failed
|
`data.status` is `ok` or `error` and `data.checks` says which check failed
|
||||||
- `GET /as/{asn}` - AS detail page (HTML): the prefixes the AS originates and
|
- `GET /as/{asn}` - AS detail page (HTML): the prefixes the AS originates and
|
||||||
its peers
|
its peers. When the database has no record of the AS, its handle and
|
||||||
|
description come from the AS list built into RouteWatch. The page then shows
|
||||||
|
no first-seen date.
|
||||||
- `GET /prefix/{prefix}/{length}` - Prefix detail page (HTML), such as
|
- `GET /prefix/{prefix}/{length}` - Prefix detail page (HTML), such as
|
||||||
`/prefix/192.0.2.0/24`: every live route for the prefix, one per peer, and its
|
`/prefix/192.0.2.0/24`: every live route for the prefix, one per peer, and its
|
||||||
origin ASes
|
origin ASes
|
||||||
@@ -63,7 +81,9 @@ Every route answers `HEAD` as it answers `GET`, without the body.
|
|||||||
### API v1
|
### API v1
|
||||||
|
|
||||||
- `GET /api/v1/stats` - The `/status.json` figures plus connection, garbage
|
- `GET /api/v1/stats` - The `/status.json` figures plus connection, garbage
|
||||||
collector, stream and handler queue statistics
|
collector, stream and handler queue statistics. `stream.route_churn_per_min`
|
||||||
|
is the number of prefix announcements and withdrawals, from all peers, in the
|
||||||
|
last full clock minute.
|
||||||
- `GET /api/v1/ip/{ip}` - Looks up each address: its most specific live route
|
- `GET /api/v1/ip/{ip}` - Looks up each address: its most specific live route
|
||||||
(`netblock`), that route's origin AS with its WHOIS details, and the address's
|
(`netblock`), that route's origin AS with its WHOIS details, and the address's
|
||||||
PTR names. `{ip}` is an IP address, a host name, or a comma-separated list of
|
PTR names. `{ip}` is an IP address, a host name, or a comma-separated list of
|
||||||
@@ -77,9 +97,23 @@ Every route answers `HEAD` as it answers `GET`, without the body.
|
|||||||
and 404 when none of the addresses has a live route. A PTR lookup gets 2
|
and 404 when none of the addresses has a live route. A PTR lookup gets 2
|
||||||
seconds; an address whose lookup takes longer is answered without PTR names.
|
seconds; an address whose lookup takes longer is answered without PTR names.
|
||||||
- `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6
|
- `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6
|
||||||
apart, each with when its newest route was updated
|
apart, each with when its newest route was updated. When the database has no
|
||||||
|
record of the AS, its handle and description come from the AS list built into
|
||||||
|
RouteWatch. The answer then has no `first_seen` or `last_seen`.
|
||||||
- `GET /api/v1/prefix/{prefix}/{length}` - Every live route for the prefix, one
|
- `GET /api/v1/prefix/{prefix}/{length}` - Every live route for the prefix, one
|
||||||
per peer, and the same routes grouped by origin AS in `origins`
|
per peer, and the same routes grouped by origin AS in `origins`
|
||||||
|
- `GET /api/v1/search/as?q={text}` - Finds the ASes whose handle or description
|
||||||
|
contains `{text}`, ignoring case. `data` holds:
|
||||||
|
- `query`: `{text}` without its leading and trailing spaces
|
||||||
|
- `total`: the number of ASes that match
|
||||||
|
- `results`: the first 50 of them in AS number order, each with `asn`,
|
||||||
|
`handle` and `description`
|
||||||
|
|
||||||
|
The search reads the AS list built into RouteWatch, not the database. The
|
||||||
|
list comes from
|
||||||
|
[ipverse/as-metadata](https://github.com/ipverse/as-metadata). A match can
|
||||||
|
therefore be an AS that the database has no record of. A `q` that is
|
||||||
|
missing, empty or only spaces answers 400.
|
||||||
|
|
||||||
A JSON answer is `{"status": "ok", "data": ...}`, or for an error
|
A JSON answer is `{"status": "ok", "data": ...}`, or for an error
|
||||||
`{"status": "error", "error": {"msg": ..., "code": ...}}`, and carries an
|
`{"status": "error", "error": {"msg": ..., "code": ...}}`, and carries an
|
||||||
@@ -90,13 +124,16 @@ took. The health check is the exception: its 503 answer is also
|
|||||||
A request that runs past its time limit answers 504: the limit is 30 seconds,
|
A request that runs past its time limit answers 504: the limit is 30 seconds,
|
||||||
and 4 seconds for the database reads behind `/status.json` and `/api/v1/stats`.
|
and 4 seconds for the database reads behind `/status.json` and `/api/v1/stats`.
|
||||||
An AS number, prefix or mask length that does not parse answers 400, as does a
|
An AS number, prefix or mask length that does not parse answers 400, as does a
|
||||||
mask length out of range for its IP version. An AS the database does not hold,
|
mask length out of range for its IP version. An AS answers 404 when it is not in
|
||||||
or a prefix with no live route, answers 404. The HTML pages give these errors as
|
the AS list built into RouteWatch and the database has neither a record of it
|
||||||
plain text. Under `/api/v1` and `/ip/`, a path no route matches answers 404 with
|
nor a live route it originates. AS 0 always answers 404, because it means no
|
||||||
the message `Not found`, and `POST`, `PUT`, `PATCH`, `DELETE`, `OPTIONS`,
|
known origin. The AS name search never lists it. A prefix with no live route
|
||||||
`TRACE` or `CONNECT` answers 405 with `Method not allowed`. A 500 answer says
|
answers 404 too. The HTML pages give these errors as plain text. Under `/api/v1`
|
||||||
only `Internal server error`, and a host name that does not resolve only
|
and `/ip/`, a path no route matches answers 404 with the message `Not found`,
|
||||||
`could not resolve host name`; the detail goes to the log.
|
and `POST`, `PUT`, `PATCH`, `DELETE`, `OPTIONS`, `TRACE` or `CONNECT` answers
|
||||||
|
405 with `Method not allowed`. A 500 answer says only `Internal server error`,
|
||||||
|
and a host name that does not resolve only `could not resolve host name`; the
|
||||||
|
detail goes to the log.
|
||||||
|
|
||||||
Every answer carries `Strict-Transport-Security`, `Content-Security-Policy`,
|
Every answer carries `Strict-Transport-Security`, `Content-Security-Policy`,
|
||||||
`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` and
|
`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` and
|
||||||
@@ -449,6 +486,11 @@ provide:
|
|||||||
or `go.sum`) and `go fmt` (writes), then `script/check`
|
or `go.sum`) and `go fmt` (writes), then `script/check`
|
||||||
- `script/install-precommit` — install the git pre-commit hook that runs
|
- `script/install-precommit` — install the git pre-commit hook that runs
|
||||||
`script/precommit`
|
`script/precommit`
|
||||||
|
- `script/asupdate` — rebuild the AS list built into RouteWatch,
|
||||||
|
`pkg/asinfo/asdata.json.gz`, from the `as.csv` file of
|
||||||
|
[ipverse/as-metadata](https://github.com/ipverse/as-metadata). It needs the
|
||||||
|
network. It runs `cmd/asinfo-gen`, which fails when it finds no AS. When
|
||||||
|
`cmd/asinfo-gen` fails, the old list stays and the script exits non-zero.
|
||||||
|
|
||||||
Tests and the linter run only in Docker, as the `test` and `lint` phases of the
|
Tests and the linter run only in Docker, as the `test` and `lint` phases of the
|
||||||
`Dockerfile`; the image cannot be built unless both pass. Every `docker build`
|
`Dockerfile`; the image cannot be built unless both pass. Every `docker build`
|
||||||
|
|||||||
@@ -33,6 +33,61 @@ tested.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-27: on the AS page, a click on the heading of the IPv4 or the IPv6
|
||||||
|
prefix list folds or unfolds the list. Both lists start open (closes #155)
|
||||||
|
- 2026-10-27: the AS page lists its IPv4 and its IPv6 prefixes by mask length,
|
||||||
|
shortest first, and prefixes of one mask length by address. They used to be
|
||||||
|
listed by address (closes #153)
|
||||||
|
- 2026-10-27: on `/` and `/status`, a spinner beside the status turns one frame
|
||||||
|
each time the stats show more messages than the time before. A page that has
|
||||||
|
not seen more messages for three refreshes shows "stale" and the time since
|
||||||
|
then instead, in red (closes #152)
|
||||||
|
- 2026-10-27: the WHOIS details of a Brazilian AS keep their accented letters.
|
||||||
|
LACNIC's answer for such an AS comes from NIC.br, Brazil's registry, with each
|
||||||
|
accented letter replaced. The details now come from NIC.br's own server,
|
||||||
|
`whois.registro.br`. When that query fails, the details from LACNIC's answer
|
||||||
|
are kept and a warning is logged. After NIC.br answers that RouteWatch is over
|
||||||
|
its query limit, RouteWatch stops asking NIC.br for an hour (closes #174)
|
||||||
|
- 2026-10-27: `make asupdate` stops with an error when the download of `as.csv`
|
||||||
|
breaks off. The generator used to retry the failed read forever. A record that
|
||||||
|
does not parse is still logged and skipped (closes #177)
|
||||||
|
- 2026-10-27: `make asupdate` runs the new `script/asupdate`. It replaces the AS
|
||||||
|
list built into RouteWatch only when the generator succeeds and finds at least
|
||||||
|
one AS. A failed download used to replace the list with an empty file. The
|
||||||
|
generator reads `as.csv` from `ipverse/as-metadata`, the source's new name
|
||||||
|
(closes #173)
|
||||||
|
- 2026-10-27: the WHOIS details of an AS at RIPE, APNIC or AFRINIC come only
|
||||||
|
from its `aut-num` object and the organisation object it names. RIPE and APNIC
|
||||||
|
start their answer with the `as-block` object of the AS's block. Its dates
|
||||||
|
were stored as the AS's (closes #168)
|
||||||
|
- 2026-10-27: AS names keep their accented letters. The AS list built into
|
||||||
|
RouteWatch is rebuilt from today's `as.csv`, which added a country code
|
||||||
|
column. That file no longer lists AS 0 or the documentation AS numbers. A
|
||||||
|
WHOIS answer that is not valid UTF-8 is decoded as Latin-1, which RIPE and
|
||||||
|
APNIC send, before it is parsed and stored (closes #165)
|
||||||
|
- 2026-10-27: the Age column on the AS and prefix pages reads in words, such as
|
||||||
|
"5 minutes ago" or "2 weeks ago", instead of Go durations such as "360h0m0s
|
||||||
|
ago". An age under a minute reads in seconds, and one over a month in months
|
||||||
|
or years instead of a date (closes #163)
|
||||||
|
- 2026-10-26: a write to a stored AS, peering or peer keeps the earlier of the
|
||||||
|
two first-seen times and the later of the two last-seen times. An old update
|
||||||
|
from a collector catching up on a backlog no longer moves last seen back
|
||||||
|
(closes #162)
|
||||||
|
- 2026-10-26: the status page shows "Route Churn/min", the number of prefix
|
||||||
|
announcements and withdrawals, from all peers, in the last full clock minute.
|
||||||
|
`/api/v1/stats` reports it as `route_churn_per_min` in `stream`, which
|
||||||
|
replaces the moving average `route_churn_per_sec`. The messages and updates
|
||||||
|
per second on `/` and `/status` are whole numbers (closes #160)
|
||||||
|
- 2026-10-26: the WHOIS fetcher backs off more slowly. A failure makes its wait
|
||||||
|
1.5 times as long, up to one minute instead of five. Its queries to RIPE,
|
||||||
|
APNIC and AFRINIC leave out person and role objects, which RIPE counts against
|
||||||
|
a daily limit per address. A refused query is logged with the RIR's error line
|
||||||
|
(closes #159)
|
||||||
|
- 2026-10-26: the home page's AS name search lists the ASes whose handle or
|
||||||
|
description holds the text, as links to their pages, from the new
|
||||||
|
`/api/v1/search/as`; the search ignores case. When the database has no record
|
||||||
|
of an AS, its page takes the handle and description from the AS list built
|
||||||
|
into RouteWatch and still lists the AS's prefixes (closes #151)
|
||||||
- 2026-10-19: README "HTTP Endpoints" lists every route with what it answers,
|
- 2026-10-19: README "HTTP Endpoints" lists every route with what it answers,
|
||||||
"Database Schema" matches `schema.sql`, and the rest of the README matches the
|
"Database Schema" matches `schema.sql`, and the rest of the README matches the
|
||||||
code; this file brought up to date (closes #72)
|
code; this file brought up to date (closes #72)
|
||||||
|
|||||||
+60
-28
@@ -6,6 +6,7 @@ import (
|
|||||||
"encoding/csv"
|
"encoding/csv"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"log"
|
"log"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -14,7 +15,21 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
const asCsvURL = "https://github.com/ipverse/asn-info/raw/refs/heads/master/as.csv"
|
const asCsvURL = "https://github.com/ipverse/as-metadata/raw/refs/heads/master/as.csv"
|
||||||
|
|
||||||
|
// The first columns of as.csv, the ones the generator reads. It reads past
|
||||||
|
// the columns after them, such as the country code.
|
||||||
|
const (
|
||||||
|
asnColumn = "asn"
|
||||||
|
handleColumn = "handle"
|
||||||
|
descriptionColumn = "description"
|
||||||
|
columnsRead = 3
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
errUnexpectedHeader = errors.New("unexpected CSV header")
|
||||||
|
errNoASes = errors.New("no AS in the CSV")
|
||||||
|
)
|
||||||
|
|
||||||
// ASInfo represents information about an Autonomous System
|
// ASInfo represents information about an Autonomous System
|
||||||
type ASInfo struct {
|
type ASInfo struct {
|
||||||
@@ -45,40 +60,66 @@ func main() {
|
|||||||
log.Fatalf("Failed to fetch CSV: HTTP %d", resp.StatusCode)
|
log.Fatalf("Failed to fetch CSV: HTTP %d", resp.StatusCode)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Parse CSV
|
asInfos, err := parseCSV(resp.Body)
|
||||||
reader := csv.NewReader(resp.Body)
|
|
||||||
|
// Closed here rather than deferred: log.Fatalf exits without running
|
||||||
|
// deferred calls.
|
||||||
|
_ = resp.Body.Close()
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
log.Fatalf("Failed to parse CSV: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Printf("Parsed %d AS records", len(asInfos))
|
||||||
|
|
||||||
|
// Convert to JSON and output to stdout
|
||||||
|
encoder := json.NewEncoder(os.Stdout)
|
||||||
|
encoder.SetIndent("", "\t")
|
||||||
|
|
||||||
|
err = encoder.Encode(asInfos)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatalf("Failed to encode JSON: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseCSV reads as.csv from r and returns its ASes, with the handle and the
|
||||||
|
// description as the file spells them. A header whose first columns are not
|
||||||
|
// the AS number, the handle and the description is an error. A record that
|
||||||
|
// does not parse is logged and skipped. A read that fails, such as on a broken
|
||||||
|
// download, is an error. A file left with no AS is an error.
|
||||||
|
func parseCSV(r io.Reader) ([]ASInfo, error) {
|
||||||
|
reader := csv.NewReader(r)
|
||||||
|
|
||||||
// Read header
|
|
||||||
header, err := reader.Read()
|
header, err := reader.Read()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Fatalf("Failed to read CSV header: %v", err)
|
return nil, fmt.Errorf("read CSV header: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(header) != 3 || header[0] != "asn" || header[1] != "handle" ||
|
if len(header) < columnsRead || header[0] != asnColumn ||
|
||||||
header[2] != "description" {
|
header[1] != handleColumn || header[2] != descriptionColumn {
|
||||||
log.Fatalf("Unexpected CSV header: %v", header)
|
return nil, fmt.Errorf("%w: %v", errUnexpectedHeader, header)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Read all records
|
|
||||||
var asInfos []ASInfo
|
var asInfos []ASInfo
|
||||||
|
|
||||||
for {
|
for {
|
||||||
|
// The reader returns an error for a record with a different number
|
||||||
|
// of fields than the header.
|
||||||
record, err := reader.Read()
|
record, err := reader.Read()
|
||||||
if errors.Is(err, io.EOF) {
|
if errors.Is(err, io.EOF) {
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
|
|
||||||
if err != nil {
|
if _, ok := errors.AsType[*csv.ParseError](err); ok {
|
||||||
log.Printf("Error reading CSV record: %v", err)
|
log.Printf("Error reading CSV record: %v", err)
|
||||||
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
const expectedFields = 3
|
// Any other error comes from the download, such as a broken
|
||||||
if len(record) != expectedFields {
|
// connection. Every later read returns it again.
|
||||||
log.Printf("Skipping invalid record: %v", record)
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read CSV record: %w", err)
|
||||||
continue
|
|
||||||
}
|
}
|
||||||
|
|
||||||
asn, err := strconv.Atoi(record[0])
|
asn, err := strconv.Atoi(record[0])
|
||||||
@@ -95,18 +136,9 @@ func main() {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// Closed here rather than deferred: log.Fatalf exits without running
|
if len(asInfos) == 0 {
|
||||||
// deferred calls.
|
return nil, errNoASes
|
||||||
_ = resp.Body.Close()
|
|
||||||
|
|
||||||
log.Printf("Parsed %d AS records", len(asInfos))
|
|
||||||
|
|
||||||
// Convert to JSON and output to stdout
|
|
||||||
encoder := json.NewEncoder(os.Stdout)
|
|
||||||
encoder.SetIndent("", "\t")
|
|
||||||
|
|
||||||
err = encoder.Encode(asInfos)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatalf("Failed to encode JSON: %v", err)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return asInfos, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,142 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"testing/iotest"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// parseTimeLimit is how long a test waits for parseCSV to return before it
|
||||||
|
// fails.
|
||||||
|
const parseTimeLimit = 2 * time.Second
|
||||||
|
|
||||||
|
// TestParseCSVReadsTodaysLayout parses an as.csv with the country code column
|
||||||
|
// that the source added after the description. It expects each AS with its
|
||||||
|
// handle and description spelled exactly as in the file, accents included.
|
||||||
|
func TestParseCSVReadsTodaysLayout(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const asCSV = `asn,handle,description,country-code
|
||||||
|
1,LVLT-1,Level 3 Parent LLC,US
|
||||||
|
1251,FUNDACAO-AMPARO-PESQUISA,FUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO SÃO PAULO,BR
|
||||||
|
`
|
||||||
|
|
||||||
|
got, err := parseCSV(strings.NewReader(asCSV))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parseCSV: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
want := []ASInfo{
|
||||||
|
{ASN: 1, Handle: "LVLT-1", Description: "Level 3 Parent LLC"},
|
||||||
|
{
|
||||||
|
ASN: 1251,
|
||||||
|
Handle: "FUNDACAO-AMPARO-PESQUISA",
|
||||||
|
Description: "FUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO " +
|
||||||
|
"SÃO PAULO",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(got, want) {
|
||||||
|
t.Errorf("parseCSV = %+v, want %+v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestParseCSVRejectsAnotherHeader parses files whose first columns are not
|
||||||
|
// the AS number, the handle and the description. It expects an error rather
|
||||||
|
// than a list read from the wrong columns.
|
||||||
|
func TestParseCSVRejectsAnotherHeader(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, asCSV := range []string{
|
||||||
|
"asn,handle\n1,LVLT-1\n",
|
||||||
|
"asn,description,handle\n1,Level 3 Parent LLC,LVLT-1\n",
|
||||||
|
} {
|
||||||
|
_, err := parseCSV(strings.NewReader(asCSV))
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("parseCSV(%q): no error", asCSV)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestParseCSVRejectsAFileWithNoASes parses files that hold no AS, one with
|
||||||
|
// only the header and one whose only record has no valid AS number. It
|
||||||
|
// expects errNoASes, so that make asupdate keeps the old list.
|
||||||
|
func TestParseCSVRejectsAFileWithNoASes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, asCSV := range []string{
|
||||||
|
"asn,handle,description,country-code\n",
|
||||||
|
"asn,handle,description,country-code\nAS1,LVLT-1,Level 3,US\n",
|
||||||
|
} {
|
||||||
|
_, err := parseCSV(strings.NewReader(asCSV))
|
||||||
|
if !errors.Is(err, errNoASes) {
|
||||||
|
t.Errorf("parseCSV(%q) = %v, want %v", asCSV, err, errNoASes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestParseCSVSkipsARecordThatDoesNotParse parses an as.csv with a record
|
||||||
|
// that has too few fields between two valid records. It expects both valid
|
||||||
|
// ASes and no error, so that one bad record does not stop make asupdate.
|
||||||
|
func TestParseCSVSkipsARecordThatDoesNotParse(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const asCSV = "asn,handle,description,country-code\n" +
|
||||||
|
"1,LVLT-1,Level 3 Parent LLC,US\n" +
|
||||||
|
"2,UDEL-DCN\n" +
|
||||||
|
"3,MIT-GATEWAYS,Massachusetts Institute of Technology,US\n"
|
||||||
|
|
||||||
|
got, err := parseCSV(strings.NewReader(asCSV))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("parseCSV: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
want := []ASInfo{
|
||||||
|
{ASN: 1, Handle: "LVLT-1", Description: "Level 3 Parent LLC"},
|
||||||
|
{
|
||||||
|
ASN: 3,
|
||||||
|
Handle: "MIT-GATEWAYS",
|
||||||
|
Description: "Massachusetts Institute of Technology",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(got, want) {
|
||||||
|
t.Errorf("parseCSV = %+v, want %+v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestParseCSVStopsWhenTheDownloadBreaksOff parses an as.csv whose download
|
||||||
|
// breaks off after the first record. From then on every read fails, as on a
|
||||||
|
// broken connection. It expects parseCSV to return that error, so that make
|
||||||
|
// asupdate keeps the old list.
|
||||||
|
func TestParseCSVStopsWhenTheDownloadBreaksOff(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const asCSV = "asn,handle,description,country-code\n" +
|
||||||
|
"1,LVLT-1,Level 3 Parent LLC,US\n"
|
||||||
|
|
||||||
|
download := io.MultiReader(
|
||||||
|
strings.NewReader(asCSV), iotest.ErrReader(io.ErrUnexpectedEOF))
|
||||||
|
|
||||||
|
// parseCSV runs on its own goroutine, so that a parseCSV that never
|
||||||
|
// returns fails the test instead of hanging it.
|
||||||
|
parsed := make(chan error, 1)
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
_, err := parseCSV(download)
|
||||||
|
parsed <- err
|
||||||
|
}()
|
||||||
|
|
||||||
|
select {
|
||||||
|
case err := <-parsed:
|
||||||
|
if !errors.Is(err, io.ErrUnexpectedEOF) {
|
||||||
|
t.Errorf("parseCSV = %v, want %v", err, io.ErrUnexpectedEOF)
|
||||||
|
}
|
||||||
|
case <-time.After(parseTimeLimit):
|
||||||
|
t.Fatalf("parseCSV did not return within %v", parseTimeLimit)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -10,6 +10,7 @@ require (
|
|||||||
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9
|
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9
|
||||||
go.uber.org/fx v1.24.0
|
go.uber.org/fx v1.24.0
|
||||||
golang.org/x/term v0.33.0
|
golang.org/x/term v0.33.0
|
||||||
|
golang.org/x/text v0.43.0
|
||||||
)
|
)
|
||||||
|
|
||||||
require (
|
require (
|
||||||
|
|||||||
@@ -28,5 +28,7 @@ golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
|
|||||||
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
|
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
|
||||||
golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
|
golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
|
||||||
golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
|
golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
|
||||||
|
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
|
||||||
|
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
|||||||
+122
-42
@@ -21,6 +21,7 @@ import (
|
|||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
||||||
_ "github.com/mattn/go-sqlite3" // CGO SQLite driver
|
_ "github.com/mattn/go-sqlite3" // CGO SQLite driver
|
||||||
)
|
)
|
||||||
@@ -169,6 +170,20 @@ func New(cfg *config.Config, logger *logger.Logger) (*Database, error) {
|
|||||||
return nil, fmt.Errorf("failed to create database directory: %w", err)
|
return nil, fmt.Errorf("failed to create database directory: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The size of the WAL file the previous run left, which Initialize logs
|
||||||
|
// with the start-up checkpoint. It is read before the database is opened,
|
||||||
|
// because opening it writes a frame to the WAL. There is no WAL file when
|
||||||
|
// the database is new or the last run closed the database, which deletes
|
||||||
|
// it.
|
||||||
|
var walBytes int64
|
||||||
|
|
||||||
|
wal, err := os.Stat(dbPath + walFileSuffix)
|
||||||
|
if err == nil {
|
||||||
|
walBytes = wal.Size()
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
logger.Warn("Failed to read the WAL file's size on startup", "error", err)
|
||||||
|
}
|
||||||
|
|
||||||
// Per-connection SQLite settings go in the DSN so every pooled connection
|
// Per-connection SQLite settings go in the DSN so every pooled connection
|
||||||
// gets them, not just the one that runs the Initialize pragmas. _txlock=
|
// gets them, not just the one that runs the Initialize pragmas. _txlock=
|
||||||
// immediate makes every transaction take the write lock at BEGIN. Without it
|
// immediate makes every transaction take the write lock at BEGIN. Without it
|
||||||
@@ -229,7 +244,7 @@ func New(cfg *config.Config, logger *logger.Logger) (*Database, error) {
|
|||||||
counts: &liveCounts{},
|
counts: &liveCounts{},
|
||||||
}
|
}
|
||||||
|
|
||||||
err = database.Initialize()
|
err = database.Initialize(walBytes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("failed to initialize database: %w", err)
|
return nil, fmt.Errorf("failed to initialize database: %w", err)
|
||||||
}
|
}
|
||||||
@@ -245,8 +260,16 @@ func New(cfg *config.Config, logger *logger.Logger) (*Database, error) {
|
|||||||
return database, nil
|
return database, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Initialize creates the database schema if it doesn't exist.
|
// The messages Initialize logs the start-up WAL checkpoint with.
|
||||||
func (d *Database) Initialize() error {
|
const (
|
||||||
|
startupCheckpointMessage = "WAL checkpoint on startup"
|
||||||
|
startupCheckpointIncompleteMessage = "WAL checkpoint on startup incomplete"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Initialize creates the database schema if it doesn't exist. walBytes is the
|
||||||
|
// size of the WAL file the previous run left, which New reads before it opens
|
||||||
|
// the database.
|
||||||
|
func (d *Database) Initialize(walBytes int64) error {
|
||||||
// Set SQLite pragmas for performance. Per-connection settings (cache_size,
|
// Set SQLite pragmas for performance. Per-connection settings (cache_size,
|
||||||
// synchronous, busy_timeout, journal_mode) live in the DSN, and New sets the
|
// synchronous, busy_timeout, journal_mode) live in the DSN, and New sets the
|
||||||
// writing connection's cache_size; temp_store is left at its default so
|
// writing connection's cache_size; temp_store is left at its default so
|
||||||
@@ -266,19 +289,33 @@ func (d *Database) Initialize() error {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Run WAL checkpoint on startup to consolidate any existing WAL
|
// The start-up checkpoint copies the WAL a previous run left into the
|
||||||
var walPages, checkpointedPages, movedPages int
|
// database file and empties the WAL file; its frame counts then read 0,
|
||||||
|
// so walBytes is logged with them.
|
||||||
|
var busy, walFrames, checkpointedFrames int
|
||||||
|
|
||||||
err := d.writer.QueryRowContext(context.Background(),
|
err := d.writer.QueryRowContext(context.Background(),
|
||||||
"PRAGMA wal_checkpoint(TRUNCATE)",
|
"PRAGMA wal_checkpoint(TRUNCATE)",
|
||||||
).Scan(&walPages, &checkpointedPages, &movedPages)
|
).Scan(&busy, &walFrames, &checkpointedFrames)
|
||||||
if err != nil {
|
|
||||||
|
switch {
|
||||||
|
case err != nil:
|
||||||
d.logger.Warn("Failed to checkpoint WAL on startup", "error", err)
|
d.logger.Warn("Failed to checkpoint WAL on startup", "error", err)
|
||||||
} else {
|
case busy != 0:
|
||||||
d.logger.Info("WAL checkpoint on startup",
|
// Another process using the database kept the checkpoint from
|
||||||
"wal_pages", walPages,
|
// copying or emptying all of the WAL.
|
||||||
"checkpointed", checkpointedPages,
|
d.logger.Warn(startupCheckpointIncompleteMessage,
|
||||||
"moved", movedPages,
|
"wal_bytes", walBytes,
|
||||||
|
"busy", busy,
|
||||||
|
"wal_frames", walFrames,
|
||||||
|
"checkpointed_frames", checkpointedFrames,
|
||||||
|
)
|
||||||
|
default:
|
||||||
|
d.logger.Info(startupCheckpointMessage,
|
||||||
|
"wal_bytes", walBytes,
|
||||||
|
"busy", busy,
|
||||||
|
"wal_frames", walFrames,
|
||||||
|
"checkpointed_frames", checkpointedFrames,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -908,6 +945,26 @@ func (d *Database) UpdatePrefixesBatch(prefixes map[string]time.Time) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// These statements update a stored AS, peering or peer. Each keeps the earlier
|
||||||
|
// of the stored and the new first_seen, and the later of the stored and the new
|
||||||
|
// last_seen. Some collectors send updates days old while they catch up on a
|
||||||
|
// backlog. MIN and MAX compare the times as text, which is time order here: the
|
||||||
|
// driver writes every time in one format, and every time is UTC.
|
||||||
|
const (
|
||||||
|
updateASNSQL = `UPDATE asns
|
||||||
|
SET first_seen = MIN(first_seen, ?), last_seen = MAX(last_seen, ?)
|
||||||
|
WHERE asn = ?`
|
||||||
|
|
||||||
|
updatePeeringSQL = `UPDATE peerings
|
||||||
|
SET first_seen = MIN(first_seen, ?), last_seen = MAX(last_seen, ?)
|
||||||
|
WHERE as_a = ? AND as_b = ?`
|
||||||
|
|
||||||
|
updatePeerSQL = `UPDATE bgp_peers
|
||||||
|
SET peer_asn = ?, last_message_type = ?,
|
||||||
|
first_seen = MIN(first_seen, ?), last_seen = MAX(last_seen, ?)
|
||||||
|
WHERE peer_ip = ?`
|
||||||
|
)
|
||||||
|
|
||||||
// newASN returns a new ASN record first seen at seen.First and last seen at
|
// newASN returns a new ASN record first seen at seen.First and last seen at
|
||||||
// seen.Last, with the handle and description from the AS info lookup when it
|
// seen.Last, with the handle and description from the AS info lookup when it
|
||||||
// knows the number.
|
// knows the number.
|
||||||
@@ -927,9 +984,8 @@ func newASN(number int, seen SeenTimes) ASN {
|
|||||||
return asn
|
return asn
|
||||||
}
|
}
|
||||||
|
|
||||||
// upsertASN sets the last_seen time of ASN number to seen.Last with the given
|
// upsertASN stores the seen times of ASN number with the given statements,
|
||||||
// statements, inserting the ASN when it is new. It reports whether it inserted
|
// inserting the ASN when it is new. It reports whether it inserted it.
|
||||||
// it.
|
|
||||||
func upsertASN(
|
func upsertASN(
|
||||||
ctx context.Context, selectStmt, updateStmt, insertStmt *sql.Stmt,
|
ctx context.Context, selectStmt, updateStmt, insertStmt *sql.Stmt,
|
||||||
number int, seen SeenTimes,
|
number int, seen SeenTimes,
|
||||||
@@ -942,8 +998,8 @@ func upsertASN(
|
|||||||
err := selectStmt.QueryRowContext(ctx, number).Scan(&asn.ASN, &handle,
|
err := selectStmt.QueryRowContext(ctx, number).Scan(&asn.ASN, &handle,
|
||||||
&description, &asn.FirstSeen, &asn.LastSeen)
|
&description, &asn.FirstSeen, &asn.LastSeen)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
// ASN exists, update last_seen
|
// ASN exists, update its seen times
|
||||||
_, err = updateStmt.ExecContext(ctx, seen.Last, number)
|
_, err = updateStmt.ExecContext(ctx, seen.First, seen.Last, number)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return false, fmt.Errorf("failed to update ASN %d: %w", number, err)
|
return false, fmt.Errorf("failed to update ASN %d: %w", number, err)
|
||||||
}
|
}
|
||||||
@@ -993,8 +1049,7 @@ func (d *Database) GetOrCreateASNBatch(asns map[int]SeenTimes) error {
|
|||||||
}
|
}
|
||||||
defer func() { _ = selectStmt.Close() }()
|
defer func() { _ = selectStmt.Close() }()
|
||||||
|
|
||||||
updateStmt, err := tx.PrepareContext(ctx,
|
updateStmt, err := tx.PrepareContext(ctx, updateASNSQL)
|
||||||
"UPDATE asns SET last_seen = ? WHERE asn = ?")
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("failed to prepare update statement: %w", err)
|
return fmt.Errorf("failed to prepare update statement: %w", err)
|
||||||
}
|
}
|
||||||
@@ -1053,17 +1108,19 @@ func (d *Database) GetOrCreateASN(number int, timestamp time.Time) (*ASN, error)
|
|||||||
"FROM asns WHERE asn = ?", number).
|
"FROM asns WHERE asn = ?", number).
|
||||||
Scan(&asn.ASN, &handle, &description, &asn.FirstSeen, &asn.LastSeen)
|
Scan(&asn.ASN, &handle, &description, &asn.FirstSeen, &asn.LastSeen)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
// ASN exists, update last_seen
|
// ASN exists, update its seen times
|
||||||
asn.Handle = handle.String
|
asn.Handle = handle.String
|
||||||
asn.Description = description.String
|
asn.Description = description.String
|
||||||
|
|
||||||
_, err = tx.Exec("UPDATE asns SET last_seen = ? WHERE asn = ?",
|
_, err = tx.Exec(updateASNSQL, timestamp, timestamp, number)
|
||||||
timestamp, number)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
asn.LastSeen = timestamp
|
// Return the seen times the update stored.
|
||||||
|
seen := SeenTimes{First: asn.FirstSeen, Last: asn.LastSeen}
|
||||||
|
seen.Add(timestamp)
|
||||||
|
asn.FirstSeen, asn.LastSeen = seen.First, seen.Last
|
||||||
|
|
||||||
err = tx.Commit()
|
err = tx.Commit()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -1222,9 +1279,7 @@ func (d *Database) RecordPeering(asA, asB int, seen SeenTimes) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if exists {
|
if exists {
|
||||||
_, err = tx.Exec(
|
_, err = tx.Exec(updatePeeringSQL, seen.First, seen.Last, asA, asB)
|
||||||
"UPDATE peerings SET last_seen = ? WHERE as_a = ? AND as_b = ?",
|
|
||||||
seen.Last, asA, asB)
|
|
||||||
} else {
|
} else {
|
||||||
_, err = tx.Exec(`
|
_, err = tx.Exec(`
|
||||||
INSERT INTO peerings (id, as_a, as_b, first_seen, last_seen)
|
INSERT INTO peerings (id, as_a, as_b, first_seen, last_seen)
|
||||||
@@ -1279,9 +1334,7 @@ func (d *Database) UpdatePeerBatch(peers map[string]PeerUpdate) error {
|
|||||||
}
|
}
|
||||||
defer func() { _ = checkStmt.Close() }()
|
defer func() { _ = checkStmt.Close() }()
|
||||||
|
|
||||||
updateStmt, err := tx.PrepareContext(ctx,
|
updateStmt, err := tx.PrepareContext(ctx, updatePeerSQL)
|
||||||
"UPDATE bgp_peers SET peer_asn = ?, last_seen = ?, "+
|
|
||||||
"last_message_type = ? WHERE peer_ip = ?")
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("failed to prepare update statement: %w", err)
|
return fmt.Errorf("failed to prepare update statement: %w", err)
|
||||||
}
|
}
|
||||||
@@ -1307,7 +1360,8 @@ func (d *Database) UpdatePeerBatch(peers map[string]PeerUpdate) error {
|
|||||||
|
|
||||||
if exists {
|
if exists {
|
||||||
_, err = updateStmt.ExecContext(ctx, update.PeerASN,
|
_, err = updateStmt.ExecContext(ctx, update.PeerASN,
|
||||||
update.Seen.Last, update.MessageType, update.PeerIP)
|
update.MessageType, update.Seen.First, update.Seen.Last,
|
||||||
|
update.PeerIP)
|
||||||
} else {
|
} else {
|
||||||
_, err = insertStmt.ExecContext(ctx,
|
_, err = insertStmt.ExecContext(ctx,
|
||||||
generateUUID().String(), update.PeerIP, update.PeerASN,
|
generateUUID().String(), update.PeerIP, update.PeerASN,
|
||||||
@@ -1356,11 +1410,8 @@ func (d *Database) UpdatePeer(
|
|||||||
}
|
}
|
||||||
|
|
||||||
if exists {
|
if exists {
|
||||||
_, err = tx.Exec(
|
_, err = tx.Exec(updatePeerSQL,
|
||||||
"UPDATE bgp_peers SET peer_asn = ?, last_seen = ?, "+
|
peerASN, messageType, timestamp, timestamp, peerIP)
|
||||||
"last_message_type = ? WHERE peer_ip = ?",
|
|
||||||
peerASN, timestamp, messageType, peerIP,
|
|
||||||
)
|
|
||||||
} else {
|
} else {
|
||||||
_, err = tx.Exec(
|
_, err = tx.Exec(
|
||||||
"INSERT INTO bgp_peers (id, peer_ip, peer_asn, first_seen, "+
|
"INSERT INTO bgp_peers (id, peer_ip, peer_asn, first_seen, "+
|
||||||
@@ -1712,9 +1763,21 @@ func (d *Database) GetASDetails(asn int) (*ASN, []LiveRoute, error) {
|
|||||||
// GetASDetailsContext returns the AS and the prefixes it originates, the IPv4
|
// GetASDetailsContext returns the AS and the prefixes it originates, the IPv4
|
||||||
// ones first, each once with the time of its newest route. It makes two index
|
// ones first, each once with the time of its newest route. It makes two index
|
||||||
// searches per prefix, however many peers carry the prefix.
|
// searches per prefix, however many peers carry the prefix.
|
||||||
|
//
|
||||||
|
// The routes of an AS can be stored before its record in the asns table. When
|
||||||
|
// the AS has no record, its handle and description come from the AS info
|
||||||
|
// lookup, and it has no seen times. An AS with no record, no lookup entry and
|
||||||
|
// no prefix is ErrNoRoute.
|
||||||
|
//
|
||||||
|
// ristypes.NoOrigin means no known origin, not an AS, so it is ErrNoRoute
|
||||||
|
// whatever is stored.
|
||||||
func (d *Database) GetASDetailsContext(
|
func (d *Database) GetASDetailsContext(
|
||||||
ctx context.Context, asn int,
|
ctx context.Context, asn int,
|
||||||
) (*ASN, []LiveRoute, error) {
|
) (*ASN, []LiveRoute, error) {
|
||||||
|
if asn == ristypes.NoOrigin {
|
||||||
|
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
|
||||||
|
}
|
||||||
|
|
||||||
// Get AS information
|
// Get AS information
|
||||||
var (
|
var (
|
||||||
asnInfo ASN
|
asnInfo ASN
|
||||||
@@ -1727,17 +1790,28 @@ func (d *Database) GetASDetailsContext(
|
|||||||
asn,
|
asn,
|
||||||
).Scan(&asnInfo.ASN, &handle, &description, &asnInfo.FirstSeen,
|
).Scan(&asnInfo.ASN, &handle, &description, &asnInfo.FirstSeen,
|
||||||
&asnInfo.LastSeen)
|
&asnInfo.LastSeen)
|
||||||
if err != nil {
|
|
||||||
if errors.Is(err, sql.ErrNoRows) {
|
// known is whether the asns table or the AS info lookup holds the AS.
|
||||||
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
|
known := true
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, sql.ErrNoRows):
|
||||||
|
asnInfo.ASN = asn
|
||||||
|
|
||||||
|
info, ok := asinfo.Get(asn)
|
||||||
|
if ok {
|
||||||
|
asnInfo.Handle = info.Handle
|
||||||
|
asnInfo.Description = info.Description
|
||||||
}
|
}
|
||||||
|
|
||||||
|
known = ok
|
||||||
|
case err != nil:
|
||||||
return nil, nil, fmt.Errorf("failed to query AS: %w", err)
|
return nil, nil, fmt.Errorf("failed to query AS: %w", err)
|
||||||
|
default:
|
||||||
|
asnInfo.Handle = handle.String
|
||||||
|
asnInfo.Description = description.String
|
||||||
}
|
}
|
||||||
|
|
||||||
asnInfo.Handle = handle.String
|
|
||||||
asnInfo.Description = description.String
|
|
||||||
|
|
||||||
allPrefixes, err := d.originPrefixes(ctx, asn, ipVersionV4)
|
allPrefixes, err := d.originPrefixes(ctx, asn, ipVersionV4)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return &asnInfo, nil, err
|
return &asnInfo, nil, err
|
||||||
@@ -1748,7 +1822,13 @@ func (d *Database) GetASDetailsContext(
|
|||||||
return &asnInfo, allPrefixes, err
|
return &asnInfo, allPrefixes, err
|
||||||
}
|
}
|
||||||
|
|
||||||
return &asnInfo, append(allPrefixes, prefixesV6...), nil
|
allPrefixes = append(allPrefixes, prefixesV6...)
|
||||||
|
|
||||||
|
if !known && len(allPrefixes) == 0 {
|
||||||
|
return nil, nil, fmt.Errorf("%w: AS%d", ErrNoRoute, asn)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &asnInfo, allPrefixes, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ASPeer represents a peering relationship with another AS including handle,
|
// ASPeer represents a peering relationship with another AS including handle,
|
||||||
|
|||||||
@@ -1,8 +1,10 @@
|
|||||||
package database
|
package database
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"database/sql"
|
"database/sql"
|
||||||
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
@@ -18,6 +20,8 @@ import (
|
|||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
|
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
||||||
"github.com/mattn/go-sqlite3"
|
"github.com/mattn/go-sqlite3"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1052,6 +1056,176 @@ func walFileSize(t *testing.T, db *Database) int64 {
|
|||||||
return wal.Size()
|
return wal.Size()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestStartupCheckpointLogsItsFigures opens a new database, writes a route,
|
||||||
|
// and opens the same file again while the first stays open, so that its WAL
|
||||||
|
// file stays. The first open finds no WAL file: it logs a size of 0 and no
|
||||||
|
// warning. The second logs the size of the WAL file the first left, and the
|
||||||
|
// checkpoint's busy flag and frame counts, each under its own name; the
|
||||||
|
// counts read 0 once the checkpoint has emptied the file.
|
||||||
|
// Initialize logged the busy flag as wal_pages and the counts under each
|
||||||
|
// other's names (https://git.eeqj.de/sneak/routewatch/issues/117).
|
||||||
|
func TestStartupCheckpointLogsItsFigures(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cfg := &config.Config{StateDir: t.TempDir()}
|
||||||
|
|
||||||
|
db, lines := openLogged(t, cfg)
|
||||||
|
|
||||||
|
for _, line := range lines {
|
||||||
|
if line[slog.LevelKey] == slog.LevelWarn.String() {
|
||||||
|
t.Errorf("opening a new database logged a warning: %v", line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
line := findLine(t, lines, startupCheckpointMessage)
|
||||||
|
if line["wal_bytes"] != 0.0 {
|
||||||
|
t.Errorf("new database: start-up checkpoint logged wal_bytes = %v, "+
|
||||||
|
"want 0", line["wal_bytes"])
|
||||||
|
}
|
||||||
|
|
||||||
|
err := db.UpsertLiveRouteBatch([]*LiveRoute{
|
||||||
|
mkV4Route(t, "192.0.2.0/24", lockTestASN, time.Now().UTC()),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to store a route: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
walBytes := walFileSize(t, db)
|
||||||
|
if walBytes == 0 {
|
||||||
|
t.Fatal("WAL file empty after the route write, want frames in it")
|
||||||
|
}
|
||||||
|
|
||||||
|
_, lines = openLogged(t, cfg)
|
||||||
|
line = findLine(t, lines, startupCheckpointMessage)
|
||||||
|
|
||||||
|
for _, key := range []string{"busy", "wal_frames", "checkpointed_frames"} {
|
||||||
|
if line[key] != 0.0 {
|
||||||
|
t.Errorf("start-up checkpoint logged %s = %v, want 0", key, line[key])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if line["wal_bytes"] != float64(walBytes) {
|
||||||
|
t.Errorf("start-up checkpoint logged wal_bytes = %v, want the %d "+
|
||||||
|
"bytes the first database left", line["wal_bytes"], walBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestStartupCheckpointWarnsWhenIncomplete keeps a read open on a database
|
||||||
|
// while a route is written after the read began, then opens the same file
|
||||||
|
// again. The second open's start-up checkpoint cannot copy the frames written
|
||||||
|
// after the read began, and gives up once the busy timeout has passed. It
|
||||||
|
// checks that the checkpoint is logged as a warning with busy 1 and fewer
|
||||||
|
// frames copied than the WAL holds.
|
||||||
|
func TestStartupCheckpointWarnsWhenIncomplete(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cfg := &config.Config{StateDir: t.TempDir()}
|
||||||
|
db, _ := openLogged(t, cfg)
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
reader, err := db.db.Conn(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to open the reading connection: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = reader.Close() }()
|
||||||
|
|
||||||
|
// A plain BEGIN starts a transaction that only reads, unlike the BEGIN
|
||||||
|
// IMMEDIATE of _txlock. It keeps the snapshot its first read takes until
|
||||||
|
// it ends.
|
||||||
|
_, err = reader.ExecContext(ctx, "BEGIN")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to begin the read transaction: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _, _ = reader.ExecContext(ctx, "ROLLBACK") }()
|
||||||
|
|
||||||
|
var routes int
|
||||||
|
|
||||||
|
err = reader.QueryRowContext(ctx,
|
||||||
|
"SELECT count(*) FROM live_routes_v4").Scan(&routes)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to read the live routes: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = db.UpsertLiveRouteBatch([]*LiveRoute{
|
||||||
|
mkV4Route(t, "192.0.2.0/24", lockTestASN, time.Now().UTC()),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to store a route after the read: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, lines := openLogged(t, cfg)
|
||||||
|
line := findLine(t, lines, startupCheckpointIncompleteMessage)
|
||||||
|
|
||||||
|
if line[slog.LevelKey] != slog.LevelWarn.String() {
|
||||||
|
t.Errorf("start-up checkpoint logged at level %v, want %s",
|
||||||
|
line[slog.LevelKey], slog.LevelWarn)
|
||||||
|
}
|
||||||
|
|
||||||
|
if line["busy"] != 1.0 {
|
||||||
|
t.Errorf("start-up checkpoint logged busy = %v, want 1", line["busy"])
|
||||||
|
}
|
||||||
|
|
||||||
|
walFrames, _ := line["wal_frames"].(float64)
|
||||||
|
checkpointedFrames, _ := line["checkpointed_frames"].(float64)
|
||||||
|
|
||||||
|
if checkpointedFrames >= walFrames {
|
||||||
|
t.Errorf("start-up checkpoint logged checkpointed_frames = %v and "+
|
||||||
|
"wal_frames = %v with a read open, want fewer copied than the WAL "+
|
||||||
|
"holds", line["checkpointed_frames"], line["wal_frames"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// openLogged opens the database of cfg with a logger that writes JSON, and
|
||||||
|
// returns it with the lines it logged while it opened, decoded.
|
||||||
|
func openLogged(t *testing.T, cfg *config.Config) (*Database, []map[string]any) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var logged bytes.Buffer
|
||||||
|
|
||||||
|
db, err := New(cfg,
|
||||||
|
&logger.Logger{Logger: slog.New(slog.NewJSONHandler(&logged, nil))})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to open the database: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
|
var lines []map[string]any
|
||||||
|
|
||||||
|
decoder := json.NewDecoder(&logged)
|
||||||
|
for decoder.More() {
|
||||||
|
var line map[string]any
|
||||||
|
|
||||||
|
err := decoder.Decode(&line)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to decode the log: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lines = append(lines, line)
|
||||||
|
}
|
||||||
|
|
||||||
|
return db, lines
|
||||||
|
}
|
||||||
|
|
||||||
|
// findLine returns the first of lines logged with message.
|
||||||
|
func findLine(
|
||||||
|
t *testing.T,
|
||||||
|
lines []map[string]any,
|
||||||
|
message string,
|
||||||
|
) map[string]any {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
for _, line := range lines {
|
||||||
|
if line[slog.MessageKey] == message {
|
||||||
|
return line
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Fatalf("no line %q in the log", message)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// TestGetRandomPrefixesByLengthListsPrefixOnce stores one prefix of each
|
// TestGetRandomPrefixesByLengthListsPrefixOnce stores one prefix of each
|
||||||
// family learned from two peers in the same second and checks that the
|
// family learned from two peers in the same second and checks that the
|
||||||
// prefix-length sample lists each of them once.
|
// prefix-length sample lists each of them once.
|
||||||
@@ -1527,6 +1701,85 @@ func TestGetASDetailsListsEachPrefixOnce(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestGetASDetailsOfASNotInDatabase asks for ASes that the database has no
|
||||||
|
// record of, some with stored routes. Each gets the prefixes of its routes and
|
||||||
|
// no seen times. One that the AS info lookup knows gets its handle and
|
||||||
|
// description from it. An AS with no record, no lookup entry and no route is
|
||||||
|
// ErrNoRoute. Before the fix every AS with no record was ErrNoRoute.
|
||||||
|
func TestGetASDetailsOfASNotInDatabase(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create database: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = db.Close() }()
|
||||||
|
|
||||||
|
ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
asn int
|
||||||
|
listed bool // in the AS info lookup
|
||||||
|
prefixes []string // of the routes stored for the AS
|
||||||
|
}{
|
||||||
|
{15169, true, []string{"192.0.2.0/24"}},
|
||||||
|
{13335, true, nil},
|
||||||
|
{4200000000, false, []string{"198.51.100.0/24"}},
|
||||||
|
}
|
||||||
|
|
||||||
|
var routes []*LiveRoute
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
if _, ok := asinfo.Get(tt.asn); ok != tt.listed {
|
||||||
|
t.Fatalf("AS%d in the AS info data: %t, want %t", tt.asn, ok, tt.listed)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, prefix := range tt.prefixes {
|
||||||
|
routes = append(routes, mkV4Route(t, prefix, tt.asn, ts))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
err = db.UpsertLiveRouteBatch(routes)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
got, prefixes, err := db.GetASDetails(tt.asn)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("GetASDetails(%d): %v", tt.asn, err)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
want := ASN{ASN: tt.asn}
|
||||||
|
if info, ok := asinfo.Get(tt.asn); ok {
|
||||||
|
want.Handle, want.Description = info.Handle, info.Description
|
||||||
|
}
|
||||||
|
|
||||||
|
var gotPrefixes []string
|
||||||
|
for _, route := range prefixes {
|
||||||
|
gotPrefixes = append(gotPrefixes, route.Prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
if *got != want || !slices.Equal(gotPrefixes, tt.prefixes) {
|
||||||
|
t.Errorf("GetASDetails(%d) = %+v with prefixes %v, want %+v with %v",
|
||||||
|
tt.asn, *got, gotPrefixes, want, tt.prefixes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const absentASN = 4200000001 // no record, no lookup entry and no route
|
||||||
|
|
||||||
|
if _, ok := asinfo.Get(absentASN); ok {
|
||||||
|
t.Fatalf("AS%d is in the AS info data", absentASN)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, _, err = db.GetASDetails(absentASN)
|
||||||
|
if !errors.Is(err, ErrNoRoute) {
|
||||||
|
t.Errorf("GetASDetails(%d): error %v, want %v", absentASN, err, ErrNoRoute)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestASPrefixQueriesUseCoveringIndexes checks the plan of each query that
|
// TestASPrefixQueriesUseCoveringIndexes checks the plan of each query that
|
||||||
// lists the prefixes an AS originates: a search of a covering index and no
|
// lists the prefixes an AS originates: a search of a covering index and no
|
||||||
// temporary B-tree, so the list reads one index entry per prefix, not one
|
// temporary B-tree, so the list reads one index entry per prefix, not one
|
||||||
@@ -1708,6 +1961,158 @@ func TestUpdatePeerBatchStoresTheEarliestAndLatestTimesOfANewPeer(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// backlogAge is how much older one write is than the other in the tests of
|
||||||
|
// first_seen and last_seen, as when a collector catches up on a backlog.
|
||||||
|
const backlogAge = 24 * time.Hour
|
||||||
|
|
||||||
|
// batchSpan is how much earlier a batch's First is than its Last in the tests
|
||||||
|
// of first_seen and last_seen.
|
||||||
|
const batchSpan = time.Hour
|
||||||
|
|
||||||
|
// TestWritesKeepEarliestFirstSeenAndLatestLastSeen writes a row of each of
|
||||||
|
// asns, peerings and bgp_peers through each function that writes one, at a
|
||||||
|
// newer time and at an older one. Each row takes the older write's First as
|
||||||
|
// first_seen and the newer write's Last as last_seen. The test writes in both
|
||||||
|
// orders, each in a database of its own. Newer then older is a collector
|
||||||
|
// catching up on a backlog; older then newer is a feed that arrives in order.
|
||||||
|
// Each function writes a row of its own. A batch write's First is batchSpan
|
||||||
|
// before its Last, so the test catches the two swapped. A single-row write
|
||||||
|
// takes one time.
|
||||||
|
func TestWritesKeepEarliestFirstSeenAndLatestLastSeen(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const (
|
||||||
|
batchASN, singleASN = 64500, 64501
|
||||||
|
peerASN = 64502
|
||||||
|
asnQuery = "SELECT first_seen, last_seen FROM asns WHERE asn = ?"
|
||||||
|
peeringQuery = "SELECT first_seen, last_seen FROM peerings " +
|
||||||
|
"WHERE as_a = ? AND as_b = ?"
|
||||||
|
peerQuery = "SELECT first_seen, last_seen FROM bgp_peers " +
|
||||||
|
"WHERE peer_ip = ?"
|
||||||
|
)
|
||||||
|
|
||||||
|
for name, olderFirst := range map[string]bool{
|
||||||
|
"newer then older": false,
|
||||||
|
"older then newer": true,
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create database: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = db.Close() }()
|
||||||
|
|
||||||
|
checkSeenTimes(t, db, olderFirst, func(seen SeenTimes) error {
|
||||||
|
return db.GetOrCreateASNBatch(map[int]SeenTimes{batchASN: seen})
|
||||||
|
}, batchSpan, asnQuery, batchASN)
|
||||||
|
|
||||||
|
checkSeenTimes(t, db, olderFirst, func(seen SeenTimes) error {
|
||||||
|
_, err := db.GetOrCreateASN(singleASN, seen.Last)
|
||||||
|
|
||||||
|
return err
|
||||||
|
}, 0, asnQuery, singleASN)
|
||||||
|
|
||||||
|
checkSeenTimes(t, db, olderFirst, func(seen SeenTimes) error {
|
||||||
|
return db.RecordPeering(batchASN, singleASN, seen)
|
||||||
|
}, batchSpan, peeringQuery, batchASN, singleASN)
|
||||||
|
|
||||||
|
checkSeenTimes(t, db, olderFirst, func(seen SeenTimes) error {
|
||||||
|
return db.UpdatePeerBatch(map[string]PeerUpdate{
|
||||||
|
testV4PeerIP: {
|
||||||
|
PeerIP: testV4PeerIP, PeerASN: peerASN,
|
||||||
|
MessageType: ristypes.MessageTypeUpdate,
|
||||||
|
Seen: seen,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}, batchSpan, peerQuery, testV4PeerIP)
|
||||||
|
|
||||||
|
checkSeenTimes(t, db, olderFirst, func(seen SeenTimes) error {
|
||||||
|
return db.UpdatePeer(testPeerIP, peerASN,
|
||||||
|
ristypes.MessageTypeUpdate, seen.Last)
|
||||||
|
}, 0, peerQuery, testPeerIP)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// checkSeenTimes calls write twice, each time with First span before Last: at
|
||||||
|
// newer times and at times backlogAge older. It writes the older times first
|
||||||
|
// when olderFirst is true. It checks that query, given args, reads the older
|
||||||
|
// write's First as first_seen and the newer write's Last as last_seen.
|
||||||
|
func checkSeenTimes(
|
||||||
|
t *testing.T, db *Database, olderFirst bool,
|
||||||
|
write func(seen SeenTimes) error,
|
||||||
|
span time.Duration, query string, args ...any,
|
||||||
|
) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
newerLast := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
||||||
|
newer := SeenTimes{First: newerLast.Add(-span), Last: newerLast}
|
||||||
|
older := SeenTimes{
|
||||||
|
First: newer.First.Add(-backlogAge),
|
||||||
|
Last: newer.Last.Add(-backlogAge),
|
||||||
|
}
|
||||||
|
|
||||||
|
writes := []SeenTimes{newer, older}
|
||||||
|
if olderFirst {
|
||||||
|
writes = []SeenTimes{older, newer}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, seen := range writes {
|
||||||
|
err := write(seen)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("writing first %s, last %s: %v", seen.First, seen.Last, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var firstSeen, lastSeen time.Time
|
||||||
|
|
||||||
|
err := db.db.QueryRowContext(t.Context(), query, args...).
|
||||||
|
Scan(&firstSeen, &lastSeen)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("reading the row: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !firstSeen.Equal(older.First) || !lastSeen.Equal(newer.Last) {
|
||||||
|
t.Errorf("first seen %s, last seen %s; want %s, %s",
|
||||||
|
firstSeen, lastSeen, older.First, newer.Last)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestGetOrCreateASNReturnsStoredSeenTimes stores an AS at a newer time, then
|
||||||
|
// at an older one. The second call returns the times the row now holds: the
|
||||||
|
// older first_seen and the newer last_seen.
|
||||||
|
func TestGetOrCreateASNReturnsStoredSeenTimes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("failed to create database: %v", err)
|
||||||
|
}
|
||||||
|
defer func() { _ = db.Close() }()
|
||||||
|
|
||||||
|
const asn = 64500
|
||||||
|
|
||||||
|
newer := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
|
||||||
|
older := newer.Add(-backlogAge)
|
||||||
|
|
||||||
|
_, err = db.GetOrCreateASN(asn, newer)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetOrCreateASN at %s: %v", newer, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, err := db.GetOrCreateASN(asn, older)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetOrCreateASN at %s: %v", older, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !got.FirstSeen.Equal(older) || !got.LastSeen.Equal(newer) {
|
||||||
|
t.Errorf("first seen %s, last seen %s; want %s, %s",
|
||||||
|
got.FirstSeen, got.LastSeen, older, newer)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestUpsertLiveRouteBatchWritesInKeyOrder writes new routes of two peers out
|
// TestUpsertLiveRouteBatchWritesInKeyOrder writes new routes of two peers out
|
||||||
// of key order, the last one a second announcement of the first, and checks
|
// of key order, the last one a second announcement of the first, and checks
|
||||||
// that the rows were inserted in key order, peer then prefix (each new row's id
|
// that the rows were inserted in key order, peer then prefix (each new row's id
|
||||||
|
|||||||
@@ -28,9 +28,10 @@ type ASN struct {
|
|||||||
RIRRegDate *time.Time `json:"rir_registration_date,omitempty"`
|
RIRRegDate *time.Time `json:"rir_registration_date,omitempty"`
|
||||||
RIRLastMod *time.Time `json:"rir_last_modified,omitempty"`
|
RIRLastMod *time.Time `json:"rir_last_modified,omitempty"`
|
||||||
WHOISRaw string `json:"whois_raw,omitempty"`
|
WHOISRaw string `json:"whois_raw,omitempty"`
|
||||||
// Timestamps
|
// Timestamps. They are zero, and left out of JSON, for an AS that has no
|
||||||
FirstSeen time.Time `json:"first_seen"`
|
// record in the asns table.
|
||||||
LastSeen time.Time `json:"last_seen"`
|
FirstSeen time.Time `json:"first_seen,omitzero"`
|
||||||
|
LastSeen time.Time `json:"last_seen,omitzero"`
|
||||||
WHOISUpdatedAt *time.Time `json:"whois_updated_at,omitempty"`
|
WHOISUpdatedAt *time.Time `json:"whois_updated_at,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -124,8 +125,8 @@ type LiveRouteDeletion struct {
|
|||||||
|
|
||||||
// SeenTimes is the earliest and the latest time one batch of updates saw an AS,
|
// SeenTimes is the earliest and the latest time one batch of updates saw an AS,
|
||||||
// a peer or a peering. A row the batch adds is stored first seen at First and
|
// a peer or a peering. A row the batch adds is stored first seen at First and
|
||||||
// last seen at Last; a row already stored keeps its first_seen and has its
|
// last seen at Last. A row already stored takes First as its first_seen only
|
||||||
// last_seen set to Last.
|
// when First is earlier, and Last as its last_seen only when Last is later.
|
||||||
type SeenTimes struct {
|
type SeenTimes struct {
|
||||||
First time.Time
|
First time.Time
|
||||||
Last time.Time
|
Last time.Time
|
||||||
|
|||||||
@@ -34,7 +34,7 @@ type Tracker struct {
|
|||||||
// Announcement/withdrawal metrics
|
// Announcement/withdrawal metrics
|
||||||
announcementCounter metrics.Counter
|
announcementCounter metrics.Counter
|
||||||
withdrawalCounter metrics.Counter
|
withdrawalCounter metrics.Counter
|
||||||
churnRate metrics.Meter // combined announcements + withdrawals per second
|
churnPerMinute minuteCounter // prefix announcements and withdrawals
|
||||||
|
|
||||||
// BGP peer tracking
|
// BGP peer tracking
|
||||||
bgpPeerCount atomic.Int32
|
bgpPeerCount atomic.Int32
|
||||||
@@ -56,7 +56,7 @@ func New() *Tracker {
|
|||||||
ipv6UpdateRate: metrics.NewMeter(),
|
ipv6UpdateRate: metrics.NewMeter(),
|
||||||
announcementCounter: metrics.NewCounter(),
|
announcementCounter: metrics.NewCounter(),
|
||||||
withdrawalCounter: metrics.NewCounter(),
|
withdrawalCounter: metrics.NewCounter(),
|
||||||
churnRate: metrics.NewMeter(),
|
churnPerMinute: minuteCounter{now: time.Now},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -156,13 +156,13 @@ func (t *Tracker) RecordIPv6Update() {
|
|||||||
// RecordAnnouncement records a route announcement
|
// RecordAnnouncement records a route announcement
|
||||||
func (t *Tracker) RecordAnnouncement() {
|
func (t *Tracker) RecordAnnouncement() {
|
||||||
t.announcementCounter.Inc(1)
|
t.announcementCounter.Inc(1)
|
||||||
t.churnRate.Mark(1)
|
t.churnPerMinute.add()
|
||||||
}
|
}
|
||||||
|
|
||||||
// RecordWithdrawal records a route withdrawal
|
// RecordWithdrawal records a route withdrawal
|
||||||
func (t *Tracker) RecordWithdrawal() {
|
func (t *Tracker) RecordWithdrawal() {
|
||||||
t.withdrawalCounter.Inc(1)
|
t.withdrawalCounter.Inc(1)
|
||||||
t.churnRate.Mark(1)
|
t.churnPerMinute.add()
|
||||||
}
|
}
|
||||||
|
|
||||||
// SetBGPPeerCount updates the current BGP peer count
|
// SetBGPPeerCount updates the current BGP peer count
|
||||||
@@ -198,9 +198,10 @@ func (t *Tracker) GetWithdrawalCount() uint64 {
|
|||||||
return uint64(count)
|
return uint64(count)
|
||||||
}
|
}
|
||||||
|
|
||||||
// GetChurnRate returns the route churn rate per second
|
// GetChurnPerMinute returns the number of prefix announcements and
|
||||||
func (t *Tracker) GetChurnRate() float64 {
|
// withdrawals, from all peers, in the last full clock minute.
|
||||||
return t.churnRate.Rate1()
|
func (t *Tracker) GetChurnPerMinute() int {
|
||||||
|
return t.churnPerMinute.lastFullMinute()
|
||||||
}
|
}
|
||||||
|
|
||||||
// GetRouteMetrics returns current route update metrics
|
// GetRouteMetrics returns current route update metrics
|
||||||
|
|||||||
@@ -0,0 +1,59 @@
|
|||||||
|
package metrics
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// minuteCounter counts events in each clock minute and reports the count of
|
||||||
|
// the last full minute. It is safe for concurrent use.
|
||||||
|
type minuteCounter struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
now func() time.Time
|
||||||
|
|
||||||
|
// minute is the start of the minute that count belongs to.
|
||||||
|
minute time.Time
|
||||||
|
count int
|
||||||
|
// lastMinuteCount is the count of the minute just before minute.
|
||||||
|
lastMinuteCount int
|
||||||
|
}
|
||||||
|
|
||||||
|
// add counts one event in the current minute.
|
||||||
|
func (c *minuteCounter) add() {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
|
||||||
|
c.moveToCurrentMinute()
|
||||||
|
c.count++
|
||||||
|
}
|
||||||
|
|
||||||
|
// lastFullMinute returns the number of events in the minute that ended most
|
||||||
|
// recently.
|
||||||
|
func (c *minuteCounter) lastFullMinute() int {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
|
||||||
|
c.moveToCurrentMinute()
|
||||||
|
|
||||||
|
return c.lastMinuteCount
|
||||||
|
}
|
||||||
|
|
||||||
|
// moveToCurrentMinute starts a new count when the clock has moved into
|
||||||
|
// another minute. The count it ends becomes the last full minute's count when
|
||||||
|
// its minute is the one just before the current minute. Otherwise the last
|
||||||
|
// full minute had no events. The caller holds mu.
|
||||||
|
func (c *minuteCounter) moveToCurrentMinute() {
|
||||||
|
minute := c.now().Truncate(time.Minute)
|
||||||
|
if minute.Equal(c.minute) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if minute.Sub(c.minute) == time.Minute {
|
||||||
|
c.lastMinuteCount = c.count
|
||||||
|
} else {
|
||||||
|
c.lastMinuteCount = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
c.minute = minute
|
||||||
|
c.count = 0
|
||||||
|
}
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
package metrics
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestChurnPerMinuteCountsLastFullMinute announces and withdraws prefixes on
|
||||||
|
// an injected clock and expects each minute's whole count once that minute
|
||||||
|
// has ended, and 0 for a minute with none.
|
||||||
|
func TestChurnPerMinuteCountsLastFullMinute(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
now := time.Date(2026, 10, 26, 12, 0, 30, 0, time.UTC)
|
||||||
|
tracker := New()
|
||||||
|
tracker.churnPerMinute.now = func() time.Time { return now }
|
||||||
|
|
||||||
|
expectChurn := func(want int, why string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
got := tracker.GetChurnPerMinute()
|
||||||
|
if got != want {
|
||||||
|
t.Fatalf("at %s churn = %d, want %d: %s",
|
||||||
|
now.Format(time.TimeOnly), got, want, why)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const firstMinuteChurn = 2
|
||||||
|
|
||||||
|
tracker.RecordAnnouncement()
|
||||||
|
tracker.RecordWithdrawal()
|
||||||
|
expectChurn(0, "12:00 has not ended")
|
||||||
|
|
||||||
|
now = time.Date(2026, 10, 26, 12, 1, 59, 0, time.UTC)
|
||||||
|
|
||||||
|
tracker.RecordAnnouncement()
|
||||||
|
expectChurn(firstMinuteChurn, "12:00 had one announcement and one withdrawal")
|
||||||
|
|
||||||
|
now = time.Date(2026, 10, 26, 12, 2, 0, 0, time.UTC)
|
||||||
|
|
||||||
|
expectChurn(1, "12:01 had one announcement")
|
||||||
|
|
||||||
|
now = time.Date(2026, 10, 26, 12, 4, 0, 0, time.UTC)
|
||||||
|
|
||||||
|
expectChurn(0, "12:03 had none")
|
||||||
|
|
||||||
|
tracker.RecordWithdrawal()
|
||||||
|
|
||||||
|
now = time.Date(2026, 10, 26, 12, 6, 0, 0, time.UTC)
|
||||||
|
|
||||||
|
expectChurn(0, "12:05 had none, though 12:04 had one withdrawal")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestChurnPerMinuteCountsConcurrentRecords announces prefixes from several
|
||||||
|
// goroutines at once and expects every one counted.
|
||||||
|
func TestChurnPerMinuteCountsConcurrentRecords(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const (
|
||||||
|
goroutines = 8
|
||||||
|
recordsPerGoroutine = 1000
|
||||||
|
)
|
||||||
|
|
||||||
|
now := time.Date(2026, 10, 26, 12, 0, 0, 0, time.UTC)
|
||||||
|
tracker := New()
|
||||||
|
tracker.churnPerMinute.now = func() time.Time { return now }
|
||||||
|
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for range goroutines {
|
||||||
|
wg.Go(func() {
|
||||||
|
for range recordsPerGoroutine {
|
||||||
|
tracker.RecordAnnouncement()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
wg.Wait()
|
||||||
|
|
||||||
|
now = now.Add(time.Minute)
|
||||||
|
|
||||||
|
want := goroutines * recordsPerGoroutine
|
||||||
|
if got := tracker.GetChurnPerMinute(); got != want {
|
||||||
|
t.Fatalf("churn = %d, want %d", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -39,12 +39,16 @@ type ASPath struct {
|
|||||||
Set []int
|
Set []int
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// NoOrigin is the origin AS of a route whose origin is not known. AS 0 is
|
||||||
|
// reserved, so no AS has this number.
|
||||||
|
const NoOrigin = 0
|
||||||
|
|
||||||
// Origin returns the AS that originated the route: the last AS of the
|
// Origin returns the AS that originated the route: the last AS of the
|
||||||
// sequence, or 0 (none) when the path ends in a set of several members or is
|
// sequence, or NoOrigin when the path ends in a set of several members or is
|
||||||
// empty.
|
// empty.
|
||||||
func (p *ASPath) Origin() int {
|
func (p *ASPath) Origin() int {
|
||||||
if len(p.Set) > 0 || len(p.Sequence) == 0 {
|
if len(p.Set) > 0 || len(p.Sequence) == 0 {
|
||||||
return 0
|
return NoOrigin
|
||||||
}
|
}
|
||||||
|
|
||||||
return p.Sequence[len(p.Sequence)-1]
|
return p.Sequence[len(p.Sequence)-1]
|
||||||
|
|||||||
@@ -21,10 +21,16 @@ const (
|
|||||||
minInterval = 1 * time.Second
|
minInterval = 1 * time.Second
|
||||||
|
|
||||||
// maxInterval is the maximum interval after failures (backoff cap).
|
// maxInterval is the maximum interval after failures (backoff cap).
|
||||||
maxInterval = 5 * time.Minute
|
maxInterval = time.Minute
|
||||||
|
|
||||||
// backoffMultiplier is how much to multiply interval on failure.
|
// A failure multiplies the interval by backoffNumerator divided by
|
||||||
backoffMultiplier = 2
|
// backoffDenominator, 1.5, up to maxInterval.
|
||||||
|
backoffNumerator = 3
|
||||||
|
backoffDenominator = 2
|
||||||
|
|
||||||
|
// A success divides the interval by successDivisor, down to
|
||||||
|
// minInterval.
|
||||||
|
successDivisor = 2
|
||||||
|
|
||||||
// whoisStaleThreshold is how old WHOIS data can be before refresh.
|
// whoisStaleThreshold is how old WHOIS data can be before refresh.
|
||||||
whoisStaleThreshold = 30 * 24 * time.Hour // 30 days
|
whoisStaleThreshold = 30 * 24 * time.Hour // 30 days
|
||||||
@@ -62,10 +68,12 @@ type ASNFetcher struct {
|
|||||||
|
|
||||||
// NewASNFetcher creates a new ASN fetcher.
|
// NewASNFetcher creates a new ASN fetcher.
|
||||||
func NewASNFetcher(db database.Store, logger *slog.Logger) *ASNFetcher {
|
func NewASNFetcher(db database.Store, logger *slog.Logger) *ASNFetcher {
|
||||||
|
logger = logger.With("component", "asn_fetcher")
|
||||||
|
|
||||||
return &ASNFetcher{
|
return &ASNFetcher{
|
||||||
db: db,
|
db: db,
|
||||||
whoisClient: whois.NewClient(),
|
whoisClient: whois.NewClient(logger),
|
||||||
logger: logger.With("component", "asn_fetcher"),
|
logger: logger,
|
||||||
immediateQueue: make(chan int, immediateQueueSize),
|
immediateQueue: make(chan int, immediateQueueSize),
|
||||||
stopCh: make(chan struct{}),
|
stopCh: make(chan struct{}),
|
||||||
done: make(chan struct{}),
|
done: make(chan struct{}),
|
||||||
@@ -160,7 +168,7 @@ func (f *ASNFetcher) recordSuccess() {
|
|||||||
f.consecutiveFails = 0
|
f.consecutiveFails = 0
|
||||||
|
|
||||||
// Decrease interval by half, but not below minimum
|
// Decrease interval by half, but not below minimum
|
||||||
newInterval := max(f.currentInterval/backoffMultiplier, minInterval)
|
newInterval := max(f.currentInterval/successDivisor, minInterval)
|
||||||
|
|
||||||
if newInterval != f.currentInterval {
|
if newInterval != f.currentInterval {
|
||||||
f.logger.Debug("Decreased fetch interval",
|
f.logger.Debug("Decreased fetch interval",
|
||||||
@@ -184,8 +192,9 @@ func (f *ASNFetcher) recordFailure() {
|
|||||||
f.intervalMu.Lock()
|
f.intervalMu.Lock()
|
||||||
f.consecutiveFails++
|
f.consecutiveFails++
|
||||||
|
|
||||||
// Exponential backoff: multiply by 2, capped at max
|
// Exponential backoff: multiply by 1.5, capped at max
|
||||||
newInterval := min(f.currentInterval*backoffMultiplier, maxInterval)
|
newInterval := min(
|
||||||
|
f.currentInterval*backoffNumerator/backoffDenominator, maxInterval)
|
||||||
|
|
||||||
if newInterval != f.currentInterval {
|
if newInterval != f.currentInterval {
|
||||||
f.logger.Debug("Increased fetch interval due to failure",
|
f.logger.Debug("Increased fetch interval due to failure",
|
||||||
|
|||||||
@@ -120,17 +120,58 @@ func TestFetchAndUpdateKeepsTheRowWhenTheLookupFails(t *testing.T) {
|
|||||||
"was stored", stats.NeverFetched)
|
"was stored", stats.NeverFetched)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
want := baseInterval * backoffNumerator / backoffDenominator
|
||||||
|
|
||||||
got := f.GetStats()
|
got := f.GetStats()
|
||||||
if got.CurrentInterval != baseInterval*backoffMultiplier ||
|
if got.CurrentInterval != want || got.ConsecutiveFails != 1 {
|
||||||
got.ConsecutiveFails != 1 {
|
|
||||||
t.Errorf("interval %v after %d failures, want %v after 1",
|
t.Errorf("interval %v after %d failures, want %v after 1",
|
||||||
got.CurrentInterval, got.ConsecutiveFails,
|
got.CurrentInterval, got.ConsecutiveFails, want)
|
||||||
baseInterval*backoffMultiplier)
|
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestRecordStepsTheInterval has the fetcher fail from its base interval of
|
||||||
|
// 15 seconds, then succeed. Each failure makes the interval 1.5 times as
|
||||||
|
// long, up to one minute, and each success halves it, down to one second.
|
||||||
|
func TestRecordStepsTheInterval(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
f := NewASNFetcher(newMockStore(), logger.New().Logger)
|
||||||
|
|
||||||
|
for i, want := range []time.Duration{
|
||||||
|
22500 * time.Millisecond,
|
||||||
|
33750 * time.Millisecond,
|
||||||
|
50625 * time.Millisecond,
|
||||||
|
time.Minute,
|
||||||
|
time.Minute,
|
||||||
|
} {
|
||||||
|
f.recordFailure()
|
||||||
|
|
||||||
|
if got := f.getInterval(); got != want {
|
||||||
|
t.Errorf("interval after %d failures = %v, want %v",
|
||||||
|
i+1, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for i, want := range []time.Duration{
|
||||||
|
30 * time.Second,
|
||||||
|
15 * time.Second,
|
||||||
|
7500 * time.Millisecond,
|
||||||
|
3750 * time.Millisecond,
|
||||||
|
1875 * time.Millisecond,
|
||||||
|
time.Second,
|
||||||
|
time.Second,
|
||||||
|
} {
|
||||||
|
f.recordSuccess()
|
||||||
|
|
||||||
|
if got := f.getInterval(); got != want {
|
||||||
|
t.Errorf("interval after %d successes = %v, want %v",
|
||||||
|
i+1, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestFetchAndUpdateMarksAnASWithoutReferral looks up a private-use AS, which
|
// TestFetchAndUpdateMarksAnASWithoutReferral looks up a private-use AS, which
|
||||||
// IANA refers to no RIR. The AS must not be looked up again until it is
|
// IANA refers to no RIR. The AS must not be looked up again until it is
|
||||||
// stale, and the lookup counts as neither a success nor a failure.
|
// stale, and the lookup counts as neither a success nor a failure.
|
||||||
|
|||||||
+83
-32
@@ -1,7 +1,7 @@
|
|||||||
package server
|
package server
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"cmp"
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
@@ -9,6 +9,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"runtime"
|
"runtime"
|
||||||
|
"slices"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -16,6 +17,7 @@ import (
|
|||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/templates"
|
"git.eeqj.de/sneak/routewatch/internal/templates"
|
||||||
asinfo "git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
asinfo "git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
||||||
@@ -56,10 +58,18 @@ const (
|
|||||||
// Error messages an answer sends, in place of the error behind it where there
|
// Error messages an answer sends, in place of the error behind it where there
|
||||||
// is one.
|
// is one.
|
||||||
const (
|
const (
|
||||||
invalidPrefixMessage = "Invalid prefix"
|
invalidPrefixMessage = "Invalid prefix"
|
||||||
hostNotResolvedMessage = "could not resolve host name"
|
hostNotResolvedMessage = "could not resolve host name"
|
||||||
hostNotPublicMessage = "host name has no public address"
|
hostNotPublicMessage = "host name has no public address"
|
||||||
tooManyTargetsMessage = "Too many addresses or host names"
|
tooManyTargetsMessage = "Too many addresses or host names"
|
||||||
|
searchQueryMissingMessage = "Missing search query"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The AS name search reads the text it looks for from the URL query parameter
|
||||||
|
// asSearchQueryParam, and answers with at most maxASSearchResults matches.
|
||||||
|
const (
|
||||||
|
asSearchQueryParam = "q"
|
||||||
|
maxASSearchResults = 50
|
||||||
)
|
)
|
||||||
|
|
||||||
// IP versions as database.LiveRoute.IPVersion records them.
|
// IP versions as database.LiveRoute.IPVersion records them.
|
||||||
@@ -432,10 +442,10 @@ type GCStats struct {
|
|||||||
//
|
//
|
||||||
//nolint:tagliatelle // field names are the existing JSON API
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
type StreamStats struct {
|
type StreamStats struct {
|
||||||
Announcements uint64 `json:"announcements"`
|
Announcements uint64 `json:"announcements"`
|
||||||
Withdrawals uint64 `json:"withdrawals"`
|
Withdrawals uint64 `json:"withdrawals"`
|
||||||
RouteChurnPerSec float64 `json:"route_churn_per_sec"`
|
RouteChurnPerMin int `json:"route_churn_per_min"`
|
||||||
BGPPeerCount int `json:"bgp_peer_count"`
|
BGPPeerCount int `json:"bgp_peer_count"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// StatsResponse represents the API statistics response
|
// StatsResponse represents the API statistics response
|
||||||
@@ -536,7 +546,7 @@ func (s *Server) streamStats() StreamStats {
|
|||||||
return StreamStats{
|
return StreamStats{
|
||||||
Announcements: metricsTracker.GetAnnouncementCount(),
|
Announcements: metricsTracker.GetAnnouncementCount(),
|
||||||
Withdrawals: metricsTracker.GetWithdrawalCount(),
|
Withdrawals: metricsTracker.GetWithdrawalCount(),
|
||||||
RouteChurnPerSec: metricsTracker.GetChurnRate(),
|
RouteChurnPerMin: metricsTracker.GetChurnPerMinute(),
|
||||||
BGPPeerCount: metricsTracker.GetBGPPeerCount(),
|
BGPPeerCount: metricsTracker.GetBGPPeerCount(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1001,11 +1011,12 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones.
|
// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones. Neither
|
||||||
|
// group is nil, so that a JSON answer holds [] for an empty one and not null.
|
||||||
func splitRoutesByIPVersion(
|
func splitRoutesByIPVersion(
|
||||||
routes []database.LiveRoute,
|
routes []database.LiveRoute,
|
||||||
) ([]database.LiveRoute, []database.LiveRoute) {
|
) ([]database.LiveRoute, []database.LiveRoute) {
|
||||||
var ipv4Routes, ipv6Routes []database.LiveRoute
|
ipv4Routes, ipv6Routes := []database.LiveRoute{}, []database.LiveRoute{}
|
||||||
|
|
||||||
for _, route := range routes {
|
for _, route := range routes {
|
||||||
if route.IPVersion == ipVersionV4 {
|
if route.IPVersion == ipVersionV4 {
|
||||||
@@ -1018,24 +1029,15 @@ func splitRoutesByIPVersion(
|
|||||||
return ipv4Routes, ipv6Routes
|
return ipv4Routes, ipv6Routes
|
||||||
}
|
}
|
||||||
|
|
||||||
// sortRoutesByNetwork sorts routes by network address, then by mask length.
|
// sortRoutesByMaskLength sorts routes by mask length, shortest first, and
|
||||||
func sortRoutesByNetwork(routes []database.LiveRoute) {
|
// routes with the same mask length by network address.
|
||||||
sort.Slice(routes, func(i, j int) bool {
|
func sortRoutesByMaskLength(routes []database.LiveRoute) {
|
||||||
// Parse the prefixes to compare network addresses
|
slices.SortFunc(routes, func(a, b database.LiveRoute) int {
|
||||||
ipI, netI, _ := net.ParseCIDR(routes[i].Prefix)
|
prefixA, _ := netip.ParsePrefix(a.Prefix)
|
||||||
ipJ, netJ, _ := net.ParseCIDR(routes[j].Prefix)
|
prefixB, _ := netip.ParsePrefix(b.Prefix)
|
||||||
|
|
||||||
// Compare by network address first
|
return cmp.Or(cmp.Compare(prefixA.Bits(), prefixB.Bits()),
|
||||||
cmp := bytes.Compare(ipI.To16(), ipJ.To16())
|
prefixA.Addr().Compare(prefixB.Addr()))
|
||||||
if cmp != 0 {
|
|
||||||
return cmp < 0
|
|
||||||
}
|
|
||||||
|
|
||||||
// If network addresses are equal, compare by mask length
|
|
||||||
onesI, _ := netI.Mask.Size()
|
|
||||||
onesJ, _ := netJ.Mask.Size()
|
|
||||||
|
|
||||||
return onesI < onesJ
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1105,6 +1107,56 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ASSearchResponse is the answer to an AS name search: the text searched for,
|
||||||
|
// the number of ASes that match it, and the first maxASSearchResults of them in
|
||||||
|
// AS number order.
|
||||||
|
type ASSearchResponse struct {
|
||||||
|
Query string `json:"query"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
Results []asinfo.Info `json:"results"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleASSearch returns a handler that searches the AS list built into
|
||||||
|
// asinfo for the text in the q parameter, without its leading and trailing
|
||||||
|
// spaces. It answers 400 when no text is left.
|
||||||
|
func (s *Server) handleASSearch() http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
query := strings.TrimSpace(r.URL.Query().Get(asSearchQueryParam))
|
||||||
|
if query == "" {
|
||||||
|
s.writeJSONError(w, http.StatusBadRequest, searchQueryMissingMessage)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// AS 0 means no known origin and has no AS page, so the answer leaves
|
||||||
|
// it out. The matches come in AS number order, so AS 0 can only be the
|
||||||
|
// first. The search asks for one match more than the answer holds, to
|
||||||
|
// take its place.
|
||||||
|
results, total := asinfo.Search(query, maxASSearchResults+1)
|
||||||
|
if len(results) > 0 && results[0].ASN == ristypes.NoOrigin {
|
||||||
|
results = results[1:]
|
||||||
|
total--
|
||||||
|
}
|
||||||
|
|
||||||
|
results = results[:min(len(results), maxASSearchResults)]
|
||||||
|
|
||||||
|
// results is an empty list rather than nil when nothing matches, so
|
||||||
|
// that the answer holds [] and not null.
|
||||||
|
if results == nil {
|
||||||
|
results = []asinfo.Info{}
|
||||||
|
}
|
||||||
|
|
||||||
|
err := writeJSONSuccess(w, ASSearchResponse{
|
||||||
|
Query: query,
|
||||||
|
Total: total,
|
||||||
|
Results: results,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("Failed to encode AS search results", "error", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// handleASDetail returns a handler that serves the AS detail HTML page
|
// handleASDetail returns a handler that serves the AS detail HTML page
|
||||||
func (s *Server) handleASDetail() http.HandlerFunc {
|
func (s *Server) handleASDetail() http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
@@ -1152,9 +1204,8 @@ func (s *Server) handleASDetail() http.HandlerFunc {
|
|||||||
// Group prefixes by IP version
|
// Group prefixes by IP version
|
||||||
ipv4Prefixes, ipv6Prefixes := splitRoutesByIPVersion(prefixes)
|
ipv4Prefixes, ipv6Prefixes := splitRoutesByIPVersion(prefixes)
|
||||||
|
|
||||||
// Sort prefixes by network address
|
sortRoutesByMaskLength(ipv4Prefixes)
|
||||||
sortRoutesByNetwork(ipv4Prefixes)
|
sortRoutesByMaskLength(ipv6Prefixes)
|
||||||
sortRoutesByNetwork(ipv6Prefixes)
|
|
||||||
|
|
||||||
// Prepare template data
|
// Prepare template data
|
||||||
data := struct {
|
data := struct {
|
||||||
|
|||||||
@@ -19,6 +19,7 @@ import (
|
|||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
"git.eeqj.de/sneak/routewatch/pkg/asinfo"
|
||||||
)
|
)
|
||||||
@@ -26,6 +27,7 @@ import (
|
|||||||
// The prefix and peer addresses the prefix page tests store routes for.
|
// The prefix and peer addresses the prefix page tests store routes for.
|
||||||
const (
|
const (
|
||||||
testPrefixV4 = "192.0.2.0/24"
|
testPrefixV4 = "192.0.2.0/24"
|
||||||
|
testPrefixV6 = "2001:db8::/32"
|
||||||
testPeerV4 = "198.51.100.1"
|
testPeerV4 = "198.51.100.1"
|
||||||
testPeerV6 = "2001:db8::1"
|
testPeerV6 = "2001:db8::1"
|
||||||
)
|
)
|
||||||
@@ -193,8 +195,53 @@ func TestStatsHandlersAnswerFromMemory(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestStatsAnswerCarriesChurnPerMinute checks that the stream object of the
|
||||||
|
// /api/v1/stats answer holds route_churn_per_min as a whole number, and no
|
||||||
|
// longer route_churn_per_sec, the moving average it replaces
|
||||||
|
// (https://git.eeqj.de/sneak/routewatch/issues/160).
|
||||||
|
func TestStatsAnswerCarriesChurnPerMinute(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTestRoutes(t)
|
||||||
|
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.handleStats()(rec,
|
||||||
|
httptest.NewRequestWithContext(t.Context(), http.MethodGet, statsPath, nil))
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status %d, want %d; body %s", rec.Code, http.StatusOK, rec.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
var body struct {
|
||||||
|
Data struct {
|
||||||
|
Stream map[string]json.RawMessage `json:"stream"`
|
||||||
|
} `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err := json.Unmarshal(rec.Body.Bytes(), &body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decoding the answer: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
churn, ok := body.Data.Stream["route_churn_per_min"]
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("stream has no route_churn_per_min: %s", rec.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
var count int
|
||||||
|
|
||||||
|
err = json.Unmarshal(churn, &count)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("route_churn_per_min = %s, want a whole number: %v", churn, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, ok := body.Data.Stream["route_churn_per_sec"]; ok {
|
||||||
|
t.Errorf("stream still has route_churn_per_sec: %s", rec.Body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// newServerWithTestRoutes returns a server on a new database that holds a
|
// newServerWithTestRoutes returns a server on a new database that holds a
|
||||||
// route for testPrefixV4 and one for 2001:db8::/32.
|
// route for testPrefixV4 and one for testPrefixV6.
|
||||||
func newServerWithTestRoutes(t *testing.T) *Server {
|
func newServerWithTestRoutes(t *testing.T) *Server {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -214,7 +261,7 @@ func newServerWithTestRoutes(t *testing.T) *Server {
|
|||||||
NextHop: testPeerV4, LastUpdated: ts,
|
NextHop: testPeerV4, LastUpdated: ts,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
Prefix: "2001:db8::/32", MaskLength: 32, IPVersion: 6,
|
Prefix: testPrefixV6, MaskLength: 32, IPVersion: 6,
|
||||||
OriginASN: 64501, PeerIP: testPeerV6, ASPath: []int{64501},
|
OriginASN: 64501, PeerIP: testPeerV6, ASPath: []int{64501},
|
||||||
NextHop: testPeerV6, LastUpdated: ts,
|
NextHop: testPeerV6, LastUpdated: ts,
|
||||||
},
|
},
|
||||||
@@ -411,8 +458,8 @@ func TestPrefixDetailShowsOriginNames(t *testing.T) {
|
|||||||
// TestPagesSayOriginZeroIsNoKnownOrigin stores a route with origin 0, as the
|
// TestPagesSayOriginZeroIsNoKnownOrigin stores a route with origin 0, as the
|
||||||
// prefix handler stores a route whose AS path ends in an AS set of several
|
// prefix handler stores a route whose AS path ends in an AS set of several
|
||||||
// members, and checks that the prefix page and the prefix length page say it
|
// members, and checks that the prefix page and the prefix length page say it
|
||||||
// has no known origin, without a link to AS0 or the description of the
|
// has no known origin, without a link to AS0
|
||||||
// reserved AS 0 (https://git.eeqj.de/sneak/routewatch/issues/75).
|
// (https://git.eeqj.de/sneak/routewatch/issues/75).
|
||||||
func TestPagesSayOriginZeroIsNoKnownOrigin(t *testing.T) {
|
func TestPagesSayOriginZeroIsNoKnownOrigin(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -432,11 +479,6 @@ func TestPagesSayOriginZeroIsNoKnownOrigin(t *testing.T) {
|
|||||||
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
reserved, ok := asinfo.Get(0)
|
|
||||||
if !ok {
|
|
||||||
t.Fatal("AS0 is not in the AS info data")
|
|
||||||
}
|
|
||||||
|
|
||||||
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
|
|
||||||
for _, path := range []string{"/prefix/" + testPrefixV4, prefixLengthPagePath} {
|
for _, path := range []string{"/prefix/" + testPrefixV4, prefixLengthPagePath} {
|
||||||
@@ -458,9 +500,70 @@ func TestPagesSayOriginZeroIsNoKnownOrigin(t *testing.T) {
|
|||||||
if strings.Contains(page, `href="/as/0"`) {
|
if strings.Contains(page, `href="/as/0"`) {
|
||||||
t.Errorf("%s links to AS0", path)
|
t.Errorf("%s links to AS0", path)
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if strings.Contains(page, html.EscapeString(reserved.Description)) {
|
// TestPagesShowAgesInWords stores routes last updated about 5 minutes, 3 hours
|
||||||
t.Errorf("%s shows AS0's description %q", path, reserved.Description)
|
// and 15 days ago, and checks that the AS page and the prefix pages show each
|
||||||
|
// age in words. Before the fix the ages read "5m0s ago", "3h0m0s ago" and
|
||||||
|
// "360h0m0s ago" (https://git.eeqj.de/sneak/routewatch/issues/163).
|
||||||
|
func TestPagesShowAgesInWords(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
db, err := database.New(&config.Config{StateDir: t.TempDir()}, logger.New())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("database.New: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
|
// Each age is well past a whole number of its unit, so the time the test
|
||||||
|
// takes to run does not change the words.
|
||||||
|
ages := []struct {
|
||||||
|
prefix string
|
||||||
|
age time.Duration
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{testPrefixV4, 5*time.Minute + 30*time.Second, "5 minutes ago"},
|
||||||
|
{"198.51.100.0/24", 3*time.Hour + 30*time.Minute, "3 hours ago"},
|
||||||
|
{"203.0.113.0/24", 15 * 24 * time.Hour, "2 weeks ago"},
|
||||||
|
}
|
||||||
|
|
||||||
|
now := time.Now().UTC()
|
||||||
|
routes := make([]*database.LiveRoute, 0, len(ages))
|
||||||
|
|
||||||
|
for _, a := range ages {
|
||||||
|
routes = append(routes, &database.LiveRoute{
|
||||||
|
Prefix: a.prefix, MaskLength: 24, IPVersion: 4,
|
||||||
|
OriginASN: 64500, PeerIP: testPeerV4, ASPath: []int{64500},
|
||||||
|
NextHop: testPeerV4, LastUpdated: now.Add(-a.age),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
err = db.UpsertLiveRouteBatch(routes)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s := New(db, streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
|
|
||||||
|
for _, a := range ages {
|
||||||
|
cell := `<td class="age">` + a.want + `</td>`
|
||||||
|
|
||||||
|
for _, path := range []string{asPagePath, "/prefix/" + a.prefix} {
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, path, nil))
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("%s: status %d, want %d; body %s",
|
||||||
|
path, rec.Code, http.StatusOK, rec.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !strings.Contains(rec.Body.String(), cell) {
|
||||||
|
t.Errorf("%s does not show the age of the route for %s as %q",
|
||||||
|
path, a.prefix, a.want)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1166,6 +1269,412 @@ func TestIPLookupHidesPrivateAddresses(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// asSearchPath is the path of the AS name search.
|
||||||
|
const asSearchPath = "/api/v1/search/as"
|
||||||
|
|
||||||
|
// searchAS sends an AS name search with the URL query rawQuery through s's
|
||||||
|
// router, expects a 200 answer, and returns its data.
|
||||||
|
func searchAS(t *testing.T, s *Server, rawQuery string) ASSearchResponse {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, asSearchPath+"?"+rawQuery, nil))
|
||||||
|
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("%s: status %d, want %d; body %s",
|
||||||
|
rawQuery, rec.Code, http.StatusOK, rec.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
var body struct {
|
||||||
|
Data ASSearchResponse `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err := json.Unmarshal(rec.Body.Bytes(), &body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: decoding the answer %q: %v", rawQuery, rec.Body, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return body.Data
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASSearchAnswersMatches searches for Cloudflare's handle in lower case,
|
||||||
|
// with spaces around it, and expects AS13335 among the matches. Before the fix
|
||||||
|
// there was no AS name search, and the home page's search card said it was
|
||||||
|
// coming soon (https://git.eeqj.de/sneak/routewatch/issues/151).
|
||||||
|
func TestASSearchAnswersMatches(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const query = "cloudflarenet"
|
||||||
|
|
||||||
|
data := searchAS(t, newServerWithTestRoutes(t), "q=%20"+query+"%20")
|
||||||
|
|
||||||
|
if data.Query != query {
|
||||||
|
t.Errorf("query %q, want %q", data.Query, query)
|
||||||
|
}
|
||||||
|
|
||||||
|
if data.Total != len(data.Results) {
|
||||||
|
t.Errorf("total %d, but %d results", data.Total, len(data.Results))
|
||||||
|
}
|
||||||
|
|
||||||
|
cloudflare := asinfo.Info{
|
||||||
|
ASN: 13335, Handle: "CLOUDFLARENET", Description: "Cloudflare Inc.",
|
||||||
|
}
|
||||||
|
if !slices.Contains(data.Results, cloudflare) {
|
||||||
|
t.Errorf("results %v do not hold %v", data.Results, cloudflare)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASSearchAnswersEmptyListWhenNothingMatches expects a search that matches
|
||||||
|
// nothing to answer 200 with a total of 0 and an empty list, not null.
|
||||||
|
func TestASSearchAnswersEmptyListWhenNothingMatches(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
data := searchAS(t, newServerWithTestRoutes(t), "q=XYZABC123456")
|
||||||
|
|
||||||
|
if data.Total != 0 || data.Results == nil || len(data.Results) != 0 {
|
||||||
|
t.Errorf("total %d and results %#v, want 0 and an empty list",
|
||||||
|
data.Total, data.Results)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASSearchAnswers400ForEmptyQuery expects a search with no q, an empty q
|
||||||
|
// or a q of only spaces to answer 400 with the JSON error body.
|
||||||
|
func TestASSearchAnswers400ForEmptyQuery(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTestRoutes(t)
|
||||||
|
|
||||||
|
for _, path := range []string{
|
||||||
|
asSearchPath, asSearchPath + "?q=", asSearchPath + "?q=%20%20",
|
||||||
|
} {
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, path, nil))
|
||||||
|
|
||||||
|
if rec.Code != http.StatusBadRequest {
|
||||||
|
t.Errorf("%s: status %d, want %d; body %s",
|
||||||
|
path, rec.Code, http.StatusBadRequest, rec.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
checkJSONErrorCode(t, path, rec, http.StatusBadRequest)
|
||||||
|
checkJSONErrorMessage(t, path, rec, searchQueryMissingMessage)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASSearchShowsFirstMatchesAndTotal searches for a word with more matches
|
||||||
|
// than an answer holds, and expects the first maxASSearchResults of them, in
|
||||||
|
// AS number order, and the number of all of them.
|
||||||
|
func TestASSearchShowsFirstMatchesAndTotal(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const query = "university"
|
||||||
|
|
||||||
|
all, _ := asinfo.Search(query, asinfo.Total())
|
||||||
|
if len(all) <= maxASSearchResults {
|
||||||
|
t.Fatalf("Search(%q) has %d matches; the test needs more than %d",
|
||||||
|
query, len(all), maxASSearchResults)
|
||||||
|
}
|
||||||
|
|
||||||
|
data := searchAS(t, newServerWithTestRoutes(t), "q="+query)
|
||||||
|
|
||||||
|
if data.Total != len(all) {
|
||||||
|
t.Errorf("total %d, want %d", data.Total, len(all))
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(data.Results, all[:maxASSearchResults]) {
|
||||||
|
t.Errorf("results %v, want the first %d matches %v",
|
||||||
|
data.Results, maxASSearchResults, all[:maxASSearchResults])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The AS numbers the AS detail tests ask for. storedASN originates the route
|
||||||
|
// for testPrefixV4 and is in the database. routeOnlyASN originates the route
|
||||||
|
// for testPrefixV6, but the database has no record of the AS itself, as when
|
||||||
|
// a route is stored before its AS. Both are documentation AS numbers, which
|
||||||
|
// the AS list built into asinfo does not hold. builtInOnlyASN is in that list
|
||||||
|
// but not in the database. unknownASN is in neither.
|
||||||
|
const (
|
||||||
|
storedASN = 64500
|
||||||
|
routeOnlyASN = 64501
|
||||||
|
builtInOnlyASN = 15169
|
||||||
|
unknownASN = 4200000000
|
||||||
|
)
|
||||||
|
|
||||||
|
// zeroDate is how a page writes a time that was never set.
|
||||||
|
const zeroDate = "0001-01-01"
|
||||||
|
|
||||||
|
// firstSeenCard is the label of the AS page's first-seen date.
|
||||||
|
const firstSeenCard = `<div class="info-label">First Seen</div>`
|
||||||
|
|
||||||
|
// newServerWithStoredAS returns newServerWithTestRoutes's server with storedASN
|
||||||
|
// added to the database. It fails the test when the AS list built into asinfo
|
||||||
|
// lacks builtInOnlyASN or holds unknownASN.
|
||||||
|
func newServerWithStoredAS(t *testing.T) *Server {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if _, ok := asinfo.Get(builtInOnlyASN); !ok {
|
||||||
|
t.Fatalf("AS%d is not in the AS info data", builtInOnlyASN)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, ok := asinfo.Get(unknownASN); ok {
|
||||||
|
t.Fatalf("AS%d is in the AS info data", unknownASN)
|
||||||
|
}
|
||||||
|
|
||||||
|
s := newServerWithTestRoutes(t)
|
||||||
|
|
||||||
|
_, err := s.db.GetOrCreateASN(storedASN, time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetOrCreateASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASDetailJSONAnswersBuiltInAS asks /api/v1/as/ for each of the ASes above.
|
||||||
|
// All but unknownASN answer 200 with the handle and description the built-in
|
||||||
|
// AS list has for them, if any, and their prefixes, as lists even when empty.
|
||||||
|
// Only storedASN has a first_seen time. unknownASN answers 404. Before the fix
|
||||||
|
// builtInOnlyASN and routeOnlyASN answered 404 too
|
||||||
|
// (https://git.eeqj.de/sneak/routewatch/issues/151).
|
||||||
|
func TestASDetailJSONAnswersBuiltInAS(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithStoredAS(t)
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
asn int
|
||||||
|
wantCode int
|
||||||
|
wantPrefixes []string
|
||||||
|
wantFirstSeen bool
|
||||||
|
}{
|
||||||
|
{storedASN, http.StatusOK, []string{testPrefixV4}, true},
|
||||||
|
{routeOnlyASN, http.StatusOK, []string{testPrefixV6}, false},
|
||||||
|
{builtInOnlyASN, http.StatusOK, nil, false},
|
||||||
|
{unknownASN, http.StatusNotFound, nil, false},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
path := "/api/v1/as/" + strconv.Itoa(tt.asn)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, path, nil))
|
||||||
|
|
||||||
|
if rec.Code != tt.wantCode {
|
||||||
|
t.Errorf("%s: status %d, want %d; body %s",
|
||||||
|
path, rec.Code, tt.wantCode, rec.Body)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if tt.wantCode != http.StatusOK {
|
||||||
|
checkJSONErrorCode(t, path, rec, tt.wantCode)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
|
var body struct {
|
||||||
|
Data struct {
|
||||||
|
ASN database.ASN `json:"asn"`
|
||||||
|
IPv4Prefixes []database.LiveRoute `json:"ipv4_prefixes"`
|
||||||
|
IPv6Prefixes []database.LiveRoute `json:"ipv6_prefixes"`
|
||||||
|
} `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err := json.Unmarshal(rec.Body.Bytes(), &body)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("%s: decoding the answer %q: %v", path, rec.Body, err)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
handle, description := asinfo.GetHandle(tt.asn), asinfo.GetDescription(tt.asn)
|
||||||
|
got := body.Data.ASN
|
||||||
|
|
||||||
|
if got.ASN != tt.asn || got.Handle != handle ||
|
||||||
|
got.Description != description {
|
||||||
|
t.Errorf("%s: AS %d %q %q, want %d %q %q", path, got.ASN,
|
||||||
|
got.Handle, got.Description, tt.asn, handle, description)
|
||||||
|
}
|
||||||
|
|
||||||
|
if body.Data.IPv4Prefixes == nil || body.Data.IPv6Prefixes == nil {
|
||||||
|
t.Errorf("%s: a prefix list is null; body %s", path, rec.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
routes := slices.Concat(body.Data.IPv4Prefixes, body.Data.IPv6Prefixes)
|
||||||
|
|
||||||
|
gotPrefixes := make([]string, 0, len(routes))
|
||||||
|
for _, route := range routes {
|
||||||
|
gotPrefixes = append(gotPrefixes, route.Prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(gotPrefixes, tt.wantPrefixes) {
|
||||||
|
t.Errorf("%s: prefixes %v, want %v", path, gotPrefixes, tt.wantPrefixes)
|
||||||
|
}
|
||||||
|
|
||||||
|
gotFirstSeen := strings.Contains(rec.Body.String(), `"first_seen"`)
|
||||||
|
if gotFirstSeen != tt.wantFirstSeen {
|
||||||
|
t.Errorf("%s: has first_seen %t, want %t; body %s",
|
||||||
|
path, gotFirstSeen, tt.wantFirstSeen, rec.Body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASDetailPageShowsBuiltInAS asks /as/ for each of the ASes above. All but
|
||||||
|
// unknownASN answer 200 with the handle and description the built-in AS list
|
||||||
|
// has for them, if any. storedASN and routeOnlyASN list their prefixes, and
|
||||||
|
// builtInOnlyASN says the AS announces none. Only storedASN shows a first-seen
|
||||||
|
// date, and no page shows a date that was never set. unknownASN answers 404.
|
||||||
|
// Before the fix builtInOnlyASN and routeOnlyASN answered 404 too
|
||||||
|
// (https://git.eeqj.de/sneak/routewatch/issues/151).
|
||||||
|
func TestASDetailPageShowsBuiltInAS(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithStoredAS(t)
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
asn int
|
||||||
|
wantCode int
|
||||||
|
wantText string
|
||||||
|
wantFirstSeen bool
|
||||||
|
}{
|
||||||
|
{storedASN, http.StatusOK, testPrefixV4, true},
|
||||||
|
{routeOnlyASN, http.StatusOK, testPrefixV6, false},
|
||||||
|
{builtInOnlyASN, http.StatusOK, "No prefixes announced by this AS", false},
|
||||||
|
{unknownASN, http.StatusNotFound, "", false},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
path := "/as/" + strconv.Itoa(tt.asn)
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, path, nil))
|
||||||
|
|
||||||
|
if rec.Code != tt.wantCode {
|
||||||
|
t.Errorf("%s: status %d, want %d; body %s",
|
||||||
|
path, rec.Code, tt.wantCode, rec.Body)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if tt.wantCode != http.StatusOK {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
page := rec.Body.String()
|
||||||
|
|
||||||
|
for _, want := range []string{
|
||||||
|
html.EscapeString(asinfo.GetHandle(tt.asn)),
|
||||||
|
html.EscapeString(asinfo.GetDescription(tt.asn)),
|
||||||
|
tt.wantText,
|
||||||
|
} {
|
||||||
|
if !strings.Contains(page, want) {
|
||||||
|
t.Errorf("%s: page does not show %q", path, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.Contains(page, zeroDate) {
|
||||||
|
t.Errorf("%s: page shows the date %s", path, zeroDate)
|
||||||
|
}
|
||||||
|
|
||||||
|
gotFirstSeen := strings.Contains(page, firstSeenCard)
|
||||||
|
if gotFirstSeen != tt.wantFirstSeen {
|
||||||
|
t.Errorf("%s: shows a first-seen date %t, want %t",
|
||||||
|
path, gotFirstSeen, tt.wantFirstSeen)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSortRoutesByMaskLength expects the AS page's prefix lists in mask length
|
||||||
|
// order, shortest first, and in address order within one mask length. Before
|
||||||
|
// the fix they were in address order, so 10.1.0.0/16 came before 11.0.0.0/8
|
||||||
|
// (https://git.eeqj.de/sneak/routewatch/issues/153).
|
||||||
|
func TestSortRoutesByMaskLength(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
prefixes []string
|
||||||
|
want []string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "IPv4",
|
||||||
|
prefixes: []string{
|
||||||
|
"100.64.1.0/24", "11.0.0.0/8", "10.1.0.0/16",
|
||||||
|
"20.0.1.0/24", "172.16.0.0/12", "9.0.0.0/8",
|
||||||
|
},
|
||||||
|
want: []string{
|
||||||
|
"9.0.0.0/8", "11.0.0.0/8", "172.16.0.0/12",
|
||||||
|
"10.1.0.0/16", "20.0.1.0/24", "100.64.1.0/24",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "IPv6",
|
||||||
|
prefixes: []string{
|
||||||
|
"2001:db8:1::/48", "2001:db8:8000::/33", "2001:db8:0:1::/64",
|
||||||
|
"2001:db8::/48", "2001:db8::/33",
|
||||||
|
},
|
||||||
|
want: []string{
|
||||||
|
"2001:db8::/33", "2001:db8:8000::/33", "2001:db8::/48",
|
||||||
|
"2001:db8:1::/48", "2001:db8:0:1::/64",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
routes := make([]database.LiveRoute, 0, len(tt.prefixes))
|
||||||
|
for _, prefix := range tt.prefixes {
|
||||||
|
routes = append(routes, database.LiveRoute{Prefix: prefix})
|
||||||
|
}
|
||||||
|
|
||||||
|
sortRoutesByMaskLength(routes)
|
||||||
|
|
||||||
|
got := make([]string, 0, len(routes))
|
||||||
|
for _, route := range routes {
|
||||||
|
got = append(got, route.Prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(got, tt.want) {
|
||||||
|
t.Errorf("%s: order %v, want %v", tt.name, got, tt.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestNoOriginHasNoASAnswers stores a route with no known origin, which has
|
||||||
|
// origin AS 0 (https://git.eeqj.de/sneak/routewatch/issues/75). It expects
|
||||||
|
// /as/0 and /api/v1/as/0 to answer 404.
|
||||||
|
func TestNoOriginHasNoASAnswers(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const noOriginPrefix = "203.0.113.0/24"
|
||||||
|
|
||||||
|
s := newServerWithTestRoutes(t)
|
||||||
|
|
||||||
|
err := s.db.UpsertLiveRouteBatch([]*database.LiveRoute{{
|
||||||
|
Prefix: noOriginPrefix, MaskLength: 24, IPVersion: 4,
|
||||||
|
OriginASN: ristypes.NoOrigin, PeerIP: testPeerV4,
|
||||||
|
ASPath: []int{64500, 64501, 64502}, NextHop: testPeerV4,
|
||||||
|
LastUpdated: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC),
|
||||||
|
}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UpsertLiveRouteBatch: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
asn := strconv.Itoa(ristypes.NoOrigin)
|
||||||
|
|
||||||
|
for _, path := range []string{"/as/" + asn, "/api/v1/as/" + asn} {
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
|
t.Context(), http.MethodGet, path, nil))
|
||||||
|
|
||||||
|
if rec.Code != http.StatusNotFound {
|
||||||
|
t.Errorf("%s: status %d, want %d; body %s",
|
||||||
|
path, rec.Code, http.StatusNotFound, rec.Body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// settledGoroutineCount lets transient goroutines finish, then reports the
|
// settledGoroutineCount lets transient goroutines finish, then reports the
|
||||||
// current count.
|
// current count.
|
||||||
func settledGoroutineCount() int {
|
func settledGoroutineCount() int {
|
||||||
|
|||||||
@@ -87,6 +87,7 @@ func (s *Server) setupRoutes() {
|
|||||||
r.Get("/ip/{ip}", s.handleIPLookup())
|
r.Get("/ip/{ip}", s.handleIPLookup())
|
||||||
r.Get("/as/{asn}", s.handleASDetailJSON())
|
r.Get("/as/{asn}", s.handleASDetailJSON())
|
||||||
r.Get("/prefix/{prefix}/{len}", s.handlePrefixDetailJSON())
|
r.Get("/prefix/{prefix}/{len}", s.handlePrefixDetailJSON())
|
||||||
|
r.Get("/search/as", s.handleASSearch())
|
||||||
})
|
})
|
||||||
|
|
||||||
s.router = r
|
s.router = r
|
||||||
|
|||||||
@@ -1,4 +1,8 @@
|
|||||||
// Package server provides HTTP endpoints for status monitoring and statistics
|
// Package server serves RouteWatch's web pages and JSON API:
|
||||||
|
// - the home page, the status figures and the health check
|
||||||
|
// - the AS, prefix and prefix length pages, and the JSON answers behind them
|
||||||
|
// - the IP lookups
|
||||||
|
// - the AS name search
|
||||||
package server
|
package server
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
|||||||
@@ -126,6 +126,26 @@
|
|||||||
margin: 0;
|
margin: 0;
|
||||||
color: #2c3e50;
|
color: #2c3e50;
|
||||||
}
|
}
|
||||||
|
/* A click on a prefix list's heading folds or unfolds the list. Not
|
||||||
|
every browser shows its own marker on a flex element, so the page
|
||||||
|
hides it and draws a triangle before the heading. The triangle
|
||||||
|
points down while the list is open. */
|
||||||
|
summary.prefix-header {
|
||||||
|
cursor: pointer;
|
||||||
|
list-style: none;
|
||||||
|
}
|
||||||
|
summary.prefix-header::-webkit-details-marker {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
summary.prefix-header h2::before {
|
||||||
|
content: "\25B8";
|
||||||
|
display: inline-block;
|
||||||
|
margin-right: 10px;
|
||||||
|
transition: transform 0.2s;
|
||||||
|
}
|
||||||
|
details[open] > summary.prefix-header h2::before {
|
||||||
|
transform: rotate(90deg);
|
||||||
|
}
|
||||||
.prefix-count {
|
.prefix-count {
|
||||||
background: #e74c3c;
|
background: #e74c3c;
|
||||||
color: white;
|
color: white;
|
||||||
@@ -233,18 +253,20 @@
|
|||||||
<div class="info-label">Peer ASNs</div>
|
<div class="info-label">Peer ASNs</div>
|
||||||
<div class="info-value">{{.PeerCount}}</div>
|
<div class="info-value">{{.PeerCount}}</div>
|
||||||
</div>
|
</div>
|
||||||
|
{{if not .ASN.FirstSeen.IsZero}}
|
||||||
<div class="info-card">
|
<div class="info-card">
|
||||||
<div class="info-label">First Seen</div>
|
<div class="info-label">First Seen</div>
|
||||||
<div class="info-value">{{.ASN.FirstSeen.Format "2006-01-02"}}</div>
|
<div class="info-value">{{.ASN.FirstSeen.Format "2006-01-02"}}</div>
|
||||||
</div>
|
</div>
|
||||||
|
{{end}}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
{{if .IPv4Prefixes}}
|
{{if .IPv4Prefixes}}
|
||||||
<div class="prefix-section">
|
<details class="prefix-section" open>
|
||||||
<div class="prefix-header">
|
<summary class="prefix-header">
|
||||||
<h2>IPv4 Prefixes</h2>
|
<h2>IPv4 Prefixes</h2>
|
||||||
<span class="prefix-count">{{.IPv4Count}}</span>
|
<span class="prefix-count">{{.IPv4Count}}</span>
|
||||||
</div>
|
</summary>
|
||||||
<table class="prefix-table">
|
<table class="prefix-table">
|
||||||
<thead>
|
<thead>
|
||||||
<tr>
|
<tr>
|
||||||
@@ -265,15 +287,15 @@
|
|||||||
{{end}}
|
{{end}}
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
</div>
|
</details>
|
||||||
{{end}}
|
{{end}}
|
||||||
|
|
||||||
{{if .IPv6Prefixes}}
|
{{if .IPv6Prefixes}}
|
||||||
<div class="prefix-section">
|
<details class="prefix-section" open>
|
||||||
<div class="prefix-header">
|
<summary class="prefix-header">
|
||||||
<h2>IPv6 Prefixes</h2>
|
<h2>IPv6 Prefixes</h2>
|
||||||
<span class="prefix-count">{{.IPv6Count}}</span>
|
<span class="prefix-count">{{.IPv6Count}}</span>
|
||||||
</div>
|
</summary>
|
||||||
<table class="prefix-table">
|
<table class="prefix-table">
|
||||||
<thead>
|
<thead>
|
||||||
<tr>
|
<tr>
|
||||||
@@ -294,7 +316,7 @@
|
|||||||
{{end}}
|
{{end}}
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
</div>
|
</details>
|
||||||
{{end}}
|
{{end}}
|
||||||
|
|
||||||
{{if eq .TotalCount 0}}
|
{{if eq .TotalCount 0}}
|
||||||
|
|||||||
+157
-16
@@ -107,9 +107,15 @@
|
|||||||
.stat-card.connected .stat-value {
|
.stat-card.connected .stat-value {
|
||||||
color: #27ae60;
|
color: #27ae60;
|
||||||
}
|
}
|
||||||
.stat-card.disconnected .stat-value {
|
.stat-card.disconnected .stat-value,
|
||||||
|
.stale {
|
||||||
color: #e74c3c;
|
color: #e74c3c;
|
||||||
}
|
}
|
||||||
|
/* The card has no room beside "Connected", so the spinner and "stale" go under it */
|
||||||
|
#freshness {
|
||||||
|
display: block;
|
||||||
|
font-size: 20px;
|
||||||
|
}
|
||||||
|
|
||||||
/* Search section */
|
/* Search section */
|
||||||
.search-section {
|
.search-section {
|
||||||
@@ -174,6 +180,38 @@
|
|||||||
#asname-results .search-message {
|
#asname-results .search-message {
|
||||||
font-size: 13px;
|
font-size: 13px;
|
||||||
}
|
}
|
||||||
|
#asname-results .error {
|
||||||
|
color: #c00;
|
||||||
|
}
|
||||||
|
.asname-list {
|
||||||
|
list-style: none;
|
||||||
|
margin: 12px 0 0 0;
|
||||||
|
padding: 0;
|
||||||
|
max-height: 400px;
|
||||||
|
overflow-y: auto;
|
||||||
|
}
|
||||||
|
.asname-list li {
|
||||||
|
border-bottom: 1px solid #eee;
|
||||||
|
}
|
||||||
|
.asname-list a {
|
||||||
|
display: block;
|
||||||
|
padding: 8px 4px;
|
||||||
|
color: #2c3e50;
|
||||||
|
text-decoration: none;
|
||||||
|
font-size: 14px;
|
||||||
|
}
|
||||||
|
.asname-list a:hover {
|
||||||
|
background: #f5f5f5;
|
||||||
|
}
|
||||||
|
.asname-list .asn {
|
||||||
|
font-weight: 600;
|
||||||
|
color: #3498db;
|
||||||
|
}
|
||||||
|
.asname-list .description {
|
||||||
|
display: block;
|
||||||
|
color: #7f8c8d;
|
||||||
|
font-size: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
/* IP Lookup result */
|
/* IP Lookup result */
|
||||||
.ip-result {
|
.ip-result {
|
||||||
@@ -265,7 +303,10 @@
|
|||||||
<main class="main-content">
|
<main class="main-content">
|
||||||
<div class="stats-grid">
|
<div class="stats-grid">
|
||||||
<div class="stat-card" id="status-card">
|
<div class="stat-card" id="status-card">
|
||||||
<div class="stat-value" id="stat-status">-</div>
|
<div class="stat-value">
|
||||||
|
<span id="stat-status">-</span>
|
||||||
|
<span id="freshness"></span>
|
||||||
|
</div>
|
||||||
<div class="stat-label">Status</div>
|
<div class="stat-label">Status</div>
|
||||||
</div>
|
</div>
|
||||||
<div class="stat-card">
|
<div class="stat-card">
|
||||||
@@ -306,7 +347,7 @@
|
|||||||
<input type="text" id="asname-input" placeholder="e.g., Google, Cloudflare" autocomplete="off">
|
<input type="text" id="asname-input" placeholder="e.g., Google, Cloudflare" autocomplete="off">
|
||||||
<button type="submit">Search</button>
|
<button type="submit">Search</button>
|
||||||
</form>
|
</form>
|
||||||
<p class="search-hint">Search for autonomous systems by organization name</p>
|
<p class="search-hint">Search for autonomous systems by organization name or handle</p>
|
||||||
<div id="asname-results"></div>
|
<div id="asname-results"></div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -337,6 +378,44 @@
|
|||||||
</footer>
|
</footer>
|
||||||
|
|
||||||
<script nonce="{{.CSPNonce}}">
|
<script nonce="{{.CSPNonce}}">
|
||||||
|
// The stats are fetched every REFRESH_PERIOD_MS. Next to the status, a
|
||||||
|
// spinner turns one frame each time an answer counts more messages
|
||||||
|
// than the one before. After STALE_AFTER_REFRESHES refreshes without
|
||||||
|
// such an answer, the spinner gives way to "stale" and the time since.
|
||||||
|
const REFRESH_PERIOD_MS = 5000;
|
||||||
|
const STALE_AFTER_REFRESHES = 3;
|
||||||
|
const MS_PER_SECOND = 1000;
|
||||||
|
const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'];
|
||||||
|
const SPINNER_COLOURS = ['#e74c3c', '#e67e22', '#f1c40f', '#27ae60', '#3498db', '#9b59b6'];
|
||||||
|
|
||||||
|
let spinnerFrame = 0;
|
||||||
|
let lastMessageCount = null;
|
||||||
|
let lastFreshAt = Date.now();
|
||||||
|
|
||||||
|
// Turn the spinner when the message count has risen since the last answer.
|
||||||
|
function noteMessageCount(count) {
|
||||||
|
if (lastMessageCount !== null && count > lastMessageCount) {
|
||||||
|
spinnerFrame = (spinnerFrame + 1) % SPINNER_FRAMES.length;
|
||||||
|
lastFreshAt = Date.now();
|
||||||
|
}
|
||||||
|
lastMessageCount = count;
|
||||||
|
showFreshness();
|
||||||
|
}
|
||||||
|
|
||||||
|
function showFreshness() {
|
||||||
|
const freshness = document.getElementById('freshness');
|
||||||
|
const sinceFreshMs = Date.now() - lastFreshAt;
|
||||||
|
const stale = sinceFreshMs >= STALE_AFTER_REFRESHES * REFRESH_PERIOD_MS;
|
||||||
|
freshness.classList.toggle('stale', stale);
|
||||||
|
if (stale) {
|
||||||
|
freshness.textContent = 'stale for ' + Math.floor(sinceFreshMs / MS_PER_SECOND) + 's';
|
||||||
|
freshness.style.color = '';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
freshness.textContent = SPINNER_FRAMES[spinnerFrame];
|
||||||
|
freshness.style.color = SPINNER_COLOURS[spinnerFrame % SPINNER_COLOURS.length];
|
||||||
|
}
|
||||||
|
|
||||||
function formatNumber(num) {
|
function formatNumber(num) {
|
||||||
if (num >= 1000000) {
|
if (num >= 1000000) {
|
||||||
return (num / 1000000).toFixed(1) + 'M';
|
return (num / 1000000).toFixed(1) + 'M';
|
||||||
@@ -358,6 +437,7 @@
|
|||||||
const statusEl = document.getElementById('stat-status');
|
const statusEl = document.getElementById('stat-status');
|
||||||
statusEl.textContent = data.connected ? 'Connected' : 'Disconnected';
|
statusEl.textContent = data.connected ? 'Connected' : 'Disconnected';
|
||||||
statusCard.className = 'stat-card ' + (data.connected ? 'connected' : 'disconnected');
|
statusCard.className = 'stat-card ' + (data.connected ? 'connected' : 'disconnected');
|
||||||
|
noteMessageCount(data.total_messages);
|
||||||
|
|
||||||
document.getElementById('stat-routes').textContent = formatNumber(data.live_routes);
|
document.getElementById('stat-routes').textContent = formatNumber(data.live_routes);
|
||||||
document.getElementById('stat-asns').textContent = formatNumber(data.asns);
|
document.getElementById('stat-asns').textContent = formatNumber(data.asns);
|
||||||
@@ -368,7 +448,7 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
const totalUpdates = data.ipv4_updates_per_sec + data.ipv6_updates_per_sec;
|
const totalUpdates = data.ipv4_updates_per_sec + data.ipv6_updates_per_sec;
|
||||||
document.getElementById('stat-updates').textContent = totalUpdates.toFixed(1);
|
document.getElementById('stat-updates').textContent = Math.round(totalUpdates).toLocaleString();
|
||||||
})
|
})
|
||||||
.catch(() => {
|
.catch(() => {
|
||||||
document.getElementById('stat-status').textContent = 'Error';
|
document.getElementById('stat-status').textContent = 'Error';
|
||||||
@@ -387,24 +467,82 @@
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// AS name search
|
// AS name search. The handles and descriptions come from outside data,
|
||||||
|
// so the results are built with textContent, never innerHTML.
|
||||||
|
const asNameResults = document.getElementById('asname-results');
|
||||||
|
|
||||||
|
// Replace the search results with one line of text.
|
||||||
|
function showASNameMessage(text, className) {
|
||||||
|
const line = document.createElement('p');
|
||||||
|
line.className = className;
|
||||||
|
line.textContent = text;
|
||||||
|
asNameResults.replaceChildren(line);
|
||||||
|
}
|
||||||
|
|
||||||
|
// List the matches of a search answer, each as a link to its AS page,
|
||||||
|
// and say how many more matched than the answer holds.
|
||||||
|
function showASNameMatches(data) {
|
||||||
|
if (data.total === 0) {
|
||||||
|
showASNameMessage('No AS matches "' + data.query + '".', 'search-message');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const list = document.createElement('ul');
|
||||||
|
list.className = 'asname-list';
|
||||||
|
for (const match of data.results) {
|
||||||
|
const asn = document.createElement('span');
|
||||||
|
asn.className = 'asn';
|
||||||
|
asn.textContent = 'AS' + match.asn;
|
||||||
|
|
||||||
|
const description = document.createElement('span');
|
||||||
|
description.className = 'description';
|
||||||
|
description.textContent = match.description;
|
||||||
|
|
||||||
|
const link = document.createElement('a');
|
||||||
|
link.href = '/as/' + match.asn;
|
||||||
|
link.append(asn, ' ' + match.handle, description);
|
||||||
|
|
||||||
|
const item = document.createElement('li');
|
||||||
|
item.append(link);
|
||||||
|
list.append(item);
|
||||||
|
}
|
||||||
|
asNameResults.replaceChildren(list);
|
||||||
|
|
||||||
|
const more = data.total - data.results.length;
|
||||||
|
if (more > 0) {
|
||||||
|
const line = document.createElement('p');
|
||||||
|
line.className = 'search-message';
|
||||||
|
line.textContent = more.toLocaleString() + ' more matched. Narrow the search to see them.';
|
||||||
|
asNameResults.append(line);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
document.getElementById('asname-form').addEventListener('submit', function(e) {
|
document.getElementById('asname-form').addEventListener('submit', function(e) {
|
||||||
e.preventDefault();
|
e.preventDefault();
|
||||||
const query = document.getElementById('asname-input').value.trim();
|
const query = document.getElementById('asname-input').value.trim();
|
||||||
if (!query) return;
|
if (!query) return;
|
||||||
|
|
||||||
const resultsDiv = document.getElementById('asname-results');
|
// A number, or AS and a number, opens that AS's page
|
||||||
resultsDiv.innerHTML = '<p class="loading">Searching...</p>';
|
const asn = query.replace(/^AS/i, '');
|
||||||
|
if (/^\d+$/.test(asn)) {
|
||||||
// Use a simple client-side search against the asinfo data
|
window.location.href = '/as/' + asn;
|
||||||
// For now, redirect to AS page if it looks like an ASN
|
|
||||||
if (/^\d+$/.test(query)) {
|
|
||||||
window.location.href = '/as/' + query;
|
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Show a message that server-side search is coming
|
showASNameMessage('Searching...', 'loading');
|
||||||
resultsDiv.innerHTML = '<p class="search-message">AS name search coming soon. For now, try an AS number.</p>';
|
|
||||||
|
fetch('/api/v1/search/as?q=' + encodeURIComponent(query))
|
||||||
|
.then(response => response.json())
|
||||||
|
.then(response => {
|
||||||
|
if (response.status === 'error') {
|
||||||
|
showASNameMessage('Error: ' + response.error.msg, 'error');
|
||||||
|
} else {
|
||||||
|
showASNameMatches(response.data);
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch(error => {
|
||||||
|
showASNameMessage('Error: ' + error.message, 'error');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
// IP lookup
|
// IP lookup
|
||||||
@@ -446,9 +584,12 @@
|
|||||||
document.getElementById('ip-input').value = '';
|
document.getElementById('ip-input').value = '';
|
||||||
});
|
});
|
||||||
|
|
||||||
// Initial load and refresh stats every 5 seconds
|
// Initial load and refresh stats every REFRESH_PERIOD_MS. The freshness
|
||||||
|
// check runs every second on its own, so a request that never returns
|
||||||
|
// still turns the spinner into "stale".
|
||||||
updateStats();
|
updateStats();
|
||||||
setInterval(updateStats, 5000);
|
setInterval(updateStats, REFRESH_PERIOD_MS);
|
||||||
|
setInterval(showFreshness, MS_PER_SECOND);
|
||||||
</script>
|
</script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|||||||
@@ -126,7 +126,8 @@
|
|||||||
.connected {
|
.connected {
|
||||||
color: #22c55e;
|
color: #22c55e;
|
||||||
}
|
}
|
||||||
.disconnected {
|
.disconnected,
|
||||||
|
.stale {
|
||||||
color: #ef4444;
|
color: #ef4444;
|
||||||
}
|
}
|
||||||
.error {
|
.error {
|
||||||
@@ -180,7 +181,10 @@
|
|||||||
<h2>RouteWatch</h2>
|
<h2>RouteWatch</h2>
|
||||||
<div class="metric">
|
<div class="metric">
|
||||||
<span class="metric-label">Status</span>
|
<span class="metric-label">Status</span>
|
||||||
<span class="metric-value" id="connected">-</span>
|
<span>
|
||||||
|
<span class="metric-value" id="connected">-</span>
|
||||||
|
<span class="metric-value" id="freshness"></span>
|
||||||
|
</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="metric">
|
<div class="metric">
|
||||||
<span class="metric-label">Uptime</span>
|
<span class="metric-label">Uptime</span>
|
||||||
@@ -231,8 +235,8 @@
|
|||||||
<span class="metric-value" id="withdrawals">-</span>
|
<span class="metric-value" id="withdrawals">-</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="metric">
|
<div class="metric">
|
||||||
<span class="metric-label">Route Churn/sec</span>
|
<span class="metric-label">Route Churn/min</span>
|
||||||
<span class="metric-value" id="route_churn_per_sec">-</span>
|
<span class="metric-value" id="route_churn_per_min">-</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="metric">
|
<div class="metric">
|
||||||
<span class="metric-label">Total Data</span>
|
<span class="metric-label">Total Data</span>
|
||||||
@@ -386,6 +390,44 @@
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<script nonce="{{.CSPNonce}}">
|
<script nonce="{{.CSPNonce}}">
|
||||||
|
// The stats are fetched every REFRESH_PERIOD_MS. Next to the status, a
|
||||||
|
// spinner turns one frame each time an answer counts more messages
|
||||||
|
// than the one before. After STALE_AFTER_REFRESHES refreshes without
|
||||||
|
// such an answer, the spinner gives way to "stale" and the time since.
|
||||||
|
const REFRESH_PERIOD_MS = 2000;
|
||||||
|
const STALE_AFTER_REFRESHES = 3;
|
||||||
|
const MS_PER_SECOND = 1000;
|
||||||
|
const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'];
|
||||||
|
const SPINNER_COLOURS = ['#e74c3c', '#e67e22', '#f1c40f', '#27ae60', '#3498db', '#9b59b6'];
|
||||||
|
|
||||||
|
let spinnerFrame = 0;
|
||||||
|
let lastMessageCount = null;
|
||||||
|
let lastFreshAt = Date.now();
|
||||||
|
|
||||||
|
// Turn the spinner when the message count has risen since the last answer.
|
||||||
|
function noteMessageCount(count) {
|
||||||
|
if (lastMessageCount !== null && count > lastMessageCount) {
|
||||||
|
spinnerFrame = (spinnerFrame + 1) % SPINNER_FRAMES.length;
|
||||||
|
lastFreshAt = Date.now();
|
||||||
|
}
|
||||||
|
lastMessageCount = count;
|
||||||
|
showFreshness();
|
||||||
|
}
|
||||||
|
|
||||||
|
function showFreshness() {
|
||||||
|
const freshness = document.getElementById('freshness');
|
||||||
|
const sinceFreshMs = Date.now() - lastFreshAt;
|
||||||
|
const stale = sinceFreshMs >= STALE_AFTER_REFRESHES * REFRESH_PERIOD_MS;
|
||||||
|
freshness.classList.toggle('stale', stale);
|
||||||
|
if (stale) {
|
||||||
|
freshness.textContent = 'stale for ' + Math.floor(sinceFreshMs / MS_PER_SECOND) + 's';
|
||||||
|
freshness.style.color = '';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
freshness.textContent = SPINNER_FRAMES[spinnerFrame];
|
||||||
|
freshness.style.color = SPINNER_COLOURS[spinnerFrame % SPINNER_COLOURS.length];
|
||||||
|
}
|
||||||
|
|
||||||
function formatBytes(bytes) {
|
function formatBytes(bytes) {
|
||||||
if (bytes === 0) return '0 B';
|
if (bytes === 0) return '0 B';
|
||||||
const k = 1024;
|
const k = 1024;
|
||||||
@@ -511,7 +553,7 @@
|
|||||||
document.getElementById('messages_per_sec').textContent = '-';
|
document.getElementById('messages_per_sec').textContent = '-';
|
||||||
document.getElementById('announcements').textContent = '-';
|
document.getElementById('announcements').textContent = '-';
|
||||||
document.getElementById('withdrawals').textContent = '-';
|
document.getElementById('withdrawals').textContent = '-';
|
||||||
document.getElementById('route_churn_per_sec').textContent = '-';
|
document.getElementById('route_churn_per_min').textContent = '-';
|
||||||
document.getElementById('total_wire_bytes').textContent = '-';
|
document.getElementById('total_wire_bytes').textContent = '-';
|
||||||
document.getElementById('wire_mbits_per_sec').textContent = '-';
|
document.getElementById('wire_mbits_per_sec').textContent = '-';
|
||||||
document.getElementById('gc_num').textContent = '-';
|
document.getElementById('gc_num').textContent = '-';
|
||||||
@@ -569,7 +611,8 @@
|
|||||||
const connectedEl = document.getElementById('connected');
|
const connectedEl = document.getElementById('connected');
|
||||||
connectedEl.textContent = data.connected ? 'Connected' : 'Disconnected';
|
connectedEl.textContent = data.connected ? 'Connected' : 'Disconnected';
|
||||||
connectedEl.className = 'metric-value ' + (data.connected ? 'connected' : 'disconnected');
|
connectedEl.className = 'metric-value ' + (data.connected ? 'connected' : 'disconnected');
|
||||||
|
noteMessageCount(data.total_messages);
|
||||||
|
|
||||||
// Update all metrics
|
// Update all metrics
|
||||||
document.getElementById('uptime').textContent = data.uptime;
|
document.getElementById('uptime').textContent = data.uptime;
|
||||||
document.getElementById('go_version').textContent = data.go_version;
|
document.getElementById('go_version').textContent = data.go_version;
|
||||||
@@ -578,7 +621,7 @@
|
|||||||
document.getElementById('connection_duration').textContent = data.connection_duration;
|
document.getElementById('connection_duration').textContent = data.connection_duration;
|
||||||
document.getElementById('reconnect_count').textContent = formatNumber(data.reconnect_count);
|
document.getElementById('reconnect_count').textContent = formatNumber(data.reconnect_count);
|
||||||
document.getElementById('total_messages').textContent = formatNumber(data.total_messages);
|
document.getElementById('total_messages').textContent = formatNumber(data.total_messages);
|
||||||
document.getElementById('messages_per_sec').textContent = data.messages_per_sec.toFixed(1);
|
document.getElementById('messages_per_sec').textContent = formatNumber(Math.round(data.messages_per_sec));
|
||||||
document.getElementById('total_wire_bytes').textContent = formatBytes(data.total_wire_bytes);
|
document.getElementById('total_wire_bytes').textContent = formatBytes(data.total_wire_bytes);
|
||||||
document.getElementById('wire_mbits_per_sec').textContent = data.wire_mbits_per_sec.toFixed(2) + ' Mbps';
|
document.getElementById('wire_mbits_per_sec').textContent = data.wire_mbits_per_sec.toFixed(2) + ' Mbps';
|
||||||
document.getElementById('asns').textContent = formatNumber(data.asns);
|
document.getElementById('asns').textContent = formatNumber(data.asns);
|
||||||
@@ -591,8 +634,8 @@
|
|||||||
document.getElementById('live_routes').textContent = formatNumber(data.live_routes);
|
document.getElementById('live_routes').textContent = formatNumber(data.live_routes);
|
||||||
document.getElementById('ipv4_routes').textContent = formatNumber(data.ipv4_routes);
|
document.getElementById('ipv4_routes').textContent = formatNumber(data.ipv4_routes);
|
||||||
document.getElementById('ipv6_routes').textContent = formatNumber(data.ipv6_routes);
|
document.getElementById('ipv6_routes').textContent = formatNumber(data.ipv6_routes);
|
||||||
document.getElementById('ipv4_updates_per_sec').textContent = data.ipv4_updates_per_sec.toFixed(1);
|
document.getElementById('ipv4_updates_per_sec').textContent = formatNumber(Math.round(data.ipv4_updates_per_sec));
|
||||||
document.getElementById('ipv6_updates_per_sec').textContent = data.ipv6_updates_per_sec.toFixed(1);
|
document.getElementById('ipv6_updates_per_sec').textContent = formatNumber(Math.round(data.ipv6_updates_per_sec));
|
||||||
document.getElementById('oldest_route').textContent = formatRelativeTime(data.oldest_route);
|
document.getElementById('oldest_route').textContent = formatRelativeTime(data.oldest_route);
|
||||||
document.getElementById('newest_route').textContent = formatRelativeTime(data.newest_route);
|
document.getElementById('newest_route').textContent = formatRelativeTime(data.newest_route);
|
||||||
|
|
||||||
@@ -601,7 +644,7 @@
|
|||||||
document.getElementById('bgp_peer_count').textContent = formatNumber(data.stream.bgp_peer_count);
|
document.getElementById('bgp_peer_count').textContent = formatNumber(data.stream.bgp_peer_count);
|
||||||
document.getElementById('announcements').textContent = formatNumber(data.stream.announcements);
|
document.getElementById('announcements').textContent = formatNumber(data.stream.announcements);
|
||||||
document.getElementById('withdrawals').textContent = formatNumber(data.stream.withdrawals);
|
document.getElementById('withdrawals').textContent = formatNumber(data.stream.withdrawals);
|
||||||
document.getElementById('route_churn_per_sec').textContent = data.stream.route_churn_per_sec.toFixed(1);
|
document.getElementById('route_churn_per_min').textContent = formatNumber(data.stream.route_churn_per_min);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Update GC stats
|
// Update GC stats
|
||||||
@@ -643,9 +686,12 @@
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// Update immediately and then every 2 seconds
|
// Update immediately and then every REFRESH_PERIOD_MS. The freshness
|
||||||
|
// check runs every second on its own, so a request that never returns
|
||||||
|
// still turns the spinner into "stale".
|
||||||
updateStatus();
|
updateStatus();
|
||||||
setInterval(updateStatus, 2000);
|
setInterval(updateStatus, REFRESH_PERIOD_MS);
|
||||||
|
setInterval(showFreshness, MS_PER_SECOND);
|
||||||
</script>
|
</script>
|
||||||
</main>
|
</main>
|
||||||
|
|
||||||
|
|||||||
@@ -7,9 +7,9 @@ import (
|
|||||||
"net/url"
|
"net/url"
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/version"
|
"git.eeqj.de/sneak/routewatch/internal/version"
|
||||||
|
"github.com/dustin/go-humanize"
|
||||||
)
|
)
|
||||||
|
|
||||||
//go:embed index.html
|
//go:embed index.html
|
||||||
@@ -48,48 +48,7 @@ var (
|
|||||||
once sync.Once
|
once sync.Once
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const cidrPartCount = 2 // A CIDR has two parts: prefix and length
|
||||||
hoursPerDay = 24
|
|
||||||
daysPerMonth = 30
|
|
||||||
cidrPartCount = 2 // A CIDR has two parts: prefix and length
|
|
||||||
)
|
|
||||||
|
|
||||||
// timeSince returns a human-readable duration since the given time
|
|
||||||
func timeSince(t time.Time) string {
|
|
||||||
duration := time.Since(t)
|
|
||||||
if duration < time.Minute {
|
|
||||||
return "just now"
|
|
||||||
}
|
|
||||||
|
|
||||||
if duration < time.Hour {
|
|
||||||
minutes := int(duration.Minutes())
|
|
||||||
if minutes == 1 {
|
|
||||||
return "1 minute ago"
|
|
||||||
}
|
|
||||||
|
|
||||||
return duration.Truncate(time.Minute).String() + " ago"
|
|
||||||
}
|
|
||||||
|
|
||||||
if duration < hoursPerDay*time.Hour {
|
|
||||||
hours := int(duration.Hours())
|
|
||||||
if hours == 1 {
|
|
||||||
return "1 hour ago"
|
|
||||||
}
|
|
||||||
|
|
||||||
return duration.Truncate(time.Hour).String() + " ago"
|
|
||||||
}
|
|
||||||
|
|
||||||
days := int(duration.Hours() / hoursPerDay)
|
|
||||||
if days == 1 {
|
|
||||||
return "1 day ago"
|
|
||||||
}
|
|
||||||
|
|
||||||
if days < daysPerMonth {
|
|
||||||
return duration.Truncate(hoursPerDay*time.Hour).String() + " ago"
|
|
||||||
}
|
|
||||||
|
|
||||||
return t.Format("2006-01-02")
|
|
||||||
}
|
|
||||||
|
|
||||||
// prefixURL generates a URL path for a prefix in CIDR notation.
|
// prefixURL generates a URL path for a prefix in CIDR notation.
|
||||||
// Takes a prefix like "192.168.1.0/24" and returns "/prefix/192.168.1.0/24"
|
// Takes a prefix like "192.168.1.0/24" and returns "/prefix/192.168.1.0/24"
|
||||||
@@ -113,7 +72,7 @@ func initTemplates() {
|
|||||||
|
|
||||||
// Create common template functions
|
// Create common template functions
|
||||||
funcs := template.FuncMap{
|
funcs := template.FuncMap{
|
||||||
"timeSince": timeSince,
|
"timeSince": humanize.Time,
|
||||||
"urlEncode": url.QueryEscape,
|
"urlEncode": url.QueryEscape,
|
||||||
"prefixURL": prefixURL,
|
"prefixURL": prefixURL,
|
||||||
"appName": func() string { return version.Name },
|
"appName": func() string { return version.Name },
|
||||||
|
|||||||
@@ -6,9 +6,24 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/templates"
|
"git.eeqj.de/sneak/routewatch/internal/templates"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// renderPage renders a page with the data every page needs.
|
||||||
|
func renderPage(t *testing.T, tmpl *template.Template) string {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var page bytes.Buffer
|
||||||
|
|
||||||
|
err := tmpl.Execute(&page, struct{ CSPNonce string }{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("rendering the page: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return page.String()
|
||||||
|
}
|
||||||
|
|
||||||
// TestFooterLicense renders the home and status pages and expects each footer
|
// TestFooterLicense renders the home and status pages and expects each footer
|
||||||
// to name the licence in LICENSE, MIT, and not WTFPL.
|
// to name the licence in LICENSE, MIT, and not WTFPL.
|
||||||
func TestFooterLicense(t *testing.T) {
|
func TestFooterLicense(t *testing.T) {
|
||||||
@@ -28,14 +43,7 @@ func TestFooterLicense(t *testing.T) {
|
|||||||
t.Run(name, func(t *testing.T) {
|
t.Run(name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
var page bytes.Buffer
|
_, footer, found := strings.Cut(renderPage(t, tmpl), "<footer")
|
||||||
|
|
||||||
err := tmpl.Execute(&page, struct{ CSPNonce string }{})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("rendering the page: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, footer, found := strings.Cut(page.String(), "<footer")
|
|
||||||
if !found {
|
if !found {
|
||||||
t.Fatal("the page has no footer")
|
t.Fatal("the page has no footer")
|
||||||
}
|
}
|
||||||
@@ -52,3 +60,121 @@ func TestFooterLicense(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestFreshnessIndicator renders the home and status pages and expects each to
|
||||||
|
// have, right after the status element, the element where the page's script
|
||||||
|
// shows its spinner or "stale".
|
||||||
|
func TestFreshnessIndicator(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
pages := map[string]struct {
|
||||||
|
tmpl *template.Template
|
||||||
|
status string
|
||||||
|
indicator string
|
||||||
|
}{
|
||||||
|
"index": {
|
||||||
|
tmpl: templates.IndexTemplate(),
|
||||||
|
status: `<span id="stat-status">-</span>`,
|
||||||
|
indicator: `<span id="freshness"></span>`,
|
||||||
|
},
|
||||||
|
"status": {
|
||||||
|
tmpl: templates.StatusTemplate(),
|
||||||
|
status: `<span class="metric-value" id="connected">-</span>`,
|
||||||
|
indicator: `<span class="metric-value" id="freshness"></span>`,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for name, page := range pages {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, afterStatus, found := strings.Cut(renderPage(t, page.tmpl), page.status)
|
||||||
|
if !found {
|
||||||
|
t.Fatalf("the page has no status element %s", page.status)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !strings.HasPrefix(strings.TrimSpace(afterStatus), page.indicator) {
|
||||||
|
t.Errorf("the status element is not followed by %s", page.indicator)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestASDetailPrefixListsFold renders the AS page with one IPv4 and one IPv6
|
||||||
|
// prefix. It expects each prefix list inside its own <details> element that
|
||||||
|
// starts open and whose <summary> holds the list's heading, so that a click on
|
||||||
|
// the heading folds the list.
|
||||||
|
func TestASDetailPrefixListsFold(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const (
|
||||||
|
asn = 64500
|
||||||
|
prefixV4 = "192.0.2.0/24"
|
||||||
|
prefixV6 = "2001:db8::/32"
|
||||||
|
)
|
||||||
|
|
||||||
|
ipv4 := []database.LiveRoute{{Prefix: prefixV4}}
|
||||||
|
ipv6 := []database.LiveRoute{{Prefix: prefixV6}}
|
||||||
|
|
||||||
|
// The fields the AS page handler gives the template.
|
||||||
|
data := struct {
|
||||||
|
ASN *database.ASN
|
||||||
|
IPv4Prefixes []database.LiveRoute
|
||||||
|
IPv6Prefixes []database.LiveRoute
|
||||||
|
TotalCount int
|
||||||
|
IPv4Count int
|
||||||
|
IPv6Count int
|
||||||
|
Peers []database.ASPeer
|
||||||
|
PeerCount int
|
||||||
|
CSPNonce string
|
||||||
|
}{
|
||||||
|
ASN: &database.ASN{ASN: asn},
|
||||||
|
IPv4Prefixes: ipv4,
|
||||||
|
IPv6Prefixes: ipv6,
|
||||||
|
TotalCount: len(ipv4) + len(ipv6),
|
||||||
|
IPv4Count: len(ipv4),
|
||||||
|
IPv6Count: len(ipv6),
|
||||||
|
}
|
||||||
|
|
||||||
|
var rendered bytes.Buffer
|
||||||
|
|
||||||
|
err := templates.ASDetailTemplate().Execute(&rendered, data)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("rendering the page: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
page := rendered.String()
|
||||||
|
|
||||||
|
lists := []struct{ heading, prefix string }{
|
||||||
|
{"IPv4 Prefixes", prefixV4},
|
||||||
|
{"IPv6 Prefixes", prefixV6},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, list := range lists {
|
||||||
|
before, _, found := strings.Cut(page, list.prefix)
|
||||||
|
if !found {
|
||||||
|
t.Fatalf("the page does not list %s", list.prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The list's section starts at the last <details before its prefix.
|
||||||
|
start := strings.LastIndex(before, "<details")
|
||||||
|
if start < 0 || strings.Contains(before[start:], "</details>") {
|
||||||
|
t.Errorf("%s: the list is not inside a <details> element", list.heading)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
tag, section, _ := strings.Cut(before[start:], ">")
|
||||||
|
if !strings.Contains(tag, " open") {
|
||||||
|
t.Errorf("%s: the <details> element does not start open: %s>",
|
||||||
|
list.heading, tag)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, summary, _ := strings.Cut(section, "<summary")
|
||||||
|
summary, _, _ = strings.Cut(summary, "</summary>")
|
||||||
|
|
||||||
|
if !strings.Contains(summary, list.heading) {
|
||||||
|
t.Errorf("%s: the <summary> does not hold the heading", list.heading)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
+284
-33
@@ -7,10 +7,15 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"log/slog"
|
||||||
"net"
|
"net"
|
||||||
"regexp"
|
"regexp"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"golang.org/x/text/encoding/charmap"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Timeout constants for WHOIS queries.
|
// Timeout constants for WHOIS queries.
|
||||||
@@ -26,10 +31,18 @@ const maxAnswerSize = 1 << 20
|
|||||||
|
|
||||||
// Parsing constants.
|
// Parsing constants.
|
||||||
const (
|
const (
|
||||||
keyValueParts = 2 // Expected parts when splitting "key: value"
|
|
||||||
lacnicDateFormatLen = 8 // Length of YYYYMMDD date format
|
lacnicDateFormatLen = 8 // Length of YYYYMMDD date format
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// The classes of the two objects that describe an AS in an answer from
|
||||||
|
// RIPE, APNIC or AFRINIC, and the key of the aut-num object's line that
|
||||||
|
// gives the ID of the AS's organisation object.
|
||||||
|
const (
|
||||||
|
autNumClass = "aut-num"
|
||||||
|
organisationClass = "organisation"
|
||||||
|
orgKey = "org"
|
||||||
|
)
|
||||||
|
|
||||||
// WHOIS server addresses.
|
// WHOIS server addresses.
|
||||||
const (
|
const (
|
||||||
whoisServerIANA = "whois.iana.org:43"
|
whoisServerIANA = "whois.iana.org:43"
|
||||||
@@ -38,8 +51,32 @@ const (
|
|||||||
whoisServerAPNIC = "whois.apnic.net:43"
|
whoisServerAPNIC = "whois.apnic.net:43"
|
||||||
whoisServerLACNIC = "whois.lacnic.net:43"
|
whoisServerLACNIC = "whois.lacnic.net:43"
|
||||||
whoisServerAFRINIC = "whois.afrinic.net:43"
|
whoisServerAFRINIC = "whois.afrinic.net:43"
|
||||||
|
|
||||||
|
// NIC.br registers Brazil's ASes under LACNIC.
|
||||||
|
whoisServerNICbr = "whois.registro.br:43"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// nicbrHeader starts the copyright notice at the top of NIC.br's WHOIS
|
||||||
|
// answers. LACNIC's answer for an AS that NIC.br registers is NIC.br's, and
|
||||||
|
// carries the same notice:
|
||||||
|
//
|
||||||
|
// % Copyright (c) Nic.br - Use of this data is governed by the Use and
|
||||||
|
// % Privacy Policy at https://registro.br/upp . Distribution,
|
||||||
|
const nicbrHeader = "% Copyright (c) Nic.br"
|
||||||
|
|
||||||
|
// nicbrLimitReached is in the line that marks NIC.br's reduced answer to a
|
||||||
|
// client over its query limit. NIC.br publishes no limit. Its Use and
|
||||||
|
// Privacy Policy, https://registro.br/upp, says only that it restricts
|
||||||
|
// queries to stop the data being taken in bulk.
|
||||||
|
const nicbrLimitReached = "Query rate limit exceeded"
|
||||||
|
|
||||||
|
// NICbrSkipTime is how long the client stops asking NIC.br after NIC.br
|
||||||
|
// answers that the client is over its query limit. NIC.br publishes
|
||||||
|
// neither the limit nor how long it lasts. An hour outlasts a limit
|
||||||
|
// counted per minute or per hour. The ASes looked up meanwhile get
|
||||||
|
// NIC.br's fields when they are next refreshed.
|
||||||
|
const NICbrSkipTime = time.Hour
|
||||||
|
|
||||||
// RIR identifiers.
|
// RIR identifiers.
|
||||||
const (
|
const (
|
||||||
RIRARIN = "ARIN"
|
RIRARIN = "ARIN"
|
||||||
@@ -49,6 +86,31 @@ const (
|
|||||||
RIRAFRNIC = "AFRINIC"
|
RIRAFRNIC = "AFRINIC"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// noPersonalFlag keeps person and role objects out of an answer. The
|
||||||
|
// organisation object stays in it. RIPE counts the person and role objects
|
||||||
|
// it returns against a daily limit per IP address. Role objects for abuse
|
||||||
|
// contacts do not count. RIPE refuses an address for good that passes the
|
||||||
|
// limit too often:
|
||||||
|
// https://docs.db.ripe.net/Types-of-Queries/Access-Control-for-Queries/
|
||||||
|
//
|
||||||
|
// RIPE documents the flag here:
|
||||||
|
// https://docs.db.ripe.net/Types-of-Queries/Referenced-Objects-in-Query-Response/
|
||||||
|
// APNIC and AFRINIC list it in their answers to the query "help". AFRINIC
|
||||||
|
// sends readers to that answer for its full list of flags:
|
||||||
|
// https://afrinic.net/whois-getting-started
|
||||||
|
const noPersonalFlag = "--no-personal"
|
||||||
|
|
||||||
|
// The error lines with which RIPE's database software refuses a query:
|
||||||
|
// access denied to the client's IP address, and that address's limit
|
||||||
|
// reached. APNIC and AFRINIC run the same software. The lines are listed
|
||||||
|
// under "Access Errors" on this page, whose URL is split over two lines:
|
||||||
|
// https://docs.db.ripe.net/Appendices/
|
||||||
|
// Appendix-C--RIPE-Database-Query-Server-Response-Codes-and-Messages/
|
||||||
|
const (
|
||||||
|
accessDeniedError = "%ERROR:201:"
|
||||||
|
accessLimitError = "%ERROR:202:"
|
||||||
|
)
|
||||||
|
|
||||||
// Errors LookupASN returns when a WHOIS answer has nothing to store.
|
// Errors LookupASN returns when a WHOIS answer has nothing to store.
|
||||||
var (
|
var (
|
||||||
// ErrNoReferral is returned when IANA refers the AS to no RIR, as it
|
// ErrNoReferral is returned when IANA refers the AS to no RIR, as it
|
||||||
@@ -58,6 +120,7 @@ var (
|
|||||||
errNoASBlock = errors.New("IANA answered with neither a referral " +
|
errNoASBlock = errors.New("IANA answered with neither a referral " +
|
||||||
"nor an AS block")
|
"nor an AS block")
|
||||||
errNoFields = errors.New("RIR answered with no WHOIS fields")
|
errNoFields = errors.New("RIR answered with no WHOIS fields")
|
||||||
|
errRefused = errors.New("RIR refused the query")
|
||||||
errUnknownServer = errors.New("IANA refers the AS to a server no RIR runs")
|
errUnknownServer = errors.New("IANA refers the AS to a server no RIR runs")
|
||||||
errAnswerTooLarge = errors.New("WHOIS answer reached the size cap")
|
errAnswerTooLarge = errors.New("WHOIS answer reached the size cap")
|
||||||
)
|
)
|
||||||
@@ -86,21 +149,42 @@ type Client struct {
|
|||||||
// host:port. NewClient sets a net.Dialer's; tests replace it to reach
|
// host:port. NewClient sets a net.Dialer's; tests replace it to reach
|
||||||
// local servers instead.
|
// local servers instead.
|
||||||
DialContext func(ctx context.Context, network, address string) (net.Conn, error)
|
DialContext func(ctx context.Context, network, address string) (net.Conn, error)
|
||||||
|
|
||||||
|
// Now returns the current time. NewClient sets time.Now; tests replace
|
||||||
|
// it to move the clock.
|
||||||
|
Now func() time.Time
|
||||||
|
|
||||||
|
logger *slog.Logger
|
||||||
|
|
||||||
|
// mu guards nicbrSkipUntil.
|
||||||
|
mu sync.Mutex
|
||||||
|
|
||||||
|
// nicbrSkipUntil is when the client next asks NIC.br, after NIC.br
|
||||||
|
// answered that the client is over its query limit.
|
||||||
|
nicbrSkipUntil time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewClient creates a new WHOIS client.
|
// NewClient creates a new WHOIS client that logs to logger.
|
||||||
func NewClient() *Client {
|
func NewClient(logger *slog.Logger) *Client {
|
||||||
dialer := &net.Dialer{Timeout: dialTimeout}
|
dialer := &net.Dialer{Timeout: dialTimeout}
|
||||||
|
|
||||||
return &Client{DialContext: dialer.DialContext}
|
return &Client{
|
||||||
|
DialContext: dialer.DialContext,
|
||||||
|
Now: time.Now,
|
||||||
|
logger: logger,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// LookupASN asks IANA which RIR holds the given ASN, asks that RIR, and
|
// LookupASN asks IANA which RIR holds the given ASN, asks that RIR, and
|
||||||
// returns what the RIR's answer says. It returns ErrNoReferral when IANA
|
// returns what the RIR's answer says. When LACNIC's answer is NIC.br's, it
|
||||||
// refers the AS to no RIR, and another error when a query fails, when IANA's
|
// returns what NIC.br's own server says instead, unless that query fails or
|
||||||
// answer has neither a referral nor the AS's block, when IANA refers the AS
|
// is skipped, as askNICbr says.
|
||||||
// to a server none of the five RIRs run, or when the RIR's answer carries
|
//
|
||||||
// none of ASNInfo's fields, as a refusal of service does.
|
// It returns ErrNoReferral when IANA refers the AS to no RIR, and another
|
||||||
|
// error when the query to IANA or to the RIR fails, when the RIR refuses
|
||||||
|
// it, when IANA's answer has neither a referral nor the AS's block, when
|
||||||
|
// IANA refers the AS to a server none of the five RIRs run, or when the
|
||||||
|
// answer carries none of ASNInfo's fields.
|
||||||
func (c *Client) LookupASN(ctx context.Context, asn int) (*ASNInfo, error) {
|
func (c *Client) LookupASN(ctx context.Context, asn int) (*ASNInfo, error) {
|
||||||
// Query IANA first to find the authoritative RIR
|
// Query IANA first to find the authoritative RIR
|
||||||
query := fmt.Sprintf("AS%d", asn)
|
query := fmt.Sprintf("AS%d", asn)
|
||||||
@@ -127,23 +211,82 @@ func (c *Client) LookupASN(ctx context.Context, asn int) (*ASNInfo, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Query the authoritative RIR
|
// Query the authoritative RIR
|
||||||
rirResp, err := c.query(ctx, whoisServer, query)
|
rirResp, err := c.queryRIR(ctx, whoisServer, rirQuery(rir, query))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("RIR query failed: %w", err)
|
return nil, fmt.Errorf("RIR query failed: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The fields come from the RIR's answer alone: IANA's describes the
|
// The fields never come from IANA's answer, which describes the block
|
||||||
// block the AS is in.
|
// the AS is in.
|
||||||
info := c.parseResponse(asn, rir, rirResp)
|
info := c.parseResponse(asn, rir, rirResp)
|
||||||
|
|
||||||
|
if rir == RIRLACNIC && strings.Contains(rirResp, nicbrHeader) {
|
||||||
|
info = c.askNICbr(ctx, query, info)
|
||||||
|
}
|
||||||
|
|
||||||
if info.isEmpty() {
|
if info.isEmpty() {
|
||||||
return nil, fmt.Errorf("%s from %s: %w", query, whoisServer, errNoFields)
|
return nil, fmt.Errorf("%s from %s: %w", query, whoisServer, errNoFields)
|
||||||
}
|
}
|
||||||
|
|
||||||
info.RawResponse = ianaResp + "\n" + rirResp
|
info.RawResponse = ianaResp + "\n" + info.RawResponse
|
||||||
|
|
||||||
return info, nil
|
return info, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// askNICbr asks NIC.br's server about an AS whose answer from LACNIC is
|
||||||
|
// NIC.br's, and returns what NIC.br's answer says. LACNIC replaces each
|
||||||
|
// accented letter in such an answer with U+FFFD. NIC.br's own server keeps
|
||||||
|
// them, in Latin-1.
|
||||||
|
//
|
||||||
|
// askNICbr returns lacnicInfo, what LACNIC's answer says, and logs a
|
||||||
|
// warning when the query fails, when NIC.br sends the reduced answer for a
|
||||||
|
// client over its limit, or when NIC.br's answer carries none of ASNInfo's
|
||||||
|
// fields. After the reduced answer, it returns lacnicInfo without asking
|
||||||
|
// NIC.br until NICbrSkipTime has passed.
|
||||||
|
func (c *Client) askNICbr(
|
||||||
|
ctx context.Context, asQuery string, lacnicInfo *ASNInfo,
|
||||||
|
) *ASNInfo {
|
||||||
|
asn := lacnicInfo.ASN
|
||||||
|
|
||||||
|
c.mu.Lock()
|
||||||
|
skipping := c.Now().Before(c.nicbrSkipUntil)
|
||||||
|
c.mu.Unlock()
|
||||||
|
|
||||||
|
if skipping {
|
||||||
|
return lacnicInfo
|
||||||
|
}
|
||||||
|
|
||||||
|
answer, err := c.query(ctx, whoisServerNICbr, asQuery)
|
||||||
|
if err != nil {
|
||||||
|
c.logger.Warn("NIC.br query failed, keeping LACNIC's fields",
|
||||||
|
"asn", asn, "error", err)
|
||||||
|
|
||||||
|
return lacnicInfo
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.Contains(answer, nicbrLimitReached) {
|
||||||
|
c.mu.Lock()
|
||||||
|
c.nicbrSkipUntil = c.Now().Add(NICbrSkipTime)
|
||||||
|
c.mu.Unlock()
|
||||||
|
|
||||||
|
c.logger.Warn("NIC.br answered that the client is over its query "+
|
||||||
|
"limit, keeping LACNIC's fields and skipping NIC.br",
|
||||||
|
"asn", asn, "skip_for", NICbrSkipTime)
|
||||||
|
|
||||||
|
return lacnicInfo
|
||||||
|
}
|
||||||
|
|
||||||
|
info := c.parseResponse(asn, RIRLACNIC, answer)
|
||||||
|
if info.isEmpty() {
|
||||||
|
c.logger.Warn("NIC.br answered with no WHOIS fields, "+
|
||||||
|
"keeping LACNIC's fields", "asn", asn)
|
||||||
|
|
||||||
|
return lacnicInfo
|
||||||
|
}
|
||||||
|
|
||||||
|
return info
|
||||||
|
}
|
||||||
|
|
||||||
// isEmpty reports whether info holds none of the fields a WHOIS answer
|
// isEmpty reports whether info holds none of the fields a WHOIS answer
|
||||||
// fills, as when the answer is only comment and error lines.
|
// fills, as when the answer is only comment and error lines.
|
||||||
func (info *ASNInfo) isEmpty() bool {
|
func (info *ASNInfo) isEmpty() bool {
|
||||||
@@ -154,8 +297,41 @@ func (info *ASNInfo) isEmpty() bool {
|
|||||||
info.RegDate == nil && info.LastMod == nil
|
info.RegDate == nil && info.LastMod == nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// query performs a raw WHOIS query to the specified server. An answer that
|
// rirQuery returns the query sent to rir's server for asQuery, an AS
|
||||||
// reaches maxAnswerSize is an error.
|
// number such as "AS3333". RIPE, APNIC and AFRINIC are asked to keep person
|
||||||
|
// and role objects out of the answer.
|
||||||
|
func rirQuery(rir, asQuery string) string {
|
||||||
|
switch rir {
|
||||||
|
case RIRRIPE, RIRAPNIC, RIRAFRNIC:
|
||||||
|
return noPersonalFlag + " " + asQuery
|
||||||
|
default:
|
||||||
|
return asQuery
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// queryRIR sends query to an RIR's server and returns the answer. An answer
|
||||||
|
// holding an error line with which the server refuses the query is an
|
||||||
|
// error, and the error holds that line.
|
||||||
|
func (c *Client) queryRIR(
|
||||||
|
ctx context.Context, server, query string,
|
||||||
|
) (string, error) {
|
||||||
|
answer, err := c.query(ctx, server, query)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
|
||||||
|
for line := range strings.SplitSeq(answer, "\n") {
|
||||||
|
if strings.HasPrefix(line, accessDeniedError) ||
|
||||||
|
strings.HasPrefix(line, accessLimitError) {
|
||||||
|
return "", fmt.Errorf("%w: %s", errRefused, line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return answer, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// query performs a raw WHOIS query to the specified server and returns the
|
||||||
|
// answer in UTF-8. An answer that reaches maxAnswerSize is an error.
|
||||||
func (c *Client) query(ctx context.Context, server, query string) (string, error) {
|
func (c *Client) query(ctx context.Context, server, query string) (string, error) {
|
||||||
conn, err := c.DialContext(ctx, "tcp", server)
|
conn, err := c.DialContext(ctx, "tcp", server)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -202,7 +378,25 @@ func (c *Client) query(ctx context.Context, server, query string) (string, error
|
|||||||
return "", fmt.Errorf("read response: %w", errAnswerTooLarge)
|
return "", fmt.Errorf("read response: %w", errAnswerTooLarge)
|
||||||
}
|
}
|
||||||
|
|
||||||
return sb.String(), nil
|
return toUTF8(sb.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
// toUTF8 returns a WHOIS answer in UTF-8. An answer that is valid UTF-8 is
|
||||||
|
// returned as it is. Any other answer is decoded as Latin-1, which RIPE,
|
||||||
|
// APNIC and NIC.br send. LACNIC, ARIN and AFRINIC send UTF-8. RIPE's
|
||||||
|
// database working group lists the five RIRs here:
|
||||||
|
// https://www.ripe.net/ripe/mail/archives/db-wg/2024-May/008075.html
|
||||||
|
func toUTF8(answer string) (string, error) {
|
||||||
|
if utf8.ValidString(answer) {
|
||||||
|
return answer, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
decoded, err := charmap.ISO8859_1.NewDecoder().String(answer)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("decode Latin-1 answer: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return decoded, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// parseIANAReferral extracts the RIR and WHOIS server from an IANA response.
|
// parseIANAReferral extracts the RIR and WHOIS server from an IANA response.
|
||||||
@@ -238,7 +432,9 @@ func (c *Client) parseIANAReferral(response string) (string, string, error) {
|
|||||||
return "", "", nil
|
return "", "", nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// parseResponse extracts ASN information from a WHOIS response.
|
// parseResponse extracts ASN information from a WHOIS response. Of an
|
||||||
|
// answer from RIPE, APNIC or AFRINIC it reads only the lines asObjectLines
|
||||||
|
// returns. Of an answer from ARIN or LACNIC it reads every line.
|
||||||
func (c *Client) parseResponse(asn int, rir, response string) *ASNInfo {
|
func (c *Client) parseResponse(asn int, rir, response string) *ASNInfo {
|
||||||
info := &ASNInfo{
|
info := &ASNInfo{
|
||||||
ASN: asn,
|
ASN: asn,
|
||||||
@@ -247,26 +443,15 @@ func (c *Client) parseResponse(asn int, rir, response string) *ASNInfo {
|
|||||||
}
|
}
|
||||||
|
|
||||||
lines := strings.Split(response, "\n")
|
lines := strings.Split(response, "\n")
|
||||||
|
if rir == RIRRIPE || rir == RIRAPNIC || rir == RIRAFRNIC {
|
||||||
|
lines = asObjectLines(lines)
|
||||||
|
}
|
||||||
|
|
||||||
var addressLines []string
|
var addressLines []string
|
||||||
|
|
||||||
for _, line := range lines {
|
for _, line := range lines {
|
||||||
line = strings.TrimSpace(line)
|
key, value, ok := splitField(line)
|
||||||
if line == "" || strings.HasPrefix(line, "%") ||
|
if !ok || value == "" {
|
||||||
strings.HasPrefix(line, "#") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
// Split on first colon
|
|
||||||
parts := strings.SplitN(line, ":", keyValueParts)
|
|
||||||
if len(parts) != keyValueParts {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
key := strings.TrimSpace(strings.ToLower(parts[0]))
|
|
||||||
value := strings.TrimSpace(parts[1])
|
|
||||||
|
|
||||||
if value == "" {
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -293,6 +478,72 @@ func (c *Client) parseResponse(asn int, rir, response string) *ASNInfo {
|
|||||||
return info
|
return info
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// splitField splits a "key: value" line of a WHOIS answer into its key, in
|
||||||
|
// lower case, and its value. The third result is false for a line that
|
||||||
|
// holds no field: a blank line, a comment line or a line without a colon.
|
||||||
|
func splitField(line string) (string, string, bool) {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if strings.HasPrefix(line, "%") || strings.HasPrefix(line, "#") {
|
||||||
|
return "", "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
key, value, ok := strings.Cut(line, ":")
|
||||||
|
|
||||||
|
return strings.ToLower(strings.TrimSpace(key)), strings.TrimSpace(value), ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// asObjectLines returns the lines of the aut-num object in an answer from
|
||||||
|
// RIPE, APNIC or AFRINIC, followed by the lines of the organisation object
|
||||||
|
// whose ID the aut-num object gives on its "org:" line. The answer's other
|
||||||
|
// objects are left out. RIPE and APNIC start their answer with the
|
||||||
|
// as-block object of the block the AS is in, whose dates are not the AS's.
|
||||||
|
//
|
||||||
|
// These answers are in RPSL. A blank line ends an object. An object's first
|
||||||
|
// line has the object's class as its key and the object's ID as its value,
|
||||||
|
// as in "organisation: ORG-APNI1-AP".
|
||||||
|
func asObjectLines(lines []string) []string {
|
||||||
|
var autNum []string
|
||||||
|
|
||||||
|
// The lines of each organisation object, by its ID.
|
||||||
|
organisations := make(map[string][]string)
|
||||||
|
|
||||||
|
// The class and ID of the object the line is in, "" between objects.
|
||||||
|
var class, id string
|
||||||
|
|
||||||
|
// The ID of the organisation object the aut-num object refers to.
|
||||||
|
var orgID string
|
||||||
|
|
||||||
|
for _, line := range lines {
|
||||||
|
if strings.TrimSpace(line) == "" {
|
||||||
|
class = ""
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
key, value, ok := splitField(line)
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if class == "" {
|
||||||
|
class, id = key, value
|
||||||
|
}
|
||||||
|
|
||||||
|
switch class {
|
||||||
|
case autNumClass:
|
||||||
|
autNum = append(autNum, line)
|
||||||
|
|
||||||
|
if key == orgKey {
|
||||||
|
orgID = value
|
||||||
|
}
|
||||||
|
case organisationClass:
|
||||||
|
organisations[id] = append(organisations[id], line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return append(autNum, organisations[orgID]...)
|
||||||
|
}
|
||||||
|
|
||||||
// setResponseField stores the value of one "key: value" line of a WHOIS
|
// setResponseField stores the value of one "key: value" line of a WHOIS
|
||||||
// response in info, keeping the first value seen for each field.
|
// response in info, keeping the first value seen for each field.
|
||||||
func (c *Client) setResponseField(info *ASNInfo, key, value string) {
|
func (c *Client) setResponseField(info *ASNInfo, key, value string) {
|
||||||
|
|||||||
+663
-28
@@ -2,15 +2,21 @@ package whois_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"io"
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"maps"
|
||||||
"net"
|
"net"
|
||||||
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"syscall"
|
"syscall"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/whois"
|
"git.eeqj.de/sneak/routewatch/internal/whois"
|
||||||
)
|
)
|
||||||
@@ -21,8 +27,20 @@ const (
|
|||||||
ripeServer = "whois.ripe.net:43"
|
ripeServer = "whois.ripe.net:43"
|
||||||
)
|
)
|
||||||
|
|
||||||
// testASN is the AS the tests look up.
|
// testASN is the AS the tests look up, and asQuery the query for it.
|
||||||
const testASN = 3333
|
const (
|
||||||
|
testASN = 3333
|
||||||
|
asQuery = "AS3333"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The fake servers' answers are keyed by a server's address, a space and a
|
||||||
|
// query. These are the keys of the queries LookupASN sends for testASN:
|
||||||
|
// IANA's, then RIPE's, which asks RIPE to leave out person and role
|
||||||
|
// objects.
|
||||||
|
const (
|
||||||
|
ianaQuery = ianaServer + " " + asQuery
|
||||||
|
ripeQuery = ripeServer + " --no-personal " + asQuery
|
||||||
|
)
|
||||||
|
|
||||||
// ianaReferral is IANA's answer for an AS in a block it gave to RIPE.
|
// ianaReferral is IANA's answer for an AS in a block it gave to RIPE.
|
||||||
const ianaReferral = `% IANA WHOIS server
|
const ianaReferral = `% IANA WHOIS server
|
||||||
@@ -36,27 +54,163 @@ changed: 2002-09-01
|
|||||||
source: IANA
|
source: IANA
|
||||||
`
|
`
|
||||||
|
|
||||||
// ripeRecord is RIPE's answer for the AS.
|
// lacnicReferral is IANA's answer for an AS in a block it gave to LACNIC,
|
||||||
|
// shortened to the referral.
|
||||||
|
const lacnicReferral = "refer: whois.lacnic.net\n"
|
||||||
|
|
||||||
|
// ripeRecord is RIPE's answer for the AS when asked to leave out person and
|
||||||
|
// role objects. It starts with the as-block object of the block the AS is
|
||||||
|
// in. Then come the abuse address as a comment, the AS's object and the
|
||||||
|
// organisation object it refers to.
|
||||||
const ripeRecord = `% This is the RIPE Database query service.
|
const ripeRecord = `% This is the RIPE Database query service.
|
||||||
|
% The objects are in RPSL format.
|
||||||
|
|
||||||
|
% Note: --no-personal means ALL personal data has been filtered from this output.
|
||||||
|
|
||||||
|
% Information related to 'AS3209 - AS3353'
|
||||||
|
|
||||||
|
as-block: AS3209 - AS3353
|
||||||
|
descr: RIPE NCC ASN block
|
||||||
|
remarks: These AS Numbers are assigned to network operators
|
||||||
|
remarks: in the RIPE NCC service region.
|
||||||
|
mnt-by: RIPE-NCC-HM-MNT
|
||||||
|
created: 2018-11-22T15:27:19Z
|
||||||
|
last-modified: 2018-11-22T15:27:19Z
|
||||||
|
source: RIPE
|
||||||
|
|
||||||
|
% Information related to 'AS3333'
|
||||||
|
|
||||||
|
% Abuse contact for 'AS3333' is 'abuse@ripe.net'
|
||||||
|
|
||||||
aut-num: AS3333
|
aut-num: AS3333
|
||||||
as-name: RIPE-NCC-AS
|
as-name: RIPE-NCC-AS
|
||||||
org: ORG-RIEN1-RIPE
|
org: ORG-RIEN1-RIPE
|
||||||
|
admin-c: BRD-RIPE
|
||||||
|
tech-c: OPS4-RIPE
|
||||||
|
abuse-c: OPS4-RIPE
|
||||||
|
mnt-by: RIPE-NCC-MNT
|
||||||
created: 2002-09-17T13:08:32Z
|
created: 2002-09-17T13:08:32Z
|
||||||
last-modified: 2024-01-09T08:55:41Z
|
last-modified: 2024-01-09T08:55:41Z
|
||||||
source: RIPE
|
source: RIPE
|
||||||
|
|
||||||
|
organisation: ORG-RIEN1-RIPE
|
||||||
|
org-name: Reseaux IP Europeens Network Coordination Centre (RIPE NCC)
|
||||||
|
country: NL
|
||||||
|
org-type: RIR
|
||||||
|
address: P.O. Box 10096
|
||||||
|
address: 1001EB Amsterdam
|
||||||
|
abuse-c: OPS4-RIPE
|
||||||
|
created: 2012-03-09T13:08:29Z
|
||||||
|
last-modified: 2024-03-26T10:55:38Z
|
||||||
|
source: RIPE
|
||||||
`
|
`
|
||||||
|
|
||||||
|
// The error lines with which RIPE refuses a query: access denied to the
|
||||||
|
// host, and the host's limit on returned objects reached.
|
||||||
|
const (
|
||||||
|
ripeAccessDenied = "%ERROR:201: access denied for 192.0.2.1"
|
||||||
|
ripeLimitReached = "%ERROR:202: access control limit reached"
|
||||||
|
)
|
||||||
|
|
||||||
// ripeRefusal is RIPE's answer to a host it no longer serves.
|
// ripeRefusal is RIPE's answer to a host it no longer serves.
|
||||||
const ripeRefusal = `% This is the RIPE Database query service.
|
const ripeRefusal = `% This is the RIPE Database query service.
|
||||||
% The objects are in RPSL format.
|
% The objects are in RPSL format.
|
||||||
%
|
%
|
||||||
%ERROR:201: access denied for 192.0.2.1
|
` + ripeAccessDenied + `
|
||||||
%
|
%
|
||||||
% Sorry, access from your host has been permanently
|
% Sorry, access from your host has been permanently
|
||||||
% denied because of a repeated excessive querying.
|
% denied because of a repeated excessive querying.
|
||||||
`
|
`
|
||||||
|
|
||||||
|
// apnicAfrinicRecord is the answer of APNIC or AFRINIC for the AS when asked
|
||||||
|
// to leave out person and role objects. Like RIPE's, it holds the abuse
|
||||||
|
// address as a comment, the AS's object and the organisation object.
|
||||||
|
const apnicAfrinicRecord = `% Abuse contact for 'AS3333' is 'abuse@example.net'
|
||||||
|
|
||||||
|
aut-num: AS3333
|
||||||
|
as-name: EXAMPLE-AS
|
||||||
|
org: ORG-EXAMPLE1
|
||||||
|
admin-c: EX1-TEST
|
||||||
|
last-modified: 2026-08-26T00:40:00Z
|
||||||
|
|
||||||
|
organisation: ORG-EXAMPLE1
|
||||||
|
org-name: Example Networks
|
||||||
|
country: ZA
|
||||||
|
address: 1 Example Road
|
||||||
|
`
|
||||||
|
|
||||||
|
// apnicServer is APNIC's WHOIS server, and apnicASN an AS that APNIC holds.
|
||||||
|
const (
|
||||||
|
apnicServer = "whois.apnic.net:43"
|
||||||
|
apnicASN = 4608
|
||||||
|
)
|
||||||
|
|
||||||
|
// The keys of the queries LookupASN sends for apnicASN: IANA's, then
|
||||||
|
// APNIC's, which asks APNIC to leave out person and role objects.
|
||||||
|
const (
|
||||||
|
apnicIANAQuery = ianaServer + " AS4608"
|
||||||
|
apnicQuery = apnicServer + " --no-personal AS4608"
|
||||||
|
)
|
||||||
|
|
||||||
|
// apnicRecord is APNIC's answer for apnicASN when asked to leave out person
|
||||||
|
// and role objects, shortened. It starts with the as-block object of the
|
||||||
|
// block the AS is in. Then come the abuse address as a comment, the AS's
|
||||||
|
// object, and the irt and organisation objects it refers to.
|
||||||
|
const apnicRecord = `% [whois.apnic.net]
|
||||||
|
% Whois data copyright terms http://www.apnic.net/db/dbcopyright.html
|
||||||
|
|
||||||
|
% Note: --no-personal means ALL personal data has been filtered from this output.
|
||||||
|
|
||||||
|
% Information related to 'AS4608 - AS4865'
|
||||||
|
|
||||||
|
as-block: AS4608 - AS4865
|
||||||
|
descr: APNIC ASN block
|
||||||
|
remarks: These AS numbers are further assigned by APNIC
|
||||||
|
remarks: to APNIC members and end-users in the APNIC region
|
||||||
|
mnt-by: APNIC-HM
|
||||||
|
mnt-lower: APNIC-HM
|
||||||
|
last-modified: 2019-05-20T03:52:26Z
|
||||||
|
source: APNIC
|
||||||
|
|
||||||
|
% Information related to 'AS4608'
|
||||||
|
|
||||||
|
% Abuse contact for 'AS4608' is 'abuse@apnic.net'
|
||||||
|
|
||||||
|
aut-num: AS4608
|
||||||
|
as-name: APNIC-SERVICES
|
||||||
|
descr: Asia Pacific Network Information Centre
|
||||||
|
country: AU
|
||||||
|
org: ORG-APNI1-AP
|
||||||
|
admin-c: AIC1-AP
|
||||||
|
abuse-c: AA1589-AP
|
||||||
|
mnt-by: APNIC-HM
|
||||||
|
mnt-irt: IRT-APNIC-IS-AP
|
||||||
|
last-modified: 2026-08-26T00:40:00Z
|
||||||
|
source: APNIC
|
||||||
|
|
||||||
|
irt: IRT-APNIC-IS-AP
|
||||||
|
address: South Brisbane, Australia
|
||||||
|
e-mail: helpdesk@apnic.net
|
||||||
|
abuse-mailbox: abuse@apnic.net
|
||||||
|
mnt-by: MAINT-APNIC-IS-AP
|
||||||
|
last-modified: 2025-11-18T00:27:08Z
|
||||||
|
source: APNIC
|
||||||
|
|
||||||
|
organisation: ORG-APNI1-AP
|
||||||
|
org-name: Asia Pacific Network Information Centre
|
||||||
|
org-type: LIR
|
||||||
|
country: AU
|
||||||
|
address: PO Box 3646
|
||||||
|
address: South Brisbane, QLD 4101,
|
||||||
|
address: Australia
|
||||||
|
e-mail: noc@apnic.net
|
||||||
|
mnt-by: APNIC-HM
|
||||||
|
last-modified: 2026-08-24T12:58:43Z
|
||||||
|
source: APNIC
|
||||||
|
|
||||||
|
% This query was served by the APNIC Whois Service version 1.88.48 (WHOIS-UK2)
|
||||||
|
`
|
||||||
|
|
||||||
// ianaPrivateUse is IANA's answer for a private-use AS: its block and no
|
// ianaPrivateUse is IANA's answer for a private-use AS: its block and no
|
||||||
// referral.
|
// referral.
|
||||||
const ianaPrivateUse = `% IANA WHOIS server
|
const ianaPrivateUse = `% IANA WHOIS server
|
||||||
@@ -80,20 +234,29 @@ const fillerLine = "remarks: and more\n"
|
|||||||
const endlessAnswerSize = 4 << 20
|
const endlessAnswerSize = 4 << 20
|
||||||
|
|
||||||
// fakeServers returns a DialContext for a whois.Client that connects to a
|
// fakeServers returns a DialContext for a whois.Client that connects to a
|
||||||
// fake server for each address in answers, which reads the query and sends
|
// fake server at each address answers names. Each key of answers is a
|
||||||
// answers[address]. Connecting to any other address is refused.
|
// server's address, a space and a query, and its value is that server's
|
||||||
|
// answer to that query. A server reads one query and sends its answer, or
|
||||||
|
// nothing for a query it has no answer for. Connecting to any other address
|
||||||
|
// is refused.
|
||||||
func fakeServers(
|
func fakeServers(
|
||||||
t *testing.T, answers map[string]string,
|
t *testing.T, answers map[string]string,
|
||||||
) func(context.Context, string, string) (net.Conn, error) {
|
) func(context.Context, string, string) (net.Conn, error) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
|
addresses := make(map[string]bool)
|
||||||
|
|
||||||
|
for key := range answers {
|
||||||
|
address, _, _ := strings.Cut(key, " ")
|
||||||
|
addresses[address] = true
|
||||||
|
}
|
||||||
|
|
||||||
var servers sync.WaitGroup
|
var servers sync.WaitGroup
|
||||||
|
|
||||||
t.Cleanup(servers.Wait)
|
t.Cleanup(servers.Wait)
|
||||||
|
|
||||||
return func(_ context.Context, _, address string) (net.Conn, error) {
|
return func(_ context.Context, _, address string) (net.Conn, error) {
|
||||||
answer, ok := answers[address]
|
if !addresses[address] {
|
||||||
if !ok {
|
|
||||||
return nil, syscall.ECONNREFUSED
|
return nil, syscall.ECONNREFUSED
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -102,7 +265,8 @@ func fakeServers(
|
|||||||
servers.Go(func() {
|
servers.Go(func() {
|
||||||
defer func() { _ = server.Close() }()
|
defer func() { _ = server.Close() }()
|
||||||
|
|
||||||
_, _ = bufio.NewReader(server).ReadString('\n')
|
query, _ := bufio.NewReader(server).ReadString('\n')
|
||||||
|
answer := answers[address+" "+strings.TrimSpace(query)]
|
||||||
_, _ = io.WriteString(server, answer)
|
_, _ = io.WriteString(server, answer)
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -116,31 +280,48 @@ func lookUp(
|
|||||||
) (*whois.ASNInfo, error) {
|
) (*whois.ASNInfo, error) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
client := whois.NewClient()
|
client := whois.NewClient(slog.New(slog.DiscardHandler))
|
||||||
client.DialContext = fakeServers(t, answers)
|
client.DialContext = fakeServers(t, answers)
|
||||||
|
|
||||||
return client.LookupASN(t.Context(), testASN)
|
return client.LookupASN(t.Context(), testASN)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestLookupASNReadsTheRIRAnswer looks up an AS IANA refers to RIPE. The
|
// TestLookupASNReadsTheRIRAnswer looks up an AS IANA refers to RIPE. RIPE
|
||||||
// fields come from RIPE's answer, the last-modified date included: IANA's
|
// answers only a query that leaves out person and role objects, and its
|
||||||
// "changed:" line dates its block, not the AS.
|
// answer to that still fills the AS's fields: the abuse address from the
|
||||||
|
// comment line, and the organisation's name and country from its object.
|
||||||
|
// The dates are the AS's. IANA's "changed:" line dates the AS's block, and
|
||||||
|
// so do the dates of the as-block object at the start of RIPE's answer.
|
||||||
func TestLookupASNReadsTheRIRAnswer(t *testing.T) {
|
func TestLookupASNReadsTheRIRAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
info, err := lookUp(t, map[string]string{
|
info, err := lookUp(t, map[string]string{
|
||||||
ianaServer: ianaReferral,
|
ianaQuery: ianaReferral,
|
||||||
ripeServer: ripeRecord,
|
ripeQuery: ripeRecord,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("LookupASN: %v", err)
|
t.Fatalf("LookupASN: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if info.ASName != "RIPE-NCC-AS" || info.OrgID != "ORG-RIEN1-RIPE" ||
|
got := []string{
|
||||||
info.RIR != whois.RIRRIPE {
|
info.ASName, info.OrgID, info.OrgName, info.CountryCode,
|
||||||
t.Errorf("LookupASN = AS name %q, org %q, RIR %q; "+
|
info.AbuseEmail, info.RIR,
|
||||||
"want RIPE-NCC-AS, ORG-RIEN1-RIPE, RIPE",
|
}
|
||||||
info.ASName, info.OrgID, info.RIR)
|
want := []string{
|
||||||
|
"RIPE-NCC-AS", "ORG-RIEN1-RIPE",
|
||||||
|
"Reseaux IP Europeens Network Coordination Centre (RIPE NCC)", "NL",
|
||||||
|
"abuse@ripe.net", whois.RIRRIPE,
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(got, want) {
|
||||||
|
t.Errorf("LookupASN = AS name, org ID, org name, country, abuse "+
|
||||||
|
"address, RIR %q; want %q", got, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
wantRegDate := time.Date(2002, 9, 17, 13, 8, 32, 0, time.UTC)
|
||||||
|
if info.RegDate == nil || !info.RegDate.Equal(wantRegDate) {
|
||||||
|
t.Errorf("LookupASN registered = %v, want %v",
|
||||||
|
info.RegDate, wantRegDate)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantLastMod := time.Date(2024, 1, 9, 8, 55, 41, 0, time.UTC)
|
wantLastMod := time.Date(2024, 1, 9, 8, 55, 41, 0, time.UTC)
|
||||||
@@ -150,6 +331,460 @@ func TestLookupASNReadsTheRIRAnswer(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestLookupASNReadsOnlyTheASAndItsOrganisation looks up an AS at APNIC,
|
||||||
|
// whose answer starts with the as-block object of the block the AS is in.
|
||||||
|
// The last-modified date is the aut-num object's, not the block's. The
|
||||||
|
// address is the organisation object's alone, without the irt object's.
|
||||||
|
func TestLookupASNReadsOnlyTheASAndItsOrganisation(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
client := whois.NewClient(slog.New(slog.DiscardHandler))
|
||||||
|
client.DialContext = fakeServers(t, map[string]string{
|
||||||
|
apnicIANAQuery: "refer: whois.apnic.net\n",
|
||||||
|
apnicQuery: apnicRecord,
|
||||||
|
})
|
||||||
|
|
||||||
|
info, err := client.LookupASN(t.Context(), apnicASN)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LookupASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
wantLastMod := time.Date(2026, 8, 26, 0, 40, 0, 0, time.UTC)
|
||||||
|
if info.LastMod == nil || !info.LastMod.Equal(wantLastMod) {
|
||||||
|
t.Errorf("LookupASN last modified = %v, want %v",
|
||||||
|
info.LastMod, wantLastMod)
|
||||||
|
}
|
||||||
|
|
||||||
|
got := []string{
|
||||||
|
info.ASName, info.OrgID, info.OrgName, info.CountryCode,
|
||||||
|
info.Address, info.AbuseEmail,
|
||||||
|
}
|
||||||
|
want := []string{
|
||||||
|
"APNIC-SERVICES", "ORG-APNI1-AP",
|
||||||
|
"Asia Pacific Network Information Centre", "AU",
|
||||||
|
"PO Box 3646\nSouth Brisbane, QLD 4101,\nAustralia", "abuse@apnic.net",
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(got, want) {
|
||||||
|
t.Errorf("LookupASN = AS name, org ID, org name, country, address, "+
|
||||||
|
"abuse address %q; want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLookupASNAsksAPNICAndAFRINICToLeaveOutPersonalData looks up an AS at
|
||||||
|
// APNIC and at AFRINIC. Each answers only a query that leaves out person
|
||||||
|
// and role objects, as RIPE does. That one answer still holds the
|
||||||
|
// organisation object, so the organisation's name and country are filled.
|
||||||
|
func TestLookupASNAsksAPNICAndAFRINICToLeaveOutPersonalData(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, server := range []string{"whois.apnic.net", "whois.afrinic.net"} {
|
||||||
|
t.Run(server, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
address := server + ":43"
|
||||||
|
|
||||||
|
info, err := lookUp(t, map[string]string{
|
||||||
|
ianaQuery: "refer: " + server + "\n",
|
||||||
|
address + " --no-personal " + asQuery: apnicAfrinicRecord,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LookupASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
got := []string{
|
||||||
|
info.ASName, info.OrgName, info.CountryCode, info.AbuseEmail,
|
||||||
|
}
|
||||||
|
want := []string{
|
||||||
|
"EXAMPLE-AS", "Example Networks", "ZA", "abuse@example.net",
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(got, want) {
|
||||||
|
t.Errorf("LookupASN = AS name, org name, country, abuse "+
|
||||||
|
"address %q; want %q", got, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The name of an organisation with accented letters, in Latin-1, one byte for
|
||||||
|
// each letter, and in UTF-8.
|
||||||
|
const (
|
||||||
|
latin1OrgName = "Fundaci\xf3n Ra\xedces de Espa\xf1a"
|
||||||
|
utf8OrgName = "Fundación Raíces de España"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestLookupASNStoresAnswersInUTF8 looks up an AS at RIPE, which answers in
|
||||||
|
// Latin-1, and at LACNIC, which answers in UTF-8. Both answers give the
|
||||||
|
// organisation's name with its accented letters in UTF-8, in the parsed field
|
||||||
|
// and in the stored answer. Before the fix the Latin-1 answer was kept as
|
||||||
|
// received, which is not valid UTF-8.
|
||||||
|
func TestLookupASNStoresAnswersInUTF8(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for name, answers := range map[string]map[string]string{
|
||||||
|
"RIPE in Latin-1": {
|
||||||
|
ianaQuery: ianaReferral,
|
||||||
|
ripeQuery: "aut-num: AS3333\n" +
|
||||||
|
"org-name: " + latin1OrgName + "\n",
|
||||||
|
},
|
||||||
|
"LACNIC in UTF-8": {
|
||||||
|
ianaQuery: lacnicReferral,
|
||||||
|
"whois.lacnic.net:43 " + asQuery: "aut-num: AS3333\n" +
|
||||||
|
"owner: " + utf8OrgName + "\n",
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
info, err := lookUp(t, answers)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LookupASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if info.OrgName != utf8OrgName {
|
||||||
|
t.Errorf("LookupASN org name = %q, want %q",
|
||||||
|
info.OrgName, utf8OrgName)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !utf8.ValidString(info.RawResponse) ||
|
||||||
|
!strings.Contains(info.RawResponse, utf8OrgName) {
|
||||||
|
t.Errorf("LookupASN answer = %q, want UTF-8 holding %q",
|
||||||
|
info.RawResponse, utf8OrgName)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// lacnicServer is LACNIC's WHOIS server and nicbrServer NIC.br's, which
|
||||||
|
// registers Brazil's ASes under LACNIC. brazilASN is one of those ASes.
|
||||||
|
const (
|
||||||
|
lacnicServer = "whois.lacnic.net:43"
|
||||||
|
nicbrServer = "whois.registro.br:43"
|
||||||
|
brazilASN = 52751
|
||||||
|
)
|
||||||
|
|
||||||
|
// The keys of the queries LookupASN sends for brazilASN: IANA's, LACNIC's,
|
||||||
|
// then NIC.br's.
|
||||||
|
const (
|
||||||
|
brazilIANAQuery = ianaServer + " AS52751"
|
||||||
|
brazilLACNICQuery = lacnicServer + " AS52751"
|
||||||
|
brazilNICbrQuery = nicbrServer + " AS52751"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The name of brazilASN's organisation in three forms: as LACNIC sends it,
|
||||||
|
// with each accented letter replaced by U+FFFD; as NIC.br sends it, in
|
||||||
|
// Latin-1; and in UTF-8.
|
||||||
|
const (
|
||||||
|
lacnicBrazilOrgName = "Brasrede Telecomunica\U0000FFFD\U0000FFFDes LTDA"
|
||||||
|
nicbrBrazilOrgName = "Brasrede Telecomunica\xe7\xf5es LTDA"
|
||||||
|
brazilOrgName = "Brasrede Telecomunicações LTDA"
|
||||||
|
)
|
||||||
|
|
||||||
|
// nicbrCopyright is the copyright notice at the top of NIC.br's answers.
|
||||||
|
const nicbrCopyright = "% Copyright (c) Nic.br - " +
|
||||||
|
"Use of this data is governed by the Use and\n" +
|
||||||
|
"% Privacy Policy at https://registro.br/upp . Distribution,\n" +
|
||||||
|
"% commercialization, reproduction, and use for advertising or similar\n" +
|
||||||
|
"% purposes are expressly prohibited.\n"
|
||||||
|
|
||||||
|
// lacnicBrazilRecord is LACNIC's answer for brazilASN, recorded on
|
||||||
|
// 2026-10-26. It is NIC.br's answer, which LACNIC passes on after a line
|
||||||
|
// of its own and a space. It is shortened. The client's address is a
|
||||||
|
// documentation address. The contact's name, the contact's object and the
|
||||||
|
// footer are left out.
|
||||||
|
const lacnicBrazilRecord = "% IP Client: 192.0.2.1\n" +
|
||||||
|
" " + nicbrCopyright + `% 2026-10-26T23:12:22-03:00 - 192.0.2.1
|
||||||
|
|
||||||
|
aut-num: AS52751
|
||||||
|
owner: ` + lacnicBrazilOrgName + `
|
||||||
|
owner-c: AVL217
|
||||||
|
routing-c: AVL217
|
||||||
|
abuse-c: AVL217
|
||||||
|
created: 20120829
|
||||||
|
changed: 20130306
|
||||||
|
inetnum: 177.38.152.0/21
|
||||||
|
inetnum: 2804:b34::/32
|
||||||
|
inetnum: 191.243.4.0/22
|
||||||
|
inetnum: 131.72.248.0/22
|
||||||
|
inetnum: 138.94.228.0/22
|
||||||
|
inetnum: 170.233.8.0/22
|
||||||
|
inetnum: 170.246.4.0/22
|
||||||
|
`
|
||||||
|
|
||||||
|
// nicbrRecord is NIC.br's answer for brazilASN, in Latin-1, recorded on
|
||||||
|
// 2026-10-26 and shortened as lacnicBrazilRecord is.
|
||||||
|
const nicbrRecord = nicbrCopyright + `% 2026-10-26T23:14:41-03:00 - 192.0.2.1
|
||||||
|
|
||||||
|
aut-num: AS52751
|
||||||
|
owner: ` + nicbrBrazilOrgName + `
|
||||||
|
owner-c: AVL217
|
||||||
|
routing-c: AVL217
|
||||||
|
abuse-c: AVL217
|
||||||
|
created: 20120829
|
||||||
|
changed: 20130306
|
||||||
|
inetnum: 177.38.152.0/21
|
||||||
|
inetnum: 2804:b34::/32
|
||||||
|
inetnum: 191.243.4.0/22
|
||||||
|
inetnum: 131.72.248.0/22
|
||||||
|
inetnum: 138.94.228.0/22
|
||||||
|
inetnum: 170.233.8.0/22
|
||||||
|
inetnum: 170.246.4.0/22
|
||||||
|
`
|
||||||
|
|
||||||
|
// nicbrOverLimit stands for NIC.br's answer for brazilASN to a client over
|
||||||
|
// its query limit, and is not recorded. Such an answer leaves information
|
||||||
|
// out, here the owner, and says so in its first line.
|
||||||
|
const nicbrOverLimit = `% Query rate limit exceeded. Reduced information.
|
||||||
|
|
||||||
|
aut-num: AS52751
|
||||||
|
created: 20120829
|
||||||
|
changed: 20130306
|
||||||
|
`
|
||||||
|
|
||||||
|
// newNICbrClient returns a client whose fake servers send answers and
|
||||||
|
// whose logger writes JSON lines to logs. It adds one to *nicbrAsks each
|
||||||
|
// time it connects to NIC.br's server.
|
||||||
|
func newNICbrClient(
|
||||||
|
t *testing.T, answers map[string]string, logs io.Writer, nicbrAsks *int,
|
||||||
|
) *whois.Client {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
dial := fakeServers(t, answers)
|
||||||
|
|
||||||
|
client := whois.NewClient(slog.New(slog.NewJSONHandler(logs, nil)))
|
||||||
|
client.DialContext = func(
|
||||||
|
ctx context.Context, network, address string,
|
||||||
|
) (net.Conn, error) {
|
||||||
|
if address == nicbrServer {
|
||||||
|
*nicbrAsks++
|
||||||
|
}
|
||||||
|
|
||||||
|
return dial(ctx, network, address)
|
||||||
|
}
|
||||||
|
|
||||||
|
return client
|
||||||
|
}
|
||||||
|
|
||||||
|
// lookUpBrazilianAS looks brazilASN up with fake servers sending answers,
|
||||||
|
// and reports whether the client connected to NIC.br's server.
|
||||||
|
func lookUpBrazilianAS(
|
||||||
|
t *testing.T, answers map[string]string,
|
||||||
|
) (*whois.ASNInfo, bool, error) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
nicbrAsks := 0
|
||||||
|
client := newNICbrClient(t, answers, io.Discard, &nicbrAsks)
|
||||||
|
|
||||||
|
info, err := client.LookupASN(t.Context(), brazilASN)
|
||||||
|
|
||||||
|
return info, nicbrAsks > 0, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLookupASNTakesABrazilianASFromNICbr looks up an AS that NIC.br
|
||||||
|
// registers. LACNIC's answer is NIC.br's with each accented letter replaced
|
||||||
|
// by U+FFFD, so the client also asks NIC.br's server, which answers in
|
||||||
|
// Latin-1. The organisation's name has its accented letters, in the parsed
|
||||||
|
// field and in the stored answer.
|
||||||
|
func TestLookupASNTakesABrazilianASFromNICbr(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
info, _, err := lookUpBrazilianAS(t, map[string]string{
|
||||||
|
brazilIANAQuery: lacnicReferral,
|
||||||
|
brazilLACNICQuery: lacnicBrazilRecord,
|
||||||
|
brazilNICbrQuery: nicbrRecord,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LookupASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if info.OrgName != brazilOrgName || info.RIR != whois.RIRLACNIC {
|
||||||
|
t.Errorf("LookupASN org name, RIR = %q, %q; want %q, %q",
|
||||||
|
info.OrgName, info.RIR, brazilOrgName, whois.RIRLACNIC)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !strings.Contains(info.RawResponse, brazilOrgName) {
|
||||||
|
t.Errorf("LookupASN answer = %q, want one holding %q",
|
||||||
|
info.RawResponse, brazilOrgName)
|
||||||
|
}
|
||||||
|
|
||||||
|
wantRegDate := time.Date(2012, 8, 29, 0, 0, 0, 0, time.UTC)
|
||||||
|
if info.RegDate == nil || !info.RegDate.Equal(wantRegDate) {
|
||||||
|
t.Errorf("LookupASN registered = %v, want %v",
|
||||||
|
info.RegDate, wantRegDate)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLookupASNAsksNICbrOnlyForItsASes looks up an AS at LACNIC whose
|
||||||
|
// answer does not carry NIC.br's copyright notice, as for an AS that LACNIC
|
||||||
|
// registers itself. The client does not connect to NIC.br's server, which
|
||||||
|
// would answer with a record, and keeps LACNIC's fields.
|
||||||
|
func TestLookupASNAsksNICbrOnlyForItsASes(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
info, askedNICbr, err := lookUpBrazilianAS(t, map[string]string{
|
||||||
|
brazilIANAQuery: lacnicReferral,
|
||||||
|
brazilLACNICQuery: "aut-num: AS52751\n" +
|
||||||
|
"owner: " + utf8OrgName + "\n",
|
||||||
|
brazilNICbrQuery: nicbrRecord,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LookupASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if askedNICbr {
|
||||||
|
t.Error("LookupASN asked NIC.br's server about an AS whose " +
|
||||||
|
"answer from LACNIC is not NIC.br's")
|
||||||
|
}
|
||||||
|
|
||||||
|
if info.OrgName != utf8OrgName {
|
||||||
|
t.Errorf("LookupASN org name = %q, want LACNIC's %q",
|
||||||
|
info.OrgName, utf8OrgName)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLookupASNKeepsLACNICsAnswerWhenNICbrFails looks up an AS that NIC.br
|
||||||
|
// registers when NIC.br's server refuses the connection, sends nothing, or
|
||||||
|
// sends the reduced answer for a client over its query limit. The client
|
||||||
|
// asks NIC.br, and the lookup still succeeds with LACNIC's fields. The
|
||||||
|
// client logs one warning, which names the AS and the reason.
|
||||||
|
func TestLookupASNKeepsLACNICsAnswerWhenNICbrFails(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for name, test := range map[string]struct {
|
||||||
|
nicbrAnswer map[string]string
|
||||||
|
reason string
|
||||||
|
}{
|
||||||
|
"connection refused": {
|
||||||
|
nicbrAnswer: map[string]string{},
|
||||||
|
reason: syscall.ECONNREFUSED.Error(),
|
||||||
|
},
|
||||||
|
"no answer": {
|
||||||
|
nicbrAnswer: map[string]string{brazilNICbrQuery: ""},
|
||||||
|
reason: "no WHOIS fields",
|
||||||
|
},
|
||||||
|
"over the query limit": {
|
||||||
|
nicbrAnswer: map[string]string{brazilNICbrQuery: nicbrOverLimit},
|
||||||
|
reason: "over its query limit",
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
answers := map[string]string{
|
||||||
|
brazilIANAQuery: lacnicReferral,
|
||||||
|
brazilLACNICQuery: lacnicBrazilRecord,
|
||||||
|
}
|
||||||
|
maps.Copy(answers, test.nicbrAnswer)
|
||||||
|
|
||||||
|
var logs bytes.Buffer
|
||||||
|
|
||||||
|
nicbrAsks := 0
|
||||||
|
client := newNICbrClient(t, answers, &logs, &nicbrAsks)
|
||||||
|
|
||||||
|
info, err := client.LookupASN(t.Context(), brazilASN)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LookupASN: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if nicbrAsks == 0 {
|
||||||
|
t.Error("LookupASN did not ask NIC.br's server")
|
||||||
|
}
|
||||||
|
|
||||||
|
if info.OrgName != lacnicBrazilOrgName {
|
||||||
|
t.Errorf("LookupASN org name = %q, want LACNIC's %q",
|
||||||
|
info.OrgName, lacnicBrazilOrgName)
|
||||||
|
}
|
||||||
|
|
||||||
|
var logged struct {
|
||||||
|
Level string `json:"level"`
|
||||||
|
ASN int `json:"asn"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err = json.Unmarshal(logs.Bytes(), &logged)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("want one log line, got %q: %v", logs.String(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if logged.Level != slog.LevelWarn.String() ||
|
||||||
|
logged.ASN != brazilASN ||
|
||||||
|
!strings.Contains(logs.String(), test.reason) {
|
||||||
|
t.Errorf("logged %q, want a warning for AS%d holding %q",
|
||||||
|
logs.String(), brazilASN, test.reason)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLookupASNSkipsNICbrAfterItsLimit looks up an AS that NIC.br
|
||||||
|
// registers three times with one client, while NIC.br answers that the
|
||||||
|
// client is over its query limit. The first lookup asks NIC.br. The
|
||||||
|
// second, a second before NICbrSkipTime has passed, does not. The third,
|
||||||
|
// once NICbrSkipTime has passed, asks again. Each keeps LACNIC's fields.
|
||||||
|
func TestLookupASNSkipsNICbrAfterItsLimit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
nicbrAsks := 0
|
||||||
|
client := newNICbrClient(t, map[string]string{
|
||||||
|
brazilIANAQuery: lacnicReferral,
|
||||||
|
brazilLACNICQuery: lacnicBrazilRecord,
|
||||||
|
brazilNICbrQuery: nicbrOverLimit,
|
||||||
|
}, io.Discard, &nicbrAsks)
|
||||||
|
|
||||||
|
start := time.Date(2026, 10, 27, 0, 0, 0, 0, time.UTC)
|
||||||
|
now := start
|
||||||
|
client.Now = func() time.Time { return now }
|
||||||
|
|
||||||
|
for _, step := range []struct {
|
||||||
|
after time.Duration
|
||||||
|
wantAsks int
|
||||||
|
}{
|
||||||
|
{after: 0, wantAsks: 1},
|
||||||
|
{after: whois.NICbrSkipTime - time.Second, wantAsks: 1},
|
||||||
|
{after: whois.NICbrSkipTime, wantAsks: 2},
|
||||||
|
} {
|
||||||
|
now = start.Add(step.after)
|
||||||
|
|
||||||
|
info, err := client.LookupASN(t.Context(), brazilASN)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LookupASN after %v: %v", step.after, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if nicbrAsks != step.wantAsks {
|
||||||
|
t.Errorf("after %v NIC.br's server was asked %d times, want %d",
|
||||||
|
step.after, nicbrAsks, step.wantAsks)
|
||||||
|
}
|
||||||
|
|
||||||
|
if info.OrgName != lacnicBrazilOrgName {
|
||||||
|
t.Errorf("LookupASN after %v org name = %q, want LACNIC's %q",
|
||||||
|
step.after, info.OrgName, lacnicBrazilOrgName)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestLookupASNReportsARefusal looks up an AS IANA refers to RIPE when RIPE
|
||||||
|
// refuses the query, with each of the error lines RIPE refuses with. The
|
||||||
|
// error holds RIPE's line, so the log says why the lookup failed.
|
||||||
|
func TestLookupASNReportsARefusal(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, line := range []string{ripeAccessDenied, ripeLimitReached} {
|
||||||
|
t.Run(line, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
_, err := lookUp(t, map[string]string{
|
||||||
|
ianaQuery: ianaReferral,
|
||||||
|
ripeQuery: "% This is the RIPE Database query service.\n" +
|
||||||
|
line + "\n",
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), line) {
|
||||||
|
t.Errorf("LookupASN error = %v, want one holding %q",
|
||||||
|
err, line)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestLookupASNFailsWhenTheRIRGivesNoAnswer looks up an AS IANA refers to
|
// TestLookupASNFailsWhenTheRIRGivesNoAnswer looks up an AS IANA refers to
|
||||||
// RIPE when RIPE's server refuses the connection, and when it answers only
|
// RIPE when RIPE's server refuses the connection, and when it answers only
|
||||||
// with an error. Neither is a lookup that found the AS.
|
// with an error. Neither is a lookup that found the AS.
|
||||||
@@ -157,8 +792,8 @@ func TestLookupASNFailsWhenTheRIRGivesNoAnswer(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for name, answers := range map[string]map[string]string{
|
for name, answers := range map[string]map[string]string{
|
||||||
"connection refused": {ianaServer: ianaReferral},
|
"connection refused": {ianaQuery: ianaReferral},
|
||||||
"only an error": {ianaServer: ianaReferral, ripeServer: ripeRefusal},
|
"only an error": {ianaQuery: ianaReferral, ripeQuery: ripeRefusal},
|
||||||
} {
|
} {
|
||||||
t.Run(name, func(t *testing.T) {
|
t.Run(name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -177,7 +812,7 @@ func TestLookupASNFailsWhenTheRIRGivesNoAnswer(t *testing.T) {
|
|||||||
func TestLookupASNReportsNoReferral(t *testing.T) {
|
func TestLookupASNReportsNoReferral(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := lookUp(t, map[string]string{ianaServer: ianaPrivateUse})
|
_, err := lookUp(t, map[string]string{ianaQuery: ianaPrivateUse})
|
||||||
if !errors.Is(err, whois.ErrNoReferral) {
|
if !errors.Is(err, whois.ErrNoReferral) {
|
||||||
t.Errorf("LookupASN error = %v, want ErrNoReferral", err)
|
t.Errorf("LookupASN error = %v, want ErrNoReferral", err)
|
||||||
}
|
}
|
||||||
@@ -189,7 +824,7 @@ func TestLookupASNReportsNoReferral(t *testing.T) {
|
|||||||
func TestLookupASNFailsWhenIANAAnswersNoBlock(t *testing.T) {
|
func TestLookupASNFailsWhenIANAAnswersNoBlock(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := lookUp(t, map[string]string{ianaServer: ianaNoObject})
|
_, err := lookUp(t, map[string]string{ianaQuery: ianaNoObject})
|
||||||
if err == nil || errors.Is(err, whois.ErrNoReferral) {
|
if err == nil || errors.Is(err, whois.ErrNoReferral) {
|
||||||
t.Errorf("LookupASN error = %v, want one that is not ErrNoReferral",
|
t.Errorf("LookupASN error = %v, want one that is not ErrNoReferral",
|
||||||
err)
|
err)
|
||||||
@@ -206,8 +841,8 @@ func TestLookupASNFailsWhenTheRIRAnswerDoesNotEnd(t *testing.T) {
|
|||||||
strings.Repeat(fillerLine, endlessAnswerSize/len(fillerLine))
|
strings.Repeat(fillerLine, endlessAnswerSize/len(fillerLine))
|
||||||
|
|
||||||
_, err := lookUp(t, map[string]string{
|
_, err := lookUp(t, map[string]string{
|
||||||
ianaServer: ianaReferral,
|
ianaQuery: ianaReferral,
|
||||||
ripeServer: endless,
|
ripeQuery: endless,
|
||||||
})
|
})
|
||||||
if err == nil || errors.Is(err, whois.ErrNoReferral) {
|
if err == nil || errors.Is(err, whois.ErrNoReferral) {
|
||||||
t.Errorf("LookupASN error = %v, want one that is not ErrNoReferral",
|
t.Errorf("LookupASN error = %v, want one that is not ErrNoReferral",
|
||||||
@@ -233,9 +868,9 @@ func TestLookupASNFailsOnAReferralToAnotherServer(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := lookUp(t, map[string]string{
|
_, err := lookUp(t, map[string]string{
|
||||||
ianaServer: "refer: " + refer + "\n\n" +
|
ianaQuery: "refer: " + refer + "\n\n" +
|
||||||
"as-block: 3154-3353\n",
|
"as-block: 3154-3353\n",
|
||||||
address: ripeRecord,
|
address + " " + asQuery: ripeRecord,
|
||||||
})
|
})
|
||||||
if err == nil || errors.Is(err, whois.ErrNoReferral) {
|
if err == nil || errors.Is(err, whois.ErrNoReferral) {
|
||||||
t.Errorf("LookupASN error = %v, want one that is not "+
|
t.Errorf("LookupASN error = %v, want one that is not "+
|
||||||
|
|||||||
Binary file not shown.
+49
-23
@@ -2,10 +2,13 @@ package asinfo
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
|
"cmp"
|
||||||
"compress/gzip"
|
"compress/gzip"
|
||||||
_ "embed"
|
_ "embed"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"io"
|
"io"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -23,6 +26,17 @@ type Info struct {
|
|||||||
type Registry struct {
|
type Registry struct {
|
||||||
mu sync.RWMutex
|
mu sync.RWMutex
|
||||||
byASN map[int]*Info
|
byASN map[int]*Info
|
||||||
|
// searchEntries holds every entry in AS number order, for Search.
|
||||||
|
searchEntries []searchEntry
|
||||||
|
}
|
||||||
|
|
||||||
|
// searchEntry is an entry with its handle and description in lower case. They
|
||||||
|
// are put in lower case once, when the data loads, so that Search only
|
||||||
|
// compares.
|
||||||
|
type searchEntry struct {
|
||||||
|
info *Info
|
||||||
|
lowerHandle string
|
||||||
|
lowerDescription string
|
||||||
}
|
}
|
||||||
|
|
||||||
var (
|
var (
|
||||||
@@ -59,9 +73,21 @@ func initRegistry() {
|
|||||||
panic("failed to unmarshal embedded AS data: " + err.Error())
|
panic("failed to unmarshal embedded AS data: " + err.Error())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
slices.SortFunc(asInfos, func(a, b Info) int {
|
||||||
|
return cmp.Compare(a.ASN, b.ASN)
|
||||||
|
})
|
||||||
|
|
||||||
|
defaultRegistry.searchEntries = make([]searchEntry, 0, len(asInfos))
|
||||||
|
|
||||||
for i := range asInfos {
|
for i := range asInfos {
|
||||||
info := &asInfos[i]
|
info := &asInfos[i]
|
||||||
defaultRegistry.byASN[info.ASN] = info
|
defaultRegistry.byASN[info.ASN] = info
|
||||||
|
defaultRegistry.searchEntries = append(defaultRegistry.searchEntries,
|
||||||
|
searchEntry{
|
||||||
|
info: info,
|
||||||
|
lowerHandle: strings.ToLower(info.Handle),
|
||||||
|
lowerDescription: strings.ToLower(info.Description),
|
||||||
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -129,36 +155,36 @@ func All() []Info {
|
|||||||
return result
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
// Search searches for AS entries where the handle or description contains the query
|
// Search finds the AS entries whose handle or description contains query,
|
||||||
// This is a simple case-sensitive substring search
|
// ignoring case. It returns the first limit of them in AS number order, and the
|
||||||
func Search(query string) []Info {
|
// number of entries that match. An empty query matches nothing.
|
||||||
|
func Search(query string, limit int) ([]Info, int) {
|
||||||
once.Do(initRegistry)
|
once.Do(initRegistry)
|
||||||
|
|
||||||
|
if query == "" {
|
||||||
|
return nil, 0
|
||||||
|
}
|
||||||
|
|
||||||
|
query = strings.ToLower(query)
|
||||||
|
|
||||||
defaultRegistry.mu.RLock()
|
defaultRegistry.mu.RLock()
|
||||||
defer defaultRegistry.mu.RUnlock()
|
defer defaultRegistry.mu.RUnlock()
|
||||||
|
|
||||||
var results []Info
|
var (
|
||||||
|
results []Info
|
||||||
|
total int
|
||||||
|
)
|
||||||
|
|
||||||
for _, info := range defaultRegistry.byASN {
|
for _, entry := range defaultRegistry.searchEntries {
|
||||||
if contains(info.Handle, query) || contains(info.Description, query) {
|
if strings.Contains(entry.lowerHandle, query) ||
|
||||||
results = append(results, *info)
|
strings.Contains(entry.lowerDescription, query) {
|
||||||
|
if total < limit {
|
||||||
|
results = append(results, *entry.info)
|
||||||
|
}
|
||||||
|
|
||||||
|
total++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return results
|
return results, total
|
||||||
}
|
|
||||||
|
|
||||||
// contains is a simple substring search
|
|
||||||
func contains(s, substr string) bool {
|
|
||||||
return len(substr) > 0 && len(s) >= len(substr) && containsImpl(s, substr)
|
|
||||||
}
|
|
||||||
|
|
||||||
func containsImpl(s, substr string) bool {
|
|
||||||
for i := 0; i <= len(s)-len(substr); i++ {
|
|
||||||
if s[i:i+len(substr)] == substr {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return false
|
|
||||||
}
|
}
|
||||||
|
|||||||
+112
-20
@@ -1,6 +1,8 @@
|
|||||||
package asinfo_test
|
package asinfo_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"cmp"
|
||||||
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
@@ -20,7 +22,7 @@ func TestGet(t *testing.T) {
|
|||||||
name: "AS1 Level 3",
|
name: "AS1 Level 3",
|
||||||
asn: 1,
|
asn: 1,
|
||||||
wantOK: true,
|
wantOK: true,
|
||||||
wantDesc: "Level 3 Parent, LLC",
|
wantDesc: "Level 3 Parent LLC",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "AS15169 Google",
|
name: "AS15169 Google",
|
||||||
@@ -34,10 +36,10 @@ func TestGet(t *testing.T) {
|
|||||||
wantOK: false,
|
wantOK: false,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "AS0 IANA Reserved",
|
name: "AS1251 with accented letters",
|
||||||
asn: 0,
|
asn: 1251,
|
||||||
wantOK: true,
|
wantOK: true,
|
||||||
wantDesc: "Internet Assigned Numbers Authority",
|
wantDesc: "FUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO SÃO PAULO",
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -67,7 +69,7 @@ func TestGetDescription(t *testing.T) {
|
|||||||
asn int
|
asn int
|
||||||
want string
|
want string
|
||||||
}{
|
}{
|
||||||
{1, "Level 3 Parent, LLC"},
|
{1, "Level 3 Parent LLC"},
|
||||||
{999999999, ""},
|
{999999999, ""},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -90,8 +92,7 @@ func TestGetHandle(t *testing.T) {
|
|||||||
asn int
|
asn int
|
||||||
want string
|
want string
|
||||||
}{
|
}{
|
||||||
{1, "LVLT"},
|
{1, "LVLT-1"},
|
||||||
{0, "IANA-RSVD"},
|
|
||||||
{999999999, ""},
|
{999999999, ""},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -141,9 +142,9 @@ func TestSearch(t *testing.T) {
|
|||||||
wantMin: 10,
|
wantMin: 10,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "Empty search",
|
name: "Search for São Paulo",
|
||||||
query: "",
|
query: "São Paulo",
|
||||||
wantMin: 0,
|
wantMin: 1,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "Non-existent org",
|
name: "Non-existent org",
|
||||||
@@ -156,26 +157,115 @@ func TestSearch(t *testing.T) {
|
|||||||
t.Run(tt.name, func(t *testing.T) {
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
results := asinfo.Search(tt.query)
|
results, _ := asinfo.Search(tt.query, asinfo.Total())
|
||||||
if len(results) < tt.wantMin {
|
if len(results) < tt.wantMin {
|
||||||
t.Errorf("Search(%q) returned %d results, want at least %d",
|
t.Errorf("Search(%q) returned %d results, want at least %d",
|
||||||
tt.query, len(results), tt.wantMin)
|
tt.query, len(results), tt.wantMin)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Verify all results contain the query
|
// Verify all results contain the query, ignoring case
|
||||||
if tt.query != "" {
|
query := strings.ToLower(tt.query)
|
||||||
for _, r := range results {
|
for _, r := range results {
|
||||||
if !strings.Contains(r.Handle, tt.query) &&
|
if !strings.Contains(strings.ToLower(r.Handle), query) &&
|
||||||
!strings.Contains(r.Description, tt.query) {
|
!strings.Contains(strings.ToLower(r.Description), query) {
|
||||||
t.Errorf("Search result ASN %d doesn't contain query %q",
|
t.Errorf("Search result ASN %d doesn't contain query %q",
|
||||||
r.ASN, tt.query)
|
r.ASN, tt.query)
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestSearchEmptyQueryMatchesNothing expects an empty query to match no AS,
|
||||||
|
// although every handle contains the empty string.
|
||||||
|
func TestSearchEmptyQueryMatchesNothing(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
results, total := asinfo.Search("", asinfo.Total())
|
||||||
|
if len(results) != 0 || total != 0 {
|
||||||
|
t.Errorf(`Search("") returned %d results and a total of %d, want none`,
|
||||||
|
len(results), total)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSearchIgnoresCase searches for Cloudflare's handle in lower case and its
|
||||||
|
// description in upper case, and expects both searches to find AS13335.
|
||||||
|
func TestSearchIgnoresCase(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const cloudflareASN = 13335 // handle CLOUDFLARENET, "Cloudflare Inc."
|
||||||
|
|
||||||
|
isCloudflare := func(info asinfo.Info) bool {
|
||||||
|
return info.ASN == cloudflareASN
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, query := range []string{"cloudflarenet", "CLOUDFLARE INC."} {
|
||||||
|
results, _ := asinfo.Search(query, asinfo.Total())
|
||||||
|
if !slices.ContainsFunc(results, isCloudflare) {
|
||||||
|
t.Errorf("Search(%q) did not find AS%d", query, cloudflareASN)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSearchOrdersByASN expects the matches of a search with many of them to
|
||||||
|
// come in AS number order.
|
||||||
|
func TestSearchOrdersByASN(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const query = "University"
|
||||||
|
|
||||||
|
results, _ := asinfo.Search(query, asinfo.Total())
|
||||||
|
|
||||||
|
for i := 1; i < len(results); i++ {
|
||||||
|
if results[i-1].ASN >= results[i].ASN {
|
||||||
|
t.Fatalf("Search(%q): AS%d comes after AS%d",
|
||||||
|
query, results[i].ASN, results[i-1].ASN)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestSearchStopsAtLimit searches for a word that matches more entries than
|
||||||
|
// the limit. It expects exactly limit entries, the first matches in AS number
|
||||||
|
// order, and the number of all matches, counted here from All.
|
||||||
|
func TestSearchStopsAtLimit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const (
|
||||||
|
query = "university"
|
||||||
|
limit = 10
|
||||||
|
)
|
||||||
|
|
||||||
|
var matches []asinfo.Info
|
||||||
|
|
||||||
|
for _, info := range asinfo.All() {
|
||||||
|
if strings.Contains(strings.ToLower(info.Handle), query) ||
|
||||||
|
strings.Contains(strings.ToLower(info.Description), query) {
|
||||||
|
matches = append(matches, info)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(matches) <= limit {
|
||||||
|
t.Fatalf("%q matches %d entries; the test needs more than %d",
|
||||||
|
query, len(matches), limit)
|
||||||
|
}
|
||||||
|
|
||||||
|
slices.SortFunc(matches, func(a, b asinfo.Info) int {
|
||||||
|
return cmp.Compare(a.ASN, b.ASN)
|
||||||
|
})
|
||||||
|
|
||||||
|
results, total := asinfo.Search(query, limit)
|
||||||
|
|
||||||
|
if total != len(matches) {
|
||||||
|
t.Errorf("Search(%q, %d): total %d, want %d",
|
||||||
|
query, limit, total, len(matches))
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(results, matches[:limit]) {
|
||||||
|
t.Errorf("Search(%q, %d) = %v, want %v",
|
||||||
|
query, limit, results, matches[:limit])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestDataIntegrity(t *testing.T) {
|
func TestDataIntegrity(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -215,11 +305,13 @@ func BenchmarkGet(b *testing.B) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func BenchmarkSearch(b *testing.B) {
|
func BenchmarkSearch(b *testing.B) {
|
||||||
|
const limit = 50
|
||||||
|
|
||||||
queries := []string{"Google", "Amazon", "Microsoft", "University"}
|
queries := []string{"Google", "Amazon", "Microsoft", "University"}
|
||||||
|
|
||||||
b.ResetTimer()
|
b.ResetTimer()
|
||||||
|
|
||||||
for i := range b.N {
|
for i := range b.N {
|
||||||
asinfo.Search(queries[i%len(queries)])
|
asinfo.Search(queries[i%len(queries)], limit)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+5
-3
@@ -2,7 +2,7 @@
|
|||||||
Package asinfo provides information about Autonomous Systems (AS).
|
Package asinfo provides information about Autonomous Systems (AS).
|
||||||
|
|
||||||
The package embeds a comprehensive database of AS numbers, handles, and descriptions
|
The package embeds a comprehensive database of AS numbers, handles, and descriptions
|
||||||
sourced from https://github.com/ipverse/asn-info. The data is embedded at compile
|
sourced from https://github.com/ipverse/as-metadata. The data is embedded at compile
|
||||||
time and provides fast, offline lookups.
|
time and provides fast, offline lookups.
|
||||||
|
|
||||||
Basic usage:
|
Basic usage:
|
||||||
@@ -16,8 +16,10 @@ Basic usage:
|
|||||||
// Get just the description
|
// Get just the description
|
||||||
desc := asinfo.GetDescription(15169) // "Google LLC"
|
desc := asinfo.GetDescription(15169) // "Google LLC"
|
||||||
|
|
||||||
// Search for AS entries
|
// Search the handles and descriptions, ignoring case, for the first 10
|
||||||
results := asinfo.Search("University")
|
// matches in AS number order and the number of all matches
|
||||||
|
results, total := asinfo.Search("university", 10)
|
||||||
|
fmt.Printf("%d matches\n", total)
|
||||||
for _, as := range results {
|
for _, as := range results {
|
||||||
fmt.Printf("AS%d: %s\n", as.ASN, as.Description)
|
fmt.Printf("AS%d: %s\n", as.ASN, as.Description)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,13 +17,20 @@ func ExampleGet() {
|
|||||||
func ExampleGetDescription() {
|
func ExampleGetDescription() {
|
||||||
desc := asinfo.GetDescription(13335)
|
desc := asinfo.GetDescription(13335)
|
||||||
fmt.Println(desc)
|
fmt.Println(desc)
|
||||||
// Output: Cloudflare, Inc.
|
// Output: Cloudflare Inc.
|
||||||
}
|
}
|
||||||
|
|
||||||
func ExampleSearch() {
|
func ExampleSearch() {
|
||||||
results := asinfo.Search("MIT-GATEWAY")
|
const limit = 10
|
||||||
|
|
||||||
|
results, total := asinfo.Search("mit-gateway", limit)
|
||||||
|
|
||||||
|
fmt.Printf("%d of %d matches\n", len(results), total)
|
||||||
|
|
||||||
for _, info := range results {
|
for _, info := range results {
|
||||||
fmt.Printf("AS%d: %s - %s\n", info.ASN, info.Handle, info.Description)
|
fmt.Printf("AS%d: %s - %s\n", info.ASN, info.Handle, info.Description)
|
||||||
}
|
}
|
||||||
// Output: AS3: MIT-GATEWAYS - Massachusetts Institute of Technology
|
// Output:
|
||||||
|
// 1 of 1 matches
|
||||||
|
// AS3: MIT-GATEWAYS - Massachusetts Institute of Technology
|
||||||
}
|
}
|
||||||
|
|||||||
Executable
+34
@@ -0,0 +1,34 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/asupdate: rebuild the AS list built into RouteWatch from
|
||||||
|
# as.csv with cmd/asinfo-gen, which fails when it finds no AS. The new
|
||||||
|
# list replaces the old one only when the generator succeeds. Otherwise
|
||||||
|
# the old list stays as it was and the script exits non-zero.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
AS_DATA="pkg/asinfo/asdata.json.gz"
|
||||||
|
# Next to AS_DATA, so the final mv is a rename on one file system.
|
||||||
|
AS_DATA_TMP="$AS_DATA.tmp"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
|
||||||
|
json="$(mktemp)"
|
||||||
|
trap 'rm -f "$json" "$AS_DATA_TMP"' EXIT
|
||||||
|
# A signal exits through the EXIT trap too.
|
||||||
|
trap 'exit 1' HUP INT TERM
|
||||||
|
|
||||||
|
echo "Updating AS info data..."
|
||||||
|
if ! go run ./cmd/asinfo-gen >"$json"; then
|
||||||
|
echo "asupdate: the generator failed; $AS_DATA is unchanged" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# -n leaves the time out of the file, so an unchanged list gives the
|
||||||
|
# same bytes.
|
||||||
|
gzip -n -c <"$json" >"$AS_DATA_TMP"
|
||||||
|
mv "$AS_DATA_TMP" "$AS_DATA"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Reference in New Issue
Block a user