Give every IP lookup result the same fields (closes #200)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
Every result of /ip/, /ip/{addr} and /api/v1/ip/{ip} now has every
field. A field without a value is null instead of left out, so an
address with no live route has all its route and AS fields set to null.
ptr is an empty list when there are no PTR names.
Two new fields, num_origins and origin_asns, give how many origin ASes
peers carry the netblock with, and which. The routing table's prefix
summary now lists them. asn still names the origin that most peers
carry.
first_seen is null, no longer the zero time, when the database has not
stored the netblock yet.
Model: opus-5-5
This commit is contained in:
@@ -141,15 +141,39 @@ one of its peers sends an update.
|
|||||||
last full clock minute.
|
last full clock minute.
|
||||||
- `GET /api/v1/ip/{ip}` - Looks up each address. `{ip}` is an IP address, a host
|
- `GET /api/v1/ip/{ip}` - Looks up each address. `{ip}` is an IP address, a host
|
||||||
name, or a comma-separated list of at most 10 of these. `results` has one
|
name, or a comma-separated list of at most 10 of these. `results` has one
|
||||||
entry for each address, with:
|
entry for each address. Every entry has the same fields:
|
||||||
- `ip` and `ptr`: the address and its PTR names
|
- `ip`: the address
|
||||||
- `netblock` and the fields beside it: the most specific prefix that holds
|
- `ptr`: the address's PTR names, or an empty list when it has none
|
||||||
the address, and its origin AS with that AS's WHOIS details. For a prefix
|
- `blocklists`: the public blocklists the address is on (see Blocklists), or
|
||||||
with several origin ASes, `asn` is the one that most peers carry.
|
an empty list when it is on none
|
||||||
`last_seen` is the time of the prefix's newest kept route.
|
|
||||||
- `blocklists`: the public blocklists the address is on (see Blocklists)
|
Every other field is about the address's live route. Each is null when the
|
||||||
|
address has no live route. The fields about the route:
|
||||||
|
- `netblock`: the most specific prefix that holds the address
|
||||||
|
- `mask_length`: the netblock's mask length
|
||||||
|
- `ip_version`: the netblock's IP version, 4 or 6
|
||||||
|
- `num_peers`: the number of peers that carry the netblock
|
||||||
|
- `asn`: the origin AS that the most peers carry the netblock with
|
||||||
|
- `num_origins`: the number of origin ASes that peers carry the netblock
|
||||||
|
with
|
||||||
|
- `origin_asns`: those origin ASes, in AS number order
|
||||||
|
- `first_seen`: when RouteWatch first saw the netblock, or null when the
|
||||||
|
database has not stored the netblock yet
|
||||||
|
- `last_seen`: the time of the netblock's newest kept route, or null when
|
||||||
|
the netblock keeps no route
|
||||||
|
|
||||||
|
The fields about the AS in `asn` come from the database. Each is also null
|
||||||
|
when the database does not have it:
|
||||||
|
- `handle`: the AS's handle
|
||||||
|
- `description`: the AS's description
|
||||||
|
- `as_name`: the AS's name in its WHOIS details
|
||||||
|
- `org_name`: the name of the organisation that holds the AS
|
||||||
|
- `org_id`: the organisation's ID at its registry
|
||||||
|
- `address`: the organisation's postal address
|
||||||
|
- `country_code`: the organisation's country code
|
||||||
|
- `abuse_email`: the address that takes abuse reports for the AS
|
||||||
|
- `rir`: the regional internet registry of the AS
|
||||||
|
|
||||||
An address with no live route has a result without the route and AS fields.
|
|
||||||
Of the addresses a host name resolves to, only public ones are looked up. A
|
Of the addresses a host name resolves to, only public ones are looked up. A
|
||||||
name with none, such as one that resolves to a private address, is reported
|
name with none, such as one that resolves to a private address, is reported
|
||||||
as `host name has no public address`, without its addresses. `errors` has
|
as `host name has no public address`, without its addresses. `errors` has
|
||||||
@@ -211,10 +235,10 @@ so each answer's policy names a random nonce made for that answer, and the
|
|||||||
browser applies or runs only the blocks that carry it. The pages therefore use
|
browser applies or runs only the blocks that carry it. The pages therefore use
|
||||||
no `style` attributes, which a nonce cannot cover.
|
no `style` attributes, which a nonce cannot cover.
|
||||||
|
|
||||||
An origin AS of 0 (`origin_asn` in a route, `asn` in an IP lookup, the key `0`
|
An origin AS of 0 (`origin_asn` in a route, `asn` and `origin_asns` in an IP
|
||||||
in a prefix's `origins`) means the route has no known origin: its AS path ends
|
lookup, the key `0` in a prefix's `origins`) means the route has no known
|
||||||
in an AS set of several members, any of which may have originated it. The pages
|
origin: its AS path ends in an AS set of several members, any of which may have
|
||||||
show such a route as having no known origin.
|
originated it. The pages show such a route as having no known origin.
|
||||||
|
|
||||||
## Blocklists
|
## Blocklists
|
||||||
|
|
||||||
|
|||||||
@@ -35,6 +35,11 @@ tested.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-27: every result of the IP lookup has the same fields. A field without
|
||||||
|
a value is null. It used to be left out. An address with no live route has
|
||||||
|
every route and AS field set to null, and `ptr` is an empty list when there
|
||||||
|
are no PTR names. New fields give the number of origin ASes of the netblock,
|
||||||
|
in `num_origins`, and which they are, in `origin_asns` (closes #200)
|
||||||
- 2026-10-27: with `TRUSTED_PROXIES` unset, the server trusts reverse proxies at
|
- 2026-10-27: with `TRUSTED_PROXIES` unset, the server trusts reverse proxies at
|
||||||
the private addresses of RFC 1918. `/ip/`, the home page's `curl` card and the
|
the private addresses of RFC 1918. `/ip/`, the home page's `curl` card and the
|
||||||
request log then give the client's address from `X-Forwarded-For`. They used
|
request log then give the client's address from `X-Forwarded-For`. They used
|
||||||
|
|||||||
@@ -14,6 +14,9 @@ type PrefixSummary struct {
|
|||||||
// OriginASN is the origin AS that the most peers carry the prefix with.
|
// OriginASN is the origin AS that the most peers carry the prefix with.
|
||||||
// When several have as many peers, it is the lowest AS number.
|
// When several have as many peers, it is the lowest AS number.
|
||||||
OriginASN int
|
OriginASN int
|
||||||
|
// OriginASNs are all the origin ASes that peers carry the prefix with, in
|
||||||
|
// AS number order.
|
||||||
|
OriginASNs []int
|
||||||
// Peers is how many peers carry the prefix.
|
// Peers is how many peers carry the prefix.
|
||||||
Peers int
|
Peers int
|
||||||
// Newest is the time of the newest kept route of the prefix, or nil when
|
// Newest is the time of the newest kept route of the prefix, or nil when
|
||||||
@@ -211,10 +214,12 @@ func (t *Table) addresses(peers *bitset.BitSet) []string {
|
|||||||
// summarize returns the summary of prefix, whose origins are in AS number
|
// summarize returns the summary of prefix, whose origins are in AS number
|
||||||
// order.
|
// order.
|
||||||
func summarize(prefix netip.Prefix, origins []*origin) PrefixSummary {
|
func summarize(prefix netip.Prefix, origins []*origin) PrefixSummary {
|
||||||
summary := PrefixSummary{Prefix: prefix}
|
summary := PrefixSummary{Prefix: prefix, OriginASNs: make([]int, 0, len(origins))}
|
||||||
mostPeers := 0
|
mostPeers := 0
|
||||||
|
|
||||||
for _, o := range origins {
|
for _, o := range origins {
|
||||||
|
summary.OriginASNs = append(summary.OriginASNs, o.asn)
|
||||||
|
|
||||||
peers := int(o.peers.Count())
|
peers := int(o.peers.Count())
|
||||||
summary.Peers += peers
|
summary.Peers += peers
|
||||||
|
|
||||||
|
|||||||
@@ -46,21 +46,22 @@ func TestLookup(t *testing.T) {
|
|||||||
tests := []struct {
|
tests := []struct {
|
||||||
addr string
|
addr string
|
||||||
// want is the summary's prefix, or empty when none is found.
|
// want is the summary's prefix, or empty when none is found.
|
||||||
want string
|
want string
|
||||||
originASN int
|
originASN int
|
||||||
peers int
|
originASNs []int
|
||||||
newest int // in seconds after the start; 0 for none
|
peers int
|
||||||
|
newest int // in seconds after the start; 0 for none
|
||||||
}{
|
}{
|
||||||
{"10.1.2.3", prefixV4, asA, 2, 20},
|
{"10.1.2.3", prefixV4, asA, []int{asA, asB}, 2, 20},
|
||||||
{"::ffff:10.1.2.3", prefixV4, asA, 2, 20},
|
{"::ffff:10.1.2.3", prefixV4, asA, []int{asA, asB}, 2, 20},
|
||||||
{"10.1.3.4", "10.1.0.0/16", asB, 3, 30},
|
{"10.1.3.4", "10.1.0.0/16", asB, []int{asA, asB}, 3, 30},
|
||||||
{"10.2.0.1", "10.0.0.0/8", asA, 1, 10},
|
{"10.2.0.1", "10.0.0.0/8", asA, []int{asA}, 1, 10},
|
||||||
{"10.9.0.1", "10.9.0.0/16", asA, 1, 0},
|
{"10.9.0.1", "10.9.0.0/16", asA, []int{asA}, 1, 0},
|
||||||
{"192.0.2.1", "", 0, 0, 0},
|
{"192.0.2.1", "", 0, nil, 0, 0},
|
||||||
{"2001:db8:1::5", prefixV6, asB, 1, 20},
|
{"2001:db8:1::5", prefixV6, asB, []int{asB}, 1, 20},
|
||||||
{"2001:db8:2::1", "2001:db8::/32", asA, 1, 10},
|
{"2001:db8:2::1", "2001:db8::/32", asA, []int{asA}, 1, 10},
|
||||||
{"3000::1", "3000::/16", asA, 1, 10},
|
{"3000::1", "3000::/16", asA, []int{asA}, 1, 10},
|
||||||
{"2001:db9::1", "", 0, 0, 0},
|
{"2001:db9::1", "", 0, nil, 0, 0},
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
@@ -77,9 +78,10 @@ func TestLookup(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
want := routetable.PrefixSummary{
|
want := routetable.PrefixSummary{
|
||||||
Prefix: netip.MustParsePrefix(tc.want),
|
Prefix: netip.MustParsePrefix(tc.want),
|
||||||
OriginASN: tc.originASN,
|
OriginASN: tc.originASN,
|
||||||
Peers: tc.peers,
|
OriginASNs: tc.originASNs,
|
||||||
|
Peers: tc.peers,
|
||||||
}
|
}
|
||||||
if tc.newest != 0 {
|
if tc.newest != 0 {
|
||||||
want.Newest = timeAt(tc.newest)
|
want.Newest = timeAt(tc.newest)
|
||||||
@@ -166,7 +168,8 @@ func TestSamplePrefixes(t *testing.T) {
|
|||||||
|
|
||||||
for _, got := range sample {
|
for _, got := range sample {
|
||||||
want := routetable.PrefixSummary{
|
want := routetable.PrefixSummary{
|
||||||
Prefix: got.Prefix, OriginASN: asA, Peers: 1, Newest: timeAt(10),
|
Prefix: got.Prefix, OriginASN: asA, OriginASNs: []int{asA},
|
||||||
|
Peers: 1, Newest: timeAt(10),
|
||||||
}
|
}
|
||||||
if seen[got.Prefix] || got.Prefix.Bits() != tc.maskLength ||
|
if seen[got.Prefix] || got.Prefix.Bits() != tc.maskLength ||
|
||||||
!reflect.DeepEqual(got, want) {
|
!reflect.DeepEqual(got, want) {
|
||||||
|
|||||||
+58
-45
@@ -752,15 +752,14 @@ type IPLookupResponse struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// IPLookupResult is the answer for one address: its PTR names, its live route
|
// IPLookupResult is the answer for one address: its PTR names, its live route
|
||||||
// and that route's AS, and the blocklists that list it. IPInfo is nil when
|
// and that route's AS, and the blocklists that list it. Every result has every
|
||||||
// the address has no live route, and the answer then has no route or AS
|
// field. The fields of IPInfo are null when the address has no live route.
|
||||||
// fields. IP hides the IPInfo field of the same name, so that an address with
|
|
||||||
// no route has it too.
|
|
||||||
type IPLookupResult struct {
|
type IPLookupResult struct {
|
||||||
*IPInfo
|
IPInfo
|
||||||
|
|
||||||
IP string `json:"ip"`
|
IP string `json:"ip"`
|
||||||
PTR []string `json:"ptr,omitempty"`
|
// PTR is empty, never nil, when the address has no PTR names.
|
||||||
|
PTR []string `json:"ptr"`
|
||||||
|
|
||||||
// Blocklists has one entry for each blocklist the address is on.
|
// Blocklists has one entry for each blocklist the address is on.
|
||||||
Blocklists []blocklist.Listing `json:"blocklists"`
|
Blocklists []blocklist.Listing `json:"blocklists"`
|
||||||
@@ -818,8 +817,8 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
|||||||
|
|
||||||
switch {
|
switch {
|
||||||
case errors.Is(err, ErrNoRoute):
|
case errors.Is(err, ErrNoRoute):
|
||||||
// The address still gets a result, without route fields.
|
// The address still gets a result, with every field of
|
||||||
ipInfo = nil
|
// ipInfo null.
|
||||||
case errors.Is(err, ErrInvalidIP):
|
case errors.Is(err, ErrInvalidIP):
|
||||||
response.Errors = append(response.Errors, ip+": "+err.Error())
|
response.Errors = append(response.Errors, ip+": "+err.Error())
|
||||||
|
|
||||||
@@ -846,8 +845,8 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
|||||||
Blocklists: s.blocklistListings(ip),
|
Blocklists: s.blocklistListings(ip),
|
||||||
})
|
})
|
||||||
|
|
||||||
if ipInfo != nil && ipInfo.NeedsWHOISRefresh {
|
if ipInfo.NeedsWHOISRefresh {
|
||||||
refreshASNs[ipInfo.ASN] = true
|
refreshASNs[*ipInfo.ASN] = true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -858,60 +857,72 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ipInfo returns the IP lookup's answer for ip. The routing table gives the
|
// ipInfo returns what the IP lookup knows about the route of ip. The routing
|
||||||
// most specific prefix that holds the address, the number of peers that carry
|
// table gives the most specific prefix that holds the address, the number of
|
||||||
// it, the origin AS that most of them carry, and the time of its newest kept
|
// peers that carry it, its origin ASes and the one that most peers carry, and
|
||||||
// route. The database gives the prefix's first-seen time and the AS's WHOIS
|
// the time of its newest kept route. The database gives the prefix's
|
||||||
// details. It returns ErrNoRoute when no prefix holds the address, and
|
// first-seen time and the AS's WHOIS details. It returns ErrNoRoute when no
|
||||||
// ErrInvalidIP when ip does not parse.
|
// prefix holds the address, and ErrInvalidIP when ip does not parse. With an
|
||||||
func (s *Server) ipInfo(ctx context.Context, ip string) (*IPInfo, error) {
|
// error, every field of the IPInfo it returns is nil.
|
||||||
|
func (s *Server) ipInfo(ctx context.Context, ip string) (IPInfo, error) {
|
||||||
addr, err := netip.ParseAddr(ip)
|
addr, err := netip.ParseAddr(ip)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("%w: %s", ErrInvalidIP, ip)
|
return IPInfo{}, fmt.Errorf("%w: %s", ErrInvalidIP, ip)
|
||||||
}
|
}
|
||||||
|
|
||||||
summary, found := s.table.Lookup(addr)
|
summary, found := s.table.Lookup(addr)
|
||||||
if !found {
|
if !found {
|
||||||
return nil, fmt.Errorf("%w for IP %s", ErrNoRoute, ip)
|
return IPInfo{}, fmt.Errorf("%w for IP %s", ErrNoRoute, ip)
|
||||||
}
|
}
|
||||||
|
|
||||||
firstSeen, err := s.db.GetPrefixFirstSeen(ctx, summary.Prefix)
|
firstSeen, err := s.db.GetPrefixFirstSeen(ctx, summary.Prefix)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return IPInfo{}, err
|
||||||
}
|
}
|
||||||
|
|
||||||
asInfo, err := s.db.GetASWHOIS(ctx, summary.OriginASN)
|
asInfo, err := s.db.GetASWHOIS(ctx, summary.OriginASN)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return IPInfo{}, err
|
||||||
}
|
}
|
||||||
|
|
||||||
info := &IPInfo{
|
info := IPInfo{
|
||||||
IP: ip,
|
Netblock: new(summary.Prefix.String()),
|
||||||
Netblock: summary.Prefix.String(),
|
MaskLength: new(summary.Prefix.Bits()),
|
||||||
MaskLength: summary.Prefix.Bits(),
|
IPVersion: new(ipVersionOf(summary.Prefix)),
|
||||||
IPVersion: ipVersionOf(summary.Prefix),
|
NumPeers: new(summary.Peers),
|
||||||
NumPeers: summary.Peers,
|
ASN: new(summary.OriginASN),
|
||||||
ASN: summary.OriginASN,
|
NumOrigins: new(len(summary.OriginASNs)),
|
||||||
ASName: asInfo.ASName,
|
OriginASNs: summary.OriginASNs,
|
||||||
Handle: asInfo.Handle,
|
ASName: nilIfEmpty(asInfo.ASName),
|
||||||
Description: asInfo.Description,
|
Handle: nilIfEmpty(asInfo.Handle),
|
||||||
OrgName: asInfo.OrgName,
|
Description: nilIfEmpty(asInfo.Description),
|
||||||
OrgID: asInfo.OrgID,
|
OrgName: nilIfEmpty(asInfo.OrgName),
|
||||||
Address: asInfo.Address,
|
OrgID: nilIfEmpty(asInfo.OrgID),
|
||||||
CountryCode: asInfo.CountryCode,
|
Address: nilIfEmpty(asInfo.Address),
|
||||||
AbuseEmail: asInfo.AbuseEmail,
|
CountryCode: nilIfEmpty(asInfo.CountryCode),
|
||||||
RIR: asInfo.RIR,
|
AbuseEmail: nilIfEmpty(asInfo.AbuseEmail),
|
||||||
FirstSeen: firstSeen,
|
RIR: nilIfEmpty(asInfo.RIR),
|
||||||
|
LastSeen: summary.Newest,
|
||||||
NeedsWHOISRefresh: needsWHOISRefresh(asInfo),
|
NeedsWHOISRefresh: needsWHOISRefresh(asInfo),
|
||||||
}
|
}
|
||||||
|
|
||||||
if summary.Newest != nil {
|
if !firstSeen.IsZero() {
|
||||||
info.LastSeen = *summary.Newest
|
info.FirstSeen = &firstSeen
|
||||||
}
|
}
|
||||||
|
|
||||||
return info, nil
|
return info, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// nilIfEmpty returns a pointer to s, or nil when s is empty, so that JSON
|
||||||
|
// gives a detail the database does not hold as null.
|
||||||
|
func nilIfEmpty(s string) *string {
|
||||||
|
if s == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &s
|
||||||
|
}
|
||||||
|
|
||||||
// needsWHOISRefresh says whether the WHOIS details of asInfo were never
|
// needsWHOISRefresh says whether the WHOIS details of asInfo were never
|
||||||
// fetched or are older than whoisStaleThreshold. AS 0, no known origin, has
|
// fetched or are older than whoisStaleThreshold. AS 0, no known origin, has
|
||||||
// none to fetch.
|
// none to fetch.
|
||||||
@@ -1117,15 +1128,17 @@ func publicAddresses(addrs []string) []string {
|
|||||||
return public
|
return public
|
||||||
}
|
}
|
||||||
|
|
||||||
// lookupPTRNames returns the PTR records of ip without their trailing dots, or
|
// lookupPTRNames returns the PTR records of ip without their trailing dots. It
|
||||||
// nil when the lookup fails or runs past ptrLookupTimeout.
|
// returns an empty list, never nil, when there are none, when the lookup fails
|
||||||
|
// and when it runs past ptrLookupTimeout, so that an answer's ptr is [] rather
|
||||||
|
// than null.
|
||||||
func (s *Server) lookupPTRNames(ctx context.Context, ip string) []string {
|
func (s *Server) lookupPTRNames(ctx context.Context, ip string) []string {
|
||||||
ctx, cancel := context.WithTimeout(ctx, ptrLookupTimeout)
|
ctx, cancel := context.WithTimeout(ctx, ptrLookupTimeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
ptrs, err := s.resolver.LookupAddr(ctx, ip)
|
ptrs, err := s.resolver.LookupAddr(ctx, ip)
|
||||||
if err != nil {
|
if err != nil || ptrs == nil {
|
||||||
return nil
|
return []string{}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Remove trailing dots from PTR records
|
// Remove trailing dots from PTR records
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"html"
|
"html"
|
||||||
|
"maps"
|
||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
@@ -1219,7 +1220,8 @@ func TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer(t *testing.T) {
|
|||||||
t.Fatalf("decoding the answer %q: %v", rec.Body, err)
|
t.Fatalf("decoding the answer %q: %v", rec.Body, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(body.Data.Results) != 1 || body.Data.Results[0].Netblock != testPrefixV4 {
|
if len(body.Data.Results) != 1 || body.Data.Results[0].Netblock == nil ||
|
||||||
|
*body.Data.Results[0].Netblock != testPrefixV4 {
|
||||||
t.Errorf("answer %s, want one result in %s", rec.Body, testPrefixV4)
|
t.Errorf("answer %s, want one result in %s", rec.Body, testPrefixV4)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1311,12 +1313,36 @@ func TestIPLookupReportsBlocklists(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// jsonNull is how JSON writes a field without a value.
|
||||||
|
const jsonNull = "null"
|
||||||
|
|
||||||
|
// resultFields decodes an IP lookup answer and returns each of its results as
|
||||||
|
// its JSON fields.
|
||||||
|
func resultFields(t *testing.T, answer []byte) []map[string]json.RawMessage {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var body struct {
|
||||||
|
Data struct {
|
||||||
|
Results []map[string]json.RawMessage `json:"results"`
|
||||||
|
} `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
err := json.Unmarshal(answer, &body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decoding the answer %s: %v", answer, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return body.Data.Results
|
||||||
|
}
|
||||||
|
|
||||||
// TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute looks up a listed
|
// TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute looks up a listed
|
||||||
// address that has no live route, alone and beside a routed address. Each
|
// address that has no live route, alone and beside a routed address. Each
|
||||||
// answer is 200 and has a result for the address, with its blocklists and no
|
// answer is 200 and has a result for the address, with its blocklists and
|
||||||
// route fields. Before the fix the lookup of that address alone answered
|
// every route field set to null. Before the fix the lookup of that address
|
||||||
// 404, and beside a routed address it was only named in errors, so its
|
// alone answered 404, and beside a routed address it was only named in
|
||||||
// blocklists were never reported.
|
// errors, so its blocklists were never reported. After that fix the result
|
||||||
|
// still left out the route fields
|
||||||
|
// (https://git.eeqj.de/sneak/routewatch/issues/200).
|
||||||
func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1324,6 +1350,15 @@ func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
|||||||
s.resolver = &fakeResolver{}
|
s.resolver = &fakeResolver{}
|
||||||
s.SetBlocklists(fakeBlocklists{})
|
s.SetBlocklists(fakeBlocklists{})
|
||||||
|
|
||||||
|
// routeFields are the fields of a result that come from the address's
|
||||||
|
// route and that route's origin AS.
|
||||||
|
routeFields := []string{
|
||||||
|
"netblock", "mask_length", "ip_version", "num_peers", "asn",
|
||||||
|
"num_origins", "origin_asns", "as_name", "handle", "description",
|
||||||
|
"org_name", "org_id", "address", "country_code", "abuse_email", "rir",
|
||||||
|
"first_seen", "last_seen",
|
||||||
|
}
|
||||||
|
|
||||||
for _, target := range []string{
|
for _, target := range []string{
|
||||||
listedUnroutedAddr,
|
listedUnroutedAddr,
|
||||||
addrInTestPrefixV4 + "," + listedUnroutedAddr,
|
addrInTestPrefixV4 + "," + listedUnroutedAddr,
|
||||||
@@ -1354,12 +1389,21 @@ func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
result := body.Data.Results[i]
|
fields := resultFields(t, rec.Body.Bytes())[i]
|
||||||
if result.IPInfo != nil {
|
for _, name := range routeFields {
|
||||||
t.Errorf("%s: result for %s has route fields %+v",
|
value, present := fields[name]
|
||||||
target, listedUnroutedAddr, result.IPInfo)
|
|
||||||
|
switch {
|
||||||
|
case !present:
|
||||||
|
t.Errorf("%s: the result for %s has no %s",
|
||||||
|
target, listedUnroutedAddr, name)
|
||||||
|
case string(value) != jsonNull:
|
||||||
|
t.Errorf("%s: %s of %s is %s, want %s",
|
||||||
|
target, name, listedUnroutedAddr, value, jsonNull)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
result := body.Data.Results[i]
|
||||||
if len(result.Blocklists) != 1 ||
|
if len(result.Blocklists) != 1 ||
|
||||||
result.Blocklists[0].Name != testListing().Name {
|
result.Blocklists[0].Name != testListing().Name {
|
||||||
t.Errorf("%s: blocklists %+v, want [%+v]",
|
t.Errorf("%s: blocklists %+v, want [%+v]",
|
||||||
@@ -1368,6 +1412,45 @@ func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// twoAddressHostName is a host name that resolves to addrInTestPrefixV4 and
|
||||||
|
// listedUnroutedAddr.
|
||||||
|
const twoAddressHostName = "two-addresses.test"
|
||||||
|
|
||||||
|
// TestIPLookupResultsHaveTheSameFields looks up a host name, addresses and
|
||||||
|
// the caller's own address, through each IP lookup route. Some of the
|
||||||
|
// addresses have a route and some do not, and none has PTR names. Every
|
||||||
|
// result has the same fields. Before the fix a result left out every field
|
||||||
|
// without a value (https://git.eeqj.de/sneak/routewatch/issues/200).
|
||||||
|
func TestIPLookupResultsHaveTheSameFields(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := newServerWithTestRoutes(t)
|
||||||
|
s.resolver = &fakeResolver{hosts: map[string][]string{
|
||||||
|
twoAddressHostName: {addrInTestPrefixV4, listedUnroutedAddr},
|
||||||
|
}}
|
||||||
|
s.SetBlocklists(fakeBlocklists{})
|
||||||
|
|
||||||
|
// want is the fields of the first result.
|
||||||
|
var want []string
|
||||||
|
|
||||||
|
for _, path := range []string{
|
||||||
|
"/api/v1/ip/" + twoAddressHostName,
|
||||||
|
"/ip/" + addrInTestPrefixV6 + "," + listedUnroutedAddr,
|
||||||
|
ownAddressPath,
|
||||||
|
} {
|
||||||
|
for _, result := range resultFields(t, get(t, s, path).Body.Bytes()) {
|
||||||
|
got := slices.Sorted(maps.Keys(result))
|
||||||
|
if want == nil {
|
||||||
|
want = got
|
||||||
|
}
|
||||||
|
|
||||||
|
if !slices.Equal(got, want) {
|
||||||
|
t.Errorf("%s: a result has the fields %v, want %v", path, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
|
// TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
|
||||||
// lookup answers. Caches may keep them for the blocklists' shortest interval,
|
// lookup answers. Caches may keep them for the blocklists' shortest interval,
|
||||||
// and only the caller's own browser may keep the answer about the caller's
|
// and only the caller's own browser may keep the answer about the caller's
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
|
"net/netip"
|
||||||
|
"reflect"
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -75,9 +77,10 @@ func get(t *testing.T, s *Server, path string) *httptest.ResponseRecorder {
|
|||||||
|
|
||||||
// TestIPLookupGivesTheOriginMostPeersCarry looks up an address in a prefix
|
// TestIPLookupGivesTheOriginMostPeersCarry looks up an address in a prefix
|
||||||
// with two origin ASes, through each IP lookup route. The answer names the
|
// with two origin ASes, through each IP lookup route. The answer names the
|
||||||
// origin AS that more peers carry, counts every peer, and gives the time of
|
// origin AS that more peers carry, counts every peer, lists both origin ASes,
|
||||||
// the prefix's newest kept route. The prefix's first-seen time and the AS's
|
// and gives the time of the prefix's newest kept route. The prefix's
|
||||||
// WHOIS details come from the database.
|
// first-seen time and the AS's WHOIS details come from the database. A WHOIS
|
||||||
|
// detail the database does not hold is null.
|
||||||
func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -105,6 +108,26 @@ func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
|||||||
t.Fatalf("UpdateASNWHOIS: %v", err)
|
t.Fatalf("UpdateASNWHOIS: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The details the database does not hold, such as org_name, are null.
|
||||||
|
want := IPInfo{
|
||||||
|
Netblock: new(testPrefixV4),
|
||||||
|
MaskLength: new(netip.MustParsePrefix(testPrefixV4).Bits()),
|
||||||
|
IPVersion: new(ipVersionV4),
|
||||||
|
NumPeers: new(4),
|
||||||
|
ASN: new(mostPeersASN),
|
||||||
|
NumOrigins: new(2),
|
||||||
|
OriginASNs: []int{mostPeersASN, fewerPeersASN},
|
||||||
|
ASName: &whois.ASName,
|
||||||
|
CountryCode: &whois.CountryCode,
|
||||||
|
FirstSeen: &firstSeen,
|
||||||
|
LastSeen: new(testRouteTime(5)),
|
||||||
|
}
|
||||||
|
|
||||||
|
wantJSON, err := json.Marshal(want)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("encoding %+v: %v", want, err)
|
||||||
|
}
|
||||||
|
|
||||||
// ownAddressPath looks up the request's own address, which httptest
|
// ownAddressPath looks up the request's own address, which httptest
|
||||||
// gives as addrInTestPrefixV4.
|
// gives as addrInTestPrefixV4.
|
||||||
for _, path := range []string{
|
for _, path := range []string{
|
||||||
@@ -115,22 +138,15 @@ func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
|||||||
Data IPLookupResponse `json:"data"`
|
Data IPLookupResponse `json:"data"`
|
||||||
}
|
}
|
||||||
|
|
||||||
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
|
answer := get(t, s, path).Body.Bytes()
|
||||||
if err != nil || len(body.Data.Results) != 1 ||
|
|
||||||
body.Data.Results[0].IPInfo == nil {
|
err := json.Unmarshal(answer, &body)
|
||||||
t.Fatalf("%s: answer %+v (%v), want one result with a route",
|
if err != nil || len(body.Data.Results) != 1 {
|
||||||
path, body.Data, err)
|
t.Fatalf("%s: answer %s (%v), want one result", path, answer, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
got := body.Data.Results[0].IPInfo
|
if !reflect.DeepEqual(body.Data.Results[0].IPInfo, want) {
|
||||||
if got.Netblock != testPrefixV4 || got.ASN != mostPeersASN ||
|
t.Errorf("%s: answer %s, want the fields %s", path, answer, wantJSON)
|
||||||
got.NumPeers != 4 || !got.LastSeen.Equal(testRouteTime(5)) ||
|
|
||||||
!got.FirstSeen.Equal(firstSeen) || got.ASName != whois.ASName ||
|
|
||||||
got.CountryCode != whois.CountryCode {
|
|
||||||
t.Errorf("%s: route %+v, want %s from AS%d, 4 peers, last seen %v, "+
|
|
||||||
"first seen %v, %s in %s", path, *got, testPrefixV4,
|
|
||||||
mostPeersASN, testRouteTime(5), firstSeen, whois.ASName,
|
|
||||||
whois.CountryCode)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+28
-21
@@ -38,30 +38,37 @@ type PrefixDistribution struct {
|
|||||||
Count int `json:"count"`
|
Count int `json:"count"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// IPInfo represents comprehensive IP information for the /ip endpoint
|
// IPInfo is what the IP lookup knows about the route of an address and that
|
||||||
|
// route's origin AS. A field is nil, null in JSON, when its value is not known.
|
||||||
|
// Every field is nil for an address with no route.
|
||||||
//
|
//
|
||||||
//nolint:tagliatelle // field names are the existing JSON API
|
//nolint:tagliatelle // field names are the existing JSON API
|
||||||
type IPInfo struct {
|
type IPInfo struct {
|
||||||
IP string `json:"ip"`
|
// Netblock is the most specific prefix that holds the address.
|
||||||
Netblock string `json:"netblock"`
|
Netblock *string `json:"netblock"`
|
||||||
MaskLength int `json:"mask_length"`
|
MaskLength *int `json:"mask_length"`
|
||||||
IPVersion int `json:"ip_version"`
|
IPVersion *int `json:"ip_version"`
|
||||||
NumPeers int `json:"num_peers"`
|
NumPeers *int `json:"num_peers"`
|
||||||
// AS information
|
// ASN is the origin AS that the most peers carry the netblock with.
|
||||||
ASN int `json:"asn"`
|
ASN *int `json:"asn"`
|
||||||
ASName string `json:"as_name,omitempty"`
|
// NumOrigins and OriginASNs are how many origin ASes peers carry the
|
||||||
Handle string `json:"handle,omitempty"`
|
// netblock with, and which, in AS number order.
|
||||||
Description string `json:"description,omitempty"`
|
NumOrigins *int `json:"num_origins"`
|
||||||
OrgName string `json:"org_name,omitempty"`
|
OriginASNs []int `json:"origin_asns"`
|
||||||
OrgID string `json:"org_id,omitempty"`
|
// The details the database holds about the AS in ASN.
|
||||||
Address string `json:"address,omitempty"`
|
ASName *string `json:"as_name"`
|
||||||
CountryCode string `json:"country_code,omitempty"`
|
Handle *string `json:"handle"`
|
||||||
AbuseEmail string `json:"abuse_email,omitempty"`
|
Description *string `json:"description"`
|
||||||
RIR string `json:"rir,omitempty"`
|
OrgName *string `json:"org_name"`
|
||||||
// Timestamps. LastSeen is zero, and left out of JSON, when the prefix
|
OrgID *string `json:"org_id"`
|
||||||
// keeps no route.
|
Address *string `json:"address"`
|
||||||
FirstSeen time.Time `json:"first_seen"`
|
CountryCode *string `json:"country_code"`
|
||||||
LastSeen time.Time `json:"last_seen,omitzero"`
|
AbuseEmail *string `json:"abuse_email"`
|
||||||
|
RIR *string `json:"rir"`
|
||||||
|
// FirstSeen is when RouteWatch first saw the netblock. LastSeen is the
|
||||||
|
// time of the netblock's newest kept route.
|
||||||
|
FirstSeen *time.Time `json:"first_seen"`
|
||||||
|
LastSeen *time.Time `json:"last_seen"`
|
||||||
// Indicates if WHOIS data needs refresh (not serialized)
|
// Indicates if WHOIS data needs refresh (not serialized)
|
||||||
NeedsWHOISRefresh bool `json:"-"`
|
NeedsWHOISRefresh bool `json:"-"`
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user