Give every IP lookup result the same fields (closes #200) #206
@@ -141,15 +141,39 @@ one of its peers sends an update.
|
||||
last full clock minute.
|
||||
- `GET /api/v1/ip/{ip}` - Looks up each address. `{ip}` is an IP address, a host
|
||||
name, or a comma-separated list of at most 10 of these. `results` has one
|
||||
entry for each address, with:
|
||||
- `ip` and `ptr`: the address and its PTR names
|
||||
- `netblock` and the fields beside it: the most specific prefix that holds
|
||||
the address, and its origin AS with that AS's WHOIS details. For a prefix
|
||||
with several origin ASes, `asn` is the one that most peers carry.
|
||||
`last_seen` is the time of the prefix's newest kept route.
|
||||
- `blocklists`: the public blocklists the address is on (see Blocklists)
|
||||
entry for each address. Every entry has the same fields:
|
||||
- `ip`: the address
|
||||
- `ptr`: the address's PTR names, or an empty list when it has none
|
||||
- `blocklists`: the public blocklists the address is on (see Blocklists), or
|
||||
an empty list when it is on none
|
||||
|
||||
Every other field is about the address's live route. Each is null when the
|
||||
address has no live route. The fields about the route:
|
||||
- `netblock`: the most specific prefix that holds the address
|
||||
- `mask_length`: the netblock's mask length
|
||||
- `ip_version`: the netblock's IP version, 4 or 6
|
||||
- `num_peers`: the number of peers that carry the netblock
|
||||
- `asn`: the origin AS that the most peers carry the netblock with
|
||||
- `num_origins`: the number of origin ASes that peers carry the netblock
|
||||
with
|
||||
- `origin_asns`: those origin ASes, in AS number order
|
||||
- `first_seen`: when RouteWatch first saw the netblock, or null when the
|
||||
database has not stored the netblock yet
|
||||
- `last_seen`: the time of the netblock's newest kept route, or null when
|
||||
the netblock keeps no route
|
||||
|
||||
The fields about the AS in `asn` come from the database. Each is also null
|
||||
when the database does not have it:
|
||||
- `handle`: the AS's handle
|
||||
- `description`: the AS's description
|
||||
- `as_name`: the AS's name in its WHOIS details
|
||||
- `org_name`: the name of the organisation that holds the AS
|
||||
- `org_id`: the organisation's ID at its registry
|
||||
- `address`: the organisation's postal address
|
||||
- `country_code`: the organisation's country code
|
||||
- `abuse_email`: the address that takes abuse reports for the AS
|
||||
- `rir`: the regional internet registry of the AS
|
||||
|
||||
An address with no live route has a result without the route and AS fields.
|
||||
Of the addresses a host name resolves to, only public ones are looked up. A
|
||||
name with none, such as one that resolves to a private address, is reported
|
||||
as `host name has no public address`, without its addresses. `errors` has
|
||||
@@ -211,10 +235,15 @@ so each answer's policy names a random nonce made for that answer, and the
|
||||
browser applies or runs only the blocks that carry it. The pages therefore use
|
||||
no `style` attributes, which a nonce cannot cover.
|
||||
|
||||
An origin AS of 0 (`origin_asn` in a route, `asn` in an IP lookup, the key `0`
|
||||
in a prefix's `origins`) means the route has no known origin: its AS path ends
|
||||
in an AS set of several members, any of which may have originated it. The pages
|
||||
show such a route as having no known origin.
|
||||
An origin AS of 0 means the route has no known origin. Its AS path ends in an AS
|
||||
set of several members, any of which may have originated it. The 0 appears in
|
||||
these fields:
|
||||
|
||||
- `origin_asn` in a route
|
||||
- `asn` and `origin_asns` in an IP lookup
|
||||
- the key `0` in a prefix's `origins`
|
||||
|
||||
The pages show such a route as having no known origin.
|
||||
|
||||
## Blocklists
|
||||
|
||||
|
||||
@@ -35,6 +35,11 @@ tested.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-27: every result of the IP lookup has the same fields. A field without
|
||||
a value is null. It used to be left out. An address with no live route has
|
||||
every route and AS field set to null, and `ptr` is an empty list when there
|
||||
are no PTR names. New fields give the number of origin ASes of the netblock,
|
||||
in `num_origins`, and which they are, in `origin_asns` (closes #200)
|
||||
- 2026-10-27: with `TRUSTED_PROXIES` unset, the server trusts reverse proxies at
|
||||
the private addresses of RFC 1918. `/ip/`, the home page's `curl` card and the
|
||||
request log then give the client's address from `X-Forwarded-For`. They used
|
||||
|
||||
@@ -14,6 +14,9 @@ type PrefixSummary struct {
|
||||
// OriginASN is the origin AS that the most peers carry the prefix with.
|
||||
// When several have as many peers, it is the lowest AS number.
|
||||
OriginASN int
|
||||
// OriginASNs are all the origin ASes that peers carry the prefix with, in
|
||||
// AS number order.
|
||||
OriginASNs []int
|
||||
// Peers is how many peers carry the prefix.
|
||||
Peers int
|
||||
// Newest is the time of the newest kept route of the prefix, or nil when
|
||||
@@ -211,10 +214,12 @@ func (t *Table) addresses(peers *bitset.BitSet) []string {
|
||||
// summarize returns the summary of prefix, whose origins are in AS number
|
||||
// order.
|
||||
func summarize(prefix netip.Prefix, origins []*origin) PrefixSummary {
|
||||
summary := PrefixSummary{Prefix: prefix}
|
||||
summary := PrefixSummary{Prefix: prefix, OriginASNs: make([]int, 0, len(origins))}
|
||||
mostPeers := 0
|
||||
|
||||
for _, o := range origins {
|
||||
summary.OriginASNs = append(summary.OriginASNs, o.asn)
|
||||
|
||||
peers := int(o.peers.Count())
|
||||
summary.Peers += peers
|
||||
|
||||
|
||||
@@ -46,21 +46,22 @@ func TestLookup(t *testing.T) {
|
||||
tests := []struct {
|
||||
addr string
|
||||
// want is the summary's prefix, or empty when none is found.
|
||||
want string
|
||||
originASN int
|
||||
peers int
|
||||
newest int // in seconds after the start; 0 for none
|
||||
want string
|
||||
originASN int
|
||||
originASNs []int
|
||||
peers int
|
||||
newest int // in seconds after the start; 0 for none
|
||||
}{
|
||||
{"10.1.2.3", prefixV4, asA, 2, 20},
|
||||
{"::ffff:10.1.2.3", prefixV4, asA, 2, 20},
|
||||
{"10.1.3.4", "10.1.0.0/16", asB, 3, 30},
|
||||
{"10.2.0.1", "10.0.0.0/8", asA, 1, 10},
|
||||
{"10.9.0.1", "10.9.0.0/16", asA, 1, 0},
|
||||
{"192.0.2.1", "", 0, 0, 0},
|
||||
{"2001:db8:1::5", prefixV6, asB, 1, 20},
|
||||
{"2001:db8:2::1", "2001:db8::/32", asA, 1, 10},
|
||||
{"3000::1", "3000::/16", asA, 1, 10},
|
||||
{"2001:db9::1", "", 0, 0, 0},
|
||||
{"10.1.2.3", prefixV4, asA, []int{asA, asB}, 2, 20},
|
||||
{"::ffff:10.1.2.3", prefixV4, asA, []int{asA, asB}, 2, 20},
|
||||
{"10.1.3.4", "10.1.0.0/16", asB, []int{asA, asB}, 3, 30},
|
||||
{"10.2.0.1", "10.0.0.0/8", asA, []int{asA}, 1, 10},
|
||||
{"10.9.0.1", "10.9.0.0/16", asA, []int{asA}, 1, 0},
|
||||
{"192.0.2.1", "", 0, nil, 0, 0},
|
||||
{"2001:db8:1::5", prefixV6, asB, []int{asB}, 1, 20},
|
||||
{"2001:db8:2::1", "2001:db8::/32", asA, []int{asA}, 1, 10},
|
||||
{"3000::1", "3000::/16", asA, []int{asA}, 1, 10},
|
||||
{"2001:db9::1", "", 0, nil, 0, 0},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
@@ -77,9 +78,10 @@ func TestLookup(t *testing.T) {
|
||||
}
|
||||
|
||||
want := routetable.PrefixSummary{
|
||||
Prefix: netip.MustParsePrefix(tc.want),
|
||||
OriginASN: tc.originASN,
|
||||
Peers: tc.peers,
|
||||
Prefix: netip.MustParsePrefix(tc.want),
|
||||
OriginASN: tc.originASN,
|
||||
OriginASNs: tc.originASNs,
|
||||
Peers: tc.peers,
|
||||
}
|
||||
if tc.newest != 0 {
|
||||
want.Newest = timeAt(tc.newest)
|
||||
@@ -166,7 +168,8 @@ func TestSamplePrefixes(t *testing.T) {
|
||||
|
||||
for _, got := range sample {
|
||||
want := routetable.PrefixSummary{
|
||||
Prefix: got.Prefix, OriginASN: asA, Peers: 1, Newest: timeAt(10),
|
||||
Prefix: got.Prefix, OriginASN: asA, OriginASNs: []int{asA},
|
||||
Peers: 1, Newest: timeAt(10),
|
||||
}
|
||||
if seen[got.Prefix] || got.Prefix.Bits() != tc.maskLength ||
|
||||
!reflect.DeepEqual(got, want) {
|
||||
|
||||
+58
-45
@@ -752,15 +752,14 @@ type IPLookupResponse struct {
|
||||
}
|
||||
|
||||
// IPLookupResult is the answer for one address: its PTR names, its live route
|
||||
// and that route's AS, and the blocklists that list it. IPInfo is nil when
|
||||
// the address has no live route, and the answer then has no route or AS
|
||||
// fields. IP hides the IPInfo field of the same name, so that an address with
|
||||
// no route has it too.
|
||||
// and that route's AS, and the blocklists that list it. Every result has every
|
||||
// field. The fields of IPInfo are null when the address has no live route.
|
||||
type IPLookupResult struct {
|
||||
*IPInfo
|
||||
IPInfo
|
||||
|
||||
IP string `json:"ip"`
|
||||
PTR []string `json:"ptr,omitempty"`
|
||||
IP string `json:"ip"`
|
||||
// PTR is empty, never nil, when the address has no PTR names.
|
||||
PTR []string `json:"ptr"`
|
||||
|
||||
// Blocklists has one entry for each blocklist the address is on.
|
||||
Blocklists []blocklist.Listing `json:"blocklists"`
|
||||
@@ -818,8 +817,8 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
||||
|
||||
switch {
|
||||
case errors.Is(err, ErrNoRoute):
|
||||
// The address still gets a result, without route fields.
|
||||
ipInfo = nil
|
||||
// The address still gets a result, with every field of
|
||||
// ipInfo null.
|
||||
case errors.Is(err, ErrInvalidIP):
|
||||
response.Errors = append(response.Errors, ip+": "+err.Error())
|
||||
|
||||
@@ -846,8 +845,8 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
||||
Blocklists: s.blocklistListings(ip),
|
||||
})
|
||||
|
||||
if ipInfo != nil && ipInfo.NeedsWHOISRefresh {
|
||||
refreshASNs[ipInfo.ASN] = true
|
||||
if ipInfo.NeedsWHOISRefresh {
|
||||
refreshASNs[*ipInfo.ASN] = true
|
||||
}
|
||||
}
|
||||
|
||||
@@ -858,60 +857,72 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// ipInfo returns the IP lookup's answer for ip. The routing table gives the
|
||||
// most specific prefix that holds the address, the number of peers that carry
|
||||
// it, the origin AS that most of them carry, and the time of its newest kept
|
||||
// route. The database gives the prefix's first-seen time and the AS's WHOIS
|
||||
// details. It returns ErrNoRoute when no prefix holds the address, and
|
||||
// ErrInvalidIP when ip does not parse.
|
||||
func (s *Server) ipInfo(ctx context.Context, ip string) (*IPInfo, error) {
|
||||
// ipInfo returns what the IP lookup knows about the route of ip. The routing
|
||||
// table gives the most specific prefix that holds the address, the number of
|
||||
// peers that carry it, its origin ASes and the one that most peers carry, and
|
||||
// the time of its newest kept route. The database gives the prefix's
|
||||
// first-seen time and the AS's WHOIS details. It returns ErrNoRoute when no
|
||||
// prefix holds the address, and ErrInvalidIP when ip does not parse. With an
|
||||
// error, every field of the IPInfo it returns is nil.
|
||||
func (s *Server) ipInfo(ctx context.Context, ip string) (IPInfo, error) {
|
||||
addr, err := netip.ParseAddr(ip)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %s", ErrInvalidIP, ip)
|
||||
return IPInfo{}, fmt.Errorf("%w: %s", ErrInvalidIP, ip)
|
||||
}
|
||||
|
||||
summary, found := s.table.Lookup(addr)
|
||||
if !found {
|
||||
return nil, fmt.Errorf("%w for IP %s", ErrNoRoute, ip)
|
||||
return IPInfo{}, fmt.Errorf("%w for IP %s", ErrNoRoute, ip)
|
||||
}
|
||||
|
||||
firstSeen, err := s.db.GetPrefixFirstSeen(ctx, summary.Prefix)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return IPInfo{}, err
|
||||
}
|
||||
|
||||
asInfo, err := s.db.GetASWHOIS(ctx, summary.OriginASN)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return IPInfo{}, err
|
||||
}
|
||||
|
||||
info := &IPInfo{
|
||||
IP: ip,
|
||||
Netblock: summary.Prefix.String(),
|
||||
MaskLength: summary.Prefix.Bits(),
|
||||
IPVersion: ipVersionOf(summary.Prefix),
|
||||
NumPeers: summary.Peers,
|
||||
ASN: summary.OriginASN,
|
||||
ASName: asInfo.ASName,
|
||||
Handle: asInfo.Handle,
|
||||
Description: asInfo.Description,
|
||||
OrgName: asInfo.OrgName,
|
||||
OrgID: asInfo.OrgID,
|
||||
Address: asInfo.Address,
|
||||
CountryCode: asInfo.CountryCode,
|
||||
AbuseEmail: asInfo.AbuseEmail,
|
||||
RIR: asInfo.RIR,
|
||||
FirstSeen: firstSeen,
|
||||
info := IPInfo{
|
||||
Netblock: new(summary.Prefix.String()),
|
||||
MaskLength: new(summary.Prefix.Bits()),
|
||||
IPVersion: new(ipVersionOf(summary.Prefix)),
|
||||
NumPeers: new(summary.Peers),
|
||||
ASN: new(summary.OriginASN),
|
||||
NumOrigins: new(len(summary.OriginASNs)),
|
||||
OriginASNs: summary.OriginASNs,
|
||||
ASName: nilIfEmpty(asInfo.ASName),
|
||||
Handle: nilIfEmpty(asInfo.Handle),
|
||||
Description: nilIfEmpty(asInfo.Description),
|
||||
OrgName: nilIfEmpty(asInfo.OrgName),
|
||||
OrgID: nilIfEmpty(asInfo.OrgID),
|
||||
Address: nilIfEmpty(asInfo.Address),
|
||||
CountryCode: nilIfEmpty(asInfo.CountryCode),
|
||||
AbuseEmail: nilIfEmpty(asInfo.AbuseEmail),
|
||||
RIR: nilIfEmpty(asInfo.RIR),
|
||||
LastSeen: summary.Newest,
|
||||
NeedsWHOISRefresh: needsWHOISRefresh(asInfo),
|
||||
}
|
||||
|
||||
if summary.Newest != nil {
|
||||
info.LastSeen = *summary.Newest
|
||||
if !firstSeen.IsZero() {
|
||||
info.FirstSeen = &firstSeen
|
||||
}
|
||||
|
||||
return info, nil
|
||||
}
|
||||
|
||||
// nilIfEmpty returns a pointer to s, or nil when s is empty, so that JSON
|
||||
// gives a detail the database does not hold as null.
|
||||
func nilIfEmpty(s string) *string {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
return &s
|
||||
}
|
||||
|
||||
// needsWHOISRefresh says whether the WHOIS details of asInfo were never
|
||||
// fetched or are older than whoisStaleThreshold. AS 0, no known origin, has
|
||||
// none to fetch.
|
||||
@@ -1117,15 +1128,17 @@ func publicAddresses(addrs []string) []string {
|
||||
return public
|
||||
}
|
||||
|
||||
// lookupPTRNames returns the PTR records of ip without their trailing dots, or
|
||||
// nil when the lookup fails or runs past ptrLookupTimeout.
|
||||
// lookupPTRNames returns the PTR records of ip without their trailing dots. It
|
||||
// returns an empty list, never nil, when there are none, when the lookup fails
|
||||
// and when it runs past ptrLookupTimeout, so that an answer's ptr is [] rather
|
||||
// than null.
|
||||
func (s *Server) lookupPTRNames(ctx context.Context, ip string) []string {
|
||||
ctx, cancel := context.WithTimeout(ctx, ptrLookupTimeout)
|
||||
defer cancel()
|
||||
|
||||
ptrs, err := s.resolver.LookupAddr(ctx, ip)
|
||||
if err != nil {
|
||||
return nil
|
||||
if err != nil || ptrs == nil {
|
||||
return []string{}
|
||||
}
|
||||
|
||||
// Remove trailing dots from PTR records
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"html"
|
||||
"maps"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
@@ -1219,7 +1220,8 @@ func TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer(t *testing.T) {
|
||||
t.Fatalf("decoding the answer %q: %v", rec.Body, err)
|
||||
}
|
||||
|
||||
if len(body.Data.Results) != 1 || body.Data.Results[0].Netblock != testPrefixV4 {
|
||||
if len(body.Data.Results) != 1 || body.Data.Results[0].Netblock == nil ||
|
||||
*body.Data.Results[0].Netblock != testPrefixV4 {
|
||||
t.Errorf("answer %s, want one result in %s", rec.Body, testPrefixV4)
|
||||
}
|
||||
}
|
||||
@@ -1311,12 +1313,39 @@ func TestIPLookupReportsBlocklists(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// jsonNull is how JSON writes a field without a value.
|
||||
const jsonNull = "null"
|
||||
|
||||
// jsonEmptyList is how JSON writes an empty list.
|
||||
const jsonEmptyList = "[]"
|
||||
|
||||
// resultFields decodes an IP lookup answer and returns each of its results as
|
||||
// its JSON fields.
|
||||
func resultFields(t *testing.T, answer []byte) []map[string]json.RawMessage {
|
||||
t.Helper()
|
||||
|
||||
var body struct {
|
||||
Data struct {
|
||||
Results []map[string]json.RawMessage `json:"results"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
err := json.Unmarshal(answer, &body)
|
||||
if err != nil {
|
||||
t.Fatalf("decoding the answer %s: %v", answer, err)
|
||||
}
|
||||
|
||||
return body.Data.Results
|
||||
}
|
||||
|
||||
// TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute looks up a listed
|
||||
// address that has no live route, alone and beside a routed address. Each
|
||||
// answer is 200 and has a result for the address, with its blocklists and no
|
||||
// route fields. Before the fix the lookup of that address alone answered
|
||||
// 404, and beside a routed address it was only named in errors, so its
|
||||
// blocklists were never reported.
|
||||
// answer is 200 and has a result for the address, with its blocklists and
|
||||
// every route field set to null. Before the fix the lookup of that address
|
||||
// alone answered 404, and beside a routed address it was only named in
|
||||
// errors, so its blocklists were never reported. After that fix the result
|
||||
// still left out the route fields
|
||||
// (https://git.eeqj.de/sneak/routewatch/issues/200).
|
||||
func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -1324,6 +1353,15 @@ func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
||||
s.resolver = &fakeResolver{}
|
||||
s.SetBlocklists(fakeBlocklists{})
|
||||
|
||||
// routeFields are the fields of a result that come from the address's
|
||||
// route and that route's origin AS.
|
||||
routeFields := []string{
|
||||
"netblock", "mask_length", "ip_version", "num_peers", "asn",
|
||||
"num_origins", "origin_asns", "as_name", "handle", "description",
|
||||
"org_name", "org_id", "address", "country_code", "abuse_email", "rir",
|
||||
"first_seen", "last_seen",
|
||||
}
|
||||
|
||||
for _, target := range []string{
|
||||
listedUnroutedAddr,
|
||||
addrInTestPrefixV4 + "," + listedUnroutedAddr,
|
||||
@@ -1354,12 +1392,21 @@ func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
||||
continue
|
||||
}
|
||||
|
||||
result := body.Data.Results[i]
|
||||
if result.IPInfo != nil {
|
||||
t.Errorf("%s: result for %s has route fields %+v",
|
||||
target, listedUnroutedAddr, result.IPInfo)
|
||||
fields := resultFields(t, rec.Body.Bytes())[i]
|
||||
for _, name := range routeFields {
|
||||
value, present := fields[name]
|
||||
|
||||
switch {
|
||||
case !present:
|
||||
t.Errorf("%s: the result for %s has no %s",
|
||||
target, listedUnroutedAddr, name)
|
||||
case string(value) != jsonNull:
|
||||
t.Errorf("%s: %s of %s is %s, want %s",
|
||||
target, name, listedUnroutedAddr, value, jsonNull)
|
||||
}
|
||||
}
|
||||
|
||||
result := body.Data.Results[i]
|
||||
if len(result.Blocklists) != 1 ||
|
||||
result.Blocklists[0].Name != testListing().Name {
|
||||
t.Errorf("%s: blocklists %+v, want [%+v]",
|
||||
@@ -1368,6 +1415,76 @@ func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// twoAddressHostName is a host name that resolves to addrInTestPrefixV4 and
|
||||
// listedUnroutedAddr.
|
||||
const twoAddressHostName = "two-addresses.test"
|
||||
|
||||
// TestIPLookupResultsHaveTheSameFields looks up a host name, addresses and
|
||||
// the caller's own address, through each IP lookup route. Some of the
|
||||
// addresses have a route and some do not, and none has PTR names. Every
|
||||
// result has the same fields. Before the fix a result left out every field
|
||||
// without a value (https://git.eeqj.de/sneak/routewatch/issues/200).
|
||||
func TestIPLookupResultsHaveTheSameFields(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := newServerWithTestRoutes(t)
|
||||
s.resolver = &fakeResolver{hosts: map[string][]string{
|
||||
twoAddressHostName: {addrInTestPrefixV4, listedUnroutedAddr},
|
||||
}}
|
||||
s.SetBlocklists(fakeBlocklists{})
|
||||
|
||||
// want is the fields of the first result.
|
||||
var want []string
|
||||
|
||||
for _, path := range []string{
|
||||
"/api/v1/ip/" + twoAddressHostName,
|
||||
"/ip/" + addrInTestPrefixV6 + "," + listedUnroutedAddr,
|
||||
ownAddressPath,
|
||||
} {
|
||||
for _, result := range resultFields(t, get(t, s, path).Body.Bytes()) {
|
||||
got := slices.Sorted(maps.Keys(result))
|
||||
if want == nil {
|
||||
want = got
|
||||
}
|
||||
|
||||
if !slices.Equal(got, want) {
|
||||
t.Errorf("%s: a result has the fields %v, want %v", path, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestIPLookupOfAnAddressWithoutPTRNamesOrStoredNetblock looks up a routed
|
||||
// address that has no PTR names, and whose netblock the database has not
|
||||
// stored. Its ptr is an empty list and its first_seen is null. Before the fix
|
||||
// ptr was left out, and first_seen was 0001-01-01T00:00:00Z
|
||||
// (https://git.eeqj.de/sneak/routewatch/issues/200).
|
||||
func TestIPLookupOfAnAddressWithoutPTRNamesOrStoredNetblock(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := newServerWithTestRoutes(t)
|
||||
s.resolver = &fakeResolver{}
|
||||
|
||||
path := "/api/v1/ip/" + addrInTestPrefixV4
|
||||
|
||||
results := resultFields(t, get(t, s, path).Body.Bytes())
|
||||
if len(results) != 1 {
|
||||
t.Fatalf("%s: %d results, want 1", path, len(results))
|
||||
}
|
||||
|
||||
want := map[string]string{
|
||||
"ptr": jsonEmptyList,
|
||||
"first_seen": jsonNull,
|
||||
}
|
||||
|
||||
for name, value := range want {
|
||||
got := string(results[0][name])
|
||||
if got != value {
|
||||
t.Errorf("%s: %s is %s, want %s", path, name, got, value)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
|
||||
// lookup answers. Caches may keep them for the blocklists' shortest interval,
|
||||
// and only the caller's own browser may keep the answer about the caller's
|
||||
|
||||
@@ -4,6 +4,8 @@ import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"reflect"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -75,9 +77,10 @@ func get(t *testing.T, s *Server, path string) *httptest.ResponseRecorder {
|
||||
|
||||
// TestIPLookupGivesTheOriginMostPeersCarry looks up an address in a prefix
|
||||
// with two origin ASes, through each IP lookup route. The answer names the
|
||||
// origin AS that more peers carry, counts every peer, and gives the time of
|
||||
// the prefix's newest kept route. The prefix's first-seen time and the AS's
|
||||
// WHOIS details come from the database.
|
||||
// origin AS that more peers carry, counts every peer, lists both origin ASes,
|
||||
// and gives the time of the prefix's newest kept route. The prefix's
|
||||
// first-seen time and the AS's WHOIS details come from the database. A WHOIS
|
||||
// detail the database does not hold is null.
|
||||
func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -105,6 +108,26 @@ func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
||||
t.Fatalf("UpdateASNWHOIS: %v", err)
|
||||
}
|
||||
|
||||
// The details the database does not hold, such as org_name, are null.
|
||||
want := IPInfo{
|
||||
Netblock: new(testPrefixV4),
|
||||
MaskLength: new(netip.MustParsePrefix(testPrefixV4).Bits()),
|
||||
IPVersion: new(ipVersionV4),
|
||||
NumPeers: new(4),
|
||||
ASN: new(mostPeersASN),
|
||||
NumOrigins: new(2),
|
||||
OriginASNs: []int{mostPeersASN, fewerPeersASN},
|
||||
ASName: &whois.ASName,
|
||||
CountryCode: &whois.CountryCode,
|
||||
FirstSeen: &firstSeen,
|
||||
LastSeen: new(testRouteTime(5)),
|
||||
}
|
||||
|
||||
wantJSON, err := json.Marshal(want)
|
||||
if err != nil {
|
||||
t.Fatalf("encoding %+v: %v", want, err)
|
||||
}
|
||||
|
||||
// ownAddressPath looks up the request's own address, which httptest
|
||||
// gives as addrInTestPrefixV4.
|
||||
for _, path := range []string{
|
||||
@@ -115,22 +138,15 @@ func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
||||
Data IPLookupResponse `json:"data"`
|
||||
}
|
||||
|
||||
err := json.Unmarshal(get(t, s, path).Body.Bytes(), &body)
|
||||
if err != nil || len(body.Data.Results) != 1 ||
|
||||
body.Data.Results[0].IPInfo == nil {
|
||||
t.Fatalf("%s: answer %+v (%v), want one result with a route",
|
||||
path, body.Data, err)
|
||||
answer := get(t, s, path).Body.Bytes()
|
||||
|
||||
err := json.Unmarshal(answer, &body)
|
||||
if err != nil || len(body.Data.Results) != 1 {
|
||||
t.Fatalf("%s: answer %s (%v), want one result", path, answer, err)
|
||||
}
|
||||
|
||||
got := body.Data.Results[0].IPInfo
|
||||
if got.Netblock != testPrefixV4 || got.ASN != mostPeersASN ||
|
||||
got.NumPeers != 4 || !got.LastSeen.Equal(testRouteTime(5)) ||
|
||||
!got.FirstSeen.Equal(firstSeen) || got.ASName != whois.ASName ||
|
||||
got.CountryCode != whois.CountryCode {
|
||||
t.Errorf("%s: route %+v, want %s from AS%d, 4 peers, last seen %v, "+
|
||||
"first seen %v, %s in %s", path, *got, testPrefixV4,
|
||||
mostPeersASN, testRouteTime(5), firstSeen, whois.ASName,
|
||||
whois.CountryCode)
|
||||
if !reflect.DeepEqual(body.Data.Results[0].IPInfo, want) {
|
||||
t.Errorf("%s: answer %s, want the fields %s", path, answer, wantJSON)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+28
-21
@@ -38,30 +38,37 @@ type PrefixDistribution struct {
|
||||
Count int `json:"count"`
|
||||
}
|
||||
|
||||
// IPInfo represents comprehensive IP information for the /ip endpoint
|
||||
// IPInfo is what the IP lookup knows about the route of an address and that
|
||||
// route's origin AS. A field is nil, null in JSON, when its value is not known.
|
||||
// Every field is nil for an address with no route.
|
||||
//
|
||||
//nolint:tagliatelle // field names are the existing JSON API
|
||||
type IPInfo struct {
|
||||
IP string `json:"ip"`
|
||||
Netblock string `json:"netblock"`
|
||||
MaskLength int `json:"mask_length"`
|
||||
IPVersion int `json:"ip_version"`
|
||||
NumPeers int `json:"num_peers"`
|
||||
// AS information
|
||||
ASN int `json:"asn"`
|
||||
ASName string `json:"as_name,omitempty"`
|
||||
Handle string `json:"handle,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
OrgName string `json:"org_name,omitempty"`
|
||||
OrgID string `json:"org_id,omitempty"`
|
||||
Address string `json:"address,omitempty"`
|
||||
CountryCode string `json:"country_code,omitempty"`
|
||||
AbuseEmail string `json:"abuse_email,omitempty"`
|
||||
RIR string `json:"rir,omitempty"`
|
||||
// Timestamps. LastSeen is zero, and left out of JSON, when the prefix
|
||||
// keeps no route.
|
||||
FirstSeen time.Time `json:"first_seen"`
|
||||
LastSeen time.Time `json:"last_seen,omitzero"`
|
||||
// Netblock is the most specific prefix that holds the address.
|
||||
Netblock *string `json:"netblock"`
|
||||
MaskLength *int `json:"mask_length"`
|
||||
IPVersion *int `json:"ip_version"`
|
||||
NumPeers *int `json:"num_peers"`
|
||||
// ASN is the origin AS that the most peers carry the netblock with.
|
||||
ASN *int `json:"asn"`
|
||||
// NumOrigins and OriginASNs are how many origin ASes peers carry the
|
||||
// netblock with, and which, in AS number order.
|
||||
NumOrigins *int `json:"num_origins"`
|
||||
OriginASNs []int `json:"origin_asns"`
|
||||
// The details the database holds about the AS in ASN.
|
||||
ASName *string `json:"as_name"`
|
||||
Handle *string `json:"handle"`
|
||||
Description *string `json:"description"`
|
||||
OrgName *string `json:"org_name"`
|
||||
OrgID *string `json:"org_id"`
|
||||
Address *string `json:"address"`
|
||||
CountryCode *string `json:"country_code"`
|
||||
AbuseEmail *string `json:"abuse_email"`
|
||||
RIR *string `json:"rir"`
|
||||
// FirstSeen is when RouteWatch first saw the netblock. LastSeen is the
|
||||
// time of the netblock's newest kept route.
|
||||
FirstSeen *time.Time `json:"first_seen"`
|
||||
LastSeen *time.Time `json:"last_seen"`
|
||||
// Indicates if WHOIS data needs refresh (not serialized)
|
||||
NeedsWHOISRefresh bool `json:"-"`
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user