Report which public blocklists list an address in the IP lookup (closes #158)

A new package, internal/blocklist, downloads seven public lists on a
schedule: DShield, Feodo Tracker, FireHOL abusers and level 3, Spamhaus
DROP for IPv4 and IPv6, and the Tor exit list. It keeps the last good
copy of each in memory and on disk. A download sends back the
Last-Modified and ETag the publisher gave for the copy, also kept on
disk.

Every IP lookup result carries blocklists. An address with no live route
now gets a result too, without route fields, so the lookup no longer
answers 404. A successful answer carries Cache-Control, private for the
caller's own address. The address sets come from go4.org/netipx.

Model: opus-5-5

README.md

@@ -87,20 +87,26 @@ Every route answers `HEAD` as it answers `GET`, without the body.
   collector, stream and handler queue statistics. `stream.route_churn_per_min`
   is the number of prefix announcements and withdrawals, from all peers, in the
   last full clock minute.
-- `GET /api/v1/ip/{ip}` - Looks up each address: its most specific live route
-  (`netblock`), that route's origin AS with its WHOIS details, and the address's
-  PTR names. Each result also lists the public blocklists the address is on, in
-  `blocklists` (see Blocklists). `{ip}` is an IP address, a host name, or a
-  comma-separated list of at most 10 of these. Of the addresses a host name
-  resolves to, only public ones are looked up: a name with none, such as one
-  that resolves to a private address, is reported as
-  `host name has no public address`, without its addresses. `results` has one
-  entry per address with a live route and `errors` one per address or name that
-  failed. The answer is 400 when `{ip}` lists more than 10, when it comes to
-  more than 10 addresses once its names are resolved, or when nothing in it is
-  an IP address or a host name with a public address; and 404 when none of the
-  addresses has a live route. A PTR lookup gets 2 seconds; an address whose
-  lookup takes longer is answered without PTR names.
+- `GET /api/v1/ip/{ip}` - Looks up each address. `{ip}` is an IP address, a host
+  name, or a comma-separated list of at most 10 of these. `results` has one
+  entry for each address, with:
+    - `ip` and `ptr`: the address and its PTR names
+    - `netblock` and the fields beside it: the address's most specific live
+      route, and that route's origin AS with its WHOIS details
+    - `blocklists`: the public blocklists the address is on (see Blocklists)
+
+    An address with no live route has a result without the route and AS fields.
+    Of the addresses a host name resolves to, only public ones are looked up. A
+    name with none, such as one that resolves to a private address, is reported
+    as `host name has no public address`, without its addresses. `errors` has
+    one entry for each host name that failed. The answer is 400 when:
+    - `{ip}` lists more than 10
+    - `{ip}` comes to more than 10 addresses once its names are resolved
+    - nothing in `{ip}` is an IP address or a host name with a public address
+
+    A PTR lookup gets 2 seconds. An address whose lookup takes longer is
+    answered without PTR names.
+
 - `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6
   apart, each with when its newest route was updated. When the database has no
   record of the AS, its handle and description come from the AS list built into
@@ -164,8 +170,8 @@ a `blocklists` list with one entry for each list the address is on:
 - `updated`: when RouteWatch last downloaded the list, or last heard from its
   publisher that its copy was current
 
-An address on no list has an empty `blocklists`. Only an address with a live
-route has a result, so an address without one gets no blocklists either.
+An address on no list has an empty `blocklists`. An address with no live route
+still has a result with its `blocklists`.
 
 The lists:
 
@@ -199,10 +205,13 @@ How RouteWatch keeps the lists:
 - It holds each list in memory, so a lookup makes no outside request.
 - It keeps a copy of each list in the `blocklists` directory of the state
   directory (see Configuration). Each copy is a file named after its list.
+- Beside each copy, a file ending in `.validators` keeps the `Last-Modified` and
+  `ETag` the publisher sent with that copy.
 - At startup it loads each list from its copy. It downloads a list at once when
   the copy is older than the list's interval, or when there is no copy.
-- A download sends `If-Modified-Since`, so a publisher does not send a list that
-  has not changed.
+- A download sends the copy's `Last-Modified` as `If-Modified-Since` and its
+  `ETag` as `If-None-Match`. The publisher then sends the list only when it has
+  changed. Without either, the download asks for the whole list.
 - A download fails when it takes over a minute or its file is over 16 MiB.
 - When a download fails, or its file does not parse, RouteWatch keeps the
   previous copy and logs the failure. It tries again one interval later.

TODO.md

@@ -38,7 +38,8 @@ tested.
   FireHOL lists, Spamhaus DROP and the Tor exit list. RouteWatch downloads each
   list on its own schedule and answers from copies in memory. The copies are
   also kept in the state directory's `blocklists` directory. IP lookup answers
-  carry `Cache-Control` (closes #158)
+  carry `Cache-Control`. An address with no live route now has a result too,
+  without route fields, so the IP lookup no longer answers 404 (closes #158)
 - 2026-10-27: the home page has an IP information API card. It shows the `curl`
   command that looks up the visitor's own address at `/ip/{addr}`, and that
   command's live answer (closes #156)

internal/blocklist/blocklist.go

@@ -64,6 +64,9 @@ type Listing struct {
 type listCopy struct {
 	addresses *netipx.IPSet
 	updated   time.Time
+
+	// validators identify the version of the list this copy holds.
+	validators validators
 }
 
 // Params is what New needs.
@@ -151,6 +154,15 @@ func (b *Blocklists) updated(name string) time.Time {
 	return b.copies[name].updated
 }
 
+// copyValidators returns the validators of the named list's copy. They are
+// empty when the list has no copy.
+func (b *Blocklists) copyValidators(name string) validators {
+	b.mu.RLock()
+	defer b.mu.RUnlock()
+
+	return b.copies[name].validators
+}
+
 // setCopy makes c the named list's copy.
 func (b *Blocklists) setCopy(name string, c listCopy) {
 	b.mu.Lock()

internal/blocklist/blocklist_internal_test.go

@@ -17,25 +17,35 @@ import (
 	"time"
 )
 
-// The test list's name and description, and the address its good version
-// lists.
+// The test list's name and description, the address its good version lists,
+// and the address a newer version lists instead.
 const (
 	testListName        = "test_list"
 	testListDescription = "On the test list"
 	listedAddr          = "192.0.2.1"
 	goodListBody        = "# A test list\n" + listedAddr + "\n"
+	newListedAddr       = "192.0.2.2"
+	newListBody         = "# A test list\n" + newListedAddr + "\n"
 )
 
+// testETag is the ETag a publisher sends with a version of the test list.
+const testETag = `"v1"`
+
 // maxWait is how long a test waits for the download goroutine.
 const maxWait = 10 * time.Second
 
 // publisher stands in for a list's publisher. It answers every request with
-// its status and body, and keeps the headers of each request.
+// its status and body, and keeps the headers of each request. A 200 answer
+// carries lastModified and etag, when set, as Last-Modified and ETag. As
+// from a real publisher, a request whose If-None-Match or If-Modified-Since
+// matches them is answered "not modified".
 type publisher struct {
-	mu      sync.Mutex
-	status  int
-	body    string
-	headers []http.Header
+	mu           sync.Mutex
+	status       int
+	body         string
+	lastModified time.Time
+	etag         string
+	headers      []http.Header
 }
 
 // newPublisher starts a publisher that answers 200 with body, and returns it
@@ -55,8 +65,22 @@ func (p *publisher) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	defer p.mu.Unlock()
 
 	p.headers = append(p.headers, r.Header.Clone())
-	w.WriteHeader(p.status)
-	_, _ = io.WriteString(w, p.body)
+
+	if p.status != http.StatusOK {
+		w.WriteHeader(p.status)
+		_, _ = io.WriteString(w, p.body)
+
+		return
+	}
+
+	if p.etag != "" {
+		w.Header().Set(etagHeader, p.etag)
+	}
+
+	// ServeContent sends Last-Modified unless lastModified is zero, and
+	// answers "not modified" when the request's conditions match.
+	http.ServeContent(w, r, testListName, p.lastModified,
+		strings.NewReader(p.body))
 }
 
 // answer makes the publisher answer later requests with status and body.
@@ -67,6 +91,17 @@ func (p *publisher) answer(status int, body string) {
 	p.status, p.body = status, body
 }
 
+// publish makes the publisher answer later requests with a version of the
+// list: body, with lastModified and etag. A zero lastModified or an empty
+// etag is not sent.
+func (p *publisher) publish(body string, lastModified time.Time, etag string) {
+	p.mu.Lock()
+	defer p.mu.Unlock()
+
+	p.status, p.body = http.StatusOK, body
+	p.lastModified, p.etag = lastModified, etag
+}
+
 // requests returns the headers of the requests the publisher has had.
 func (p *publisher) requests() []http.Header {
 	p.mu.Lock()
@@ -280,8 +315,9 @@ func TestRestartLoadsCopiesFromDisk(t *testing.T) {
 
 // TestStartRefreshesAnOldCopy starts on a copy older than its list's
 // interval. The list is downloaded at once, with routewatch's User-Agent and
-// the copy's time in If-Modified-Since. The publisher answers that the copy
-// is still current, so the copy is kept and marked as updated now.
+// the Last-Modified and ETag the publisher sent with the copy, kept on disk.
+// The publisher answers that the copy is still current, so the copy is kept
+// and marked as updated now.
 func TestStartRefreshesAnOldCopy(t *testing.T) {
 	t.Parallel()
 
@@ -289,6 +325,9 @@ func TestStartRefreshesAnOldCopy(t *testing.T) {
 	p, url := newPublisher(t, goodListBody)
 	list := testList(url, FormatLines)
 
+	lastModified := time.Now().Add(-3 * list.Interval).UTC().Truncate(time.Second)
+	p.publish(goodListBody, lastModified, testETag)
+
 	err := newTestBlocklists(dir, list).download(t.Context(), list)
 	if err != nil {
 		t.Fatalf("download: %v", err)
@@ -301,8 +340,6 @@ func TestStartRefreshesAnOldCopy(t *testing.T) {
 		t.Fatalf("Chtimes: %v", err)
 	}
 
-	p.answer(http.StatusNotModified, "")
-
 	b := newTestBlocklists(dir, list)
 	b.Start(t.Context())
 
@@ -320,9 +357,13 @@ func TestStartRefreshesAnOldCopy(t *testing.T) {
 	requests := p.requests()
 	last := requests[len(requests)-1]
 
-	if got := last.Get(ifModifiedSinceHeader); got != old.Format(http.TimeFormat) {
-		t.Errorf("If-Modified-Since %q, want %q", got,
-			old.Format(http.TimeFormat))
+	want := lastModified.Format(http.TimeFormat)
+	if got := last.Get(ifModifiedSinceHeader); got != want {
+		t.Errorf("If-Modified-Since %q, want %q", got, want)
+	}
+
+	if got := last.Get(ifNoneMatchHeader); got != testETag {
+		t.Errorf("If-None-Match %q, want %q", got, testETag)
 	}
 
 	if got := last.Get(userAgentHeader); got != userAgent {
@@ -330,6 +371,73 @@ func TestStartRefreshesAnOldCopy(t *testing.T) {
 	}
 }
 
+// TestNewVersionOlderThanTheLastDownloadReplacesTheCopy downloads a list.
+// The publisher then puts out a new version whose Last-Modified is earlier
+// than that download, as when a cache in front of the publisher passes the
+// version on late. The next download sends the Last-Modified of the copy,
+// not the time of the download, so it gets the new version.
+func TestNewVersionOlderThanTheLastDownloadReplacesTheCopy(t *testing.T) {
+	t.Parallel()
+
+	p, url := newPublisher(t, goodListBody)
+	list := testList(url, FormatLines)
+	b := newTestBlocklists(t.TempDir(), list)
+
+	firstVersion := time.Now().Add(-2 * list.Interval).UTC().Truncate(time.Second)
+	p.publish(goodListBody, firstVersion, "")
+
+	err := b.download(t.Context(), list)
+	if err != nil {
+		t.Fatalf("first download: %v", err)
+	}
+
+	p.publish(newListBody, firstVersion.Add(list.Interval), "")
+
+	err = b.download(t.Context(), list)
+	if err != nil {
+		t.Fatalf("second download: %v", err)
+	}
+
+	if listings := lookup(t, b, newListedAddr); len(listings) != 1 {
+		t.Errorf("%s: listings %+v, want one from the new version",
+			newListedAddr, listings)
+	}
+
+	want := firstVersion.Format(http.TimeFormat)
+	if got := p.requests()[1].Get(ifModifiedSinceHeader); got != want {
+		t.Errorf("If-Modified-Since %q, want %q", got, want)
+	}
+}
+
+// TestDownloadWithoutLastModifiedOrETagSendsNoCondition downloads a list
+// twice from a publisher that sends neither Last-Modified nor ETag. Neither
+// request carries If-Modified-Since or If-None-Match, so the second download
+// gets the whole list again.
+func TestDownloadWithoutLastModifiedOrETagSendsNoCondition(t *testing.T) {
+	t.Parallel()
+
+	p, url := newPublisher(t, goodListBody)
+	list := testList(url, FormatLines)
+	b := newTestBlocklists(t.TempDir(), list)
+
+	err := b.download(t.Context(), list)
+	if err != nil {
+		t.Fatalf("first download: %v", err)
+	}
+
+	err = b.download(t.Context(), list)
+	if err != nil {
+		t.Fatalf("second download: %v", err)
+	}
+
+	for i, request := range p.requests() {
+		if request.Get(ifModifiedSinceHeader) != "" ||
+			request.Get(ifNoneMatchHeader) != "" {
+			t.Errorf("request %d has a condition: %v", i, request)
+		}
+	}
+}
+
 // TestStopCancelsADownloadInProgress stops while the publisher has not
 // answered. Stop cancels the download and returns once the download
 // goroutine has stopped.

internal/blocklist/download.go

@@ -2,6 +2,7 @@ package blocklist
 
 import (
 	"context"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
@@ -28,20 +29,46 @@ const (
 const (
 	userAgentHeader       = "User-Agent"
 	ifModifiedSinceHeader = "If-Modified-Since"
+	ifNoneMatchHeader     = "If-None-Match"
 
 	// userAgent names routewatch to the publishers.
 	userAgent = "routewatch (+https://git.eeqj.de/sneak/routewatch)"
 )
 
+// The response headers that identify the version of a list a publisher sent.
+const (
+	lastModifiedHeader = "Last-Modified"
+	etagHeader         = "ETag"
+)
+
 const (
 	// downloadSuffix ends the name of the file a download is written to,
 	// beside the list's copy, until it has parsed.
 	downloadSuffix = ".download"
 
+	// validatorsSuffix ends the name of the file beside a list's copy that
+	// keeps the copy's validators.
+	validatorsSuffix = ".validators"
+
 	// dirPermissions are those of the directory that holds the copies.
 	dirPermissions = 0o750
+
+	// validatorsPermissions are those of the file that keeps a copy's
+	// validators.
+	validatorsPermissions = 0o600
 )
 
+// validators are the Last-Modified and ETag headers a publisher sent with a
+// version of a list. A download sends them back, as If-Modified-Since and
+// If-None-Match, so that the publisher sends the list only when it has
+// changed. They are kept as the publisher wrote them.
+//
+//nolint:tagliatelle // snake_case keys, as the Go style guide asks
+type validators struct {
+	LastModified string `json:"last_modified,omitempty"`
+	ETag         string `json:"etag,omitempty"`
+}
+
 var (
 	errUnexpectedStatus = errors.New("unexpected HTTP status")
 	errTooLarge         = errors.New("list is over the size limit")
@@ -137,10 +164,11 @@ func (b *Blocklists) run(ctx context.Context, nextDownload []time.Time) {
 	}
 }
 
-// download asks the publisher for list. A new version that parses becomes
-// the list's copy, on disk and in memory. An answer that the copy is still
-// current marks the copy as updated now. Anything else leaves the copy as it
-// is and returns an error.
+// download asks the publisher for list, sending the validators of the list's
+// copy when it has them. A new version that parses becomes the list's copy,
+// on disk and in memory. An answer that the copy is still current marks the
+// copy as updated now. Anything else leaves the copy as it is and returns an
+// error.
 func (b *Blocklists) download(ctx context.Context, list List) error {
 	ctx, cancel := context.WithTimeout(ctx, downloadTimeout)
 	defer cancel()
@@ -152,9 +180,13 @@ func (b *Blocklists) download(ctx context.Context, list List) error {
 
 	req.Header.Set(userAgentHeader, userAgent)
 
-	updated := b.updated(list.Name)
-	if !updated.IsZero() {
-		req.Header.Set(ifModifiedSinceHeader, updated.Format(http.TimeFormat))
+	held := b.copyValidators(list.Name)
+	if held.LastModified != "" {
+		req.Header.Set(ifModifiedSinceHeader, held.LastModified)
+	}
+
+	if held.ETag != "" {
+		req.Header.Set(ifNoneMatchHeader, held.ETag)
 	}
 
 	resp, err := http.DefaultClient.Do(req)
@@ -166,7 +198,10 @@ func (b *Blocklists) download(ctx context.Context, list List) error {
 
 	switch resp.StatusCode {
 	case http.StatusOK:
-		return b.replaceCopy(list, resp.Body)
+		return b.replaceCopy(list, resp.Body, validators{
+			LastModified: resp.Header.Get(lastModifiedHeader),
+			ETag:         resp.Header.Get(etagHeader),
+		})
 	case http.StatusNotModified:
 		return b.markCurrent(list)
 	default:
@@ -176,7 +211,8 @@ func (b *Blocklists) download(ctx context.Context, list List) error {
 
 // replaceCopy writes a new version of list, read from body, beside the list's
 // copy. Only when it parses does it replace the copy, on disk and in memory.
-func (b *Blocklists) replaceCopy(list List, body io.Reader) error {
+// sent are the validators the publisher sent with the new version.
+func (b *Blocklists) replaceCopy(list List, body io.Reader, sent validators) error {
 	root, err := os.OpenRoot(b.dir)
 	if err != nil {
 		return err
@@ -193,14 +229,31 @@ func (b *Blocklists) replaceCopy(list List, body io.Reader) error {
 		return err
 	}
 
+	// The old copy's validators are removed before the copy is replaced, so
+	// that they are never kept with the new copy.
+	err = root.Remove(list.Name + validatorsSuffix)
+	if err != nil && !errors.Is(err, fs.ErrNotExist) {
+		return err
+	}
+
 	err = root.Rename(downloadName, list.Name)
 	if err != nil {
 		return err
 	}
 
-	b.setCopy(list.Name, listCopy{addresses: addresses, updated: time.Now().UTC()})
+	b.setCopy(list.Name, listCopy{
+		addresses: addresses, updated: time.Now().UTC(), validators: sent,
+	})
 	b.logger.Info("Downloaded a new version of a blocklist", "list", list.Name)
 
+	// Without this file, the first download after a restart asks for the
+	// whole list.
+	err = writeValidators(root, list.Name, sent)
+	if err != nil {
+		b.logger.Error("Failed to keep the validators of a blocklist copy",
+			"list", list.Name, "error", err)
+	}
+
 	return nil
 }
 
@@ -256,10 +309,18 @@ func (b *Blocklists) markCurrent(list List) error {
 	return nil
 }
 
-// loadCopy makes the list's copy on disk its copy in memory. The file's
-// modification time is when the copy was updated.
+// loadCopy makes the list's copy on disk, with the validators kept beside it,
+// its copy in memory. The file's modification time is when the copy was
+// updated.
 func (b *Blocklists) loadCopy(list List) error {
-	f, err := os.OpenInRoot(b.dir, list.Name)
+	root, err := os.OpenRoot(b.dir)
+	if err != nil {
+		return err
+	}
+
+	defer func() { _ = root.Close() }()
+
+	f, err := root.Open(list.Name)
 	if err != nil {
 		return err
 	}
@@ -276,9 +337,43 @@ func (b *Blocklists) loadCopy(list List) error {
 		return err
 	}
 
+	held, err := readValidators(root, list.Name)
+	if err != nil {
+		return err
+	}
+
 	b.setCopy(list.Name, listCopy{
-		addresses: addresses, updated: info.ModTime().UTC(),
+		addresses: addresses, updated: info.ModTime().UTC(), validators: held,
 	})
 
 	return nil
 }
+
+// writeValidators keeps held in the file beside the named list's copy.
+func writeValidators(root *os.Root, name string, held validators) error {
+	data, err := json.Marshal(held)
+	if err != nil {
+		return err
+	}
+
+	return root.WriteFile(name+validatorsSuffix, data, validatorsPermissions)
+}
+
+// readValidators returns the validators kept beside the named list's copy.
+// They are empty when there is no such file.
+func readValidators(root *os.Root, name string) (validators, error) {
+	var held validators
+
+	data, err := root.ReadFile(name + validatorsSuffix)
+	if errors.Is(err, fs.ErrNotExist) {
+		return held, nil
+	}
+
+	if err != nil {
+		return held, err
+	}
+
+	err = json.Unmarshal(data, &held)
+
+	return held, err
+}

internal/database/models.go

@@ -92,12 +92,11 @@ type PrefixDistribution struct {
 //
 //nolint:tagliatelle // field names are the existing JSON API
 type IPInfo struct {
-	IP         string   `json:"ip"`
-	PTR        []string `json:"ptr,omitempty"`
-	Netblock   string   `json:"netblock"`
-	MaskLength int      `json:"mask_length"`
-	IPVersion  int      `json:"ip_version"`
-	NumPeers   int      `json:"num_peers"`
+	IP         string `json:"ip"`
+	Netblock   string `json:"netblock"`
+	MaskLength int    `json:"mask_length"`
+	IPVersion  int    `json:"ip_version"`
+	NumPeers   int    `json:"num_peers"`
 	// AS information
 	ASN         int    `json:"asn"`
 	ASName      string `json:"as_name,omitempty"`

internal/server/handlers.go

@@ -708,11 +708,17 @@ type IPLookupResponse struct {
 	Errors  []string         `json:"errors,omitempty"`
 }
 
-// IPLookupResult is the answer for one address: its route and AS from the
-// database, and the blocklists that list it.
+// IPLookupResult is the answer for one address: its PTR names, its live route
+// and that route's AS, and the blocklists that list it. IPInfo is nil when
+// the address has no live route, and the answer then has no route or AS
+// fields. IP hides the IPInfo field of the same name, so that an address with
+// no route has it too.
 type IPLookupResult struct {
 	*database.IPInfo
 
+	IP  string   `json:"ip"`
+	PTR []string `json:"ptr,omitempty"`
+
 	// Blocklists has one entry for each blocklist the address is on.
 	Blocklists []blocklist.Listing `json:"blocklists"`
 }
@@ -768,7 +774,10 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
 			}
 
 			switch {
-			case errors.Is(err, database.ErrNoRoute), errors.Is(err, database.ErrInvalidIP):
+			case errors.Is(err, database.ErrNoRoute):
+				// The address still gets a result, without route fields.
+				ipInfo = nil
+			case errors.Is(err, database.ErrInvalidIP):
 				response.Errors = append(response.Errors, ip+": "+err.Error())
 
 				continue
@@ -787,16 +796,14 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
 				return
 			}
 
-			if len(ptrs) > 0 {
-				ipInfo.PTR = ptrs
-			}
-
 			response.Results = append(response.Results, IPLookupResult{
+				IP:         ip,
+				PTR:        ptrs,
 				IPInfo:     ipInfo,
 				Blocklists: s.blocklistListings(ip),
 			})
 
-			if ipInfo.NeedsWHOISRefresh {
+			if ipInfo != nil && ipInfo.NeedsWHOISRefresh {
 				refreshASNs[ipInfo.ASN] = true
 			}
 		}
@@ -851,18 +858,11 @@ func noTargetsMessage(resolveErrors []string) string {
 	return message + ": " + resolveErrors[0]
 }
 
-// writeIPLookupAnswer sends response, or 404 with its first error when no
-// address had a route. ownAddress says the request asked about the caller's
-// own address.
+// writeIPLookupAnswer sends response with its Cache-Control header.
+// ownAddress says the request asked about the caller's own address.
 func (s *Server) writeIPLookupAnswer(
 	w http.ResponseWriter, response IPLookupResponse, ownAddress bool,
 ) {
-	if len(response.Results) == 0 && len(response.Errors) > 0 {
-		s.writeJSONError(w, http.StatusNotFound, "No routes found: "+response.Errors[0])
-
-		return
-	}
-
 	s.setIPLookupCacheControl(w, ownAddress)
 
 	err := writeJSONSuccess(w, response)

internal/server/handlers_internal_test.go

@@ -1042,14 +1042,15 @@ func (failingIPLookupDB) GetIPInfoContext(
 // server's router with a store whose read of one address fails with the
 // driver's error. A lookup that asks for that address answers 500 with the
 // JSON error body, as the AS and prefix JSON routes do; an address with no
-// route still answers 404. Before the fix the failed read was reported as if
-// the address had no route: 404 for that address alone, and 200 beside a
-// routed address (https://git.eeqj.de/sneak/routewatch/issues/109).
+// route answers 200. Before the fix the failed read was reported as if the
+// address had no route: 404 for that address alone, and 200 beside a routed
+// address (https://git.eeqj.de/sneak/routewatch/issues/109).
 func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
 	t.Parallel()
 
 	s := New(failingIPLookupDB{}, streamer.New(logger.New(), metrics.New()),
 		logger.New(), &config.Config{})
+	s.resolver = &fakeResolver{}
 
 	tests := []struct {
 		path     string
@@ -1065,7 +1066,7 @@ func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
 		},
 		{
 			path:     "/api/v1/ip/203.0.113.1",
-			wantCode: http.StatusNotFound,
+			wantCode: http.StatusOK,
 		},
 	}
 
@@ -1079,7 +1080,9 @@ func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
 				tt.path, rec.Code, tt.wantCode, rec.Body)
 		}
 
-		checkJSONErrorCode(t, tt.path, rec, tt.wantCode)
+		if tt.wantCode != http.StatusOK {
+			checkJSONErrorCode(t, tt.path, rec, tt.wantCode)
+		}
 	}
 }
 
@@ -1167,10 +1170,10 @@ func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
 	}{
 		{
 			strings.Join(addrs[:maxLookupTargets], ","),
-			http.StatusNotFound, maxLookupTargets, 0,
+			http.StatusOK, maxLookupTargets, 0,
 		},
 		{strings.Join(addrs, ","), http.StatusBadRequest, 0, 0},
-		{"most.test", http.StatusNotFound, maxLookupTargets, 1},
+		{"most.test", http.StatusOK, maxLookupTargets, 1},
 		{"too-many.test", http.StatusBadRequest, 0, 1},
 		{strings.Join(names, ","), http.StatusBadRequest, 0, 0},
 	}
@@ -1191,7 +1194,9 @@ func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
 				tt.target, rec.Code, tt.wantCode, rec.Body)
 		}
 
-		checkJSONErrorCode(t, tt.target, rec, tt.wantCode)
+		if tt.wantCode != http.StatusOK {
+			checkJSONErrorCode(t, tt.target, rec, tt.wantCode)
+		}
 
 		got := db.lookups.Load()
 		if got != tt.wantLookups {
@@ -1258,8 +1263,12 @@ const testBlocklistInterval = 15 * time.Minute
 // ownAddressPath is the IP lookup of the caller's own address.
 const ownAddressPath = "/ip/"
 
-// fakeBlocklists has one list, which lists addrInTestPrefixV4 and no other
-// address.
+// listedUnroutedAddr is an address that fakeBlocklists lists, outside the
+// routes newServerWithTestRoutes stores.
+const listedUnroutedAddr = "203.0.113.1"
+
+// fakeBlocklists has one list, which lists addrInTestPrefixV4 and
+// listedUnroutedAddr and no other address.
 type fakeBlocklists struct{}
 
 func testListing() blocklist.Listing {
@@ -1271,7 +1280,8 @@ func testListing() blocklist.Listing {
 }
 
 func (fakeBlocklists) Lookup(addr netip.Addr) []blocklist.Listing {
-	if addr == netip.MustParseAddr(addrInTestPrefixV4) {
+	if addr == netip.MustParseAddr(addrInTestPrefixV4) ||
+		addr == netip.MustParseAddr(listedUnroutedAddr) {
 		return []blocklist.Listing{testListing()}
 	}
 
@@ -1330,6 +1340,63 @@ func TestIPLookupReportsBlocklists(t *testing.T) {
 	}
 }
 
+// TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute looks up a listed
+// address that has no live route, alone and beside a routed address. Each
+// answer is 200 and has a result for the address, with its blocklists and no
+// route fields. Before the fix the lookup of that address alone answered
+// 404, and beside a routed address it was only named in errors, so its
+// blocklists were never reported.
+func TestIPLookupReportsBlocklistsOfAnAddressWithNoRoute(t *testing.T) {
+	t.Parallel()
+
+	s := newServerWithTestRoutes(t)
+	s.resolver = &fakeResolver{}
+	s.SetBlocklists(fakeBlocklists{})
+
+	for _, target := range []string{
+		listedUnroutedAddr,
+		addrInTestPrefixV4 + "," + listedUnroutedAddr,
+	} {
+		rec := httptest.NewRecorder()
+		s.router.ServeHTTP(rec, httptest.NewRequestWithContext(t.Context(),
+			http.MethodGet, "/api/v1/ip/"+target, nil))
+
+		var body struct {
+			Data IPLookupResponse `json:"data"`
+		}
+
+		err := json.Unmarshal(rec.Body.Bytes(), &body)
+		if rec.Code != http.StatusOK || err != nil {
+			t.Errorf("%s: status %d (%v), want %d; body %s",
+				target, rec.Code, err, http.StatusOK, rec.Body)
+
+			continue
+		}
+
+		i := slices.IndexFunc(body.Data.Results, func(r IPLookupResult) bool {
+			return r.IP == listedUnroutedAddr
+		})
+		if i < 0 || len(body.Data.Errors) != 0 {
+			t.Errorf("%s: answer %s, want a result for %s and no errors",
+				target, rec.Body, listedUnroutedAddr)
+
+			continue
+		}
+
+		result := body.Data.Results[i]
+		if result.IPInfo != nil {
+			t.Errorf("%s: result for %s has route fields %+v",
+				target, listedUnroutedAddr, result.IPInfo)
+		}
+
+		if len(result.Blocklists) != 1 ||
+			result.Blocklists[0].Name != testListing().Name {
+			t.Errorf("%s: blocklists %+v, want [%+v]",
+				target, result.Blocklists, testListing())
+		}
+	}
+}
+
 // TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
 // lookup answers. Caches may keep them for the blocklists' shortest interval,
 // and only the caller's own browser may keep the answer about the caller's

internal/templates/index.html

@@ -606,8 +606,8 @@
 
         // IP information API card: show the curl command that asks /ip/ about
         // the visitor's own address, then make that same request and show the
-        // JSON it answers, an error answer such as a 404 included. The answer
-        // holds outside data, so it is set with textContent, never innerHTML.
+        // JSON it answers, an error answer included. The answer holds outside
+        // data, so it is set with textContent, never innerHTML.
         const ipAPICommand = document.getElementById('ip-api-command');
         const ipAPIAnswer = document.getElementById('ip-api-answer');
         const ipAPIURL = window.location.origin + '/ip/' + ipAPICommand.dataset.clientIp;