Write batches through one connection with a 1 GiB cache, in key order (closes #96)

Every write now uses one connection taken from the pool when the
database opens, under the database lock. Its 1 GiB page cache keeps
the pages the route writes touch from one batch to the next. A reading
connection's cache is emptied whenever another connection has written,
so the 9 reading connections drop to 16 MiB each. The soft and hard
heap limits rise to 1.5 and 2 GiB, above the caches; README "Memory"
adds up the new budget. Vacuum, Analyze and Checkpoint check their
context after the lock, which the kept connection does not. Batch
writes sort their rows by peer, then prefix. The benchmark also times
the largest prefix handler batch, 250,000 updates.

Model: opus-5-5

.dockerignore

@@ -73,8 +73,7 @@
 
 # This repository's host-built artifacts: `make build` writes bin/, `go
 # build` in the root the binaries, `make run` log.txt, `make asupdate` its
-# .tmp file; logs, test binaries, coverage profiles, source archives, and
+# .tmp file; test binaries, coverage profiles and source archives.
-# the binaries and libraries a build for another platform writes.
 /bin
 /routewatch
 /asinfo-gen
@@ -83,24 +82,15 @@
 /out
 /pkg/asinfo/asdata.json
 **/*.tar.zst
-**/*.log
 **/*.test
 **/*.out
 **/*.tmp
-**/*.exe
-**/*.dll
-**/*.so
-**/*.dylib
 
 # Local databases. The image carries a source archive of the whole build
-# context, so these would otherwise ship inside it. The daemon's database
+# context, so these would otherwise ship inside it.
-# is db.sqlite, with its -wal and -shm files while it runs.
 **/*.db
 **/*.db-journal
 **/*.db-wal
-**/*.sqlite
-**/*.sqlite-wal
-**/*.sqlite-shm
 
 # A local Go workspace points at directories outside the build context.
 /go.work

.gitignore

@@ -67,14 +67,10 @@ node_modules/
 go.work
 go.work.sum
 
-# Local databases, temporary files and debug output. The daemon's
+# Local databases, temporary files and debug output.
-# database is db.sqlite, with its -wal and -shm files while it runs.
 *.db
 *.db-journal
 *.db-wal
-*.sqlite
-*.sqlite-wal
-*.sqlite-shm
 *.tmp
 log.txt
 out

Dockerfile

@@ -24,8 +24,8 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
 # image ships and the alpine one does not. -count=1 because the mounted
 # cache also holds test results, which go test would otherwise replay.
 # -short skips the live RIS Live feed test, so the suite needs no network.
-# golang:1.27.2-bookworm, 2026-10-19
+# golang:1.24-bookworm, 2026-09-21
-FROM golang@sha256:5cf287a799e6b94384bad13d16b14904c531f51ba65792237e122ce42b392f61 AS test
+FROM golang@sha256:1a6d4452c65dea36aac2e2d606b01b4a029ec90cc1ae53890540ce6173ea77ac AS test
 WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
@@ -39,8 +39,8 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
 # Build stage. Nothing is wanted from either phase above; the copies
 # are what make BuildKit build them first, so nothing after them can
 # run unless lint and test passed.
-# golang:1.27.2-bookworm, 2026-10-19
+# golang:1.24-bookworm, 2026-09-21
-FROM golang@sha256:5cf287a799e6b94384bad13d16b14904c531f51ba65792237e122ce42b392f61 AS builder
+FROM golang@sha256:1a6d4452c65dea36aac2e2d606b01b4a029ec90cc1ae53890540ce6173ea77ac AS builder
 
 # Install build dependencies (zstd for archive, gcc for CGO/sqlite3)
 RUN apt-get update && apt-get install -y --no-install-recommends \

Makefile

@@ -52,6 +52,7 @@ clean:
 run: build
 	DEBUG=routewatch ./bin/routewatch 2>&1 | tee log.txt
 
-# Rebuild the AS list built into RouteWatch from as.csv (needs the network).
 asupdate:
-	@script/asupdate
+	@echo "Updating AS info data..."
+	@go run cmd/asinfo-gen/main.go | gzip > pkg/asinfo/asdata.json.gz.tmp && \
+		mv pkg/asinfo/asdata.json.gz.tmp pkg/asinfo/asdata.json.gz

README.md

@@ -1,27 +1,9 @@
 # RouteWatch
 
 RouteWatch is an MIT-licensed Go daemon by @sneak that monitors the BGP routing
-table in real time. It works like this:
+table in real time: it streams BGP UPDATE messages from the RIPE RIS Live
-
+service, maintains a live routing table in SQLite, and provides HTTP APIs for
-- It streams BGP UPDATE messages from the RIPE RIS Live service.
+querying routing information.
-- It keeps a live routing table in SQLite.
-- It serves web pages and a JSON API for looking routes up.
-
-## Rationale
-
-RIPE RIS Live streams the BGP updates that RIPE's route collectors receive, as
-they happen, from the moment a client connects. It reports changes, not the
-routing table they add up to, so it cannot say which AS an IP address is routed
-to now or which routes a prefix has. RouteWatch applies the feed's updates to
-live routing tables in its own SQLite database and answers IP-to-AS lookups and
-prefix pages from them.
-
-The feed sends no table when a client connects, so these tables hold only the
-routes the feed has announced since RouteWatch first started: an IP-to-AS lookup
-finds nothing for an address whose routes have not been announced since then.
-The tables also miss the updates RouteWatch drops when it falls behind under
-load and those sent while it is stopped, so a lookup or a prefix page can show a
-route the feed has withdrawn.
 
 ## Features
 
@@ -29,8 +11,6 @@ route the feed has withdrawn.
 - Maintains live IPv4 and IPv6 routing tables
 - Tracks AS peering relationships
 - HTTP API for IP-to-AS lookups, prefix details, and AS information
-- IP lookups report the public blocklists an address is on, from copies kept in
-  memory and on disk
 - Automatic reconnection with exponential backoff
 - Batched database writes for high performance
 - Backpressure handling to prevent memory exhaustion
@@ -56,172 +36,29 @@ DEBUG=routewatch ./routewatch
 
 ## HTTP Endpoints
 
-Every route answers `HEAD` as it answers `GET`, without the body.
-
 ### Web Interface
 
-- `GET /` - Home page (HTML): headline figures, lookups by AS number and by IP
+- `GET /` - Redirects to /status
-  address, and a search of AS names. A card shows the `curl` command that looks
+- `GET /status` - HTML status dashboard
-  up the visitor's own address at `/ip/{addr}`, and that command's live answer.
+- `GET /status.json` - JSON statistics
-- `GET /status` - HTML status dashboard, which reads `/api/v1/stats`
+- `GET /as/{asn}` - AS detail page (HTML)
-- `GET /status.json` - JSON statistics: feed, memory, database and route figures
+- `GET /prefix/{prefix}` - Prefix detail page (HTML)
-- `GET /.well-known/healthcheck.json` - Health check (JSON): 200 when the
+- `GET /prefixlength/{length}` - IPv4 prefixes by mask length
-  database answers and the RIS Live feed is connected, otherwise 503;
+- `GET /prefixlength6/{length}` - IPv6 prefixes by mask length
-  `data.status` is `ok` or `error` and `data.checks` says which check failed
+- `GET /ip/{ip}` - Redirects to prefix containing the IP
-- `GET /as/{asn}` - AS detail page (HTML): the prefixes the AS originates and
-  its peers. When the database has no record of the AS, its handle and
-  description come from the AS list built into RouteWatch. The page then shows
-  no first-seen date.
-- `GET /prefix/{prefix}/{length}` - Prefix detail page (HTML), such as
-  `/prefix/192.0.2.0/24`: every live route for the prefix, one per peer, and its
-  origin ASes
-- `GET /prefixlength/{length}` - Up to 500 IPv4 prefixes with that mask length,
-  picked at random, each with its newest route (HTML)
-- `GET /prefixlength6/{length}` - The same for IPv6
-- `GET /ip/{addr}` - The same JSON answer as `/api/v1/ip/{ip}`
-- `GET /ip/` - The same, for the caller's own address: the address the request
-  came from or, when that is a trusted proxy (`TRUSTED_PROXIES`, see
-  Configuration), the client address in its `X-Forwarded-For`
 
 ### API v1
 
-- `GET /api/v1/stats` - The `/status.json` figures plus connection, garbage
+- `GET /api/v1/stats` - Detailed statistics with handler metrics
-  collector, stream and handler queue statistics. `stream.route_churn_per_min`
+- `GET /api/v1/ip/{ip}` - Look up AS information for an IP address
-  is the number of prefix announcements and withdrawals, from all peers, in the
+- `GET /api/v1/as/{asn}` - Get prefixes announced by an AS
-  last full clock minute.
+- `GET /api/v1/prefix/{prefix}` - Get routes for a specific prefix
-- `GET /api/v1/ip/{ip}` - Looks up each address. `{ip}` is an IP address, a host
-  name, or a comma-separated list of at most 10 of these. `results` has one
-  entry for each address, with:
-    - `ip` and `ptr`: the address and its PTR names
-    - `netblock` and the fields beside it: the address's most specific live
-      route, and that route's origin AS with its WHOIS details
-    - `blocklists`: the public blocklists the address is on (see Blocklists)
-
-    An address with no live route has a result without the route and AS fields.
-    Of the addresses a host name resolves to, only public ones are looked up. A
-    name with none, such as one that resolves to a private address, is reported
-    as `host name has no public address`, without its addresses. `errors` has
-    one entry for each host name that failed. The answer is 400 when:
-    - `{ip}` lists more than 10
-    - `{ip}` comes to more than 10 addresses once its names are resolved
-    - nothing in `{ip}` is an IP address or a host name with a public address
-
-    A PTR lookup gets 2 seconds. An address whose lookup takes longer is
-    answered without PTR names.
-
-- `GET /api/v1/as/{asn}` - The AS and the prefixes it originates, IPv4 and IPv6
-  apart, each with when its newest route was updated. When the database has no
-  record of the AS, its handle and description come from the AS list built into
-  RouteWatch. The answer then has no `first_seen` or `last_seen`.
-- `GET /api/v1/prefix/{prefix}/{length}` - Every live route for the prefix, one
-  per peer, and the same routes grouped by origin AS in `origins`
-- `GET /api/v1/search/as?q={text}` - Finds the ASes whose handle or description
-  contains `{text}`, ignoring case. `data` holds:
-    - `query`: `{text}` without its leading and trailing spaces
-    - `total`: the number of ASes that match
-    - `results`: the first 50 of them in AS number order, each with `asn`,
-      `handle` and `description`
-
-    The search reads the AS list built into RouteWatch, not the database. The
-    list comes from
-    [ipverse/as-metadata](https://github.com/ipverse/as-metadata). A match can
-    therefore be an AS that the database has no record of. A `q` that is
-    missing, empty or only spaces answers 400.
-
-A JSON answer is `{"status": "ok", "data": ...}`, or for an error
-`{"status": "error", "error": {"msg": ..., "code": ...}}`, and carries an
-`@meta` object with the time zone, the API version and the time the request
-took. The health check is the exception: its 503 answer is also
-`{"status": "ok", "data": ...}`, with `error` in `data.status`.
-
-A request that runs past its time limit answers 504: the limit is 30 seconds,
-and 4 seconds for the database reads behind `/status.json` and `/api/v1/stats`.
-An AS number, prefix or mask length that does not parse answers 400, as does a
-mask length out of range for its IP version. An AS answers 404 when it is not in
-the AS list built into RouteWatch and the database has neither a record of it
-nor a live route it originates. AS 0 always answers 404, because it means no
-known origin. The AS name search never lists it. A prefix with no live route
-answers 404 too. The HTML pages give these errors as plain text. Under `/api/v1`
-and `/ip/`, a path no route matches answers 404 with the message `Not found`,
-and `POST`, `PUT`, `PATCH`, `DELETE`, `OPTIONS`, `TRACE` or `CONNECT` answers
-405 with `Method not allowed`. A 500 answer says only `Internal server error`,
-and a host name that does not resolve only `could not resolve host name`; the
-detail goes to the log.
-
-Every answer carries `Strict-Transport-Security`, `Content-Security-Policy`,
-`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy` and
-`Permissions-Policy`. The Content-Security-Policy lets a page load content only
-from the server, and does not allow inline content with `'unsafe-inline'`: the
-pages keep their CSS and JavaScript in inline `<style>` and `<script>` blocks,
-so each answer's policy names a random nonce made for that answer, and the
-browser applies or runs only the blocks that carry it. The pages therefore use
-no `style` attributes, which a nonce cannot cover.
 
 An origin AS of 0 (`origin_asn` in a route, `asn` in an IP lookup, the key `0`
 in a prefix's `origins`) means the route has no known origin: its AS path ends
 in an AS set of several members, any of which may have originated it. The pages
 show such a route as having no known origin.
 
-## Blocklists
-
-The IP lookup reports which public blocklists list each address. Each result has
-a `blocklists` list with one entry for each list the address is on:
-
-- `name`: the list's name, from the lists below
-- `description`: what being on the list means
-- `updated`: when RouteWatch last downloaded the list, or last heard from its
-  publisher that its copy was current
-
-An address on no list has an empty `blocklists`. An address with no live route
-still has a result with its `blocklists`.
-
-The lists:
-
-- `dshield`: the 20 networks that DShield saw scan the most targets in the last
-  three days. From [DShield](https://isc.sans.edu/feeds_doc.html), downloaded
-  every hour.
-- `feodo_tracker`: botnet command-and-control servers. From abuse.ch
-  [Feodo Tracker](https://feodotracker.abuse.ch/blocklist/), downloaded every 15
-  minutes.
-- `firehol_abusers_30d`: addresses reported for comment, form or forum spam in
-  the last 30 days. From
-  [FireHOL](https://iplists.firehol.org/?ipset=firehol_abusers_30d), downloaded
-  every 5 hours.
-- `firehol_level3`: addresses reported for attacks, spyware or viruses in the
-  last 30 days. From
-  [FireHOL](https://iplists.firehol.org/?ipset=firehol_level3), downloaded every
-  3 hours.
-- `spamhaus_drop` and `spamhaus_dropv6`: IPv4 and IPv6 networks that Spamhaus
-  says spammers or cyber criminals have hijacked or lease. From
-  [Spamhaus DROP](https://www.spamhaus.org/blocklists/do-not-route-or-peer/),
-  downloaded once a day. Spamhaus merged its EDROP list into DROP.
-- `tor_exits`: Tor exit nodes, where traffic from the Tor network leaves. From
-  the [Tor Project](https://check.torproject.org/torbulkexitlist), downloaded
-  every hour.
-
-No list is downloaded more often than its publisher updates it or allows. Each
-can be downloaded whole without an account or a key.
-
-How RouteWatch keeps the lists:
-
-- It holds each list in memory, so a lookup makes no outside request.
-- It keeps a copy of each list in the `blocklists` directory of the state
-  directory (see Configuration). Each copy is a file named after its list.
-- Beside each copy, a file ending in `.validators` keeps the `Last-Modified` and
-  `ETag` the publisher sent with that copy.
-- At startup it loads each list from its copy. It downloads a list at once when
-  the copy is older than the list's interval, or when there is no copy.
-- A download sends the copy's `Last-Modified` as `If-Modified-Since` and its
-  `ETag` as `If-None-Match`. The publisher then sends the list only when it has
-  changed. Without either, the download asks for the whole list.
-- A download fails when it takes over a minute or its file is over 16 MiB.
-- When a download fails, or its file does not parse, RouteWatch keeps the
-  previous copy and logs the failure. It tries again one interval later.
-
-A successful IP lookup answer carries `Cache-Control` with a `max-age` of 15
-minutes, the shortest list interval. The answer for the caller's own address
-(`/ip/`) is `private`, and the others are `public`.
-
 ## Code Structure
 
 ```
@@ -231,19 +68,15 @@ routewatch/
 │   ├── asinfo-gen/         # Utility to generate AS info data
 │   └── streamdumper/       # Debug utility for raw stream output
 ├── internal/
-│   ├── blocklist/          # Public IP blocklists: downloads and lookups
 │   ├── routewatch/         # Core application logic
 │   ├── server/             # HTTP server and handlers
 │   ├── database/           # SQLite storage layer
-│   ├── routetable/         # The whole routing table in memory
 │   ├── streamer/           # RIPE RIS Live client
 │   ├── ristypes/           # BGP message data structures
 │   ├── logger/             # Structured logging wrapper
 │   ├── metrics/            # Performance metrics tracking
 │   ├── config/             # Configuration management
-│   ├── templates/          # HTML templates
+│   └── templates/          # HTML templates
-│   ├── version/            # Build version information
-│   └── whois/              # WHOIS lookups of AS details
 └── pkg/
     └── asinfo/             # AS information lookup (public API)
 ```
@@ -262,7 +95,7 @@ routewatch/
 │  │   Streamer   │───▶│   Handlers   │───▶│   Database   │       │
 │  │              │    │              │    │              │       │
 │  │ RIS Live     │    │ - ASHandler  │    │ SQLite with  │       │
-│  │ HTTP stream  │    │ - PeerHandler│    │ WAL mode     │       │
+│  │ WebSocket    │    │ - PeerHandler│    │ WAL mode     │       │
 │  │ client       │    │ - PrefixHdlr │    │              │       │
 │  │              │    │ - PeeringHdlr│    │ Tables:      │       │
 │  └──────────────┘    └──────────────┘    │ - asns       │       │
@@ -298,17 +131,13 @@ routewatch/
     - Parses BGP UPDATE messages from JSON stream
     - Dispatches to registered handlers based on message type
     - Each handler has its own queue with backpressure handling
-    - Handlers batch writes for efficiency: the AS, peer and prefix handlers
+    - Handlers batch writes for efficiency (25K-30K ops, 1-2s timeout)
-      write 30,000, 10,000 and 25,000 updates at a time, or after 2, 2 and 1
-      seconds; the peering handler writes the peerings it has collected every 30
-      seconds
 
 4. **Handler Details**
     - **ASHandler**: Tracks all ASNs seen in AS paths
     - **PeerHandler**: Records BGP peer information
     - **PrefixHandler**: Maintains live routing table (upserts on announcement,
-      deletes on withdrawal or on an announcement whose AS path is empty or
+      deletes on withdrawal, deletes all of a peer's routes when its BGP session
-      holds AS 0 anywhere, deletes all of a peer's routes when its BGP session
       goes down). Routes are kept per peer IP, so when one peer IP has sessions
       on two collectors, a session going down on one also removes the routes the
       other still carries, until they are announced again.
@@ -321,15 +150,10 @@ routewatch/
 
 ### Key Design Patterns
 
-- **Batched Writes**: The handlers write the feed's updates to the database in
+- **Batched Writes**: All database operations are batched for performance
-  batches for performance
 - **Backpressure**: Probabilistic message dropping when queues exceed 50%
   capacity
-- **Graceful Shutdown**: on SIGTERM or SIGINT the streamer stops, the handlers
+- **Graceful Shutdown**: 60-second timeout, flushes all pending batches
-  write the batches they hold, the HTTP server stops and the database is
-  checkpointed and closed. The daemon gives this 8 seconds and exits with status
-  0; "Running under upaas" says what a stop loses, logs and leaves undone when
-  the 8 seconds run out.
 - **Reconnection**: Any end of the stream other than shutdown, including RIS
   Live ending the response normally, marks the feed disconnected and reconnects
   with exponential backoff (5s-320s). The backoff resets after a stream that
@@ -345,35 +169,21 @@ routewatch/
 
 ```sql
 -- Core tables
-asns(asn PRIMARY KEY, handle, description,
+asns(id, number, handle, description, first_seen, last_seen)
-     as_name, org_name, org_id, address, country_code, abuse_email,
+prefixes_v4(id, prefix, mask_length, first_seen, last_seen)
-     abuse_phone, tech_email, tech_phone, rir, rir_registration_date,
+prefixes_v6(id, prefix, mask_length, first_seen, last_seen)
-     rir_last_modified, whois_raw,
-     first_seen, last_seen, whois_updated_at)
-prefixes_v4(prefix PRIMARY KEY, first_seen) WITHOUT ROWID
-prefixes_v6(prefix PRIMARY KEY, first_seen) WITHOUT ROWID
 
--- Live routing tables (one per IP version), one route per peer and prefix
+-- Live routing tables (one per IP version)
-live_routes_v4(id INTEGER PRIMARY KEY, prefix, mask_length, origin_asn,
+live_routes_v4(id, prefix, mask_length, origin_asn, peer_ip, as_path,
-               peer_ip, as_path, next_hop, last_updated,
+               next_hop, last_updated)
-               UNIQUE(peer_ip, prefix))
+live_routes_v6(id, prefix, mask_length, origin_asn, peer_ip, as_path,
-live_routes_v6(id INTEGER PRIMARY KEY, prefix, mask_length, origin_asn,
+               next_hop, last_updated)
-               peer_ip, as_path, next_hop, last_updated,
-               UNIQUE(peer_ip, prefix))
 
 -- Relationship tracking
-peerings(id PRIMARY KEY, as_a, as_b, first_seen, last_seen,
+peerings(id, as_a, as_b, first_seen, last_seen)
-         UNIQUE(as_a, as_b))
+bgp_peers(id, peer_ip, peer_asn, last_message_type, last_seen)
-bgp_peers(id PRIMARY KEY, peer_ip UNIQUE, peer_asn, first_seen, last_seen,
-          last_message_type)
-
--- Nothing in the daemon writes this table
-announcements(id PRIMARY KEY, prefix_id, peer_asn, origin_asn, path,
-              next_hop, timestamp, is_withdrawal)
 ```
 
-The indexes are in `internal/database/schema.sql`.
-
 ## Configuration
 
 Configuration is handled via environment variables and OS-specific paths:
@@ -385,23 +195,13 @@ Configuration is handled via environment variables and OS-specific paths:
 | `XDG_DATA_HOME`    | `/var/lib` (in the Docker image) | Base of the state directory; must be an absolute path       |
 | `GOMEMLIMIT`       | `1536MiB` (in the Docker image)  | Go soft memory limit; see Memory                            |
 | `MALLOC_ARENA_MAX` | `2` (in the Docker image)        | glibc malloc arena cap, a positive whole number; see Memory |
-| `TRUSTED_PROXIES`  | (empty)                          | Addresses and CIDR ranges of trusted proxies; see below     |
 
 A variable that is set to an invalid value stops the start with an error and a
 non-zero exit. An empty variable counts as unset.
 
-`TRUSTED_PROXIES` lists the reverse proxies whose `X-Forwarded-For` the server
+State directory (database location):
-believes, comma-separated, such as `10.0.0.0/8,192.0.2.1`. For a request from
-one of them, the request log and `/ip/` use the last address in
-`X-Forwarded-For` that is not a trusted proxy: each proxy adds the address it
-received the request from to the end, so the entries before it are whatever the
-client sent. For a request from any other address they use the address the
-request came from, whatever forwarding headers it carries. `X-Real-IP` and
-`True-Client-IP` are not read.
 
-State directory (the database and the blocklist copies):
+- macOS: `~/Library/Application Support/routewatch/`
-
-- macOS: `~/Library/Application Support/berlin.sneak.app.routewatch/`
 - Linux: `/var/lib/berlin.sneak.app.routewatch/` when running as root, otherwise
   `$XDG_DATA_HOME/berlin.sneak.app.routewatch/` (with `XDG_DATA_HOME` unset,
   `~/.local/share/berlin.sneak.app.routewatch/`). In the Docker image this is
@@ -466,14 +266,8 @@ What happens at each limit:
   (`peerBatchMaxSize`, about 10 MiB) and 250,000 for the prefix handler
   (`prefixBatchMaxSize`, about 60 MiB). At that size the handler takes nothing
   more off its queue until the write ends, so the queue fills and the streamer
-  drops messages as above. When the delete of a peer's live routes after its
+  drops messages as above. With the batches being written, the three handlers'
-  session went down fails, the prefix handler keeps the updates the peer sent
+  batches take at most about 0.25 GiB of the Go heap's 1.5 GiB.
-  after the down for the next write, which retries the delete first; on top of
-  its 250,000 it keeps at most the last 25,000 of them (`prefixBatchSize`, about
-  6 MiB) and drops and logs the older ones, so while the delete keeps failing
-  the other peers' updates still come off the queue. With the batches being
-  written, the three handlers' batches take at most about 0.25 GiB of the Go
-  heap's 1.5 GiB.
 
 With `DEBUG=routewatch` the daemon logs a `System stats` line every 60 seconds
 with the goroutine count and Go memory figures.
@@ -484,36 +278,15 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app needs:
 
 - Container port: `8080`.
 - Volume: one, at container path `/var/lib/berlin.sneak.app.routewatch`.
-- Environment: `TRUSTED_PROXIES`, set to the address the reverse proxy in front
+- Environment: nothing is required. Leave `XDG_DATA_HOME`, `GOMEMLIMIT` and
-  of the app connects from, as the container sees it; without it the request log
+  `MALLOC_ARENA_MAX` at the image's values. `DEBUG=routewatch` is optional and
-  and `/ip/` give the proxy's address for every request. Nothing else is needed:
+  adds the `System stats` memory line to the log.
-  leave `XDG_DATA_HOME`, `GOMEMLIMIT` and `MALLOC_ARENA_MAX` at the image's
-  values. `DEBUG=routewatch` is optional and adds the `System stats` memory line
-  to the log.
 - Memory Limit: `5g`, the 5 GiB limit from Memory above. upaas sets no swap
   limit, so on a host with swap Docker allows the same amount of swap again.
 - Health check: the image's `HEALTHCHECK` requests
   `/.well-known/healthcheck.json` on the container port. upaas reads the
   container's health 60 seconds after a deploy and fails the deploy unless it is
   `healthy`.
-- Stop grace: the container needs 9 seconds between SIGTERM and SIGKILL. upaas
-  gives 10 (`stopTimeoutSeconds` in its `internal/docker/client.go`), as
-  `docker stop` does by default. The daemon allows 8 seconds for the handlers to
-  write what they hold and for the requests in progress to finish, then stops
-  the HTTP server, closes the database and exits with status 0. A longer grace
-  changes nothing. What is not written after 8 seconds is lost, and the streamer
-  and each handler log how much. When the prefix handler has fallen behind the
-  feed, a stop loses far more: its batch is full at 250,000 updates and the
-  streamer's worker for it waits for room, so the streamer's stop takes the
-  whole 8 seconds and every handler then skips its last write. Lost then are the
-  messages left on the four handlers' queues, what the AS, peer and peering
-  handlers hold, the prefix handler's batch, and the write it has in progress,
-  up to another 250,000 updates, which take about 20 seconds and are cut off by
-  the exit; withdrawals and peer downs are among them. A route withdrawn in them
-  stays in the live route table until its peer announces or withdraws the prefix
-  again or its session goes down. When the 8 seconds run out the database is not
-  closed, and SQLite rolls back a write the exit cut off when the database next
-  opens.
 
 ## Development
 
@@ -528,7 +301,7 @@ make fmt
 make lint
 
 # Build
-make build
+make
 ```
 
 ## Entrypoints
@@ -557,15 +330,10 @@ provide:
 - `script/docker` — build the Docker image tagged via `script/projectname`
 - `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`,
   then builds the image
-- `script/precommit` — pre-commit gate: `go mod tidy` (must not change `go.mod`
+- `script/precommit` — pre-commit gate: `go mod tidy` + `go fmt` (must not
-  or `go.sum`) and `go fmt` (writes), then `script/check`
+  change files), then `script/check`
 - `script/install-precommit` — install the git pre-commit hook that runs
   `script/precommit`
-- `script/asupdate` — rebuild the AS list built into RouteWatch,
-  `pkg/asinfo/asdata.json.gz`, from the `as.csv` file of
-  [ipverse/as-metadata](https://github.com/ipverse/as-metadata). It needs the
-  network. It runs `cmd/asinfo-gen`, which fails when it finds no AS. When
-  `cmd/asinfo-gen` fails, the old list stays and the script exits non-zero.
 
 Tests and the linter run only in Docker, as the `test` and `lint` phases of the
 `Dockerfile`; the image cannot be built unless both pass. Every `docker build`

TODO.md

@@ -10,165 +10,28 @@
 
 # Status
 
-pre-1.0. No git tags. `main` last took `next` on 2026-10-06, through
+pre-1.0. No git tags. The linter and the tests are phases of the Docker build,
-https://git.eeqj.de/sneak/routewatch/pulls/6. The linter and the tests are
+and the Gitea workflow runs `script/cibuild` (the format check of the Go code
-phases of the Docker build, and the Gitea workflow runs `script/cibuild` (the
+and the markdown, both phases, then the image) on every push. The image sets
-format check of the Go code and the markdown, both phases, then the image) on
+memory ceilings for a 5 GiB container (README "Memory") and the README says how
-every push. The image sets memory ceilings for a 5 GiB container (README
+to run it under upaas (README "Running under upaas"). A 35-hour run of `3898daa`
-"Memory") and the README says how to run it under upaas (README "Running under
+on the live feed peaked at about 1 GiB, without a container memory limit.
-upaas"). A 25-hour run of `a9b6860` on the live feed peaked at about 0.6 GiB
-(https://git.eeqj.de/sneak/routewatch/issues/3). A run of `825afa2` on the live
-feed found the prefix handler dropping about a quarter of its updates because
-its route writes were slower than the feed
-(https://git.eeqj.de/sneak/routewatch/issues/96); the cheaper writes since then
-are measured by a benchmark, not yet on the live feed.
 
 # Next Step
 
-Run `next` on the live feed, as README "Running under upaas" describes, for the
+`next` waits for sneak to merge it to `main` through
-final check https://git.eeqj.de/sneak/routewatch/issues/96 names: whether the
+https://git.eeqj.de/sneak/routewatch/pulls/6. After that, setting routewatch up
-prefix handler still drops updates after the first hour. If it does, a new issue
+under upaas on fsn1app1 and deploying it are his
-follows with the numbers. Then an audit of the code at the commit that run
+(https://git.eeqj.de/sneak/routewatch/issues/31), and so is the run under a real
-tested.
+5 GiB limit (https://git.eeqj.de/sneak/routewatch/issues/3).
 
 # Completed Steps
 
-- 2026-10-27: the database keeps a copy of the whole routing table in a new
-  `routes` table, one row per prefix and origin AS. The peers' numbers are in a
-  new `route_peers` table. One function writes the pairs that changed in memory,
-  and another reads the whole table back into memory at startup. Nothing uses
-  them yet (part of #122)
-- 2026-10-27: a new package, `internal/routetable`, holds the whole routing
-  table in memory. For each prefix and origin AS it keeps the peers that carry
-  it, one bit per peer, and the two newest of their routes. Nothing uses it yet
-  (part of #122)
-- 2026-10-27: each result of the IP lookup lists the public blocklists the
-  address is on, in `blocklists`. The lists are DShield, Feodo Tracker, two
-  FireHOL lists, Spamhaus DROP and the Tor exit list. RouteWatch downloads each
-  list on its own schedule and answers from copies in memory. The copies are
-  also kept in the state directory's `blocklists` directory. IP lookup answers
-  carry `Cache-Control`. An address with no live route now has a result too,
-  without route fields, so the IP lookup no longer answers 404 (closes #158)
-- 2026-10-27: the home page has an IP information API card. It shows the `curl`
-  command that looks up the visitor's own address at `/ip/{addr}`, and that
-  command's live answer (closes #156)
-- 2026-10-27: on the AS page, a click on the heading of the IPv4 or the IPv6
-  prefix list folds or unfolds the list. Both lists start open (closes #155)
-- 2026-10-27: the AS page lists its IPv4 and its IPv6 prefixes by mask length,
-  shortest first, and prefixes of one mask length by address. They used to be
-  listed by address (closes #153)
-- 2026-10-27: on `/` and `/status`, a spinner beside the status turns one frame
-  each time the stats show more messages than the time before. A page that has
-  not seen more messages for three refreshes shows "stale" and the time since
-  then instead, in red (closes #152)
-- 2026-10-27: the WHOIS details of a Brazilian AS keep their accented letters.
-  LACNIC's answer for such an AS comes from NIC.br, Brazil's registry, with each
-  accented letter replaced. The details now come from NIC.br's own server,
-  `whois.registro.br`. When that query fails, the details from LACNIC's answer
-  are kept and a warning is logged. After NIC.br answers that RouteWatch is over
-  its query limit, RouteWatch stops asking NIC.br for an hour (closes #174)
-- 2026-10-27: `make asupdate` stops with an error when the download of `as.csv`
-  breaks off. The generator used to retry the failed read forever. A record that
-  does not parse is still logged and skipped (closes #177)
-- 2026-10-27: `make asupdate` runs the new `script/asupdate`. It replaces the AS
-  list built into RouteWatch only when the generator succeeds and finds at least
-  one AS. A failed download used to replace the list with an empty file. The
-  generator reads `as.csv` from `ipverse/as-metadata`, the source's new name
-  (closes #173)
-- 2026-10-27: the WHOIS details of an AS at RIPE, APNIC or AFRINIC come only
-  from its `aut-num` object and the organisation object it names. RIPE and APNIC
-  start their answer with the `as-block` object of the AS's block. Its dates
-  were stored as the AS's (closes #168)
-- 2026-10-27: AS names keep their accented letters. The AS list built into
-  RouteWatch is rebuilt from today's `as.csv`, which added a country code
-  column. That file no longer lists AS 0 or the documentation AS numbers. A
-  WHOIS answer that is not valid UTF-8 is decoded as Latin-1, which RIPE and
-  APNIC send, before it is parsed and stored (closes #165)
-- 2026-10-27: the Age column on the AS and prefix pages reads in words, such as
-  "5 minutes ago" or "2 weeks ago", instead of Go durations such as "360h0m0s
-  ago". An age under a minute reads in seconds, and one over a month in months
-  or years instead of a date (closes #163)
-- 2026-10-26: a write to a stored AS, peering or peer keeps the earlier of the
-  two first-seen times and the later of the two last-seen times. An old update
-  from a collector catching up on a backlog no longer moves last seen back
-  (closes #162)
-- 2026-10-26: the status page shows "Route Churn/min", the number of prefix
-  announcements and withdrawals, from all peers, in the last full clock minute.
-  `/api/v1/stats` reports it as `route_churn_per_min` in `stream`, which
-  replaces the moving average `route_churn_per_sec`. The messages and updates
-  per second on `/` and `/status` are whole numbers (closes #160)
-- 2026-10-26: the WHOIS fetcher backs off more slowly. A failure makes its wait
-  1.5 times as long, up to one minute instead of five. Its queries to RIPE,
-  APNIC and AFRINIC leave out person and role objects, which RIPE counts against
-  a daily limit per address. A refused query is logged with the RIR's error line
-  (closes #159)
-- 2026-10-26: the home page's AS name search lists the ASes whose handle or
-  description holds the text, as links to their pages, from the new
-  `/api/v1/search/as`; the search ignores case. When the database has no record
-  of an AS, its page takes the handle and description from the AS list built
-  into RouteWatch and still lists the AS's prefixes (closes #151)
-- 2026-10-19: README "HTTP Endpoints" lists every route with what it answers,
-  "Database Schema" matches `schema.sql`, and the rest of the README matches the
-  code; this file brought up to date (closes #72)
-- 2026-10-19: route writes are cheaper: the live route and prefix tables lose
-  their random UUID keys and the prefix tables their duplicate index, a route
-  update sets only the columns that change, and every write goes through one
-  connection with a page cache of 1 GiB (README "Memory"). A benchmark writes
-  the prefix handler's largest batch faster than the feed sends it (closes #96)
-- 2026-10-19: the WAL checkpoint, the incremental vacuum and ANALYZE take the
-  database lock every batch write holds, so a vacuum no longer fails with
-  "database is locked" and an ANALYZE no longer makes a batch write lose its
-  updates; ANALYZE runs as `PRAGMA optimize` with an analysis limit (closes #94)
-- 2026-10-19: shutdown stops the streamer first, and each handler writes
-  everything left on its queue before it stops, so the updates waiting there are
-  no longer lost (closes #69)
-- 2026-10-19: a path that ends in an AS set of several members has origin AS 0,
-  no known origin, instead of the set's last member, and the set adds no
-  peerings; the pages show such a route as having no known origin (closes #75)
-- 2026-10-19: `bgp_peer_count` drops to 0 when the last peer goes down (closes
-  #92)
-- 2026-10-19: `bgp_peer_count` counts every peer the feed has heard from, not
-  only those whose session opened since startup, and drops a peer whose session
-  goes down (closes #74)
 - 2026-10-19: the shared files are re-vendored from `sneak/prompts` at
   `0653579`: golangci-lint v2.14.0 and its configuration, lint and test as
   `Dockerfile` phases that keep Go's and golangci-lint's caches, markdown
   formatted with prettier, agent guidance in `AGENTS.md`; the code is fixed for
   what the new linter configuration finds (closes #52)
-- 2026-10-19: the streamer reads feed lines up to 1 MiB, so a line over 64 KiB
-  no longer ends the connection (closes #76)
-- 2026-10-19: a peer's live routes are removed when its BGP session goes down;
-  `STATE` messages are never dropped for backpressure, and one that finds its
-  queue full waits for room ahead of the handler's other messages (closes #56)
-- 2026-10-19: the prefix-length pages pick their sample with a bounded number of
-  index lookups instead of reading every route of that length, and list each
-  prefix once (closes #66)
-- 2026-10-19: a request past its time limit answers 504 instead of 408, and
-  `/api/v1/stats` answers 504 instead of 500 when its read runs out of time
-  (closes #70)
-- 2026-10-19: the AS, peer and prefix handlers write each batch outside the lock
-  their message handling needs, so messages keep coming off their queues during
-  a write; meanwhile the next batch grows to at most ten batch sizes (closes
-  #65)
-- 2026-10-19: a live route is identified by prefix and peer, so an announcement
-  with a new origin AS replaces the peer's route and a withdrawal removes it
-  whatever its origin AS (closes #55)
-- 2026-10-14: the streamer reconnects when RIS Live ends the stream or sends
-  nothing for 60 seconds, and the health check reports the feed disconnected
-  meanwhile (closes #53)
-- 2026-10-14: the prefix page takes the names of the origin ASes from the prefix
-  query instead of reading every route of each origin AS (closes #67)
-- 2026-10-14: the daemon exits with status 1 when its web server cannot listen
-  on its port, instead of running on without it (closes #73)
-- 2026-10-14: the prefix pages parse the prefix before looking it up: one that
-  does not parse answers 400, and any valid way of writing a prefix finds its
-  routes (closes #71)
-- 2026-10-14: the IP lookup ignores routes shorter than /8 for IPv4 and /16 for
-  IPv6, so a default route is no longer named as an address's owner (closes #57)
-- 2026-10-14: a prefix batch keeps each peer's last update for a prefix, instead
-  of one update per prefix across all peers (closes #54)
-- 2026-10-14: `reconnect_count` in `/api/v1/stats` counts every reconnect
-  (closes #68)
 - 2026-10-03: `/api/v1/stats` serves the prefix distribution from memory, seeded
   at startup and adjusted on every live-route write, so a request no longer
   reads every live route (closes #30)
@@ -253,6 +116,6 @@ tested.
 
 # Future Steps
 
-- the rest of https://git.eeqj.de/sneak/routewatch/issues/122, one PR each: the
+- Production memory under 5 GiB: whether to test under a real 5 GiB container
-  prefix handler writing to the routing table, the pages and the API reading
+  limit on fsn1app1 is open for sneak
-  from it, then removing the per-peer route tables
+  (https://git.eeqj.de/sneak/routewatch/issues/3)

cmd/asinfo-gen/main.go

@@ -6,7 +6,6 @@ import (
 	"encoding/csv"
 	"encoding/json"
 	"errors"
-	"fmt"
 	"io"
 	"log"
 	"net/http"
@@ -15,21 +14,7 @@ import (
 	"strings"
 )
 
-const asCsvURL = "https://github.com/ipverse/as-metadata/raw/refs/heads/master/as.csv"
+const asCsvURL = "https://github.com/ipverse/asn-info/raw/refs/heads/master/as.csv"
-
-// The first columns of as.csv, the ones the generator reads. It reads past
-// the columns after them, such as the country code.
-const (
-	asnColumn         = "asn"
-	handleColumn      = "handle"
-	descriptionColumn = "description"
-	columnsRead       = 3
-)
-
-var (
-	errUnexpectedHeader = errors.New("unexpected CSV header")
-	errNoASes           = errors.New("no AS in the CSV")
-)
 
 // ASInfo represents information about an Autonomous System
 type ASInfo struct {
@@ -60,66 +45,40 @@ func main() {
 		log.Fatalf("Failed to fetch CSV: HTTP %d", resp.StatusCode)
 	}
 
-	asInfos, err := parseCSV(resp.Body)
+	// Parse CSV
-
+	reader := csv.NewReader(resp.Body)
-	// Closed here rather than deferred: log.Fatalf exits without running
-	// deferred calls.
-	_ = resp.Body.Close()
-
-	if err != nil {
-		log.Fatalf("Failed to parse CSV: %v", err)
-	}
-
-	log.Printf("Parsed %d AS records", len(asInfos))
-
-	// Convert to JSON and output to stdout
-	encoder := json.NewEncoder(os.Stdout)
-	encoder.SetIndent("", "\t")
-
-	err = encoder.Encode(asInfos)
-	if err != nil {
-		log.Fatalf("Failed to encode JSON: %v", err)
-	}
-}
-
-// parseCSV reads as.csv from r and returns its ASes, with the handle and the
-// description as the file spells them. A header whose first columns are not
-// the AS number, the handle and the description is an error. A record that
-// does not parse is logged and skipped. A read that fails, such as on a broken
-// download, is an error. A file left with no AS is an error.
-func parseCSV(r io.Reader) ([]ASInfo, error) {
-	reader := csv.NewReader(r)
 
+	// Read header
 	header, err := reader.Read()
 	if err != nil {
-		return nil, fmt.Errorf("read CSV header: %w", err)
+		log.Fatalf("Failed to read CSV header: %v", err)
 	}
 
-	if len(header) < columnsRead || header[0] != asnColumn ||
+	if len(header) != 3 || header[0] != "asn" || header[1] != "handle" ||
-		header[1] != handleColumn || header[2] != descriptionColumn {
+		header[2] != "description" {
-		return nil, fmt.Errorf("%w: %v", errUnexpectedHeader, header)
+		log.Fatalf("Unexpected CSV header: %v", header)
 	}
 
+	// Read all records
 	var asInfos []ASInfo
 
 	for {
-		// The reader returns an error for a record with a different number
-		// of fields than the header.
 		record, err := reader.Read()
 		if errors.Is(err, io.EOF) {
 			break
 		}
 
-		if _, ok := errors.AsType[*csv.ParseError](err); ok {
+		if err != nil {
 			log.Printf("Error reading CSV record: %v", err)
 
 			continue
 		}
 
-		// Any other error comes from the download, such as a broken
+		const expectedFields = 3
-		// connection. Every later read returns it again.
+		if len(record) != expectedFields {
-		if err != nil {
+			log.Printf("Skipping invalid record: %v", record)
-			return nil, fmt.Errorf("read CSV record: %w", err)
+
+			continue
 		}
 
 		asn, err := strconv.Atoi(record[0])
@@ -136,9 +95,18 @@ func parseCSV(r io.Reader) ([]ASInfo, error) {
 		})
 	}
 
-	if len(asInfos) == 0 {
+	// Closed here rather than deferred: log.Fatalf exits without running
-		return nil, errNoASes
+	// deferred calls.
-	}
+	_ = resp.Body.Close()
 
-	return asInfos, nil
+	log.Printf("Parsed %d AS records", len(asInfos))
+
+	// Convert to JSON and output to stdout
+	encoder := json.NewEncoder(os.Stdout)
+	encoder.SetIndent("", "\t")
+
+	err = encoder.Encode(asInfos)
+	if err != nil {
+		log.Fatalf("Failed to encode JSON: %v", err)
+	}
 }

cmd/asinfo-gen/main_test.go

@@ -1,142 +0,0 @@
-package main
-
-import (
-	"errors"
-	"io"
-	"slices"
-	"strings"
-	"testing"
-	"testing/iotest"
-	"time"
-)
-
-// parseTimeLimit is how long a test waits for parseCSV to return before it
-// fails.
-const parseTimeLimit = 2 * time.Second
-
-// TestParseCSVReadsTodaysLayout parses an as.csv with the country code column
-// that the source added after the description. It expects each AS with its
-// handle and description spelled exactly as in the file, accents included.
-func TestParseCSVReadsTodaysLayout(t *testing.T) {
-	t.Parallel()
-
-	const asCSV = `asn,handle,description,country-code
-1,LVLT-1,Level 3 Parent LLC,US
-1251,FUNDACAO-AMPARO-PESQUISA,FUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO SÃO PAULO,BR
-`
-
-	got, err := parseCSV(strings.NewReader(asCSV))
-	if err != nil {
-		t.Fatalf("parseCSV: %v", err)
-	}
-
-	want := []ASInfo{
-		{ASN: 1, Handle: "LVLT-1", Description: "Level 3 Parent LLC"},
-		{
-			ASN:    1251,
-			Handle: "FUNDACAO-AMPARO-PESQUISA",
-			Description: "FUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO " +
-				"SÃO PAULO",
-		},
-	}
-
-	if !slices.Equal(got, want) {
-		t.Errorf("parseCSV = %+v, want %+v", got, want)
-	}
-}
-
-// TestParseCSVRejectsAnotherHeader parses files whose first columns are not
-// the AS number, the handle and the description. It expects an error rather
-// than a list read from the wrong columns.
-func TestParseCSVRejectsAnotherHeader(t *testing.T) {
-	t.Parallel()
-
-	for _, asCSV := range []string{
-		"asn,handle\n1,LVLT-1\n",
-		"asn,description,handle\n1,Level 3 Parent LLC,LVLT-1\n",
-	} {
-		_, err := parseCSV(strings.NewReader(asCSV))
-		if err == nil {
-			t.Errorf("parseCSV(%q): no error", asCSV)
-		}
-	}
-}
-
-// TestParseCSVRejectsAFileWithNoASes parses files that hold no AS, one with
-// only the header and one whose only record has no valid AS number. It
-// expects errNoASes, so that make asupdate keeps the old list.
-func TestParseCSVRejectsAFileWithNoASes(t *testing.T) {
-	t.Parallel()
-
-	for _, asCSV := range []string{
-		"asn,handle,description,country-code\n",
-		"asn,handle,description,country-code\nAS1,LVLT-1,Level 3,US\n",
-	} {
-		_, err := parseCSV(strings.NewReader(asCSV))
-		if !errors.Is(err, errNoASes) {
-			t.Errorf("parseCSV(%q) = %v, want %v", asCSV, err, errNoASes)
-		}
-	}
-}
-
-// TestParseCSVSkipsARecordThatDoesNotParse parses an as.csv with a record
-// that has too few fields between two valid records. It expects both valid
-// ASes and no error, so that one bad record does not stop make asupdate.
-func TestParseCSVSkipsARecordThatDoesNotParse(t *testing.T) {
-	t.Parallel()
-
-	const asCSV = "asn,handle,description,country-code\n" +
-		"1,LVLT-1,Level 3 Parent LLC,US\n" +
-		"2,UDEL-DCN\n" +
-		"3,MIT-GATEWAYS,Massachusetts Institute of Technology,US\n"
-
-	got, err := parseCSV(strings.NewReader(asCSV))
-	if err != nil {
-		t.Fatalf("parseCSV: %v", err)
-	}
-
-	want := []ASInfo{
-		{ASN: 1, Handle: "LVLT-1", Description: "Level 3 Parent LLC"},
-		{
-			ASN:         3,
-			Handle:      "MIT-GATEWAYS",
-			Description: "Massachusetts Institute of Technology",
-		},
-	}
-
-	if !slices.Equal(got, want) {
-		t.Errorf("parseCSV = %+v, want %+v", got, want)
-	}
-}
-
-// TestParseCSVStopsWhenTheDownloadBreaksOff parses an as.csv whose download
-// breaks off after the first record. From then on every read fails, as on a
-// broken connection. It expects parseCSV to return that error, so that make
-// asupdate keeps the old list.
-func TestParseCSVStopsWhenTheDownloadBreaksOff(t *testing.T) {
-	t.Parallel()
-
-	const asCSV = "asn,handle,description,country-code\n" +
-		"1,LVLT-1,Level 3 Parent LLC,US\n"
-
-	download := io.MultiReader(
-		strings.NewReader(asCSV), iotest.ErrReader(io.ErrUnexpectedEOF))
-
-	// parseCSV runs on its own goroutine, so that a parseCSV that never
-	// returns fails the test instead of hanging it.
-	parsed := make(chan error, 1)
-
-	go func() {
-		_, err := parseCSV(download)
-		parsed <- err
-	}()
-
-	select {
-	case err := <-parsed:
-		if !errors.Is(err, io.ErrUnexpectedEOF) {
-			t.Errorf("parseCSV = %v, want %v", err, io.ErrUnexpectedEOF)
-		}
-	case <-time.After(parseTimeLimit):
-		t.Fatalf("parseCSV did not return within %v", parseTimeLimit)
-	}
-}

cmd/streamdumper/main.go

@@ -35,7 +35,7 @@ func main() {
 	if err != nil {
 		log.Fatal("Failed to start streamer:", err)
 	}
-	defer s.Stop(context.Background())
+	defer s.Stop()
 
 	// Set up context with cancellation
 	ctx, cancel := context.WithCancel(context.Background())

go.mod

@@ -1,19 +1,15 @@
 module git.eeqj.de/sneak/routewatch
 
-go 1.27.1
+go 1.24.4
 
 require (
-	github.com/bits-and-blooms/bitset v1.26.0
 	github.com/dustin/go-humanize v1.0.1
 	github.com/go-chi/chi/v5 v5.2.2
-	github.com/google/btree v1.1.3
 	github.com/google/uuid v1.6.0
 	github.com/mattn/go-sqlite3 v1.14.29
 	github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9
 	go.uber.org/fx v1.24.0
-	go4.org/netipx v0.0.0-20260823151212-3075585bcbeb
 	golang.org/x/term v0.33.0
-	golang.org/x/text v0.43.0
 )
 
 require (

go.sum

@@ -1,13 +1,9 @@
-github.com/bits-and-blooms/bitset v1.26.0 h1:WPvcPZbP1yuKofmjs2SYIFO1EjzIGhCP9ducElLsh5Q=
-github.com/bits-and-blooms/bitset v1.26.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
 github.com/go-chi/chi/v5 v5.2.2 h1:CMwsvRVTbXVytCk1Wd72Zy1LAsAh9GxMmSNWLHCG618=
 github.com/go-chi/chi/v5 v5.2.2/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
-github.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg=
-github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/mattn/go-sqlite3 v1.14.29 h1:1O6nRLJKvsi1H2Sj0Hzdfojwt8GiGKm+LOfLaBFaouQ=
@@ -28,13 +24,9 @@ go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=
 go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
 go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
 go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
-go4.org/netipx v0.0.0-20260823151212-3075585bcbeb h1:XBM4hvfwGAttkkiTIFfeigdfcL1xIfdKXqFdgiHGtDs=
-go4.org/netipx v0.0.0-20260823151212-3075585bcbeb/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
 golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
 golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
 golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
 golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
-golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
-golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

internal/blocklist/blocklist.go

@@ -1,184 +0,0 @@
-// Package blocklist downloads public IP blocklists on a schedule, keeps the
-// last good copy of each on disk and in memory, and answers which lists
-// contain an address.
-package blocklist
-
-import (
-	"context"
-	"log/slog"
-	"net/netip"
-	"sync"
-	"time"
-
-	"go4.org/netipx"
-)
-
-// Format is how a list's file is written.
-type Format int
-
-const (
-	// FormatLines has one address or network per line, such as 192.0.2.1 or
-	// 192.0.2.0/24. Lines that start with # are comments.
-	FormatLines Format = iota
-
-	// FormatSpamhausJSON has one JSON object per line, with a network in its
-	// cidr field. The last line holds the file's metadata and no network.
-	FormatSpamhausJSON
-
-	// FormatDShield has one network per line, as tab-separated fields that
-	// start with the network's first and last address. Lines that start
-	// with # are comments.
-	FormatDShield
-)
-
-// List is one public blocklist.
-type List struct {
-	// Name names the list in lookup answers, and names its copy on disk.
-	Name string
-
-	// Description says in one line what being on the list means.
-	Description string
-
-	// URL is where the publisher serves the whole list.
-	URL string
-
-	// Interval is how long a copy is used before the list is downloaded
-	// again.
-	Interval time.Duration
-
-	// Format is how the list's file is written.
-	Format Format
-}
-
-// Listing says that a list contains an address.
-type Listing struct {
-	Name        string `json:"name"`
-	Description string `json:"description"`
-
-	// Updated is when routewatch last downloaded the list, or last heard from
-	// its publisher that its copy was still current.
-	Updated time.Time `json:"updated"`
-}
-
-// listCopy is a list's last good copy in memory.
-type listCopy struct {
-	addresses *netipx.IPSet
-	updated   time.Time
-
-	// validators identify the version of the list this copy holds.
-	validators validators
-}
-
-// Params is what New needs.
-type Params struct {
-	// Dir is the directory that holds the copies on disk.
-	Dir string
-
-	// Lists are the lists to download and look addresses up in.
-	Lists []List
-
-	Logger *slog.Logger
-}
-
-// Blocklists keeps a copy of each list and answers which lists contain an
-// address. Start loads the copies from disk and starts downloading the lists.
-type Blocklists struct {
-	dir    string
-	lists  []List
-	logger *slog.Logger
-
-	// copies holds each list's last good copy, by list name. A list with no
-	// good copy yet has none.
-	mu     sync.RWMutex
-	copies map[string]listCopy
-
-	// stop ends the goroutine Start runs, and done is closed when it returns.
-	stop context.CancelFunc
-	done chan struct{}
-}
-
-// New returns a Blocklists with no copies. Start loads them.
-func New(params Params) *Blocklists {
-	return &Blocklists{
-		dir:    params.Dir,
-		lists:  params.Lists,
-		logger: params.Logger.With("component", "blocklists"),
-		copies: make(map[string]listCopy),
-		done:   make(chan struct{}),
-	}
-}
-
-// Lookup returns a Listing for each list whose copy contains addr, in the
-// order of the lists given to New. It makes no outside request. When no list
-// contains addr, it returns an empty list, not nil.
-func (b *Blocklists) Lookup(addr netip.Addr) []Listing {
-	b.mu.RLock()
-	defer b.mu.RUnlock()
-
-	listings := []Listing{}
-
-	for _, list := range b.lists {
-		c, ok := b.copies[list.Name]
-		if ok && c.addresses.Contains(addr.Unmap()) {
-			listings = append(listings, Listing{
-				Name:        list.Name,
-				Description: list.Description,
-				Updated:     c.updated,
-			})
-		}
-	}
-
-	return listings
-}
-
-// ShortestInterval returns the shortest Interval of the lists. That is the
-// soonest a copy can change, and so how long a lookup answer stays current.
-func (b *Blocklists) ShortestInterval() time.Duration {
-	var shortest time.Duration
-
-	for i, list := range b.lists {
-		if i == 0 || list.Interval < shortest {
-			shortest = list.Interval
-		}
-	}
-
-	return shortest
-}
-
-// updated returns when the named list's copy was last downloaded or found
-// current, or the zero time when the list has no copy.
-func (b *Blocklists) updated(name string) time.Time {
-	b.mu.RLock()
-	defer b.mu.RUnlock()
-
-	return b.copies[name].updated
-}
-
-// copyValidators returns the validators of the named list's copy. They are
-// empty when the list has no copy.
-func (b *Blocklists) copyValidators(name string) validators {
-	b.mu.RLock()
-	defer b.mu.RUnlock()
-
-	return b.copies[name].validators
-}
-
-// setCopy makes c the named list's copy.
-func (b *Blocklists) setCopy(name string, c listCopy) {
-	b.mu.Lock()
-	defer b.mu.Unlock()
-
-	b.copies[name] = c
-}
-
-// setUpdated records that the named list's copy was found current at t.
-func (b *Blocklists) setUpdated(name string, t time.Time) {
-	b.mu.Lock()
-	defer b.mu.Unlock()
-
-	c, ok := b.copies[name]
-	if ok {
-		c.updated = t
-		b.copies[name] = c
-	}
-}

internal/blocklist/blocklist_internal_test.go

@@ -1,494 +0,0 @@
-package blocklist
-
-import (
-	"context"
-	"errors"
-	"io"
-	"io/fs"
-	"log/slog"
-	"net/http"
-	"net/http/httptest"
-	"net/netip"
-	"os"
-	"path/filepath"
-	"strings"
-	"sync"
-	"testing"
-	"time"
-)
-
-// The test list's name and description, the address its good version lists,
-// and the address a newer version lists instead.
-const (
-	testListName        = "test_list"
-	testListDescription = "On the test list"
-	listedAddr          = "192.0.2.1"
-	goodListBody        = "# A test list\n" + listedAddr + "\n"
-	newListedAddr       = "192.0.2.2"
-	newListBody         = "# A test list\n" + newListedAddr + "\n"
-)
-
-// testETag is the ETag a publisher sends with a version of the test list.
-const testETag = `"v1"`
-
-// maxWait is how long a test waits for the download goroutine.
-const maxWait = 10 * time.Second
-
-// publisher stands in for a list's publisher. It answers every request with
-// its status and body, and keeps the headers of each request. A 200 answer
-// carries lastModified and etag, when set, as Last-Modified and ETag. As
-// from a real publisher, a request whose If-None-Match or If-Modified-Since
-// matches them is answered "not modified".
-type publisher struct {
-	mu           sync.Mutex
-	status       int
-	body         string
-	lastModified time.Time
-	etag         string
-	headers      []http.Header
-}
-
-// newPublisher starts a publisher that answers 200 with body, and returns it
-// and its URL.
-func newPublisher(t *testing.T, body string) (*publisher, string) {
-	t.Helper()
-
-	p := &publisher{status: http.StatusOK, body: body}
-	srv := httptest.NewServer(p)
-	t.Cleanup(srv.Close)
-
-	return p, srv.URL
-}
-
-func (p *publisher) ServeHTTP(w http.ResponseWriter, r *http.Request) {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-
-	p.headers = append(p.headers, r.Header.Clone())
-
-	if p.status != http.StatusOK {
-		w.WriteHeader(p.status)
-		_, _ = io.WriteString(w, p.body)
-
-		return
-	}
-
-	if p.etag != "" {
-		w.Header().Set(etagHeader, p.etag)
-	}
-
-	// ServeContent sends Last-Modified unless lastModified is zero, and
-	// answers "not modified" when the request's conditions match.
-	http.ServeContent(w, r, testListName, p.lastModified,
-		strings.NewReader(p.body))
-}
-
-// answer makes the publisher answer later requests with status and body.
-func (p *publisher) answer(status int, body string) {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-
-	p.status, p.body = status, body
-}
-
-// publish makes the publisher answer later requests with a version of the
-// list: body, with lastModified and etag. A zero lastModified or an empty
-// etag is not sent.
-func (p *publisher) publish(body string, lastModified time.Time, etag string) {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-
-	p.status, p.body = http.StatusOK, body
-	p.lastModified, p.etag = lastModified, etag
-}
-
-// requests returns the headers of the requests the publisher has had.
-func (p *publisher) requests() []http.Header {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-
-	return append([]http.Header(nil), p.headers...)
-}
-
-// testList returns a list served at url, written in format.
-func testList(url string, format Format) List {
-	return List{
-		Name:        testListName,
-		Description: testListDescription,
-		URL:         url,
-		Interval:    time.Hour,
-		Format:      format,
-	}
-}
-
-func newTestBlocklists(dir string, list List) *Blocklists {
-	return New(Params{
-		Dir:    dir,
-		Lists:  []List{list},
-		Logger: slog.New(slog.DiscardHandler),
-	})
-}
-
-// lookup looks addr up in b.
-func lookup(t *testing.T, b *Blocklists, addr string) []Listing {
-	t.Helper()
-
-	return b.Lookup(netip.MustParseAddr(addr))
-}
-
-// TestLookupFindsListedAddresses downloads a list in each format and looks
-// up addresses on it, inside its networks, and off it, in IPv4 and IPv6.
-// A listed address has one listing that names the list; any other has an
-// empty list of listings.
-func TestLookupFindsListedAddresses(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		name     string
-		format   Format
-		body     string
-		listed   []string
-		unlisted []string
-	}{
-		{
-			name:     "single addresses",
-			format:   FormatLines,
-			body:     "# comment\n192.0.2.1\n\n2001:db8::1\n",
-			listed:   []string{"192.0.2.1", "::ffff:192.0.2.1", "2001:db8::1"},
-			unlisted: []string{"192.0.2.2", "2001:db8::2"},
-		},
-		{
-			name:     "networks",
-			format:   FormatLines,
-			body:     "# comment\n198.51.100.0/24\n2001:db8:1::/48\n",
-			listed:   []string{"198.51.100.0", "198.51.100.255", "2001:db8:1::5"},
-			unlisted: []string{"198.51.101.0", "2001:db8:2::1"},
-		},
-		{
-			name:   "Spamhaus networks",
-			format: FormatSpamhausJSON,
-			body: `{"cidr":"203.0.113.0/25","sblid":"SBL1","rir":"arin"}` + "\n" +
-				`{"cidr":"2001:db8:3::/48","sblid":"SBL2","rir":"ripencc"}` + "\n" +
-				`{"type":"metadata","timestamp":1793074442,"records":2}` + "\n",
-			listed:   []string{"203.0.113.1", "2001:db8:3::1"},
-			unlisted: []string{"203.0.113.200", "2001:db8:4::1"},
-		},
-		{
-			name:   "DShield networks",
-			format: FormatDShield,
-			body: "#   DShield.org Recommended Block List\n" +
-				"192.0.2.0\t192.0.2.255\t24\t327\tEXAMPLE\tUS\tNone\n",
-			listed:   []string{"192.0.2.0", "192.0.2.77"},
-			unlisted: []string{"192.0.3.1", "2001:db8::1"},
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			_, url := newPublisher(t, tt.body)
-			b := newTestBlocklists(t.TempDir(), testList(url, tt.format))
-
-			err := b.download(t.Context(), b.lists[0])
-			if err != nil {
-				t.Fatalf("download: %v", err)
-			}
-
-			for _, addr := range tt.listed {
-				listings := lookup(t, b, addr)
-				if len(listings) != 1 || listings[0].Name != testListName ||
-					listings[0].Description != testListDescription ||
-					listings[0].Updated.IsZero() {
-					t.Errorf("%s: listings %+v, want one for %s", addr,
-						listings, testListName)
-				}
-			}
-
-			for _, addr := range tt.unlisted {
-				listings := lookup(t, b, addr)
-				if listings == nil || len(listings) != 0 {
-					t.Errorf("%s: listings %#v, want an empty list", addr, listings)
-				}
-			}
-		})
-	}
-}
-
-// TestFailedDownloadKeepsThePreviousCopy downloads a good version of a list,
-// then one that fails. The good copy stays in memory and on disk, and no
-// part of the failed download is left beside it.
-func TestFailedDownloadKeepsThePreviousCopy(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		name   string
-		status int
-		body   string
-	}{
-		{"server error", http.StatusInternalServerError, "unavailable"},
-		{"not a list", http.StatusOK, "<!DOCTYPE html>\n<html></html>\n"},
-		{
-			"over the size limit", http.StatusOK,
-			strings.Repeat(listedAddr+"\n", maxDownloadSize/len(listedAddr)),
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			dir := t.TempDir()
-			p, url := newPublisher(t, goodListBody)
-			b := newTestBlocklists(dir, testList(url, FormatLines))
-
-			err := b.download(t.Context(), b.lists[0])
-			if err != nil {
-				t.Fatalf("first download: %v", err)
-			}
-
-			before := lookup(t, b, listedAddr)
-
-			p.answer(tt.status, tt.body)
-
-			err = b.download(t.Context(), b.lists[0])
-			if err == nil {
-				t.Fatal("second download returned no error")
-			}
-
-			after := lookup(t, b, listedAddr)
-			if len(after) != 1 || len(before) != 1 ||
-				!after[0].Updated.Equal(before[0].Updated) {
-				t.Errorf("listings %+v after the failure, want %+v", after, before)
-			}
-
-			copies := os.DirFS(dir)
-
-			onDisk, err := fs.ReadFile(copies, testListName)
-			if err != nil || string(onDisk) != goodListBody {
-				t.Errorf("copy on disk %q (%v), want %q", onDisk, err, goodListBody)
-			}
-
-			_, err = fs.Stat(copies, testListName+downloadSuffix)
-			if !errors.Is(err, fs.ErrNotExist) {
-				t.Errorf("the failed download's file is still there (%v)", err)
-			}
-		})
-	}
-}
-
-// TestRestartLoadsCopiesFromDisk downloads a list, then starts a new
-// Blocklists on the same directory, as a restart does. The new one answers
-// from the copy on disk at once, and does not download the list again while
-// the copy is younger than the list's interval.
-func TestRestartLoadsCopiesFromDisk(t *testing.T) {
-	t.Parallel()
-
-	dir := t.TempDir()
-	p, url := newPublisher(t, goodListBody)
-	list := testList(url, FormatLines)
-
-	first := newTestBlocklists(dir, list)
-
-	err := first.download(t.Context(), list)
-	if err != nil {
-		t.Fatalf("download: %v", err)
-	}
-
-	downloaded := lookup(t, first, listedAddr)[0].Updated
-
-	second := newTestBlocklists(dir, list)
-	second.Start(t.Context())
-	second.Stop(t.Context())
-
-	listings := lookup(t, second, listedAddr)
-	if len(listings) != 1 ||
-		listings[0].Updated.Sub(downloaded).Abs() > time.Second {
-		t.Errorf("after the restart, listings %+v, want one updated at %s",
-			listings, downloaded)
-	}
-
-	if got := len(p.requests()); got != 1 {
-		t.Errorf("the publisher had %d requests, want 1", got)
-	}
-}
-
-// TestStartRefreshesAnOldCopy starts on a copy older than its list's
-// interval. The list is downloaded at once, with routewatch's User-Agent and
-// the Last-Modified and ETag the publisher sent with the copy, kept on disk.
-// The publisher answers that the copy is still current, so the copy is kept
-// and marked as updated now.
-func TestStartRefreshesAnOldCopy(t *testing.T) {
-	t.Parallel()
-
-	dir := t.TempDir()
-	p, url := newPublisher(t, goodListBody)
-	list := testList(url, FormatLines)
-
-	lastModified := time.Now().Add(-3 * list.Interval).UTC().Truncate(time.Second)
-	p.publish(goodListBody, lastModified, testETag)
-
-	err := newTestBlocklists(dir, list).download(t.Context(), list)
-	if err != nil {
-		t.Fatalf("download: %v", err)
-	}
-
-	old := time.Now().Add(-2 * list.Interval).UTC().Truncate(time.Second)
-
-	err = os.Chtimes(filepath.Join(dir, testListName), old, old)
-	if err != nil {
-		t.Fatalf("Chtimes: %v", err)
-	}
-
-	b := newTestBlocklists(dir, list)
-	b.Start(t.Context())
-
-	deadline := time.Now().Add(maxWait)
-	for !lookup(t, b, listedAddr)[0].Updated.After(old) {
-		if time.Now().After(deadline) {
-			t.Fatalf("the copy was not marked updated within %s", maxWait)
-		}
-
-		time.Sleep(time.Millisecond)
-	}
-
-	b.Stop(t.Context())
-
-	requests := p.requests()
-	last := requests[len(requests)-1]
-
-	want := lastModified.Format(http.TimeFormat)
-	if got := last.Get(ifModifiedSinceHeader); got != want {
-		t.Errorf("If-Modified-Since %q, want %q", got, want)
-	}
-
-	if got := last.Get(ifNoneMatchHeader); got != testETag {
-		t.Errorf("If-None-Match %q, want %q", got, testETag)
-	}
-
-	if got := last.Get(userAgentHeader); got != userAgent {
-		t.Errorf("User-Agent %q, want %q", got, userAgent)
-	}
-}
-
-// TestNewVersionOlderThanTheLastDownloadReplacesTheCopy downloads a list.
-// The publisher then puts out a new version whose Last-Modified is earlier
-// than that download, as when a cache in front of the publisher passes the
-// version on late. The next download sends the Last-Modified of the copy,
-// not the time of the download, so it gets the new version.
-func TestNewVersionOlderThanTheLastDownloadReplacesTheCopy(t *testing.T) {
-	t.Parallel()
-
-	p, url := newPublisher(t, goodListBody)
-	list := testList(url, FormatLines)
-	b := newTestBlocklists(t.TempDir(), list)
-
-	firstVersion := time.Now().Add(-2 * list.Interval).UTC().Truncate(time.Second)
-	p.publish(goodListBody, firstVersion, "")
-
-	err := b.download(t.Context(), list)
-	if err != nil {
-		t.Fatalf("first download: %v", err)
-	}
-
-	p.publish(newListBody, firstVersion.Add(list.Interval), "")
-
-	err = b.download(t.Context(), list)
-	if err != nil {
-		t.Fatalf("second download: %v", err)
-	}
-
-	if listings := lookup(t, b, newListedAddr); len(listings) != 1 {
-		t.Errorf("%s: listings %+v, want one from the new version",
-			newListedAddr, listings)
-	}
-
-	want := firstVersion.Format(http.TimeFormat)
-	if got := p.requests()[1].Get(ifModifiedSinceHeader); got != want {
-		t.Errorf("If-Modified-Since %q, want %q", got, want)
-	}
-}
-
-// TestDownloadWithoutLastModifiedOrETagSendsNoCondition downloads a list
-// twice from a publisher that sends neither Last-Modified nor ETag. Neither
-// request carries If-Modified-Since or If-None-Match, so the second download
-// gets the whole list again.
-func TestDownloadWithoutLastModifiedOrETagSendsNoCondition(t *testing.T) {
-	t.Parallel()
-
-	p, url := newPublisher(t, goodListBody)
-	list := testList(url, FormatLines)
-	b := newTestBlocklists(t.TempDir(), list)
-
-	err := b.download(t.Context(), list)
-	if err != nil {
-		t.Fatalf("first download: %v", err)
-	}
-
-	err = b.download(t.Context(), list)
-	if err != nil {
-		t.Fatalf("second download: %v", err)
-	}
-
-	for i, request := range p.requests() {
-		if request.Get(ifModifiedSinceHeader) != "" ||
-			request.Get(ifNoneMatchHeader) != "" {
-			t.Errorf("request %d has a condition: %v", i, request)
-		}
-	}
-}
-
-// TestStopCancelsADownloadInProgress stops while the publisher has not
-// answered. Stop cancels the download and returns once the download
-// goroutine has stopped.
-func TestStopCancelsADownloadInProgress(t *testing.T) {
-	t.Parallel()
-
-	arrived := make(chan struct{})
-
-	var once sync.Once
-
-	srv := httptest.NewServer(http.HandlerFunc(
-		func(_ http.ResponseWriter, r *http.Request) {
-			once.Do(func() { close(arrived) })
-			<-r.Context().Done()
-		}))
-	t.Cleanup(srv.Close)
-
-	b := newTestBlocklists(t.TempDir(), testList(srv.URL, FormatLines))
-	b.Start(t.Context())
-
-	select {
-	case <-arrived:
-	case <-time.After(maxWait):
-		t.Fatalf("no download began within %s", maxWait)
-	}
-
-	ctx, cancel := context.WithTimeout(t.Context(), maxWait)
-	defer cancel()
-
-	b.Stop(ctx)
-
-	select {
-	case <-b.done:
-	default:
-		t.Error("Stop returned before the download goroutine stopped")
-	}
-}
-
-// TestPublicListsAreComplete checks that every public list has what a
-// download and a lookup answer need, and a unique name.
-func TestPublicListsAreComplete(t *testing.T) {
-	t.Parallel()
-
-	names := make(map[string]bool)
-
-	for _, list := range Lists() {
-		if list.Name == "" || list.Description == "" || list.URL == "" ||
-			list.Interval <= 0 || names[list.Name] {
-			t.Errorf("list %+v is incomplete or has a name used before", list)
-		}
-
-		names[list.Name] = true
-	}
-}

internal/blocklist/download.go

@@ -1,379 +0,0 @@
-package blocklist
-
-import (
-	"context"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"io/fs"
-	"net/http"
-	"os"
-	"slices"
-	"time"
-
-	"go4.org/netipx"
-)
-
-// Limits of one download.
-const (
-	// downloadTimeout is how long one download may take.
-	downloadTimeout = time.Minute
-
-	// maxDownloadSize is the size of the largest list file accepted, in
-	// bytes. The largest list, FireHOL abusers, is about 2.5 MiB.
-	maxDownloadSize = 16 << 20
-)
-
-// The request headers a download sets.
-const (
-	userAgentHeader       = "User-Agent"
-	ifModifiedSinceHeader = "If-Modified-Since"
-	ifNoneMatchHeader     = "If-None-Match"
-
-	// userAgent names routewatch to the publishers.
-	userAgent = "routewatch (+https://git.eeqj.de/sneak/routewatch)"
-)
-
-// The response headers that identify the version of a list a publisher sent.
-const (
-	lastModifiedHeader = "Last-Modified"
-	etagHeader         = "ETag"
-)
-
-const (
-	// downloadSuffix ends the name of the file a download is written to,
-	// beside the list's copy, until it has parsed.
-	downloadSuffix = ".download"
-
-	// validatorsSuffix ends the name of the file beside a list's copy that
-	// keeps the copy's validators.
-	validatorsSuffix = ".validators"
-
-	// dirPermissions are those of the directory that holds the copies.
-	dirPermissions = 0o750
-
-	// validatorsPermissions are those of the file that keeps a copy's
-	// validators.
-	validatorsPermissions = 0o600
-)
-
-// validators are the Last-Modified and ETag headers a publisher sent with a
-// version of a list. A download sends them back, as If-Modified-Since and
-// If-None-Match, so that the publisher sends the list only when it has
-// changed. They are kept as the publisher wrote them.
-//
-//nolint:tagliatelle // snake_case keys, as the Go style guide asks
-type validators struct {
-	LastModified string `json:"last_modified,omitempty"`
-	ETag         string `json:"etag,omitempty"`
-}
-
-var (
-	errUnexpectedStatus = errors.New("unexpected HTTP status")
-	errTooLarge         = errors.New("list is over the size limit")
-)
-
-// Start loads each list's copy from disk. It then starts the goroutine that
-// downloads each list once its copy is older than its Interval, and a list
-// with no copy at once. The goroutine runs until Stop: the end of ctx does
-// not stop it.
-func (b *Blocklists) Start(ctx context.Context) {
-	err := os.MkdirAll(b.dir, dirPermissions)
-	if err != nil {
-		b.logger.Error("Failed to create the blocklists directory",
-			"dir", b.dir, "error", err)
-	}
-
-	nextDownload := make([]time.Time, len(b.lists))
-
-	for i, list := range b.lists {
-		loadErr := b.loadCopy(list)
-		if loadErr != nil && !errors.Is(loadErr, fs.ErrNotExist) {
-			b.logger.Error("Failed to load a blocklist copy",
-				"list", list.Name, "error", loadErr)
-		}
-
-		// A list with no copy has the zero time, so it is due at once.
-		nextDownload[i] = b.updated(list.Name).Add(list.Interval)
-	}
-
-	ctx, b.stop = context.WithCancel(context.WithoutCancel(ctx))
-
-	go func() {
-		b.run(ctx, nextDownload)
-	}()
-
-	b.logger.Info("Blocklist downloads started", "dir", b.dir)
-}
-
-// Stop stops the downloads, cancelling one in progress, and waits until the
-// goroutine has returned or ctx ends. The copies in memory stay, so Lookup
-// still answers.
-func (b *Blocklists) Stop(ctx context.Context) {
-	b.stop()
-
-	select {
-	case <-b.done:
-		b.logger.Info("Blocklist downloads stopped")
-	case <-ctx.Done():
-		b.logger.Warn("Stop deadline passed before the blocklist downloads " +
-			"stopped")
-	}
-}
-
-// run downloads each list when nextDownload says, until ctx ends. After each
-// try, whether it worked or not, the list's next download is an Interval
-// later.
-func (b *Blocklists) run(ctx context.Context, nextDownload []time.Time) {
-	defer close(b.done)
-
-	if len(nextDownload) == 0 {
-		return
-	}
-
-	for {
-		soonest := slices.MinFunc(nextDownload, time.Time.Compare)
-		timer := time.NewTimer(time.Until(soonest))
-
-		select {
-		case <-ctx.Done():
-			timer.Stop()
-
-			return
-		case <-timer.C:
-		}
-
-		for i, list := range b.lists {
-			if time.Now().Before(nextDownload[i]) {
-				continue
-			}
-
-			err := b.download(ctx, list)
-			if ctx.Err() != nil {
-				return
-			}
-
-			if err != nil {
-				b.logger.Error("Blocklist download failed; keeping the "+
-					"previous copy", "list", list.Name, "error", err)
-			}
-
-			nextDownload[i] = time.Now().Add(list.Interval)
-		}
-	}
-}
-
-// download asks the publisher for list, sending the validators of the list's
-// copy when it has them. A new version that parses becomes the list's copy,
-// on disk and in memory. An answer that the copy is still current marks the
-// copy as updated now. Anything else leaves the copy as it is and returns an
-// error.
-func (b *Blocklists) download(ctx context.Context, list List) error {
-	ctx, cancel := context.WithTimeout(ctx, downloadTimeout)
-	defer cancel()
-
-	req, err := http.NewRequestWithContext(ctx, http.MethodGet, list.URL, nil)
-	if err != nil {
-		return err
-	}
-
-	req.Header.Set(userAgentHeader, userAgent)
-
-	held := b.copyValidators(list.Name)
-	if held.LastModified != "" {
-		req.Header.Set(ifModifiedSinceHeader, held.LastModified)
-	}
-
-	if held.ETag != "" {
-		req.Header.Set(ifNoneMatchHeader, held.ETag)
-	}
-
-	resp, err := http.DefaultClient.Do(req)
-	if err != nil {
-		return err
-	}
-
-	defer func() { _ = resp.Body.Close() }()
-
-	switch resp.StatusCode {
-	case http.StatusOK:
-		return b.replaceCopy(list, resp.Body, validators{
-			LastModified: resp.Header.Get(lastModifiedHeader),
-			ETag:         resp.Header.Get(etagHeader),
-		})
-	case http.StatusNotModified:
-		return b.markCurrent(list)
-	default:
-		return fmt.Errorf("%w: %s", errUnexpectedStatus, resp.Status)
-	}
-}
-
-// replaceCopy writes a new version of list, read from body, beside the list's
-// copy. Only when it parses does it replace the copy, on disk and in memory.
-// sent are the validators the publisher sent with the new version.
-func (b *Blocklists) replaceCopy(list List, body io.Reader, sent validators) error {
-	root, err := os.OpenRoot(b.dir)
-	if err != nil {
-		return err
-	}
-
-	defer func() { _ = root.Close() }()
-
-	downloadName := list.Name + downloadSuffix
-
-	addresses, err := writeAndParse(root, downloadName, body, list.Format)
-	if err != nil {
-		_ = root.Remove(downloadName)
-
-		return err
-	}
-
-	// The old copy's validators are removed before the copy is replaced, so
-	// that they are never kept with the new copy.
-	err = root.Remove(list.Name + validatorsSuffix)
-	if err != nil && !errors.Is(err, fs.ErrNotExist) {
-		return err
-	}
-
-	err = root.Rename(downloadName, list.Name)
-	if err != nil {
-		return err
-	}
-
-	b.setCopy(list.Name, listCopy{
-		addresses: addresses, updated: time.Now().UTC(), validators: sent,
-	})
-	b.logger.Info("Downloaded a new version of a blocklist", "list", list.Name)
-
-	// Without this file, the first download after a restart asks for the
-	// whole list.
-	err = writeValidators(root, list.Name, sent)
-	if err != nil {
-		b.logger.Error("Failed to keep the validators of a blocklist copy",
-			"list", list.Name, "error", err)
-	}
-
-	return nil
-}
-
-// writeAndParse writes body to the file name in root, up to maxDownloadSize
-// bytes, and parses the file as format.
-func writeAndParse(
-	root *os.Root, name string, body io.Reader, format Format,
-) (*netipx.IPSet, error) {
-	f, err := root.Create(name)
-	if err != nil {
-		return nil, err
-	}
-
-	defer func() { _ = f.Close() }()
-
-	written, err := io.Copy(f, io.LimitReader(body, maxDownloadSize+1))
-	if err != nil {
-		return nil, err
-	}
-
-	if written > maxDownloadSize {
-		return nil, errTooLarge
-	}
-
-	_, err = f.Seek(0, io.SeekStart)
-	if err != nil {
-		return nil, err
-	}
-
-	return parse(f, format)
-}
-
-// markCurrent records that the publisher says the list's copy is still
-// current: the copy, on disk and in memory, is marked as updated now.
-func (b *Blocklists) markCurrent(list List) error {
-	root, err := os.OpenRoot(b.dir)
-	if err != nil {
-		return err
-	}
-
-	defer func() { _ = root.Close() }()
-
-	now := time.Now().UTC()
-
-	err = root.Chtimes(list.Name, now, now)
-	if err != nil {
-		return err
-	}
-
-	b.setUpdated(list.Name, now)
-	b.logger.Debug("Blocklist copy is still current", "list", list.Name)
-
-	return nil
-}
-
-// loadCopy makes the list's copy on disk, with the validators kept beside it,
-// its copy in memory. The file's modification time is when the copy was
-// updated.
-func (b *Blocklists) loadCopy(list List) error {
-	root, err := os.OpenRoot(b.dir)
-	if err != nil {
-		return err
-	}
-
-	defer func() { _ = root.Close() }()
-
-	f, err := root.Open(list.Name)
-	if err != nil {
-		return err
-	}
-
-	defer func() { _ = f.Close() }()
-
-	info, err := f.Stat()
-	if err != nil {
-		return err
-	}
-
-	addresses, err := parse(f, list.Format)
-	if err != nil {
-		return err
-	}
-
-	held, err := readValidators(root, list.Name)
-	if err != nil {
-		return err
-	}
-
-	b.setCopy(list.Name, listCopy{
-		addresses: addresses, updated: info.ModTime().UTC(), validators: held,
-	})
-
-	return nil
-}
-
-// writeValidators keeps held in the file beside the named list's copy.
-func writeValidators(root *os.Root, name string, held validators) error {
-	data, err := json.Marshal(held)
-	if err != nil {
-		return err
-	}
-
-	return root.WriteFile(name+validatorsSuffix, data, validatorsPermissions)
-}
-
-// readValidators returns the validators kept beside the named list's copy.
-// They are empty when there is no such file.
-func readValidators(root *os.Root, name string) (validators, error) {
-	var held validators
-
-	data, err := root.ReadFile(name + validatorsSuffix)
-	if errors.Is(err, fs.ErrNotExist) {
-		return held, nil
-	}
-
-	if err != nil {
-		return held, err
-	}
-
-	err = json.Unmarshal(data, &held)
-
-	return held, err
-}

internal/blocklist/lists.go

@@ -1,93 +0,0 @@
-package blocklist
-
-import "time"
-
-// How often each list is downloaded. No list is downloaded more often than
-// its publisher updates it or allows.
-const (
-	// DShield asks that its feeds be downloaded at most once an hour:
-	// https://isc.sans.edu/feeds_doc.html
-	dshieldInterval = time.Hour
-
-	// abuse.ch makes the list every 5 minutes and recommends a download at
-	// least every 15: https://feodotracker.abuse.ch/blocklist/
-	feodoTrackerInterval = 15 * time.Minute
-
-	// FireHOL's page for each list gives how often the list changes on
-	// average: about every 5 hours for abusers and every 3 hours for level
-	// 3. FireHOL sets no download limit.
-	// https://iplists.firehol.org/?ipset=firehol_abusers_30d
-	// https://iplists.firehol.org/?ipset=firehol_level3
-	fireHOLAbusersInterval = 5 * time.Hour
-	fireHOLLevel3Interval  = 3 * time.Hour
-
-	// Spamhaus asks that DROP be downloaded at most once a day:
-	// https://www.spamhaus.org/blocklists/do-not-route-or-peer/
-	spamhausDROPInterval = 24 * time.Hour
-
-	// Tor publishes a new exit list every hour and sets no download limit:
-	// https://collector.torproject.org/recent/exit-lists/
-	torExitsInterval = time.Hour
-)
-
-// Lists returns the public blocklists routewatch downloads. Each can be
-// downloaded whole without an account or a key. Spamhaus merged its EDROP
-// list into DROP, so DROP covers both.
-func Lists() []List {
-	return []List{
-		{
-			Name: "dshield",
-			Description: "In one of the 20 networks that DShield saw scan " +
-				"the most targets in the last three days",
-			URL:      "https://feeds.dshield.org/block.txt",
-			Interval: dshieldInterval,
-			Format:   FormatDShield,
-		},
-		{
-			Name:        "feodo_tracker",
-			Description: "A botnet command-and-control server",
-			URL:         "https://feodotracker.abuse.ch/downloads/ipblocklist.txt",
-			Interval:    feodoTrackerInterval,
-			Format:      FormatLines,
-		},
-		{
-			Name: "firehol_abusers_30d",
-			Description: "Reported for comment, form or forum spam in the " +
-				"last 30 days",
-			URL:      "https://iplists.firehol.org/files/firehol_abusers_30d.netset",
-			Interval: fireHOLAbusersInterval,
-			Format:   FormatLines,
-		},
-		{
-			Name: "firehol_level3",
-			Description: "Reported for attacks, spyware or viruses in the " +
-				"last 30 days",
-			URL:      "https://iplists.firehol.org/files/firehol_level3.netset",
-			Interval: fireHOLLevel3Interval,
-			Format:   FormatLines,
-		},
-		{
-			Name: "spamhaus_drop",
-			Description: "In an IPv4 network that Spamhaus says spammers " +
-				"or cyber criminals have hijacked or lease",
-			URL:      "https://www.spamhaus.org/drop/drop_v4.json",
-			Interval: spamhausDROPInterval,
-			Format:   FormatSpamhausJSON,
-		},
-		{
-			Name: "spamhaus_dropv6",
-			Description: "In an IPv6 network that Spamhaus says spammers " +
-				"or cyber criminals have hijacked or lease",
-			URL:      "https://www.spamhaus.org/drop/drop_v6.json",
-			Interval: spamhausDROPInterval,
-			Format:   FormatSpamhausJSON,
-		},
-		{
-			Name:        "tor_exits",
-			Description: "A Tor exit node, where traffic from the Tor network leaves",
-			URL:         "https://check.torproject.org/torbulkexitlist",
-			Interval:    torExitsInterval,
-			Format:      FormatLines,
-		},
-	}
-}

internal/blocklist/parse.go

@@ -1,158 +0,0 @@
-package blocklist
-
-import (
-	"bufio"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"net/netip"
-	"strings"
-
-	"go4.org/netipx"
-)
-
-const (
-	// commentPrefix starts a comment line in a list file.
-	commentPrefix = "#"
-
-	// networkSeparator separates the address and the mask length of a
-	// network, as in 192.0.2.0/24.
-	networkSeparator = "/"
-
-	// dshieldFieldSeparator separates the fields of a DShield line.
-	dshieldFieldSeparator = "\t"
-
-	// spamhausMetadataType is the type of the line that ends a Spamhaus
-	// file. That line holds the file's metadata and no network.
-	spamhausMetadataType = "metadata"
-)
-
-var (
-	errUnknownFormat = errors.New("unknown list format")
-	errNoNetwork     = errors.New("no network on the line")
-	errBadRange      = errors.New("first and last address make no range")
-)
-
-// parse reads a list file written in format and returns the addresses it
-// holds. A line it cannot read fails the whole file, so a file that is not
-// the list, such as an error page, never replaces a good copy.
-func parse(r io.Reader, format Format) (*netipx.IPSet, error) {
-	var builder netipx.IPSetBuilder
-
-	scanner := bufio.NewScanner(r)
-
-	for lineNumber := 1; scanner.Scan(); lineNumber++ {
-		line := strings.TrimSpace(scanner.Text())
-		if line == "" || strings.HasPrefix(line, commentPrefix) {
-			continue
-		}
-
-		err := addLine(&builder, line, format)
-		if err != nil {
-			return nil, fmt.Errorf("line %d: %w", lineNumber, err)
-		}
-	}
-
-	err := scanner.Err()
-	if err != nil {
-		return nil, err
-	}
-
-	return builder.IPSet()
-}
-
-// addLine adds the address or network on one line of a list file to builder.
-func addLine(builder *netipx.IPSetBuilder, line string, format Format) error {
-	switch format {
-	case FormatLines:
-		return addAddressOrNetwork(builder, line)
-	case FormatSpamhausJSON:
-		return addSpamhausNetwork(builder, line)
-	case FormatDShield:
-		return addDShieldNetwork(builder, line)
-	default:
-		return errUnknownFormat
-	}
-}
-
-// addAddressOrNetwork adds an address, such as 192.0.2.1, or a network, such
-// as 192.0.2.0/24, to builder.
-func addAddressOrNetwork(builder *netipx.IPSetBuilder, text string) error {
-	if strings.Contains(text, networkSeparator) {
-		network, err := netip.ParsePrefix(text)
-		if err != nil {
-			return err
-		}
-
-		builder.AddPrefix(network)
-
-		return nil
-	}
-
-	addr, err := netip.ParseAddr(text)
-	if err != nil {
-		return err
-	}
-
-	builder.Add(addr)
-
-	return nil
-}
-
-// addSpamhausNetwork adds the network on a line of a Spamhaus file to
-// builder. The metadata line adds nothing.
-func addSpamhausNetwork(builder *netipx.IPSetBuilder, line string) error {
-	var entry struct {
-		CIDR string `json:"cidr"`
-		Type string `json:"type"`
-	}
-
-	err := json.Unmarshal([]byte(line), &entry)
-	if err != nil {
-		return err
-	}
-
-	if entry.Type == spamhausMetadataType {
-		return nil
-	}
-
-	network, err := netip.ParsePrefix(entry.CIDR)
-	if err != nil {
-		return err
-	}
-
-	builder.AddPrefix(network)
-
-	return nil
-}
-
-// addDShieldNetwork adds the network on a line of a DShield file to builder.
-// The line's first two fields are the network's first and last address.
-func addDShieldNetwork(builder *netipx.IPSetBuilder, line string) error {
-	firstField, rest, found := strings.Cut(line, dshieldFieldSeparator)
-	if !found {
-		return errNoNetwork
-	}
-
-	lastField, _, _ := strings.Cut(rest, dshieldFieldSeparator)
-
-	first, err := netip.ParseAddr(firstField)
-	if err != nil {
-		return err
-	}
-
-	last, err := netip.ParseAddr(lastField)
-	if err != nil {
-		return err
-	}
-
-	network := netipx.IPRangeFrom(first, last)
-	if !network.IsValid() {
-		return errBadRange
-	}
-
-	builder.AddRange(network)
-
-	return nil
-}

internal/config/config.go

@@ -4,12 +4,10 @@ package config
 import (
 	"errors"
 	"fmt"
-	"net/netip"
 	"os"
 	"path/filepath"
 	"runtime"
 	"strconv"
-	"strings"
 	"time"
 )
 
@@ -34,8 +32,6 @@ var (
 	errRelativeXDGDataHome = errors.New("XDG_DATA_HOME must be an absolute path")
 	errUnsupportedOS       = errors.New("unsupported operating system")
 	errInvalidPort         = errors.New("PORT must be a whole number")
-	errInvalidProxy        = errors.New(
-		"TRUSTED_PROXIES must list IP addresses and CIDR ranges, comma-separated")
 )
 
 // Config holds configuration for the entire application
@@ -46,10 +42,6 @@ type Config struct {
 	// Port is the TCP port the HTTP server listens on
 	Port int
 
-	// TrustedProxies are the reverse proxies whose X-Forwarded-For header
-	// the HTTP server believes
-	TrustedProxies []netip.Prefix
-
 	// MaxRuntime is the maximum runtime (0 = run forever)
 	MaxRuntime time.Duration
 
@@ -76,15 +68,9 @@ func New() (*Config, error) {
 		return nil, err
 	}
 
-	trustedProxies, err := getTrustedProxies()
-	if err != nil {
-		return nil, err
-	}
-
 	return &Config{
 		StateDir: stateDir,
 		Port:     port,
-		TrustedProxies: trustedProxies,
 		// Run forever by default
 		MaxRuntime: 0,
 		// Enable batching by default
@@ -159,38 +145,6 @@ func getPort() (int, error) {
 	return int(port), nil
 }
 
-// getTrustedProxies returns the addresses and CIDR ranges listed in
-// TRUSTED_PROXIES, such as "10.0.0.0/8,192.0.2.1", or none when it is not set.
-// An address is returned as a range holding only that address.
-func getTrustedProxies() ([]netip.Prefix, error) {
-	value := os.Getenv("TRUSTED_PROXIES")
-	if value == "" {
-		return nil, nil
-	}
-
-	var proxies []netip.Prefix
-
-	for entry := range strings.SplitSeq(value, ",") {
-		entry = strings.TrimSpace(entry)
-
-		addr, err := netip.ParseAddr(entry)
-		if err == nil {
-			proxies = append(proxies, netip.PrefixFrom(addr, addr.BitLen()))
-
-			continue
-		}
-
-		prefix, err := netip.ParsePrefix(entry)
-		if err != nil {
-			return nil, fmt.Errorf("%w, got %q", errInvalidProxy, entry)
-		}
-
-		proxies = append(proxies, prefix)
-	}
-
-	return proxies, nil
-}
-
 // EnsureDirectories creates all necessary directories if they don't exist
 func (c *Config) EnsureDirectories() error {
 	// Ensure state directory exists

internal/config/config_internal_test.go

@@ -1,9 +1,7 @@
 package config
 
 import (
-	"net/netip"
 	"runtime"
-	"slices"
 	"testing"
 )
 
@@ -46,55 +44,6 @@ func TestNewRefusesInvalidPort(t *testing.T) {
 	}
 }
 
-func TestNewReadsTrustedProxies(t *testing.T) {
-	t.Setenv("PORT", "")
-	t.Setenv("XDG_DATA_HOME", "")
-
-	t.Setenv("TRUSTED_PROXIES", "")
-
-	cfg, err := New()
-	if err != nil {
-		t.Fatalf("New() without TRUSTED_PROXIES: %v", err)
-	}
-
-	if len(cfg.TrustedProxies) != 0 {
-		t.Errorf("New() without TRUSTED_PROXIES: TrustedProxies = %v, want none",
-			cfg.TrustedProxies)
-	}
-
-	t.Setenv("TRUSTED_PROXIES", "10.0.0.0/8, 192.0.2.1,2001:db8::/32")
-
-	cfg, err = New()
-	if err != nil {
-		t.Fatalf("New() with TRUSTED_PROXIES: %v", err)
-	}
-
-	want := []netip.Prefix{
-		netip.MustParsePrefix("10.0.0.0/8"),
-		netip.MustParsePrefix("192.0.2.1/32"),
-		netip.MustParsePrefix("2001:db8::/32"),
-	}
-	if !slices.Equal(cfg.TrustedProxies, want) {
-		t.Errorf("TrustedProxies = %v, want %v", cfg.TrustedProxies, want)
-	}
-}
-
-func TestNewRefusesInvalidTrustedProxies(t *testing.T) {
-	invalid := []string{"proxy.example", "10.0.0.0/33", "192.0.2.1:80", "192.0.2.1,"}
-	for _, value := range invalid {
-		t.Run(value, func(t *testing.T) {
-			t.Setenv("PORT", "")
-			t.Setenv("XDG_DATA_HOME", "")
-			t.Setenv("TRUSTED_PROXIES", value)
-
-			_, err := New()
-			if err == nil {
-				t.Errorf("New() with TRUSTED_PROXIES=%q returned no error", value)
-			}
-		})
-	}
-}
-
 func TestNewRefusesRelativeXDGDataHome(t *testing.T) {
 	if runtime.GOOS == "darwin" {
 		t.Skip("macOS does not read XDG_DATA_HOME")

internal/database/counts_internal_test.go

@@ -2,7 +2,6 @@ package database
 
 import (
 	"context"
-	"fmt"
 	"slices"
 	"sync"
 	"testing"
@@ -148,9 +147,7 @@ func TestLiveCountsTrackWritesInRealtime(t *testing.T) {
 func writeASNsPrefixesAndPeers(t *testing.T, db *Database, ts time.Time) {
 	t.Helper()
 
-	seen := SeenTimes{First: ts, Last: ts}
+	err := db.GetOrCreateASNBatch(map[int]time.Time{64500: ts, 64501: ts})
-
-	err := db.GetOrCreateASNBatch(map[int]SeenTimes{64500: seen, 64501: seen})
 	if err != nil {
 		t.Fatalf("GetOrCreateASNBatch: %v", err)
 	}
@@ -166,14 +163,14 @@ func writeASNsPrefixesAndPeers(t *testing.T, db *Database, ts time.Time) {
 	err = db.UpdatePeerBatch(map[string]PeerUpdate{
 		testV4PeerIP: {
 			PeerIP: testV4PeerIP, PeerASN: 64500,
-			MessageType: "UPDATE", Seen: seen,
+			MessageType: "UPDATE", Timestamp: ts,
 		},
 	})
 	if err != nil {
 		t.Fatalf("UpdatePeerBatch: %v", err)
 	}
 
-	err = db.RecordPeering(64500, 64501, seen)
+	err = db.RecordPeering(64500, 64501, ts)
 	if err != nil {
 		t.Fatalf("RecordPeering: %v", err)
 	}
@@ -194,10 +191,7 @@ func TestLiveCountsSeededFromDatabaseAtStartup(t *testing.T) {
 
 	ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
 
-	seen := SeenTimes{First: ts, Last: ts}
+	err = db.GetOrCreateASNBatch(map[int]time.Time{64500: ts, 64501: ts, 64502: ts})
-
-	err = db.GetOrCreateASNBatch(
-		map[int]SeenTimes{64500: seen, 64501: seen, 64502: seen})
 	if err != nil {
 		t.Fatalf("GetOrCreateASNBatch: %v", err)
 	}
@@ -500,124 +494,6 @@ func TestDeleteLiveRoutesOfPeer(t *testing.T) {
 	checkOneLiveRoutePerIPVersion(t, db)
 }
 
-// TestDeleteLiveRoutesOfPeerDeletesInChunks gives a peer one IPv4 route more
-// than a chunk holds, two of them for prefixes another peer also has. One chunk
-// deletes peerRouteChunkSize of the peer's routes and commits, leaving the last
-// one. The routes are written again, and DeleteLiveRoutesOfPeer deletes all of
-// them, a chunk and then the last one. After each step the route count and
-// prefix distribution held in memory and those read from the route table are
-// the ones wanted.
-func TestDeleteLiveRoutesOfPeerDeletesInChunks(t *testing.T) {
-	t.Parallel()
-
-	const (
-		peerIP      = "192.0.2.1"
-		otherPeerIP = "192.0.2.2"
-		// How many of the peer's prefixes the other peer also has, the first
-		// ones in prefix order.
-		sharedPrefixes = 2
-	)
-
-	db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
-	if err != nil {
-		t.Fatalf("failed to create database: %v", err)
-	}
-	defer func() { _ = db.Close() }()
-
-	ts := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
-
-	peerRoutes := make([]*LiveRoute, 0, peerRouteChunkSize+1)
-	otherPeerRoutes := make([]*LiveRoute, 0, sharedPrefixes)
-
-	for i := range peerRouteChunkSize + 1 {
-		prefix := fmt.Sprintf("10.%d.%d.0/24", i/256, i%256)
-
-		route := mkV4Route(t, prefix, 64500, ts)
-		route.PeerIP = peerIP
-		peerRoutes = append(peerRoutes, route)
-
-		if i < sharedPrefixes {
-			route := mkV4Route(t, prefix, 64500, ts)
-			route.PeerIP = otherPeerIP
-			otherPeerRoutes = append(otherPeerRoutes, route)
-		}
-	}
-
-	err = db.UpsertLiveRouteBatch(slices.Concat(peerRoutes, otherPeerRoutes))
-	if err != nil {
-		t.Fatalf("UpsertLiveRouteBatch: %v", err)
-	}
-
-	deletedV4, deletedV6, err := db.deletePeerRouteChunk(peerIP)
-	if err != nil {
-		t.Fatalf("deletePeerRouteChunk: %v", err)
-	}
-
-	if deletedV4 != peerRouteChunkSize || deletedV6 != 0 {
-		t.Errorf("deletePeerRouteChunk deleted (%d, %d) routes, want (%d, 0)",
-			deletedV4, deletedV6, peerRouteChunkSize)
-	}
-
-	// The shared prefixes went with the chunk, so each route left has a
-	// prefix of its own.
-	checkV4RoutesOfOwnPrefixes(t, db, "after one chunk", 1+sharedPrefixes)
-
-	err = db.UpsertLiveRouteBatch(peerRoutes)
-	if err != nil {
-		t.Fatalf("UpsertLiveRouteBatch: %v", err)
-	}
-
-	deleted, err := db.DeleteLiveRoutesOfPeer(peerIP)
-	if err != nil {
-		t.Fatalf("DeleteLiveRoutesOfPeer: %v", err)
-	}
-
-	if deleted != len(peerRoutes) {
-		t.Errorf("DeleteLiveRoutesOfPeer deleted %d routes, want %d",
-			deleted, len(peerRoutes))
-	}
-
-	checkV4RoutesOfOwnPrefixes(t, db, "after DeleteLiveRoutesOfPeer",
-		sharedPrefixes)
-}
-
-// checkV4RoutesOfOwnPrefixes checks that the database has wantRoutes IPv4 live
-// routes, each for a /24 no other live route has, both as counted in memory and
-// as read from the route table. when says which step of the test checks.
-func checkV4RoutesOfOwnPrefixes(
-	t *testing.T, db *Database, when string, wantRoutes int,
-) {
-	t.Helper()
-
-	ctx := context.Background()
-
-	stats, err := db.GetStatsContext(ctx)
-	if err != nil {
-		t.Fatalf("GetStatsContext: %v", err)
-	}
-
-	tableRoutes, _, err := db.GetLiveRouteCountsContext(ctx)
-	if err != nil {
-		t.Fatalf("GetLiveRouteCountsContext: %v", err)
-	}
-
-	if stats.IPv4Routes != wantRoutes || tableRoutes != wantRoutes {
-		t.Errorf("%s: IPv4 live routes: %d in memory, %d in the table, want %d",
-			when, stats.IPv4Routes, tableRoutes, wantRoutes)
-	}
-
-	queryDistribution, _, err := db.GetPrefixDistributionContext(ctx)
-	if err != nil {
-		t.Fatalf("GetPrefixDistributionContext: %v", err)
-	}
-
-	want := []PrefixDistribution{{MaskLength: 24, Count: wantRoutes}}
-	assertDistribution(t, when+": IPv4 distribution",
-		stats.IPv4PrefixDistribution, want)
-	assertDistribution(t, when+": IPv4 distribution query",
-		queryDistribution, want)
-}
-
 // checkOneLiveRoutePerIPVersion checks that the database has one IPv4 live
 // route, for a /24, and one IPv6 live route, for a /32, both as counted in
 // memory and as read from the route tables.

internal/database/dockerignore_internal_test.go

@@ -1,61 +0,0 @@
-package database
-
-import (
-	"os"
-	"path/filepath"
-	"strings"
-	"testing"
-
-	"git.eeqj.de/sneak/routewatch/internal/config"
-	"git.eeqj.de/sneak/routewatch/internal/logger"
-)
-
-// TestDockerignoreExcludesDatabaseFiles checks that .dockerignore keeps the
-// database file, and the WAL and shared-memory files SQLite writes next to it,
-// out of the build context wherever they sit in the clone. The image ships the
-// whole build context as a source archive.
-func TestDockerignoreExcludesDatabaseFiles(t *testing.T) {
-	t.Parallel()
-
-	db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
-	if err != nil {
-		t.Fatalf("failed to create database: %v", err)
-	}
-	defer func() { _ = db.Close() }()
-
-	data, err := os.ReadFile("../../.dockerignore")
-	if err != nil {
-		t.Fatalf("failed to read .dockerignore: %v", err)
-	}
-
-	// Docker reads `**/` as any number of directories and the rest of the
-	// pattern as filepath.Match does, so `**/` followed by a pattern that
-	// matches a file's name excludes that file at any depth.
-	var anyDepth []string
-
-	for line := range strings.Lines(string(data)) {
-		pattern, ok := strings.CutPrefix(strings.TrimSpace(line), "**/")
-		if ok {
-			anyDepth = append(anyDepth, pattern)
-		}
-	}
-
-	name := filepath.Base(db.path)
-
-	for _, file := range []string{name, name + "-wal", name + "-shm"} {
-		excluded := false
-
-		for _, pattern := range anyDepth {
-			matched, err := filepath.Match(pattern, file)
-			if err != nil {
-				t.Fatalf("bad .dockerignore pattern **/%s: %v", pattern, err)
-			}
-
-			excluded = excluded || matched
-		}
-
-		if !excluded {
-			t.Errorf(".dockerignore has no **/ pattern that excludes %s", file)
-		}
-	}
-}

internal/database/interface.go

@@ -1,7 +1,5 @@
 // Package database provides SQLite storage for BGP routing data including ASNs,
-// prefixes, announcements, peerings, and live route tables. Its routes table
+// prefixes, announcements, peerings, and live route tables.
-// holds a copy of the whole routing table that internal/routetable keeps in
-// memory.
 package database
 
 import (
@@ -36,7 +34,7 @@ type Stats struct {
 type Store interface {
 	// ASN operations
 	GetOrCreateASN(number int, timestamp time.Time) (*ASN, error)
-	GetOrCreateASNBatch(asns map[int]SeenTimes) error
+	GetOrCreateASNBatch(asns map[int]time.Time) error
 
 	// Prefix operations
 	GetOrCreatePrefix(prefix string, timestamp time.Time) (*Prefix, error)
@@ -46,7 +44,7 @@ type Store interface {
 	RecordAnnouncement(announcement *Announcement) error
 
 	// Peering operations
-	RecordPeering(asA, asB int, seen SeenTimes) error
+	RecordPeering(asA, asB int, timestamp time.Time) error
 
 	// Statistics
 	GetStats() (Stats, error)

internal/database/models.go

@@ -28,19 +28,19 @@ type ASN struct {
 	RIRRegDate  *time.Time `json:"rir_registration_date,omitempty"`
 	RIRLastMod  *time.Time `json:"rir_last_modified,omitempty"`
 	WHOISRaw    string     `json:"whois_raw,omitempty"`
-	// Timestamps. They are zero, and left out of JSON, for an AS that has no
+	// Timestamps
-	// record in the asns table.
+	FirstSeen      time.Time  `json:"first_seen"`
-	FirstSeen      time.Time  `json:"first_seen,omitzero"`
+	LastSeen       time.Time  `json:"last_seen"`
-	LastSeen       time.Time  `json:"last_seen,omitzero"`
 	WHOISUpdatedAt *time.Time `json:"whois_updated_at,omitempty"`
 }
 
 // Prefix represents an IP prefix (CIDR block) with its IP version (4 or 6)
-// and the time it was first seen.
+// and first/last seen timestamps.
 type Prefix struct {
 	Prefix    string
 	IPVersion int // 4 or 6
 	FirstSeen time.Time
+	LastSeen  time.Time
 }
 
 // Announcement represents a BGP announcement or withdrawal event,
@@ -93,6 +93,7 @@ type PrefixDistribution struct {
 //nolint:tagliatelle // field names are the existing JSON API
 type IPInfo struct {
 	IP         string   `json:"ip"`
+	PTR        []string `json:"ptr,omitempty"`
 	Netblock   string   `json:"netblock"`
 	MaskLength int      `json:"mask_length"`
 	IPVersion  int      `json:"ip_version"`
@@ -122,35 +123,12 @@ type LiveRouteDeletion struct {
 	IPVersion int
 }
 
-// SeenTimes is the earliest and the latest time one batch of updates saw an AS,
+// PeerUpdate represents parameters for updating a peer
-// a peer or a peering. A row the batch adds is stored first seen at First and
-// last seen at Last. A row already stored takes First as its first_seen only
-// when First is earlier, and Last as its last_seen only when Last is later.
-type SeenTimes struct {
-	First time.Time
-	Last  time.Time
-}
-
-// Add makes t the earliest time of s when it is earlier than First, and the
-// latest when it is later than Last. The zero SeenTimes holds no time yet, so
-// the first Add sets both.
-func (s *SeenTimes) Add(t time.Time) {
-	if s.First.IsZero() || t.Before(s.First) {
-		s.First = t
-	}
-
-	if s.Last.IsZero() || t.After(s.Last) {
-		s.Last = t
-	}
-}
-
-// PeerUpdate represents parameters for updating a peer. PeerASN and
-// MessageType are those of the peer's latest update.
 type PeerUpdate struct {
 	PeerIP      string
 	PeerASN     int
 	MessageType string
-	Seen        SeenTimes
+	Timestamp   time.Time
 }
 
 // ASNWHOISUpdate contains WHOIS data for updating an ASN record.

internal/database/routes.go

@@ -1,339 +0,0 @@
-package database
-
-import (
-	"context"
-	"database/sql"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"net/netip"
-	"time"
-
-	"github.com/bits-and-blooms/bitset"
-
-	"git.eeqj.de/sneak/routewatch/internal/routetable"
-)
-
-// keptRoutesPerRow is how many kept routes a row of the routes table has
-// columns for: the newest and the older one.
-const keptRoutesPerRow = 2
-
-// errUnknownRoutePeer is returned for a peer that route_peers lacks: on a
-// write, the peer of a kept route; on a read, a peer number in a row.
-var errUnknownRoutePeer = errors.New("peer not in route_peers")
-
-// The statements of the routes and route_peers tables (schema.sql).
-const (
-	insertRoutePeerSQL = `INSERT INTO route_peers (number, address)
-		VALUES (?, ?)`
-	routePeerNumberSQL = `SELECT number FROM route_peers WHERE address = ?`
-	routePeersSQL      = `SELECT number, address FROM route_peers`
-
-	upsertRouteSQL = `INSERT INTO routes (prefix, origin_asn, peers,
-			newest_peer, newest_as_path, newest_time,
-			older_peer, older_as_path, older_time)
-		VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
-		ON CONFLICT (prefix, origin_asn) DO UPDATE SET peers = excluded.peers,
-			newest_peer = excluded.newest_peer,
-			newest_as_path = excluded.newest_as_path,
-			newest_time = excluded.newest_time,
-			older_peer = excluded.older_peer,
-			older_as_path = excluded.older_as_path,
-			older_time = excluded.older_time`
-	deleteRouteSQL = `DELETE FROM routes WHERE prefix = ? AND origin_asn = ?`
-	routesSQL      = `SELECT prefix, origin_asn, peers,
-		newest_peer, newest_as_path, newest_time,
-		older_peer, older_as_path, older_time FROM routes`
-)
-
-// WriteRouteChanges writes, in one transaction, what one call of the routing
-// table's TakeChanges returned. It stores each new peer with its number, writes
-// the row of each changed prefix and origin AS pair, and deletes the row of
-// each removed one. Route times are stored to the second, as the feed gives
-// them.
-//
-// When it fails, it writes nothing. The caller then writes these changes and
-// peers again, because later changes can name the peers it did not store.
-func (d *Database) WriteRouteChanges(
-	changes []routetable.Change, newPeers map[string]uint,
-) error {
-	if len(changes) == 0 && len(newPeers) == 0 {
-		return nil
-	}
-
-	d.lock("WriteRouteChanges")
-	defer d.unlock()
-
-	tx, err := d.beginTx()
-	if err != nil {
-		return fmt.Errorf("failed to begin transaction: %w", err)
-	}
-	defer d.rollback(tx)
-
-	ctx := context.Background()
-
-	// The new peers first, so that the rows can look up their numbers.
-	for address, number := range newPeers {
-		_, err := tx.ExecContext(ctx, insertRoutePeerSQL, number, address)
-		if err != nil {
-			return fmt.Errorf("failed to store peer %s: %w", address, err)
-		}
-	}
-
-	err = writeRouteRows(ctx, tx, changes)
-	if err != nil {
-		return err
-	}
-
-	err = tx.Commit()
-	if err != nil {
-		return fmt.Errorf("failed to commit transaction: %w", err)
-	}
-
-	return nil
-}
-
-// writeRouteRows upserts the row of each change in tx, or deletes it for a
-// removed pair.
-func writeRouteRows(
-	ctx context.Context, tx *loggingTx, changes []routetable.Change,
-) error {
-	upsert, err := tx.PrepareContext(ctx, upsertRouteSQL)
-	if err != nil {
-		return fmt.Errorf("failed to prepare route upsert: %w", err)
-	}
-	defer func() { _ = upsert.Close() }()
-
-	del, err := tx.PrepareContext(ctx, deleteRouteSQL)
-	if err != nil {
-		return fmt.Errorf("failed to prepare route delete: %w", err)
-	}
-	defer func() { _ = del.Close() }()
-
-	peerNumber, err := tx.PrepareContext(ctx, routePeerNumberSQL)
-	if err != nil {
-		return fmt.Errorf("failed to prepare peer number lookup: %w", err)
-	}
-	defer func() { _ = peerNumber.Close() }()
-
-	for _, change := range changes {
-		prefix := change.Prefix.String()
-
-		if change.Removed {
-			_, err := del.ExecContext(ctx, prefix, change.OriginASN)
-			if err != nil {
-				return fmt.Errorf("failed to delete route %s: %w", prefix, err)
-			}
-
-			continue
-		}
-
-		row, err := routeRow(ctx, peerNumber, change)
-		if err != nil {
-			return fmt.Errorf("failed to write route %s: %w", prefix, err)
-		}
-
-		_, err = upsert.ExecContext(ctx, row...)
-		if err != nil {
-			return fmt.Errorf("failed to write route %s: %w", prefix, err)
-		}
-	}
-
-	return nil
-}
-
-// routeRow returns the values of the routes row of change, in the order of
-// upsertRouteSQL. peerNumber looks up a peer's number by its address. The
-// table keeps at most keptRoutesPerRow routes of a pair.
-func routeRow(
-	ctx context.Context, peerNumber *sql.Stmt, change routetable.Change,
-) ([]any, error) {
-	peers, err := change.Peers.MarshalBinary()
-	if err != nil {
-		return nil, err
-	}
-
-	row := []any{change.Prefix.String(), change.OriginASN, peers}
-
-	for i := range keptRoutesPerRow {
-		if i >= len(change.Routes) {
-			row = append(row, nil, nil, nil)
-
-			continue
-		}
-
-		r := change.Routes[i]
-
-		var number uint
-
-		err := peerNumber.QueryRowContext(ctx, r.Peer).Scan(&number)
-		if errors.Is(err, sql.ErrNoRows) {
-			return nil, fmt.Errorf("%w: %s", errUnknownRoutePeer, r.Peer)
-		}
-
-		if err != nil {
-			return nil, err
-		}
-
-		path, err := json.Marshal(r.ASPath)
-		if err != nil {
-			return nil, err
-		}
-
-		row = append(row, number, string(path), r.Time.Unix())
-	}
-
-	return row, nil
-}
-
-// ReadRouteTable returns the routing table that the routes and route_peers
-// tables hold, as startup reads it. It reads the peers first, so that the
-// table keeps their numbers, then loads each row into the table as it reads
-// it. The two reads are separate, so nothing may write the tables meanwhile.
-// It fails on a row that names a peer number route_peers lacks.
-func (d *Database) ReadRouteTable(ctx context.Context) (*routetable.Table, error) {
-	peerNumbers, err := d.readRoutePeers(ctx)
-	if err != nil {
-		return nil, err
-	}
-
-	addresses := make(map[uint]string, len(peerNumbers))
-	numbers := new(bitset.BitSet)
-
-	for address, number := range peerNumbers {
-		addresses[number] = address
-		numbers.Set(number)
-	}
-
-	table := routetable.New(routetable.Params{PeerNumbers: peerNumbers})
-
-	rows, err := d.db.QueryContext(ctx, routesSQL)
-	if err != nil {
-		return nil, fmt.Errorf("failed to read routes: %w", err)
-	}
-	defer func() { _ = rows.Close() }()
-
-	for rows.Next() {
-		change, err := scanRouteRow(rows, addresses, numbers)
-		if err != nil {
-			return nil, err
-		}
-
-		table.Load(change)
-	}
-
-	err = rows.Err()
-	if err != nil {
-		return nil, fmt.Errorf("failed to read routes: %w", err)
-	}
-
-	return table, nil
-}
-
-// readRoutePeers returns the number of each peer in route_peers, by address.
-func (d *Database) readRoutePeers(ctx context.Context) (map[string]uint, error) {
-	rows, err := d.db.QueryContext(ctx, routePeersSQL)
-	if err != nil {
-		return nil, fmt.Errorf("failed to read route peers: %w", err)
-	}
-	defer func() { _ = rows.Close() }()
-
-	peerNumbers := make(map[string]uint)
-
-	for rows.Next() {
-		var (
-			number  uint
-			address string
-		)
-
-		err := rows.Scan(&number, &address)
-		if err != nil {
-			return nil, fmt.Errorf("failed to read a route peer: %w", err)
-		}
-
-		peerNumbers[address] = number
-	}
-
-	err = rows.Err()
-	if err != nil {
-		return nil, fmt.Errorf("failed to read route peers: %w", err)
-	}
-
-	return peerNumbers, nil
-}
-
-// keptRouteColumns are the three columns of one kept route in a routes row.
-type keptRouteColumns struct {
-	peer   sql.Null[uint]
-	asPath sql.NullString
-	time   sql.NullInt64
-}
-
-// scanRouteRow returns the content of the routes row rows is at, with each kept
-// route's peer named by its address in addresses. It fails when the row names a
-// peer number that addresses lacks. numbers holds the numbers of addresses as a
-// set, against which a row's whole peer set is checked at once.
-func scanRouteRow(
-	rows *sql.Rows, addresses map[uint]string, numbers *bitset.BitSet,
-) (routetable.Change, error) {
-	var (
-		change routetable.Change
-		prefix string
-		peers  []byte
-		kept   [keptRoutesPerRow]keptRouteColumns
-	)
-
-	err := rows.Scan(&prefix, &change.OriginASN, &peers,
-		&kept[0].peer, &kept[0].asPath, &kept[0].time,
-		&kept[1].peer, &kept[1].asPath, &kept[1].time)
-	if err != nil {
-		return change, fmt.Errorf("failed to read a route: %w", err)
-	}
-
-	change.Prefix, err = netip.ParsePrefix(prefix)
-	if err != nil {
-		return change, fmt.Errorf("failed to read route %s: %w", prefix, err)
-	}
-
-	change.Peers = new(bitset.BitSet)
-
-	err = change.Peers.UnmarshalBinary(peers)
-	if err != nil {
-		return change, fmt.Errorf("failed to read the peers of route %s: %w",
-			prefix, err)
-	}
-
-	if !numbers.IsSuperSet(change.Peers) {
-		unknown, _ := change.Peers.Difference(numbers).NextSet(0)
-
-		return change, fmt.Errorf("%w: number %d in the peers of route %s",
-			errUnknownRoutePeer, unknown, prefix)
-	}
-
-	for _, columns := range kept {
-		if !columns.peer.Valid {
-			continue
-		}
-
-		address, known := addresses[columns.peer.V]
-		if !known {
-			return change, fmt.Errorf("%w: number %d in route %s",
-				errUnknownRoutePeer, columns.peer.V, prefix)
-		}
-
-		r := routetable.Route{
-			Peer: address,
-			Time: time.Unix(columns.time.Int64, 0).UTC(),
-		}
-
-		err := json.Unmarshal([]byte(columns.asPath.String), &r.ASPath)
-		if err != nil {
-			return change, fmt.Errorf("failed to read an AS path of route %s: %w",
-				prefix, err)
-		}
-
-		change.Routes = append(change.Routes, r)
-	}
-
-	return change, nil
-}

internal/database/routes_internal_test.go

@@ -1,526 +0,0 @@
-package database
-
-import (
-	"errors"
-	"fmt"
-	"math/rand/v2"
-	"net/netip"
-	"os"
-	"reflect"
-	"slices"
-	"testing"
-	"time"
-
-	"github.com/bits-and-blooms/bitset"
-
-	"git.eeqj.de/sneak/routewatch/internal/config"
-	"git.eeqj.de/sneak/routewatch/internal/logger"
-	"git.eeqj.de/sneak/routewatch/internal/routetable"
-)
-
-// Peers, origin ASes and prefixes of the routes table tests.
-const (
-	peerA = "192.0.2.1"
-	peerB = "192.0.2.2"
-	peerC = "2001:db8::3"
-	peerD = "2001:db8::4"
-	peerE = "2001:db8::5"
-
-	asA = 64500
-	asB = 64501
-	// transitAS is the first AS of every test path, the peer's own.
-	transitAS = 64496
-
-	tablePrefixV4      = "10.1.2.0/24"
-	tablePrefixShortV4 = "10.9.0.0/16"
-	tablePrefixV6      = "2001:db8:1::/48"
-)
-
-// tableTime returns the time second seconds after the start of every routes
-// table test.
-func tableTime(second int) time.Time {
-	return time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC).
-		Add(time.Duration(second) * time.Second)
-}
-
-// announceTo has peer announce prefix with origin AS asn second seconds after
-// the start.
-func announceTo(
-	table *routetable.Table, prefix string, asn int, peer string, second int,
-) {
-	table.Announce(netip.MustParsePrefix(prefix), asn, routetable.Route{
-		Peer: peer, ASPath: []int{transitAS, asn}, Time: tableTime(second),
-	})
-}
-
-// newRoutesTestDatabase returns a new database in a temporary directory.
-func newRoutesTestDatabase(t *testing.T) *Database {
-	t.Helper()
-
-	db, err := New(&config.Config{StateDir: t.TempDir()}, logger.New())
-	if err != nil {
-		t.Fatalf("failed to create database: %v", err)
-	}
-
-	t.Cleanup(func() { _ = db.Close() })
-
-	return db
-}
-
-// writeTableChanges writes what table changed since the last call.
-func writeTableChanges(t *testing.T, db *Database, table *routetable.Table) {
-	t.Helper()
-
-	changes, newPeers := table.TakeChanges()
-
-	err := db.WriteRouteChanges(changes, newPeers)
-	if err != nil {
-		t.Fatalf("WriteRouteChanges: %v", err)
-	}
-}
-
-func TestReadRouteTableGivesBackWhatWasWritten(t *testing.T) {
-	t.Parallel()
-
-	db := newRoutesTestDatabase(t)
-	source := routetable.New(routetable.Params{})
-
-	// Two origins of one prefix: the routes of peerC and peerB are kept for
-	// asA, peerA keeps only its bit.
-	announceTo(source, tablePrefixV4, asA, peerA, 10)
-	announceTo(source, tablePrefixV4, asA, peerB, 20)
-	announceTo(source, tablePrefixV4, asA, peerC, 30)
-	announceTo(source, tablePrefixV4, asB, peerD, 40)
-	// Two routes with the same time.
-	announceTo(source, tablePrefixV6, asA, peerA, 50)
-	announceTo(source, tablePrefixV6, asA, peerC, 50)
-	// An origin whose kept routes were withdrawn while peerA carries it.
-	announceTo(source, tablePrefixShortV4, asB, peerA, 10)
-	announceTo(source, tablePrefixShortV4, asB, peerB, 20)
-	announceTo(source, tablePrefixShortV4, asB, peerC, 30)
-	source.Withdraw(netip.MustParsePrefix(tablePrefixShortV4), peerB)
-	source.Withdraw(netip.MustParsePrefix(tablePrefixShortV4), peerC)
-
-	writeTableChanges(t, db, source)
-	checkReadTable(t, db, source)
-
-	// A peer moves to another origin, one goes down and a new one comes.
-	announceTo(source, tablePrefixV4, asB, peerA, 60)
-	source.PeerDown(peerC)
-	announceTo(source, tablePrefixV6, asB, peerE, 70)
-
-	writeTableChanges(t, db, source)
-	checkReadTable(t, db, source)
-}
-
-func TestReadRouteTableGivesATableThatTakesLaterUpdates(t *testing.T) {
-	t.Parallel()
-
-	db := newRoutesTestDatabase(t)
-	source := routetable.New(routetable.Params{})
-
-	// peerA and peerB carry tablePrefixV4, peerB and peerC tablePrefixV6.
-	announceTo(source, tablePrefixV4, asA, peerA, 10)
-	announceTo(source, tablePrefixV4, asA, peerB, 20)
-	announceTo(source, tablePrefixV6, asA, peerB, 30)
-	announceTo(source, tablePrefixV6, asA, peerC, 40)
-
-	writeTableChanges(t, db, source)
-	read := checkReadTable(t, db, source)
-
-	// The same updates to both tables: a loaded peer withdraws a prefix,
-	// another one goes down, and a new peer comes.
-	for _, table := range []*routetable.Table{source, read} {
-		table.Withdraw(netip.MustParsePrefix(tablePrefixV4), peerA)
-		table.PeerDown(peerB)
-		announceTo(table, tablePrefixShortV4, asB, peerD, 50)
-	}
-
-	checkSameReads(t, read, source)
-}
-
-func TestReadRouteTableFailsOnAPeerWithoutANumber(t *testing.T) {
-	t.Parallel()
-
-	db := newRoutesTestDatabase(t)
-	table := routetable.New(routetable.Params{})
-
-	// Only the routes of peerB and peerC are kept, so only the peer set names
-	// peerA.
-	announceTo(table, tablePrefixV4, asA, peerA, 10)
-	announceTo(table, tablePrefixV4, asA, peerB, 20)
-	announceTo(table, tablePrefixV4, asA, peerC, 30)
-	writeTableChanges(t, db, table)
-
-	_, err := db.db.ExecContext(t.Context(),
-		"DELETE FROM route_peers WHERE address = ?", peerA)
-	if err != nil {
-		t.Fatalf("deleting a route peer: %v", err)
-	}
-
-	_, err = db.ReadRouteTable(t.Context())
-	if !errors.Is(err, errUnknownRoutePeer) {
-		t.Fatalf("ReadRouteTable = %v, want %v", err, errUnknownRoutePeer)
-	}
-}
-
-// checkReadTable checks that the table ReadRouteTable reads answers every read
-// as want does, and that reading it recorded no change. It returns that table.
-func checkReadTable(
-	t *testing.T, db *Database, want *routetable.Table,
-) *routetable.Table {
-	t.Helper()
-
-	got, err := db.ReadRouteTable(t.Context())
-	if err != nil {
-		t.Fatalf("ReadRouteTable: %v", err)
-	}
-
-	checkSameReads(t, got, want)
-
-	changes, newPeers := got.TakeChanges()
-	if len(changes) != 0 || len(newPeers) != 0 {
-		t.Errorf("the read recorded changes %+v and new peers %v, want none",
-			changes, newPeers)
-	}
-
-	return got
-}
-
-// checkSameReads checks that got answers every read as want does, for the
-// prefixes and the origin ASes the tests use.
-func checkSameReads(t *testing.T, got, want *routetable.Table) {
-	t.Helper()
-
-	if !reflect.DeepEqual(got.Counts(), want.Counts()) {
-		t.Errorf("Counts() = %+v, want %+v", got.Counts(), want.Counts())
-	}
-
-	for _, text := range []string{
-		tablePrefixV4, tablePrefixShortV4, tablePrefixV6,
-	} {
-		prefix := netip.MustParsePrefix(text)
-
-		if !reflect.DeepEqual(got.Origins(prefix), want.Origins(prefix)) {
-			t.Errorf("Origins(%s) = %+v, want %+v",
-				prefix, got.Origins(prefix), want.Origins(prefix))
-		}
-
-		gotSummary, gotFound := got.Lookup(prefix.Addr())
-		wantSummary, wantFound := want.Lookup(prefix.Addr())
-
-		if gotFound != wantFound || !reflect.DeepEqual(gotSummary, wantSummary) {
-			t.Errorf("Lookup(%s) = %+v, %v; want %+v, %v", prefix.Addr(),
-				gotSummary, gotFound, wantSummary, wantFound)
-		}
-
-		ipVersion := ipVersionV6
-		if prefix.Addr().Is4() {
-			ipVersion = ipVersionV4
-		}
-
-		gotSample := sortedSample(got, ipVersion, prefix.Bits())
-		wantSample := sortedSample(want, ipVersion, prefix.Bits())
-
-		if !reflect.DeepEqual(gotSample, wantSample) {
-			t.Errorf("SamplePrefixes(%d, %d) = %+v, want %+v",
-				ipVersion, prefix.Bits(), gotSample, wantSample)
-		}
-	}
-
-	for _, asn := range []int{asA, asB} {
-		if !reflect.DeepEqual(got.OriginatedPrefixes(asn),
-			want.OriginatedPrefixes(asn)) {
-			t.Errorf("OriginatedPrefixes(%d) = %+v, want %+v", asn,
-				got.OriginatedPrefixes(asn), want.OriginatedPrefixes(asn))
-		}
-	}
-}
-
-// sortedSample returns every prefix of the IP version and mask length in
-// table, in prefix order.
-func sortedSample(
-	table *routetable.Table, ipVersion, maskLength int,
-) []routetable.PrefixSummary {
-	// More than the tests have of any mask length.
-	const all = 100
-
-	sample := table.SamplePrefixes(ipVersion, maskLength, all)
-	slices.SortFunc(sample, func(a, b routetable.PrefixSummary) int {
-		return a.Prefix.Compare(b.Prefix)
-	})
-
-	return sample
-}
-
-func TestWriteRouteChangesDeletesTheRowOfARemovedPair(t *testing.T) {
-	t.Parallel()
-
-	db := newRoutesTestDatabase(t)
-	table := routetable.New(routetable.Params{})
-
-	announceTo(table, tablePrefixV4, asA, peerA, 10)
-	announceTo(table, tablePrefixV4, asB, peerB, 20)
-	writeTableChanges(t, db, table)
-
-	// The prefix's only peer with asA withdraws it.
-	table.Withdraw(netip.MustParsePrefix(tablePrefixV4), peerA)
-	writeTableChanges(t, db, table)
-
-	var origins []int
-
-	rows, err := db.db.QueryContext(t.Context(),
-		"SELECT origin_asn FROM routes WHERE prefix = ?", tablePrefixV4)
-	if err != nil {
-		t.Fatalf("reading the routes: %v", err)
-	}
-	defer func() { _ = rows.Close() }()
-
-	for rows.Next() {
-		var origin int
-
-		err := rows.Scan(&origin)
-		if err != nil {
-			t.Fatalf("reading a route: %v", err)
-		}
-
-		origins = append(origins, origin)
-	}
-
-	err = rows.Err()
-	if err != nil {
-		t.Fatalf("reading the routes: %v", err)
-	}
-
-	if !slices.Equal(origins, []int{asB}) {
-		t.Errorf("rows of %s have origins %v, want only %d",
-			tablePrefixV4, origins, asB)
-	}
-}
-
-// routePeer is one row of route_peers.
-type routePeer struct {
-	number  uint
-	address string
-}
-
-// readRoutePeerRows returns the rows of route_peers in number order.
-func readRoutePeerRows(t *testing.T, db *Database) []routePeer {
-	t.Helper()
-
-	rows, err := db.db.QueryContext(t.Context(),
-		"SELECT number, address FROM route_peers ORDER BY number")
-	if err != nil {
-		t.Fatalf("reading the route peers: %v", err)
-	}
-	defer func() { _ = rows.Close() }()
-
-	var peers []routePeer
-
-	for rows.Next() {
-		var peer routePeer
-
-		err := rows.Scan(&peer.number, &peer.address)
-		if err != nil {
-			t.Fatalf("reading a route peer: %v", err)
-		}
-
-		peers = append(peers, peer)
-	}
-
-	err = rows.Err()
-	if err != nil {
-		t.Fatalf("reading the route peers: %v", err)
-	}
-
-	return peers
-}
-
-func TestWriteRouteChangesStoresNewPeers(t *testing.T) {
-	t.Parallel()
-
-	db := newRoutesTestDatabase(t)
-	table := routetable.New(routetable.Params{})
-
-	announceTo(table, tablePrefixV4, asA, peerA, 10)
-	announceTo(table, tablePrefixV6, asA, peerC, 20)
-	writeTableChanges(t, db, table)
-
-	// peerA is stored already; peerB is new.
-	announceTo(table, tablePrefixV4, asA, peerA, 30)
-	announceTo(table, tablePrefixV4, asA, peerB, 40)
-	writeTableChanges(t, db, table)
-
-	want := []routePeer{{0, peerA}, {1, peerC}, {2, peerB}}
-
-	got := readRoutePeerRows(t, db)
-	if !slices.Equal(got, want) {
-		t.Errorf("route_peers = %+v, want %+v", got, want)
-	}
-}
-
-func TestWriteRouteChangesWritesNothingWhenItFails(t *testing.T) {
-	t.Parallel()
-
-	db := newRoutesTestDatabase(t)
-	table := routetable.New(routetable.Params{})
-
-	announceTo(table, tablePrefixV4, asA, peerA, 10)
-	announceTo(table, tablePrefixV6, asA, peerB, 20)
-
-	// peerB's number is missing, so its kept route cannot be written.
-	changes, newPeers := table.TakeChanges()
-	delete(newPeers, peerB)
-
-	err := db.WriteRouteChanges(changes, newPeers)
-	if !errors.Is(err, errUnknownRoutePeer) {
-		t.Fatalf("WriteRouteChanges = %v, want %v", err, errUnknownRoutePeer)
-	}
-
-	peers := readRoutePeerRows(t, db)
-	if len(peers) != 0 {
-		t.Errorf("route_peers = %+v after the failed write, want none", peers)
-	}
-
-	var routes int
-
-	err = db.db.QueryRowContext(t.Context(),
-		"SELECT COUNT(*) FROM routes").Scan(&routes)
-	if err != nil {
-		t.Fatalf("counting the routes: %v", err)
-	}
-
-	if routes != 0 {
-		t.Errorf("routes holds %d rows after the failed write, want none", routes)
-	}
-}
-
-// Parameters for BenchmarkReadRouteTable.
-const (
-	// benchTablePairs is how many prefix and origin AS pairs the table holds:
-	// one origin AS for each prefix. Every benchTableV6Every-th prefix is an
-	// IPv6 /48, the others IPv4 /24s.
-	benchTablePairs   = 200000
-	benchTableV6Every = 5
-	// benchTablePeers is how many peers there are, about as many as RIS has.
-	// benchTablePeersPerPair is how many of them, picked at random, carry each
-	// pair: RIS's 460 million routes over 1.4 million pairs.
-	benchTablePeers        = 1400
-	benchTablePeersPerPair = 330
-	// benchTableBatch is how many changes each write holds.
-	benchTableBatch = 10000
-)
-
-// BenchmarkReadRouteTable times the read of a routing table of 200,000 prefix
-// and origin AS pairs, each carried by 330 of 1,400 peers and keeping two
-// routes. It writes that table first. It reports the bytes of the database
-// file per pair. It is run by hand, not by make test, for example in the test
-// image:
-//
-//	go test -run '^$' -bench ReadRouteTable -benchtime 5x ./internal/database/
-func BenchmarkReadRouteTable(b *testing.B) {
-	db, err := New(&config.Config{StateDir: b.TempDir()}, logger.New())
-	if err != nil {
-		b.Fatalf("failed to create database: %v", err)
-	}
-	defer func() { _ = db.Close() }()
-
-	//nolint:gosec // The benchmark needs the same table every run.
-	rng := rand.New(rand.NewPCG(1, 2))
-
-	newPeers := make(map[string]uint, benchTablePeers)
-	for number := range uint(benchTablePeers) {
-		newPeers[benchPeerAddress(number)] = number
-	}
-
-	for first := 0; first < benchTablePairs; first += benchTableBatch {
-		changes := make([]routetable.Change, 0, benchTableBatch)
-		for pair := first; pair < first+benchTableBatch; pair++ {
-			changes = append(changes, benchTableChange(rng, pair))
-		}
-
-		err := db.WriteRouteChanges(changes, newPeers)
-		if err != nil {
-			b.Fatalf("WriteRouteChanges: %v", err)
-		}
-
-		newPeers = nil
-	}
-
-	// The WAL is copied into the database file, whose size is reported.
-	err = db.Checkpoint(b.Context())
-	if err != nil {
-		b.Fatalf("Checkpoint: %v", err)
-	}
-
-	file, err := os.Stat(db.path)
-	if err != nil {
-		b.Fatalf("reading the database file's size: %v", err)
-	}
-
-	for b.Loop() {
-		table, err := db.ReadRouteTable(b.Context())
-		if err != nil {
-			b.Fatalf("ReadRouteTable: %v", err)
-		}
-
-		counts := table.Counts()
-		if counts.IPv4Prefixes+counts.IPv6Prefixes != benchTablePairs {
-			b.Fatalf("read %d prefixes, want %d",
-				counts.IPv4Prefixes+counts.IPv6Prefixes, benchTablePairs)
-		}
-	}
-
-	// After the loop: b.Loop drops the metrics reported before it starts.
-	b.ReportMetric(float64(file.Size())/benchTablePairs, "file-bytes/pair")
-}
-
-// benchPeerAddress returns the address of the peer numbered number.
-func benchPeerAddress(number uint) string {
-	const addressesPerBlock = 256
-
-	return fmt.Sprintf("198.18.%d.%d",
-		number/addressesPerBlock, number%addressesPerBlock)
-}
-
-// benchTableChange returns the content of the pair numbered pair: its prefix,
-// its origin AS, benchTablePeersPerPair peers picked at random, and the routes
-// of two of them, each with an AS path of four ASes.
-func benchTableChange(rng *rand.Rand, pair int) routetable.Change {
-	const (
-		firstOrigin  = 64500
-		transitASes  = 50000
-		olderSeconds = 10
-	)
-
-	text := fmt.Sprintf("%d.%d.%d.0/24", 1+pair>>16, (pair>>8)%256, pair%256)
-	if pair%benchTableV6Every == 0 {
-		text = fmt.Sprintf("2001:%x:%x::/48", pair>>16, pair%65536)
-	}
-
-	prefix := netip.MustParsePrefix(text)
-	origin := firstOrigin + pair%transitASes
-	peers := bitset.New(benchTablePeers)
-
-	for peers.Count() < benchTablePeersPerPair {
-		peers.Set(rng.UintN(benchTablePeers))
-	}
-
-	change := routetable.Change{Prefix: prefix, OriginASN: origin, Peers: peers}
-
-	// The routes of the first two peers, newest first.
-	for i, number := range slices.Collect(peers.EachSet())[:keptRoutesPerRow] {
-		change.Routes = append(change.Routes, routetable.Route{
-			Peer: benchPeerAddress(number),
-			ASPath: []int{
-				firstOrigin + rng.IntN(transitASes),
-				firstOrigin + rng.IntN(transitASes),
-				firstOrigin + rng.IntN(transitASes),
-				origin,
-			},
-			Time: tableTime(pair - i*olderSeconds),
-		})
-	}
-
-	return change
-}

internal/database/schema.sql

@@ -31,13 +31,15 @@ CREATE TABLE IF NOT EXISTS asns (
 -- rows are stored by prefix (WITHOUT ROWID) and need no index of their own.
 CREATE TABLE IF NOT EXISTS prefixes_v4 (
 	prefix TEXT PRIMARY KEY,
-	first_seen DATETIME NOT NULL
+	first_seen DATETIME NOT NULL,
+	last_seen DATETIME NOT NULL
 ) WITHOUT ROWID;
 
 -- IPv6 prefixes table, stored as prefixes_v4 is.
 CREATE TABLE IF NOT EXISTS prefixes_v6 (
 	prefix TEXT PRIMARY KEY,
-	first_seen DATETIME NOT NULL
+	first_seen DATETIME NOT NULL,
+	last_seen DATETIME NOT NULL
 ) WITHOUT ROWID;
 
 CREATE TABLE IF NOT EXISTS announcements (
@@ -128,7 +130,7 @@ CREATE TABLE IF NOT EXISTS live_routes_v6 (
 -- prefix, last_updated: a prefix's routes, for the prefix page and its API
 -- answer, the IP lookup and its peer count, and the check whether a prefix
 -- still has a live route; and a prefix's newest route for the prefix-length
--- pages and the AS page, in one search whatever the number of peers.
+-- pages, in one search whatever the number of peers.
 CREATE INDEX IF NOT EXISTS idx_live_routes_v4_prefix_last_updated ON live_routes_v4(prefix, last_updated);
 CREATE INDEX IF NOT EXISTS idx_live_routes_v6_prefix_last_updated ON live_routes_v6(prefix, last_updated);
 -- mask_length: the prefix-length pages' random sample, the row id range of a
@@ -139,40 +141,10 @@ CREATE INDEX IF NOT EXISTS idx_live_routes_v6_mask_length ON live_routes_v6(mask
 -- prefixes in text order, and the prefix distribution read at startup.
 CREATE INDEX IF NOT EXISTS idx_live_routes_v4_mask_prefix ON live_routes_v4(mask_length, prefix);
 CREATE INDEX IF NOT EXISTS idx_live_routes_v6_mask_prefix ON live_routes_v6(mask_length, prefix);
--- origin_asn, prefix: the prefixes an AS originates, each in one search
+-- origin_asn: the prefixes an AS originates, for the AS page and its API answer.
--- whatever the number of peers, for the AS page and its API answer.
+CREATE INDEX IF NOT EXISTS idx_live_routes_v4_origin_asn ON live_routes_v4(origin_asn);
-CREATE INDEX IF NOT EXISTS idx_live_routes_v4_origin_prefix ON live_routes_v4(origin_asn, prefix);
+CREATE INDEX IF NOT EXISTS idx_live_routes_v6_origin_asn ON live_routes_v6(origin_asn);
-CREATE INDEX IF NOT EXISTS idx_live_routes_v6_origin_prefix ON live_routes_v6(origin_asn, prefix);
 -- last_updated: the oldest and newest route times in /api/v1/stats and
 -- /status.json.
 CREATE INDEX IF NOT EXISTS idx_live_routes_v4_last_updated ON live_routes_v4(last_updated);
 CREATE INDEX IF NOT EXISTS idx_live_routes_v6_last_updated ON live_routes_v6(last_updated);
-
--- The whole routing table, as internal/routetable keeps it in memory: one row
--- for each prefix and origin AS that some peer carries. Startup reads every
--- row, and the rows of the pairs that change are written back.
-CREATE TABLE IF NOT EXISTS routes (
-	prefix TEXT NOT NULL, -- in the text form Go's net/netip prints
-	origin_asn INTEGER NOT NULL,
-	-- The peers whose current route to the prefix has this origin AS: a set
-	-- of their numbers in route_peers, as the bitset package's MarshalBinary
-	-- writes it.
-	peers BLOB NOT NULL,
-	-- The newest of those routes and the one before it, the two kept in full.
-	-- Each has its peer's number in route_peers, its AS path as a JSON array,
-	-- and its time in seconds since 1970, UTC. A route that is not kept has
-	-- NULL in its three columns.
-	newest_peer INTEGER,
-	newest_as_path TEXT,
-	newest_time INTEGER,
-	older_peer INTEGER,
-	older_as_path TEXT,
-	older_time INTEGER,
-	PRIMARY KEY (prefix, origin_asn)
-) WITHOUT ROWID;
-
--- The peers of the routes table, each with the number it has there.
-CREATE TABLE IF NOT EXISTS route_peers (
-	number INTEGER PRIMARY KEY,
-	address TEXT NOT NULL UNIQUE
-);

internal/metrics/metrics.go

@@ -34,7 +34,7 @@ type Tracker struct {
 	// Announcement/withdrawal metrics
 	announcementCounter metrics.Counter
 	withdrawalCounter   metrics.Counter
-	churnPerMinute      minuteCounter // prefix announcements and withdrawals
+	churnRate           metrics.Meter // combined announcements + withdrawals per second
 
 	// BGP peer tracking
 	bgpPeerCount atomic.Int32
@@ -56,7 +56,7 @@ func New() *Tracker {
 		ipv6UpdateRate:      metrics.NewMeter(),
 		announcementCounter: metrics.NewCounter(),
 		withdrawalCounter:   metrics.NewCounter(),
-		churnPerMinute:      minuteCounter{now: time.Now},
+		churnRate:           metrics.NewMeter(),
 	}
 }
 
@@ -156,13 +156,13 @@ func (t *Tracker) RecordIPv6Update() {
 // RecordAnnouncement records a route announcement
 func (t *Tracker) RecordAnnouncement() {
 	t.announcementCounter.Inc(1)
-	t.churnPerMinute.add()
+	t.churnRate.Mark(1)
 }
 
 // RecordWithdrawal records a route withdrawal
 func (t *Tracker) RecordWithdrawal() {
 	t.withdrawalCounter.Inc(1)
-	t.churnPerMinute.add()
+	t.churnRate.Mark(1)
 }
 
 // SetBGPPeerCount updates the current BGP peer count
@@ -198,10 +198,9 @@ func (t *Tracker) GetWithdrawalCount() uint64 {
 	return uint64(count)
 }
 
-// GetChurnPerMinute returns the number of prefix announcements and
+// GetChurnRate returns the route churn rate per second
-// withdrawals, from all peers, in the last full clock minute.
+func (t *Tracker) GetChurnRate() float64 {
-func (t *Tracker) GetChurnPerMinute() int {
+	return t.churnRate.Rate1()
-	return t.churnPerMinute.lastFullMinute()
 }
 
 // GetRouteMetrics returns current route update metrics

internal/metrics/minutecounter.go

@@ -1,59 +0,0 @@
-package metrics
-
-import (
-	"sync"
-	"time"
-)
-
-// minuteCounter counts events in each clock minute and reports the count of
-// the last full minute. It is safe for concurrent use.
-type minuteCounter struct {
-	mu  sync.Mutex
-	now func() time.Time
-
-	// minute is the start of the minute that count belongs to.
-	minute time.Time
-	count  int
-	// lastMinuteCount is the count of the minute just before minute.
-	lastMinuteCount int
-}
-
-// add counts one event in the current minute.
-func (c *minuteCounter) add() {
-	c.mu.Lock()
-	defer c.mu.Unlock()
-
-	c.moveToCurrentMinute()
-	c.count++
-}
-
-// lastFullMinute returns the number of events in the minute that ended most
-// recently.
-func (c *minuteCounter) lastFullMinute() int {
-	c.mu.Lock()
-	defer c.mu.Unlock()
-
-	c.moveToCurrentMinute()
-
-	return c.lastMinuteCount
-}
-
-// moveToCurrentMinute starts a new count when the clock has moved into
-// another minute. The count it ends becomes the last full minute's count when
-// its minute is the one just before the current minute. Otherwise the last
-// full minute had no events. The caller holds mu.
-func (c *minuteCounter) moveToCurrentMinute() {
-	minute := c.now().Truncate(time.Minute)
-	if minute.Equal(c.minute) {
-		return
-	}
-
-	if minute.Sub(c.minute) == time.Minute {
-		c.lastMinuteCount = c.count
-	} else {
-		c.lastMinuteCount = 0
-	}
-
-	c.minute = minute
-	c.count = 0
-}

internal/metrics/minutecounter_internal_test.go

@@ -1,86 +0,0 @@
-package metrics
-
-import (
-	"sync"
-	"testing"
-	"time"
-)
-
-// TestChurnPerMinuteCountsLastFullMinute announces and withdraws prefixes on
-// an injected clock and expects each minute's whole count once that minute
-// has ended, and 0 for a minute with none.
-func TestChurnPerMinuteCountsLastFullMinute(t *testing.T) {
-	t.Parallel()
-
-	now := time.Date(2026, 10, 26, 12, 0, 30, 0, time.UTC)
-	tracker := New()
-	tracker.churnPerMinute.now = func() time.Time { return now }
-
-	expectChurn := func(want int, why string) {
-		t.Helper()
-
-		got := tracker.GetChurnPerMinute()
-		if got != want {
-			t.Fatalf("at %s churn = %d, want %d: %s",
-				now.Format(time.TimeOnly), got, want, why)
-		}
-	}
-
-	const firstMinuteChurn = 2
-
-	tracker.RecordAnnouncement()
-	tracker.RecordWithdrawal()
-	expectChurn(0, "12:00 has not ended")
-
-	now = time.Date(2026, 10, 26, 12, 1, 59, 0, time.UTC)
-
-	tracker.RecordAnnouncement()
-	expectChurn(firstMinuteChurn, "12:00 had one announcement and one withdrawal")
-
-	now = time.Date(2026, 10, 26, 12, 2, 0, 0, time.UTC)
-
-	expectChurn(1, "12:01 had one announcement")
-
-	now = time.Date(2026, 10, 26, 12, 4, 0, 0, time.UTC)
-
-	expectChurn(0, "12:03 had none")
-
-	tracker.RecordWithdrawal()
-
-	now = time.Date(2026, 10, 26, 12, 6, 0, 0, time.UTC)
-
-	expectChurn(0, "12:05 had none, though 12:04 had one withdrawal")
-}
-
-// TestChurnPerMinuteCountsConcurrentRecords announces prefixes from several
-// goroutines at once and expects every one counted.
-func TestChurnPerMinuteCountsConcurrentRecords(t *testing.T) {
-	t.Parallel()
-
-	const (
-		goroutines          = 8
-		recordsPerGoroutine = 1000
-	)
-
-	now := time.Date(2026, 10, 26, 12, 0, 0, 0, time.UTC)
-	tracker := New()
-	tracker.churnPerMinute.now = func() time.Time { return now }
-
-	var wg sync.WaitGroup
-	for range goroutines {
-		wg.Go(func() {
-			for range recordsPerGoroutine {
-				tracker.RecordAnnouncement()
-			}
-		})
-	}
-
-	wg.Wait()
-
-	now = now.Add(time.Minute)
-
-	want := goroutines * recordsPerGoroutine
-	if got := tracker.GetChurnPerMinute(); got != want {
-		t.Fatalf("churn = %d, want %d", got, want)
-	}
-}

internal/ristypes/ris.go

@@ -4,7 +4,6 @@ package ristypes
 
 import (
 	"encoding/json"
-	"slices"
 	"time"
 )
 
@@ -25,8 +24,7 @@ const (
 )
 
 // ASPath is a BGP AS path as the feed writes it: an array of AS numbers in
-// which an AS set is an array of its members, such as [1, 2, [3, 4]]. A path
+// which an AS set is an array of its members, such as [1, 2, [3, 4]].
-// that holds AS 0 decodes as an empty one (UnmarshalJSON).
 type ASPath struct {
 	// Sequence is the path's AS numbers in order. A set of one member that
 	// ends the path counts as that AS and is its last element. The members of
@@ -39,56 +37,34 @@ type ASPath struct {
 	Set []int
 }
 
-// NoOrigin is the origin AS of a route whose origin is not known. AS 0 is
-// reserved, so no AS has this number.
-const NoOrigin = 0
-
 // Origin returns the AS that originated the route: the last AS of the
-// sequence, or NoOrigin when the path ends in a set of several members or is
+// sequence, or 0 (none) when the path ends in a set of several members or is
 // empty.
 func (p *ASPath) Origin() int {
 	if len(p.Set) > 0 || len(p.Sequence) == 0 {
-		return NoOrigin
+		return 0
 	}
 
 	return p.Sequence[len(p.Sequence)-1]
 }
 
-// UnmarshalJSON implements the json.Unmarshaler interface for ASPath. A path
+// UnmarshalJSON implements the json.Unmarshaler interface for ASPath.
-// that holds AS 0 anywhere, a member of an AS set included, decodes as an empty
-// path: RFC 7607 makes such a path malformed and its announcements
-// withdrawals. On an announcement with an empty path the prefix handler
-// deletes the peer's route, and the AS and peering handlers record nothing.
 func (p *ASPath) UnmarshalJSON(data []byte) error {
-	path, err := decodeASPath(data)
-	if err != nil {
-		return err
-	}
-
-	if slices.Contains(path.Sequence, 0) || slices.Contains(path.Set, 0) {
-		path = ASPath{}
-	}
-
-	*p = path
-
-	return nil
-}
-
-// decodeASPath decodes an AS path as the feed writes it.
-func decodeASPath(data []byte) (ASPath, error) {
 	// Most paths have no AS set and decode as a plain array of integers.
 	var sequence []int
 
 	err := json.Unmarshal(data, &sequence)
 	if err == nil {
-		return ASPath{Sequence: sequence}, nil
+		*p = ASPath{Sequence: sequence}
+
+		return nil
 	}
 
 	var raw []any
 
 	err = json.Unmarshal(data, &raw)
 	if err != nil {
-		return ASPath{}, err
+		return err
 	}
 
 	path := ASPath{Sequence: make([]int, 0, len(raw))}
@@ -114,7 +90,9 @@ func decodeASPath(data []byte) (ASPath, error) {
 		}
 	}
 
-	return path, nil
+	*p = path
+
+	return nil
 }
 
 // RISLiveMessage represents the outer wrapper message from the RIPE RIS Live stream.

internal/ristypes/ris_test.go

@@ -145,33 +145,3 @@ func TestASPathKeepsATrailingSetApart(t *testing.T) {
 		})
 	}
 }
-
-// TestASPathWithASZeroDecodesAsAnEmptyPath decodes AS paths that hold AS 0
-// before their end, at their end, or in an AS set. RFC 7607 makes such a path
-// malformed, so each decodes as an empty path.
-func TestASPathWithASZeroDecodesAsAnEmptyPath(t *testing.T) {
-	t.Parallel()
-
-	for _, path := range []string{
-		"[64496, 0, 64497]",
-		"[64496, 0]",
-		"[64496, [0, 64497], 64498]",
-		"[64496, [64497, 0]]",
-	} {
-		t.Run(path, func(t *testing.T) {
-			t.Parallel()
-
-			var got ristypes.ASPath
-
-			err := json.Unmarshal([]byte(path), &got)
-			if err != nil {
-				t.Fatalf("unmarshal %s: %v", path, err)
-			}
-
-			if len(got.Sequence) > 0 || len(got.Set) > 0 {
-				t.Errorf("%s decoded to sequence %v and set %v, want an empty path",
-					path, got.Sequence, got.Set)
-			}
-		})
-	}
-}

internal/routetable/benchmark_test.go

@@ -1,154 +0,0 @@
-package routetable_test
-
-import (
-	"fmt"
-	"math/rand/v2"
-	"net/netip"
-	"runtime"
-	"testing"
-	"time"
-
-	"git.eeqj.de/sneak/routewatch/internal/routetable"
-)
-
-// Parameters for BenchmarkApplyMixedBatch.
-const (
-	// benchPeers is how many peers there are, about as many as RIS has.
-	benchPeers = 1400
-	// benchPrefixes is how many IPv4 /24 prefixes the table holds, and
-	// benchPeersPerPrefix how many peers, picked at random, carry each.
-	benchPrefixes       = 100000
-	benchPeersPerPrefix = 20
-	// benchBatchSize is how many updates the timed batch holds, as many as
-	// the prefix handler's largest batch (prefixBatchMaxSize in
-	// internal/routewatch).
-	benchBatchSize = 250000
-	// In the timed batch every benchWithdrawalEvery-th update is a
-	// withdrawal, and every benchNewOriginEvery-th announcement has an
-	// origin AS other than the prefix's.
-	benchWithdrawalEvery = 5
-	benchNewOriginEvery  = 7
-	// benchOrigins is how many origin ASes the prefixes are spread over,
-	// from benchFirstASN on: 20 prefixes each, about as many as an origin AS
-	// has on average in the full table.
-	benchOrigins  = 5000
-	benchFirstASN = 64500
-	// benchUpdatesPerSecond is how many updates share one time.
-	benchUpdatesPerSecond = 1000
-	benchMaskLength       = 24
-)
-
-// benchUpdate is one update of the benchmark.
-type benchUpdate struct {
-	prefix   netip.Prefix
-	origin   int
-	route    routetable.Route
-	withdraw bool
-}
-
-// BenchmarkApplyMixedBatch times one batch of 250,000 updates of routes picked
-// at random: announcements, withdrawals and announcements with a new origin AS.
-// The table it applies them to holds 100,000 IPv4 prefixes from 5,000 origin
-// ASes, each carried by 20 of 1,400 peers. It reports the heap bytes that table
-// takes per prefix and origin AS pair. It is run by hand, not by make test, for
-// example in the test image:
-//
-//	go test -run '^$' -bench MixedBatch -benchtime 10x ./internal/routetable/
-func BenchmarkApplyMixedBatch(b *testing.B) {
-	//nolint:gosec // The benchmark needs the same updates every run.
-	rng := rand.New(rand.NewPCG(1, 2))
-	peers := make([]string, benchPeers)
-
-	for i := range peers {
-		peers[i] = fmt.Sprintf("198.18.%d.%d", i/256, i%256)
-	}
-
-	before := heapBytes()
-	table := routetable.New(routetable.Params{})
-	updates := 0
-
-	for prefix := range benchPrefixes {
-		for range benchPeersPerPrefix {
-			peer := peers[rng.IntN(benchPeers)]
-			apply(table, benchAnnouncement(rng, prefix, peer, updates, false))
-			updates++
-		}
-	}
-
-	// The record of changes is handed over every second while routewatch
-	// runs, so it is not part of the table's size.
-	table.TakeChanges()
-
-	bytesPerPair := float64(heapBytes()-before) / benchPrefixes
-
-	batch := make([]benchUpdate, 0, benchBatchSize)
-	for i := range benchBatchSize {
-		update := benchAnnouncement(rng, rng.IntN(benchPrefixes),
-			peers[rng.IntN(benchPeers)], updates+i, i%benchNewOriginEvery == 0)
-		update.withdraw = i%benchWithdrawalEvery == 0
-		batch = append(batch, update)
-	}
-
-	for b.Loop() {
-		for _, update := range batch {
-			apply(table, update)
-		}
-	}
-
-	// After the loop: b.Loop drops the metrics reported before it starts.
-	b.ReportMetric(bytesPerPair, "bytes/pair")
-}
-
-// benchAnnouncement returns the announcement of the prefix numbered prefix, a
-// /24 from 1.0.0.0/24 on, by peer. It is the updates-th update of the
-// benchmark, and its AS path holds four ASes from the peer's AS to the origin
-// AS. newOrigin gives it an origin AS other than the prefix's.
-func benchAnnouncement(
-	rng *rand.Rand, prefix int, peer string, updates int, newOrigin bool,
-) benchUpdate {
-	origin := benchFirstASN + prefix%benchOrigins
-	if newOrigin {
-		origin += benchOrigins
-	}
-
-	start := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
-
-	return benchUpdate{
-		prefix: netip.MustParsePrefix(fmt.Sprintf("%d.%d.%d.0/%d",
-			1+prefix>>16, (prefix>>8)%256, prefix%256, benchMaskLength)),
-		origin: origin,
-		route: routetable.Route{
-			Peer: peer,
-			ASPath: []int{
-				benchFirstASN + rng.IntN(benchOrigins),
-				benchFirstASN + rng.IntN(benchOrigins),
-				benchFirstASN + rng.IntN(benchOrigins),
-				origin,
-			},
-			Time: start.Add(time.Duration(updates/benchUpdatesPerSecond) *
-				time.Second),
-		},
-	}
-}
-
-// apply applies update to table.
-func apply(table *routetable.Table, update benchUpdate) {
-	if update.withdraw {
-		table.Withdraw(update.prefix, update.route.Peer)
-
-		return
-	}
-
-	table.Announce(update.prefix, update.origin, update.route)
-}
-
-// heapBytes returns the bytes of the live heap objects, after a collection.
-func heapBytes() uint64 {
-	runtime.GC()
-
-	var stats runtime.MemStats
-
-	runtime.ReadMemStats(&stats)
-
-	return stats.HeapAlloc
-}

internal/routetable/reads.go

@@ -1,223 +0,0 @@
-package routetable
-
-import (
-	"net/netip"
-	"slices"
-	"time"
-
-	"github.com/bits-and-blooms/bitset"
-)
-
-// PrefixSummary is what the table answers about one prefix.
-type PrefixSummary struct {
-	Prefix netip.Prefix
-	// OriginASN is the origin AS that the most peers carry the prefix with.
-	// When several have as many peers, it is the lowest AS number.
-	OriginASN int
-	// Peers is how many peers carry the prefix.
-	Peers int
-	// Newest is the time of the newest kept route of the prefix, or nil when
-	// it keeps none.
-	Newest *time.Time
-}
-
-// OriginatedPrefix is one prefix an AS originates.
-type OriginatedPrefix struct {
-	Prefix netip.Prefix
-	// Newest is the time of the newest kept route of the prefix with that
-	// origin AS, or nil when it keeps none.
-	Newest *time.Time
-}
-
-// Counts are the table's figures for the stats.
-type Counts struct {
-	IPv4Prefixes int
-	IPv6Prefixes int
-	// IPv4PrefixesByLength[n] and IPv6PrefixesByLength[n] are how many
-	// prefixes of mask length n some peer carries.
-	IPv4PrefixesByLength []int
-	IPv6PrefixesByLength []int
-	// IPv4Routes and IPv6Routes count peer and prefix pairs: each peer that
-	// carries a prefix counts once for it.
-	IPv4Routes int
-	IPv6Routes int
-	// OriginsWithoutKeptRoute counts the prefix and origin AS pairs that peers
-	// carry but whose kept routes were all withdrawn.
-	OriginsWithoutKeptRoute int
-	// OldestKeptRoute and NewestKeptRoute are the times of the oldest and the
-	// newest kept routes, or nil when the table keeps none.
-	OldestKeptRoute *time.Time
-	NewestKeptRoute *time.Time
-}
-
-// Lookup returns the most specific prefix that holds addr and that some peer
-// carries. It ignores prefixes shorter than /8 for IPv4 and /16 for IPv6. It
-// makes one map lookup per mask length, and reports false when no prefix
-// holds addr.
-func (t *Table) Lookup(addr netip.Addr) (PrefixSummary, bool) {
-	t.mu.RLock()
-	defer t.mu.RUnlock()
-
-	// An IPv4 address written in IPv6 form (::ffff:192.0.2.1) is looked up as
-	// IPv4.
-	addr = addr.Unmap()
-	f := t.familyOf(addr)
-
-	for bits := addr.BitLen(); bits >= f.minLookupLength; bits-- {
-		prefix := netip.PrefixFrom(addr, bits).Masked()
-
-		origins, found := f.byLength[bits][prefix]
-		if found {
-			return summarize(prefix, origins), true
-		}
-	}
-
-	return PrefixSummary{}, false
-}
-
-// OriginatedPrefixes returns the prefixes AS asn originates, in prefix order.
-func (t *Table) OriginatedPrefixes(asn int) []OriginatedPrefix {
-	t.mu.RLock()
-	defer t.mu.RUnlock()
-
-	prefixes := make([]OriginatedPrefix, 0, len(t.byOrigin[asn]))
-	for prefix, o := range t.byOrigin[asn] {
-		prefixes = append(prefixes,
-			OriginatedPrefix{Prefix: prefix, Newest: o.newestTime()})
-	}
-
-	slices.SortFunc(prefixes, func(a, b OriginatedPrefix) int {
-		return a.Prefix.Compare(b.Prefix)
-	})
-
-	return prefixes
-}
-
-// Origins returns each origin AS of prefix, in AS number order, with its peers
-// and its kept routes. It returns nil when no peer carries the prefix.
-func (t *Table) Origins(prefix netip.Prefix) []Origin {
-	t.mu.RLock()
-	defer t.mu.RUnlock()
-
-	if !prefix.IsValid() {
-		return nil
-	}
-
-	var origins []Origin
-
-	for _, o := range t.origins(prefix.Masked()) {
-		origins = append(origins, Origin{
-			ASN:    o.asn,
-			Peers:  t.addresses(&o.peers),
-			Routes: t.exportRoutes(o.routes),
-		})
-	}
-
-	return origins
-}
-
-// SamplePrefixes returns up to limit of the prefixes of IP version ipVersion
-// (4 or 6) and mask length maskLength. It reads only the prefixes it returns.
-// Go starts each walk over a map at a random place, so each call returns
-// another sample.
-func (t *Table) SamplePrefixes(ipVersion, maskLength, limit int) []PrefixSummary {
-	t.mu.RLock()
-	defer t.mu.RUnlock()
-
-	var f *family
-
-	switch ipVersion {
-	case ipVersion4:
-		f = &t.ipv4
-	case ipVersion6:
-		f = &t.ipv6
-	default:
-		return nil
-	}
-
-	if maskLength < 0 || maskLength >= len(f.byLength) || limit <= 0 {
-		return nil
-	}
-
-	prefixes := f.byLength[maskLength]
-	sample := make([]PrefixSummary, 0, min(limit, len(prefixes)))
-
-	for prefix, origins := range prefixes {
-		if len(sample) == limit {
-			break
-		}
-
-		sample = append(sample, summarize(prefix, origins))
-	}
-
-	return sample
-}
-
-// Counts returns the table's figures for the stats. The writes keep them up to
-// date, so this reads no prefix.
-func (t *Table) Counts() Counts {
-	t.mu.RLock()
-	defer t.mu.RUnlock()
-
-	counts := Counts{
-		IPv4PrefixesByLength:    t.ipv4.prefixesByLength(),
-		IPv6PrefixesByLength:    t.ipv6.prefixesByLength(),
-		IPv4Routes:              t.ipv4.routes,
-		IPv6Routes:              t.ipv6.routes,
-		OriginsWithoutKeptRoute: t.originsWithoutKeptRoute,
-	}
-
-	for _, n := range counts.IPv4PrefixesByLength {
-		counts.IPv4Prefixes += n
-	}
-
-	for _, n := range counts.IPv6PrefixesByLength {
-		counts.IPv6Prefixes += n
-	}
-
-	oldest, found := t.keptTimes.Min()
-	if found {
-		counts.OldestKeptRoute = &oldest.time
-	}
-
-	newest, found := t.keptTimes.Max()
-	if found {
-		counts.NewestKeptRoute = &newest.time
-	}
-
-	return counts
-}
-
-// addresses returns the addresses of the peers in peers, in number order.
-func (t *Table) addresses(peers *bitset.BitSet) []string {
-	addresses := make([]string, 0, peers.Count())
-	for number := range peers.EachSet() {
-		addresses = append(addresses, t.peerAddresses[number])
-	}
-
-	return addresses
-}
-
-// summarize returns the summary of prefix, whose origins are in AS number
-// order.
-func summarize(prefix netip.Prefix, origins []*origin) PrefixSummary {
-	summary := PrefixSummary{Prefix: prefix}
-	mostPeers := 0
-
-	for _, o := range origins {
-		peers := int(o.peers.Count())
-		summary.Peers += peers
-
-		if peers > mostPeers {
-			mostPeers = peers
-			summary.OriginASN = o.asn
-		}
-
-		newest := o.newestTime()
-		if newest != nil && (summary.Newest == nil || newest.After(*summary.Newest)) {
-			summary.Newest = newest
-		}
-	}
-
-	return summary
-}

internal/routetable/reads_test.go

@@ -1,259 +0,0 @@
-package routetable_test
-
-import (
-	"net/netip"
-	"reflect"
-	"testing"
-
-	"git.eeqj.de/sneak/routewatch/internal/routetable"
-)
-
-// announceAll has each listed peer announce prefix with origin AS asn, the
-// peer at index i i*10 seconds after start.
-func announceAll(
-	table *routetable.Table, prefix string, asn, start int, peers ...string,
-) {
-	const secondsBetweenPeers = 10
-
-	for i, peer := range peers {
-		table.Announce(netip.MustParsePrefix(prefix), asn,
-			routeOf(peer, asn, start+i*secondsBetweenPeers))
-	}
-}
-
-func TestLookup(t *testing.T) {
-	t.Parallel()
-
-	table := routetable.New(routetable.Params{})
-	announceAll(table, "10.0.0.0/8", asA, 10, peerA)
-	// Three peers with two origins: asB has more of them.
-	announceAll(table, "10.1.0.0/16", asA, 10, peerA)
-	announceAll(table, "10.1.0.0/16", asB, 20, peerB, peerC)
-	// Two origins with one peer each: the lower AS number wins.
-	announceAll(table, prefixV4, asB, 20, peerB)
-	announceAll(table, prefixV4, asA, 10, peerA)
-	// An origin whose kept routes were withdrawn: no newest time.
-	announceAll(table, "10.9.0.0/16", asA, 10, peerA, peerB, peerC)
-	table.Withdraw(netip.MustParsePrefix("10.9.0.0/16"), peerB)
-	table.Withdraw(netip.MustParsePrefix("10.9.0.0/16"), peerC)
-	announceAll(table, "2001:db8::/32", asA, 10, peerA)
-	announceAll(table, prefixV6, asB, 20, peerB)
-	announceAll(table, "3000::/16", asA, 10, peerA)
-	// Shorter than the lookup uses.
-	announceAll(table, "192.0.0.0/7", asA, 10, peerA)
-	announceAll(table, "2000::/15", asA, 10, peerA)
-
-	tests := []struct {
-		addr string
-		// want is the summary's prefix, or empty when none is found.
-		want      string
-		originASN int
-		peers     int
-		newest    int // in seconds after the start; 0 for none
-	}{
-		{"10.1.2.3", prefixV4, asA, 2, 20},
-		{"::ffff:10.1.2.3", prefixV4, asA, 2, 20},
-		{"10.1.3.4", "10.1.0.0/16", asB, 3, 30},
-		{"10.2.0.1", "10.0.0.0/8", asA, 1, 10},
-		{"10.9.0.1", "10.9.0.0/16", asA, 1, 0},
-		{"192.0.2.1", "", 0, 0, 0},
-		{"2001:db8:1::5", prefixV6, asB, 1, 20},
-		{"2001:db8:2::1", "2001:db8::/32", asA, 1, 10},
-		{"3000::1", "3000::/16", asA, 1, 10},
-		{"2001:db9::1", "", 0, 0, 0},
-	}
-
-	for _, tc := range tests {
-		t.Run(tc.addr, func(t *testing.T) {
-			t.Parallel()
-
-			got, found := table.Lookup(netip.MustParseAddr(tc.addr))
-			if tc.want == "" {
-				if found {
-					t.Errorf("Lookup(%s) = %+v, want none", tc.addr, got)
-				}
-
-				return
-			}
-
-			want := routetable.PrefixSummary{
-				Prefix:    netip.MustParsePrefix(tc.want),
-				OriginASN: tc.originASN,
-				Peers:     tc.peers,
-			}
-			if tc.newest != 0 {
-				want.Newest = timeAt(tc.newest)
-			}
-
-			if !found || !reflect.DeepEqual(got, want) {
-				t.Errorf("Lookup(%s) = %+v, %v; want %+v", tc.addr, got, found, want)
-			}
-		})
-	}
-}
-
-func TestOriginatedPrefixes(t *testing.T) {
-	t.Parallel()
-
-	table := routetable.New(routetable.Params{})
-	announceAll(table, "2001:db8::/32", asA, 30, peerA)
-	announceAll(table, prefixV4, asA, 10, peerA)
-	// asB's newer route for the same prefix does not count for asA.
-	announceAll(table, prefixV4, asB, 40, peerB)
-	announceAll(table, "10.1.0.0/16", asA, 10, peerA, peerB)
-	announceAll(table, "10.9.0.0/16", asA, 10, peerA, peerB, peerC)
-	table.Withdraw(netip.MustParsePrefix("10.9.0.0/16"), peerB)
-	table.Withdraw(netip.MustParsePrefix("10.9.0.0/16"), peerC)
-
-	want := []routetable.OriginatedPrefix{
-		{Prefix: netip.MustParsePrefix("10.1.0.0/16"), Newest: timeAt(20)},
-		{Prefix: netip.MustParsePrefix(prefixV4), Newest: timeAt(10)},
-		{Prefix: netip.MustParsePrefix("10.9.0.0/16")},
-		{Prefix: netip.MustParsePrefix("2001:db8::/32"), Newest: timeAt(30)},
-	}
-
-	got := table.OriginatedPrefixes(asA)
-	if !reflect.DeepEqual(got, want) {
-		t.Errorf("OriginatedPrefixes(%d) = %+v, want %+v", asA, got, want)
-	}
-
-	got = table.OriginatedPrefixes(asC)
-	if len(got) != 0 {
-		t.Errorf("OriginatedPrefixes(%d) = %+v, want none", asC, got)
-	}
-}
-
-func TestSamplePrefixes(t *testing.T) {
-	t.Parallel()
-
-	table := routetable.New(routetable.Params{})
-
-	announced := []string{
-		"10.0.0.0/24", "10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24",
-		"10.0.4.0/24", "10.0.0.0/16",
-		"2001:db8::/48", "2001:db8:1::/48", "2001:db8:2::/48",
-	}
-	for _, prefix := range announced {
-		announceAll(table, prefix, asA, 10, peerA)
-	}
-
-	tests := []struct {
-		name                         string
-		ipVersion, maskLength, limit int
-		want                         int
-	}{
-		{"fewer than there are", 4, 24, 3, 3},
-		{"more than there are", 4, 24, 10, 5},
-		{"IPv6", 6, 48, 10, 3},
-		{"a mask length with none", 4, 20, 10, 0},
-		{"no such IP version", 5, 24, 10, 0},
-		{"a mask length too long", 4, 33, 10, 0},
-		{"a negative mask length", 4, -1, 10, 0},
-		{"a limit of zero", 4, 24, 0, 0},
-	}
-
-	for _, tc := range tests {
-		t.Run(tc.name, func(t *testing.T) {
-			t.Parallel()
-
-			sample := table.SamplePrefixes(tc.ipVersion, tc.maskLength, tc.limit)
-			if len(sample) != tc.want {
-				t.Fatalf("SamplePrefixes(%d, %d, %d) returned %d prefixes, want %d",
-					tc.ipVersion, tc.maskLength, tc.limit, len(sample), tc.want)
-			}
-
-			seen := make(map[netip.Prefix]bool)
-
-			for _, got := range sample {
-				want := routetable.PrefixSummary{
-					Prefix: got.Prefix, OriginASN: asA, Peers: 1, Newest: timeAt(10),
-				}
-				if seen[got.Prefix] || got.Prefix.Bits() != tc.maskLength ||
-					!reflect.DeepEqual(got, want) {
-					t.Errorf("SamplePrefixes(%d, %d, %d) returned %+v",
-						tc.ipVersion, tc.maskLength, tc.limit, got)
-				}
-
-				seen[got.Prefix] = true
-			}
-		})
-	}
-}
-
-func TestCounts(t *testing.T) {
-	t.Parallel()
-
-	table := routetable.New(routetable.Params{})
-	checkCounts(t, table, routetable.Counts{
-		IPv4PrefixesByLength: byLength(ipv4Bits, nil),
-		IPv6PrefixesByLength: byLength(ipv6Bits, nil),
-	})
-
-	announceAll(table, prefixV4, asA, 10, peerA, peerB)
-	announceAll(table, "10.2.0.0/16", asA, 15, peerA)
-	announceAll(table, "2001:db8::/32", asA, 5, peerA)
-	announceAll(table, "2001:db8::/32", asA, 25, peerC)
-	checkCounts(t, table, routetable.Counts{
-		IPv4Prefixes:         2,
-		IPv6Prefixes:         1,
-		IPv4PrefixesByLength: byLength(ipv4Bits, map[int]int{16: 1, 24: 1}),
-		IPv6PrefixesByLength: byLength(ipv6Bits, map[int]int{32: 1}),
-		IPv4Routes:           3,
-		IPv6Routes:           2,
-		OldestKeptRoute:      timeAt(5),
-		NewestKeptRoute:      timeAt(25),
-	})
-
-	// peerC's route drops peerA's from the kept ones, and peerA leaves the
-	// IPv6 prefix.
-	announceAll(table, prefixV4, asA, 30, peerC)
-	table.Withdraw(netip.MustParsePrefix("2001:db8::/32"), peerA)
-	checkCounts(t, table, routetable.Counts{
-		IPv4Prefixes:         2,
-		IPv6Prefixes:         1,
-		IPv4PrefixesByLength: byLength(ipv4Bits, map[int]int{16: 1, 24: 1}),
-		IPv6PrefixesByLength: byLength(ipv6Bits, map[int]int{32: 1}),
-		IPv4Routes:           4,
-		IPv6Routes:           1,
-		OldestKeptRoute:      timeAt(15),
-		NewestKeptRoute:      timeAt(30),
-	})
-
-	// peerC was the last peer of the IPv6 prefix.
-	table.PeerDown(peerC)
-	checkCounts(t, table, routetable.Counts{
-		IPv4Prefixes:         2,
-		IPv4PrefixesByLength: byLength(ipv4Bits, map[int]int{16: 1, 24: 1}),
-		IPv6PrefixesByLength: byLength(ipv6Bits, nil),
-		IPv4Routes:           3,
-		OldestKeptRoute:      timeAt(15),
-		NewestKeptRoute:      timeAt(20),
-	})
-}
-
-// The address lengths of the two IP versions.
-const (
-	ipv4Bits = 32
-	ipv6Bits = 128
-)
-
-// byLength returns the prefix counts by mask length, for addresses of bits
-// bits, with counts[n] prefixes of mask length n and none of any other.
-func byLength(bits int, counts map[int]int) []int {
-	all := make([]int, bits+1)
-	for n, count := range counts {
-		all[n] = count
-	}
-
-	return all
-}
-
-// checkCounts checks that the table's counts are want.
-func checkCounts(t *testing.T, table *routetable.Table, want routetable.Counts) {
-	t.Helper()
-
-	got := table.Counts()
-	if !reflect.DeepEqual(got, want) {
-		t.Errorf("Counts() = %+v, want %+v", got, want)
-	}
-}

internal/routetable/routetable.go

@@ -1,632 +0,0 @@
-// Package routetable holds the whole routing table in memory. For each prefix
-// and origin AS, it keeps the peers whose current route to the prefix has that
-// origin AS. It also keeps the two newest of those routes in full. It applies
-// announcements, withdrawals and peers' sessions going down. It answers the
-// route reads that the pages and the API need.
-package routetable
-
-import (
-	"cmp"
-	"net/netip"
-	"slices"
-	"sync"
-	"time"
-
-	"github.com/bits-and-blooms/bitset"
-	"github.com/google/btree"
-)
-
-const (
-	// keptRoutesPerOrigin is how many routes of each prefix and origin AS are
-	// kept in full: the newest ones. Every other peer that carries the prefix
-	// with that origin AS keeps only its bit in the peer set.
-	keptRoutesPerOrigin = 2
-
-	// keptTimesDegree is the degree of the B-tree that holds the times of the
-	// kept routes in order: each of its nodes holds up to 63 times.
-	keptTimesDegree = 32
-
-	ipv4Bits = 32
-	ipv6Bits = 128
-
-	// The address lookup ignores prefixes shorter than these mask lengths. A
-	// default route or a route this short only says where a peer sends traffic
-	// it has no better route for; it does not say who holds the address.
-	minLookupMaskLengthV4 = 8
-	minLookupMaskLengthV6 = 16
-
-	// The IP versions SamplePrefixes takes.
-	ipVersion4 = 4
-	ipVersion6 = 6
-)
-
-// Route is one peer's route to one prefix. The next hop is the peer itself, so
-// it is not stored.
-type Route struct {
-	// Peer is the peer's address.
-	Peer   string
-	ASPath []int
-	// Time is when the peer announced the route.
-	Time time.Time
-}
-
-// Origin is one origin AS of a prefix: the peers whose current route to the
-// prefix has that origin AS, and the newest of those routes in full.
-type Origin struct {
-	// ASN is the origin AS. 0 means no known origin: the path ends in an AS
-	// set of several members.
-	ASN int
-	// Peers are the peers' addresses, in the order of their numbers: the
-	// order they were first seen in.
-	Peers []string
-	// Routes are the kept routes, newest first: at most two, and none when
-	// both were withdrawn while other peers still carry the prefix.
-	Routes []Route
-}
-
-// Change is the current content of one prefix and origin AS pair that changed:
-// what a database row for the pair holds.
-type Change struct {
-	Prefix    netip.Prefix
-	OriginASN int
-	// Removed says that no peer carries the prefix with this origin AS any
-	// more. Peers and Routes are then nil.
-	Removed bool
-	// Peers holds the number of each peer that carries the prefix with this
-	// origin AS.
-	Peers *bitset.BitSet
-	// Routes are the kept routes, newest first.
-	Routes []Route
-}
-
-// Params configures New.
-type Params struct {
-	// PeerNumbers holds the number each peer was given before, by address. A
-	// peer not in it gets the next number above all of them the first time it
-	// announces a route.
-	PeerNumbers map[string]uint
-}
-
-// Table is the routing table. Its methods are safe for concurrent use.
-type Table struct {
-	mu sync.RWMutex
-
-	ipv4 family
-	ipv6 family
-	// byOrigin holds, for each origin AS, the prefixes it originates.
-	byOrigin map[int]map[netip.Prefix]*origin
-
-	// peerNumbers and peerAddresses give each peer's number by its address
-	// and its address by its number. nextPeerNumber is the number the next
-	// new peer gets.
-	peerNumbers    map[string]uint
-	peerAddresses  map[uint]string
-	nextPeerNumber uint
-	// peerPrefixes counts the prefixes each peer carries, by its number.
-	peerPrefixes map[uint]int
-
-	// newPeers and changed are what TakeChanges hands over next: the peers
-	// numbered, and the prefix and origin AS pairs changed, since it last ran.
-	newPeers map[string]uint
-	changed  map[pair]struct{}
-
-	// keptTimes counts the kept routes at each time, in time order, so its
-	// first and last items are the oldest and newest kept routes.
-	keptTimes *btree.BTreeG[keptTime]
-	// originsWithoutKeptRoute counts the origins that peers carry but that
-	// keep no route.
-	originsWithoutKeptRoute int
-}
-
-// New returns an empty table that numbers new peers on from
-// params.PeerNumbers.
-func New(params Params) *Table {
-	t := &Table{
-		ipv4:          newFamily(ipv4Bits, minLookupMaskLengthV4),
-		ipv6:          newFamily(ipv6Bits, minLookupMaskLengthV6),
-		byOrigin:      make(map[int]map[netip.Prefix]*origin),
-		peerNumbers:   make(map[string]uint, len(params.PeerNumbers)),
-		peerAddresses: make(map[uint]string, len(params.PeerNumbers)),
-		peerPrefixes:  make(map[uint]int),
-		newPeers:      make(map[string]uint),
-		changed:       make(map[pair]struct{}),
-		keptTimes: btree.NewG(keptTimesDegree, func(a, b keptTime) bool {
-			return a.time.Before(b.time)
-		}),
-	}
-
-	for address, number := range params.PeerNumbers {
-		t.peerNumbers[address] = number
-		t.peerAddresses[number] = address
-		t.nextPeerNumber = max(t.nextPeerNumber, number+1)
-	}
-
-	return t
-}
-
-// Announce applies an announcement: the peer's current route to prefix is now
-// r, with origin AS originASN. A peer that carried the prefix with another
-// origin AS leaves that origin's peer set, and its route there is dropped. r
-// is kept in full when it is one of the two newest routes of its prefix and
-// origin AS; otherwise only the peer's bit is.
-//
-// An announcement whose path is empty or holds AS 0 is a withdrawal, and the
-// caller passes it to Withdraw instead.
-func (t *Table) Announce(prefix netip.Prefix, originASN int, r Route) {
-	t.mu.Lock()
-	defer t.mu.Unlock()
-
-	if !prefix.IsValid() {
-		return
-	}
-
-	prefix = prefix.Masked()
-	peer := t.numberPeer(r.Peer)
-
-	current := originWithPeer(t.origins(prefix), peer)
-	if current != nil && current.asn != originASN {
-		t.leave(prefix, current, peer)
-	}
-
-	t.join(prefix, t.findOrAddOrigin(prefix, originASN), route{
-		peer: peer,
-		path: slices.Clone(r.ASPath),
-		time: r.Time,
-	})
-}
-
-// Withdraw applies a withdrawal: the peer no longer carries prefix. It leaves
-// the prefix's peer set, and its kept route, if any, is dropped.
-func (t *Table) Withdraw(prefix netip.Prefix, peer string) {
-	t.mu.Lock()
-	defer t.mu.Unlock()
-
-	number, known := t.peerNumbers[peer]
-	if !known || !prefix.IsValid() {
-		return
-	}
-
-	prefix = prefix.Masked()
-
-	o := originWithPeer(t.origins(prefix), number)
-	if o != nil {
-		t.leave(prefix, o, number)
-	}
-}
-
-// PeerDown applies a peer's session going down: the peer leaves every peer
-// set, and every route of it is dropped. When the peer carries any prefix, it
-// visits every prefix in the table.
-func (t *Table) PeerDown(peer string) {
-	t.mu.Lock()
-	defer t.mu.Unlock()
-
-	number, known := t.peerNumbers[peer]
-	if !known || t.peerPrefixes[number] == 0 {
-		return
-	}
-
-	for _, f := range []*family{&t.ipv4, &t.ipv6} {
-		for _, prefixes := range f.byLength {
-			for prefix, origins := range prefixes {
-				o := originWithPeer(origins, number)
-				if o != nil {
-					t.leave(prefix, o, number)
-				}
-			}
-		}
-	}
-}
-
-// Load puts one prefix and origin AS pair that the database holds into the
-// table, as startup does with each row it reads. The change's peers join the
-// pair's peer set, and its routes are kept as an announcement's would be.
-// Unlike an update, Load records no change, so TakeChanges does not hand the
-// pair back to be written again. Loading a pair twice changes nothing.
-//
-// The caller makes sure that every peer change names, in its peer set or in
-// its routes, has a number in Params.PeerNumbers.
-func (t *Table) Load(change Change) {
-	t.mu.Lock()
-	defer t.mu.Unlock()
-
-	// A removed pair has no peers, and a pair without peers is not in the
-	// table.
-	if change.Peers == nil || change.Peers.None() || !change.Prefix.IsValid() {
-		return
-	}
-
-	prefix := change.Prefix.Masked()
-	o := t.findOrAddOrigin(prefix, change.OriginASN)
-	wasWithoutKeptRoute := o.withoutKeptRoute()
-
-	for peer := range change.Peers.EachSet() {
-		if !o.peers.Test(peer) {
-			o.peers.Set(peer)
-
-			t.familyOf(prefix.Addr()).routes++
-			t.peerPrefixes[peer]++
-		}
-	}
-
-	// Oldest first, as they were announced, so that two routes with the same
-	// time are kept in the order they had.
-	for _, r := range slices.Backward(change.Routes) {
-		peer := t.peerNumbers[r.Peer]
-
-		t.dropKeptRoute(o, peer)
-		t.keepIfNewest(o, route{
-			peer: peer,
-			path: slices.Clone(r.ASPath),
-			time: r.Time,
-		})
-	}
-
-	t.countOriginWithoutKeptRoute(o, wasWithoutKeptRoute)
-}
-
-// TakeChanges returns the prefix and origin AS pairs that changed since it last
-// ran, in prefix order, each with its current content. It also returns the
-// peers numbered since then, by address. It then clears both records.
-func (t *Table) TakeChanges() ([]Change, map[string]uint) {
-	t.mu.Lock()
-	defer t.mu.Unlock()
-
-	changes := make([]Change, 0, len(t.changed))
-	for key := range t.changed {
-		changes = append(changes, t.currentContent(key))
-	}
-
-	slices.SortFunc(changes, func(a, b Change) int {
-		return cmp.Or(a.Prefix.Compare(b.Prefix),
-			cmp.Compare(a.OriginASN, b.OriginASN))
-	})
-
-	newPeers := t.newPeers
-	t.newPeers = make(map[string]uint)
-	t.changed = make(map[pair]struct{})
-
-	return changes, newPeers
-}
-
-// numberPeer returns the peer's number, and gives it the next one the first
-// time the table sees it.
-func (t *Table) numberPeer(address string) uint {
-	number, known := t.peerNumbers[address]
-	if known {
-		return number
-	}
-
-	number = t.nextPeerNumber
-	t.nextPeerNumber++
-	t.peerNumbers[address] = number
-	t.peerAddresses[number] = address
-	t.newPeers[address] = number
-
-	return number
-}
-
-// familyOf returns the part of the table for addr's IP version.
-func (t *Table) familyOf(addr netip.Addr) *family {
-	if addr.Is4() {
-		return &t.ipv4
-	}
-
-	return &t.ipv6
-}
-
-// prefixesOfLength returns the prefixes of the IP version and the mask length
-// of prefix, a valid masked prefix.
-func (t *Table) prefixesOfLength(prefix netip.Prefix) map[netip.Prefix][]*origin {
-	return t.familyOf(prefix.Addr()).byLength[prefix.Bits()]
-}
-
-// origins returns the origins of prefix, a valid masked prefix.
-func (t *Table) origins(prefix netip.Prefix) []*origin {
-	return t.prefixesOfLength(prefix)[prefix]
-}
-
-// findOrAddOrigin returns the origin of prefix with AS asn, added with no
-// peers when the prefix has none with that AS.
-func (t *Table) findOrAddOrigin(prefix netip.Prefix, asn int) *origin {
-	prefixes := t.prefixesOfLength(prefix)
-	origins := prefixes[prefix]
-
-	i, found := slices.BinarySearchFunc(origins, asn,
-		func(o *origin, asn int) int { return cmp.Compare(o.asn, asn) })
-	if found {
-		return origins[i]
-	}
-
-	o := &origin{asn: asn}
-	prefixes[prefix] = slices.Insert(origins, i, o)
-
-	if t.byOrigin[asn] == nil {
-		t.byOrigin[asn] = make(map[netip.Prefix]*origin)
-	}
-
-	t.byOrigin[asn][prefix] = o
-
-	return o
-}
-
-// removeOrigin removes o, which no peer carries any more, from prefix, and the
-// prefix from the table when it has no origin left.
-func (t *Table) removeOrigin(prefix netip.Prefix, o *origin) {
-	prefixes := t.prefixesOfLength(prefix)
-
-	origins := slices.DeleteFunc(prefixes[prefix],
-		func(other *origin) bool { return other == o })
-	if len(origins) == 0 {
-		delete(prefixes, prefix)
-	} else {
-		prefixes[prefix] = origins
-	}
-
-	delete(t.byOrigin[o.asn], prefix)
-
-	if len(t.byOrigin[o.asn]) == 0 {
-		delete(t.byOrigin, o.asn)
-	}
-}
-
-// join puts the peer of r in the peer set of o, an origin of prefix, and keeps
-// r in full when it is one of the two newest routes of o. The peer's earlier
-// route, if o keeps one, is replaced: it is no longer the peer's current route.
-// The pair is recorded as changed only when its peer set or kept routes change.
-func (t *Table) join(prefix netip.Prefix, o *origin, r route) {
-	if slices.ContainsFunc(o.routes, r.equal) {
-		// The peer announced its kept route again.
-		return
-	}
-
-	wasWithoutKeptRoute := o.withoutKeptRoute()
-
-	joined := !o.peers.Test(r.peer)
-	if joined {
-		o.peers.Set(r.peer)
-
-		t.familyOf(prefix.Addr()).routes++
-		t.peerPrefixes[r.peer]++
-	}
-
-	dropped := t.dropKeptRoute(o, r.peer)
-	kept := t.keepIfNewest(o, r)
-
-	if joined || dropped || kept {
-		t.recordChange(prefix, o, wasWithoutKeptRoute)
-	}
-}
-
-// leave takes peer out of the peer set of o, an origin of prefix, and drops
-// its kept route, if any.
-func (t *Table) leave(prefix netip.Prefix, o *origin, peer uint) {
-	wasWithoutKeptRoute := o.withoutKeptRoute()
-
-	o.peers.Clear(peer)
-	t.dropKeptRoute(o, peer)
-
-	t.familyOf(prefix.Addr()).routes--
-	t.peerPrefixes[peer]--
-
-	t.recordChange(prefix, o, wasWithoutKeptRoute)
-}
-
-// keepIfNewest keeps r in o unless both kept routes of o are newer, and says
-// whether it kept r. When o already keeps two routes, the older one is
-// dropped, and its peer keeps only its bit.
-func (t *Table) keepIfNewest(o *origin, r route) bool {
-	// The kept routes stay newest first: r goes before the first one that is
-	// not newer than it.
-	i := 0
-	for i < len(o.routes) && o.routes[i].time.After(r.time) {
-		i++
-	}
-
-	if i == keptRoutesPerOrigin {
-		return false
-	}
-
-	if len(o.routes) == keptRoutesPerOrigin {
-		t.dropKeptRoute(o, o.routes[keptRoutesPerOrigin-1].peer)
-	}
-
-	o.routes = slices.Insert(o.routes, i, r)
-	t.countKeptTime(r.time, 1)
-
-	return true
-}
-
-// dropKeptRoute drops the peer's route from the kept routes of o, if o keeps
-// one, and says whether it did.
-func (t *Table) dropKeptRoute(o *origin, peer uint) bool {
-	i := slices.IndexFunc(o.routes, func(r route) bool { return r.peer == peer })
-	if i < 0 {
-		return false
-	}
-
-	t.countKeptTime(o.routes[i].time, -1)
-	o.routes = slices.Delete(o.routes, i, i+1)
-
-	return true
-}
-
-// recordChange records that the pair of prefix and the AS of o changed, and
-// brings the count of origins without a kept route up to date.
-// wasWithoutKeptRoute says whether o was such an origin before the change. An
-// origin that no peer carries any more is removed.
-func (t *Table) recordChange(
-	prefix netip.Prefix, o *origin, wasWithoutKeptRoute bool,
-) {
-	t.changed[pair{prefix: prefix, asn: o.asn}] = struct{}{}
-
-	t.countOriginWithoutKeptRoute(o, wasWithoutKeptRoute)
-
-	if o.peers.None() {
-		t.removeOrigin(prefix, o)
-	}
-}
-
-// countOriginWithoutKeptRoute brings the count of origins without a kept route
-// up to date after a change to o. wasWithoutKeptRoute says whether o was such
-// an origin before the change.
-func (t *Table) countOriginWithoutKeptRoute(o *origin, wasWithoutKeptRoute bool) {
-	if wasWithoutKeptRoute {
-		t.originsWithoutKeptRoute--
-	}
-
-	if o.withoutKeptRoute() {
-		t.originsWithoutKeptRoute++
-	}
-}
-
-// countKeptTime adds delta to the number of kept routes with time at.
-func (t *Table) countKeptTime(at time.Time, delta int) {
-	count, _ := t.keptTimes.Get(keptTime{time: at})
-	count.time = at
-	count.routes += delta
-
-	if count.routes == 0 {
-		t.keptTimes.Delete(count)
-
-		return
-	}
-
-	t.keptTimes.ReplaceOrInsert(count)
-}
-
-// currentContent returns what the table holds for the pair now.
-func (t *Table) currentContent(key pair) Change {
-	o := t.byOrigin[key.asn][key.prefix]
-	if o == nil {
-		return Change{Prefix: key.prefix, OriginASN: key.asn, Removed: true}
-	}
-
-	return Change{
-		Prefix:    key.prefix,
-		OriginASN: key.asn,
-		Peers:     o.peers.Clone(),
-		Routes:    t.exportRoutes(o.routes),
-	}
-}
-
-// exportRoutes returns copies of routes that name each peer by its address, or
-// nil when there are none.
-func (t *Table) exportRoutes(routes []route) []Route {
-	if len(routes) == 0 {
-		return nil
-	}
-
-	exported := make([]Route, 0, len(routes))
-	for _, r := range routes {
-		exported = append(exported, Route{
-			Peer:   t.peerAddresses[r.peer],
-			ASPath: slices.Clone(r.path),
-			Time:   r.time,
-		})
-	}
-
-	return exported
-}
-
-// family is the part of the table for one IP version.
-type family struct {
-	// byLength[n] holds the prefixes of mask length n that some peer carries,
-	// each with its origins in AS number order.
-	byLength []map[netip.Prefix][]*origin
-	// minLookupLength is the shortest mask length the address lookup uses.
-	minLookupLength int
-	// routes counts the peer and prefix pairs: each peer that carries a
-	// prefix counts once for it.
-	routes int
-}
-
-// newFamily returns an empty family for addresses of bits bits.
-func newFamily(bits, minLookupLength int) family {
-	f := family{
-		byLength:        make([]map[netip.Prefix][]*origin, bits+1),
-		minLookupLength: minLookupLength,
-	}
-
-	for n := range f.byLength {
-		f.byLength[n] = make(map[netip.Prefix][]*origin)
-	}
-
-	return f
-}
-
-// prefixesByLength returns how many prefixes of each mask length f holds.
-func (f *family) prefixesByLength() []int {
-	counts := make([]int, len(f.byLength))
-	for n, prefixes := range f.byLength {
-		counts[n] = len(prefixes)
-	}
-
-	return counts
-}
-
-// origin is what the table holds for one prefix and origin AS.
-type origin struct {
-	asn int
-	// peers holds the number of each peer whose current route to the prefix
-	// has this origin AS.
-	peers bitset.BitSet
-	// routes are the newest of those routes, newest first, at most
-	// keptRoutesPerOrigin.
-	routes []route
-}
-
-// withoutKeptRoute says whether peers carry o but o keeps no route.
-func (o *origin) withoutKeptRoute() bool {
-	return o.peers.Any() && len(o.routes) == 0
-}
-
-// newestTime returns the time of the newest kept route of o, or nil when o
-// keeps none.
-func (o *origin) newestTime() *time.Time {
-	if len(o.routes) == 0 {
-		return nil
-	}
-
-	newest := o.routes[0].time
-
-	return &newest
-}
-
-// originWithPeer returns the origin whose peer set holds peer, or nil when
-// none does. A peer carries a prefix with one origin AS at most.
-func originWithPeer(origins []*origin, peer uint) *origin {
-	for _, o := range origins {
-		if o.peers.Test(peer) {
-			return o
-		}
-	}
-
-	return nil
-}
-
-// route is a kept route. peer is the peer's number.
-type route struct {
-	peer uint
-	path []int
-	time time.Time
-}
-
-// equal says whether r and other are the same peer's route with the same path
-// and time.
-func (r route) equal(other route) bool {
-	return r.peer == other.peer && r.time.Equal(other.time) &&
-		slices.Equal(r.path, other.path)
-}
-
-// pair names one prefix and origin AS.
-type pair struct {
-	prefix netip.Prefix
-	asn    int
-}
-
-// keptTime counts the kept routes with one time.
-type keptTime struct {
-	time   time.Time
-	routes int
-}

internal/routetable/routetable_test.go

@@ -1,590 +0,0 @@
-package routetable_test
-
-import (
-	"net/netip"
-	"reflect"
-	"slices"
-	"sync"
-	"testing"
-	"time"
-
-	"github.com/bits-and-blooms/bitset"
-
-	"git.eeqj.de/sneak/routewatch/internal/routetable"
-)
-
-const (
-	peerA = "192.0.2.1"
-	peerB = "192.0.2.2"
-	peerC = "2001:db8::3"
-	peerD = "2001:db8::4"
-
-	asA = 64500
-	asB = 64501
-	asC = 64502
-	// transitAS is the first AS of every test path, the peer's own.
-	transitAS = 64496
-
-	prefixV4 = "10.1.2.0/24"
-	prefixV6 = "2001:db8:1::/48"
-)
-
-// at returns the time second seconds after the start of every test.
-func at(second int) time.Time {
-	return time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC).
-		Add(time.Duration(second) * time.Second)
-}
-
-// timeAt returns at(second) for a field that may be nil.
-func timeAt(second int) *time.Time {
-	t := at(second)
-
-	return &t
-}
-
-// routeOf returns the route peer announces with origin AS asn second seconds
-// after the start.
-func routeOf(peer string, asn, second int) routetable.Route {
-	return routetable.Route{
-		Peer: peer, ASPath: []int{transitAS, asn}, Time: at(second),
-	}
-}
-
-// step is one update a test applies to a table for one prefix.
-type step func(table *routetable.Table, prefix netip.Prefix)
-
-// announce is the step of peer announcing prefix with origin AS asn second
-// seconds after the start.
-func announce(peer string, asn, second int) step {
-	return func(table *routetable.Table, prefix netip.Prefix) {
-		table.Announce(prefix, asn, routeOf(peer, asn, second))
-	}
-}
-
-// withdraw is the step of peer withdrawing prefix.
-func withdraw(peer string) step {
-	return func(table *routetable.Table, prefix netip.Prefix) {
-		table.Withdraw(prefix, peer)
-	}
-}
-
-// peerDown is the step of peer's session going down.
-func peerDown(peer string) step {
-	return func(table *routetable.Table, _ netip.Prefix) {
-		table.PeerDown(peer)
-	}
-}
-
-// rule is one case of the rule that decides what the table holds for a prefix:
-// the steps applied to an empty table, and the prefix's origins after them.
-// The peers are numbered in the order they first announce, which is the order
-// Origin.Peers lists them in.
-type rule struct {
-	name  string
-	steps []step
-	want  []routetable.Origin
-}
-
-// checkRules runs each case for an IPv4 and an IPv6 prefix.
-func checkRules(t *testing.T, rules []rule) {
-	t.Helper()
-
-	for _, tc := range rules {
-		for _, text := range []string{prefixV4, prefixV6} {
-			t.Run(tc.name+" "+text, func(t *testing.T) {
-				t.Parallel()
-
-				prefix := netip.MustParsePrefix(text)
-				table := routetable.New(routetable.Params{})
-
-				for _, apply := range tc.steps {
-					apply(table, prefix)
-				}
-
-				got := table.Origins(prefix)
-				if !reflect.DeepEqual(got, tc.want) {
-					t.Errorf("Origins(%s) = %+v, want %+v", prefix, got, tc.want)
-				}
-			})
-		}
-	}
-}
-
-func TestAnnouncementsKeepTheTwoNewestRoutes(t *testing.T) {
-	t.Parallel()
-
-	checkRules(t, []rule{
-		{
-			name:  "an announcement is kept",
-			steps: []step{announce(peerA, asA, 10)},
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA},
-				Routes: []routetable.Route{routeOf(peerA, asA, 10)}}},
-		},
-		{
-			name: "a lagging peer's older route is not kept",
-			steps: []step{announce(peerA, asA, 20), announce(peerB, asA, 30),
-				announce(peerC, asA, 10)},
-			want: []routetable.Origin{{ASN: asA,
-				Peers: []string{peerA, peerB, peerC}, Routes: []routetable.Route{
-					routeOf(peerB, asA, 30), routeOf(peerA, asA, 20),
-				}}},
-		},
-		{
-			name: "a third newer route drops the older kept one, whose peer " +
-				"keeps its bit",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				announce(peerC, asA, 30)},
-			want: []routetable.Origin{{ASN: asA,
-				Peers: []string{peerA, peerB, peerC}, Routes: []routetable.Route{
-					routeOf(peerC, asA, 30), routeOf(peerB, asA, 20),
-				}}},
-		},
-		{
-			name: "a route as old as the older kept one replaces it",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				announce(peerC, asA, 10)},
-			want: []routetable.Origin{{ASN: asA,
-				Peers: []string{peerA, peerB, peerC}, Routes: []routetable.Route{
-					routeOf(peerB, asA, 20), routeOf(peerC, asA, 10),
-				}}},
-		},
-		{
-			name: "a peer's new route replaces its kept one",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				announce(peerA, asA, 30)},
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA, peerB},
-				Routes: []routetable.Route{
-					routeOf(peerA, asA, 30), routeOf(peerB, asA, 20),
-				}}},
-		},
-		{
-			name: "a peer's new route replaces its kept one even when older",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				announce(peerB, asA, 5)},
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA, peerB},
-				Routes: []routetable.Route{
-					routeOf(peerA, asA, 10), routeOf(peerB, asA, 5),
-				}}},
-		},
-	})
-}
-
-func TestAPeerCarriesAPrefixWithOneOrigin(t *testing.T) {
-	t.Parallel()
-
-	checkRules(t, []rule{
-		{
-			name: "a peer moving to a new origin leaves the old one",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				announce(peerA, asB, 30)},
-			want: []routetable.Origin{
-				{ASN: asA, Peers: []string{peerB},
-					Routes: []routetable.Route{routeOf(peerB, asA, 20)}},
-				{ASN: asB, Peers: []string{peerA},
-					Routes: []routetable.Route{routeOf(peerA, asB, 30)}},
-			},
-		},
-		{
-			name: "a peer whose route is not kept moves to a new origin",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				announce(peerC, asA, 30), announce(peerA, asB, 40)},
-			want: []routetable.Origin{
-				{ASN: asA, Peers: []string{peerB, peerC},
-					Routes: []routetable.Route{
-						routeOf(peerC, asA, 30), routeOf(peerB, asA, 20),
-					}},
-				{ASN: asB, Peers: []string{peerA},
-					Routes: []routetable.Route{routeOf(peerA, asB, 40)}},
-			},
-		},
-		{
-			name: "origins are listed in AS number order",
-			steps: []step{announce(peerA, asC, 10), announce(peerB, asA, 20),
-				announce(peerC, asB, 30)},
-			want: []routetable.Origin{
-				{ASN: asA, Peers: []string{peerB},
-					Routes: []routetable.Route{routeOf(peerB, asA, 20)}},
-				{ASN: asB, Peers: []string{peerC},
-					Routes: []routetable.Route{routeOf(peerC, asB, 30)}},
-				{ASN: asC, Peers: []string{peerA},
-					Routes: []routetable.Route{routeOf(peerA, asC, 10)}},
-			},
-		},
-	})
-}
-
-func TestWithdrawalsAndPeerDowns(t *testing.T) {
-	t.Parallel()
-
-	// threePeers is peerA, peerB and peerC announcing in turn; the routes of
-	// peerC and peerB are kept.
-	threePeers := []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-		announce(peerC, asA, 30)}
-
-	checkRules(t, []rule{
-		{
-			name:  "a withdrawal drops the peer's kept route",
-			steps: append(slices.Clone(threePeers), withdraw(peerC)),
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA, peerB},
-				Routes: []routetable.Route{routeOf(peerB, asA, 20)}}},
-		},
-		{
-			name:  "a withdrawal of a route that is not kept leaves the kept ones",
-			steps: append(slices.Clone(threePeers), withdraw(peerA)),
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerB, peerC},
-				Routes: []routetable.Route{
-					routeOf(peerC, asA, 30), routeOf(peerB, asA, 20),
-				}}},
-		},
-		{
-			name: "an origin whose kept routes are withdrawn keeps its other " +
-				"peers",
-			steps: append(slices.Clone(threePeers), withdraw(peerB),
-				withdraw(peerC)),
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA}}},
-		},
-		{
-			name: "an origin without kept routes keeps the next update",
-			steps: append(slices.Clone(threePeers), withdraw(peerB),
-				withdraw(peerC), announce(peerA, asA, 5)),
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA},
-				Routes: []routetable.Route{routeOf(peerA, asA, 5)}}},
-		},
-		{
-			name: "an origin no peer carries is removed",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asB, 20),
-				withdraw(peerA)},
-			want: []routetable.Origin{{ASN: asB, Peers: []string{peerB},
-				Routes: []routetable.Route{routeOf(peerB, asB, 20)}}},
-		},
-		{
-			name:  "a prefix no peer carries is removed",
-			steps: []step{announce(peerA, asA, 10), withdraw(peerA)},
-			want:  nil,
-		},
-		{
-			name:  "a withdrawal from a peer that does not carry it changes nothing",
-			steps: []step{announce(peerA, asA, 10), withdraw(peerB)},
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA},
-				Routes: []routetable.Route{routeOf(peerA, asA, 10)}}},
-		},
-		{
-			name: "a peer going down leaves the prefix",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				peerDown(peerB)},
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA},
-				Routes: []routetable.Route{routeOf(peerA, asA, 10)}}},
-		},
-		{
-			name: "a peer going down after it withdrew changes nothing",
-			steps: []step{announce(peerA, asA, 10), announce(peerB, asA, 20),
-				withdraw(peerB), peerDown(peerB)},
-			want: []routetable.Origin{{ASN: asA, Peers: []string{peerA},
-				Routes: []routetable.Route{routeOf(peerA, asA, 10)}}},
-		},
-	})
-}
-
-func TestPeerDownLeavesEveryPrefix(t *testing.T) {
-	t.Parallel()
-
-	table := routetable.New(routetable.Params{})
-	prefixes := []netip.Prefix{
-		netip.MustParsePrefix(prefixV4),
-		netip.MustParsePrefix("10.2.0.0/16"),
-		netip.MustParsePrefix(prefixV6),
-	}
-
-	// peerA carries every prefix with origin asA, peerB the first one with
-	// origin asB.
-	for _, prefix := range prefixes {
-		table.Announce(prefix, asA, routeOf(peerA, asA, 10))
-	}
-
-	table.Announce(prefixes[0], asB, routeOf(peerB, asB, 20))
-	table.PeerDown(peerA)
-
-	want := []routetable.Origin{{ASN: asB, Peers: []string{peerB},
-		Routes: []routetable.Route{routeOf(peerB, asB, 20)}}}
-
-	got := table.Origins(prefixes[0])
-	if !reflect.DeepEqual(got, want) {
-		t.Errorf("Origins(%s) = %+v, want %+v", prefixes[0], got, want)
-	}
-
-	for _, prefix := range prefixes[1:] {
-		got := table.Origins(prefix)
-		if got != nil {
-			t.Errorf("Origins(%s) = %+v after the only peer went down, want nil",
-				prefix, got)
-		}
-	}
-
-	counts := table.Counts()
-	if counts.IPv4Prefixes != 1 || counts.IPv6Prefixes != 0 ||
-		counts.IPv4Routes != 1 || counts.IPv6Routes != 0 {
-		t.Errorf("Counts() = %+v, want one IPv4 prefix and route", counts)
-	}
-
-	if len(table.OriginatedPrefixes(asA)) != 0 {
-		t.Errorf("OriginatedPrefixes(%d) = %+v, want none",
-			asA, table.OriginatedPrefixes(asA))
-	}
-}
-
-func TestCountsOriginsWithoutKeptRoute(t *testing.T) {
-	t.Parallel()
-
-	prefix := netip.MustParsePrefix(prefixV4)
-	table := routetable.New(routetable.Params{})
-
-	steps := []struct {
-		apply step
-		want  int
-	}{
-		{announce(peerA, asA, 10), 0},
-		{announce(peerB, asA, 20), 0},
-		{announce(peerC, asA, 30), 0},
-		{withdraw(peerC), 0},
-		{withdraw(peerB), 1},
-		// An update from the last peer is kept.
-		{announce(peerA, asA, 40), 0},
-		{withdraw(peerA), 0},
-		// Once more without kept routes, then without peers: the origin
-		// is removed.
-		{announce(peerA, asA, 10), 0},
-		{announce(peerB, asA, 20), 0},
-		{announce(peerC, asA, 30), 0},
-		{withdraw(peerB), 0},
-		{withdraw(peerC), 1},
-		{peerDown(peerA), 0},
-	}
-
-	for i, s := range steps {
-		s.apply(table, prefix)
-
-		got := table.Counts().OriginsWithoutKeptRoute
-		if got != s.want {
-			t.Errorf("after step %d, OriginsWithoutKeptRoute = %d, want %d",
-				i, got, s.want)
-		}
-	}
-}
-
-// change is a routetable.Change with its peer set as a list of peer numbers.
-type change struct {
-	Prefix    string
-	OriginASN int
-	Removed   bool
-	Peers     []uint
-	Routes    []routetable.Route
-}
-
-// peerNumbers returns the numbers in set, or nil for a nil set.
-func peerNumbers(set *bitset.BitSet) []uint {
-	if set == nil {
-		return nil
-	}
-
-	return slices.Collect(set.EachSet())
-}
-
-func TestTakeChanges(t *testing.T) {
-	t.Parallel()
-
-	// peerA was numbered 5 before; the next new peer is 6.
-	table := routetable.New(routetable.Params{
-		PeerNumbers: map[string]uint{peerA: 5},
-	})
-	v4 := netip.MustParsePrefix(prefixV4)
-	v6 := netip.MustParsePrefix(prefixV6)
-
-	table.Announce(v6, asA, routeOf(peerA, asA, 10))
-	table.Announce(v4, asA, routeOf(peerA, asA, 10))
-	table.Announce(v4, asA, routeOf(peerB, asA, 20))
-
-	checkChanges(t, table, []change{
-		{Prefix: prefixV4, OriginASN: asA, Peers: []uint{5, 6},
-			Routes: []routetable.Route{
-				routeOf(peerB, asA, 20), routeOf(peerA, asA, 10),
-			}},
-		{Prefix: prefixV6, OriginASN: asA, Peers: []uint{5},
-			Routes: []routetable.Route{routeOf(peerA, asA, 10)}},
-	}, map[string]uint{peerB: 6})
-
-	// TakeChanges cleared the record.
-	checkChanges(t, table, []change{}, map[string]uint{})
-
-	// peerB moves to asB; peerA withdraws, which removes the pair of v4 and
-	// asA.
-	table.Announce(v4, asB, routeOf(peerB, asB, 30))
-	table.Withdraw(v4, peerA)
-
-	checkChanges(t, table, []change{
-		{Prefix: prefixV4, OriginASN: asA, Removed: true},
-		{Prefix: prefixV4, OriginASN: asB, Peers: []uint{6},
-			Routes: []routetable.Route{routeOf(peerB, asB, 30)}},
-	}, map[string]uint{})
-}
-
-func TestTakeChangesHoldsOnlyPairsThatChanged(t *testing.T) {
-	t.Parallel()
-
-	table := routetable.New(routetable.Params{})
-	v4 := netip.MustParsePrefix(prefixV4)
-	v6 := netip.MustParsePrefix(prefixV6)
-
-	// peerA, peerB and peerC are numbered 0, 1 and 2. The routes of peerC and
-	// peerB to v4 are kept; peerA keeps only its bit.
-	announceAll(table, prefixV4, asA, 10, peerA, peerB, peerC)
-	table.Announce(v6, asA, routeOf(peerB, asA, 10))
-	table.TakeChanges()
-
-	// peerA announces again, with a time older than both kept routes.
-	table.Announce(v4, asA, routeOf(peerA, asA, 15))
-	checkChanges(t, table, []change{}, map[string]uint{})
-
-	// peerC announces its kept route again.
-	table.Announce(v4, asA, routeOf(peerC, asA, 30))
-	checkChanges(t, table, []change{}, map[string]uint{})
-
-	// peerC announces the same path with a newer time, so its kept route
-	// changes.
-	table.Announce(v4, asA, routeOf(peerC, asA, 40))
-	checkChanges(t, table, []change{
-		{Prefix: prefixV4, OriginASN: asA, Peers: []uint{0, 1, 2},
-			Routes: []routetable.Route{
-				routeOf(peerC, asA, 40), routeOf(peerB, asA, 20),
-			}},
-	}, map[string]uint{})
-
-	// peerA withdraws v6, which it does not carry.
-	table.Withdraw(v6, peerA)
-	checkChanges(t, table, []change{}, map[string]uint{})
-
-	// peerA going down changes v4, the one prefix it carries.
-	table.PeerDown(peerA)
-	checkChanges(t, table, []change{
-		{Prefix: prefixV4, OriginASN: asA, Peers: []uint{1, 2},
-			Routes: []routetable.Route{
-				routeOf(peerC, asA, 40), routeOf(peerB, asA, 20),
-			}},
-	}, map[string]uint{})
-
-	// peerA, which carries nothing now, goes down again.
-	table.PeerDown(peerA)
-	checkChanges(t, table, []change{}, map[string]uint{})
-}
-
-func TestLoad(t *testing.T) {
-	t.Parallel()
-
-	const otherV4 = "10.9.0.0/16"
-
-	v4 := netip.MustParsePrefix(prefixV4)
-	v6 := netip.MustParsePrefix(prefixV6)
-	other := netip.MustParsePrefix(otherV4)
-
-	// The routes of peerC and peerB to v4 are kept; peerA keeps only its bit.
-	// The two routes to v6 have the same time. The routes of other were
-	// withdrawn while peerA still carries it.
-	source := routetable.New(routetable.Params{})
-	announceAll(source, prefixV4, asA, 10, peerA, peerB, peerC)
-	announceAll(source, prefixV4, asB, 40, peerD)
-	source.Announce(v6, asA, routeOf(peerA, asA, 50))
-	source.Announce(v6, asA, routeOf(peerC, asA, 50))
-	announceAll(source, otherV4, asB, 10, peerA, peerB, peerC)
-	source.Withdraw(other, peerB)
-	source.Withdraw(other, peerC)
-
-	changes, peers := source.TakeChanges()
-	loaded := routetable.New(routetable.Params{PeerNumbers: peers})
-
-	// Loading a pair twice changes nothing.
-	for range 2 {
-		for _, change := range changes {
-			loaded.Load(change)
-		}
-	}
-
-	for _, prefix := range []netip.Prefix{v4, v6, other} {
-		got, want := loaded.Origins(prefix), source.Origins(prefix)
-		if !reflect.DeepEqual(got, want) {
-			t.Errorf("Origins(%s) = %+v after the load, want %+v", prefix, got, want)
-		}
-	}
-
-	if !reflect.DeepEqual(loaded.Counts(), source.Counts()) {
-		t.Errorf("Counts() = %+v after the load, want %+v",
-			loaded.Counts(), source.Counts())
-	}
-
-	checkChanges(t, loaded, []change{}, map[string]uint{})
-}
-
-// TestConcurrentUse has goroutines update one table while others read it. The
-// race detector, which make test runs, fails it when a method uses the table
-// without its lock.
-func TestConcurrentUse(t *testing.T) {
-	t.Parallel()
-
-	const (
-		// rounds is how many times each goroutine calls its method.
-		rounds     = 1000
-		ipVersion  = 4
-		sampleSize = 10
-	)
-
-	table := routetable.New(routetable.Params{})
-	prefix := netip.MustParsePrefix(prefixV4)
-
-	uses := []func(round int){
-		func(round int) { table.Announce(prefix, asA, routeOf(peerA, asA, round)) },
-		func(round int) { table.Announce(prefix, asB, routeOf(peerB, asB, round)) },
-		func(int) { table.Withdraw(prefix, peerA) },
-		func(int) { table.PeerDown(peerB) },
-		func(int) { table.TakeChanges() },
-		func(int) { table.Lookup(prefix.Addr()) },
-		func(int) { table.Origins(prefix) },
-		func(int) { table.OriginatedPrefixes(asA) },
-		func(int) { table.SamplePrefixes(ipVersion, prefix.Bits(), sampleSize) },
-		func(int) { table.Counts() },
-	}
-
-	var wg sync.WaitGroup
-	for _, use := range uses {
-		wg.Go(func() {
-			for round := range rounds {
-				use(round)
-			}
-		})
-	}
-
-	wg.Wait()
-}
-
-// checkChanges checks that TakeChanges returns want and wantPeers.
-func checkChanges(
-	t *testing.T, table *routetable.Table, want []change,
-	wantPeers map[string]uint,
-) {
-	t.Helper()
-
-	changes, newPeers := table.TakeChanges()
-
-	got := make([]change, 0, len(changes))
-	for _, c := range changes {
-		got = append(got, change{
-			Prefix: c.Prefix.String(), OriginASN: c.OriginASN, Removed: c.Removed,
-			Peers: peerNumbers(c.Peers), Routes: c.Routes,
-		})
-	}
-
-	if !reflect.DeepEqual(got, want) {
-		t.Errorf("TakeChanges() changes = %+v, want %+v", got, want)
-	}
-
-	if !reflect.DeepEqual(newPeers, wantPeers) {
-		t.Errorf("TakeChanges() new peers = %v, want %v", newPeers, wantPeers)
-	}
-}

internal/routewatch/app.go

@@ -8,11 +8,9 @@ package routewatch
 import (
 	"context"
 	"errors"
-	"path/filepath"
 	"sync"
 	"time"
 
-	"git.eeqj.de/sneak/routewatch/internal/blocklist"
 	"git.eeqj.de/sneak/routewatch/internal/config"
 	"git.eeqj.de/sneak/routewatch/internal/database"
 	"git.eeqj.de/sneak/routewatch/internal/logger"
@@ -54,13 +52,8 @@ type RouteWatch struct {
 	peeringHandler *PeeringHandler
 	asnFetcher     *ASNFetcher
 	dbMaintainer   *DBMaintainer
-	blocklists     *blocklist.Blocklists
 }
 
-// blocklistsDirName is the directory in the state directory that holds the
-// blocklist copies.
-const blocklistsDirName = "blocklists"
-
 // New creates a new RouteWatch instance
 func New(deps Dependencies) *RouteWatch {
 	rw := &RouteWatch{
@@ -123,30 +116,17 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
 		return err
 	}
 
-	// Start ASN WHOIS fetcher for background updates. The HTTP handlers read
-	// the server's fetcher without a lock, so it is set before they run.
-	rw.asnFetcher = NewASNFetcher(rw.db, rw.logger.Logger)
-	rw.asnFetcher.Start(ctx)
-	rw.server.SetASNFetcher(rw.asnFetcher)
-
-	// Load the blocklist copies from disk and start downloading the lists.
-	// The HTTP handlers read the server's blocklists without a lock, so they
-	// are set before the handlers run.
-	blocklists := blocklist.New(blocklist.Params{
-		Dir:    filepath.Join(rw.config.StateDir, blocklistsDirName),
-		Lists:  blocklist.Lists(),
-		Logger: rw.logger.Logger,
-	})
-	blocklists.Start(ctx)
-	rw.blocklists = blocklists
-	rw.server.SetBlocklists(blocklists)
-
 	// Start HTTP server
 	err = rw.server.Start(ctx)
 	if err != nil {
 		return err
 	}
 
+	// Start ASN WHOIS fetcher for background updates
+	rw.asnFetcher = NewASNFetcher(rw.db, rw.logger.Logger)
+	rw.asnFetcher.Start(ctx)
+	rw.server.SetASNFetcher(rw.asnFetcher)
+
 	// Start database maintenance goroutine
 	rw.dbMaintainer = NewDBMaintainer(rw.db, rw.logger.Logger)
 	rw.dbMaintainer.Start(ctx)
@@ -157,13 +137,7 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
 	return nil
 }
 
-// Shutdown stops all services in order: the streamer, the handlers, which
+// Shutdown performs graceful shutdown of all services
-// write what they hold, the WHOIS fetcher, the database maintainer, the
-// blocklist downloads and the HTTP server, and last the database, which it
-// checkpoints and closes. Each stage waits at most until ctx ends, so
-// Shutdown returns soon after: a handler logs what it could not write by
-// then, and the HTTP server closes the connections of requests still
-// running.
 func (rw *RouteWatch) Shutdown(ctx context.Context) {
 	rw.mu.Lock()
 	if rw.shutdown {
@@ -176,52 +150,51 @@ func (rw *RouteWatch) Shutdown(ctx context.Context) {
 	rw.mu.Unlock()
 
 	// Stop the streamer first: it returns once the handlers have been handed
-	// every message it queued, or when ctx ends. Then each handler writes
+	// every message it queued. Then each handler writes what it holds.
-	// what it holds.
+	rw.streamer.Stop()
-	rw.streamer.Stop(ctx)
 
 	if rw.dbHandler != nil {
 		rw.logger.Info("Flushing database handler")
-		rw.dbHandler.Stop(ctx)
+		rw.dbHandler.Stop()
 	}
 
 	if rw.peerHandler != nil {
 		rw.logger.Info("Flushing peer handler")
-		rw.peerHandler.Stop(ctx)
+		rw.peerHandler.Stop()
 	}
 
 	if rw.prefixHandler != nil {
 		rw.logger.Info("Flushing prefix handler")
-		rw.prefixHandler.Stop(ctx)
+		rw.prefixHandler.Stop()
 	}
 
 	if rw.peeringHandler != nil {
 		rw.logger.Info("Flushing peering handler")
-		rw.peeringHandler.Stop(ctx)
+		rw.peeringHandler.Stop()
 	}
 
 	// Stop ASN WHOIS fetcher
 	if rw.asnFetcher != nil {
-		rw.asnFetcher.Stop(ctx)
+		rw.asnFetcher.Stop()
 	}
 
 	// Stop database maintainer
 	if rw.dbMaintainer != nil {
-		rw.dbMaintainer.Stop(ctx)
+		rw.dbMaintainer.Stop()
 	}
 
-	if rw.blocklists != nil {
+	// Stop HTTP server with a timeout of its own, however much of ctx is left
-		rw.blocklists.Stop(ctx)
+	const serverStopTimeout = 5 * time.Second
-	}
 
-	err := rw.server.Stop(ctx)
+	stopCtx, cancel := context.WithTimeout(
+		context.WithoutCancel(ctx), serverStopTimeout)
+	defer cancel()
+
+	err := rw.server.Stop(stopCtx)
 	if err != nil {
-		rw.logger.Error("Stopped the HTTP server before every request finished",
+		rw.logger.Error("Failed to stop HTTP server gracefully", "error", err)
-			"error", err)
 	}
 
-	rw.closeDatabase(ctx)
-
 	// Log final metrics
 	metrics := rw.streamer.GetMetrics()
 	rw.logger.Info("Final metrics",
@@ -233,27 +206,6 @@ func (rw *RouteWatch) Shutdown(ctx context.Context) {
 	)
 }
 
-// closeDatabase checkpoints and closes the database, unless ctx has ended: a
-// write may then still be running, and both would wait for it. SQLite rolls
-// back a write cut off by the exit when the database next opens.
-func (rw *RouteWatch) closeDatabase(ctx context.Context) {
-	if ctx.Err() != nil {
-		rw.logger.Error("Stop deadline passed; the database is not closed")
-
-		return
-	}
-
-	err := rw.db.Checkpoint(ctx)
-	if err != nil {
-		rw.logger.Error("Failed to checkpoint the database", "error", err)
-	}
-
-	err = rw.db.Close()
-	if err != nil {
-		rw.logger.Error("Failed to close the database", "error", err)
-	}
-}
-
 // getModule provides all fx dependencies
 func getModule() []fx.Option {
 	return []fx.Option{

internal/routewatch/app_integration_internal_test.go

@@ -4,11 +4,8 @@ import (
 	"context"
 	"fmt"
 	"io"
-	"net"
 	"net/http"
 	"net/http/httptest"
-	"slices"
-	"strconv"
 	"strings"
 	"sync"
 	"testing"
@@ -85,7 +82,9 @@ func (m *mockStore) UpdatePrefixesBatch(prefixes map[string]time.Time) error {
 	defer m.mu.Unlock()
 
 	for prefix, timestamp := range prefixes {
-		if _, exists := m.Prefixes[prefix]; !exists {
+		if p, exists := m.Prefixes[prefix]; exists {
+			p.LastSeen = timestamp
+		} else {
 			const (
 				ipVersionV4 = 4
 				ipVersionV6 = 6
@@ -100,6 +99,7 @@ func (m *mockStore) UpdatePrefixesBatch(prefixes map[string]time.Time) error {
 				Prefix:    prefix,
 				IPVersion: ipVersion,
 				FirstSeen: timestamp,
+				LastSeen:  timestamp,
 			}
 		}
 	}
@@ -115,6 +115,8 @@ func (m *mockStore) GetOrCreatePrefix(
 	defer m.mu.Unlock()
 
 	if p, exists := m.Prefixes[prefix]; exists {
+		p.LastSeen = timestamp
+
 		return p, nil
 	}
 
@@ -132,6 +134,7 @@ func (m *mockStore) GetOrCreatePrefix(
 		Prefix:    prefix,
 		IPVersion: ipVersion,
 		FirstSeen: timestamp,
+		LastSeen:  timestamp,
 	}
 	m.Prefixes[prefix] = p
 	m.PrefixCount++
@@ -152,7 +155,7 @@ func (m *mockStore) RecordAnnouncement(_ *database.Announcement) error {
 }
 
 // RecordPeering mock implementation
-func (m *mockStore) RecordPeering(asA, asB int, _ database.SeenTimes) error {
+func (m *mockStore) RecordPeering(asA, asB int, _ time.Time) error {
 	m.mu.Lock()
 	defer m.mu.Unlock()
 
@@ -367,6 +370,7 @@ func (m *mockStore) UpsertLiveRouteBatch(routes []*database.LiveRoute) error {
 				Prefix:    route.Prefix,
 				IPVersion: route.IPVersion,
 				FirstSeen: route.LastUpdated,
+				LastSeen:  route.LastUpdated,
 			}
 			m.PrefixCount++
 
@@ -396,16 +400,16 @@ func (m *mockStore) DeleteLiveRoutesOfPeer(_ string) (int, error) {
 }
 
 // GetOrCreateASNBatch mock implementation
-func (m *mockStore) GetOrCreateASNBatch(asns map[int]database.SeenTimes) error {
+func (m *mockStore) GetOrCreateASNBatch(asns map[int]time.Time) error {
 	m.mu.Lock()
 	defer m.mu.Unlock()
 
-	for number, seen := range asns {
+	for number, timestamp := range asns {
 		if _, exists := m.ASNs[number]; !exists {
 			m.ASNs[number] = &database.ASN{
 				ASN:       number,
-				FirstSeen: seen.First,
+				FirstSeen: timestamp,
-				LastSeen:  seen.Last,
+				LastSeen:  timestamp,
 			}
 			m.ASNCount++
 		}
@@ -466,9 +470,9 @@ func TestRouteWatchLiveFeed(t *testing.T) {
 	// Create streamer
 	s := streamer.New(logger, metricsTracker)
 
-	// The state directory holds the blocklist copies Run downloads.
+	// Create test config with empty state dir (no snapshot loading)
 	cfg := &config.Config{
-		StateDir:                    t.TempDir(),
+		StateDir:                    "",
 		MaxRuntime:                  5 * time.Second,
 		EnableBatchedDatabaseWrites: true,
 	}
@@ -646,189 +650,3 @@ func TestShutdownWritesTheUpdatesQueuedInTheStreamer(t *testing.T) {
 		}
 	}
 }
-
-// freePort returns a TCP port that was free a moment ago, for a server the
-// test starts.
-func freePort(t *testing.T) int {
-	t.Helper()
-
-	var listenConfig net.ListenConfig
-
-	listener, err := listenConfig.Listen(t.Context(), "tcp", ":0")
-	if err != nil {
-		t.Fatalf("failed to find a free port: %v", err)
-	}
-	defer func() { _ = listener.Close() }()
-
-	addr, ok := listener.Addr().(*net.TCPAddr)
-	if !ok {
-		t.Fatalf("listener address %v is not a TCP address", listener.Addr())
-	}
-
-	return addr.Port
-}
-
-// TestShutdownReturnsByItsDeadlineWhileAWriteIsBlocked starts the HTTP server
-// and has the AS handler start a write that the store holds until the test
-// ends, past Shutdown's deadline. Shutdown must return within its deadline
-// plus a small margin, with the HTTP server stopped, and must neither
-// checkpoint nor close the database: the real one would wait for the write.
-// Before the fix Shutdown waited for the write.
-func TestShutdownReturnsByItsDeadlineWhileAWriteIsBlocked(t *testing.T) {
-	t.Parallel()
-
-	const (
-		deadline = 100 * time.Millisecond
-		// margin is how long Shutdown may take after its deadline, to stop
-		// the HTTP server.
-		margin = 2 * time.Second
-	)
-
-	blocking := newBlockingStore()
-	defer blocking.releaseWrites()
-
-	store := &closeOrderStore{Store: blocking}
-	log := logger.New()
-	port := freePort(t)
-	cfg := &config.Config{Port: port}
-	s := streamer.New(log, metrics.New())
-
-	rw := New(Dependencies{
-		DB:       store,
-		Streamer: s,
-		Server:   server.New(store, s, log, cfg),
-		Logger:   log,
-		Config:   cfg,
-	})
-
-	err := rw.server.Start(t.Context())
-	if err != nil {
-		t.Fatalf("failed to start the HTTP server: %v", err)
-	}
-
-	rw.dbHandler = NewASHandler(store, log)
-	fill := ristypes.ASPath{Sequence: slices.Repeat([]int{testASNA}, asnBatchSize)}
-	startBlockedWrite(t, blocking, rw.dbHandler,
-		[]*ristypes.RISMessage{{Path: fill}})
-
-	ctx, cancel := context.WithTimeout(t.Context(), deadline)
-	defer cancel()
-
-	shutDown := make(chan struct{})
-
-	go func() {
-		rw.Shutdown(ctx)
-		close(shutDown)
-	}()
-
-	select {
-	case <-shutDown:
-	case <-time.After(deadline + margin):
-		t.Fatalf("Shutdown did not return within %s of its deadline while "+
-			"a write was blocked", margin)
-	}
-
-	var dialer net.Dialer
-
-	addr := net.JoinHostPort("127.0.0.1", strconv.Itoa(port))
-
-	conn, err := dialer.DialContext(t.Context(), "tcp", addr)
-	if err == nil {
-		_ = conn.Close()
-
-		t.Errorf("the HTTP server still accepts connections on %s after "+
-			"Shutdown returned", addr)
-	}
-
-	store.callsMu.Lock()
-	defer store.callsMu.Unlock()
-
-	if slices.Contains(store.calls, callCheckpoint) ||
-		slices.Contains(store.calls, callClose) {
-		t.Errorf("Shutdown made the calls %q past its deadline, want no %s "+
-			"and no %s", store.calls, callCheckpoint, callClose)
-	}
-}
-
-// closeOrderStore is a store that records, in order, the AS writes,
-// checkpoints and closes it is given. It passes the AS writes on to Store.
-type closeOrderStore struct {
-	database.Store
-
-	callsMu sync.Mutex
-	calls   []string
-}
-
-// The calls closeOrderStore records.
-const (
-	callWrite      = "AS write"
-	callCheckpoint = "checkpoint"
-	callClose      = "close"
-)
-
-func (s *closeOrderStore) GetOrCreateASNBatch(
-	asns map[int]database.SeenTimes,
-) error {
-	s.record(callWrite)
-
-	return s.Store.GetOrCreateASNBatch(asns)
-}
-
-func (s *closeOrderStore) Checkpoint(_ context.Context) error {
-	s.record(callCheckpoint)
-
-	return nil
-}
-
-func (s *closeOrderStore) Close() error {
-	s.record(callClose)
-
-	return nil
-}
-
-func (s *closeOrderStore) record(call string) {
-	s.callsMu.Lock()
-	defer s.callsMu.Unlock()
-
-	s.calls = append(s.calls, call)
-}
-
-// TestShutdownCheckpointsAndClosesTheDatabaseLast hands the AS handler one
-// message and shuts down with time to spare. The handler's last write must
-// come first, then a checkpoint, then the close. Before the fix Shutdown
-// neither checkpointed nor closed the database.
-func TestShutdownCheckpointsAndClosesTheDatabaseLast(t *testing.T) {
-	t.Parallel()
-
-	const deadline = 10 * time.Second
-
-	store := &closeOrderStore{Store: newMockStore()}
-	log := logger.New()
-	cfg := &config.Config{}
-	s := streamer.New(log, metrics.New())
-
-	rw := New(Dependencies{
-		DB:       store,
-		Streamer: s,
-		Server:   server.New(store, s, log, cfg),
-		Logger:   log,
-		Config:   cfg,
-	})
-	rw.dbHandler = NewASHandler(store, log)
-	rw.dbHandler.HandleMessage(&ristypes.RISMessage{
-		Path: ristypes.ASPath{Sequence: []int{testASNA}},
-	})
-
-	ctx, cancel := context.WithTimeout(t.Context(), deadline)
-	defer cancel()
-
-	rw.Shutdown(ctx)
-
-	store.callsMu.Lock()
-	defer store.callsMu.Unlock()
-
-	want := []string{callWrite, callCheckpoint, callClose}
-	if !slices.Equal(store.calls, want) {
-		t.Errorf("Shutdown made the calls %q, want %q", store.calls, want)
-	}
-}

internal/routewatch/ashandler.go

@@ -1,7 +1,6 @@
 package routewatch
 
 import (
-	"context"
 	"slices"
 	"sync"
 	"time"
@@ -40,23 +39,18 @@ type ASHandler struct {
 	db     database.Store
 	logger *logger.Logger
 
-	// Batching. mu guards batch and writing only: the flush goroutine takes
+	// Batching. mu guards batch only: the flush goroutine takes the batch
-	// the batch under mu and writes it after releasing mu, so HandleMessage
+	// under mu and writes it after releasing mu, so HandleMessage waits for
-	// waits for a database write only while batch is at asnBatchMaxSize.
+	// a database write only while batch is at asnBatchMaxSize.
 	mu    sync.Mutex
 	batch []asnOp
-	// writing is how many operations the flush goroutine took from batch
-	// and has not finished writing.
-	writing int
 	// batchTaken is signalled when the flush goroutine takes batch; its L is
 	// &mu.
 	batchTaken  sync.Cond
 	batchFullCh chan struct{} // wakes the flush goroutine when batch is full
 	lastFlush   time.Time     // used only by the flush goroutine
-	// stopCh hands Stop's context to the flush goroutine. It holds one, so
+	stopCh      chan struct{}
-	// Stop does not wait for a write in progress to hand it over.
+	wg          sync.WaitGroup
-	stopCh chan context.Context
-	done   chan struct{} // closed when the flush goroutine returns
 }
 
 type asnOp struct {
@@ -76,12 +70,12 @@ func NewASHandler(db database.Store, logger *logger.Logger) *ASHandler {
 		batch:       make([]asnOp, 0, asnBatchSize),
 		batchFullCh: make(chan struct{}, 1),
 		lastFlush:   time.Now(),
-		stopCh:      make(chan context.Context, 1),
+		stopCh:      make(chan struct{}),
-		done:        make(chan struct{}),
 	}
 	h.batchTaken.L = &h.mu
 
 	// Start the flush timer goroutine
+	h.wg.Add(1)
 	go h.flushLoop()
 
 	return h
@@ -153,32 +147,20 @@ func (h *ASHandler) HandleMessage(msg *ristypes.RISMessage) {
 	}
 }
 
-// Stop stops the flush goroutine, which writes the operations the handler holds
+// Stop gracefully shuts down the ASHandler by signaling the background flush
-// before it returns, and waits for it until ctx ends. Once ctx has ended the
+// goroutine to terminate and waiting for it to complete. Any pending ASN
-// flush goroutine skips that last write, and a write still running is lost
+// operations in the current batch are flushed to the database before Stop
-// when the process exits, so Stop logs how many operations are not written.
+// returns. This method should be called during application shutdown to ensure
-func (h *ASHandler) Stop(ctx context.Context) {
+// no data is lost.
-	h.stopCh <- ctx
+func (h *ASHandler) Stop() {
-
+	close(h.stopCh)
-	select {
+	h.wg.Wait()
-	case <-h.done:
-	case <-ctx.Done():
-	}
-
-	h.mu.Lock()
-	unwritten := h.writing + len(h.batch)
-	h.mu.Unlock()
-
-	if unwritten > 0 {
-		h.logger.Error("Stop deadline passed before the AS handler wrote "+
-			"its updates", "updates", unwritten)
-	}
 }
 
 // flushLoop runs in a goroutine, the only one that writes the batch: it
 // flushes when the batch is full, periodically, and once more on Stop.
 func (h *ASHandler) flushLoop() {
-	defer close(h.done)
+	defer h.wg.Done()
 
 	ticker := time.NewTicker(asnBatchTimeout)
 	defer ticker.Stop()
@@ -191,11 +173,9 @@ func (h *ASHandler) flushLoop() {
 			}
 		case <-h.batchFullCh:
 			h.flushBatch()
-		case ctx := <-h.stopCh:
+		case <-h.stopCh:
-			// Final flush, unless the stop deadline has passed
+			// Final flush
-			if ctx.Err() == nil {
 			h.flushBatch()
-			}
 
 			return
 		}
@@ -203,8 +183,7 @@ func (h *ASHandler) flushLoop() {
 }
 
 // takeBatch returns the pending operations and leaves an empty batch in their
-// place for HandleMessage to fill. They count in writing until flushBatch has
+// place for HandleMessage to fill.
-// written them.
 func (h *ASHandler) takeBatch() []asnOp {
 	h.mu.Lock()
 	defer h.mu.Unlock()
@@ -214,7 +193,6 @@ func (h *ASHandler) takeBatch() []asnOp {
 	}
 
 	batch := h.batch
-	h.writing = len(batch)
 	h.batch = make([]asnOp, 0, asnBatchSize)
 	h.batchTaken.Broadcast()
 
@@ -229,26 +207,20 @@ func (h *ASHandler) flushBatch() {
 		return
 	}
 
-	// The earliest and the latest time the batch saw each AS. The batch holds
+	// Process ASNs first (deduped)
-	// several updates for an AS, from many peers, so a new AS is stored first
+	asnMap := make(map[int]time.Time)
-	// seen at the earliest of them and last seen at the latest.
-	asns := make(map[int]database.SeenTimes)
 	for _, op := range batch {
-		seen := asns[op.number]
+		if existing, ok := asnMap[op.number]; !ok || op.timestamp.After(existing) {
-		seen.Add(op.timestamp)
+			asnMap[op.number] = op.timestamp
-		asns[op.number] = seen
+		}
 	}
 
 	// Process all ASNs in a single batch transaction
-	err := h.db.GetOrCreateASNBatch(asns)
+	err := h.db.GetOrCreateASNBatch(asnMap)
 	if err != nil {
 		h.logger.Error("Failed to process ASN batch",
-			"error", err, "count", len(asns))
+			"error", err, "count", len(asnMap))
 	}
 
-	h.mu.Lock()
-	h.writing = 0
-	h.mu.Unlock()
-
 	h.lastFlush = time.Now()
 }

internal/routewatch/ashandler_internal_test.go

@@ -1,68 +0,0 @@
-package routewatch
-
-import (
-	"maps"
-	"slices"
-	"testing"
-
-	"git.eeqj.de/sneak/routewatch/internal/logger"
-)
-
-// TestASHandlerStoresTheEarliestAndLatestTimesOfANewAS handles two updates
-// whose path carries an AS the database does not hold yet, the second 5
-// seconds after the first, in one batch into a real database. The AS is stored
-// first seen at the time of the first update and last seen at the time of the
-// second.
-func TestASHandlerStoresTheEarliestAndLatestTimesOfANewAS(t *testing.T) {
-	t.Parallel()
-
-	db := newTestDatabase(t)
-
-	// Built without NewASHandler's flush goroutine, so the test flushes itself.
-	h := &ASHandler{db: db, logger: logger.New()}
-
-	first := peerAnnouncement(testPeerIP, testV4Prefix)
-	second := peerAnnouncement(testPeerIP, testV4Prefix)
-	second.ParsedTimestamp = first.ParsedTimestamp.Add(secondUpdateAfter)
-
-	h.HandleMessage(first)
-	h.HandleMessage(second)
-	h.flushBatch()
-
-	asn, _, err := db.GetASDetails(testASNA)
-	if err != nil {
-		t.Fatalf("GetASDetails(%d): %v", testASNA, err)
-	}
-
-	if !asn.FirstSeen.Equal(first.ParsedTimestamp) {
-		t.Errorf("first_seen of AS%d = %s, want %s", testASNA,
-			asn.FirstSeen, first.ParsedTimestamp)
-	}
-
-	if !asn.LastSeen.Equal(second.ParsedTimestamp) {
-		t.Errorf("last_seen of AS%d = %s, want %s", testASNA,
-			asn.LastSeen, second.ParsedTimestamp)
-	}
-}
-
-// TestASHandlerRecordsNothingFromAPathWithASZero handles an update whose AS
-// path holds AS 0 between two other ASes. RFC 7607 makes such a path
-// malformed, so neither AS is recorded.
-func TestASHandlerRecordsNothingFromAPathWithASZero(t *testing.T) {
-	t.Parallel()
-
-	store := newMockStore()
-
-	// Built without NewASHandler's flush goroutine, so the test flushes itself.
-	h := &ASHandler{db: store, logger: logger.New()}
-
-	msg := peerAnnouncement(testPeerIP, testV4Prefix)
-	msg.Path = decodePath(t, testASNA, 0, testASNB)
-	h.HandleMessage(msg)
-	h.flushBatch()
-
-	if len(store.ASNs) > 0 {
-		t.Errorf("ASes recorded = %v, want none",
-			slices.Sorted(maps.Keys(store.ASNs)))
-	}
-}

internal/routewatch/asnfetcher.go

@@ -21,16 +21,10 @@ const (
 	minInterval = 1 * time.Second
 
 	// maxInterval is the maximum interval after failures (backoff cap).
-	maxInterval = time.Minute
+	maxInterval = 5 * time.Minute
 
-	// A failure multiplies the interval by backoffNumerator divided by
+	// backoffMultiplier is how much to multiply interval on failure.
-	// backoffDenominator, 1.5, up to maxInterval.
+	backoffMultiplier = 2
-	backoffNumerator   = 3
-	backoffDenominator = 2
-
-	// A success divides the interval by successDivisor, down to
-	// minInterval.
-	successDivisor = 2
 
 	// whoisStaleThreshold is how old WHOIS data can be before refresh.
 	whoisStaleThreshold = 30 * 24 * time.Hour // 30 days
@@ -49,7 +43,7 @@ type ASNFetcher struct {
 	logger         *slog.Logger
 	immediateQueue chan int
 	stopCh         chan struct{}
-	done           chan struct{} // closed when the fetcher goroutine returns
+	wg             sync.WaitGroup
 
 	// fetchMu ensures only one fetch runs at a time
 	fetchMu sync.Mutex
@@ -59,8 +53,7 @@ type ASNFetcher struct {
 	currentInterval  time.Duration
 	consecutiveFails int
 
-	// hourly stats tracking; now is the clock, time.Now except in tests
+	// hourly stats tracking
-	now          func() time.Time
 	statsMu      sync.Mutex
 	successTimes []time.Time
 	errorTimes   []time.Time
@@ -68,17 +61,13 @@ type ASNFetcher struct {
 
 // NewASNFetcher creates a new ASN fetcher.
 func NewASNFetcher(db database.Store, logger *slog.Logger) *ASNFetcher {
-	logger = logger.With("component", "asn_fetcher")
-
 	return &ASNFetcher{
 		db:              db,
-		whoisClient:     whois.NewClient(logger),
+		whoisClient:     whois.NewClient(),
-		logger:          logger,
+		logger:          logger.With("component", "asn_fetcher"),
 		immediateQueue:  make(chan int, immediateQueueSize),
 		stopCh:          make(chan struct{}),
-		done:            make(chan struct{}),
 		currentInterval: baseInterval,
-		now:             time.Now,
 		successTimes:    make([]time.Time, 0),
 		errorTimes:      make([]time.Time, 0),
 	}
@@ -87,6 +76,8 @@ func NewASNFetcher(db database.Store, logger *slog.Logger) *ASNFetcher {
 // Start begins the background ASN fetcher goroutine. It runs until Stop:
 // the end of ctx does not stop it.
 func (f *ASNFetcher) Start(ctx context.Context) {
+	f.wg.Add(1)
+
 	go f.run(context.WithoutCancel(ctx))
 
 	f.logger.Info("ASN fetcher started",
@@ -96,17 +87,11 @@ func (f *ASNFetcher) Start(ctx context.Context) {
 	)
 }
 
-// Stop stops the fetcher and waits, until ctx ends, for a lookup in progress to
+// Stop gracefully shuts down the fetcher.
-// finish.
+func (f *ASNFetcher) Stop() {
-func (f *ASNFetcher) Stop(ctx context.Context) {
 	close(f.stopCh)
-
+	f.wg.Wait()
-	select {
-	case <-f.done:
 	f.logger.Info("ASN fetcher stopped")
-	case <-ctx.Done():
-		f.logger.Warn("Stop deadline passed before the ASN fetcher stopped")
-	}
 }
 
 // QueueImmediate queues an ASN for immediate WHOIS lookup.
@@ -131,7 +116,7 @@ func (f *ASNFetcher) GetStats() server.ASNFetcherStats {
 	f.intervalMu.Unlock()
 
 	// Prune old entries and count
-	cutoff := f.now().Add(-statsWindow)
+	cutoff := time.Now().Add(-statsWindow)
 	f.successTimes = pruneOldTimes(f.successTimes, cutoff)
 	f.errorTimes = pruneOldTimes(f.errorTimes, cutoff)
 
@@ -143,15 +128,16 @@ func (f *ASNFetcher) GetStats() server.ASNFetcherStats {
 	}
 }
 
-// pruneOldTimes drops the times at or before cutoff from the front of times,
+// pruneOldTimes removes times older than cutoff and returns the pruned slice.
-// which holds times in the order they were recorded, and returns the rest. It
-// copies nothing, so it is cheap enough to run on every append.
 func pruneOldTimes(times []time.Time, cutoff time.Time) []time.Time {
-	for len(times) > 0 && !times[0].After(cutoff) {
+	result := make([]time.Time, 0, len(times))
-		times = times[1:]
+	for _, t := range times {
+		if t.After(cutoff) {
+			result = append(result, t)
+		}
 	}
 
-	return times
+	return result
 }
 
 // getInterval returns the current fetch interval.
@@ -168,7 +154,7 @@ func (f *ASNFetcher) recordSuccess() {
 	f.consecutiveFails = 0
 
 	// Decrease interval by half, but not below minimum
-	newInterval := max(f.currentInterval/successDivisor, minInterval)
+	newInterval := max(f.currentInterval/backoffMultiplier, minInterval)
 
 	if newInterval != f.currentInterval {
 		f.logger.Debug("Decreased fetch interval",
@@ -179,11 +165,9 @@ func (f *ASNFetcher) recordSuccess() {
 	}
 	f.intervalMu.Unlock()
 
-	// Record success time for stats, dropping those older than statsWindow
+	// Record success time for stats
 	f.statsMu.Lock()
-	now := f.now()
+	f.successTimes = append(f.successTimes, time.Now())
-	f.successTimes = pruneOldTimes(f.successTimes, now.Add(-statsWindow))
-	f.successTimes = append(f.successTimes, now)
 	f.statsMu.Unlock()
 }
 
@@ -192,9 +176,8 @@ func (f *ASNFetcher) recordFailure() {
 	f.intervalMu.Lock()
 	f.consecutiveFails++
 
-	// Exponential backoff: multiply by 1.5, capped at max
+	// Exponential backoff: multiply by 2, capped at max
-	newInterval := min(
+	newInterval := min(f.currentInterval*backoffMultiplier, maxInterval)
-		f.currentInterval*backoffNumerator/backoffDenominator, maxInterval)
 
 	if newInterval != f.currentInterval {
 		f.logger.Debug("Increased fetch interval due to failure",
@@ -206,17 +189,15 @@ func (f *ASNFetcher) recordFailure() {
 	}
 	f.intervalMu.Unlock()
 
-	// Record error time for stats, dropping those older than statsWindow
+	// Record error time for stats
 	f.statsMu.Lock()
-	now := f.now()
+	f.errorTimes = append(f.errorTimes, time.Now())
-	f.errorTimes = pruneOldTimes(f.errorTimes, now.Add(-statsWindow))
-	f.errorTimes = append(f.errorTimes, now)
 	f.statsMu.Unlock()
 }
 
 // run is the main background loop.
 func (f *ASNFetcher) run(ctx context.Context) {
-	defer close(f.done)
+	defer f.wg.Done()
 
 	timer := time.NewTimer(f.getInterval())
 	defer timer.Stop()
@@ -283,10 +264,7 @@ func (f *ASNFetcher) fetchNextStale(ctx context.Context) {
 }
 
 // fetchAndUpdate performs a WHOIS lookup and updates the database.
-// Returns true if successful. A failed lookup leaves the AS's row as it was,
+// Returns true if successful.
-// so the AS stays stale. An AS IANA refers to no RIR is stored with no
-// WHOIS data, so it is not looked up again until it is stale; that counts
-// as neither a success nor a failure.
 func (f *ASNFetcher) fetchAndUpdate(ctx context.Context, asn int) bool {
 	ctx, cancel := context.WithTimeout(ctx, time.Minute)
 	defer cancel()
@@ -294,19 +272,6 @@ func (f *ASNFetcher) fetchAndUpdate(ctx context.Context, asn int) bool {
 	f.logger.Info("Fetching WHOIS data", "asn", asn)
 
 	info, err := f.whoisClient.LookupASN(ctx, asn)
-	if errors.Is(err, whois.ErrNoReferral) {
-		f.logger.Info("IANA refers the AS to no RIR", "asn", asn)
-
-		err = f.db.UpdateASNWHOIS(ctx, &database.ASNWHOISUpdate{ASN: asn})
-		if err != nil {
-			f.logger.Error("Failed to update ASN WHOIS data",
-				"asn", asn, "error", err)
-			f.recordFailure()
-		}
-
-		return false
-	}
-
 	if err != nil {
 		f.logger.Error("WHOIS lookup failed", "asn", asn, "error", err)
 		f.recordFailure()

internal/routewatch/asnfetcher_internal_test.go

@@ -1,278 +0,0 @@
-package routewatch
-
-import (
-	"bufio"
-	"context"
-	"errors"
-	"io"
-	"net"
-	"sync"
-	"syscall"
-	"testing"
-	"time"
-
-	"git.eeqj.de/sneak/routewatch/internal/database"
-	"git.eeqj.de/sneak/routewatch/internal/logger"
-)
-
-// The WHOIS servers a lookup asks: IANA first, then the RIR IANA refers it
-// to.
-const (
-	ianaWHOISServer = "whois.iana.org:43"
-	ripeWHOISServer = "whois.ripe.net:43"
-)
-
-// ianaRefersToRIPE is IANA's answer for an AS in a block it gave to RIPE.
-const ianaRefersToRIPE = `refer:        whois.ripe.net
-
-as-block:     64496-64511
-organisation: Assigned by RIPE NCC
-changed:      2002-09-01
-source:       IANA
-`
-
-// ripeAccessDenied is RIPE's answer to a host it no longer serves.
-const ripeAccessDenied = `% This is the RIPE Database query service.
-%ERROR:201: access denied for 192.0.2.1
-`
-
-// ianaPrivateUseBlock is IANA's answer for a private-use AS: its block and
-// no referral.
-const ianaPrivateUseBlock = `as-block:     64512-65534
-organisation: Reserved for Private Use
-source:       IANA
-`
-
-// newFetcherWithWHOISAnswers returns an ASN fetcher on a new database that
-// holds testASNA, never looked up. The fetcher's WHOIS client connects to a
-// fake server for each address in answers, which reads the query and sends
-// answers[address]; connecting to any other address is refused.
-func newFetcherWithWHOISAnswers(
-	t *testing.T, answers map[string]string,
-) (*ASNFetcher, *database.Database) {
-	t.Helper()
-
-	db := newTestDatabase(t)
-
-	_, err := db.GetOrCreateASN(testASNA, time.Now())
-	if err != nil {
-		t.Fatalf("GetOrCreateASN: %v", err)
-	}
-
-	var servers sync.WaitGroup
-
-	t.Cleanup(servers.Wait)
-
-	f := NewASNFetcher(db, logger.New().Logger)
-	f.whoisClient.DialContext = func(
-		_ context.Context, _, address string,
-	) (net.Conn, error) {
-		answer, ok := answers[address]
-		if !ok {
-			return nil, syscall.ECONNREFUSED
-		}
-
-		client, server := net.Pipe()
-
-		servers.Go(func() {
-			defer func() { _ = server.Close() }()
-
-			_, _ = bufio.NewReader(server).ReadString('\n')
-			_, _ = io.WriteString(server, answer)
-		})
-
-		return client, nil
-	}
-
-	return f, db
-}
-
-// TestFetchAndUpdateKeepsTheRowWhenTheLookupFails looks up an AS IANA refers
-// to RIPE when RIPE's server refuses the connection, and when it answers only
-// with an error. The AS must stay never looked up, and the fetcher must back
-// off.
-func TestFetchAndUpdateKeepsTheRowWhenTheLookupFails(t *testing.T) {
-	t.Parallel()
-
-	for name, answers := range map[string]map[string]string{
-		"connection refused": {ianaWHOISServer: ianaRefersToRIPE},
-		"only an error": {
-			ianaWHOISServer: ianaRefersToRIPE,
-			ripeWHOISServer: ripeAccessDenied,
-		},
-	} {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			f, db := newFetcherWithWHOISAnswers(t, answers)
-
-			if f.fetchAndUpdate(t.Context(), testASNA) {
-				t.Error("fetchAndUpdate reported success")
-			}
-
-			stats, err := db.GetWHOISStats(t.Context(), whoisStaleThreshold)
-			if err != nil {
-				t.Fatalf("GetWHOISStats: %v", err)
-			}
-
-			if stats.NeverFetched != 1 {
-				t.Errorf("never looked up = %d, want 1: the failed lookup "+
-					"was stored", stats.NeverFetched)
-			}
-
-			want := baseInterval * backoffNumerator / backoffDenominator
-
-			got := f.GetStats()
-			if got.CurrentInterval != want || got.ConsecutiveFails != 1 {
-				t.Errorf("interval %v after %d failures, want %v after 1",
-					got.CurrentInterval, got.ConsecutiveFails, want)
-			}
-		})
-	}
-}
-
-// TestRecordStepsTheInterval has the fetcher fail from its base interval of
-// 15 seconds, then succeed. Each failure makes the interval 1.5 times as
-// long, up to one minute, and each success halves it, down to one second.
-func TestRecordStepsTheInterval(t *testing.T) {
-	t.Parallel()
-
-	f := NewASNFetcher(newMockStore(), logger.New().Logger)
-
-	for i, want := range []time.Duration{
-		22500 * time.Millisecond,
-		33750 * time.Millisecond,
-		50625 * time.Millisecond,
-		time.Minute,
-		time.Minute,
-	} {
-		f.recordFailure()
-
-		if got := f.getInterval(); got != want {
-			t.Errorf("interval after %d failures = %v, want %v",
-				i+1, got, want)
-		}
-	}
-
-	for i, want := range []time.Duration{
-		30 * time.Second,
-		15 * time.Second,
-		7500 * time.Millisecond,
-		3750 * time.Millisecond,
-		1875 * time.Millisecond,
-		time.Second,
-		time.Second,
-	} {
-		f.recordSuccess()
-
-		if got := f.getInterval(); got != want {
-			t.Errorf("interval after %d successes = %v, want %v",
-				i+1, got, want)
-		}
-	}
-}
-
-// TestFetchAndUpdateMarksAnASWithoutReferral looks up a private-use AS, which
-// IANA refers to no RIR. The AS must not be looked up again until it is
-// stale, and the lookup counts as neither a success nor a failure.
-func TestFetchAndUpdateMarksAnASWithoutReferral(t *testing.T) {
-	t.Parallel()
-
-	f, db := newFetcherWithWHOISAnswers(t,
-		map[string]string{ianaWHOISServer: ianaPrivateUseBlock})
-
-	f.fetchAndUpdate(t.Context(), testASNA)
-
-	asn, err := db.GetNextStaleASN(t.Context(), whoisStaleThreshold)
-	if !errors.Is(err, database.ErrNoStaleASN) {
-		t.Errorf("GetNextStaleASN = %d, %v; want ErrNoStaleASN", asn, err)
-	}
-
-	got := f.GetStats()
-	if got.CurrentInterval != baseInterval || got.SuccessesLastHour != 0 ||
-		got.ErrorsLastHour != 0 {
-		t.Errorf("interval %v, %d successes, %d errors; want %v, 0, 0",
-			got.CurrentInterval, got.SuccessesLastHour, got.ErrorsLastHour,
-			baseInterval)
-	}
-}
-
-// TestASNFetcherStopReturnsByItsDeadline starts a fetcher, has it look up an
-// AS whose WHOIS server does not answer until the test ends, and stops it with
-// a short deadline while that lookup is in progress. Stop must return all the
-// same instead of waiting for the lookup.
-func TestASNFetcherStopReturnsByItsDeadline(t *testing.T) {
-	t.Parallel()
-
-	const stopDeadline = 100 * time.Millisecond
-
-	lookupStarted := make(chan struct{})
-	release := make(chan struct{})
-
-	f := NewASNFetcher(newMockStore(), logger.New().Logger)
-	f.whoisClient.DialContext = func(
-		context.Context, string, string,
-	) (net.Conn, error) {
-		close(lookupStarted)
-		<-release
-
-		return nil, syscall.ECONNREFUSED
-	}
-
-	f.Start(t.Context())
-	f.QueueImmediate(testASNA)
-
-	select {
-	case <-lookupStarted:
-	case <-time.After(handleMessageTimeout):
-		t.Fatal("the fetcher did not start the lookup")
-	}
-
-	ctx, cancel := context.WithTimeout(t.Context(), stopDeadline)
-	defer cancel()
-
-	if !returnsInTime(func() { f.Stop(ctx) }) {
-		t.Fatalf("Stop with a deadline of %s did not return while a lookup "+
-			"was in progress", stopDeadline)
-	}
-
-	close(release)
-
-	select {
-	case <-f.done:
-	case <-time.After(handleMessageTimeout):
-		t.Fatal("the fetcher goroutine did not return after the lookup ended")
-	}
-}
-
-// TestRecordKeepsOnlyTheStatsWindow records a success and a failure every
-// second, by a clock the test sets, far longer than statsWindow, and never
-// asks for the stats. The fetcher must keep only the times within the last
-// statsWindow.
-func TestRecordKeepsOnlyTheStatsWindow(t *testing.T) {
-	t.Parallel()
-
-	const (
-		records = 100_000
-		step    = time.Second
-		want    = int(statsWindow / step)
-	)
-
-	clock := time.Now()
-
-	f := NewASNFetcher(newMockStore(), logger.New().Logger)
-	f.now = func() time.Time { return clock }
-
-	for range records {
-		clock = clock.Add(step)
-
-		f.recordSuccess()
-		f.recordFailure()
-	}
-
-	if len(f.successTimes) != want || len(f.errorTimes) != want {
-		t.Errorf("%d success and %d error times kept, want %d of each, "+
-			"those of the last %s", len(f.successTimes), len(f.errorTimes),
-			want, statsWindow)
-	}
-}

internal/routewatch/batchwrite_internal_test.go

@@ -1,10 +1,6 @@
 package routewatch
 
 import (
-	"bytes"
-	"context"
-	"encoding/json"
-	"log/slog"
 	"slices"
 	"sync"
 	"testing"
@@ -26,9 +22,8 @@ const (
 )
 
 // blockingStore is a mockStore whose batch writes for the AS, peer and prefix
-// handlers, delete of a peer's routes and peering write wait until the test
+// handlers wait until the test calls releaseWrites. writeStarted is closed when
-// calls releaseWrites. writeStarted is closed when the first of them starts. It
+// the first of them starts. It records what each write was given.
-// records what each batch write was given.
 type blockingStore struct {
 	*mockStore
 
@@ -66,7 +61,7 @@ func (s *blockingStore) UpsertLiveRouteBatch(routes []*database.LiveRoute) error
 	return nil
 }
 
-func (s *blockingStore) GetOrCreateASNBatch(asns map[int]database.SeenTimes) error {
+func (s *blockingStore) GetOrCreateASNBatch(asns map[int]time.Time) error {
 	s.waitForRelease()
 	s.mu.Lock()
 	defer s.mu.Unlock()
@@ -90,20 +85,6 @@ func (s *blockingStore) UpdatePeerBatch(peers map[string]database.PeerUpdate) er
 	return nil
 }
 
-func (s *blockingStore) DeleteLiveRoutesOfPeer(peerIP string) (int, error) {
-	s.waitForRelease()
-
-	return s.mockStore.DeleteLiveRoutesOfPeer(peerIP)
-}
-
-func (s *blockingStore) RecordPeering(
-	asA, asB int, seen database.SeenTimes,
-) error {
-	s.waitForRelease()
-
-	return s.mockStore.RecordPeering(asA, asB, seen)
-}
-
 func (s *blockingStore) waitForRelease() {
 	s.startOnce.Do(func() { close(s.writeStarted) })
 	<-s.release
@@ -115,40 +96,6 @@ func (s *blockingStore) releaseWrites() {
 	s.releaseOnce.Do(func() { close(s.release) })
 }
 
-// stopLog holds what a handler's Stop logs it did not write.
-//
-//nolint:tagliatelle // field names are the keys the handlers log
-type stopLog struct {
-	Msg       string `json:"msg"`
-	Updates   int    `json:"updates"`
-	PeersDown int    `json:"peers_down"`
-	Paths     int    `json:"paths"`
-}
-
-// newErrorLogger returns a logger that writes only errors, as JSON lines, to
-// logs.
-func newErrorLogger(logs *bytes.Buffer) *logger.Logger {
-	return &logger.Logger{Logger: slog.New(slog.NewJSONHandler(logs,
-		&slog.HandlerOptions{Level: slog.LevelError}))}
-}
-
-// lastLogged decodes the last line in logs, which is the one Stop logs: a
-// handler logs nothing after it.
-func lastLogged(t *testing.T, logs *bytes.Buffer) stopLog {
-	t.Helper()
-
-	lines := bytes.Split(bytes.TrimSpace(logs.Bytes()), []byte("\n"))
-
-	var logged stopLog
-
-	err := json.Unmarshal(lines[len(lines)-1], &logged)
-	if err != nil {
-		t.Fatalf("want the error Stop logs, got %q: %v", logs.String(), err)
-	}
-
-	return logged
-}
-
 // returnsInTime reports whether f returns within handleMessageTimeout.
 func returnsInTime(f func()) bool {
 	done := make(chan struct{})
@@ -204,14 +151,14 @@ func handleWhileAWriteIsBlocked(
 	store *blockingStore,
 	h interface {
 		HandleMessage(msg *ristypes.RISMessage)
-		Stop(ctx context.Context)
+		Stop()
 	},
 	fill []*ristypes.RISMessage,
 	extra *ristypes.RISMessage,
 ) {
 	t.Helper()
 
-	defer h.Stop(t.Context())
+	defer h.Stop()
 	defer store.releaseWrites()
 
 	startBlockedWrite(t, store, h, fill)
@@ -231,14 +178,14 @@ func handleAtMaxBatchSize(
 	store *blockingStore,
 	h interface {
 		HandleMessage(msg *ristypes.RISMessage)
-		Stop(ctx context.Context)
+		Stop()
 	},
 	fill, toMax []*ristypes.RISMessage,
 	extra *ristypes.RISMessage,
 ) {
 	t.Helper()
 
-	defer h.Stop(t.Context())
+	defer h.Stop()
 	defer store.releaseWrites()
 
 	startBlockedWrite(t, store, h, fill)
@@ -270,173 +217,6 @@ func handleAtMaxBatchSize(
 	}
 }
 
-// stopWhileAWriteIsBlocked starts a blocked write with the messages in fill,
-// hands h the message extra, and stops h with a deadline that passes while
-// store holds the write. Stop must return all the same. Then it lets the write
-// finish and waits until done, closed when h's flush goroutine returns. Past
-// the deadline that goroutine skips its last write, so extra is not written,
-// and Stop logs as not written both extra and what the held write took.
-func stopWhileAWriteIsBlocked(
-	t *testing.T,
-	store *blockingStore,
-	h interface {
-		HandleMessage(msg *ristypes.RISMessage)
-		Stop(ctx context.Context)
-	},
-	done <-chan struct{},
-	fill []*ristypes.RISMessage,
-	extra *ristypes.RISMessage,
-) {
-	t.Helper()
-
-	const stopDeadline = 100 * time.Millisecond
-
-	defer store.releaseWrites()
-
-	startBlockedWrite(t, store, h, fill)
-	h.HandleMessage(extra)
-
-	ctx, cancel := context.WithTimeout(t.Context(), stopDeadline)
-	defer cancel()
-
-	if !returnsInTime(func() { h.Stop(ctx) }) {
-		t.Fatalf("Stop with a deadline of %s did not return while a write "+
-			"was blocked", stopDeadline)
-	}
-
-	store.releaseWrites()
-
-	select {
-	case <-done:
-	case <-time.After(handleMessageTimeout):
-		t.Fatal("the flush goroutine did not return after the write finished")
-	}
-}
-
-func TestPrefixHandlerSkipsItsLastWriteAfterTheStopDeadline(t *testing.T) {
-	t.Parallel()
-
-	const extraPrefix = "198.51.100.0/24"
-
-	var logs bytes.Buffer
-
-	store := newBlockingStore()
-	h := NewPrefixHandler(store, newErrorLogger(&logs))
-	fill := slices.Repeat([]string{testV4Prefix}, prefixBatchSize)
-	// The peer's session goes down first, so the write held is the delete of
-	// its routes, and Stop must log that peer down too.
-	stopWhileAWriteIsBlocked(t, store, h, h.done,
-		[]*ristypes.RISMessage{sessionDown(), prefixAnnouncement(fill)},
-		prefixAnnouncement([]string{extraPrefix}))
-
-	store.mu.Lock()
-	defer store.mu.Unlock()
-
-	if store.prefixes[extraPrefix] {
-		t.Errorf("%s was written after the stop deadline", extraPrefix)
-	}
-
-	logged := lastLogged(t, &logs)
-	if logged.Updates != prefixBatchSize+1 || logged.PeersDown != 1 {
-		t.Errorf("Stop logged %d updates and %d peer downs not written, "+
-			"want %d and 1", logged.Updates, logged.PeersDown, prefixBatchSize+1)
-	}
-}
-
-func TestASHandlerSkipsItsLastWriteAfterTheStopDeadline(t *testing.T) {
-	t.Parallel()
-
-	// Each message gives an update for its origin AS and one for each AS
-	// in its path: fill asnBatchSize+1, extra 2.
-	const wantUpdates = asnBatchSize + 1 + 2
-
-	var logs bytes.Buffer
-
-	store := newBlockingStore()
-	h := NewASHandler(store, newErrorLogger(&logs))
-	fill := ristypes.ASPath{Sequence: slices.Repeat([]int{testASNA}, asnBatchSize)}
-	stopWhileAWriteIsBlocked(t, store, h, h.done,
-		[]*ristypes.RISMessage{{Path: fill}},
-		&ristypes.RISMessage{Path: ristypes.ASPath{Sequence: []int{testASNB}}})
-
-	store.mu.Lock()
-	defer store.mu.Unlock()
-
-	if store.asns[testASNB] {
-		t.Errorf("AS%d was written after the stop deadline", testASNB)
-	}
-
-	if logged := lastLogged(t, &logs); logged.Updates != wantUpdates {
-		t.Errorf("Stop logged %d updates not written, want %d",
-			logged.Updates, wantUpdates)
-	}
-}
-
-func TestPeerHandlerSkipsItsLastWriteAfterTheStopDeadline(t *testing.T) {
-	t.Parallel()
-
-	const otherPeerIP = "2001:db8:ffff::2"
-
-	var logs bytes.Buffer
-
-	store := newBlockingStore()
-	h := NewPeerHandler(store, newErrorLogger(&logs))
-	stopWhileAWriteIsBlocked(t, store, h, h.done,
-		slices.Repeat([]*ristypes.RISMessage{{Peer: testPeerIP}}, peerBatchSize),
-		&ristypes.RISMessage{Peer: otherPeerIP})
-
-	store.mu.Lock()
-	defer store.mu.Unlock()
-
-	if store.peers[otherPeerIP] {
-		t.Errorf("peer %s was written after the stop deadline", otherPeerIP)
-	}
-
-	if logged := lastLogged(t, &logs); logged.Updates != peerBatchSize+1 {
-		t.Errorf("Stop logged %d updates not written, want %d",
-			logged.Updates, peerBatchSize+1)
-	}
-}
-
-// TestPeeringHandlerStopCountsTheWriteInProgress hands a running peering
-// handler one AS path and stops it with a deadline that passes while store
-// holds Stop's last write, of that path's peering. Stop must log that path as
-// not written.
-func TestPeeringHandlerStopCountsTheWriteInProgress(t *testing.T) {
-	t.Parallel()
-
-	const stopDeadline = 100 * time.Millisecond
-
-	var logs bytes.Buffer
-
-	store := newBlockingStore()
-	defer store.releaseWrites()
-
-	h := NewPeeringHandler(store, newErrorLogger(&logs))
-	handle(h, time.Now().UTC(), testASNA, testASNB)
-
-	ctx, cancel := context.WithTimeout(t.Context(), stopDeadline)
-	defer cancel()
-
-	if !returnsInTime(func() { h.Stop(ctx) }) {
-		t.Fatalf("Stop with a deadline of %s did not return while a write "+
-			"was blocked", stopDeadline)
-	}
-
-	store.releaseWrites()
-
-	select {
-	case <-h.done:
-	case <-time.After(handleMessageTimeout):
-		t.Fatal("the processing goroutine did not return after the write " +
-			"finished")
-	}
-
-	if logged := lastLogged(t, &logs); logged.Paths != 1 {
-		t.Errorf("Stop logged %d paths not written, want 1", logged.Paths)
-	}
-}
-
 // prefixAnnouncement returns a message in which testPeerIP announces prefixes
 // with origin AS testASNA.
 func prefixAnnouncement(prefixes []string) *ristypes.RISMessage {

internal/routewatch/cli.go

@@ -3,8 +3,10 @@ package routewatch
 import (
 	"context"
 	"os"
+	"os/signal"
 	"runtime"
 	"strings"
+	"syscall"
 	"time"
 
 	"git.eeqj.de/sneak/routewatch/internal/logger"
@@ -12,14 +14,8 @@ import (
 )
 
 const (
-	// shutdownTimeout is how long Shutdown has after a stop signal. upaas,
+	// shutdownTimeout is the maximum time allowed for graceful shutdown
-	// like docker stop by default, kills the container 10 seconds after the
+	shutdownTimeout = 60 * time.Second
-	// stop signal (README "Running under upaas").
-	shutdownTimeout = 8 * time.Second
-	// stopTimeout is fx's limit on the stop hook, past which fx exits with
-	// status 1: shutdownTimeout and time for Shutdown to close the HTTP
-	// server after its deadline.
-	stopTimeout = 9 * time.Second
 	// debugInterval is how often to log debug stats
 	debugInterval = 60 * time.Second
 	// bytesPerMB is bytes per megabyte
@@ -60,14 +56,14 @@ func logDebugStats(logger *logger.Logger) {
 }
 
 // CLIEntry is the main entry point for the routewatch command-line interface.
-// It initializes the application using the fx dependency injection framework
+// It initializes the application using the fx dependency injection framework,
-// and starts the RouteWatch service. It blocks until fx receives SIGINT or
+// sets up signal handling for graceful shutdown, and starts the RouteWatch
-// SIGTERM or RouteWatch stops on an error; fx then runs the stop hook and sets
+// service. This function blocks until the application receives a shutdown
-// the exit code.
+// signal or encounters a fatal error.
 func CLIEntry() {
 	app := fx.New(
 		fx.Options(getModule()...),
-		fx.StopTimeout(stopTimeout),
+		fx.StopTimeout(shutdownTimeout), // Allow 60 seconds for graceful shutdown
 		fx.Invoke(func(
 			lc fx.Lifecycle,
 			rw *RouteWatch,
@@ -79,6 +75,20 @@ func CLIEntry() {
 					// Start debug stats logging
 					go logDebugStats(logger)
 
+					// Handle shutdown signals
+					sigCh := make(chan os.Signal, 1)
+					signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)
+
+					go func() {
+						<-sigCh
+						logger.Info("Received shutdown signal")
+
+						err := shutdowner.Shutdown()
+						if err != nil {
+							logger.Error("Failed to shutdown gracefully", "error", err)
+						}
+					}()
+
 					go func() {
 						err := rw.Run(ctx)
 						if err != nil {
@@ -96,10 +106,6 @@ func CLIEntry() {
 				},
 				OnStop: func(ctx context.Context) error {
 					logger.Info("Shutting down RouteWatch")
-
-					ctx, cancel := context.WithTimeout(ctx, shutdownTimeout)
-					defer cancel()
-
 					rw.Shutdown(ctx)
 
 					return nil

internal/routewatch/dbmaintainer.go

@@ -2,7 +2,6 @@ package routewatch
 
 import (
 	"context"
-	"errors"
 	"log/slog"
 	"sync"
 	"time"
@@ -35,18 +34,12 @@ const (
 	analyzeTimeout = 5 * time.Minute
 )
 
-// checkpointIncompleteMessage is logged, at debug level like a completed
-// checkpoint, when a checkpoint leaves part of the WAL for a later one: while
-// reads overlap, most do. The database warns when the WAL file stays over its
-// size limit.
-const checkpointIncompleteMessage = "WAL checkpoint incomplete"
-
 // DBMaintainer handles background database maintenance tasks.
 type DBMaintainer struct {
 	db     database.Store
 	logger *slog.Logger
 	stopCh chan struct{}
-	done   chan struct{} // closed when the maintenance goroutine returns
+	wg     sync.WaitGroup
 
 	// Stats tracking
 	statsMu             sync.Mutex
@@ -67,13 +60,14 @@ func NewDBMaintainer(db database.Store, logger *slog.Logger) *DBMaintainer {
 		db:     db,
 		logger: logger.With("component", "db_maintainer"),
 		stopCh: make(chan struct{}),
-		done:   make(chan struct{}),
 	}
 }
 
 // Start begins the background maintenance goroutine. It runs until Stop:
 // the end of ctx does not stop it.
 func (m *DBMaintainer) Start(ctx context.Context) {
+	m.wg.Add(1)
+
 	go m.run(context.WithoutCancel(ctx))
 
 	m.logger.Info("Database maintainer started",
@@ -83,23 +77,16 @@ func (m *DBMaintainer) Start(ctx context.Context) {
 	)
 }
 
-// Stop stops the maintainer and waits, until ctx ends, for a maintenance task
+// Stop gracefully shuts down the maintainer.
-// in progress to finish.
+func (m *DBMaintainer) Stop() {
-func (m *DBMaintainer) Stop(ctx context.Context) {
 	close(m.stopCh)
-
+	m.wg.Wait()
-	select {
-	case <-m.done:
 	m.logger.Info("Database maintainer stopped")
-	case <-ctx.Done():
-		m.logger.Warn("Stop deadline passed before the database maintainer " +
-			"stopped")
-	}
 }
 
 // run is the main background loop.
 func (m *DBMaintainer) run(ctx context.Context) {
-	defer close(m.done)
+	defer m.wg.Done()
 
 	// Use different timers for each task
 	checkpointTimer := time.NewTimer(checkpointInterval)
@@ -136,13 +123,9 @@ func (m *DBMaintainer) runCheckpoint(ctx context.Context) {
 	duration, err := runWithTimeout(ctx, checkpointTimeout, m.db.Checkpoint)
 	m.recordResult(&m.lastCheckpoint, &m.lastCheckpointError, &m.checkpointCount, err)
 
-	switch {
+	if err != nil {
-	case errors.Is(err, database.ErrCheckpointIncomplete):
-		m.logger.Debug(checkpointIncompleteMessage,
-			"error", err, "duration", duration)
-	case err != nil:
 		m.logger.Error("WAL checkpoint failed", "error", err, "duration", duration)
-	default:
+	} else {
 		m.logger.Debug("WAL checkpoint completed", "duration", duration)
 	}
 }

internal/routewatch/dbmaintainer_internal_test.go

@@ -1,120 +0,0 @@
-package routewatch
-
-import (
-	"bytes"
-	"context"
-	"encoding/json"
-	"log/slog"
-	"testing"
-	"time"
-
-	"git.eeqj.de/sneak/routewatch/internal/database"
-)
-
-// incompleteCheckpointStore is a store whose every checkpoint leaves part of
-// the WAL uncopied.
-type incompleteCheckpointStore struct{ *mockStore }
-
-// Checkpoint reports the checkpoint incomplete.
-func (incompleteCheckpointStore) Checkpoint(context.Context) error {
-	return database.ErrCheckpointIncomplete
-}
-
-// TestIncompleteCheckpointNotLoggedAsCompleted runs a checkpoint that leaves
-// part of the WAL uncopied and checks that the maintainer logs it as
-// incomplete, not as completed, and does not count it as a checkpoint that
-// completed.
-func TestIncompleteCheckpointNotLoggedAsCompleted(t *testing.T) {
-	t.Parallel()
-
-	var logs bytes.Buffer
-
-	logger := slog.New(slog.NewJSONHandler(&logs,
-		&slog.HandlerOptions{Level: slog.LevelDebug}))
-	m := NewDBMaintainer(incompleteCheckpointStore{newMockStore()}, logger)
-
-	m.runCheckpoint(t.Context())
-
-	var logged struct {
-		Msg string `json:"msg"`
-	}
-
-	err := json.Unmarshal(logs.Bytes(), &logged)
-	if err != nil {
-		t.Fatalf("want one log message, got %q: %v", logs.String(), err)
-	}
-
-	if logged.Msg != checkpointIncompleteMessage {
-		t.Errorf("logged %q, want %q", logged.Msg, checkpointIncompleteMessage)
-	}
-
-	if m.checkpointCount != 0 {
-		t.Errorf("checkpointCount = %d after an incomplete checkpoint, want 0",
-			m.checkpointCount)
-	}
-}
-
-// heldCheckpointStore is a store whose checkpoint closes started and then
-// waits until release is closed.
-type heldCheckpointStore struct {
-	*mockStore
-
-	started chan struct{}
-	release chan struct{}
-}
-
-// Checkpoint waits until release is closed.
-func (s heldCheckpointStore) Checkpoint(context.Context) error {
-	close(s.started)
-	<-s.release
-
-	return nil
-}
-
-// TestDBMaintainerStopReturnsByItsDeadline stops the maintainer with a short
-// deadline while its goroutine runs a checkpoint that the store holds until
-// the test ends. Stop must return all the same instead of waiting for the
-// checkpoint.
-func TestDBMaintainerStopReturnsByItsDeadline(t *testing.T) {
-	t.Parallel()
-
-	const stopDeadline = 100 * time.Millisecond
-
-	store := heldCheckpointStore{
-		mockStore: newMockStore(),
-		started:   make(chan struct{}),
-		release:   make(chan struct{}),
-	}
-	m := NewDBMaintainer(store, slog.Default())
-
-	// Start would run the first checkpoint only after checkpointInterval.
-	// This goroutine stands for the one Start runs and checkpoints at once.
-	go func() {
-		defer close(m.done)
-
-		m.runCheckpoint(t.Context())
-	}()
-
-	select {
-	case <-store.started:
-	case <-time.After(handleMessageTimeout):
-		t.Fatal("the maintainer did not start the checkpoint")
-	}
-
-	ctx, cancel := context.WithTimeout(t.Context(), stopDeadline)
-	defer cancel()
-
-	if !returnsInTime(func() { m.Stop(ctx) }) {
-		t.Fatalf("Stop with a deadline of %s did not return while a "+
-			"checkpoint was in progress", stopDeadline)
-	}
-
-	close(store.release)
-
-	select {
-	case <-m.done:
-	case <-time.After(handleMessageTimeout):
-		t.Fatal("the maintenance goroutine did not return after the " +
-			"checkpoint ended")
-	}
-}

internal/routewatch/peerhandler.go

@@ -5,7 +5,6 @@ package routewatch
 // peer state in the database.
 
 import (
-	"context"
 	"strconv"
 	"sync"
 	"time"
@@ -37,30 +36,24 @@ const (
 
 // PeerHandler tracks BGP peers from UPDATE messages using batched operations.
 // It maintains a queue of peer updates and periodically flushes them to the database
-// in batches to improve performance. The handler writes one row per peer IP
+// in batches to improve performance. The handler deduplicates peer updates within
-// address for each batch, with the earliest and the latest time of its updates.
+// each batch, keeping only the most recent update for each peer IP address.
 type PeerHandler struct {
 	db     database.Store
 	logger *logger.Logger
 
-	// Batching. mu guards peerBatch and writing only: the flush goroutine
+	// Batching. mu guards peerBatch only: the flush goroutine takes the batch
-	// takes the batch under mu and writes it after releasing mu, so
+	// under mu and writes it after releasing mu, so HandleMessage waits for
-	// HandleMessage waits for a database write only while peerBatch is at
+	// a database write only while peerBatch is at peerBatchMaxSize.
-	// peerBatchMaxSize.
 	mu        sync.Mutex
 	peerBatch []peerUpdate
-	// writing is how many updates the flush goroutine took from peerBatch
-	// and has not finished writing.
-	writing int
 	// batchTaken is signalled when the flush goroutine takes peerBatch; its L
 	// is &mu.
 	batchTaken  sync.Cond
 	batchFullCh chan struct{} // wakes the flush goroutine when peerBatch is full
 	lastFlush   time.Time     // used only by the flush goroutine
-	// stopCh hands Stop's context to the flush goroutine. It holds one, so
+	stopCh      chan struct{}
-	// Stop does not wait for a write in progress to hand it over.
+	wg          sync.WaitGroup
-	stopCh chan context.Context
-	done   chan struct{} // closed when the flush goroutine returns
 }
 
 type peerUpdate struct {
@@ -81,12 +74,12 @@ func NewPeerHandler(db database.Store, logger *logger.Logger) *PeerHandler {
 		peerBatch:   make([]peerUpdate, 0, peerBatchSize),
 		batchFullCh: make(chan struct{}, 1),
 		lastFlush:   time.Now(),
-		stopCh:      make(chan context.Context, 1),
+		stopCh:      make(chan struct{}),
-		done:        make(chan struct{}),
 	}
 	h.batchTaken.L = &h.mu
 
 	// Start the flush timer goroutine
+	h.wg.Add(1)
 	go h.flushLoop()
 
 	return h
@@ -151,32 +144,16 @@ func (h *PeerHandler) HandleMessage(msg *ristypes.RISMessage) {
 	}
 }
 
-// Stop stops the flush goroutine, which writes the updates the handler holds
+// Stop gracefully stops the handler and flushes remaining batches
-// before it returns, and waits for it until ctx ends. Once ctx has ended the
+func (h *PeerHandler) Stop() {
-// flush goroutine skips that last write, and a write still running is lost
+	close(h.stopCh)
-// when the process exits, so Stop logs how many updates are not written.
+	h.wg.Wait()
-func (h *PeerHandler) Stop(ctx context.Context) {
-	h.stopCh <- ctx
-
-	select {
-	case <-h.done:
-	case <-ctx.Done():
-	}
-
-	h.mu.Lock()
-	unwritten := h.writing + len(h.peerBatch)
-	h.mu.Unlock()
-
-	if unwritten > 0 {
-		h.logger.Error("Stop deadline passed before the peer handler wrote "+
-			"its updates", "updates", unwritten)
-	}
 }
 
 // flushLoop runs in a goroutine, the only one that writes the batch: it
 // flushes when the batch is full, periodically, and once more on Stop.
 func (h *PeerHandler) flushLoop() {
-	defer close(h.done)
+	defer h.wg.Done()
 
 	ticker := time.NewTicker(peerBatchTimeout)
 	defer ticker.Stop()
@@ -189,11 +166,9 @@ func (h *PeerHandler) flushLoop() {
 			}
 		case <-h.batchFullCh:
 			h.flushBatch()
-		case ctx := <-h.stopCh:
+		case <-h.stopCh:
-			// Final flush, unless the stop deadline has passed
+			// Final flush
-			if ctx.Err() == nil {
 			h.flushBatch()
-			}
 
 			return
 		}
@@ -201,8 +176,7 @@ func (h *PeerHandler) flushLoop() {
 }
 
 // takeBatch returns the pending updates and leaves an empty batch in their
-// place for HandleMessage to fill. They count in writing until flushBatch has
+// place for HandleMessage to fill.
-// written them.
 func (h *PeerHandler) takeBatch() []peerUpdate {
 	h.mu.Lock()
 	defer h.mu.Unlock()
@@ -212,7 +186,6 @@ func (h *PeerHandler) takeBatch() []peerUpdate {
 	}
 
 	batch := h.peerBatch
-	h.writing = len(batch)
 	h.peerBatch = make([]peerUpdate, 0, peerBatchSize)
 	h.batchTaken.Broadcast()
 
@@ -227,33 +200,32 @@ func (h *PeerHandler) flushBatch() {
 		return
 	}
 
-	// The earliest and the latest time the batch saw each peer, with the peer
+	// Deduplicate by peer IP, keeping the latest update for each peer
-	// AS and message type of its latest update. The batch holds several
+	peerMap := make(map[string]peerUpdate)
-	// updates for a peer, so a new peer is stored first seen at the earliest of
-	// them and last seen at the latest.
-	peers := make(map[string]database.PeerUpdate)
 	for _, update := range batch {
-		peer, ok := peers[update.peerIP]
+		existing, ok := peerMap[update.peerIP]
-		if !ok || update.timestamp.After(peer.Seen.Last) {
+		if !ok || update.timestamp.After(existing.timestamp) {
-			peer.PeerIP = update.peerIP
+			peerMap[update.peerIP] = update
-			peer.PeerASN = update.peerASN
+		}
-			peer.MessageType = update.messageType
 	}
 
-		peer.Seen.Add(update.timestamp)
+	// Convert to database format
-		peers[update.peerIP] = peer
+	dbPeerMap := make(map[string]database.PeerUpdate)
+	for peerIP, update := range peerMap {
+		dbPeerMap[peerIP] = database.PeerUpdate{
+			PeerIP:      update.peerIP,
+			PeerASN:     update.peerASN,
+			MessageType: update.messageType,
+			Timestamp:   update.timestamp,
+		}
 	}
 
 	// Process all peers in a single batch transaction
-	err := h.db.UpdatePeerBatch(peers)
+	err := h.db.UpdatePeerBatch(dbPeerMap)
 	if err != nil {
 		h.logger.Error("Failed to process peer batch",
-			"error", err, "count", len(peers))
+			"error", err, "count", len(dbPeerMap))
 	}
 
-	h.mu.Lock()
-	h.writing = 0
-	h.mu.Unlock()
-
 	h.lastFlush = time.Now()
 }

internal/routewatch/peerhandler_internal_test.go

@@ -1,59 +0,0 @@
-package routewatch
-
-import (
-	"testing"
-
-	"git.eeqj.de/sneak/routewatch/internal/database"
-	"git.eeqj.de/sneak/routewatch/internal/logger"
-)
-
-// peerBatchStore keeps the peers of the last UpdatePeerBatch call. The
-// database has no read of the peer table, so the peer handler's test checks
-// what the handler writes.
-type peerBatchStore struct {
-	*mockStore
-
-	peers map[string]database.PeerUpdate
-}
-
-func (s *peerBatchStore) UpdatePeerBatch(
-	peers map[string]database.PeerUpdate,
-) error {
-	s.peers = peers
-
-	return nil
-}
-
-// TestPeerHandlerWritesTheEarliestAndLatestTimesOfANewPeer handles two updates
-// from one peer, the second 5 seconds after the first, in one batch. The peer
-// handler writes the peer first seen at the time of the first update and last
-// seen at the time of the second.
-func TestPeerHandlerWritesTheEarliestAndLatestTimesOfANewPeer(t *testing.T) {
-	t.Parallel()
-
-	store := &peerBatchStore{mockStore: newMockStore()}
-
-	// Built without NewPeerHandler's flush goroutine, so the test flushes
-	// itself.
-	h := &PeerHandler{db: store, logger: logger.New()}
-
-	first := peerAnnouncement(testPeerIP, testV4Prefix)
-	second := peerAnnouncement(testPeerIP, testV4Prefix)
-	second.ParsedTimestamp = first.ParsedTimestamp.Add(secondUpdateAfter)
-
-	h.HandleMessage(first)
-	h.HandleMessage(second)
-	h.flushBatch()
-
-	seen := store.peers[testPeerIP].Seen
-
-	if !seen.First.Equal(first.ParsedTimestamp) {
-		t.Errorf("first_seen of peer %s = %s, want %s", testPeerIP,
-			seen.First, first.ParsedTimestamp)
-	}
-
-	if !seen.Last.Equal(second.ParsedTimestamp) {
-		t.Errorf("last_seen of peer %s = %s, want %s", testPeerIP,
-			seen.Last, second.ParsedTimestamp)
-	}
-}

internal/routewatch/peeringhandler.go

@@ -1,7 +1,6 @@
 package routewatch
 
 import (
-	"context"
 	"encoding/json"
 	"sync"
 	"time"
@@ -42,19 +41,13 @@ type PeeringHandler struct {
 	db     database.Store
 	logger *logger.Logger
 
-	// In-memory AS path tracking: each path, JSON-encoded, with the earliest
+	// In-memory AS path tracking
-	// and the latest time an update carried it.
 	mu           sync.Mutex
-	asPaths      map[string]database.SeenTimes
+	asPaths      map[string]time.Time // key is JSON-encoded AS path
-	droppedPaths int // paths dropped since the last run because the map was full
+	droppedPaths int                  // paths dropped because the map was full
-	// writing is how many paths processPeerings took from asPaths and has
-	// not finished writing the peerings of.
-	writing int
 
-	// stopCh hands Stop's context to the processing goroutine. It holds one,
+	stopCh chan struct{}
-	// so Stop does not wait for a write in progress to hand it over.
+	wg     sync.WaitGroup
-	stopCh chan context.Context
-	done   chan struct{} // closed when the processing goroutine returns
 }
 
 // NewPeeringHandler creates and initializes a new PeeringHandler with the
@@ -65,12 +58,13 @@ func NewPeeringHandler(db database.Store, logger *logger.Logger) *PeeringHandler
 	h := &PeeringHandler{
 		db:      db,
 		logger:  logger,
-		asPaths: make(map[string]database.SeenTimes),
+		asPaths: make(map[string]time.Time),
-		stopCh:  make(chan context.Context, 1),
+		stopCh:  make(chan struct{}),
-		done:    make(chan struct{}),
 	}
 
 	// Start the periodic processing goroutine
+	h.wg.Add(1)
+
 	go h.processLoop()
 
 	return h
@@ -119,18 +113,16 @@ func (h *PeeringHandler) HandleMessage(msg *ristypes.RISMessage) {
 	key := string(pathJSON)
 
 	h.mu.Lock()
-	defer h.mu.Unlock()
+	if _, exists := h.asPaths[key]; exists {
-
+		// Already tracked: refresh its timestamp.
-	seen, exists := h.asPaths[key]
+		h.asPaths[key] = timestamp
-	if !exists && len(h.asPaths) >= maxTrackedPaths {
+	} else if len(h.asPaths) >= maxTrackedPaths {
 		// Map is full; drop this new path and count it.
 		h.droppedPaths++
-
+	} else {
-		return
+		h.asPaths[key] = timestamp
 	}
-
+	h.mu.Unlock()
-	seen.Add(timestamp)
-	h.asPaths[key] = seen
 }
 
 // ProcessPeeringsNow triggers immediate processing of all accumulated AS
@@ -140,33 +132,18 @@ func (h *PeeringHandler) ProcessPeeringsNow() {
 	h.processPeerings()
 }
 
-// Stop stops the processing goroutine, which writes the peerings of the AS
+// Stop gracefully shuts down the handler by signaling the background
-// paths the handler holds before it returns, and waits for it until ctx ends.
+// goroutine to stop and waiting while it processes the remaining AS paths,
-// Once ctx has ended the processing goroutine skips that last write, and a
+// so no peering data is lost during shutdown.
-// write still running is lost when the process exits, so Stop logs how many
+func (h *PeeringHandler) Stop() {
-// paths' peerings are not written.
+	close(h.stopCh)
-func (h *PeeringHandler) Stop(ctx context.Context) {
+	h.wg.Wait()
-	h.stopCh <- ctx
-
-	select {
-	case <-h.done:
-	case <-ctx.Done():
-	}
-
-	h.mu.Lock()
-	unwritten := h.writing + len(h.asPaths)
-	h.mu.Unlock()
-
-	if unwritten > 0 {
-		h.logger.Error("Stop deadline passed before the peering handler wrote "+
-			"its peerings", "paths", unwritten)
-	}
 }
 
 // processLoop runs periodically to process AS paths into peerings, and once
 // more on Stop.
 func (h *PeeringHandler) processLoop() {
-	defer close(h.done)
+	defer h.wg.Done()
 
 	ticker := time.NewTicker(peeringProcessInterval)
 	defer ticker.Stop()
@@ -175,11 +152,9 @@ func (h *PeeringHandler) processLoop() {
 		select {
 		case <-ticker.C:
 			h.processPeerings()
-		case ctx := <-h.stopCh:
+		case <-h.stopCh:
-			// Final processing, unless the stop deadline has passed
+			// Final processing
-			if ctx.Err() == nil {
 			h.processPeerings()
-			}
 
 			return
 		}
@@ -190,14 +165,11 @@ func (h *PeeringHandler) processLoop() {
 func (h *PeeringHandler) processPeerings() {
 	// Take the accumulated paths and replace the map with a fresh empty one
 	// under the lock. Each path is processed exactly once and the memory is
-	// released, so the map never grows past a single interval's traffic. The
+	// released, so the map never grows past a single interval's traffic.
-	// count of dropped paths starts again with the new map.
 	h.mu.Lock()
 	paths := h.asPaths
-	h.asPaths = make(map[string]database.SeenTimes)
+	h.asPaths = make(map[string]time.Time)
-	h.writing = len(paths)
 	dropped := h.droppedPaths
-	h.droppedPaths = 0
 	h.mu.Unlock()
 
 	if len(paths) == 0 {
@@ -209,13 +181,9 @@ func (h *PeeringHandler) processPeerings() {
 		low, high int
 	}
 
-	// The earliest and the latest time an update carried each peering. Several
+	peerings := make(map[peeringKey]time.Time)
-	// paths, each carried at several times, can hold a peering, so a new
-	// peering is stored first seen at the earliest of them and last seen at the
-	// latest.
-	peerings := make(map[peeringKey]database.SeenTimes)
 
-	for pathJSON, pathSeen := range paths {
+	for pathJSON, timestamp := range paths {
 		var path []int
 
 		err := json.Unmarshal([]byte(pathJSON), &path)
@@ -242,11 +210,10 @@ func (h *PeeringHandler) processPeerings() {
 			}
 
 			key := peeringKey{low: low, high: high}
-
+			// Update timestamp if this is newer
-			seen := peerings[key]
+			if existing, ok := peerings[key]; !ok || timestamp.After(existing) {
-			seen.Add(pathSeen.First)
+				peerings[key] = timestamp
-			seen.Add(pathSeen.Last)
+			}
-			peerings[key] = seen
 		}
 	}
 
@@ -254,8 +221,8 @@ func (h *PeeringHandler) processPeerings() {
 	start := time.Now()
 	successCount := 0
 
-	for key, seen := range peerings {
+	for key, ts := range peerings {
-		err := h.db.RecordPeering(key.low, key.high, seen)
+		err := h.db.RecordPeering(key.low, key.high, ts)
 		if err != nil {
 			h.logger.Error("Failed to record peering",
 				"as_a", key.low,
@@ -267,10 +234,6 @@ func (h *PeeringHandler) processPeerings() {
 		}
 	}
 
-	h.mu.Lock()
-	h.writing = 0
-	h.mu.Unlock()
-
 	h.logger.Info("Processed AS peerings",
 		"paths", len(paths),
 		"unique_peerings", len(peerings),

internal/routewatch/peeringhandler_internal_test.go

@@ -31,14 +31,12 @@ type recordingStore struct {
 	calls int
 }
 
-func (r *recordingStore) RecordPeering(
+func (r *recordingStore) RecordPeering(asA, asB int, ts time.Time) error {
-	asA, asB int, seen database.SeenTimes,
-) error {
 	r.mu.Lock()
 	r.calls++
 	r.mu.Unlock()
 
-	return r.mockStore.RecordPeering(asA, asB, seen)
+	return r.mockStore.RecordPeering(asA, asB, ts)
 }
 
 func (r *recordingStore) callCount() int {
@@ -54,7 +52,8 @@ func newTestHandler(db database.Store) *PeeringHandler {
 	return &PeeringHandler{
 		db:      db,
 		logger:  logger.New(),
-		asPaths: make(map[string]database.SeenTimes),
+		asPaths: make(map[string]time.Time),
+		stopCh:  make(chan struct{}),
 	}
 }
 
@@ -153,43 +152,6 @@ func TestPeeringHandlerLeavesATrailingASSetOutOfThePeerings(t *testing.T) {
 	}
 }
 
-// TestPeeringHandlerRecordsNothingFromAPathWithASZero handles AS paths that
-// hold AS 0. RFC 7607 makes such a path malformed, so no peering is recorded
-// from it, not even one between two other ASes of the path.
-func TestPeeringHandlerRecordsNothingFromAPathWithASZero(t *testing.T) {
-	t.Parallel()
-
-	tests := []struct {
-		name string
-		path []any
-	}{
-		{name: "AS 0 between two ASes", path: []any{testASNA, 0, testASNB}},
-		{
-			name: "AS 0 after two ASes",
-			path: []any{testASNA, testASNB, 0, testASNC},
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			store := newMockStore()
-			h := newTestHandler(store)
-			h.HandleMessage(&ristypes.RISMessage{
-				Path:            decodePath(t, tt.path...),
-				ParsedTimestamp: time.Now().UTC(),
-			})
-			h.ProcessPeeringsNow()
-
-			if len(store.Peerings) > 0 {
-				t.Errorf("peerings recorded = %v, want none",
-					slices.Sorted(maps.Keys(store.Peerings)))
-			}
-		})
-	}
-}
-
 // TestPeeringHandlerProcessesEachRunAndEmpties verifies that a processing run
 // empties the path map (the swap) and that a path seen again after a run is
 // recorded in the next run too.
@@ -235,7 +197,7 @@ func TestPeeringHandlerStopWritesTheRemainingPaths(t *testing.T) {
 	h := NewPeeringHandler(store, logger.New())
 
 	handle(h, time.Now().UTC(), testASNA, testASNB)
-	h.Stop(t.Context())
+	h.Stop()
 
 	if got := store.callCount(); got != 1 {
 		t.Errorf("want 1 RecordPeering call when Stop returns, got %d", got)
@@ -259,10 +221,10 @@ func TestPeeringHandlerCapDropsAndCounts(t *testing.T) {
 	existing := pathKey(t, testASNA, testASNB)
 
 	h.mu.Lock()
-	h.asPaths[existing] = database.SeenTimes{First: now, Last: now}
+	h.asPaths[existing] = now
 
 	for i := 0; len(h.asPaths) < maxTrackedPaths; i++ {
-		h.asPaths[strconv.Itoa(i)] = database.SeenTimes{First: now, Last: now}
+		h.asPaths[strconv.Itoa(i)] = now
 	}
 	h.mu.Unlock()
 
@@ -292,83 +254,11 @@ func TestPeeringHandlerCapDropsAndCounts(t *testing.T) {
 	}
 
 	h.mu.Lock()
-	gotSeen := h.asPaths[existing]
+	gotTS := h.asPaths[existing]
 	h.mu.Unlock()
 
-	if !gotSeen.Last.Equal(refreshed) {
+	if !gotTS.Equal(refreshed) {
-		t.Fatalf("existing path last seen not refreshed: want %v, got %v",
+		t.Fatalf("existing path timestamp not refreshed: want %v, got %v",
-			refreshed, gotSeen.Last)
+			refreshed, gotTS)
-	}
-}
-
-// TestPeeringHandlerRunResetsTheDroppedCount fills the map, has the handler
-// drop one path, and runs the processing. The count of dropped paths is the
-// run's own, so it is 0 again after the run.
-func TestPeeringHandlerRunResetsTheDroppedCount(t *testing.T) {
-	t.Parallel()
-
-	h := newTestHandler(newMockStore())
-
-	now := time.Now().UTC()
-
-	// Paths of one AS carry no peering, so the run records nothing.
-	h.mu.Lock()
-	for asn := 1; len(h.asPaths) < maxTrackedPaths; asn++ {
-		h.asPaths[pathKey(t, asn)] = database.SeenTimes{First: now, Last: now}
-	}
-	h.mu.Unlock()
-
-	handle(h, now, testASNA, testASNB)
-
-	if _, dropped := snapshot(h); dropped != 1 {
-		t.Fatalf("want dropped count 1 with the map full, got %d", dropped)
-	}
-
-	h.ProcessPeeringsNow()
-
-	if _, dropped := snapshot(h); dropped != 0 {
-		t.Errorf("want dropped count 0 after a run, got %d", dropped)
-	}
-}
-
-// TestPeeringHandlerStoresTheEarliestAndLatestTimesOfANewPeering handles two
-// updates whose path carries a peering the database does not hold yet, the
-// second 5 seconds after the first, in one run into a real database. The
-// peering is stored first seen at the time of the first update and last seen
-// at the time of the second.
-func TestPeeringHandlerStoresTheEarliestAndLatestTimesOfANewPeering(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	db := newTestDatabase(t)
-	h := newTestHandler(db)
-
-	first := peerAnnouncement(testPeerIP, testV4Prefix)
-	second := peerAnnouncement(testPeerIP, testV4Prefix)
-	second.ParsedTimestamp = first.ParsedTimestamp.Add(secondUpdateAfter)
-
-	h.HandleMessage(first)
-	h.HandleMessage(second)
-	h.ProcessPeeringsNow()
-
-	peers, err := db.GetASPeers(testASNA)
-	if err != nil {
-		t.Fatalf("GetASPeers(%d): %v", testASNA, err)
-	}
-
-	if len(peers) != 1 || peers[0].ASN != testASNB {
-		t.Fatalf("peers of AS%d = %+v, want AS%d alone", testASNA, peers,
-			testASNB)
-	}
-
-	if !peers[0].FirstSeen.Equal(first.ParsedTimestamp) {
-		t.Errorf("first_seen of peering AS%d-AS%d = %s, want %s", testASNA,
-			testASNB, peers[0].FirstSeen, first.ParsedTimestamp)
-	}
-
-	if !peers[0].LastSeen.Equal(second.ParsedTimestamp) {
-		t.Errorf("last_seen of peering AS%d-AS%d = %s, want %s", testASNA,
-			testASNB, peers[0].LastSeen, second.ParsedTimestamp)
 	}
 }

internal/routewatch/prefixhandler.go

@@ -1,7 +1,6 @@
 package routewatch
 
 import (
-	"context"
 	"net"
 	"net/netip"
 	"slices"
@@ -29,9 +28,7 @@ const (
 	// prefixBatchMaxSize is the most prefix updates the batch holds while the
 	// flush goroutine writes the previous one: ten batches, about 60 MiB
 	// (README "Memory"). At this size HandleMessage waits until the flush
-	// goroutine takes the batch. On top of these, the handler holds at most
+	// goroutine takes the batch.
-	// prefixBatchSize updates, about 6 MiB, that a flush kept for the peers
-	// whose route delete failed (held).
 	prefixBatchMaxSize = 250000
 
 	// prefixBatchTimeout is the maximum time to wait before flushing a batch
@@ -44,45 +41,28 @@ const (
 )
 
 // PrefixHandler tracks BGP prefixes and maintains a live routing table in the
-// database. Routes are added on announcement and deleted on withdrawal or on an
+// database. Routes are added on announcement and deleted on withdrawal, and all
-// announcement whose AS path is empty or holds AS 0 anywhere, and all routes of
+// routes of a peer are deleted when its BGP session goes down.
-// a peer are deleted when its BGP session goes down.
 type PrefixHandler struct {
 	db      database.Store
 	logger  *logger.Logger
 	metrics *metrics.Tracker
 
-	// Batching. mu guards batch, held, peersDown, writing and deleting only:
+	// Batching. mu guards batch and peersDown only: the flush goroutine takes
-	// the flush goroutine takes them under mu and writes them after releasing
+	// both under mu and writes them after releasing mu, so HandleMessage
-	// mu, so HandleMessage waits for a database write only while batch is at
+	// waits for a database write only while batch is at prefixBatchMaxSize.
-	// prefixBatchMaxSize.
 	mu    sync.Mutex
 	batch []prefixUpdate
-	// held holds the updates that the peers whose route delete failed in the
+	// peersDown holds the peers whose BGP session went down since the batch
-	// last flush sent after their session went down, for the next flush to
+	// was taken.
-	// write after it retries the delete. They are not in batch, so they do
-	// not count toward a full batch: a delete that keeps failing is retried
-	// at the pace of the flush timer.
-	held []prefixUpdate
-	// peersDown holds, each once, the peers whose BGP session went down
-	// since the batch was taken, and those whose route delete failed in the
-	// last flush.
 	peersDown []string
-	// writing is how many updates the flush goroutine took and has not
-	// finished writing.
-	writing int
-	// deleting is how many peers from peersDown the flush goroutine took and
-	// has not yet deleted the routes of.
-	deleting int
 	// batchTaken is signalled when the flush goroutine takes batch; its L is
 	// &mu.
 	batchTaken  sync.Cond
 	batchFullCh chan struct{} // wakes the flush goroutine when batch is full
 	lastFlush   time.Time     // used only by the flush goroutine
-	// stopCh hands Stop's context to the flush goroutine. It holds one, so
+	stopCh      chan struct{}
-	// Stop does not wait for a write in progress to hand it over.
+	wg          sync.WaitGroup
-	stopCh chan context.Context
-	done   chan struct{} // closed when the flush goroutine returns
 }
 
 type prefixUpdate struct {
@@ -105,12 +85,12 @@ func NewPrefixHandler(db database.Store, logger *logger.Logger) *PrefixHandler {
 		batch:       make([]prefixUpdate, 0, prefixBatchSize),
 		batchFullCh: make(chan struct{}, 1),
 		lastFlush:   time.Now(),
-		stopCh:      make(chan context.Context, 1),
+		stopCh:      make(chan struct{}),
-		done:        make(chan struct{}),
 	}
 	h.batchTaken.L = &h.mu
 
 	// Start the flush timer goroutine
+	h.wg.Add(1)
 	go h.flushLoop()
 
 	return h
@@ -212,64 +192,31 @@ func (h *PrefixHandler) HandleMessage(msg *ristypes.RISMessage) {
 	}
 }
 
-// Stop stops the flush goroutine, which writes the updates and peer downs the
+// Stop gracefully stops the handler and flushes remaining batches
-// handler holds before it returns, and waits for it until ctx ends. Once ctx
+func (h *PrefixHandler) Stop() {
-// has ended the flush goroutine skips that last write, and a write still
+	close(h.stopCh)
-// running is lost when the process exits. A peer whose route delete fails in
+	h.wg.Wait()
-// the last write is not retried either. Stop logs how many updates are not
-// written and how many peer downs have not had their routes deleted.
-func (h *PrefixHandler) Stop(ctx context.Context) {
-	h.stopCh <- ctx
-
-	select {
-	case <-h.done:
-	case <-ctx.Done():
-	}
-
-	h.mu.Lock()
-	unwritten := h.writing + len(h.batch) + len(h.held)
-	peersDown := h.deleting + len(h.peersDown)
-	h.mu.Unlock()
-
-	if unwritten == 0 && peersDown == 0 {
-		return
-	}
-
-	if ctx.Err() != nil {
-		h.logger.Error("Stop deadline passed before the prefix handler wrote "+
-			"its updates", "updates", unwritten, "peers_down", peersDown)
-
-		return
-	}
-
-	h.logger.Error("The prefix handler stopped without deleting the routes "+
-		"of peers whose session went down", "updates", unwritten,
-		"peers_down", peersDown)
 }
 
 // peerSessionDown notes that the peer's BGP session went down, so that the next
 // flush deletes all of the peer's live routes before it writes the batch. The
-// peer's updates still in the batch or held came before the down, which
+// peer's updates still in the batch came before the down, which replaces them,
-// replaces them, so they are dropped; updates the peer sends later stay in the
+// so they are dropped; updates the peer sends later stay in the batch and are
-// batch and are written after the delete. A peer already in peersDown is not
+// written after the delete.
-// added again: one delete removes all of its routes.
 func (h *PrefixHandler) peerSessionDown(peer string) {
 	h.mu.Lock()
 	defer h.mu.Unlock()
 
-	isPeers := func(update prefixUpdate) bool { return update.peer == peer }
+	h.batch = slices.DeleteFunc(h.batch, func(update prefixUpdate) bool {
-	h.batch = slices.DeleteFunc(h.batch, isPeers)
+		return update.peer == peer
-	h.held = slices.DeleteFunc(h.held, isPeers)
+	})
-
-	if !slices.Contains(h.peersDown, peer) {
 	h.peersDown = append(h.peersDown, peer)
 }
-}
 
 // flushLoop runs in a goroutine, the only one that writes the batch: it
 // flushes when the batch is full, periodically, and once more on Stop.
 func (h *PrefixHandler) flushLoop() {
-	defer close(h.done)
+	defer h.wg.Done()
 
 	ticker := time.NewTicker(prefixBatchTimeout)
 	defer ticker.Stop()
@@ -282,38 +229,27 @@ func (h *PrefixHandler) flushLoop() {
 			}
 		case <-h.batchFullCh:
 			h.flushBatch()
-		case ctx := <-h.stopCh:
+		case <-h.stopCh:
-			// Final flush, unless the stop deadline has passed
+			// Final flush
-			if ctx.Err() == nil {
 			h.flushBatch()
-			}
 
 			return
 		}
 	}
 }
 
-// takeBatch returns the pending updates, the held ones first, and the peers in
+// takeBatch returns the pending updates and the peers whose session went down
-// peersDown, and leaves an empty batch, no held updates and no peers in their
+// since the last flush, and leaves an empty batch and no peers in their place
-// place for HandleMessage to fill. The updates count in writing until
+// for HandleMessage to fill.
-// flushBatch has written them, and the peers in deleting until their routes
-// are deleted.
 func (h *PrefixHandler) takeBatch() ([]prefixUpdate, []string) {
 	h.mu.Lock()
 	defer h.mu.Unlock()
 
-	if len(h.held) == 0 && len(h.batch) == 0 && len(h.peersDown) == 0 {
+	if len(h.batch) == 0 && len(h.peersDown) == 0 {
 		return nil, nil
 	}
 
 	batch, peersDown := h.batch, h.peersDown
-	if len(h.held) > 0 {
-		batch = slices.Concat(h.held, h.batch)
-	}
-
-	h.writing = len(batch)
-	h.deleting = len(peersDown)
-	h.held = nil
 	h.batch = make([]prefixUpdate, 0, prefixBatchSize)
 	h.peersDown = nil
 	h.batchTaken.Broadcast()
@@ -333,12 +269,8 @@ func (h *PrefixHandler) flushBatch() {
 	batchSize := len(batch)
 
 	// The routes of a peer whose session went down are deleted first: the
-	// batch holds only the updates the peer sent after the down. A peer whose
+	// batch holds only the updates the peer sent after the down.
-	// delete failed is kept, with those updates, for the next flush to retry.
+	h.deleteLiveRoutesOfPeers(peersDown)
-	failedPeers := h.deleteLiveRoutesOfPeers(peersDown)
-	if len(failedPeers) > 0 {
-		batch = h.keepForNextFlush(batch, failedPeers)
-	}
 
 	// For each prefix and peer, keep only the last update the feed delivered.
 	// Updates from different peers for the same prefix are all kept. Arrival
@@ -348,19 +280,8 @@ func (h *PrefixHandler) flushBatch() {
 
 	lastUpdates := make(map[prefixAndPeer]prefixUpdate)
 
-	// Each prefix the batch names, announced or withdrawn, with the earliest
-	// timestamp among all of its updates, not only each peer's last one: the
-	// prefix tables store only first_seen, written when the prefix is new and
-	// never changed after.
-	prefixesToUpdate := make(map[string]time.Time)
-
 	for _, update := range batch {
 		lastUpdates[prefixAndPeer{update.prefix, update.peer}] = update
-
-		seen, exists := prefixesToUpdate[update.prefix]
-		if !exists || update.timestamp.Before(seen) {
-			prefixesToUpdate[update.prefix] = update.timestamp
-		}
 	}
 
 	// Collect routes to upsert and delete
@@ -369,14 +290,19 @@ func (h *PrefixHandler) flushBatch() {
 		routesToDelete []database.LiveRouteDeletion
 	)
 
+	// Collect unique prefixes to update
+	prefixesToUpdate := make(map[string]time.Time)
+
 	for _, update := range lastUpdates {
+		// Track prefix for both announcements and withdrawals
+		seen, exists := prefixesToUpdate[update.prefix]
+		if !exists || update.timestamp.After(seen) {
+			prefixesToUpdate[update.prefix] = update.timestamp
+		}
+
 		// An announcement is stored when its path gives an origin, or when it
 		// ends in an AS set of several members: that route is stored with
-		// origin 0 (none). Any other update deletes the peer's route for the
+		// origin 0 (none). An announcement with an empty path is not stored.
-		// prefix: a withdrawal, or an announcement whose path is empty or
-		// holds AS 0 anywhere (the decoder empties such a path). Such an
-		// announcement replaces the peer's route, as any announcement does,
-		// with none (RFC 7607 treats AS 0 in a path as a withdrawal).
 		if update.messageType == "announcement" &&
 			(update.originASN > 0 || update.endsInSet) {
 			// Create live route for batch upsert
@@ -384,11 +310,11 @@ func (h *PrefixHandler) flushBatch() {
 			if route != nil {
 				routesToUpsert = append(routesToUpsert, route)
 			}
-		} else {
+		} else if update.messageType == "withdrawal" {
 			// Parse CIDR to get IP version
 			_, ipVersion, err := parseCIDR(update.prefix)
 			if err != nil {
-				h.logger.Error("Failed to parse CIDR for route deletion",
+				h.logger.Error("Failed to parse CIDR for withdrawal",
 					"prefix", update.prefix, "error", err)
 
 				continue
@@ -405,10 +331,6 @@ func (h *PrefixHandler) flushBatch() {
 
 	successCount := h.writeBatch(routesToUpsert, routesToDelete, prefixesToUpdate)
 
-	h.mu.Lock()
-	h.writing = 0
-	h.mu.Unlock()
-
 	elapsed := time.Since(startTime)
 	h.logger.Debug("Flushed prefix batch",
 		"batch_size", batchSize,
@@ -422,97 +344,25 @@ func (h *PrefixHandler) flushBatch() {
 }
 
 // deleteLiveRoutesOfPeers deletes all live routes of each of peers, the peers
-// whose BGP session went down, and returns those whose delete failed. Most
+// whose BGP session went down. Most downs come from peers that have no routes,
-// downs come from peers that have no routes, so only a delete that removed
+// so only a delete that removed routes is logged.
-// routes is logged. A peer whose delete succeeded no longer counts in deleting;
+func (h *PrefixHandler) deleteLiveRoutesOfPeers(peers []string) {
-// a failed one counts until keepForNextFlush puts it back in peersDown.
-func (h *PrefixHandler) deleteLiveRoutesOfPeers(peers []string) []string {
-	var failed []string
-
 	for _, peer := range peers {
 		deleted, err := h.db.DeleteLiveRoutesOfPeer(peer)
 		if err != nil {
 			h.logger.Error(
-				"Failed to delete the live routes of a peer whose session "+
+				"Failed to delete the live routes of a peer whose session went down",
-					"went down; the next flush retries",
+				"peer", peer, "error", err)
-				"peer", peer, "routes_deleted", deleted, "error", err)
-
-			failed = append(failed, peer)
 
 			continue
 		}
 
-		h.mu.Lock()
-		h.deleting--
-		h.mu.Unlock()
-
 		if deleted > 0 {
 			h.logger.Info(
 				"Deleted the live routes of a peer whose session went down",
 				"peer", peer, "routes", deleted)
 		}
 	}
-
-	return failed
-}
-
-// keepForNextFlush puts the peers whose route delete failed back in peersDown,
-// and their updates from batch in held, so that the next flush retries each
-// delete before it writes the peer's updates; written now, they would be
-// deleted by the retry. The updates of a peer whose session went down again
-// since the batch was taken came before that down, so they are dropped, as
-// peerSessionDown drops them, and the peer is already in peersDown. It returns
-// the updates of the other peers, for this flush to write.
-//
-// While a delete keeps failing, the held updates would grow with every flush,
-// so only the last prefixBatchSize of them are kept and the older ones are
-// dropped. This never leaves in place a route the peer withdrew: a prefix with
-// a kept update keeps its last one, and the retry deletes all of the peer's
-// routes, so a prefix whose updates were all dropped has no route from the peer
-// until the peer announces it again.
-func (h *PrefixHandler) keepForNextFlush(
-	batch []prefixUpdate, failedPeers []string,
-) []prefixUpdate {
-	var kept, toWrite []prefixUpdate
-
-	for _, update := range batch {
-		if slices.Contains(failedPeers, update.peer) {
-			kept = append(kept, update)
-		} else {
-			toWrite = append(toWrite, update)
-		}
-	}
-
-	h.mu.Lock()
-	defer h.mu.Unlock()
-
-	kept = slices.DeleteFunc(kept, func(update prefixUpdate) bool {
-		return slices.Contains(h.peersDown, update.peer)
-	})
-
-	if len(kept) > prefixBatchSize {
-		h.logger.Error(
-			"Dropped the oldest updates of peers whose route delete failed",
-			"peers", failedPeers, "dropped", len(kept)-prefixBatchSize)
-
-		// A copy, so that the dropped updates are freed.
-		kept = slices.Clone(kept[len(kept)-prefixBatchSize:])
-	}
-
-	// takeBatch left held empty, and only this flush fills it. The kept
-	// updates now count in held, not in writing, and the failed peers in
-	// peersDown, not in deleting.
-	h.held = kept
-	h.writing = len(toWrite)
-	h.deleting = 0
-
-	for _, peer := range failedPeers {
-		if !slices.Contains(h.peersDown, peer) {
-			h.peersDown = append(h.peersDown, peer)
-		}
-	}
-
-	return toWrite
 }
 
 // writeBatch writes the collected route upserts, route deletions and prefix

internal/routewatch/prefixhandler_internal_test.go

@@ -1,13 +1,10 @@
 package routewatch
 
 import (
-	"bytes"
 	"context"
 	"errors"
 	"fmt"
-	"net/netip"
 	"slices"
-	"strings"
 	"sync"
 	"testing"
 	"time"
@@ -26,10 +23,6 @@ const (
 	testV6Prefix = "2001:db8:1::/48"
 )
 
-// secondUpdateAfter is how long after the first update the tests of first_seen
-// and last_seen send the second, in the same batch.
-const secondUpdateAfter = 5 * time.Second
-
 // TestPrefixHandlerStoresPrefixesTheIPLookupFinds runs announcements and
 // withdrawals through the prefix handler into a real database and looks the
 // addresses up. The prefixes are written the way the feed sends them: IPv6
@@ -87,49 +80,6 @@ func TestPrefixHandlerStoresPrefixesTheIPLookupFinds(t *testing.T) {
 	}
 }
 
-// TestPrefixHandlerStoresANewPrefixFirstSeenAtItsEarliestUpdate announces a
-// prefix the database does not hold yet twice in one batch, the second time 5
-// seconds after the first, through the prefix handler into a real database. The
-// prefix is stored first seen at the time of the first announcement, whether
-// the two come from two peers or from one, and the IP lookup answers with that
-// time.
-func TestPrefixHandlerStoresANewPrefixFirstSeenAtItsEarliestUpdate(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	const otherPeerIP = "2001:db8:ffff::2"
-
-	tests := []struct {
-		name                  string
-		firstPeer, secondPeer string
-	}{
-		{name: "two peers", firstPeer: testPeerIP, secondPeer: otherPeerIP},
-		{name: "one peer", firstPeer: testPeerIP, secondPeer: testPeerIP},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			first := peerAnnouncement(tt.firstPeer, testV4Prefix)
-			second := peerAnnouncement(tt.secondPeer, testV4Prefix)
-			second.ParsedTimestamp = first.ParsedTimestamp.Add(secondUpdateAfter)
-			db := handleBatches(t, [][]*ristypes.RISMessage{{first, second}})
-
-			info, err := db.GetIPInfo("192.0.2.1")
-			if err != nil {
-				t.Fatalf("GetIPInfo(192.0.2.1): %v", err)
-			}
-
-			if !info.FirstSeen.Equal(first.ParsedTimestamp) {
-				t.Errorf("first_seen of %s = %s, want %s", testV4Prefix,
-					info.FirstSeen, first.ParsedTimestamp)
-			}
-		})
-	}
-}
-
 // TestPrefixHandlerAppliesEachPeersLastUpdateInABatch runs updates for one
 // prefix through the prefix handler into a real database. All updates carry the
 // same timestamp, as updates within one second of the feed do. Within a batch,
@@ -378,133 +328,6 @@ func TestPrefixHandlerDoesNotStoreAnAnnouncementWithAnEmptyPath(t *testing.T) {
 	assertPeers(t, db, map[string][]string{testV4Prefix: nil})
 }
 
-// TestPrefixHandlerRemovesTheRouteOnAnAnnouncementWithoutAnOrigin runs, through
-// the prefix handler into a real database, a peer's announcement of a prefix
-// with origin AS testASNB and then the peer's announcement of the prefix with
-// an AS path that gives no origin: one whose last AS is 0, an empty one, or one
-// that holds AS 0 before its end, which decodes as an empty path. As any later
-// announcement from the peer does, the second replaces the peer's route, here
-// with none, whether it comes in a later batch or follows a withdrawal in one
-// batch. The in-memory route count and prefix distribution must match the
-// route table.
-func TestPrefixHandlerRemovesTheRouteOnAnAnnouncementWithoutAnOrigin(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	const prefixMaskLength = 24
-
-	noOrigin := func(path ristypes.ASPath) *ristypes.RISMessage {
-		msg := peerAnnouncement(testPeerIP, testV4Prefix)
-		msg.Path = path
-
-		return msg
-	}
-	endsInZero := ristypes.ASPath{Sequence: []int{testASNA, 0}}
-	withdrawal := &ristypes.RISMessage{
-		Type:            ristypes.MessageTypeUpdate,
-		Peer:            testPeerIP,
-		ParsedTimestamp: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC),
-		Withdrawals:     []string{testV4Prefix},
-	}
-
-	tests := []struct {
-		name string
-		// Handled and flushed as one batch after the announcement with origin
-		// AS testASNB.
-		batch []*ristypes.RISMessage
-	}{
-		{
-			name:  "a path whose last AS is 0",
-			batch: []*ristypes.RISMessage{noOrigin(endsInZero)},
-		},
-		{
-			name:  "an empty path",
-			batch: []*ristypes.RISMessage{noOrigin(ristypes.ASPath{})},
-		},
-		{
-			name: "a path with AS 0 before its end",
-			batch: []*ristypes.RISMessage{
-				noOrigin(decodePath(t, testASNA, 0, testASNB)),
-			},
-		},
-		{
-			name:  "a withdrawal and then a path whose last AS is 0",
-			batch: []*ristypes.RISMessage{withdrawal, noOrigin(endsInZero)},
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			db := handleBatches(t, [][]*ristypes.RISMessage{
-				{peerAnnouncement(testPeerIP, testV4Prefix)},
-				tt.batch,
-			})
-			checkLivePeers(t, db, testV4Prefix, prefixMaskLength, nil)
-		})
-	}
-}
-
-// TestPrefixHandlerDoesNotStoreAPrefixThatDoesNotParse runs an announcement of
-// "not-a-prefix" and a withdrawal of "also/bad" through the prefix handler into
-// a real database. Neither is stored in the prefix tables: the prefix count in
-// the stats stays 0, both as kept in memory and as counted from the tables when
-// the database is opened again.
-func TestPrefixHandlerDoesNotStoreAPrefixThatDoesNotParse(t *testing.T) {
-	t.Parallel()
-
-	cfg := &config.Config{StateDir: t.TempDir()}
-
-	db, err := database.New(cfg, logger.New())
-	if err != nil {
-		t.Fatalf("failed to create database: %v", err)
-	}
-
-	// Built without NewPrefixHandler's flush timer, so the test flushes itself.
-	h := &PrefixHandler{db: db, logger: logger.New()}
-
-	withdrawal := &ristypes.RISMessage{
-		Type:            ristypes.MessageTypeUpdate,
-		Peer:            testPeerIP,
-		ParsedTimestamp: time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC),
-		Withdrawals:     []string{"also/bad"},
-	}
-	h.HandleMessage(peerAnnouncement(testPeerIP, "not-a-prefix"))
-	h.HandleMessage(withdrawal)
-	h.flushBatch()
-
-	checkNoPrefixes := func(when string) {
-		t.Helper()
-
-		stats, err := db.GetStatsContext(context.Background())
-		if err != nil {
-			t.Fatalf("GetStatsContext %s: %v", when, err)
-		}
-
-		if stats.Prefixes != 0 {
-			t.Errorf("prefix count %s = %d, want 0", when, stats.Prefixes)
-		}
-	}
-
-	checkNoPrefixes("after the flush")
-
-	err = db.Close()
-	if err != nil {
-		t.Fatalf("Close: %v", err)
-	}
-
-	db, err = database.New(cfg, logger.New())
-	if err != nil {
-		t.Fatalf("failed to open the database again: %v", err)
-	}
-
-	t.Cleanup(func() { _ = db.Close() })
-
-	checkNoPrefixes("after opening the database again")
-}
-
 // TestStateMessagesGoToThePrefixHandlerNotThePeerHandler checks that STATE
 // messages go to the prefix handler, which acts on a down, and not to the peer
 // handler, which would record a peer whose session never comes up as active.
@@ -737,258 +560,6 @@ func TestPrefixHandlerHandlesADownWhileAFlushWrites(t *testing.T) {
 	})
 }
 
-// errTestPeerDelete is the error a test store's failing delete of a peer's
-// routes returns.
-var errTestPeerDelete = errors.New("test: the delete of a peer's routes failed")
-
-// firstPeerDeleteFailsStore is a real database whose first delete of a peer's
-// live routes fails, deleting nothing. deletes counts the deletes asked for.
-type firstPeerDeleteFailsStore struct {
-	*database.Database
-
-	// whileFailing, if not nil, runs during the delete that fails.
-	whileFailing func()
-	deletes      int
-}
-
-// DeleteLiveRoutesOfPeer fails the first call and deletes the peer's routes on
-// every later one.
-func (s *firstPeerDeleteFailsStore) DeleteLiveRoutesOfPeer(
-	peerIP string,
-) (int, error) {
-	s.deletes++
-	if s.deletes > 1 {
-		return s.Database.DeleteLiveRoutesOfPeer(peerIP)
-	}
-
-	if s.whileFailing != nil {
-		s.whileFailing()
-	}
-
-	return 0, errTestPeerDelete
-}
-
-// TestPrefixHandlerRetriesAFailedDeleteOfAPeersRoutes runs, through the prefix
-// handler into a database whose first delete of a peer's routes fails, one flush
-// after another: a peer's announcement, then its session going down and a new
-// announcement from the peer, then nothing. The failed delete must be retried
-// on the next flush, and the announcement the peer sent after the down written
-// only after that retry, so it is kept. In the other cases the peer's session
-// goes down again, and the peer announces another prefix, while the delete
-// fails or after the flush whose delete failed: the announcement sent between
-// the two downs must not be written, and the next flush must delete the peer's
-// routes once, not once for the retry and once for the down.
-func TestPrefixHandlerRetriesAFailedDeleteOfAPeersRoutes(t *testing.T) {
-	t.Parallel()
-
-	const (
-		beforeDown      = "192.0.2.0/24"
-		afterDown       = "198.51.100.0/24"
-		afterSecondDown = "203.0.113.0/24"
-		maskLength      = 24
-		// The delete that fails and its retry, which is also the delete for
-		// a second down.
-		wantDeletes = 2
-	)
-
-	tests := []struct {
-		name string
-		// Handled while the delete fails, if not nil.
-		whileFailing []*ristypes.RISMessage
-		// Handled after the flush whose delete fails, if not nil.
-		afterFailing []*ristypes.RISMessage
-		// The peers left with a live route for each prefix.
-		wantPeers map[string][]string
-	}{
-		{
-			name: "the delete fails once",
-			wantPeers: map[string][]string{
-				beforeDown: nil, afterDown: {testPeerIP}, afterSecondDown: nil,
-			},
-		},
-		{
-			name: "the session goes down again while the delete fails",
-			whileFailing: []*ristypes.RISMessage{
-				sessionDown(), peerAnnouncement(testPeerIP, afterSecondDown),
-			},
-			wantPeers: map[string][]string{
-				beforeDown: nil, afterDown: nil, afterSecondDown: {testPeerIP},
-			},
-		},
-		{
-			name: "the session goes down again after the delete failed",
-			afterFailing: []*ristypes.RISMessage{
-				sessionDown(), peerAnnouncement(testPeerIP, afterSecondDown),
-			},
-			wantPeers: map[string][]string{
-				beforeDown: nil, afterDown: nil, afterSecondDown: {testPeerIP},
-			},
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			t.Parallel()
-
-			db := newTestDatabase(t)
-			store := &firstPeerDeleteFailsStore{Database: db}
-
-			// Built without NewPrefixHandler's flush timer, so the test
-			// flushes itself.
-			h := &PrefixHandler{db: store, logger: logger.New()}
-			store.whileFailing = func() {
-				for _, msg := range tt.whileFailing {
-					h.HandleMessage(msg)
-				}
-			}
-
-			h.HandleMessage(peerAnnouncement(testPeerIP, beforeDown))
-			h.flushBatch()
-			h.HandleMessage(sessionDown())
-			h.HandleMessage(peerAnnouncement(testPeerIP, afterDown))
-			h.flushBatch() // the delete fails
-
-			for _, msg := range tt.afterFailing {
-				h.HandleMessage(msg)
-			}
-
-			h.flushBatch()
-
-			if store.deletes != wantDeletes {
-				t.Errorf("the store was asked for %d deletes of the peer's "+
-					"routes, want %d", store.deletes, wantDeletes)
-			}
-
-			assertPeers(t, db, tt.wantPeers)
-			assertCounts(t, db, ipv4Version, 1, []database.PrefixDistribution{
-				{MaskLength: maskLength, Count: 1},
-			})
-		})
-	}
-}
-
-// peerDeleteAlwaysFailsStore is a real database whose every delete of a peer's
-// live routes fails, deleting nothing.
-type peerDeleteAlwaysFailsStore struct {
-	*database.Database
-}
-
-// DeleteLiveRoutesOfPeer fails.
-func (peerDeleteAlwaysFailsStore) DeleteLiveRoutesOfPeer(string) (int, error) {
-	return 0, errTestPeerDelete
-}
-
-// TestPrefixHandlerKeepsAtMostABatchOfUpdatesWhileADeleteKeepsFailing runs,
-// through the prefix handler into a database whose every delete of a peer's
-// routes fails, a peer's session going down and then three flushes. Before each
-// flush the peer announces prefixBatchSize new prefixes and another peer
-// announces one. After each flush the handler must hold only the peer's last
-// prefixBatchSize updates, not every update since the down, and the other
-// peer's route must be written. After the last flush, one more update must not
-// wake the flush goroutine: held updates do not count toward a full batch.
-func TestPrefixHandlerKeepsAtMostABatchOfUpdatesWhileADeleteKeepsFailing(
-	t *testing.T,
-) {
-	t.Parallel()
-
-	const otherPeerIP = "2001:db8:ffff::2"
-
-	// One prefix the other peer announces before each flush.
-	otherPeersPrefixes := []string{testV4Prefix, testV6Prefix, "2001:db8:2::/48"}
-
-	db := newTestDatabase(t)
-
-	// Built without NewPrefixHandler's flush timer, so the test flushes itself.
-	h := &PrefixHandler{
-		db: peerDeleteAlwaysFailsStore{Database: db}, logger: logger.New(),
-	}
-	h.HandleMessage(sessionDown())
-
-	// The peer announces a /32 for each address from 10.0.0.0 on, each
-	// address once.
-	addr := netip.MustParseAddr("10.0.0.0")
-
-	for flush, otherPeersPrefix := range otherPeersPrefixes {
-		prefixes := make([]string, prefixBatchSize)
-		for i := range prefixes {
-			prefixes[i] = netip.PrefixFrom(addr, addr.BitLen()).String()
-			addr = addr.Next()
-		}
-
-		h.HandleMessage(peerAnnouncement(testPeerIP, prefixes...))
-		h.HandleMessage(peerAnnouncement(otherPeerIP, otherPeersPrefix))
-		h.flushBatch()
-
-		kept := make([]string, 0, len(h.held))
-		for _, update := range h.held {
-			kept = append(kept, update.prefix)
-		}
-
-		if !slices.Equal(kept, prefixes) {
-			t.Fatalf("after flush %d the handler holds %d updates, not just "+
-				"the peer's last %d", flush+1, len(kept), len(prefixes))
-		}
-
-		assertPeers(t, db, map[string][]string{otherPeersPrefix: {otherPeerIP}})
-	}
-
-	// A new channel, so that a wake-up in it comes from the update below.
-	h.batchFullCh = make(chan struct{}, 1)
-	h.HandleMessage(peerAnnouncement(otherPeerIP, testV4Prefix))
-
-	if len(h.batchFullCh) > 0 {
-		t.Error("one update after a flush that held a batch of updates " +
-			"woke the flush goroutine")
-	}
-}
-
-// TestPrefixHandlerStopLogsAFailedDeleteWithoutTheDeadline stops a prefix
-// handler, with no deadline, after a peer's session went down and the peer
-// announced a prefix, into a database whose every delete of a peer's routes
-// fails. Stop must log the peer down and the announcement as not written, and
-// must not say that the stop deadline passed.
-func TestPrefixHandlerStopLogsAFailedDeleteWithoutTheDeadline(t *testing.T) {
-	t.Parallel()
-
-	var logs bytes.Buffer
-
-	h := NewPrefixHandler(peerDeleteAlwaysFailsStore{Database: newTestDatabase(t)},
-		newErrorLogger(&logs))
-	h.HandleMessage(sessionDown())
-	h.HandleMessage(peerAnnouncement(testPeerIP, testV4Prefix))
-	h.Stop(t.Context())
-
-	logged := lastLogged(t, &logs)
-	if strings.Contains(logged.Msg, "deadline") {
-		t.Errorf("Stop logged %q although its deadline had not passed",
-			logged.Msg)
-	}
-
-	if logged.Updates != 1 || logged.PeersDown != 1 {
-		t.Errorf("Stop logged %d updates and %d peer downs not written, "+
-			"want 1 and 1", logged.Updates, logged.PeersDown)
-	}
-}
-
-// TestPrefixHandlerStopLogsNothingAfterADeleteThatSucceeds stops a prefix
-// handler, with no deadline, after a peer's session went down, into a database
-// whose delete of the peer's routes succeeds. Stop must log nothing: a peer
-// whose routes are deleted no longer counts as a peer down left undone.
-func TestPrefixHandlerStopLogsNothingAfterADeleteThatSucceeds(t *testing.T) {
-	t.Parallel()
-
-	var logs bytes.Buffer
-
-	h := NewPrefixHandler(newTestDatabase(t), newErrorLogger(&logs))
-	h.HandleMessage(sessionDown())
-	h.Stop(t.Context())
-
-	if logs.Len() != 0 {
-		t.Errorf("Stop logged %q after the routes of the peer whose session "+
-			"went down were deleted", logs.String())
-	}
-}
-
 // newTestDatabase opens a database in a new temporary directory. It is closed
 // when the test ends.
 func newTestDatabase(t *testing.T) *database.Database {

internal/server/handlers.go

@@ -1,7 +1,7 @@
 package server
 
 import (
-	"cmp"
+	"bytes"
 	"context"
 	"encoding/json"
 	"errors"
@@ -9,16 +9,13 @@ import (
 	"net/http"
 	"net/netip"
 	"runtime"
-	"slices"
 	"sort"
 	"strconv"
 	"strings"
 	"time"
 
-	"git.eeqj.de/sneak/routewatch/internal/blocklist"
 	"git.eeqj.de/sneak/routewatch/internal/database"
 	"git.eeqj.de/sneak/routewatch/internal/metrics"
-	"git.eeqj.de/sneak/routewatch/internal/ristypes"
 	"git.eeqj.de/sneak/routewatch/internal/streamer"
 	"git.eeqj.de/sneak/routewatch/internal/templates"
 	asinfo "git.eeqj.de/sneak/routewatch/pkg/asinfo"
@@ -34,27 +31,6 @@ const (
 	healthCheckTimeout = 2 * time.Second
 )
 
-// Limits of one IP lookup.
-const (
-	// maxLookupTargets is the most addresses and host names an IP lookup may
-	// list, and the most addresses it may have once the names are resolved:
-	// each address costs up to 113 database reads and a PTR lookup.
-	maxLookupTargets = 10
-
-	// ptrLookupTimeout is each PTR lookup's own time limit, so that ten
-	// addresses whose PTR lookups never answer fit in the 30-second limit.
-	ptrLookupTimeout = 2 * time.Second
-)
-
-// The Cache-Control header of an IP lookup answer, such as
-// "public, max-age=900".
-const (
-	cacheControlHeader  = "Cache-Control"
-	cacheControlPublic  = "public"
-	cacheControlPrivate = "private"
-	cacheControlMaxAge  = ", max-age="
-)
-
 // Keys of the JSON envelope the handlers and middleware write, and the status
 // value of an error answer.
 const (
@@ -65,23 +41,6 @@ const (
 	jsonStatusError = "error"
 )
 
-// Error messages an answer sends, in place of the error behind it where there
-// is one.
-const (
-	invalidPrefixMessage      = "Invalid prefix"
-	hostNotResolvedMessage    = "could not resolve host name"
-	hostNotPublicMessage      = "host name has no public address"
-	tooManyTargetsMessage     = "Too many addresses or host names"
-	searchQueryMissingMessage = "Missing search query"
-)
-
-// The AS name search reads the text it looks for from the URL query parameter
-// asSearchQueryParam, and answers with at most maxASSearchResults matches.
-const (
-	asSearchQueryParam = "q"
-	maxASSearchResults = 50
-)
-
 // IP versions as database.LiveRoute.IPVersion records them.
 const (
 	ipVersionV4 = 4
@@ -109,9 +68,7 @@ func (s *Server) handleHealthCheck() http.HandlerFunc {
 		// Check database connectivity with lightweight ping
 		err := s.db.Ping(ctx)
 		if err != nil {
-			s.logger.Error("Health check database ping failed", "error", err)
+			checks["database"] = "error: " + err.Error()
-
-			checks["database"] = jsonStatusError
 			healthy = false
 		} else {
 			checks["database"] = "ok"
@@ -151,15 +108,12 @@ func (s *Server) handleHealthCheck() http.HandlerFunc {
 
 // handleIndex returns a handler that serves the home page.
 func (s *Server) handleIndex() http.HandlerFunc {
-	return func(w http.ResponseWriter, r *http.Request) {
+	return func(w http.ResponseWriter, _ *http.Request) {
 		w.Header().Set("Content-Type", "text/html; charset=utf-8")
 
 		tmpl := templates.IndexTemplate()
 
-		err := tmpl.Execute(w, indexPageData{
+		err := tmpl.Execute(w, nil)
-			CSPNonce: cspNonce(r.Context()),
-			ClientIP: extractClientIP(r),
-		})
 		if err != nil {
 			s.logger.Error("Failed to render index template", "error", err)
 			http.Error(w, "Internal Server Error", http.StatusInternalServerError)
@@ -167,21 +121,6 @@ func (s *Server) handleIndex() http.HandlerFunc {
 	}
 }
 
-// pageData is what the pages that show no data are rendered with: the nonce
-// their script and style blocks carry.
-type pageData struct {
-	CSPNonce string
-}
-
-// indexPageData is what the home page is rendered with.
-type indexPageData struct {
-	// CSPNonce is the nonce the page's script and style blocks carry.
-	CSPNonce string
-	// ClientIP is the visitor's address, which the IP information API card
-	// looks up: the address /ip/ would look up for the same request.
-	ClientIP string
-}
-
 // writeJSONError writes a standardized JSON error response with the given
 // status code and error message.
 func (s *Server) writeJSONError(w http.ResponseWriter, statusCode int, message string) {
@@ -312,7 +251,7 @@ func (s *Server) handleStatusJSON() http.HandlerFunc {
 
 		if err != nil {
 			s.logger.Error("Failed to get database stats", "error", err)
-			s.writeJSONError(w, http.StatusInternalServerError, internalErrorMessage)
+			s.writeJSONError(w, http.StatusInternalServerError, err.Error())
 
 			return
 		}
@@ -466,7 +405,7 @@ type GCStats struct {
 type StreamStats struct {
 	Announcements    uint64  `json:"announcements"`
 	Withdrawals      uint64  `json:"withdrawals"`
-	RouteChurnPerMin int    `json:"route_churn_per_min"`
+	RouteChurnPerSec float64 `json:"route_churn_per_sec"`
 	BGPPeerCount     int     `json:"bgp_peer_count"`
 }
 
@@ -568,7 +507,7 @@ func (s *Server) streamStats() StreamStats {
 	return StreamStats{
 		Announcements:    metricsTracker.GetAnnouncementCount(),
 		Withdrawals:      metricsTracker.GetWithdrawalCount(),
-		RouteChurnPerMin: metricsTracker.GetChurnPerMinute(),
+		RouteChurnPerSec: metricsTracker.GetChurnRate(),
 		BGPPeerCount:     metricsTracker.GetBGPPeerCount(),
 	}
 }
@@ -602,7 +541,7 @@ func (s *Server) handleStats() http.HandlerFunc {
 
 		if err != nil {
 			s.logger.Error("Failed to get database stats", "error", err)
-			s.writeJSONError(w, http.StatusInternalServerError, internalErrorMessage)
+			s.writeJSONError(w, http.StatusInternalServerError, err.Error())
 
 			return
 		}
@@ -683,12 +622,12 @@ func (s *Server) statsResponse(
 // handleStatusHTML returns a handler that serves the HTML status page,
 // which displays real-time statistics fetched via JavaScript.
 func (s *Server) handleStatusHTML() http.HandlerFunc {
-	return func(w http.ResponseWriter, r *http.Request) {
+	return func(w http.ResponseWriter, _ *http.Request) {
 		w.Header().Set("Content-Type", "text/html; charset=utf-8")
 
 		tmpl := templates.StatusTemplate()
 
-		err := tmpl.Execute(w, pageData{CSPNonce: cspNonce(r.Context())})
+		err := tmpl.Execute(w, nil)
 		if err != nil {
 			s.logger.Error("Failed to render template", "error", err)
 			http.Error(w, "Internal Server Error", http.StatusInternalServerError)
@@ -704,32 +643,17 @@ func (s *Server) handleIPLookup() http.HandlerFunc {
 // IPLookupResponse is the standard response for IP/hostname lookups.
 type IPLookupResponse struct {
 	Query   string             `json:"query"`
-	Results []IPLookupResult `json:"results"`
+	Results []*database.IPInfo `json:"results"`
 	Errors  []string           `json:"errors,omitempty"`
 }
 
-// IPLookupResult is the answer for one address: its PTR names, its live route
-// and that route's AS, and the blocklists that list it. IPInfo is nil when
-// the address has no live route, and the answer then has no route or AS
-// fields. IP hides the IPInfo field of the same name, so that an address with
-// no route has it too.
-type IPLookupResult struct {
-	*database.IPInfo
-
-	IP  string   `json:"ip"`
-	PTR []string `json:"ptr,omitempty"`
-
-	// Blocklists has one entry for each blocklist the address is on.
-	Blocklists []blocklist.Listing `json:"blocklists"`
-}
-
 // handleIPInfo returns a handler that provides comprehensive IP information.
 // Used for /ip, /ip/{addr}, and /api/v1/ip/{ip} endpoints.
 // Accepts IP addresses (single or comma-separated) and hostnames.
 // Always returns the same response structure with PTR records for each IP.
 func (s *Server) handleIPInfo() http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
-		target, ownAddress := lookupTarget(r)
+		target := lookupTarget(r)
 		if target == "" {
 			s.writeJSONError(w, http.StatusBadRequest, "Could not determine IP address")
 
@@ -737,18 +661,15 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
 		}
 
 		ctx := r.Context()
-		response := IPLookupResponse{Query: target, Results: make([]IPLookupResult, 0)}
+		response := IPLookupResponse{
-
+			Query:   target,
-		// Collect all IPs to look up
+			Results: make([]*database.IPInfo, 0),
-		ipsToLookup, resolveErrors, err := s.resolveLookupTargets(ctx, target)
-		if deadlinePassed(ctx) {
-			s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
-
-			return
 		}
 
-		if err != nil {
+		// Collect all IPs to look up
-			s.writeJSONError(w, http.StatusBadRequest, tooManyTargetsMessage)
+		ipsToLookup, resolveErrors := resolveLookupTargets(ctx, target)
+		if deadlinePassed(ctx) {
+			s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
 
 			return
 		}
@@ -756,7 +677,7 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
 		response.Errors = append(response.Errors, resolveErrors...)
 
 		if len(ipsToLookup) == 0 {
-			s.writeJSONError(w, http.StatusBadRequest, noTargetsMessage(resolveErrors))
+			s.writeJSONError(w, http.StatusBadRequest, "No valid IPs or hostnames provided")
 
 			return
 		}
@@ -773,37 +694,27 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
 				return
 			}
 
-			switch {
+			if err != nil {
-			case errors.Is(err, database.ErrNoRoute):
-				// The address still gets a result, without route fields.
-				ipInfo = nil
-			case errors.Is(err, database.ErrInvalidIP):
 				response.Errors = append(response.Errors, ip+": "+err.Error())
 
 				continue
-			case err != nil:
-				s.logger.Error("Failed to look up IP", "ip", ip, "error", err)
-				s.writeJSONError(w, http.StatusInternalServerError, internalErrorMessage)
-
-				return
 			}
 
 			// Do PTR lookup for this IP
-			ptrs := s.lookupPTRNames(ctx, ip)
+			ptrs := lookupPTRNames(ctx, ip)
 			if deadlinePassed(ctx) {
 				s.writeJSONError(w, http.StatusGatewayTimeout, requestTimeoutMessage)
 
 				return
 			}
 
-			response.Results = append(response.Results, IPLookupResult{
+			if len(ptrs) > 0 {
-				IP:         ip,
+				ipInfo.PTR = ptrs
-				PTR:        ptrs,
+			}
-				IPInfo:     ipInfo,
-				Blocklists: s.blocklistListings(ip),
-			})
 
-			if ipInfo != nil && ipInfo.NeedsWHOISRefresh {
+			response.Results = append(response.Results, ipInfo)
+
+			if ipInfo.NeedsWHOISRefresh {
 				refreshASNs[ipInfo.ASN] = true
 			}
 		}
@@ -811,70 +722,23 @@ func (s *Server) handleIPInfo() http.HandlerFunc {
 		// Queue WHOIS refresh for stale ASNs (non-blocking)
 		s.queueWHOISRefresh(refreshASNs)
 
-		s.writeIPLookupAnswer(w, response, ownAddress)
+		// Return response (even if no results, include errors)
-	}
+		if len(response.Results) == 0 && len(response.Errors) > 0 {
-}
+			s.writeJSONError(w, http.StatusNotFound, "No routes found: "+response.Errors[0])
 
-// blocklistListings returns the blocklists that list ip. It returns an empty
-// list, never nil, so that an answer's blocklists is [] rather than null.
-func (s *Server) blocklistListings(ip string) []blocklist.Listing {
-	addr, err := netip.ParseAddr(ip)
-	if s.blocklists == nil || err != nil {
-		return []blocklist.Listing{}
-	}
-
-	return s.blocklists.Lookup(addr)
-}
-
-// setIPLookupCacheControl lets caches keep an IP lookup answer for the
-// shortest blocklist interval, the soonest a blocklist copy can change. The
-// answer about the caller's own address is private, so that a shared cache
-// does not give it to another caller. Without blocklists it sets nothing.
-func (s *Server) setIPLookupCacheControl(w http.ResponseWriter, ownAddress bool) {
-	if s.blocklists == nil {
 			return
 		}
 
-	visibility := cacheControlPublic
-	if ownAddress {
-		visibility = cacheControlPrivate
-	}
-
-	maxAgeSeconds := int(s.blocklists.ShortestInterval().Seconds())
-	w.Header().Set(cacheControlHeader,
-		visibility+cacheControlMaxAge+strconv.Itoa(maxAgeSeconds))
-}
-
-// noTargetsMessage returns the message of the 400 an IP lookup answers when
-// no address is left to look up, followed by the first of resolveErrors, which
-// says why, when there is one.
-func noTargetsMessage(resolveErrors []string) string {
-	const message = "No valid IPs or hostnames provided"
-
-	if len(resolveErrors) == 0 {
-		return message
-	}
-
-	return message + ": " + resolveErrors[0]
-}
-
-// writeIPLookupAnswer sends response with its Cache-Control header.
-// ownAddress says the request asked about the caller's own address.
-func (s *Server) writeIPLookupAnswer(
-	w http.ResponseWriter, response IPLookupResponse, ownAddress bool,
-) {
-	s.setIPLookupCacheControl(w, ownAddress)
-
 		err := writeJSONSuccess(w, response)
 		if err != nil {
 			s.logger.Error("Failed to encode IP lookup response", "error", err)
 		}
 	}
+}
 
 // lookupTarget returns what an IP lookup request asks for: the ip or addr URL
-// parameter, or else the client's own address. The bool is true for the
+// parameter, or else the client's own address.
-// client's own address.
+func lookupTarget(r *http.Request) string {
-func lookupTarget(r *http.Request) (string, bool) {
 	target := chi.URLParam(r, "ip")
 	if target == "" {
 		target = chi.URLParam(r, "addr")
@@ -882,10 +746,10 @@ func lookupTarget(r *http.Request) (string, bool) {
 
 	if target == "" {
 		// Use client IP (RealIP middleware has already processed this)
-		return extractClientIP(r), true
+		target = extractClientIP(r)
 	}
 
-	return target, false
+	return target
 }
 
 // queueWHOISRefresh asks the WHOIS fetcher, when there is one, to refresh
@@ -900,16 +764,12 @@ func (s *Server) queueWHOISRefresh(asns map[int]bool) {
 	}
 }
 
-// errTooManyTargets is what resolveLookupTargets returns for a list of more
+// resolveLookupTargets splits a comma-separated list of IP addresses and
-// than maxLookupTargets addresses and host names, or of names that resolve to
+// hostnames and returns the IP addresses to look up, resolving each hostname.
-// more than maxLookupTargets addresses.
+// A hostname that does not resolve is reported in the returned errors instead.
-var errTooManyTargets = errors.New("too many addresses or host names")
+// It stops once ctx's deadline has passed; the caller then answers 504.
-
+func resolveLookupTargets(ctx context.Context, target string) ([]string, []string) {
-// splitLookupTargets returns the addresses and host names in a comma-separated
+	var ipsToLookup, resolveErrors []string
-// list, leaving out blank entries, or errTooManyTargets when there are more
-// than maxLookupTargets.
-func splitLookupTargets(target string) ([]string, error) {
-	var targets []string
 
 	for t := range strings.SplitSeq(target, ",") {
 		t = strings.TrimSpace(t)
@@ -917,36 +777,6 @@ func splitLookupTargets(target string) ([]string, error) {
 			continue
 		}
 
-		if len(targets) == maxLookupTargets {
-			return nil, errTooManyTargets
-		}
-
-		targets = append(targets, t)
-	}
-
-	return targets, nil
-}
-
-// resolveLookupTargets splits a comma-separated list of IP addresses and
-// hostnames and returns the IP addresses to look up, resolving each hostname.
-// A hostname that does not resolve is reported in the returned errors instead,
-// with the resolver's own error only in the log, since it names the DNS server.
-// Only the public addresses a hostname resolves to are kept, so the answer
-// does not show the deployment's private addresses; a hostname with none is
-// reported in the errors. More than maxLookupTargets entries, or addresses once
-// resolved, return errTooManyTargets. It stops once ctx's deadline has passed;
-// the caller then answers 504.
-func (s *Server) resolveLookupTargets(
-	ctx context.Context, target string,
-) ([]string, []string, error) {
-	targets, err := splitLookupTargets(target)
-	if err != nil {
-		return nil, nil, err
-	}
-
-	var ipsToLookup, resolveErrors []string
-
-	for _, t := range targets {
 		// Check if this target is an IP address
 		if net.ParseIP(t) != nil {
 			ipsToLookup = append(ipsToLookup, t)
@@ -955,57 +785,27 @@ func (s *Server) resolveLookupTargets(
 		}
 
 		// It's a hostname - resolve it
-		resolved, err := s.resolver.LookupHost(ctx, t)
+		resolved, err := net.DefaultResolver.LookupHost(ctx, t)
 		if deadlinePassed(ctx) {
-			return nil, nil, nil
+			return nil, nil
 		}
 
 		if err != nil {
-			s.logger.Warn("Host name lookup failed", "host", t, "error", err)
+			resolveErrors = append(resolveErrors, t+": "+err.Error())
-			resolveErrors = append(resolveErrors, t+": "+hostNotResolvedMessage)
 
 			continue
 		}
 
-		public := publicAddresses(resolved)
+		ipsToLookup = append(ipsToLookup, resolved...)
-		if len(public) == 0 {
-			resolveErrors = append(resolveErrors, t+": "+hostNotPublicMessage)
-
-			continue
 	}
 
-		ipsToLookup = append(ipsToLookup, public...)
+	return ipsToLookup, resolveErrors
-	}
-
-	if len(ipsToLookup) > maxLookupTargets {
-		return nil, nil, errTooManyTargets
-	}
-
-	return ipsToLookup, resolveErrors, nil
-}
-
-// publicAddresses returns the addresses in addrs that are global unicast and
-// not private; Go counts the private ranges as global unicast.
-func publicAddresses(addrs []string) []string {
-	var public []string
-
-	for _, a := range addrs {
-		ip, err := netip.ParseAddr(a)
-		if err == nil && ip.IsGlobalUnicast() && !ip.IsPrivate() {
-			public = append(public, a)
-		}
-	}
-
-	return public
 }
 
 // lookupPTRNames returns the PTR records of ip without their trailing dots, or
-// nil when the lookup fails or runs past ptrLookupTimeout.
+// nil when the lookup fails.
-func (s *Server) lookupPTRNames(ctx context.Context, ip string) []string {
+func lookupPTRNames(ctx context.Context, ip string) []string {
-	ctx, cancel := context.WithTimeout(ctx, ptrLookupTimeout)
+	ptrs, err := net.DefaultResolver.LookupAddr(ctx, ip)
-	defer cancel()
-
-	ptrs, err := s.resolver.LookupAddr(ctx, ip)
 	if err != nil {
 		return nil
 	}
@@ -1019,7 +819,7 @@ func (s *Server) lookupPTRNames(ctx context.Context, ip string) []string {
 }
 
 // extractClientIP extracts the client IP from the request.
-// Works with RealIPMiddleware, which sets RemoteAddr.
+// Works with chi's RealIP middleware which sets RemoteAddr.
 func extractClientIP(r *http.Request) string {
 	// RemoteAddr is in the form "IP:port" or just "IP" for unix sockets
 	host, _, err := net.SplitHostPort(r.RemoteAddr)
@@ -1054,9 +854,7 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
 			if errors.Is(err, database.ErrNoRoute) {
 				s.writeJSONError(w, http.StatusNotFound, err.Error())
 			} else {
-				s.logger.Error("Failed to get AS details", "asn", asn, "error", err)
+				s.writeJSONError(w, http.StatusInternalServerError, err.Error())
-				s.writeJSONError(w, http.StatusInternalServerError,
-					internalErrorMessage)
 			}
 
 			return
@@ -1079,12 +877,11 @@ func (s *Server) handleASDetailJSON() http.HandlerFunc {
 	}
 }
 
-// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones. Neither
+// splitRoutesByIPVersion groups routes into the IPv4 and the IPv6 ones.
-// group is nil, so that a JSON answer holds [] for an empty one and not null.
 func splitRoutesByIPVersion(
 	routes []database.LiveRoute,
 ) ([]database.LiveRoute, []database.LiveRoute) {
-	ipv4Routes, ipv6Routes := []database.LiveRoute{}, []database.LiveRoute{}
+	var ipv4Routes, ipv6Routes []database.LiveRoute
 
 	for _, route := range routes {
 		if route.IPVersion == ipVersionV4 {
@@ -1097,15 +894,24 @@ func splitRoutesByIPVersion(
 	return ipv4Routes, ipv6Routes
 }
 
-// sortRoutesByMaskLength sorts routes by mask length, shortest first, and
+// sortRoutesByNetwork sorts routes by network address, then by mask length.
-// routes with the same mask length by network address.
+func sortRoutesByNetwork(routes []database.LiveRoute) {
-func sortRoutesByMaskLength(routes []database.LiveRoute) {
+	sort.Slice(routes, func(i, j int) bool {
-	slices.SortFunc(routes, func(a, b database.LiveRoute) int {
+		// Parse the prefixes to compare network addresses
-		prefixA, _ := netip.ParsePrefix(a.Prefix)
+		ipI, netI, _ := net.ParseCIDR(routes[i].Prefix)
-		prefixB, _ := netip.ParsePrefix(b.Prefix)
+		ipJ, netJ, _ := net.ParseCIDR(routes[j].Prefix)
 
-		return cmp.Or(cmp.Compare(prefixA.Bits(), prefixB.Bits()),
+		// Compare by network address first
-			prefixA.Addr().Compare(prefixB.Addr()))
+		cmp := bytes.Compare(ipI.To16(), ipJ.To16())
+		if cmp != 0 {
+			return cmp < 0
+		}
+
+		// If network addresses are equal, compare by mask length
+		onesI, _ := netI.Mask.Size()
+		onesJ, _ := netJ.Mask.Size()
+
+		return onesI < onesJ
 	})
 }
 
@@ -1127,7 +933,7 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
 		// tables store, so that any valid way of writing it finds the route.
 		parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
 		if err != nil {
-			s.writeJSONError(w, http.StatusBadRequest, invalidPrefixMessage)
+			s.writeJSONError(w, http.StatusBadRequest, err.Error())
 
 			return
 		}
@@ -1145,10 +951,7 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
 			if errors.Is(err, database.ErrNoRoute) {
 				s.writeJSONError(w, http.StatusNotFound, err.Error())
 			} else {
-				s.logger.Error("Failed to get prefix details",
+				s.writeJSONError(w, http.StatusInternalServerError, err.Error())
-					"prefix", prefix, "error", err)
-				s.writeJSONError(w, http.StatusInternalServerError,
-					internalErrorMessage)
 			}
 
 			return
@@ -1175,56 +978,6 @@ func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
 	}
 }
 
-// ASSearchResponse is the answer to an AS name search: the text searched for,
-// the number of ASes that match it, and the first maxASSearchResults of them in
-// AS number order.
-type ASSearchResponse struct {
-	Query   string        `json:"query"`
-	Total   int           `json:"total"`
-	Results []asinfo.Info `json:"results"`
-}
-
-// handleASSearch returns a handler that searches the AS list built into
-// asinfo for the text in the q parameter, without its leading and trailing
-// spaces. It answers 400 when no text is left.
-func (s *Server) handleASSearch() http.HandlerFunc {
-	return func(w http.ResponseWriter, r *http.Request) {
-		query := strings.TrimSpace(r.URL.Query().Get(asSearchQueryParam))
-		if query == "" {
-			s.writeJSONError(w, http.StatusBadRequest, searchQueryMissingMessage)
-
-			return
-		}
-
-		// AS 0 means no known origin and has no AS page, so the answer leaves
-		// it out. The matches come in AS number order, so AS 0 can only be the
-		// first. The search asks for one match more than the answer holds, to
-		// take its place.
-		results, total := asinfo.Search(query, maxASSearchResults+1)
-		if len(results) > 0 && results[0].ASN == ristypes.NoOrigin {
-			results = results[1:]
-			total--
-		}
-
-		results = results[:min(len(results), maxASSearchResults)]
-
-		// results is an empty list rather than nil when nothing matches, so
-		// that the answer holds [] and not null.
-		if results == nil {
-			results = []asinfo.Info{}
-		}
-
-		err := writeJSONSuccess(w, ASSearchResponse{
-			Query:   query,
-			Total:   total,
-			Results: results,
-		})
-		if err != nil {
-			s.logger.Error("Failed to encode AS search results", "error", err)
-		}
-	}
-}
-
 // handleASDetail returns a handler that serves the AS detail HTML page
 func (s *Server) handleASDetail() http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
@@ -1249,7 +1002,7 @@ func (s *Server) handleASDetail() http.HandlerFunc {
 				http.Error(w, "AS not found", http.StatusNotFound)
 			} else {
 				s.logger.Error("Failed to get AS details", "error", err)
-				http.Error(w, internalErrorMessage, http.StatusInternalServerError)
+				http.Error(w, "Internal server error", http.StatusInternalServerError)
 			}
 
 			return
@@ -1272,8 +1025,9 @@ func (s *Server) handleASDetail() http.HandlerFunc {
 		// Group prefixes by IP version
 		ipv4Prefixes, ipv6Prefixes := splitRoutesByIPVersion(prefixes)
 
-		sortRoutesByMaskLength(ipv4Prefixes)
+		// Sort prefixes by network address
-		sortRoutesByMaskLength(ipv6Prefixes)
+		sortRoutesByNetwork(ipv4Prefixes)
+		sortRoutesByNetwork(ipv6Prefixes)
 
 		// Prepare template data
 		data := struct {
@@ -1285,7 +1039,6 @@ func (s *Server) handleASDetail() http.HandlerFunc {
 			IPv6Count    int
 			Peers        []database.ASPeer
 			PeerCount    int
-			CSPNonce     string
 		}{
 			ASN:          asInfo,
 			IPv4Prefixes: ipv4Prefixes,
@@ -1295,7 +1048,6 @@ func (s *Server) handleASDetail() http.HandlerFunc {
 			IPv6Count:    len(ipv6Prefixes),
 			Peers:        peers,
 			PeerCount:    len(peers),
-			CSPNonce:     cspNonce(r.Context()),
 		}
 
 		// Check if context is still valid before writing response
@@ -1306,7 +1058,9 @@ func (s *Server) handleASDetail() http.HandlerFunc {
 
 		w.Header().Set("Content-Type", "text/html; charset=utf-8")
 
-		err = templates.ASDetailTemplate().Execute(w, data)
+		tmpl := templates.ASDetailTemplate()
+
+		err = tmpl.Execute(w, data)
 		if err != nil {
 			s.logger.Error("Failed to render AS detail template", "error", err)
 			http.Error(w, "Internal Server Error", http.StatusInternalServerError)
@@ -1331,7 +1085,7 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
 		// tables store, so that any valid way of writing it finds the route.
 		parsed, err := netip.ParsePrefix(prefixParam + "/" + lenParam)
 		if err != nil {
-			http.Error(w, invalidPrefixMessage, http.StatusBadRequest)
+			http.Error(w, "Invalid prefix", http.StatusBadRequest)
 
 			return
 		}
@@ -1350,7 +1104,7 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
 				http.Error(w, "Prefix not found", http.StatusNotFound)
 			} else {
 				s.logger.Error("Failed to get prefix details", "error", err)
-				http.Error(w, internalErrorMessage, http.StatusInternalServerError)
+				http.Error(w, "Internal server error", http.StatusInternalServerError)
 			}
 
 			return
@@ -1377,7 +1131,6 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
 			Origins     []*ASNInfo
 			PeerCount   int
 			OriginCount int
-			CSPNonce    string
 		}{
 			Prefix:      prefix,
 			MaskLength:  maskLength,
@@ -1386,7 +1139,6 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
 			Origins:     origins,
 			PeerCount:   len(routes),
 			OriginCount: len(origins),
-			CSPNonce:    cspNonce(r.Context()),
 		}
 
 		// Check if context is still valid before writing response
@@ -1399,7 +1151,9 @@ func (s *Server) handlePrefixDetail() http.HandlerFunc {
 
 		w.Header().Set("Content-Type", "text/html; charset=utf-8")
 
-		err = templates.PrefixDetailTemplate().Execute(w, data)
+		tmpl := templates.PrefixDetailTemplate()
+
+		err = tmpl.Execute(w, data)
 		if err != nil {
 			s.logger.Error("Failed to render prefix detail template", "error", err)
 			http.Error(w, "Internal Server Error", http.StatusInternalServerError)
@@ -1549,7 +1303,7 @@ func (s *Server) prefixLengthHandler(ipVersion, maxMaskLength int) http.HandlerF
 
 		if err != nil {
 			s.logger.Error("Failed to get prefixes by length", "error", err)
-			http.Error(w, internalErrorMessage, http.StatusInternalServerError)
+			http.Error(w, "Internal server error", http.StatusInternalServerError)
 
 			return
 		}
@@ -1570,7 +1324,6 @@ func (s *Server) prefixLengthHandler(ipVersion, maxMaskLength int) http.HandlerF
 			"IPVersion":  ipVersion,
 			"Prefixes":   prefixesWithASDescriptions(prefixes),
 			"Count":      len(prefixes),
-			"CSPNonce":   cspNonce(r.Context()),
 		}
 
 		// Check if context is still valid before writing response

internal/server/middleware.go

@@ -3,19 +3,13 @@ package server
 import (
 	"bytes"
 	"context"
-	"crypto/rand"
 	"encoding/json"
 	"errors"
 	"fmt"
 	"log/slog"
 	"net/http"
-	"net/netip"
-	"slices"
-	"strings"
 	"sync"
 	"time"
-
-	"github.com/go-chi/chi/v5"
 )
 
 // responseWriter wraps http.ResponseWriter to capture the response
@@ -122,10 +116,6 @@ func JSONResponseMiddleware(next http.Handler) http.Handler {
 // requestTimeoutMessage is the error message sent with 504 Gateway Timeout.
 const requestTimeoutMessage = "Request timeout"
 
-// internalErrorMessage is the error message sent with 500 Internal Server
-// Error, whatever went wrong; the handler logs the detail.
-const internalErrorMessage = "Internal server error"
-
 // deadlinePassed reports whether ctx ended because its deadline passed.
 //
 // A handler calls it right after each database read or host name lookup,
@@ -142,9 +132,8 @@ func deadlinePassed(ctx context.Context) bool {
 //
 // If a response is not valid JSON or is empty when JSON is expected, the middleware
 // returns a properly formatted JSON error response. If the request's time limit
-// has passed, it answers 504 with the error message "Request timeout". A 4xx
+// has passed, it answers 504 with the error message "Request timeout". For other
-// keeps its status, with the status's standard text as the message. For other
+// errors, it returns "Internal server error" with status 500 if the original
-// errors, it returns "Internal server error", with status 500 if the original
 // status was 200.
 //
 // Non-JSON responses are passed through unchanged.
@@ -203,14 +192,9 @@ func JSONValidationMiddleware(next http.Handler) http.Handler {
 
 		w.WriteHeader(statusCode)
 
-		errorMsg := internalErrorMessage
+		errorMsg := "Internal server error"
-
+		if statusCode == http.StatusGatewayTimeout {
-		switch {
-		case statusCode == http.StatusGatewayTimeout:
 			errorMsg = requestTimeoutMessage
-		case statusCode >= http.StatusBadRequest &&
-			statusCode < http.StatusInternalServerError:
-			errorMsg = http.StatusText(statusCode)
 		}
 
 		response := map[string]any{
@@ -297,118 +281,3 @@ func RequestLoggerMiddleware(logger *slog.Logger) func(http.Handler) http.Handle
 		})
 	}
 }
-
-// DecodedPathMiddleware makes the router match routes against the decoded
-// path, so that the handlers read their URL parameters decoded. Without it,
-// chi matches against the escaped path whenever the request has one and hands
-// the handlers their parameters still escaped: /ip/2001%3Adb8%3A%3A1 would
-// look up "2001%3Adb8%3A%3A1".
-func DecodedPathMiddleware(next http.Handler) http.Handler {
-	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		chi.RouteContext(r.Context()).RoutePath = r.URL.Path
-
-		next.ServeHTTP(w, r)
-	})
-}
-
-// The values of the headers SecurityHeadersMiddleware sets.
-const (
-	// strictTransportSecurity tells browsers to reach this host and its
-	// subdomains only over HTTPS for a year. The server listens on plain
-	// HTTP behind a TLS-terminating proxy, which passes the header on.
-	strictTransportSecurity = "max-age=31536000; includeSubDomains"
-
-	// contentSecurityPolicy lets a page load content only from this server,
-	// and run or apply an inline script or style block only when the block
-	// carries the nonce made for that answer, which replaces %[1]s. No other
-	// site may frame the pages.
-	contentSecurityPolicy = "default-src 'self'; " +
-		"script-src 'nonce-%[1]s'; style-src 'nonce-%[1]s'; " +
-		"frame-ancestors 'none'; base-uri 'none'; form-action 'self'"
-
-	frameOptions       = "DENY"
-	contentTypeOptions = "nosniff"
-	referrerPolicy     = "strict-origin-when-cross-origin"
-
-	// permissionsPolicy turns off browser features the pages do not use.
-	permissionsPolicy = "accelerometer=(), camera=(), geolocation=(), " +
-		"gyroscope=(), magnetometer=(), microphone=(), payment=(), usb=()"
-)
-
-// cspNonceKey is the request context key of the nonce SecurityHeadersMiddleware
-// made for the answer.
-type cspNonceKey struct{}
-
-// SecurityHeadersMiddleware sets the security headers REPO_POLICIES.md requires
-// on every answer. Each answer's Content-Security-Policy names a nonce of its
-// own; a page reads it with cspNonce and writes it on its script and style
-// blocks.
-func SecurityHeadersMiddleware(next http.Handler) http.Handler {
-	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		nonce := rand.Text()
-
-		header := w.Header()
-		header.Set("Strict-Transport-Security", strictTransportSecurity)
-		header.Set("Content-Security-Policy",
-			fmt.Sprintf(contentSecurityPolicy, nonce))
-		header.Set("X-Frame-Options", frameOptions)
-		header.Set("X-Content-Type-Options", contentTypeOptions)
-		header.Set("Referrer-Policy", referrerPolicy)
-		header.Set("Permissions-Policy", permissionsPolicy)
-
-		ctx := context.WithValue(r.Context(), cspNonceKey{}, nonce)
-		next.ServeHTTP(w, r.WithContext(ctx))
-	})
-}
-
-// cspNonce returns the nonce SecurityHeadersMiddleware made for the answer to
-// the request with ctx.
-func cspNonce(ctx context.Context) string {
-	nonce, _ := ctx.Value(cspNonceKey{}).(string)
-
-	return nonce
-}
-
-// RealIPMiddleware gives a request from one of trustedProxies the address of
-// the client the proxies forwarded it for as its RemoteAddr: the last
-// X-Forwarded-For entry that is not a trusted proxy. Each proxy appends the
-// address it took the request from, so the entries left of that one are
-// whatever the client sent. A request from any other peer keeps the peer's
-// address, whatever headers it carries.
-func RealIPMiddleware(trustedProxies []netip.Prefix) func(http.Handler) http.Handler {
-	isTrusted := func(addr netip.Addr) bool {
-		return slices.ContainsFunc(trustedProxies, func(proxy netip.Prefix) bool {
-			return proxy.Contains(addr.Unmap())
-		})
-	}
-
-	return func(next http.Handler) http.Handler {
-		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			peer, err := netip.ParseAddrPort(r.RemoteAddr)
-			if err != nil || !isTrusted(peer.Addr()) {
-				next.ServeHTTP(w, r)
-
-				return
-			}
-
-			// A header sent on several lines is one list.
-			entries := strings.Split(
-				strings.Join(r.Header.Values("X-Forwarded-For"), ","), ",")
-
-			for _, entry := range slices.Backward(entries) {
-				addr, err := netip.ParseAddr(strings.TrimSpace(entry))
-				if err != nil {
-					break
-				}
-
-				r.RemoteAddr = addr.String()
-
-				if !isTrusted(addr) {
-					break
-				}
-			}
-
-			next.ServeHTTP(w, r)
-		})
-	}
-}

internal/server/middleware_internal_test.go

@@ -1,234 +0,0 @@
-package server
-
-import (
-	"encoding/json"
-	"net/http"
-	"net/http/httptest"
-	"net/netip"
-	"strings"
-	"testing"
-	"time"
-)
-
-// The addresses the forwarding tests use: the client a proxy names in
-// X-Forwarded-For, the range of trusted proxies, a peer inside it, and a peer
-// outside it, which lies in testPrefixV4 so that an IP lookup of it finds a
-// route.
-const (
-	forwardedClient   = "203.0.113.9"
-	trustedProxyRange = "10.0.0.0/8"
-	trustedPeer       = "10.0.0.2:1234"
-	untrustedPeerAddr = "192.0.2.1"
-	untrustedPeer     = untrustedPeerAddr + ":1234"
-)
-
-// TestResponsesCarrySecurityHeaders checks that a page, a JSON route and a
-// path no route matches each answer with the security headers REPO_POLICIES.md
-// requires, and that the Content-Security-Policy allows no inline content
-// without a nonce (https://git.eeqj.de/sneak/routewatch/issues/108).
-func TestResponsesCarrySecurityHeaders(t *testing.T) {
-	t.Parallel()
-
-	s := newServerWithTestRoutes(t)
-
-	headers := []string{
-		"Strict-Transport-Security",
-		"Content-Security-Policy",
-		"X-Frame-Options",
-		"X-Content-Type-Options",
-		"Referrer-Policy",
-		"Permissions-Policy",
-	}
-
-	for _, path := range []string{"/", statsPath, "/no-such-page"} {
-		rec := httptest.NewRecorder()
-		s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
-			t.Context(), http.MethodGet, path, nil))
-
-		for _, name := range headers {
-			if rec.Header().Get(name) == "" {
-				t.Errorf("%s: no %s header", path, name)
-			}
-		}
-
-		policy := rec.Header().Get("Content-Security-Policy")
-		if strings.Contains(policy, "'unsafe-") {
-			t.Errorf("%s: Content-Security-Policy %q allows unsafe content",
-				path, policy)
-		}
-	}
-}
-
-// TestJSONValidationKeeps4xxWithoutBody wraps a handler that answers 400
-// without a body in JSONValidationMiddleware and checks that the JSON error
-// body keeps the 400 and gives the status's own text. Before the fix it said
-// "Internal server error"
-// (https://git.eeqj.de/sneak/routewatch/issues/111).
-func TestJSONValidationKeeps4xxWithoutBody(t *testing.T) {
-	t.Parallel()
-
-	handler := JSONValidationMiddleware(http.HandlerFunc(
-		func(w http.ResponseWriter, _ *http.Request) {
-			w.WriteHeader(http.StatusBadRequest)
-		}))
-
-	rec := httptest.NewRecorder()
-	handler.ServeHTTP(rec, httptest.NewRequestWithContext(
-		t.Context(), http.MethodGet, "/", nil))
-
-	const name = "400 without a body"
-
-	if rec.Code != http.StatusBadRequest {
-		t.Errorf("%s: status %d, want %d; body %s",
-			name, rec.Code, http.StatusBadRequest, rec.Body)
-	}
-
-	checkJSONErrorCode(t, name, rec, http.StatusBadRequest)
-	checkJSONErrorMessage(t, name, rec, http.StatusText(http.StatusBadRequest))
-}
-
-// TestPagesCarryTheCSPNonce checks that every script and style block of each
-// page carries the nonce its Content-Security-Policy header names, so that the
-// browser runs and applies them, that each answer has a nonce of its own, and
-// that no page uses a style attribute, which the policy does not allow.
-func TestPagesCarryTheCSPNonce(t *testing.T) {
-	t.Parallel()
-
-	s := newServerWithTestRoutes(t)
-
-	_, err := s.db.GetOrCreateASN(64500, time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC))
-	if err != nil {
-		t.Fatalf("GetOrCreateASN: %v", err)
-	}
-
-	paths := []string{
-		"/", "/status", asPagePath, "/prefix/" + testPrefixV4,
-		prefixLengthPagePath, "/prefixlength6/32",
-	}
-	seen := make(map[string]bool)
-
-	for _, path := range paths {
-		rec := httptest.NewRecorder()
-		s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
-			t.Context(), http.MethodGet, path, nil))
-
-		if rec.Code != http.StatusOK {
-			t.Errorf("%s: status %d, want %d; body %s",
-				path, rec.Code, http.StatusOK, rec.Body)
-
-			continue
-		}
-
-		policy := rec.Header().Get("Content-Security-Policy")
-
-		_, nonce, found := strings.Cut(policy, "'nonce-")
-		if !found {
-			t.Errorf("%s: Content-Security-Policy %q names no nonce", path, policy)
-
-			continue
-		}
-
-		nonce, _, _ = strings.Cut(nonce, "'")
-		if seen[nonce] {
-			t.Errorf("%s: nonce %q was already used for another answer", path, nonce)
-		}
-
-		seen[nonce] = true
-		page := rec.Body.String()
-
-		for _, tag := range []string{"<script", "<style"} {
-			withNonce := tag + ` nonce="` + nonce + `"`
-			if strings.Count(page, tag) != strings.Count(page, withNonce) {
-				t.Errorf("%s: a %s tag does not carry the nonce %q", path, tag, nonce)
-			}
-		}
-
-		if strings.Contains(page, "style=") {
-			t.Errorf("%s: the page uses a style attribute", path)
-		}
-	}
-}
-
-// TestForwardedForFromUntrustedPeerIsIgnored asks /ip/ for the caller's own
-// address from a peer that is not a trusted proxy, with forwarding headers
-// naming another address, and checks that the lookup is for the TCP peer.
-// Before the fix any client could choose the address it was looked up and
-// logged as (https://git.eeqj.de/sneak/routewatch/issues/108).
-func TestForwardedForFromUntrustedPeerIsIgnored(t *testing.T) {
-	t.Parallel()
-
-	s := newServerWithTestRoutes(t)
-
-	req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/ip/", nil)
-	req.RemoteAddr = untrustedPeer
-	req.Header.Set("X-Forwarded-For", forwardedClient)
-	req.Header.Set("X-Real-IP", forwardedClient)
-	req.Header.Set("True-Client-IP", forwardedClient)
-
-	rec := httptest.NewRecorder()
-	s.router.ServeHTTP(rec, req)
-
-	var body struct {
-		Data IPLookupResponse `json:"data"`
-	}
-
-	err := json.Unmarshal(rec.Body.Bytes(), &body)
-	if err != nil {
-		t.Fatalf("decoding the answer %q: %v", rec.Body, err)
-	}
-
-	if body.Data.Query != untrustedPeerAddr {
-		t.Errorf("looked up %q, want the TCP peer %s; answer %s",
-			body.Data.Query, untrustedPeerAddr, rec.Body)
-	}
-}
-
-// TestRealIPMiddlewareTrustsOnlyConfiguredProxies checks which address
-// RealIPMiddleware gives a request: the TCP peer, unless the peer is a trusted
-// proxy; then the last X-Forwarded-For entry that is not a trusted proxy,
-// since the entries left of it are whatever the client sent.
-func TestRealIPMiddlewareTrustsOnlyConfiguredProxies(t *testing.T) {
-	t.Parallel()
-
-	trusted := []netip.Prefix{netip.MustParsePrefix(trustedProxyRange)}
-
-	tests := []struct {
-		name         string
-		peer         string
-		forwardedFor string
-		want         string
-	}{
-		{"untrusted peer", untrustedPeer, forwardedClient, untrustedPeer},
-		{"trusted peer", trustedPeer, forwardedClient, forwardedClient},
-		{
-			"entries the client wrote", trustedPeer,
-			"198.51.100.7, " + forwardedClient, forwardedClient,
-		},
-		{
-			"two trusted proxies", trustedPeer,
-			forwardedClient + ", 10.0.0.3", forwardedClient,
-		},
-		{"no header", trustedPeer, "", trustedPeer},
-		{"entry not an address", trustedPeer, "client", trustedPeer},
-	}
-
-	for _, tt := range tests {
-		var got string
-
-		handler := RealIPMiddleware(trusted)(http.HandlerFunc(
-			func(_ http.ResponseWriter, r *http.Request) { got = r.RemoteAddr }))
-
-		req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
-		req.RemoteAddr = tt.peer
-
-		if tt.forwardedFor != "" {
-			req.Header.Set("X-Forwarded-For", tt.forwardedFor)
-		}
-
-		handler.ServeHTTP(httptest.NewRecorder(), req)
-
-		if got != tt.want {
-			t.Errorf("%s: RemoteAddr %q, want %q", tt.name, got, tt.want)
-		}
-	}
-}

internal/server/routes.go

@@ -1,36 +1,19 @@
 package server
 
 import (
-	"net/http"
 	"time"
 
 	"github.com/go-chi/chi/v5"
 	"github.com/go-chi/chi/v5/middleware"
 )
 
-// The error messages the JSON routes under /api/v1 and /ip answer with when no
-// route matches the path, and when none takes the method.
-const (
-	notFoundMessage         = "Not found"
-	methodNotAllowedMessage = "Method not allowed"
-
-	// jsonRoutesAllow is the Allow header of their 405 answer: every one of
-	// them takes GET and HEAD, and no other method.
-	jsonRoutesAllow = "GET, HEAD"
-)
-
 // setupRoutes configures the HTTP routes
 func (s *Server) setupRoutes() {
 	r := chi.NewRouter()
 
 	// Middleware
-	r.Use(DecodedPathMiddleware)
-	// GetHead serves HEAD with the GET handler. It looks the route up by the
-	// path DecodedPathMiddleware sets, so it comes after it.
-	r.Use(middleware.GetHead)
-	r.Use(SecurityHeadersMiddleware)
 	r.Use(middleware.RequestID)
-	r.Use(RealIPMiddleware(s.trustedProxies))
+	r.Use(middleware.RealIP)
 	r.Use(RequestLoggerMiddleware(s.logger.Logger)) // Structured request logging
 	r.Use(middleware.Recoverer)
 	r.Use(JSONResponseMiddleware)
@@ -55,39 +38,20 @@ func (s *Server) setupRoutes() {
 	r.Get("/prefixlength/{length}", s.handlePrefixLength())
 	r.Get("/prefixlength6/{length}", s.handlePrefixLength6())
 
-	// The JSON routes answer a path or method they do not take with the JSON
-	// error body, as they answer everything else.
-	jsonNotFound := func(w http.ResponseWriter, _ *http.Request) {
-		s.writeJSONError(w, http.StatusNotFound, notFoundMessage)
-	}
-	jsonMethodNotAllowed := func(w http.ResponseWriter, _ *http.Request) {
-		// chi's own 405 answer sets Allow; one set in its place must too.
-		w.Header().Set("Allow", jsonRoutesAllow)
-		s.writeJSONError(w, http.StatusMethodNotAllowed, methodNotAllowedMessage)
-	}
-
 	// IP info JSON endpoints (replaces old /ip redirect)
 	r.Route("/ip", func(r chi.Router) {
 		r.Use(JSONValidationMiddleware)
-		r.NotFound(jsonNotFound)
-		r.MethodNotAllowed(jsonMethodNotAllowed)
 		r.Get("/", s.handleIPInfo())       // Client IP
-		// GetHead does not reach this route: mounting the group registers
-		// /ip and /ip/ for every method, so HEAD arrives here as HEAD.
-		r.Head("/", s.handleIPInfo())
 		r.Get("/{addr}", s.handleIPInfo()) // Specified IP
 	})
 
 	// API routes
 	r.Route("/api/v1", func(r chi.Router) {
 		r.Use(JSONValidationMiddleware)
-		r.NotFound(jsonNotFound)
-		r.MethodNotAllowed(jsonMethodNotAllowed)
 		r.Get("/stats", s.handleStats())
 		r.Get("/ip/{ip}", s.handleIPLookup())
 		r.Get("/as/{asn}", s.handleASDetailJSON())
 		r.Get("/prefix/{prefix}/{len}", s.handlePrefixDetailJSON())
-		r.Get("/search/as", s.handleASSearch())
 	})
 
 	s.router = r

internal/server/routes_internal_test.go

@@ -1,100 +0,0 @@
-package server
-
-import (
-	"encoding/json"
-	"net/http"
-	"net/http/httptest"
-	"testing"
-)
-
-// TestRoutesAnswerHEAD sends HEAD for a page and JSON routes through the
-// server's router and checks that each answers as GET does. /ip and /ip/, the
-// lookup of the caller's own address, are in the list because chi's GetHead
-// does not reach them. Before the fix every route answered HEAD with 405
-// (https://git.eeqj.de/sneak/routewatch/issues/111).
-func TestRoutesAnswerHEAD(t *testing.T) {
-	t.Parallel()
-
-	s := newServerWithTestRoutes(t)
-
-	for _, path := range []string{"/", statsPath, "/ip", "/ip/"} {
-		rec := httptest.NewRecorder()
-		s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
-			t.Context(), http.MethodHead, path, nil))
-
-		if rec.Code != http.StatusOK {
-			t.Errorf("HEAD %s: status %d, want %d; body %s",
-				path, rec.Code, http.StatusOK, rec.Body)
-		}
-	}
-}
-
-// TestJSONRoutesAnswerUnmatchedRequestsWithJSONErrors sends a method the JSON
-// routes do not take, and a path none of them matches, to each group of JSON
-// routes through the server's router. Each must answer with the JSON error
-// body, and a 405 must name the methods the routes take in its Allow header.
-// Before the fix a 405 said "Internal server error" and a 404 was chi's plain
-// text answer (https://git.eeqj.de/sneak/routewatch/issues/111).
-func TestJSONRoutesAnswerUnmatchedRequestsWithJSONErrors(t *testing.T) {
-	t.Parallel()
-
-	s := newServerWithTestRoutes(t)
-
-	tests := []struct {
-		method   string
-		path     string
-		wantCode int
-		wantMsg  string
-	}{
-		{http.MethodPost, statsPath, http.StatusMethodNotAllowed, methodNotAllowedMessage},
-		{http.MethodPost, "/ip/", http.StatusMethodNotAllowed, methodNotAllowedMessage},
-		{http.MethodGet, "/api/v1/nothing", http.StatusNotFound, notFoundMessage},
-		{http.MethodGet, "/ip/192.0.2.1/nothing", http.StatusNotFound, notFoundMessage},
-	}
-
-	for _, tt := range tests {
-		name := tt.method + " " + tt.path
-
-		rec := httptest.NewRecorder()
-		s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
-			t.Context(), tt.method, tt.path, nil))
-
-		if rec.Code != tt.wantCode {
-			t.Errorf("%s: status %d, want %d; body %s",
-				name, rec.Code, tt.wantCode, rec.Body)
-		}
-
-		checkJSONErrorCode(t, name, rec, tt.wantCode)
-		checkJSONErrorMessage(t, name, rec, tt.wantMsg)
-
-		allow := rec.Header().Get("Allow")
-		if tt.wantCode == http.StatusMethodNotAllowed && allow != jsonRoutesAllow {
-			t.Errorf("%s: Allow %q, want %q", name, allow, jsonRoutesAllow)
-		}
-	}
-}
-
-// checkJSONErrorMessage checks that rec holds the JSON error body with the
-// given message.
-func checkJSONErrorMessage(
-	t *testing.T, name string, rec *httptest.ResponseRecorder, want string,
-) {
-	t.Helper()
-
-	var body struct {
-		Error struct {
-			Msg string `json:"msg"`
-		} `json:"error"`
-	}
-
-	err := json.Unmarshal(rec.Body.Bytes(), &body)
-	if err != nil {
-		t.Errorf("%s: decoding the answer %q: %v", name, rec.Body, err)
-
-		return
-	}
-
-	if body.Error.Msg != want {
-		t.Errorf("%s: error message %q, want %q", name, body.Error.Msg, want)
-	}
-}

internal/server/server.go

@@ -1,8 +1,4 @@
-// Package server serves RouteWatch's web pages and JSON API:
+// Package server provides HTTP endpoints for status monitoring and statistics
-//   - the home page, the status figures and the health check
-//   - the AS, prefix and prefix length pages, and the JSON answers behind them
-//   - the IP lookups
-//   - the AS name search
 package server
 
 import (
@@ -11,11 +7,9 @@ import (
 	"fmt"
 	"net"
 	"net/http"
-	"net/netip"
 	"strconv"
 	"time"
 
-	"git.eeqj.de/sneak/routewatch/internal/blocklist"
 	"git.eeqj.de/sneak/routewatch/internal/config"
 	"git.eeqj.de/sneak/routewatch/internal/database"
 	"git.eeqj.de/sneak/routewatch/internal/logger"
@@ -37,20 +31,6 @@ type ASNFetcher interface {
 	GetStats() ASNFetcherStats
 }
 
-// Blocklists tells which public blocklists list an address, from copies it
-// keeps in memory, as *blocklist.Blocklists does.
-type Blocklists interface {
-	Lookup(addr netip.Addr) []blocklist.Listing
-	ShortestInterval() time.Duration
-}
-
-// dnsResolver looks up the addresses of host names and the names of addresses
-// in DNS, as *net.Resolver does.
-type dnsResolver interface {
-	LookupHost(ctx context.Context, host string) ([]string, error)
-	LookupAddr(ctx context.Context, addr string) ([]string, error)
-}
-
 // Server provides HTTP endpoints for status monitoring
 type Server struct {
 	router     *chi.Mux
@@ -60,15 +40,6 @@ type Server struct {
 	port       int
 	srv        *http.Server
 	asnFetcher ASNFetcher
-	blocklists Blocklists
-
-	// trustedProxies are the reverse proxies whose X-Forwarded-For header
-	// RealIPMiddleware believes
-	trustedProxies []netip.Prefix
-
-	// resolver makes the IP lookup's DNS lookups: net.DefaultResolver, or a
-	// fake in tests
-	resolver dnsResolver
 }
 
 // New creates a new HTTP server
@@ -83,8 +54,6 @@ func New(
 		streamer: streamer,
 		logger:   logger,
 		port:     cfg.Port,
-		trustedProxies: cfg.TrustedProxies,
-		resolver:       net.DefaultResolver,
 	}
 
 	s.setupRoutes()
@@ -134,9 +103,7 @@ func (s *Server) Start(ctx context.Context) error {
 	return nil
 }
 
-// Stop stops the HTTP server. It waits, until ctx ends, for the requests in
+// Stop gracefully stops the HTTP server
-// progress to finish, and then closes the connections of those still running
-// and returns ctx's error.
 func (s *Server) Stop(ctx context.Context) error {
 	if s.srv == nil {
 		return nil
@@ -144,22 +111,10 @@ func (s *Server) Stop(ctx context.Context) error {
 
 	s.logger.Info("Stopping HTTP server")
 
-	err := s.srv.Shutdown(ctx)
+	return s.srv.Shutdown(ctx)
-	if err != nil {
-		return errors.Join(err, s.srv.Close())
 }
 
-	return nil
+// SetASNFetcher sets the ASN WHOIS fetcher for on-demand lookups.
-}
-
-// SetASNFetcher sets the ASN WHOIS fetcher for on-demand lookups. Call it
-// before Start: the handlers read the fetcher without a lock.
 func (s *Server) SetASNFetcher(fetcher ASNFetcher) {
 	s.asnFetcher = fetcher
 }
-
-// SetBlocklists sets the blocklists the IP lookups report. Call it before
-// Start: the handlers read the blocklists without a lock.
-func (s *Server) SetBlocklists(blocklists Blocklists) {
-	s.blocklists = blocklists
-}

internal/server/server_internal_test.go

@@ -1,92 +0,0 @@
-package server
-
-import (
-	"context"
-	"net"
-	"net/http"
-	"testing"
-	"time"
-
-	"git.eeqj.de/sneak/routewatch/internal/logger"
-)
-
-// TestStopClosesTheConnectionOfARequestStillRunning serves a request whose
-// handler does not return until the test ends, and stops the server with a
-// deadline that passes while the request runs. The client must then see its
-// connection closed. Before the fix Stop returned at the deadline and left the
-// connection open until the handler returned.
-func TestStopClosesTheConnectionOfARequestStillRunning(t *testing.T) {
-	t.Parallel()
-
-	const (
-		stopDeadline = 100 * time.Millisecond
-		// waitTimeout is how long the test waits for the request to reach
-		// the handler, and then for the client to see its connection closed.
-		waitTimeout = 2 * time.Second
-		// readHeaderTimeout bounds the test server's read of the request
-		// header, as every http.Server must.
-		readHeaderTimeout = time.Second
-	)
-
-	release := make(chan struct{})
-	defer close(release)
-
-	started := make(chan struct{})
-
-	var listenConfig net.ListenConfig
-
-	listener, err := listenConfig.Listen(t.Context(), "tcp", "127.0.0.1:0")
-	if err != nil {
-		t.Fatalf("failed to open the test listener: %v", err)
-	}
-
-	s := &Server{logger: logger.New(), srv: &http.Server{
-		Handler: http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
-			close(started)
-			<-release
-		}),
-		ReadHeaderTimeout: readHeaderTimeout,
-	}}
-
-	go func() { _ = s.srv.Serve(listener) }()
-
-	answered := make(chan error, 1)
-
-	go func() {
-		req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
-			"http://"+listener.Addr().String()+"/", nil)
-		if err != nil {
-			answered <- err
-
-			return
-		}
-
-		resp, err := http.DefaultClient.Do(req)
-		if err == nil {
-			_ = resp.Body.Close()
-		}
-
-		answered <- err
-	}()
-
-	select {
-	case <-started:
-	case <-time.After(waitTimeout):
-		t.Fatal("the request did not reach the handler")
-	}
-
-	ctx, cancel := context.WithTimeout(t.Context(), stopDeadline)
-	defer cancel()
-
-	_ = s.Stop(ctx)
-
-	select {
-	case err := <-answered:
-		if err == nil {
-			t.Error("the request still running at the deadline was answered")
-		}
-	case <-time.After(waitTimeout):
-		t.Fatal("the connection of the request still running at the " +
-			"deadline was open after Stop returned")
-	}
-}

internal/streamer/streamer.go

@@ -106,12 +106,12 @@ type RawMessageHandler func(line string)
 // handlerMetrics tracks performance metrics for a handler
 type handlerMetrics struct {
 	processedCount     uint64        // Total messages processed
-	droppedCount       atomic.Uint64 // Total messages dropped; not under mu
+	droppedCount       uint64        // Total messages dropped
 	totalTime          time.Duration // Total processing time (for average calculation)
 	minTime            time.Duration // Minimum processing time
 	maxTime            time.Duration // Maximum processing time
 	queueHighWaterMark int           // Maximum queue length seen
-	mu                 sync.Mutex    // Protects the other metrics
+	mu                 sync.Mutex    // Protects the metrics
 }
 
 // handlerInfo wraps a handler with its queue and metrics
@@ -253,10 +253,9 @@ func (s *Streamer) Start(ctx context.Context) error {
 
 // Stop ends the stream and stops the handler workers. It returns once each
 // worker has handed its handler every message left on its queue and then the
-// STATE messages held for it (see queueMessage), or once ctx ends, and then
+// STATE messages held for it (see queueMessage). This method is safe to call
-// logs how many messages were not handed over. This method is safe to call
 // multiple times.
-func (s *Streamer) Stop(ctx context.Context) {
+func (s *Streamer) Stop() {
 	s.mu.Lock()
 	if s.cancel == nil {
 		// Not started, or already stopped: closing the queues again would panic.
@@ -265,56 +264,28 @@ func (s *Streamer) Stop(ctx context.Context) {
 		return
 	}
 
-	// Once the stream's context is cancelled the read loop queues and holds
+	// Once ctx is cancelled the read loop queues and holds nothing more (see
-	// nothing more (see dispatchMessage).
+	// dispatchMessage).
 	s.cancel()
 	s.cancel = nil
 
-	notHandedOver := 0
-
 	for _, info := range s.handlers {
 		// The held messages came after everything on the queue. Each waits
-		// for the worker, which does not take s.mu, to make room, until ctx
+		// for the worker, which does not take s.mu, to make room.
-		// ends.
+		for _, msg := range info.heldState {
-		for len(info.heldState) > 0 && ctx.Err() == nil {
+			info.queue <- msg
-			select {
-			case info.queue <- info.heldState[0]:
-				info.heldState = info.heldState[1:]
-			case <-ctx.Done():
-			}
 		}
 
-		notHandedOver += len(info.heldState)
 		info.heldState = nil
 
 		// The worker returns once it has emptied the closed queue.
 		close(info.queue)
 	}
 
-	handlers := s.handlers
 	s.running = false
 	s.mu.Unlock()
 	s.metrics.SetConnected(false)
-
-	workersDone := make(chan struct{})
-
-	go func() {
 	s.workers.Wait()
-		close(workersDone)
-	}()
-
-	select {
-	case <-workersDone:
-	case <-ctx.Done():
-		for _, info := range handlers {
-			notHandedOver += len(info.queue)
-		}
-	}
-
-	if notHandedOver > 0 {
-		s.logger.Error("Stop deadline passed before the handlers took every "+
-			"message", "messages", notHandedOver)
-	}
 }
 
 // IsRunning reports whether the streamer is currently connected and processing
@@ -371,7 +342,7 @@ func (s *Streamer) GetHandlerStats() []HandlerStats {
 			QueueCapacity:      cap(info.queue),
 			QueueHighWaterMark: info.metrics.queueHighWaterMark,
 			ProcessedCount:     info.metrics.processedCount,
-			DroppedCount:       info.metrics.droppedCount.Load(),
+			DroppedCount:       info.metrics.droppedCount,
 			MinProcessTime:     info.metrics.minTime,
 			MaxProcessTime:     info.metrics.maxTime,
 		}
@@ -488,7 +459,7 @@ func (s *Streamer) logMetrics() {
 				"queue_len", len(info.queue),
 				"queue_cap", cap(info.queue),
 				"processed", info.metrics.processedCount,
-				"dropped", info.metrics.droppedCount.Load(),
+				"dropped", info.metrics.droppedCount,
 				"avg_time", avgTime,
 				"min_time", info.metrics.minTime,
 				"max_time", info.metrics.maxTime,
@@ -858,7 +829,7 @@ func (s *Streamer) dispatchMessage(
 		}
 
 		if !s.queueMessage(info, msg) {
-			info.metrics.droppedCount.Add(1)
+			atomic.AddUint64(&info.metrics.droppedCount, 1)
 			s.totalDropped.Add(1)
 
 			continue

internal/streamer/streamer_internal_test.go

@@ -1,12 +1,9 @@
 package streamer
 
 import (
-	"bytes"
 	"context"
-	"encoding/json"
 	"errors"
 	"io"
-	"log/slog"
 	"net/http"
 	"net/http/httptest"
 	"runtime"
@@ -380,7 +377,7 @@ func TestStopQueuesTheHeldStateMessages(t *testing.T) {
 	stopped := make(chan struct{})
 
 	go func() {
-		s.Stop(t.Context())
+		s.Stop()
 		close(stopped)
 	}()
 
@@ -415,68 +412,6 @@ func TestStopQueuesTheHeldStateMessages(t *testing.T) {
 	}
 }
 
-// TestStopReturnsByItsDeadline stops a streamer whose handler has a full queue
-// and a STATE message held for it, and a worker that stays busy until the test
-// ends, as one does whose handler waits for a database write. Stop must return
-// within its deadline plus a small margin instead of waiting for room on the
-// queue and for the worker, and log as not handed over the held message and
-// every message on the queue.
-func TestStopReturnsByItsDeadline(t *testing.T) {
-	t.Parallel()
-
-	const (
-		deadline = 100 * time.Millisecond
-		margin   = 2 * time.Second
-	)
-
-	var logs bytes.Buffer
-
-	s := newStreamerWithFullStateQueue("")
-	s.logger = &logger.Logger{Logger: slog.New(slog.NewJSONHandler(&logs,
-		&slog.HandlerOptions{Level: slog.LevelError}))}
-	s.handlers[0].heldState = []*ristypes.RISMessage{
-		{Type: ristypes.MessageTypeState},
-	}
-	wantNotHandedOver := 1 + len(s.handlers[0].queue)
-	// Stop cancels the stream Start would have started; there is none here.
-	s.cancel = func() {}
-
-	release := make(chan struct{})
-	defer close(release)
-
-	s.workers.Go(func() { <-release })
-
-	ctx, cancel := context.WithTimeout(t.Context(), deadline)
-	defer cancel()
-
-	stopped := make(chan struct{})
-
-	go func() {
-		s.Stop(ctx)
-		close(stopped)
-	}()
-
-	select {
-	case <-stopped:
-	case <-time.After(deadline + margin):
-		t.Fatalf("Stop did not return within %s of its deadline", margin)
-	}
-
-	var logged struct {
-		Messages int `json:"messages"`
-	}
-
-	err := json.Unmarshal(logs.Bytes(), &logged)
-	if err != nil {
-		t.Fatalf("want the error Stop logs, got %q: %v", logs.String(), err)
-	}
-
-	if logged.Messages != wantNotHandedOver {
-		t.Errorf("Stop logged %d messages not handed over, want %d",
-			logged.Messages, wantNotHandedOver)
-	}
-}
-
 // TestStopBeforeMessageReachesQueues stops the streamer after the read loop
 // has checked for cancellation but before it hands the message to the handler
 // queues. That is the gap a stop from another goroutine can land in, and it
@@ -494,7 +429,7 @@ func TestStopBeforeMessageReachesQueues(t *testing.T) {
 	s := New(logger.New(), metrics.New())
 	s.url = srv.URL
 	s.RegisterHandler(updateHandler{})
-	s.RegisterRawHandler(func(string) { s.Stop(t.Context()) })
+	s.RegisterRawHandler(func(string) { s.Stop() })
 
 	// Start would run the stream in the background, where the test cannot
 	// wait for it. Setting cancel as Start does lets Stop cancel the stream
@@ -508,7 +443,7 @@ func TestStopBeforeMessageReachesQueues(t *testing.T) {
 	}
 
 	// A second Stop must not close the queues again.
-	s.Stop(t.Context())
+	s.Stop()
 }
 
 // TestReconnectWhenStreamEnds has the server end the first response
@@ -621,68 +556,6 @@ func TestStreamReadsLongLine(t *testing.T) {
 	}
 }
 
-// blockedHandler wants UPDATE messages, has room on its queue for one, and
-// does not return from HandleMessage until release is closed.
-type blockedHandler struct {
-	release chan struct{}
-}
-
-func (blockedHandler) WantsMessage(messageType string) bool {
-	return messageType == ristypes.MessageTypeUpdate
-}
-
-func (blockedHandler) QueueCapacity() int { return 1 }
-
-func (h blockedHandler) HandleMessage(*ristypes.RISMessage) { <-h.release }
-
-// TestHandlerStatsReadWhileMessagesAreDropped has the server send a few UPDATE
-// lines to a blockedHandler, so the read loop drops all but the one the
-// handler holds and the one on its queue, while the test reads the handler's
-// statistics. The drop count must reach the number dropped, and the race
-// detector must find the count read and written safely.
-func TestHandlerStatsReadWhileMessagesAreDropped(t *testing.T) {
-	t.Parallel()
-
-	const (
-		lines       = 5
-		wantDropped = lines - 2
-		maxWait     = 5 * time.Second
-	)
-
-	srv := httptest.NewServer(http.HandlerFunc(
-		func(w http.ResponseWriter, r *http.Request) {
-			_, _ = io.WriteString(w, strings.Repeat(updateLine, lines))
-
-			flush(w)
-			<-r.Context().Done()
-		}))
-	defer srv.Close()
-
-	s := New(logger.New(), metrics.New())
-	s.url = srv.URL
-	handler := blockedHandler{release: make(chan struct{})}
-	s.RegisterHandler(handler)
-
-	err := s.Start(t.Context())
-	if err != nil {
-		t.Fatalf("Start returned error: %v", err)
-	}
-	defer s.Stop(t.Context())
-	defer close(handler.release)
-
-	// The test polls instead of waiting for a signal from the read loop: a
-	// signal would order the drops before the reads here, and the race
-	// detector would have nothing to report.
-	deadline := time.Now().Add(maxWait)
-
-	for s.GetHandlerStats()[0].DroppedCount < wantDropped {
-		if time.Now().After(deadline) {
-			t.Fatalf("fewer than %d messages counted as dropped within %s",
-				wantDropped, maxWait)
-		}
-	}
-}
-
 // TestPeerCountCountsEveryPeerHeardFrom has the server send an UPDATE from
 // stateDownPeer and a KEEPALIVE from another peer, with no OPEN from either,
 // and then a message that ends stateDownPeer's session: a NOTIFICATION, or a
@@ -803,7 +676,7 @@ func checkReconnect(
 	if err != nil {
 		t.Fatalf("Start returned error: %v", err)
 	}
-	defer s.Stop(t.Context())
+	defer s.Stop()
 
 	select {
 	case <-handler.handled:

internal/templates/as_detail.html

@@ -4,7 +4,7 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>AS{{.ASN.ASN}} - {{.ASN.Handle}} - RouteWatch</title>
-    <style nonce="{{.CSPNonce}}">
+    <style>
         * {
             box-sizing: border-box;
         }
@@ -126,26 +126,6 @@
             margin: 0;
             color: #2c3e50;
         }
-        /* A click on a prefix list's heading folds or unfolds the list. Not
-           every browser shows its own marker on a flex element, so the page
-           hides it and draws a triangle before the heading. The triangle
-           points down while the list is open. */
-        summary.prefix-header {
-            cursor: pointer;
-            list-style: none;
-        }
-        summary.prefix-header::-webkit-details-marker {
-            display: none;
-        }
-        summary.prefix-header h2::before {
-            content: "\25B8";
-            display: inline-block;
-            margin-right: 10px;
-            transition: transform 0.2s;
-        }
-        details[open] > summary.prefix-header h2::before {
-            transform: rotate(90deg);
-        }
         .prefix-count {
             background: #e74c3c;
             color: white;
@@ -253,20 +233,18 @@
                 <div class="info-label">Peer ASNs</div>
                 <div class="info-value">{{.PeerCount}}</div>
             </div>
-            {{if not .ASN.FirstSeen.IsZero}}
             <div class="info-card">
                 <div class="info-label">First Seen</div>
                 <div class="info-value">{{.ASN.FirstSeen.Format "2006-01-02"}}</div>
             </div>
-            {{end}}
         </div>
 
         {{if .IPv4Prefixes}}
-        <details class="prefix-section" open>
+        <div class="prefix-section">
-            <summary class="prefix-header">
+            <div class="prefix-header">
                 <h2>IPv4 Prefixes</h2>
                 <span class="prefix-count">{{.IPv4Count}}</span>
-            </summary>
+            </div>
             <table class="prefix-table">
                 <thead>
                     <tr>
@@ -287,15 +265,15 @@
                     {{end}}
                 </tbody>
             </table>
-        </details>
+        </div>
         {{end}}
 
         {{if .IPv6Prefixes}}
-        <details class="prefix-section" open>
+        <div class="prefix-section">
-            <summary class="prefix-header">
+            <div class="prefix-header">
                 <h2>IPv6 Prefixes</h2>
                 <span class="prefix-count">{{.IPv6Count}}</span>
-            </summary>
+            </div>
             <table class="prefix-table">
                 <thead>
                     <tr>
@@ -316,7 +294,7 @@
                     {{end}}
                 </tbody>
             </table>
-        </details>
+        </div>
         {{end}}
 
         {{if eq .TotalCount 0}}

internal/templates/index.html

@@ -4,7 +4,7 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>RouteWatch - BGP Route Monitor</title>
-    <style nonce="{{.CSPNonce}}">
+    <style>
         * {
             box-sizing: border-box;
         }
@@ -107,15 +107,9 @@
         .stat-card.connected .stat-value {
             color: #27ae60;
         }
-        .stat-card.disconnected .stat-value,
+        .stat-card.disconnected .stat-value {
-        .stale {
             color: #e74c3c;
         }
-        /* The card has no room beside "Connected", so the spinner and "stale" go under it */
-        #freshness {
-            display: block;
-            font-size: 20px;
-        }
 
         /* Search section */
         .search-section {
@@ -173,45 +167,6 @@
             color: #95a5a6;
             margin-top: 8px;
         }
-        #asname-results p {
-            color: #7f8c8d;
-            margin-top: 12px;
-        }
-        #asname-results .search-message {
-            font-size: 13px;
-        }
-        #asname-results .error {
-            color: #c00;
-        }
-        .asname-list {
-            list-style: none;
-            margin: 12px 0 0 0;
-            padding: 0;
-            max-height: 400px;
-            overflow-y: auto;
-        }
-        .asname-list li {
-            border-bottom: 1px solid #eee;
-        }
-        .asname-list a {
-            display: block;
-            padding: 8px 4px;
-            color: #2c3e50;
-            text-decoration: none;
-            font-size: 14px;
-        }
-        .asname-list a:hover {
-            background: #f5f5f5;
-        }
-        .asname-list .asn {
-            font-weight: 600;
-            color: #3498db;
-        }
-        .asname-list .description {
-            display: block;
-            color: #7f8c8d;
-            font-size: 12px;
-        }
 
         /* IP Lookup result */
         .ip-result {
@@ -239,8 +194,7 @@
             cursor: pointer;
             font-size: 12px;
         }
-        .ip-result pre,
+        .ip-result pre {
-        #ip-api pre {
             background: #2c3e50;
             color: #ecf0f1;
             padding: 16px;
@@ -252,30 +206,18 @@
             max-height: 400px;
             overflow-y: auto;
         }
-        .ip-result .error,
+        .ip-result .error {
-        #ip-api .error {
             background: #fee;
             color: #c00;
             padding: 12px;
             border-radius: 6px;
             font-size: 14px;
         }
-        .ip-result .loading,
+        .ip-result .loading {
-        #ip-api .loading {
             color: #7f8c8d;
             font-style: italic;
         }
 
-        /* IP information API card: the command, then its answer */
-        #ip-api p {
-            margin: 0;
-            color: #7f8c8d;
-            font-size: 14px;
-        }
-        #ip-api pre {
-            margin-top: 12px;
-        }
-
         /* Footer */
         .footer {
             margin-top: 40px;
@@ -316,10 +258,7 @@
     <main class="main-content">
         <div class="stats-grid">
             <div class="stat-card" id="status-card">
-                <div class="stat-value">
+                <div class="stat-value" id="stat-status">-</div>
-                    <span id="stat-status">-</span>
-                    <span id="freshness"></span>
-                </div>
                 <div class="stat-label">Status</div>
             </div>
             <div class="stat-card">
@@ -360,7 +299,7 @@
                     <input type="text" id="asname-input" placeholder="e.g., Google, Cloudflare" autocomplete="off">
                     <button type="submit">Search</button>
                 </form>
-                <p class="search-hint">Search for autonomous systems by organization name or handle</p>
+                <p class="search-hint">Search for autonomous systems by organization name</p>
                 <div id="asname-results"></div>
             </div>
 
@@ -380,13 +319,6 @@
                 </div>
             </div>
         </div>
-
-        <div class="search-card" id="ip-api">
-            <h2>IP Information API</h2>
-            <p>The API answers what RouteWatch knows about an IP address, such as its AS number and country. <code>/ip/</code> with no address answers for the caller's own address.</p>
-            <pre id="ip-api-command" data-client-ip="{{.ClientIP}}"></pre>
-            <pre id="ip-api-answer" class="loading">Loading...</pre>
-        </div>
     </main>
 
     <footer class="footer">
@@ -397,45 +329,7 @@
         <span><a href="{{appGitCommitURL}}">{{appGitRevision}}</a></span>
     </footer>
 
-    <script nonce="{{.CSPNonce}}">
+    <script>
-        // The stats are fetched every REFRESH_PERIOD_MS. Next to the status, a
-        // spinner turns one frame each time an answer counts more messages
-        // than the one before. After STALE_AFTER_REFRESHES refreshes without
-        // such an answer, the spinner gives way to "stale" and the time since.
-        const REFRESH_PERIOD_MS = 5000;
-        const STALE_AFTER_REFRESHES = 3;
-        const MS_PER_SECOND = 1000;
-        const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'];
-        const SPINNER_COLOURS = ['#e74c3c', '#e67e22', '#f1c40f', '#27ae60', '#3498db', '#9b59b6'];
-
-        let spinnerFrame = 0;
-        let lastMessageCount = null;
-        let lastFreshAt = Date.now();
-
-        // Turn the spinner when the message count has risen since the last answer.
-        function noteMessageCount(count) {
-            if (lastMessageCount !== null && count > lastMessageCount) {
-                spinnerFrame = (spinnerFrame + 1) % SPINNER_FRAMES.length;
-                lastFreshAt = Date.now();
-            }
-            lastMessageCount = count;
-            showFreshness();
-        }
-
-        function showFreshness() {
-            const freshness = document.getElementById('freshness');
-            const sinceFreshMs = Date.now() - lastFreshAt;
-            const stale = sinceFreshMs >= STALE_AFTER_REFRESHES * REFRESH_PERIOD_MS;
-            freshness.classList.toggle('stale', stale);
-            if (stale) {
-                freshness.textContent = 'stale for ' + Math.floor(sinceFreshMs / MS_PER_SECOND) + 's';
-                freshness.style.color = '';
-                return;
-            }
-            freshness.textContent = SPINNER_FRAMES[spinnerFrame];
-            freshness.style.color = SPINNER_COLOURS[spinnerFrame % SPINNER_COLOURS.length];
-        }
-
         function formatNumber(num) {
             if (num >= 1000000) {
                 return (num / 1000000).toFixed(1) + 'M';
@@ -457,7 +351,6 @@
                     const statusEl = document.getElementById('stat-status');
                     statusEl.textContent = data.connected ? 'Connected' : 'Disconnected';
                     statusCard.className = 'stat-card ' + (data.connected ? 'connected' : 'disconnected');
-                    noteMessageCount(data.total_messages);
 
                     document.getElementById('stat-routes').textContent = formatNumber(data.live_routes);
                     document.getElementById('stat-asns').textContent = formatNumber(data.asns);
@@ -468,7 +361,7 @@
                     }
 
                     const totalUpdates = data.ipv4_updates_per_sec + data.ipv6_updates_per_sec;
-                    document.getElementById('stat-updates').textContent = Math.round(totalUpdates).toLocaleString();
+                    document.getElementById('stat-updates').textContent = totalUpdates.toFixed(1);
                 })
                 .catch(() => {
                     document.getElementById('stat-status').textContent = 'Error';
@@ -487,82 +380,24 @@
             }
         });
 
-        // AS name search. The handles and descriptions come from outside data,
+        // AS name search
-        // so the results are built with textContent, never innerHTML.
-        const asNameResults = document.getElementById('asname-results');
-
-        // Replace the search results with one line of text.
-        function showASNameMessage(text, className) {
-            const line = document.createElement('p');
-            line.className = className;
-            line.textContent = text;
-            asNameResults.replaceChildren(line);
-        }
-
-        // List the matches of a search answer, each as a link to its AS page,
-        // and say how many more matched than the answer holds.
-        function showASNameMatches(data) {
-            if (data.total === 0) {
-                showASNameMessage('No AS matches "' + data.query + '".', 'search-message');
-                return;
-            }
-
-            const list = document.createElement('ul');
-            list.className = 'asname-list';
-            for (const match of data.results) {
-                const asn = document.createElement('span');
-                asn.className = 'asn';
-                asn.textContent = 'AS' + match.asn;
-
-                const description = document.createElement('span');
-                description.className = 'description';
-                description.textContent = match.description;
-
-                const link = document.createElement('a');
-                link.href = '/as/' + match.asn;
-                link.append(asn, ' ' + match.handle, description);
-
-                const item = document.createElement('li');
-                item.append(link);
-                list.append(item);
-            }
-            asNameResults.replaceChildren(list);
-
-            const more = data.total - data.results.length;
-            if (more > 0) {
-                const line = document.createElement('p');
-                line.className = 'search-message';
-                line.textContent = more.toLocaleString() + ' more matched. Narrow the search to see them.';
-                asNameResults.append(line);
-            }
-        }
-
         document.getElementById('asname-form').addEventListener('submit', function(e) {
             e.preventDefault();
             const query = document.getElementById('asname-input').value.trim();
             if (!query) return;
 
-            // A number, or AS and a number, opens that AS's page
+            const resultsDiv = document.getElementById('asname-results');
-            const asn = query.replace(/^AS/i, '');
+            resultsDiv.innerHTML = '<p class="loading" style="color: #7f8c8d; margin-top: 12px;">Searching...</p>';
-            if (/^\d+$/.test(asn)) {
+
-                window.location.href = '/as/' + asn;
+            // Use a simple client-side search against the asinfo data
+            // For now, redirect to AS page if it looks like an ASN
+            if (/^\d+$/.test(query)) {
+                window.location.href = '/as/' + query;
                 return;
             }
 
-            showASNameMessage('Searching...', 'loading');
+            // Show a message that server-side search is coming
-
+            resultsDiv.innerHTML = '<p style="color: #7f8c8d; margin-top: 12px; font-size: 13px;">AS name search coming soon. For now, try an AS number.</p>';
-            fetch('/api/v1/search/as?q=' + encodeURIComponent(query))
-                .then(response => response.json())
-                .then(response => {
-                    if (response.status === 'error') {
-                        showASNameMessage('Error: ' + response.error.msg, 'error');
-                    } else {
-                        showASNameMatches(response.data);
-                    }
-                })
-                .catch(error => {
-                    showASNameMessage('Error: ' + error.message, 'error');
-                });
         });
 
         // IP lookup
@@ -604,33 +439,9 @@
             document.getElementById('ip-input').value = '';
         });
 
-        // IP information API card: show the curl command that asks /ip/ about
+        // Initial load and refresh stats every 5 seconds
-        // the visitor's own address, then make that same request and show the
-        // JSON it answers, an error answer included. The answer holds outside
-        // data, so it is set with textContent, never innerHTML.
-        const ipAPICommand = document.getElementById('ip-api-command');
-        const ipAPIAnswer = document.getElementById('ip-api-answer');
-        const ipAPIURL = window.location.origin + '/ip/' + ipAPICommand.dataset.clientIp;
-
-        ipAPICommand.textContent = 'curl ' + ipAPIURL;
-
-        fetch(ipAPIURL)
-            .then(response => response.json())
-            .then(answer => {
-                ipAPIAnswer.className = '';
-                ipAPIAnswer.textContent = JSON.stringify(answer, null, 2);
-            })
-            .catch(error => {
-                ipAPIAnswer.className = 'error';
-                ipAPIAnswer.textContent = 'Error: ' + error.message;
-            });
-
-        // Initial load and refresh stats every REFRESH_PERIOD_MS. The freshness
-        // check runs every second on its own, so a request that never returns
-        // still turns the spinner into "stale".
         updateStats();
-        setInterval(updateStats, REFRESH_PERIOD_MS);
+        setInterval(updateStats, 5000);
-        setInterval(showFreshness, MS_PER_SECOND);
     </script>
 </body>
 </html>

internal/templates/prefix_detail.html

@@ -4,7 +4,7 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>{{.Prefix}} - RouteWatch</title>
-    <style nonce="{{.CSPNonce}}">
+    <style>
         * {
             box-sizing: border-box;
         }
@@ -221,10 +221,6 @@
             border-radius: 4px;
             border: 1px solid #e0e0e0;
         }
-        .origin-item .peer-count {
-            color: #7f8c8d;
-            margin-left: 10px;
-        }
         .empty-state {
             text-align: center;
             padding: 40px;
@@ -292,7 +288,7 @@
                     {{else}}
                     No known origin
                     {{end}}
-                    <span class="peer-count">{{.PeerCount}} peer{{if ne .PeerCount 1}}s{{end}}</span>
+                    <span style="color: #7f8c8d; margin-left: 10px;">{{.PeerCount}} peer{{if ne .PeerCount 1}}s{{end}}</span>
                 </div>
                 {{end}}
             </div>

internal/templates/prefix_length.html

@@ -4,7 +4,7 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>Prefixes with /{{ .MaskLength }} - RouteWatch</title>
-    <style nonce="{{.CSPNonce}}">
+    <style>
         * {
             box-sizing: border-box;
         }

internal/templates/status.html

@@ -4,7 +4,7 @@
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>RouteWatch Status</title>
-    <style nonce="{{.CSPNonce}}">
+    <style>
         * {
             box-sizing: border-box;
         }
@@ -126,12 +126,10 @@
         .connected {
             color: #22c55e;
         }
-        .disconnected,
+        .disconnected {
-        .stale {
             color: #ef4444;
         }
         .error {
-            display: none;
             background: #fee;
             color: #c00;
             padding: 10px;
@@ -175,16 +173,13 @@
     </nav>
 
     <main class="main-content">
-    <div id="error" class="error"></div>
+    <div id="error" class="error" style="display: none;"></div>
     <div class="status-grid">
         <div class="status-card">
             <h2>RouteWatch</h2>
             <div class="metric">
                 <span class="metric-label">Status</span>
-                <span>
                 <span class="metric-value" id="connected">-</span>
-                    <span class="metric-value" id="freshness"></span>
-                </span>
             </div>
             <div class="metric">
                 <span class="metric-label">Uptime</span>
@@ -235,8 +230,8 @@
                 <span class="metric-value" id="withdrawals">-</span>
             </div>
             <div class="metric">
-                <span class="metric-label">Route Churn/min</span>
+                <span class="metric-label">Route Churn/sec</span>
-                <span class="metric-value" id="route_churn_per_min">-</span>
+                <span class="metric-value" id="route_churn_per_sec">-</span>
             </div>
             <div class="metric">
                 <span class="metric-label">Total Data</span>
@@ -389,45 +384,7 @@
         <!-- Handler stats will be dynamically added here -->
     </div>
     
-    <script nonce="{{.CSPNonce}}">
+    <script>
-        // The stats are fetched every REFRESH_PERIOD_MS. Next to the status, a
-        // spinner turns one frame each time an answer counts more messages
-        // than the one before. After STALE_AFTER_REFRESHES refreshes without
-        // such an answer, the spinner gives way to "stale" and the time since.
-        const REFRESH_PERIOD_MS = 2000;
-        const STALE_AFTER_REFRESHES = 3;
-        const MS_PER_SECOND = 1000;
-        const SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'];
-        const SPINNER_COLOURS = ['#e74c3c', '#e67e22', '#f1c40f', '#27ae60', '#3498db', '#9b59b6'];
-
-        let spinnerFrame = 0;
-        let lastMessageCount = null;
-        let lastFreshAt = Date.now();
-
-        // Turn the spinner when the message count has risen since the last answer.
-        function noteMessageCount(count) {
-            if (lastMessageCount !== null && count > lastMessageCount) {
-                spinnerFrame = (spinnerFrame + 1) % SPINNER_FRAMES.length;
-                lastFreshAt = Date.now();
-            }
-            lastMessageCount = count;
-            showFreshness();
-        }
-
-        function showFreshness() {
-            const freshness = document.getElementById('freshness');
-            const sinceFreshMs = Date.now() - lastFreshAt;
-            const stale = sinceFreshMs >= STALE_AFTER_REFRESHES * REFRESH_PERIOD_MS;
-            freshness.classList.toggle('stale', stale);
-            if (stale) {
-                freshness.textContent = 'stale for ' + Math.floor(sinceFreshMs / MS_PER_SECOND) + 's';
-                freshness.style.color = '';
-                return;
-            }
-            freshness.textContent = SPINNER_FRAMES[spinnerFrame];
-            freshness.style.color = SPINNER_COLOURS[spinnerFrame % SPINNER_COLOURS.length];
-        }
-
         function formatBytes(bytes) {
             if (bytes === 0) return '0 B';
             const k = 1024;
@@ -553,7 +510,7 @@
             document.getElementById('messages_per_sec').textContent = '-';
             document.getElementById('announcements').textContent = '-';
             document.getElementById('withdrawals').textContent = '-';
-            document.getElementById('route_churn_per_min').textContent = '-';
+            document.getElementById('route_churn_per_sec').textContent = '-';
             document.getElementById('total_wire_bytes').textContent = '-';
             document.getElementById('wire_mbits_per_sec').textContent = '-';
             document.getElementById('gc_num').textContent = '-';
@@ -611,7 +568,6 @@
                     const connectedEl = document.getElementById('connected');
                     connectedEl.textContent = data.connected ? 'Connected' : 'Disconnected';
                     connectedEl.className = 'metric-value ' + (data.connected ? 'connected' : 'disconnected');
-                    noteMessageCount(data.total_messages);
                     
                     // Update all metrics
                     document.getElementById('uptime').textContent = data.uptime;
@@ -621,7 +577,7 @@
                     document.getElementById('connection_duration').textContent = data.connection_duration;
                     document.getElementById('reconnect_count').textContent = formatNumber(data.reconnect_count);
                     document.getElementById('total_messages').textContent = formatNumber(data.total_messages);
-                    document.getElementById('messages_per_sec').textContent = formatNumber(Math.round(data.messages_per_sec));
+                    document.getElementById('messages_per_sec').textContent = data.messages_per_sec.toFixed(1);
                     document.getElementById('total_wire_bytes').textContent = formatBytes(data.total_wire_bytes);
                     document.getElementById('wire_mbits_per_sec').textContent = data.wire_mbits_per_sec.toFixed(2) + ' Mbps';
                     document.getElementById('asns').textContent = formatNumber(data.asns);
@@ -634,8 +590,8 @@
                     document.getElementById('live_routes').textContent = formatNumber(data.live_routes);
                     document.getElementById('ipv4_routes').textContent = formatNumber(data.ipv4_routes);
                     document.getElementById('ipv6_routes').textContent = formatNumber(data.ipv6_routes);
-                    document.getElementById('ipv4_updates_per_sec').textContent = formatNumber(Math.round(data.ipv4_updates_per_sec));
+                    document.getElementById('ipv4_updates_per_sec').textContent = data.ipv4_updates_per_sec.toFixed(1);
-                    document.getElementById('ipv6_updates_per_sec').textContent = formatNumber(Math.round(data.ipv6_updates_per_sec));
+                    document.getElementById('ipv6_updates_per_sec').textContent = data.ipv6_updates_per_sec.toFixed(1);
                     document.getElementById('oldest_route').textContent = formatRelativeTime(data.oldest_route);
                     document.getElementById('newest_route').textContent = formatRelativeTime(data.newest_route);
 
@@ -644,7 +600,7 @@
                         document.getElementById('bgp_peer_count').textContent = formatNumber(data.stream.bgp_peer_count);
                         document.getElementById('announcements').textContent = formatNumber(data.stream.announcements);
                         document.getElementById('withdrawals').textContent = formatNumber(data.stream.withdrawals);
-                        document.getElementById('route_churn_per_min').textContent = formatNumber(data.stream.route_churn_per_min);
+                        document.getElementById('route_churn_per_sec').textContent = data.stream.route_churn_per_sec.toFixed(1);
                     }
 
                     // Update GC stats
@@ -686,12 +642,9 @@
                 });
         }
         
-        // Update immediately and then every REFRESH_PERIOD_MS. The freshness
+        // Update immediately and then every 2 seconds
-        // check runs every second on its own, so a request that never returns
-        // still turns the spinner into "stale".
         updateStatus();
-        setInterval(updateStatus, REFRESH_PERIOD_MS);
+        setInterval(updateStatus, 2000);
-        setInterval(showFreshness, MS_PER_SECOND);
     </script>
     </main>
 

internal/templates/templates.go

@@ -7,9 +7,9 @@ import (
 	"net/url"
 	"strings"
 	"sync"
+	"time"
 
 	"git.eeqj.de/sneak/routewatch/internal/version"
-	"github.com/dustin/go-humanize"
 )
 
 //go:embed index.html
@@ -48,7 +48,48 @@ var (
 	once sync.Once
 )
 
-const cidrPartCount = 2 // A CIDR has two parts: prefix and length
+const (
+	hoursPerDay   = 24
+	daysPerMonth  = 30
+	cidrPartCount = 2 // A CIDR has two parts: prefix and length
+)
+
+// timeSince returns a human-readable duration since the given time
+func timeSince(t time.Time) string {
+	duration := time.Since(t)
+	if duration < time.Minute {
+		return "just now"
+	}
+
+	if duration < time.Hour {
+		minutes := int(duration.Minutes())
+		if minutes == 1 {
+			return "1 minute ago"
+		}
+
+		return duration.Truncate(time.Minute).String() + " ago"
+	}
+
+	if duration < hoursPerDay*time.Hour {
+		hours := int(duration.Hours())
+		if hours == 1 {
+			return "1 hour ago"
+		}
+
+		return duration.Truncate(time.Hour).String() + " ago"
+	}
+
+	days := int(duration.Hours() / hoursPerDay)
+	if days == 1 {
+		return "1 day ago"
+	}
+
+	if days < daysPerMonth {
+		return duration.Truncate(hoursPerDay*time.Hour).String() + " ago"
+	}
+
+	return t.Format("2006-01-02")
+}
 
 // prefixURL generates a URL path for a prefix in CIDR notation.
 // Takes a prefix like "192.168.1.0/24" and returns "/prefix/192.168.1.0/24"
@@ -72,7 +113,7 @@ func initTemplates() {
 
 	// Create common template functions
 	funcs := template.FuncMap{
-		"timeSince":       humanize.Time,
+		"timeSince":       timeSince,
 		"urlEncode":       url.QueryEscape,
 		"prefixURL":       prefixURL,
 		"appName":         func() string { return version.Name },

internal/templates/templates_test.go

@@ -1,181 +0,0 @@
-package templates_test
-
-import (
-	"bytes"
-	"html/template"
-	"strings"
-	"testing"
-
-	"git.eeqj.de/sneak/routewatch/internal/database"
-	"git.eeqj.de/sneak/routewatch/internal/templates"
-)
-
-// renderPage renders a page with the data every page needs, and the visitor's
-// address, which the home page also needs.
-func renderPage(t *testing.T, tmpl *template.Template) string {
-	t.Helper()
-
-	var page bytes.Buffer
-
-	err := tmpl.Execute(&page, struct{ CSPNonce, ClientIP string }{})
-	if err != nil {
-		t.Fatalf("rendering the page: %v", err)
-	}
-
-	return page.String()
-}
-
-// TestFooterLicense renders the home and status pages and expects each footer
-// to name the licence in LICENSE, MIT, and not WTFPL.
-func TestFooterLicense(t *testing.T) {
-	t.Parallel()
-
-	const (
-		license      = "MIT"
-		wrongLicense = "WTFPL"
-	)
-
-	pages := map[string]*template.Template{
-		"index":  templates.IndexTemplate(),
-		"status": templates.StatusTemplate(),
-	}
-
-	for name, tmpl := range pages {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			_, footer, found := strings.Cut(renderPage(t, tmpl), "<footer")
-			if !found {
-				t.Fatal("the page has no footer")
-			}
-
-			footer, _, _ = strings.Cut(footer, "</footer>")
-
-			if !strings.Contains(footer, license) {
-				t.Errorf("footer does not name %s: %q", license, footer)
-			}
-
-			if strings.Contains(footer, wrongLicense) {
-				t.Errorf("footer names %s: %q", wrongLicense, footer)
-			}
-		})
-	}
-}
-
-// TestFreshnessIndicator renders the home and status pages and expects each to
-// have, right after the status element, the element where the page's script
-// shows its spinner or "stale".
-func TestFreshnessIndicator(t *testing.T) {
-	t.Parallel()
-
-	pages := map[string]struct {
-		tmpl      *template.Template
-		status    string
-		indicator string
-	}{
-		"index": {
-			tmpl:      templates.IndexTemplate(),
-			status:    `<span id="stat-status">-</span>`,
-			indicator: `<span id="freshness"></span>`,
-		},
-		"status": {
-			tmpl:      templates.StatusTemplate(),
-			status:    `<span class="metric-value" id="connected">-</span>`,
-			indicator: `<span class="metric-value" id="freshness"></span>`,
-		},
-	}
-
-	for name, page := range pages {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			_, afterStatus, found := strings.Cut(renderPage(t, page.tmpl), page.status)
-			if !found {
-				t.Fatalf("the page has no status element %s", page.status)
-			}
-
-			if !strings.HasPrefix(strings.TrimSpace(afterStatus), page.indicator) {
-				t.Errorf("the status element is not followed by %s", page.indicator)
-			}
-		})
-	}
-}
-
-// TestASDetailPrefixListsFold renders the AS page with one IPv4 and one IPv6
-// prefix. It expects each prefix list inside its own <details> element that
-// starts open and whose <summary> holds the list's heading, so that a click on
-// the heading folds the list.
-func TestASDetailPrefixListsFold(t *testing.T) {
-	t.Parallel()
-
-	const (
-		asn      = 64500
-		prefixV4 = "192.0.2.0/24"
-		prefixV6 = "2001:db8::/32"
-	)
-
-	ipv4 := []database.LiveRoute{{Prefix: prefixV4}}
-	ipv6 := []database.LiveRoute{{Prefix: prefixV6}}
-
-	// The fields the AS page handler gives the template.
-	data := struct {
-		ASN          *database.ASN
-		IPv4Prefixes []database.LiveRoute
-		IPv6Prefixes []database.LiveRoute
-		TotalCount   int
-		IPv4Count    int
-		IPv6Count    int
-		Peers        []database.ASPeer
-		PeerCount    int
-		CSPNonce     string
-	}{
-		ASN:          &database.ASN{ASN: asn},
-		IPv4Prefixes: ipv4,
-		IPv6Prefixes: ipv6,
-		TotalCount:   len(ipv4) + len(ipv6),
-		IPv4Count:    len(ipv4),
-		IPv6Count:    len(ipv6),
-	}
-
-	var rendered bytes.Buffer
-
-	err := templates.ASDetailTemplate().Execute(&rendered, data)
-	if err != nil {
-		t.Fatalf("rendering the page: %v", err)
-	}
-
-	page := rendered.String()
-
-	lists := []struct{ heading, prefix string }{
-		{"IPv4 Prefixes", prefixV4},
-		{"IPv6 Prefixes", prefixV6},
-	}
-
-	for _, list := range lists {
-		before, _, found := strings.Cut(page, list.prefix)
-		if !found {
-			t.Fatalf("the page does not list %s", list.prefix)
-		}
-
-		// The list's section starts at the last <details before its prefix.
-		start := strings.LastIndex(before, "<details")
-		if start < 0 || strings.Contains(before[start:], "</details>") {
-			t.Errorf("%s: the list is not inside a <details> element", list.heading)
-
-			continue
-		}
-
-		tag, section, _ := strings.Cut(before[start:], ">")
-		if !strings.Contains(tag, " open") {
-			t.Errorf("%s: the <details> element does not start open: %s>",
-				list.heading, tag)
-		}
-
-		_, summary, _ := strings.Cut(section, "<summary")
-		summary, _, _ = strings.Cut(summary, "</summary>")
-
-		if !strings.Contains(summary, list.heading) {
-			t.Errorf("%s: the <summary> does not hold the heading", list.heading)
-		}
-	}
-}

internal/version/version.go

@@ -19,8 +19,8 @@ const (
 	Author = "@sneak"
 	// AuthorURL is the author's website
 	AuthorURL = "https://sneak.berlin"
-	// License is the program license, as LICENSE grants it
+	// License is the program license
-	License = "MIT"
+	License = "WTFPL"
 	// RepoURL is the git repository URL
 	RepoURL = "https://git.eeqj.de/sneak/routewatch"
 )

internal/whois/whois.go

@@ -4,18 +4,11 @@ package whois
 import (
 	"bufio"
 	"context"
-	"errors"
 	"fmt"
-	"io"
-	"log/slog"
 	"net"
 	"regexp"
 	"strings"
-	"sync"
 	"time"
-	"unicode/utf8"
-
-	"golang.org/x/text/encoding/charmap"
 )
 
 // Timeout constants for WHOIS queries.
@@ -25,24 +18,12 @@ const (
 	writeTimeout = 5 * time.Second
 )
 
-// maxAnswerSize caps the bytes read of one WHOIS answer: a real one is a
-// few KiB, so one this large is from a server that does not stop sending.
-const maxAnswerSize = 1 << 20
-
 // Parsing constants.
 const (
+	keyValueParts       = 2 // Expected parts when splitting "key: value"
 	lacnicDateFormatLen = 8 // Length of YYYYMMDD date format
 )
 
-// The classes of the two objects that describe an AS in an answer from
-// RIPE, APNIC or AFRINIC, and the key of the aut-num object's line that
-// gives the ID of the AS's organisation object.
-const (
-	autNumClass       = "aut-num"
-	organisationClass = "organisation"
-	orgKey            = "org"
-)
-
 // WHOIS server addresses.
 const (
 	whoisServerIANA    = "whois.iana.org:43"
@@ -51,32 +32,8 @@ const (
 	whoisServerAPNIC   = "whois.apnic.net:43"
 	whoisServerLACNIC  = "whois.lacnic.net:43"
 	whoisServerAFRINIC = "whois.afrinic.net:43"
-
-	// NIC.br registers Brazil's ASes under LACNIC.
-	whoisServerNICbr = "whois.registro.br:43"
 )
 
-// nicbrHeader starts the copyright notice at the top of NIC.br's WHOIS
-// answers. LACNIC's answer for an AS that NIC.br registers is NIC.br's, and
-// carries the same notice:
-//
-//	% Copyright (c) Nic.br - Use of this data is governed by the Use and
-//	% Privacy Policy at https://registro.br/upp . Distribution,
-const nicbrHeader = "% Copyright (c) Nic.br"
-
-// nicbrLimitReached is in the line that marks NIC.br's reduced answer to a
-// client over its query limit. NIC.br publishes no limit. Its Use and
-// Privacy Policy, https://registro.br/upp, says only that it restricts
-// queries to stop the data being taken in bulk.
-const nicbrLimitReached = "Query rate limit exceeded"
-
-// NICbrSkipTime is how long the client stops asking NIC.br after NIC.br
-// answers that the client is over its query limit. NIC.br publishes
-// neither the limit nor how long it lasts. An hour outlasts a limit
-// counted per minute or per hour. The ASes looked up meanwhile get
-// NIC.br's fields when they are next refreshed.
-const NICbrSkipTime = time.Hour
-
 // RIR identifiers.
 const (
 	RIRARIN   = "ARIN"
@@ -86,45 +43,6 @@ const (
 	RIRAFRNIC = "AFRINIC"
 )
 
-// noPersonalFlag keeps person and role objects out of an answer. The
-// organisation object stays in it. RIPE counts the person and role objects
-// it returns against a daily limit per IP address. Role objects for abuse
-// contacts do not count. RIPE refuses an address for good that passes the
-// limit too often:
-// https://docs.db.ripe.net/Types-of-Queries/Access-Control-for-Queries/
-//
-// RIPE documents the flag here:
-// https://docs.db.ripe.net/Types-of-Queries/Referenced-Objects-in-Query-Response/
-// APNIC and AFRINIC list it in their answers to the query "help". AFRINIC
-// sends readers to that answer for its full list of flags:
-// https://afrinic.net/whois-getting-started
-const noPersonalFlag = "--no-personal"
-
-// The error lines with which RIPE's database software refuses a query:
-// access denied to the client's IP address, and that address's limit
-// reached. APNIC and AFRINIC run the same software. The lines are listed
-// under "Access Errors" on this page, whose URL is split over two lines:
-// https://docs.db.ripe.net/Appendices/
-// Appendix-C--RIPE-Database-Query-Server-Response-Codes-and-Messages/
-const (
-	accessDeniedError = "%ERROR:201:"
-	accessLimitError  = "%ERROR:202:"
-)
-
-// Errors LookupASN returns when a WHOIS answer has nothing to store.
-var (
-	// ErrNoReferral is returned when IANA refers the AS to no RIR, as it
-	// does for a reserved, private-use or unallocated AS number.
-	ErrNoReferral = errors.New("IANA refers the AS to no RIR")
-
-	errNoASBlock = errors.New("IANA answered with neither a referral " +
-		"nor an AS block")
-	errNoFields       = errors.New("RIR answered with no WHOIS fields")
-	errRefused        = errors.New("RIR refused the query")
-	errUnknownServer  = errors.New("IANA refers the AS to a server no RIR runs")
-	errAnswerTooLarge = errors.New("WHOIS answer reached the size cap")
-)
-
 // ASNInfo contains parsed WHOIS information for an ASN.
 type ASNInfo struct {
 	ASN         int
@@ -145,46 +63,20 @@ type ASNInfo struct {
 
 // Client performs WHOIS lookups for ASNs.
 type Client struct {
-	// DialContext opens the connection to a WHOIS server, given as
+	// Dialer for creating connections (can be overridden for testing)
-	// host:port. NewClient sets a net.Dialer's; tests replace it to reach
+	dialer *net.Dialer
-	// local servers instead.
-	DialContext func(ctx context.Context, network, address string) (net.Conn, error)
-
-	// Now returns the current time. NewClient sets time.Now; tests replace
-	// it to move the clock.
-	Now func() time.Time
-
-	logger *slog.Logger
-
-	// mu guards nicbrSkipUntil.
-	mu sync.Mutex
-
-	// nicbrSkipUntil is when the client next asks NIC.br, after NIC.br
-	// answered that the client is over its query limit.
-	nicbrSkipUntil time.Time
 }
 
-// NewClient creates a new WHOIS client that logs to logger.
+// NewClient creates a new WHOIS client.
-func NewClient(logger *slog.Logger) *Client {
+func NewClient() *Client {
-	dialer := &net.Dialer{Timeout: dialTimeout}
-
 	return &Client{
-		DialContext: dialer.DialContext,
+		dialer: &net.Dialer{
-		Now:         time.Now,
+			Timeout: dialTimeout,
-		logger:      logger,
+		},
 	}
 }
 
-// LookupASN asks IANA which RIR holds the given ASN, asks that RIR, and
+// LookupASN queries WHOIS for the given ASN and returns parsed information.
-// returns what the RIR's answer says. When LACNIC's answer is NIC.br's, it
-// returns what NIC.br's own server says instead, unless that query fails or
-// is skipped, as askNICbr says.
-//
-// It returns ErrNoReferral when IANA refers the AS to no RIR, and another
-// error when the query to IANA or to the RIR fails, when the RIR refuses
-// it, when IANA's answer has neither a referral nor the AS's block, when
-// IANA refers the AS to a server none of the five RIRs run, or when the
-// answer carries none of ASNInfo's fields.
 func (c *Client) LookupASN(ctx context.Context, asn int) (*ASNInfo, error) {
 	// Query IANA first to find the authoritative RIR
 	query := fmt.Sprintf("AS%d", asn)
@@ -195,145 +87,34 @@ func (c *Client) LookupASN(ctx context.Context, asn int) (*ASNInfo, error) {
 	}
 
 	// Determine RIR from IANA response
-	rir, whoisServer, err := c.parseIANAReferral(ianaResp)
+	rir, whoisServer := c.parseIANAReferral(ianaResp)
-	if err != nil {
-		return nil, fmt.Errorf("%s: %w", query, err)
-	}
-
 	if whoisServer == "" {
-		// IANA names the block of an AS it refers to no RIR. An answer
+		// No referral, try to parse what we have
-		// without that, such as an empty one, says nothing about the AS.
+		return c.parseResponse(asn, rir, ianaResp), nil
-		if !strings.Contains(ianaResp, "as-block:") {
-			return nil, fmt.Errorf("%s: %w", query, errNoASBlock)
-		}
-
-		return nil, fmt.Errorf("%s: %w", query, ErrNoReferral)
 	}
 
 	// Query the authoritative RIR
-	rirResp, err := c.queryRIR(ctx, whoisServer, rirQuery(rir, query))
+	rirResp, err := c.query(ctx, whoisServer, query)
 	if err != nil {
-		return nil, fmt.Errorf("RIR query failed: %w", err)
+		// Return partial data from IANA if RIR query fails
-	}
+		info := c.parseResponse(asn, rir, ianaResp)
-
+		info.RawResponse = fmt.Sprintf("%s\n--- RIR query failed: %v ---\n",
-	// The fields never come from IANA's answer, which describes the block
+			ianaResp, err)
-	// the AS is in.
-	info := c.parseResponse(asn, rir, rirResp)
-
-	if rir == RIRLACNIC && strings.Contains(rirResp, nicbrHeader) {
-		info = c.askNICbr(ctx, query, info)
-	}
-
-	if info.isEmpty() {
-		return nil, fmt.Errorf("%s from %s: %w", query, whoisServer, errNoFields)
-	}
-
-	info.RawResponse = ianaResp + "\n" + info.RawResponse
 
 		return info, nil
 	}
 
-// askNICbr asks NIC.br's server about an AS whose answer from LACNIC is
+	// Combine responses and parse
-// NIC.br's, and returns what NIC.br's answer says. LACNIC replaces each
+	fullResponse := ianaResp + "\n" + rirResp
-// accented letter in such an answer with U+FFFD. NIC.br's own server keeps
+	info := c.parseResponse(asn, rir, fullResponse)
-// them, in Latin-1.
+	info.RawResponse = fullResponse
-//
-// askNICbr returns lacnicInfo, what LACNIC's answer says, and logs a
-// warning when the query fails, when NIC.br sends the reduced answer for a
-// client over its limit, or when NIC.br's answer carries none of ASNInfo's
-// fields. After the reduced answer, it returns lacnicInfo without asking
-// NIC.br until NICbrSkipTime has passed.
-func (c *Client) askNICbr(
-	ctx context.Context, asQuery string, lacnicInfo *ASNInfo,
-) *ASNInfo {
-	asn := lacnicInfo.ASN
 
-	c.mu.Lock()
+	return info, nil
-	skipping := c.Now().Before(c.nicbrSkipUntil)
-	c.mu.Unlock()
-
-	if skipping {
-		return lacnicInfo
 }
 
-	answer, err := c.query(ctx, whoisServerNICbr, asQuery)
+// query performs a raw WHOIS query to the specified server.
-	if err != nil {
-		c.logger.Warn("NIC.br query failed, keeping LACNIC's fields",
-			"asn", asn, "error", err)
-
-		return lacnicInfo
-	}
-
-	if strings.Contains(answer, nicbrLimitReached) {
-		c.mu.Lock()
-		c.nicbrSkipUntil = c.Now().Add(NICbrSkipTime)
-		c.mu.Unlock()
-
-		c.logger.Warn("NIC.br answered that the client is over its query "+
-			"limit, keeping LACNIC's fields and skipping NIC.br",
-			"asn", asn, "skip_for", NICbrSkipTime)
-
-		return lacnicInfo
-	}
-
-	info := c.parseResponse(asn, RIRLACNIC, answer)
-	if info.isEmpty() {
-		c.logger.Warn("NIC.br answered with no WHOIS fields, "+
-			"keeping LACNIC's fields", "asn", asn)
-
-		return lacnicInfo
-	}
-
-	return info
-}
-
-// isEmpty reports whether info holds none of the fields a WHOIS answer
-// fills, as when the answer is only comment and error lines.
-func (info *ASNInfo) isEmpty() bool {
-	return info.ASName == "" && info.OrgName == "" && info.OrgID == "" &&
-		info.Address == "" && info.CountryCode == "" &&
-		info.AbuseEmail == "" && info.AbusePhone == "" &&
-		info.TechEmail == "" && info.TechPhone == "" &&
-		info.RegDate == nil && info.LastMod == nil
-}
-
-// rirQuery returns the query sent to rir's server for asQuery, an AS
-// number such as "AS3333". RIPE, APNIC and AFRINIC are asked to keep person
-// and role objects out of the answer.
-func rirQuery(rir, asQuery string) string {
-	switch rir {
-	case RIRRIPE, RIRAPNIC, RIRAFRNIC:
-		return noPersonalFlag + " " + asQuery
-	default:
-		return asQuery
-	}
-}
-
-// queryRIR sends query to an RIR's server and returns the answer. An answer
-// holding an error line with which the server refuses the query is an
-// error, and the error holds that line.
-func (c *Client) queryRIR(
-	ctx context.Context, server, query string,
-) (string, error) {
-	answer, err := c.query(ctx, server, query)
-	if err != nil {
-		return "", err
-	}
-
-	for line := range strings.SplitSeq(answer, "\n") {
-		if strings.HasPrefix(line, accessDeniedError) ||
-			strings.HasPrefix(line, accessLimitError) {
-			return "", fmt.Errorf("%w: %s", errRefused, line)
-		}
-	}
-
-	return answer, nil
-}
-
-// query performs a raw WHOIS query to the specified server and returns the
-// answer in UTF-8. An answer that reaches maxAnswerSize is an error.
 func (c *Client) query(ctx context.Context, server, query string) (string, error) {
-	conn, err := c.DialContext(ctx, "tcp", server)
+	conn, err := c.dialer.DialContext(ctx, "tcp", server)
 	if err != nil {
 		return "", fmt.Errorf("dial %s: %w", server, err)
 	}
@@ -359,11 +140,7 @@ func (c *Client) query(ctx context.Context, server, query string) (string, error
 
 	var sb strings.Builder
 
-	// The reader ends at the cap; N, the bytes it has left, is 0 once the
+	scanner := bufio.NewScanner(conn)
-	// answer has reached it.
-	answer := &io.LimitedReader{R: conn, N: maxAnswerSize}
-
-	scanner := bufio.NewScanner(answer)
 	for scanner.Scan() {
 		sb.WriteString(scanner.Text())
 		sb.WriteString("\n")
@@ -374,36 +151,13 @@ func (c *Client) query(ctx context.Context, server, query string) (string, error
 		return sb.String(), fmt.Errorf("read response: %w", err)
 	}
 
-	if answer.N == 0 {
+	return sb.String(), nil
-		return "", fmt.Errorf("read response: %w", errAnswerTooLarge)
-	}
-
-	return toUTF8(sb.String())
-}
-
-// toUTF8 returns a WHOIS answer in UTF-8. An answer that is valid UTF-8 is
-// returned as it is. Any other answer is decoded as Latin-1, which RIPE,
-// APNIC and NIC.br send. LACNIC, ARIN and AFRINIC send UTF-8. RIPE's
-// database working group lists the five RIRs here:
-// https://www.ripe.net/ripe/mail/archives/db-wg/2024-May/008075.html
-func toUTF8(answer string) (string, error) {
-	if utf8.ValidString(answer) {
-		return answer, nil
-	}
-
-	decoded, err := charmap.ISO8859_1.NewDecoder().String(answer)
-	if err != nil {
-		return "", fmt.Errorf("decode Latin-1 answer: %w", err)
-	}
-
-	return decoded, nil
 }
 
 // parseIANAReferral extracts the RIR and WHOIS server from an IANA response.
-// The server is "" when the response has no referral. A referral to a server
+func (c *Client) parseIANAReferral(response string) (string, string) {
-// that is none of the five RIRs' is an error, so a forged or broken answer
+	var rir string
-// cannot send the client to any other host or port.
+
-func (c *Client) parseIANAReferral(response string) (string, string, error) {
 	for line := range strings.SplitSeq(response, "\n") {
 		line = strings.TrimSpace(line)
 
@@ -414,27 +168,57 @@ func (c *Client) parseIANAReferral(response string) (string, string, error) {
 
 			switch {
 			case strings.Contains(server, "arin"):
-				return RIRARIN, whoisServerARIN, nil
+				return RIRARIN, whoisServerARIN
 			case strings.Contains(server, "ripe"):
-				return RIRRIPE, whoisServerRIPE, nil
+				return RIRRIPE, whoisServerRIPE
 			case strings.Contains(server, "apnic"):
-				return RIRAPNIC, whoisServerAPNIC, nil
+				return RIRAPNIC, whoisServerAPNIC
 			case strings.Contains(server, "lacnic"):
-				return RIRLACNIC, whoisServerLACNIC, nil
+				return RIRLACNIC, whoisServerLACNIC
 			case strings.Contains(server, "afrinic"):
-				return RIRAFRNIC, whoisServerAFRINIC, nil
+				return RIRAFRNIC, whoisServerAFRINIC
 			default:
-				return "", "", fmt.Errorf("%w: %q", errUnknownServer, server)
+				// Unknown server, add port if missing
+				if !strings.Contains(server, ":") {
+					server += ":43"
+				}
+
+				return "", server
+			}
+		}
+
+		// Also check organisation line for RIR hints
+		if strings.HasPrefix(strings.ToLower(line), "organisation:") {
+			orgRIR := rirInOrganisationLine(strings.ToLower(line))
+			if orgRIR != "" {
+				rir = orgRIR
 			}
 		}
 	}
 
-	return "", "", nil
+	return rir, ""
 }
 
-// parseResponse extracts ASN information from a WHOIS response. Of an
+// rirInOrganisationLine returns the RIR a lowercased IANA "organisation:"
-// answer from RIPE, APNIC or AFRINIC it reads only the lines asObjectLines
+// line names, or "" when it names none.
-// returns. Of an answer from ARIN or LACNIC it reads every line.
+func rirInOrganisationLine(org string) string {
+	switch {
+	case strings.Contains(org, "arin"):
+		return RIRARIN
+	case strings.Contains(org, "ripe"):
+		return RIRRIPE
+	case strings.Contains(org, "apnic"):
+		return RIRAPNIC
+	case strings.Contains(org, "lacnic"):
+		return RIRLACNIC
+	case strings.Contains(org, "afrinic"):
+		return RIRAFRNIC
+	default:
+		return ""
+	}
+}
+
+// parseResponse extracts ASN information from a WHOIS response.
 func (c *Client) parseResponse(asn int, rir, response string) *ASNInfo {
 	info := &ASNInfo{
 		ASN:         asn,
@@ -443,15 +227,26 @@ func (c *Client) parseResponse(asn int, rir, response string) *ASNInfo {
 	}
 
 	lines := strings.Split(response, "\n")
-	if rir == RIRRIPE || rir == RIRAPNIC || rir == RIRAFRNIC {
-		lines = asObjectLines(lines)
-	}
 
 	var addressLines []string
 
 	for _, line := range lines {
-		key, value, ok := splitField(line)
+		line = strings.TrimSpace(line)
-		if !ok || value == "" {
+		if line == "" || strings.HasPrefix(line, "%") ||
+			strings.HasPrefix(line, "#") {
+			continue
+		}
+
+		// Split on first colon
+		parts := strings.SplitN(line, ":", keyValueParts)
+		if len(parts) != keyValueParts {
+			continue
+		}
+
+		key := strings.TrimSpace(strings.ToLower(parts[0]))
+		value := strings.TrimSpace(parts[1])
+
+		if value == "" {
 			continue
 		}
 
@@ -478,72 +273,6 @@ func (c *Client) parseResponse(asn int, rir, response string) *ASNInfo {
 	return info
 }
 
-// splitField splits a "key: value" line of a WHOIS answer into its key, in
-// lower case, and its value. The third result is false for a line that
-// holds no field: a blank line, a comment line or a line without a colon.
-func splitField(line string) (string, string, bool) {
-	line = strings.TrimSpace(line)
-	if strings.HasPrefix(line, "%") || strings.HasPrefix(line, "#") {
-		return "", "", false
-	}
-
-	key, value, ok := strings.Cut(line, ":")
-
-	return strings.ToLower(strings.TrimSpace(key)), strings.TrimSpace(value), ok
-}
-
-// asObjectLines returns the lines of the aut-num object in an answer from
-// RIPE, APNIC or AFRINIC, followed by the lines of the organisation object
-// whose ID the aut-num object gives on its "org:" line. The answer's other
-// objects are left out. RIPE and APNIC start their answer with the
-// as-block object of the block the AS is in, whose dates are not the AS's.
-//
-// These answers are in RPSL. A blank line ends an object. An object's first
-// line has the object's class as its key and the object's ID as its value,
-// as in "organisation: ORG-APNI1-AP".
-func asObjectLines(lines []string) []string {
-	var autNum []string
-
-	// The lines of each organisation object, by its ID.
-	organisations := make(map[string][]string)
-
-	// The class and ID of the object the line is in, "" between objects.
-	var class, id string
-
-	// The ID of the organisation object the aut-num object refers to.
-	var orgID string
-
-	for _, line := range lines {
-		if strings.TrimSpace(line) == "" {
-			class = ""
-
-			continue
-		}
-
-		key, value, ok := splitField(line)
-		if !ok {
-			continue
-		}
-
-		if class == "" {
-			class, id = key, value
-		}
-
-		switch class {
-		case autNumClass:
-			autNum = append(autNum, line)
-
-			if key == orgKey {
-				orgID = value
-			}
-		case organisationClass:
-			organisations[id] = append(organisations[id], line)
-		}
-	}
-
-	return append(autNum, organisations[orgID]...)
-}
-
 // setResponseField stores the value of one "key: value" line of a WHOIS
 // response in info, keeping the first value seen for each field.
 func (c *Client) setResponseField(info *ASNInfo, key, value string) {

internal/whois/whois_test.go

@@ -1,881 +0,0 @@
-package whois_test
-
-import (
-	"bufio"
-	"bytes"
-	"context"
-	"encoding/json"
-	"errors"
-	"io"
-	"log/slog"
-	"maps"
-	"net"
-	"slices"
-	"strings"
-	"sync"
-	"syscall"
-	"testing"
-	"time"
-	"unicode/utf8"
-
-	"git.eeqj.de/sneak/routewatch/internal/whois"
-)
-
-// The servers LookupASN asks: IANA first, then the RIR IANA refers it to.
-const (
-	ianaServer = "whois.iana.org:43"
-	ripeServer = "whois.ripe.net:43"
-)
-
-// testASN is the AS the tests look up, and asQuery the query for it.
-const (
-	testASN = 3333
-	asQuery = "AS3333"
-)
-
-// The fake servers' answers are keyed by a server's address, a space and a
-// query. These are the keys of the queries LookupASN sends for testASN:
-// IANA's, then RIPE's, which asks RIPE to leave out person and role
-// objects.
-const (
-	ianaQuery = ianaServer + " " + asQuery
-	ripeQuery = ripeServer + " --no-personal " + asQuery
-)
-
-// ianaReferral is IANA's answer for an AS in a block it gave to RIPE.
-const ianaReferral = `% IANA WHOIS server
-
-refer:        whois.ripe.net
-
-as-block:     3154-3353
-organisation: Assigned by RIPE NCC
-whois:        whois.ripe.net
-changed:      2002-09-01
-source:       IANA
-`
-
-// lacnicReferral is IANA's answer for an AS in a block it gave to LACNIC,
-// shortened to the referral.
-const lacnicReferral = "refer:        whois.lacnic.net\n"
-
-// ripeRecord is RIPE's answer for the AS when asked to leave out person and
-// role objects. It starts with the as-block object of the block the AS is
-// in. Then come the abuse address as a comment, the AS's object and the
-// organisation object it refers to.
-const ripeRecord = `% This is the RIPE Database query service.
-% The objects are in RPSL format.
-
-% Note: --no-personal means ALL personal data has been filtered from this output.
-
-% Information related to 'AS3209 - AS3353'
-
-as-block:       AS3209 - AS3353
-descr:          RIPE NCC ASN block
-remarks:        These AS Numbers are assigned to network operators
-remarks:        in the RIPE NCC service region.
-mnt-by:         RIPE-NCC-HM-MNT
-created:        2018-11-22T15:27:19Z
-last-modified:  2018-11-22T15:27:19Z
-source:         RIPE
-
-% Information related to 'AS3333'
-
-% Abuse contact for 'AS3333' is 'abuse@ripe.net'
-
-aut-num:        AS3333
-as-name:        RIPE-NCC-AS
-org:            ORG-RIEN1-RIPE
-admin-c:        BRD-RIPE
-tech-c:         OPS4-RIPE
-abuse-c:        OPS4-RIPE
-mnt-by:         RIPE-NCC-MNT
-created:        2002-09-17T13:08:32Z
-last-modified:  2024-01-09T08:55:41Z
-source:         RIPE
-
-organisation:   ORG-RIEN1-RIPE
-org-name:       Reseaux IP Europeens Network Coordination Centre (RIPE NCC)
-country:        NL
-org-type:       RIR
-address:        P.O. Box 10096
-address:        1001EB Amsterdam
-abuse-c:        OPS4-RIPE
-created:        2012-03-09T13:08:29Z
-last-modified:  2024-03-26T10:55:38Z
-source:         RIPE
-`
-
-// The error lines with which RIPE refuses a query: access denied to the
-// host, and the host's limit on returned objects reached.
-const (
-	ripeAccessDenied = "%ERROR:201: access denied for 192.0.2.1"
-	ripeLimitReached = "%ERROR:202: access control limit reached"
-)
-
-// ripeRefusal is RIPE's answer to a host it no longer serves.
-const ripeRefusal = `% This is the RIPE Database query service.
-% The objects are in RPSL format.
-%
-` + ripeAccessDenied + `
-%
-% Sorry, access from your host has been permanently
-% denied because of a repeated excessive querying.
-`
-
-// apnicAfrinicRecord is the answer of APNIC or AFRINIC for the AS when asked
-// to leave out person and role objects. Like RIPE's, it holds the abuse
-// address as a comment, the AS's object and the organisation object.
-const apnicAfrinicRecord = `% Abuse contact for 'AS3333' is 'abuse@example.net'
-
-aut-num:        AS3333
-as-name:        EXAMPLE-AS
-org:            ORG-EXAMPLE1
-admin-c:        EX1-TEST
-last-modified:  2026-08-26T00:40:00Z
-
-organisation:   ORG-EXAMPLE1
-org-name:       Example Networks
-country:        ZA
-address:        1 Example Road
-`
-
-// apnicServer is APNIC's WHOIS server, and apnicASN an AS that APNIC holds.
-const (
-	apnicServer = "whois.apnic.net:43"
-	apnicASN    = 4608
-)
-
-// The keys of the queries LookupASN sends for apnicASN: IANA's, then
-// APNIC's, which asks APNIC to leave out person and role objects.
-const (
-	apnicIANAQuery = ianaServer + " AS4608"
-	apnicQuery     = apnicServer + " --no-personal AS4608"
-)
-
-// apnicRecord is APNIC's answer for apnicASN when asked to leave out person
-// and role objects, shortened. It starts with the as-block object of the
-// block the AS is in. Then come the abuse address as a comment, the AS's
-// object, and the irt and organisation objects it refers to.
-const apnicRecord = `% [whois.apnic.net]
-% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html
-
-% Note: --no-personal means ALL personal data has been filtered from this output.
-
-% Information related to 'AS4608 - AS4865'
-
-as-block:       AS4608 - AS4865
-descr:          APNIC ASN block
-remarks:        These AS numbers are further assigned by APNIC
-remarks:        to APNIC members and end-users in the APNIC region
-mnt-by:         APNIC-HM
-mnt-lower:      APNIC-HM
-last-modified:  2019-05-20T03:52:26Z
-source:         APNIC
-
-% Information related to 'AS4608'
-
-% Abuse contact for 'AS4608' is 'abuse@apnic.net'
-
-aut-num:        AS4608
-as-name:        APNIC-SERVICES
-descr:          Asia Pacific Network Information Centre
-country:        AU
-org:            ORG-APNI1-AP
-admin-c:        AIC1-AP
-abuse-c:        AA1589-AP
-mnt-by:         APNIC-HM
-mnt-irt:        IRT-APNIC-IS-AP
-last-modified:  2026-08-26T00:40:00Z
-source:         APNIC
-
-irt:            IRT-APNIC-IS-AP
-address:        South Brisbane, Australia
-e-mail:         helpdesk@apnic.net
-abuse-mailbox:  abuse@apnic.net
-mnt-by:         MAINT-APNIC-IS-AP
-last-modified:  2025-11-18T00:27:08Z
-source:         APNIC
-
-organisation:   ORG-APNI1-AP
-org-name:       Asia Pacific Network Information Centre
-org-type:       LIR
-country:        AU
-address:        PO Box 3646
-address:        South Brisbane, QLD 4101,
-address:        Australia
-e-mail:         noc@apnic.net
-mnt-by:         APNIC-HM
-last-modified:  2026-08-24T12:58:43Z
-source:         APNIC
-
-% This query was served by the APNIC Whois Service version 1.88.48 (WHOIS-UK2)
-`
-
-// ianaPrivateUse is IANA's answer for a private-use AS: its block and no
-// referral.
-const ianaPrivateUse = `% IANA WHOIS server
-
-as-block:     64512-65534
-organisation: Reserved for Private Use
-descr:        Reserved for Private Use
-source:       IANA
-`
-
-// ianaNoObject is IANA's answer when it has nothing for the query.
-const ianaNoObject = `% IANA WHOIS server
-% This query returned 0 objects.
-`
-
-// fillerLine is one of the lines a server that does not stop keeps sending.
-const fillerLine = "remarks:        and more\n"
-
-// endlessAnswerSize is how many bytes of filler lines that server sends:
-// four times the client's cap on an answer.
-const endlessAnswerSize = 4 << 20
-
-// fakeServers returns a DialContext for a whois.Client that connects to a
-// fake server at each address answers names. Each key of answers is a
-// server's address, a space and a query, and its value is that server's
-// answer to that query. A server reads one query and sends its answer, or
-// nothing for a query it has no answer for. Connecting to any other address
-// is refused.
-func fakeServers(
-	t *testing.T, answers map[string]string,
-) func(context.Context, string, string) (net.Conn, error) {
-	t.Helper()
-
-	addresses := make(map[string]bool)
-
-	for key := range answers {
-		address, _, _ := strings.Cut(key, " ")
-		addresses[address] = true
-	}
-
-	var servers sync.WaitGroup
-
-	t.Cleanup(servers.Wait)
-
-	return func(_ context.Context, _, address string) (net.Conn, error) {
-		if !addresses[address] {
-			return nil, syscall.ECONNREFUSED
-		}
-
-		client, server := net.Pipe()
-
-		servers.Go(func() {
-			defer func() { _ = server.Close() }()
-
-			query, _ := bufio.NewReader(server).ReadString('\n')
-			answer := answers[address+" "+strings.TrimSpace(query)]
-			_, _ = io.WriteString(server, answer)
-		})
-
-		return client, nil
-	}
-}
-
-// lookUp looks testASN up with fake servers sending answers.
-func lookUp(
-	t *testing.T, answers map[string]string,
-) (*whois.ASNInfo, error) {
-	t.Helper()
-
-	client := whois.NewClient(slog.New(slog.DiscardHandler))
-	client.DialContext = fakeServers(t, answers)
-
-	return client.LookupASN(t.Context(), testASN)
-}
-
-// TestLookupASNReadsTheRIRAnswer looks up an AS IANA refers to RIPE. RIPE
-// answers only a query that leaves out person and role objects, and its
-// answer to that still fills the AS's fields: the abuse address from the
-// comment line, and the organisation's name and country from its object.
-// The dates are the AS's. IANA's "changed:" line dates the AS's block, and
-// so do the dates of the as-block object at the start of RIPE's answer.
-func TestLookupASNReadsTheRIRAnswer(t *testing.T) {
-	t.Parallel()
-
-	info, err := lookUp(t, map[string]string{
-		ianaQuery: ianaReferral,
-		ripeQuery: ripeRecord,
-	})
-	if err != nil {
-		t.Fatalf("LookupASN: %v", err)
-	}
-
-	got := []string{
-		info.ASName, info.OrgID, info.OrgName, info.CountryCode,
-		info.AbuseEmail, info.RIR,
-	}
-	want := []string{
-		"RIPE-NCC-AS", "ORG-RIEN1-RIPE",
-		"Reseaux IP Europeens Network Coordination Centre (RIPE NCC)", "NL",
-		"abuse@ripe.net", whois.RIRRIPE,
-	}
-
-	if !slices.Equal(got, want) {
-		t.Errorf("LookupASN = AS name, org ID, org name, country, abuse "+
-			"address, RIR %q; want %q", got, want)
-	}
-
-	wantRegDate := time.Date(2002, 9, 17, 13, 8, 32, 0, time.UTC)
-	if info.RegDate == nil || !info.RegDate.Equal(wantRegDate) {
-		t.Errorf("LookupASN registered = %v, want %v",
-			info.RegDate, wantRegDate)
-	}
-
-	wantLastMod := time.Date(2024, 1, 9, 8, 55, 41, 0, time.UTC)
-	if info.LastMod == nil || !info.LastMod.Equal(wantLastMod) {
-		t.Errorf("LookupASN last modified = %v, want %v",
-			info.LastMod, wantLastMod)
-	}
-}
-
-// TestLookupASNReadsOnlyTheASAndItsOrganisation looks up an AS at APNIC,
-// whose answer starts with the as-block object of the block the AS is in.
-// The last-modified date is the aut-num object's, not the block's. The
-// address is the organisation object's alone, without the irt object's.
-func TestLookupASNReadsOnlyTheASAndItsOrganisation(t *testing.T) {
-	t.Parallel()
-
-	client := whois.NewClient(slog.New(slog.DiscardHandler))
-	client.DialContext = fakeServers(t, map[string]string{
-		apnicIANAQuery: "refer:        whois.apnic.net\n",
-		apnicQuery:     apnicRecord,
-	})
-
-	info, err := client.LookupASN(t.Context(), apnicASN)
-	if err != nil {
-		t.Fatalf("LookupASN: %v", err)
-	}
-
-	wantLastMod := time.Date(2026, 8, 26, 0, 40, 0, 0, time.UTC)
-	if info.LastMod == nil || !info.LastMod.Equal(wantLastMod) {
-		t.Errorf("LookupASN last modified = %v, want %v",
-			info.LastMod, wantLastMod)
-	}
-
-	got := []string{
-		info.ASName, info.OrgID, info.OrgName, info.CountryCode,
-		info.Address, info.AbuseEmail,
-	}
-	want := []string{
-		"APNIC-SERVICES", "ORG-APNI1-AP",
-		"Asia Pacific Network Information Centre", "AU",
-		"PO Box 3646\nSouth Brisbane, QLD 4101,\nAustralia", "abuse@apnic.net",
-	}
-
-	if !slices.Equal(got, want) {
-		t.Errorf("LookupASN = AS name, org ID, org name, country, address, "+
-			"abuse address %q; want %q", got, want)
-	}
-}
-
-// TestLookupASNAsksAPNICAndAFRINICToLeaveOutPersonalData looks up an AS at
-// APNIC and at AFRINIC. Each answers only a query that leaves out person
-// and role objects, as RIPE does. That one answer still holds the
-// organisation object, so the organisation's name and country are filled.
-func TestLookupASNAsksAPNICAndAFRINICToLeaveOutPersonalData(t *testing.T) {
-	t.Parallel()
-
-	for _, server := range []string{"whois.apnic.net", "whois.afrinic.net"} {
-		t.Run(server, func(t *testing.T) {
-			t.Parallel()
-
-			address := server + ":43"
-
-			info, err := lookUp(t, map[string]string{
-				ianaQuery:                             "refer:        " + server + "\n",
-				address + " --no-personal " + asQuery: apnicAfrinicRecord,
-			})
-			if err != nil {
-				t.Fatalf("LookupASN: %v", err)
-			}
-
-			got := []string{
-				info.ASName, info.OrgName, info.CountryCode, info.AbuseEmail,
-			}
-			want := []string{
-				"EXAMPLE-AS", "Example Networks", "ZA", "abuse@example.net",
-			}
-
-			if !slices.Equal(got, want) {
-				t.Errorf("LookupASN = AS name, org name, country, abuse "+
-					"address %q; want %q", got, want)
-			}
-		})
-	}
-}
-
-// The name of an organisation with accented letters, in Latin-1, one byte for
-// each letter, and in UTF-8.
-const (
-	latin1OrgName = "Fundaci\xf3n Ra\xedces de Espa\xf1a"
-	utf8OrgName   = "Fundación Raíces de España"
-)
-
-// TestLookupASNStoresAnswersInUTF8 looks up an AS at RIPE, which answers in
-// Latin-1, and at LACNIC, which answers in UTF-8. Both answers give the
-// organisation's name with its accented letters in UTF-8, in the parsed field
-// and in the stored answer. Before the fix the Latin-1 answer was kept as
-// received, which is not valid UTF-8.
-func TestLookupASNStoresAnswersInUTF8(t *testing.T) {
-	t.Parallel()
-
-	for name, answers := range map[string]map[string]string{
-		"RIPE in Latin-1": {
-			ianaQuery: ianaReferral,
-			ripeQuery: "aut-num:        AS3333\n" +
-				"org-name:       " + latin1OrgName + "\n",
-		},
-		"LACNIC in UTF-8": {
-			ianaQuery: lacnicReferral,
-			"whois.lacnic.net:43 " + asQuery: "aut-num:     AS3333\n" +
-				"owner:       " + utf8OrgName + "\n",
-		},
-	} {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			info, err := lookUp(t, answers)
-			if err != nil {
-				t.Fatalf("LookupASN: %v", err)
-			}
-
-			if info.OrgName != utf8OrgName {
-				t.Errorf("LookupASN org name = %q, want %q",
-					info.OrgName, utf8OrgName)
-			}
-
-			if !utf8.ValidString(info.RawResponse) ||
-				!strings.Contains(info.RawResponse, utf8OrgName) {
-				t.Errorf("LookupASN answer = %q, want UTF-8 holding %q",
-					info.RawResponse, utf8OrgName)
-			}
-		})
-	}
-}
-
-// lacnicServer is LACNIC's WHOIS server and nicbrServer NIC.br's, which
-// registers Brazil's ASes under LACNIC. brazilASN is one of those ASes.
-const (
-	lacnicServer = "whois.lacnic.net:43"
-	nicbrServer  = "whois.registro.br:43"
-	brazilASN    = 52751
-)
-
-// The keys of the queries LookupASN sends for brazilASN: IANA's, LACNIC's,
-// then NIC.br's.
-const (
-	brazilIANAQuery   = ianaServer + " AS52751"
-	brazilLACNICQuery = lacnicServer + " AS52751"
-	brazilNICbrQuery  = nicbrServer + " AS52751"
-)
-
-// The name of brazilASN's organisation in three forms: as LACNIC sends it,
-// with each accented letter replaced by U+FFFD; as NIC.br sends it, in
-// Latin-1; and in UTF-8.
-const (
-	lacnicBrazilOrgName = "Brasrede Telecomunica\U0000FFFD\U0000FFFDes LTDA"
-	nicbrBrazilOrgName  = "Brasrede Telecomunica\xe7\xf5es LTDA"
-	brazilOrgName       = "Brasrede Telecomunicações LTDA"
-)
-
-// nicbrCopyright is the copyright notice at the top of NIC.br's answers.
-const nicbrCopyright = "% Copyright (c) Nic.br - " +
-	"Use of this data is governed by the Use and\n" +
-	"% Privacy Policy at https://registro.br/upp . Distribution,\n" +
-	"% commercialization, reproduction, and use for advertising or similar\n" +
-	"% purposes are expressly prohibited.\n"
-
-// lacnicBrazilRecord is LACNIC's answer for brazilASN, recorded on
-// 2026-10-26. It is NIC.br's answer, which LACNIC passes on after a line
-// of its own and a space. It is shortened. The client's address is a
-// documentation address. The contact's name, the contact's object and the
-// footer are left out.
-const lacnicBrazilRecord = "% IP Client: 192.0.2.1\n" +
-	" " + nicbrCopyright + `% 2026-10-26T23:12:22-03:00 - 192.0.2.1
-
-aut-num:     AS52751
-owner:       ` + lacnicBrazilOrgName + `
-owner-c:     AVL217
-routing-c:   AVL217
-abuse-c:     AVL217
-created:     20120829
-changed:     20130306
-inetnum:     177.38.152.0/21
-inetnum:     2804:b34::/32
-inetnum:     191.243.4.0/22
-inetnum:     131.72.248.0/22
-inetnum:     138.94.228.0/22
-inetnum:     170.233.8.0/22
-inetnum:     170.246.4.0/22
-`
-
-// nicbrRecord is NIC.br's answer for brazilASN, in Latin-1, recorded on
-// 2026-10-26 and shortened as lacnicBrazilRecord is.
-const nicbrRecord = nicbrCopyright + `% 2026-10-26T23:14:41-03:00 - 192.0.2.1
-
-aut-num:     AS52751
-owner:       ` + nicbrBrazilOrgName + `
-owner-c:     AVL217
-routing-c:   AVL217
-abuse-c:     AVL217
-created:     20120829
-changed:     20130306
-inetnum:     177.38.152.0/21
-inetnum:     2804:b34::/32
-inetnum:     191.243.4.0/22
-inetnum:     131.72.248.0/22
-inetnum:     138.94.228.0/22
-inetnum:     170.233.8.0/22
-inetnum:     170.246.4.0/22
-`
-
-// nicbrOverLimit stands for NIC.br's answer for brazilASN to a client over
-// its query limit, and is not recorded. Such an answer leaves information
-// out, here the owner, and says so in its first line.
-const nicbrOverLimit = `% Query rate limit exceeded. Reduced information.
-
-aut-num:     AS52751
-created:     20120829
-changed:     20130306
-`
-
-// newNICbrClient returns a client whose fake servers send answers and
-// whose logger writes JSON lines to logs. It adds one to *nicbrAsks each
-// time it connects to NIC.br's server.
-func newNICbrClient(
-	t *testing.T, answers map[string]string, logs io.Writer, nicbrAsks *int,
-) *whois.Client {
-	t.Helper()
-
-	dial := fakeServers(t, answers)
-
-	client := whois.NewClient(slog.New(slog.NewJSONHandler(logs, nil)))
-	client.DialContext = func(
-		ctx context.Context, network, address string,
-	) (net.Conn, error) {
-		if address == nicbrServer {
-			*nicbrAsks++
-		}
-
-		return dial(ctx, network, address)
-	}
-
-	return client
-}
-
-// lookUpBrazilianAS looks brazilASN up with fake servers sending answers,
-// and reports whether the client connected to NIC.br's server.
-func lookUpBrazilianAS(
-	t *testing.T, answers map[string]string,
-) (*whois.ASNInfo, bool, error) {
-	t.Helper()
-
-	nicbrAsks := 0
-	client := newNICbrClient(t, answers, io.Discard, &nicbrAsks)
-
-	info, err := client.LookupASN(t.Context(), brazilASN)
-
-	return info, nicbrAsks > 0, err
-}
-
-// TestLookupASNTakesABrazilianASFromNICbr looks up an AS that NIC.br
-// registers. LACNIC's answer is NIC.br's with each accented letter replaced
-// by U+FFFD, so the client also asks NIC.br's server, which answers in
-// Latin-1. The organisation's name has its accented letters, in the parsed
-// field and in the stored answer.
-func TestLookupASNTakesABrazilianASFromNICbr(t *testing.T) {
-	t.Parallel()
-
-	info, _, err := lookUpBrazilianAS(t, map[string]string{
-		brazilIANAQuery:   lacnicReferral,
-		brazilLACNICQuery: lacnicBrazilRecord,
-		brazilNICbrQuery:  nicbrRecord,
-	})
-	if err != nil {
-		t.Fatalf("LookupASN: %v", err)
-	}
-
-	if info.OrgName != brazilOrgName || info.RIR != whois.RIRLACNIC {
-		t.Errorf("LookupASN org name, RIR = %q, %q; want %q, %q",
-			info.OrgName, info.RIR, brazilOrgName, whois.RIRLACNIC)
-	}
-
-	if !strings.Contains(info.RawResponse, brazilOrgName) {
-		t.Errorf("LookupASN answer = %q, want one holding %q",
-			info.RawResponse, brazilOrgName)
-	}
-
-	wantRegDate := time.Date(2012, 8, 29, 0, 0, 0, 0, time.UTC)
-	if info.RegDate == nil || !info.RegDate.Equal(wantRegDate) {
-		t.Errorf("LookupASN registered = %v, want %v",
-			info.RegDate, wantRegDate)
-	}
-}
-
-// TestLookupASNAsksNICbrOnlyForItsASes looks up an AS at LACNIC whose
-// answer does not carry NIC.br's copyright notice, as for an AS that LACNIC
-// registers itself. The client does not connect to NIC.br's server, which
-// would answer with a record, and keeps LACNIC's fields.
-func TestLookupASNAsksNICbrOnlyForItsASes(t *testing.T) {
-	t.Parallel()
-
-	info, askedNICbr, err := lookUpBrazilianAS(t, map[string]string{
-		brazilIANAQuery: lacnicReferral,
-		brazilLACNICQuery: "aut-num:     AS52751\n" +
-			"owner:       " + utf8OrgName + "\n",
-		brazilNICbrQuery: nicbrRecord,
-	})
-	if err != nil {
-		t.Fatalf("LookupASN: %v", err)
-	}
-
-	if askedNICbr {
-		t.Error("LookupASN asked NIC.br's server about an AS whose " +
-			"answer from LACNIC is not NIC.br's")
-	}
-
-	if info.OrgName != utf8OrgName {
-		t.Errorf("LookupASN org name = %q, want LACNIC's %q",
-			info.OrgName, utf8OrgName)
-	}
-}
-
-// TestLookupASNKeepsLACNICsAnswerWhenNICbrFails looks up an AS that NIC.br
-// registers when NIC.br's server refuses the connection, sends nothing, or
-// sends the reduced answer for a client over its query limit. The client
-// asks NIC.br, and the lookup still succeeds with LACNIC's fields. The
-// client logs one warning, which names the AS and the reason.
-func TestLookupASNKeepsLACNICsAnswerWhenNICbrFails(t *testing.T) {
-	t.Parallel()
-
-	for name, test := range map[string]struct {
-		nicbrAnswer map[string]string
-		reason      string
-	}{
-		"connection refused": {
-			nicbrAnswer: map[string]string{},
-			reason:      syscall.ECONNREFUSED.Error(),
-		},
-		"no answer": {
-			nicbrAnswer: map[string]string{brazilNICbrQuery: ""},
-			reason:      "no WHOIS fields",
-		},
-		"over the query limit": {
-			nicbrAnswer: map[string]string{brazilNICbrQuery: nicbrOverLimit},
-			reason:      "over its query limit",
-		},
-	} {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			answers := map[string]string{
-				brazilIANAQuery:   lacnicReferral,
-				brazilLACNICQuery: lacnicBrazilRecord,
-			}
-			maps.Copy(answers, test.nicbrAnswer)
-
-			var logs bytes.Buffer
-
-			nicbrAsks := 0
-			client := newNICbrClient(t, answers, &logs, &nicbrAsks)
-
-			info, err := client.LookupASN(t.Context(), brazilASN)
-			if err != nil {
-				t.Fatalf("LookupASN: %v", err)
-			}
-
-			if nicbrAsks == 0 {
-				t.Error("LookupASN did not ask NIC.br's server")
-			}
-
-			if info.OrgName != lacnicBrazilOrgName {
-				t.Errorf("LookupASN org name = %q, want LACNIC's %q",
-					info.OrgName, lacnicBrazilOrgName)
-			}
-
-			var logged struct {
-				Level string `json:"level"`
-				ASN   int    `json:"asn"`
-			}
-
-			err = json.Unmarshal(logs.Bytes(), &logged)
-			if err != nil {
-				t.Fatalf("want one log line, got %q: %v", logs.String(), err)
-			}
-
-			if logged.Level != slog.LevelWarn.String() ||
-				logged.ASN != brazilASN ||
-				!strings.Contains(logs.String(), test.reason) {
-				t.Errorf("logged %q, want a warning for AS%d holding %q",
-					logs.String(), brazilASN, test.reason)
-			}
-		})
-	}
-}
-
-// TestLookupASNSkipsNICbrAfterItsLimit looks up an AS that NIC.br
-// registers three times with one client, while NIC.br answers that the
-// client is over its query limit. The first lookup asks NIC.br. The
-// second, a second before NICbrSkipTime has passed, does not. The third,
-// once NICbrSkipTime has passed, asks again. Each keeps LACNIC's fields.
-func TestLookupASNSkipsNICbrAfterItsLimit(t *testing.T) {
-	t.Parallel()
-
-	nicbrAsks := 0
-	client := newNICbrClient(t, map[string]string{
-		brazilIANAQuery:   lacnicReferral,
-		brazilLACNICQuery: lacnicBrazilRecord,
-		brazilNICbrQuery:  nicbrOverLimit,
-	}, io.Discard, &nicbrAsks)
-
-	start := time.Date(2026, 10, 27, 0, 0, 0, 0, time.UTC)
-	now := start
-	client.Now = func() time.Time { return now }
-
-	for _, step := range []struct {
-		after    time.Duration
-		wantAsks int
-	}{
-		{after: 0, wantAsks: 1},
-		{after: whois.NICbrSkipTime - time.Second, wantAsks: 1},
-		{after: whois.NICbrSkipTime, wantAsks: 2},
-	} {
-		now = start.Add(step.after)
-
-		info, err := client.LookupASN(t.Context(), brazilASN)
-		if err != nil {
-			t.Fatalf("LookupASN after %v: %v", step.after, err)
-		}
-
-		if nicbrAsks != step.wantAsks {
-			t.Errorf("after %v NIC.br's server was asked %d times, want %d",
-				step.after, nicbrAsks, step.wantAsks)
-		}
-
-		if info.OrgName != lacnicBrazilOrgName {
-			t.Errorf("LookupASN after %v org name = %q, want LACNIC's %q",
-				step.after, info.OrgName, lacnicBrazilOrgName)
-		}
-	}
-}
-
-// TestLookupASNReportsARefusal looks up an AS IANA refers to RIPE when RIPE
-// refuses the query, with each of the error lines RIPE refuses with. The
-// error holds RIPE's line, so the log says why the lookup failed.
-func TestLookupASNReportsARefusal(t *testing.T) {
-	t.Parallel()
-
-	for _, line := range []string{ripeAccessDenied, ripeLimitReached} {
-		t.Run(line, func(t *testing.T) {
-			t.Parallel()
-
-			_, err := lookUp(t, map[string]string{
-				ianaQuery: ianaReferral,
-				ripeQuery: "% This is the RIPE Database query service.\n" +
-					line + "\n",
-			})
-			if err == nil || !strings.Contains(err.Error(), line) {
-				t.Errorf("LookupASN error = %v, want one holding %q",
-					err, line)
-			}
-		})
-	}
-}
-
-// TestLookupASNFailsWhenTheRIRGivesNoAnswer looks up an AS IANA refers to
-// RIPE when RIPE's server refuses the connection, and when it answers only
-// with an error. Neither is a lookup that found the AS.
-func TestLookupASNFailsWhenTheRIRGivesNoAnswer(t *testing.T) {
-	t.Parallel()
-
-	for name, answers := range map[string]map[string]string{
-		"connection refused": {ianaQuery: ianaReferral},
-		"only an error":      {ianaQuery: ianaReferral, ripeQuery: ripeRefusal},
-	} {
-		t.Run(name, func(t *testing.T) {
-			t.Parallel()
-
-			_, err := lookUp(t, answers)
-			if err == nil || errors.Is(err, whois.ErrNoReferral) {
-				t.Errorf("LookupASN error = %v, want one that is not "+
-					"ErrNoReferral", err)
-			}
-		})
-	}
-}
-
-// TestLookupASNReportsNoReferral looks up a private-use AS, which IANA
-// refers to no RIR.
-func TestLookupASNReportsNoReferral(t *testing.T) {
-	t.Parallel()
-
-	_, err := lookUp(t, map[string]string{ianaQuery: ianaPrivateUse})
-	if !errors.Is(err, whois.ErrNoReferral) {
-		t.Errorf("LookupASN error = %v, want ErrNoReferral", err)
-	}
-}
-
-// TestLookupASNFailsWhenIANAAnswersNoBlock looks up an AS when IANA answers
-// with neither a referral nor the AS's block. That is not IANA saying no
-// RIR holds the AS.
-func TestLookupASNFailsWhenIANAAnswersNoBlock(t *testing.T) {
-	t.Parallel()
-
-	_, err := lookUp(t, map[string]string{ianaQuery: ianaNoObject})
-	if err == nil || errors.Is(err, whois.ErrNoReferral) {
-		t.Errorf("LookupASN error = %v, want one that is not ErrNoReferral",
-			err)
-	}
-}
-
-// TestLookupASNFailsWhenTheRIRAnswerDoesNotEnd looks up an AS whose RIR
-// server follows the record with filler lines far past any real answer.
-// The client stops reading at its cap and fails instead of keeping them.
-func TestLookupASNFailsWhenTheRIRAnswerDoesNotEnd(t *testing.T) {
-	t.Parallel()
-
-	endless := ripeRecord +
-		strings.Repeat(fillerLine, endlessAnswerSize/len(fillerLine))
-
-	_, err := lookUp(t, map[string]string{
-		ianaQuery: ianaReferral,
-		ripeQuery: endless,
-	})
-	if err == nil || errors.Is(err, whois.ErrNoReferral) {
-		t.Errorf("LookupASN error = %v, want one that is not ErrNoReferral",
-			err)
-	}
-}
-
-// TestLookupASNFailsOnAReferralToAnotherServer looks up an AS when IANA's
-// answer refers it to a server none of the five RIRs run, with a port and
-// without. The client follows a referral only to an RIR's server, so it
-// fails instead of asking that server, which would answer with a record.
-func TestLookupASNFailsOnAReferralToAnotherServer(t *testing.T) {
-	t.Parallel()
-
-	// Each case maps the server on the "refer:" line to the address the
-	// client would dial to follow it. IANA's answer also names the AS's
-	// block, so a client that dropped the referral would report no RIR.
-	for refer, address := range map[string]string{
-		"whois.example.net:8080": "whois.example.net:8080",
-		"whois.example.net":      "whois.example.net:43",
-	} {
-		t.Run(refer, func(t *testing.T) {
-			t.Parallel()
-
-			_, err := lookUp(t, map[string]string{
-				ianaQuery: "refer:        " + refer + "\n\n" +
-					"as-block:     3154-3353\n",
-				address + " " + asQuery: ripeRecord,
-			})
-			if err == nil || errors.Is(err, whois.ErrNoReferral) {
-				t.Errorf("LookupASN error = %v, want one that is not "+
-					"ErrNoReferral", err)
-			}
-		})
-	}
-}

pkg/asinfo/asinfo.go

@@ -2,13 +2,10 @@ package asinfo
 
 import (
 	"bytes"
-	"cmp"
 	"compress/gzip"
 	_ "embed"
 	"encoding/json"
 	"io"
-	"slices"
-	"strings"
 	"sync"
 )
 
@@ -26,17 +23,6 @@ type Info struct {
 type Registry struct {
 	mu    sync.RWMutex
 	byASN map[int]*Info
-	// searchEntries holds every entry in AS number order, for Search.
-	searchEntries []searchEntry
-}
-
-// searchEntry is an entry with its handle and description in lower case. They
-// are put in lower case once, when the data loads, so that Search only
-// compares.
-type searchEntry struct {
-	info             *Info
-	lowerHandle      string
-	lowerDescription string
 }
 
 var (
@@ -73,21 +59,9 @@ func initRegistry() {
 		panic("failed to unmarshal embedded AS data: " + err.Error())
 	}
 
-	slices.SortFunc(asInfos, func(a, b Info) int {
-		return cmp.Compare(a.ASN, b.ASN)
-	})
-
-	defaultRegistry.searchEntries = make([]searchEntry, 0, len(asInfos))
-
 	for i := range asInfos {
 		info := &asInfos[i]
 		defaultRegistry.byASN[info.ASN] = info
-		defaultRegistry.searchEntries = append(defaultRegistry.searchEntries,
-			searchEntry{
-				info:             info,
-				lowerHandle:      strings.ToLower(info.Handle),
-				lowerDescription: strings.ToLower(info.Description),
-			})
 	}
 }
 
@@ -155,36 +129,36 @@ func All() []Info {
 	return result
 }
 
-// Search finds the AS entries whose handle or description contains query,
+// Search searches for AS entries where the handle or description contains the query
-// ignoring case. It returns the first limit of them in AS number order, and the
+// This is a simple case-sensitive substring search
-// number of entries that match. An empty query matches nothing.
+func Search(query string) []Info {
-func Search(query string, limit int) ([]Info, int) {
 	once.Do(initRegistry)
 
-	if query == "" {
-		return nil, 0
-	}
-
-	query = strings.ToLower(query)
-
 	defaultRegistry.mu.RLock()
 	defer defaultRegistry.mu.RUnlock()
 
-	var (
+	var results []Info
-		results []Info
-		total   int
-	)
 
-	for _, entry := range defaultRegistry.searchEntries {
+	for _, info := range defaultRegistry.byASN {
-		if strings.Contains(entry.lowerHandle, query) ||
+		if contains(info.Handle, query) || contains(info.Description, query) {
-			strings.Contains(entry.lowerDescription, query) {
+			results = append(results, *info)
-			if total < limit {
-				results = append(results, *entry.info)
-			}
-
-			total++
 		}
 	}
 
-	return results, total
+	return results
+}
+
+// contains is a simple substring search
+func contains(s, substr string) bool {
+	return len(substr) > 0 && len(s) >= len(substr) && containsImpl(s, substr)
+}
+
+func containsImpl(s, substr string) bool {
+	for i := 0; i <= len(s)-len(substr); i++ {
+		if s[i:i+len(substr)] == substr {
+			return true
+		}
+	}
+
+	return false
 }

pkg/asinfo/asinfo_test.go

@@ -1,8 +1,6 @@
 package asinfo_test
 
 import (
-	"cmp"
-	"slices"
 	"strings"
 	"testing"
 
@@ -22,7 +20,7 @@ func TestGet(t *testing.T) {
 			name:     "AS1 Level 3",
 			asn:      1,
 			wantOK:   true,
-			wantDesc: "Level 3 Parent LLC",
+			wantDesc: "Level 3 Parent, LLC",
 		},
 		{
 			name:     "AS15169 Google",
@@ -36,10 +34,10 @@ func TestGet(t *testing.T) {
 			wantOK: false,
 		},
 		{
-			name:     "AS1251 with accented letters",
+			name:     "AS0 IANA Reserved",
-			asn:      1251,
+			asn:      0,
 			wantOK:   true,
-			wantDesc: "FUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO SÃO PAULO",
+			wantDesc: "Internet Assigned Numbers Authority",
 		},
 	}
 
@@ -69,7 +67,7 @@ func TestGetDescription(t *testing.T) {
 		asn  int
 		want string
 	}{
-		{1, "Level 3 Parent LLC"},
+		{1, "Level 3 Parent, LLC"},
 		{999999999, ""},
 	}
 
@@ -92,7 +90,8 @@ func TestGetHandle(t *testing.T) {
 		asn  int
 		want string
 	}{
-		{1, "LVLT-1"},
+		{1, "LVLT"},
+		{0, "IANA-RSVD"},
 		{999999999, ""},
 	}
 
@@ -142,9 +141,9 @@ func TestSearch(t *testing.T) {
 			wantMin: 10,
 		},
 		{
-			name:    "Search for São Paulo",
+			name:    "Empty search",
-			query:   "São Paulo",
+			query:   "",
-			wantMin: 1,
+			wantMin: 0,
 		},
 		{
 			name:    "Non-existent org",
@@ -157,115 +156,26 @@ func TestSearch(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			t.Parallel()
 
-			results, _ := asinfo.Search(tt.query, asinfo.Total())
+			results := asinfo.Search(tt.query)
 			if len(results) < tt.wantMin {
 				t.Errorf("Search(%q) returned %d results, want at least %d",
 					tt.query, len(results), tt.wantMin)
 			}
 
-			// Verify all results contain the query, ignoring case
+			// Verify all results contain the query
-			query := strings.ToLower(tt.query)
+			if tt.query != "" {
 				for _, r := range results {
-				if !strings.Contains(strings.ToLower(r.Handle), query) &&
+					if !strings.Contains(r.Handle, tt.query) &&
-					!strings.Contains(strings.ToLower(r.Description), query) {
+						!strings.Contains(r.Description, tt.query) {
 						t.Errorf("Search result ASN %d doesn't contain query %q",
 							r.ASN, tt.query)
 					}
 				}
+			}
 		})
 	}
 }
 
-// TestSearchEmptyQueryMatchesNothing expects an empty query to match no AS,
-// although every handle contains the empty string.
-func TestSearchEmptyQueryMatchesNothing(t *testing.T) {
-	t.Parallel()
-
-	results, total := asinfo.Search("", asinfo.Total())
-	if len(results) != 0 || total != 0 {
-		t.Errorf(`Search("") returned %d results and a total of %d, want none`,
-			len(results), total)
-	}
-}
-
-// TestSearchIgnoresCase searches for Cloudflare's handle in lower case and its
-// description in upper case, and expects both searches to find AS13335.
-func TestSearchIgnoresCase(t *testing.T) {
-	t.Parallel()
-
-	const cloudflareASN = 13335 // handle CLOUDFLARENET, "Cloudflare Inc."
-
-	isCloudflare := func(info asinfo.Info) bool {
-		return info.ASN == cloudflareASN
-	}
-
-	for _, query := range []string{"cloudflarenet", "CLOUDFLARE INC."} {
-		results, _ := asinfo.Search(query, asinfo.Total())
-		if !slices.ContainsFunc(results, isCloudflare) {
-			t.Errorf("Search(%q) did not find AS%d", query, cloudflareASN)
-		}
-	}
-}
-
-// TestSearchOrdersByASN expects the matches of a search with many of them to
-// come in AS number order.
-func TestSearchOrdersByASN(t *testing.T) {
-	t.Parallel()
-
-	const query = "University"
-
-	results, _ := asinfo.Search(query, asinfo.Total())
-
-	for i := 1; i < len(results); i++ {
-		if results[i-1].ASN >= results[i].ASN {
-			t.Fatalf("Search(%q): AS%d comes after AS%d",
-				query, results[i].ASN, results[i-1].ASN)
-		}
-	}
-}
-
-// TestSearchStopsAtLimit searches for a word that matches more entries than
-// the limit. It expects exactly limit entries, the first matches in AS number
-// order, and the number of all matches, counted here from All.
-func TestSearchStopsAtLimit(t *testing.T) {
-	t.Parallel()
-
-	const (
-		query = "university"
-		limit = 10
-	)
-
-	var matches []asinfo.Info
-
-	for _, info := range asinfo.All() {
-		if strings.Contains(strings.ToLower(info.Handle), query) ||
-			strings.Contains(strings.ToLower(info.Description), query) {
-			matches = append(matches, info)
-		}
-	}
-
-	if len(matches) <= limit {
-		t.Fatalf("%q matches %d entries; the test needs more than %d",
-			query, len(matches), limit)
-	}
-
-	slices.SortFunc(matches, func(a, b asinfo.Info) int {
-		return cmp.Compare(a.ASN, b.ASN)
-	})
-
-	results, total := asinfo.Search(query, limit)
-
-	if total != len(matches) {
-		t.Errorf("Search(%q, %d): total %d, want %d",
-			query, limit, total, len(matches))
-	}
-
-	if !slices.Equal(results, matches[:limit]) {
-		t.Errorf("Search(%q, %d) = %v, want %v",
-			query, limit, results, matches[:limit])
-	}
-}
-
 func TestDataIntegrity(t *testing.T) {
 	t.Parallel()
 
@@ -305,13 +215,11 @@ func BenchmarkGet(b *testing.B) {
 }
 
 func BenchmarkSearch(b *testing.B) {
-	const limit = 50
-
 	queries := []string{"Google", "Amazon", "Microsoft", "University"}
 
 	b.ResetTimer()
 
 	for i := range b.N {
-		asinfo.Search(queries[i%len(queries)], limit)
+		asinfo.Search(queries[i%len(queries)])
 	}
 }

pkg/asinfo/doc.go

@@ -2,7 +2,7 @@
 Package asinfo provides information about Autonomous Systems (AS).
 
 The package embeds a comprehensive database of AS numbers, handles, and descriptions
-sourced from https://github.com/ipverse/as-metadata. The data is embedded at compile
+sourced from https://github.com/ipverse/asn-info. The data is embedded at compile
 time and provides fast, offline lookups.
 
 Basic usage:
@@ -16,10 +16,8 @@ Basic usage:
 	// Get just the description
 	desc := asinfo.GetDescription(15169) // "Google LLC"
 
-	// Search the handles and descriptions, ignoring case, for the first 10
+	// Search for AS entries
-	// matches in AS number order and the number of all matches
+	results := asinfo.Search("University")
-	results, total := asinfo.Search("university", 10)
-	fmt.Printf("%d matches\n", total)
 	for _, as := range results {
 		fmt.Printf("AS%d: %s\n", as.ASN, as.Description)
 	}

pkg/asinfo/example_test.go

@@ -17,20 +17,13 @@ func ExampleGet() {
 func ExampleGetDescription() {
 	desc := asinfo.GetDescription(13335)
 	fmt.Println(desc)
-	// Output: Cloudflare Inc.
+	// Output: Cloudflare, Inc.
 }
 
 func ExampleSearch() {
-	const limit = 10
+	results := asinfo.Search("MIT-GATEWAY")
-
-	results, total := asinfo.Search("mit-gateway", limit)
-
-	fmt.Printf("%d of %d matches\n", len(results), total)
-
 	for _, info := range results {
 		fmt.Printf("AS%d: %s - %s\n", info.ASN, info.Handle, info.Description)
 	}
-	// Output:
+	// Output: AS3: MIT-GATEWAYS - Massachusetts Institute of Technology
-	// 1 of 1 matches
-	// AS3: MIT-GATEWAYS - Massachusetts Institute of Technology
 }

script/asupdate

@@ -1,34 +0,0 @@
-#!/bin/sh
-# script/asupdate: rebuild the AS list built into RouteWatch from
-# as.csv with cmd/asinfo-gen, which fails when it finds no AS. The new
-# list replaces the old one only when the generator succeeds. Otherwise
-# the old list stays as it was and the script exits non-zero.
-set -eu
-
-ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
-
-AS_DATA="pkg/asinfo/asdata.json.gz"
-# Next to AS_DATA, so the final mv is a rename on one file system.
-AS_DATA_TMP="$AS_DATA.tmp"
-
-main() {
-    cd "$ROOT"
-
-    json="$(mktemp)"
-    trap 'rm -f "$json" "$AS_DATA_TMP"' EXIT
-    # A signal exits through the EXIT trap too.
-    trap 'exit 1' HUP INT TERM
-
-    echo "Updating AS info data..."
-    if ! go run ./cmd/asinfo-gen >"$json"; then
-        echo "asupdate: the generator failed; $AS_DATA is unchanged" >&2
-        exit 1
-    fi
-
-    # -n leaves the time out of the file, so an unchanged list gives the
-    # same bytes.
-    gzip -n -c <"$json" >"$AS_DATA_TMP"
-    mv "$AS_DATA_TMP" "$AS_DATA"
-}
-
-main "$@"