Labelled critical: once routewatch has run about a day on the live feed, looking up any IPv6 address on /ip/ or /api/v1/ip/ takes longer than the 30-second request timeout and fails for everyone.
Found by the repo-manager on 2026-10-03 while reading the open items of the memory analysis on #3 (section 2), which listed this as out of scope for memory and did not file it.
What goes wrong
getIPv6Info in internal/database/database.go (behind /ip/{addr} and /api/v1/ip/{ip}) selects every row of live_routes_v6 joined with asns, with DISTINCT and no WHERE, and checks each prefix in Go. Measured on 2026-09-21: 22.8 s per lookup at 1.0 M IPv6 live routes. The test database for #30 held 3.5 M, at which a lookup would take over a minute (derived, not measured); the request timeout is 30 s (internal/server/routes.go), so the lookup fails.
getIPv4Info uses WHERE ip_start <= ? AND ip_end >= ? on the (ip_start, ip_end) index, which reads every entry with ip_start at or below the address (about half the index on average) and then sorts by mask length. At 18.6 M IPv4 live routes that is likely seconds per lookup too. Not measured.
Plan
Find the most specific live route with point lookups on the existing prefix index: for each mask length from the longest (32 or 128) down to 0, form the address's network prefix at that length in the text form the routes are stored in, and look it up with WHERE prefix = ?; the first length with a live route wins. At most 33 or 129 indexed lookups, whatever the table size. One plain function serves both families; the IPv4 range query goes.
The text form must match: check how the write path stores prefixes. If they are stored as received and the feed can send a non-canonical form, store them in Go's net.IPNet string form on write.
If nothing else reads ip_start and ip_end afterwards, the columns, their index and the code that fills them go too (pre-1.0, schema changed in place).
GetASInfoForIP and GetASInfoForIPContext run the same whole-table IPv6 query, and nothing outside the test mock calls them: remove them from the store interface and the mock instead of fixing them.
The peer count and first-seen queries after the match stay; they already use the prefix index.
Definition of done
IPv4 and IPv6 lookups find the most specific live route with a bounded number of indexed lookups, independent of the number of live routes.
Tests, for both families: the most specific of several nested live prefixes wins; an address with no covering route returns the no-route error; a prefix whose last route was withdrawn is not matched.
A check on a database of about 2 M live routes, several hundred thousand of them IPv6, built within a 2-3 GiB memory budget: lookups for both families answer well under a second.
make check passes.
Model: opus-5-5
Labelled critical: once routewatch has run about a day on the live feed, looking up any IPv6 address on `/ip/` or `/api/v1/ip/` takes longer than the 30-second request timeout and fails for everyone.
Found by the repo-manager on 2026-10-03 while reading the open items of the memory analysis on https://git.eeqj.de/sneak/routewatch/issues/3 (section 2), which listed this as out of scope for memory and did not file it.
## What goes wrong
- `getIPv6Info` in `internal/database/database.go` (behind `/ip/{addr}` and `/api/v1/ip/{ip}`) selects every row of `live_routes_v6` joined with `asns`, with `DISTINCT` and no `WHERE`, and checks each prefix in Go. Measured on 2026-09-21: 22.8 s per lookup at 1.0 M IPv6 live routes. The test database for https://git.eeqj.de/sneak/routewatch/issues/30 held 3.5 M, at which a lookup would take over a minute (derived, not measured); the request timeout is 30 s (`internal/server/routes.go`), so the lookup fails.
- `getIPv4Info` uses `WHERE ip_start <= ? AND ip_end >= ?` on the `(ip_start, ip_end)` index, which reads every entry with `ip_start` at or below the address (about half the index on average) and then sorts by mask length. At 18.6 M IPv4 live routes that is likely seconds per lookup too. Not measured.
## Plan
- Find the most specific live route with point lookups on the existing `prefix` index: for each mask length from the longest (32 or 128) down to 0, form the address's network prefix at that length in the text form the routes are stored in, and look it up with `WHERE prefix = ?`; the first length with a live route wins. At most 33 or 129 indexed lookups, whatever the table size. One plain function serves both families; the IPv4 range query goes.
- The text form must match: check how the write path stores prefixes. If they are stored as received and the feed can send a non-canonical form, store them in Go's `net.IPNet` string form on write.
- If nothing else reads `ip_start` and `ip_end` afterwards, the columns, their index and the code that fills them go too (pre-1.0, schema changed in place).
- `GetASInfoForIP` and `GetASInfoForIPContext` run the same whole-table IPv6 query, and nothing outside the test mock calls them: remove them from the store interface and the mock instead of fixing them.
- The peer count and first-seen queries after the match stay; they already use the prefix index.
## Definition of done
- IPv4 and IPv6 lookups find the most specific live route with a bounded number of indexed lookups, independent of the number of live routes.
- Tests, for both families: the most specific of several nested live prefixes wins; an address with no covering route returns the no-route error; a prefix whose last route was withdrawn is not matched.
- A check on a database of about 2 M live routes, several hundred thousand of them IPv6, built within a 2-3 GiB memory budget: lookups for both families answer well under a second.
- `make check` passes.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Labelled critical: once routewatch has run about a day on the live feed, looking up any IPv6 address on
/ip/or/api/v1/ip/takes longer than the 30-second request timeout and fails for everyone.Found by the repo-manager on 2026-10-03 while reading the open items of the memory analysis on #3 (section 2), which listed this as out of scope for memory and did not file it.
What goes wrong
getIPv6Infoininternal/database/database.go(behind/ip/{addr}and/api/v1/ip/{ip}) selects every row oflive_routes_v6joined withasns, withDISTINCTand noWHERE, and checks each prefix in Go. Measured on 2026-09-21: 22.8 s per lookup at 1.0 M IPv6 live routes. The test database for #30 held 3.5 M, at which a lookup would take over a minute (derived, not measured); the request timeout is 30 s (internal/server/routes.go), so the lookup fails.getIPv4InfousesWHERE ip_start <= ? AND ip_end >= ?on the(ip_start, ip_end)index, which reads every entry withip_startat or below the address (about half the index on average) and then sorts by mask length. At 18.6 M IPv4 live routes that is likely seconds per lookup too. Not measured.Plan
prefixindex: for each mask length from the longest (32 or 128) down to 0, form the address's network prefix at that length in the text form the routes are stored in, and look it up withWHERE prefix = ?; the first length with a live route wins. At most 33 or 129 indexed lookups, whatever the table size. One plain function serves both families; the IPv4 range query goes.net.IPNetstring form on write.ip_startandip_endafterwards, the columns, their index and the code that fills them go too (pre-1.0, schema changed in place).GetASInfoForIPandGetASInfoForIPContextrun the same whole-table IPv6 query, and nothing outside the test mock calls them: remove them from the store interface and the mock instead of fixing them.Definition of done
make checkpasses.Model: opus-5-5