Every result of /ip/, /ip/{addr} and /api/v1/ip/{ip} now has every
field. A field without a value is null instead of left out, so an
address with no live route has all its route and AS fields set to null.
ptr is an empty list when there are no PTR names.
Two new fields, num_origins and origin_asns, give how many origin ASes
peers carry the netblock with, and which. The routing table's prefix
summary now lists them. asn still names the origin that most peers
carry.
first_seen is null, no longer the zero time, when the database has not
stored the netblock yet.
Model: opus-5-5
The home page gets cards for the newest and oldest kept route, the newest
and oldest updated AS, the AS with the most prefixes, the ASes with a
single prefix and fifteen well-known networks. Two more cards count the
bogon announcements and the prefixes with several origin ASes. They link
to the new pages /bogons and /multiple-origins, which list up to 1,000
entries each.
The new package internal/bogon tells reserved prefixes and AS numbers,
from the IANA special-purpose and address space registries. Table.Overview
walks the table and lets the read lock go every 10,000 prefixes. The
server works the cards out once a minute and serves that copy. A new index
on asns(last_seen) finds the newest and oldest updated AS.
Model: opus-5-5
The routing table in memory now holds every route, so the two per-peer route
tables go, with every write, delete, count and read that served them. The
prefix handler applies each update to the table, writes the table's changes
every second, and stores each new prefix with its first-seen time. The peer
delete, its retry on the next flush and the updates held for it go too. The
types and errors of the server's answers move to internal/server.
The image's memory settings fit a full table of about 1 GB: GOMEMLIMIT of
2.5 GiB, a 512 MiB page cache for the writing connection, and SQLite heap
limits of 768 MiB and 1 GiB. The README describes the new table, its
database copy and the memory budget.
Model: opus-5-5
The IP lookups, the AS and prefix pages and their API answers, the prefix-length pages and the stats read routes only from the routing table the prefix handler keeps. fx reads the table at startup and gives it to both. AS names, WHOIS details and first-seen times still come from the database.
The IP lookup names the origin AS most peers carry, with the time of the newest kept route. The prefix answers list peers whose routes are not kept without a path or time. The stats gain origins_without_kept_route. They no longer read route times from the per-peer tables, so the two helpers that did are gone.
Model: opus-5-5
A new package, internal/blocklist, downloads seven public lists on a
schedule: DShield, Feodo Tracker, FireHOL abusers and level 3, Spamhaus
DROP for IPv4 and IPv6, and the Tor exit list. It keeps the last good
copy of each in memory and on disk. A download sends back the
Last-Modified and ETag the publisher gave for the copy, also kept on
disk.
Every IP lookup result carries blocklists. An address with no live route
now gets a result too, without route fields, so the lookup no longer
answers 404. A successful answer carries Cache-Control, private for the
caller's own address. The address sets come from go4.org/netipx.
Model: opus-5-5
The home page gets an IP information API card. It shows the curl
command that looks up the visitor's own address at /ip/{addr}. The page
then makes that same request and shows the JSON answer, pretty-printed.
An error answer, such as a 404 for an address with no route, is shown
the same way. A request that fails shows an error line.
The server gives the page the visitor's address, found as /ip/ finds
it, so a trusted proxy's X-Forwarded-For is honoured. The command takes
its origin from the browser, so it names the host the visitor used.
Model: opus-5-5
The AS page sorted its IPv4 and IPv6 prefix lists by network address,
then by mask length. It now sorts them by mask length, shortest first,
and by network address within one mask length, so a /16 comes before a
/24. The sort function is renamed from sortRoutesByNetwork to
sortRoutesByMaskLength to say so.
The handler of /api/v1/as/{asn} does not sort its lists, so it is
unchanged.
Model: opus-5-5
The status page showed route churn per second as a one-minute moving
average, so a quiet feed gave a fraction such as 0.1. The average was
correct.
The metrics tracker now also counts prefix announcements and
withdrawals, from all peers, in each clock minute. /api/v1/stats
reports the count for the last full minute as route_churn_per_min in
stream, in place of route_churn_per_sec. The status page shows it as
"Route Churn/min".
The status and home pages round messages and updates per second to
whole numbers.
Model: opus-5-5
The AS Name Search card on the home page said the search was coming soon.
It now asks the new GET /api/v1/search/as?q=<text> and lists the matches
as links to their AS pages. asinfo.Search ignores case, returns the first
matches up to a limit in AS number order, and counts all matches. An
empty q answers 400.
When the database has no record of an AS, its page and /api/v1/as/{asn}
take its handle and description from asinfo and still list its prefixes.
They answer 404 only for an AS with no record, no asinfo entry and no
prefix. AS 0 means no known origin: it always answers 404, and the search
never lists it.
Model: opus-5-5
An IP lookup took any number of addresses and host names, gave each PTR
lookup until the request's 30-second limit, and looked up and named
every address a host name resolved to. It now answers 400 for more than
10 entries, or more than 10 addresses once the names are resolved,
before reading the database. Each PTR lookup gets 2 seconds; one that
runs out leaves the address's result without PTR names. Of a host
name's addresses only public ones are kept: global unicast and not
private. A name with none is reported as having no public address. The
tests give the server a fake resolver.
Model: opus-5-5
A failed database read for an address was listed in the answer's errors
as if the address had no route, so the lookup answered 404, or 200 next
to a routed address. Only a missing route or an invalid address is
listed now; any other read error is logged and answers 500 with the JSON
error body, as the AS and prefix JSON routes do.
Model: opus-5-5
Every answer carries the six security headers REPO_POLICIES.md lists.
The Content-Security-Policy allows inline content only with a nonce made
for that answer; the pages put it on their style and script blocks and
drop their four style attributes.
The request log and /ip/ take a forwarded address only from a proxy
listed in TRUSTED_PROXIES, which stops the start when set but invalid:
the last X-Forwarded-For entry that is not a trusted proxy. X-Real-IP
and True-Client-IP are no longer read.
The JSON routes and the health check send fixed messages in place of
database, parser and resolver errors, and log the detail.
Model: opus-5-5
Takes .golangci.yml, .dockerignore, .gitignore, .editorconfig, the
workflow and REPO_POLICIES.md from sneak/prompts 0653579, with this
repository's own entries after the shared content. Lint (golangci-lint
v2.14.0) and test are Dockerfile phases that script/lint and script/test
build alone with a new CACHEBUST value, keeping Go's and golangci-lint's
caches; the image build depends on both. make fmt and make fmt-check also
format the markdown with prettier, pinned in yarn.lock. The agent
guidance moves to AGENTS.md. The Go code is fixed for what the new
configuration finds, and the AS and prefix detail reads now return the
rows' error after their loops.
Model: opus-5-5
A request past the server's 30-second limit answered 408, which tells the
client it was too slow sending its request, and /api/v1/stats answered 500
when its 4-second read ran out. The limit is now chi's middleware.Timeout,
which answers 504 and never writes from a second goroutine while the
handler is writing.
Right after each database read or host name lookup, whatever came back, a
handler asks deadlinePassed whether its time limit has passed and, if so,
answers 504, with the JSON error body on JSON routes: a cut-off read can
return part of its rows without an error. The health check is unchanged.
Model: opus-5-5
The prefix page looked up each origin AS's details only to show its handle and description, and that lookup reads every live route the AS originates. `GetPrefixDetailsContext` already selected the handle and description and dropped them; it now returns them keyed by AS number, and the page uses them. The origin list now also shows the description after the handle, which it never rendered. The JSON prefix answer is unchanged. The test serves the page from a store whose AS lookup always fails.
Model: opus-5-5
`/prefix/{prefix}/{len}` and `/api/v1/prefix/{prefix}/{len}` passed the URL text to the lookup as typed: a prefix that does not parse answered 500, and a valid prefix written in another form (`2001:DB8:0:0::/32`, `192.0.2.1/24`) answered 404. Both now parse the prefix with `net/netip`, mask it and look up the form the tables store; one that does not parse answers 400. The answer shows the stored form of the prefix. The test drives both pages through the router against a real database.
Model: opus-5-5
Realtime in-memory counters seeded at startup and adjusted on every insert, update and delete; no periodic recompute. Independent review passed: #29 (comment)
model: claude-opus-4-8 (implementation and review); merged by claude-fable-5
The stats handlers ran the database query in a goroutine that sent on an
unbuffered channel. When the 4s request timeout won, nothing received and
the goroutine blocked forever; the status page polls every 2s, so once the
query exceeds the timeout every poll leaked one goroutine. Give both
channels capacity 1 so the send always completes.
The streamer started two ticker goroutines per connection that exited only
with the streamer's lifetime context, leaking two on every reconnect. Scope
them to a per-connection context cancelled when the stream call returns.
Tests force the stats timeout repeatedly and drive many reconnects, then
assert the goroutine count settles back to its starting value. The streamer
gains an internal endpoint field so a test can point it at a local server.
Model: opus-4-8
- Log slow requests (>1s) at WARNING level with slow=true flag
- Log request timeouts at WARNING level in TimeoutMiddleware
- Replace heavy GetStatsContext with lightweight Ping in health check
- Add Ping method to database interface (SELECT 1)
Display oldest and newest route timestamps in the Routing Table card
on the /status page. Timestamps are shown as relative times (e.g.,
"5m ago", "2h 30m ago").
Changes:
- Add OldestRoute and NewestRoute fields to database.Stats
- Query MIN/MAX of last_updated across live_routes_v4 and v6 tables
- Include timestamps in both /status.json and /api/v1/stats responses
- Add formatRelativeTime JavaScript function for display
- Create new home page (/) with overview stats, ASN lookup,
AS name search, and IP address lookup with JSON display
- Add responsive navbar to all pages with app branding
- Navbar shows "routewatch by @sneak" with link to author
- Status page accessible via navbar link
- Remove redirect from / to /status, serve home page instead
Change route from wildcard /prefix/* to explicit /prefix/{prefix}/{len}
to properly handle URL-encoded IPv6 addresses with CIDR notation.
- Separate prefix and length into individual path parameters
- Add prefixURL template function for generating correct links
- Remove url.QueryUnescape from handlers (chi handles decoding)
- Add GC statistics (run count, total/last pause, heap usage)
- Add BGP peer count tracking from RIS Live OPEN/NOTIFICATION messages
- Add route churn rate metric (announcements + withdrawals per second)
- Add announcement and withdrawal counters
- Add footer with attribution, license, and git revision
- Embed git revision at build time via ldflags
- Update HTML template to display all new metrics
- Track reconnection count in metrics tracker
- Display connection duration under Stream Statistics
- Display reconnect count since app startup
- Update both JSON API and HTML status page
- Add health check endpoint at /.well-known/healthcheck.json that
verifies database and RIS Live connectivity, returns 200/503
- Fix panic in streamer when encountering unknown RIS message types
by logging a warning and continuing instead of crashing
- Add DBMaintainer for periodic database maintenance:
- VACUUM every 6 hours to reclaim space
- ANALYZE every hour to update query statistics
- Graceful shutdown support
- Add Vacuum() and Analyze() methods to database interface
- Add WHOIS Fetcher card showing fresh/stale/never-fetched ASN counts
- Display hourly success/error counts and current fetch interval
- Increase max WHOIS rate to 1/sec (down from 10 sec minimum)
- Select random stale ASN instead of oldest for better distribution
- Add index on whois_updated_at for query performance
- Track success/error timestamps for hourly stats
- Add GetWHOISStats database method for freshness statistics
- Always return consistent JSON structure with query and results array
- Add PTR field to IPInfo for reverse DNS records
- Support comma-separated IPs and hostnames in single query
- Do PTR lookup for all IPs (direct, resolved from hostname, or listed)
- Remove trailing dots from PTR records
- Accept hostnames in addition to IP addresses for /ip endpoints
- Resolve A and AAAA records for hostnames
- Return list of results with info for each resolved IP
- Include hostname in response when resolving hostnames
- Report per-IP errors while still returning successful lookups
- Add /ip and /ip/{addr} JSON endpoints returning comprehensive IP info
- Include ASN, netblock, country code, org name, abuse contact, RIR data
- Extend ASN schema with WHOIS fields (country, org, abuse contact, etc)
- Create background WHOIS fetcher for rate-limited ASN info updates
- Store raw WHOIS responses for debugging and data preservation
- Queue on-demand WHOIS lookups when stale data is requested
- Refactor handleIPInfo to serve all IP endpoints consistently
The stream stats were showing decompressed data sizes, not actual wire
bandwidth. This change adds wire byte tracking by disabling automatic
gzip decompression in the HTTP client and wrapping the response body
with a counting reader before decompression. Both wire (compressed) and
decompressed bytes are now tracked and exposed in the API responses.
Expand the documentation comment for CLIEntry to provide more context
about what the function does, including its use of the fx dependency
injection framework, signal handling, and blocking behavior.
The prefix length links for IPv6 prefixes were incorrectly pointing to
/prefixlength/<len> which would show IPv4 prefixes. Added a new route
/prefixlength6/<len> specifically for IPv6 prefixes and updated the
template to use the correct URL based on whether displaying IPv4 or IPv6
prefix distributions.
Also refactored handlePrefixLength to explicitly handle only IPv4 prefixes
and created handlePrefixLength6 for IPv6 prefixes, removing the ambiguous
auto-detection based on mask length value.
- HTTP request timeout: 2s -> 8s
- Stats collection context timeout: 1s -> 4s
- HTTP read header timeout: 10s -> 40s
This should prevent timeout errors when the database is under load
or when complex queries take longer than expected (slow query
threshold is 50ms).
- Added GetASPeers method to database to fetch all peering relationships
- Updated AS detail handler to fetch and pass peers to template
- Added peers section to AS detail page showing all peer ASNs with their info
- Added peer count to the info cards at the top of the page
- Shows handle, description, and first/last seen dates for each peer
- Track the maximum queue length seen for each handler
- Display high water marks on the status page with percentage
- Helps identify which handlers are experiencing queue pressure
- Remove UUID primary keys from ASNs table, use ASN number as primary key
- Update announcements table to reference ASN numbers directly
- Rename asns.number column to asns.asn for consistency
- Add prefix tracking to PrefixHandler to populate prefixes_v4/v6 tables
- Add UpdatePrefixesBatch method for efficient batch updates
- Update all database methods and models to use new schema
- Fix all references in code to use ASN field instead of Number
- Update test mocks to match new interfaces
- Update status page JavaScript to reset all fields to '-' on error
- Fix status page to not show 'Connected' when API returns error
- Update remaining database methods to use new live_routes_v4/v6 tables
- Fix GetStatsContext to count routes from both IPv4 and IPv6 tables
- Fix UpsertLiveRoute to insert into correct table based on IP version
- Fix DeleteLiveRoute to determine table from prefix IP version
- Add urlEncode template function to properly encode prefix URLs
- Move prefix_length.html to embedded templates with function map
- Prevents broken links for prefixes containing slashes
- Add thread-safe header wrapper in timeoutWriter
- Check context cancellation before writing responses in handlers
- Protect header access after timeout with mutex
- Prevents race condition when requests timeout while handlers are still running
- Add context-aware versions of all read operations in the database
- Update handlers to use context from HTTP requests
- Allows database queries to be cancelled when HTTP requests timeout
- Prevents database operations from continuing after client disconnects
- Remove duplicate http.Error call when context times out
- The timeout middleware already handles writing the response
- Prevents "concurrent write to websocket connection" panic
- Fix GetPrefixDistribution to count unique prefixes using COUNT(DISTINCT prefix) instead of COUNT(*)
- Add /prefixlength/<length> route showing random sample of 500 prefixes
- Make prefix counts on status page clickable links to prefix length pages
- Add GetRandomPrefixesByLength database method
- Create prefix_length.html template with sortable table
- Show prefix age and origin AS with descriptions
- Change AS path from descriptions to handles (short names)
- Make each AS in the path a clickable link to /as/<asn>
- Add font-weight to AS links in path for better visibility
- Prevent word wrapping on all table columns except AS path
- Remove unused maxASDescriptionLength constant
- Display AS descriptions alongside AS numbers in format: Description (ASN)
- Truncate descriptions longer than 20 characters with ellipsis
- Increase container max width to 1600px for better display
- Enable word wrapping for AS path cells to handle long paths
- Update mobile responsive styles for AS path display
- Fix GetASDetails to properly handle timestamp from MAX(last_updated)
- Parse timestamp string from SQLite aggregate function result
- Add natural sorting of prefixes by IP address in AS detail view
- Sort IPv4 and IPv6 prefixes separately by network address
- Remove SQL ORDER BY since we're sorting in Go
- This fixes the issue where AS detail pages showed no prefixes
- Replace slash-to-dash conversion with proper URL encoding
- Update handlePrefixDetail and handlePrefixDetailJSON to URL decode prefix parameter
- Update handleIPRedirect to URL encode the prefix in the redirect
- Add urlEncode template function for use in templates
- Update AS detail template to URL encode prefix links
- This properly handles the slash in CIDR notation (e.g., /prefix/192.168.1.0%2F24)
- Implement handleIPRedirect handler that looks up the prefix containing an IP
- Add /ip/{ip} route to routes.go
- Reuse existing GetASInfoForIP database method which returns prefix info
- Redirect to /prefix/<prefix> page with HTTP 303 See Other status
- Handle invalid IPs (400) and IPs with no route (404)
clawbot
sneak