RouteWatch

RouteWatch is an MIT-licensed Go daemon by @sneak that monitors the BGP routing table in real time: it streams BGP UPDATE messages from the RIPE RIS Live service, maintains a live routing table in SQLite, and provides HTTP APIs for querying routing information.

Features

• Real-time streaming of BGP updates from RIPE RIS Live
• Maintains live IPv4 and IPv6 routing tables
• Tracks AS peering relationships
• HTTP API for IP-to-AS lookups, prefix details, and AS information
• Automatic reconnection with exponential backoff
• Batched database writes for high performance
• Backpressure handling to prevent memory exhaustion

Installation

go build -o routewatch ./cmd/routewatch

Usage

# Run the daemon (listens on port 8080 by default)
./routewatch

# Set custom port
PORT=3000 ./routewatch

# Enable debug logging
DEBUG=routewatch ./routewatch

HTTP Endpoints

Web Interface

• GET / - Redirects to /status
• GET /status - HTML status dashboard
• GET /status.json - JSON statistics
• GET /as/{asn} - AS detail page (HTML)
• GET /prefix/{prefix} - Prefix detail page (HTML)
• GET /prefixlength/{length} - IPv4 prefixes by mask length
• GET /prefixlength6/{length} - IPv6 prefixes by mask length
• GET /ip/{ip} - Redirects to prefix containing the IP

API v1

• GET /api/v1/stats - Detailed statistics with handler metrics
• GET /api/v1/ip/{ip} - Look up AS information for an IP address
• GET /api/v1/as/{asn} - Get prefixes announced by an AS
• GET /api/v1/prefix/{prefix} - Get routes for a specific prefix

An origin AS of 0 (origin_asn in a route, asn in an IP lookup, the key 0 in a prefix's origins) means the route has no known origin: its AS path ends in an AS set of several members, any of which may have originated it. The pages show such a route as having no known origin.

Code Structure

routewatch/
├── cmd/
│   ├── routewatch/         # Main daemon entry point
│   ├── asinfo-gen/         # Utility to generate AS info data
│   └── streamdumper/       # Debug utility for raw stream output
├── internal/
│   ├── routewatch/         # Core application logic
│   ├── server/             # HTTP server and handlers
│   ├── database/           # SQLite storage layer
│   ├── streamer/           # RIPE RIS Live client
│   ├── ristypes/           # BGP message data structures
│   ├── logger/             # Structured logging wrapper
│   ├── metrics/            # Performance metrics tracking
│   ├── config/             # Configuration management
│   └── templates/          # HTML templates
└── pkg/
    └── asinfo/             # AS information lookup (public API)

Architecture Overview

Component Relationships

┌─────────────────────────────────────────────────────────────────┐
│                         RouteWatch                               │
│  (internal/routewatch/app.go - main orchestrator)               │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐       │
│  │   Streamer   │───▶│   Handlers   │───▶│   Database   │       │
│  │              │    │              │    │              │       │
│  │ RIS Live     │    │ - ASHandler  │    │ SQLite with  │       │
│  │ WebSocket    │    │ - PeerHandler│    │ WAL mode     │       │
│  │ client       │    │ - PrefixHdlr │    │              │       │
│  │              │    │ - PeeringHdlr│    │ Tables:      │       │
│  └──────────────┘    └──────────────┘    │ - asns       │       │
│                                          │ - prefixes   │       │
│  ┌──────────────┐    ┌──────────────┐    │ - live_routes│       │
│  │    Server    │───▶│   Handlers   │───▶│ - peerings   │       │
│  │              │    │              │    │ - bgp_peers  │       │
│  │ Chi router   │    │ Status, API  │    └──────────────┘       │
│  │ port 8080    │    │ AS, Prefix   │                           │
│  └──────────────┘    └──────────────┘                           │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Execution Flow

1. Startup (cmd/routewatch/main.go → internal/routewatch/cli.go)

   • Uber fx dependency injection initializes all components
   • Signal handlers registered for graceful shutdown

2. Initialization (internal/routewatch/app.go)

   • Database created with SQLite schema (WAL mode, 3GB cache)
   • Message handlers registered with the streamer
   • HTTP server started on configured port

3. Message Processing Pipeline

   RIS Live Stream → JSON Parser → Message Dispatcher → Handler Queues → Batch Writers → SQLite
   

   • Streamer connects to ris-live.ripe.net via HTTP
   • Parses BGP UPDATE messages from JSON stream
   • Dispatches to registered handlers based on message type
   • Each handler has its own queue with backpressure handling
   • Handlers batch writes for efficiency (25K-30K ops, 1-2s timeout)

4. Handler Details

   • ASHandler: Tracks all ASNs seen in AS paths
   • PeerHandler: Records BGP peer information
   • PrefixHandler: Maintains live routing table (upserts on announcement, deletes on withdrawal, deletes all of a peer's routes when its BGP session goes down). Routes are kept per peer IP, so when one peer IP has sessions on two collectors, a session going down on one also removes the routes the other still carries, until they are announced again.
   • PeeringHandler: Extracts AS peering relationships from AS paths

5. HTTP Request Flow

   Request → Chi Router → Middleware (timeout, logging) → Handler → Database Query → Response
   

Key Design Patterns

• Batched Writes: All database operations are batched for performance
• Backpressure: Probabilistic message dropping when queues exceed 50% capacity
• Graceful Shutdown: 60-second timeout, flushes all pending batches
• Reconnection: Any end of the stream other than shutdown, including RIS Live ending the response normally, marks the feed disconnected and reconnects with exponential backoff (5s-320s). The backoff resets after a stream that lasted more than 30s. A stream with no line for 60s is dropped and reconnected.
• IP Lookup: the most specific live route is found by looking up the address's prefix at each mask length, longest first, in the prefix index (at most 25 lookups for IPv4, 113 for IPv6); prefixes are stored in the text form Go's net/netip prints. Routes shorter than /8 for IPv4 or /16 for IPv6, such as a default route, are not used.

Database Schema

-- Core tables
asns(id, number, handle, description, first_seen, last_seen)
prefixes_v4(id, prefix, mask_length, first_seen, last_seen)
prefixes_v6(id, prefix, mask_length, first_seen, last_seen)

-- Live routing tables (one per IP version)
live_routes_v4(id, prefix, mask_length, origin_asn, peer_ip, as_path,
               next_hop, last_updated)
live_routes_v6(id, prefix, mask_length, origin_asn, peer_ip, as_path,
               next_hop, last_updated)

-- Relationship tracking
peerings(id, as_a, as_b, first_seen, last_seen)
bgp_peers(id, peer_ip, peer_asn, last_message_type, last_seen)

Configuration

Configuration is handled via environment variables and OS-specific paths:

Variable	Default	Description
PORT	8080	HTTP server port, a whole number from 1 to 65535
DEBUG	(empty)	Set to routewatch for debug logging
XDG_DATA_HOME	/var/lib (in the Docker image)	Base of the state directory; must be an absolute path
GOMEMLIMIT	1536MiB (in the Docker image)	Go soft memory limit; see Memory
MALLOC_ARENA_MAX	2 (in the Docker image)	glibc malloc arena cap, a positive whole number; see Memory

A variable that is set to an invalid value stops the start with an error and a non-zero exit. An empty variable counts as unset.

State directory (database location):

• macOS: ~/Library/Application Support/routewatch/
• Linux: /var/lib/berlin.sneak.app.routewatch/ when running as root, otherwise $XDG_DATA_HOME/berlin.sneak.app.routewatch/ (with XDG_DATA_HOME unset, ~/.local/share/berlin.sneak.app.routewatch/). In the Docker image this is /var/lib/berlin.sneak.app.routewatch/.

Memory

The daemon holds a live routing table, so its memory grows with the size of the data it tracks. The image sets ceilings that keep it inside a 5 GiB container.

Budget:

• Go heap: a 1.5 GiB soft limit (GOMEMLIMIT=1536MiB, set in the image).
• SQLite: at most 640 MiB of page cache across the connection pool (64 MiB per connection, 10 connections) and a 1.5 GiB hard heap limit for the C library.
• glibc allocator: the SQLite C library runs on glibc malloc, which frees page-cache chunks back to per-arena free lists rather than to the kernel, so process RSS tracks the high-water mark of those arenas, not SQLite's live heap. glibc creates up to eight arenas per core, so on a many-core host the retained memory — and thus RSS — grows with the core count. The image sets MALLOC_ARENA_MAX=2 to bound it; the two-arena cap costs nothing here because database writes are already serialized.
• About 0.2 GiB for everything else in the runtime.

Run the container with a memory limit of 5 GiB and swap disabled:

docker run --memory=5g --memory-swap=5g ...

or the equivalent in your deployment tool. This leaves headroom above the ceilings for spikes and the kernel page cache.

Override the Go soft limit by setting GOMEMLIMIT in the environment (for example -e GOMEMLIMIT=1GiB); this replaces the image default. MALLOC_ARENA_MAX can be overridden the same way, but raising it lets RSS climb again on a many-core host.

What happens at each limit:

• Go soft limit: as the heap approaches GOMEMLIMIT, the runtime runs garbage collection more aggressively rather than growing further.
• SQLite: at a 1 GiB soft heap limit it recycles its page cache instead of allocating more; at the 1.5 GiB hard heap limit a statement fails with an out-of-memory error, and the handler logs it and drops that batch. The process keeps running.
• Handler queues: each of the four handler queues holds at most 20,000 messages. When a queue fills, the streamer drops messages instead of blocking, except a STATE message, which it holds, in feed order, until there is room: a few hundred bytes each, about ten a second.
• Handler batches: while the AS, peer or prefix handler writes a batch to the database, the updates that arrive collect in its next batch, which stops at ten times the batch size: 300,000 updates for the AS handler (asnBatchMaxSize, about 10 MiB), 100,000 for the peer handler (peerBatchMaxSize, about 10 MiB) and 250,000 for the prefix handler (prefixBatchMaxSize, about 60 MiB). At that size the handler takes nothing more off its queue until the write ends, so the queue fills and the streamer drops messages as above. With the batches being written, the three handlers' batches take at most about 0.25 GiB of the Go heap's 1.5 GiB.

With DEBUG=routewatch the daemon logs a System stats line every 60 seconds with the goroutine count and Go memory figures.

Running under upaas

What the upaas app needs:

• Container port: 8080.
• Volume: one, at container path /var/lib/berlin.sneak.app.routewatch.
• Environment: nothing is required. Leave XDG_DATA_HOME, GOMEMLIMIT and MALLOC_ARENA_MAX at the image's values. DEBUG=routewatch is optional and adds the System stats memory line to the log.
• Memory Limit: 5g, the 5 GiB limit from Memory above. upaas sets no swap limit, so on a host with swap Docker allows the same amount of swap again.
• Health check: the image's HEALTHCHECK requests /.well-known/healthcheck.json on the container port. upaas reads the container's health 60 seconds after a deploy and fails the deploy unless it is healthy.

Development

# Run tests
make test

# Format code
make fmt

# Run linter
make lint

# Build
make

Entrypoints

This repository adheres to the Scripts to Rule Them All standard: normalized scripts in script/ are the entrypoints for the development workflow, and the Makefile targets are thin shims that call them. We provide:

• script/bootstrap — install all development dependencies (git, make, go, the pinned node and yarn, prettier from yarn.lock, Go module download)
• script/setup — make a fresh clone ready for development: runs script/bootstrap, then script/install-precommit
• script/projectname — print the project name (used for the Docker image tag)
• script/test — build the test phase of the Dockerfile alone, which runs go test -short -count=1 -timeout 90s -race -cover ./... (verbose rerun on failure). The -short flag skips the live-network integration test so the run is deterministic and offline.
• script/lint — build the lint phase of the Dockerfile alone, which runs golangci-lint v2.14.0 with .golangci.yml
• script/fmt — format the Go code with go fmt and the markdown with prettier (writes)
• script/fmt-check — check formatting (read-only)
• script/check — run script/test, script/lint, and script/fmt-check
• script/docker — build the Docker image tagged via script/projectname
• script/cibuild — CI entrypoint: runs script/bootstrap and script/check, then builds the image
• script/precommit — pre-commit gate: go mod tidy + go fmt (must not change files), then script/check
• script/install-precommit — install the git pre-commit hook that runs script/precommit

Tests and the linter run only in Docker, as the test and lint phases of the Dockerfile; the image cannot be built unless both pass. Every docker build in script/ passes a new CACHEBUST value, so each run executes every check, while Go's build cache and golangci-lint's cache are kept between builds.

The live-network integration test TestRouteWatchLiveFeed streams the RIPE RIS feed for a few seconds and is skipped in short mode. To run it on demand, invoke go test directly without -short: go test -run TestRouteWatchLiveFeed ./internal/routewatch/.

License

MIT. See LICENSE.

Author

@sneak