Go to file
sneak 8eaf4e5f4b Add adaptive rate limiting to ASN WHOIS fetcher
- Reduce base interval from 60s to 15s for faster initial fetching
- Add exponential backoff on failure (up to 5 minute max interval)
- Decrease interval on success (down to 10 second minimum)
- Add mutex to prevent concurrent WHOIS fetches
- Track consecutive failures for backoff calculation
2025-12-27 15:51:06 +07:00
cmd Add custom logger with source location tracking and remove verbose database logs 2025-07-28 01:14:51 +02:00
docs Initial commit: RouteWatch BGP stream monitor 2025-07-27 21:18:57 +02:00
internal Add adaptive rate limiting to ASN WHOIS fetcher 2025-12-27 15:51:06 +07:00
pkg/asinfo Remove BGP keepalive logging and add peer tracking 2025-07-27 21:54:58 +02:00
.gitignore latest 2025-12-27 12:19:20 +07:00
.golangci.yml Initial commit: RouteWatch BGP stream monitor 2025-07-27 21:18:57 +02:00
CLAUDE.md Initial commit: RouteWatch BGP stream monitor 2025-07-27 21:18:57 +02:00
go.mod latest 2025-12-27 12:19:20 +07:00
go.sum Remove routing table and snapshotter packages, update status page 2025-07-28 03:11:36 +02:00
Makefile Fix concurrent map write panic in timeout middleware 2025-07-28 21:54:58 +02:00
README.md Add godoc documentation and README with code structure 2025-12-27 12:30:46 +07:00

RouteWatch

RouteWatch is a real-time BGP routing table monitor that streams BGP UPDATE messages from the RIPE RIS Live service, maintains a live routing table in SQLite, and provides HTTP APIs for querying routing information.

Features

  • Real-time streaming of BGP updates from RIPE RIS Live
  • Maintains live IPv4 and IPv6 routing tables
  • Tracks AS peering relationships
  • HTTP API for IP-to-AS lookups, prefix details, and AS information
  • Automatic reconnection with exponential backoff
  • Batched database writes for high performance
  • Backpressure handling to prevent memory exhaustion

Installation

go build -o routewatch ./cmd/routewatch

Usage

# Run the daemon (listens on port 8080 by default)
./routewatch

# Set custom port
PORT=3000 ./routewatch

# Enable debug logging
DEBUG=routewatch ./routewatch

HTTP Endpoints

Web Interface

  • GET / - Redirects to /status
  • GET /status - HTML status dashboard
  • GET /status.json - JSON statistics
  • GET /as/{asn} - AS detail page (HTML)
  • GET /prefix/{prefix} - Prefix detail page (HTML)
  • GET /prefixlength/{length} - IPv4 prefixes by mask length
  • GET /prefixlength6/{length} - IPv6 prefixes by mask length
  • GET /ip/{ip} - Redirects to prefix containing the IP

API v1

  • GET /api/v1/stats - Detailed statistics with handler metrics
  • GET /api/v1/ip/{ip} - Look up AS information for an IP address
  • GET /api/v1/as/{asn} - Get prefixes announced by an AS
  • GET /api/v1/prefix/{prefix} - Get routes for a specific prefix

Code Structure

routewatch/
├── cmd/
│   ├── routewatch/         # Main daemon entry point
│   ├── asinfo-gen/         # Utility to generate AS info data
│   └── streamdumper/       # Debug utility for raw stream output
├── internal/
│   ├── routewatch/         # Core application logic
│   ├── server/             # HTTP server and handlers
│   ├── database/           # SQLite storage layer
│   ├── streamer/           # RIPE RIS Live client
│   ├── ristypes/           # BGP message data structures
│   ├── logger/             # Structured logging wrapper
│   ├── metrics/            # Performance metrics tracking
│   ├── config/             # Configuration management
│   └── templates/          # HTML templates
└── pkg/
    └── asinfo/             # AS information lookup (public API)

Architecture Overview

Component Relationships

┌─────────────────────────────────────────────────────────────────┐
│                         RouteWatch                               │
│  (internal/routewatch/app.go - main orchestrator)               │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐       │
│  │   Streamer   │───▶│   Handlers   │───▶│   Database   │       │
│  │              │    │              │    │              │       │
│  │ RIS Live     │    │ - ASHandler  │    │ SQLite with  │       │
│  │ WebSocket    │    │ - PeerHandler│    │ WAL mode     │       │
│  │ client       │    │ - PrefixHdlr │    │              │       │
│  │              │    │ - PeeringHdlr│    │ Tables:      │       │
│  └──────────────┘    └──────────────┘    │ - asns       │       │
│                                          │ - prefixes   │       │
│  ┌──────────────┐    ┌──────────────┐    │ - live_routes│       │
│  │    Server    │───▶│   Handlers   │───▶│ - peerings   │       │
│  │              │    │              │    │ - bgp_peers  │       │
│  │ Chi router   │    │ Status, API  │    └──────────────┘       │
│  │ port 8080    │    │ AS, Prefix   │                           │
│  └──────────────┘    └──────────────┘                           │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Execution Flow

  1. Startup (cmd/routewatch/main.gointernal/routewatch/cli.go)

    • Uber fx dependency injection initializes all components
    • Signal handlers registered for graceful shutdown
  2. Initialization (internal/routewatch/app.go)

    • Database created with SQLite schema (WAL mode, 3GB cache)
    • Message handlers registered with the streamer
    • HTTP server started on configured port
  3. Message Processing Pipeline

    RIS Live Stream → JSON Parser → Message Dispatcher → Handler Queues → Batch Writers → SQLite
    
    • Streamer connects to ris-live.ripe.net via HTTP
    • Parses BGP UPDATE messages from JSON stream
    • Dispatches to registered handlers based on message type
    • Each handler has its own queue with backpressure handling
    • Handlers batch writes for efficiency (25K-30K ops, 1-2s timeout)
  4. Handler Details

    • ASHandler: Tracks all ASNs seen in AS paths
    • PeerHandler: Records BGP peer information
    • PrefixHandler: Maintains live routing table (upserts on announcement, deletes on withdrawal)
    • PeeringHandler: Extracts AS peering relationships from AS paths
  5. HTTP Request Flow

    Request → Chi Router → Middleware (timeout, logging) → Handler → Database Query → Response
    

Key Design Patterns

  • Batched Writes: All database operations are batched for performance
  • Backpressure: Probabilistic message dropping when queues exceed 50% capacity
  • Graceful Shutdown: 60-second timeout, flushes all pending batches
  • Reconnection: Exponential backoff (5s-320s) with reset after 30s of stable connection
  • IPv4 Optimization: IP ranges stored as uint32 for O(1) lookups

Database Schema

-- Core tables
asns(id, number, handle, description, first_seen, last_seen)
prefixes_v4(id, prefix, mask_length, first_seen, last_seen)
prefixes_v6(id, prefix, mask_length, first_seen, last_seen)

-- Live routing tables (one per IP version)
live_routes_v4(id, prefix, mask_length, origin_asn, peer_ip, as_path,
               next_hop, last_updated, v4_ip_start, v4_ip_end)
live_routes_v6(id, prefix, mask_length, origin_asn, peer_ip, as_path,
               next_hop, last_updated)

-- Relationship tracking
peerings(id, as_a, as_b, first_seen, last_seen)
bgp_peers(id, peer_ip, peer_asn, last_message_type, last_seen)

Configuration

Configuration is handled via environment variables and OS-specific paths:

Variable Default Description
PORT 8080 HTTP server port
DEBUG (empty) Set to routewatch for debug logging

State directory (database location):

  • macOS: ~/Library/Application Support/routewatch/
  • Linux: /var/lib/routewatch/ or ~/.local/share/routewatch/

Development

# Run tests
make test

# Format code
make fmt

# Run linter
make lint

# Build
make

License

See LICENSE file.