Commit Graph

13 Commits

Author SHA1 Message Date
67f6b78aaa Add custom logger with source location tracking and remove verbose database logs
- Create internal/logger package with Logger wrapper around slog
- Logger automatically adds source file, line number, and function name to all log entries
- Use golang.org/x/term to properly detect if stdout is a terminal
- Replace all slog.Logger usage with logger.Logger throughout the codebase
- Remove verbose logging from database GetStats() method
- Update all constructors and dependencies to use the new logger
2025-07-28 01:14:51 +02:00
3f06955214 Increase batch sizes and timeouts for better throughput
- Increase batch size from 100/50 to 500 for both handlers
- Increase batch timeout from 100-200ms to 5 seconds
- This will reduce database write frequency and improve throughput
  for high-volume BGP update streams
2025-07-28 01:03:09 +02:00
155c08d735 Implement batched database operations for improved performance
- Add BatchedDatabaseHandler that batches prefix, ASN, and peering operations
- Add BatchedPeerHandler that batches peer update operations
- Batch operations are deduped and flushed every 100-200ms or when batch size is reached
- Add EnableBatchedDatabaseWrites config option (enabled by default)
- Properly flush remaining batches on shutdown
- This significantly reduces database write pressure and improves throughput
2025-07-28 01:01:27 +02:00
d15a5e91b9 Inject Config as dependency for database, routing table, and snapshotter
- Remove old database Config struct and related functions
- Update database.New() to accept config.Config parameter
- Update routingtable.New() to accept config.Config parameter
- Update snapshotter.New() to accept config.Config parameter
- Simplify fx module providers in app.go
- Fix truthiness check for environment variables
- Handle empty state directory gracefully in routing table and snapshotter
- Update all tests to use empty state directory for testing
2025-07-28 00:55:09 +02:00
6593a7be76 Add database file size and reorganize status page
- Add database file size tracking to Stats struct and GetStats()
- Move routing table metrics to separate 'Routing Table' status box
- Add IPv4/IPv6 updates per second to routing table metrics
- Database box now shows: ASNs, prefixes, peerings, and database size
- Routing table box shows: live routes, IPv4/IPv6 counts, and update rates
2025-07-28 00:27:54 +02:00
fa9b086629 Fix shutdown sequence to ensure final snapshot is taken
- Add Shutdown() method to RouteWatch with mutex-protected shutdown flag
- Move all cleanup logic from Run() to Shutdown()
- Call Shutdown() from fx OnStop hook
- This ensures snapshotter gets called during graceful shutdown
2025-07-28 00:11:54 +02:00
52cdcd5785 Fix snapshotter initialization and remove initial snapshot on startup
- Remove immediate snapshot when periodic goroutine starts
- Fix variable shadowing issue in snapshotter creation
- Add debug logging for snapshotter shutdown
- Snapshots now only occur after 10 minutes or on shutdown
2025-07-28 00:06:39 +02:00
ae2ef2ae0c Implement routing table snapshotter with automatic loading on startup
- Create snapshotter package with periodic (10 min) and on-demand snapshots
- Add JSON serialization with gzip compression and atomic file writes
- Update routing table to track AddedAt time for each route
- Load snapshots on startup, filtering out stale routes (>30 minutes old)
- Add ROUTEWATCH_DISABLE_SNAPSHOTTER env var for tests
- Use OS-appropriate state directories (macOS: ~/Library/Application Support, Linux: /var/lib or XDG_STATE_HOME)
2025-07-28 00:03:19 +02:00
1d05372899 Fix linting errors for magic numbers in handler queue sizes
- Define constants for all handler queue capacities
- Fix integer overflow warning in metrics calculation
- Add missing blank lines before continue statements
2025-07-27 23:38:38 +02:00
76ec9f68b7 Add ASN info lookup and periodic routing table statistics
- Add handle and description columns to asns table
- Look up ASN info using asinfo package when creating new ASNs
- Remove noisy debug logging for individual route updates
- Add IPv4/IPv6 route counters and update rate tracking
- Log routing table statistics every 15 seconds with IPv4/IPv6 breakdown
- Track updates per second for both IPv4 and IPv6 routes separately
2025-07-27 23:25:23 +02:00
a555a1dee2 Replace live_routes database table with in-memory routing table
- Remove live_routes table from SQL schema and all related indexes
- Create new internal/routingtable package with thread-safe RoutingTable
- Implement RouteKey-based indexing with secondary indexes for efficient lookups
- Add RoutingTableHandler to manage in-memory routes separately from database
- Update DatabaseHandler to only handle persistent database operations
- Wire up RoutingTable through fx dependency injection
- Update server to get live route count from routing table instead of database
- Remove LiveRoutes field from database.Stats struct
- Update tests to work with new architecture
2025-07-27 23:16:19 +02:00
585ff63fae Remove BGP keepalive logging and add peer tracking
- Created bgp_peers table to track all BGP peers
- Added PeerHandler to update peer last seen times for all message types
- Removed verbose BGP keepalive debug logging
- BGP keepalive messages now silently update peer tracking

Refactor HTML templates to use go:embed

- Created internal/templates package with embedded templates
- Moved status.html from inline const to separate file
- Templates are parsed once on startup
- Server now uses parsed template instead of raw string

Optimize AS data embedding with gzip compression

- Changed asinfo package to embed gzipped data (2.4MB vs 12MB)
- Updated Makefile to gzip AS data during update
- Added decompression during initialization
- Raw JSON file excluded from git
2025-07-27 21:54:58 +02:00
92f7527cc5 Initial commit: RouteWatch BGP stream monitor
- Connects to RIPE RIS Live stream to receive real-time BGP updates
- Stores BGP data in SQLite database:
  - ASNs with first/last seen timestamps
  - Prefixes with IPv4/IPv6 classification
  - BGP announcements and withdrawals
  - AS-to-AS peering relationships from AS paths
  - Live routing table tracking active routes
- HTTP server with statistics endpoints
- Metrics tracking with go-metrics
- Custom JSON unmarshaling to handle nested AS sets in paths
- Dependency injection with uber/fx
- Pure Go implementation (no CGO)
- Includes streamdumper utility for debugging raw messages
2025-07-27 21:18:57 +02:00