58 Commits

Author SHA1 Message Date
9518519208 Fix prefix links on prefix length page with URL encoding
- Add urlEncode template function to properly encode prefix URLs
- Move prefix_length.html to embedded templates with function map
- Prevents broken links for prefixes containing slashes
2025-07-28 22:00:27 +02:00
7d39bd18bc Fix concurrent map write panic in timeout middleware
- Add thread-safe header wrapper in timeoutWriter
- Check context cancellation before writing responses in handlers
- Protect header access after timeout with mutex
- Prevents race condition when requests timeout while handlers are still running
2025-07-28 21:54:58 +02:00
e0a4c8642e Add context cancellation support to database operations
- Add context-aware versions of all read operations in the database
- Update handlers to use context from HTTP requests
- Allows database queries to be cancelled when HTTP requests timeout
- Prevents database operations from continuing after client disconnects
2025-07-28 19:27:55 +02:00
0196251906 Fix race condition crash in timeout middleware
- Remove duplicate http.Error call when context times out
- The timeout middleware already handles writing the response
- Prevents "concurrent write to websocket connection" panic
2025-07-28 19:07:30 +02:00
62ed5e08aa Improve prefix count link styling on status page
- Add dashed underline to prefix count links to indicate they are clickable
- Change to solid blue underline on hover for better UX
- Remove inline styles and use CSS classes instead
2025-07-28 19:05:45 +02:00
5fb3fc0381 Fix prefix length page to show unique prefixes only
- Change GetRandomPrefixesByLength to return unique prefixes instead of all routes
- Use CTE to first select random unique prefixes, then join to get their latest route info
- This ensures each prefix appears only once in the list
2025-07-28 19:04:19 +02:00
9a63553f8d Add index to optimize COUNT(DISTINCT prefix) queries
- Add compound index on (ip_version, mask_length, prefix) to speed up prefix distribution queries
- This index will significantly improve performance of COUNT(DISTINCT prefix) operations
- Note: Existing databases will need to manually create this index or recreate the database
2025-07-28 19:01:45 +02:00
ba13c76c53 Fix prefix distribution bug and add prefix length pages
- Fix GetPrefixDistribution to count unique prefixes using COUNT(DISTINCT prefix) instead of COUNT(*)
- Add /prefixlength/<length> route showing random sample of 500 prefixes
- Make prefix counts on status page clickable links to prefix length pages
- Add GetRandomPrefixesByLength database method
- Create prefix_length.html template with sortable table
- Show prefix age and origin AS with descriptions
2025-07-28 18:42:38 +02:00
1dcde74a90 Update AS path display to show handles with clickable links
- Change AS path from descriptions to handles (short names)
- Make each AS in the path a clickable link to /as/<asn>
- Add font-weight to AS links in path for better visibility
- Prevent word wrapping on all table columns except AS path
- Remove unused maxASDescriptionLength constant
2025-07-28 18:31:35 +02:00
81267431f7 Increase batch sizes to improve write throughput
- Increase prefix batch size from 5K to 20K
- Increase ASN batch size from 10K to 30K
- Add comments warning not to reduce batch timeouts
- Add comments warning not to increase queue sizes above 100K
- Maintains existing batch timeouts for efficiency
2025-07-28 18:27:42 +02:00
dc3ceb8d94 Show AS descriptions in AS path on prefix detail page
- Display AS descriptions alongside AS numbers in format: Description (ASN)
- Truncate descriptions longer than 20 characters with ellipsis
- Increase container max width to 1600px for better display
- Enable word wrapping for AS path cells to handle long paths
- Update mobile responsive styles for AS path display
2025-07-28 18:25:26 +02:00
a78e5c6e92 Add log.txt to .gitignore 2025-07-28 18:13:07 +02:00
9ef2a22db3 Remove SQLite pragmas that set default values
- Remove page_size, wal_autocheckpoint, locking_mode, mmap_size
- Keep only pragmas that change behavior from defaults
- Increase cache size to 3GB (upper limit for 2.4GB database)
2025-07-28 18:12:25 +02:00
05805b8847 Optimize SQLite settings for better balance
- Reduce cache size from 8GB to 512MB (still plenty for 2.4GB DB)
- Reduce mmap_size from 10GB to 256MB (reasonable default)
- Use default page size (4KB) instead of 8KB
- Use default WAL checkpoint interval (1000 pages)
- Remove redundant pragmas (threads, cache_spill, read_uncommitted)
- Clean up connection string to only use _txlock parameter
- Keep synchronous=OFF for performance (since we have mutex protection)
2025-07-28 18:06:31 +02:00
ddb3cfa4f0 Add mutex to serialize database access
- Add internal mutex to Database struct with lock/unlock wrappers
- Add debug logging for lock acquisition and release with timing
- Wrap all write operations with database mutex
- Use _txlock=immediate in SQLite connection string

This works around apparent issues with SQLite's internal locking
not properly respecting busy_timeout in production environment.
2025-07-28 17:56:26 +02:00
3ef60459b2 Fix SQLite transaction deadlocks with immediate mode
- Add _txlock=immediate to SQLite connection string
- This prevents deadlocks by acquiring write locks at transaction start
- Multiple concurrent writers now queue properly instead of failing instantly
- Resolves 'database table is locked' errors in production
2025-07-28 17:26:42 +02:00
40d7f0185b Optimize database batch operations with prepared statements
- Add prepared statements to all batch operations for better performance
- Fix database lock contention by properly batching operations
- Update SQLite settings for extreme performance (8GB cache, sync OFF)
- Add proper error handling for statement closing
- Update tests to properly track batch operations
2025-07-28 17:21:40 +02:00
b9b0792df9 Fix shutdown handling and optimize SQLite settings
- Fix Ctrl-C shutdown by using fx.Shutdowner instead of just canceling context
- Pass context from fx lifecycle to rw.Run() for proper cancellation
- Adjust WAL settings: checkpoint at 50MB, max size 100MB
- Reduce busy timeout from 30s to 2s to fail fast on lock contention

This should fix the issue where Ctrl-C doesn't cause shutdown and improve
database responsiveness under heavy load.
2025-07-28 16:52:52 +02:00
21921a170c Optimize database performance to fix slow queries
- Add VACUUM on startup to defragment database
- Increase cache size from 256MB to 2GB for better performance
- Increase mmap_size from 256MB to 512MB
- Add PRAGMA analysis_limit=0 to disable automatic ANALYZE
- Remove PRAGMA optimize which could trigger slow ANALYZE

These changes should dramatically improve query performance and prevent
the 5+ second query times seen in production.
2025-07-28 16:47:59 +02:00
78d6e17c76 Add debug logging and optimize SQLite performance
- Add debug logging for goroutines and memory usage (enabled via DEBUG=routewatch)
- Increase SQLite connection pool from 1 to 10 connections for better concurrency
- Optimize SQLite pragmas for balanced performance and safety
- Add proper shutdown handling for peering handler
- Define constants to avoid magic numbers in code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-28 15:45:06 +02:00
9b649c98c9 Fix AS detail view and add prefix sorting
- Fix GetASDetails to properly handle timestamp from MAX(last_updated)
- Parse timestamp string from SQLite aggregate function result
- Add natural sorting of prefixes by IP address in AS detail view
- Sort IPv4 and IPv6 prefixes separately by network address
- Remove SQL ORDER BY since we're sorting in Go
- This fixes the issue where AS detail pages showed no prefixes
2025-07-28 04:42:10 +02:00
48db8b9edf Fix AS detail view to show unique prefixes
- Update GetASDetails query to GROUP BY prefix instead of using DISTINCT
- Use MAX(last_updated) to get the most recent update time for each prefix
- This prevents duplicate prefixes from appearing when announced by multiple peers
- Maintains the same prefix count and ordering
2025-07-28 04:36:22 +02:00
df31cf880a Fix prefix URL routing by using URL encoding
- Replace slash-to-dash conversion with proper URL encoding
- Update handlePrefixDetail and handlePrefixDetailJSON to URL decode prefix parameter
- Update handleIPRedirect to URL encode the prefix in the redirect
- Add urlEncode template function for use in templates
- Update AS detail template to URL encode prefix links
- This properly handles the slash in CIDR notation (e.g., /prefix/192.168.1.0%2F24)
2025-07-28 04:34:34 +02:00
af9ff258b1 Add /ip/<ip> route that redirects to prefix detail page
- Implement handleIPRedirect handler that looks up the prefix containing an IP
- Add /ip/{ip} route to routes.go
- Reuse existing GetASInfoForIP database method which returns prefix info
- Redirect to /prefix/<prefix> page with HTTP 303 See Other status
- Handle invalid IPs (400) and IPs with no route (404)
2025-07-28 04:31:22 +02:00
aeeb5e7d7d Implement AS and prefix detail pages
- Implement handleASDetail() and handlePrefixDetail() HTML handlers
- Create AS detail HTML template with prefix listings
- Create prefix detail HTML template with route information
- Add timeSince template function for human-readable durations
- Update templates.go to include new templates
- Server-side rendered pages as requested (no client-side API calls)
2025-07-28 04:26:20 +02:00
27ae80ea2e Refactor server package: split handlers and routes into separate files
- Move all handler functions to handlers.go
- Move setupRoutes to routes.go
- Clean up server.go to only contain core server logic
- Add missing GetASDetails and GetPrefixDetails to mockStore for tests
- Fix linter errors (magic numbers, unused parameters, blank lines)
2025-07-28 04:00:12 +02:00
2fc24bb937 Add route age information to IP lookup API
- Add last_updated timestamp and age fields to ASInfo
- Include route's last_updated time from live_routes table
- Calculate and display age as human-readable duration
- Update both IPv4 and IPv6 queries to fetch timestamp
- Fix error handling to return 400 for invalid IPs
2025-07-28 03:44:19 +02:00
691710bc7c Add /api/v1/ip/<ip> endpoint for IP to AS lookups
- Add handleIPLookup handler that uses GetASInfoForIP
- Create writeJSONError and writeJSONSuccess helper functions
- Refactor all JSON error responses to use the helpers
- Add GetASInfoForIP to Store interface
- Add mock implementation for tests
- Fix all linter warnings
2025-07-28 03:31:53 +02:00
afb916036c Fix handler processing time display for sub-millisecond values
- Add formatProcessingTime function to display microseconds for values < 1ms
- Show 0 µs for times < 0.001ms, X.X µs for times < 0.01ms
- Show X.XXX ms for times < 1ms, X.XX ms for times >= 1ms
- Apply formatting to both average and min/max time displays
2025-07-28 03:26:16 +02:00
13047b5cb9 Add IPv4 range optimization for IP to AS lookups
- Add v4_ip_start and v4_ip_end columns to live_routes table
- Calculate IPv4 CIDR ranges as 32-bit integers for fast lookups
- Update PrefixHandler to populate IPv4 range fields
- Add GetASInfoForIP method with optimized IPv4 queries
- Add comprehensive tests for IP conversion functions
2025-07-28 03:23:25 +02:00
ae89468a1b Remove routing table and snapshotter packages, update status page
- Remove routingtable package entirely as database handles all routing data
- Remove snapshotter package as database contains all information
- Rename 'Connection Status' box to 'RouteWatch' and add Go version, goroutines, memory usage
- Move IPv4/IPv6 prefix counts from Database Statistics to Routing Table box
- Add Peers count to Database Statistics box
- Add go-humanize dependency for memory formatting
- Update server to include new metrics in API responses
2025-07-28 03:11:36 +02:00
d929f24f80 Remove RoutingTableHandler and snapshotter, use database for route stats
- Remove RoutingTableHandler as PrefixHandler maintains live_routes table
- Update server to get route counts from database instead of in-memory routing table
- Add GetLiveRouteCounts method to database for IPv4/IPv6 route counts
- Use metrics tracker in PrefixHandler for route update rates
- Remove snapshotter entirely as database contains all information
- Update tests to work without routing table
2025-07-28 03:02:44 +02:00
cb1f4d9052 Add route update metrics tracking to PrefixHandler
- Add RecordIPv4Update and RecordIPv6Update to metrics package
- Add SetMetricsTracker method to PrefixHandler
- Track IPv4/IPv6 route updates when processing announcements
- Add GetMetricsTracker method to Streamer to expose metrics
2025-07-28 02:55:27 +02:00
bc640b0b37 Reduce all handler queue sizes to 100,000 2025-07-28 02:50:05 +02:00
7d814c9d2d Optimize SQLite and PrefixHandler for better performance
- Increase PrefixHandler queue size to 500k and batch size to 25k
- Set SQLite PRAGMA synchronous=OFF for faster writes (trades durability)
- Increase SQLite cache to 1GB and mmap to 512MB
- Increase WAL checkpoint interval to 10000 pages
- Set page size to 8KB for better performance
- Increase busy timeout to 30 seconds
- Keep single connection to avoid SQLite locking issues
2025-07-28 02:40:17 +02:00
54bb0ba1cb Simplify peerings table to store AS numbers directly
- Rename asn_peerings table to peerings
- Change columns from from_asn_id/to_asn_id to as_a/as_b (integers)
- Remove foreign key constraints to asns table
- Update RecordPeering to use AS numbers directly
- Add validation in RecordPeering to ensure:
  - Both AS numbers are > 0
  - AS numbers are different
  - as_a is always lower than as_b (normalized)
- Update PeeringHandler to no longer need ASN cache
- Simplify the code by removing unnecessary ASN lookups
2025-07-28 02:36:15 +02:00
1157003db7 Refactor database handlers and optimize PeeringHandler
- Create PeeringHandler for asn_peerings table maintenance
- Rename DBHandler to ASHandler (now only handles asns table)
- Move prefixes table maintenance to PrefixHandler
- Optimize PeeringHandler with in-memory AS path tracking:
  - Stores AS paths in memory with timestamps
  - Processes peerings in batch every 2 minutes
  - Prunes old paths (>30 minutes) every 5 minutes
  - Normalizes peerings with lower AS number first
- Each handler now has a single responsibility:
  - ASHandler: asns table
  - PeerHandler: bgp_peers table
  - PrefixHandler: prefixes and live_routes tables
  - PeeringHandler: asn_peerings table
2025-07-28 02:31:04 +02:00
eaa11b5f8d Move prefixes table maintenance from DBHandler to PrefixHandler
- DBHandler now only maintains asns and asn_peerings tables
- PrefixHandler maintains both prefixes and live_routes tables
- This consolidates all prefix-related operations in one handler
2025-07-28 02:16:12 +02:00
8b43882526 Increase batch sizes to 10000 and queue sizes to 200000, reduce timeout to 2s 2025-07-28 02:11:05 +02:00
eda90d96a9 Remove debug logging for withdrawals without origin ASN 2025-07-28 02:07:33 +02:00
3c46087976 Add live routing table with CIDR mask length tracking
- Added new live_routes table with mask_length column for tracking CIDR prefix lengths
- Updated PrefixHandler to maintain live routing table with additions and deletions
- Added route expiration functionality (5 minute timeout) to in-memory routing table
- Added prefix distribution stats showing count of prefixes by mask length
- Added IPv4/IPv6 prefix distribution cards to status page
- Updated database interface with UpsertLiveRoute, DeleteLiveRoute, and GetPrefixDistribution
- Set all handler queue depths to 50000 for consistency
- Doubled DBHandler batch size to 32000 for better throughput
- Fixed withdrawal handling to delete routes when origin ASN is available
2025-07-28 01:51:42 +02:00
cea7c3dfd3 Rename handlers and add PrefixHandler for database routing table
- Renamed BatchedDatabaseHandler to DBHandler
- Renamed BatchedPeerHandler to PeerHandler
- Quadrupled DBHandler batch size from 4000 to 16000
- Created new PrefixHandler using same batching strategy to maintain routing table in database
- Removed verbose batch flush logging from all handlers
- Updated app.go to use renamed handlers and register PrefixHandler
- Fixed test configuration to enable batched database writes
2025-07-28 01:37:19 +02:00
3aef3f9a07 Format logger source location as file.go:linenum
- Change source location format from separate source and line fields
  to a single source field with format 'file.go:linenum'
- This provides a more concise and standard format for source locations
2025-07-28 01:16:29 +02:00
67f6b78aaa Add custom logger with source location tracking and remove verbose database logs
- Create internal/logger package with Logger wrapper around slog
- Logger automatically adds source file, line number, and function name to all log entries
- Use golang.org/x/term to properly detect if stdout is a terminal
- Replace all slog.Logger usage with logger.Logger throughout the codebase
- Remove verbose logging from database GetStats() method
- Update all constructors and dependencies to use the new logger
2025-07-28 01:14:51 +02:00
3f06955214 Increase batch sizes and timeouts for better throughput
- Increase batch size from 100/50 to 500 for both handlers
- Increase batch timeout from 100-200ms to 5 seconds
- This will reduce database write frequency and improve throughput
  for high-volume BGP update streams
2025-07-28 01:03:09 +02:00
155c08d735 Implement batched database operations for improved performance
- Add BatchedDatabaseHandler that batches prefix, ASN, and peering operations
- Add BatchedPeerHandler that batches peer update operations
- Batch operations are deduped and flushed every 100-200ms or when batch size is reached
- Add EnableBatchedDatabaseWrites config option (enabled by default)
- Properly flush remaining batches on shutdown
- This significantly reduces database write pressure and improves throughput
2025-07-28 01:01:27 +02:00
d15a5e91b9 Inject Config as dependency for database, routing table, and snapshotter
- Remove old database Config struct and related functions
- Update database.New() to accept config.Config parameter
- Update routingtable.New() to accept config.Config parameter
- Update snapshotter.New() to accept config.Config parameter
- Simplify fx module providers in app.go
- Fix truthiness check for environment variables
- Handle empty state directory gracefully in routing table and snapshotter
- Update all tests to use empty state directory for testing
2025-07-28 00:55:09 +02:00
1a0622efaa Add handler queue metrics to status page
- Add GetHandlerStats() method to streamer to expose handler metrics
- Include queue length/capacity, processed/dropped counts, timing stats
- Update API to include handler_stats in response
- Add dynamic handler stats display to status page HTML
- Shows separate status box for each handler with all metrics
2025-07-28 00:32:37 +02:00
6593a7be76 Add database file size and reorganize status page
- Add database file size tracking to Stats struct and GetStats()
- Move routing table metrics to separate 'Routing Table' status box
- Add IPv4/IPv6 updates per second to routing table metrics
- Database box now shows: ASNs, prefixes, peerings, and database size
- Routing table box shows: live routes, IPv4/IPv6 counts, and update rates
2025-07-28 00:27:54 +02:00
5bd3add59b Fix deadlock in snapshotter and add timing logs
- Move GetDetailedStats() call outside of read lock to avoid deadlock
- Add timing logs to identify performance bottlenecks during snapshot
- Log duration for copying routes, marshaling JSON, and writing to disk
2025-07-28 00:16:01 +02:00
fa9b086629 Fix shutdown sequence to ensure final snapshot is taken
- Add Shutdown() method to RouteWatch with mutex-protected shutdown flag
- Move all cleanup logic from Run() to Shutdown()
- Call Shutdown() from fx OnStop hook
- This ensures snapshotter gets called during graceful shutdown
2025-07-28 00:11:54 +02:00
52cdcd5785 Fix snapshotter initialization and remove initial snapshot on startup
- Remove immediate snapshot when periodic goroutine starts
- Fix variable shadowing issue in snapshotter creation
- Add debug logging for snapshotter shutdown
- Snapshots now only occur after 10 minutes or on shutdown
2025-07-28 00:06:39 +02:00
ae2ef2ae0c Implement routing table snapshotter with automatic loading on startup
- Create snapshotter package with periodic (10 min) and on-demand snapshots
- Add JSON serialization with gzip compression and atomic file writes
- Update routing table to track AddedAt time for each route
- Load snapshots on startup, filtering out stale routes (>30 minutes old)
- Add ROUTEWATCH_DISABLE_SNAPSHOTTER env var for tests
- Use OS-appropriate state directories (macOS: ~/Library/Application Support, Linux: /var/lib or XDG_STATE_HOME)
2025-07-28 00:03:19 +02:00
283f2ddbf2 Add separate IPv4/IPv6 route counts to status page and API
- Update server Stats and StatsResponse structs to include ipv4_routes and ipv6_routes
- Fetch detailed routing table stats to get IPv4/IPv6 breakdown
- Add IPv4 Routes and IPv6 Routes display to HTML status page
- Change metric values to monospace font and remove bold styling
2025-07-27 23:46:47 +02:00
1d05372899 Fix linting errors for magic numbers in handler queue sizes
- Define constants for all handler queue capacities
- Fix integer overflow warning in metrics calculation
- Add missing blank lines before continue statements
2025-07-27 23:38:38 +02:00
76ec9f68b7 Add ASN info lookup and periodic routing table statistics
- Add handle and description columns to asns table
- Look up ASN info using asinfo package when creating new ASNs
- Remove noisy debug logging for individual route updates
- Add IPv4/IPv6 route counters and update rate tracking
- Log routing table statistics every 15 seconds with IPv4/IPv6 breakdown
- Track updates per second for both IPv4 and IPv6 routes separately
2025-07-27 23:25:23 +02:00
a555a1dee2 Replace live_routes database table with in-memory routing table
- Remove live_routes table from SQL schema and all related indexes
- Create new internal/routingtable package with thread-safe RoutingTable
- Implement RouteKey-based indexing with secondary indexes for efficient lookups
- Add RoutingTableHandler to manage in-memory routes separately from database
- Update DatabaseHandler to only handle persistent database operations
- Wire up RoutingTable through fx dependency injection
- Update server to get live route count from routing table instead of database
- Remove LiveRoutes field from database.Stats struct
- Update tests to work with new architecture
2025-07-27 23:16:19 +02:00
b49d3ce88c Switch back to CGO SQLite driver
- Replace modernc.org/sqlite with github.com/mattn/go-sqlite3
- Update connection string for go-sqlite3 syntax
- Keep all performance optimizations and pragmas

The CGO driver may provide better performance for write-heavy
workloads compared to the pure Go implementation.
2025-07-27 22:57:53 +02:00
37 changed files with 183557 additions and 984 deletions

1
.gitignore vendored
View File

@@ -35,3 +35,4 @@ pkg/asinfo/asdata.json
# Debug output files # Debug output files
out out
log.txt

View File

@@ -15,13 +15,13 @@ lint:
golangci-lint run golangci-lint run
build: build:
go build -o bin/routewatch cmd/routewatch/main.go CGO_ENABLED=1 go build -o bin/routewatch cmd/routewatch/main.go
clean: clean:
rm -rf bin/ rm -rf bin/
run: build run: build
./bin/routewatch DEBUG=routewatch ./bin/routewatch 2>&1 | tee log.txt
asupdate: asupdate:
@echo "Updating AS info data..." @echo "Updating AS info data..."

View File

@@ -9,16 +9,14 @@ import (
"os/signal" "os/signal"
"syscall" "syscall"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/metrics" "git.eeqj.de/sneak/routewatch/internal/metrics"
"git.eeqj.de/sneak/routewatch/internal/streamer" "git.eeqj.de/sneak/routewatch/internal/streamer"
"log/slog"
) )
func main() { func main() {
// Set up logger to only show errors // Set up logger
logger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{ logger := logger.New()
Level: slog.LevelError,
}))
// Create metrics tracker // Create metrics tracker
metricsTracker := metrics.New() metricsTracker := metrics.New()

14
go.mod
View File

@@ -3,24 +3,18 @@ module git.eeqj.de/sneak/routewatch
go 1.24.4 go 1.24.4
require ( require (
github.com/go-chi/chi/v5 v5.2.2
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/mattn/go-sqlite3 v1.14.29
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9
go.uber.org/fx v1.24.0 go.uber.org/fx v1.24.0
modernc.org/sqlite v1.38.1
) )
require ( require (
github.com/dustin/go-humanize v1.0.1 // indirect github.com/dustin/go-humanize v1.0.1 // indirect
github.com/go-chi/chi/v5 v5.2.2 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
go.uber.org/dig v1.19.0 // indirect go.uber.org/dig v1.19.0 // indirect
go.uber.org/multierr v1.10.0 // indirect go.uber.org/multierr v1.10.0 // indirect
go.uber.org/zap v1.26.0 // indirect go.uber.org/zap v1.26.0 // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
golang.org/x/sys v0.34.0 // indirect golang.org/x/sys v0.34.0 // indirect
modernc.org/libc v1.66.3 // indirect golang.org/x/term v0.33.0 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
) )

47
go.sum
View File

@@ -4,20 +4,14 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkp
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-chi/chi/v5 v5.2.2 h1:CMwsvRVTbXVytCk1Wd72Zy1LAsAh9GxMmSNWLHCG618= github.com/go-chi/chi/v5 v5.2.2 h1:CMwsvRVTbXVytCk1Wd72Zy1LAsAh9GxMmSNWLHCG618=
github.com/go-chi/chi/v5 v5.2.2/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops= github.com/go-chi/chi/v5 v5.2.2/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-sqlite3 v1.14.29 h1:1O6nRLJKvsi1H2Sj0Hzdfojwt8GiGKm+LOfLaBFaouQ=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/mattn/go-sqlite3 v1.14.29/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 h1:bsUq1dX0N8AOIL7EB/X911+m4EHsnWEHeJ0c+3TTBrg= github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 h1:bsUq1dX0N8AOIL7EB/X911+m4EHsnWEHeJ0c+3TTBrg=
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
go.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4= go.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4=
@@ -30,42 +24,9 @@ go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=
go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo= go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so= go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b h1:M2rDM6z3Fhozi9O7NWsxAkg/yqS/lQJ6PmkyIV3YP+o=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b/go.mod h1:3//PLf8L/X+8b4vuAfHzxeRUl04Adcb341+IGKfnqS8=
golang.org/x/mod v0.25.0 h1:n7a+ZbQKQA/Ysbyb0/6IbB1H/X41mKgbhfv7AfG/44w=
golang.org/x/mod v0.25.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/sync v0.15.0 h1:KWH3jNZsfyT6xfAfKiz6MRNmd46ByHDYaZ7KSkCtdW8=
golang.org/x/sync v0.15.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA= golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo= golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg= golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.26.2 h1:991HMkLjJzYBIfha6ECZdjrIYz2/1ayr+FL8GN+CNzM=
modernc.org/cc/v4 v4.26.2/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccgo/v4 v4.28.0 h1:rjznn6WWehKq7dG4JtLRKxb52Ecv8OUGah8+Z/SfpNU=
modernc.org/ccgo/v4 v4.28.0/go.mod h1:JygV3+9AV6SmPhDasu4JgquwU81XAKLd3OKTUDNOiKE=
modernc.org/fileutil v1.3.8 h1:qtzNm7ED75pd1C7WgAGcK4edm4fvhtBsEiI/0NQ54YM=
modernc.org/fileutil v1.3.8/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.66.3 h1:cfCbjTUcdsKyyZZfEUKfoHcP3S0Wkvz3jgSzByEWVCQ=
modernc.org/libc v1.66.3/go.mod h1:XD9zO8kt59cANKvHPXpx7yS2ELPheAey0vjIuZOhOU8=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.38.1 h1:jNnIjleVta+DKSAr3TnkKK87EEhjPhBLzi6hvIX9Bas=
modernc.org/sqlite v1.38.1/go.mod h1:cPTJYSlgg3Sfg046yBShXENNtPrWrDX8bsbAQBzgQ5E=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=

103
internal/config/config.go Normal file
View File

@@ -0,0 +1,103 @@
// Package config provides centralized configuration management for RouteWatch
package config
import (
"fmt"
"os"
"path/filepath"
"runtime"
"time"
)
const (
// AppIdentifier is the reverse domain name identifier for the app
AppIdentifier = "berlin.sneak.app.routewatch"
// dirPermissions for creating directories
dirPermissions = 0750 // rwxr-x---
// defaultRouteExpirationMinutes is the default route expiration timeout in minutes
defaultRouteExpirationMinutes = 5
)
// Config holds configuration for the entire application
type Config struct {
// StateDir is the directory for all application state (database, snapshots)
StateDir string
// MaxRuntime is the maximum runtime (0 = run forever)
MaxRuntime time.Duration
// EnableBatchedDatabaseWrites enables batched database operations for better performance
EnableBatchedDatabaseWrites bool
// RouteExpirationTimeout is how long a route can go without being refreshed before expiring
// Default is 2 hours which is conservative for BGP (typical BGP hold time is 90-180 seconds)
RouteExpirationTimeout time.Duration
}
// New creates a new Config with default paths based on the OS
func New() (*Config, error) {
stateDir, err := getStateDirectory()
if err != nil {
return nil, fmt.Errorf("failed to determine state directory: %w", err)
}
return &Config{
StateDir: stateDir,
MaxRuntime: 0, // Run forever by default
EnableBatchedDatabaseWrites: true, // Enable batching by default
RouteExpirationTimeout: defaultRouteExpirationMinutes * time.Minute, // For active route monitoring
}, nil
}
// GetStateDir returns the state directory path
func (c *Config) GetStateDir() string {
return c.StateDir
}
// getStateDirectory returns the appropriate state directory based on the OS
func getStateDirectory() (string, error) {
switch runtime.GOOS {
case "darwin":
// macOS: ~/Library/Application Support/berlin.sneak.app.routewatch
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
return filepath.Join(home, "Library", "Application Support", AppIdentifier), nil
case "linux", "freebsd", "openbsd", "netbsd":
// Unix-like: /var/lib/berlin.sneak.app.routewatch if root, else XDG_DATA_HOME
if os.Geteuid() == 0 {
return filepath.Join("/var/lib", AppIdentifier), nil
}
// Check XDG_DATA_HOME first
if xdgData := os.Getenv("XDG_DATA_HOME"); xdgData != "" {
return filepath.Join(xdgData, AppIdentifier), nil
}
// Fall back to ~/.local/share/berlin.sneak.app.routewatch
home, err := os.UserHomeDir()
if err != nil {
return "", err
}
return filepath.Join(home, ".local", "share", AppIdentifier), nil
default:
return "", fmt.Errorf("unsupported operating system: %s", runtime.GOOS)
}
}
// EnsureDirectories creates all necessary directories if they don't exist
func (c *Config) EnsureDirectories() error {
// Ensure state directory exists
if err := os.MkdirAll(c.StateDir, dirPermissions); err != nil {
return fmt.Errorf("failed to create state directory: %w", err)
}
return nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,301 @@
package database
import (
"net"
"testing"
)
func TestIPToUint32(t *testing.T) {
tests := []struct {
name string
ip string
expected uint32
}{
{
name: "Simple IP",
ip: "192.168.1.1",
expected: 3232235777, // 192<<24 + 168<<16 + 1<<8 + 1
},
{
name: "Minimum IP",
ip: "0.0.0.0",
expected: 0,
},
{
name: "Maximum IP",
ip: "255.255.255.255",
expected: 4294967295,
},
{
name: "10.0.0.0",
ip: "10.0.0.0",
expected: 167772160,
},
{
name: "172.16.0.0",
ip: "172.16.0.0",
expected: 2886729728,
},
{
name: "8.8.8.8",
ip: "8.8.8.8",
expected: 134744072,
},
{
name: "1.2.3.4",
ip: "1.2.3.4",
expected: 16909060,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ip := net.ParseIP(tt.ip)
if ip == nil {
t.Fatalf("Failed to parse IP: %s", tt.ip)
}
result := ipToUint32(ip)
if result != tt.expected {
t.Errorf("ipToUint32(%s) = %d, want %d", tt.ip, result, tt.expected)
}
// Test with IPv4-mapped IPv6 address
ip6 := net.ParseIP(tt.ip).To16()
if ip6 != nil {
result6 := ipToUint32(ip6)
if result6 != tt.expected {
t.Errorf("ipToUint32(%s as IPv6) = %d, want %d", tt.ip, result6, tt.expected)
}
}
})
}
}
func TestCalculateIPv4Range(t *testing.T) {
tests := []struct {
name string
cidr string
wantStart uint32
wantEnd uint32
wantErr bool
}{
{
name: "Single IP /32",
cidr: "192.168.1.1/32",
wantStart: 3232235777,
wantEnd: 3232235777,
},
{
name: "Class C /24",
cidr: "192.168.1.0/24",
wantStart: 3232235776, // 192.168.1.0
wantEnd: 3232236031, // 192.168.1.255
},
{
name: "Class B /16",
cidr: "192.168.0.0/16",
wantStart: 3232235520, // 192.168.0.0
wantEnd: 3232301055, // 192.168.255.255
},
{
name: "Class A /8",
cidr: "10.0.0.0/8",
wantStart: 167772160, // 10.0.0.0
wantEnd: 184549375, // 10.255.255.255
},
{
name: "Entire IPv4 space /0",
cidr: "0.0.0.0/0",
wantStart: 0,
wantEnd: 4294967295,
},
{
name: "Small subnet /30",
cidr: "192.168.1.0/30",
wantStart: 3232235776, // 192.168.1.0
wantEnd: 3232235779, // 192.168.1.3
},
{
name: "Medium subnet /20",
cidr: "172.16.0.0/20",
wantStart: 2886729728, // 172.16.0.0
wantEnd: 2886733823, // 172.16.15.255
},
{
name: "Private range 172.16/12",
cidr: "172.16.0.0/12",
wantStart: 2886729728, // 172.16.0.0
wantEnd: 2887778303, // 172.31.255.255
},
{
name: "Google DNS /29",
cidr: "8.8.8.8/29",
wantStart: 134744072, // 8.8.8.8 (network is actually 8.8.8.8 with /29)
wantEnd: 134744079, // 8.8.8.15
},
{
name: "Non-zero host bits",
cidr: "192.168.1.5/24",
wantStart: 3232235776, // 192.168.1.0 (network address)
wantEnd: 3232236031, // 192.168.1.255
},
{
name: "Invalid CIDR",
cidr: "192.168.1.1/33",
wantErr: true,
},
{
name: "Invalid IP",
cidr: "256.256.256.256/24",
wantErr: true,
},
{
name: "IPv6 CIDR",
cidr: "2001:db8::/32",
wantErr: true,
},
{
name: "Empty CIDR",
cidr: "",
wantErr: true,
},
{
name: "Missing mask",
cidr: "192.168.1.1",
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
start, end, err := CalculateIPv4Range(tt.cidr)
if tt.wantErr {
if err == nil {
t.Errorf("CalculateIPv4Range(%s) expected error, got nil", tt.cidr)
}
return
}
if err != nil {
t.Errorf("CalculateIPv4Range(%s) unexpected error: %v", tt.cidr, err)
return
}
if start != tt.wantStart {
t.Errorf("CalculateIPv4Range(%s) start = %d, want %d", tt.cidr, start, tt.wantStart)
}
if end != tt.wantEnd {
t.Errorf("CalculateIPv4Range(%s) end = %d, want %d", tt.cidr, end, tt.wantEnd)
}
// Verify that start <= end
if start > end {
t.Errorf("CalculateIPv4Range(%s) start (%d) > end (%d)", tt.cidr, start, end)
}
// Verify the range size matches the CIDR mask
if !tt.wantErr && tt.cidr != "" {
_, ipNet, _ := net.ParseCIDR(tt.cidr)
if ipNet != nil {
ones, bits := ipNet.Mask.Size()
expectedSize := uint32(1) << uint(bits-ones)
actualSize := end - start + 1
if actualSize != expectedSize {
t.Errorf("CalculateIPv4Range(%s) range size = %d, want %d", tt.cidr, actualSize, expectedSize)
}
}
}
})
}
}
func TestIPv4RangeIntegration(t *testing.T) {
// Test that our functions work correctly together
tests := []struct {
name string
cidr string
testIPs []string
shouldContain []bool
}{
{
name: "192.168.1.0/24",
cidr: "192.168.1.0/24",
testIPs: []string{
"192.168.1.0",
"192.168.1.1",
"192.168.1.255",
"192.168.0.255",
"192.168.2.0",
},
shouldContain: []bool{true, true, true, false, false},
},
{
name: "10.0.0.0/8",
cidr: "10.0.0.0/8",
testIPs: []string{
"10.0.0.0",
"10.255.255.255",
"10.1.2.3",
"9.255.255.255",
"11.0.0.0",
},
shouldContain: []bool{true, true, true, false, false},
},
{
name: "172.16.0.0/12",
cidr: "172.16.0.0/12",
testIPs: []string{
"172.16.0.0",
"172.31.255.255",
"172.20.1.1",
"172.15.255.255",
"172.32.0.0",
},
shouldContain: []bool{true, true, true, false, false},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
start, end, err := CalculateIPv4Range(tt.cidr)
if err != nil {
t.Fatalf("Failed to calculate range for %s: %v", tt.cidr, err)
}
for i, testIP := range tt.testIPs {
ip := net.ParseIP(testIP)
if ip == nil {
t.Fatalf("Failed to parse test IP: %s", testIP)
}
ipUint := ipToUint32(ip)
contained := ipUint >= start && ipUint <= end
if contained != tt.shouldContain[i] {
t.Errorf("IP %s in range %s: got %v, want %v", testIP, tt.cidr, contained, tt.shouldContain[i])
}
}
})
}
}
func BenchmarkIPToUint32(b *testing.B) {
ip := net.ParseIP("192.168.1.1")
b.ResetTimer()
for i := 0; i < b.N; i++ {
_ = ipToUint32(ip)
}
}
func BenchmarkCalculateIPv4Range(b *testing.B) {
cidr := "192.168.0.0/16"
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, _, _ = CalculateIPv4Range(cidr)
}
}

View File

@@ -1,25 +1,29 @@
package database package database
import ( import (
"context"
"time" "time"
"github.com/google/uuid"
) )
// Stats contains database statistics // Stats contains database statistics
type Stats struct { type Stats struct {
ASNs int ASNs int
Prefixes int Prefixes int
IPv4Prefixes int IPv4Prefixes int
IPv6Prefixes int IPv6Prefixes int
Peerings int Peerings int
LiveRoutes int Peers int
FileSizeBytes int64
LiveRoutes int
IPv4PrefixDistribution []PrefixDistribution
IPv6PrefixDistribution []PrefixDistribution
} }
// Store defines the interface for database operations // Store defines the interface for database operations
type Store interface { type Store interface {
// ASN operations // ASN operations
GetOrCreateASN(number int, timestamp time.Time) (*ASN, error) GetOrCreateASN(number int, timestamp time.Time) (*ASN, error)
GetOrCreateASNBatch(asns map[int]time.Time) error
// Prefix operations // Prefix operations
GetOrCreatePrefix(prefix string, timestamp time.Time) (*Prefix, error) GetOrCreatePrefix(prefix string, timestamp time.Time) (*Prefix, error)
@@ -28,18 +32,37 @@ type Store interface {
RecordAnnouncement(announcement *Announcement) error RecordAnnouncement(announcement *Announcement) error
// Peering operations // Peering operations
RecordPeering(fromASNID, toASNID string, timestamp time.Time) error RecordPeering(asA, asB int, timestamp time.Time) error
// Live route operations
UpdateLiveRoute(prefixID, originASNID uuid.UUID, peerASN int, nextHop string, timestamp time.Time) error
WithdrawLiveRoute(prefixID uuid.UUID, peerASN int, timestamp time.Time) error
GetActiveLiveRoutes() ([]LiveRoute, error)
// Statistics // Statistics
GetStats() (Stats, error) GetStats() (Stats, error)
GetStatsContext(ctx context.Context) (Stats, error)
// Peer operations // Peer operations
UpdatePeer(peerIP string, peerASN int, messageType string, timestamp time.Time) error UpdatePeer(peerIP string, peerASN int, messageType string, timestamp time.Time) error
UpdatePeerBatch(peers map[string]PeerUpdate) error
// Live route operations
UpsertLiveRoute(route *LiveRoute) error
UpsertLiveRouteBatch(routes []*LiveRoute) error
DeleteLiveRoute(prefix string, originASN int, peerIP string) error
DeleteLiveRouteBatch(deletions []LiveRouteDeletion) error
GetPrefixDistribution() (ipv4 []PrefixDistribution, ipv6 []PrefixDistribution, err error)
GetPrefixDistributionContext(ctx context.Context) (ipv4 []PrefixDistribution, ipv6 []PrefixDistribution, err error)
GetLiveRouteCounts() (ipv4Count, ipv6Count int, err error)
GetLiveRouteCountsContext(ctx context.Context) (ipv4Count, ipv6Count int, err error)
// IP lookup operations
GetASInfoForIP(ip string) (*ASInfo, error)
GetASInfoForIPContext(ctx context.Context, ip string) (*ASInfo, error)
// AS and prefix detail operations
GetASDetails(asn int) (*ASN, []LiveRoute, error)
GetASDetailsContext(ctx context.Context, asn int) (*ASN, []LiveRoute, error)
GetPrefixDetails(prefix string) ([]LiveRoute, error)
GetPrefixDetailsContext(ctx context.Context, prefix string) ([]LiveRoute, error)
GetRandomPrefixesByLength(maskLength, ipVersion, limit int) ([]LiveRoute, error)
GetRandomPrefixesByLengthContext(ctx context.Context, maskLength, ipVersion, limit int) ([]LiveRoute, error)
// Lifecycle // Lifecycle
Close() error Close() error

View File

@@ -8,10 +8,12 @@ import (
// ASN represents an Autonomous System Number // ASN represents an Autonomous System Number
type ASN struct { type ASN struct {
ID uuid.UUID `json:"id"` ID uuid.UUID `json:"id"`
Number int `json:"number"` Number int `json:"number"`
FirstSeen time.Time `json:"first_seen"` Handle string `json:"handle"`
LastSeen time.Time `json:"last_seen"` Description string `json:"description"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
} }
// Prefix represents an IP prefix (CIDR block) // Prefix represents an IP prefix (CIDR block)
@@ -44,14 +46,49 @@ type ASNPeering struct {
LastSeen time.Time `json:"last_seen"` LastSeen time.Time `json:"last_seen"`
} }
// LiveRoute represents the current state of a route in the live routing table // LiveRoute represents a route in the live routing table
type LiveRoute struct { type LiveRoute struct {
ID uuid.UUID `json:"id"` ID uuid.UUID `json:"id"`
PrefixID uuid.UUID `json:"prefix_id"` Prefix string `json:"prefix"`
OriginASNID uuid.UUID `json:"origin_asn_id"` MaskLength int `json:"mask_length"`
PeerASN int `json:"peer_asn"` IPVersion int `json:"ip_version"`
Path string `json:"path"` OriginASN int `json:"origin_asn"`
NextHop string `json:"next_hop"` PeerIP string `json:"peer_ip"`
AnnouncedAt time.Time `json:"announced_at"` ASPath []int `json:"as_path"`
WithdrawnAt *time.Time `json:"withdrawn_at"` NextHop string `json:"next_hop"`
LastUpdated time.Time `json:"last_updated"`
// IPv4 range fields for fast lookups (nil for IPv6)
V4IPStart *uint32 `json:"v4_ip_start,omitempty"`
V4IPEnd *uint32 `json:"v4_ip_end,omitempty"`
}
// PrefixDistribution represents the distribution of prefixes by mask length
type PrefixDistribution struct {
MaskLength int `json:"mask_length"`
Count int `json:"count"`
}
// ASInfo represents AS information for an IP lookup
type ASInfo struct {
ASN int `json:"asn"`
Handle string `json:"handle"`
Description string `json:"description"`
Prefix string `json:"prefix"`
LastUpdated time.Time `json:"last_updated"`
Age string `json:"age"`
}
// LiveRouteDeletion represents parameters for deleting a live route
type LiveRouteDeletion struct {
Prefix string
OriginASN int
PeerIP string
}
// PeerUpdate represents parameters for updating a peer
type PeerUpdate struct {
PeerIP string
PeerASN int
MessageType string
Timestamp time.Time
} }

View File

@@ -1,6 +1,8 @@
CREATE TABLE IF NOT EXISTS asns ( CREATE TABLE IF NOT EXISTS asns (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
number INTEGER UNIQUE NOT NULL, number INTEGER UNIQUE NOT NULL,
handle TEXT,
description TEXT,
first_seen DATETIME NOT NULL, first_seen DATETIME NOT NULL,
last_seen DATETIME NOT NULL last_seen DATETIME NOT NULL
); );
@@ -27,15 +29,13 @@ CREATE TABLE IF NOT EXISTS announcements (
FOREIGN KEY (origin_asn_id) REFERENCES asns(id) FOREIGN KEY (origin_asn_id) REFERENCES asns(id)
); );
CREATE TABLE IF NOT EXISTS asn_peerings ( CREATE TABLE IF NOT EXISTS peerings (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
from_asn_id TEXT NOT NULL, as_a INTEGER NOT NULL,
to_asn_id TEXT NOT NULL, as_b INTEGER NOT NULL,
first_seen DATETIME NOT NULL, first_seen DATETIME NOT NULL,
last_seen DATETIME NOT NULL, last_seen DATETIME NOT NULL,
FOREIGN KEY (from_asn_id) REFERENCES asns(id), UNIQUE(as_a, as_b)
FOREIGN KEY (to_asn_id) REFERENCES asns(id),
UNIQUE(from_asn_id, to_asn_id)
); );
-- BGP peers that send us messages -- BGP peers that send us messages
@@ -48,65 +48,14 @@ CREATE TABLE IF NOT EXISTS bgp_peers (
last_message_type TEXT last_message_type TEXT
); );
-- Live routing table: current state of announced routes
CREATE TABLE IF NOT EXISTS live_routes (
id TEXT PRIMARY KEY,
prefix_id TEXT NOT NULL,
origin_asn_id TEXT NOT NULL,
peer_asn INTEGER NOT NULL,
next_hop TEXT,
announced_at DATETIME NOT NULL,
withdrawn_at DATETIME,
FOREIGN KEY (prefix_id) REFERENCES prefixes(id),
FOREIGN KEY (origin_asn_id) REFERENCES asns(id),
UNIQUE(prefix_id, origin_asn_id, peer_asn)
);
CREATE INDEX IF NOT EXISTS idx_prefixes_ip_version ON prefixes(ip_version); CREATE INDEX IF NOT EXISTS idx_prefixes_ip_version ON prefixes(ip_version);
CREATE INDEX IF NOT EXISTS idx_prefixes_version_prefix ON prefixes(ip_version, prefix); CREATE INDEX IF NOT EXISTS idx_prefixes_version_prefix ON prefixes(ip_version, prefix);
CREATE INDEX IF NOT EXISTS idx_announcements_timestamp ON announcements(timestamp); CREATE INDEX IF NOT EXISTS idx_announcements_timestamp ON announcements(timestamp);
CREATE INDEX IF NOT EXISTS idx_announcements_prefix_id ON announcements(prefix_id); CREATE INDEX IF NOT EXISTS idx_announcements_prefix_id ON announcements(prefix_id);
CREATE INDEX IF NOT EXISTS idx_announcements_asn_id ON announcements(asn_id); CREATE INDEX IF NOT EXISTS idx_announcements_asn_id ON announcements(asn_id);
CREATE INDEX IF NOT EXISTS idx_asn_peerings_from_asn ON asn_peerings(from_asn_id); CREATE INDEX IF NOT EXISTS idx_peerings_as_a ON peerings(as_a);
CREATE INDEX IF NOT EXISTS idx_asn_peerings_to_asn ON asn_peerings(to_asn_id); CREATE INDEX IF NOT EXISTS idx_peerings_as_b ON peerings(as_b);
CREATE INDEX IF NOT EXISTS idx_asn_peerings_lookup ON asn_peerings(from_asn_id, to_asn_id); CREATE INDEX IF NOT EXISTS idx_peerings_lookup ON peerings(as_a, as_b);
-- Indexes for live routes table
CREATE INDEX IF NOT EXISTS idx_live_routes_active
ON live_routes(prefix_id, origin_asn_id)
WHERE withdrawn_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_live_routes_origin
ON live_routes(origin_asn_id)
WHERE withdrawn_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_live_routes_prefix
ON live_routes(prefix_id)
WHERE withdrawn_at IS NULL;
-- Critical index for the most common query pattern
CREATE INDEX IF NOT EXISTS idx_live_routes_lookup
ON live_routes(prefix_id, origin_asn_id, peer_asn)
WHERE withdrawn_at IS NULL;
-- Index for withdrawal updates by prefix and peer
CREATE INDEX IF NOT EXISTS idx_live_routes_withdraw
ON live_routes(prefix_id, peer_asn)
WHERE withdrawn_at IS NULL;
-- Covering index for SELECT id queries (includes id in index)
CREATE INDEX IF NOT EXISTS idx_live_routes_covering
ON live_routes(prefix_id, origin_asn_id, peer_asn, id)
WHERE withdrawn_at IS NULL;
-- Index for UPDATE by id operations
CREATE INDEX IF NOT EXISTS idx_live_routes_id
ON live_routes(id);
-- Index for stats queries
CREATE INDEX IF NOT EXISTS idx_live_routes_stats
ON live_routes(withdrawn_at)
WHERE withdrawn_at IS NULL;
-- Additional indexes for prefixes table -- Additional indexes for prefixes table
CREATE INDEX IF NOT EXISTS idx_prefixes_prefix ON prefixes(prefix); CREATE INDEX IF NOT EXISTS idx_prefixes_prefix ON prefixes(prefix);
@@ -118,3 +67,30 @@ CREATE INDEX IF NOT EXISTS idx_asns_number ON asns(number);
CREATE INDEX IF NOT EXISTS idx_bgp_peers_asn ON bgp_peers(peer_asn); CREATE INDEX IF NOT EXISTS idx_bgp_peers_asn ON bgp_peers(peer_asn);
CREATE INDEX IF NOT EXISTS idx_bgp_peers_last_seen ON bgp_peers(last_seen); CREATE INDEX IF NOT EXISTS idx_bgp_peers_last_seen ON bgp_peers(last_seen);
CREATE INDEX IF NOT EXISTS idx_bgp_peers_ip ON bgp_peers(peer_ip); CREATE INDEX IF NOT EXISTS idx_bgp_peers_ip ON bgp_peers(peer_ip);
-- Live routing table maintained by PrefixHandler
CREATE TABLE IF NOT EXISTS live_routes (
id TEXT PRIMARY KEY,
prefix TEXT NOT NULL,
mask_length INTEGER NOT NULL, -- CIDR mask length (0-32 for IPv4, 0-128 for IPv6)
ip_version INTEGER NOT NULL, -- 4 or 6
origin_asn INTEGER NOT NULL,
peer_ip TEXT NOT NULL,
as_path TEXT NOT NULL, -- JSON array
next_hop TEXT NOT NULL,
last_updated DATETIME NOT NULL,
-- IPv4 range columns for fast lookups (NULL for IPv6)
v4_ip_start INTEGER, -- Start of IPv4 range as 32-bit unsigned int
v4_ip_end INTEGER, -- End of IPv4 range as 32-bit unsigned int
UNIQUE(prefix, origin_asn, peer_ip)
);
-- Indexes for live_routes table
CREATE INDEX IF NOT EXISTS idx_live_routes_prefix ON live_routes(prefix);
CREATE INDEX IF NOT EXISTS idx_live_routes_mask_length ON live_routes(mask_length);
CREATE INDEX IF NOT EXISTS idx_live_routes_ip_version_mask ON live_routes(ip_version, mask_length);
CREATE INDEX IF NOT EXISTS idx_live_routes_last_updated ON live_routes(last_updated);
-- Indexes for IPv4 range queries
CREATE INDEX IF NOT EXISTS idx_live_routes_ipv4_range ON live_routes(v4_ip_start, v4_ip_end) WHERE ip_version = 4;
-- Index to optimize COUNT(DISTINCT prefix) queries
CREATE INDEX IF NOT EXISTS idx_live_routes_ip_mask_prefix ON live_routes(ip_version, mask_length, prefix);

View File

@@ -3,14 +3,15 @@ package database
import ( import (
"context" "context"
"database/sql" "database/sql"
"log/slog"
"time" "time"
"git.eeqj.de/sneak/routewatch/internal/logger"
) )
const slowQueryThreshold = 50 * time.Millisecond const slowQueryThreshold = 50 * time.Millisecond
// logSlowQuery logs queries that take longer than slowQueryThreshold // logSlowQuery logs queries that take longer than slowQueryThreshold
func logSlowQuery(logger *slog.Logger, query string, start time.Time) { func logSlowQuery(logger *logger.Logger, query string, start time.Time) {
elapsed := time.Since(start) elapsed := time.Since(start)
if elapsed > slowQueryThreshold { if elapsed > slowQueryThreshold {
logger.Debug("Slow query", "query", query, "duration", elapsed) logger.Debug("Slow query", "query", query, "duration", elapsed)
@@ -18,6 +19,7 @@ func logSlowQuery(logger *slog.Logger, query string, start time.Time) {
} }
// queryRow wraps QueryRow with slow query logging // queryRow wraps QueryRow with slow query logging
// nolint:unused // kept for consistency with other query wrappers
func (d *Database) queryRow(query string, args ...interface{}) *sql.Row { func (d *Database) queryRow(query string, args ...interface{}) *sql.Row {
start := time.Now() start := time.Now()
defer logSlowQuery(d.logger, query, start) defer logSlowQuery(d.logger, query, start)
@@ -26,6 +28,7 @@ func (d *Database) queryRow(query string, args ...interface{}) *sql.Row {
} }
// query wraps Query with slow query logging // query wraps Query with slow query logging
// nolint:unused // kept for future use to ensure all queries go through slow query logging
func (d *Database) query(query string, args ...interface{}) (*sql.Rows, error) { func (d *Database) query(query string, args ...interface{}) (*sql.Rows, error) {
start := time.Now() start := time.Now()
defer logSlowQuery(d.logger, query, start) defer logSlowQuery(d.logger, query, start)
@@ -46,7 +49,7 @@ func (d *Database) exec(query string, args ...interface{}) error {
// loggingTx wraps sql.Tx to log slow queries // loggingTx wraps sql.Tx to log slow queries
type loggingTx struct { type loggingTx struct {
*sql.Tx *sql.Tx
logger *slog.Logger logger *logger.Logger
} }
// QueryRow wraps sql.Tx.QueryRow to log slow queries // QueryRow wraps sql.Tx.QueryRow to log slow queries

View File

@@ -10,11 +10,6 @@ func generateUUID() uuid.UUID {
return uuid.New() return uuid.New()
} }
const (
ipVersionV4 = 4
ipVersionV6 = 6
)
// detectIPVersion determines if a prefix is IPv4 (returns 4) or IPv6 (returns 6) // detectIPVersion determines if a prefix is IPv4 (returns 4) or IPv6 (returns 6)
func detectIPVersion(prefix string) int { func detectIPVersion(prefix string) int {
if strings.Contains(prefix, ":") { if strings.Contains(prefix, ":") {

150
internal/logger/logger.go Normal file
View File

@@ -0,0 +1,150 @@
// Package logger provides a structured logger with source location tracking
package logger
import (
"fmt"
"log/slog"
"os"
"path/filepath"
"runtime"
"strings"
"golang.org/x/term"
)
// Logger wraps slog.Logger to add source location information
type Logger struct {
*slog.Logger
}
// AsSlog returns the underlying slog.Logger
func (l *Logger) AsSlog() *slog.Logger {
return l.Logger
}
// New creates a new logger with appropriate handler based on environment
func New() *Logger {
level := slog.LevelInfo
if debug := os.Getenv("DEBUG"); strings.Contains(debug, "routewatch") {
level = slog.LevelDebug
}
opts := &slog.HandlerOptions{
Level: level,
}
var handler slog.Handler
if term.IsTerminal(int(os.Stdout.Fd())) {
// Terminal, use text
handler = slog.NewTextHandler(os.Stdout, opts)
} else {
// Not a terminal, use JSON
handler = slog.NewJSONHandler(os.Stdout, opts)
}
return &Logger{Logger: slog.New(handler)}
}
const sourceSkipLevel = 2 // Skip levels for source location tracking
// getSourceAttrs returns attributes for the calling source location
func getSourceAttrs() []slog.Attr {
pc, file, line, ok := runtime.Caller(sourceSkipLevel)
if !ok {
return nil
}
// Get just the filename without the full path
file = filepath.Base(file)
// Get the function name
fn := runtime.FuncForPC(pc)
var funcName string
if fn != nil {
funcName = filepath.Base(fn.Name())
}
attrs := []slog.Attr{
slog.String("source", fmt.Sprintf("%s:%d", file, line)),
}
if funcName != "" {
attrs = append(attrs, slog.String("func", funcName))
}
return attrs
}
// Debug logs at debug level with source location
func (l *Logger) Debug(msg string, args ...any) {
sourceAttrs := getSourceAttrs()
allArgs := make([]any, 0, len(args)+len(sourceAttrs)*2)
// Add source attributes first
for _, attr := range sourceAttrs {
allArgs = append(allArgs, attr)
}
// Add user args
allArgs = append(allArgs, args...)
l.Logger.Debug(msg, allArgs...)
}
// Info logs at info level with source location
func (l *Logger) Info(msg string, args ...any) {
sourceAttrs := getSourceAttrs()
allArgs := make([]any, 0, len(args)+len(sourceAttrs)*2)
// Add source attributes first
for _, attr := range sourceAttrs {
allArgs = append(allArgs, attr)
}
// Add user args
allArgs = append(allArgs, args...)
l.Logger.Info(msg, allArgs...)
}
// Warn logs at warn level with source location
func (l *Logger) Warn(msg string, args ...any) {
sourceAttrs := getSourceAttrs()
allArgs := make([]any, 0, len(args)+len(sourceAttrs)*2)
// Add source attributes first
for _, attr := range sourceAttrs {
allArgs = append(allArgs, attr)
}
// Add user args
allArgs = append(allArgs, args...)
l.Logger.Warn(msg, allArgs...)
}
// Error logs at error level with source location
func (l *Logger) Error(msg string, args ...any) {
sourceAttrs := getSourceAttrs()
allArgs := make([]any, 0, len(args)+len(sourceAttrs)*2)
// Add source attributes first
for _, attr := range sourceAttrs {
allArgs = append(allArgs, attr)
}
// Add user args
allArgs = append(allArgs, args...)
l.Logger.Error(msg, allArgs...)
}
// With returns a new logger with additional attributes
func (l *Logger) With(args ...any) *Logger {
return &Logger{Logger: l.Logger.With(args...)}
}
// WithGroup returns a new logger with a group prefix
func (l *Logger) WithGroup(name string) *Logger {
return &Logger{Logger: l.Logger.WithGroup(name)}
}

View File

@@ -21,6 +21,10 @@ type Tracker struct {
byteCounter metrics.Counter byteCounter metrics.Counter
messageRate metrics.Meter messageRate metrics.Meter
byteRate metrics.Meter byteRate metrics.Meter
// Route update metrics
ipv4UpdateRate metrics.Meter
ipv6UpdateRate metrics.Meter
} }
// New creates a new metrics tracker // New creates a new metrics tracker
@@ -33,6 +37,8 @@ func New() *Tracker {
byteCounter: metrics.NewCounter(), byteCounter: metrics.NewCounter(),
messageRate: metrics.NewMeter(), messageRate: metrics.NewMeter(),
byteRate: metrics.NewMeter(), byteRate: metrics.NewMeter(),
ipv4UpdateRate: metrics.NewMeter(),
ipv6UpdateRate: metrics.NewMeter(),
} }
} }
@@ -89,6 +95,24 @@ func (t *Tracker) GetStreamMetrics() StreamMetrics {
} }
} }
// RecordIPv4Update records an IPv4 route update
func (t *Tracker) RecordIPv4Update() {
t.ipv4UpdateRate.Mark(1)
}
// RecordIPv6Update records an IPv6 route update
func (t *Tracker) RecordIPv6Update() {
t.ipv6UpdateRate.Mark(1)
}
// GetRouteMetrics returns current route update metrics
func (t *Tracker) GetRouteMetrics() RouteMetrics {
return RouteMetrics{
IPv4UpdatesPerSec: t.ipv4UpdateRate.Rate1(),
IPv6UpdatesPerSec: t.ipv6UpdateRate.Rate1(),
}
}
// StreamMetrics contains streaming statistics // StreamMetrics contains streaming statistics
type StreamMetrics struct { type StreamMetrics struct {
TotalMessages uint64 TotalMessages uint64
@@ -98,3 +122,9 @@ type StreamMetrics struct {
MessagesPerSec float64 MessagesPerSec float64
BitsPerSec float64 BitsPerSec float64
} }
// RouteMetrics contains route update statistics
type RouteMetrics struct {
IPv4UpdatesPerSec float64
IPv6UpdatesPerSec float64
}

View File

@@ -4,12 +4,13 @@ package routewatch
import ( import (
"context" "context"
"log/slog" "fmt"
"os" "sync"
"strings"
"time" "time"
"git.eeqj.de/sneak/routewatch/internal/config"
"git.eeqj.de/sneak/routewatch/internal/database" "git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/metrics" "git.eeqj.de/sneak/routewatch/internal/metrics"
"git.eeqj.de/sneak/routewatch/internal/server" "git.eeqj.de/sneak/routewatch/internal/server"
"git.eeqj.de/sneak/routewatch/internal/streamer" "git.eeqj.de/sneak/routewatch/internal/streamer"
@@ -17,18 +18,6 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
) )
// Config contains runtime configuration for RouteWatch
type Config struct {
MaxRuntime time.Duration // Maximum runtime (0 = run forever)
}
// NewConfig provides default configuration
func NewConfig() Config {
return Config{
MaxRuntime: 0, // Run forever by default
}
}
// Dependencies contains all dependencies for RouteWatch // Dependencies contains all dependencies for RouteWatch
type Dependencies struct { type Dependencies struct {
fx.In fx.In
@@ -36,28 +25,38 @@ type Dependencies struct {
DB database.Store DB database.Store
Streamer *streamer.Streamer Streamer *streamer.Streamer
Server *server.Server Server *server.Server
Logger *slog.Logger Logger *logger.Logger
Config Config `optional:"true"` Config *config.Config
} }
// RouteWatch represents the main application instance // RouteWatch represents the main application instance
type RouteWatch struct { type RouteWatch struct {
db database.Store db database.Store
streamer *streamer.Streamer streamer *streamer.Streamer
server *server.Server server *server.Server
logger *slog.Logger logger *logger.Logger
maxRuntime time.Duration maxRuntime time.Duration
shutdown bool
mu sync.Mutex
config *config.Config
dbHandler *ASHandler
peerHandler *PeerHandler
prefixHandler *PrefixHandler
peeringHandler *PeeringHandler
} }
// New creates a new RouteWatch instance // New creates a new RouteWatch instance
func New(deps Dependencies) *RouteWatch { func New(deps Dependencies) *RouteWatch {
return &RouteWatch{ rw := &RouteWatch{
db: deps.DB, db: deps.DB,
streamer: deps.Streamer, streamer: deps.Streamer,
server: deps.Server, server: deps.Server,
logger: deps.Logger, logger: deps.Logger,
maxRuntime: deps.Config.MaxRuntime, maxRuntime: deps.Config.MaxRuntime,
config: deps.Config,
} }
return rw
} }
// Run starts the RouteWatch application // Run starts the RouteWatch application
@@ -73,12 +72,32 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
} }
// Register database handler to process BGP UPDATE messages // Register database handler to process BGP UPDATE messages
dbHandler := NewDatabaseHandler(rw.db, rw.logger) if rw.config.EnableBatchedDatabaseWrites {
rw.streamer.RegisterHandler(dbHandler) rw.logger.Info("Using batched database handlers for improved performance")
// ASHandler maintains the asns table
rw.dbHandler = NewASHandler(rw.db, rw.logger)
rw.streamer.RegisterHandler(rw.dbHandler)
// Register peer tracking handler to track all peers // PeerHandler maintains the bgp_peers table
peerHandler := NewPeerHandler(rw.db, rw.logger) rw.peerHandler = NewPeerHandler(rw.db, rw.logger)
rw.streamer.RegisterHandler(peerHandler) rw.streamer.RegisterHandler(rw.peerHandler)
// PrefixHandler maintains the prefixes and live_routes tables
rw.prefixHandler = NewPrefixHandler(rw.db, rw.logger)
rw.prefixHandler.SetMetricsTracker(rw.streamer.GetMetricsTracker())
rw.streamer.RegisterHandler(rw.prefixHandler)
// PeeringHandler maintains the asn_peerings table
rw.peeringHandler = NewPeeringHandler(rw.db, rw.logger)
rw.streamer.RegisterHandler(rw.peeringHandler)
} else {
// Non-batched handlers not implemented yet
rw.logger.Error("Non-batched handlers not implemented")
return fmt.Errorf("non-batched handlers not implemented")
}
// No longer need routing table handler - PrefixHandler maintains live_routes table
// Start streaming // Start streaming
if err := rw.streamer.Start(); err != nil { if err := rw.streamer.Start(); err != nil {
@@ -93,6 +112,38 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
// Wait for context cancellation // Wait for context cancellation
<-ctx.Done() <-ctx.Done()
return nil
}
// Shutdown performs graceful shutdown of all services
func (rw *RouteWatch) Shutdown() {
rw.mu.Lock()
if rw.shutdown {
rw.mu.Unlock()
return
}
rw.shutdown = true
rw.mu.Unlock()
// Stop batched handlers first to flush remaining batches
if rw.dbHandler != nil {
rw.logger.Info("Flushing database handler")
rw.dbHandler.Stop()
}
if rw.peerHandler != nil {
rw.logger.Info("Flushing peer handler")
rw.peerHandler.Stop()
}
if rw.prefixHandler != nil {
rw.logger.Info("Flushing prefix handler")
rw.prefixHandler.Stop()
}
if rw.peeringHandler != nil {
rw.logger.Info("Flushing peering handler")
rw.peeringHandler.Stop()
}
// Stop services // Stop services
rw.streamer.Stop() rw.streamer.Stop()
@@ -114,44 +165,17 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
"duration", time.Since(metrics.ConnectedSince), "duration", time.Since(metrics.ConnectedSince),
) )
return nil
}
// NewLogger creates a structured logger
func NewLogger() *slog.Logger {
level := slog.LevelInfo
if debug := os.Getenv("DEBUG"); strings.Contains(debug, "routewatch") {
level = slog.LevelDebug
}
opts := &slog.HandlerOptions{
Level: level,
}
var handler slog.Handler
if os.Stdout.Name() != "/dev/stdout" || os.Getenv("TERM") == "" {
// Not a terminal, use JSON
handler = slog.NewJSONHandler(os.Stdout, opts)
} else {
// Terminal, use text
handler = slog.NewTextHandler(os.Stdout, opts)
}
return slog.New(handler)
} }
// getModule provides all fx dependencies // getModule provides all fx dependencies
func getModule() fx.Option { func getModule() fx.Option {
return fx.Options( return fx.Options(
fx.Provide( fx.Provide(
NewLogger, logger.New,
NewConfig, config.New,
metrics.New, metrics.New,
database.New,
fx.Annotate( fx.Annotate(
func(db *database.Database) database.Store { database.New,
return db
},
fx.As(new(database.Store)), fx.As(new(database.Store)),
), ),
streamer.New, streamer.New,

View File

@@ -2,12 +2,15 @@ package routewatch
import ( import (
"context" "context"
"fmt"
"strings" "strings"
"sync" "sync"
"testing" "testing"
"time" "time"
"git.eeqj.de/sneak/routewatch/internal/config"
"git.eeqj.de/sneak/routewatch/internal/database" "git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/metrics" "git.eeqj.de/sneak/routewatch/internal/metrics"
"git.eeqj.de/sneak/routewatch/internal/server" "git.eeqj.de/sneak/routewatch/internal/server"
"git.eeqj.de/sneak/routewatch/internal/streamer" "git.eeqj.de/sneak/routewatch/internal/streamer"
@@ -116,11 +119,16 @@ func (m *mockStore) RecordAnnouncement(_ *database.Announcement) error {
} }
// RecordPeering mock implementation // RecordPeering mock implementation
func (m *mockStore) RecordPeering(fromASNID, toASNID string, _ time.Time) error { func (m *mockStore) RecordPeering(asA, asB int, _ time.Time) error {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
key := fromASNID + "_" + toASNID // Normalize
if asA > asB {
asA, asB = asB, asA
}
key := fmt.Sprintf("%d_%d", asA, asB)
if !m.Peerings[key] { if !m.Peerings[key] {
m.Peerings[key] = true m.Peerings[key] = true
m.PeeringCount++ m.PeeringCount++
@@ -129,35 +137,6 @@ func (m *mockStore) RecordPeering(fromASNID, toASNID string, _ time.Time) error
return nil return nil
} }
// UpdateLiveRoute mock implementation
func (m *mockStore) UpdateLiveRoute(prefixID, originASNID uuid.UUID, peerASN int, _ string, _ time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
key := prefixID.String() + "_" + originASNID.String() + "_" + string(rune(peerASN))
if !m.Routes[key] {
m.Routes[key] = true
m.RouteCount++
}
return nil
}
// WithdrawLiveRoute mock implementation
func (m *mockStore) WithdrawLiveRoute(_ uuid.UUID, _ int, _ time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
m.WithdrawalCount++
return nil
}
// GetActiveLiveRoutes mock implementation
func (m *mockStore) GetActiveLiveRoutes() ([]database.LiveRoute, error) {
return []database.LiveRoute{}, nil
}
// UpdatePeer mock implementation // UpdatePeer mock implementation
func (m *mockStore) UpdatePeer(peerIP string, peerASN int, messageType string, timestamp time.Time) error { func (m *mockStore) UpdatePeer(peerIP string, peerASN int, messageType string, timestamp time.Time) error {
// Simple mock - just return nil // Simple mock - just return nil
@@ -180,16 +159,173 @@ func (m *mockStore) GetStats() (database.Stats, error) {
IPv4Prefixes: m.IPv4Prefixes, IPv4Prefixes: m.IPv4Prefixes,
IPv6Prefixes: m.IPv6Prefixes, IPv6Prefixes: m.IPv6Prefixes,
Peerings: m.PeeringCount, Peerings: m.PeeringCount,
LiveRoutes: m.RouteCount, Peers: 10, // Mock peer count
}, nil }, nil
} }
// GetStatsContext returns statistics about the mock store with context support
func (m *mockStore) GetStatsContext(ctx context.Context) (database.Stats, error) {
return m.GetStats()
}
// UpsertLiveRoute mock implementation
func (m *mockStore) UpsertLiveRoute(route *database.LiveRoute) error {
// Simple mock - just return nil
return nil
}
// DeleteLiveRoute mock implementation
func (m *mockStore) DeleteLiveRoute(prefix string, originASN int, peerIP string) error {
// Simple mock - just return nil
return nil
}
// GetPrefixDistribution mock implementation
func (m *mockStore) GetPrefixDistribution() (ipv4 []database.PrefixDistribution, ipv6 []database.PrefixDistribution, err error) {
// Return empty distributions for now
return nil, nil, nil
}
// GetPrefixDistributionContext mock implementation with context support
func (m *mockStore) GetPrefixDistributionContext(ctx context.Context) (ipv4 []database.PrefixDistribution, ipv6 []database.PrefixDistribution, err error) {
return m.GetPrefixDistribution()
}
// GetLiveRouteCounts mock implementation
func (m *mockStore) GetLiveRouteCounts() (ipv4Count, ipv6Count int, err error) {
// Return mock counts
return m.RouteCount / 2, m.RouteCount / 2, nil
}
// GetLiveRouteCountsContext mock implementation with context support
func (m *mockStore) GetLiveRouteCountsContext(ctx context.Context) (ipv4Count, ipv6Count int, err error) {
return m.GetLiveRouteCounts()
}
// GetASInfoForIP mock implementation
func (m *mockStore) GetASInfoForIP(ip string) (*database.ASInfo, error) {
// Simple mock - return a test AS
now := time.Now()
return &database.ASInfo{
ASN: 15169,
Handle: "GOOGLE",
Description: "Google LLC",
Prefix: "8.8.8.0/24",
LastUpdated: now.Add(-5 * time.Minute),
Age: "5m0s",
}, nil
}
// GetASInfoForIPContext mock implementation with context support
func (m *mockStore) GetASInfoForIPContext(ctx context.Context, ip string) (*database.ASInfo, error) {
return m.GetASInfoForIP(ip)
}
// GetASDetails mock implementation
func (m *mockStore) GetASDetails(asn int) (*database.ASN, []database.LiveRoute, error) {
m.mu.Lock()
defer m.mu.Unlock()
// Check if ASN exists
if asnInfo, exists := m.ASNs[asn]; exists {
// Return empty prefixes for now
return asnInfo, []database.LiveRoute{}, nil
}
return nil, nil, database.ErrNoRoute
}
// GetASDetailsContext mock implementation with context support
func (m *mockStore) GetASDetailsContext(ctx context.Context, asn int) (*database.ASN, []database.LiveRoute, error) {
return m.GetASDetails(asn)
}
// GetPrefixDetails mock implementation
func (m *mockStore) GetPrefixDetails(prefix string) ([]database.LiveRoute, error) {
// Return empty routes for now
return []database.LiveRoute{}, nil
}
// GetPrefixDetailsContext mock implementation with context support
func (m *mockStore) GetPrefixDetailsContext(ctx context.Context, prefix string) ([]database.LiveRoute, error) {
return m.GetPrefixDetails(prefix)
}
func (m *mockStore) GetRandomPrefixesByLength(maskLength, ipVersion, limit int) ([]database.LiveRoute, error) {
// Return empty routes for now
return []database.LiveRoute{}, nil
}
// GetRandomPrefixesByLengthContext mock implementation with context support
func (m *mockStore) GetRandomPrefixesByLengthContext(ctx context.Context, maskLength, ipVersion, limit int) ([]database.LiveRoute, error) {
return m.GetRandomPrefixesByLength(maskLength, ipVersion, limit)
}
// UpsertLiveRouteBatch mock implementation
func (m *mockStore) UpsertLiveRouteBatch(routes []*database.LiveRoute) error {
m.mu.Lock()
defer m.mu.Unlock()
for _, route := range routes {
// Track prefix
if _, exists := m.Prefixes[route.Prefix]; !exists {
m.Prefixes[route.Prefix] = &database.Prefix{
ID: uuid.New(),
Prefix: route.Prefix,
IPVersion: route.IPVersion,
FirstSeen: route.LastUpdated,
LastSeen: route.LastUpdated,
}
m.PrefixCount++
if route.IPVersion == 4 {
m.IPv4Prefixes++
} else {
m.IPv6Prefixes++
}
}
m.RouteCount++
}
return nil
}
// DeleteLiveRouteBatch mock implementation
func (m *mockStore) DeleteLiveRouteBatch(deletions []database.LiveRouteDeletion) error {
// Simple mock - just return nil
return nil
}
// GetOrCreateASNBatch mock implementation
func (m *mockStore) GetOrCreateASNBatch(asns map[int]time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
for number, timestamp := range asns {
if _, exists := m.ASNs[number]; !exists {
m.ASNs[number] = &database.ASN{
ID: uuid.New(),
Number: number,
FirstSeen: timestamp,
LastSeen: timestamp,
}
m.ASNCount++
}
}
return nil
}
// UpdatePeerBatch mock implementation
func (m *mockStore) UpdatePeerBatch(peers map[string]database.PeerUpdate) error {
// Simple mock - just return nil
return nil
}
func TestRouteWatchLiveFeed(t *testing.T) { func TestRouteWatchLiveFeed(t *testing.T) {
// Create mock database // Create mock database
mockDB := newMockStore() mockDB := newMockStore()
defer mockDB.Close() defer mockDB.Close()
logger := NewLogger() logger := logger.New()
// Create metrics tracker // Create metrics tracker
metricsTracker := metrics.New() metricsTracker := metrics.New()
@@ -197,6 +333,13 @@ func TestRouteWatchLiveFeed(t *testing.T) {
// Create streamer // Create streamer
s := streamer.New(logger, metricsTracker) s := streamer.New(logger, metricsTracker)
// Create test config with empty state dir (no snapshot loading)
cfg := &config.Config{
StateDir: "",
MaxRuntime: 5 * time.Second,
EnableBatchedDatabaseWrites: true,
}
// Create server // Create server
srv := server.New(mockDB, s, logger) srv := server.New(mockDB, s, logger)
@@ -206,9 +349,7 @@ func TestRouteWatchLiveFeed(t *testing.T) {
Streamer: s, Streamer: s,
Server: srv, Server: srv,
Logger: logger, Logger: logger,
Config: Config{ Config: cfg,
MaxRuntime: 5 * time.Second,
},
} }
rw := New(deps) rw := New(deps)
@@ -221,6 +362,11 @@ func TestRouteWatchLiveFeed(t *testing.T) {
// Wait for the configured duration // Wait for the configured duration
time.Sleep(5 * time.Second) time.Sleep(5 * time.Second)
// Force peering processing for test
if rw.peeringHandler != nil {
rw.peeringHandler.ProcessPeeringsNow()
}
// Get statistics // Get statistics
stats, err := mockDB.GetStats() stats, err := mockDB.GetStats()
if err != nil { if err != nil {
@@ -242,8 +388,4 @@ func TestRouteWatchLiveFeed(t *testing.T) {
} }
t.Logf("Recorded %d AS peering relationships in 5 seconds", stats.Peerings) t.Logf("Recorded %d AS peering relationships in 5 seconds", stats.Peerings)
if stats.LiveRoutes == 0 {
t.Error("Expected to have some active routes")
}
t.Logf("Active routes: %d", stats.LiveRoutes)
} }

View File

@@ -1,12 +1,3 @@
package routewatch package routewatch
import ( // Tests for routewatch package are in app_integration_test.go
"testing"
)
func TestNewLogger(t *testing.T) {
logger := NewLogger()
if logger == nil {
t.Fatal("NewLogger returned nil")
}
}

View File

@@ -0,0 +1,163 @@
package routewatch
import (
"sync"
"time"
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/ristypes"
)
const (
// asHandlerQueueSize is the queue capacity for ASN operations
// DO NOT set this higher than 100000 without explicit instructions
asHandlerQueueSize = 100000
// asnBatchSize is the number of ASN operations to batch together
asnBatchSize = 30000
// asnBatchTimeout is the maximum time to wait before flushing a batch
// DO NOT reduce this timeout - larger batches are more efficient
asnBatchTimeout = 2 * time.Second
)
// ASHandler handles ASN information from BGP messages using batched operations
type ASHandler struct {
db database.Store
logger *logger.Logger
// Batching
mu sync.Mutex
batch []asnOp
lastFlush time.Time
stopCh chan struct{}
wg sync.WaitGroup
}
type asnOp struct {
number int
timestamp time.Time
}
// NewASHandler creates a new batched ASN handler
func NewASHandler(db database.Store, logger *logger.Logger) *ASHandler {
h := &ASHandler{
db: db,
logger: logger,
batch: make([]asnOp, 0, asnBatchSize),
lastFlush: time.Now(),
stopCh: make(chan struct{}),
}
// Start the flush timer goroutine
h.wg.Add(1)
go h.flushLoop()
return h
}
// WantsMessage returns true if this handler wants to process messages of the given type
func (h *ASHandler) WantsMessage(messageType string) bool {
// We only care about UPDATE messages for the database
return messageType == "UPDATE"
}
// QueueCapacity returns the desired queue capacity for this handler
func (h *ASHandler) QueueCapacity() int {
// Batching allows us to use a larger queue
return asHandlerQueueSize
}
// HandleMessage processes a RIS message and queues database operations
func (h *ASHandler) HandleMessage(msg *ristypes.RISMessage) {
// Use the pre-parsed timestamp
timestamp := msg.ParsedTimestamp
// Get origin ASN from path (last element)
var originASN int
if len(msg.Path) > 0 {
originASN = msg.Path[len(msg.Path)-1]
}
h.mu.Lock()
defer h.mu.Unlock()
// Queue origin ASN operation
if originASN > 0 {
h.batch = append(h.batch, asnOp{
number: originASN,
timestamp: timestamp,
})
}
// Also track all ASNs in the path
for _, asn := range msg.Path {
if asn > 0 {
h.batch = append(h.batch, asnOp{
number: asn,
timestamp: timestamp,
})
}
}
// Check if we need to flush
if len(h.batch) >= asnBatchSize {
h.flushBatchLocked()
}
}
// flushLoop runs in a goroutine and periodically flushes batches
func (h *ASHandler) flushLoop() {
defer h.wg.Done()
ticker := time.NewTicker(asnBatchTimeout)
defer ticker.Stop()
for {
select {
case <-ticker.C:
h.mu.Lock()
if time.Since(h.lastFlush) >= asnBatchTimeout {
h.flushBatchLocked()
}
h.mu.Unlock()
case <-h.stopCh:
// Final flush
h.mu.Lock()
h.flushBatchLocked()
h.mu.Unlock()
return
}
}
}
// flushBatchLocked flushes the ASN batch to the database (must be called with mutex held)
func (h *ASHandler) flushBatchLocked() {
if len(h.batch) == 0 {
return
}
// Process ASNs first (deduped)
asnMap := make(map[int]time.Time)
for _, op := range h.batch {
if existing, ok := asnMap[op.number]; !ok || op.timestamp.After(existing) {
asnMap[op.number] = op.timestamp
}
}
// Process all ASNs in a single batch transaction
if err := h.db.GetOrCreateASNBatch(asnMap); err != nil {
h.logger.Error("Failed to process ASN batch", "error", err, "count", len(asnMap))
}
// Clear batch
h.batch = h.batch[:0]
h.lastFlush = time.Now()
}
// Stop gracefully stops the handler and flushes remaining batches
func (h *ASHandler) Stop() {
close(h.stopCh)
h.wg.Wait()
}

View File

@@ -2,35 +2,81 @@ package routewatch
import ( import (
"context" "context"
"log/slog"
"os" "os"
"os/signal" "os/signal"
"runtime"
"strings"
"syscall" "syscall"
"time"
"git.eeqj.de/sneak/routewatch/internal/logger"
"go.uber.org/fx" "go.uber.org/fx"
) )
const (
// shutdownTimeout is the maximum time allowed for graceful shutdown
shutdownTimeout = 60 * time.Second
// debugInterval is how often to log debug stats
debugInterval = 60 * time.Second
// bytesPerMB is bytes per megabyte
bytesPerMB = 1024 * 1024
)
// logDebugStats logs goroutine count and memory usage
func logDebugStats(logger *logger.Logger) {
// Only run if DEBUG env var contains "routewatch"
debugEnv := os.Getenv("DEBUG")
if !strings.Contains(debugEnv, "routewatch") {
return
}
ticker := time.NewTicker(debugInterval)
defer ticker.Stop()
for range ticker.C {
var m runtime.MemStats
runtime.ReadMemStats(&m)
logger.Debug("System stats",
"goroutines", runtime.NumGoroutine(),
"alloc_mb", m.Alloc/bytesPerMB,
"total_alloc_mb", m.TotalAlloc/bytesPerMB,
"sys_mb", m.Sys/bytesPerMB,
"num_gc", m.NumGC,
"heap_alloc_mb", m.HeapAlloc/bytesPerMB,
"heap_sys_mb", m.HeapSys/bytesPerMB,
"heap_idle_mb", m.HeapIdle/bytesPerMB,
"heap_inuse_mb", m.HeapInuse/bytesPerMB,
"heap_released_mb", m.HeapReleased/bytesPerMB,
"stack_inuse_mb", m.StackInuse/bytesPerMB,
)
}
}
// CLIEntry is the main entry point for the CLI // CLIEntry is the main entry point for the CLI
func CLIEntry() { func CLIEntry() {
app := fx.New( app := fx.New(
getModule(), getModule(),
fx.Invoke(func(lc fx.Lifecycle, rw *RouteWatch, logger *slog.Logger) { fx.StopTimeout(shutdownTimeout), // Allow 60 seconds for graceful shutdown
fx.Invoke(func(lc fx.Lifecycle, rw *RouteWatch, logger *logger.Logger, shutdowner fx.Shutdowner) {
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(_ context.Context) error { OnStart: func(ctx context.Context) error {
// Start debug stats logging
go logDebugStats(logger)
// Handle shutdown signals
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)
go func() { go func() {
ctx, cancel := context.WithCancel(context.Background()) <-sigCh
defer cancel() logger.Info("Received shutdown signal")
if err := shutdowner.Shutdown(); err != nil {
// Handle shutdown signals logger.Error("Failed to shutdown gracefully", "error", err)
sigCh := make(chan os.Signal, 1) }
signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM) }()
go func() {
<-sigCh
logger.Info("Received shutdown signal")
cancel()
}()
go func() {
if err := rw.Run(ctx); err != nil { if err := rw.Run(ctx); err != nil {
logger.Error("RouteWatch error", "error", err) logger.Error("RouteWatch error", "error", err)
} }
@@ -40,6 +86,7 @@ func CLIEntry() {
}, },
OnStop: func(_ context.Context) error { OnStop: func(_ context.Context) error {
logger.Info("Shutting down RouteWatch") logger.Info("Shutting down RouteWatch")
rw.Shutdown()
return nil return nil
}, },

View File

@@ -1,144 +0,0 @@
package routewatch
import (
"log/slog"
"strconv"
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/ristypes"
)
// DatabaseHandler handles BGP messages and stores them in the database
type DatabaseHandler struct {
db database.Store
logger *slog.Logger
}
// NewDatabaseHandler creates a new database handler
func NewDatabaseHandler(db database.Store, logger *slog.Logger) *DatabaseHandler {
return &DatabaseHandler{
db: db,
logger: logger,
}
}
// WantsMessage returns true if this handler wants to process messages of the given type
func (h *DatabaseHandler) WantsMessage(messageType string) bool {
// We only care about UPDATE messages for the database
return messageType == "UPDATE"
}
// HandleMessage processes a RIS message and updates the database
func (h *DatabaseHandler) HandleMessage(msg *ristypes.RISMessage) {
// Use the pre-parsed timestamp
timestamp := msg.ParsedTimestamp
// Parse peer ASN
peerASN, err := strconv.Atoi(msg.PeerASN)
if err != nil {
h.logger.Error("Failed to parse peer ASN", "peer_asn", msg.PeerASN, "error", err)
return
}
// Get origin ASN from path (last element)
var originASN int
if len(msg.Path) > 0 {
originASN = msg.Path[len(msg.Path)-1]
}
// Process announcements
for _, announcement := range msg.Announcements {
for _, prefix := range announcement.Prefixes {
// Get or create prefix
p, err := h.db.GetOrCreatePrefix(prefix, timestamp)
if err != nil {
h.logger.Error("Failed to get/create prefix", "prefix", prefix, "error", err)
continue
}
// Get or create origin ASN
asn, err := h.db.GetOrCreateASN(originASN, timestamp)
if err != nil {
h.logger.Error("Failed to get/create ASN", "asn", originASN, "error", err)
continue
}
// Update live route
err = h.db.UpdateLiveRoute(
p.ID,
asn.ID,
peerASN,
announcement.NextHop,
timestamp,
)
if err != nil {
h.logger.Error("Failed to update live route",
"prefix", prefix,
"origin_asn", originASN,
"peer_asn", peerASN,
"error", err,
)
}
// TODO: Record the announcement in the announcements table
// Process AS path to update peerings
if len(msg.Path) > 1 {
for i := range len(msg.Path) - 1 {
fromASN := msg.Path[i]
toASN := msg.Path[i+1]
// Get or create both ASNs
fromAS, err := h.db.GetOrCreateASN(fromASN, timestamp)
if err != nil {
h.logger.Error("Failed to get/create from ASN", "asn", fromASN, "error", err)
continue
}
toAS, err := h.db.GetOrCreateASN(toASN, timestamp)
if err != nil {
h.logger.Error("Failed to get/create to ASN", "asn", toASN, "error", err)
continue
}
// Record the peering
err = h.db.RecordPeering(fromAS.ID.String(), toAS.ID.String(), timestamp)
if err != nil {
h.logger.Error("Failed to record peering",
"from_asn", fromASN,
"to_asn", toASN,
"error", err,
)
}
}
}
}
}
// Process withdrawals
for _, prefix := range msg.Withdrawals {
// Get prefix
p, err := h.db.GetOrCreatePrefix(prefix, timestamp)
if err != nil {
h.logger.Error("Failed to get prefix for withdrawal", "prefix", prefix, "error", err)
continue
}
// Withdraw the route
err = h.db.WithdrawLiveRoute(p.ID, peerASN, timestamp)
if err != nil {
h.logger.Error("Failed to withdraw route",
"prefix", prefix,
"peer_asn", peerASN,
"error", err,
)
}
// TODO: Record the withdrawal in the withdrawals table
}
}

View File

@@ -1,19 +1,23 @@
package routewatch package routewatch
import ( import (
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/ristypes" "git.eeqj.de/sneak/routewatch/internal/ristypes"
"log/slog"
) )
// SimpleHandler is a basic implementation of streamer.MessageHandler // SimpleHandler is a basic implementation of streamer.MessageHandler
type SimpleHandler struct { type SimpleHandler struct {
logger *slog.Logger logger *logger.Logger
messageTypes []string messageTypes []string
callback func(*ristypes.RISMessage) callback func(*ristypes.RISMessage)
} }
// NewSimpleHandler creates a handler that accepts specific message types // NewSimpleHandler creates a handler that accepts specific message types
func NewSimpleHandler(logger *slog.Logger, messageTypes []string, callback func(*ristypes.RISMessage)) *SimpleHandler { func NewSimpleHandler(
logger *logger.Logger,
messageTypes []string,
callback func(*ristypes.RISMessage),
) *SimpleHandler {
return &SimpleHandler{ return &SimpleHandler{
logger: logger, logger: logger,
messageTypes: messageTypes, messageTypes: messageTypes,

View File

@@ -1,25 +1,61 @@
package routewatch package routewatch
import ( import (
"log/slog"
"strconv" "strconv"
"sync"
"time"
"git.eeqj.de/sneak/routewatch/internal/database" "git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/ristypes" "git.eeqj.de/sneak/routewatch/internal/ristypes"
) )
// PeerHandler tracks BGP peers from all message types const (
// peerHandlerQueueSize is the queue capacity for peer tracking operations
peerHandlerQueueSize = 100000
// peerBatchSize is the number of peer updates to batch together
peerBatchSize = 10000
// peerBatchTimeout is the maximum time to wait before flushing a batch
peerBatchTimeout = 2 * time.Second
)
// PeerHandler tracks BGP peers from all message types using batched operations
type PeerHandler struct { type PeerHandler struct {
db database.Store db database.Store
logger *slog.Logger logger *logger.Logger
// Batching
mu sync.Mutex
peerBatch []peerUpdate
lastFlush time.Time
stopCh chan struct{}
wg sync.WaitGroup
} }
// NewPeerHandler creates a new peer tracking handler type peerUpdate struct {
func NewPeerHandler(db database.Store, logger *slog.Logger) *PeerHandler { peerIP string
return &PeerHandler{ peerASN int
db: db, messageType string
logger: logger, timestamp time.Time
}
// NewPeerHandler creates a new batched peer tracking handler
func NewPeerHandler(db database.Store, logger *logger.Logger) *PeerHandler {
h := &PeerHandler{
db: db,
logger: logger,
peerBatch: make([]peerUpdate, 0, peerBatchSize),
lastFlush: time.Now(),
stopCh: make(chan struct{}),
} }
// Start the flush timer goroutine
h.wg.Add(1)
go h.flushLoop()
return h
} }
// WantsMessage returns true for all message types since we track peers from all messages // WantsMessage returns true for all message types since we track peers from all messages
@@ -27,6 +63,12 @@ func (h *PeerHandler) WantsMessage(_ string) bool {
return true return true
} }
// QueueCapacity returns the desired queue capacity for this handler
func (h *PeerHandler) QueueCapacity() int {
// Batching allows us to use a larger queue
return peerHandlerQueueSize
}
// HandleMessage processes a message to track peer information // HandleMessage processes a message to track peer information
func (h *PeerHandler) HandleMessage(msg *ristypes.RISMessage) { func (h *PeerHandler) HandleMessage(msg *ristypes.RISMessage) {
// Parse peer ASN from string // Parse peer ASN from string
@@ -37,13 +79,85 @@ func (h *PeerHandler) HandleMessage(msg *ristypes.RISMessage) {
} }
} }
// Update peer in database h.mu.Lock()
if err := h.db.UpdatePeer(msg.Peer, peerASN, msg.Type, msg.ParsedTimestamp); err != nil { defer h.mu.Unlock()
h.logger.Error("Failed to update peer",
"peer", msg.Peer, // Add to batch
"peer_asn", peerASN, h.peerBatch = append(h.peerBatch, peerUpdate{
"message_type", msg.Type, peerIP: msg.Peer,
"error", err, peerASN: peerASN,
) messageType: msg.Type,
timestamp: msg.ParsedTimestamp,
})
// Check if we need to flush
if len(h.peerBatch) >= peerBatchSize {
h.flushBatchLocked()
} }
} }
// flushLoop runs in a goroutine and periodically flushes batches
func (h *PeerHandler) flushLoop() {
defer h.wg.Done()
ticker := time.NewTicker(peerBatchTimeout)
defer ticker.Stop()
for {
select {
case <-ticker.C:
h.mu.Lock()
if time.Since(h.lastFlush) >= peerBatchTimeout {
h.flushBatchLocked()
}
h.mu.Unlock()
case <-h.stopCh:
// Final flush
h.mu.Lock()
h.flushBatchLocked()
h.mu.Unlock()
return
}
}
}
// flushBatchLocked flushes the peer batch to the database (must be called with mutex held)
func (h *PeerHandler) flushBatchLocked() {
if len(h.peerBatch) == 0 {
return
}
// Deduplicate by peer IP, keeping the latest update for each peer
peerMap := make(map[string]peerUpdate)
for _, update := range h.peerBatch {
if existing, ok := peerMap[update.peerIP]; !ok || update.timestamp.After(existing.timestamp) {
peerMap[update.peerIP] = update
}
}
// Convert to database format
dbPeerMap := make(map[string]database.PeerUpdate)
for peerIP, update := range peerMap {
dbPeerMap[peerIP] = database.PeerUpdate{
PeerIP: update.peerIP,
PeerASN: update.peerASN,
MessageType: update.messageType,
Timestamp: update.timestamp,
}
}
// Process all peers in a single batch transaction
if err := h.db.UpdatePeerBatch(dbPeerMap); err != nil {
h.logger.Error("Failed to process peer batch", "error", err, "count", len(dbPeerMap))
}
// Clear batch
h.peerBatch = h.peerBatch[:0]
h.lastFlush = time.Now()
}
// Stop gracefully stops the handler and flushes remaining batches
func (h *PeerHandler) Stop() {
close(h.stopCh)
h.wg.Wait()
}

View File

@@ -0,0 +1,230 @@
package routewatch
import (
"encoding/json"
"sync"
"time"
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/ristypes"
)
const (
// peeringHandlerQueueSize is the queue capacity for peering operations
peeringHandlerQueueSize = 100000
// minPathLengthForPeering is the minimum AS path length to extract peerings
minPathLengthForPeering = 2
// pathExpirationTime is how long to keep AS paths in memory
pathExpirationTime = 30 * time.Minute
// peeringProcessInterval is how often to process AS paths into peerings
peeringProcessInterval = 2 * time.Minute
// pathPruneInterval is how often to prune old AS paths
pathPruneInterval = 5 * time.Minute
)
// PeeringHandler handles AS peering relationships from BGP path data
type PeeringHandler struct {
db database.Store
logger *logger.Logger
// In-memory AS path tracking
mu sync.RWMutex
asPaths map[string]time.Time // key is JSON-encoded AS path
stopCh chan struct{}
}
// NewPeeringHandler creates a new batched peering handler
func NewPeeringHandler(db database.Store, logger *logger.Logger) *PeeringHandler {
h := &PeeringHandler{
db: db,
logger: logger,
asPaths: make(map[string]time.Time),
stopCh: make(chan struct{}),
}
// Start the periodic processing goroutines
go h.processLoop()
go h.pruneLoop()
return h
}
// WantsMessage returns true if this handler wants to process messages of the given type
func (h *PeeringHandler) WantsMessage(messageType string) bool {
// We only care about UPDATE messages that have AS paths
return messageType == "UPDATE"
}
// QueueCapacity returns the desired queue capacity for this handler
func (h *PeeringHandler) QueueCapacity() int {
return peeringHandlerQueueSize
}
// HandleMessage processes a message to extract AS paths
func (h *PeeringHandler) HandleMessage(msg *ristypes.RISMessage) {
// Skip if no AS path or only one AS
if len(msg.Path) < minPathLengthForPeering {
return
}
timestamp := msg.ParsedTimestamp
// Encode AS path as JSON for use as map key
pathJSON, err := json.Marshal(msg.Path)
if err != nil {
h.logger.Error("Failed to encode AS path", "error", err)
return
}
h.mu.Lock()
h.asPaths[string(pathJSON)] = timestamp
h.mu.Unlock()
}
// processLoop runs periodically to process AS paths into peerings
func (h *PeeringHandler) processLoop() {
ticker := time.NewTicker(peeringProcessInterval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
h.processPeerings()
case <-h.stopCh:
// Final processing
h.processPeerings()
return
}
}
}
// pruneLoop runs periodically to remove old AS paths
func (h *PeeringHandler) pruneLoop() {
ticker := time.NewTicker(pathPruneInterval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
h.prunePaths()
case <-h.stopCh:
return
}
}
}
// prunePaths removes AS paths older than pathExpirationTime
func (h *PeeringHandler) prunePaths() {
cutoff := time.Now().Add(-pathExpirationTime)
var removed int
h.mu.Lock()
for pathKey, timestamp := range h.asPaths {
if timestamp.Before(cutoff) {
delete(h.asPaths, pathKey)
removed++
}
}
pathCount := len(h.asPaths)
h.mu.Unlock()
if removed > 0 {
h.logger.Debug("Pruned old AS paths", "removed", removed, "remaining", pathCount)
}
}
// ProcessPeeringsNow forces immediate processing of peerings (for testing)
func (h *PeeringHandler) ProcessPeeringsNow() {
h.processPeerings()
}
// processPeerings extracts peerings from AS paths and writes to database
func (h *PeeringHandler) processPeerings() {
// Take a snapshot of current AS paths
h.mu.RLock()
pathsCopy := make(map[string]time.Time, len(h.asPaths))
for k, v := range h.asPaths {
pathsCopy[k] = v
}
h.mu.RUnlock()
if len(pathsCopy) == 0 {
return
}
// Extract unique peerings from AS paths
type peeringKey struct {
low, high int
}
peerings := make(map[peeringKey]time.Time)
for pathJSON, timestamp := range pathsCopy {
var path []int
if err := json.Unmarshal([]byte(pathJSON), &path); err != nil {
h.logger.Error("Failed to decode AS path", "error", err)
continue
}
// Extract peerings from path
for i := range len(path) - 1 {
asn1 := path[i]
asn2 := path[i+1]
// Skip invalid ASNs
if asn1 <= 0 || asn2 <= 0 || asn1 == asn2 {
continue
}
// Normalize: lower AS number first
low, high := asn1, asn2
if low > high {
low, high = high, low
}
key := peeringKey{low: low, high: high}
// Update timestamp if this is newer
if existing, ok := peerings[key]; !ok || timestamp.After(existing) {
peerings[key] = timestamp
}
}
}
// Record peerings in database
start := time.Now()
successCount := 0
for key, ts := range peerings {
err := h.db.RecordPeering(key.low, key.high, ts)
if err != nil {
h.logger.Error("Failed to record peering",
"as_a", key.low,
"as_b", key.high,
"error", err,
)
} else {
successCount++
}
}
h.logger.Info("Processed AS peerings",
"paths", len(pathsCopy),
"unique_peerings", len(peerings),
"success", successCount,
"duration", time.Since(start),
)
}
// Stop gracefully stops the handler and processes remaining peerings
func (h *PeeringHandler) Stop() {
close(h.stopCh)
// Process any remaining peerings synchronously
h.processPeerings()
}

View File

@@ -0,0 +1,473 @@
package routewatch
import (
"net"
"strings"
"sync"
"time"
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/metrics"
"git.eeqj.de/sneak/routewatch/internal/ristypes"
"github.com/google/uuid"
)
const (
// prefixHandlerQueueSize is the queue capacity for prefix tracking operations
// DO NOT set this higher than 100000 without explicit instructions
prefixHandlerQueueSize = 100000
// prefixBatchSize is the number of prefix updates to batch together
prefixBatchSize = 20000
// prefixBatchTimeout is the maximum time to wait before flushing a batch
// DO NOT reduce this timeout - larger batches are more efficient
prefixBatchTimeout = 1 * time.Second
// IP version constants
ipv4Version = 4
ipv6Version = 6
)
// PrefixHandler tracks BGP prefixes and maintains a live routing table in the database.
// Routes are added on announcement and deleted on withdrawal.
type PrefixHandler struct {
db database.Store
logger *logger.Logger
metrics *metrics.Tracker
// Batching
mu sync.Mutex
batch []prefixUpdate
lastFlush time.Time
stopCh chan struct{}
wg sync.WaitGroup
}
type prefixUpdate struct {
prefix string
originASN int
peer string
messageType string
timestamp time.Time
path []int
}
// NewPrefixHandler creates a new batched prefix tracking handler
func NewPrefixHandler(db database.Store, logger *logger.Logger) *PrefixHandler {
h := &PrefixHandler{
db: db,
logger: logger,
batch: make([]prefixUpdate, 0, prefixBatchSize),
lastFlush: time.Now(),
stopCh: make(chan struct{}),
}
// Start the flush timer goroutine
h.wg.Add(1)
go h.flushLoop()
return h
}
// SetMetricsTracker sets the metrics tracker for recording route updates
func (h *PrefixHandler) SetMetricsTracker(metrics *metrics.Tracker) {
h.metrics = metrics
}
// WantsMessage returns true if this handler wants to process messages of the given type
func (h *PrefixHandler) WantsMessage(messageType string) bool {
// We only care about UPDATE messages for the routing table
return messageType == "UPDATE"
}
// QueueCapacity returns the desired queue capacity for this handler
func (h *PrefixHandler) QueueCapacity() int {
// Batching allows us to use a larger queue
return prefixHandlerQueueSize
}
// HandleMessage processes a message to track prefix information
func (h *PrefixHandler) HandleMessage(msg *ristypes.RISMessage) {
// Use the pre-parsed timestamp
timestamp := msg.ParsedTimestamp
// Get origin ASN from path (last element)
var originASN int
if len(msg.Path) > 0 {
originASN = msg.Path[len(msg.Path)-1]
}
h.mu.Lock()
defer h.mu.Unlock()
// Process announcements
for _, announcement := range msg.Announcements {
for _, prefix := range announcement.Prefixes {
h.batch = append(h.batch, prefixUpdate{
prefix: prefix,
originASN: originASN,
peer: msg.Peer,
messageType: "announcement",
timestamp: timestamp,
path: msg.Path,
})
}
}
// Process withdrawals
for _, prefix := range msg.Withdrawals {
h.batch = append(h.batch, prefixUpdate{
prefix: prefix,
originASN: originASN, // Use the originASN from path if available
peer: msg.Peer,
messageType: "withdrawal",
timestamp: timestamp,
path: msg.Path,
})
}
// Check if we need to flush
if len(h.batch) >= prefixBatchSize {
h.flushBatchLocked()
}
}
// flushLoop runs in a goroutine and periodically flushes batches
func (h *PrefixHandler) flushLoop() {
defer h.wg.Done()
ticker := time.NewTicker(prefixBatchTimeout)
defer ticker.Stop()
for {
select {
case <-ticker.C:
h.mu.Lock()
if time.Since(h.lastFlush) >= prefixBatchTimeout {
h.flushBatchLocked()
}
h.mu.Unlock()
case <-h.stopCh:
// Final flush
h.mu.Lock()
h.flushBatchLocked()
h.mu.Unlock()
return
}
}
}
// flushBatchLocked flushes the prefix batch to the database (must be called with mutex held)
func (h *PrefixHandler) flushBatchLocked() {
if len(h.batch) == 0 {
return
}
startTime := time.Now()
batchSize := len(h.batch)
// Group updates by prefix to deduplicate
// For each prefix, keep the latest update
prefixMap := make(map[string]prefixUpdate)
for _, update := range h.batch {
key := update.prefix
if existing, ok := prefixMap[key]; !ok || update.timestamp.After(existing.timestamp) {
prefixMap[key] = update
}
}
// Collect routes to upsert and delete
var routesToUpsert []*database.LiveRoute
var routesToDelete []database.LiveRouteDeletion
// Skip the prefix table updates entirely - just update live_routes
// The prefix table is not critical for routing lookups
for _, update := range prefixMap {
if update.messageType == "announcement" && update.originASN > 0 {
// Create live route for batch upsert
route := h.createLiveRoute(update)
if route != nil {
routesToUpsert = append(routesToUpsert, route)
}
} else if update.messageType == "withdrawal" {
// Create deletion record for batch delete
routesToDelete = append(routesToDelete, database.LiveRouteDeletion{
Prefix: update.prefix,
OriginASN: update.originASN,
PeerIP: update.peer,
})
}
}
// Process batch operations
successCount := 0
if len(routesToUpsert) > 0 {
if err := h.db.UpsertLiveRouteBatch(routesToUpsert); err != nil {
h.logger.Error("Failed to upsert route batch", "error", err, "count", len(routesToUpsert))
} else {
successCount += len(routesToUpsert)
}
}
if len(routesToDelete) > 0 {
if err := h.db.DeleteLiveRouteBatch(routesToDelete); err != nil {
h.logger.Error("Failed to delete route batch", "error", err, "count", len(routesToDelete))
} else {
successCount += len(routesToDelete)
}
}
elapsed := time.Since(startTime)
h.logger.Debug("Flushed prefix batch",
"batch_size", batchSize,
"unique_prefixes", len(prefixMap),
"success", successCount,
"duration_ms", elapsed.Milliseconds(),
)
// Clear batch
h.batch = h.batch[:0]
h.lastFlush = time.Now()
}
// parseCIDR extracts the mask length and IP version from a prefix string
func parseCIDR(prefix string) (maskLength int, ipVersion int, err error) {
_, ipNet, err := net.ParseCIDR(prefix)
if err != nil {
return 0, 0, err
}
ones, _ := ipNet.Mask.Size()
if strings.Contains(prefix, ":") {
return ones, ipv6Version, nil
}
return ones, ipv4Version, nil
}
// processAnnouncement handles storing an announcement in the database
// nolint:unused // kept for potential future use
func (h *PrefixHandler) processAnnouncement(_ *database.Prefix, update prefixUpdate) {
// Parse CIDR to get mask length
maskLength, ipVersion, err := parseCIDR(update.prefix)
if err != nil {
h.logger.Error("Failed to parse CIDR",
"prefix", update.prefix,
"error", err,
)
return
}
// Track route update metrics
if h.metrics != nil {
if ipVersion == ipv4Version {
h.metrics.RecordIPv4Update()
} else {
h.metrics.RecordIPv6Update()
}
}
// Create live route record
liveRoute := &database.LiveRoute{
ID: uuid.New(),
Prefix: update.prefix,
MaskLength: maskLength,
IPVersion: ipVersion,
OriginASN: update.originASN,
PeerIP: update.peer,
ASPath: update.path,
NextHop: update.peer, // Using peer as next hop
LastUpdated: update.timestamp,
}
// For IPv4, calculate the IP range
if ipVersion == ipv4Version {
start, end, err := database.CalculateIPv4Range(update.prefix)
if err == nil {
liveRoute.V4IPStart = &start
liveRoute.V4IPEnd = &end
} else {
h.logger.Error("Failed to calculate IPv4 range",
"prefix", update.prefix,
"error", err,
)
}
}
if err := h.db.UpsertLiveRoute(liveRoute); err != nil {
h.logger.Error("Failed to upsert live route",
"prefix", update.prefix,
"error", err,
)
}
}
// createLiveRoute creates a LiveRoute from a prefix update
func (h *PrefixHandler) createLiveRoute(update prefixUpdate) *database.LiveRoute {
// Parse CIDR to get mask length
maskLength, ipVersion, err := parseCIDR(update.prefix)
if err != nil {
h.logger.Error("Failed to parse CIDR",
"prefix", update.prefix,
"error", err,
)
return nil
}
// Track route update metrics
if h.metrics != nil {
if ipVersion == ipv4Version {
h.metrics.RecordIPv4Update()
} else {
h.metrics.RecordIPv6Update()
}
}
// Create live route record
liveRoute := &database.LiveRoute{
ID: uuid.New(),
Prefix: update.prefix,
MaskLength: maskLength,
IPVersion: ipVersion,
OriginASN: update.originASN,
PeerIP: update.peer,
ASPath: update.path,
NextHop: update.peer, // Using peer as next hop
LastUpdated: update.timestamp,
}
// For IPv4, calculate the IP range
if ipVersion == ipv4Version {
start, end, err := database.CalculateIPv4Range(update.prefix)
if err == nil {
liveRoute.V4IPStart = &start
liveRoute.V4IPEnd = &end
} else {
h.logger.Error("Failed to calculate IPv4 range",
"prefix", update.prefix,
"error", err,
)
}
}
return liveRoute
}
// processAnnouncementDirect handles storing an announcement directly without prefix table
// nolint:unused // kept for potential future use
func (h *PrefixHandler) processAnnouncementDirect(update prefixUpdate) {
// Parse CIDR to get mask length
maskLength, ipVersion, err := parseCIDR(update.prefix)
if err != nil {
h.logger.Error("Failed to parse CIDR",
"prefix", update.prefix,
"error", err,
)
return
}
// Track route update metrics
if h.metrics != nil {
if ipVersion == ipv4Version {
h.metrics.RecordIPv4Update()
} else {
h.metrics.RecordIPv6Update()
}
}
// Create live route record
liveRoute := &database.LiveRoute{
ID: uuid.New(),
Prefix: update.prefix,
MaskLength: maskLength,
IPVersion: ipVersion,
OriginASN: update.originASN,
PeerIP: update.peer,
ASPath: update.path,
NextHop: update.peer, // Using peer as next hop
LastUpdated: update.timestamp,
}
// For IPv4, calculate the IP range
if ipVersion == ipv4Version {
start, end, err := database.CalculateIPv4Range(update.prefix)
if err == nil {
liveRoute.V4IPStart = &start
liveRoute.V4IPEnd = &end
} else {
h.logger.Error("Failed to calculate IPv4 range",
"prefix", update.prefix,
"error", err,
)
}
}
if err := h.db.UpsertLiveRoute(liveRoute); err != nil {
h.logger.Error("Failed to upsert live route",
"prefix", update.prefix,
"error", err,
)
}
}
// processWithdrawalDirect handles removing a route directly without prefix table
// nolint:unused // kept for potential future use
func (h *PrefixHandler) processWithdrawalDirect(update prefixUpdate) {
// For withdrawals, we need to delete the route from live_routes
if update.originASN > 0 {
if err := h.db.DeleteLiveRoute(update.prefix, update.originASN, update.peer); err != nil {
h.logger.Error("Failed to delete live route",
"prefix", update.prefix,
"origin_asn", update.originASN,
"peer", update.peer,
"error", err,
)
}
} else {
// If no origin ASN, just delete all routes for this prefix from this peer
if err := h.db.DeleteLiveRoute(update.prefix, 0, update.peer); err != nil {
h.logger.Error("Failed to delete live route (no origin ASN)",
"prefix", update.prefix,
"peer", update.peer,
"error", err,
)
}
}
}
// processWithdrawal handles removing a route from the live routing table
// nolint:unused // kept for potential future use
func (h *PrefixHandler) processWithdrawal(_ *database.Prefix, update prefixUpdate) {
// For withdrawals, we need to delete the route from live_routes
// Since we have the origin ASN from the update, we can delete the specific route
if update.originASN > 0 {
if err := h.db.DeleteLiveRoute(update.prefix, update.originASN, update.peer); err != nil {
h.logger.Error("Failed to delete live route",
"prefix", update.prefix,
"origin_asn", update.originASN,
"peer", update.peer,
"error", err,
)
}
} else {
// If no origin ASN, just delete all routes for this prefix from this peer
if err := h.db.DeleteLiveRoute(update.prefix, 0, update.peer); err != nil {
h.logger.Error("Failed to delete live route (no origin ASN)",
"prefix", update.prefix,
"peer", update.peer,
"error", err,
)
}
}
}
// Stop gracefully stops the handler and flushes remaining batches
func (h *PrefixHandler) Stop() {
close(h.stopCh)
h.wg.Wait()
}

869
internal/server/handlers.go Normal file
View File

@@ -0,0 +1,869 @@
package server
import (
"bytes"
"context"
"encoding/json"
"errors"
"net"
"net/http"
"net/url"
"runtime"
"sort"
"strconv"
"time"
"git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/templates"
asinfo "git.eeqj.de/sneak/routewatch/pkg/asinfo"
"github.com/dustin/go-humanize"
"github.com/go-chi/chi/v5"
)
// handleRoot returns a handler that redirects to /status
func (s *Server) handleRoot() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/status", http.StatusSeeOther)
}
}
// writeJSONError writes a standardized JSON error response
func writeJSONError(w http.ResponseWriter, statusCode int, message string) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(statusCode)
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"status": "error",
"error": map[string]interface{}{
"msg": message,
"code": statusCode,
},
})
}
// writeJSONSuccess writes a standardized JSON success response
func writeJSONSuccess(w http.ResponseWriter, data interface{}) error {
w.Header().Set("Content-Type", "application/json")
return json.NewEncoder(w).Encode(map[string]interface{}{
"status": "ok",
"data": data,
})
}
// handleStatusJSON returns a handler that serves JSON statistics
func (s *Server) handleStatusJSON() http.HandlerFunc {
// Stats represents the statistics response
type Stats struct {
Uptime string `json:"uptime"`
TotalMessages uint64 `json:"total_messages"`
TotalBytes uint64 `json:"total_bytes"`
MessagesPerSec float64 `json:"messages_per_sec"`
MbitsPerSec float64 `json:"mbits_per_sec"`
Connected bool `json:"connected"`
GoVersion string `json:"go_version"`
Goroutines int `json:"goroutines"`
MemoryUsage string `json:"memory_usage"`
ASNs int `json:"asns"`
Prefixes int `json:"prefixes"`
IPv4Prefixes int `json:"ipv4_prefixes"`
IPv6Prefixes int `json:"ipv6_prefixes"`
Peerings int `json:"peerings"`
Peers int `json:"peers"`
DatabaseSizeBytes int64 `json:"database_size_bytes"`
LiveRoutes int `json:"live_routes"`
IPv4Routes int `json:"ipv4_routes"`
IPv6Routes int `json:"ipv6_routes"`
IPv4UpdatesPerSec float64 `json:"ipv4_updates_per_sec"`
IPv6UpdatesPerSec float64 `json:"ipv6_updates_per_sec"`
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
}
return func(w http.ResponseWriter, r *http.Request) {
// Create a 1 second timeout context for this request
ctx, cancel := context.WithTimeout(r.Context(), 1*time.Second)
defer cancel()
metrics := s.streamer.GetMetrics()
// Get database stats with timeout
statsChan := make(chan database.Stats)
errChan := make(chan error)
go func() {
dbStats, err := s.db.GetStatsContext(ctx)
if err != nil {
s.logger.Debug("Database stats query failed", "error", err)
errChan <- err
return
}
statsChan <- dbStats
}()
var dbStats database.Stats
select {
case <-ctx.Done():
s.logger.Error("Database stats timeout in status.json")
writeJSONError(w, http.StatusRequestTimeout, "Database timeout")
return
case err := <-errChan:
s.logger.Error("Failed to get database stats", "error", err)
writeJSONError(w, http.StatusInternalServerError, err.Error())
return
case dbStats = <-statsChan:
// Success
}
uptime := time.Since(metrics.ConnectedSince).Truncate(time.Second).String()
if metrics.ConnectedSince.IsZero() {
uptime = "0s"
}
const bitsPerMegabit = 1000000.0
// Get route counts from database
ipv4Routes, ipv6Routes, err := s.db.GetLiveRouteCountsContext(ctx)
if err != nil {
s.logger.Warn("Failed to get live route counts", "error", err)
// Continue with zero counts
}
// Get route update metrics
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
// Get memory stats
var memStats runtime.MemStats
runtime.ReadMemStats(&memStats)
stats := Stats{
Uptime: uptime,
TotalMessages: metrics.TotalMessages,
TotalBytes: metrics.TotalBytes,
MessagesPerSec: metrics.MessagesPerSec,
MbitsPerSec: metrics.BitsPerSec / bitsPerMegabit,
Connected: metrics.Connected,
GoVersion: runtime.Version(),
Goroutines: runtime.NumGoroutine(),
MemoryUsage: humanize.Bytes(memStats.Alloc),
ASNs: dbStats.ASNs,
Prefixes: dbStats.Prefixes,
IPv4Prefixes: dbStats.IPv4Prefixes,
IPv6Prefixes: dbStats.IPv6Prefixes,
Peerings: dbStats.Peerings,
Peers: dbStats.Peers,
DatabaseSizeBytes: dbStats.FileSizeBytes,
LiveRoutes: dbStats.LiveRoutes,
IPv4Routes: ipv4Routes,
IPv6Routes: ipv6Routes,
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
IPv4PrefixDistribution: dbStats.IPv4PrefixDistribution,
IPv6PrefixDistribution: dbStats.IPv6PrefixDistribution,
}
if err := writeJSONSuccess(w, stats); err != nil {
s.logger.Error("Failed to encode stats", "error", err)
}
}
}
// handleStats returns a handler that serves API v1 statistics
func (s *Server) handleStats() http.HandlerFunc {
// HandlerStatsInfo represents handler statistics in the API response
type HandlerStatsInfo struct {
Name string `json:"name"`
QueueLength int `json:"queue_length"`
QueueCapacity int `json:"queue_capacity"`
ProcessedCount uint64 `json:"processed_count"`
DroppedCount uint64 `json:"dropped_count"`
AvgProcessTimeMs float64 `json:"avg_process_time_ms"`
MinProcessTimeMs float64 `json:"min_process_time_ms"`
MaxProcessTimeMs float64 `json:"max_process_time_ms"`
}
// StatsResponse represents the API statistics response
type StatsResponse struct {
Uptime string `json:"uptime"`
TotalMessages uint64 `json:"total_messages"`
TotalBytes uint64 `json:"total_bytes"`
MessagesPerSec float64 `json:"messages_per_sec"`
MbitsPerSec float64 `json:"mbits_per_sec"`
Connected bool `json:"connected"`
GoVersion string `json:"go_version"`
Goroutines int `json:"goroutines"`
MemoryUsage string `json:"memory_usage"`
ASNs int `json:"asns"`
Prefixes int `json:"prefixes"`
IPv4Prefixes int `json:"ipv4_prefixes"`
IPv6Prefixes int `json:"ipv6_prefixes"`
Peerings int `json:"peerings"`
Peers int `json:"peers"`
DatabaseSizeBytes int64 `json:"database_size_bytes"`
LiveRoutes int `json:"live_routes"`
IPv4Routes int `json:"ipv4_routes"`
IPv6Routes int `json:"ipv6_routes"`
IPv4UpdatesPerSec float64 `json:"ipv4_updates_per_sec"`
IPv6UpdatesPerSec float64 `json:"ipv6_updates_per_sec"`
HandlerStats []HandlerStatsInfo `json:"handler_stats"`
IPv4PrefixDistribution []database.PrefixDistribution `json:"ipv4_prefix_distribution"`
IPv6PrefixDistribution []database.PrefixDistribution `json:"ipv6_prefix_distribution"`
}
return func(w http.ResponseWriter, r *http.Request) {
// Create a 1 second timeout context for this request
ctx, cancel := context.WithTimeout(r.Context(), 1*time.Second)
defer cancel()
// Check if context is already cancelled
select {
case <-ctx.Done():
http.Error(w, "Request timeout", http.StatusRequestTimeout)
return
default:
}
metrics := s.streamer.GetMetrics()
// Get database stats with timeout
statsChan := make(chan database.Stats)
errChan := make(chan error)
go func() {
dbStats, err := s.db.GetStatsContext(ctx)
if err != nil {
s.logger.Debug("Database stats query failed", "error", err)
errChan <- err
return
}
statsChan <- dbStats
}()
var dbStats database.Stats
select {
case <-ctx.Done():
s.logger.Error("Database stats timeout")
// Don't write response here - timeout middleware already handles it
return
case err := <-errChan:
s.logger.Error("Failed to get database stats", "error", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
case dbStats = <-statsChan:
// Success
}
uptime := time.Since(metrics.ConnectedSince).Truncate(time.Second).String()
if metrics.ConnectedSince.IsZero() {
uptime = "0s"
}
const bitsPerMegabit = 1000000.0
// Get route counts from database
ipv4Routes, ipv6Routes, err := s.db.GetLiveRouteCountsContext(ctx)
if err != nil {
s.logger.Warn("Failed to get live route counts", "error", err)
// Continue with zero counts
}
// Get route update metrics
routeMetrics := s.streamer.GetMetricsTracker().GetRouteMetrics()
// Get handler stats
handlerStats := s.streamer.GetHandlerStats()
handlerStatsInfo := make([]HandlerStatsInfo, 0, len(handlerStats))
const microsecondsPerMillisecond = 1000.0
for _, hs := range handlerStats {
handlerStatsInfo = append(handlerStatsInfo, HandlerStatsInfo{
Name: hs.Name,
QueueLength: hs.QueueLength,
QueueCapacity: hs.QueueCapacity,
ProcessedCount: hs.ProcessedCount,
DroppedCount: hs.DroppedCount,
AvgProcessTimeMs: float64(hs.AvgProcessTime.Microseconds()) / microsecondsPerMillisecond,
MinProcessTimeMs: float64(hs.MinProcessTime.Microseconds()) / microsecondsPerMillisecond,
MaxProcessTimeMs: float64(hs.MaxProcessTime.Microseconds()) / microsecondsPerMillisecond,
})
}
// Get memory stats
var memStats runtime.MemStats
runtime.ReadMemStats(&memStats)
stats := StatsResponse{
Uptime: uptime,
TotalMessages: metrics.TotalMessages,
TotalBytes: metrics.TotalBytes,
MessagesPerSec: metrics.MessagesPerSec,
MbitsPerSec: metrics.BitsPerSec / bitsPerMegabit,
Connected: metrics.Connected,
GoVersion: runtime.Version(),
Goroutines: runtime.NumGoroutine(),
MemoryUsage: humanize.Bytes(memStats.Alloc),
ASNs: dbStats.ASNs,
Prefixes: dbStats.Prefixes,
IPv4Prefixes: dbStats.IPv4Prefixes,
IPv6Prefixes: dbStats.IPv6Prefixes,
Peerings: dbStats.Peerings,
Peers: dbStats.Peers,
DatabaseSizeBytes: dbStats.FileSizeBytes,
LiveRoutes: dbStats.LiveRoutes,
IPv4Routes: ipv4Routes,
IPv6Routes: ipv6Routes,
IPv4UpdatesPerSec: routeMetrics.IPv4UpdatesPerSec,
IPv6UpdatesPerSec: routeMetrics.IPv6UpdatesPerSec,
HandlerStats: handlerStatsInfo,
IPv4PrefixDistribution: dbStats.IPv4PrefixDistribution,
IPv6PrefixDistribution: dbStats.IPv6PrefixDistribution,
}
if err := writeJSONSuccess(w, stats); err != nil {
s.logger.Error("Failed to encode stats", "error", err)
}
}
}
// handleStatusHTML returns a handler that serves the HTML status page
func (s *Server) handleStatusHTML() http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
tmpl := templates.StatusTemplate()
if err := tmpl.Execute(w, nil); err != nil {
s.logger.Error("Failed to render template", "error", err)
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
}
}
}
// handleIPLookup returns a handler that looks up AS information for an IP address
func (s *Server) handleIPLookup() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ip := chi.URLParam(r, "ip")
if ip == "" {
writeJSONError(w, http.StatusBadRequest, "IP parameter is required")
return
}
// Look up AS information for the IP
asInfo, err := s.db.GetASInfoForIPContext(r.Context(), ip)
if err != nil {
// Check if it's an invalid IP error
if errors.Is(err, database.ErrInvalidIP) {
writeJSONError(w, http.StatusBadRequest, err.Error())
} else {
// All other errors (including ErrNoRoute) are 404
writeJSONError(w, http.StatusNotFound, err.Error())
}
return
}
// Return successful response
if err := writeJSONSuccess(w, asInfo); err != nil {
s.logger.Error("Failed to encode AS info", "error", err)
}
}
}
// handleASDetailJSON returns AS details as JSON
func (s *Server) handleASDetailJSON() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
asnStr := chi.URLParam(r, "asn")
asn, err := strconv.Atoi(asnStr)
if err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid ASN")
return
}
asInfo, prefixes, err := s.db.GetASDetailsContext(r.Context(), asn)
if err != nil {
if errors.Is(err, database.ErrNoRoute) {
writeJSONError(w, http.StatusNotFound, err.Error())
} else {
writeJSONError(w, http.StatusInternalServerError, err.Error())
}
return
}
// Group prefixes by IP version
const ipVersionV4 = 4
var ipv4Prefixes, ipv6Prefixes []database.LiveRoute
for _, p := range prefixes {
if p.IPVersion == ipVersionV4 {
ipv4Prefixes = append(ipv4Prefixes, p)
} else {
ipv6Prefixes = append(ipv6Prefixes, p)
}
}
response := map[string]interface{}{
"asn": asInfo,
"ipv4_prefixes": ipv4Prefixes,
"ipv6_prefixes": ipv6Prefixes,
"total_count": len(prefixes),
}
if err := writeJSONSuccess(w, response); err != nil {
s.logger.Error("Failed to encode AS details", "error", err)
}
}
}
// handlePrefixDetailJSON returns prefix details as JSON
func (s *Server) handlePrefixDetailJSON() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
prefixParam := chi.URLParam(r, "prefix")
if prefixParam == "" {
writeJSONError(w, http.StatusBadRequest, "Prefix parameter is required")
return
}
// URL decode the prefix parameter
prefix, err := url.QueryUnescape(prefixParam)
if err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid prefix parameter")
return
}
routes, err := s.db.GetPrefixDetailsContext(r.Context(), prefix)
if err != nil {
if errors.Is(err, database.ErrNoRoute) {
writeJSONError(w, http.StatusNotFound, err.Error())
} else {
writeJSONError(w, http.StatusInternalServerError, err.Error())
}
return
}
// Group by origin AS
originMap := make(map[int][]database.LiveRoute)
for _, route := range routes {
originMap[route.OriginASN] = append(originMap[route.OriginASN], route)
}
response := map[string]interface{}{
"prefix": prefix,
"routes": routes,
"origins": originMap,
"peer_count": len(routes),
"origin_count": len(originMap),
}
if err := writeJSONSuccess(w, response); err != nil {
s.logger.Error("Failed to encode prefix details", "error", err)
}
}
}
// handleASDetail returns a handler that serves the AS detail HTML page
func (s *Server) handleASDetail() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
asnStr := chi.URLParam(r, "asn")
asn, err := strconv.Atoi(asnStr)
if err != nil {
http.Error(w, "Invalid ASN", http.StatusBadRequest)
return
}
asInfo, prefixes, err := s.db.GetASDetailsContext(r.Context(), asn)
if err != nil {
if errors.Is(err, database.ErrNoRoute) {
http.Error(w, "AS not found", http.StatusNotFound)
} else {
s.logger.Error("Failed to get AS details", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
return
}
// Group prefixes by IP version
const ipVersionV4 = 4
var ipv4Prefixes, ipv6Prefixes []database.LiveRoute
for _, p := range prefixes {
if p.IPVersion == ipVersionV4 {
ipv4Prefixes = append(ipv4Prefixes, p)
} else {
ipv6Prefixes = append(ipv6Prefixes, p)
}
}
// Sort prefixes by network address
sort.Slice(ipv4Prefixes, func(i, j int) bool {
// Parse the prefixes to compare network addresses
ipI, netI, _ := net.ParseCIDR(ipv4Prefixes[i].Prefix)
ipJ, netJ, _ := net.ParseCIDR(ipv4Prefixes[j].Prefix)
// Compare by network address first
cmp := bytes.Compare(ipI.To4(), ipJ.To4())
if cmp != 0 {
return cmp < 0
}
// If network addresses are equal, compare by mask length
onesI, _ := netI.Mask.Size()
onesJ, _ := netJ.Mask.Size()
return onesI < onesJ
})
sort.Slice(ipv6Prefixes, func(i, j int) bool {
// Parse the prefixes to compare network addresses
ipI, netI, _ := net.ParseCIDR(ipv6Prefixes[i].Prefix)
ipJ, netJ, _ := net.ParseCIDR(ipv6Prefixes[j].Prefix)
// Compare by network address first
cmp := bytes.Compare(ipI.To16(), ipJ.To16())
if cmp != 0 {
return cmp < 0
}
// If network addresses are equal, compare by mask length
onesI, _ := netI.Mask.Size()
onesJ, _ := netJ.Mask.Size()
return onesI < onesJ
})
// Prepare template data
data := struct {
ASN *database.ASN
IPv4Prefixes []database.LiveRoute
IPv6Prefixes []database.LiveRoute
TotalCount int
IPv4Count int
IPv6Count int
}{
ASN: asInfo,
IPv4Prefixes: ipv4Prefixes,
IPv6Prefixes: ipv6Prefixes,
TotalCount: len(prefixes),
IPv4Count: len(ipv4Prefixes),
IPv6Count: len(ipv6Prefixes),
}
// Check if context is still valid before writing response
select {
case <-r.Context().Done():
// Request was cancelled, don't write response
return
default:
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
tmpl := templates.ASDetailTemplate()
if err := tmpl.Execute(w, data); err != nil {
s.logger.Error("Failed to render AS detail template", "error", err)
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
}
}
}
// handlePrefixDetail returns a handler that serves the prefix detail HTML page
func (s *Server) handlePrefixDetail() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
prefixParam := chi.URLParam(r, "prefix")
if prefixParam == "" {
http.Error(w, "Prefix parameter is required", http.StatusBadRequest)
return
}
// URL decode the prefix parameter
prefix, err := url.QueryUnescape(prefixParam)
if err != nil {
http.Error(w, "Invalid prefix parameter", http.StatusBadRequest)
return
}
routes, err := s.db.GetPrefixDetailsContext(r.Context(), prefix)
if err != nil {
if errors.Is(err, database.ErrNoRoute) {
http.Error(w, "Prefix not found", http.StatusNotFound)
} else {
s.logger.Error("Failed to get prefix details", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
return
}
// Group by origin AS and collect unique AS info
type ASNInfo struct {
Number int
Handle string
Description string
PeerCount int
}
originMap := make(map[int]*ASNInfo)
for _, route := range routes {
if _, exists := originMap[route.OriginASN]; !exists {
// Get AS info from database
asInfo, _, _ := s.db.GetASDetailsContext(r.Context(), route.OriginASN)
handle := ""
description := ""
if asInfo != nil {
handle = asInfo.Handle
description = asInfo.Description
}
originMap[route.OriginASN] = &ASNInfo{
Number: route.OriginASN,
Handle: handle,
Description: description,
PeerCount: 0,
}
}
originMap[route.OriginASN].PeerCount++
}
// Get the first route to extract some common info
var maskLength, ipVersion int
if len(routes) > 0 {
// Parse CIDR to get mask length and IP version
_, ipNet, err := net.ParseCIDR(prefix)
if err == nil {
ones, _ := ipNet.Mask.Size()
maskLength = ones
if ipNet.IP.To4() != nil {
ipVersion = 4
} else {
ipVersion = 6
}
}
}
// Convert origin map to sorted slice
var origins []*ASNInfo
for _, origin := range originMap {
origins = append(origins, origin)
}
// Create enhanced routes with AS path handles
type ASPathEntry struct {
Number int
Handle string
}
type EnhancedRoute struct {
database.LiveRoute
ASPathWithHandle []ASPathEntry
}
enhancedRoutes := make([]EnhancedRoute, len(routes))
for i, route := range routes {
enhancedRoute := EnhancedRoute{
LiveRoute: route,
ASPathWithHandle: make([]ASPathEntry, len(route.ASPath)),
}
// Look up handle for each AS in the path
for j, asn := range route.ASPath {
handle := asinfo.GetHandle(asn)
enhancedRoute.ASPathWithHandle[j] = ASPathEntry{
Number: asn,
Handle: handle,
}
}
enhancedRoutes[i] = enhancedRoute
}
// Prepare template data
data := struct {
Prefix string
MaskLength int
IPVersion int
Routes []EnhancedRoute
Origins []*ASNInfo
PeerCount int
OriginCount int
}{
Prefix: prefix,
MaskLength: maskLength,
IPVersion: ipVersion,
Routes: enhancedRoutes,
Origins: origins,
PeerCount: len(routes),
OriginCount: len(originMap),
}
// Check if context is still valid before writing response
select {
case <-r.Context().Done():
// Request was cancelled, don't write response
return
default:
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
tmpl := templates.PrefixDetailTemplate()
if err := tmpl.Execute(w, data); err != nil {
s.logger.Error("Failed to render prefix detail template", "error", err)
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
}
}
}
// handleIPRedirect looks up the prefix containing the IP and redirects to its detail page
func (s *Server) handleIPRedirect() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ip := chi.URLParam(r, "ip")
if ip == "" {
http.Error(w, "IP parameter is required", http.StatusBadRequest)
return
}
// Look up AS information for the IP (which includes the prefix)
asInfo, err := s.db.GetASInfoForIP(ip)
if err != nil {
if errors.Is(err, database.ErrInvalidIP) {
http.Error(w, "Invalid IP address", http.StatusBadRequest)
} else if errors.Is(err, database.ErrNoRoute) {
http.Error(w, "No route found for this IP", http.StatusNotFound)
} else {
s.logger.Error("Failed to look up IP", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
return
}
// Redirect to the prefix detail page (URL encode the prefix)
http.Redirect(w, r, "/prefix/"+url.QueryEscape(asInfo.Prefix), http.StatusSeeOther)
}
}
// handlePrefixLength shows a random sample of prefixes with the specified mask length
func (s *Server) handlePrefixLength() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
lengthStr := chi.URLParam(r, "length")
if lengthStr == "" {
http.Error(w, "Length parameter is required", http.StatusBadRequest)
return
}
maskLength, err := strconv.Atoi(lengthStr)
if err != nil {
http.Error(w, "Invalid mask length", http.StatusBadRequest)
return
}
// Determine IP version based on mask length
const (
maxIPv4MaskLength = 32
maxIPv6MaskLength = 128
)
var ipVersion int
if maskLength <= maxIPv4MaskLength {
ipVersion = 4
} else if maskLength <= maxIPv6MaskLength {
ipVersion = 6
} else {
http.Error(w, "Invalid mask length", http.StatusBadRequest)
return
}
// Get random sample of prefixes
const maxPrefixes = 500
prefixes, err := s.db.GetRandomPrefixesByLengthContext(r.Context(), maskLength, ipVersion, maxPrefixes)
if err != nil {
s.logger.Error("Failed to get prefixes by length", "error", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
return
}
// Sort prefixes for display
sort.Slice(prefixes, func(i, j int) bool {
// First compare by IP version
if prefixes[i].IPVersion != prefixes[j].IPVersion {
return prefixes[i].IPVersion < prefixes[j].IPVersion
}
// Then by prefix
return prefixes[i].Prefix < prefixes[j].Prefix
})
// Create enhanced prefixes with AS descriptions
type EnhancedPrefix struct {
database.LiveRoute
OriginASDescription string
Age string
}
enhancedPrefixes := make([]EnhancedPrefix, len(prefixes))
for i, prefix := range prefixes {
enhancedPrefixes[i] = EnhancedPrefix{
LiveRoute: prefix,
Age: formatAge(prefix.LastUpdated),
}
// Get AS description
if asInfo, ok := asinfo.Get(prefix.OriginASN); ok {
enhancedPrefixes[i].OriginASDescription = asInfo.Description
}
}
// Render template
data := map[string]interface{}{
"MaskLength": maskLength,
"IPVersion": ipVersion,
"Prefixes": enhancedPrefixes,
"Count": len(prefixes),
}
// Check if context is still valid before writing response
select {
case <-r.Context().Done():
// Request was cancelled, don't write response
return
default:
}
tmpl := templates.PrefixLengthTemplate()
if err := tmpl.Execute(w, data); err != nil {
s.logger.Error("Failed to render prefix length template", "error", err)
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
}
}
}
// formatAge returns a human-readable age string
func formatAge(timestamp time.Time) string {
age := time.Since(timestamp)
const hoursPerDay = 24
if age < time.Minute {
return "< 1m"
} else if age < time.Hour {
minutes := int(age.Minutes())
return strconv.Itoa(minutes) + "m"
} else if age < hoursPerDay*time.Hour {
hours := int(age.Hours())
return strconv.Itoa(hours) + "h"
}
days := int(age.Hours() / hoursPerDay)
return strconv.Itoa(days) + "d"
}

View File

@@ -108,6 +108,7 @@ type timeoutWriter struct {
http.ResponseWriter http.ResponseWriter
mu sync.Mutex mu sync.Mutex
written bool written bool
header http.Header // cached header to prevent concurrent access
} }
func (tw *timeoutWriter) Write(b []byte) (int, error) { func (tw *timeoutWriter) Write(b []byte) (int, error) {
@@ -133,6 +134,18 @@ func (tw *timeoutWriter) WriteHeader(statusCode int) {
} }
func (tw *timeoutWriter) Header() http.Header { func (tw *timeoutWriter) Header() http.Header {
tw.mu.Lock()
defer tw.mu.Unlock()
if tw.written {
// Return a copy to prevent modifications after timeout
if tw.header == nil {
tw.header = make(http.Header)
}
return tw.header
}
return tw.ResponseWriter.Header() return tw.ResponseWriter.Header()
} }
@@ -153,6 +166,7 @@ func TimeoutMiddleware(timeout time.Duration) func(http.Handler) http.Handler {
tw := &timeoutWriter{ tw := &timeoutWriter{
ResponseWriter: w, ResponseWriter: w,
header: make(http.Header),
} }
done := make(chan struct{}) done := make(chan struct{})
@@ -178,8 +192,12 @@ func TimeoutMiddleware(timeout time.Duration) func(http.Handler) http.Handler {
tw.markWritten() // Prevent the handler from writing after timeout tw.markWritten() // Prevent the handler from writing after timeout
execTime := time.Since(startTime) execTime := time.Since(startTime)
// Write directly to the underlying writer since we've marked tw as written
// This is safe because markWritten() prevents the handler from writing
tw.mu.Lock()
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusRequestTimeout) w.WriteHeader(http.StatusRequestTimeout)
tw.mu.Unlock()
response := map[string]interface{}{ response := map[string]interface{}{
"status": "error", "status": "error",

43
internal/server/routes.go Normal file
View File

@@ -0,0 +1,43 @@
package server
import (
"time"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
)
// setupRoutes configures the HTTP routes
func (s *Server) setupRoutes() {
r := chi.NewRouter()
// Middleware
r.Use(middleware.RequestID)
r.Use(middleware.RealIP)
r.Use(middleware.Logger)
r.Use(middleware.Recoverer)
const requestTimeout = 2 * time.Second
r.Use(TimeoutMiddleware(requestTimeout))
r.Use(JSONResponseMiddleware)
// Routes
r.Get("/", s.handleRoot())
r.Get("/status", s.handleStatusHTML())
r.Get("/status.json", s.handleStatusJSON())
// AS and prefix detail pages
r.Get("/as/{asn}", s.handleASDetail())
r.Get("/prefix/{prefix}", s.handlePrefixDetail())
r.Get("/prefixlength/{length}", s.handlePrefixLength())
r.Get("/ip/{ip}", s.handleIPRedirect())
// API routes
r.Route("/api/v1", func(r chi.Router) {
r.Get("/stats", s.handleStats())
r.Get("/ip/{ip}", s.handleIPLookup())
r.Get("/as/{asn}", s.handleASDetailJSON())
r.Get("/prefix/{prefix}", s.handlePrefixDetailJSON())
})
s.router = r
}

View File

@@ -3,17 +3,14 @@ package server
import ( import (
"context" "context"
"encoding/json"
"log/slog"
"net/http" "net/http"
"os" "os"
"time" "time"
"git.eeqj.de/sneak/routewatch/internal/database" "git.eeqj.de/sneak/routewatch/internal/database"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/streamer" "git.eeqj.de/sneak/routewatch/internal/streamer"
"git.eeqj.de/sneak/routewatch/internal/templates"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
) )
// Server provides HTTP endpoints for status monitoring // Server provides HTTP endpoints for status monitoring
@@ -21,12 +18,12 @@ type Server struct {
router *chi.Mux router *chi.Mux
db database.Store db database.Store
streamer *streamer.Streamer streamer *streamer.Streamer
logger *slog.Logger logger *logger.Logger
srv *http.Server srv *http.Server
} }
// New creates a new HTTP server // New creates a new HTTP server
func New(db database.Store, streamer *streamer.Streamer, logger *slog.Logger) *Server { func New(db database.Store, streamer *streamer.Streamer, logger *logger.Logger) *Server {
s := &Server{ s := &Server{
db: db, db: db,
streamer: streamer, streamer: streamer,
@@ -38,32 +35,6 @@ func New(db database.Store, streamer *streamer.Streamer, logger *slog.Logger) *S
return s return s
} }
// setupRoutes configures the HTTP routes
func (s *Server) setupRoutes() {
r := chi.NewRouter()
// Middleware
r.Use(middleware.RequestID)
r.Use(middleware.RealIP)
r.Use(middleware.Logger)
r.Use(middleware.Recoverer)
const requestTimeout = 2 * time.Second
r.Use(TimeoutMiddleware(requestTimeout))
r.Use(JSONResponseMiddleware)
// Routes
r.Get("/", s.handleRoot())
r.Get("/status", s.handleStatusHTML())
r.Get("/status.json", s.handleStatusJSON())
// API routes
r.Route("/api/v1", func(r chi.Router) {
r.Get("/stats", s.handleStats())
})
s.router = r
}
// Start starts the HTTP server // Start starts the HTTP server
func (s *Server) Start() error { func (s *Server) Start() error {
port := os.Getenv("PORT") port := os.Getenv("PORT")
@@ -99,230 +70,3 @@ func (s *Server) Stop(ctx context.Context) error {
return s.srv.Shutdown(ctx) return s.srv.Shutdown(ctx)
} }
// handleRoot returns a handler that redirects to /status
func (s *Server) handleRoot() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/status", http.StatusSeeOther)
}
}
// handleStatusJSON returns a handler that serves JSON statistics
func (s *Server) handleStatusJSON() http.HandlerFunc {
// Stats represents the statistics response
type Stats struct {
Uptime string `json:"uptime"`
TotalMessages uint64 `json:"total_messages"`
TotalBytes uint64 `json:"total_bytes"`
MessagesPerSec float64 `json:"messages_per_sec"`
MbitsPerSec float64 `json:"mbits_per_sec"`
Connected bool `json:"connected"`
ASNs int `json:"asns"`
Prefixes int `json:"prefixes"`
IPv4Prefixes int `json:"ipv4_prefixes"`
IPv6Prefixes int `json:"ipv6_prefixes"`
Peerings int `json:"peerings"`
LiveRoutes int `json:"live_routes"`
}
return func(w http.ResponseWriter, r *http.Request) {
// Create a 1 second timeout context for this request
ctx, cancel := context.WithTimeout(r.Context(), 1*time.Second)
defer cancel()
metrics := s.streamer.GetMetrics()
// Get database stats with timeout
statsChan := make(chan database.Stats)
errChan := make(chan error)
go func() {
s.logger.Debug("Starting database stats query")
dbStats, err := s.db.GetStats()
if err != nil {
s.logger.Debug("Database stats query failed", "error", err)
errChan <- err
return
}
s.logger.Debug("Database stats query completed")
statsChan <- dbStats
}()
var dbStats database.Stats
select {
case <-ctx.Done():
s.logger.Error("Database stats timeout in status.json")
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusRequestTimeout)
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"status": "error",
"error": map[string]interface{}{
"msg": "Database timeout",
"code": http.StatusRequestTimeout,
},
})
return
case err := <-errChan:
s.logger.Error("Failed to get database stats", "error", err)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusInternalServerError)
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"status": "error",
"error": map[string]interface{}{
"msg": err.Error(),
"code": http.StatusInternalServerError,
},
})
return
case dbStats = <-statsChan:
// Success
}
uptime := time.Since(metrics.ConnectedSince).Truncate(time.Second).String()
if metrics.ConnectedSince.IsZero() {
uptime = "0s"
}
const bitsPerMegabit = 1000000.0
stats := Stats{
Uptime: uptime,
TotalMessages: metrics.TotalMessages,
TotalBytes: metrics.TotalBytes,
MessagesPerSec: metrics.MessagesPerSec,
MbitsPerSec: metrics.BitsPerSec / bitsPerMegabit,
Connected: metrics.Connected,
ASNs: dbStats.ASNs,
Prefixes: dbStats.Prefixes,
IPv4Prefixes: dbStats.IPv4Prefixes,
IPv6Prefixes: dbStats.IPv6Prefixes,
Peerings: dbStats.Peerings,
LiveRoutes: dbStats.LiveRoutes,
}
w.Header().Set("Content-Type", "application/json")
response := map[string]interface{}{
"status": "ok",
"data": stats,
}
if err := json.NewEncoder(w).Encode(response); err != nil {
s.logger.Error("Failed to encode stats", "error", err)
}
}
}
// handleStats returns a handler that serves API v1 statistics
func (s *Server) handleStats() http.HandlerFunc {
// StatsResponse represents the API statistics response
type StatsResponse struct {
Uptime string `json:"uptime"`
TotalMessages uint64 `json:"total_messages"`
TotalBytes uint64 `json:"total_bytes"`
MessagesPerSec float64 `json:"messages_per_sec"`
MbitsPerSec float64 `json:"mbits_per_sec"`
Connected bool `json:"connected"`
ASNs int `json:"asns"`
Prefixes int `json:"prefixes"`
IPv4Prefixes int `json:"ipv4_prefixes"`
IPv6Prefixes int `json:"ipv6_prefixes"`
Peerings int `json:"peerings"`
LiveRoutes int `json:"live_routes"`
}
return func(w http.ResponseWriter, r *http.Request) {
// Create a 1 second timeout context for this request
ctx, cancel := context.WithTimeout(r.Context(), 1*time.Second)
defer cancel()
// Check if context is already cancelled
select {
case <-ctx.Done():
http.Error(w, "Request timeout", http.StatusRequestTimeout)
return
default:
}
metrics := s.streamer.GetMetrics()
// Get database stats with timeout
statsChan := make(chan database.Stats)
errChan := make(chan error)
go func() {
s.logger.Debug("Starting database stats query")
dbStats, err := s.db.GetStats()
if err != nil {
s.logger.Debug("Database stats query failed", "error", err)
errChan <- err
return
}
s.logger.Debug("Database stats query completed")
statsChan <- dbStats
}()
var dbStats database.Stats
select {
case <-ctx.Done():
s.logger.Error("Database stats timeout")
http.Error(w, "Database timeout", http.StatusRequestTimeout)
return
case err := <-errChan:
s.logger.Error("Failed to get database stats", "error", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
case dbStats = <-statsChan:
// Success
}
uptime := time.Since(metrics.ConnectedSince).Truncate(time.Second).String()
if metrics.ConnectedSince.IsZero() {
uptime = "0s"
}
const bitsPerMegabit = 1000000.0
stats := StatsResponse{
Uptime: uptime,
TotalMessages: metrics.TotalMessages,
TotalBytes: metrics.TotalBytes,
MessagesPerSec: metrics.MessagesPerSec,
MbitsPerSec: metrics.BitsPerSec / bitsPerMegabit,
Connected: metrics.Connected,
ASNs: dbStats.ASNs,
Prefixes: dbStats.Prefixes,
IPv4Prefixes: dbStats.IPv4Prefixes,
IPv6Prefixes: dbStats.IPv6Prefixes,
Peerings: dbStats.Peerings,
LiveRoutes: dbStats.LiveRoutes,
}
w.Header().Set("Content-Type", "application/json")
response := map[string]interface{}{
"status": "ok",
"data": stats,
}
if err := json.NewEncoder(w).Encode(response); err != nil {
s.logger.Error("Failed to encode stats", "error", err)
}
}
}
// handleStatusHTML returns a handler that serves the HTML status page
func (s *Server) handleStatusHTML() http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
tmpl := templates.StatusTemplate()
if err := tmpl.Execute(w, nil); err != nil {
s.logger.Error("Failed to render template", "error", err)
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
}
}
}

View File

@@ -7,25 +7,28 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"log/slog" "math"
"net/http" "net/http"
"os"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time" "time"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/metrics" "git.eeqj.de/sneak/routewatch/internal/metrics"
"git.eeqj.de/sneak/routewatch/internal/ristypes" "git.eeqj.de/sneak/routewatch/internal/ristypes"
) )
const ( const (
risLiveURL = "https://ris-live.ripe.net/v1/stream/?format=json" risLiveURL = "https://ris-live.ripe.net/v1/stream/?format=json&" +
"client=https%3A%2F%2Fgit.eeqj.de%2Fsneak%2Froutewatch"
metricsWindowSize = 60 // seconds for rolling average metricsWindowSize = 60 // seconds for rolling average
metricsUpdateRate = time.Second metricsUpdateRate = time.Second
minBackoffDelay = 5 * time.Second
maxBackoffDelay = 320 * time.Second
metricsLogInterval = 10 * time.Second metricsLogInterval = 10 * time.Second
bytesPerKB = 1024 bytesPerKB = 1024
bytesPerMB = 1024 * 1024 bytesPerMB = 1024 * 1024
maxConcurrentHandlers = 100 // Maximum number of concurrent message handlers maxConcurrentHandlers = 800 // Maximum number of concurrent message handlers
) )
// MessageHandler is an interface for handling RIS messages // MessageHandler is an interface for handling RIS messages
@@ -35,35 +38,54 @@ type MessageHandler interface {
// HandleMessage processes a RIS message // HandleMessage processes a RIS message
HandleMessage(msg *ristypes.RISMessage) HandleMessage(msg *ristypes.RISMessage)
// QueueCapacity returns the desired queue capacity for this handler
// Handlers that process quickly can have larger queues
QueueCapacity() int
} }
// RawMessageHandler is a callback for handling raw JSON lines from the stream // RawMessageHandler is a callback for handling raw JSON lines from the stream
type RawMessageHandler func(line string) type RawMessageHandler func(line string)
// handlerMetrics tracks performance metrics for a handler
type handlerMetrics struct {
processedCount uint64 // Total messages processed
droppedCount uint64 // Total messages dropped
totalTime time.Duration // Total processing time (for average calculation)
minTime time.Duration // Minimum processing time
maxTime time.Duration // Maximum processing time
mu sync.Mutex // Protects the metrics
}
// handlerInfo wraps a handler with its queue and metrics
type handlerInfo struct {
handler MessageHandler
queue chan *ristypes.RISMessage
metrics handlerMetrics
}
// Streamer handles streaming BGP updates from RIS Live // Streamer handles streaming BGP updates from RIS Live
type Streamer struct { type Streamer struct {
logger *slog.Logger logger *logger.Logger
client *http.Client client *http.Client
handlers []MessageHandler handlers []*handlerInfo
rawHandler RawMessageHandler rawHandler RawMessageHandler
mu sync.RWMutex mu sync.RWMutex
cancel context.CancelFunc cancel context.CancelFunc
running bool running bool
metrics *metrics.Tracker metrics *metrics.Tracker
semaphore chan struct{} // Limits concurrent message processing totalDropped uint64 // Total dropped messages across all handlers
droppedMessages uint64 // Atomic counter for dropped messages
} }
// New creates a new RIS streamer // New creates a new RIS streamer
func New(logger *slog.Logger, metrics *metrics.Tracker) *Streamer { func New(logger *logger.Logger, metrics *metrics.Tracker) *Streamer {
return &Streamer{ return &Streamer{
logger: logger, logger: logger,
client: &http.Client{ client: &http.Client{
Timeout: 0, // No timeout for streaming Timeout: 0, // No timeout for streaming
}, },
handlers: make([]MessageHandler, 0), handlers: make([]*handlerInfo, 0),
metrics: metrics, metrics: metrics,
semaphore: make(chan struct{}, maxConcurrentHandlers),
} }
} }
@@ -71,7 +93,22 @@ func New(logger *slog.Logger, metrics *metrics.Tracker) *Streamer {
func (s *Streamer) RegisterHandler(handler MessageHandler) { func (s *Streamer) RegisterHandler(handler MessageHandler) {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
s.handlers = append(s.handlers, handler)
// Create handler info with its own queue based on capacity
info := &handlerInfo{
handler: handler,
queue: make(chan *ristypes.RISMessage, handler.QueueCapacity()),
metrics: handlerMetrics{
minTime: time.Duration(math.MaxInt64), // Initialize to max so first value sets the floor
},
}
s.handlers = append(s.handlers, info)
// If we're already running, start a worker for this handler
if s.running {
go s.runHandlerWorker(info)
}
} }
// RegisterRawHandler sets a callback for raw message lines // RegisterRawHandler sets a callback for raw message lines
@@ -94,10 +131,13 @@ func (s *Streamer) Start() error {
s.cancel = cancel s.cancel = cancel
s.running = true s.running = true
// Start workers for each handler
for _, info := range s.handlers {
go s.runHandlerWorker(info)
}
go func() { go func() {
if err := s.stream(ctx); err != nil { s.streamWithReconnect(ctx)
s.logger.Error("Streaming error", "error", err)
}
s.mu.Lock() s.mu.Lock()
s.running = false s.running = false
s.mu.Unlock() s.mu.Unlock()
@@ -112,10 +152,40 @@ func (s *Streamer) Stop() {
if s.cancel != nil { if s.cancel != nil {
s.cancel() s.cancel()
} }
// Close all handler queues to signal workers to stop
for _, info := range s.handlers {
close(info.queue)
}
s.running = false
s.mu.Unlock() s.mu.Unlock()
s.metrics.SetConnected(false) s.metrics.SetConnected(false)
} }
// runHandlerWorker processes messages for a specific handler
func (s *Streamer) runHandlerWorker(info *handlerInfo) {
for msg := range info.queue {
start := time.Now()
info.handler.HandleMessage(msg)
elapsed := time.Since(start)
// Update metrics
info.metrics.mu.Lock()
info.metrics.processedCount++
info.metrics.totalTime += elapsed
// Update min time
if elapsed < info.metrics.minTime {
info.metrics.minTime = elapsed
}
// Update max time
if elapsed > info.metrics.maxTime {
info.metrics.maxTime = elapsed
}
info.metrics.mu.Unlock()
}
}
// IsRunning returns whether the streamer is currently active // IsRunning returns whether the streamer is currently active
func (s *Streamer) IsRunning() bool { func (s *Streamer) IsRunning() bool {
s.mu.RLock() s.mu.RLock()
@@ -129,9 +199,65 @@ func (s *Streamer) GetMetrics() metrics.StreamMetrics {
return s.metrics.GetStreamMetrics() return s.metrics.GetStreamMetrics()
} }
// GetMetricsTracker returns the metrics tracker instance
func (s *Streamer) GetMetricsTracker() *metrics.Tracker {
return s.metrics
}
// HandlerStats represents metrics for a single handler
type HandlerStats struct {
Name string
QueueLength int
QueueCapacity int
ProcessedCount uint64
DroppedCount uint64
AvgProcessTime time.Duration
MinProcessTime time.Duration
MaxProcessTime time.Duration
}
// GetHandlerStats returns current handler statistics
func (s *Streamer) GetHandlerStats() []HandlerStats {
s.mu.RLock()
defer s.mu.RUnlock()
stats := make([]HandlerStats, 0, len(s.handlers))
for _, info := range s.handlers {
info.metrics.mu.Lock()
hs := HandlerStats{
Name: fmt.Sprintf("%T", info.handler),
QueueLength: len(info.queue),
QueueCapacity: cap(info.queue),
ProcessedCount: info.metrics.processedCount,
DroppedCount: info.metrics.droppedCount,
MinProcessTime: info.metrics.minTime,
MaxProcessTime: info.metrics.maxTime,
}
// Calculate average time
if info.metrics.processedCount > 0 {
processedCount := info.metrics.processedCount
const maxInt64 = 1<<63 - 1
if processedCount > maxInt64 {
processedCount = maxInt64
}
//nolint:gosec // processedCount is explicitly bounded above
hs.AvgProcessTime = info.metrics.totalTime / time.Duration(processedCount)
}
info.metrics.mu.Unlock()
stats = append(stats, hs)
}
return stats
}
// GetDroppedMessages returns the total number of dropped messages // GetDroppedMessages returns the total number of dropped messages
func (s *Streamer) GetDroppedMessages() uint64 { func (s *Streamer) GetDroppedMessages() uint64 {
return atomic.LoadUint64(&s.droppedMessages) return atomic.LoadUint64(&s.totalDropped)
} }
// logMetrics logs the current streaming statistics // logMetrics logs the current streaming statistics
@@ -140,18 +266,57 @@ func (s *Streamer) logMetrics() {
uptime := time.Since(metrics.ConnectedSince) uptime := time.Since(metrics.ConnectedSince)
const bitsPerMegabit = 1000000 const bitsPerMegabit = 1000000
droppedMessages := atomic.LoadUint64(&s.droppedMessages) totalDropped := atomic.LoadUint64(&s.totalDropped)
s.logger.Info("Stream statistics",
"uptime", uptime, s.logger.Info(
"total_messages", metrics.TotalMessages, "Stream statistics",
"total_bytes", metrics.TotalBytes, "uptime",
"total_mb", fmt.Sprintf("%.2f", float64(metrics.TotalBytes)/bytesPerMB), uptime,
"messages_per_sec", fmt.Sprintf("%.2f", metrics.MessagesPerSec), "total_messages",
"bits_per_sec", fmt.Sprintf("%.0f", metrics.BitsPerSec), metrics.TotalMessages,
"mbps", fmt.Sprintf("%.2f", metrics.BitsPerSec/bitsPerMegabit), "total_bytes",
"dropped_messages", droppedMessages, metrics.TotalBytes,
"active_handlers", len(s.semaphore), "total_mb",
fmt.Sprintf("%.2f", float64(metrics.TotalBytes)/bytesPerMB),
"messages_per_sec",
fmt.Sprintf("%.2f", metrics.MessagesPerSec),
"bits_per_sec",
fmt.Sprintf("%.0f", metrics.BitsPerSec),
"mbps",
fmt.Sprintf("%.2f", metrics.BitsPerSec/bitsPerMegabit),
"total_dropped",
totalDropped,
) )
// Log per-handler statistics
s.mu.RLock()
for i, info := range s.handlers {
info.metrics.mu.Lock()
if info.metrics.processedCount > 0 {
// Safe conversion: processedCount is bounded by maxInt64
processedCount := info.metrics.processedCount
const maxInt64 = 1<<63 - 1
if processedCount > maxInt64 {
processedCount = maxInt64
}
//nolint:gosec // processedCount is explicitly bounded above
avgTime := info.metrics.totalTime / time.Duration(processedCount)
s.logger.Info(
"Handler statistics",
"handler", fmt.Sprintf("%T", info.handler),
"index", i,
"queue_len", len(info.queue),
"queue_cap", cap(info.queue),
"processed", info.metrics.processedCount,
"dropped", info.metrics.droppedCount,
"avg_time", avgTime,
"min_time", info.metrics.minTime,
"max_time", info.metrics.maxTime,
)
}
info.metrics.mu.Unlock()
}
s.mu.RUnlock()
} }
// updateMetrics updates the metrics counters and rates // updateMetrics updates the metrics counters and rates
@@ -159,6 +324,72 @@ func (s *Streamer) updateMetrics(messageBytes int) {
s.metrics.RecordMessage(int64(messageBytes)) s.metrics.RecordMessage(int64(messageBytes))
} }
// streamWithReconnect handles streaming with automatic reconnection and exponential backoff
func (s *Streamer) streamWithReconnect(ctx context.Context) {
backoffDelay := minBackoffDelay
consecutiveFailures := 0
for {
select {
case <-ctx.Done():
s.logger.Info("Stream context cancelled, stopping reconnection attempts")
return
default:
}
// Attempt to stream
startTime := time.Now()
err := s.stream(ctx)
streamDuration := time.Since(startTime)
if err == nil {
// Clean exit (context cancelled)
return
}
// Log the error
s.logger.Error("Stream disconnected",
"error", err,
"consecutive_failures", consecutiveFailures+1,
"stream_duration", streamDuration)
s.metrics.SetConnected(false)
// Check if context is cancelled
if ctx.Err() != nil {
return
}
// If we streamed for more than 30 seconds, reset the backoff
// This indicates we had a successful connection that received data
if streamDuration > 30*time.Second {
s.logger.Info("Resetting backoff delay due to successful connection",
"stream_duration", streamDuration)
backoffDelay = minBackoffDelay
consecutiveFailures = 0
} else {
// Increment consecutive failures
consecutiveFailures++
}
// Wait with exponential backoff
s.logger.Info("Waiting before reconnection attempt",
"delay_seconds", backoffDelay.Seconds(),
"consecutive_failures", consecutiveFailures)
select {
case <-ctx.Done():
return
case <-time.After(backoffDelay):
// Double the backoff delay for next time, up to max
backoffDelay *= 2
if backoffDelay > maxBackoffDelay {
backoffDelay = maxBackoffDelay
}
}
}
}
func (s *Streamer) stream(ctx context.Context) error { func (s *Streamer) stream(ctx context.Context) error {
req, err := http.NewRequestWithContext(ctx, "GET", risLiveURL, nil) req, err := http.NewRequestWithContext(ctx, "GET", risLiveURL, nil)
if err != nil { if err != nil {
@@ -226,92 +457,87 @@ func (s *Streamer) stream(ctx context.Context) error {
rawHandler(string(line)) rawHandler(string(line))
} }
// Get current handlers // Parse the message first
s.mu.RLock() var wrapper ristypes.RISLiveMessage
handlers := make([]MessageHandler, len(s.handlers)) if err := json.Unmarshal(line, &wrapper); err != nil {
copy(handlers, s.handlers) // Log the error and return to trigger reconnection
s.mu.RUnlock() s.logger.Error("Failed to parse JSON",
"error", err,
"line", string(line),
"line_length", len(line))
// Try to acquire semaphore, drop message if at capacity return fmt.Errorf("JSON parse error: %w", err)
select { }
case s.semaphore <- struct{}{}:
// Successfully acquired semaphore, process message
go func(rawLine []byte, messageHandlers []MessageHandler) {
defer func() { <-s.semaphore }() // Release semaphore when done
// Parse the outer wrapper first // Check if it's a ris_message wrapper
var wrapper ristypes.RISLiveMessage if wrapper.Type != "ris_message" {
if err := json.Unmarshal(rawLine, &wrapper); err != nil { s.logger.Error("Unexpected wrapper type",
// Output the raw line and panic on parse failure "type", wrapper.Type,
fmt.Fprintf(os.Stderr, "Failed to parse JSON: %v\n", err) "line", string(line),
fmt.Fprintf(os.Stderr, "Raw line: %s\n", string(rawLine)) )
panic(fmt.Sprintf("JSON parse error: %v", err))
}
// Check if it's a ris_message wrapper continue
if wrapper.Type != "ris_message" { }
s.logger.Error("Unexpected wrapper type",
"type", wrapper.Type,
"line", string(rawLine),
)
return // Get the actual message
} msg := wrapper.Data
// Get the actual message // Parse the timestamp
msg := wrapper.Data msg.ParsedTimestamp = time.Unix(int64(msg.Timestamp), 0).UTC()
// Parse the timestamp // Process based on message type
msg.ParsedTimestamp = time.Unix(int64(msg.Timestamp), 0).UTC() switch msg.Type {
case "UPDATE":
// Process BGP UPDATE messages
// Will be dispatched to handlers
case "RIS_PEER_STATE":
// RIS peer state messages - silently ignore
continue
case "KEEPALIVE":
// BGP keepalive messages - silently process
continue
case "OPEN":
// BGP open messages
s.logger.Info("BGP session opened",
"peer", msg.Peer,
"peer_asn", msg.PeerASN,
)
// Process based on message type continue
switch msg.Type { case "NOTIFICATION":
case "UPDATE": // BGP notification messages (errors)
// Process BGP UPDATE messages s.logger.Warn("BGP notification",
// Will be handled by registered handlers "peer", msg.Peer,
case "RIS_PEER_STATE": "peer_asn", msg.PeerASN,
// RIS peer state messages - silently ignore )
case "KEEPALIVE":
// BGP keepalive messages - silently process
case "OPEN":
// BGP open messages
s.logger.Info("BGP session opened",
"peer", msg.Peer,
"peer_asn", msg.PeerASN,
)
case "NOTIFICATION":
// BGP notification messages (errors)
s.logger.Warn("BGP notification",
"peer", msg.Peer,
"peer_asn", msg.PeerASN,
)
case "STATE":
// Peer state changes - silently ignore
default:
fmt.Fprintf(
os.Stderr,
"UNKNOWN MESSAGE TYPE: %s\nRAW MESSAGE: %s\n",
msg.Type,
string(rawLine),
)
panic(fmt.Sprintf("Unknown RIS message type: %s", msg.Type))
}
// Call handlers synchronously within this goroutine continue
// This prevents unbounded goroutine growth at the handler level case "STATE":
for _, handler := range messageHandlers { // Peer state changes - silently ignore
if handler.WantsMessage(msg.Type) { continue
handler.HandleMessage(&msg)
}
}
}(append([]byte(nil), line...), handlers) // Copy the line to avoid data races
default: default:
// Semaphore is full, drop the message s.logger.Error("Unknown message type",
dropped := atomic.AddUint64(&s.droppedMessages, 1) "type", msg.Type,
if dropped%1000 == 0 { // Log every 1000 dropped messages "line", string(line),
s.logger.Warn("Dropping messages due to overload", "total_dropped", dropped, "max_handlers", maxConcurrentHandlers) )
panic(fmt.Sprintf("Unknown RIS message type: %s", msg.Type))
}
// Dispatch to interested handlers
s.mu.RLock()
for _, info := range s.handlers {
if info.handler.WantsMessage(msg.Type) {
select {
case info.queue <- &msg:
// Message queued successfully
default:
// Queue is full, drop the message
atomic.AddUint64(&info.metrics.droppedCount, 1)
atomic.AddUint64(&s.totalDropped, 1)
}
} }
} }
s.mu.RUnlock()
} }
if err := scanner.Err(); err != nil { if err := scanner.Err(); err != nil {

View File

@@ -3,12 +3,12 @@ package streamer
import ( import (
"testing" "testing"
"git.eeqj.de/sneak/routewatch/internal/logger"
"git.eeqj.de/sneak/routewatch/internal/metrics" "git.eeqj.de/sneak/routewatch/internal/metrics"
"log/slog"
) )
func TestNewStreamer(t *testing.T) { func TestNewStreamer(t *testing.T) {
logger := slog.Default() logger := logger.New()
metricsTracker := metrics.New() metricsTracker := metrics.New()
s := New(logger, metricsTracker) s := New(logger, metricsTracker)

View File

@@ -0,0 +1,228 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>AS{{.ASN.Number}} - {{.ASN.Handle}} - RouteWatch</title>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
margin: 0;
padding: 20px;
background: #f5f5f5;
color: #333;
}
.container {
max-width: 1200px;
margin: 0 auto;
background: white;
padding: 30px;
border-radius: 8px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
}
h1 {
margin: 0 0 10px 0;
color: #2c3e50;
}
.subtitle {
color: #7f8c8d;
margin-bottom: 30px;
}
.info-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
gap: 20px;
margin-bottom: 30px;
}
.info-card {
background: #f8f9fa;
padding: 20px;
border-radius: 6px;
border-left: 4px solid #3498db;
}
.info-label {
font-size: 14px;
color: #7f8c8d;
margin-bottom: 5px;
}
.info-value {
font-size: 24px;
font-weight: bold;
color: #2c3e50;
}
.prefix-section {
margin-top: 30px;
}
.prefix-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 20px;
}
.prefix-header h2 {
margin: 0;
color: #2c3e50;
}
.prefix-count {
background: #e74c3c;
color: white;
padding: 5px 12px;
border-radius: 20px;
font-size: 14px;
font-weight: bold;
}
.prefix-table {
width: 100%;
border-collapse: collapse;
background: white;
border: 1px solid #e0e0e0;
border-radius: 6px;
overflow: hidden;
}
.prefix-table th {
background: #34495e;
color: white;
padding: 12px;
text-align: left;
font-weight: 600;
}
.prefix-table td {
padding: 12px;
border-bottom: 1px solid #e0e0e0;
}
.prefix-table tr:hover {
background: #f8f9fa;
}
.prefix-table tr:last-child td {
border-bottom: none;
}
.prefix-link {
color: #3498db;
text-decoration: none;
font-family: monospace;
}
.prefix-link:hover {
text-decoration: underline;
}
.age {
color: #7f8c8d;
font-size: 14px;
}
.nav-link {
display: inline-block;
margin-bottom: 20px;
color: #3498db;
text-decoration: none;
}
.nav-link:hover {
text-decoration: underline;
}
.empty-state {
text-align: center;
padding: 40px;
color: #7f8c8d;
}
@media (max-width: 768px) {
.container {
padding: 20px;
}
.info-grid {
grid-template-columns: 1fr;
}
}
</style>
</head>
<body>
<div class="container">
<a href="/status" class="nav-link">← Back to Status</a>
<h1>AS{{.ASN.Number}}{{if .ASN.Handle}} - {{.ASN.Handle}}{{end}}</h1>
{{if .ASN.Description}}
<p class="subtitle">{{.ASN.Description}}</p>
{{end}}
<div class="info-grid">
<div class="info-card">
<div class="info-label">Total Prefixes</div>
<div class="info-value">{{.TotalCount}}</div>
</div>
<div class="info-card">
<div class="info-label">IPv4 Prefixes</div>
<div class="info-value">{{.IPv4Count}}</div>
</div>
<div class="info-card">
<div class="info-label">IPv6 Prefixes</div>
<div class="info-value">{{.IPv6Count}}</div>
</div>
<div class="info-card">
<div class="info-label">First Seen</div>
<div class="info-value">{{.ASN.FirstSeen.Format "2006-01-02"}}</div>
</div>
</div>
{{if .IPv4Prefixes}}
<div class="prefix-section">
<div class="prefix-header">
<h2>IPv4 Prefixes</h2>
<span class="prefix-count">{{.IPv4Count}}</span>
</div>
<table class="prefix-table">
<thead>
<tr>
<th>Prefix</th>
<th>Mask Length</th>
<th>Last Updated</th>
<th>Age</th>
</tr>
</thead>
<tbody>
{{range .IPv4Prefixes}}
<tr>
<td><a href="/prefix/{{.Prefix | urlEncode}}" class="prefix-link">{{.Prefix}}</a></td>
<td>/{{.MaskLength}}</td>
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
<td class="age">{{.LastUpdated | timeSince}}</td>
</tr>
{{end}}
</tbody>
</table>
</div>
{{end}}
{{if .IPv6Prefixes}}
<div class="prefix-section">
<div class="prefix-header">
<h2>IPv6 Prefixes</h2>
<span class="prefix-count">{{.IPv6Count}}</span>
</div>
<table class="prefix-table">
<thead>
<tr>
<th>Prefix</th>
<th>Mask Length</th>
<th>Last Updated</th>
<th>Age</th>
</tr>
</thead>
<tbody>
{{range .IPv6Prefixes}}
<tr>
<td><a href="/prefix/{{.Prefix | urlEncode}}" class="prefix-link">{{.Prefix}}</a></td>
<td>/{{.MaskLength}}</td>
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
<td class="age">{{.LastUpdated | timeSince}}</td>
</tr>
{{end}}
</tbody>
</table>
</div>
{{end}}
{{if eq .TotalCount 0}}
<div class="empty-state">
<p>No prefixes announced by this AS</p>
</div>
{{end}}
</div>
</body>
</html>

View File

@@ -0,0 +1,259 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{{.Prefix}} - RouteWatch</title>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
margin: 0;
padding: 20px;
background: #f5f5f5;
color: #333;
}
.container {
width: 90%;
max-width: 1600px;
margin: 0 auto;
background: white;
padding: 30px;
border-radius: 8px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
}
h1 {
margin: 0 0 10px 0;
color: #2c3e50;
font-family: monospace;
font-size: 28px;
}
.subtitle {
color: #7f8c8d;
margin-bottom: 30px;
}
.info-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 20px;
margin-bottom: 30px;
}
.info-card {
background: #f8f9fa;
padding: 20px;
border-radius: 6px;
border-left: 4px solid #3498db;
}
.info-label {
font-size: 14px;
color: #7f8c8d;
margin-bottom: 5px;
}
.info-value {
font-size: 24px;
font-weight: bold;
color: #2c3e50;
}
.routes-section {
margin-top: 30px;
}
.routes-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 20px;
}
.routes-header h2 {
margin: 0;
color: #2c3e50;
}
.route-count {
background: #e74c3c;
color: white;
padding: 5px 12px;
border-radius: 20px;
font-size: 14px;
font-weight: bold;
}
.route-table {
width: 100%;
border-collapse: collapse;
background: white;
border: 1px solid #e0e0e0;
border-radius: 6px;
overflow: hidden;
}
.route-table th {
background: #34495e;
color: white;
padding: 12px;
text-align: left;
font-weight: 600;
}
.route-table td {
padding: 12px;
border-bottom: 1px solid #e0e0e0;
white-space: nowrap;
}
.route-table tr:hover {
background: #f8f9fa;
}
.route-table tr:last-child td {
border-bottom: none;
}
.as-link {
color: #3498db;
text-decoration: none;
}
.as-link:hover {
text-decoration: underline;
}
.peer-ip {
font-family: monospace;
font-size: 14px;
color: #555;
}
.as-path {
font-family: monospace;
font-size: 13px;
color: #666;
max-width: 600px;
word-wrap: break-word;
white-space: normal !important;
line-height: 1.5;
}
.as-path .as-link {
font-weight: 600;
}
.age {
color: #7f8c8d;
font-size: 14px;
}
.nav-link {
display: inline-block;
margin-bottom: 20px;
color: #3498db;
text-decoration: none;
}
.nav-link:hover {
text-decoration: underline;
}
.origins-section {
margin-top: 30px;
background: #f8f9fa;
padding: 20px;
border-radius: 6px;
}
.origins-section h3 {
margin-top: 0;
color: #2c3e50;
}
.origin-list {
display: flex;
flex-wrap: wrap;
gap: 10px;
}
.origin-item {
background: white;
padding: 10px 15px;
border-radius: 4px;
border: 1px solid #e0e0e0;
}
.empty-state {
text-align: center;
padding: 40px;
color: #7f8c8d;
}
@media (max-width: 768px) {
.container {
padding: 20px;
}
.info-grid {
grid-template-columns: 1fr;
}
.route-table {
font-size: 14px;
}
.as-path {
max-width: 100%;
}
}
</style>
</head>
<body>
<div class="container">
<a href="/status" class="nav-link">← Back to Status</a>
<h1>{{.Prefix}}</h1>
<p class="subtitle">IPv{{.IPVersion}} Prefix{{if .MaskLength}} • /{{.MaskLength}}{{end}}</p>
<div class="info-grid">
<div class="info-card">
<div class="info-label">Seen from Peers</div>
<div class="info-value">{{.PeerCount}}</div>
</div>
<div class="info-card">
<div class="info-label">Origin ASNs</div>
<div class="info-value">{{.OriginCount}}</div>
</div>
<div class="info-card">
<div class="info-label">IP Version</div>
<div class="info-value">IPv{{.IPVersion}}</div>
</div>
</div>
{{if .Origins}}
<div class="origins-section">
<h3>Origin ASNs</h3>
<div class="origin-list">
{{range .Origins}}
<div class="origin-item">
<a href="/as/{{.Number}}" class="as-link">AS{{.Number}}</a>
{{if .Handle}} ({{.Handle}}){{end}}
<span style="color: #7f8c8d; margin-left: 10px;">{{.PeerCount}} peer{{if ne .PeerCount 1}}s{{end}}</span>
</div>
{{end}}
</div>
</div>
{{end}}
{{if .Routes}}
<div class="routes-section">
<div class="routes-header">
<h2>Route Details</h2>
<span class="route-count">{{.PeerCount}} route{{if ne .PeerCount 1}}s{{end}}</span>
</div>
<table class="route-table">
<thead>
<tr>
<th>Origin AS</th>
<th>Peer IP</th>
<th>AS Path</th>
<th>Next Hop</th>
<th>Last Updated</th>
<th>Age</th>
</tr>
</thead>
<tbody>
{{range .Routes}}
<tr>
<td>
<a href="/as/{{.OriginASN}}" class="as-link">AS{{.OriginASN}}</a>
</td>
<td class="peer-ip">{{.PeerIP}}</td>
<td class="as-path">{{range $i, $as := .ASPathWithHandle}}{{if $i}} → {{end}}<a href="/as/{{$as.Number}}" class="as-link">{{if $as.Handle}}{{$as.Handle}}{{else}}AS{{$as.Number}}{{end}}</a>{{end}}</td>
<td class="peer-ip">{{.NextHop}}</td>
<td>{{.LastUpdated.Format "2006-01-02 15:04:05"}}</td>
<td class="age">{{.LastUpdated | timeSince}}</td>
</tr>
{{end}}
</tbody>
</table>
</div>
{{else}}
<div class="empty-state">
<p>No routes found for this prefix</p>
</div>
{{end}}
</div>
</body>
</html>

View File

@@ -0,0 +1,108 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Prefixes with /{{ .MaskLength }} - RouteWatch</title>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
max-width: 1200px;
margin: 0 auto;
padding: 20px;
background: #f5f5f5;
}
h1 {
color: #333;
margin-bottom: 10px;
}
.subtitle {
color: #666;
margin-bottom: 30px;
}
.info-card {
background: white;
padding: 20px;
border-radius: 8px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
margin-bottom: 20px;
}
table {
width: 100%;
border-collapse: collapse;
background: white;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
border-radius: 8px;
overflow: hidden;
}
th {
background: #f8f9fa;
padding: 12px;
text-align: left;
font-weight: 600;
color: #333;
border-bottom: 2px solid #dee2e6;
}
td {
padding: 12px;
border-bottom: 1px solid #eee;
}
tr:last-child td {
border-bottom: none;
}
tr:hover {
background: #f8f9fa;
}
a {
color: #0066cc;
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
.prefix-link {
font-family: 'SF Mono', Monaco, 'Cascadia Mono', 'Roboto Mono', Consolas, 'Courier New', monospace;
}
.as-link {
white-space: nowrap;
}
.age {
color: #666;
font-size: 0.9em;
}
.back-link {
display: inline-block;
margin-bottom: 20px;
color: #0066cc;
}
</style>
</head>
<body>
<a href="/status" class="back-link">← Back to Status</a>
<h1>IPv{{ .IPVersion }} Prefixes with /{{ .MaskLength }}</h1>
<p class="subtitle">Showing {{ .Count }} randomly selected prefixes</p>
<table>
<thead>
<tr>
<th>Prefix</th>
<th>Age</th>
<th>Origin AS</th>
</tr>
</thead>
<tbody>
{{ range .Prefixes }}
<tr>
<td><a href="/prefix/{{ .Prefix | urlEncode }}" class="prefix-link">{{ .Prefix }}</a></td>
<td class="age">{{ .Age }}</td>
<td>
<a href="/as/{{ .OriginASN }}" class="as-link">
AS{{ .OriginASN }}{{ if .OriginASDescription }} ({{ .OriginASDescription }}){{ end }}
</a>
</td>
</tr>
{{ end }}
</tbody>
</table>
</body>
</html>

View File

@@ -46,9 +46,19 @@
color: #666; color: #666;
} }
.metric-value { .metric-value {
font-weight: 600; font-family: 'SF Mono', Monaco, 'Cascadia Mono', 'Roboto Mono', Consolas, 'Courier New', monospace;
color: #333; color: #333;
} }
.metric-value.metric-link {
text-decoration: underline;
text-decoration-style: dashed;
text-underline-offset: 2px;
cursor: pointer;
}
.metric-value.metric-link:hover {
color: #0066cc;
text-decoration-style: solid;
}
.connected { .connected {
color: #22c55e; color: #22c55e;
} }
@@ -69,7 +79,7 @@
<div id="error" class="error" style="display: none;"></div> <div id="error" class="error" style="display: none;"></div>
<div class="status-grid"> <div class="status-grid">
<div class="status-card"> <div class="status-card">
<h2>Connection Status</h2> <h2>RouteWatch</h2>
<div class="metric"> <div class="metric">
<span class="metric-label">Status</span> <span class="metric-label">Status</span>
<span class="metric-value" id="connected">-</span> <span class="metric-value" id="connected">-</span>
@@ -78,6 +88,18 @@
<span class="metric-label">Uptime</span> <span class="metric-label">Uptime</span>
<span class="metric-value" id="uptime">-</span> <span class="metric-value" id="uptime">-</span>
</div> </div>
<div class="metric">
<span class="metric-label">Go Version</span>
<span class="metric-value" id="go_version">-</span>
</div>
<div class="metric">
<span class="metric-label">Goroutines</span>
<span class="metric-value" id="goroutines">-</span>
</div>
<div class="metric">
<span class="metric-label">Memory Usage</span>
<span class="metric-value" id="memory_usage">-</span>
</div>
</div> </div>
<div class="status-card"> <div class="status-card">
@@ -110,6 +132,26 @@
<span class="metric-label">Total Prefixes</span> <span class="metric-label">Total Prefixes</span>
<span class="metric-value" id="prefixes">-</span> <span class="metric-value" id="prefixes">-</span>
</div> </div>
<div class="metric">
<span class="metric-label">Peerings</span>
<span class="metric-value" id="peerings">-</span>
</div>
<div class="metric">
<span class="metric-label">Peers</span>
<span class="metric-value" id="peers">-</span>
</div>
<div class="metric">
<span class="metric-label">Database Size</span>
<span class="metric-value" id="database_size">-</span>
</div>
</div>
<div class="status-card">
<h2>Routing Table</h2>
<div class="metric">
<span class="metric-label">Live Routes</span>
<span class="metric-value" id="live_routes">-</span>
</div>
<div class="metric"> <div class="metric">
<span class="metric-label">IPv4 Prefixes</span> <span class="metric-label">IPv4 Prefixes</span>
<span class="metric-value" id="ipv4_prefixes">-</span> <span class="metric-value" id="ipv4_prefixes">-</span>
@@ -119,16 +161,44 @@
<span class="metric-value" id="ipv6_prefixes">-</span> <span class="metric-value" id="ipv6_prefixes">-</span>
</div> </div>
<div class="metric"> <div class="metric">
<span class="metric-label">Peerings</span> <span class="metric-label">IPv4 Routes</span>
<span class="metric-value" id="peerings">-</span> <span class="metric-value" id="ipv4_routes">-</span>
</div> </div>
<div class="metric"> <div class="metric">
<span class="metric-label">Live Routes</span> <span class="metric-label">IPv6 Routes</span>
<span class="metric-value" id="live_routes">-</span> <span class="metric-value" id="ipv6_routes">-</span>
</div>
<div class="metric">
<span class="metric-label">IPv4 Updates/sec</span>
<span class="metric-value" id="ipv4_updates_per_sec">-</span>
</div>
<div class="metric">
<span class="metric-label">IPv6 Updates/sec</span>
<span class="metric-value" id="ipv6_updates_per_sec">-</span>
</div> </div>
</div> </div>
</div> </div>
<div class="status-grid">
<div class="status-card">
<h2>IPv4 Prefix Distribution</h2>
<div id="ipv4-prefix-distribution">
<!-- Will be populated dynamically -->
</div>
</div>
<div class="status-card">
<h2>IPv6 Prefix Distribution</h2>
<div id="ipv6-prefix-distribution">
<!-- Will be populated dynamically -->
</div>
</div>
</div>
<div id="handler-stats-container" class="status-grid">
<!-- Handler stats will be dynamically added here -->
</div>
<script> <script>
function formatBytes(bytes) { function formatBytes(bytes) {
if (bytes === 0) return '0 B'; if (bytes === 0) return '0 B';
@@ -142,6 +212,80 @@
return num.toLocaleString(); return num.toLocaleString();
} }
function formatProcessingTime(ms) {
if (ms < 0.001) {
return (ms * 1000).toFixed(0) + ' µs';
} else if (ms < 0.01) {
return (ms * 1000).toFixed(1) + ' µs';
} else if (ms < 1) {
return ms.toFixed(3) + ' ms';
} else {
return ms.toFixed(2) + ' ms';
}
}
function updatePrefixDistribution(elementId, distribution) {
const container = document.getElementById(elementId);
container.innerHTML = '';
if (!distribution || distribution.length === 0) {
container.innerHTML = '<div class="metric"><span class="metric-label">No data</span></div>';
return;
}
// Sort by mask length
distribution.sort((a, b) => a.mask_length - b.mask_length);
distribution.forEach(item => {
const metric = document.createElement('div');
metric.className = 'metric';
metric.innerHTML = `
<span class="metric-label">/${item.mask_length}</span>
<a href="/prefixlength/${item.mask_length}" class="metric-value metric-link">${formatNumber(item.count)}</a>
`;
container.appendChild(metric);
});
}
function updateHandlerStats(handlerStats) {
const container = document.getElementById('handler-stats-container');
container.innerHTML = '';
handlerStats.forEach(handler => {
const card = document.createElement('div');
card.className = 'status-card';
// Extract handler name (remove package prefix)
const handlerName = handler.name.split('.').pop();
card.innerHTML = `
<h2>${handlerName}</h2>
<div class="metric">
<span class="metric-label">Queue</span>
<span class="metric-value">${handler.queue_length}/${handler.queue_capacity}</span>
</div>
<div class="metric">
<span class="metric-label">Processed</span>
<span class="metric-value">${formatNumber(handler.processed_count)}</span>
</div>
<div class="metric">
<span class="metric-label">Dropped</span>
<span class="metric-value ${handler.dropped_count > 0 ? 'disconnected' : ''}">${formatNumber(handler.dropped_count)}</span>
</div>
<div class="metric">
<span class="metric-label">Avg Time</span>
<span class="metric-value">${formatProcessingTime(handler.avg_process_time_ms)}</span>
</div>
<div class="metric">
<span class="metric-label">Min/Max Time</span>
<span class="metric-value">${formatProcessingTime(handler.min_process_time_ms)} / ${formatProcessingTime(handler.max_process_time_ms)}</span>
</div>
`;
container.appendChild(card);
});
}
function updateStatus() { function updateStatus() {
fetch('/api/v1/stats') fetch('/api/v1/stats')
.then(response => response.json()) .then(response => response.json())
@@ -163,6 +307,9 @@
// Update all metrics // Update all metrics
document.getElementById('uptime').textContent = data.uptime; document.getElementById('uptime').textContent = data.uptime;
document.getElementById('go_version').textContent = data.go_version;
document.getElementById('goroutines').textContent = formatNumber(data.goroutines);
document.getElementById('memory_usage').textContent = data.memory_usage;
document.getElementById('total_messages').textContent = formatNumber(data.total_messages); document.getElementById('total_messages').textContent = formatNumber(data.total_messages);
document.getElementById('messages_per_sec').textContent = data.messages_per_sec.toFixed(1); document.getElementById('messages_per_sec').textContent = data.messages_per_sec.toFixed(1);
document.getElementById('total_bytes').textContent = formatBytes(data.total_bytes); document.getElementById('total_bytes').textContent = formatBytes(data.total_bytes);
@@ -172,7 +319,20 @@
document.getElementById('ipv4_prefixes').textContent = formatNumber(data.ipv4_prefixes); document.getElementById('ipv4_prefixes').textContent = formatNumber(data.ipv4_prefixes);
document.getElementById('ipv6_prefixes').textContent = formatNumber(data.ipv6_prefixes); document.getElementById('ipv6_prefixes').textContent = formatNumber(data.ipv6_prefixes);
document.getElementById('peerings').textContent = formatNumber(data.peerings); document.getElementById('peerings').textContent = formatNumber(data.peerings);
document.getElementById('peers').textContent = formatNumber(data.peers);
document.getElementById('database_size').textContent = formatBytes(data.database_size_bytes);
document.getElementById('live_routes').textContent = formatNumber(data.live_routes); document.getElementById('live_routes').textContent = formatNumber(data.live_routes);
document.getElementById('ipv4_routes').textContent = formatNumber(data.ipv4_routes);
document.getElementById('ipv6_routes').textContent = formatNumber(data.ipv6_routes);
document.getElementById('ipv4_updates_per_sec').textContent = data.ipv4_updates_per_sec.toFixed(1);
document.getElementById('ipv6_updates_per_sec').textContent = data.ipv6_updates_per_sec.toFixed(1);
// Update handler stats
updateHandlerStats(data.handler_stats || []);
// Update prefix distribution
updatePrefixDistribution('ipv4-prefix-distribution', data.ipv4_prefix_distribution);
updatePrefixDistribution('ipv6-prefix-distribution', data.ipv6_prefix_distribution);
// Clear any errors // Clear any errors
document.getElementById('error').style.display = 'none'; document.getElementById('error').style.display = 'none';

View File

@@ -4,15 +4,29 @@ package templates
import ( import (
_ "embed" _ "embed"
"html/template" "html/template"
"net/url"
"sync" "sync"
"time"
) )
//go:embed status.html //go:embed status.html
var statusHTML string var statusHTML string
//go:embed as_detail.html
var asDetailHTML string
//go:embed prefix_detail.html
var prefixDetailHTML string
//go:embed prefix_length.html
var prefixLengthHTML string
// Templates contains all parsed templates // Templates contains all parsed templates
type Templates struct { type Templates struct {
Status *template.Template Status *template.Template
ASDetail *template.Template
PrefixDetail *template.Template
PrefixLength *template.Template
} }
var ( var (
@@ -22,17 +36,79 @@ var (
once sync.Once once sync.Once
) )
const (
hoursPerDay = 24
daysPerMonth = 30
)
// timeSince returns a human-readable duration since the given time
func timeSince(t time.Time) string {
duration := time.Since(t)
if duration < time.Minute {
return "just now"
}
if duration < time.Hour {
minutes := int(duration.Minutes())
if minutes == 1 {
return "1 minute ago"
}
return duration.Truncate(time.Minute).String() + " ago"
}
if duration < hoursPerDay*time.Hour {
hours := int(duration.Hours())
if hours == 1 {
return "1 hour ago"
}
return duration.Truncate(time.Hour).String() + " ago"
}
days := int(duration.Hours() / hoursPerDay)
if days == 1 {
return "1 day ago"
}
if days < daysPerMonth {
return duration.Truncate(hoursPerDay*time.Hour).String() + " ago"
}
return t.Format("2006-01-02")
}
// initTemplates parses all embedded templates // initTemplates parses all embedded templates
func initTemplates() { func initTemplates() {
var err error var err error
defaultTemplates = &Templates{} defaultTemplates = &Templates{}
// Create common template functions
funcs := template.FuncMap{
"timeSince": timeSince,
"urlEncode": url.QueryEscape,
}
// Parse status template // Parse status template
defaultTemplates.Status, err = template.New("status").Parse(statusHTML) defaultTemplates.Status, err = template.New("status").Parse(statusHTML)
if err != nil { if err != nil {
panic("failed to parse status template: " + err.Error()) panic("failed to parse status template: " + err.Error())
} }
// Parse AS detail template
defaultTemplates.ASDetail, err = template.New("asDetail").Funcs(funcs).Parse(asDetailHTML)
if err != nil {
panic("failed to parse AS detail template: " + err.Error())
}
// Parse prefix detail template
defaultTemplates.PrefixDetail, err = template.New("prefixDetail").Funcs(funcs).Parse(prefixDetailHTML)
if err != nil {
panic("failed to parse prefix detail template: " + err.Error())
}
// Parse prefix length template
defaultTemplates.PrefixLength, err = template.New("prefixLength").Funcs(funcs).Parse(prefixLengthHTML)
if err != nil {
panic("failed to parse prefix length template: " + err.Error())
}
} }
// Get returns the singleton Templates instance // Get returns the singleton Templates instance
@@ -46,3 +122,18 @@ func Get() *Templates {
func StatusTemplate() *template.Template { func StatusTemplate() *template.Template {
return Get().Status return Get().Status
} }
// ASDetailTemplate returns the parsed AS detail template
func ASDetailTemplate() *template.Template {
return Get().ASDetail
}
// PrefixDetailTemplate returns the parsed prefix detail template
func PrefixDetailTemplate() *template.Template {
return Get().PrefixDetail
}
// PrefixLengthTemplate returns the parsed prefix length template
func PrefixLengthTemplate() *template.Template {
return Get().PrefixLength
}

178395
log.txt Normal file

File diff suppressed because it is too large Load Diff