sneak/routewatch
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
88 Commits 3 Branches 0 Tags 41 MiB
1ec0b3e7ca
Commit Graph

4 Commits

Author SHA1 Message Date
sneak
81267431f7 Increase batch sizes to improve write throughput 
- Increase prefix batch size from 5K to 20K
- Increase ASN batch size from 10K to 30K
- Add comments warning not to reduce batch timeouts
- Add comments warning not to increase queue sizes above 100K
- Maintains existing batch timeouts for efficiency
 2025-07-28 18:27:42 +02:00
sneak
40d7f0185b Optimize database batch operations with prepared statements 
- Add prepared statements to all batch operations for better performance
- Fix database lock contention by properly batching operations
- Update SQLite settings for extreme performance (8GB cache, sync OFF)
- Add proper error handling for statement closing
- Update tests to properly track batch operations
 2025-07-28 17:21:40 +02:00
sneak
bc640b0b37 Reduce all handler queue sizes to 100,000 2025-07-28 02:50:05 +02:00
sneak
1157003db7 Refactor database handlers and optimize PeeringHandler 
- Create PeeringHandler for asn_peerings table maintenance
- Rename DBHandler to ASHandler (now only handles asns table)
- Move prefixes table maintenance to PrefixHandler
- Optimize PeeringHandler with in-memory AS path tracking:
  - Stores AS paths in memory with timestamps
  - Processes peerings in batch every 2 minutes
  - Prunes old paths (>30 minutes) every 5 minutes
  - Normalizes peerings with lower AS number first
- Each handler now has a single responsibility:
  - ASHandler: asns table
  - PeerHandler: bgp_peers table
  - PrefixHandler: prefixes and live_routes tables
  - PeeringHandler: asn_peerings table
 2025-07-28 02:31:04 +02:00