Shrink the four handler queues from 100,000 to 20,000 (closes #11)
check / check (push) Successful in 2m52s
check / check (push) Successful in 2m52s
Each handler queue held 100,000 message pointers; a message is retained until the slowest handler drains it, so all four full was a derived worst case near 800 MiB. Twenty thousand is about four seconds of feed at peak and caps that at roughly 160 MiB. The streamer already drops rather than blocks on a full queue, so the smaller bound is safe. Batch sizes are unchanged. The largest, asnBatchSize, is 30,000 and now exceeds its queue, but each queued message contributes every ASN in its path, and every handler also flushes on its own timer regardless of fill, so batches still flush and no size change is warranted. Model: opus-4-8
This commit was merged in pull request #21.
This commit is contained in:
@@ -14,9 +14,11 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
// prefixHandlerQueueSize is the queue capacity for prefix tracking operations
|
||||
// DO NOT set this higher than 100000 without explicit instructions
|
||||
prefixHandlerQueueSize = 100000
|
||||
// prefixHandlerQueueSize is the queue capacity for prefix tracking
|
||||
// operations, about 4 seconds of feed at peak. The streamer drops rather
|
||||
// than blocks when a queue is full, so this bounds memory. Batches still
|
||||
// flush on a timer (prefixBatchTimeout).
|
||||
prefixHandlerQueueSize = 20000
|
||||
|
||||
// prefixBatchSize is the number of prefix updates to batch together
|
||||
prefixBatchSize = 25000
|
||||
|
||||
Reference in New Issue
Block a user