In PrefixHandler, ASHandler and PeerHandler (internal/routewatch/prefixhandler.go, ashandler.go, peerhandler.go), flushBatchLocked writes the batch to the database while holding the handler's mutex, and HandleMessage needs that mutex to add an update; when the batch is full, HandleMessage does the write itself. For as long as a write takes, including any wait for another handler's write to finish, the handler's worker takes nothing off its queue. A queue holds 20,000 messages, about four seconds of feed, and the streamer starts dropping when it is half full (shouldDropForBackpressure in internal/streamer/streamer.go), so any write longer than about two seconds loses updates.
Failure scenario: the 24-hour run on #3 (last comment) dropped updates all day: the prefix handler 16 percent (5 to 30 percent in any hour), the AS handler 27 million, the peer handler 11 million. A dropped update loses every announcement and withdrawal in it, and a lost withdrawal leaves its route in the table for good, since nothing else removes it. Drops at the AS and peer handlers only delay last_seen and new rows.
Severity: high
Definition of done
A test, failing before the fix, for each of the three handlers: with a store whose batch write blocks until the test releases it, HandleMessage returns while a write is blocked.
Each handler writes only from its flush goroutine: it takes the pending batch under the mutex, releases the mutex, then writes. HandleMessage never writes; a full batch wakes the flush goroutine.
make check passes.
Model: opus-5-5
In `PrefixHandler`, `ASHandler` and `PeerHandler` (`internal/routewatch/prefixhandler.go`, `ashandler.go`, `peerhandler.go`), `flushBatchLocked` writes the batch to the database while holding the handler's mutex, and `HandleMessage` needs that mutex to add an update; when the batch is full, `HandleMessage` does the write itself. For as long as a write takes, including any wait for another handler's write to finish, the handler's worker takes nothing off its queue. A queue holds 20,000 messages, about four seconds of feed, and the streamer starts dropping when it is half full (`shouldDropForBackpressure` in `internal/streamer/streamer.go`), so any write longer than about two seconds loses updates.
Failure scenario: the 24-hour run on https://git.eeqj.de/sneak/routewatch/issues/3 (last comment) dropped updates all day: the prefix handler 16 percent (5 to 30 percent in any hour), the AS handler 27 million, the peer handler 11 million. A dropped update loses every announcement and withdrawal in it, and a lost withdrawal leaves its route in the table for good, since nothing else removes it. Drops at the AS and peer handlers only delay `last_seen` and new rows.
Severity: high
## Definition of done
- A test, failing before the fix, for each of the three handlers: with a store whose batch write blocks until the test releases it, `HandleMessage` returns while a write is blocked.
- Each handler writes only from its flush goroutine: it takes the pending batch under the mutex, releases the mutex, then writes. `HandleMessage` never writes; a full batch wakes the flush goroutine.
- `make check` passes.
Model: opus-5-5
clawbot
self-assigned this 2026-10-14 14:50:31 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
In
PrefixHandler,ASHandlerandPeerHandler(internal/routewatch/prefixhandler.go,ashandler.go,peerhandler.go),flushBatchLockedwrites the batch to the database while holding the handler's mutex, andHandleMessageneeds that mutex to add an update; when the batch is full,HandleMessagedoes the write itself. For as long as a write takes, including any wait for another handler's write to finish, the handler's worker takes nothing off its queue. A queue holds 20,000 messages, about four seconds of feed, and the streamer starts dropping when it is half full (shouldDropForBackpressureininternal/streamer/streamer.go), so any write longer than about two seconds loses updates.Failure scenario: the 24-hour run on #3 (last comment) dropped updates all day: the prefix handler 16 percent (5 to 30 percent in any hour), the AS handler 27 million, the peer handler 11 million. A dropped update loses every announcement and withdrawal in it, and a lost withdrawal leaves its route in the table for good, since nothing else removes it. Drops at the AS and peer handlers only delay
last_seenand new rows.Severity: high
Definition of done
HandleMessagereturns while a write is blocked.HandleMessagenever writes; a full batch wakes the flush goroutine.make checkpasses.Model: opus-5-5