Write each handler batch from its flush goroutine, outside the mutex #83

Open
clawbot wants to merge 1 commits from issue-65-batch-write-off-mutex into next
Collaborator

Fixes #65.

The AS, peer and prefix handlers wrote their batch to the database while holding the mutex that HandleMessage needs, and HandleMessage wrote a full batch itself. While a write ran, the handler took nothing off its queue, and the streamer dropped updates once the queue was half full.

Now each handler writes only from its flush goroutine: it takes the batch under the mutex, puts an empty one in its place, releases the mutex, then writes. HandleMessage only adds to the batch; a full batch wakes the flush goroutine through a one-slot channel, so a wake-up sent during a write is picked up as soon as that write ends. Stop still has the flush goroutine write what the handler holds before it returns.

The new tests use a store whose batch writes wait until the test releases them. They check that HandleMessage returns while a write is held, and that Stop writes the update handled meanwhile.

  • Judgement call: while a write runs, the next batch has no size limit. A database that stays slower than the feed now grows memory instead of dropping updates; the README memory section says so.
  • Deviation: an update that reaches a handler after its Stop is no longer written, even when it fills a batch, since only the flush goroutine writes. #69 changes the shutdown order so that none arrive.

Model: opus-5-5

Fixes https://git.eeqj.de/sneak/routewatch/issues/65. The AS, peer and prefix handlers wrote their batch to the database while holding the mutex that `HandleMessage` needs, and `HandleMessage` wrote a full batch itself. While a write ran, the handler took nothing off its queue, and the streamer dropped updates once the queue was half full. Now each handler writes only from its flush goroutine: it takes the batch under the mutex, puts an empty one in its place, releases the mutex, then writes. `HandleMessage` only adds to the batch; a full batch wakes the flush goroutine through a one-slot channel, so a wake-up sent during a write is picked up as soon as that write ends. `Stop` still has the flush goroutine write what the handler holds before it returns. The new tests use a store whose batch writes wait until the test releases them. They check that `HandleMessage` returns while a write is held, and that `Stop` writes the update handled meanwhile. - Judgement call: while a write runs, the next batch has no size limit. A database that stays slower than the feed now grows memory instead of dropping updates; the README memory section says so. - Deviation: an update that reaches a handler after its `Stop` is no longer written, even when it fills a batch, since only the flush goroutine writes. https://git.eeqj.de/sneak/routewatch/issues/69 changes the shutdown order so that none arrive. Model: opus-5-5
clawbot added the needs-review label 2026-10-14 17:17:25 +02:00
clawbot self-assigned this 2026-10-14 17:17:25 +02:00
clawbot force-pushed issue-65-batch-write-off-mutex from e33371b74b to f602994a81 2026-10-14 17:47:39 +02:00 Compare
clawbot force-pushed issue-65-batch-write-off-mutex from f602994a81 to d8b65e9018 2026-10-14 19:19:31 +02:00 Compare
clawbot added 1 commit 2026-10-14 19:50:51 +02:00
The AS, peer and prefix handlers wrote their batch to the database while
holding the mutex HandleMessage needs, and HandleMessage wrote a full batch
itself. For as long as a write took, the handler took nothing off its queue,
and the streamer dropped updates once the queue was half full.

Now only the flush goroutine writes: it takes the batch under the mutex,
releases it, then writes. A full batch wakes the flush goroutine instead.
Stop still writes what the handler holds.

While a write runs, the next batch has no size limit; the README memory
section says so.

Model: opus-5-5
clawbot force-pushed issue-65-batch-write-off-mutex from d8b65e9018 to 73793e68e0 2026-10-14 19:50:51 +02:00 Compare
Some checks are pending
check / check (push) Waiting to run
You are not authorized to merge this pull request.
This pull request can be merged automatically.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin issue-65-batch-write-off-mutex:issue-65-batch-write-off-mutex
git checkout issue-65-batch-write-off-mutex
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/routewatch#83