Take every down peer out of the table in one walk (closes #225)

The prefix handler ran one PeerDown per peer, one after another, and
stopped taking messages at 20,000 held-back messages. Sessions going
down together queued up their walks, so a returning full-table peer
could reach that limit.

The table's PeersDown now takes several peers out of every prefix in
one walk. One walk runs at a time. The next one takes every peer whose
session went down meanwhile, so a message is held back for about two
walks at most. The limit is now 100,000 messages, about 100 MiB. The
status page shows how many messages each handler holds back.

Model: opus-5-5

README.md

@@ -652,10 +652,9 @@ What happens at each limit:
 
 - Held-back messages: the prefix handler holds back a peer's messages after the
   peer's session goes down. It applies them once the peer has left every prefix.
-  It holds at most 150,000 such messages (`maxHeldBackMessages`), about 150 MiB.
-  The limit is enough when the sessions of four full-table peers go down
-  together. At the limit it takes nothing more off its queue until it has
-  applied one of them.
+  It holds at most 100,000 such messages (`maxHeldBackMessages`), about 100 MiB.
+  At that number it takes nothing more off its queue until it has applied one of
+  them.
 
 With `DEBUG=routewatch` the daemon logs a `System stats` line every 60 seconds
 with the goroutine count and Go memory figures.

TODO.md

@@ -39,12 +39,10 @@ tested.
   goes on taking messages. The table's `PeersDown` takes several peers out of
   every prefix in one walk over the table. One walk runs at a time. The next one
   takes every peer whose session went down meanwhile. So a peer's messages are
-  held back for two walks at most. Each walk takes about 2.6 seconds for each
-  full-table peer it takes out. The handler holds back at most 150,000 messages,
-  about 150 MiB. The limit is enough when the sessions of four full-table peers
-  go down together. The status page shows how many messages the handler holds
-  back. Each peer used to have a walk of its own, and the handler stopped taking
-  messages at 20,000 held back (closes #225)
+  held back for about two walks at most. The handler holds back at most 100,000
+  messages, about 100 MiB. The status page shows how many it holds back. Each
+  peer used to have a walk of its own, and the handler stopped taking messages
+  at 20,000 held back (closes #225)
 - 2026-11-02: like a failed WHOIS lookup, a lookup skipped because its RIR is
   paused makes the fetcher wait longer before the next lookup. The longest wait
   is one minute. The status page does not count a skip as a failure. A skip used

internal/routetable/routetable.go

@@ -242,8 +242,7 @@ func (t *Table) PeersDown(peers []string) {
 					if o != nil {
 						t.leave(prefix, o, number)
 
-						// leave may have removed an origin of the
-						// prefix, or the prefix itself.
+						// leave may have removed an origin of the prefix.
 						origins = prefixes[prefix]
 					}
 				}

internal/routetable/routetable_test.go

@@ -284,6 +284,7 @@ func TestWithdrawalsAndPeerDowns(t *testing.T) {
 				Routes: []routetable.Route{routeOf(peerA, asA, 10)}}},
 		},
 		{
+			// peerA's leaving removes the origin asA, before peerB leaves.
 			name: "peers going down together each leave the prefix",
 			steps: []step{announce(peerA, asA, 10), announce(peerB, asB, 20),
 				announce(peerC, asB, 30), peersDown(peerA, peerB)},
@@ -340,32 +341,6 @@ func TestPeersDownLeavesEveryPrefix(t *testing.T) {
 	}
 }
 
-// TestPeersDownTakesOutPeersOfDifferentPrefixes takes peerA and peerB out in
-// one PeersDown. peerA carries only prefixV4, and peerB only prefixV6. So
-// peerA's leaving removes prefixV4 before PeersDown looks for peerB there. Both
-// prefixes must then be gone.
-func TestPeersDownTakesOutPeersOfDifferentPrefixes(t *testing.T) {
-	t.Parallel()
-
-	table := routetable.New(routetable.Params{})
-	prefixes := []netip.Prefix{
-		netip.MustParsePrefix(prefixV4),
-		netip.MustParsePrefix(prefixV6),
-	}
-
-	table.Announce(prefixes[0], asA, routeOf(peerA, asA, 10))
-	table.Announce(prefixes[1], asB, routeOf(peerB, asB, 20))
-	table.PeersDown([]string{peerA, peerB})
-
-	for _, prefix := range prefixes {
-		got := table.Origins(prefix)
-		if got != nil {
-			t.Errorf("Origins(%s) = %+v after its only peer went down, want nil",
-				prefix, got)
-		}
-	}
-}
-
 func TestCountsOriginsWithoutKeptRoute(t *testing.T) {
 	t.Parallel()
 

internal/routewatch/prefixhandler.go

@@ -46,19 +46,13 @@ const (
 	maxTableChangesPerWrite = 20000
 
 	// maxHeldBackMessages is the most messages the handler holds back from
-	// peers whose session went down. A peer's messages are held back while
-	// up to two PeersDown calls run: the one running when its session goes
-	// down, and the next, which takes the peer out. A call takes about 2.6
-	// seconds for each full-table peer it takes out. Take a full-table peer
-	// whose session goes down together with those of three other full-table
-	// peers. The two calls then take about 10.4 seconds, 2.6 for each of the
-	// four. The feed peaks at about 14,000 messages a second, so the four
-	// peers send at most about 146,000 messages in that time. Once the second
-	// call has returned, the handler applies the held-back messages faster
-	// than the feed sends more. A message takes about 1 KiB, so 150,000 take
-	// about 150 MiB (README "Memory"). At this number HandleMessage waits
-	// until one of them is applied.
-	maxHeldBackMessages = 150000
+	// peers whose session went down. A message is held back for about two
+	// full-table PeersDown calls at most, about 5 seconds. At its peak of
+	// about 14,000 messages a second, the feed sends about 70,000 in that
+	// time. A message takes about 1 KiB, so 100,000 take about 100 MiB
+	// (README "Memory"). At this number HandleMessage waits until one of them
+	// is applied.
+	maxHeldBackMessages = 100000
 )
 
 // PrefixHandler keeps the routing table in memory current from the feed. It
@@ -451,8 +445,7 @@ func (h *PrefixHandler) peerDown() {
 // mu for one message at a time, so HandleMessage takes new messages in
 // between. The peer's new messages join the end of the held-back ones. A
 // held-back message that says the peer's session went down again adds the
-// peer to downPeers. The rest stay held back until the next PeersDown has
-// returned.
+// peer to downPeers. The rest stay held back until the next PeersDown.
 func (h *PrefixHandler) applyHeldBack(peer string) {
 	for {
 		h.mu.Lock()