Bound the peering handler AS-path map and stop copying it every 30 s #10

New Issue

2026-09-21T14:54:36+02:00

clawbot commented

2026-09-21 14:54:36 +02:00

Part of #3 (analysis and plan in the comment dated 2026-09-21 14:34 there).

asPaths in internal/routewatch/peeringhandler.go:50 keeps every distinct AS path of the last 30 minutes (measured 1.1 M entries, ~184 bytes each, still growing). Every 30 s processPeerings copies the whole map and calls RecordPeering for every peering in the whole window, 129-150 k transactions per run, so it runs almost continuously and drives a 300 MiB heap sawtooth.

Requirements

processPeerings takes the current map and replaces it with a fresh empty one under the lock (no copy), so each path is processed once. RecordPeering already upserts last_seen, so stored results are unchanged.
Add a constant maxTrackedPaths = 500000; when the map is full, new paths are dropped and counted; expose the count where the handler's other metrics are exposed, if there is such a place, otherwise log it once per run.
Remove the 30-minute prune and its ticker, which the swap makes dead code.
Files: internal/routewatch/peeringhandler.go and its test.

Definition of done

Tests: the map is empty after ProcessPeeringsNow; a path seen before and after a run is recorded in both runs; the cap is enforced and the drop counter counts.
make check green. Commit title ends (closes #N).

Model: fable-5-1

Part of https://git.eeqj.de/sneak/routewatch/issues/3 (analysis and plan in the comment dated 2026-09-21 14:34 there). `asPaths` in `internal/routewatch/peeringhandler.go:50` keeps every distinct AS path of the last 30 minutes (measured 1.1 M entries, ~184 bytes each, still growing). Every 30 s `processPeerings` copies the whole map and calls `RecordPeering` for every peering in the whole window, 129-150 k transactions per run, so it runs almost continuously and drives a 300 MiB heap sawtooth. ## Requirements - `processPeerings` takes the current map and replaces it with a fresh empty one under the lock (no copy), so each path is processed once. `RecordPeering` already upserts `last_seen`, so stored results are unchanged. - Add a constant `maxTrackedPaths = 500000`; when the map is full, new paths are dropped and counted; expose the count where the handler's other metrics are exposed, if there is such a place, otherwise log it once per run. - Remove the 30-minute prune and its ticker, which the swap makes dead code. - Files: `internal/routewatch/peeringhandler.go` and its test. ## Definition of done - Tests: the map is empty after `ProcessPeeringsNow`; a path seen before and after a run is recorded in both runs; the cap is enforced and the drop counter counts. - `make check` green. Commit title ends ` (closes #N)`. Model: fable-5-1

clawbot referenced this issue

2026-09-21 14:55:52 +02:00

Production memory: run under 5 GiB on real internet feed (was killed at ~12 GiB) #3

clawbot referenced this issue from a commit

2026-09-21 15:10:51 +02:00

Bound peering AS-path map and swap it instead of copying (closes #10)

clawbot referenced this issue

2026-09-21 15:11:17 +02:00

Bound peering AS-path map and swap it instead of copying #16

clawbot commented

2026-09-21 15:11:28 +02:00

PR: #16

processPeerings now swaps the AS-path map for a fresh empty one under the lock instead of copying it, so each path is processed once and the memory is freed each run; RecordPeering still upserts, so stored peerings are unchanged. New paths are dropped and counted once the map reaches maxTrackedPaths (500000), with the drop count logged each run (the handler has no metrics sink). The 30-minute prune and its ticker are gone as dead code.

Tests: a run empties the map; a path seen before and after a run is recorded in both runs; the cap drops and counts new paths while refreshing an already-tracked one.

Disclosure: host make check cannot run here (host golangci-lint built with Go 1.25 panics on the Go 1.26 host toolchain); gated instead on the repo's authoritative Docker build, which is green.

Model: opus-4-8

PR: https://git.eeqj.de/sneak/routewatch/pulls/16 `processPeerings` now swaps the AS-path map for a fresh empty one under the lock instead of copying it, so each path is processed once and the memory is freed each run; `RecordPeering` still upserts, so stored peerings are unchanged. New paths are dropped and counted once the map reaches `maxTrackedPaths` (500000), with the drop count logged each run (the handler has no metrics sink). The 30-minute prune and its ticker are gone as dead code. Tests: a run empties the map; a path seen before and after a run is recorded in both runs; the cap drops and counts new paths while refreshing an already-tracked one. Disclosure: host `make check` cannot run here (host `golangci-lint` built with Go 1.25 panics on the Go 1.26 host toolchain); gated instead on the repo's authoritative Docker build, which is green. Model: opus-4-8

clawbot referenced this issue

2026-09-21 15:54:57 +02:00

Bound peering AS-path map and swap it instead of copying #16

clawbot referenced this issue from a commit

2026-09-21 16:01:10 +02:00

Bound peering AS-path map and swap it instead of copying (closes #10)

clawbot closed this issue

2026-09-21 16:02:15 +02:00

Sign in to join this conversation.