Next milestone: production memory under 5 GiB, ready for upaas #6
@@ -1,6 +1,9 @@
|
|||||||
# RouteWatch
|
# RouteWatch
|
||||||
|
|
||||||
RouteWatch is a real-time BGP routing table monitor that streams BGP UPDATE messages from the RIPE RIS Live service, maintains a live routing table in SQLite, and provides HTTP APIs for querying routing information.
|
RouteWatch is an MIT-licensed Go daemon by @sneak that monitors the BGP routing
|
||||||
|
table in real time: it streams BGP UPDATE messages from the RIPE RIS Live
|
||||||
|
service, maintains a live routing table in SQLite, and provides HTTP APIs for
|
||||||
|
querying routing information.
|
||||||
|
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
@@ -299,4 +302,8 @@ invoke `go test` directly without `-short`:
|
|||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
See LICENSE file.
|
MIT. See [`LICENSE`](LICENSE).
|
||||||
|
|
||||||
|
## Author
|
||||||
|
|
||||||
|
[@sneak](https://sneak.berlin)
|
||||||
|
|||||||
@@ -10,19 +10,30 @@
|
|||||||
|
|
||||||
# Status
|
# Status
|
||||||
|
|
||||||
pre-1.0. No git tags. Runs in production-style Docker deployment, but
|
pre-1.0. No git tags. The Docker build runs the format check, the linter
|
||||||
the policy compliance branch (repo-policies-compliance, make check
|
and the tests, and the Gitea workflow runs that build on every push. The
|
||||||
passing, clean tree) is unmerged to main and the CI workflow is missing.
|
image sets memory ceilings for a 5 GiB container (README "Memory") and the
|
||||||
|
README says how to run it under upaas (README "Running under upaas"). A
|
||||||
|
35-hour run of `3898daa` on the live feed peaked at about 1 GiB, without a
|
||||||
|
container memory limit.
|
||||||
|
|
||||||
# Next Step
|
# Next Step
|
||||||
|
|
||||||
Merge repo-policies-compliance into main (3 commits: policy files and
|
`next` waits for sneak to merge it to `main` through
|
||||||
.gitignore, Makefile targets fmt-check/check/docker/hooks, gofmt pass),
|
https://git.eeqj.de/sneak/routewatch/pulls/6. After that, setting
|
||||||
then add .gitea/workflows/check.yml as a small follow-up commit so CI
|
routewatch up under upaas on fsn1app1 and deploying it are his
|
||||||
runs make check on main.
|
(https://git.eeqj.de/sneak/routewatch/issues/31), and so is the run under a
|
||||||
|
real 5 GiB limit (https://git.eeqj.de/sneak/routewatch/issues/3).
|
||||||
|
Nothing on the record orders
|
||||||
|
https://git.eeqj.de/sneak/routewatch/issues/39 and
|
||||||
|
https://git.eeqj.de/sneak/routewatch/issues/30.
|
||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-09-29: README first line names the MIT license and the author;
|
||||||
|
the License section now says MIT and links `LICENSE`, and an Author
|
||||||
|
section was added; this file brought up to date (closes #38)
|
||||||
|
- 2026-09-29: MIT `LICENSE` (closes #1)
|
||||||
- 2026-09-28: stopping the daemon while the feed is flowing no longer
|
- 2026-09-28: stopping the daemon while the feed is flowing no longer
|
||||||
panics with "send on closed channel": the read loop checks for a stop
|
panics with "send on closed channel": the read loop checks for a stop
|
||||||
just before handing a message to the handler queues, and a second
|
just before handing a message to the handler queues, and a second
|
||||||
@@ -40,6 +51,33 @@ runs make check on main.
|
|||||||
startup and adjusted on every write, oldest/newest route timestamps via
|
startup and adjusted on every write, oldest/newest route timestamps via
|
||||||
index-end lookups; `/api/v1/stats` no longer scans the tables (closes
|
index-end lookups; `/api/v1/stats` no longer scans the tables (closes
|
||||||
#27)
|
#27)
|
||||||
|
- 2026-09-21: batch writes take the write lock when their transaction
|
||||||
|
begins (`_txlock=immediate`), so they wait out a WAL checkpoint instead
|
||||||
|
of failing with "database is locked" (closes #25)
|
||||||
|
- 2026-09-21: `MALLOC_ARENA_MAX=2` in the image caps glibc malloc arenas,
|
||||||
|
so memory outside the Go runtime no longer grows with the core count
|
||||||
|
(closes #23)
|
||||||
|
- 2026-09-21: `GOMEMLIMIT=1536MiB` in the image; README Memory section
|
||||||
|
with the memory budget and the 5 GiB container limit (closes #13)
|
||||||
|
- 2026-09-21: the four handler queues hold at most 20,000 messages each,
|
||||||
|
down from 100,000 (closes #11)
|
||||||
|
- 2026-09-21: two goroutine leaks fixed: the stats handlers after a
|
||||||
|
timeout and the streamer's tickers on every reconnect (closes #12)
|
||||||
|
- 2026-09-21: parsed RIS messages no longer keep the unused `Community`
|
||||||
|
and `Raw` fields (closes #9)
|
||||||
|
- 2026-09-21: `.editorconfig`, and a Gitea workflow that runs
|
||||||
|
`script/cibuild` on every push (closes #14)
|
||||||
|
- 2026-09-21: the peering handler's AS-path map holds at most 500,000
|
||||||
|
paths and is swapped for an empty one every 30 seconds instead of copied
|
||||||
|
(closes #10)
|
||||||
|
- 2026-09-21: SQLite memory bounded across the whole connection pool: a
|
||||||
|
64 MiB page cache on each connection, 1 GiB soft and 1.5 GiB hard heap
|
||||||
|
limits (closes #8)
|
||||||
|
- 2026-09-21: the Docker build runs the format check, the linter and the
|
||||||
|
tests, linting in a separate stage on a golangci-lint image pinned by
|
||||||
|
digest (closes #5)
|
||||||
|
- 2026-09-21: `make test` skips the live-network feed test (`-short`), so
|
||||||
|
`make check` no longer depends on the network (closes #2)
|
||||||
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
|
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
|
||||||
Makefile shims, README Entrypoints section
|
Makefile shims, README Entrypoints section
|
||||||
- 2026-02-22: repo policy compliance: required policy files, .gitignore
|
- 2026-02-22: repo policy compliance: required policy files, .gitignore
|
||||||
@@ -58,8 +96,10 @@ runs make check on main.
|
|||||||
|
|
||||||
# Future Steps
|
# Future Steps
|
||||||
|
|
||||||
- Verify main is green after the merge: make check locally and the new
|
- Production memory under 5 GiB: whether to test under a real 5 GiB
|
||||||
CI workflow passing
|
container limit on fsn1app1 is open for sneak
|
||||||
- Review stale remote branches fix-min-time-calculation and
|
(https://git.eeqj.de/sneak/routewatch/issues/3)
|
||||||
optimize-sqlite-settings: land or delete
|
- `/api/v1/stats` answered HTTP 500 after 35 hours on the live feed, seen
|
||||||
- Clean up the tmp/ directory at the repo root: gitignore or remove
|
on `3898daa`, which predates the 2026-09-22 in-memory statistics
|
||||||
|
(https://git.eeqj.de/sneak/routewatch/issues/30)
|
||||||
|
- Add a `.dockerignore` (https://git.eeqj.de/sneak/routewatch/issues/39)
|
||||||
|
|||||||
Reference in New Issue
Block a user