sneak/routewatch
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions 25 Packages Projects Releases Wiki Activity

Re-vendor the shared files from sneak/prompts at 0653579 (closes #52)
check / check (push) Canceled after 0s
Details

Browse Source 
Takes .golangci.yml, .dockerignore, .gitignore, .editorconfig, the
workflow and REPO_POLICIES.md from sneak/prompts 0653579, with this
repository's own entries after the shared content. Lint (golangci-lint
v2.14.0) and test are Dockerfile phases that script/lint and script/test
build alone with a new CACHEBUST value, keeping Go's and golangci-lint's
caches; the image build depends on both. make fmt and make fmt-check also
format the markdown with prettier, pinned in yarn.lock. The agent
guidance moves to AGENTS.md. The Go code is fixed for what the new
configuration finds, and the AS and prefix detail reads now return the
rows' error after their loops.

Model: opus-5-5
This commit was merged in pull request #85.
This commit is contained in:
clawbot
2026-10-19 12:10:11 +02:00
parent 4f515d718b
commit b89987e356
65 changed files with 5624 additions and 3509 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TODO.md
+94 -95
View File
@@ -1,122 +1,121 @@
# Workflow
* branch (from `main`)
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* push
- branch (from `main`)
- do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work)
- merge to `main` if the branch is not protected, otherwise open a PR
- push
# Status
pre-1.0. No git tags. The Docker build runs the format check, the linter
and the tests, and the Gitea workflow runs that build on every push. The
image sets memory ceilings for a 5 GiB container (README "Memory") and the
README says how to run it under upaas (README "Running under upaas"). A
35-hour run of `3898daa` on the live feed peaked at about 1 GiB, without a
container memory limit.
pre-1.0. No git tags. The linter and the tests are phases of the Docker build,
and the Gitea workflow runs `script/cibuild` (the format check of the Go code
and the markdown, both phases, then the image) on every push. The image sets
memory ceilings for a 5 GiB container (README "Memory") and the README says how
to run it under upaas (README "Running under upaas"). A 35-hour run of `3898daa`
on the live feed peaked at about 1 GiB, without a container memory limit.
# Next Step
`next` waits for sneak to merge it to `main` through
https://git.eeqj.de/sneak/routewatch/pulls/6. After that, setting
routewatch up under upaas on fsn1app1 and deploying it are his
(https://git.eeqj.de/sneak/routewatch/issues/31), and so is the run under a
real 5 GiB limit (https://git.eeqj.de/sneak/routewatch/issues/3).
https://git.eeqj.de/sneak/routewatch/pulls/6. After that, setting routewatch up
under upaas on fsn1app1 and deploying it are his
(https://git.eeqj.de/sneak/routewatch/issues/31), and so is the run under a real
5 GiB limit (https://git.eeqj.de/sneak/routewatch/issues/3).
# Completed Steps
- 2026-10-03: `/api/v1/stats` serves the prefix distribution from memory,
seeded at startup and adjusted on every live-route write, so a request no
longer reads every live route (closes #30)
- 2026-10-03: a new database is created with `auto_vacuum` set to
incremental, through the connection string so it is set before the file
is first written, and each periodic incremental vacuum now returns up to
1000 free pages; the late `PRAGMA auto_vacuum` in `Initialize`, which
SQLite ignored, is gone (closes #43)
- 2026-10-03: looking up an IP address no longer reads every IPv6 route:
both families find the most specific live route with at most 33 or 129
lookups on the prefix index, and the IPv4 range columns are gone. Prefixes
from the feed are stored in one text form, so an IPv6 withdrawal, which the
feed sends uncompressed, now removes its route (closes #48)
- 2026-10-02: a plain `docker build .` stamps the commit's tag or short
commit (`git describe --tags --always`) into the page footer instead of
`unknown`: `.dockerignore` sends `.git` without `.git/config`, a `VERSION`
build argument takes precedence, and `make build` stamps the same value
(closes #46)
- 2026-10-19: the shared files are re-vendored from `sneak/prompts` at
`0653579`: golangci-lint v2.14.0 and its configuration, lint and test as
`Dockerfile` phases that keep Go's and golangci-lint's caches, markdown
formatted with prettier, agent guidance in `AGENTS.md`; the code is fixed for
what the new linter configuration finds (closes #52)
- 2026-10-03: `/api/v1/stats` serves the prefix distribution from memory, seeded
at startup and adjusted on every live-route write, so a request no longer
reads every live route (closes #30)
- 2026-10-03: a new database is created with `auto_vacuum` set to incremental,
through the connection string so it is set before the file is first written,
and each periodic incremental vacuum now returns up to 1000 free pages; the
late `PRAGMA auto_vacuum` in `Initialize`, which SQLite ignored, is gone
(closes #43)
- 2026-10-03: looking up an IP address no longer reads every IPv6 route: both
families find the most specific live route with at most 33 or 129 lookups on
the prefix index, and the IPv4 range columns are gone. Prefixes from the feed
are stored in one text form, so an IPv6 withdrawal, which the feed sends
uncompressed, now removes its route (closes #48)
- 2026-10-02: a plain `docker build .` stamps the commit's tag or short commit
(`git describe --tags --always`) into the page footer instead of `unknown`:
`.dockerignore` sends `.git` without `.git/config`, a `VERSION` build argument
takes precedence, and `make build` stamps the same value (closes #46)
- 2026-09-29: the entrypoint creates the data directory if it is missing and
stops the start if a step fails; README "Running under upaas" no longer
asks for the host directory to be created first (closes #42)
- 2026-09-29: `.dockerignore` keeps `.git`, local build output, local
databases and `.env` out of the Docker build context, and so out of the
source archive in the image (closes #39)
- 2026-09-29: README first line names the MIT license and the author;
the License section now says MIT and links `LICENSE`, and an Author
section was added; this file brought up to date (closes #38)
stops the start if a step fails; README "Running under upaas" no longer asks
for the host directory to be created first (closes #42)
- 2026-09-29: `.dockerignore` keeps `.git`, local build output, local databases
and `.env` out of the Docker build context, and so out of the source archive
in the image (closes #39)
- 2026-09-29: README first line names the MIT license and the author; the
License section now says MIT and links `LICENSE`, and an Author section was
added; this file brought up to date (closes #38)
- 2026-09-29: MIT `LICENSE` (closes #1)
- 2026-09-28: stopping the daemon while the feed is flowing no longer
panics with "send on closed channel": the read loop checks for a stop
just before handing a message to the handler queues, and a second
`Stop` no longer closes the queues again (closes #34)
- 2026-09-28: `docker stop` no longer kills the daemon 2 seconds after the
stop signal: the entrypoint switches to the `routewatch` user with
`setpriv` instead of `runuser`, so the daemon receives the signal itself
and gets the whole wait `docker stop` allows, up to its own 60-second
limit (closes #33)
- 2026-09-28: stopping the daemon while the feed is flowing no longer panics
with "send on closed channel": the read loop checks for a stop just before
handing a message to the handler queues, and a second `Stop` no longer closes
the queues again (closes #34)
- 2026-09-28: `docker stop` no longer kills the daemon 2 seconds after the stop
signal: the entrypoint switches to the `routewatch` user with `setpriv`
instead of `runuser`, so the daemon receives the signal itself and gets the
whole wait `docker stop` allows, up to its own 60-second limit (closes #33)
- 2026-09-28: ready to run under upaas: a set but invalid `PORT`,
`XDG_DATA_HOME` or `MALLOC_ARENA_MAX` stops the start, the health
check follows `PORT`, README "Running under upaas" section (closes
#31)
- 2026-09-22: realtime in-memory database statistics: counts seeded at
startup and adjusted on every write, oldest/newest route timestamps via
index-end lookups; `/api/v1/stats` no longer scans the tables (closes
#27)
- 2026-09-21: batch writes take the write lock when their transaction
begins (`_txlock=immediate`), so they wait out a WAL checkpoint instead
of failing with "database is locked" (closes #25)
- 2026-09-21: `MALLOC_ARENA_MAX=2` in the image caps glibc malloc arenas,
so memory outside the Go runtime no longer grows with the core count
(closes #23)
- 2026-09-21: `GOMEMLIMIT=1536MiB` in the image; README Memory section
with the memory budget and the 5 GiB container limit (closes #13)
- 2026-09-21: the four handler queues hold at most 20,000 messages each,
down from 100,000 (closes #11)
- 2026-09-21: two goroutine leaks fixed: the stats handlers after a
timeout and the streamer's tickers on every reconnect (closes #12)
- 2026-09-21: parsed RIS messages no longer keep the unused `Community`
and `Raw` fields (closes #9)
- 2026-09-21: `.editorconfig`, and a Gitea workflow that runs
`script/cibuild` on every push (closes #14)
- 2026-09-21: the peering handler's AS-path map holds at most 500,000
paths and is swapped for an empty one every 30 seconds instead of copied
(closes #10)
- 2026-09-21: SQLite memory bounded across the whole connection pool: a
64 MiB page cache on each connection, 1 GiB soft and 1.5 GiB hard heap
limits (closes #8)
- 2026-09-21: the Docker build runs the format check, the linter and the
tests, linting in a separate stage on a golangci-lint image pinned by
digest (closes #5)
`XDG_DATA_HOME` or `MALLOC_ARENA_MAX` stops the start, the health check
follows `PORT`, README "Running under upaas" section (closes #31)
- 2026-09-22: realtime in-memory database statistics: counts seeded at startup
and adjusted on every write, oldest/newest route timestamps via index-end
lookups; `/api/v1/stats` no longer scans the tables (closes #27)
- 2026-09-21: batch writes take the write lock when their transaction begins
(`_txlock=immediate`), so they wait out a WAL checkpoint instead of failing
with "database is locked" (closes #25)
- 2026-09-21: `MALLOC_ARENA_MAX=2` in the image caps glibc malloc arenas, so
memory outside the Go runtime no longer grows with the core count (closes #23)
- 2026-09-21: `GOMEMLIMIT=1536MiB` in the image; README Memory section with the
memory budget and the 5 GiB container limit (closes #13)
- 2026-09-21: the four handler queues hold at most 20,000 messages each, down
from 100,000 (closes #11)
- 2026-09-21: two goroutine leaks fixed: the stats handlers after a timeout and
the streamer's tickers on every reconnect (closes #12)
- 2026-09-21: parsed RIS messages no longer keep the unused `Community` and
`Raw` fields (closes #9)
- 2026-09-21: `.editorconfig`, and a Gitea workflow that runs `script/cibuild`
on every push (closes #14)
- 2026-09-21: the peering handler's AS-path map holds at most 500,000 paths and
is swapped for an empty one every 30 seconds instead of copied (closes #10)
- 2026-09-21: SQLite memory bounded across the whole connection pool: a 64 MiB
page cache on each connection, 1 GiB soft and 1.5 GiB hard heap limits (closes
#8)
- 2026-09-21: the Docker build runs the format check, the linter and the tests,
linting in a separate stage on a golangci-lint image pinned by digest (closes
#5)
- 2026-09-21: `make test` skips the live-network feed test (`-short`), so
`make check` no longer depends on the network (closes #2)
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
Makefile shims, README Entrypoints section
- 2026-02-22: repo policy compliance: required policy files, .gitignore
update, Makefile fmt-check/check/docker/hooks targets, gofmt pass
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
shims, README Entrypoints section
- 2026-02-22: repo policy compliance: required policy files, .gitignore update,
Makefile fmt-check/check/docker/hooks targets, gofmt pass
(repo-policies-compliance, unmerged)
- 2026-01-01: WAL checkpointing: periodic (5s), on startup with logging,
TRUNCATE mode; fixed slow queries
- 2025-12-31: status page: navbar, home page with search, oldest/newest
route timestamps, NULL handling in WHOIS stats; container runs as
routewatch user with proper state dir
- 2025-12-30: structured HTTP request logging, increased timeouts,
CIDR prefix URL routing fixes, status page metrics and footer
- 2025-12-31: status page: navbar, home page with search, oldest/newest route
timestamps, NULL handling in WHOIS stats; container runs as routewatch user
with proper state dir
- 2025-12-30: structured HTTP request logging, increased timeouts, CIDR prefix
URL routing fixes, status page metrics and footer
- 2025-12-29: Dockerfile multi-stage build with source archive; SQLite
incremental vacuum for non-blocking space reclamation
- 2025-07-27: initial RouteWatch BGP stream monitor
# Future Steps
- Production memory under 5 GiB: whether to test under a real 5 GiB
container limit on fsn1app1 is open for sneak
- Production memory under 5 GiB: whether to test under a real 5 GiB container
limit on fsn1app1 is open for sneak
(https://git.eeqj.de/sneak/routewatch/issues/3)