The routing table records a row each time a prefix and origin AS gets its first peer (announced) or loses its last (withdrawn), with the time and how many peers carry the prefix just after. The prefix handler writes the rows to a new origin_changes table, apart from routes. Every hour the database maintainer keeps each prefix's rows from the last 72 hours or its newest 500, whichever keeps more. The prefix page lists them newest first, at most 5,000.
At most 100,000 rows wait in memory to be written; past that the oldest are dropped and logged. A stop loses the rows past its last write.
Model: deepseek-v4-pro (implementation); opus-5-5 (commit message)
Shutdown passes its context to every stage: the streamer, the four
handlers, the WHOIS fetcher, the database maintainer and the HTTP server
each stop waiting when it ends. A handler then skips its last write and
logs how many updates it did not write, and the HTTP server closes the
connections still open. The database is checkpointed and closed last,
and left open once the deadline has passed, since a write may still
hold it. The stop hook gives Shutdown 8 seconds and fx 9, inside the 10
seconds upaas and docker stop allow, so a stop signal exits 0.
Model: opus-5-5
The 5-second checkpoint ran in TRUNCATE mode, which waits up to the
5-second busy timeout for every open read while holding the database
lock every batch write needs. It now runs in PASSIVE mode, which never
waits. A WAL file over 64 MiB instead gets one TRUNCATE checkpoint whose
wait for reads is capped at 500 ms and logged; it empties the file once
no read is open, and while reads follow one another with no pause it
can give up every time. A checkpoint that leaves part of the WAL
uncopied returns ErrCheckpointIncomplete, which the maintainer logs as
incomplete instead of completed. The writing connection's
journal_size_limit is 64 MiB.
Model: opus-5-5
Takes .golangci.yml, .dockerignore, .gitignore, .editorconfig, the
workflow and REPO_POLICIES.md from sneak/prompts 0653579, with this
repository's own entries after the shared content. Lint (golangci-lint
v2.14.0) and test are Dockerfile phases that script/lint and script/test
build alone with a new CACHEBUST value, keeping Go's and golangci-lint's
caches; the image build depends on both. make fmt and make fmt-check also
format the markdown with prettier, pinned in yarn.lock. The agent
guidance moves to AGENTS.md. The Go code is fixed for what the new
configuration finds, and the AS and prefix detail reads now return the
rows' error after their loops.
Model: opus-5-5
Under heavy write load, 30 seconds is too long between checkpoints,
causing the WAL to grow and slow down read queries. More aggressive
checkpointing keeps the WAL small and maintains read performance.
The WAL file was growing to 700MB+ which caused COUNT(*) queries to
timeout. Reads must scan the WAL to find current page versions, and
a large WAL makes this slow.
Add Checkpoint method to database interface and run PASSIVE checkpoints
every 30 seconds via the DBMaintainer. This keeps the WAL small and
maintains fast read performance under heavy write load.
- Use PRAGMA incremental_vacuum instead of full VACUUM
- Frees ~1000 pages (~4MB) per run without blocking writes
- Run every 10 minutes instead of 6 hours since it's lightweight
- Set auto_vacuum=INCREMENTAL pragma for new databases
- Remove blocking VACUUM on startup
- Add health check endpoint at /.well-known/healthcheck.json that
verifies database and RIS Live connectivity, returns 200/503
- Fix panic in streamer when encountering unknown RIS message types
by logging a warning and continuing instead of crashing
- Add DBMaintainer for periodic database maintenance:
- VACUUM every 6 hours to reclaim space
- ANALYZE every hour to update query statistics
- Graceful shutdown support
- Add Vacuum() and Analyze() methods to database interface
clawbot
sneak