diff --git a/README.md b/README.md index 086febe..fe1786c 100644 --- a/README.md +++ b/README.md @@ -1100,8 +1100,7 @@ calls one of them. The scripts: ## TODO -`TODO.md` holds the working list: the next step, the steps done and the steps -planned. +`TODO.md` holds the working list: the next step and the steps planned. ## License diff --git a/TODO.md b/TODO.md index 6dd87e1..10b517f 100644 --- a/TODO.md +++ b/TODO.md @@ -2,7 +2,7 @@ - branch (from `main`) - do the work in Next Step -- move Next Step to the top of Completed Steps +- delete Next Step from `TODO.md` - move the top item of Future Steps into Next Step - commit (`TODO.md` changes in the same commit as the work) - merge to `main` if the branch is not protected, otherwise open a PR @@ -10,41 +10,20 @@ # Status -RouteWatch is pre-1.0, with no git tags. `main` last took `next` on 2026-10-27, -through https://git.eeqj.de/sneak/routewatch/pulls/166. -https://git.eeqj.de/sneak/routewatch/pulls/202 carries the commits since then -into `main`. It is labelled `merge-ready` and waits for sneak. - -The linter and the tests are phases of the Docker build. The Gitea workflow runs -`script/cibuild` on every push. That script runs the two phases and checks the -formatting of the Go code and the markdown. Then it builds the image. - -The image sets memory ceilings for a 5 GiB container (README "Memory"). README -"Running under upaas" says how to run it. - -Five runs of `next` on the live feed have ended since 2026-10-27, from three -hours to five days long. The runs found the handlers dropping updates for five -causes, and the WHOIS fetcher asking a registry that had refused it. Each is -fixed on `next`, and its Completed Steps entry below names its issue. A 21-hour -run of `742e759`, which has every fix but one, dropped no update except after -the feed stalled. That cause, https://git.eeqj.de/sneak/routewatch/issues/229, -is fixed on `next`. - -A run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10 -(https://git.eeqj.de/sneak/routewatch/issues/232). It is to confirm that fix. -Next Step lists what its outcome reports. The seven IP lookup additions of -https://git.eeqj.de/sneak/routewatch/issues/203 landed after it began, so it -does not exercise them. - -The order of the project's remaining work is kept on +RouteWatch is pre-1.0. The order of the remaining work is kept on https://git.eeqj.de/sneak/routewatch/issues/86. # Next Step -Post the outcome of the run of `e58c663` on -https://git.eeqj.de/sneak/routewatch/issues/232. The run lasts at least 24 -hours. If the feed has not stalled or reconnected by then, the run goes on until -it does, up to 72 hours in all. The outcome says: +Post the outcome of the run of `e58c663` on the live feed on +https://git.eeqj.de/sneak/routewatch/issues/232. Work it out from the run's +data: + +- the log of the stopped container `routewatch-run-e58c663`, read with + `docker logs` +- the database in the Docker volume `routewatch-run-e58c663` + +Keep the container until the outcome is posted. The outcome says: - whether the feed stalled or reconnected - whether any handler dropped an update, and the most messages the prefix @@ -53,442 +32,9 @@ it does, up to 72 hours in all. The outcome says: - the size of the `origin_changes` table on disk - the process's peak memory against 5 GiB -File an issue for each problem the run finds. +The run's data does not show the most messages the prefix handler held back at +once, or the process's peak memory. -# Completed Steps - -- 2026-11-11: two more tests no longer fail on a busy host. The test of the home - page's card refresh runs on a fake clock. The geo test in which geojs never - answers counts each request as it is sent, not when it reaches the server - (closes https://git.eeqj.de/sneak/routewatch/issues/256). -- 2026-11-10: `github.com/mattn/go-sqlite3` is at v1.14.53, so building on the - host no longer prints its two C compiler warnings (closes - https://git.eeqj.de/sneak/routewatch/issues/245). -- 2026-11-10: `README.md` and this file are read whole against the rules of - https://git.eeqj.de/sneak/routewatch/issues/157, and every statement except - measured figures and what outside services do is checked against the code on - `next`. Four README sentences failed the prose rules. Five disagreed with the - code. Each is rewritten. Status and Next Step describe the live-feed run in - progress (closes https://git.eeqj.de/sneak/routewatch/issues/251). -- 2026-11-10: the image creates the `routewatch` user as a regular user with UID - 1000, so the build no longer warns about the system UID range (closes - https://git.eeqj.de/sneak/routewatch/issues/248). -- 2026-11-10: each result of the IP lookup has `geo`, where the address is. It - comes from geojs, with DB-IP's City Lite database as the fallback. The geojs - answers stay in memory and in `cache.db` for 7 days (closes - https://git.eeqj.de/sneak/routewatch/issues/239). -- 2026-11-10: the IP lookup checks the `icloud_private_relay` list of Apple's - iCloud Private Relay networks, downloaded once a day. Each result has - `flags.is_relay` (closes https://git.eeqj.de/sneak/routewatch/issues/237). -- 2026-11-10: the IP lookup checks two ProtonVPN lists from a public mirror of - Proton's server list. `protonvpn` holds the /24 networks of ProtonVPN's - servers, and `protonvpn_entry` holds the servers' IPv4 and IPv6 addresses - (closes https://git.eeqj.de/sneak/routewatch/issues/236). -- 2026-11-10: the IP lookup finds PTR names over DNS-over-HTTPS and caches them - in memory and in `cache.db` for the record's TTL. A TTL is the time for which - a DNS record may be kept (closes - https://git.eeqj.de/sneak/routewatch/issues/238). -- 2026-11-10: the tests no longer fail when a busy host stalls for several - seconds. Most waits for something that should happen are now 15 seconds. Tests - that depend on a short idle timeout or a request limit run on a fake clock - (closes https://git.eeqj.de/sneak/routewatch/issues/240) -- 2026-11-10: the IP lookup checks the public VPN server lists of Mullvad, - NordVPN, Private Internet Access and IVPN, and X4BNet's community list. Each - result has `flags.is_vpn` and `vpn_providers` (closes - https://git.eeqj.de/sneak/routewatch/issues/235). -- 2026-11-10: `tor_exits` comes from Onionoo, so it holds IPv6 exits and - `is_tor` is true for them (closes - https://git.eeqj.de/sneak/routewatch/issues/234). -- 2026-11-10: each result of the IP lookup has a `flags` object with `is_tor` - and `is_bogon`. The public IP lists are named `ip_lists` in the answer, - `internal/iplist` in the code and `iplists` in the state directory. A list - file may be up to 200 MiB (closes - https://git.eeqj.de/sneak/routewatch/issues/233). -- 2026-11-05: the prefix page lists the prefix's origin changes, newest first. - The routing table records each change, and the prefix handler writes them to a - new `origin_changes` table. The database maintainer prunes each prefix to its - newest 500 rows or its rows from the last 72 hours, whichever keeps more. The - page shows at most the newest 5,000. Each row shows: - - the time - - announced or withdrawn - - the origin AS - - the peer count - - A row's peer count is how many peers carry the prefix just after the change, - with any origin AS (closes #161). - -- 2026-11-03: the streamer drops an update for a handler only when that - handler's queue is full. It used to drop a random share of the updates once - the queue was half full. The prefix handler's queue now holds 100,000 - messages, about 100 MiB. It used to hold 20,000. While the routing table's - `Overview` walks every prefix, the prefix handler applies updates more slowly. - In a run on the live feed, the feed paused during such a walk and then sent - its backlog. The streamer then dropped about 10,000 of the prefix handler's - updates (closes #229) -- 2026-11-02: `README.md` is rewritten from scratch, sentence by sentence, - against the rules of https://git.eeqj.de/sneak/routewatch/issues/157. Every - statement was checked against the code on `next` (closes #157) -- 2026-11-02: `TODO.md` is rewritten from scratch, sentence by sentence, against - the rules of https://git.eeqj.de/sneak/routewatch/issues/157. Every Completed - Steps entry keeps its date, its facts and its issue. Status and Next Step - describe the repo as the tracker shows it (part of #157) -- 2026-11-02: when the sessions of several peers go down, the prefix handler - takes the peers out of the routing table together. The table's `PeersDown` - takes several peers out of every prefix in one walk. One walk runs at a time. - The next one takes every peer whose session went down since the previous one - began. Each peer used to have a walk of its own. A peer's messages stayed held - back while the walks queued ahead of its own ran. The handler holds back at - most 150,000 messages, about 150 MiB. It used to hold back at most 20,000. The - status page shows how many messages the handler holds back (closes #225) -- 2026-11-02: working out the home page's cards no longer stops the prefix - handler. The walk over the routing table holds the table's lock for 10,000 - prefixes at a time, then lets the lock go for as long again, as `PeerDown` - does. It used to take the lock back at once. The prefix handler then applied - about one update per 10,000 prefixes walked. During a walk, its queue could - fill, and the streamer then dropped updates for it (closes #226) -- 2026-11-02: A skipped WHOIS lookup now makes the fetcher wait longer before - the next one, as a failed lookup does. A lookup is skipped when its RIR is - paused. An RIR is one of the five regional Internet registries. The longest - wait is one minute. The status page does not count a skip as a failure. A skip - used to leave the wait as it was. Each lookup asks IANA which RIR holds the AS - before it finds that RIR paused. So while most stale ASes belonged to a paused - RIR, the fetcher could ask IANA every second (closes #222) -- 2026-11-02: A peer's session going down no longer stops the prefix handler. - The handler runs the routing table's `PeerDown` on a goroutine of its own, one - peer at a time. Meanwhile it goes on taking messages. `PeerDown` holds the - table's lock for 10,000 prefixes at a time, then lets the lock go for as long - again. The handler holds back the down peer's later messages until that peer's - `PeerDown` returns. Then the handler applies them in order, one at a time, and - takes new messages in between. It holds back at most 20,000 messages. At that - number, `HandleMessage` waits until one of them is applied. When `Stop` runs - past its deadline, it logs how many held-back messages were not applied - (closes #216) -- 2026-11-02: RouteWatch pauses an RIR's or NIC.br's WHOIS server for an hour - after the server refuses a query. NIC.br is Brazil's registry. A refused or - reset connection is a refusal. So is an answer with an `%ERROR:201:` or - `%ERROR:202:` line. Lookups at the other servers go on. An AS whose RIR is - paused is looked up after the pause. While NIC.br is paused, a Brazilian AS - gets the details from LACNIC's answer. IANA is never paused, because a pause - of IANA would stop every lookup. A refusal by IANA is an ordinary failure. - NIC.br is also paused for an hour after it says RouteWatch has passed its - query limit. Before, the fetcher asked a server again within seconds of a - refusal (closes #218) -- 2026-11-02: The AS handler never waits for a database write. Its batch holds - one entry per AS, with the earliest and the latest time of its updates. So the - batch no longer grows with the feed's rate. It used to hold one entry for each - AS of each path. At 300,000 entries it waited for the write in progress. In a - burst of the feed its queue then filled, and the streamer dropped messages for - it (closes #217) -- 2026-10-27: The tests of `internal/routewatch` no longer fail on a loaded - host. Five of them gave `HandleMessage` two seconds to apply a full prefix - batch. Under the race detector that can take longer. A test now waits up to 15 - seconds for each step it expects. The tests of the handlers' limits now give - the handlers small limits. So the package no longer runs past its 90-second - limit (closes #213, #215) -- 2026-10-27: The server works out the home page's cards every minute until it - stops. It used to work them out only once, at startup. The end of the context - given to `Start` stopped the refreshes (closes #211) -- 2026-10-27: README "Rationale" says why RouteWatch exists. RouteWatch answers - our other services' questions about IP addresses, from its own routing table - and the WHOIS details it keeps. The section still says what RIS Live does not - give (closes #204) -- 2026-10-27: Every JSON answer carries `@meta`. A panic on a JSON route answers - 500 with the JSON error body. `/status.json` and the health check answer 405 - with the JSON error body to any method they do not take. The 500 and the 405 - used to have an empty body. The home page's IP lookup shows the whole JSON - answer, `@meta` included. It used to show only the answer's `data`, or only - the message of an error answer. A test checks `@meta` on every JSON route, in - its successes and its errors (closes #201) -- 2026-10-27: Every value with a unit in a JSON answer is a number. The unit is - at the end of the field name. `@meta.execution_time` is now - `execution_time_ms`. In the stats, `uptime` is now `uptime_seconds`, - `connection_duration` is now `connection_duration_seconds`, and `memory_usage` - is now `memory_usage_bytes`. `whois_stats.current_interval` is now - `whois_stats.current_interval_seconds`. They used to be strings such as "12 - ms", "1h18m34s" or "1.1 GB". `connection_duration_seconds` is null while the - feed is not connected. `API_POLICIES.md` is vendored from `sneak/prompts`, and - `REPO_POLICIES.md` is re-vendored (closes #199) -- 2026-10-27: Every result of the IP lookup has the same fields. A field without - a value is null. It used to be left out. An address with no route in the - routing table has every route and AS field set to null. `ptr` is an empty list - when there are no PTR names. The new field `num_origins` gives the number of - origin ASes of the netblock, and `origin_asns` gives which they are (closes - #200) -- 2026-10-27: With `TRUSTED_PROXIES` unset, the server trusts reverse proxies at - the private addresses of RFC 1918. `/ip/`, the home page's `curl` card and the - request log then give the client's address from `X-Forwarded-For`. They used - to give the proxy's address. A list in `TRUSTED_PROXIES` replaces the default. - `TRUSTED_PROXIES` set but empty trusts no proxy (closes #198) -- 2026-10-27: The home page has cards with figures of the routing table. One - card shows the newest and the oldest kept route. One shows the newest and the - oldest updated AS. One shows the AS with the most prefixes. One shows the ASes - with a single prefix. One shows fifteen well-known networks. Two cards count - the bogon announcements and the prefixes with more than one origin AS. They - link to the new pages `/bogons` and `/multiple-origins`, which list them. A - new package, `internal/bogon`, tells which prefixes and AS numbers are - reserved. The server works the figures out once a minute (closes #154) -- 2026-10-27: The per-peer route tables are gone, with the code that wrote, - counted and read them. The prefix handler applies each update to the routing - table in memory. It writes the table's changes every second. It also stores - each new prefix with its first-seen time. The image's memory limits fit a full - routing table of about 1 GB. `GOMEMLIMIT` is 2.5 GiB. The writing connection's - page cache is 512 MiB. SQLite's soft and hard heap limits are 768 MiB and 1 - GiB (closes #122) -- 2026-10-27: The pages, the API and the stats read every route from the routing - table in memory. For a prefix with several origin ASes, the IP lookup gives - the one most peers carry. Its `last_seen` is the newest kept route's time. The - prefix page and API list every peer that carries the prefix. Only the kept - routes show an AS path, a next hop and a time. The stats count the origins - whose kept routes were all withdrawn, in `origins_without_kept_route` (part of - #122) -- 2026-10-27: The prefix, AS and peer handlers make a timed write on every tick - of their timers. The prefix handler's timer ticks every second. The other two - tick every two seconds. They used to skip every other tick. A full batch's - write restarts the timer. So the next timed write comes one interval after - that write began (closes #192) -- 2026-10-27: Once the stop deadline has passed, the AS, peer and peering - handlers start no write either. Before, a full batch or a timer tick could - wait while a write ran past the deadline. Either could then start one more - write. The flush goroutines of the prefix, AS and peer handlers now run the - same code (closes #189) -- 2026-10-27: Once the stop deadline has passed, the prefix handler starts no - flush. Before, a batch that filled during a late write could still be written - after the deadline (closes #188) -- 2026-10-27: The prefix handler applies each update to the routing table in - memory. It writes the table's changes to the database every second. A write - takes at most 20,000 changed pairs of prefix and origin AS. The rest follow in - the next writes. A failed write is written again later. Startup reads the - table before the streamer connects. The pages still read the per-peer route - tables (part of #122) -- 2026-10-27: The database keeps a copy of the whole routing table in a new - `routes` table, one row per prefix and origin AS. The peers' numbers are in a - new `route_peers` table. One function writes the pairs that changed in memory. - Another reads the whole table back into memory at startup. Nothing uses them - yet (part of #122) -- 2026-10-27: A new package, `internal/routetable`, holds the whole routing - table in memory. For each prefix and origin AS it keeps the peers that carry - it, one bit per peer. It also keeps the two newest of their routes. Nothing - uses it yet (part of #122) -- 2026-10-27: Each result of the IP lookup lists the public blocklists the - address is on, in `blocklists`. The lists are DShield, Feodo Tracker, two - FireHOL lists, two Spamhaus DROP lists and the Tor exit list. RouteWatch - downloads each list on its own schedule and answers from copies in memory. The - copies are also kept in the state directory's `blocklists` directory. IP - lookup answers carry `Cache-Control`. An address with no route now has a - result too, without route fields. So the IP lookup no longer answers 404 - (closes #158) -- 2026-10-27: The home page has an IP information API card. It shows the `curl` - command that looks up the visitor's own address at `/ip/{addr}`. It also shows - that command's live answer (closes #156) -- 2026-10-27: On the AS page, a click on the heading of the IPv4 or the IPv6 - prefix list folds or unfolds the list. Both lists start open (closes #155) -- 2026-10-27: The AS page lists its IPv4 and its IPv6 prefixes by mask length, - shortest first. Prefixes of one mask length are listed by address. They used - to be sorted by address first, then by mask length (closes #153) -- 2026-10-27: The connection status on `/` and `/status` gets a spinner. It - turns one frame each time the stats show more messages than the time before. - After three refreshes with no new messages, the page shows "stale" in red - instead. Next to it is the time since the last fresh answer (closes #152) -- 2026-10-27: The WHOIS details of a Brazilian AS keep their accented letters. - LACNIC's answer for such an AS is NIC.br's answer with each accented letter - replaced. NIC.br is Brazil's registry. The details now come from NIC.br's own - server, `whois.registro.br`. When that query fails, the details from LACNIC's - answer are kept and a warning is logged. RouteWatch stops asking NIC.br for an - hour after NIC.br says RouteWatch is over its query limit (closes #174) -- 2026-10-27: `make asupdate` stops with an error when the download of `as.csv` - breaks off. The generator used to retry the failed read forever. A record that - does not parse is still logged and skipped (closes #177) -- 2026-10-27: `make asupdate` runs the new `script/asupdate`. It replaces the AS - list built into RouteWatch only when the generator succeeds and finds at least - one AS. A failed download used to replace the list with an empty file. The - generator reads `as.csv` from `ipverse/as-metadata`, the source's new name - (closes #173) -- 2026-10-27: The WHOIS details of an AS at RIPE, APNIC or AFRINIC come only - from its `aut-num` object and the organisation object it names. RIPE and APNIC - start their answer with the `as-block` object of the AS's block. Its dates - were stored as the AS's (closes #168) -- 2026-10-27: AS names keep their accented letters. The AS list built into - RouteWatch is rebuilt from today's `as.csv`, which added a country code - column. That file no longer lists AS 0 or the documentation AS numbers. A - WHOIS answer that is not valid UTF-8 is decoded as Latin-1 before it is parsed - and stored. RIPE and APNIC send Latin-1 (closes #165) -- 2026-10-27: The Age column on the AS and prefix pages reads in words, such as - "5 minutes ago" or "2 weeks ago". It used to print Go durations such as - "360h0m0s ago". An age under a minute reads "now" or in seconds. An age over a - month reads in months or years instead of a date (closes #163) -- 2026-10-26: A write to a stored AS, peering or peer keeps the earlier of the - two first-seen times. It keeps the later of the two last-seen times. An old - update from a collector catching up on a backlog no longer moves last seen - back (closes #162) -- 2026-10-26: The status page shows "Route Churn/min". That is the number of - prefix announcements and withdrawals from all peers in the last full clock - minute. `/api/v1/stats` reports it as `route_churn_per_min` in `stream`. It - replaces the moving average `route_churn_per_sec`. The messages and updates - per second on `/` and `/status` are whole numbers (closes #160) -- 2026-10-26: The WHOIS fetcher backs off more slowly. A failure makes its wait - 1.5 times as long, up to one minute instead of five. Its queries to RIPE, - APNIC and AFRINIC leave out person and role objects. RIPE counts those objects - against a daily limit per address. A refused query is logged with the RIR's - error line (closes #159) -- 2026-10-26: The home page's AS name search works. It lists the ASes whose - handle or description holds the text, as links to their pages. The new - `/api/v1/search/as` answers it. The search ignores case. When the database has - no record of an AS, its page takes the handle and description from the AS list - built into RouteWatch. It still lists the AS's prefixes (closes #151) -- 2026-10-19: README "HTTP Endpoints" lists every route with what it answers. - README "Database Schema" matches `schema.sql`. The rest of the README matches - the code. This file is brought up to date (closes #72) -- 2026-10-19: Route writes are cheaper. The live route and prefix tables lose - their random UUID keys. The prefix tables lose their duplicate index. A route - update sets only the columns that change. Every write goes through one - connection with a page cache of 1 GiB (README "Memory"). A benchmark writes - the prefix handler's largest batch faster than the feed sends it (closes #96) -- 2026-10-19: The checkpoint of the WAL (SQLite's write-ahead log), the - incremental vacuum and `ANALYZE` take the database lock that every batch write - holds. So a vacuum no longer fails with "database is locked". An `ANALYZE` no - longer makes a batch write lose its updates. `ANALYZE` runs as - `PRAGMA optimize` with an analysis limit (closes #94) -- 2026-10-19: Shutdown stops the streamer first. Each handler then writes - everything left on its queue before it stops. So the updates waiting there are - no longer lost (closes #69) -- 2026-10-19: A path that ends in an AS set of several members has origin AS 0, - which means no known origin. It used to have the set's last member as its - origin. The set adds no peerings. The pages show such a route as having no - known origin (closes #75) -- 2026-10-19: `bgp_peer_count` drops to 0 when the last peer goes down (closes - #92) -- 2026-10-19: `bgp_peer_count` counts every peer the feed has heard from. It - used to count only the peers whose session opened since startup. It drops a - peer whose session goes down (closes #74) -- 2026-10-19: The shared files are re-vendored from `sneak/prompts` at - `0653579`. The linter is `golangci-lint` v2.14.0 with its configuration. Lint - and test are `Dockerfile` phases that keep Go's and `golangci-lint`'s caches. - `make fmt` also formats the markdown with `prettier`. The agent guidance moves - to `AGENTS.md`. The code is fixed for what the new linter configuration finds - (closes #52) -- 2026-10-19: The streamer reads feed lines up to 1 MiB. A line over 64 KiB used - to end the connection (closes #76) -- 2026-10-19: A peer's live routes are removed when its BGP session goes down. - `STATE` messages are never dropped when a queue is full. One that finds its - queue full waits for room ahead of the handler's other messages (closes #56) -- 2026-10-19: The prefix-length pages pick their sample with a bounded number of - index lookups. They used to read every route of that length. They list each - prefix once (closes #66) -- 2026-10-19: A request past its time limit answers 504 instead of 408. - `/api/v1/stats` answers 504 instead of 500 when its read runs out of time - (closes #70) -- 2026-10-19: The AS, peer and prefix handlers write each batch outside the lock - their message handling needs. So messages keep coming off their queues during - a write. Meanwhile the next batch grows to at most ten batch sizes (closes - #65) -- 2026-10-19: A live route is identified by prefix and peer. So an announcement - with a new origin AS replaces the peer's route. A withdrawal removes it - whatever its origin AS (closes #55) -- 2026-10-14: The streamer reconnects when RIS Live ends the stream or sends - nothing for 60 seconds. The health check reports the feed disconnected - meanwhile (closes #53) -- 2026-10-14: The prefix page takes the names of the origin ASes from the prefix - query. It used to read every route of each origin AS (closes #67) -- 2026-10-14: The daemon exits with status 1 when its web server cannot listen - on its port. It used to run on without a web server (closes #73) -- 2026-10-14: The prefix pages parse the prefix before looking it up. One that - does not parse answers 400. Any valid way of writing a prefix finds its routes - (closes #71) -- 2026-10-14: The IP lookup ignores routes shorter than /8 for IPv4 and /16 for - IPv6. So a default route is no longer named as an address's owner (closes #57) -- 2026-10-14: A prefix batch keeps each peer's last update for a prefix. It used - to keep one update per prefix across all peers (closes #54) -- 2026-10-14: `reconnect_count` in `/api/v1/stats` counts every reconnect - (closes #68) -- 2026-10-03: `/api/v1/stats` serves the prefix distribution from memory. The - distribution is seeded at startup and adjusted on every live-route write. So a - request no longer reads every live route (closes #30) -- 2026-10-03: A new database is created with `auto_vacuum` set to incremental. - The setting goes in the connection string, so it is set before the file is - first written. Each periodic incremental vacuum now returns up to 1000 free - pages. The late `PRAGMA auto_vacuum` in `Initialize`, which SQLite ignored, is - gone (closes #43) -- 2026-10-03: Looking up an IP address no longer reads every IPv6 route. Both - families find the most specific live route with at most 33 or 129 lookups on - the prefix index. The IPv4 range columns are gone. Prefixes from the feed are - stored in one text form. So an IPv6 withdrawal, which the feed sends - uncompressed, now removes its route (closes #48) -- 2026-10-02: A plain `docker build .` stamps the commit's tag or short commit - into the page footer. It used to stamp `unknown`. The value comes from - `git describe --tags --always`. `.dockerignore` sends `.git` without - `.git/config`. A `VERSION` build argument takes precedence. `make build` - stamps the same value (closes #46) -- 2026-09-29: The entrypoint creates the data directory if it is missing. It - stops the start if a step fails. README "Running under upaas" no longer asks - for the host directory to be created first (closes #42) -- 2026-09-29: `.dockerignore` keeps `.git`, local build output, local databases - and `.env` out of the Docker build context. So they stay out of the source - archive in the image (closes #39) -- 2026-09-29: The README's first line names the MIT licence and the author. The - License section says MIT and links `LICENSE`. An Author section is added. This - file is brought up to date (closes #38) -- 2026-09-29: `LICENSE` is added, with the MIT licence (closes #1) -- 2026-09-28: Stopping the daemon while the feed is flowing no longer panics - with "send on closed channel". The read loop checks for a stop just before - handing a message to the handler queues. A second `Stop` no longer closes the - queues again (closes #34) -- 2026-09-28: `docker stop` no longer kills the daemon 2 seconds after the stop - signal. The entrypoint switches to the `routewatch` user with `setpriv` - instead of `runuser`. So the daemon receives the signal itself. It gets the - whole wait `docker stop` allows, up to its own 60-second limit (closes #33) -- 2026-09-28: RouteWatch is ready to run under upaas. A set but invalid `PORT`, - `XDG_DATA_HOME` or `MALLOC_ARENA_MAX` stops the start. The health check - follows `PORT`. README gains a "Running under upaas" section (closes #31) -- 2026-09-22: The database statistics are kept in memory. The counts are seeded - at startup and adjusted on every write. The oldest and newest route times come - from a lookup at each end of an index. `/api/v1/stats` no longer scans the - tables (closes #27) -- 2026-09-21: Batch writes take the write lock when their transaction begins - (`_txlock=immediate`). So they wait out a WAL checkpoint instead of failing - with "database is locked" (closes #25) -- 2026-09-21: `MALLOC_ARENA_MAX=2` in the image caps the number of glibc malloc - arenas. So memory outside the Go runtime no longer grows with the core count - (closes #23) -- 2026-09-21: The image sets `GOMEMLIMIT=1536MiB`. README "Memory" gives the - memory budget and the 5 GiB container limit (closes #13) -- 2026-09-21: The four handler queues hold at most 20,000 messages each, down - from 100,000 (closes #11) -- 2026-09-21: Two goroutine leaks are fixed. The stats handlers leaked after a - timeout. The streamer's tickers leaked on every reconnect (closes #12) -- 2026-09-21: Parsed RIS messages no longer keep the unused `Community` and - `Raw` fields (closes #9) -- 2026-09-21: `.editorconfig` is added. A Gitea workflow runs `script/cibuild` - on every push (closes #14) -- 2026-09-21: The peering handler's AS-path map holds at most 500,000 paths. - Every 30 seconds it is swapped for an empty one instead of copied (closes #10) -- 2026-09-21: SQLite's memory is bounded across the whole connection pool. Each - connection has a 64 MiB page cache. The heap limits are 1 GiB soft and 1.5 GiB - hard (closes #8) -- 2026-09-21: The Docker build runs the format check, the linter and the tests. - Linting runs in a separate stage on a `golangci-lint` image pinned by digest - (closes #5) -- 2026-09-21: `make test` skips the live feed test, with `-short`. So - `make check` no longer depends on the network (closes #2) -- 2026-07-07: The scripts-to-rule-them-all layout is adopted. `script/` holds - the entrypoints. The `Makefile` targets `check`, `test`, `lint`, `fmt` and - `fmt-check` call them. So do `bootstrap`, `setup`, `docker` and `hooks`. - README gains an "Entrypoints" section -- 2026-02-22: The repo is brought in line with the repo policies. The required - policy files are added. `.gitignore` is updated. The `Makefile` gains - `fmt-check`, `check`, `docker` and `hooks` targets. The Go code is formatted - with `gofmt`. The work was on the branch `repo-policies-compliance`, unmerged - when this entry was written -- 2026-01-01: The WAL is checkpointed every 5 seconds, and at startup with a log - line. Checkpoints run in `TRUNCATE` mode. This fixed slow queries -- 2025-12-31: The pages get a navigation bar and a home page with a search. The - status page shows the oldest and newest route times. The WHOIS stats query - handles `NULL`. The container runs as the `routewatch` user with the right - state directory -- 2025-12-30: HTTP requests are logged as structured log lines. The HTTP - timeouts are longer. Prefix URLs with a slash in them route correctly. The - status page gets more metrics and a footer -- 2025-12-29: The `Dockerfile` builds in stages and puts a source archive in the - image. SQLite reclaims space with incremental vacuum, which does not block -- 2025-07-27: The first commit: RouteWatch, a BGP stream monitor +File an issue for each problem the run found. # Future Steps