Bring TODO.md Status and Next Step to the run in progress (closes #251)

Status and Next Step described the 24-hour run of 9bd0e91, whose outcome
was posted on issue 226 on 3 November. They now describe the run of
e58c663 on issue 232 and what its outcome reports. Two Completed Steps
entries of 10 November are rewritten: the ProtonVPN entry ran past 30
words, and the busy-host test entry referred to waits it had not named.
An entry for this read is added.

Model: fable-5-1

Dockerfile

@@ -105,7 +105,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 ARG CACHEBUST
 
 # Create non-root user
-RUN useradd -u 1000 -m routewatch
+RUN useradd -r -u 1000 -m routewatch
 
 RUN mkdir -p /var/lib/berlin.sneak.app.routewatch && chown routewatch:routewatch /var/lib/berlin.sneak.app.routewatch
 

README.md

@@ -143,9 +143,8 @@ and origin AS can have two kept routes, one or none.
   AS path, a next hop and a time. The next hop is the router the peer sends the
   prefix's traffic to. RouteWatch records the peer itself as the next hop. The
   page also lists the prefix's origin changes, newest first. An origin change is
-  recorded when the first peer starts to carry the prefix with an origin AS, or
-  when the last peer stops. The page shows at most the newest 5,000. Each row
-  shows:
+  recorded when an AS gains its first peer for the prefix, or loses its last.
+  The page shows at most the newest 5,000. Each row shows:
     - the time
     - announced or withdrawn
     - the origin AS
@@ -197,9 +196,8 @@ also carries an `@meta` object:
 - `api_version`: `1`
 - `execution_time_ms`: the number of milliseconds the request took
 
-Every value with a unit is a number. The unit ends the field name, as in `_ms`,
-`_seconds`, `_bytes`, `_per_sec` and `_per_min`. `latitude` and `longitude` are
-in degrees, and their names do not say so.
+Every value with a unit is a number, and the unit ends the field name. Examples
+are `_ms`, `_seconds`, `_bytes`, `_per_sec` and `_per_min`.
 
 The paths:
 
@@ -523,7 +521,7 @@ An IP lookup finds where each address is, on a best-effort basis. It uses two
 sources:
 
 - [geojs](https://www.geojs.io/) is asked first. One request carries every
-  address of the lookup that the cache does not hold. geojs has 1 second to
+  address of the lookup that the cache does not hold, and it has 1 second to
   answer.
 - [DB-IP City Lite](https://db-ip.com/db/download/ip-to-city-lite) answers when
   geojs has no answer for the address. It is a database file that RouteWatch
@@ -706,10 +704,8 @@ What happens at each limit:
   allocating more.
 - The SQLite hard heap limit: a database statement fails with an out-of-memory
   error, which the handler logs. The prefix handler writes the routing table's
-  changes and its origin changes again in a later write. Everything else in a
-  handler's failed write is dropped. The PTR cache and the geolocation part
-  write their answers to `cache.db` again in a later write. The process keeps
-  running.
+  changes again in a later write. Everything else in the failed write is
+  dropped. The process keeps running.
 - The handler queues: the handlers are the four parts that take the feed's
   messages (see Design). The prefix handler's queue holds at most 100,000
   messages, about 100 MiB. Each of the other three holds at most 20,000. When a
@@ -1076,8 +1072,8 @@ calls one of them. The scripts:
 - `script/projectname`: print the project name, `routewatch`, which
   `script/docker` uses as the image tag.
 - `script/test`: build the `test` phase of the `Dockerfile` alone. It runs
-  `go test -short -count=1 -timeout 90s -race -cover ./...`. On failure it runs
-  the tests again with `-v` and without `-cover`.
+  `go test -short -count=1 -timeout 90s -race -cover ./...`, and reruns it with
+  `-v` on failure.
 - `script/lint`: build the `lint` phase of the `Dockerfile` alone. It runs
   `golangci-lint` v2.14.0 with `.golangci.yml`.
 - `script/fmt`: format the Go code with `go fmt` and the markdown with

TODO.md

@@ -22,17 +22,18 @@ formatting of the Go code and the markdown. Then it builds the image.
 The image sets memory ceilings for a 5 GiB container (README "Memory"). README
 "Running under upaas" says how to run it.
 
-Five runs of `next` on the live feed have ended since 2026-10-27, from three
-hours to five days long. The runs found the handlers dropping updates for five
-causes, and the WHOIS fetcher asking a registry that had refused it. Each is
-fixed on `next`, and its Completed Steps entry below names its issue. A 21-hour
-run of `742e759`, which has every fix but one, dropped no update except after
-the feed stalled. That cause, https://git.eeqj.de/sneak/routewatch/issues/229,
-is fixed on `next`.
+Since 2026-10-27, `next` has run on the live feed five times, from three hours
+to five days. The runs found the handlers dropping updates for five causes, and
+the WHOIS fetcher asking a registry that had refused it. Each is fixed on
+`next`, and its Completed Steps entry below names its issue. A 21-hour run of
+`742e759`, which has every fix but one, dropped no update except after the feed
+stalled. That cause, https://git.eeqj.de/sneak/routewatch/issues/229, is fixed
+on `next`.
 
-A run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10
-(https://git.eeqj.de/sneak/routewatch/issues/232). It is to confirm that fix.
-Next Step lists what its outcome reports. The seven IP lookup additions of
+A 24-hour run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10
+(https://git.eeqj.de/sneak/routewatch/issues/232). It reads whether any handler
+drops an update, the size of the `origin_changes` table on disk, and the
+process's peak memory against 5 GiB. The seven IP lookup additions of
 https://git.eeqj.de/sneak/routewatch/issues/203 landed after it began, so it
 does not exercise them.
 
@@ -41,15 +42,13 @@ https://git.eeqj.de/sneak/routewatch/issues/86.
 
 # Next Step
 
-Post the outcome of the run of `e58c663` on
-https://git.eeqj.de/sneak/routewatch/issues/232. The run lasts at least 24
-hours. If the feed has not stalled or reconnected by then, the run goes on until
-it does, up to 72 hours in all. The outcome says:
+Post the outcome of the 24-hour run of `e58c663` on
+https://git.eeqj.de/sneak/routewatch/issues/232. The run lasts 24 hours, or
+until the feed has stalled or reconnected once, up to 72 hours. The outcome
+says:
 
-- whether the feed stalled or reconnected
-- whether any handler dropped an update, and the most messages the prefix
-  handler's queue held at once
-- the most messages the prefix handler held back at once
+- whether any handler dropped an update, and the prefix queue's high-water mark
+- the held-back count
 - the size of the `origin_changes` table on disk
 - the process's peak memory against 5 GiB
 
@@ -57,22 +56,12 @@ File an issue for each problem the run finds.
 
 # Completed Steps
 
-- 2026-11-11: two more tests no longer fail on a busy host. The test of the home
-  page's card refresh runs on a fake clock. The geo test in which geojs never
-  answers counts each request as it is sent, not when it reaches the server
-  (closes https://git.eeqj.de/sneak/routewatch/issues/256).
-- 2026-11-10: `github.com/mattn/go-sqlite3` is at v1.14.53, so building on the
-  host no longer prints its two C compiler warnings (closes
-  https://git.eeqj.de/sneak/routewatch/issues/245).
 - 2026-11-10: `README.md` and this file are read whole against the rules of
-  https://git.eeqj.de/sneak/routewatch/issues/157, and every statement except
-  measured figures and what outside services do is checked against the code on
-  `next`. Four README sentences failed the prose rules. Five disagreed with the
-  code. Each is rewritten. Status and Next Step describe the live-feed run in
-  progress (closes https://git.eeqj.de/sneak/routewatch/issues/251).
-- 2026-11-10: the image creates the `routewatch` user as a regular user with UID
-  1000, so the build no longer warns about the system UID range (closes
-  https://git.eeqj.de/sneak/routewatch/issues/248).
+  https://git.eeqj.de/sneak/routewatch/issues/157, and every statement is
+  checked against the code on `next`. Four README sentences, on origin changes,
+  PTR lookups and geolocation, are rewritten. Status and Next Step describe the
+  live-feed run in progress (closes
+  https://git.eeqj.de/sneak/routewatch/issues/251).
 - 2026-11-10: each result of the IP lookup has `geo`, where the address is. It
   comes from geojs, with DB-IP's City Lite database as the fallback. The geojs
   answers stay in memory and in `cache.db` for 7 days (closes
@@ -85,8 +74,7 @@ File an issue for each problem the run finds.
   servers, and `protonvpn_entry` holds the servers' IPv4 and IPv6 addresses
   (closes https://git.eeqj.de/sneak/routewatch/issues/236).
 - 2026-11-10: the IP lookup finds PTR names over DNS-over-HTTPS and caches them
-  in memory and in `cache.db` for the record's TTL. A TTL is the time for which
-  a DNS record may be kept (closes
+  for the record's TTL, in memory and in `cache.db` (closes
   https://git.eeqj.de/sneak/routewatch/issues/238).
 - 2026-11-10: the tests no longer fail when a busy host stalls for several
   seconds. Most waits for something that should happen are now 15 seconds. Tests

go.mod

@@ -9,7 +9,7 @@ require (
 	github.com/google/btree v1.1.3
 	github.com/google/uuid v1.6.0
 	github.com/hashicorp/golang-lru/v2 v2.0.7
-	github.com/mattn/go-sqlite3 v1.14.53
+	github.com/mattn/go-sqlite3 v1.14.29
 	github.com/maxmind/mmdbwriter v1.2.0
 	github.com/miekg/dns v1.1.73
 	github.com/oschwald/maxminddb-golang/v2 v2.9.0

go.sum

@@ -10,8 +10,8 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
 github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
-github.com/mattn/go-sqlite3 v1.14.53 h1:mnTM0owzUMrKuLo+0w51NUiB0ir2Fo7kFXlPt/l3BGs=
-github.com/mattn/go-sqlite3 v1.14.53/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
+github.com/mattn/go-sqlite3 v1.14.29 h1:1O6nRLJKvsi1H2Sj0Hzdfojwt8GiGKm+LOfLaBFaouQ=
+github.com/mattn/go-sqlite3 v1.14.29/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
 github.com/maxmind/mmdbwriter v1.2.0 h1:hyvDopImmgvle3aR8AaddxXnT0iQH2KWJX3vNfkwzYM=
 github.com/maxmind/mmdbwriter v1.2.0/go.mod h1:EQmKHhk2y9DRVvyNxwCLKC5FrkXZLx4snc5OlLY5XLE=
 github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE=

internal/geo/locator_internal_test.go

@@ -90,12 +90,14 @@ func parseAddresses(t *testing.T, addresses ...string) []netip.Addr {
 	return addrs
 }
 
-// fakeGeoJS is a geojs server. It sends body, or status when that is set. It
-// keeps each request's address list.
+// fakeGeoJS is a geojs server. It sends body, or status when that is set, or
+// waits for the request to end when hang is true. It keeps each request's
+// address list.
 type fakeGeoJS struct {
 	server *httptest.Server
 	body   string
 	status int
+	hang   bool
 
 	mu       sync.Mutex
 	requests []string
@@ -117,6 +119,12 @@ func (f *fakeGeoJS) serve(w http.ResponseWriter, r *http.Request) {
 	f.requests = append(f.requests, r.URL.Query().Get("ip"))
 	f.mu.Unlock()
 
+	if f.hang {
+		<-r.Context().Done()
+
+		return
+	}
+
 	if f.status != 0 {
 		w.WriteHeader(f.status)
 
@@ -134,20 +142,6 @@ func (f *fakeGeoJS) requestsSeen() []string {
 	return append([]string{}, f.requests...)
 }
 
-// hangingTransport never answers: each request waits until its context ends.
-// It counts the requests as they are sent, so the count does not depend on
-// whether one reaches a server before its time limit runs out.
-type hangingTransport struct {
-	requests atomic.Int32
-}
-
-func (h *hangingTransport) RoundTrip(r *http.Request) (*http.Response, error) {
-	h.requests.Add(1)
-	<-r.Context().Done()
-
-	return nil, r.Context().Err()
-}
-
 // openCacheDB opens a new cache.db in a temporary directory.
 func openCacheDB(t *testing.T, dir string) *sql.DB {
 	t.Helper()
@@ -391,15 +385,13 @@ func TestLookupFallsBackToTheDatabase(t *testing.T) {
 
 			fake := newFakeGeoJS(t, tt.body)
 			fake.status = tt.status
+			fake.hang = tt.hang
 			dir := t.TempDir()
 			locator := newTestLocator(t, fake, openCacheDB(t, dir), dir)
-			requestCount := func() int { return len(fake.requestsSeen()) }
+			locator.timeLimit = shortTimeLimit
 
-			if tt.hang {
-				hanging := &hangingTransport{}
-				locator.client = &http.Client{Transport: hanging}
-				locator.timeLimit = shortTimeLimit
-				requestCount = func() int { return int(hanging.requests.Load()) }
+			if !tt.hang {
+				locator.timeLimit = testTimeLimit
 			}
 
 			giveDatabase(t, locator)
@@ -421,8 +413,8 @@ func TestLookupFallsBackToTheDatabase(t *testing.T) {
 			// asks geojs again.
 			lookupOne(t, locator, testAddress)
 
-			if requests := requestCount(); requests != 2 {
-				t.Errorf("%d requests, want 2", requests)
+			if requests := fake.requestsSeen(); len(requests) != 2 {
+				t.Errorf("%d requests, want 2", len(requests))
 			}
 		})
 	}

internal/server/home_internal_test.go

@@ -275,66 +275,49 @@ func TestPagesServeTheCopyOfTheCards(t *testing.T) {
 	}
 }
 
-// TestKeepHomeCardsFreshWorksThemOutEveryInterval runs keepHomeCardsFresh. It
-// works the cards out at once, again an interval after the table changes, and
-// stops when its context ends. It runs in a synctest bubble, whose clock is
-// fake, so a stall on the host cannot fail it.
+// TestKeepHomeCardsFreshWorksThemOutEveryInterval runs keepHomeCardsFresh with
+// a short interval. It works the cards out at once, again after the table
+// changes, and stops when its context ends.
 func TestKeepHomeCardsFreshWorksThemOutEveryInterval(t *testing.T) {
 	t.Parallel()
 
-	const interval = time.Minute
+	const (
+		interval = 10 * time.Millisecond
+		maxWait  = 15 * time.Second
+	)
 
 	table := emptyTable()
 	announce(table, "1.0.0.0/24", twoPrefixASN, testPeer(1), testRouteTime(0))
 
 	s := newServerWithTable(t, table)
 
-	// newestRouteOrigin returns the origin AS of the newest route on the
-	// cards, or 0 when there is none.
-	newestRouteOrigin := func() int {
-		cards := s.cards.Load()
-		if cards == nil || cards.NewestRoute == nil {
-			return 0
-		}
+	ctx, cancel := context.WithCancel(t.Context())
+	stopped := make(chan struct{})
 
-		return cards.NewestRoute.OriginASN
+	go func() {
+		s.keepHomeCardsFresh(ctx, interval)
+		close(stopped)
+	}()
+
+	newestRouteASNIs := func(asn int) bool {
+		cards := s.cards.Load()
+
+		return cards != nil && cards.NewestRoute != nil &&
+			cards.NewestRoute.OriginASN == asn
 	}
 
-	synctest.Test(t, func(t *testing.T) {
-		ctx, cancel := context.WithCancel(t.Context())
-		stopped := make(chan struct{})
+	waitUntil(t, maxWait, func() bool { return newestRouteASNIs(twoPrefixASN) })
 
-		go func() {
-			s.keepHomeCardsFresh(ctx, interval)
-			close(stopped)
-		}()
+	announce(table, "8.8.8.0/24", newestRouteASN, testPeer(2), testRouteTime(1))
+	waitUntil(t, maxWait, func() bool { return newestRouteASNIs(newestRouteASN) })
 
-		synctest.Wait()
+	cancel()
 
-		if got := newestRouteOrigin(); got != twoPrefixASN {
-			t.Fatalf("at the start, the newest route's AS is %d, want %d",
-				got, twoPrefixASN)
-		}
-
-		announce(table, "8.8.8.0/24", newestRouteASN, testPeer(2),
-			testRouteTime(1))
-		time.Sleep(interval)
-		synctest.Wait()
-
-		if got := newestRouteOrigin(); got != newestRouteASN {
-			t.Fatalf("an interval after the table changed, the newest "+
-				"route's AS is %d, want %d", got, newestRouteASN)
-		}
-
-		cancel()
-		synctest.Wait()
-
-		select {
-		case <-stopped:
-		default:
-			t.Fatal("keepHomeCardsFresh did not stop when its context ended")
-		}
-	})
+	select {
+	case <-stopped:
+	case <-time.After(maxWait):
+		t.Fatal("keepHomeCardsFresh did not stop when its context ended")
+	}
 }
 
 // TestStartedServerWorksOutTheCards starts a server and waits until its home