Author SHA1 Message Date
sneak b1bd1d97cf Make two tests independent of host load (closes #256)
check / check (push) Successful in 3m55s
The home-card refresh test runs keepHomeCardsFresh in a synctest bubble,
so its waits are exact on a fake clock instead of 15-second polls.

The geo fallback subtest in which geojs never answers used a 50 ms time
limit and counted requests at the fake server, which a slow host could
miss. It now uses a transport that counts each request as it is sent and
holds it until the time limit ends it.

Model: opus-5-5
2026-11-11 18:15:10 +00:00
clawbot 760ba4746b Update go-sqlite3 to v1.14.53 to drop two C compiler warnings (closes #245)
check / check (push) Successful in 1m2s
`go-sqlite3` v1.14.29 printed two `-Wdiscarded-qualifiers` warnings from its copy of SQLite when compiled with a recent `gcc`. v1.14.53 prints neither.

Model: opus-5-5
2026-11-10 18:13:50 +01:00
clawbot dcb3935643 Rewrite the README and TODO.md sentences that fail the prose rules (closes #251)
check / check (push) Successful in 1m7s
A fresh read of `README.md` and `TODO.md` against the prose rules, with every statement except measured figures and what outside services do checked against the code. Four README sentences failed the prose rules and five disagreed with the code; each is rewritten. The Status and Next Step of `TODO.md` now describe the run of `e58c663`.

Model: fable-5-1 (rewrite); opus-5-5 (commit message)
2026-11-10 18:01:51 +01:00
6 changed files with 91 additions and 57 deletions

No files matched your search

+7 -4
View File
@@ -197,8 +197,9 @@ also carries an `@meta` object:
- `api_version`: `1` - `api_version`: `1`
- `execution_time_ms`: the number of milliseconds the request took - `execution_time_ms`: the number of milliseconds the request took
Every value with a unit is a number, and the unit ends the field name. Examples Every value with a unit is a number. The unit ends the field name, as in `_ms`,
are `_ms`, `_seconds`, `_bytes`, `_per_sec` and `_per_min`. `_seconds`, `_bytes`, `_per_sec` and `_per_min`. `latitude` and `longitude` are
in degrees, and their names do not say so.
The paths: The paths:
@@ -705,8 +706,10 @@ What happens at each limit:
allocating more. allocating more.
- The SQLite hard heap limit: a database statement fails with an out-of-memory - The SQLite hard heap limit: a database statement fails with an out-of-memory
error, which the handler logs. The prefix handler writes the routing table's error, which the handler logs. The prefix handler writes the routing table's
changes and its origin changes again in a later write. Everything else in the changes and its origin changes again in a later write. Everything else in a
failed write is dropped. The process keeps running. handler's failed write is dropped. The PTR cache and the geolocation part
write their answers to `cache.db` again in a later write. The process keeps
running.
- The handler queues: the handlers are the four parts that take the feed's - The handler queues: the handlers are the four parts that take the feed's
messages (see Design). The prefix handler's queue holds at most 100,000 messages (see Design). The prefix handler's queue holds at most 100,000
messages, about 100 MiB. Each of the other three holds at most 20,000. When a messages, about 100 MiB. Each of the other three holds at most 20,000. When a
+13 -7
View File
@@ -30,7 +30,7 @@ run of `742e759`, which has every fix but one, dropped no update except after
the feed stalled. That cause, https://git.eeqj.de/sneak/routewatch/issues/229, the feed stalled. That cause, https://git.eeqj.de/sneak/routewatch/issues/229,
is fixed on `next`. is fixed on `next`.
A 24-hour run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10 A run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10
(https://git.eeqj.de/sneak/routewatch/issues/232). It is to confirm that fix. (https://git.eeqj.de/sneak/routewatch/issues/232). It is to confirm that fix.
Next Step lists what its outcome reports. The seven IP lookup additions of Next Step lists what its outcome reports. The seven IP lookup additions of
https://git.eeqj.de/sneak/routewatch/issues/203 landed after it began, so it https://git.eeqj.de/sneak/routewatch/issues/203 landed after it began, so it
@@ -41,7 +41,7 @@ https://git.eeqj.de/sneak/routewatch/issues/86.
# Next Step # Next Step
Post the outcome of the 24-hour run of `e58c663` on Post the outcome of the run of `e58c663` on
https://git.eeqj.de/sneak/routewatch/issues/232. The run lasts at least 24 https://git.eeqj.de/sneak/routewatch/issues/232. The run lasts at least 24
hours. If the feed has not stalled or reconnected by then, the run goes on until hours. If the feed has not stalled or reconnected by then, the run goes on until
it does, up to 72 hours in all. The outcome says: it does, up to 72 hours in all. The outcome says:
@@ -57,13 +57,19 @@ File an issue for each problem the run finds.
# Completed Steps # Completed Steps
- 2026-11-11: two more tests no longer fail on a busy host. The test of the home
page's card refresh runs on a fake clock. The geo test in which geojs never
answers counts each request as it is sent, not when it reaches the server
(closes https://git.eeqj.de/sneak/routewatch/issues/256).
- 2026-11-10: `github.com/mattn/go-sqlite3` is at v1.14.53, so building on the
host no longer prints its two C compiler warnings (closes
https://git.eeqj.de/sneak/routewatch/issues/245).
- 2026-11-10: `README.md` and this file are read whole against the rules of - 2026-11-10: `README.md` and this file are read whole against the rules of
https://git.eeqj.de/sneak/routewatch/issues/157, and every statement except https://git.eeqj.de/sneak/routewatch/issues/157, and every statement except
measured figures is checked against the code on `next`. Six README sentences measured figures and what outside services do is checked against the code on
are rewritten. Four failed the prose rules, on origin changes, PTR lookups and `next`. Four README sentences failed the prose rules. Five disagreed with the
geolocation. Two disagreed with the code, on a failed database write and code. Each is rewritten. Status and Next Step describe the live-feed run in
`script/test`. Status and Next Step describe the live-feed run in progress progress (closes https://git.eeqj.de/sneak/routewatch/issues/251).
(closes https://git.eeqj.de/sneak/routewatch/issues/251).
- 2026-11-10: the image creates the `routewatch` user as a regular user with UID - 2026-11-10: the image creates the `routewatch` user as a regular user with UID
1000, so the build no longer warns about the system UID range (closes 1000, so the build no longer warns about the system UID range (closes
https://git.eeqj.de/sneak/routewatch/issues/248). https://git.eeqj.de/sneak/routewatch/issues/248).
+1 -1
View File
@@ -9,7 +9,7 @@ require (
github.com/google/btree v1.1.3 github.com/google/btree v1.1.3
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/hashicorp/golang-lru/v2 v2.0.7 github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/mattn/go-sqlite3 v1.14.29 github.com/mattn/go-sqlite3 v1.14.53
github.com/maxmind/mmdbwriter v1.2.0 github.com/maxmind/mmdbwriter v1.2.0
github.com/miekg/dns v1.1.73 github.com/miekg/dns v1.1.73
github.com/oschwald/maxminddb-golang/v2 v2.9.0 github.com/oschwald/maxminddb-golang/v2 v2.9.0
+2 -2
View File
@@ -10,8 +10,8 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-sqlite3 v1.14.29 h1:1O6nRLJKvsi1H2Sj0Hzdfojwt8GiGKm+LOfLaBFaouQ= github.com/mattn/go-sqlite3 v1.14.53 h1:mnTM0owzUMrKuLo+0w51NUiB0ir2Fo7kFXlPt/l3BGs=
github.com/mattn/go-sqlite3 v1.14.29/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y= github.com/mattn/go-sqlite3 v1.14.53/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/maxmind/mmdbwriter v1.2.0 h1:hyvDopImmgvle3aR8AaddxXnT0iQH2KWJX3vNfkwzYM= github.com/maxmind/mmdbwriter v1.2.0 h1:hyvDopImmgvle3aR8AaddxXnT0iQH2KWJX3vNfkwzYM=
github.com/maxmind/mmdbwriter v1.2.0/go.mod h1:EQmKHhk2y9DRVvyNxwCLKC5FrkXZLx4snc5OlLY5XLE= github.com/maxmind/mmdbwriter v1.2.0/go.mod h1:EQmKHhk2y9DRVvyNxwCLKC5FrkXZLx4snc5OlLY5XLE=
github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE= github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE=
+24 -16
View File
@@ -90,14 +90,12 @@ func parseAddresses(t *testing.T, addresses ...string) []netip.Addr {
return addrs return addrs
} }
// fakeGeoJS is a geojs server. It sends body, or status when that is set, or // fakeGeoJS is a geojs server. It sends body, or status when that is set. It
// waits for the request to end when hang is true. It keeps each request's // keeps each request's address list.
// address list.
type fakeGeoJS struct { type fakeGeoJS struct {
server *httptest.Server server *httptest.Server
body string body string
status int status int
hang bool
mu sync.Mutex mu sync.Mutex
requests []string requests []string
@@ -119,12 +117,6 @@ func (f *fakeGeoJS) serve(w http.ResponseWriter, r *http.Request) {
f.requests = append(f.requests, r.URL.Query().Get("ip")) f.requests = append(f.requests, r.URL.Query().Get("ip"))
f.mu.Unlock() f.mu.Unlock()
if f.hang {
<-r.Context().Done()
return
}
if f.status != 0 { if f.status != 0 {
w.WriteHeader(f.status) w.WriteHeader(f.status)
@@ -142,6 +134,20 @@ func (f *fakeGeoJS) requestsSeen() []string {
return append([]string{}, f.requests...) return append([]string{}, f.requests...)
} }
// hangingTransport never answers: each request waits until its context ends.
// It counts the requests as they are sent, so the count does not depend on
// whether one reaches a server before its time limit runs out.
type hangingTransport struct {
requests atomic.Int32
}
func (h *hangingTransport) RoundTrip(r *http.Request) (*http.Response, error) {
h.requests.Add(1)
<-r.Context().Done()
return nil, r.Context().Err()
}
// openCacheDB opens a new cache.db in a temporary directory. // openCacheDB opens a new cache.db in a temporary directory.
func openCacheDB(t *testing.T, dir string) *sql.DB { func openCacheDB(t *testing.T, dir string) *sql.DB {
t.Helper() t.Helper()
@@ -385,13 +391,15 @@ func TestLookupFallsBackToTheDatabase(t *testing.T) {
fake := newFakeGeoJS(t, tt.body) fake := newFakeGeoJS(t, tt.body)
fake.status = tt.status fake.status = tt.status
fake.hang = tt.hang
dir := t.TempDir() dir := t.TempDir()
locator := newTestLocator(t, fake, openCacheDB(t, dir), dir) locator := newTestLocator(t, fake, openCacheDB(t, dir), dir)
locator.timeLimit = shortTimeLimit requestCount := func() int { return len(fake.requestsSeen()) }
if !tt.hang { if tt.hang {
locator.timeLimit = testTimeLimit hanging := &hangingTransport{}
locator.client = &http.Client{Transport: hanging}
locator.timeLimit = shortTimeLimit
requestCount = func() int { return int(hanging.requests.Load()) }
} }
giveDatabase(t, locator) giveDatabase(t, locator)
@@ -413,8 +421,8 @@ func TestLookupFallsBackToTheDatabase(t *testing.T) {
// asks geojs again. // asks geojs again.
lookupOne(t, locator, testAddress) lookupOne(t, locator, testAddress)
if requests := fake.requestsSeen(); len(requests) != 2 { if requests := requestCount(); requests != 2 {
t.Errorf("%d requests, want 2", len(requests)) t.Errorf("%d requests, want 2", requests)
} }
}) })
} }
+44 -27
View File
@@ -275,49 +275,66 @@ func TestPagesServeTheCopyOfTheCards(t *testing.T) {
} }
} }
// TestKeepHomeCardsFreshWorksThemOutEveryInterval runs keepHomeCardsFresh with // TestKeepHomeCardsFreshWorksThemOutEveryInterval runs keepHomeCardsFresh. It
// a short interval. It works the cards out at once, again after the table // works the cards out at once, again an interval after the table changes, and
// changes, and stops when its context ends. // stops when its context ends. It runs in a synctest bubble, whose clock is
// fake, so a stall on the host cannot fail it.
func TestKeepHomeCardsFreshWorksThemOutEveryInterval(t *testing.T) { func TestKeepHomeCardsFreshWorksThemOutEveryInterval(t *testing.T) {
t.Parallel() t.Parallel()
const ( const interval = time.Minute
interval = 10 * time.Millisecond
maxWait = 15 * time.Second
)
table := emptyTable() table := emptyTable()
announce(table, "1.0.0.0/24", twoPrefixASN, testPeer(1), testRouteTime(0)) announce(table, "1.0.0.0/24", twoPrefixASN, testPeer(1), testRouteTime(0))
s := newServerWithTable(t, table) s := newServerWithTable(t, table)
ctx, cancel := context.WithCancel(t.Context()) // newestRouteOrigin returns the origin AS of the newest route on the
stopped := make(chan struct{}) // cards, or 0 when there is none.
newestRouteOrigin := func() int {
go func() {
s.keepHomeCardsFresh(ctx, interval)
close(stopped)
}()
newestRouteASNIs := func(asn int) bool {
cards := s.cards.Load() cards := s.cards.Load()
if cards == nil || cards.NewestRoute == nil {
return 0
}
return cards != nil && cards.NewestRoute != nil && return cards.NewestRoute.OriginASN
cards.NewestRoute.OriginASN == asn
} }
waitUntil(t, maxWait, func() bool { return newestRouteASNIs(twoPrefixASN) }) synctest.Test(t, func(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
stopped := make(chan struct{})
announce(table, "8.8.8.0/24", newestRouteASN, testPeer(2), testRouteTime(1)) go func() {
waitUntil(t, maxWait, func() bool { return newestRouteASNIs(newestRouteASN) }) s.keepHomeCardsFresh(ctx, interval)
close(stopped)
}()
cancel() synctest.Wait()
select { if got := newestRouteOrigin(); got != twoPrefixASN {
case <-stopped: t.Fatalf("at the start, the newest route's AS is %d, want %d",
case <-time.After(maxWait): got, twoPrefixASN)
t.Fatal("keepHomeCardsFresh did not stop when its context ended") }
}
announce(table, "8.8.8.0/24", newestRouteASN, testPeer(2),
testRouteTime(1))
time.Sleep(interval)
synctest.Wait()
if got := newestRouteOrigin(); got != newestRouteASN {
t.Fatalf("an interval after the table changed, the newest "+
"route's AS is %d, want %d", got, newestRouteASN)
}
cancel()
synctest.Wait()
select {
case <-stopped:
default:
t.Fatal("keepHomeCardsFresh did not stop when its context ended")
}
})
} }
// TestStartedServerWorksOutTheCards starts a server and waits until its home // TestStartedServerWorksOutTheCards starts a server and waits until its home