Author SHA1 Message Date
sneak b1bd1d97cf Make two tests independent of host load (closes #256)
check / check (push) Successful in 3m55s
The home-card refresh test runs keepHomeCardsFresh in a synctest bubble,
so its waits are exact on a fake clock instead of 15-second polls.

The geo fallback subtest in which geojs never answers used a 50 ms time
limit and counted requests at the fake server, which a slow host could
miss. It now uses a transport that counts each request as it is sent and
holds it until the time limit ends it.

Model: opus-5-5
2026-11-11 18:15:10 +00:00
clawbot 760ba4746b Update go-sqlite3 to v1.14.53 to drop two C compiler warnings (closes #245)
check / check (push) Successful in 1m2s
`go-sqlite3` v1.14.29 printed two `-Wdiscarded-qualifiers` warnings from its copy of SQLite when compiled with a recent `gcc`. v1.14.53 prints neither.

Model: opus-5-5
2026-11-10 18:13:50 +01:00
clawbot dcb3935643 Rewrite the README and TODO.md sentences that fail the prose rules (closes #251)
check / check (push) Successful in 1m7s
A fresh read of `README.md` and `TODO.md` against the prose rules, with every statement except measured figures and what outside services do checked against the code. Four README sentences failed the prose rules and five disagreed with the code; each is rewritten. The Status and Next Step of `TODO.md` now describe the run of `e58c663`.

Model: fable-5-1 (rewrite); opus-5-5 (commit message)
2026-11-10 18:01:51 +01:00
6 changed files with 91 additions and 57 deletions

No files matched your search

+7 -4
View File
@@ -197,8 +197,9 @@ also carries an `@meta` object:
- `api_version`: `1`
- `execution_time_ms`: the number of milliseconds the request took
Every value with a unit is a number, and the unit ends the field name. Examples
are `_ms`, `_seconds`, `_bytes`, `_per_sec` and `_per_min`.
Every value with a unit is a number. The unit ends the field name, as in `_ms`,
`_seconds`, `_bytes`, `_per_sec` and `_per_min`. `latitude` and `longitude` are
in degrees, and their names do not say so.
The paths:
@@ -705,8 +706,10 @@ What happens at each limit:
allocating more.
- The SQLite hard heap limit: a database statement fails with an out-of-memory
error, which the handler logs. The prefix handler writes the routing table's
changes and its origin changes again in a later write. Everything else in the
failed write is dropped. The process keeps running.
changes and its origin changes again in a later write. Everything else in a
handler's failed write is dropped. The PTR cache and the geolocation part
write their answers to `cache.db` again in a later write. The process keeps
running.
- The handler queues: the handlers are the four parts that take the feed's
messages (see Design). The prefix handler's queue holds at most 100,000
messages, about 100 MiB. Each of the other three holds at most 20,000. When a
+13 -7
View File
@@ -30,7 +30,7 @@ run of `742e759`, which has every fix but one, dropped no update except after
the feed stalled. That cause, https://git.eeqj.de/sneak/routewatch/issues/229,
is fixed on `next`.
A 24-hour run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10
A run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10
(https://git.eeqj.de/sneak/routewatch/issues/232). It is to confirm that fix.
Next Step lists what its outcome reports. The seven IP lookup additions of
https://git.eeqj.de/sneak/routewatch/issues/203 landed after it began, so it
@@ -41,7 +41,7 @@ https://git.eeqj.de/sneak/routewatch/issues/86.
# Next Step
Post the outcome of the 24-hour run of `e58c663` on
Post the outcome of the run of `e58c663` on
https://git.eeqj.de/sneak/routewatch/issues/232. The run lasts at least 24
hours. If the feed has not stalled or reconnected by then, the run goes on until
it does, up to 72 hours in all. The outcome says:
@@ -57,13 +57,19 @@ File an issue for each problem the run finds.
# Completed Steps
- 2026-11-11: two more tests no longer fail on a busy host. The test of the home
page's card refresh runs on a fake clock. The geo test in which geojs never
answers counts each request as it is sent, not when it reaches the server
(closes https://git.eeqj.de/sneak/routewatch/issues/256).
- 2026-11-10: `github.com/mattn/go-sqlite3` is at v1.14.53, so building on the
host no longer prints its two C compiler warnings (closes
https://git.eeqj.de/sneak/routewatch/issues/245).
- 2026-11-10: `README.md` and this file are read whole against the rules of
https://git.eeqj.de/sneak/routewatch/issues/157, and every statement except
measured figures is checked against the code on `next`. Six README sentences
are rewritten. Four failed the prose rules, on origin changes, PTR lookups and
geolocation. Two disagreed with the code, on a failed database write and
`script/test`. Status and Next Step describe the live-feed run in progress
(closes https://git.eeqj.de/sneak/routewatch/issues/251).
measured figures and what outside services do is checked against the code on
`next`. Four README sentences failed the prose rules. Five disagreed with the
code. Each is rewritten. Status and Next Step describe the live-feed run in
progress (closes https://git.eeqj.de/sneak/routewatch/issues/251).
- 2026-11-10: the image creates the `routewatch` user as a regular user with UID
1000, so the build no longer warns about the system UID range (closes
https://git.eeqj.de/sneak/routewatch/issues/248).
+1 -1
View File
@@ -9,7 +9,7 @@ require (
github.com/google/btree v1.1.3
github.com/google/uuid v1.6.0
github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/mattn/go-sqlite3 v1.14.29
github.com/mattn/go-sqlite3 v1.14.53
github.com/maxmind/mmdbwriter v1.2.0
github.com/miekg/dns v1.1.73
github.com/oschwald/maxminddb-golang/v2 v2.9.0
+2 -2
View File
@@ -10,8 +10,8 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-sqlite3 v1.14.29 h1:1O6nRLJKvsi1H2Sj0Hzdfojwt8GiGKm+LOfLaBFaouQ=
github.com/mattn/go-sqlite3 v1.14.29/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mattn/go-sqlite3 v1.14.53 h1:mnTM0owzUMrKuLo+0w51NUiB0ir2Fo7kFXlPt/l3BGs=
github.com/mattn/go-sqlite3 v1.14.53/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/maxmind/mmdbwriter v1.2.0 h1:hyvDopImmgvle3aR8AaddxXnT0iQH2KWJX3vNfkwzYM=
github.com/maxmind/mmdbwriter v1.2.0/go.mod h1:EQmKHhk2y9DRVvyNxwCLKC5FrkXZLx4snc5OlLY5XLE=
github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE=
+24 -16
View File
@@ -90,14 +90,12 @@ func parseAddresses(t *testing.T, addresses ...string) []netip.Addr {
return addrs
}
// fakeGeoJS is a geojs server. It sends body, or status when that is set, or
// waits for the request to end when hang is true. It keeps each request's
// address list.
// fakeGeoJS is a geojs server. It sends body, or status when that is set. It
// keeps each request's address list.
type fakeGeoJS struct {
server *httptest.Server
body string
status int
hang bool
mu sync.Mutex
requests []string
@@ -119,12 +117,6 @@ func (f *fakeGeoJS) serve(w http.ResponseWriter, r *http.Request) {
f.requests = append(f.requests, r.URL.Query().Get("ip"))
f.mu.Unlock()
if f.hang {
<-r.Context().Done()
return
}
if f.status != 0 {
w.WriteHeader(f.status)
@@ -142,6 +134,20 @@ func (f *fakeGeoJS) requestsSeen() []string {
return append([]string{}, f.requests...)
}
// hangingTransport never answers: each request waits until its context ends.
// It counts the requests as they are sent, so the count does not depend on
// whether one reaches a server before its time limit runs out.
type hangingTransport struct {
requests atomic.Int32
}
func (h *hangingTransport) RoundTrip(r *http.Request) (*http.Response, error) {
h.requests.Add(1)
<-r.Context().Done()
return nil, r.Context().Err()
}
// openCacheDB opens a new cache.db in a temporary directory.
func openCacheDB(t *testing.T, dir string) *sql.DB {
t.Helper()
@@ -385,13 +391,15 @@ func TestLookupFallsBackToTheDatabase(t *testing.T) {
fake := newFakeGeoJS(t, tt.body)
fake.status = tt.status
fake.hang = tt.hang
dir := t.TempDir()
locator := newTestLocator(t, fake, openCacheDB(t, dir), dir)
locator.timeLimit = shortTimeLimit
requestCount := func() int { return len(fake.requestsSeen()) }
if !tt.hang {
locator.timeLimit = testTimeLimit
if tt.hang {
hanging := &hangingTransport{}
locator.client = &http.Client{Transport: hanging}
locator.timeLimit = shortTimeLimit
requestCount = func() int { return int(hanging.requests.Load()) }
}
giveDatabase(t, locator)
@@ -413,8 +421,8 @@ func TestLookupFallsBackToTheDatabase(t *testing.T) {
// asks geojs again.
lookupOne(t, locator, testAddress)
if requests := fake.requestsSeen(); len(requests) != 2 {
t.Errorf("%d requests, want 2", len(requests))
if requests := requestCount(); requests != 2 {
t.Errorf("%d requests, want 2", requests)
}
})
}
+44 -27
View File
@@ -275,49 +275,66 @@ func TestPagesServeTheCopyOfTheCards(t *testing.T) {
}
}
// TestKeepHomeCardsFreshWorksThemOutEveryInterval runs keepHomeCardsFresh with
// a short interval. It works the cards out at once, again after the table
// changes, and stops when its context ends.
// TestKeepHomeCardsFreshWorksThemOutEveryInterval runs keepHomeCardsFresh. It
// works the cards out at once, again an interval after the table changes, and
// stops when its context ends. It runs in a synctest bubble, whose clock is
// fake, so a stall on the host cannot fail it.
func TestKeepHomeCardsFreshWorksThemOutEveryInterval(t *testing.T) {
t.Parallel()
const (
interval = 10 * time.Millisecond
maxWait = 15 * time.Second
)
const interval = time.Minute
table := emptyTable()
announce(table, "1.0.0.0/24", twoPrefixASN, testPeer(1), testRouteTime(0))
s := newServerWithTable(t, table)
ctx, cancel := context.WithCancel(t.Context())
stopped := make(chan struct{})
go func() {
s.keepHomeCardsFresh(ctx, interval)
close(stopped)
}()
newestRouteASNIs := func(asn int) bool {
// newestRouteOrigin returns the origin AS of the newest route on the
// cards, or 0 when there is none.
newestRouteOrigin := func() int {
cards := s.cards.Load()
if cards == nil || cards.NewestRoute == nil {
return 0
}
return cards != nil && cards.NewestRoute != nil &&
cards.NewestRoute.OriginASN == asn
return cards.NewestRoute.OriginASN
}
waitUntil(t, maxWait, func() bool { return newestRouteASNIs(twoPrefixASN) })
synctest.Test(t, func(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
stopped := make(chan struct{})
announce(table, "8.8.8.0/24", newestRouteASN, testPeer(2), testRouteTime(1))
waitUntil(t, maxWait, func() bool { return newestRouteASNIs(newestRouteASN) })
go func() {
s.keepHomeCardsFresh(ctx, interval)
close(stopped)
}()
cancel()
synctest.Wait()
select {
case <-stopped:
case <-time.After(maxWait):
t.Fatal("keepHomeCardsFresh did not stop when its context ended")
}
if got := newestRouteOrigin(); got != twoPrefixASN {
t.Fatalf("at the start, the newest route's AS is %d, want %d",
got, twoPrefixASN)
}
announce(table, "8.8.8.0/24", newestRouteASN, testPeer(2),
testRouteTime(1))
time.Sleep(interval)
synctest.Wait()
if got := newestRouteOrigin(); got != newestRouteASN {
t.Fatalf("an interval after the table changed, the newest "+
"route's AS is %d, want %d", got, newestRouteASN)
}
cancel()
synctest.Wait()
select {
case <-stopped:
default:
t.Fatal("keepHomeCardsFresh did not stop when its context ended")
}
})
}
// TestStartedServerWorksOutTheCards starts a server and waits until its home