Once the database passed about 4.5 GiB, every /api/v1/stats request ran a COUNT(*) over each table plus a MIN/MAX union scan of both route tables, hit the 4 s timeout and returned HTTP 500, so the status page went blank (#27).
The server now keeps the last database statistics in memory and recomputes them at most once every 30 s. A request serves the cached copy; a stale copy triggers a single background refresh, so no request runs the scans (the first request after startup computes once to have data to serve). The IPv4/IPv6 route-count split is folded into the cached stats, so the separate per-request live-route count query is gone too.
The oldest/newest route timestamps now read one row from each end of the last_updated index instead of scanning both tables, and select the column directly so the driver parses it into time.Time. The old aggregate returned an untyped string that failed to scan and logged Failed to get route timestamps on every call; that warning is gone.
JSON shape of /api/v1/stats and the status page is unchanged.
Notes / disclosures
Also touched beyond the issue's named files: internal/database/interface.go (two fields on Stats), internal/server/server.go, and a new internal/server/statscache.go.
Removed the per-request stats goroutine and its buffered-channel timeout path (from #12); requests no longer launch a database goroutine, so that leak cannot recur on this path.
Judgement call: a timestamp-query error now logs and continues (display-only fields), matching the prefix-distribution handling.
Model: opus-4-8
## What and why
Once the database passed about 4.5 GiB, every `/api/v1/stats` request ran a `COUNT(*)` over each table plus a `MIN`/`MAX` union scan of both route tables, hit the 4 s timeout and returned HTTP 500, so the status page went blank (https://git.eeqj.de/sneak/routewatch/issues/27).
The server now keeps the last database statistics in memory and recomputes them at most once every 30 s. A request serves the cached copy; a stale copy triggers a single background refresh, so no request runs the scans (the first request after startup computes once to have data to serve). The IPv4/IPv6 route-count split is folded into the cached stats, so the separate per-request live-route count query is gone too.
The oldest/newest route timestamps now read one row from each end of the `last_updated` index instead of scanning both tables, and select the column directly so the driver parses it into `time.Time`. The old aggregate returned an untyped string that failed to scan and logged `Failed to get route timestamps` on every call; that warning is gone.
JSON shape of `/api/v1/stats` and the status page is unchanged.
## Notes / disclosures
- Also touched beyond the issue's named files: `internal/database/interface.go` (two fields on `Stats`), `internal/server/server.go`, and a new `internal/server/statscache.go`.
- Removed the per-request stats goroutine and its buffered-channel timeout path (from https://git.eeqj.de/sneak/routewatch/issues/12); requests no longer launch a database goroutine, so that leak cannot recur on this path.
- Judgement call: a timestamp-query error now logs and continues (display-only fields), matching the prefix-distribution handling.
Model: opus-4-8
Once the database passed about 4.5 GiB every stats request ran a COUNT(*)
over each table plus a MIN/MAX union scan of both route tables, took the
full timeout and returned HTTP 500, so the status page went blank.
The server now keeps the last database statistics in memory and recomputes
them at most once every 30 seconds; requests serve the cached copy and a
stale copy triggers a single background refresh, so no request runs the
scans. The route-count split is folded into the cached stats, removing the
separate per-request live-route count query.
The oldest/newest route timestamps now read one row from each end of the
last_updated index instead of scanning both tables, and select the column
directly so the driver parses it into time.Time; the old aggregate returned
an untyped string that failed to scan and logged a warning every call.
Model: opus-4-8
Superseded by #29. The owner ruled on issue 27 that the stats endpoint must serve realtime in-memory counters updated as messages arrive; a cache-and-rescan approach like this one is not wanted. Closing unmerged.
model: claude-fable-5
Superseded by https://git.eeqj.de/sneak/routewatch/pulls/29. The owner ruled on issue 27 that the stats endpoint must serve realtime in-memory counters updated as messages arrive; a cache-and-rescan approach like this one is not wanted. Closing unmerged.
model: claude-fable-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
What and why
Once the database passed about 4.5 GiB, every
/api/v1/statsrequest ran aCOUNT(*)over each table plus aMIN/MAXunion scan of both route tables, hit the 4 s timeout and returned HTTP 500, so the status page went blank (#27).The server now keeps the last database statistics in memory and recomputes them at most once every 30 s. A request serves the cached copy; a stale copy triggers a single background refresh, so no request runs the scans (the first request after startup computes once to have data to serve). The IPv4/IPv6 route-count split is folded into the cached stats, so the separate per-request live-route count query is gone too.
The oldest/newest route timestamps now read one row from each end of the
last_updatedindex instead of scanning both tables, and select the column directly so the driver parses it intotime.Time. The old aggregate returned an untyped string that failed to scan and loggedFailed to get route timestampson every call; that warning is gone.JSON shape of
/api/v1/statsand the status page is unchanged.Notes / disclosures
internal/database/interface.go(two fields onStats),internal/server/server.go, and a newinternal/server/statscache.go.Model: opus-4-8
Superseded by #29. The owner ruled on issue 27 that the stats endpoint must serve realtime in-memory counters updated as messages arrive; a cache-and-rescan approach like this one is not wanted. Closing unmerged.
model: claude-fable-5
Pull request closed