Bound the WHOIS fetcher's success and error time lists (closes #118) #143

Merged
clawbot merged 1 commits from issue-118-fetcher-stats into next 2026-10-20 03:38:52 +02:00
Collaborator

The WHOIS fetcher kept the time of every success and failure, and only a request to /status.json or /api/v1/stats dropped those older than an hour, so a daemon nobody polled kept them all. Recording a success or a failure now first drops the times older than statsWindow, so each list holds at most an hour of fetches whether or not anyone asks for the stats.

What the diff does not show:

  • The old times are dropped from the front of the list instead of copying the ones kept. That is correct because each list holds its times in the order they were recorded: the time is read under the same lock as the append, and time.Now never goes backwards within one process. The copying version would have copied up to an hour of entries on every fetch.
  • The fetcher reads the time through a now field, time.Now outside tests, so the test moves the clock without sleeping. GetStats reads the same field.

Disclosures:

  • Judgement call: pruning on append rather than per-minute counters; the issue allows either, and this is the smaller change.
  • The test records failures as well as successes, since recordFailure changed the same way; the issue names only recordSuccess.

Closes #118

Model: opus-5-5

The WHOIS fetcher kept the time of every success and failure, and only a request to `/status.json` or `/api/v1/stats` dropped those older than an hour, so a daemon nobody polled kept them all. Recording a success or a failure now first drops the times older than `statsWindow`, so each list holds at most an hour of fetches whether or not anyone asks for the stats. What the diff does not show: - The old times are dropped from the front of the list instead of copying the ones kept. That is correct because each list holds its times in the order they were recorded: the time is read under the same lock as the append, and `time.Now` never goes backwards within one process. The copying version would have copied up to an hour of entries on every fetch. - The fetcher reads the time through a `now` field, `time.Now` outside tests, so the test moves the clock without sleeping. `GetStats` reads the same field. Disclosures: - Judgement call: pruning on append rather than per-minute counters; the issue allows either, and this is the smaller change. - The test records failures as well as successes, since `recordFailure` changed the same way; the issue names only `recordSuccess`. Closes https://git.eeqj.de/sneak/routewatch/issues/118 Model: opus-5-5
clawbot added the needs-review label 2026-10-20 03:33:53 +02:00
clawbot self-assigned this 2026-10-20 03:33:53 +02:00
clawbot added 1 commit 2026-10-20 03:33:53 +02:00
recordSuccess and recordFailure appended a time on every fetch, and only
GetStats, run when a stats endpoint is requested, dropped the times older
than statsWindow, so a daemon nobody polled kept every one. Each now drops
those times before it appends. The drop takes them off the front of the
list instead of copying the rest: the times are in the order they were
recorded, and it now runs on every fetch.

The fetcher reads the time through a now field so a test can set the
clock. The test records 100,000 successes and failures a second apart
without asking for the stats and finds one hour's worth of each kept.

Model: opus-5-5
Author
Collaborator

Review passed.

Model: opus-5-5

Review passed. Model: opus-5-5
clawbot merged commit 55ecb22846 into next 2026-10-20 03:38:52 +02:00
clawbot deleted branch issue-118-fetcher-stats 2026-10-20 03:38:52 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/routewatch#143