A WHOIS lookup whose RIR query fails or returns nothing is stored as complete and the AS is not retried for 30 days #105

Closed
opened 2026-10-19 22:35:11 +02:00 by clawbot · 1 comment
Collaborator

Where: Client.LookupASN in internal/whois/whois.go; ASNFetcher.fetchAndUpdate in internal/routewatch/asnfetcher.go; Database.UpdateASNWHOIS and GetNextStaleASN in internal/database/database.go.

LookupASN asks IANA, follows the refer: line to the RIR, and when the RIR query fails (refused, reset, dial or read timeout, a line over 64 KiB) returns the IANA-only data with a nil error. It also returns nil when IANA gives no referral, or when the RIR answers only with comment and error lines, such as a rate-limit refusal: nobody checks that any field was parsed. fetchAndUpdate then calls UpdateASNWHOIS, which sets whois_updated_at to now and overwrites as_name, org_name, country_code and the contacts with empty strings. GetNextStaleASN and the IP lookup's NeedsWHOISRefresh consider only rows older than 30 days, so the AS is not looked up again for a month. recordSuccess also halves the interval down to 1 s, so during a refusal the fetcher speeds up against the refusing server and marks one AS per second as done.

Effect: an outage or rate limit of one RIR for an hour silently blanks the WHOIS details of thousands of ASes in the IP lookup for 30 days. Lookups queued by IP requests (QueueImmediate, up to 100) run back to back with no pacing, so a burst of lookups is what brings the rate limit on. The whois package has no tests.

Severity: medium.

Definition of done

  • A lookup whose RIR query fails, or whose answer carries none of the stored fields, returns an error; the fetcher records a failure (backing off) and the row keeps its data and stale time. An AS IANA has no referral for is marked so it is not retried every second, without counting as fetched.
  • Tests, failing before the fix: a fake IANA server referring to a closed port, and one whose RIR answers only an error line; fetchAndUpdate leaves whois_updated_at unset and the interval unchanged (today both write an empty row and shorten it).
  • make check passes.

Model: fable-5-1

Where: `Client.LookupASN` in `internal/whois/whois.go`; `ASNFetcher.fetchAndUpdate` in `internal/routewatch/asnfetcher.go`; `Database.UpdateASNWHOIS` and `GetNextStaleASN` in `internal/database/database.go`. `LookupASN` asks IANA, follows the `refer:` line to the RIR, and when the RIR query fails (refused, reset, dial or read timeout, a line over 64 KiB) returns the IANA-only data with a nil error. It also returns nil when IANA gives no referral, or when the RIR answers only with comment and error lines, such as a rate-limit refusal: nobody checks that any field was parsed. `fetchAndUpdate` then calls `UpdateASNWHOIS`, which sets `whois_updated_at` to now and overwrites `as_name`, `org_name`, `country_code` and the contacts with empty strings. `GetNextStaleASN` and the IP lookup's `NeedsWHOISRefresh` consider only rows older than 30 days, so the AS is not looked up again for a month. `recordSuccess` also halves the interval down to 1 s, so during a refusal the fetcher speeds up against the refusing server and marks one AS per second as done. Effect: an outage or rate limit of one RIR for an hour silently blanks the WHOIS details of thousands of ASes in the IP lookup for 30 days. Lookups queued by IP requests (`QueueImmediate`, up to 100) run back to back with no pacing, so a burst of lookups is what brings the rate limit on. The `whois` package has no tests. Severity: medium. ## Definition of done - A lookup whose RIR query fails, or whose answer carries none of the stored fields, returns an error; the fetcher records a failure (backing off) and the row keeps its data and stale time. An AS IANA has no referral for is marked so it is not retried every second, without counting as fetched. - Tests, failing before the fix: a fake IANA server referring to a closed port, and one whose RIR answers only an error line; `fetchAndUpdate` leaves `whois_updated_at` unset and the interval unchanged (today both write an empty row and shorten it). - `make check` passes. Model: fable-5-1
clawbot self-assigned this 2026-10-19 22:35:11 +02:00
Author
Collaborator

Implemented in #127: a failed lookup, or an RIR answer with no WHOIS fields, is now an error the fetcher backs off on, leaving the row as it was; an AS IANA refers to no RIR is stored empty without counting as a success. Readings taken are listed in the PR.

Model: opus-5-5

Implemented in https://git.eeqj.de/sneak/routewatch/pulls/127: a failed lookup, or an RIR answer with no WHOIS fields, is now an error the fetcher backs off on, leaving the row as it was; an AS IANA refers to no RIR is stored empty without counting as a success. Readings taken are listed in the PR. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/routewatch#105