sneak, 22 October (chat): "review the ip tags/blocklists sources here and incorporate them into the routewatch ip lookup api, with aggressive caching. https://hackertarget.com/as-ip-lookup/ - we want routewatch to be able to serve a similar purpose wrt ip reputation/info."
What. routewatch's IP lookup API, and the IP page in the web UI, also report which public blocklists list the address. One of routewatch's main purposes is an IP information API for antispam tools (#156); this adds reputation to the country and AS number it already gives.
The sources that hackertarget's lookup tags addresses with, under "IP Tags / Blocklists" on that page:
Each list is downloaded whole on its own schedule, no more often than its publisher updates it, and never once per lookup.
Lists are held in memory for lookup by address or network, so a lookup makes no outside request at all.
The last good copy of each list is kept on disk and loaded at startup. A failed download keeps the previous copy and is logged.
Lookup answers carry HTTP caching headers suited to the list refresh times.
What a lookup returns. For each list the address is on: the list's name, a one-line description of what that list means, and when routewatch last downloaded it. An address on no list returns an empty set, not an error. The IP page in the web UI shows the same tags. Nothing is shown in the UI that does not work (#151).
Definition of done.
The lookup API and the IP page report every listing above that can be downloaded without an account or a key, from cached copies.
Tests cover a listed address, an unlisted one, an address inside a listed network, IPv6, a failed download keeping the old copy, and a restart loading the copies from disk.
The README documents the tags, their sources and the refresh schedule, following the prose rules in #157.
One or more reviewed PRs into next, the last ending its title with (closes #N), N being this issue.
Model: opus-5-5
sneak, 22 October (chat): "review the ip tags/blocklists sources here and incorporate them into the routewatch ip lookup api, with aggressive caching. https://hackertarget.com/as-ip-lookup/ - we want routewatch to be able to serve a similar purpose wrt ip reputation/info."
**What.** routewatch's IP lookup API, and the IP page in the web UI, also report which public blocklists list the address. One of routewatch's main purposes is an IP information API for antispam tools (https://git.eeqj.de/sneak/routewatch/issues/156); this adds reputation to the country and AS number it already gives.
**The sources** that hackertarget's lookup tags addresses with, under "IP Tags / Blocklists" on that page:
- DShield (SANS Institute): addresses reported for suspicious activity. https://www.dshield.org/
- AlienVault OTX: community threat data. https://otx.alienvault.com/
- FireHOL Abusers (30 days): comment and form spam addresses. https://iplists.firehol.org/?ipset=firehol_abusers_30d
- abuse.ch Feodo Tracker: botnet command-and-control servers. https://abuse.ch/
- Spamhaus DROP and EDROP: networks run by spammers and criminals. https://www.spamhaus.org/drop/
- FireHOL level 3: an aggregate of feeds against the most severe attacks. https://iplists.firehol.org/?ipset=firehol_level3
- Tor: Tor exit nodes. https://www.torproject.org/
Starting points for the downloadable lists, to check before use: https://check.torproject.org/torbulkexitlist, https://feodotracker.abuse.ch/downloads/ipblocklist.txt, https://www.spamhaus.org/drop/drop_v4.json and https://www.spamhaus.org/drop/drop_v6.json, https://iplists.firehol.org/files/firehol_level3.netset, https://iplists.firehol.org/files/firehol_abusers_30d.netset and https://feeds.dshield.org/block.txt. Use only lists that can be downloaded whole without an account or a key. A source that needs either, AlienVault OTX possibly among them, is left out entirely, and is not raised as a question (sneak, 22 October: "no sources gated by keys or accounts.").
**Aggressive caching.**
- Each list is downloaded whole on its own schedule, no more often than its publisher updates it, and never once per lookup.
- Lists are held in memory for lookup by address or network, so a lookup makes no outside request at all.
- The last good copy of each list is kept on disk and loaded at startup. A failed download keeps the previous copy and is logged.
- Lookup answers carry HTTP caching headers suited to the list refresh times.
**What a lookup returns.** For each list the address is on: the list's name, a one-line description of what that list means, and when routewatch last downloaded it. An address on no list returns an empty set, not an error. The IP page in the web UI shows the same tags. Nothing is shown in the UI that does not work (https://git.eeqj.de/sneak/routewatch/issues/151).
**Definition of done.**
- The lookup API and the IP page report every listing above that can be downloaded without an account or a key, from cached copies.
- Tests cover a listed address, an unlisted one, an address inside a listed network, IPv6, a failed download keeping the old copy, and a restart loading the copies from disk.
- The README documents the tags, their sources and the refresh schedule, following the prose rules in https://git.eeqj.de/sneak/routewatch/issues/157.
- One or more reviewed PRs into `next`, the last ending its title with (closes #N), N being this issue.
Model: opus-5-5
clawbot
self-assigned this 2026-10-22 20:20:27 +02:00
Recorded: no source that needs a key or an account is used. Any such source is left out and not raised again. The issue text above is updated to match.
Model: opus-5-5
Recorded: no source that needs a key or an account is used. Any such source is left out and not raised again. The issue text above is updated to match.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
sneak, 22 October (chat): "review the ip tags/blocklists sources here and incorporate them into the routewatch ip lookup api, with aggressive caching. https://hackertarget.com/as-ip-lookup/ - we want routewatch to be able to serve a similar purpose wrt ip reputation/info."
What. routewatch's IP lookup API, and the IP page in the web UI, also report which public blocklists list the address. One of routewatch's main purposes is an IP information API for antispam tools (#156); this adds reputation to the country and AS number it already gives.
The sources that hackertarget's lookup tags addresses with, under "IP Tags / Blocklists" on that page:
Starting points for the downloadable lists, to check before use: https://check.torproject.org/torbulkexitlist, https://feodotracker.abuse.ch/downloads/ipblocklist.txt, https://www.spamhaus.org/drop/drop_v4.json and https://www.spamhaus.org/drop/drop_v6.json, https://iplists.firehol.org/files/firehol_level3.netset, https://iplists.firehol.org/files/firehol_abusers_30d.netset and https://feeds.dshield.org/block.txt. Use only lists that can be downloaded whole without an account or a key. A source that needs either, AlienVault OTX possibly among them, is left out entirely, and is not raised as a question (sneak, 22 October: "no sources gated by keys or accounts.").
Aggressive caching.
What a lookup returns. For each list the address is on: the list's name, a one-line description of what that list means, and when routewatch last downloaded it. An address on no list returns an empty set, not an error. The IP page in the web UI shows the same tags. Nothing is shown in the UI that does not work (#151).
Definition of done.
next, the last ending its title with (closes #N), N being this issue.Model: opus-5-5
Recorded: no source that needs a key or an account is used. Any such source is left out and not raised again. The issue text above is updated to match.
Model: opus-5-5