IP lookup: add the public VPN server lists, with is_vpn and vpn_providers #235

Closed
opened 2026-11-10 10:15:08 +01:00 by clawbot · 1 comment
Collaborator

Part of #203, from item 2 of the proposal sneak agreed to (#203 (comment), #203 (comment); he added "coverage does matter more than attribution", so x4bnet is in). Depends on the rename issue linked from the plan on that issue.

Change.

  1. internal/iplist/lists.go gains these lists, each downloaded once a day with its own parser in internal/iplist/parse.go:
  2. List gains a field saying a list holds VPN servers.
  3. Each result gains:
    • flags.is_vpn: true when any VPN list holds the address.
    • vpn_providers: the names of those lists, empty and never null when there are none.

ProtonVPN is its own issue. Check each URL's shape with a real download before writing its parser; the proposal's shapes date from October.

before: "flags": {"is_tor": false, "is_bogon": false}, "ip_lists": []
after:  "flags": {"is_tor": false, "is_vpn": true, "is_bogon": false}, "vpn_providers": ["mullvad"], "ip_lists": [{"name": "mullvad", "description": "...", "updated": "..."}]

Tests. A fixture per format with documentation addresses, IPv4 and IPv6 where the format has them; a wrong document fails each parse; is_vpn and vpn_providers for an address on none, one and two lists.

Done when. A real download of each list parses (one line on the PR saying how many addresses each gave); the README lists each source and the two new fields.

Model: opus-5-5

Part of https://git.eeqj.de/sneak/routewatch/issues/203, from item 2 of the proposal sneak agreed to (https://git.eeqj.de/sneak/routewatch/issues/203#issuecomment-159864, https://git.eeqj.de/sneak/routewatch/issues/203#issuecomment-165190; he added "coverage does matter more than attribution", so `x4bnet` is in). Depends on the rename issue linked from the plan on that issue. **Change.** 1. `internal/iplist/lists.go` gains these lists, each downloaded once a day with its own parser in `internal/iplist/parse.go`: - `mullvad`: https://api.mullvad.net/www/relays/all/ - `nordvpn`: https://api.nordvpn.com/v1/servers?limit=0&fields[servers.station]&fields[servers.ipv6_station]&fields[servers.hostname] - `pia`: https://serverlist.piaservers.net/vpninfo/servers/v6 (JSON, then a signature block after a blank line) - `ivpn`: https://api.ivpn.net/v5/servers.json - `x4bnet`: https://github.com/X4BNet/lists_vpn (its plain list of ranges; find the raw file's URL and use it) 2. `List` gains a field saying a list holds VPN servers. 3. Each result gains: - `flags.is_vpn`: true when any VPN list holds the address. - `vpn_providers`: the names of those lists, empty and never null when there are none. ProtonVPN is its own issue. Check each URL's shape with a real download before writing its parser; the proposal's shapes date from October. ```json before: "flags": {"is_tor": false, "is_bogon": false}, "ip_lists": [] after: "flags": {"is_tor": false, "is_vpn": true, "is_bogon": false}, "vpn_providers": ["mullvad"], "ip_lists": [{"name": "mullvad", "description": "...", "updated": "..."}] ``` **Tests.** A fixture per format with documentation addresses, IPv4 and IPv6 where the format has them; a wrong document fails each parse; `is_vpn` and `vpn_providers` for an address on none, one and two lists. **Done when.** A real download of each list parses (one line on the PR saying how many addresses each gave); the README lists each source and the two new fields. Model: opus-5-5
clawbot self-assigned this 2026-11-10 10:15:08 +01:00
Author
Collaborator

Built in #246: the six lists (X4BNet's IPv4 and IPv6 files are x4bnet and x4bnet_v6), flags.is_vpn and vpn_providers. Details are on the PR.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/routewatch/pulls/246: the six lists (X4BNet's IPv4 and IPv6 files are `x4bnet` and `x4bnet_v6`), `flags.is_vpn` and `vpn_providers`. Details are on the PR. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/routewatch#235