Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c149de1d9a |
No files matched your search
+2
-3
@@ -93,12 +93,11 @@
|
|||||||
**/*.dylib
|
**/*.dylib
|
||||||
|
|
||||||
# Local databases. The image carries a source archive of the whole build
|
# Local databases. The image carries a source archive of the whole build
|
||||||
# context, so these would otherwise ship inside it. The daemon's databases
|
# context, so these would otherwise ship inside it. The daemon's database
|
||||||
# are db.sqlite and cache.db, with their -wal and -shm files while it runs.
|
# is db.sqlite, with its -wal and -shm files while it runs.
|
||||||
**/*.db
|
**/*.db
|
||||||
**/*.db-journal
|
**/*.db-journal
|
||||||
**/*.db-wal
|
**/*.db-wal
|
||||||
**/*.db-shm
|
|
||||||
**/*.sqlite
|
**/*.sqlite
|
||||||
**/*.sqlite-wal
|
**/*.sqlite-wal
|
||||||
**/*.sqlite-shm
|
**/*.sqlite-shm
|
||||||
|
|||||||
+1
-3
@@ -68,12 +68,10 @@ go.work
|
|||||||
go.work.sum
|
go.work.sum
|
||||||
|
|
||||||
# Local databases, temporary files and debug output. The daemon's
|
# Local databases, temporary files and debug output. The daemon's
|
||||||
# databases are db.sqlite and cache.db, with their -wal and -shm files
|
# database is db.sqlite, with its -wal and -shm files while it runs.
|
||||||
# while it runs.
|
|
||||||
*.db
|
*.db
|
||||||
*.db-journal
|
*.db-journal
|
||||||
*.db-wal
|
*.db-wal
|
||||||
*.db-shm
|
|
||||||
*.sqlite
|
*.sqlite
|
||||||
*.sqlite-wal
|
*.sqlite-wal
|
||||||
*.sqlite-shm
|
*.sqlite-shm
|
||||||
|
|||||||
@@ -281,8 +281,8 @@ answer holds `query`, the text looked up, and `results`, with one entry for each
|
|||||||
address. Every entry has the same fields:
|
address. Every entry has the same fields:
|
||||||
|
|
||||||
- `ip`: the address
|
- `ip`: the address
|
||||||
- `ptr`: the address's PTR names, the host names that DNS gives for it (see PTR
|
- `ptr`: the address's PTR names, the host names that DNS gives for it, or an
|
||||||
lookups), or an empty list when it has none
|
empty list when it has none
|
||||||
- `flags`: yes-or-no facts about the address:
|
- `flags`: yes-or-no facts about the address:
|
||||||
- `is_tor`: true when the address is on the `tor_exits` list
|
- `is_tor`: true when the address is on the `tor_exits` list
|
||||||
- `is_vpn`: true when the address is on any list of VPN servers (see IP
|
- `is_vpn`: true when the address is on any list of VPN servers (see IP
|
||||||
@@ -334,6 +334,9 @@ address, such as one that resolves to a private address, is reported as
|
|||||||
- `{ip}` comes to more than 10 addresses once its names are resolved
|
- `{ip}` comes to more than 10 addresses once its names are resolved
|
||||||
- nothing in `{ip}` is an IP address or a host name with a public address
|
- nothing in `{ip}` is an IP address or a host name with a public address
|
||||||
|
|
||||||
|
Each PTR lookup gets 2 seconds. An address whose lookup takes longer is answered
|
||||||
|
without PTR names.
|
||||||
|
|
||||||
A successful answer carries `Cache-Control` with a `max-age` of 15 minutes, the
|
A successful answer carries `Cache-Control` with a `max-age` of 15 minutes, the
|
||||||
shortest IP list interval. The answer for the caller's own address (`/ip/`) is
|
shortest IP list interval. The answer for the caller's own address (`/ip/`) is
|
||||||
`private`, and the others are `public`.
|
`private`, and the others are `public`.
|
||||||
@@ -431,6 +434,14 @@ The lists:
|
|||||||
- `pia`: VPN servers run by Private Internet Access. From
|
- `pia`: VPN servers run by Private Internet Access. From
|
||||||
[Private Internet Access](https://serverlist.piaservers.net/vpninfo/servers/v6),
|
[Private Internet Access](https://serverlist.piaservers.net/vpninfo/servers/v6),
|
||||||
downloaded once a day.
|
downloaded once a day.
|
||||||
|
- `protonvpn`: the /24 networks that hold ProtonVPN servers. The whole /24 is
|
||||||
|
listed because the ProtonVPN-IPs README says traffic leaves from unpublished
|
||||||
|
addresses in it. From the public
|
||||||
|
[ProtonVPN-IPs](https://github.com/tn3w/ProtonVPN-IPs) mirror of Proton's
|
||||||
|
server list, downloaded once a day.
|
||||||
|
- `protonvpn_entry`: the IPv4 and IPv6 addresses of ProtonVPN servers. From the
|
||||||
|
same [ProtonVPN-IPs](https://github.com/tn3w/ProtonVPN-IPs) mirror, downloaded
|
||||||
|
once a day.
|
||||||
- `spamhaus_drop` and `spamhaus_dropv6`: IPv4 and IPv6 networks hijacked or
|
- `spamhaus_drop` and `spamhaus_dropv6`: IPv4 and IPv6 networks hijacked or
|
||||||
leased by spammers or cyber criminals, according to Spamhaus. From
|
leased by spammers or cyber criminals, according to Spamhaus. From
|
||||||
[Spamhaus DROP](https://www.spamhaus.org/blocklists/do-not-route-or-peer/),
|
[Spamhaus DROP](https://www.spamhaus.org/blocklists/do-not-route-or-peer/),
|
||||||
@@ -462,45 +473,6 @@ How RouteWatch keeps the lists:
|
|||||||
- When a download fails, or its file does not parse, RouteWatch keeps the
|
- When a download fails, or its file does not parse, RouteWatch keeps the
|
||||||
previous copy and logs the failure. It tries again one interval later.
|
previous copy and logs the failure. It tries again one interval later.
|
||||||
|
|
||||||
## PTR lookups
|
|
||||||
|
|
||||||
An IP lookup finds each address's PTR names over DNS-over-HTTPS. It asks these
|
|
||||||
public resolvers in order, until one answers:
|
|
||||||
|
|
||||||
- Google: `https://dns.google/dns-query`
|
|
||||||
- Cloudflare: `https://cloudflare-dns.com/dns-query`
|
|
||||||
- AdGuard: `https://unfiltered.adguard-dns.com/dns-query`
|
|
||||||
- Quad9: `https://dns.quad9.net/dns-query`
|
|
||||||
|
|
||||||
One address's lookup gets 2 seconds across all four. Each resolver gets an equal
|
|
||||||
share of the time left, so one that does not answer leaves time for the next.
|
|
||||||
When none answers in time, the address's `ptr` is an empty list.
|
|
||||||
|
|
||||||
A reserved address, one whose `is_bogon` flag is true, is not sent to the
|
|
||||||
resolvers. Its `ptr` is an empty list.
|
|
||||||
|
|
||||||
What goes outside: each resolver asked receives the address being looked up. For
|
|
||||||
`/ip/` and the home page, that address is the visitor's own. A lookup answered
|
|
||||||
from the cache sends nothing.
|
|
||||||
|
|
||||||
How RouteWatch keeps the answers:
|
|
||||||
|
|
||||||
- It holds answers in memory. An answer that has not expired is served from
|
|
||||||
there.
|
|
||||||
- It writes new answers every 5 seconds to `cache.db` in the state directory
|
|
||||||
(see Configuration), and loads them from there at startup.
|
|
||||||
- An answer expires after the TTL of its records, the shortest one when there
|
|
||||||
are several. RouteWatch keeps it for at least 60 seconds and at most 24 hours.
|
|
||||||
- When an address has no PTR names, that answer expires after the SOA record's
|
|
||||||
minimum field, or the record's own TTL when that is shorter. RouteWatch keeps
|
|
||||||
it for at most 1 hour.
|
|
||||||
- An answer with no PTR names and no SOA record is not kept. Neither is one
|
|
||||||
whose SOA record has a TTL or minimum field of 0.
|
|
||||||
- A failed lookup is not kept, so the next lookup of the address asks again.
|
|
||||||
- Every 10 minutes it deletes expired answers from memory and from `cache.db`.
|
|
||||||
- It keeps answers for at most 250,000 addresses, and keeps no answer for a new
|
|
||||||
address while it holds that many.
|
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
RouteWatch reads its settings from environment variables. It takes no
|
RouteWatch reads its settings from environment variables. It takes no
|
||||||
@@ -549,13 +521,7 @@ by the client. For a request from any other address, they use the address the
|
|||||||
request came from, whatever headers it carries. `X-Real-IP` and `True-Client-IP`
|
request came from, whatever headers it carries. `X-Real-IP` and `True-Client-IP`
|
||||||
are not read.
|
are not read.
|
||||||
|
|
||||||
The state directory holds:
|
The state directory holds the database, `db.sqlite`, and the IP list copies:
|
||||||
|
|
||||||
- `db.sqlite`: the database
|
|
||||||
- `cache.db`: the answers of PTR lookups
|
|
||||||
- `iplists`: the IP list copies
|
|
||||||
|
|
||||||
It is at:
|
|
||||||
|
|
||||||
- macOS: `~/Library/Application Support/berlin.sneak.app.routewatch/`
|
- macOS: `~/Library/Application Support/berlin.sneak.app.routewatch/`
|
||||||
- Linux and the BSDs, as root: `/var/lib/berlin.sneak.app.routewatch/`
|
- Linux and the BSDs, as root: `/var/lib/berlin.sneak.app.routewatch/`
|
||||||
@@ -712,8 +678,6 @@ The parts:
|
|||||||
with old or missing details asks the fetcher for it at once, without the wait.
|
with old or missing details asks the fetcher for it at once, without the wait.
|
||||||
- The IP lists part downloads the IP lists and answers which lists hold an
|
- The IP lists part downloads the IP lists and answers which lists hold an
|
||||||
address.
|
address.
|
||||||
- The PTR cache looks up the PTR names of addresses and keeps the answers (see
|
|
||||||
PTR lookups).
|
|
||||||
- The database maintainer checkpoints the database every 5 seconds: it folds the
|
- The database maintainer checkpoints the database every 5 seconds: it folds the
|
||||||
write-ahead log, where SQLite first writes changes, into the database file. It
|
write-ahead log, where SQLite first writes changes, into the database file. It
|
||||||
reclaims free space every 10 minutes. Every hour it refreshes the statistics
|
reclaims free space every 10 minutes. Every hour it refreshes the statistics
|
||||||
@@ -823,7 +787,6 @@ On `SIGTERM` or `SIGINT` the daemon stops its parts in this order:
|
|||||||
4. the database maintainer
|
4. the database maintainer
|
||||||
5. the IP list downloads
|
5. the IP list downloads
|
||||||
6. the HTTP server
|
6. the HTTP server
|
||||||
7. the PTR cache, which writes what it holds
|
|
||||||
|
|
||||||
Last, the daemon checkpoints and closes the database. It gives all this 8
|
Last, the daemon checkpoints and closes the database. It gives all this 8
|
||||||
seconds and then exits with status 0. A stop that is still running after 9
|
seconds and then exits with status 0. A stop that is still running after 9
|
||||||
@@ -844,8 +807,6 @@ What a stop can lose:
|
|||||||
- When the 8 seconds run out, the streamer and the handlers log what they could
|
- When the 8 seconds run out, the streamer and the handlers log what they could
|
||||||
not hand over or write. The database is then not closed. When it next opens,
|
not hand over or write. The database is then not closed. When it next opens,
|
||||||
SQLite rolls back any write the exit cut off.
|
SQLite rolls back any write the exit cut off.
|
||||||
- When the 8 seconds run out, the PTR answers not yet in `cache.db` are lost.
|
|
||||||
RouteWatch asks the resolvers for them again the next time they are needed.
|
|
||||||
|
|
||||||
### Code layout
|
### Code layout
|
||||||
|
|
||||||
@@ -859,13 +820,11 @@ routewatch/
|
|||||||
│ └── message-examples.json # Examples of RIS Live messages
|
│ └── message-examples.json # Examples of RIS Live messages
|
||||||
├── internal/
|
├── internal/
|
||||||
│ ├── bogon/ # Reserved prefixes and AS numbers
|
│ ├── bogon/ # Reserved prefixes and AS numbers
|
||||||
│ ├── cachedb/ # cache.db, the caches of outside answers
|
|
||||||
│ ├── config/ # Settings from the environment
|
│ ├── config/ # Settings from the environment
|
||||||
│ ├── database/ # SQLite storage
|
│ ├── database/ # SQLite storage
|
||||||
│ ├── iplist/ # Public IP lists: downloads and lookups
|
│ ├── iplist/ # Public IP lists: downloads and lookups
|
||||||
│ ├── logger/ # Structured logging
|
│ ├── logger/ # Structured logging
|
||||||
│ ├── metrics/ # Feed and route update figures
|
│ ├── metrics/ # Feed and route update figures
|
||||||
│ ├── ptr/ # PTR lookups over DNS-over-HTTPS, and their cache
|
|
||||||
│ ├── ristypes/ # The RIS Live message types
|
│ ├── ristypes/ # The RIS Live message types
|
||||||
│ ├── routetable/ # The whole routing table in memory
|
│ ├── routetable/ # The whole routing table in memory
|
||||||
│ ├── routewatch/ # The daemon: startup, handlers, WHOIS fetcher
|
│ ├── routewatch/ # The daemon: startup, handlers, WHOIS fetcher
|
||||||
@@ -914,13 +873,6 @@ announcements(id PRIMARY KEY, prefix_id, peer_asn, origin_asn, path,
|
|||||||
|
|
||||||
The indexes are in `internal/database/schema.sql`.
|
The indexes are in `internal/database/schema.sql`.
|
||||||
|
|
||||||
`cache.db` is a separate file, with one table:
|
|
||||||
|
|
||||||
```sql
|
|
||||||
-- The answers of PTR lookups. names is empty for an address with no PTR names.
|
|
||||||
hostnames(address PRIMARY KEY, names, expires_at)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|
||||||
The `make` targets:
|
The `make` targets:
|
||||||
|
|||||||
@@ -53,9 +53,9 @@ problem the run finds.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
- 2026-11-10: the IP lookup finds PTR names over DNS-over-HTTPS and caches them
|
- 2026-11-10: the IP lookup checks the `protonvpn` list of ProtonVPN's /24
|
||||||
for the record's TTL, in memory and in `cache.db` (closes
|
networks, from a public mirror of Proton's server list (closes
|
||||||
https://git.eeqj.de/sneak/routewatch/issues/238).
|
https://git.eeqj.de/sneak/routewatch/issues/236).
|
||||||
- 2026-11-10: the tests no longer fail when a busy host stalls for several
|
- 2026-11-10: the tests no longer fail when a busy host stalls for several
|
||||||
seconds. Most such waits are now 15 seconds. The test of the stream's idle
|
seconds. Most such waits are now 15 seconds. The test of the stream's idle
|
||||||
timeout runs on a fake clock (closes #240)
|
timeout runs on a fake clock (closes #240)
|
||||||
|
|||||||
@@ -9,11 +9,10 @@ require (
|
|||||||
github.com/google/btree v1.1.3
|
github.com/google/btree v1.1.3
|
||||||
github.com/google/uuid v1.6.0
|
github.com/google/uuid v1.6.0
|
||||||
github.com/mattn/go-sqlite3 v1.14.29
|
github.com/mattn/go-sqlite3 v1.14.29
|
||||||
github.com/miekg/dns v1.1.73
|
|
||||||
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9
|
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9
|
||||||
go.uber.org/fx v1.24.0
|
go.uber.org/fx v1.24.0
|
||||||
go4.org/netipx v0.0.0-20260823151212-3075585bcbeb
|
go4.org/netipx v0.0.0-20260823151212-3075585bcbeb
|
||||||
golang.org/x/term v0.45.0
|
golang.org/x/term v0.33.0
|
||||||
golang.org/x/text v0.43.0
|
golang.org/x/text v0.43.0
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -21,6 +20,5 @@ require (
|
|||||||
go.uber.org/dig v1.19.0 // indirect
|
go.uber.org/dig v1.19.0 // indirect
|
||||||
go.uber.org/multierr v1.10.0 // indirect
|
go.uber.org/multierr v1.10.0 // indirect
|
||||||
go.uber.org/zap v1.26.0 // indirect
|
go.uber.org/zap v1.26.0 // indirect
|
||||||
golang.org/x/net v0.57.0 // indirect
|
golang.org/x/sys v0.34.0 // indirect
|
||||||
golang.org/x/sys v0.47.0 // indirect
|
|
||||||
)
|
)
|
||||||
@@ -12,8 +12,6 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
|||||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||||
github.com/mattn/go-sqlite3 v1.14.29 h1:1O6nRLJKvsi1H2Sj0Hzdfojwt8GiGKm+LOfLaBFaouQ=
|
github.com/mattn/go-sqlite3 v1.14.29 h1:1O6nRLJKvsi1H2Sj0Hzdfojwt8GiGKm+LOfLaBFaouQ=
|
||||||
github.com/mattn/go-sqlite3 v1.14.29/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
github.com/mattn/go-sqlite3 v1.14.29/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
||||||
github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE=
|
|
||||||
github.com/miekg/dns v1.1.73/go.mod h1:RW2Obtfd5NZHvOFe3zYG0W8koWOQtAzyHaLo8vASBuQ=
|
|
||||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 h1:bsUq1dX0N8AOIL7EB/X911+m4EHsnWEHeJ0c+3TTBrg=
|
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 h1:bsUq1dX0N8AOIL7EB/X911+m4EHsnWEHeJ0c+3TTBrg=
|
||||||
@@ -32,14 +30,10 @@ go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
|
|||||||
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
|
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
|
||||||
go4.org/netipx v0.0.0-20260823151212-3075585bcbeb h1:XBM4hvfwGAttkkiTIFfeigdfcL1xIfdKXqFdgiHGtDs=
|
go4.org/netipx v0.0.0-20260823151212-3075585bcbeb h1:XBM4hvfwGAttkkiTIFfeigdfcL1xIfdKXqFdgiHGtDs=
|
||||||
go4.org/netipx v0.0.0-20260823151212-3075585bcbeb/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
|
go4.org/netipx v0.0.0-20260823151212-3075585bcbeb/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
|
||||||
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
|
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
|
||||||
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
|
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
|
||||||
golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
|
golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
|
||||||
golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
|
golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
|
||||||
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
|
|
||||||
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
|
||||||
golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
|
|
||||||
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
|
|
||||||
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
|
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
|
||||||
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
|
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
|||||||
+2
-11
@@ -1,5 +1,5 @@
|
|||||||
// Package bogon tells whether a prefix, an address or an AS number is reserved:
|
// Package bogon tells whether a prefix or an AS number is reserved: set aside
|
||||||
// set aside for a special purpose, so that no network should announce it on the
|
// for a special purpose, so that no network should announce it on the
|
||||||
// internet. A route is a bogon announcement when its prefix is reserved or its
|
// internet. A route is a bogon announcement when its prefix is reserved or its
|
||||||
// origin AS is.
|
// origin AS is.
|
||||||
//
|
//
|
||||||
@@ -42,15 +42,6 @@ func Prefix(prefix netip.Prefix) (string, bool) {
|
|||||||
return holder.name + " (" + holder.prefix.String() + ")", true
|
return holder.name + " (" + holder.prefix.String() + ")", true
|
||||||
}
|
}
|
||||||
|
|
||||||
// Addr returns why addr is reserved, and true, when it lies inside a reserved
|
|
||||||
// block, as Prefix decides. An IPv4 address written in IPv6 form counts as the
|
|
||||||
// IPv4 address it holds. It returns "" and false when addr is not reserved.
|
|
||||||
func Addr(addr netip.Addr) (string, bool) {
|
|
||||||
addr = addr.Unmap()
|
|
||||||
|
|
||||||
return Prefix(netip.PrefixFrom(addr, addr.BitLen()))
|
|
||||||
}
|
|
||||||
|
|
||||||
// ASN returns why asn is reserved, and true, when it is. It returns "" and
|
// ASN returns why asn is reserved, and true, when it is. It returns "" and
|
||||||
// false when asn is not reserved.
|
// false when asn is not reserved.
|
||||||
func ASN(asn int) (string, bool) {
|
func ASN(asn int) (string, bool) {
|
||||||
|
|||||||
@@ -8,9 +8,6 @@ import (
|
|||||||
"git.eeqj.de/sneak/routewatch/internal/bogon"
|
"git.eeqj.de/sneak/routewatch/internal/bogon"
|
||||||
)
|
)
|
||||||
|
|
||||||
// privateUseTen is why a prefix or an address inside 10.0.0.0/8 is reserved.
|
|
||||||
const privateUseTen = "Private-Use (10.0.0.0/8)"
|
|
||||||
|
|
||||||
func TestPrefix(t *testing.T) {
|
func TestPrefix(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -19,8 +16,8 @@ func TestPrefix(t *testing.T) {
|
|||||||
// want is why the prefix is reserved, or "" when it is not.
|
// want is why the prefix is reserved, or "" when it is not.
|
||||||
want string
|
want string
|
||||||
}{
|
}{
|
||||||
{"10.1.2.0/24", privateUseTen},
|
{"10.1.2.0/24", "Private-Use (10.0.0.0/8)"},
|
||||||
{"10.0.0.0/8", privateUseTen},
|
{"10.0.0.0/8", "Private-Use (10.0.0.0/8)"},
|
||||||
{"192.0.2.0/24", "Documentation (TEST-NET-1) (192.0.2.0/24)"},
|
{"192.0.2.0/24", "Documentation (TEST-NET-1) (192.0.2.0/24)"},
|
||||||
{"2001:db8:1::/48", "Documentation (2001:db8::/32)"},
|
{"2001:db8:1::/48", "Documentation (2001:db8::/32)"},
|
||||||
{"fd00::/8", "Unique-Local (fc00::/7)"},
|
{"fd00::/8", "Unique-Local (fc00::/7)"},
|
||||||
@@ -59,39 +56,6 @@ func TestPrefix(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAddr(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
addr string
|
|
||||||
// want is why the address is reserved, or "" when it is not.
|
|
||||||
want string
|
|
||||||
}{
|
|
||||||
{"10.0.0.1", privateUseTen},
|
|
||||||
{"fd00::1", "Unique-Local (fc00::/7)"},
|
|
||||||
// An IPv4 address written in IPv6 form counts as the IPv4 address.
|
|
||||||
{"::ffff:10.0.0.1", privateUseTen},
|
|
||||||
{"::ffff:8.8.8.8", ""},
|
|
||||||
// An address that may be announced, inside a reserved block.
|
|
||||||
{"192.0.0.9", ""},
|
|
||||||
// Addresses anyone may announce.
|
|
||||||
{"8.8.8.8", ""},
|
|
||||||
{"2001:4860:4860::8888", ""},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tc := range tests {
|
|
||||||
t.Run(tc.addr, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
reason, reserved := bogon.Addr(netip.MustParseAddr(tc.addr))
|
|
||||||
if reason != tc.want || reserved != (tc.want != "") {
|
|
||||||
t.Errorf("Addr(%s) = %q, %v; want %q", tc.addr, reason,
|
|
||||||
reserved, tc.want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestASN(t *testing.T) {
|
func TestASN(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -1,51 +0,0 @@
|
|||||||
// Package cachedb opens cache.db, the SQLite file in the state directory that
|
|
||||||
// holds RouteWatch's caches of answers from outside services. It is separate
|
|
||||||
// from the routes database so that a cache write never queues behind a write
|
|
||||||
// to that database.
|
|
||||||
package cachedb
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"database/sql"
|
|
||||||
_ "embed" // for schema.sql
|
|
||||||
"fmt"
|
|
||||||
"path/filepath"
|
|
||||||
|
|
||||||
_ "github.com/mattn/go-sqlite3" // CGO SQLite driver
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
// fileName is the database's file in the state directory.
|
|
||||||
fileName = "cache.db"
|
|
||||||
|
|
||||||
// busyTimeoutMs is how long a write waits for another write to finish.
|
|
||||||
busyTimeoutMs = 5000
|
|
||||||
)
|
|
||||||
|
|
||||||
//go:embed schema.sql
|
|
||||||
var schema string
|
|
||||||
|
|
||||||
// Open opens cache.db in stateDir, creating the file and its tables when they
|
|
||||||
// do not exist. The caller closes the database.
|
|
||||||
func Open(ctx context.Context, stateDir string) (*sql.DB, error) {
|
|
||||||
dsn := fmt.Sprintf(
|
|
||||||
"file:%s?_journal_mode=WAL&_synchronous=NORMAL&_busy_timeout=%d"+
|
|
||||||
"&_txlock=immediate",
|
|
||||||
filepath.Join(stateDir, fileName),
|
|
||||||
busyTimeoutMs,
|
|
||||||
)
|
|
||||||
|
|
||||||
db, err := sql.Open("sqlite3", dsn)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("opening the cache database: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = db.ExecContext(ctx, schema)
|
|
||||||
if err != nil {
|
|
||||||
_ = db.Close()
|
|
||||||
|
|
||||||
return nil, fmt.Errorf("creating the cache database's tables: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return db, nil
|
|
||||||
}
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
package cachedb
|
|
||||||
|
|
||||||
import (
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestOpenCreatesTheFileInWALMode opens a new database, and then opens it
|
|
||||||
// again, as a restart does.
|
|
||||||
func TestOpenCreatesTheFileInWALMode(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
dir := t.TempDir()
|
|
||||||
|
|
||||||
for range 2 {
|
|
||||||
db, err := Open(t.Context(), dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Open: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var mode string
|
|
||||||
|
|
||||||
err = db.QueryRowContext(t.Context(), "PRAGMA journal_mode").Scan(&mode)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("reading the journal mode: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if mode != "wal" {
|
|
||||||
t.Errorf("journal mode %q, want wal", mode)
|
|
||||||
}
|
|
||||||
|
|
||||||
var rows int
|
|
||||||
|
|
||||||
err = db.QueryRowContext(t.Context(),
|
|
||||||
"SELECT COUNT(*) FROM hostnames").Scan(&rows)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("reading the hostnames table: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = db.Close()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Close: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err := os.Stat(filepath.Join(dir, fileName))
|
|
||||||
if err != nil {
|
|
||||||
t.Errorf("the file is not in the state directory: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestOpenFailsWhenTheDirectoryDoesNotExist checks that a state directory that
|
|
||||||
// is not there is reported, not ignored.
|
|
||||||
func TestOpenFailsWhenTheDirectoryDoesNotExist(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
db, err := Open(t.Context(), filepath.Join(t.TempDir(), "missing"))
|
|
||||||
if err == nil {
|
|
||||||
_ = db.Close()
|
|
||||||
|
|
||||||
t.Fatal("Open succeeded in a directory that does not exist")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
-- hostnames holds the PTR names of addresses. names has the names separated by
|
|
||||||
-- spaces, and is empty for an address with no PTR names. expires_at is in
|
|
||||||
-- seconds since 1970.
|
|
||||||
CREATE TABLE IF NOT EXISTS hostnames (
|
|
||||||
address TEXT PRIMARY KEY,
|
|
||||||
names TEXT NOT NULL,
|
|
||||||
expires_at INTEGER NOT NULL
|
|
||||||
);
|
|
||||||
@@ -51,6 +51,11 @@ const (
|
|||||||
// FormatIVPNJSON is IVPN's JSON server list. The hosts of its wireguard
|
// FormatIVPNJSON is IVPN's JSON server list. The hosts of its wireguard
|
||||||
// and openvpn gateways have their addresses in host and v2ray.
|
// and openvpn gateways have their addresses in host and v2ray.
|
||||||
FormatIVPNJSON
|
FormatIVPNJSON
|
||||||
|
|
||||||
|
// FormatProtonVPN has one address or network per line, with comment lines
|
||||||
|
// that start with #. Unlike FormatLines, a file with no addresses or
|
||||||
|
// networks fails.
|
||||||
|
FormatProtonVPN
|
||||||
)
|
)
|
||||||
|
|
||||||
// List is one public IP list.
|
// List is one public IP list.
|
||||||
|
|||||||
@@ -70,6 +70,19 @@ const (
|
|||||||
"config":{"api":{"ips":["192.0.2.73"]}}}`
|
"config":{"api":{"ips":["192.0.2.73"]}}}`
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// protonVPNBody is a small file in the format of the ProtonVPN-IPs list.
|
||||||
|
const protonVPNBody = "#\n# protonvpn_ip_ranges.txt\n" +
|
||||||
|
"# https://github.com/tn3w/ProtonVPN-IPs\n#\n" +
|
||||||
|
"# The /24 block of every advertised IPv4.\n#\n" +
|
||||||
|
"192.0.2.0/24\n198.51.100.0/24\n"
|
||||||
|
|
||||||
|
// protonVPNEntryBody is a small file in the format of the ProtonVPN-IPs list
|
||||||
|
// of server addresses.
|
||||||
|
const protonVPNEntryBody = "#\n# protonvpn_entry_ips.txt\n" +
|
||||||
|
"# https://github.com/tn3w/ProtonVPN-IPs\n#\n" +
|
||||||
|
"# Entry IPs clients connect to, resolved from ProtonVPN hostnames.\n#\n" +
|
||||||
|
"192.0.2.80\n2001:db8::80\n"
|
||||||
|
|
||||||
// errorPage is a file that is no list, as a publisher's error page is.
|
// errorPage is a file that is no list, as a publisher's error page is.
|
||||||
const errorPage = "<!DOCTYPE html>\n<html></html>\n"
|
const errorPage = "<!DOCTYPE html>\n<html></html>\n"
|
||||||
|
|
||||||
@@ -267,6 +280,20 @@ func TestLookupFindsVPNServers(t *testing.T) {
|
|||||||
listed: []string{"192.0.2.70", "192.0.2.71", "192.0.2.72"},
|
listed: []string{"192.0.2.70", "192.0.2.71", "192.0.2.72"},
|
||||||
unlisted: []string{"192.0.2.73", "2001:db8::70"},
|
unlisted: []string{"192.0.2.73", "2001:db8::70"},
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
name: "ProtonVPN networks",
|
||||||
|
format: FormatProtonVPN,
|
||||||
|
body: protonVPNBody,
|
||||||
|
listed: []string{"192.0.2.0", "192.0.2.255", "198.51.100.7"},
|
||||||
|
unlisted: []string{"192.0.3.1", "2001:db8::90"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "ProtonVPN server addresses",
|
||||||
|
format: FormatProtonVPN,
|
||||||
|
body: protonVPNEntryBody,
|
||||||
|
listed: []string{"192.0.2.80", "2001:db8::80"},
|
||||||
|
unlisted: []string{"192.0.2.81", "2001:db8::81"},
|
||||||
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -479,6 +506,31 @@ func TestWrongVPNDocumentFailsToParse(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestWrongProtonVPNDocumentFailsToParse parses documents that are not a
|
||||||
|
// ProtonVPN-IPs list. Each fails, so none can replace a good copy.
|
||||||
|
func TestWrongProtonVPNDocumentFailsToParse(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
bodies := map[string]string{
|
||||||
|
"error page": errorPage,
|
||||||
|
"JSON": `{"status":"ok"}`,
|
||||||
|
"comments only": "#\n# protonvpn_ip_ranges.txt\n#\n",
|
||||||
|
"bad network": "192.0.2.0/24\nexample\n",
|
||||||
|
"an empty file": "",
|
||||||
|
}
|
||||||
|
|
||||||
|
for name, body := range bodies {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
addresses, err := parse(strings.NewReader(body), FormatProtonVPN)
|
||||||
|
if err == nil {
|
||||||
|
t.Errorf("parse returned %v and no error", addresses.Ranges())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestLookupMarksVPNLists looks up an address on a VPN list and on another
|
// TestLookupMarksVPNLists looks up an address on a VPN list and on another
|
||||||
// list. Only the VPN list's listing says it is one.
|
// list. Only the VPN list's listing says it is one.
|
||||||
func TestLookupMarksVPNLists(t *testing.T) {
|
func TestLookupMarksVPNLists(t *testing.T) {
|
||||||
@@ -523,7 +575,10 @@ func TestPublicListsMarkTheVPNLists(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
want := []string{"ivpn", "mullvad", "nordvpn", "pia", "x4bnet", "x4bnet_v6"}
|
want := []string{
|
||||||
|
"ivpn", "mullvad", "nordvpn", "pia", "protonvpn", "protonvpn_entry",
|
||||||
|
"x4bnet", "x4bnet_v6",
|
||||||
|
}
|
||||||
if !slices.Equal(got, want) {
|
if !slices.Equal(got, want) {
|
||||||
t.Errorf("VPN lists %v, want %v", got, want)
|
t.Errorf("VPN lists %v, want %v", got, want)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -146,6 +146,30 @@ func vpnLists() []List {
|
|||||||
Format: FormatPIAJSON,
|
Format: FormatPIAJSON,
|
||||||
VPN: true,
|
VPN: true,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
// Proton's own server list needs a logged-in session, so this is
|
||||||
|
// the public ProtonVPN-IPs mirror of it. Proton publishes only
|
||||||
|
// the addresses clients connect to. The mirror's README says
|
||||||
|
// traffic leaves from others in the same /24, so this file of
|
||||||
|
// /24 networks is the one it says to use for detecting exits.
|
||||||
|
Name: "protonvpn",
|
||||||
|
Description: "In a /24 network that holds a server of the " +
|
||||||
|
"ProtonVPN service",
|
||||||
|
URL: "https://raw.githubusercontent.com/tn3w/ProtonVPN-IPs/master/" +
|
||||||
|
"protonvpn_ip_ranges.txt",
|
||||||
|
Interval: vpnListInterval,
|
||||||
|
Format: FormatProtonVPN,
|
||||||
|
VPN: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Name: "protonvpn_entry",
|
||||||
|
Description: "A server of the ProtonVPN service",
|
||||||
|
URL: "https://raw.githubusercontent.com/tn3w/ProtonVPN-IPs/master/" +
|
||||||
|
"protonvpn_entry_ips.txt",
|
||||||
|
Interval: vpnListInterval,
|
||||||
|
Format: FormatProtonVPN,
|
||||||
|
VPN: true,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
Name: "x4bnet",
|
Name: "x4bnet",
|
||||||
Description: "In an IPv4 network of a VPN service, according to " +
|
Description: "In an IPv4 network of a VPN service, according to " +
|
||||||
|
|||||||
@@ -58,6 +58,8 @@ func parse(r io.Reader, format Format) (*netipx.IPSet, error) {
|
|||||||
return parsePIA(r)
|
return parsePIA(r)
|
||||||
case FormatIVPNJSON:
|
case FormatIVPNJSON:
|
||||||
return parseIVPN(r)
|
return parseIVPN(r)
|
||||||
|
case FormatProtonVPN:
|
||||||
|
return parseProtonVPN(r)
|
||||||
default:
|
default:
|
||||||
return nil, errUnknownFormat
|
return nil, errUnknownFormat
|
||||||
}
|
}
|
||||||
@@ -329,6 +331,22 @@ func parseIVPN(r io.Reader) (*netipx.IPSet, error) {
|
|||||||
return addressSet(texts)
|
return addressSet(texts)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// parseProtonVPN returns the addresses or networks in a ProtonVPN-IPs list. A
|
||||||
|
// file with no addresses or networks fails, because the service always has
|
||||||
|
// servers.
|
||||||
|
func parseProtonVPN(r io.Reader) (*netipx.IPSet, error) {
|
||||||
|
addresses, err := parseLines(r, addAddressOrNetwork)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(addresses.Ranges()) == 0 {
|
||||||
|
return nil, errNoAddresses
|
||||||
|
}
|
||||||
|
|
||||||
|
return addresses, nil
|
||||||
|
}
|
||||||
|
|
||||||
// readJSON reads r to its end and decodes it, as one JSON document, into v.
|
// readJSON reads r to its end and decodes it, as one JSON document, into v.
|
||||||
func readJSON(r io.Reader, v any) error {
|
func readJSON(r io.Reader, v any) error {
|
||||||
data, err := io.ReadAll(r)
|
data, err := io.ReadAll(r)
|
||||||
|
|||||||
@@ -1,324 +0,0 @@
|
|||||||
// Package ptr looks up the PTR names of addresses over DNS-over-HTTPS. It
|
|
||||||
// keeps answers in memory for as long as DNS allows, and in cache.db so that
|
|
||||||
// the answers survive a restart.
|
|
||||||
package ptr
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"database/sql"
|
|
||||||
"fmt"
|
|
||||||
"log/slog"
|
|
||||||
"maps"
|
|
||||||
"net/http"
|
|
||||||
"net/netip"
|
|
||||||
"slices"
|
|
||||||
"strings"
|
|
||||||
"sync"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/bogon"
|
|
||||||
"github.com/miekg/dns"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
// flushInterval is how often new answers are written to cache.db.
|
|
||||||
flushInterval = 5 * time.Second
|
|
||||||
|
|
||||||
// sweepInterval is how often expired answers are deleted.
|
|
||||||
sweepInterval = 10 * time.Minute
|
|
||||||
|
|
||||||
// nameSeparator separates the names in a row of the hostnames table.
|
|
||||||
nameSeparator = " "
|
|
||||||
|
|
||||||
// maxAddresses limits how many addresses the cache holds, so that a public
|
|
||||||
// lookup cannot grow memory and cache.db without bound.
|
|
||||||
maxAddresses = 250_000
|
|
||||||
)
|
|
||||||
|
|
||||||
// Params is what New needs.
|
|
||||||
type Params struct {
|
|
||||||
// DB is the open cache.db.
|
|
||||||
DB *sql.DB
|
|
||||||
|
|
||||||
// Resolvers are the URLs of the DNS-over-HTTPS resolvers to ask, in order.
|
|
||||||
Resolvers []string
|
|
||||||
|
|
||||||
Logger *slog.Logger
|
|
||||||
}
|
|
||||||
|
|
||||||
// entry is one address's cached answer.
|
|
||||||
type entry struct {
|
|
||||||
// names are the address's PTR names, nil when it has none.
|
|
||||||
names []string
|
|
||||||
expires time.Time
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cache answers PTR lookups from memory, and asks the resolvers for an address
|
|
||||||
// it holds no current answer for. Start loads it from cache.db.
|
|
||||||
type Cache struct {
|
|
||||||
db *sql.DB
|
|
||||||
resolvers []string
|
|
||||||
timeLimit time.Duration
|
|
||||||
maxEntries int
|
|
||||||
logger *slog.Logger
|
|
||||||
client *http.Client
|
|
||||||
now func() time.Time
|
|
||||||
|
|
||||||
// entries holds the kept answers by address, at most maxEntries of them.
|
|
||||||
// pending holds those that cache.db does not have yet.
|
|
||||||
mu sync.RWMutex
|
|
||||||
entries map[string]entry
|
|
||||||
pending map[string]entry
|
|
||||||
|
|
||||||
// stop ends the goroutine Start runs, and done is closed when it returns.
|
|
||||||
stop context.CancelFunc
|
|
||||||
done chan struct{}
|
|
||||||
}
|
|
||||||
|
|
||||||
// New returns an empty Cache. Start loads it.
|
|
||||||
func New(params Params) *Cache {
|
|
||||||
return &Cache{
|
|
||||||
db: params.DB,
|
|
||||||
resolvers: params.Resolvers,
|
|
||||||
timeLimit: LookupTimeout,
|
|
||||||
maxEntries: maxAddresses,
|
|
||||||
logger: params.Logger.With("component", "ptr"),
|
|
||||||
client: &http.Client{},
|
|
||||||
now: time.Now,
|
|
||||||
entries: make(map[string]entry),
|
|
||||||
pending: make(map[string]entry),
|
|
||||||
done: make(chan struct{}),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// LookupAddr returns the PTR names of addr, fully qualified, or none when it
|
|
||||||
// has none. A reserved address has none, and is not sent to the resolvers. A
|
|
||||||
// lookup takes at most LookupTimeout. A cached answer makes no outside request.
|
|
||||||
// It returns an error when no resolver answered, and nothing is cached then.
|
|
||||||
func (c *Cache) LookupAddr(ctx context.Context, addr string) ([]string, error) {
|
|
||||||
parsed, err := netip.ParseAddr(addr)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("parsing the address: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, reserved := bogon.Addr(parsed)
|
|
||||||
if reserved {
|
|
||||||
return []string{}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
address := parsed.Unmap().String()
|
|
||||||
|
|
||||||
cached, ok := c.current(address)
|
|
||||||
if ok {
|
|
||||||
return slices.Clone(cached.names), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
reverseName, err := dns.ReverseAddr(address)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("building the reverse name: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
result, err := c.lookup(ctx, reverseName)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if result.ttl > 0 {
|
|
||||||
c.store(address, entry{names: result.names, expires: c.now().Add(result.ttl)})
|
|
||||||
}
|
|
||||||
|
|
||||||
return slices.Clone(result.names), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// Start loads the answers that have not expired from cache.db. It then starts
|
|
||||||
// the goroutine that writes new answers there and deletes expired ones. The
|
|
||||||
// goroutine runs until Stop: the end of ctx does not stop it.
|
|
||||||
func (c *Cache) Start(ctx context.Context) error {
|
|
||||||
err := c.load(ctx)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
c.logger.Info("PTR cache loaded", "entries", len(c.entries))
|
|
||||||
|
|
||||||
ctx, c.stop = context.WithCancel(context.WithoutCancel(ctx))
|
|
||||||
|
|
||||||
go c.run(ctx)
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// Stop ends the goroutine, which first writes the answers cache.db does not
|
|
||||||
// have yet, and waits until it has returned or ctx ends.
|
|
||||||
func (c *Cache) Stop(ctx context.Context) {
|
|
||||||
c.stop()
|
|
||||||
|
|
||||||
select {
|
|
||||||
case <-c.done:
|
|
||||||
c.logger.Info("PTR cache stopped")
|
|
||||||
case <-ctx.Done():
|
|
||||||
c.logger.Warn("Stop deadline passed before the PTR cache was written")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// current returns the answer held for address, unless it has expired.
|
|
||||||
func (c *Cache) current(address string) (entry, bool) {
|
|
||||||
c.mu.RLock()
|
|
||||||
defer c.mu.RUnlock()
|
|
||||||
|
|
||||||
found, ok := c.entries[address]
|
|
||||||
if !ok || !c.now().Before(found.expires) {
|
|
||||||
return entry{}, false
|
|
||||||
}
|
|
||||||
|
|
||||||
return found, true
|
|
||||||
}
|
|
||||||
|
|
||||||
// store keeps an answer in memory and queues it for cache.db. Once entries
|
|
||||||
// holds maxEntries addresses it keeps no new address. It still replaces the
|
|
||||||
// answer of an address already held.
|
|
||||||
func (c *Cache) store(address string, answer entry) {
|
|
||||||
c.mu.Lock()
|
|
||||||
defer c.mu.Unlock()
|
|
||||||
|
|
||||||
_, held := c.entries[address]
|
|
||||||
if !held && len(c.entries) >= c.maxEntries {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
c.entries[address] = answer
|
|
||||||
c.pending[address] = answer
|
|
||||||
}
|
|
||||||
|
|
||||||
// run writes new answers and deletes expired ones until ctx ends.
|
|
||||||
func (c *Cache) run(ctx context.Context) {
|
|
||||||
defer close(c.done)
|
|
||||||
|
|
||||||
flushes := time.NewTicker(flushInterval)
|
|
||||||
defer flushes.Stop()
|
|
||||||
|
|
||||||
sweeps := time.NewTicker(sweepInterval)
|
|
||||||
defer sweeps.Stop()
|
|
||||||
|
|
||||||
for {
|
|
||||||
select {
|
|
||||||
case <-ctx.Done():
|
|
||||||
c.flush(context.WithoutCancel(ctx))
|
|
||||||
|
|
||||||
return
|
|
||||||
case <-flushes.C:
|
|
||||||
c.flush(ctx)
|
|
||||||
case <-sweeps.C:
|
|
||||||
c.sweep(ctx)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// load reads the rows that have not expired into memory.
|
|
||||||
func (c *Cache) load(ctx context.Context) error {
|
|
||||||
rows, err := c.db.QueryContext(ctx,
|
|
||||||
"SELECT address, names, expires_at FROM hostnames WHERE expires_at > ?",
|
|
||||||
c.now().Unix())
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("reading the PTR cache: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
defer func() { _ = rows.Close() }()
|
|
||||||
|
|
||||||
for rows.Next() {
|
|
||||||
var (
|
|
||||||
address, names string
|
|
||||||
expires int64
|
|
||||||
)
|
|
||||||
|
|
||||||
err = rows.Scan(&address, &names, &expires)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("reading a PTR cache row: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
c.entries[address] = entry{
|
|
||||||
names: strings.Fields(names),
|
|
||||||
expires: time.Unix(expires, 0).UTC(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
err = rows.Err()
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("reading the PTR cache: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// flush writes the pending answers to cache.db. Those it cannot write stay
|
|
||||||
// pending, unless a newer answer for the address has arrived meanwhile.
|
|
||||||
func (c *Cache) flush(ctx context.Context) {
|
|
||||||
c.mu.Lock()
|
|
||||||
batch := c.pending
|
|
||||||
c.pending = make(map[string]entry)
|
|
||||||
c.mu.Unlock()
|
|
||||||
|
|
||||||
if len(batch) == 0 {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
err := c.writeRows(ctx, batch)
|
|
||||||
if err == nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
c.logger.Error("Failed to write the PTR cache", "error", err)
|
|
||||||
|
|
||||||
c.mu.Lock()
|
|
||||||
defer c.mu.Unlock()
|
|
||||||
|
|
||||||
for address, answer := range batch {
|
|
||||||
_, newer := c.pending[address]
|
|
||||||
if !newer {
|
|
||||||
c.pending[address] = answer
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// writeRows writes batch to cache.db in one transaction.
|
|
||||||
func (c *Cache) writeRows(ctx context.Context, batch map[string]entry) error {
|
|
||||||
tx, err := c.db.BeginTx(ctx, nil)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
defer func() { _ = tx.Rollback() }()
|
|
||||||
|
|
||||||
for address, answer := range batch {
|
|
||||||
_, err = tx.ExecContext(ctx,
|
|
||||||
"INSERT OR REPLACE INTO hostnames (address, names, expires_at) "+
|
|
||||||
"VALUES (?, ?, ?)",
|
|
||||||
address,
|
|
||||||
strings.Join(answer.names, nameSeparator),
|
|
||||||
answer.expires.Unix())
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return tx.Commit()
|
|
||||||
}
|
|
||||||
|
|
||||||
// sweep deletes the expired answers from memory and cache.db.
|
|
||||||
func (c *Cache) sweep(ctx context.Context) {
|
|
||||||
now := c.now()
|
|
||||||
expired := func(_ string, answer entry) bool {
|
|
||||||
return !now.Before(answer.expires)
|
|
||||||
}
|
|
||||||
|
|
||||||
c.mu.Lock()
|
|
||||||
maps.DeleteFunc(c.entries, expired)
|
|
||||||
maps.DeleteFunc(c.pending, expired)
|
|
||||||
c.mu.Unlock()
|
|
||||||
|
|
||||||
_, err := c.db.ExecContext(ctx,
|
|
||||||
"DELETE FROM hostnames WHERE expires_at <= ?", now.Unix())
|
|
||||||
if err != nil {
|
|
||||||
c.logger.Error("Failed to delete expired PTR cache rows", "error", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,951 +0,0 @@
|
|||||||
package ptr
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"database/sql"
|
|
||||||
"errors"
|
|
||||||
"io"
|
|
||||||
"log/slog"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"slices"
|
|
||||||
"strconv"
|
|
||||||
"sync"
|
|
||||||
"testing"
|
|
||||||
"testing/synctest"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/cachedb"
|
|
||||||
"github.com/miekg/dns"
|
|
||||||
)
|
|
||||||
|
|
||||||
// These addresses are public, because a reserved address is not sent to the
|
|
||||||
// resolvers. That rules out the documentation ranges.
|
|
||||||
const (
|
|
||||||
testAddress = "8.8.4.4"
|
|
||||||
testReverse = "4.4.8.8.in-addr.arpa."
|
|
||||||
mappedAddress = "::ffff:" + testAddress
|
|
||||||
otherAddress = "8.8.8.8"
|
|
||||||
testV6Address = "2606:4700:4700::1111"
|
|
||||||
testV6Reverse = "1.1.1.1.0.0.0.0.0.0.0.0.0.0.0.0." +
|
|
||||||
"0.0.0.0.0.0.7.4.0.0.7.4.6.0.6.2.ip6.arpa."
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
testName = "host.example.com."
|
|
||||||
otherName = "other.example.com."
|
|
||||||
|
|
||||||
// testTimeLimit replaces LookupTimeout in the tests that use a real
|
|
||||||
// server. A stall on a busy host cannot use it up, and no test waits for
|
|
||||||
// it.
|
|
||||||
testTimeLimit = 2 * time.Minute
|
|
||||||
)
|
|
||||||
|
|
||||||
// classlessOwner is the name that holds the PTR records of a CNAME chain.
|
|
||||||
const classlessOwner = "4.0-25.4.8.8.in-addr.arpa."
|
|
||||||
|
|
||||||
// fixedTime returns the time a test's clock starts at.
|
|
||||||
func fixedTime() time.Time {
|
|
||||||
return time.Date(2026, time.January, 2, 3, 4, 5, 0, time.UTC)
|
|
||||||
}
|
|
||||||
|
|
||||||
// fakeResolver is a DNS-over-HTTPS server. It answers each query with reply's
|
|
||||||
// result, and counts its requests.
|
|
||||||
type fakeResolver struct {
|
|
||||||
server *httptest.Server
|
|
||||||
|
|
||||||
mu sync.Mutex
|
|
||||||
requests int
|
|
||||||
reply func(http.ResponseWriter, *http.Request, *dns.Msg)
|
|
||||||
}
|
|
||||||
|
|
||||||
// newFakeResolver starts a fake resolver that answers with reply.
|
|
||||||
func newFakeResolver(
|
|
||||||
t *testing.T,
|
|
||||||
reply func(http.ResponseWriter, *http.Request, *dns.Msg),
|
|
||||||
) *fakeResolver {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
resolver := &fakeResolver{reply: reply}
|
|
||||||
resolver.server = httptest.NewServer(http.HandlerFunc(resolver.serve))
|
|
||||||
t.Cleanup(resolver.server.Close)
|
|
||||||
|
|
||||||
return resolver
|
|
||||||
}
|
|
||||||
|
|
||||||
// serve checks that the request is RFC 8484 wire format and passes the
|
|
||||||
// question on to the current reply.
|
|
||||||
func (f *fakeResolver) serve(w http.ResponseWriter, r *http.Request) {
|
|
||||||
f.mu.Lock()
|
|
||||||
f.requests++
|
|
||||||
reply := f.reply
|
|
||||||
f.mu.Unlock()
|
|
||||||
|
|
||||||
body, err := io.ReadAll(r.Body)
|
|
||||||
if err != nil {
|
|
||||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
var query dns.Msg
|
|
||||||
|
|
||||||
err = query.Unpack(body)
|
|
||||||
if err != nil || r.Header.Get("Content-Type") != dnsMessageType ||
|
|
||||||
r.Header.Get("Accept") != dnsMessageType || r.Method != http.MethodPost {
|
|
||||||
http.Error(w, "not an RFC 8484 request", http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
reply(w, r, &query)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *fakeResolver) setReply(
|
|
||||||
reply func(http.ResponseWriter, *http.Request, *dns.Msg),
|
|
||||||
) {
|
|
||||||
f.mu.Lock()
|
|
||||||
defer f.mu.Unlock()
|
|
||||||
|
|
||||||
f.reply = reply
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *fakeResolver) requestCount() int {
|
|
||||||
f.mu.Lock()
|
|
||||||
defer f.mu.Unlock()
|
|
||||||
|
|
||||||
return f.requests
|
|
||||||
}
|
|
||||||
|
|
||||||
// writeReply packs reply and writes it as the answer to query.
|
|
||||||
func writeReply(w http.ResponseWriter, query, reply *dns.Msg) {
|
|
||||||
reply.SetReply(query)
|
|
||||||
|
|
||||||
wire, err := reply.Pack()
|
|
||||||
if err != nil {
|
|
||||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
w.Header().Set("Content-Type", dnsMessageType)
|
|
||||||
_, _ = w.Write(wire)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ptrReply answers with one PTR record for each name, each with the TTL ttl.
|
|
||||||
func ptrReply(
|
|
||||||
ttl uint32, names ...string,
|
|
||||||
) func(http.ResponseWriter, *http.Request, *dns.Msg) {
|
|
||||||
return func(w http.ResponseWriter, _ *http.Request, query *dns.Msg) {
|
|
||||||
reply := new(dns.Msg)
|
|
||||||
|
|
||||||
for _, name := range names {
|
|
||||||
reply.Answer = append(reply.Answer, &dns.PTR{
|
|
||||||
Hdr: dns.RR_Header{
|
|
||||||
Name: query.Question[0].Name, Rrtype: dns.TypePTR,
|
|
||||||
Class: dns.ClassINET, Ttl: ttl,
|
|
||||||
},
|
|
||||||
Ptr: name,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
writeReply(w, query, reply)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// rcodeReply answers with the response code and nothing else.
|
|
||||||
func rcodeReply(rcode int) func(http.ResponseWriter, *http.Request, *dns.Msg) {
|
|
||||||
return func(w http.ResponseWriter, _ *http.Request, query *dns.Msg) {
|
|
||||||
reply := new(dns.Msg)
|
|
||||||
reply.SetRcode(query, rcode)
|
|
||||||
|
|
||||||
wire, err := reply.Pack()
|
|
||||||
if err != nil {
|
|
||||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
_, _ = w.Write(wire)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// soaReply answers with the response code and a SOA record with these TTLs.
|
|
||||||
func soaReply(
|
|
||||||
rcode int, soaTTL, minimum uint32,
|
|
||||||
) func(http.ResponseWriter, *http.Request, *dns.Msg) {
|
|
||||||
return func(w http.ResponseWriter, _ *http.Request, query *dns.Msg) {
|
|
||||||
reply := new(dns.Msg)
|
|
||||||
reply.SetRcode(query, rcode)
|
|
||||||
reply.Ns = []dns.RR{&dns.SOA{
|
|
||||||
Hdr: dns.RR_Header{
|
|
||||||
Name: "arpa.", Rrtype: dns.TypeSOA, Class: dns.ClassINET, Ttl: soaTTL,
|
|
||||||
},
|
|
||||||
Ns: "ns.example.com.", Mbox: "hostmaster.example.com.", Minttl: minimum,
|
|
||||||
}}
|
|
||||||
|
|
||||||
wire, err := reply.Pack()
|
|
||||||
if err != nil {
|
|
||||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
_, _ = w.Write(wire)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// hangingReply never answers: it waits until the client gives up.
|
|
||||||
func hangingReply(_ http.ResponseWriter, r *http.Request, _ *dns.Msg) {
|
|
||||||
<-r.Context().Done()
|
|
||||||
}
|
|
||||||
|
|
||||||
// statusReply answers with an HTTP status.
|
|
||||||
func statusReply(status int) func(http.ResponseWriter, *http.Request, *dns.Msg) {
|
|
||||||
return func(w http.ResponseWriter, _ *http.Request, _ *dns.Msg) {
|
|
||||||
w.WriteHeader(status)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// openTestDB opens a cache.db in dir.
|
|
||||||
func openTestDB(t *testing.T, dir string) *sql.DB {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
db, err := cachedb.Open(t.Context(), dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("opening the cache database: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Cleanup(func() { _ = db.Close() })
|
|
||||||
|
|
||||||
return db
|
|
||||||
}
|
|
||||||
|
|
||||||
// testClock is a clock a test moves by hand.
|
|
||||||
type testClock struct {
|
|
||||||
mu sync.Mutex
|
|
||||||
now time.Time
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *testClock) time() time.Time {
|
|
||||||
c.mu.Lock()
|
|
||||||
defer c.mu.Unlock()
|
|
||||||
|
|
||||||
return c.now
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *testClock) advance(d time.Duration) {
|
|
||||||
c.mu.Lock()
|
|
||||||
defer c.mu.Unlock()
|
|
||||||
|
|
||||||
c.now = c.now.Add(d)
|
|
||||||
}
|
|
||||||
|
|
||||||
// newTestCache returns a Cache that asks the resolvers in order, reads the
|
|
||||||
// time from a clock the test holds, has testTimeLimit, and is not started.
|
|
||||||
func newTestCache(
|
|
||||||
t *testing.T, db *sql.DB, resolvers ...*fakeResolver,
|
|
||||||
) (*Cache, *testClock) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
urls := make([]string, 0, len(resolvers))
|
|
||||||
for _, resolver := range resolvers {
|
|
||||||
urls = append(urls, resolver.server.URL)
|
|
||||||
}
|
|
||||||
|
|
||||||
cache := New(Params{DB: db, Resolvers: urls, Logger: slog.Default()})
|
|
||||||
clock := &testClock{now: fixedTime()}
|
|
||||||
cache.now = clock.time
|
|
||||||
cache.timeLimit = testTimeLimit
|
|
||||||
|
|
||||||
return cache, clock
|
|
||||||
}
|
|
||||||
|
|
||||||
// handlerTransport hands each request to the fake resolver for its host, with
|
|
||||||
// no network between them. A test in a synctest bubble needs this: the bubble's
|
|
||||||
// clock stands still while one of its goroutines waits on a real connection. As
|
|
||||||
// over a network, a request whose time is up is not sent.
|
|
||||||
type handlerTransport map[string]*fakeResolver
|
|
||||||
|
|
||||||
func (h handlerTransport) RoundTrip(r *http.Request) (*http.Response, error) {
|
|
||||||
recorder := httptest.NewRecorder()
|
|
||||||
|
|
||||||
if r.Context().Err() == nil {
|
|
||||||
h[r.URL.Host].serve(recorder, r)
|
|
||||||
}
|
|
||||||
|
|
||||||
err := r.Context().Err()
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return recorder.Result(), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// newBubbleCache returns a Cache with the real time limit, for a test in a
|
|
||||||
// synctest bubble. It has one fake resolver for each reply, in order, reached
|
|
||||||
// through a handlerTransport.
|
|
||||||
func newBubbleCache(
|
|
||||||
replies ...func(http.ResponseWriter, *http.Request, *dns.Msg),
|
|
||||||
) (*Cache, []*fakeResolver) {
|
|
||||||
transport := make(handlerTransport, len(replies))
|
|
||||||
resolvers := make([]*fakeResolver, 0, len(replies))
|
|
||||||
urls := make([]string, 0, len(replies))
|
|
||||||
|
|
||||||
for i, reply := range replies {
|
|
||||||
host := "resolver" + strconv.Itoa(i) + ".example.com"
|
|
||||||
resolver := &fakeResolver{reply: reply}
|
|
||||||
|
|
||||||
transport[host] = resolver
|
|
||||||
resolvers = append(resolvers, resolver)
|
|
||||||
urls = append(urls, "https://"+host+"/dns-query")
|
|
||||||
}
|
|
||||||
|
|
||||||
cache := New(Params{Resolvers: urls, Logger: slog.Default()})
|
|
||||||
cache.client = &http.Client{Transport: transport}
|
|
||||||
|
|
||||||
return cache, resolvers
|
|
||||||
}
|
|
||||||
|
|
||||||
// lookupOK looks up address and fails the test on an error.
|
|
||||||
func lookupOK(t *testing.T, cache *Cache, address string) []string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
names, err := cache.LookupAddr(t.Context(), address)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("LookupAddr(%s): %v", address, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return names
|
|
||||||
}
|
|
||||||
|
|
||||||
func countRows(t *testing.T, db *sql.DB) int {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var rows int
|
|
||||||
|
|
||||||
err := db.QueryRowContext(t.Context(),
|
|
||||||
"SELECT COUNT(*) FROM hostnames").Scan(&rows)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("counting rows: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return rows
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLookupAsksForTheReverseName(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
address string
|
|
||||||
want string
|
|
||||||
}{
|
|
||||||
{testAddress, testReverse},
|
|
||||||
{mappedAddress, testReverse},
|
|
||||||
{testV6Address, testV6Reverse},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
var asked string
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t,
|
|
||||||
func(w http.ResponseWriter, r *http.Request, query *dns.Msg) {
|
|
||||||
asked = query.Question[0].Name
|
|
||||||
if query.Question[0].Qtype != dns.TypePTR {
|
|
||||||
t.Errorf("query type %d, want PTR", query.Question[0].Qtype)
|
|
||||||
}
|
|
||||||
|
|
||||||
ptrReply(300, testName)(w, r, query)
|
|
||||||
})
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, tt.address)
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("%s: names %v, want %s", tt.address, names, testName)
|
|
||||||
}
|
|
||||||
|
|
||||||
if asked != tt.want {
|
|
||||||
t.Errorf("%s: asked for %s, want %s", tt.address, asked, tt.want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestOnlyPublicAddressesAreSentToTheResolvers gives every resolver an answer
|
|
||||||
// to send, which a reserved address must not get.
|
|
||||||
func TestOnlyPublicAddressesAreSentToTheResolvers(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
address string
|
|
||||||
want []string
|
|
||||||
requests int
|
|
||||||
}{
|
|
||||||
{"private IPv4", "10.0.0.1", []string{}, 0},
|
|
||||||
{"documentation IPv4", "192.0.2.1", []string{}, 0},
|
|
||||||
{"private IPv4 in IPv6 form", "::ffff:10.0.0.1", []string{}, 0},
|
|
||||||
{"unique-local IPv6", "fd00::1", []string{}, 0},
|
|
||||||
{"documentation IPv6", "2001:db8::1", []string{}, 0},
|
|
||||||
{"public IPv4", testAddress, []string{testName}, 1},
|
|
||||||
{"public IPv4 in IPv6 form", mappedAddress, []string{testName}, 1},
|
|
||||||
{"public IPv6", testV6Address, []string{testName}, 1},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
resolver := newFakeResolver(t, ptrReply(300, testName))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, tt.address)
|
|
||||||
if names == nil || !slices.Equal(names, tt.want) {
|
|
||||||
t.Errorf("%s: names %#v, want %#v", tt.name, names, tt.want)
|
|
||||||
}
|
|
||||||
|
|
||||||
if got := resolver.requestCount(); got != tt.requests {
|
|
||||||
t.Errorf("%s: %d requests, want %d", tt.name, got, tt.requests)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestResolversAreTriedInOrderUntilOneAnswers has the first two resolvers fail,
|
|
||||||
// in different ways, and the third answer. The fourth is not asked.
|
|
||||||
func TestResolversAreTriedInOrderUntilOneAnswers(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
var (
|
|
||||||
mu sync.Mutex
|
|
||||||
order []int
|
|
||||||
)
|
|
||||||
|
|
||||||
asked := func(
|
|
||||||
index int, reply func(http.ResponseWriter, *http.Request, *dns.Msg),
|
|
||||||
) func(http.ResponseWriter, *http.Request, *dns.Msg) {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request, query *dns.Msg) {
|
|
||||||
mu.Lock()
|
|
||||||
|
|
||||||
order = append(order, index)
|
|
||||||
|
|
||||||
mu.Unlock()
|
|
||||||
|
|
||||||
reply(w, r, query)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
resolvers := []*fakeResolver{
|
|
||||||
newFakeResolver(t, asked(0, statusReply(http.StatusServiceUnavailable))),
|
|
||||||
newFakeResolver(t, asked(1, rcodeReply(dns.RcodeServerFailure))),
|
|
||||||
newFakeResolver(t, asked(2, ptrReply(300, testName))),
|
|
||||||
newFakeResolver(t, asked(3, ptrReply(300, otherName))),
|
|
||||||
}
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolvers...)
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("names %v, want %s", names, testName)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !slices.Equal(order, []int{0, 1, 2}) {
|
|
||||||
t.Errorf("resolvers asked %v, want [0 1 2]", order)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestResolverThatNeverAnswersLeavesTimeForTheNext would fail if the first
|
|
||||||
// resolver could take the whole time limit. The first gets half of it, the
|
|
||||||
// share of one resolver in two. The test runs in a synctest bubble, whose clock
|
|
||||||
// is fake, so a stall on the host uses none of the limit.
|
|
||||||
func TestResolverThatNeverAnswersLeavesTimeForTheNext(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
|
||||||
cache, _ := newBubbleCache(hangingReply, ptrReply(300, testName))
|
|
||||||
|
|
||||||
start := time.Now()
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
elapsed := time.Since(start)
|
|
||||||
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("names %v, want %s", names, testName)
|
|
||||||
}
|
|
||||||
|
|
||||||
if elapsed != time.Second {
|
|
||||||
t.Errorf("took %s, want 1s", elapsed)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestLookupStopsAtTheTimeLimit has no resolver answer. The lookup fails when
|
|
||||||
// 2 seconds have passed, having asked every resolver. The test runs in a
|
|
||||||
// synctest bubble, whose clock is fake, so a stall on the host uses none of the
|
|
||||||
// limit.
|
|
||||||
func TestLookupStopsAtTheTimeLimit(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
|
||||||
cache, resolvers := newBubbleCache(
|
|
||||||
hangingReply, hangingReply, hangingReply, hangingReply)
|
|
||||||
|
|
||||||
start := time.Now()
|
|
||||||
_, err := cache.LookupAddr(t.Context(), testAddress)
|
|
||||||
elapsed := time.Since(start)
|
|
||||||
|
|
||||||
if !errors.Is(err, context.DeadlineExceeded) {
|
|
||||||
t.Errorf("error %v, want %v", err, context.DeadlineExceeded)
|
|
||||||
}
|
|
||||||
|
|
||||||
if elapsed != 2*time.Second {
|
|
||||||
t.Errorf("took %s, want 2s", elapsed)
|
|
||||||
}
|
|
||||||
|
|
||||||
for i, resolver := range resolvers {
|
|
||||||
if resolver.requestCount() != 1 {
|
|
||||||
t.Errorf("resolver %d asked %d times, want once",
|
|
||||||
i, resolver.requestCount())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestAnswerIsServedFromMemoryUntilItExpires makes a second lookup within the
|
|
||||||
// TTL, which asks nobody, and a third after it, which asks again.
|
|
||||||
func TestAnswerIsServedFromMemoryUntilItExpires(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t, ptrReply(300, testName))
|
|
||||||
cache, clock := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("cached names %v, want %s", names, testName)
|
|
||||||
}
|
|
||||||
|
|
||||||
if got := resolver.requestCount(); got != 1 {
|
|
||||||
t.Errorf("%d requests after the second lookup, want 1", got)
|
|
||||||
}
|
|
||||||
|
|
||||||
clock.advance(300 * time.Second)
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
if got := resolver.requestCount(); got != 2 {
|
|
||||||
t.Errorf("%d requests after the TTL passed, want 2", got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestAnAddressHasOneCacheEntryHoweverItIsWritten looks up an IPv4 address in
|
|
||||||
// its IPv4-mapped IPv6 form, then in its usual form.
|
|
||||||
func TestAnAddressHasOneCacheEntryHoweverItIsWritten(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t, ptrReply(300, testName))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
lookupOK(t, cache, mappedAddress)
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
if got := resolver.requestCount(); got != 1 {
|
|
||||||
t.Errorf("%d requests, want 1", got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestCallerCannotChangeACachedAnswer changes the list each lookup returned, as
|
|
||||||
// the server does when it removes the trailing dots. The second lookup is the
|
|
||||||
// first answered from the cache. The third shows that the cached list did not
|
|
||||||
// change with the list the second returned.
|
|
||||||
func TestCallerCannotChangeACachedAnswer(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t, ptrReply(300, testName))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
for range 3 {
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("names %v, want %s", names, testName)
|
|
||||||
}
|
|
||||||
|
|
||||||
names[0] = "changed"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestTTLIsClamped(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
ttl uint32
|
|
||||||
want time.Duration
|
|
||||||
}{
|
|
||||||
{"below the minimum", 1, time.Minute},
|
|
||||||
{"zero", 0, time.Minute},
|
|
||||||
{"at the minimum", 60, time.Minute},
|
|
||||||
{"in range", 300, 300 * time.Second},
|
|
||||||
{"at the maximum", 86400, 24 * time.Hour},
|
|
||||||
{"above the maximum", 604800, 24 * time.Hour},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
resolver := newFakeResolver(t, ptrReply(tt.ttl, testName))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
got := cache.entries[testAddress].expires.Sub(fixedTime())
|
|
||||||
if got != tt.want {
|
|
||||||
t.Errorf("%s: kept for %s, want %s", tt.name, got, tt.want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSmallestTTLOfTheChainCounts answers with a CNAME record and two PTR
|
|
||||||
// records, whose TTLs all differ.
|
|
||||||
func TestSmallestTTLOfTheChainCounts(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
reply := func(w http.ResponseWriter, _ *http.Request, query *dns.Msg) {
|
|
||||||
owner := query.Question[0].Name
|
|
||||||
answer := new(dns.Msg)
|
|
||||||
answer.Answer = []dns.RR{
|
|
||||||
&dns.CNAME{
|
|
||||||
Hdr: dns.RR_Header{
|
|
||||||
Name: owner, Rrtype: dns.TypeCNAME, Class: dns.ClassINET, Ttl: 200,
|
|
||||||
},
|
|
||||||
Target: classlessOwner,
|
|
||||||
},
|
|
||||||
&dns.PTR{
|
|
||||||
Hdr: dns.RR_Header{
|
|
||||||
Name: classlessOwner, Rrtype: dns.TypePTR,
|
|
||||||
Class: dns.ClassINET, Ttl: 900,
|
|
||||||
},
|
|
||||||
Ptr: testName,
|
|
||||||
},
|
|
||||||
&dns.PTR{
|
|
||||||
Hdr: dns.RR_Header{
|
|
||||||
Name: classlessOwner, Rrtype: dns.TypePTR,
|
|
||||||
Class: dns.ClassINET, Ttl: 600,
|
|
||||||
},
|
|
||||||
Ptr: otherName,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
writeReply(w, query, answer)
|
|
||||||
}
|
|
||||||
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), newFakeResolver(t, reply))
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
if !slices.Equal(names, []string{testName, otherName}) {
|
|
||||||
t.Errorf("names %v, want both PTR names", names)
|
|
||||||
}
|
|
||||||
|
|
||||||
got := cache.entries[testAddress].expires.Sub(fixedTime())
|
|
||||||
if got != 200*time.Second {
|
|
||||||
t.Errorf("kept for %s, want 200s", got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestNegativeTTL(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
rcode int
|
|
||||||
soaTTL uint32
|
|
||||||
minimum uint32
|
|
||||||
want time.Duration
|
|
||||||
}{
|
|
||||||
{"name does not exist", dns.RcodeNameError, 3600, 300, 300 * time.Second},
|
|
||||||
{"no PTR record", dns.RcodeSuccess, 3600, 300, 300 * time.Second},
|
|
||||||
{"limited to an hour", dns.RcodeNameError, 86400, 7200, time.Hour},
|
|
||||||
{"SOA's own TTL is smaller", dns.RcodeNameError, 100, 300, 100 * time.Second},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
resolver := newFakeResolver(t, soaReply(tt.rcode, tt.soaTTL, tt.minimum))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
if len(names) != 0 {
|
|
||||||
t.Errorf("%s: names %v, want none", tt.name, names)
|
|
||||||
}
|
|
||||||
|
|
||||||
got := cache.entries[testAddress].expires.Sub(fixedTime())
|
|
||||||
if got != tt.want {
|
|
||||||
t.Errorf("%s: kept for %s, want %s", tt.name, got, tt.want)
|
|
||||||
}
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
if count := resolver.requestCount(); count != 1 {
|
|
||||||
t.Errorf("%s: %d requests, want 1", tt.name, count)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestAnswerWithoutNamesOrSOAIsNotCached has nothing to say how long the
|
|
||||||
// answer may be kept.
|
|
||||||
func TestAnswerWithoutNamesOrSOAIsNotCached(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t, rcodeReply(dns.RcodeNameError))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
if got := resolver.requestCount(); got != 2 {
|
|
||||||
t.Errorf("%d requests, want 2", got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFailedQueryIsNotCached has the only resolver fail, and then recover.
|
|
||||||
func TestFailedQueryIsNotCached(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t, rcodeReply(dns.RcodeServerFailure))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
_, err := cache.LookupAddr(t.Context(), testAddress)
|
|
||||||
if !errors.Is(err, errBadResponse) {
|
|
||||||
t.Fatalf("error %v, want %v", err, errBadResponse)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(cache.entries)+len(cache.pending) != 0 {
|
|
||||||
t.Error("a failed query left an entry in the cache")
|
|
||||||
}
|
|
||||||
|
|
||||||
resolver.setReply(ptrReply(300, testName))
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("names %v, want %s", names, testName)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamesThatAreNotHostNamesAreDropped answers with a name that holds markup
|
|
||||||
// beside a good one, as a PTR record can.
|
|
||||||
func TestNamesThatAreNotHostNamesAreDropped(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t,
|
|
||||||
ptrReply(300, "bad<script>.example.com.", testName, "."))
|
|
||||||
cache, _ := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, testAddress)
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("names %v, want only %s", names, testName)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestCacheSurvivesARestart stops a cache that holds a positive and a negative
|
|
||||||
// answer, and starts a new one on the reopened file. It makes no request.
|
|
||||||
func TestCacheSurvivesARestart(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
dir := t.TempDir()
|
|
||||||
db := openTestDB(t, dir)
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t,
|
|
||||||
func(w http.ResponseWriter, r *http.Request, query *dns.Msg) {
|
|
||||||
if query.Question[0].Name == testReverse {
|
|
||||||
ptrReply(300, testName, otherName)(w, r, query)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
soaReply(dns.RcodeNameError, 3600, 300)(w, r, query)
|
|
||||||
})
|
|
||||||
|
|
||||||
cache, _ := newTestCache(t, db, resolver)
|
|
||||||
|
|
||||||
err := cache.Start(t.Context())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Start: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
lookupOK(t, cache, otherAddress)
|
|
||||||
cache.Stop(t.Context())
|
|
||||||
|
|
||||||
err = db.Close()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("closing the database: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
restarted, _ := newTestCache(t, openTestDB(t, dir), resolver)
|
|
||||||
|
|
||||||
err = restarted.Start(t.Context())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Start after the restart: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
defer restarted.Stop(t.Context())
|
|
||||||
|
|
||||||
requestsBefore := resolver.requestCount()
|
|
||||||
|
|
||||||
names := lookupOK(t, restarted, testAddress)
|
|
||||||
if !slices.Equal(names, []string{testName, otherName}) {
|
|
||||||
t.Errorf("names after restart %v", names)
|
|
||||||
}
|
|
||||||
|
|
||||||
if names := lookupOK(t, restarted, otherAddress); len(names) != 0 {
|
|
||||||
t.Errorf("names of the address with none after restart: %v", names)
|
|
||||||
}
|
|
||||||
|
|
||||||
if got := resolver.requestCount(); got != requestsBefore {
|
|
||||||
t.Errorf("%d requests after the restart, want none",
|
|
||||||
got-requestsBefore)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSweepDeletesExpiredRows also checks that a restart does not load a row
|
|
||||||
// that has expired since it was written.
|
|
||||||
func TestSweepDeletesExpiredRows(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
db := openTestDB(t, t.TempDir())
|
|
||||||
resolver := newFakeResolver(t,
|
|
||||||
func(w http.ResponseWriter, r *http.Request, query *dns.Msg) {
|
|
||||||
ttl := uint32(300)
|
|
||||||
if query.Question[0].Name == testReverse {
|
|
||||||
ttl = 60
|
|
||||||
}
|
|
||||||
|
|
||||||
ptrReply(ttl, testName)(w, r, query)
|
|
||||||
})
|
|
||||||
cache, clock := newTestCache(t, db, resolver)
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
lookupOK(t, cache, otherAddress)
|
|
||||||
cache.flush(t.Context())
|
|
||||||
|
|
||||||
if got := countRows(t, db); got != 2 {
|
|
||||||
t.Fatalf("%d rows before the sweep, want 2", got)
|
|
||||||
}
|
|
||||||
|
|
||||||
clock.advance(90 * time.Second)
|
|
||||||
|
|
||||||
loaded, loadedClock := newTestCache(t, db, resolver)
|
|
||||||
loadedClock.advance(90 * time.Second)
|
|
||||||
|
|
||||||
err := loaded.load(t.Context())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("load: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(loaded.entries) != 1 {
|
|
||||||
t.Errorf("loaded %d entries, want the 1 not expired", len(loaded.entries))
|
|
||||||
}
|
|
||||||
|
|
||||||
cache.sweep(t.Context())
|
|
||||||
|
|
||||||
if got := countRows(t, db); got != 1 {
|
|
||||||
t.Errorf("%d rows after the sweep, want 1", got)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(cache.entries) != 1 {
|
|
||||||
t.Errorf("%d entries in memory after the sweep, want 1", len(cache.entries))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFlushKeepsAnswersItCouldNotWrite makes one write fail by hiding the
|
|
||||||
// table. The next write, once the table is back, stores the answer.
|
|
||||||
func TestFlushKeepsAnswersItCouldNotWrite(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
db := openTestDB(t, t.TempDir())
|
|
||||||
resolver := newFakeResolver(t, ptrReply(300, testName))
|
|
||||||
cache, _ := newTestCache(t, db, resolver)
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
_, err := db.ExecContext(t.Context(), "ALTER TABLE hostnames RENAME TO hidden")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("renaming the table: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
cache.flush(t.Context())
|
|
||||||
|
|
||||||
_, err = db.ExecContext(t.Context(), "ALTER TABLE hidden RENAME TO hostnames")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("renaming the table back: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if got := countRows(t, db); got != 0 {
|
|
||||||
t.Fatalf("%d rows after the failed write, want 0", got)
|
|
||||||
}
|
|
||||||
|
|
||||||
cache.flush(t.Context())
|
|
||||||
|
|
||||||
if got := countRows(t, db); got != 1 {
|
|
||||||
t.Errorf("%d rows after the second write, want 1", got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNewAddressIsNotKeptAtTheLimit fills a cache that holds one address. A
|
|
||||||
// lookup of a second address still returns its answer, and the cache keeps
|
|
||||||
// nothing for it until the sweep has deleted the first address's expired
|
|
||||||
// answer.
|
|
||||||
func TestNewAddressIsNotKeptAtTheLimit(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
db := openTestDB(t, t.TempDir())
|
|
||||||
resolver := newFakeResolver(t, ptrReply(300, testName))
|
|
||||||
cache, clock := newTestCache(t, db, resolver)
|
|
||||||
cache.maxEntries = 1
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
names := lookupOK(t, cache, otherAddress)
|
|
||||||
if !slices.Equal(names, []string{testName}) {
|
|
||||||
t.Errorf("names %v at the limit, want %s", names, testName)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, inEntries := cache.entries[otherAddress]
|
|
||||||
_, inPending := cache.pending[otherAddress]
|
|
||||||
|
|
||||||
if inEntries || inPending {
|
|
||||||
t.Errorf("a new address was kept at the limit: in entries %t, in pending %t",
|
|
||||||
inEntries, inPending)
|
|
||||||
}
|
|
||||||
|
|
||||||
cache.flush(t.Context())
|
|
||||||
|
|
||||||
if got := countRows(t, db); got != 1 {
|
|
||||||
t.Errorf("%d rows at the limit, want 1", got)
|
|
||||||
}
|
|
||||||
|
|
||||||
clock.advance(300 * time.Second)
|
|
||||||
cache.sweep(t.Context())
|
|
||||||
lookupOK(t, cache, otherAddress)
|
|
||||||
|
|
||||||
_, inEntries = cache.entries[otherAddress]
|
|
||||||
if !inEntries {
|
|
||||||
t.Error("a new address was not kept after the sweep made room")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHeldAddressIsReplacedAtTheLimit lets the answer of the only address a
|
|
||||||
// cache holds expire. The new answer replaces it in memory and is queued for
|
|
||||||
// cache.db.
|
|
||||||
func TestHeldAddressIsReplacedAtTheLimit(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
resolver := newFakeResolver(t, ptrReply(300, testName))
|
|
||||||
cache, clock := newTestCache(t, openTestDB(t, t.TempDir()), resolver)
|
|
||||||
cache.maxEntries = 1
|
|
||||||
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
clock.advance(300 * time.Second)
|
|
||||||
resolver.setReply(ptrReply(300, otherName))
|
|
||||||
lookupOK(t, cache, testAddress)
|
|
||||||
|
|
||||||
want := []string{otherName}
|
|
||||||
|
|
||||||
if got := cache.entries[testAddress].names; !slices.Equal(got, want) {
|
|
||||||
t.Errorf("entries holds %v, want %v", got, want)
|
|
||||||
}
|
|
||||||
|
|
||||||
if got := cache.pending[testAddress].names; !slices.Equal(got, want) {
|
|
||||||
t.Errorf("pending holds %v, want %v", got, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,200 +0,0 @@
|
|||||||
package ptr
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"math"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/miekg/dns"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
// LookupTimeout limits one address's lookup, every resolver included.
|
|
||||||
LookupTimeout = 2 * time.Second
|
|
||||||
|
|
||||||
// minTTL and maxTTL limit how long PTR names are kept.
|
|
||||||
minTTL = time.Minute
|
|
||||||
maxTTL = 24 * time.Hour
|
|
||||||
|
|
||||||
// maxNegativeTTL limits how long an address with no PTR names is
|
|
||||||
// remembered as having none.
|
|
||||||
maxNegativeTTL = time.Hour
|
|
||||||
|
|
||||||
// dnsMessageType is the content type of RFC 8484 requests and answers.
|
|
||||||
dnsMessageType = "application/dns-message"
|
|
||||||
|
|
||||||
// maxMessageBytes is the largest possible DNS message.
|
|
||||||
maxMessageBytes = 65535
|
|
||||||
)
|
|
||||||
|
|
||||||
var (
|
|
||||||
errBadStatus = errors.New("unexpected HTTP status")
|
|
||||||
errBadResponse = errors.New("resolver reported an error")
|
|
||||||
)
|
|
||||||
|
|
||||||
// Resolvers returns the URLs of the public DNS-over-HTTPS resolvers, in the
|
|
||||||
// order they are tried.
|
|
||||||
func Resolvers() []string {
|
|
||||||
return []string{
|
|
||||||
"https://dns.google/dns-query",
|
|
||||||
"https://cloudflare-dns.com/dns-query",
|
|
||||||
"https://unfiltered.adguard-dns.com/dns-query",
|
|
||||||
"https://dns.quad9.net/dns-query",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// answer is a resolver's answer for one address.
|
|
||||||
type answer struct {
|
|
||||||
// names are the address's PTR names, nil when it has none.
|
|
||||||
names []string
|
|
||||||
|
|
||||||
// ttl is how long the answer may be kept. Zero means it is not kept.
|
|
||||||
ttl time.Duration
|
|
||||||
}
|
|
||||||
|
|
||||||
// lookup asks the resolvers in turn for the PTR names of the reverse name
|
|
||||||
// until one answers. It fails when none has answered within the time limit.
|
|
||||||
func (c *Cache) lookup(ctx context.Context, reverseName string) (answer, error) {
|
|
||||||
ctx, cancel := context.WithTimeout(ctx, c.timeLimit)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
query := new(dns.Msg).SetQuestion(reverseName, dns.TypePTR)
|
|
||||||
query.Id = 0 // RFC 8484 asks for 0, so that equal queries can be cached.
|
|
||||||
|
|
||||||
wire, err := query.Pack()
|
|
||||||
if err != nil {
|
|
||||||
return answer{}, fmt.Errorf("packing the query: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var failures []error
|
|
||||||
|
|
||||||
for i, resolver := range c.resolvers {
|
|
||||||
// Each resolver gets an equal share of the time left, so that one
|
|
||||||
// that never answers leaves time for the next.
|
|
||||||
deadline, _ := ctx.Deadline()
|
|
||||||
share := time.Until(deadline) / time.Duration(len(c.resolvers)-i)
|
|
||||||
|
|
||||||
result, err := c.ask(ctx, resolver, wire, share)
|
|
||||||
if err == nil {
|
|
||||||
return result, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
failures = append(failures, fmt.Errorf("%s: %w", resolver, err))
|
|
||||||
}
|
|
||||||
|
|
||||||
return answer{}, errors.Join(failures...)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ask sends the query to one resolver and waits at most timeout for its answer.
|
|
||||||
func (c *Cache) ask(
|
|
||||||
ctx context.Context, resolver string, query []byte, timeout time.Duration,
|
|
||||||
) (answer, error) {
|
|
||||||
ctx, cancel := context.WithTimeout(ctx, timeout)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
request, err := http.NewRequestWithContext(
|
|
||||||
ctx, http.MethodPost, resolver, bytes.NewReader(query))
|
|
||||||
if err != nil {
|
|
||||||
return answer{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
request.Header.Set("Content-Type", dnsMessageType)
|
|
||||||
request.Header.Set("Accept", dnsMessageType)
|
|
||||||
|
|
||||||
response, err := c.client.Do(request)
|
|
||||||
if err != nil {
|
|
||||||
return answer{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
defer func() { _ = response.Body.Close() }()
|
|
||||||
|
|
||||||
if response.StatusCode != http.StatusOK {
|
|
||||||
return answer{}, fmt.Errorf("%w: %d", errBadStatus, response.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
body, err := io.ReadAll(io.LimitReader(response.Body, maxMessageBytes))
|
|
||||||
if err != nil {
|
|
||||||
return answer{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
var reply dns.Msg
|
|
||||||
|
|
||||||
err = reply.Unpack(body)
|
|
||||||
if err != nil {
|
|
||||||
return answer{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return answerFrom(&reply)
|
|
||||||
}
|
|
||||||
|
|
||||||
// answerFrom reads a resolver's reply. A name that does not exist and a name
|
|
||||||
// with no PTR records are answers. Every other response code is a failure.
|
|
||||||
func answerFrom(reply *dns.Msg) (answer, error) {
|
|
||||||
if reply.Rcode != dns.RcodeSuccess && reply.Rcode != dns.RcodeNameError {
|
|
||||||
return answer{}, fmt.Errorf("%w: %s",
|
|
||||||
errBadResponse, dns.RcodeToString[reply.Rcode])
|
|
||||||
}
|
|
||||||
|
|
||||||
var names []string
|
|
||||||
|
|
||||||
seconds := uint32(math.MaxUint32)
|
|
||||||
|
|
||||||
for _, record := range reply.Answer {
|
|
||||||
// The chain to a PTR record can hold CNAME records, and the answer
|
|
||||||
// is good only as long as every record in it.
|
|
||||||
seconds = min(seconds, record.Header().Ttl)
|
|
||||||
|
|
||||||
ptr, ok := record.(*dns.PTR)
|
|
||||||
if ok && isHostName(ptr.Ptr) {
|
|
||||||
names = append(names, ptr.Ptr)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(names) == 0 {
|
|
||||||
return answer{ttl: negativeTTL(reply.Ns)}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
ttl := time.Duration(seconds) * time.Second
|
|
||||||
|
|
||||||
return answer{names: names, ttl: min(max(ttl, minTTL), maxTTL)}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// negativeTTL returns how long a reply with no PTR names may be kept: the
|
|
||||||
// smaller of the SOA record's own TTL and its minimum field (RFC 2308), at most
|
|
||||||
// maxNegativeTTL. Without a SOA record it is zero.
|
|
||||||
func negativeTTL(authority []dns.RR) time.Duration {
|
|
||||||
for _, record := range authority {
|
|
||||||
soa, ok := record.(*dns.SOA)
|
|
||||||
if ok {
|
|
||||||
ttl := time.Duration(min(soa.Hdr.Ttl, soa.Minttl)) * time.Second
|
|
||||||
|
|
||||||
return min(ttl, maxNegativeTTL)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
// isHostName reports whether name is a host name. A PTR record can hold any
|
|
||||||
// bytes, and the names go into web pages, so anything else is dropped, as
|
|
||||||
// net.Resolver.LookupAddr drops it. The root name is not a host name.
|
|
||||||
func isHostName(name string) bool {
|
|
||||||
_, ok := dns.IsDomainName(name)
|
|
||||||
|
|
||||||
return ok && name != "." && !strings.ContainsFunc(name, notInHostName)
|
|
||||||
}
|
|
||||||
|
|
||||||
// notInHostName reports whether r cannot be part of a host name.
|
|
||||||
func notInHostName(r rune) bool {
|
|
||||||
return !strings.ContainsRune(hostNameCharacters, r)
|
|
||||||
}
|
|
||||||
|
|
||||||
// hostNameCharacters are the characters of a host name, in presentation format.
|
|
||||||
const hostNameCharacters = "abcdefghijklmnopqrstuvwxyz" +
|
|
||||||
"ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_."
|
|
||||||
@@ -9,20 +9,17 @@ package routewatch
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"database/sql"
|
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/cachedb"
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/config"
|
"git.eeqj.de/sneak/routewatch/internal/config"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/database"
|
"git.eeqj.de/sneak/routewatch/internal/database"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/ptr"
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/server"
|
"git.eeqj.de/sneak/routewatch/internal/server"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
@@ -65,8 +62,6 @@ type RouteWatch struct {
|
|||||||
asnFetcher *ASNFetcher
|
asnFetcher *ASNFetcher
|
||||||
dbMaintainer *DBMaintainer
|
dbMaintainer *DBMaintainer
|
||||||
ipLists *iplist.IPLists
|
ipLists *iplist.IPLists
|
||||||
cacheDB *sql.DB
|
|
||||||
ptrCache *ptr.Cache
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ipListsDirName is the directory in the state directory that holds the IP
|
// ipListsDirName is the directory in the state directory that holds the IP
|
||||||
@@ -155,11 +150,6 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
|
|||||||
rw.ipLists = ipLists
|
rw.ipLists = ipLists
|
||||||
rw.server.SetIPLists(ipLists)
|
rw.server.SetIPLists(ipLists)
|
||||||
|
|
||||||
err = rw.startPTRCache(ctx)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Start HTTP server
|
// Start HTTP server
|
||||||
err = rw.server.Start(ctx)
|
err = rw.server.Start(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -178,8 +168,8 @@ func (rw *RouteWatch) Run(ctx context.Context) error {
|
|||||||
|
|
||||||
// Shutdown stops all services in order: the streamer, the handlers, which
|
// Shutdown stops all services in order: the streamer, the handlers, which
|
||||||
// write what they hold, the WHOIS fetcher, the database maintainer, the
|
// write what they hold, the WHOIS fetcher, the database maintainer, the
|
||||||
// IP list downloads, the HTTP server and the PTR cache, and last the database,
|
// IP list downloads and the HTTP server, and last the database, which it
|
||||||
// which it checkpoints and closes. Each stage waits at most until ctx ends, so
|
// checkpoints and closes. Each stage waits at most until ctx ends, so
|
||||||
// Shutdown returns soon after: a handler logs what it could not write by
|
// Shutdown returns soon after: a handler logs what it could not write by
|
||||||
// then, and the HTTP server closes the connections of requests still
|
// then, and the HTTP server closes the connections of requests still
|
||||||
// running.
|
// running.
|
||||||
@@ -239,8 +229,6 @@ func (rw *RouteWatch) Shutdown(ctx context.Context) {
|
|||||||
"error", err)
|
"error", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
rw.stopPTRCache(ctx)
|
|
||||||
|
|
||||||
rw.closeDatabase(ctx)
|
rw.closeDatabase(ctx)
|
||||||
|
|
||||||
// Log final metrics
|
// Log final metrics
|
||||||
@@ -254,55 +242,6 @@ func (rw *RouteWatch) Shutdown(ctx context.Context) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// startPTRCache opens cache.db, loads the PTR cache from it and gives the
|
|
||||||
// server the cache. The HTTP handlers read it without a lock, so it is set
|
|
||||||
// before they run.
|
|
||||||
func (rw *RouteWatch) startPTRCache(ctx context.Context) error {
|
|
||||||
cacheDB, err := cachedb.Open(ctx, rw.config.StateDir)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
ptrCache := ptr.New(ptr.Params{
|
|
||||||
DB: cacheDB,
|
|
||||||
Resolvers: ptr.Resolvers(),
|
|
||||||
Logger: rw.logger.Logger,
|
|
||||||
})
|
|
||||||
|
|
||||||
err = ptrCache.Start(ctx)
|
|
||||||
if err != nil {
|
|
||||||
_ = cacheDB.Close()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
rw.cacheDB = cacheDB
|
|
||||||
rw.ptrCache = ptrCache
|
|
||||||
rw.server.SetPTRResolver(ptrCache)
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// stopPTRCache stops the PTR cache, which writes the answers cache.db does not
|
|
||||||
// have yet, and closes cache.db, unless ctx has ended and a write may still be
|
|
||||||
// running.
|
|
||||||
func (rw *RouteWatch) stopPTRCache(ctx context.Context) {
|
|
||||||
if rw.ptrCache == nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
rw.ptrCache.Stop(ctx)
|
|
||||||
|
|
||||||
if ctx.Err() != nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
err := rw.cacheDB.Close()
|
|
||||||
if err != nil {
|
|
||||||
rw.logger.Error("Failed to close the cache database", "error", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// closeDatabase checkpoints and closes the database, unless ctx has ended: a
|
// closeDatabase checkpoints and closes the database, unless ctx has ended: a
|
||||||
// write may then still be running, and both would wait for it. SQLite rolls
|
// write may then still be running, and both would wait for it. SQLite rolls
|
||||||
// back a write cut off by the exit when the database next opens.
|
// back a write cut off by the exit when the database next opens.
|
||||||
|
|||||||
@@ -8,8 +8,6 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -785,74 +783,3 @@ func TestShutdownCheckpointsAndClosesTheDatabaseLast(t *testing.T) {
|
|||||||
t.Errorf("Shutdown made the calls %q, want %q", store.calls, want)
|
t.Errorf("Shutdown made the calls %q, want %q", store.calls, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestPTRCacheAnswersTheIPLookup starts the PTR cache and the HTTP server as
|
|
||||||
// Run does, and looks up a reserved address, which asks no resolver. Without
|
|
||||||
// the PTR cache the server has no PTR resolver, and every IP lookup answers
|
|
||||||
// 500.
|
|
||||||
func TestPTRCacheAnswersTheIPLookup(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
store := newMockStore()
|
|
||||||
log := logger.New()
|
|
||||||
port := freePort(t)
|
|
||||||
cfg := &config.Config{Port: port, StateDir: t.TempDir()}
|
|
||||||
s := streamer.New(log, metrics.New())
|
|
||||||
|
|
||||||
// A started server reads its routing table in the background, so it
|
|
||||||
// needs one.
|
|
||||||
rw := New(Dependencies{
|
|
||||||
DB: store,
|
|
||||||
Streamer: s,
|
|
||||||
Server: server.New(store, routetable.New(routetable.Params{}), s, log,
|
|
||||||
cfg),
|
|
||||||
Logger: log,
|
|
||||||
Config: cfg,
|
|
||||||
})
|
|
||||||
|
|
||||||
err := rw.startPTRCache(t.Context())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to start the PTR cache: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
defer rw.stopPTRCache(t.Context())
|
|
||||||
|
|
||||||
err = rw.server.Start(t.Context())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to start the HTTP server: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
defer func() { _ = rw.server.Stop(t.Context()) }()
|
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
|
|
||||||
"http://"+net.JoinHostPort("127.0.0.1", strconv.Itoa(port))+
|
|
||||||
"/api/v1/ip/10.0.0.1", nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to build the request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := http.DefaultClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("the IP lookup failed: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
defer func() { _ = resp.Body.Close() }()
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to read the answer: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
t.Fatalf("status %d, want %d; body %s", resp.StatusCode, http.StatusOK, body)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !strings.Contains(string(body), `"ptr":[]`) {
|
|
||||||
t.Errorf("answer %s, want an empty ptr", body)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = os.Stat(filepath.Join(cfg.StateDir, "cache.db"))
|
|
||||||
if err != nil {
|
|
||||||
t.Errorf("cache.db is not in the state directory: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+16
-13
@@ -39,9 +39,12 @@ const (
|
|||||||
const (
|
const (
|
||||||
// maxLookupTargets is the most addresses and host names an IP lookup may
|
// maxLookupTargets is the most addresses and host names an IP lookup may
|
||||||
// list, and the most addresses it may have once the names are resolved:
|
// list, and the most addresses it may have once the names are resolved:
|
||||||
// each address costs two database reads and a PTR lookup. Ten PTR lookups
|
// each address costs two database reads and a PTR lookup.
|
||||||
// that run to ptr.LookupTimeout fit in the request's 30-second limit.
|
|
||||||
maxLookupTargets = 10
|
maxLookupTargets = 10
|
||||||
|
|
||||||
|
// ptrLookupTimeout is each PTR lookup's own time limit, so that ten
|
||||||
|
// addresses whose PTR lookups never answer fit in the 30-second limit.
|
||||||
|
ptrLookupTimeout = 2 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
// The Cache-Control header of an IP lookup answer, such as
|
// The Cache-Control header of an IP lookup answer, such as
|
||||||
@@ -1029,14 +1032,16 @@ func ipLookupFlags(ip string, listings []iplist.Listing) IPLookupFlags {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// isBogon says whether the address ip is reserved, as package bogon tells.
|
// isBogon says whether the address ip is reserved. An IPv4 address written in
|
||||||
|
// IPv6 form counts as the IPv4 address it holds.
|
||||||
func isBogon(ip string) bool {
|
func isBogon(ip string) bool {
|
||||||
addr, err := netip.ParseAddr(ip)
|
addr, err := netip.ParseAddr(ip)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
_, reserved := bogon.Addr(addr)
|
addr = addr.Unmap()
|
||||||
|
_, reserved := bogon.Prefix(netip.PrefixFrom(addr, addr.BitLen()))
|
||||||
|
|
||||||
return reserved
|
return reserved
|
||||||
}
|
}
|
||||||
@@ -1215,17 +1220,15 @@ func publicAddresses(addrs []string) []string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// lookupPTRNames returns the PTR records of ip without their trailing dots. It
|
// lookupPTRNames returns the PTR records of ip without their trailing dots. It
|
||||||
// returns an empty list, never nil, when there are none and when the lookup
|
// returns an empty list, never nil, when there are none, when the lookup fails
|
||||||
// fails, so that an answer's ptr is [] rather than null.
|
// and when it runs past ptrLookupTimeout, so that an answer's ptr is [] rather
|
||||||
|
// than null.
|
||||||
func (s *Server) lookupPTRNames(ctx context.Context, ip string) []string {
|
func (s *Server) lookupPTRNames(ctx context.Context, ip string) []string {
|
||||||
ptrs, err := s.ptrResolver.LookupAddr(ctx, ip)
|
ctx, cancel := context.WithTimeout(ctx, ptrLookupTimeout)
|
||||||
if err != nil {
|
defer cancel()
|
||||||
s.logger.Warn("PTR lookup failed", "ip", ip, "error", err)
|
|
||||||
|
|
||||||
return []string{}
|
ptrs, err := s.resolver.LookupAddr(ctx, ip)
|
||||||
}
|
if err != nil || ptrs == nil {
|
||||||
|
|
||||||
if ptrs == nil {
|
|
||||||
return []string{}
|
return []string{}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -25,7 +25,6 @@ import (
|
|||||||
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
"git.eeqj.de/sneak/routewatch/internal/iplist"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/logger"
|
"git.eeqj.de/sneak/routewatch/internal/logger"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
"git.eeqj.de/sneak/routewatch/internal/metrics"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/ptr"
|
|
||||||
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
"git.eeqj.de/sneak/routewatch/internal/ristypes"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
"git.eeqj.de/sneak/routewatch/internal/routetable"
|
||||||
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
"git.eeqj.de/sneak/routewatch/internal/streamer"
|
||||||
@@ -558,11 +557,8 @@ func newServerWithTable(t *testing.T, table *routetable.Table) *Server {
|
|||||||
|
|
||||||
t.Cleanup(func() { _ = db.Close() })
|
t.Cleanup(func() { _ = db.Close() })
|
||||||
|
|
||||||
s := New(db, table, streamer.New(logger.New(), metrics.New()), logger.New(),
|
return New(db, table, streamer.New(logger.New(), metrics.New()), logger.New(),
|
||||||
&config.Config{})
|
&config.Config{})
|
||||||
useFakeResolver(s, &fakeResolver{})
|
|
||||||
|
|
||||||
return s
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestPrefixDetailHandlersParsePrefix checks that both prefix pages find a
|
// TestPrefixDetailHandlersParsePrefix checks that both prefix pages find a
|
||||||
@@ -1256,7 +1252,7 @@ func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
|
|||||||
|
|
||||||
s := New(failingIPLookupDB{}, newTestTable(),
|
s := New(failingIPLookupDB{}, newTestTable(),
|
||||||
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
path string
|
path string
|
||||||
@@ -1292,19 +1288,16 @@ func TestIPLookupAnswers500WhenDatabaseReadFails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// errNoSuchHost is what fakeResolver answers a host name lookup it has no
|
// errNoSuchHost is what fakeResolver answers a lookup it has no answer for
|
||||||
// addresses for with.
|
// with.
|
||||||
var errNoSuchHost = errors.New("no such host")
|
var errNoSuchHost = errors.New("no such host")
|
||||||
|
|
||||||
// fakeResolver answers a host name lookup with the addresses hosts lists for
|
// fakeResolver answers a host name lookup with the addresses hosts lists for
|
||||||
// the name. It answers a PTR lookup with the names ptrs lists for the address,
|
// the name, and a PTR lookup with no names or, when ptrNeverAnswers is set,
|
||||||
// and with no names and no error, as the real one does, when ptrs lists none.
|
// not at all: it waits until the lookup's context ends. hostLookups counts the
|
||||||
// When ptrNeverAnswers is set, a PTR lookup fails after ptr.LookupTimeout, as
|
// host name lookups.
|
||||||
// the real one does when no resolver answers. hostLookups counts the host name
|
|
||||||
// lookups.
|
|
||||||
type fakeResolver struct {
|
type fakeResolver struct {
|
||||||
hosts map[string][]string
|
hosts map[string][]string
|
||||||
ptrs map[string][]string
|
|
||||||
ptrNeverAnswers bool
|
ptrNeverAnswers bool
|
||||||
|
|
||||||
hostLookups atomic.Int64
|
hostLookups atomic.Int64
|
||||||
@@ -1321,23 +1314,14 @@ func (r *fakeResolver) LookupHost(_ context.Context, host string) ([]string, err
|
|||||||
return addrs, nil
|
return addrs, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *fakeResolver) LookupAddr(ctx context.Context, addr string) ([]string, error) {
|
func (r *fakeResolver) LookupAddr(ctx context.Context, _ string) ([]string, error) {
|
||||||
if r.ptrNeverAnswers {
|
if r.ptrNeverAnswers {
|
||||||
ctx, cancel := context.WithTimeout(ctx, ptr.LookupTimeout)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
<-ctx.Done()
|
<-ctx.Done()
|
||||||
|
|
||||||
return nil, ctx.Err()
|
return nil, ctx.Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
return r.ptrs[addr], nil
|
return nil, errNoSuchHost
|
||||||
}
|
|
||||||
|
|
||||||
// useFakeResolver makes r answer the server's host name and PTR lookups.
|
|
||||||
func useFakeResolver(s *Server, r *fakeResolver) {
|
|
||||||
s.resolver = r
|
|
||||||
s.ptrResolver = r
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// countingIPLookupDB embeds database.Store (left nil) and answers the IP
|
// countingIPLookupDB embeds database.Store (left nil) and answers the IP
|
||||||
@@ -1408,7 +1392,7 @@ func TestIPLookupRefusesTooManyAddresses(t *testing.T) {
|
|||||||
s := New(db, newTestTable(), streamer.New(logger.New(), metrics.New()),
|
s := New(db, newTestTable(), streamer.New(logger.New(), metrics.New()),
|
||||||
logger.New(), &config.Config{})
|
logger.New(), &config.Config{})
|
||||||
resolver := &fakeResolver{hosts: hosts}
|
resolver := &fakeResolver{hosts: hosts}
|
||||||
useFakeResolver(s, resolver)
|
s.resolver = resolver
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
s.router.ServeHTTP(rec, httptest.NewRequestWithContext(
|
||||||
@@ -1443,19 +1427,18 @@ const addrInTestPrefixV4 = "192.0.2.1"
|
|||||||
|
|
||||||
// TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer looks up a routed address
|
// TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer looks up a routed address
|
||||||
// with a resolver that never answers its PTR lookup. Once the PTR lookup's own
|
// with a resolver that never answers its PTR lookup. Once the PTR lookup's own
|
||||||
// limit passes, the lookup answers 200 with the route and an empty ptr. Before
|
// limit passes, the lookup answers 200 with the route. Before the fix the PTR
|
||||||
// the fix the PTR lookup ran until the request's time limit, and the lookup
|
// lookup ran until the request's time limit, and the lookup answered 504
|
||||||
// answered 504 without the route
|
// without the route (https://git.eeqj.de/sneak/routewatch/issues/110).
|
||||||
// (https://git.eeqj.de/sneak/routewatch/issues/110).
|
|
||||||
func TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer(t *testing.T) {
|
func TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{ptrNeverAnswers: true})
|
s.resolver = &fakeResolver{ptrNeverAnswers: true}
|
||||||
|
|
||||||
// The request's time limit passes well after the PTR lookup's, so that a
|
// The request's time limit passes well after the PTR lookup's, so that a
|
||||||
// stall on a busy host does not use up the difference.
|
// stall on a busy host does not use up the difference.
|
||||||
const requestTimeLimit = ptr.LookupTimeout + 15*time.Second
|
const requestTimeLimit = ptrLookupTimeout + 15*time.Second
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(t.Context(), requestTimeLimit)
|
ctx, cancel := context.WithTimeout(t.Context(), requestTimeLimit)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
@@ -1481,10 +1464,6 @@ func TestIPLookupKeepsRouteWhenPTRLookupDoesNotAnswer(t *testing.T) {
|
|||||||
*body.Data.Results[0].Netblock != testPrefixV4 {
|
*body.Data.Results[0].Netblock != testPrefixV4 {
|
||||||
t.Errorf("answer %s, want one result in %s", rec.Body, testPrefixV4)
|
t.Errorf("answer %s, want one result in %s", rec.Body, testPrefixV4)
|
||||||
}
|
}
|
||||||
|
|
||||||
if !strings.Contains(rec.Body.String(), `"ptr":[]`) {
|
|
||||||
t.Errorf("answer %s, want an empty ptr list", rec.Body)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// addrInTestPrefixV6 is an address in testPrefixV6, which
|
// addrInTestPrefixV6 is an address in testPrefixV6, which
|
||||||
@@ -1562,7 +1541,7 @@ func TestIPLookupFlags(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
s.SetIPLists(fakeIPLists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
@@ -1675,7 +1654,7 @@ func TestIPLookupReportsIPLists(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
s.SetIPLists(fakeIPLists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
@@ -1752,7 +1731,7 @@ func TestIPLookupReportsIPListsOfAnAddressWithNoRoute(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
s.SetIPLists(fakeIPLists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
// routeFields are the fields of a result that come from the address's
|
// routeFields are the fields of a result that come from the address's
|
||||||
@@ -1830,9 +1809,9 @@ func TestIPLookupResultsHaveTheSameFields(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{hosts: map[string][]string{
|
s.resolver = &fakeResolver{hosts: map[string][]string{
|
||||||
twoAddressHostName: {addrInTestPrefixV4, listedUnroutedAddr},
|
twoAddressHostName: {addrInTestPrefixV4, listedUnroutedAddr},
|
||||||
}})
|
}}
|
||||||
s.SetIPLists(fakeIPLists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
// want is the fields of the first result.
|
// want is the fields of the first result.
|
||||||
@@ -1865,7 +1844,7 @@ func TestIPLookupOfAnAddressWithoutPTRNamesOrStoredNetblock(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
|
|
||||||
path := "/api/v1/ip/" + addrInTestPrefixV4
|
path := "/api/v1/ip/" + addrInTestPrefixV4
|
||||||
|
|
||||||
@@ -1887,32 +1866,6 @@ func TestIPLookupOfAnAddressWithoutPTRNamesOrStoredNetblock(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestIPLookupGivesPTRNamesWithoutTrailingDots looks up an address whose PTR
|
|
||||||
// name the resolver gives fully qualified. The result's ptr holds the name
|
|
||||||
// without its trailing dot.
|
|
||||||
func TestIPLookupGivesPTRNamesWithoutTrailingDots(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
|
||||||
useFakeResolver(s, &fakeResolver{ptrs: map[string][]string{
|
|
||||||
addrInTestPrefixV4: {"host.example.com."},
|
|
||||||
}})
|
|
||||||
|
|
||||||
path := "/api/v1/ip/" + addrInTestPrefixV4
|
|
||||||
|
|
||||||
results := resultFields(t, get(t, s, path).Body.Bytes())
|
|
||||||
if len(results) != 1 {
|
|
||||||
t.Fatalf("%s: %d results, want 1", path, len(results))
|
|
||||||
}
|
|
||||||
|
|
||||||
got := string(results[0]["ptr"])
|
|
||||||
|
|
||||||
want := `["host.example.com"]`
|
|
||||||
if got != want {
|
|
||||||
t.Errorf("%s: ptr is %s, want %s", path, got, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
|
// TestIPLookupAnswersMayBeCached checks the Cache-Control header of the IP
|
||||||
// lookup answers. Caches may keep them for the IP lists' shortest interval,
|
// lookup answers. Caches may keep them for the IP lists' shortest interval,
|
||||||
// and only the caller's own browser may keep the answer about the caller's
|
// and only the caller's own browser may keep the answer about the caller's
|
||||||
@@ -1921,7 +1874,7 @@ func TestIPLookupAnswersMayBeCached(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
s.SetIPLists(fakeIPLists{})
|
s.SetIPLists(fakeIPLists{})
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
@@ -1972,10 +1925,10 @@ func TestIPLookupHidesPrivateAddresses(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{hosts: map[string][]string{
|
s.resolver = &fakeResolver{hosts: map[string][]string{
|
||||||
internalHostName: {internalHostAddr},
|
internalHostName: {internalHostAddr},
|
||||||
"mixed.test": {internalHostAddr, addrInTestPrefixV4},
|
"mixed.test": {internalHostAddr, addrInTestPrefixV4},
|
||||||
}})
|
}}
|
||||||
|
|
||||||
notPublic := internalHostName + ": " + hostNotPublicMessage
|
notPublic := internalHostName + ": " + hostNotPublicMessage
|
||||||
|
|
||||||
|
|||||||
@@ -85,7 +85,7 @@ func TestIPLookupGivesTheOriginMostPeersCarry(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := newServerWithTable(t, newTwoOriginTable())
|
s := newServerWithTable(t, newTwoOriginTable())
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
|
|
||||||
firstSeen := testRouteTime(-1)
|
firstSeen := testRouteTime(-1)
|
||||||
|
|
||||||
|
|||||||
@@ -115,7 +115,7 @@ func TestJSONAnswersPutNoUnitInAString(t *testing.T) {
|
|||||||
`^([0-9]+(\.[0-9]+)? ?(ns|µs|ms|s|m|h|B|kB|MB|GB|TB))+$`)
|
`^([0-9]+(\.[0-9]+)? ?(ns|µs|ms|s|m|h|B|kB|MB|GB|TB))+$`)
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
s.SetASNFetcher(fixedIntervalASNFetcher{interval: time.Second})
|
s.SetASNFetcher(fixedIntervalASNFetcher{interval: time.Second})
|
||||||
|
|
||||||
paths := []string{
|
paths := []string{
|
||||||
@@ -179,7 +179,7 @@ func TestJSONAnswersCarryMeta(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
s := newServerWithTestRoutes(t)
|
s := newServerWithTestRoutes(t)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
s.streamer.GetMetricsTracker().SetConnected(true)
|
s.streamer.GetMetricsTracker().SetConnected(true)
|
||||||
s.router.Get(writesNothingPath, JSONValidationMiddleware(http.HandlerFunc(
|
s.router.Get(writesNothingPath, JSONValidationMiddleware(http.HandlerFunc(
|
||||||
func(http.ResponseWriter, *http.Request) {})).ServeHTTP)
|
func(http.ResponseWriter, *http.Request) {})).ServeHTTP)
|
||||||
@@ -188,7 +188,7 @@ func TestJSONAnswersCarryMeta(t *testing.T) {
|
|||||||
|
|
||||||
failing := New(failingDB{}, newTestTable(),
|
failing := New(failingDB{}, newTestTable(),
|
||||||
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
streamer.New(logger.New(), metrics.New()), logger.New(), &config.Config{})
|
||||||
useFakeResolver(failing, &fakeResolver{})
|
failing.resolver = &fakeResolver{}
|
||||||
|
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
method string
|
method string
|
||||||
@@ -521,7 +521,7 @@ func newServerWithConfig(
|
|||||||
|
|
||||||
s := New(db, newTestTable(), streamer.New(logger.New(), metrics.New()),
|
s := New(db, newTestTable(), streamer.New(logger.New(), metrics.New()),
|
||||||
&logger.Logger{Logger: slog.New(slog.NewJSONHandler(requestLog, nil))}, cfg)
|
&logger.Logger{Logger: slog.New(slog.NewJSONHandler(requestLog, nil))}, cfg)
|
||||||
useFakeResolver(s, &fakeResolver{})
|
s.resolver = &fakeResolver{}
|
||||||
|
|
||||||
return s
|
return s
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -52,14 +52,10 @@ type IPLists interface {
|
|||||||
ShortestInterval() time.Duration
|
ShortestInterval() time.Duration
|
||||||
}
|
}
|
||||||
|
|
||||||
// dnsResolver looks up the addresses of host names in DNS, as *net.Resolver
|
// dnsResolver looks up the addresses of host names and the names of addresses
|
||||||
// does.
|
// in DNS, as *net.Resolver does.
|
||||||
type dnsResolver interface {
|
type dnsResolver interface {
|
||||||
LookupHost(ctx context.Context, host string) ([]string, error)
|
LookupHost(ctx context.Context, host string) ([]string, error)
|
||||||
}
|
|
||||||
|
|
||||||
// PTRResolver looks up the PTR names of addresses, as *ptr.Cache does.
|
|
||||||
type PTRResolver interface {
|
|
||||||
LookupAddr(ctx context.Context, addr string) ([]string, error)
|
LookupAddr(ctx context.Context, addr string) ([]string, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -79,13 +75,10 @@ type Server struct {
|
|||||||
// RealIPMiddleware believes
|
// RealIPMiddleware believes
|
||||||
trustedProxies []netip.Prefix
|
trustedProxies []netip.Prefix
|
||||||
|
|
||||||
// resolver looks up the IP lookup's host names: net.DefaultResolver, or a
|
// resolver makes the IP lookup's DNS lookups: net.DefaultResolver, or a
|
||||||
// fake in tests
|
// fake in tests
|
||||||
resolver dnsResolver
|
resolver dnsResolver
|
||||||
|
|
||||||
// ptrResolver looks up the IP lookup's PTR names. SetPTRResolver sets it.
|
|
||||||
ptrResolver PTRResolver
|
|
||||||
|
|
||||||
// cards is the copy of the home page's cards that the pages serve, nil
|
// cards is the copy of the home page's cards that the pages serve, nil
|
||||||
// until the server first works them out
|
// until the server first works them out
|
||||||
cards atomic.Pointer[homeCards]
|
cards atomic.Pointer[homeCards]
|
||||||
@@ -203,12 +196,6 @@ func (s *Server) Stop(ctx context.Context) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// SetPTRResolver sets what looks up the IP lookup's PTR names. Call it before
|
|
||||||
// Start: the handlers read it without a lock.
|
|
||||||
func (s *Server) SetPTRResolver(resolver PTRResolver) {
|
|
||||||
s.ptrResolver = resolver
|
|
||||||
}
|
|
||||||
|
|
||||||
// SetASNFetcher sets the ASN WHOIS fetcher for on-demand lookups. Call it
|
// SetASNFetcher sets the ASN WHOIS fetcher for on-demand lookups. Call it
|
||||||
// before Start: the handlers read the fetcher without a lock.
|
// before Start: the handlers read the fetcher without a lock.
|
||||||
func (s *Server) SetASNFetcher(fetcher ASNFetcher) {
|
func (s *Server) SetASNFetcher(fetcher ASNFetcher) {
|
||||||
|
|||||||
Reference in new issue
Block a user