Add best-effort geolocation to the IP lookup from geojs and DB-IP City Lite (closes #239)

Each result gains `geo`. One request to geojs carries every address, 1 second in all. On a timeout, an error or an answer with no country, city or coordinates, the lookup reads DB-IP City Lite, downloaded monthly into the state directory. geojs answers stay in memory (250,000, least recently used dropped) and in a new `cache.db` table for 7 days.

Judgement call: a reserved address goes to neither source, and its `geo` is null.
Judgement call: `updated` for DB-IP is when the copy was downloaded.
Deviation: the cache is SQLite, by sneak's words.

Model: opus-5-5

README.md

@@ -63,8 +63,8 @@ docker run --memory=5g --memory-swap=5g -p 8080:8080 \
     -v routewatch:/var/lib/berlin.sneak.app.routewatch routewatch
 ```
 
-The memory limit is explained in Memory. The volume holds the database and the
-copies of the IP lists.
+The memory limit is explained in Memory. The volume holds the database, the
+copies of the IP lists and the geolocation files.
 
 To run the binary on the host instead, build it with `make build`. The build
 needs Go and a C compiler, because the SQLite library is written in C:
@@ -294,6 +294,7 @@ address. Every entry has the same fields:
   an empty list when it is on none
 - `ip_lists`: the public IP lists the address is on (see IP lists), or an empty
   list when it is on none
+- `geo`: where the address is (see Geolocation), or null when no source answers
 
 Every other field is about the prefix in the routing table that holds the
 address. Each is null when no prefix in the routing table holds the address:
@@ -513,6 +514,72 @@ How RouteWatch keeps the answers:
 - It keeps answers for at most 250,000 addresses, and keeps no answer for a new
   address while it holds that many.
 
+## Geolocation
+
+An IP lookup finds where each address is, on a best-effort basis. It uses two
+sources:
+
+- [geojs](https://www.geojs.io/) is asked first. One request carries every
+  address of the lookup, and it has 1 second to answer.
+- [DB-IP City Lite](https://db-ip.com/db/download/ip-to-city-lite) answers when
+  geojs has no answer for the address. It is a database file that RouteWatch
+  reads itself.
+
+geojs has no answer when its request fails or runs out of time. An answer with
+no country, city or coordinates also counts as no answer, and RouteWatch does
+not cache it.
+
+The `geo` object of a result has these fields:
+
+- `country_code`: the two-letter country code
+- `country`: the country's name in English
+- `continent_code`: the two-letter continent code
+- `region`: the region or state
+- `city`: the city
+- `latitude` and `longitude`: the location in degrees
+- `accuracy_radius_km`: how far from that location the address may be, in km
+- `time_zone`: the time zone's name, such as `Europe/Berlin`
+- `source`: the source of the answer, `geojs` or `dbip`
+- `updated`: when the answer was fetched from geojs, or when the database copy
+  was downloaded
+
+A field the source does not know is null. DB-IP has no `accuracy_radius_km` and
+no `time_zone`, so both are null in its answers. `geo` itself is null when
+neither source answers. A reserved address, one whose `is_bogon` flag is true,
+goes to neither source, so its `geo` is null.
+
+What goes outside: geojs receives each address that the cache does not hold. For
+`/ip/` and the home page, that address is the visitor's own. A lookup answered
+from the cache sends nothing. DB-IP receives only the monthly request for its
+file.
+
+How RouteWatch keeps the answers of geojs:
+
+- It holds at most 250,000 addresses in memory, and drops the least recently
+  used first. That takes about 100 MB of the Go heap.
+- It writes new answers every 5 seconds to `cache.db` in the state directory.
+- It loads the newest 250,000 answers from `cache.db` at startup.
+- An address not in memory is read from `cache.db` before geojs is asked.
+- An answer expires 7 days after its fetch. Every 10 minutes RouteWatch deletes
+  expired answers from memory and from `cache.db`.
+
+Answers from the database are not cached, because reading the file is local.
+
+How RouteWatch keeps the database:
+
+- It keeps one copy in the state directory, named `dbip-city-lite.mmdb`.
+- It downloads the current month's file at the first start, and then once a
+  month. Until the first download finishes, only geojs answers.
+- A download fails when it takes over 10 minutes or unpacks to over 200 MiB.
+- A new file replaces the old copy only once it opens. After a failed download
+  RouteWatch keeps the old copy and tries again an hour later.
+- The file is about 130 MB. RouteWatch maps it into memory, outside the Go heap.
+  The kernel loads the parts that are read, and can drop them again when memory
+  is short.
+
+The home page's footer says "IP Geolocation by DB-IP" with a link to
+[db-ip.com](https://db-ip.com).
+
 ## Configuration
 
 RouteWatch reads its settings from environment variables. It takes no
@@ -564,7 +631,9 @@ are not read.
 The state directory holds:
 
 - `db.sqlite`: the database
-- `cache.db`: the answers of PTR lookups
+- `cache.db`: the answers of PTR lookups and of geojs
+- `dbip-city-lite.mmdb`: the copy of DB-IP's City Lite database (see
+  Geolocation)
 - `iplists`: the IP list copies
 
 It is at:
@@ -589,8 +658,9 @@ a memory limit of 5 GiB and swap disabled:
 docker run --memory=5g --memory-swap=5g ...
 ```
 
-The 5 GiB leaves room above the ceilings for spikes and for the kernel's cache
-of the database file.
+The 5 GiB leaves room above the ceilings for spikes, for the kernel's cache of
+the database file, and for the mapped copy of DB-IP's database (see
+Geolocation).
 
 The ceilings add up to 3.7 GiB:
 
@@ -733,6 +803,8 @@ The parts:
   address.
 - The PTR cache looks up the PTR names of addresses and keeps the answers (see
   PTR lookups).
+- The geolocation part finds where addresses are, keeps the answers of geojs and
+  downloads DB-IP's database (see Geolocation).
 - The database maintainer checkpoints the database every 5 seconds: it folds the
   write-ahead log, where SQLite first writes changes, into the database file. It
   reclaims free space every 10 minutes. Every hour it refreshes the statistics
@@ -842,7 +914,8 @@ On `SIGTERM` or `SIGINT` the daemon stops its parts in this order:
 4. the database maintainer
 5. the IP list downloads
 6. the HTTP server
-7. the PTR cache, which writes what it holds
+7. the geolocation part, which writes the geojs answers it holds
+8. the PTR cache, which writes what it holds
 
 Last, the daemon checkpoints and closes the database. It gives all this 8
 seconds and then exits with status 0. A stop that is still running after 9
@@ -865,6 +938,8 @@ What a stop can lose:
   SQLite rolls back any write the exit cut off.
 - When the 8 seconds run out, the PTR answers not yet in `cache.db` are lost.
   RouteWatch asks the resolvers for them again the next time they are needed.
+- When the 8 seconds run out, the geojs answers not yet in `cache.db` are lost.
+  RouteWatch asks geojs for them again the next time they are needed.
 
 ### Code layout
 
@@ -881,6 +956,7 @@ routewatch/
 │   ├── cachedb/            # cache.db, the caches of outside answers
 │   ├── config/             # Settings from the environment
 │   ├── database/           # SQLite storage
+│   ├── geo/                # Geolocation: geojs, DB-IP City Lite, the cache
 │   ├── iplist/             # Public IP lists: downloads and lookups
 │   ├── logger/             # Structured logging
 │   ├── metrics/            # Feed and route update figures
@@ -933,11 +1009,16 @@ announcements(id PRIMARY KEY, prefix_id, peer_asn, origin_asn, path,
 
 The indexes are in `internal/database/schema.sql`.
 
-`cache.db` is a separate file, with one table:
+`cache.db` is a separate file, with two tables:
 
 ```sql
 -- The answers of PTR lookups. names is empty for an address with no PTR names.
 hostnames(address PRIMARY KEY, names, expires_at)
+
+-- The answers of geojs. A part geojs does not know is NULL.
+geolocations(address PRIMARY KEY, country_code, country, continent_code,
+             region, city, latitude, longitude, accuracy_radius_km, time_zone,
+             fetched_at)
 ```
 
 ## Development