Compare commits
3
Commits
d0d997d88e
...
a389579d7a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a389579d7a | ||
|
|
6024aee2c4 | ||
|
|
6de44ac054 |
No files matched your search
+1
-1
@@ -105,7 +105,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ARG CACHEBUST
|
||||
|
||||
# Create non-root user
|
||||
RUN useradd -r -u 1000 -m routewatch
|
||||
RUN useradd -u 1000 -m routewatch
|
||||
|
||||
RUN mkdir -p /var/lib/berlin.sneak.app.routewatch && chown routewatch:routewatch /var/lib/berlin.sneak.app.routewatch
|
||||
|
||||
|
||||
@@ -143,9 +143,9 @@ and origin AS can have two kept routes, one or none.
|
||||
AS path, a next hop and a time. The next hop is the router the peer sends the
|
||||
prefix's traffic to. RouteWatch records the peer itself as the next hop. The
|
||||
page also lists the prefix's origin changes, newest first. An origin change is
|
||||
an AS starting to announce the prefix, when its first peer carries it, or
|
||||
stopping, when its last peer no longer does. The page shows at most the newest
|
||||
5,000. Each row shows:
|
||||
recorded when the first peer starts to carry the prefix with an origin AS, or
|
||||
when the last peer stops. The page shows at most the newest 5,000. Each row
|
||||
shows:
|
||||
- the time
|
||||
- announced or withdrawn
|
||||
- the origin AS
|
||||
@@ -502,11 +502,13 @@ How RouteWatch keeps the answers:
|
||||
there.
|
||||
- It writes new answers every 5 seconds to `cache.db` in the state directory
|
||||
(see Configuration), and loads them from there at startup.
|
||||
- An answer expires after the TTL of its records, the shortest one when there
|
||||
are several. RouteWatch keeps it for at least 60 seconds and at most 24 hours.
|
||||
- When an address has no PTR names, that answer expires after the SOA record's
|
||||
minimum field, or the record's own TTL when that is shorter. RouteWatch keeps
|
||||
it for at most 1 hour.
|
||||
- Each DNS record carries a TTL, the time for which it may be kept. An answer
|
||||
expires after the shortest TTL of its records. RouteWatch keeps it for at
|
||||
least 60 seconds and at most 24 hours.
|
||||
- When an address has no PTR names, the answer's SOA record says how long that
|
||||
answer may be kept. The answer expires after the SOA record's minimum field,
|
||||
or the record's own TTL when that is shorter. RouteWatch keeps it for at most
|
||||
1 hour.
|
||||
- An answer with no PTR names and no SOA record is not kept. Neither is one
|
||||
whose SOA record has a TTL or minimum field of 0.
|
||||
- A failed lookup is not kept, so the next lookup of the address asks again.
|
||||
@@ -520,7 +522,8 @@ An IP lookup finds where each address is, on a best-effort basis. It uses two
|
||||
sources:
|
||||
|
||||
- [geojs](https://www.geojs.io/) is asked first. One request carries every
|
||||
address of the lookup, and it has 1 second to answer.
|
||||
address of the lookup that the cache does not hold. geojs has 1 second to
|
||||
answer.
|
||||
- [DB-IP City Lite](https://db-ip.com/db/download/ip-to-city-lite) answers when
|
||||
geojs has no answer for the address. It is a database file that RouteWatch
|
||||
reads itself.
|
||||
@@ -702,8 +705,8 @@ What happens at each limit:
|
||||
allocating more.
|
||||
- The SQLite hard heap limit: a database statement fails with an out-of-memory
|
||||
error, which the handler logs. The prefix handler writes the routing table's
|
||||
changes again in a later write. Everything else in the failed write is
|
||||
dropped. The process keeps running.
|
||||
changes and its origin changes again in a later write. Everything else in the
|
||||
failed write is dropped. The process keeps running.
|
||||
- The handler queues: the handlers are the four parts that take the feed's
|
||||
messages (see Design). The prefix handler's queue holds at most 100,000
|
||||
messages, about 100 MiB. Each of the other three holds at most 20,000. When a
|
||||
@@ -1070,8 +1073,8 @@ calls one of them. The scripts:
|
||||
- `script/projectname`: print the project name, `routewatch`, which
|
||||
`script/docker` uses as the image tag.
|
||||
- `script/test`: build the `test` phase of the `Dockerfile` alone. It runs
|
||||
`go test -short -count=1 -timeout 90s -race -cover ./...`, and reruns it with
|
||||
`-v` on failure.
|
||||
`go test -short -count=1 -timeout 90s -race -cover ./...`. On failure it runs
|
||||
the tests again with `-v` and without `-cover`.
|
||||
- `script/lint`: build the `lint` phase of the `Dockerfile` alone. It runs
|
||||
`golangci-lint` v2.14.0 with `.golangci.yml`.
|
||||
- `script/fmt`: format the Go code with `go fmt` and the markdown with
|
||||
|
||||
@@ -22,37 +22,51 @@ formatting of the Go code and the markdown. Then it builds the image.
|
||||
The image sets memory ceilings for a 5 GiB container (README "Memory"). README
|
||||
"Running under upaas" says how to run it.
|
||||
|
||||
A five-day run of `1c65656` on the live feed, from 2026-10-27 to 2026-11-01,
|
||||
found three problems:
|
||||
Five runs of `next` on the live feed have ended since 2026-10-27, from three
|
||||
hours to five days long. The runs found the handlers dropping updates for five
|
||||
causes, and the WHOIS fetcher asking a registry that had refused it. Each is
|
||||
fixed on `next`, and its Completed Steps entry below names its issue. A 21-hour
|
||||
run of `742e759`, which has every fix but one, dropped no update except after
|
||||
the feed stalled. That cause, https://git.eeqj.de/sneak/routewatch/issues/229,
|
||||
is fixed on `next`.
|
||||
|
||||
- The prefix handler dropped updates while a peer's session went down
|
||||
(https://git.eeqj.de/sneak/routewatch/issues/216).
|
||||
- The AS handler dropped updates when the feed burst
|
||||
(https://git.eeqj.de/sneak/routewatch/issues/217).
|
||||
- The WHOIS fetcher kept asking a registry that had refused it
|
||||
(https://git.eeqj.de/sneak/routewatch/issues/218).
|
||||
|
||||
All three are fixed on `next`. So is
|
||||
https://git.eeqj.de/sneak/routewatch/issues/222, which the third fix brought up.
|
||||
A run of `26d939a` on the live feed showed the prefix handler still dropping
|
||||
updates while the home page's cards were worked out.
|
||||
https://git.eeqj.de/sneak/routewatch/issues/226 fixed that on `next`.
|
||||
https://git.eeqj.de/sneak/routewatch/issues/225, also on `next`, takes the peers
|
||||
whose sessions go down together out of the table in one walk. A 24-hour run of
|
||||
`9bd0e91` on the live feed is checking whether any handler still drops updates.
|
||||
A 24-hour run of `e58c663` on the live feed began at 09:19 UTC on 2026-11-10
|
||||
(https://git.eeqj.de/sneak/routewatch/issues/232). It is to confirm that fix.
|
||||
Next Step lists what its outcome reports. The seven IP lookup additions of
|
||||
https://git.eeqj.de/sneak/routewatch/issues/203 landed after it began, so it
|
||||
does not exercise them.
|
||||
|
||||
The order of the project's remaining work is kept on
|
||||
https://git.eeqj.de/sneak/routewatch/issues/86.
|
||||
|
||||
# Next Step
|
||||
|
||||
Post the outcome of the 24-hour run of `9bd0e91` on
|
||||
https://git.eeqj.de/sneak/routewatch/issues/226: whether any handler still drops
|
||||
updates, and the process's peak memory against 5 GiB. File an issue for each
|
||||
problem the run finds.
|
||||
Post the outcome of the 24-hour run of `e58c663` on
|
||||
https://git.eeqj.de/sneak/routewatch/issues/232. The run lasts at least 24
|
||||
hours. If the feed has not stalled or reconnected by then, the run goes on until
|
||||
it does, up to 72 hours in all. The outcome says:
|
||||
|
||||
- whether the feed stalled or reconnected
|
||||
- whether any handler dropped an update, and the most messages the prefix
|
||||
handler's queue held at once
|
||||
- the most messages the prefix handler held back at once
|
||||
- the size of the `origin_changes` table on disk
|
||||
- the process's peak memory against 5 GiB
|
||||
|
||||
File an issue for each problem the run finds.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-11-10: `README.md` and this file are read whole against the rules of
|
||||
https://git.eeqj.de/sneak/routewatch/issues/157, and every statement except
|
||||
measured figures is checked against the code on `next`. Six README sentences
|
||||
are rewritten. Four failed the prose rules, on origin changes, PTR lookups and
|
||||
geolocation. Two disagreed with the code, on a failed database write and
|
||||
`script/test`. Status and Next Step describe the live-feed run in progress
|
||||
(closes https://git.eeqj.de/sneak/routewatch/issues/251).
|
||||
- 2026-11-10: the image creates the `routewatch` user as a regular user with UID
|
||||
1000, so the build no longer warns about the system UID range (closes
|
||||
https://git.eeqj.de/sneak/routewatch/issues/248).
|
||||
- 2026-11-10: each result of the IP lookup has `geo`, where the address is. It
|
||||
comes from geojs, with DB-IP's City Lite database as the fallback. The geojs
|
||||
answers stay in memory and in `cache.db` for 7 days (closes
|
||||
@@ -60,16 +74,18 @@ problem the run finds.
|
||||
- 2026-11-10: the IP lookup checks the `icloud_private_relay` list of Apple's
|
||||
iCloud Private Relay networks, downloaded once a day. Each result has
|
||||
`flags.is_relay` (closes https://git.eeqj.de/sneak/routewatch/issues/237).
|
||||
- 2026-11-10: the IP lookup checks the `protonvpn` list of ProtonVPN's /24
|
||||
networks and the `protonvpn_entry` list of its servers' IPv4 and IPv6
|
||||
addresses, from a public mirror of Proton's server list (closes
|
||||
https://git.eeqj.de/sneak/routewatch/issues/236).
|
||||
- 2026-11-10: the IP lookup checks two ProtonVPN lists from a public mirror of
|
||||
Proton's server list. `protonvpn` holds the /24 networks of ProtonVPN's
|
||||
servers, and `protonvpn_entry` holds the servers' IPv4 and IPv6 addresses
|
||||
(closes https://git.eeqj.de/sneak/routewatch/issues/236).
|
||||
- 2026-11-10: the IP lookup finds PTR names over DNS-over-HTTPS and caches them
|
||||
for the record's TTL, in memory and in `cache.db` (closes
|
||||
in memory and in `cache.db` for the record's TTL. A TTL is the time for which
|
||||
a DNS record may be kept (closes
|
||||
https://git.eeqj.de/sneak/routewatch/issues/238).
|
||||
- 2026-11-10: the tests no longer fail when a busy host stalls for several
|
||||
seconds. Most such waits are now 15 seconds. The test of the stream's idle
|
||||
timeout runs on a fake clock (closes #240)
|
||||
seconds. Most waits for something that should happen are now 15 seconds. Tests
|
||||
that depend on a short idle timeout or a request limit run on a fake clock
|
||||
(closes https://git.eeqj.de/sneak/routewatch/issues/240)
|
||||
- 2026-11-10: the IP lookup checks the public VPN server lists of Mullvad,
|
||||
NordVPN, Private Internet Access and IVPN, and X4BNet's community list. Each
|
||||
result has `flags.is_vpn` and `vpn_providers` (closes
|
||||
|
||||
Reference in new issue
Block a user