Keep the AS list when the asupdate generator fails (closes #173)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
make asupdate piped the generator into gzip, so the target saw gzip's exit status. A failed download replaced pkg/asinfo/asdata.json.gz with an empty file and the target still exited 0. The target now runs the new script/asupdate. It writes the generator's output to a temporary file and checks that the generator succeeded and wrote at least one AS. Only then does it compress the list and move it into place. Otherwise it leaves the file alone and exits non-zero. The generator reads as.csv from ipverse/as-metadata, the source's new name. The list it builds there is unchanged, so no new list is committed. Model: opus-5-5
This commit is contained in:
@@ -52,7 +52,6 @@ clean:
|
||||
run: build
|
||||
DEBUG=routewatch ./bin/routewatch 2>&1 | tee log.txt
|
||||
|
||||
# Rebuild the AS list built into RouteWatch from as.csv (needs the network).
|
||||
asupdate:
|
||||
@echo "Updating AS info data..."
|
||||
@go run cmd/asinfo-gen/main.go | gzip > pkg/asinfo/asdata.json.gz.tmp && \
|
||||
mv pkg/asinfo/asdata.json.gz.tmp pkg/asinfo/asdata.json.gz
|
||||
@script/asupdate
|
||||
|
||||
@@ -110,9 +110,10 @@ Every route answers `HEAD` as it answers `GET`, without the body.
|
||||
`handle` and `description`
|
||||
|
||||
The search reads the AS list built into RouteWatch, not the database. The
|
||||
list comes from [ipverse/asn-info](https://github.com/ipverse/asn-info). A
|
||||
match can therefore be an AS that the database has no record of. A `q` that
|
||||
is missing, empty or only spaces answers 400.
|
||||
list comes from
|
||||
[ipverse/as-metadata](https://github.com/ipverse/as-metadata). A match can
|
||||
therefore be an AS that the database has no record of. A `q` that is
|
||||
missing, empty or only spaces answers 400.
|
||||
|
||||
A JSON answer is `{"status": "ok", "data": ...}`, or for an error
|
||||
`{"status": "error", "error": {"msg": ..., "code": ...}}`, and carries an
|
||||
@@ -485,6 +486,11 @@ provide:
|
||||
or `go.sum`) and `go fmt` (writes), then `script/check`
|
||||
- `script/install-precommit` — install the git pre-commit hook that runs
|
||||
`script/precommit`
|
||||
- `script/asupdate` — rebuild the AS list built into RouteWatch,
|
||||
`pkg/asinfo/asdata.json.gz`, from the `as.csv` file of
|
||||
[ipverse/as-metadata](https://github.com/ipverse/as-metadata). It needs the
|
||||
network. When the generator fails or finds no ASes, the old list stays and the
|
||||
script exits non-zero.
|
||||
|
||||
Tests and the linter run only in Docker, as the `test` and `lint` phases of the
|
||||
`Dockerfile`; the image cannot be built unless both pass. Every `docker build`
|
||||
|
||||
@@ -33,6 +33,11 @@ tested.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-27: `make asupdate` runs the new `script/asupdate`. It replaces the AS
|
||||
list built into RouteWatch only when the generator succeeds and finds at least
|
||||
one AS. A failed download used to replace the list with an empty file. The
|
||||
generator reads `as.csv` from `ipverse/as-metadata`, the source's new name
|
||||
(closes #173)
|
||||
- 2026-10-27: the WHOIS details of an AS at RIPE, APNIC or AFRINIC come only
|
||||
from its `aut-num` object and the organisation object it names. RIPE and APNIC
|
||||
start their answer with the `as-block` object of the AS's block. Its dates
|
||||
|
||||
@@ -15,7 +15,7 @@ import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
const asCsvURL = "https://github.com/ipverse/asn-info/raw/refs/heads/master/as.csv"
|
||||
const asCsvURL = "https://github.com/ipverse/as-metadata/raw/refs/heads/master/as.csv"
|
||||
|
||||
// The first columns of as.csv, the ones the generator reads. It reads past
|
||||
// the columns after them, such as the country code.
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
Package asinfo provides information about Autonomous Systems (AS).
|
||||
|
||||
The package embeds a comprehensive database of AS numbers, handles, and descriptions
|
||||
sourced from https://github.com/ipverse/asn-info. The data is embedded at compile
|
||||
sourced from https://github.com/ipverse/as-metadata. The data is embedded at compile
|
||||
time and provides fast, offline lookups.
|
||||
|
||||
Basic usage:
|
||||
|
||||
Executable
+43
@@ -0,0 +1,43 @@
|
||||
#!/bin/sh
|
||||
# script/asupdate: rebuild the AS list built into RouteWatch from
|
||||
# as.csv with cmd/asinfo-gen. The new list replaces the old one only
|
||||
# when the generator succeeds and writes at least one AS. Otherwise
|
||||
# the old list stays as it was and the script exits non-zero.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
AS_DATA="pkg/asinfo/asdata.json.gz"
|
||||
# Next to AS_DATA, so the final mv is a rename on one file system.
|
||||
AS_DATA_TMP="$AS_DATA.tmp"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
|
||||
json="$(mktemp)"
|
||||
trap 'rm -f "$json" "$AS_DATA_TMP"' EXIT
|
||||
# A signal exits through the EXIT trap too.
|
||||
trap 'exit 1' HUP INT TERM
|
||||
|
||||
echo "Updating AS info data..."
|
||||
if ! go run ./cmd/asinfo-gen >"$json"; then
|
||||
echo "asupdate: the generator failed; $AS_DATA is unchanged" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# The generator writes a list of ASes with "[" alone on the first
|
||||
# line. An empty list is "[]", and no list at all is "null".
|
||||
first_line=""
|
||||
read -r first_line <"$json" || true
|
||||
if [ "$first_line" != "[" ]; then
|
||||
echo "asupdate: the generator wrote no ASes; $AS_DATA is unchanged" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# -n leaves the time out of the file, so an unchanged list gives the
|
||||
# same bytes.
|
||||
gzip -n -c <"$json" >"$AS_DATA_TMP"
|
||||
mv "$AS_DATA_TMP" "$AS_DATA"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user