Keep the AS list when the asupdate generator fails (closes #173)
check / check (push) Canceled after 0s

make asupdate piped the generator into gzip, so the target saw gzip's
exit status. A failed download replaced pkg/asinfo/asdata.json.gz with
an empty file and the target still exited 0.

The target now runs the new script/asupdate. It writes the output of
cmd/asinfo-gen to a temporary file. Only when the generator succeeds
does it compress the list and move it into place. Otherwise it leaves
the file alone and exits non-zero. The generator now fails when as.csv
holds no AS.

The generator reads as.csv from ipverse/as-metadata, the source's new
name. The list it builds there is unchanged, so no new list is
committed.

Model: opus-5-5
This commit was merged in pull request #176.
This commit is contained in:
2026-10-27 02:43:35 +01:00
parent 57469b6370
commit 023c3f9b41
7 changed files with 79 additions and 10 deletions
+2 -3
View File
@@ -52,7 +52,6 @@ clean:
run: build
DEBUG=routewatch ./bin/routewatch 2>&1 | tee log.txt
# Rebuild the AS list built into RouteWatch from as.csv (needs the network).
asupdate:
@echo "Updating AS info data..."
@go run cmd/asinfo-gen/main.go | gzip > pkg/asinfo/asdata.json.gz.tmp && \
mv pkg/asinfo/asdata.json.gz.tmp pkg/asinfo/asdata.json.gz
@script/asupdate
+9 -3
View File
@@ -110,9 +110,10 @@ Every route answers `HEAD` as it answers `GET`, without the body.
`handle` and `description`
The search reads the AS list built into RouteWatch, not the database. The
list comes from [ipverse/asn-info](https://github.com/ipverse/asn-info). A
match can therefore be an AS that the database has no record of. A `q` that
is missing, empty or only spaces answers 400.
list comes from
[ipverse/as-metadata](https://github.com/ipverse/as-metadata). A match can
therefore be an AS that the database has no record of. A `q` that is
missing, empty or only spaces answers 400.
A JSON answer is `{"status": "ok", "data": ...}`, or for an error
`{"status": "error", "error": {"msg": ..., "code": ...}}`, and carries an
@@ -485,6 +486,11 @@ provide:
or `go.sum`) and `go fmt` (writes), then `script/check`
- `script/install-precommit` — install the git pre-commit hook that runs
`script/precommit`
- `script/asupdate` — rebuild the AS list built into RouteWatch,
`pkg/asinfo/asdata.json.gz`, from the `as.csv` file of
[ipverse/as-metadata](https://github.com/ipverse/as-metadata). It needs the
network. It runs `cmd/asinfo-gen`, which fails when it finds no AS. When
`cmd/asinfo-gen` fails, the old list stays and the script exits non-zero.
Tests and the linter run only in Docker, as the `test` and `lint` phases of the
`Dockerfile`; the image cannot be built unless both pass. Every `docker build`
+5
View File
@@ -33,6 +33,11 @@ tested.
# Completed Steps
- 2026-10-27: `make asupdate` runs the new `script/asupdate`. It replaces the AS
list built into RouteWatch only when the generator succeeds and finds at least
one AS. A failed download used to replace the list with an empty file. The
generator reads `as.csv` from `ipverse/as-metadata`, the source's new name
(closes #173)
- 2026-10-27: the WHOIS details of an AS at RIPE, APNIC or AFRINIC come only
from its `aut-num` object and the organisation object it names. RIPE and APNIC
start their answer with the `as-block` object of the AS's block. Its dates
+10 -3
View File
@@ -15,7 +15,7 @@ import (
"strings"
)
const asCsvURL = "https://github.com/ipverse/asn-info/raw/refs/heads/master/as.csv"
const asCsvURL = "https://github.com/ipverse/as-metadata/raw/refs/heads/master/as.csv"
// The first columns of as.csv, the ones the generator reads. It reads past
// the columns after them, such as the country code.
@@ -26,7 +26,10 @@ const (
columnsRead = 3
)
var errUnexpectedHeader = errors.New("unexpected CSV header")
var (
errUnexpectedHeader = errors.New("unexpected CSV header")
errNoASes = errors.New("no AS in the CSV")
)
// ASInfo represents information about an Autonomous System
type ASInfo struct {
@@ -82,7 +85,7 @@ func main() {
// parseCSV reads as.csv from r and returns its ASes, with the handle and the
// description as the file spells them. A header whose first columns are not
// the AS number, the handle and the description is an error. A record that
// does not parse is logged and skipped.
// does not parse is logged and skipped. A file left with no AS is an error.
func parseCSV(r io.Reader) ([]ASInfo, error) {
reader := csv.NewReader(r)
@@ -126,5 +129,9 @@ func parseCSV(r io.Reader) ([]ASInfo, error) {
})
}
if len(asInfos) == 0 {
return nil, errNoASes
}
return asInfos, nil
}
+18
View File
@@ -1,6 +1,7 @@
package main
import (
"errors"
"slices"
"strings"
"testing"
@@ -53,3 +54,20 @@ func TestParseCSVRejectsAnotherHeader(t *testing.T) {
}
}
}
// TestParseCSVRejectsAFileWithNoASes parses files that hold no AS, one with
// only the header and one whose only record has no valid AS number. It
// expects errNoASes, so that make asupdate keeps the old list.
func TestParseCSVRejectsAFileWithNoASes(t *testing.T) {
t.Parallel()
for _, asCSV := range []string{
"asn,handle,description,country-code\n",
"asn,handle,description,country-code\nAS1,LVLT-1,Level 3,US\n",
} {
_, err := parseCSV(strings.NewReader(asCSV))
if !errors.Is(err, errNoASes) {
t.Errorf("parseCSV(%q) = %v, want %v", asCSV, err, errNoASes)
}
}
}
+1 -1
View File
@@ -2,7 +2,7 @@
Package asinfo provides information about Autonomous Systems (AS).
The package embeds a comprehensive database of AS numbers, handles, and descriptions
sourced from https://github.com/ipverse/asn-info. The data is embedded at compile
sourced from https://github.com/ipverse/as-metadata. The data is embedded at compile
time and provides fast, offline lookups.
Basic usage:
+34
View File
@@ -0,0 +1,34 @@
#!/bin/sh
# script/asupdate: rebuild the AS list built into RouteWatch from
# as.csv with cmd/asinfo-gen, which fails when it finds no AS. The new
# list replaces the old one only when the generator succeeds. Otherwise
# the old list stays as it was and the script exits non-zero.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
AS_DATA="pkg/asinfo/asdata.json.gz"
# Next to AS_DATA, so the final mv is a rename on one file system.
AS_DATA_TMP="$AS_DATA.tmp"
main() {
cd "$ROOT"
json="$(mktemp)"
trap 'rm -f "$json" "$AS_DATA_TMP"' EXIT
# A signal exits through the EXIT trap too.
trap 'exit 1' HUP INT TERM
echo "Updating AS info data..."
if ! go run ./cmd/asinfo-gen >"$json"; then
echo "asupdate: the generator failed; $AS_DATA is unchanged" >&2
exit 1
fi
# -n leaves the time out of the file, so an unchanged list gives the
# same bytes.
gzip -n -c <"$json" >"$AS_DATA_TMP"
mv "$AS_DATA_TMP" "$AS_DATA"
}
main "$@"