embed blogs.json instead of fetching it at runtime (closes #1)

The dataset is vendored as blogs.json and compiled in with go:embed, so
the package does no network I/O on any code path.

FetchBlogs is removed rather than kept as a no-op wrapper, since nothing
is fetched any more: GetBlogs decodes the embedded bytes on first call
and memoizes the result, and RandomBlog, RandomBlogs and NthBlog go
through it. Callers of FetchBlogs have to switch to GetBlogs.

make update-data refreshes the vendored copy from the BlogsURL constant,
downloading to a temporary file and replacing blogs.json only once that
file parses as a non-empty JSON array of blog entries, so a
complete-but-wrong response cannot overwrite the good dataset. A test
walks the dependency graph of the non-test build and fails if any net/*
package is reachable.

Model: opus-5
This commit is contained in:
2026-09-05 03:08:25 +00:00
parent 44b915c54b
commit 7e1b010c4f
6 changed files with 248449 additions and 45 deletions

View File

@@ -1,5 +1,5 @@
# Targets
.PHONY: all run test clean
.PHONY: all run test clean docker lint update-data
all: run
@@ -13,10 +13,29 @@ test:
go test -v ./...
clean:
rm -f example
rm -f example blogs.json.tmp
docker:
docker build --progress plain .
lint:
golangci-lint run
# Refresh the vendored dataset from the BlogsURL constant in hnblogs.go, which
# is the single source of truth for the upstream location. The download lands
# on a temporary file and only replaces blogs.json once it has been checked to
# be a non-empty JSON array of blog entries, so neither an interrupted transfer
# nor a complete-but-wrong response (an error page, a redirect landing page)
# can overwrite the good dataset.
update-data:
@command -v jq >/dev/null || { echo "update-data requires jq" >&2; exit 1; }
@url=$$(sed -n 's/^const BlogsURL = "\(.*\)"$$/\1/p' hnblogs.go); \
test -n "$$url" || { echo "could not parse BlogsURL from hnblogs.go" >&2; exit 1; }; \
echo "downloading $$url"; \
curl -fsSL "$$url" -o blogs.json.tmp
@jq -e 'type == "array" and length > 0 and all(.[]; type == "object" and has("url"))' \
blogs.json.tmp >/dev/null 2>&1 \
|| { echo "download is not a non-empty JSON array of blog entries; blogs.json left unchanged" >&2; \
rm -f blogs.json.tmp; exit 1; }
mv blogs.json.tmp blogs.json
$(MAKE) test