embed blogs.json instead of fetching it at runtime (closes #1)
The dataset is vendored as blogs.json and compiled in with go:embed, so the package does no network I/O on any code path. FetchBlogs is removed rather than kept as a no-op wrapper, since nothing is fetched any more: GetBlogs decodes the embedded bytes on first call and memoizes the result, and RandomBlog, RandomBlogs and NthBlog go through it. Callers of FetchBlogs have to switch to GetBlogs. make update-data refreshes the vendored copy from the BlogsURL constant, downloading to a temporary file and replacing blogs.json only once that file parses as a non-empty JSON array of blog entries, so a complete-but-wrong response cannot overwrite the good dataset. A test walks the dependency graph of the non-test build and fails if any net/* package is reachable. Model: opus-5
This commit is contained in:
23
Makefile
23
Makefile
@@ -1,5 +1,5 @@
|
||||
# Targets
|
||||
.PHONY: all run test clean
|
||||
.PHONY: all run test clean docker lint update-data
|
||||
|
||||
all: run
|
||||
|
||||
@@ -13,10 +13,29 @@ test:
|
||||
go test -v ./...
|
||||
|
||||
clean:
|
||||
rm -f example
|
||||
rm -f example blogs.json.tmp
|
||||
|
||||
docker:
|
||||
docker build --progress plain .
|
||||
|
||||
lint:
|
||||
golangci-lint run
|
||||
|
||||
# Refresh the vendored dataset from the BlogsURL constant in hnblogs.go, which
|
||||
# is the single source of truth for the upstream location. The download lands
|
||||
# on a temporary file and only replaces blogs.json once it has been checked to
|
||||
# be a non-empty JSON array of blog entries, so neither an interrupted transfer
|
||||
# nor a complete-but-wrong response (an error page, a redirect landing page)
|
||||
# can overwrite the good dataset.
|
||||
update-data:
|
||||
@command -v jq >/dev/null || { echo "update-data requires jq" >&2; exit 1; }
|
||||
@url=$$(sed -n 's/^const BlogsURL = "\(.*\)"$$/\1/p' hnblogs.go); \
|
||||
test -n "$$url" || { echo "could not parse BlogsURL from hnblogs.go" >&2; exit 1; }; \
|
||||
echo "downloading $$url"; \
|
||||
curl -fsSL "$$url" -o blogs.json.tmp
|
||||
@jq -e 'type == "array" and length > 0 and all(.[]; type == "object" and has("url"))' \
|
||||
blogs.json.tmp >/dev/null 2>&1 \
|
||||
|| { echo "download is not a non-empty JSON array of blog entries; blogs.json left unchanged" >&2; \
|
||||
rm -f blogs.json.tmp; exit 1; }
|
||||
mv blogs.json.tmp blogs.json
|
||||
$(MAKE) test
|
||||
|
||||
Reference in New Issue
Block a user