embed blogs.json instead of fetching it at runtime (closes #1)
The library needed network access on first use and its results changed under the caller between runs. blogs.json is now vendored and compiled in with go:embed, so the dataset is fixed for a given build. FetchBlogs keeps its name, signature and sync.Once memoization but now decodes the embedded bytes; its error return is only reachable if the committed blogs.json is malformed. net/http is gone from the package, and a test asserts no net/* import returns to non-test code. make update-data refreshes the vendored file, reading the upstream location from the BlogsURL constant so the URL has one definition. It downloads to a temporary file, replaces blogs.json only on a complete download, and runs the test suite against the new data. The dataset is committed verbatim as upstream serves it, which is ~8 MB of JSON in the repo and in every linking binary; most of that is per-blog post history that the Blog struct does not expose. Model: opus-5
This commit is contained in:
17
Makefile
17
Makefile
@@ -1,5 +1,5 @@
|
||||
# Targets
|
||||
.PHONY: all run test clean
|
||||
.PHONY: all run test clean docker lint update-data
|
||||
|
||||
all: run
|
||||
|
||||
@@ -13,10 +13,23 @@ test:
|
||||
go test -v ./...
|
||||
|
||||
clean:
|
||||
rm -f example
|
||||
rm -f example blogs.json.tmp
|
||||
|
||||
docker:
|
||||
docker build --progress plain .
|
||||
|
||||
lint:
|
||||
golangci-lint run
|
||||
|
||||
# Refresh the vendored dataset from the BlogsURL constant in hnblogs.go, which
|
||||
# is the single source of truth for the upstream location. The download lands
|
||||
# on a temporary file and only replaces blogs.json once it is complete, so an
|
||||
# interrupted fetch cannot leave a truncated dataset committed.
|
||||
update-data:
|
||||
@url=$$(sed -n 's/^const BlogsURL = "\(.*\)"$$/\1/p' hnblogs.go); \
|
||||
test -n "$$url" || { echo "could not parse BlogsURL from hnblogs.go" >&2; exit 1; }; \
|
||||
echo "fetching $$url"; \
|
||||
curl -fsSL "$$url" -o blogs.json.tmp
|
||||
@test -s blogs.json.tmp || { echo "downloaded dataset is empty" >&2; rm -f blogs.json.tmp; exit 1; }
|
||||
mv blogs.json.tmp blogs.json
|
||||
$(MAKE) test
|
||||
|
||||
Reference in New Issue
Block a user