sneak f84378c426 embed blogs.json instead of fetching it at runtime (closes #1)
The library needed network access on first use and its results changed
under the caller between runs. blogs.json is now vendored and compiled in
with go:embed, so the dataset is fixed for a given build.

FetchBlogs keeps its name, signature and sync.Once memoization but now
decodes the embedded bytes; its error return is only reachable if the
committed blogs.json is malformed. net/http is gone from the package, and
a test asserts no net/* import returns to non-test code.

make update-data refreshes the vendored file, reading the upstream
location from the BlogsURL constant so the URL has one definition. It
downloads to a temporary file, replaces blogs.json only on a complete
download, and runs the test suite against the new data.

The dataset is committed verbatim as upstream serves it, which is ~8 MB
of JSON in the repo and in every linking binary; most of that is per-blog
post history that the Blog struct does not expose.

Model: opus-5
2026-09-05 03:08:25 +00:00
2024-06-02 15:21:39 -07:00
2024-06-02 15:21:39 -07:00
2024-06-02 15:21:39 -07:00

hnblogs

A Go library for the blogs.hn dataset: a list of personal blogs collected from Hacker News.

import "sneak.berlin/go/hnblogs"

blog, err := hnblogs.RandomBlog()

Embedded data

The dataset is vendored into this repository as blogs.json and compiled into the package with go:embed. The library performs no network I/O: importing it does not reach out to anything, results do not change under a caller between runs of the same build, and go test works offline.

FetchBlogs keeps its name and its sync.Once memoization, but on first call it decodes the embedded bytes rather than issuing an HTTP request. Its error return is now only reachable if the committed blogs.json is malformed.

The trade-off is that the dataset is a build-time artifact: it is roughly 8 MB of JSON, it lands in every binary that links the package, and it is only as fresh as the last commit that refreshed it.

Refreshing the dataset

make update-data

That target reads the upstream location from the BlogsURL constant in hnblogs.go — the single source of truth — downloads to a temporary file, replaces blogs.json only once the download completes, and then runs the test suite against the new data. Commit the resulting blogs.json to publish the update.

Development

make test
make lint
make docker

make docker runs lint and tests in containers, matching CI.

Description
go library for accessing the hnblogs dataset
Readme WTFPL 2.3 MiB
Languages
Go 73.3%
Makefile 15.9%
Dockerfile 10.8%