embed blogs.json instead of fetching it at runtime (closes #1)

The library needed network access on first use and its results changed
under the caller between runs. blogs.json is now vendored and compiled in
with go:embed, so the dataset is fixed for a given build.

FetchBlogs keeps its name, signature and sync.Once memoization but now
decodes the embedded bytes; its error return is only reachable if the
committed blogs.json is malformed. net/http is gone from the package, and
a test asserts no net/* import returns to non-test code.

make update-data refreshes the vendored file, reading the upstream
location from the BlogsURL constant so the URL has one definition. It
downloads to a temporary file, replaces blogs.json only on a complete
download, and runs the test suite against the new data.

The dataset is committed verbatim as upstream serves it, which is ~8 MB
of JSON in the repo and in every linking binary; most of that is per-blog
post history that the Blog struct does not expose.

Model: opus-5
This commit is contained in:
2026-09-05 03:08:25 +00:00
parent 44b915c54b
commit f84378c426
6 changed files with 248423 additions and 31 deletions

47
README.md Normal file
View File

@@ -0,0 +1,47 @@
# hnblogs
A Go library for the [blogs.hn](https://blogs.hn) dataset: a list of personal
blogs collected from Hacker News.
```go
import "sneak.berlin/go/hnblogs"
blog, err := hnblogs.RandomBlog()
```
## Embedded data
The dataset is vendored into this repository as `blogs.json` and compiled into
the package with `go:embed`. The library performs no network I/O: importing it
does not reach out to anything, results do not change under a caller between
runs of the same build, and `go test` works offline.
`FetchBlogs` keeps its name and its `sync.Once` memoization, but on first call
it decodes the embedded bytes rather than issuing an HTTP request. Its error
return is now only reachable if the committed `blogs.json` is malformed.
The trade-off is that the dataset is a build-time artifact: it is roughly 8 MB
of JSON, it lands in every binary that links the package, and it is only as
fresh as the last commit that refreshed it.
## Refreshing the dataset
```sh
make update-data
```
That target reads the upstream location from the `BlogsURL` constant in
`hnblogs.go` — the single source of truth — downloads to a temporary file,
replaces `blogs.json` only once the download completes, and then runs the test
suite against the new data. Commit the resulting `blogs.json` to publish the
update.
## Development
```sh
make test
make lint
make docker
```
`make docker` runs lint and tests in containers, matching CI.