sneak
f84378c426
The library needed network access on first use and its results changed under the caller between runs. blogs.json is now vendored and compiled in with go:embed, so the dataset is fixed for a given build. FetchBlogs keeps its name, signature and sync.Once memoization but now decodes the embedded bytes; its error return is only reachable if the committed blogs.json is malformed. net/http is gone from the package, and a test asserts no net/* import returns to non-test code. make update-data refreshes the vendored file, reading the upstream location from the BlogsURL constant so the URL has one definition. It downloads to a temporary file, replaces blogs.json only on a complete download, and runs the test suite against the new data. The dataset is committed verbatim as upstream serves it, which is ~8 MB of JSON in the repo and in every linking binary; most of that is per-blog post history that the Blog struct does not expose. Model: opus-5
48 lines
1.4 KiB
Markdown
48 lines
1.4 KiB
Markdown
# hnblogs
|
|
|
|
A Go library for the [blogs.hn](https://blogs.hn) dataset: a list of personal
|
|
blogs collected from Hacker News.
|
|
|
|
```go
|
|
import "sneak.berlin/go/hnblogs"
|
|
|
|
blog, err := hnblogs.RandomBlog()
|
|
```
|
|
|
|
## Embedded data
|
|
|
|
The dataset is vendored into this repository as `blogs.json` and compiled into
|
|
the package with `go:embed`. The library performs no network I/O: importing it
|
|
does not reach out to anything, results do not change under a caller between
|
|
runs of the same build, and `go test` works offline.
|
|
|
|
`FetchBlogs` keeps its name and its `sync.Once` memoization, but on first call
|
|
it decodes the embedded bytes rather than issuing an HTTP request. Its error
|
|
return is now only reachable if the committed `blogs.json` is malformed.
|
|
|
|
The trade-off is that the dataset is a build-time artifact: it is roughly 8 MB
|
|
of JSON, it lands in every binary that links the package, and it is only as
|
|
fresh as the last commit that refreshed it.
|
|
|
|
## Refreshing the dataset
|
|
|
|
```sh
|
|
make update-data
|
|
```
|
|
|
|
That target reads the upstream location from the `BlogsURL` constant in
|
|
`hnblogs.go` — the single source of truth — downloads to a temporary file,
|
|
replaces `blogs.json` only once the download completes, and then runs the test
|
|
suite against the new data. Commit the resulting `blogs.json` to publish the
|
|
update.
|
|
|
|
## Development
|
|
|
|
```sh
|
|
make test
|
|
make lint
|
|
make docker
|
|
```
|
|
|
|
`make docker` runs lint and tests in containers, matching CI.
|