embed blogs.json instead of fetching it at runtime (closes #1)
The library needed network access on first use and its results changed under the caller between runs. blogs.json is now vendored and compiled in with go:embed, so the dataset is fixed for a given build. FetchBlogs keeps its name, signature and sync.Once memoization but now decodes the embedded bytes; its error return is only reachable if the committed blogs.json is malformed. net/http is gone from the package, and a test asserts no net/* import returns to non-test code. make update-data refreshes the vendored file, reading the upstream location from the BlogsURL constant so the URL has one definition. It downloads to a temporary file, replaces blogs.json only on a complete download, and runs the test suite against the new data. The dataset is committed verbatim as upstream serves it, which is ~8 MB of JSON in the repo and in every linking binary; most of that is per-blog post history that the Blog struct does not expose. Model: opus-5
This commit is contained in:
47
README.md
Normal file
47
README.md
Normal file
@@ -0,0 +1,47 @@
|
||||
# hnblogs
|
||||
|
||||
A Go library for the [blogs.hn](https://blogs.hn) dataset: a list of personal
|
||||
blogs collected from Hacker News.
|
||||
|
||||
```go
|
||||
import "sneak.berlin/go/hnblogs"
|
||||
|
||||
blog, err := hnblogs.RandomBlog()
|
||||
```
|
||||
|
||||
## Embedded data
|
||||
|
||||
The dataset is vendored into this repository as `blogs.json` and compiled into
|
||||
the package with `go:embed`. The library performs no network I/O: importing it
|
||||
does not reach out to anything, results do not change under a caller between
|
||||
runs of the same build, and `go test` works offline.
|
||||
|
||||
`FetchBlogs` keeps its name and its `sync.Once` memoization, but on first call
|
||||
it decodes the embedded bytes rather than issuing an HTTP request. Its error
|
||||
return is now only reachable if the committed `blogs.json` is malformed.
|
||||
|
||||
The trade-off is that the dataset is a build-time artifact: it is roughly 8 MB
|
||||
of JSON, it lands in every binary that links the package, and it is only as
|
||||
fresh as the last commit that refreshed it.
|
||||
|
||||
## Refreshing the dataset
|
||||
|
||||
```sh
|
||||
make update-data
|
||||
```
|
||||
|
||||
That target reads the upstream location from the `BlogsURL` constant in
|
||||
`hnblogs.go` — the single source of truth — downloads to a temporary file,
|
||||
replaces `blogs.json` only once the download completes, and then runs the test
|
||||
suite against the new data. Commit the resulting `blogs.json` to publish the
|
||||
update.
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
make test
|
||||
make lint
|
||||
make docker
|
||||
```
|
||||
|
||||
`make docker` runs lint and tests in containers, matching CI.
|
||||
Reference in New Issue
Block a user