b100814f8e
All checks were successful
check / check (push) Successful in 29s
Root `make check` only ever ran the frontend, so the "main is always green" policy was satisfied vacuously: the Go backend could be entirely broken and the root gate stayed green. - The backend moves onto scripts-to-rule-them-all. Its test, lint, fmt, fmt-check, build, run and clean implementations now live in `backend/script/`, and `backend/Makefile` is thin shims. The backend is its own project (own module, README, LICENSE, linter config, Dockerfile stage), and `Dockerfile.backend` only copies `backend/` into its builder, so its scripts have to live under `backend/`. - The root `script/test`, `script/lint`, `script/fmt` and `script/fmt-check` now run the frontend step and then the matching `backend/script/*` step, so `script/check` — and therefore the pre-commit hook — gates both halves. The frontend-only steps moved into `script/frontend-*` so nothing is duplicated. - `script/bootstrap` now provisions the backend's toolchain as well, because widening the gate without widening bootstrap left the documented fresh-clone path (`make setup`) installing a pre-commit hook that rejected every commit with `golangci-lint: not found`. Go is reused when the installed version is new enough, otherwise it is installed from the official `go1.25.7` release archive; golangci-lint is installed at exactly `2.7.2`, the version `Dockerfile.backend` pins, so local findings match CI. Both come from a specific release archive whose sha256 is hardcoded here and verified before anything is unpacked — never an install script piped to a shell — and both are symlinked onto `PATH`, since nvm-style activation does not reach `make` or the git hook. - `script/frontend-check` is the frontend half of the gate, exposed as the `frontend-check` target, for the frontend Dockerfile: its build stage is a node image with no Go toolchain. The backend half is gated by `Dockerfile.backend`, and `script/cibuild` builds both images, so the two Dockerfiles together still gate the whole repo. The `backend-check` target is the mirror of it. Both targets are named after the script they shim, like every other target. - `script/cibuild` builds both images through one `build_image` helper, and the Gitea workflow's only build step is `script/cibuild`; the raw `docker build -f Dockerfile.backend .` is gone from the workflow. `script/docker` likewise builds and tags both images. - `backend/Makefile`'s `hooks` target is removed. It wrote the same `.git/hooks/pre-commit` as `script/install-precommit`, so the two clobbered each other and the developer silently ended up gating on only one half of the repo. `script/install-precommit` is now the only installer, and the hook it writes runs the repo-wide `script/check`. - `backend/Makefile`'s `docker` target is removed too: the backend image builds from the repo root with a root-level Dockerfile, so it belongs to the root `script/docker` and `script/cibuild` rather than to a backend script that would have to reach outside `backend/`. - `backend/script/lint` verifies that `.golangci.yml` still matches its pinned sha256 before running the linter. Offline hash comparison, no network. The pin is marked provisional in the file: it is the config currently on `main`, and the comment names PR #31 and the canonical hash that must replace it when #31 lands. - Every script locates the repo root with the mandated `$(cd "$(dirname "$0")/.." && pwd -P)` idiom, `cd`s there, and calls siblings as `"$ROOT/script/<name>"`; the `SCRIPT_DIR` variant is gone. READMEs at the root and in `backend/` document every script, the backend's Getting Started separates commands run from `backend/` from those run at the repo root, and `TODO.md` records the change.
107 lines
3.9 KiB
Markdown
107 lines
3.9 KiB
Markdown
netwatch-server is an MIT-licensed Go HTTP backend by
|
|
[@sneak](https://sneak.berlin) that receives telemetry reports from the NetWatch
|
|
SPA and persists them as zstd-compressed JSONL files on disk.
|
|
|
|
## Getting Started
|
|
|
|
From this directory (`backend/`):
|
|
|
|
```bash
|
|
# Build and run locally
|
|
make run
|
|
|
|
# Run the backend's tests, lint, and format check
|
|
make check
|
|
```
|
|
|
|
From the repo root, one directory up — `Dockerfile.backend` lives there and its
|
|
build context is the repo root, so there is no `docker` target here:
|
|
|
|
```bash
|
|
# Build both images, including netwatch-server
|
|
make docker
|
|
|
|
# Run the backend image
|
|
docker run -p 8080:8080 netwatch-server
|
|
```
|
|
|
|
## Entrypoints
|
|
|
|
This project follows the same
|
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
|
pattern as the repo root: the implementations live in `backend/script/` and the
|
|
targets in `backend/Makefile` are thin shims that call them. The repo root's
|
|
`script/test`, `script/lint`, `script/fmt` and `script/fmt-check` call these
|
|
too, so the root `make check` covers the backend.
|
|
|
|
- `script/build` — compile `netwatch-server` with the version and architecture
|
|
stamped in via ldflags (statically linked on Linux)
|
|
- `script/test` — run the Go tests under a 30-second timeout
|
|
- `script/lint` — assert `.golangci.yml` still matches its pinned sha256, then
|
|
run golangci-lint
|
|
- `script/fmt` — format the Go sources (writes)
|
|
- `script/fmt-check` — check Go formatting (read-only)
|
|
- `script/check` — run test, lint, and fmt-check
|
|
- `script/run` — build and run the server locally
|
|
- `script/clean` — remove build artifacts
|
|
|
|
There is deliberately no `hooks` target here: the repo has exactly one
|
|
pre-commit hook installer, the root `script/install-precommit`, and the hook it
|
|
installs runs the root `script/check`, which gates both halves of the repo.
|
|
There is no `docker` target either: `Dockerfile.backend` lives at the repo root
|
|
and builds with the repo root as its context, so the backend image is built by
|
|
the root `make docker` and by `script/cibuild`.
|
|
|
|
## Rationale
|
|
|
|
The NetWatch frontend collects latency measurements from the browser but has no
|
|
way to persist or aggregate them. This backend provides a minimal
|
|
`POST /api/v1/reports` endpoint that buffers incoming reports in memory and
|
|
flushes them to compressed files on disk for later analysis.
|
|
|
|
## Design
|
|
|
|
The server is structured as an `fx`-wired Go application under `cmd/netwatch-server/`.
|
|
Internal packages in `internal/` follow standard Go project layout:
|
|
|
|
- **`config`**: Loads configuration from environment variables and config files
|
|
via Viper.
|
|
- **`handlers`**: HTTP request handlers for the API (health check, report
|
|
ingestion).
|
|
- **`reportbuf`**: In-memory buffer that accumulates JSONL report lines and
|
|
flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60
|
|
seconds.
|
|
- **`server`**: Chi-based HTTP server with middleware wiring and route
|
|
registration.
|
|
- **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting
|
|
infrastructure.
|
|
|
|
### Configuration
|
|
|
|
| Variable | Default | Description |
|
|
| ---------- | ------------------ | --------------------------------- |
|
|
| `PORT` | `8080` | HTTP listen port |
|
|
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
|
|
| `DEBUG` | `false` | Enable debug logging |
|
|
|
|
### Report storage
|
|
|
|
Reports are written as `reports-<timestamp>.jsonl.zst` files in `DATA_DIR`.
|
|
Each file contains one JSON object per line, compressed with zstd. Files are
|
|
created with `O_EXCL` to prevent overwrites.
|
|
|
|
## TODO
|
|
|
|
- Add integration test that POSTs a report and verifies the compressed output
|
|
- Add report decompression/query endpoint
|
|
- Add metrics (Prometheus) for buffer size, flush count, report count
|
|
- Add retention policy to prune old report files
|
|
|
|
## License
|
|
|
|
MIT. See [LICENSE](LICENSE).
|
|
|
|
## Author
|
|
|
|
[@sneak](https://sneak.berlin)
|