netwatch-server is an MIT-licensed Go HTTP backend by [@sneak](https://sneak.berlin) that receives telemetry reports from the NetWatch SPA and persists them as zstd-compressed JSONL files on disk. ## Getting Started From this directory: ```bash # Build and run locally make run ``` From the repo root, which is also the build context of `Dockerfile.backend`: ```bash # Run tests, lint, and format check over the frontend and this backend make check # Build both images, including netwatch-server make docker docker run -p 8080:8080 netwatch-server ``` ## Entrypoints This directory follows the same [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) pattern as the repo root: the targets in `backend/Makefile` are thin shims over `backend/script/`. `Dockerfile.backend` runs them, and the root scripts call `test`, `fmt` and `fmt-check`: - `script/build` — compile the static `netwatch-server` binary with its version and architecture stamped in. The version is `VERSION` from the environment; when that is unset or empty, it falls back to `git describe` inside a git checkout, then to `dev` - `script/test` — run the Go tests under a 30-second timeout - `script/lint` — check `.golangci.yml` against its pinned sha256, then run golangci-lint. It runs inside the golangci-lint image of the lint stage of `Dockerfile.backend`; from a checkout, run `make lint` at the repo root, which builds that stage - `script/fmt` — format the Go sources (writes) - `script/fmt-check` — check Go formatting (read-only) - `script/run` — build and run the server locally - `script/clean` — remove build artifacts There is no `check`, `hooks` or `docker` target here: the root `make check` covers this directory, the root `make hooks` installs the repo's only pre-commit hook, and the root `make docker` builds this image. ## Rationale The NetWatch frontend collects latency measurements from the browser but has no way to persist or aggregate them. This backend provides a minimal `POST /api/v1/reports` endpoint that buffers incoming reports in memory and flushes them to compressed files on disk for later analysis. ## Design The server is structured as an `fx`-wired Go application under `cmd/netwatch-server/`. Internal packages in `internal/` follow standard Go project layout: - **`config`**: Loads configuration from environment variables and config files via Viper. - **`handlers`**: HTTP request handlers for the API (health check, report ingestion). - **`reportbuf`**: In-memory buffer that accumulates JSONL report lines and flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60 seconds. - **`server`**: Chi-based HTTP server with middleware wiring and route registration. - **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting infrastructure. ### Configuration | Variable | Default | Description | | ----------------- | -------------------- | -------------------------------------------------------------------------------------------------------- | | `PORT` | `8080` | HTTP listen port | | `DATA_DIR` | `./data/reports` | Directory for compressed reports | | `DEBUG` | `false` | Enable debug logging | | `TRUSTED_PROXIES` | loopback + RFC1918 | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution | `TRUSTED_PROXIES` defaults to `127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. The loopback entries cover the reverse proxy that shares the container; the RFC1918 ranges match `nginx.conf`. A request whose direct peer is outside this set has its forwarded headers ignored, and the direct peer is logged instead. ### Report storage Reports are written as `reports-.jsonl.zst` files in `DATA_DIR`. Each file contains one JSON object per line, compressed with zstd. Files are created with `O_EXCL` to prevent overwrites. ## TODO - Add integration test that POSTs a report and verifies the compressed output - Add report decompression/query endpoint - Add metrics (Prometheus) for buffer size, flush count, report count - Add retention policy to prune old report files ## License MIT. See [LICENSE](LICENSE). ## Author [@sneak](https://sneak.berlin)