netwatch/backend/README.md

netwatch-server is an MIT-licensed Go HTTP backend by @sneak that receives telemetry reports from the NetWatch SPA and persists them as zstd-compressed JSONL files on disk.

Getting Started

From this directory:

# Build and run locally
make run

From the repo root, which is also the build context of Dockerfile.backend:

# Run tests, lint, and format check over the frontend and this backend
make check

# Build both images, including netwatch-server
make docker
docker run -p 8080:8080 netwatch-server

Entrypoints

This directory follows the same Scripts to Rule Them All pattern as the repo root: the targets in backend/Makefile are thin shims over backend/script/. Dockerfile.backend runs them, and the root scripts call test, fmt and fmt-check:

• script/build — compile the static netwatch-server binary with its version and architecture stamped in. The version is VERSION from the environment; when that is unset or empty, it falls back to git describe inside a git checkout, then to dev
• script/test — run the Go tests under a 30-second timeout
• script/lint — check .golangci.yml against its pinned sha256, then run golangci-lint. It runs inside the golangci-lint image of the lint stage of Dockerfile.backend; from a checkout, run make lint at the repo root, which builds that stage
• script/fmt — format the Go sources (writes)
• script/fmt-check — check Go formatting (read-only)
• script/run — build and run the server locally
• script/clean — remove build artifacts

There is no check, hooks or docker target here: the root make check covers this directory, the root make hooks installs the repo's only pre-commit hook, and the root make docker builds this image.

Rationale

The NetWatch frontend collects latency measurements from the browser but has no way to persist or aggregate them. This backend provides a minimal POST /api/v1/reports endpoint that buffers incoming reports in memory and flushes them to compressed files on disk for later analysis.

Design

The server is structured as an fx-wired Go application under cmd/netwatch-server/. Internal packages in internal/ follow standard Go project layout:

• config: Loads configuration from environment variables and config files via Viper.
• handlers: HTTP request handlers for the API (health check, report ingestion).
• reportbuf: In-memory buffer that accumulates JSONL report lines and flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60 seconds.
• server: Chi-based HTTP server with middleware wiring and route registration.
• healthcheck, middleware, logger, globals: Supporting infrastructure.

Configuration

Variable	Default	Description
PORT	8080	HTTP listen port
DATA_DIR	./data/reports	Directory for compressed reports
DEBUG	false	Enable debug logging
TRUSTED_PROXIES	loopback + RFC1918	Comma-separated CIDRs whose X-Forwarded-For / X-Real-IP headers are trusted for client IP resolution

TRUSTED_PROXIES defaults to 127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16. The loopback entries cover the reverse proxy that shares the container; the RFC1918 ranges match nginx.conf. A request whose direct peer is outside this set has its forwarded headers ignored, and the direct peer is logged instead.

Report storage

Reports are written as reports-<timestamp>.jsonl.zst files in DATA_DIR. Each file contains one JSON object per line, compressed with zstd. Files are created with O_EXCL to prevent overwrites.

TODO

• Add integration test that POSTs a report and verifies the compressed output
• Add report decompression/query endpoint
• Add metrics (Prometheus) for buffer size, flush count, report count
• Add retention policy to prune old report files

License

MIT. See LICENSE.

Author

@sneak