netwatch/backend

netwatch-server is an MIT-licensed Go HTTP backend by @sneak that receives telemetry reports from the NetWatch SPA and persists them as zstd-compressed JSONL files on disk.

Getting Started

From this directory:

# Build and run locally
make run

From the repo root, whose Dockerfile builds the one image that ships this backend behind nginx (see Container image):

# Run tests, lint, and format check over the frontend and this backend
make check

# Build the image: nginx, the frontend and this backend
make docker
docker run -p 8080:8080 netwatch

Entrypoints

This directory follows the same Scripts to Rule Them All pattern as the repo root: the targets in backend/Makefile are thin shims over backend/script/. The root Dockerfile runs them, and the root scripts call test, fmt and fmt-check:

• script/build — compile the static netwatch-server binary with its version and architecture stamped in. The version is VERSION from the environment; when that is unset or empty, it falls back to git describe inside a git checkout, then to dev
• script/test — run the Go tests under a 30-second timeout
• script/lint — check .golangci.yml against its pinned sha256, then run golangci-lint. It runs inside the golangci-lint image of the lint stage of the root Dockerfile; from a checkout, run make lint at the repo root, which builds that stage
• script/fmt — format the Go sources (writes)
• script/fmt-check — check Go formatting (read-only)
• script/run — build and run the server locally
• script/clean — remove build artifacts

There is no check, hooks or docker target here: the root make check covers this directory, the root make hooks installs the repo's only pre-commit hook, and the root make docker builds the image that contains this backend.

Rationale

The NetWatch frontend collects latency measurements from the browser but has no way to persist or aggregate them. This backend provides a minimal POST /api/v1/reports endpoint that buffers incoming reports in memory and flushes them to compressed files on disk for later analysis.

Design

The server is structured as an fx-wired Go application under cmd/netwatch-server/. Internal packages in internal/ follow standard Go project layout:

• config: Loads configuration from environment variables and config files via Viper.
• handlers: HTTP request handlers for the API (health check, report ingestion).
• reportbuf: In-memory buffer that accumulates JSONL report lines and flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60 seconds.
• server: Chi-based HTTP server with middleware wiring and route registration.
• healthcheck, middleware, logger, globals: Supporting infrastructure.

Configuration

Variable	Default	Description
BIND_ADDRESS	empty	IP address to listen on; empty listens on every interface
PORT	8080	HTTP listen port
DATA_DIR	./data/reports	Directory for compressed reports
DEBUG	false	Enable debug logging
TRUSTED_PROXIES	loopback + RFC1918	Comma-separated CIDRs whose X-Forwarded-For / X-Real-IP headers are trusted for client IP resolution

TRUSTED_PROXIES defaults to 127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16. The loopback entries cover the reverse proxy that shares the container; the RFC1918 ranges match nginx.conf. A request whose direct peer is outside this set has its forwarded headers ignored, and the direct peer is logged instead.

Container image

The root Dockerfile builds one image in which nginx listens on the public port 8080, serves the frontend, and proxies /api/ and /.well-known/healthcheck to this server. The image's entrypoint, bin/entrypoint.sh, starts the server as user netwatch (uid 1000) with BIND_ADDRESS=127.0.0.1 and PORT=8081, so only nginx reaches it. DATA_DIR is /data/reports, on the /data volume, which netwatch owns.

Report storage

Reports are written as reports-<timestamp>.jsonl.zst files in DATA_DIR. Each file contains one JSON object per line, compressed with zstd. Files are created with O_EXCL to prevent overwrites.

TODO

• Add integration test that POSTs a report and verifies the compressed output
• Add report decompression/query endpoint
• Add metrics (Prometheus) for buffer size, flush count, report count
• Add retention policy to prune old report files

License

MIT. See LICENSE.

Author

@sneak