sneak
95c6ec16b4
A buffer failure on POST /api/v1/reports now returns 500 instead of a
false {"status":"ok"}, so clients can retry. Decode errors are split:
an over-limit body returns 413 (via errors.As on *http.MaxBytesError),
malformed JSON stays 400. A new MaxBodyBytes middleware caps every
route (1 MiB default; rejects an oversized Content-Length up-front and
caps the read otherwise), replacing the per-route reader so the health
check and future routes are bounded too. The raw attacker-controlled
geo blob is no longer logged — only its byte length — and client_id and
timestamp are length-bounded before logging. A decodeJSON handler helper
is added per the HTTP server conventions. Panic recovery is now a local
middleware that routes the stack through slog as structured JSON rather
than chi's plain-text stderr. Error bodies still leak nothing internal.
Chosen failure code for a storage failure: 500, since a full buffer or
write error is server-side and retryable, not the client's fault.
Model: opus-4-8
netwatch-server is an MIT-licensed Go HTTP backend by @sneak that receives telemetry reports from the NetWatch SPA and persists them as zstd-compressed JSONL files on disk.
Getting Started
# Build and run locally
make run
# Run tests, lint, and format check
make check
# Docker
docker build -t netwatch-server .
docker run -p 8080:8080 netwatch-server
Rationale
The NetWatch frontend collects latency measurements from the browser but has no
way to persist or aggregate them. This backend provides a minimal
POST /api/v1/reports endpoint that buffers incoming reports in memory and
flushes them to compressed files on disk for later analysis.
Design
The server is structured as an fx-wired Go application under cmd/netwatch-server/.
Internal packages in internal/ follow standard Go project layout:
config: Loads configuration from environment variables and config files via Viper.handlers: HTTP request handlers for the API (health check, report ingestion).reportbuf: In-memory buffer that accumulates JSONL report lines and flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60 seconds.server: Chi-based HTTP server with middleware wiring and route registration.healthcheck,middleware,logger,globals: Supporting infrastructure.
Configuration
| Variable | Default | Description |
|---|---|---|
PORT |
8080 |
HTTP listen port |
DATA_DIR |
./data/reports |
Directory for compressed reports |
DEBUG |
false |
Enable debug logging |
TRUSTED_PROXIES |
loopback + RFC1918 | Comma-separated CIDRs whose X-Forwarded-For / X-Real-IP headers are trusted for client IP resolution |
TRUSTED_PROXIES defaults to 127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16.
The loopback entries cover the reverse proxy that shares the container; the
RFC1918 ranges match nginx.conf. A request whose direct peer is outside this
set has its forwarded headers ignored, and the direct peer is logged instead.
Report storage
Reports are written as reports-<timestamp>.jsonl.zst files in DATA_DIR.
Each file contains one JSON object per line, compressed with zstd. Files are
created with O_EXCL to prevent overwrites.
TODO
- Add integration test that POSTs a report and verifies the compressed output
- Add report decompression/query endpoint
- Add metrics (Prometheus) for buffer size, flush count, report count
- Add retention policy to prune old report files
License
MIT. See LICENSE.