clawbot
d7cf010e00
The server ran os.Exit at the end of its own goroutine, racing fx's teardown and sometimes killing the process before reportbuf's OnStop flushed — silently losing a full flush window of telemetry on every restart, at exit 0. Shutdown now goes through fx.Shutdowner, so every OnStop runs in order. The http.Server is built synchronously in OnStart before the serving goroutine, so shutdown can no longer race or nil-deref it. A listen failure exits non-zero via fx.ExitCode(1). reportbuf's OnStop is guarded by sync.Once. writeTimeout now exceeds the chi per-request budget so that budget is reachable. Dead startupTime, exitCode, and cancelFunc fields are gone. A new test asserts a buffered report reaches disk after the lifecycle stops. Model: opus-4-8
netwatch-server is an MIT-licensed Go HTTP backend by @sneak that receives telemetry reports from the NetWatch SPA and persists them as zstd-compressed JSONL files on disk.
Getting Started
# Build and run locally
make run
# Run tests, lint, and format check
make check
# Docker
docker build -t netwatch-server .
docker run -p 8080:8080 netwatch-server
Rationale
The NetWatch frontend collects latency measurements from the browser but has no
way to persist or aggregate them. This backend provides a minimal
POST /api/v1/reports endpoint that buffers incoming reports in memory and
flushes them to compressed files on disk for later analysis.
Design
The server is structured as an fx-wired Go application under cmd/netwatch-server/.
Internal packages in internal/ follow standard Go project layout:
config: Loads configuration from environment variables and config files via Viper.handlers: HTTP request handlers for the API (health check, report ingestion).reportbuf: In-memory buffer that accumulates JSONL report lines and flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60 seconds.server: Chi-based HTTP server with middleware wiring and route registration.healthcheck,middleware,logger,globals: Supporting infrastructure.
Configuration
| Variable | Default | Description |
|---|---|---|
PORT |
8080 |
HTTP listen port |
DATA_DIR |
./data/reports |
Directory for compressed reports |
DEBUG |
false |
Enable debug logging |
TRUSTED_PROXIES |
loopback + RFC1918 | Comma-separated CIDRs whose X-Forwarded-For / X-Real-IP headers are trusted for client IP resolution |
TRUSTED_PROXIES defaults to 127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16.
The loopback entries cover the reverse proxy that shares the container; the
RFC1918 ranges match nginx.conf. A request whose direct peer is outside this
set has its forwarded headers ignored, and the direct peer is logged instead.
Report storage
Reports are written as reports-<timestamp>.jsonl.zst files in DATA_DIR.
Each file contains one JSON object per line, compressed with zstd. Files are
created with O_EXCL to prevent overwrites.
TODO
- Add integration test that POSTs a report and verifies the compressed output
- Add report decompression/query endpoint
- Add metrics (Prometheus) for buffer size, flush count, report count
- Add retention policy to prune old report files
License
MIT. See LICENSE.