clawbot
64e142c17f
check / check (push) Successful in 2m57s
README.md: Getting Started leads with make targets; a Backend section gives netwatch-server's routes and how the image builds and runs it; the checks are GET requests; the WAN host list, health states, summary figures, sorting and missing features match src/main.js; the TODO section points to TODO.md, which holds the one to-do list. backend/README.md: its TODO section points to TODO.md too, whose Future Steps take its three open items. TODO.md: Workflow branches from next and opens the PR against next; Status, Next Step and Future Steps describe the open work, linked to its issue where one exists. test/viewport/README.md: the unit tests run on Node's test runner, not vitest. Model: opus-5-5
232 lines
13 KiB
Markdown
232 lines
13 KiB
Markdown
netwatch-server is an MIT-licensed Go HTTP backend by
|
|
[@sneak](https://sneak.berlin) that receives telemetry reports from the NetWatch
|
|
SPA and persists them as zstd-compressed JSONL files on disk.
|
|
|
|
## Getting Started
|
|
|
|
From this directory:
|
|
|
|
```bash
|
|
# Build and run locally
|
|
make run
|
|
```
|
|
|
|
From the repo root, whose `Dockerfile` builds the one image that ships this
|
|
backend behind nginx (see [Container image](#container-image)):
|
|
|
|
```bash
|
|
# Run tests, lint, and format check over the frontend and this backend
|
|
make check
|
|
|
|
# Build the image: nginx, the frontend and this backend
|
|
make docker
|
|
docker run -p 8080:8080 netwatch
|
|
```
|
|
|
|
## Entrypoints
|
|
|
|
This directory follows the same
|
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
|
pattern as the repo root: the targets in `backend/Makefile` are thin shims over
|
|
`backend/script/`. The root `Dockerfile` runs them, and the root scripts call
|
|
`test`, `fmt` and `fmt-check`:
|
|
|
|
- `script/build` — compile the static `netwatch-server` binary with its version
|
|
stamped in. The version is `VERSION` from the environment; when that is unset
|
|
or empty, it falls back to `git describe` inside a git checkout, then to `dev`
|
|
- `script/test` — run the Go tests with the race detector and coverage. Go's
|
|
`-timeout 30s` bounds the tests, not their compile. If they fail, they run
|
|
again with `-v` for the details, and the script fails. The race detector needs
|
|
a C compiler
|
|
- `script/lint` — check `.golangci.yml` against its pinned sha256, then run
|
|
golangci-lint. It runs inside the golangci-lint image of the lint stage of the
|
|
root `Dockerfile`; from a checkout, run `make lint` at the repo root, which
|
|
builds that stage
|
|
- `script/fmt` — format the Go sources (writes)
|
|
- `script/fmt-check` — check Go formatting (read-only)
|
|
- `script/run` — build and run the server locally
|
|
- `script/clean` — remove build artifacts
|
|
|
|
There is no `check`, `hooks` or `docker` target here: the root `make check`
|
|
covers this directory, the root `make hooks` installs the repo's only pre-commit
|
|
hook, and the root `make docker` builds the image that contains this backend.
|
|
|
|
## Rationale
|
|
|
|
The NetWatch frontend collects latency measurements from the browser but has no
|
|
way to persist or aggregate them. This backend provides a minimal
|
|
`POST /api/v1/reports` endpoint that buffers incoming reports in memory and
|
|
flushes them to compressed files on disk for later analysis.
|
|
|
|
## Design
|
|
|
|
The server is structured as an `fx`-wired Go application under
|
|
`cmd/netwatch-server/`. Internal packages in `internal/` follow standard Go
|
|
project layout:
|
|
|
|
- **`config`**: Loads configuration from environment variables and config files
|
|
via Viper.
|
|
- **`handlers`**: HTTP request handlers for the API (health check, report
|
|
ingestion).
|
|
- **`reportbuf`**: In-memory buffer that accumulates JSONL report lines and
|
|
flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60
|
|
seconds.
|
|
- **`server`**: Chi-based HTTP server with middleware wiring and route
|
|
registration.
|
|
- **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting
|
|
infrastructure.
|
|
|
|
### Configuration
|
|
|
|
| Variable | Default | Description |
|
|
| ---------------------- | -------------------- | -------------------------------------------------------------------------------------------------------- |
|
|
| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface |
|
|
| `PORT` | `8080` | HTTP listen port |
|
|
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
|
|
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) |
|
|
| `DEBUG` | `false` | Enable debug logging |
|
|
| `TRUSTED_PROXIES` | loopback + RFC1918 | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
|
|
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
|
|
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
|
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
|
|
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
|
|
| `SENTRY_DSN` | empty | DSN of the Sentry project to send errors to; see [Sentry](#sentry) |
|
|
|
|
`TRUSTED_PROXIES` defaults to
|
|
`127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. The loopback
|
|
entries cover a reverse proxy on the same host. A request whose direct peer is
|
|
outside this set has its forwarded headers ignored, and the direct peer is
|
|
logged and rate-limited instead. The container image does not use this default;
|
|
see [Container image](#container-image).
|
|
|
|
A variable set to a value the server cannot use, such as `PORT=abc`,
|
|
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
|
|
starting, with an error naming the variable. An empty variable counts as unset.
|
|
|
|
### Container image
|
|
|
|
The root `Dockerfile` builds one image in which nginx listens on the public port
|
|
8080, serves the frontend, and proxies `/api/`, `/.well-known/healthcheck` and
|
|
`/metrics` to this server. The image's entrypoint, `bin/entrypoint.sh`, starts
|
|
the server as user `netwatch` (uid 1000) with `BIND_ADDRESS=127.0.0.1` and
|
|
`PORT=8081`, so only nginx reaches it, and with `TRUSTED_PROXIES=127.0.0.1/32`,
|
|
so it takes the client address nginx passes on and no other. `DATA_DIR` is
|
|
`/data/reports`, on the `/data` volume; before starting the server, the
|
|
entrypoint creates it and gives it and `/data` to `netwatch` with
|
|
`netwatch-server prepare-data-dir`, which acts on nothing outside `/data`. nginx
|
|
replaces the security headers this server sets with those in the root
|
|
`security-headers.conf`, so those are what clients of the image see.
|
|
|
|
The container's own `TRUSTED_PROXIES` goes to nginx instead: IP addresses or
|
|
CIDRs, separated by commas, of the reverse proxies in front of the container.
|
|
nginx takes the client address from `X-Forwarded-For` only on a request from one
|
|
of them. Unset or empty, nginx trusts no proxy, and the client address is the
|
|
one each request comes from, so every client behind a proxy shares one rate
|
|
limit. An entry that is not an IP address or CIDR, such as a hostname or
|
|
`1.2.3`, stops the container at start with an error naming `TRUSTED_PROXIES`:
|
|
the entrypoint checks each entry with `netwatch-server check-cidr`, which parses
|
|
it as this server parses its own `TRUSTED_PROXIES`.
|
|
|
|
### Report storage
|
|
|
|
Reports are written as `reports-<timestamp>-<number>.jsonl.zst` files in
|
|
`DATA_DIR`. The timestamp is in UTC to the millisecond, so the names sort by
|
|
time. The number starts at 1 when the server starts and goes up by one for each
|
|
file the server starts to write, so two files written in the same millisecond
|
|
still get different names. A failed write uses up its number and leaves a gap in
|
|
the numbers: its file, if it was created, is removed. The file stays, counted
|
|
toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too.
|
|
Each file contains one JSON object per line, compressed with zstd. Files are
|
|
created with `O_EXCL` to prevent overwrites.
|
|
|
|
### Report limits
|
|
|
|
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
|
credentials, so it is bounded instead. Both refusals below answer with the same
|
|
`{"status":"error"}` body as any other error.
|
|
|
|
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
|
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
|
`Retry-After: 60`. The minute slides: reports from the minute before still
|
|
count, fading out over the current one, so an address is sure never to be
|
|
refused only while it sends at most half of `REPORTS_PER_MINUTE` in any 60
|
|
seconds. The page sends one report a minute from each open tab, so the default
|
|
of 60 refuses nothing from up to 30 tabs behind one address, such as a
|
|
household or an office sharing it, however their reports bunch up. Report
|
|
responses also carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and
|
|
`X-RateLimit-Reset` headers.
|
|
- **Size cap.** The report files in `DATA_DIR` may total at most
|
|
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
|
|
waiting in memory count at their uncompressed size until they are written;
|
|
those lost to a failed write stop counting, and the part of its file written
|
|
is removed. When a report would take the total past the cap, the oldest report
|
|
files are deleted to make room, and each deletion is logged with the file's
|
|
name and size; a file still being written is never deleted. A report is
|
|
refused with 507, and nothing of it is stored, only when the reports waiting
|
|
to be written fill the cap on their own, and then no file is deleted. At
|
|
start, report files past the cap, as after lowering it, are deleted the same
|
|
way. So the cap is how much of the newest reports is kept: the default of 1
|
|
GiB is small enough for any host; set it to the space you can give `DATA_DIR`.
|
|
|
|
### CORS
|
|
|
|
The page calls the API from the origin it is served from, so by default the
|
|
server sends no CORS headers, and browsers let no other origin's pages call it.
|
|
To serve the page from elsewhere, list that origin in `CORS_ALLOWED_ORIGINS`
|
|
(for example `https://netwatch.example.com`); pages from a listed origin may
|
|
`GET` and `POST` with a `Content-Type` header. Each entry must be a plain
|
|
origin, `scheme://host` with an optional `:port`, as browsers send it: no path,
|
|
not even a trailing `/`, and no `*`. Any other entry stops the server from
|
|
starting, with an error naming `CORS_ALLOWED_ORIGINS`.
|
|
|
|
### Metrics
|
|
|
|
With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves
|
|
Prometheus metrics at `GET /metrics` to requests with those as their basic auth
|
|
credentials, and answers any other with 401. For each request that reaches the
|
|
health check or `POST /api/v1/reports`, those the rate limit refuses included,
|
|
the metrics record its duration and response size, labelled with its path,
|
|
method and status; they also count those requests in progress, and include Go's
|
|
runtime and process metrics. No other request is recorded: not those to
|
|
`/metrics` itself, and not those answered before they reach either route, such
|
|
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
|
for a method its route does not take, or 413 for declaring a body length over
|
|
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
|
length reaches the route, is answered 413 there, and is recorded with that
|
|
status. Clients can make up any number of paths and methods, and each would add
|
|
labels to the metrics for as long as the server runs. With neither set, nothing
|
|
is recorded and `/metrics` answers 404. One without the other stops the server
|
|
from starting, with an error naming both; so does a `METRICS_USERNAME`
|
|
containing `:`, which basic auth cannot carry, with an error naming it.
|
|
|
|
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
|
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
|
minute, whatever their credentials. Past that it gets 429 with
|
|
`Retry-After: 60`, and its credentials are not checked. The minute slides as it
|
|
does for reports (see [Report limits](#report-limits)), so a scraper polling
|
|
every 2 seconds or less often is never refused. This allowance is apart from the
|
|
one for reports.
|
|
|
|
### Sentry
|
|
|
|
With `SENTRY_DSN` set, the server sends its errors to that Sentry project: each
|
|
panic in a handler is reported there, under the release `netwatch-server-`
|
|
followed by the server's version, and the request still gets 500 from the
|
|
server's panic recovery. On shutdown the server waits up to 2 seconds for Sentry
|
|
to finish sending. A DSN Sentry refuses stops the server from starting, with an
|
|
error naming `SENTRY_DSN`. With it empty, Sentry is not set up, and nothing is
|
|
sent to it.
|
|
|
|
## TODO
|
|
|
|
The to-do list, this backend's open work included, is [TODO.md](../TODO.md) at
|
|
the repo root.
|
|
|
|
## License
|
|
|
|
MIT. See [LICENSE](LICENSE).
|
|
|
|
## Author
|
|
|
|
[@sneak](https://sneak.berlin)
|