clawbot
2a47fb9a8e
check / check (push) Waiting to run
The backend's own TRUSTED_PROXIES now follows the ruling for every app: unset, it trusts 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16, and no longer loopback; a list replaces that; set and empty, it trusts no proxy. viper reads an empty environment variable as unset, so an empty value got the default. config.New now reads this one setting with os.LookupEnv first. viper's AllowEmptyEnv would apply to every setting, so an empty PORT or DEBUG would stop the start instead of counting as unset, as README.md says it does. bin/entrypoint.sh still starts the backend with TRUSTED_PROXIES=127.0.0.1/32. Model: opus-5-5
518 lines
28 KiB
Markdown
518 lines
28 KiB
Markdown
NetWatch is an MIT-licensed JavaScript single-page application by
|
||
[@sneak](https://sneak.berlin) that provides real-time network latency
|
||
monitoring to common internet hosts, displayed with color-coded figures and
|
||
sparkline graphs, served from a static bucket or from its Docker image, where a
|
||
small Go backend stores the measurements the page reports.
|
||
|
||
## Getting Started
|
||
|
||
```bash
|
||
# Install the dependencies and the git pre-commit hook
|
||
make setup
|
||
|
||
# Run the page on the Vite dev server
|
||
make dev
|
||
|
||
# Build netwatch-server and run it on port 8080
|
||
make run
|
||
|
||
# Run the tests, both linters and the format check
|
||
make check
|
||
|
||
# Build the page into dist/ and netwatch-server into the repo root
|
||
make build
|
||
|
||
# Build the image and run it
|
||
make docker
|
||
docker run -p 8080:8080 netwatch
|
||
```
|
||
|
||
`make check` and `make docker` need Docker. `make dev` passes `/api` to
|
||
`http://127.0.0.1:8080`, where `make run` starts `netwatch-server` with its
|
||
defaults, so the reports the page posts are stored in `data/reports`.
|
||
|
||
## Entrypoints
|
||
|
||
This repository adheres to the
|
||
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
||
standard: normalized scripts in `script/` are the entrypoints for the
|
||
development workflow, and the Makefile targets are thin shims that call them.
|
||
The scripts cover both the frontend and the Go backend, so `make check` fails if
|
||
either one is broken. We provide:
|
||
|
||
- `script/bootstrap` — install all dependencies (the pinned node via nvm unless
|
||
one new enough for the frontend's dependencies is installed, yarn via
|
||
corepack, `yarn install --frozen-lockfile`, Go `GO_VERSION` unless the `go` on
|
||
`PATH` is exactly that version, and the Go modules), linking what it installs
|
||
itself into `~/.local/bin`, which has to be on `PATH`. It installs no Go
|
||
linter and not Docker: `make test` and `make lint` run in Docker
|
||
- `script/setup` — make a fresh clone ready for development: bootstrap plus the
|
||
git pre-commit hook
|
||
- `script/dev` — run the Vite dev server, which proxies `/api` to a locally
|
||
running `netwatch-server`
|
||
- `script/build` — build the frontend for production into `dist/`, then the
|
||
static `netwatch-server` binary into the repo root with its version stamped
|
||
in: `VERSION` from the environment, or when that is unset or empty,
|
||
`git describe` inside a git checkout, then `dev`
|
||
- `script/run` — build with `script/build`, then run `netwatch-server` with its
|
||
defaults (see [Settings](#settings))
|
||
- `script/clean` — remove what `script/build` writes
|
||
- `script/projectname` — print the project name (used for the Docker image tag)
|
||
- `script/test` — build the `test` phase of `Dockerfile` without the cache: the
|
||
frontend's unit tests and production build in its `frontend` stage, and the
|
||
backend's Go tests with the race detector and coverage
|
||
- `script/lint` — build the `lint` phase of `Dockerfile` without the cache:
|
||
eslint in its `frontend-lint` stage, and golangci-lint over the Go packages
|
||
- `script/fmt` — format all files (writes): prettier over the JavaScript, CSS,
|
||
HTML and Markdown, then gofmt over `cmd/` and `internal/`. It runs on the
|
||
host, as `script/fmt-check` does, with `~/.local/bin` put on `PATH`
|
||
- `script/fmt-check` — check formatting (read-only): prettier, then gofmt
|
||
- `script/check` — run test, lint, and fmt-check
|
||
- `script/add-dependency` — add a frontend package, or move one to another
|
||
version: `make add-dependency PACKAGE=<name>@<version>` runs `yarn add --dev`,
|
||
which changes `package.json` and `yarn.lock` together, then
|
||
`yarn install --frozen-lockfile`
|
||
- `script/tidy` — run `go mod tidy`: to add a Go module, import it and run
|
||
`make tidy`; to move one to another version, edit its `require` line in
|
||
`go.mod`, then run `make tidy`
|
||
- `script/frontend-test` — run the unit tests in `test/unit/` on the host with
|
||
Node's built-in test runner, through the `test` script in `package.json`, and
|
||
if any fails, run them again listing every test, and fail; then the production
|
||
build. Each test run has a 90-second timeout. `make test` runs the same in
|
||
Docker
|
||
- `script/frontend-fmt` — format the JavaScript, CSS, HTML and Markdown with
|
||
prettier (writes)
|
||
- `script/frontend-fmt-check` — check prettier formatting (read-only)
|
||
- `script/frontend-viewport-test` — responsive-layout verification of the built
|
||
frontend in a containerised headless Chrome (see
|
||
[test/viewport/README.md](test/viewport/README.md)). Not part of
|
||
`script/check`: it takes minutes.
|
||
- `script/docker` — build the image from `Dockerfile` without the build cache,
|
||
tagged `netwatch` via `script/projectname`
|
||
- `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`,
|
||
then builds the image as `script/docker` does, without the build cache
|
||
- `script/precommit` — run by the git pre-commit hook; runs `script/check`
|
||
- `script/install-precommit` — install the git pre-commit hook
|
||
|
||
## Responsive layout
|
||
|
||
The narrow-viewport layout lives in the `max-width: 768px` media block in
|
||
`web/src/styles.css`. It is verified automatically by
|
||
`make frontend-viewport-test`, which drives a digest-pinned headless Chrome
|
||
against the built `dist/` and asserts on computed layout at widths derived from
|
||
that CSS — on every breakpoint it declares and one pixel either side of it, plus
|
||
a 320px floor, a desktop baseline and two landscape sizes. See
|
||
[test/viewport/README.md](test/viewport/README.md) for what it covers and what
|
||
it genuinely cannot.
|
||
|
||
## Rationale
|
||
|
||
When debugging network issues, it's useful to have a persistent at-a-glance view
|
||
of latency and reachability to multiple well-known internet endpoints. NetWatch
|
||
provides this as a single page that does all its measuring in the browser, so it
|
||
can be served from anywhere static files are served. The backend in its Docker
|
||
image only stores the measurements the page reports; without it, the page works
|
||
the same and nothing is stored.
|
||
|
||
## Design
|
||
|
||
The page's source is in `web/`, the Go backend's in `cmd/` and `internal/`, and
|
||
nginx's configuration in `configs/`.
|
||
|
||
The page is built with Vite and Tailwind CSS v4. Its code is all in
|
||
`web/src/main.js`, with a class-based architecture:
|
||
|
||
- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks,
|
||
etc.). The interval menu sets `updateInterval`, the one value the page writes
|
||
into `CONFIG`; the timeouts, the time the history spans and the x-axis ticks
|
||
are computed from it
|
||
- **`HostState`**: Per-host state management — history buffer, latency tracking,
|
||
status transitions
|
||
- **`AppState`**: Top-level state container — WAN hosts, local hosts, pause
|
||
state, aggregate stats
|
||
- **`SparklineRenderer`**: Canvas 2D sparkline drawing with fixed axes,
|
||
color-coded line segments, error regions, and DPR-aware scaling
|
||
- **UI functions**: `buildUI()` constructs the DOM, `updateHostRow()` /
|
||
`updateSummary()` / `updateHealthBox()` handle incremental updates
|
||
- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's
|
||
sample and redrawing its row as soon as its check ends, then redraws every
|
||
row, the summary and the health box once the last check ends. The first round,
|
||
after loading or an interval change, is discarded. The rows are sorted when
|
||
the last check ends in round 2, the first one kept, and in rounds 11, 21, 31
|
||
and so on. When paused, pushes blank markers (no probes, no false outage)
|
||
- **`Reporter`**: Posts collected samples to the backend
|
||
|
||
### Reporting
|
||
|
||
Every `reportInterval` (default 60s) the page POSTs a JSON report to the
|
||
same-origin path `/api/v1/reports`: a random per-browser `clientId` kept in
|
||
`localStorage`, `geo` sent as null, and each host's unreported, non-paused
|
||
samples (timestamp, latency, error). A per-host high-water mark makes every
|
||
report a delta, so only new samples are sent; the mark advances only on a
|
||
delivered report, and while paused nothing is sent. Changing the interval clears
|
||
each host's history on the page; the samples in it not yet reported still go in
|
||
the next report. Delivery failure is quiet — one debug-log line per outage,
|
||
retried at the next interval, never blocking probing. The report-building step
|
||
is a pure function of host state.
|
||
|
||
### Backend
|
||
|
||
`netwatch-server` is a small Go HTTP server that stores the reports the page
|
||
posts. It keeps them in memory and writes them to `DATA_DIR` as zstd-compressed
|
||
files of JSON lines: every minute, whenever 10 MiB are waiting, and when it
|
||
stops. Its routes:
|
||
|
||
- `POST /api/v1/reports` — takes a report, without credentials; each client
|
||
address may send a limited number a minute, and the report files are capped in
|
||
size, the oldest deleted first
|
||
- `GET /.well-known/healthcheck` — answers 200 with `"status":"ok"`, the
|
||
server's version and its uptime
|
||
- `GET /metrics` — Prometheus metrics behind basic auth, only when
|
||
`METRICS_USERNAME` and `METRICS_PASSWORD` are set; each client address may
|
||
make a limited number of requests to it a minute
|
||
|
||
It is an `fx`-wired Go application: `cmd/netwatch-server/` starts it, and its
|
||
packages are in `internal/`:
|
||
|
||
- **`config`**: Loads configuration from environment variables and config files
|
||
via Viper.
|
||
- **`handlers`**: HTTP request handlers for the API (health check, report
|
||
ingestion).
|
||
- **`reportbuf`**: The in-memory buffer of reports and the files it writes them
|
||
to.
|
||
- **`server`**: Chi-based HTTP server with middleware wiring and route
|
||
registration.
|
||
- **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting
|
||
infrastructure.
|
||
|
||
In the image, the `test` phase of `Dockerfile` tests it, the `builder` stage
|
||
builds it, and `bin/entrypoint.sh` runs it as user `netwatch` on
|
||
`127.0.0.1:8081`, behind nginx (see [Deployment](#deployment)). Outside the
|
||
image, `make run` builds it and runs it on port 8080.
|
||
|
||
#### Settings
|
||
|
||
| Variable | Default | Description |
|
||
| ---------------------- | -------------------- | -------------------------------------------------------------------------------------------------------- |
|
||
| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface |
|
||
| `PORT` | `8080` | HTTP listen port |
|
||
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
|
||
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) |
|
||
| `DEBUG` | `false` | Enable debug logging |
|
||
| `TRUSTED_PROXIES` | RFC 1918 ranges | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
|
||
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
|
||
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
||
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
|
||
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
|
||
| `SENTRY_DSN` | empty | DSN of the Sentry project to send errors to; see [Sentry](#sentry) |
|
||
|
||
`TRUSTED_PROXIES` defaults to `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. A list
|
||
replaces the default; set and empty, the server trusts no proxy. Loopback is not
|
||
in the default, so a reverse proxy on the same host has to be named, as in
|
||
`TRUSTED_PROXIES=127.0.0.1/32`. A request whose direct peer is outside this set
|
||
has its forwarded headers ignored, and the direct peer is logged and
|
||
rate-limited instead. The container image does not use this default; see
|
||
[Deployment](#deployment).
|
||
|
||
A variable set to a value the server cannot use, such as `PORT=abc`,
|
||
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
|
||
starting, with an error naming the variable. An empty variable counts as unset,
|
||
except `TRUSTED_PROXIES`.
|
||
|
||
#### Report storage
|
||
|
||
Reports are written as `reports-<timestamp>-<number>.jsonl.zst` files in
|
||
`DATA_DIR`. The timestamp is in UTC to the millisecond, so the names sort by
|
||
time. The number starts at 1 when the server starts and goes up by one for each
|
||
file the server starts to write, so two files written in the same millisecond
|
||
still get different names. A failed write uses up its number and leaves a gap in
|
||
the numbers: its file, if it was created, is removed. The file stays, counted
|
||
toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too.
|
||
Files are created with `O_EXCL` to prevent overwrites.
|
||
|
||
The reports of a failed write stay in memory, and the next flush, within a
|
||
minute, tries again to write them. From the failed write until a write succeeds,
|
||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
||
the page keeps its samples and sends them again at its next report. Reports
|
||
already answered 200 are lost only if the server stops before a write succeeds;
|
||
the final flush at stop tries them too, and if it fails, the process exits
|
||
non-zero.
|
||
|
||
#### Report limits
|
||
|
||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||
credentials, so it is bounded instead. Each refusal below answers with the same
|
||
`{"status":"error"}` body as any other error.
|
||
|
||
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
||
parameters such as `charset` allowed, as the page sends it; any other is
|
||
refused with 415, and nothing of it is stored. A page on another origin can
|
||
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
||
keeps any site from making its visitors' browsers store reports (see
|
||
[CORS](#cors)).
|
||
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
||
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
||
`Retry-After: 60`. The minute slides: reports from the minute before still
|
||
count, fading out over the current one, so an address is sure never to be
|
||
refused only while it sends at most half of `REPORTS_PER_MINUTE` in any 60
|
||
seconds. The page sends one report a minute from each open tab, so the default
|
||
of 60 refuses nothing from up to 30 tabs behind one address, such as a
|
||
household or an office sharing it, however their reports bunch up. Report
|
||
responses also carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and
|
||
`X-RateLimit-Reset` headers.
|
||
- **Size cap.** The report files in `DATA_DIR` may total at most
|
||
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
|
||
waiting in memory count at their uncompressed size until they are written,
|
||
those of a failed write included; the part of its file written is removed.
|
||
When a report would take the total past the cap, the oldest report files are
|
||
deleted to make room, and each deletion is logged with the file's name and
|
||
size; a file still being written is never deleted. A report is refused with
|
||
507, and nothing of it is stored, only when the reports waiting to be written
|
||
fill the cap on their own, and then no file is deleted. At start, report files
|
||
past the cap, as after lowering it, are deleted the same way. So the cap is
|
||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||
for any host; set it to the space you can give `DATA_DIR`.
|
||
|
||
#### CORS
|
||
|
||
The page calls the API from the origin it is served from, so by default the
|
||
server sends no CORS headers, and browsers let no other origin's pages call it.
|
||
To serve the page from elsewhere, list that origin in `CORS_ALLOWED_ORIGINS`
|
||
(for example `https://netwatch.example.com`); pages from a listed origin may
|
||
`GET` and `POST` with a `Content-Type` header. Each entry must be a plain
|
||
origin, `scheme://host` with an optional `:port`, as browsers send it: no path,
|
||
not even a trailing `/`, and no `*`. Any other entry stops the server from
|
||
starting, with an error naming `CORS_ALLOWED_ORIGINS`.
|
||
|
||
#### Metrics
|
||
|
||
With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves
|
||
Prometheus metrics at `GET /metrics` to requests with those as their basic auth
|
||
credentials, and answers any other with 401. For each request that reaches the
|
||
health check or `POST /api/v1/reports`, those the rate limit refuses included,
|
||
the metrics record its duration and response size, labelled with its path,
|
||
method and status; they also count those requests in progress, and include Go's
|
||
runtime and process metrics. No other request is recorded: not those to
|
||
`/metrics` itself, and not those answered before they reach either route, such
|
||
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
||
for a method its route does not take, or 413 for declaring a body length over
|
||
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
||
length reaches the route, is answered 413 there, and is recorded with that
|
||
status. Clients can make up any number of paths and methods, and each would add
|
||
labels to the metrics for as long as the server runs.
|
||
|
||
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
||
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
||
minute, whatever their credentials. Past that it gets 429 with
|
||
`Retry-After: 60`, and its credentials are not checked. The minute slides as it
|
||
does for reports (see [Report limits](#report-limits)), so a scraper polling
|
||
every 2 seconds or less often is never refused. This allowance is apart from the
|
||
one for reports.
|
||
|
||
#### Sentry
|
||
|
||
With `SENTRY_DSN` set, each panic in a handler is reported to that Sentry
|
||
project under the release `netwatch-server-` followed by the server's version.
|
||
On shutdown the server waits up to 2 seconds for Sentry to finish sending.
|
||
|
||
### Monitoring targets
|
||
|
||
- **26 WAN hosts**: datavi.be (pinned at start), Anthropic API, OpenAI API, AWS
|
||
Console, Google Cloud Console, Microsoft Azure, Cloudflare, Fastly CDN,
|
||
Akamai, Google, GitHub, B2, 8 S3 regional endpoints (Cape Town, London,
|
||
Bahrain, Tokyo, Singapore, Sydney, Oregon, São Paulo) and 6 Hetzner speed test
|
||
servers (Nuremberg, Falkenstein, Helsinki, Ashburn, Hillsboro, Singapore)
|
||
- **Local CPE**: Cable modem at 192.168.100.1 (always monitored)
|
||
- **Local Gateway**: Auto-detected on startup by probing common default gateway
|
||
addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder
|
||
wins. Note: modern browsers enforce Private Network Access restrictions that
|
||
block public-origin pages from reaching RFC1918 addresses, so local targets
|
||
only work when NetWatch is served from localhost or a private address.
|
||
|
||
Local hosts are tracked separately from WAN stats.
|
||
|
||
### Latency measurement
|
||
|
||
GET requests to each target's URL as written, with `mode: 'no-cors'` and
|
||
`cache: 'no-store'`, timed with `performance.now()`. No query string is added:
|
||
the Hetzner speed-test servers close the connection without an answer when the
|
||
URL has one, and `no-store` keeps the browser's cache out of the measurement.
|
||
Each check times out after 80% of the refresh interval (24 seconds at 30
|
||
seconds) and is then recorded as a timeout, so a round's checks have all
|
||
finished before the next round is due. When no WAN host answers, a recovery
|
||
probe checks 4 WAN hosts, picked at random when it starts, every half second,
|
||
giving up the checks it started half a second before. As soon as one answers, a
|
||
new round starts at once, as it does after an interval change. A round started
|
||
early gives up the last round's checks if they are still waiting, and that round
|
||
records nothing more, so rounds never overlap. The browser chooses between IPv4
|
||
and IPv6 for each target, as for any request; the local targets are IPv4
|
||
addresses.
|
||
|
||
Each recorded check that fails writes one line to the browser console with
|
||
`console.error`, and the same line to the debug log: the target's name and URL,
|
||
the time, whether it timed out, answered over the time limit or hit a network
|
||
error (with the error the browser gives the page, such as
|
||
`TypeError: Failed to fetch`, which does not say why), and how long the request
|
||
took. A target that answers after failed checks writes one `console.info` line
|
||
with its latency and how many checks in a row had failed.
|
||
|
||
### Color coding
|
||
|
||
| Latency | Color |
|
||
| ----------- | ------ |
|
||
| < 50ms | Green |
|
||
| < 100ms | Lime |
|
||
| < 200ms | Yellow |
|
||
| < 500ms | Orange |
|
||
| >= 500ms | Red |
|
||
| Unreachable | Gray |
|
||
|
||
### Output structure
|
||
|
||
```
|
||
dist/
|
||
├── index.html
|
||
└── assets/
|
||
├── index-*.css
|
||
└── index-*.js
|
||
```
|
||
|
||
## Features
|
||
|
||
- A round of checks every 3 seconds by default; the interval menu sets 1, 2, 3,
|
||
5, 10, 15, 30 or 60 seconds and clears the history
|
||
- Sparklines of each target's last 100 rounds: 300 seconds at 3 seconds
|
||
- The first round after loading or an interval change is discarded, as DNS and
|
||
TLS setup inflate its latencies
|
||
- Health indicator from the WAN hosts' latest results: OFFLINE (red) when more
|
||
than 10 fail and at most 4 answer, otherwise DEGRADED (orange) when more than
|
||
4 fail, otherwise SLOW (yellow) when more than 3 take over 1000ms, otherwise
|
||
HEALTHY (green)
|
||
- Summary stats across WAN hosts only: how many answered, the min, median,
|
||
average and max of their latest latencies, the min and max over the whole
|
||
history, and the number of rounds run (`Checks`)
|
||
- Fixed chart axes: Y-axis 0–1000ms, higher latencies drawn at the top; X-axis
|
||
the time the history spans
|
||
- Color-coded latency figures and sparkline line segments
|
||
- WAN host rows sorted by latest latency, unreachable last; pinned rows stay on
|
||
top, in name order
|
||
- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no
|
||
false outage)
|
||
- Debug log panel, behind a checkbox in the footer, with five levels (error,
|
||
warning, notice, info, debug) and the last 1000 lines
|
||
- Local and UTC clocks
|
||
- Clickable service URLs
|
||
- A footer link to the commit the page was built from
|
||
- Canvas-based sparkline rendering with devicePixelRatio scaling
|
||
- Zero runtime dependencies: all resources bundled into build artifacts
|
||
|
||
## Deployment
|
||
|
||
`make build` writes the page to `dist/`, which any static file host (S3, GCS,
|
||
Cloudflare Pages, Vercel, Netlify, GitHub Pages) can serve; with no backend
|
||
there, its reports fail quietly and nothing is stored. Or run the Docker image
|
||
behind a reverse proxy.
|
||
|
||
The Docker image, built from `Dockerfile` by `make docker`, is the whole service
|
||
in one container: nginx serves the built frontend and passes `/api/`,
|
||
`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`.
|
||
`bin/entrypoint.sh` starts it listening only inside the container, on
|
||
`127.0.0.1:8081`, with `TRUSTED_PROXIES=127.0.0.1/32`, so it takes the client
|
||
address nginx passes on and no other. The image:
|
||
|
||
- Listens on port 8080 by default (override with `PORT` env var)
|
||
- Takes the client address from `X-Forwarded-For` only on requests from the
|
||
reverse proxies named in `TRUSTED_PROXIES`, by default from any address in
|
||
`10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`. `bin/entrypoint.sh` checks
|
||
each entry with `netwatch-server check-cidr`, which parses it as the backend
|
||
parses its own `TRUSTED_PROXIES`
|
||
- Sends access logs to stdout
|
||
- Caches static assets with immutable headers
|
||
- Sends the security headers `REPO_POLICIES.md` requires on every response, as
|
||
`configs/security-headers.conf` sets them, in place of the backend's own
|
||
- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
|
||
volume. Before the backend starts, the image creates `DATA_DIR` and gives it
|
||
and `/data` to user `netwatch` (uid 1000), which the backend runs as, with
|
||
`netwatch-server prepare-data-dir`, which acts on nothing outside `/data`, so
|
||
a host directory bind-mounted at `/data` ends up owned by uid 1000
|
||
- Writes buffered reports to disk on `docker stop`, and exits non-zero if nginx
|
||
or the backend exits on its own, so the platform restarts it
|
||
|
||
## Running under upaas
|
||
|
||
What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
||
|
||
- **Port:** container port `8080`.
|
||
- **Volume:** container path `/data`; the reports are kept in `/data/reports`.
|
||
- **Environment variables:** none is required. An empty one counts as unset,
|
||
except `TRUSTED_PROXIES`, and one set to a value netwatch cannot use stops the
|
||
container at start, with the reason in its log.
|
||
- `PORT`, default `8080`: the container port, from 1 to 65535. `8081` cannot
|
||
be used: the backend listens on it inside the container
|
||
- `REPORTS_PER_MINUTE`, default `60`: reports each client address may send a
|
||
minute
|
||
- `DATA_DIR_MAX_BYTES`, default `1073741824` (1 GiB): the most room the
|
||
report files may take; the oldest are deleted to stay under it
|
||
- `CORS_ALLOWED_ORIGINS`, default empty: other origins whose pages may call
|
||
the API
|
||
- `DEBUG`, default `false`: debug logging
|
||
- `DATA_DIR`, default `/data/reports`: the directory the reports are kept
|
||
in: `/data` or a path below it, with no `.` or `..` part and no extra `/`.
|
||
The container also stops if the path goes through a symbolic link that
|
||
leads out of `/data` or is written as a full path
|
||
- `TRUSTED_PROXIES`, default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`: the
|
||
addresses the reverse proxy in front of the container connects from, as IP
|
||
addresses or CIDRs separated by commas. nginx takes the client address
|
||
from `X-Forwarded-For` only on a request from one of them, and the rate
|
||
limit counts that address. A list replaces the default. Set and empty,
|
||
`X-Forwarded-For` is ignored and every client behind the proxy shares the
|
||
proxy's one allowance of `REPORTS_PER_MINUTE`. Any client that connects
|
||
from one of these addresses can write its own `X-Forwarded-For`, and
|
||
through a port Docker publishes, every client may connect from the Docker
|
||
network's gateway, such as `172.17.0.1`, which the default includes; where
|
||
that matters, name only the addresses nothing but the proxy connects from.
|
||
- `METRICS_USERNAME` and `METRICS_PASSWORD`, default empty: with both set,
|
||
the backend records Prometheus metrics of its requests and serves them at
|
||
`/metrics` on the container port, to requests with this user name and
|
||
password as their basic auth credentials. With neither set, there are no
|
||
metrics and `/metrics` is not found. One set without the other, or a user
|
||
name containing `:`, stops the container
|
||
- `SENTRY_DSN`, default empty: set to a Sentry project's DSN, the backend
|
||
sends its errors to that Sentry project: each request whose handling
|
||
crashes, which still gets a 500 response. A value Sentry does not accept
|
||
stops the container. Empty, the backend sends nothing to Sentry
|
||
- **Health check:** the image's `HEALTHCHECK` requests
|
||
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
|
||
both nginx and the backend answer. upaas reads the container's health 60
|
||
seconds after a deploy and fails the deploy unless it is `healthy`. The
|
||
container also stops when either process exits.
|
||
|
||
## Browser Compatibility
|
||
|
||
Requires a modern browser with ES modules, Fetch API, Canvas API, and CSS custom
|
||
properties.
|
||
|
||
## Limitations
|
||
|
||
- **CORS**: The checks are cross-origin requests in `no-cors` mode, so the page
|
||
cannot read the answer, only time it: any answer counts as reachable, an error
|
||
page included.
|
||
- **Local targets**: The cable modem at 192.168.100.1 and the detected gateway
|
||
answer only on a network that has them, and only when NetWatch is served from
|
||
localhost or a private address (see Monitoring targets).
|
||
- **Network conditions**: Measurements reflect browser-to-endpoint latency,
|
||
which includes your local network, ISP, and internet routing.
|
||
|
||
## TODO
|
||
|
||
The to-do list is [TODO.md](TODO.md): where the work stands, the next step, the
|
||
open work, and what has been done.
|
||
|
||
## License
|
||
|
||
MIT. See [LICENSE](LICENSE).
|
||
|
||
## Author
|
||
|
||
[@sneak](https://sneak.berlin)
|