Compare commits
1
Commits
next
..
1a7966b97a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1a7966b97a |
+1
-4
@@ -73,12 +73,9 @@
|
||||
|
||||
# This repository's own entries, after the shared content above: the
|
||||
# frontend build, the viewport test's output, the binary `make build`
|
||||
# writes and the reports `make run` stores, and Go test binaries,
|
||||
# coverage output and log files at the root.
|
||||
# writes and the reports `make run` stores, and log files at the root.
|
||||
/dist
|
||||
/tmp
|
||||
/netwatch-server
|
||||
/data
|
||||
/*.test
|
||||
/*.out
|
||||
/*.log
|
||||
|
||||
+2
-3
@@ -11,8 +11,7 @@ insert_final_newline = true
|
||||
[Makefile]
|
||||
indent_style = tab
|
||||
|
||||
# This repository's own entries, after the shared content above.
|
||||
|
||||
[*.go]
|
||||
indent_style = tab
|
||||
|
||||
# This repository's own sections, such as one for another language it
|
||||
# uses, go below this comment, and a re-vendor keeps them.
|
||||
|
||||
@@ -1,20 +1,9 @@
|
||||
name: check
|
||||
on: [push]
|
||||
# Free the shared runner: a new push cancels only the same branch's older run.
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
check:
|
||||
runs-on: ubuntu-latest
|
||||
# Free the shared runner from a hung build.
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
# actions/checkout v4.2.2, 2026-02-22
|
||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||
# script/cibuild needs no token, so none is left in .git/config.
|
||||
with:
|
||||
persist-credentials: false
|
||||
# All history and tags, so git describe finds the version tag.
|
||||
fetch-depth: 0
|
||||
- run: script/cibuild
|
||||
|
||||
+2
-4
@@ -27,7 +27,7 @@ node_modules/
|
||||
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
||||
# convention. Only the templates `example.env` and `sample.env` are
|
||||
# re-included below. A repository that commits any other template adds
|
||||
# its own negation at the end of this file, for example `!.env.example`.
|
||||
# its own negation after these lines, for example `!.env.example`.
|
||||
*.[eE][nN][vV]
|
||||
.[eE][nN][vV].*
|
||||
.[eE][nN][vV][rR][cC]
|
||||
@@ -46,9 +46,7 @@ node_modules/
|
||||
[iI][dD]_[eE][dD]25519
|
||||
[iI][dD]_[eE][dD]25519_[sS][kK]
|
||||
|
||||
# This repository's own entries, such as its build outputs, go below
|
||||
# this comment, and a re-vendor keeps them. Anchor a binary built at the
|
||||
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
|
||||
# This repository's own entries, after the shared content above.
|
||||
|
||||
# Build output
|
||||
dist/
|
||||
|
||||
@@ -25,8 +25,6 @@ linters:
|
||||
# silenced by disabling that name, not by enabling the successor.
|
||||
- wsl # Deprecated, replaced by wsl_v5
|
||||
- gomodguard # Deprecated, replaced by gomodguard_v2
|
||||
# Misses findings at random in v2.14.0; back once a pinned release fixes it
|
||||
- canonicalheader
|
||||
settings:
|
||||
lll:
|
||||
line-length: 88
|
||||
|
||||
@@ -1,17 +1,8 @@
|
||||
NetWatch is an MIT-licensed single-page web application by
|
||||
[@sneak](https://sneak.berlin) that shows the latency from your browser to 26
|
||||
well-known internet hosts. The page is JavaScript and does all its measuring in
|
||||
the browser. A small Go backend, `netwatch-server`, stores the measurements the
|
||||
page reports. The page and the backend ship together in one Docker image.
|
||||
|
||||
## Rationale
|
||||
|
||||
When a network misbehaves, it helps to see at a glance which hosts answer and
|
||||
how fast. NetWatch is one page with a latency figure and a small graph for each
|
||||
host, refreshed every few seconds. The page measures from the browser, so it
|
||||
shows the path your own traffic takes. It can be served from any static file
|
||||
host. The backend only stores what the page reports. Without it, the page works
|
||||
the same and nothing is stored.
|
||||
NetWatch is an MIT-licensed JavaScript single-page application by
|
||||
[@sneak](https://sneak.berlin) that provides real-time network latency
|
||||
monitoring to common internet hosts, displayed with color-coded figures and
|
||||
sparkline graphs, served from a static bucket or from its Docker image, where a
|
||||
small Go backend stores the measurements the page reports.
|
||||
|
||||
## Getting Started
|
||||
|
||||
@@ -19,10 +10,10 @@ the same and nothing is stored.
|
||||
# Install the dependencies and the git pre-commit hook
|
||||
make setup
|
||||
|
||||
# Serve the page from the Vite dev server
|
||||
# Run the page on the Vite dev server
|
||||
make dev
|
||||
|
||||
# Build the page and netwatch-server, then run the server on port 8080
|
||||
# Build netwatch-server and run it on port 8080
|
||||
make run
|
||||
|
||||
# Run the tests, both linters and the format check
|
||||
@@ -31,417 +22,557 @@ make check
|
||||
# Build the page into dist/ and netwatch-server into the repo root
|
||||
make build
|
||||
|
||||
# Build the Docker image and run it
|
||||
# Build the image and run it
|
||||
make docker
|
||||
docker run -p 8080:8080 netwatch
|
||||
```
|
||||
|
||||
`make check` and `make docker` need Docker. `make run` starts `netwatch-server`
|
||||
on port 8080 with its defaults, so the page's reports are stored in
|
||||
`data/reports`. `make dev` passes the page's `/api` requests there.
|
||||
`make check` and `make docker` need Docker. `make dev` passes `/api` to
|
||||
`http://127.0.0.1:8080`, where `make run` starts `netwatch-server` with its
|
||||
defaults, so the reports the page posts are stored in `data/reports`.
|
||||
|
||||
## The page
|
||||
|
||||
The page shows one row per host: the host's name, its URL, its latest latency in
|
||||
milliseconds and a graph of its last 100 checks. Below the figure are the
|
||||
minimum, median, average and maximum of the host's answers in that history, or
|
||||
`timeout` or `unreachable` after a failed check. A round of checks runs every 3
|
||||
seconds by default. The header shows the interval, the history's span, and
|
||||
whether the page is running or paused.
|
||||
|
||||
### Targets
|
||||
|
||||
The page checks 26 hosts on the internet, the WAN hosts:
|
||||
|
||||
- `datavi.be`, pinned to the top at start
|
||||
- the Anthropic API and the OpenAI API
|
||||
- the AWS Console, the Google Cloud Console and Microsoft Azure
|
||||
- Cloudflare, Fastly, Akamai, Google, GitHub and Backblaze B2
|
||||
- 8 S3 regional endpoints: Cape Town, London, Bahrain, Tokyo, Singapore, Sydney,
|
||||
Oregon and São Paulo
|
||||
- 6 Hetzner speed-test servers: Nuremberg, Falkenstein, Helsinki, Ashburn,
|
||||
Hillsboro and Singapore
|
||||
|
||||
Two local targets follow under the heading "Local Network", left out of the
|
||||
summary and the health box. `Local CPE` is the cable modem at
|
||||
`http://192.168.100.1`. `Local Gateway` is the first of `http://192.168.1.1`,
|
||||
`http://192.168.0.1`, `http://192.168.8.1` and `http://10.0.0.1` to answer
|
||||
within 1.5 seconds of the page loading; with no answer, it has no row. Browsers
|
||||
block a page served from a public address from reaching private addresses. So
|
||||
the local targets answer only when the page is served from `localhost` or from a
|
||||
private address.
|
||||
|
||||
The host list is written in `web/src/main.js`; there is no setting for it.
|
||||
|
||||
### What a figure measures
|
||||
|
||||
A check is a request to the target's URL: `GET`, or `HEAD` for the S3 regions.
|
||||
The figure is the time from sending the request until the response headers
|
||||
arrive, in milliseconds. Unlike ping, it includes the time the server takes to
|
||||
answer. On a new connection it also includes the DNS lookup, the connection
|
||||
setup and the TLS handshake. Any answer counts as reachable, an error page
|
||||
included. The browser follows redirects, so a target whose home page redirects
|
||||
is checked at `/robots.txt` on the same host.
|
||||
|
||||
A check times out after 80% of the interval: 2.4 seconds at the default. A check
|
||||
that answers after that limit counts as a timeout too. A check that fails with a
|
||||
network error is recorded as unreachable. The first round after the page loads
|
||||
is discarded, as its figures include the connection setup. So is the first round
|
||||
after an interval change.
|
||||
|
||||
### Health and summary
|
||||
|
||||
Above the rows, a health box sums up the WAN hosts' latest results. A host
|
||||
counts as failed when its last check timed out or found it unreachable.
|
||||
|
||||
- `OFFLINE` (red): more than 10 hosts failed and at most 4 answered
|
||||
- `DEGRADED` (orange): otherwise, more than 4 hosts failed
|
||||
- `SLOW` (yellow): otherwise, more than 3 hosts took over 1000 ms
|
||||
- `HEALTHY` (green): otherwise
|
||||
|
||||
Below it, a summary line covers the WAN hosts only:
|
||||
|
||||
- `Reachable`: how many of the 26 answered their last check
|
||||
- `Now`: the minimum, median, average and maximum of their latest figures
|
||||
- the history's span, such as `5m`: the minimum and maximum over the whole
|
||||
history
|
||||
- `Checks`: the rounds run since the page loaded or the interval changed
|
||||
|
||||
When no WAN host answers, the page checks 4 of them, picked at random, every
|
||||
half second. As soon as one answers, a new round starts.
|
||||
|
||||
### Rows, colors and controls
|
||||
|
||||
A figure and its line in the graph take the color of the legend in the footer,
|
||||
from green under 50 ms to red from 500 ms up; gray is no answer. The graph's
|
||||
axis ends at 1000 ms; a higher latency is drawn at the top, and the figure still
|
||||
shows its real value.
|
||||
|
||||
From time to time, the page sorts the WAN rows by their latest latency: fastest
|
||||
first, failed last. The pin button at the left of a row keeps the row at the
|
||||
top; pinned rows are sorted by name. `datavi.be` is pinned at start.
|
||||
|
||||
`Pause` stops the checks: the graphs keep scrolling with a gap for each paused
|
||||
round, and the figures turn gray. The `Interval` menu sets the time between
|
||||
rounds: 1, 2, 3, 5, 10, 15, 30 or 60 seconds. Changing it clears the history and
|
||||
starts a new round at once. The `Debug log` checkbox in the footer opens a panel
|
||||
with the page's last 1000 log lines.
|
||||
|
||||
## Reporting
|
||||
|
||||
Every 60 seconds the page posts its undelivered samples as a report to
|
||||
`/api/v1/reports` on the origin it was served from. The report is JSON with four
|
||||
fields:
|
||||
|
||||
- `clientId`: a random id, made once per browser and kept in `localStorage`, or
|
||||
once per session when the browser has no usable storage
|
||||
- `geo`: always `null`
|
||||
- `hosts`: one entry per host with new samples: its `name`, `url` and `status`,
|
||||
and `history`, its samples. A sample has the check's time `t` in Unix
|
||||
milliseconds, its `latency` (`null` for a failed check) and its `error`
|
||||
(`"timeout"`, `"unreachable"` or `null`).
|
||||
- `timestamp`: the time of the report
|
||||
|
||||
Each report carries only the samples no earlier report has delivered. A sample
|
||||
waits as long as the history holds it: the last 100 rounds of each host. An
|
||||
interval change clears the history. The cleared samples are kept, the last 100
|
||||
of each host, until a report delivers them. Nothing is sent while the page is
|
||||
paused. A report with no answer after 30 seconds is abandoned, and its samples
|
||||
go in the next one. So a server that stored them but answered late gets them
|
||||
twice.
|
||||
|
||||
A failed delivery is quiet, as on a static file host with no backend: one debug
|
||||
log line when deliveries start failing, one when they recover. The checks go on.
|
||||
|
||||
## Deployment
|
||||
|
||||
`make docker` builds the image `netwatch` from `Dockerfile`: the whole service
|
||||
in one container. nginx serves the page and passes `/api/`,
|
||||
`/.well-known/healthcheck` and `/metrics` to `netwatch-server` in the same
|
||||
container. The backend listens on `127.0.0.1:8081` only, as user `netwatch` (uid
|
||||
1000). `bin/entrypoint.sh` starts both and watches them.
|
||||
|
||||
Before the backend starts, `bin/entrypoint.sh` runs two subcommands of
|
||||
`netwatch-server`. `check-cidr` checks each `TRUSTED_PROXIES` entry (see
|
||||
[Running under upaas](#running-under-upaas)). `prepare-data-dir` creates the
|
||||
report directory and gives `/data`, with the directories, regular files and
|
||||
symbolic links in it, to user `netwatch`. It changes nothing outside `/data`.
|
||||
|
||||
The image also:
|
||||
|
||||
- takes the client address from `X-Forwarded-For` only on a request from a proxy
|
||||
named in `TRUSTED_PROXIES`, and passes that one address to the backend. The
|
||||
backend is started with `TRUSTED_PROXIES=127.0.0.1/32`, so it trusts nginx
|
||||
alone.
|
||||
- writes nginx's access log and the backend's JSON log to stdout
|
||||
- refuses a request body over 1 MiB with nginx's own 413 error page
|
||||
- requests `/.well-known/healthcheck` through nginx every 30 seconds, as its
|
||||
`HEALTHCHECK`. The check fails unless both nginx and the backend answer, and
|
||||
while report files cannot be written.
|
||||
- on `docker stop`, writes the waiting reports to disk and exits 0, or non-zero
|
||||
if that write fails. If nginx or the backend exits on its own, the other is
|
||||
stopped and the container exits non-zero.
|
||||
|
||||
### Running under upaas
|
||||
|
||||
The [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
||||
|
||||
- Port: container port `8080`.
|
||||
- Volume: container path `/data`.
|
||||
- Environment variables: none is required. Every setting in
|
||||
[Settings](#settings) applies, with four differences in the image:
|
||||
- `BIND_ADDRESS` has no effect. The backend listens on `127.0.0.1` inside
|
||||
the container.
|
||||
- `PORT` is nginx's public port, from 1 to 65535 and not `8081`, where the
|
||||
backend listens inside the container.
|
||||
- `DATA_DIR` must be `/data` or a path below it, written in full, with no
|
||||
`.` or `..` part and no extra `/`. A path through a symbolic link is
|
||||
refused unless the link is relative and stays inside `/data`.
|
||||
- `TRUSTED_PROXIES` names the proxies nginx trusts, as IP addresses or CIDRs
|
||||
separated by commas; the backend's own setting is fixed to `127.0.0.1/32`.
|
||||
`bin/entrypoint.sh` gives a lone address a `/32` or `/128` and checks each
|
||||
entry with `netwatch-server check-cidr`, which refuses what the backend
|
||||
would refuse. Set and empty, nginx trusts no proxy, and every client
|
||||
behind a proxy then shares the proxy's one allowance of
|
||||
`REPORTS_PER_MINUTE`. A client that connects from a trusted address can
|
||||
write its own `X-Forwarded-For`. A port Docker publishes can bring
|
||||
connections from the Docker network's gateway, such as `172.17.0.1`, which
|
||||
the default includes. Where that matters, name only the proxy's own
|
||||
addresses.
|
||||
- Health: the image's `HEALTHCHECK`, described above.
|
||||
|
||||
The container stops at start when a variable holds a value it cannot use. The
|
||||
reason is in its log.
|
||||
|
||||
## Settings
|
||||
|
||||
`netwatch-server` reads its settings from environment variables. It also reads a
|
||||
`.env` file in the directory it starts in, and a YAML file named
|
||||
`netwatch-server.yaml` in `/etc/netwatch-server/` or
|
||||
`~/.config/netwatch-server/`, when one exists. A variable in the environment
|
||||
wins over a file.
|
||||
|
||||
The server does not start when a variable holds a value it cannot use. Its error
|
||||
names the variable. An empty variable counts as unset, except `TRUSTED_PROXIES`.
|
||||
|
||||
- `BIND_ADDRESS`, default empty: the IP address to listen on. Empty listens on
|
||||
every interface. The image sets it to `127.0.0.1`.
|
||||
- `PORT`, default `8080`: the port to listen on, from 1 to 65535. In the image
|
||||
this is nginx's port; the backend listens on `8081`.
|
||||
- `DATA_DIR`, default `./data/reports` (`/data/reports` in the image): the
|
||||
directory for the report files. The server creates it if it is missing, and
|
||||
stops at start if it cannot create a file in it.
|
||||
- `DATA_DIR_MAX_BYTES`, default `1073741824` (1 GiB): the most bytes of report
|
||||
files kept in `DATA_DIR`, oldest deleted first. A positive whole number. See
|
||||
[Report limits](#report-limits).
|
||||
- `REPORTS_PER_MINUTE`, default `60`: how many reports each client address may
|
||||
send a minute. A positive whole number. See [Report limits](#report-limits).
|
||||
- `TRUSTED_PROXIES`, default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`: the
|
||||
reverse proxies whose `X-Forwarded-For` and `X-Real-IP` headers are trusted,
|
||||
as CIDRs separated by commas. See [Trusted proxies](#trusted-proxies).
|
||||
- `CORS_ALLOWED_ORIGINS`, default empty: other origins whose pages may call the
|
||||
API, separated by commas. See [CORS](#cors).
|
||||
- `METRICS_USERNAME` and `METRICS_PASSWORD`, default empty: the basic auth
|
||||
credentials for `/metrics`. See [Metrics](#metrics).
|
||||
- `SENTRY_DSN`, default empty: the DSN of a Sentry project. Each request whose
|
||||
handling crashes is reported there; the request still gets a 500.
|
||||
- `DEBUG`, default empty: any non-empty value turns debug logging on, `false`
|
||||
and `0` included. Every value is accepted.
|
||||
|
||||
### Trusted proxies
|
||||
|
||||
Each `TRUSTED_PROXIES` entry must be a CIDR, such as `127.0.0.1/32`; a lone
|
||||
address is refused. A list replaces the default; set and empty, the server
|
||||
trusts no proxy. Loopback is not in the default, so a reverse proxy on the same
|
||||
host has to be named.
|
||||
|
||||
A request from a peer outside this set has its forwarded headers ignored; the
|
||||
peer's own address is logged and rate limited. From a peer inside the set, the
|
||||
client address is the right-most `X-Forwarded-For` entry that is not in the set.
|
||||
The peer's address is used when every entry is in the set, or when the search
|
||||
from the right meets an entry that is not an address, such as `unknown`.
|
||||
`X-Real-IP` is read only when `X-Forwarded-For` is missing or empty.
|
||||
|
||||
### Routes
|
||||
|
||||
- `POST /api/v1/reports` takes a report from the page. It needs no credentials;
|
||||
[Report limits](#report-limits) bound it instead.
|
||||
- `GET /.well-known/healthcheck` answers 200 with a JSON object: `appname`,
|
||||
`version`, `now`, `uptime_seconds`, `uptime_human` and `status`, which is
|
||||
`"ok"`. While report files cannot be written, it answers 503 with the same
|
||||
fields and `"status":"error"`.
|
||||
- `GET /metrics` serves Prometheus metrics, only when `METRICS_USERNAME` and
|
||||
`METRICS_PASSWORD` are set. See [Metrics](#metrics).
|
||||
|
||||
Every request body is capped at 1 MiB, and every request at 60 seconds.
|
||||
|
||||
### Report storage
|
||||
|
||||
The server keeps the reports it takes in memory and writes them to `DATA_DIR` as
|
||||
files of JSON lines, compressed with zstd. It writes a new file every minute,
|
||||
whenever 10 MiB of reports are waiting, and when it stops. With none waiting, it
|
||||
writes no file. The files are named `reports-<timestamp>-<number>.jsonl.zst`;
|
||||
the timestamp is UTC to the millisecond, so the names sort by time.
|
||||
|
||||
The reports of a failed write, as on a full disk, stay in memory for the next
|
||||
write to try again. Until a write succeeds, `POST /api/v1/reports` refuses every
|
||||
report with 500 and the health check answers 503.
|
||||
|
||||
### Report limits
|
||||
|
||||
`POST /api/v1/reports` takes reports from anyone who can reach it, so it is
|
||||
bounded in four ways. Each refusal by the backend answers with the body
|
||||
`{"status":"error"}`.
|
||||
|
||||
- JSON only. A report must be sent with `Content-Type: application/json`; any
|
||||
other content type is refused with 415, and malformed JSON with 400.
|
||||
- Body limit. A report body may be at most 1 MiB. A request that declares a
|
||||
longer body, or a report that runs past that, is refused with 413.
|
||||
- Rate limit. Each client address may send `REPORTS_PER_MINUTE` reports a
|
||||
minute; past that it gets 429 with `Retry-After: 60`. The page sends one
|
||||
report a minute from each open tab, so the default of 60 refuses nothing from
|
||||
up to 30 tabs behind one address.
|
||||
- Size cap. The report files in `DATA_DIR` may total at most
|
||||
`DATA_DIR_MAX_BYTES`, counting the files already there at start and the
|
||||
reports waiting in memory. When a report would take the total past the cap,
|
||||
the oldest files are deleted to make room. A report is refused with 507 when
|
||||
deleting the oldest files cannot make room.
|
||||
|
||||
At the defaults, one address can fill the 1 GiB cap with its own reports in
|
||||
about 17 minutes, and every report stored before them is deleted to make room.
|
||||
|
||||
### CORS
|
||||
|
||||
The page needs no entry here: it calls the API from the origin it is served
|
||||
from. By default the server sends no CORS headers, and browsers let no other
|
||||
origin's pages call it. `CORS_ALLOWED_ORIGINS` lists other origins, such as
|
||||
`https://netwatch.example.com`, whose pages may then call the API with `GET` and
|
||||
`POST`. Each entry must be a plain origin as browsers send it: `scheme://host`,
|
||||
with an optional `:port`, no path, no trailing `/` and no `*`.
|
||||
|
||||
### Metrics
|
||||
|
||||
With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves
|
||||
Prometheus metrics at `GET /metrics`. A request with those as its basic auth
|
||||
credentials gets them; any other gets 401. With neither set, `/metrics`
|
||||
answers 404. The metrics record each request that reaches the health check or
|
||||
`POST /api/v1/reports`: its duration and response size, labelled with its path,
|
||||
method and status. Each client address may make 60 requests to `/metrics` a
|
||||
minute. Past that it gets 429 with `Retry-After: 60`, and its credentials are
|
||||
not checked.
|
||||
|
||||
### Logging
|
||||
|
||||
The backend logs JSON to stdout, or plain text when stdout is a terminal. Each
|
||||
request is logged with its method, URL, status, client address and duration.
|
||||
Every value taken from a request is cut to 128 bytes before it is logged.
|
||||
|
||||
## Design
|
||||
|
||||
The repository holds the page, the backend and the image:
|
||||
|
||||
- `web/`: the page's source. `index.html` loads `src/main.js` and
|
||||
`src/styles.css`.
|
||||
- `cmd/netwatch-server/` and `internal/`: the Go backend. `cmd/` holds only
|
||||
`main`, which calls into `internal/cli`.
|
||||
- `configs/`: nginx's configuration for the image, `nginx.conf` and
|
||||
`security-headers.conf`.
|
||||
- `bin/entrypoint.sh`: the image's entrypoint.
|
||||
- `Dockerfile`: the lint and test phases, the two build stages and the image.
|
||||
- `script/`: the development commands; `Makefile` targets call them.
|
||||
- `test/unit/`: the page's unit tests. `test/viewport/`: the layout test.
|
||||
|
||||
### The page
|
||||
|
||||
The page is built with Vite and Tailwind CSS v4. All its code is in
|
||||
`web/src/main.js`:
|
||||
|
||||
- `CONFIG`: the timing and display constants. The interval menu sets its
|
||||
`updateInterval`; the check timeout and the history span follow from it.
|
||||
- `WAN_HOSTS`: the WAN targets, each with its URL and, for the S3 entries, the
|
||||
method `HEAD`.
|
||||
- `HostState` and `AppState`: one host's history and status, and the whole
|
||||
page's state.
|
||||
- `measureLatency`: one check. `tick`: one round, which checks every host in
|
||||
parallel and draws each row as soon as its check ends.
|
||||
- `SparklineRenderer`: the graphs. `buildUI`, `updateHostRow`, `updateSummary`
|
||||
and `updateHealthBox`: the page and its parts.
|
||||
- `Reporter`: builds and posts the reports. `buildReport` is a pure function of
|
||||
the hosts' state.
|
||||
|
||||
### The backend
|
||||
|
||||
`netwatch-server` is a Go HTTP server. The `fx` library starts and stops its
|
||||
parts in dependency order. The backend's packages are in `internal/`:
|
||||
|
||||
- `cli`: the command line: the server when given no arguments, or the
|
||||
`check-cidr` and `prepare-data-dir` subcommands that `bin/entrypoint.sh` runs.
|
||||
- `config`: reads the settings and refuses values the server cannot use.
|
||||
- `server`: the router, the routes and the HTTP server.
|
||||
- `middleware`: the request log, the security headers, CORS, the body limit, the
|
||||
rate limit, the client address resolution and the metrics.
|
||||
- `handlers`: the health check and the report route.
|
||||
- `reportbuf`: the in-memory buffer of reports and the files it writes them to.
|
||||
It also holds `PrepareDataDir`, which `prepare-data-dir` runs.
|
||||
- `healthcheck`, `logger` and `globals`: the health status, the logger, and the
|
||||
name and version the build stamps in.
|
||||
|
||||
## Development
|
||||
|
||||
### Entrypoints
|
||||
## Entrypoints
|
||||
|
||||
This repository adheres to the
|
||||
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
||||
standard. The scripts in `script/` are the entrypoints for the development
|
||||
workflow, and the `Makefile` targets are thin shims that call them.
|
||||
standard: normalized scripts in `script/` are the entrypoints for the
|
||||
development workflow, and the Makefile targets are thin shims that call them.
|
||||
The scripts cover both the frontend and the Go backend, so `make check` fails if
|
||||
either one is broken. We provide:
|
||||
|
||||
- `script/bootstrap`: install the dependencies. It installs the pinned node,
|
||||
yarn and Go when the ones on `PATH` do not pass its checks, and links them
|
||||
into `~/.local/bin`, which has to be on `PATH`. It installs no Go linter and
|
||||
no Docker.
|
||||
- `script/setup`: `script/bootstrap`, then the git pre-commit hook.
|
||||
- `script/dev`: run the Vite dev server.
|
||||
- `script/build`: build the page into `dist/`, then the static `netwatch-server`
|
||||
binary into the repo root. Its version is `VERSION` from the environment, else
|
||||
`git describe --tags --always --dirty`, else `dev`.
|
||||
- `script/run`: `script/build`, then run `netwatch-server` with its defaults.
|
||||
- `script/clean`: remove what `script/build` writes.
|
||||
- `script/projectname`: print `netwatch`, the image tag `script/docker` uses.
|
||||
- `script/yarn`: run yarn under the node `script/bootstrap` would use.
|
||||
- `script/test`: build the `test` phase of `Dockerfile`, writing no image: the
|
||||
Go tests with the race detector and coverage, and the page's unit tests.
|
||||
- `script/lint`: build the `lint` phase of `Dockerfile`, writing no image:
|
||||
golangci-lint over the Go packages and eslint over the JavaScript.
|
||||
- `script/fmt`: run the formatters: prettier over the JavaScript, CSS, HTML and
|
||||
Markdown, then gofmt over `cmd/` and `internal/`. They write the files in
|
||||
place.
|
||||
- `script/fmt-check`: check the formatting (read-only), with the same scope as
|
||||
`script/fmt`.
|
||||
- `script/check`: `script/test`, `script/lint` and `script/fmt-check`.
|
||||
- `script/add-dependency`: add a page dependency, or move one to another
|
||||
version: `make add-dependency PACKAGE=<name>@<version>`.
|
||||
- `script/tidy`: run `go mod tidy`.
|
||||
- `script/frontend-test`: run the unit tests in `test/unit/` on the host, with
|
||||
Node's built-in test runner, then the production build.
|
||||
- `script/frontend-fmt` and `script/frontend-fmt-check`: the prettier half of
|
||||
`script/fmt` and `script/fmt-check`.
|
||||
- `script/frontend-viewport-test`: check the built page's layout in a
|
||||
containerised headless Chrome, not as part of `script/check`; see
|
||||
[test/viewport/README.md](test/viewport/README.md).
|
||||
- `script/docker`: build the image from `Dockerfile`, tagged `netwatch`.
|
||||
- `script/cibuild`: `script/bootstrap`, `script/check`, then the image build.
|
||||
The Gitea workflow `.gitea/workflows/check.yml` runs it on every push.
|
||||
- `script/precommit`: run by the git pre-commit hook; it runs `script/check`.
|
||||
- `script/install-precommit`: install the git pre-commit hook. `make hooks`
|
||||
calls it.
|
||||
- `script/bootstrap` — install all dependencies (the pinned node via nvm unless
|
||||
one new enough for the frontend's dependencies is installed, yarn via
|
||||
corepack, `yarn install --frozen-lockfile`, Go `GO_VERSION` unless the `go` on
|
||||
`PATH` is exactly that version, and the Go modules), linking what it installs
|
||||
itself into `~/.local/bin`, which has to be on `PATH`. It installs no Go
|
||||
linter and not Docker: `make test` and `make lint` run in Docker
|
||||
- `script/setup` — make a fresh clone ready for development: bootstrap plus the
|
||||
git pre-commit hook
|
||||
- `script/dev` — run the Vite dev server, which proxies `/api` to a locally
|
||||
running `netwatch-server`
|
||||
- `script/build` — build the frontend for production into `dist/`, then the
|
||||
static `netwatch-server` binary into the repo root with its version stamped
|
||||
in: `VERSION` from the environment, or when that is unset or empty,
|
||||
`git describe` inside a git checkout, then `dev`
|
||||
- `script/run` — build with `script/build`, then run `netwatch-server` with its
|
||||
defaults (see [Settings](#settings))
|
||||
- `script/clean` — remove what `script/build` writes
|
||||
- `script/projectname` — print the project name (used for the Docker image tag)
|
||||
- `script/test` — build the `test` phase of `Dockerfile`, writing no image: the
|
||||
frontend's unit tests in its `frontend-test` stage, and the backend's Go tests
|
||||
with the race detector and coverage. The frontend's production build is left
|
||||
to the image build, in the `frontend-build` stage
|
||||
- `script/lint` — build the `lint` phase of `Dockerfile`, writing no image:
|
||||
eslint in its `frontend-lint` stage, and golangci-lint over the Go packages
|
||||
- `script/fmt` — format all files (writes): prettier over the JavaScript, CSS,
|
||||
HTML and Markdown, then gofmt over `cmd/` and `internal/`. It runs on the
|
||||
host, as `script/fmt-check` does, with `~/.local/bin` put on `PATH`
|
||||
- `script/fmt-check` — check formatting (read-only): prettier, then gofmt
|
||||
- `script/check` — run test, lint, and fmt-check
|
||||
- `script/add-dependency` — add a frontend package, or move one to another
|
||||
version: `make add-dependency PACKAGE=<name>@<version>` runs `yarn add --dev`,
|
||||
which changes `package.json` and `yarn.lock` together, then
|
||||
`yarn install --frozen-lockfile`
|
||||
- `script/tidy` — run `go mod tidy`: to add a Go module, import it and run
|
||||
`make tidy`; to move one to another version, edit its `require` line in
|
||||
`go.mod`, then run `make tidy`
|
||||
- `script/frontend-test` — run the unit tests in `test/unit/` on the host with
|
||||
Node's built-in test runner, through the `test` script in `package.json`, and
|
||||
if any fails, run them again listing every test, and fail; then the production
|
||||
build. Each test run has a 90-second timeout. `make test` runs the same tests
|
||||
in Docker, without the production build
|
||||
- `script/frontend-fmt` — format the JavaScript, CSS, HTML and Markdown with
|
||||
prettier (writes)
|
||||
- `script/frontend-fmt-check` — check prettier formatting (read-only)
|
||||
- `script/frontend-viewport-test` — responsive-layout verification of the built
|
||||
frontend in a containerised headless Chrome (see
|
||||
[test/viewport/README.md](test/viewport/README.md)). Before the browser
|
||||
starts, it checks that the page, and a path that falls back to it, are served
|
||||
with `Cache-Control: no-cache` and `Content-Security-Policy`. Not part of
|
||||
`script/check`: it takes minutes.
|
||||
- `script/docker` — build the image from `Dockerfile`, tagged `netwatch` via
|
||||
`script/projectname`
|
||||
- `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`,
|
||||
then builds the image as `script/docker` does
|
||||
- `script/precommit` — run by the git pre-commit hook; runs `script/check`
|
||||
- `script/install-precommit` — install the git pre-commit hook
|
||||
|
||||
### Tests
|
||||
`script/test`, `script/lint`, `script/docker` and `script/cibuild` each pass a
|
||||
new `CACHEBUST` value, so every step in `Dockerfile` that lints, tests or
|
||||
compiles runs again even when nothing has changed. Docker keeps the system
|
||||
packages and the Go modules from earlier builds until the base image, the step
|
||||
that installs them, or `go.mod` and `go.sum` change. The `frontend-build` stage
|
||||
is the exception: it installs git again on every image build, because it starts
|
||||
from the `frontend-test` stage, whose steps run again on every build. Go's build
|
||||
cache and golangci-lint's are kept as well, and each reuses a result only when
|
||||
everything it was made from is unchanged. Both `go test` runs carry `-count=1`,
|
||||
so no test result is reused. The yarn packages are installed again on every
|
||||
build.
|
||||
|
||||
`make test` runs the backend's Go tests and the page's unit tests in Docker. The
|
||||
unit tests import `web/src/main.js` in Node and call the functions it exports;
|
||||
no browser is involved. `make frontend-viewport-test` checks the built page's
|
||||
layout in a headless Chrome; [test/viewport/README.md](test/viewport/README.md)
|
||||
says what it checks and what it cannot.
|
||||
## Responsive layout
|
||||
|
||||
The narrow-viewport layout lives in the `max-width: 768px` media block in
|
||||
`web/src/styles.css`. It is verified automatically by
|
||||
`make frontend-viewport-test`, which drives a digest-pinned headless Chrome
|
||||
against the built `dist/` and asserts on computed layout at widths derived from
|
||||
that CSS — on every breakpoint it declares and one pixel either side of it, plus
|
||||
a 320px floor, a desktop baseline and two landscape sizes. See
|
||||
[test/viewport/README.md](test/viewport/README.md) for what it covers and what
|
||||
it genuinely cannot.
|
||||
|
||||
## Rationale
|
||||
|
||||
When debugging network issues, it's useful to have a persistent at-a-glance view
|
||||
of latency and reachability to multiple well-known internet endpoints. NetWatch
|
||||
provides this as a single page that does all its measuring in the browser, so it
|
||||
can be served from anywhere static files are served. The backend in its Docker
|
||||
image only stores the measurements the page reports; without it, the page works
|
||||
the same and nothing is stored.
|
||||
|
||||
## Design
|
||||
|
||||
The page's source is in `web/`, the Go backend's in `cmd/` and `internal/`, and
|
||||
nginx's configuration in `configs/`.
|
||||
|
||||
The page is built with Vite and Tailwind CSS v4. Its code is all in
|
||||
`web/src/main.js`, with a class-based architecture:
|
||||
|
||||
- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks,
|
||||
etc.). The interval menu sets `updateInterval`, the one value the page writes
|
||||
into `CONFIG`; the timeouts, the time the history spans and the x-axis ticks
|
||||
are computed from it
|
||||
- **`HostState`**: Per-host state management — history buffer, latency tracking,
|
||||
status transitions
|
||||
- **`AppState`**: Top-level state container — WAN hosts, local hosts, pause
|
||||
state, aggregate stats
|
||||
- **`SparklineRenderer`**: Canvas 2D sparkline drawing with fixed axes,
|
||||
color-coded line segments, error regions, and DPR-aware scaling
|
||||
- **UI functions**: `buildUI()` constructs the DOM, `updateHostRow()` /
|
||||
`updateSummary()` / `updateHealthBox()` handle incremental updates
|
||||
- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's
|
||||
sample and redrawing its row as soon as its check ends, then redraws every
|
||||
row, the summary and the health box once the last check ends. The first round,
|
||||
after loading or an interval change, is discarded. The rows are sorted when
|
||||
the last check ends in round 2, the first one kept, and in rounds 11, 21, 31
|
||||
and so on. When paused, pushes blank markers (no probes, no false outage)
|
||||
- **`Reporter`**: Posts collected samples to the backend
|
||||
|
||||
### Reporting
|
||||
|
||||
Every `reportInterval` (default 60s) the page POSTs a JSON report to the
|
||||
same-origin path `/api/v1/reports`: a random per-browser `clientId` kept in
|
||||
`localStorage`, `geo` sent as null, and each host's unreported, non-paused
|
||||
samples (timestamp, latency, error). A per-host high-water mark makes every
|
||||
report a delta, so only new samples are sent; the mark advances only on a
|
||||
delivered report, and while paused nothing is sent. Changing the interval clears
|
||||
each host's history on the page; the samples in it not yet reported still go in
|
||||
the next report. Delivery failure is quiet — one debug-log line per outage,
|
||||
never blocking probing. The report-building step is a pure function of host
|
||||
state.
|
||||
|
||||
A sample in the history that no report has delivered is sent only as long as the
|
||||
history holds it, so after a refused or failed report the next report carries
|
||||
the samples the history still has. Samples an interval change cleared from the
|
||||
history are kept apart, the last 100 per host, until they are delivered. The
|
||||
history keeps each host's last 100 rounds: 100 seconds at the 1-second interval,
|
||||
200 seconds at 2 seconds, 5 minutes at the default of 3 seconds and 100 minutes
|
||||
at 60 seconds. At 1 second, one refused or failed report loses the oldest 20
|
||||
seconds of each host's samples, as the next delivered report comes 2 minutes
|
||||
after the last. At 2 seconds and longer one loses nothing; samples are lost from
|
||||
the third refused report in a row at 2 seconds, from the fifth at 3 seconds, and
|
||||
later still at the longer intervals.
|
||||
|
||||
No report is sent while the page is paused, but each paused round still adds an
|
||||
entry to the history, so each report due during a pause counts as a refused one.
|
||||
A pause longer than the history spans loses the samples in the history recorded
|
||||
between the last delivered report and the pause: up to a minute's worth for each
|
||||
host if the report before the pause was delivered.
|
||||
|
||||
### Backend
|
||||
|
||||
`netwatch-server` is a small Go HTTP server that stores the reports the page
|
||||
posts. It keeps them in memory and writes them to `DATA_DIR` as zstd-compressed
|
||||
files of JSON lines: every minute, whenever 10 MiB are waiting, and when it
|
||||
stops. Its routes:
|
||||
|
||||
- `POST /api/v1/reports` — takes a report, without credentials; each client
|
||||
address may send a limited number a minute, and the report files are capped in
|
||||
size, the oldest deleted first
|
||||
- `GET /.well-known/healthcheck` — answers 200 with `"status":"ok"`, the
|
||||
server's version and its uptime
|
||||
- `GET /metrics` — Prometheus metrics behind basic auth, only when
|
||||
`METRICS_USERNAME` and `METRICS_PASSWORD` are set; each client address may
|
||||
make a limited number of requests to it a minute
|
||||
|
||||
It is an `fx`-wired Go application: `cmd/netwatch-server/` starts it, and its
|
||||
packages are in `internal/`:
|
||||
|
||||
- **`cli`**: Runs netwatch-server from its command line: the `check-cidr` and
|
||||
`prepare-data-dir` subcommands, which `bin/entrypoint.sh` runs before it
|
||||
starts the server, and otherwise the server, with the `fx` wiring.
|
||||
- **`config`**: Loads configuration from environment variables and config files
|
||||
via Viper.
|
||||
- **`handlers`**: HTTP request handlers for the API (health check, report
|
||||
ingestion).
|
||||
- **`reportbuf`**: The in-memory buffer of reports and the files it writes them
|
||||
to.
|
||||
- **`server`**: Chi-based HTTP server with middleware wiring and route
|
||||
registration.
|
||||
- **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting
|
||||
infrastructure.
|
||||
|
||||
In the image, the `test` phase of `Dockerfile` tests it, the `builder` stage
|
||||
builds it, and `bin/entrypoint.sh` runs it as user `netwatch` on
|
||||
`127.0.0.1:8081`, behind nginx (see [Deployment](#deployment)). Outside the
|
||||
image, `make run` builds it and runs it on port 8080.
|
||||
|
||||
#### Settings
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ---------------------- | -------------------- | -------------------------------------------------------------------------------------------------------- |
|
||||
| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface |
|
||||
| `PORT` | `8080` | HTTP listen port |
|
||||
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
|
||||
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) |
|
||||
| `DEBUG` | empty | Debug logging; any non-empty value, `false` and `0` included, turns it on |
|
||||
| `TRUSTED_PROXIES` | RFC 1918 ranges | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
|
||||
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
|
||||
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
||||
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
|
||||
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
|
||||
| `SENTRY_DSN` | empty | DSN of the Sentry project to send errors to; see [Sentry](#sentry) |
|
||||
|
||||
`TRUSTED_PROXIES` defaults to `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. A list
|
||||
replaces the default; set and empty, the server trusts no proxy. Loopback is not
|
||||
in the default, so a reverse proxy on the same host has to be named, as in
|
||||
`TRUSTED_PROXIES=127.0.0.1/32`. A request whose direct peer is outside this set
|
||||
has its forwarded headers ignored, and the direct peer is logged and
|
||||
rate-limited instead. From a direct peer inside the set, the client address is
|
||||
the right-most `X-Forwarded-For` entry that is not in the set: each proxy
|
||||
appends the address it took the request from, so the entries further left were
|
||||
written by the client and are not used. An entry written as an address with a
|
||||
port, such as `198.51.100.7:5000` or `[2001:db8::1]:443`, is read as that
|
||||
address. If the search from the right finds only addresses in the set, or meets
|
||||
an entry that is not an address at all, such as `unknown`, the direct peer is
|
||||
used. `X-Real-IP` is read only when `X-Forwarded-For` is missing or empty. The
|
||||
container image does not use this default; see [Deployment](#deployment).
|
||||
|
||||
A variable set to a value the server cannot use, such as `PORT=abc` or a
|
||||
`BIND_ADDRESS` that is not an IP address, stops it from starting, with an error
|
||||
naming the variable. An empty variable counts as unset, except
|
||||
`TRUSTED_PROXIES`. No value of `DEBUG` stops the start.
|
||||
|
||||
#### Report storage
|
||||
|
||||
Reports are written as `reports-<timestamp>-<number>.jsonl.zst` files in
|
||||
`DATA_DIR`. The timestamp is in UTC to the millisecond, so the names sort by
|
||||
time. The number starts at 1 when the server starts and goes up by one for each
|
||||
file the server starts to write, so two files written in the same millisecond
|
||||
still get different names. A failed write uses up its number and leaves a gap in
|
||||
the numbers: its file, if it was created, is removed. The file stays, counted
|
||||
toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too.
|
||||
Files are created with `O_EXCL` to prevent overwrites.
|
||||
|
||||
The reports of a failed write stay in memory, and the next flush, within a
|
||||
minute, tries again to write them. The server writes one file at a time, so a
|
||||
file holds its reports in the order they arrived, those of a failed write
|
||||
included. From the failed write until a write succeeds, `POST /api/v1/reports`
|
||||
refuses every report with 500 and keeps nothing of it. The page sends the
|
||||
refused samples again in its next report, as long as it still holds them: at the
|
||||
1-second interval one refused report already loses some of those in the history
|
||||
(see [Reporting](#reporting)). Reports already answered 200 are lost only if the
|
||||
server stops before a write succeeds; the final flush at stop tries them too,
|
||||
and if it fails, the process exits non-zero.
|
||||
|
||||
#### Report limits
|
||||
|
||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||
credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
`{"status":"error"}` body as any other error, except nginx's refusal of a body
|
||||
over the limit in the image.
|
||||
|
||||
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
||||
parameters such as `charset` allowed, as the page sends it; any other is
|
||||
refused with 415, and nothing of it is stored. A page on another origin can
|
||||
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
||||
keeps any site from making its visitors' browsers store reports (see
|
||||
[CORS](#cors)).
|
||||
- **Body limit.** A report body may be at most 1 MiB, the backend's limit for
|
||||
any request. Run on its own, the backend answers 413 before the route to a
|
||||
request that declares a longer body. Without a declared length, the route
|
||||
answers 413 only when the report it reads runs past 1 MiB; nothing after the
|
||||
end of the report is read, so a longer body whose report ends sooner is
|
||||
stored, and one that stops being valid JSON sooner is answered 400. In the
|
||||
image, nginx refuses a body over 1 MiB first, with or without a declared
|
||||
length, as its `client_max_body_size`, which `configs/nginx.conf` does not
|
||||
set, is 1 MiB by default. nginx answers 413 with its own HTML error page, and
|
||||
the backend never sees the request.
|
||||
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
||||
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
||||
`Retry-After: 60`. The minute slides: reports from the minute before still
|
||||
count, fading out over the current one, so an address is sure never to be
|
||||
refused only while it sends at most half of `REPORTS_PER_MINUTE` in any 60
|
||||
seconds. The page sends one report a minute from each open tab, so the default
|
||||
of 60 refuses nothing from up to 30 tabs behind one address, such as a
|
||||
household or an office sharing it, however their reports bunch up. Report
|
||||
responses also carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and
|
||||
`X-RateLimit-Reset` headers.
|
||||
- **Size cap.** The report files in `DATA_DIR` may total at most
|
||||
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
|
||||
waiting in memory count at their uncompressed size until they are written,
|
||||
those of a failed write included; the part of its file written is removed.
|
||||
When a report would take the total past the cap, the oldest report files are
|
||||
deleted to make room, and each deletion is logged with the file's name and
|
||||
size; a file still being written is never deleted. A report is refused with
|
||||
507, and nothing of it is stored, only when the reports waiting to be written
|
||||
fill the cap on their own, and then no file is deleted. At start, report files
|
||||
past the cap, as after lowering it, are deleted the same way. So the cap is
|
||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||||
for any host; set it to the space you can give `DATA_DIR`.
|
||||
|
||||
At the defaults, one address may send 60 reports of up to 1 MiB a minute, and a
|
||||
report made up to compress badly takes about as much room in the report files as
|
||||
its request body, so in about 17 minutes one address can fill the 1 GiB cap with
|
||||
its own reports, every report stored before them deleted to make room. Lowering
|
||||
`REPORTS_PER_MINUTE` or raising `DATA_DIR_MAX_BYTES` makes that take longer.
|
||||
|
||||
#### CORS
|
||||
|
||||
The page calls the API from the origin it is served from, so by default the
|
||||
server sends no CORS headers, and browsers let no other origin's pages call it.
|
||||
To serve the page from elsewhere, list that origin in `CORS_ALLOWED_ORIGINS`
|
||||
(for example `https://netwatch.example.com`); pages from a listed origin may
|
||||
`GET` and `POST` with a `Content-Type` header. Each entry must be a plain
|
||||
origin, `scheme://host` with an optional `:port`, as browsers send it: no path,
|
||||
not even a trailing `/`, and no `*`. Any other entry stops the server from
|
||||
starting, with an error naming `CORS_ALLOWED_ORIGINS`.
|
||||
|
||||
#### Metrics
|
||||
|
||||
With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves
|
||||
Prometheus metrics at `GET /metrics` to requests with those as their basic auth
|
||||
credentials, and answers any other with 401. For each request that reaches the
|
||||
health check or `POST /api/v1/reports`, those the rate limit refuses included,
|
||||
the metrics record its duration and response size, labelled with its path,
|
||||
method and status; they also count those requests in progress, and include Go's
|
||||
runtime and process metrics. No other request is recorded: not those to
|
||||
`/metrics` itself, and not those answered before they reach either route, such
|
||||
as a CORS preflight, or a request refused with 404 for a path no route has or
|
||||
405 for a method its route does not take. Run on its own, the backend also
|
||||
answers 413 before the route to a request declaring a body length over the 1 MiB
|
||||
limit, which is not recorded. A body over the limit sent without a declared
|
||||
length reaches the route and is recorded with the status the route answers,
|
||||
which is 413 only when the report it reads runs past the limit: nothing after
|
||||
the end of the report is read. In the image neither reaches the backend: nginx
|
||||
answers both with 413 itself (see [Report limits](#report-limits)), and nothing
|
||||
of them is recorded. Clients can make up any number of paths and methods, and
|
||||
each would add labels to the metrics for as long as the server runs.
|
||||
|
||||
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
||||
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
||||
minute, whatever their credentials. Past that it gets 429 with
|
||||
`Retry-After: 60`, and its credentials are not checked. The minute slides as it
|
||||
does for reports (see [Report limits](#report-limits)), so a scraper polling
|
||||
every 2 seconds or less often is never refused. This allowance is apart from the
|
||||
one for reports.
|
||||
|
||||
#### Sentry
|
||||
|
||||
With `SENTRY_DSN` set, each panic in a handler is reported to that Sentry
|
||||
project under the release `netwatch-server-` followed by the server's version.
|
||||
On shutdown the server waits up to 2 seconds for Sentry to finish sending.
|
||||
|
||||
### Monitoring targets
|
||||
|
||||
- **26 WAN hosts**: datavi.be (pinned at start), Anthropic API, OpenAI API, AWS
|
||||
Console, Google Cloud Console, Microsoft Azure, Cloudflare, Fastly CDN,
|
||||
Akamai, Google, GitHub, B2, 8 S3 regional endpoints (Cape Town, London,
|
||||
Bahrain, Tokyo, Singapore, Sydney, Oregon, São Paulo) and 6 Hetzner speed test
|
||||
servers (Nuremberg, Falkenstein, Helsinki, Ashburn, Hillsboro, Singapore)
|
||||
- **Local CPE**: Cable modem at 192.168.100.1 (always monitored)
|
||||
- **Local Gateway**: Auto-detected on startup by probing common default gateway
|
||||
addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder
|
||||
wins. Note: modern browsers enforce Private Network Access restrictions that
|
||||
block public-origin pages from reaching RFC1918 addresses, so local targets
|
||||
only work when NetWatch is served from localhost or a private address.
|
||||
|
||||
Local hosts are tracked separately from WAN stats.
|
||||
|
||||
### Latency measurement
|
||||
|
||||
GET requests to each target's URL as written, with `mode: 'no-cors'` and
|
||||
`cache: 'no-store'`, timed with `performance.now()`. No query string is added:
|
||||
the Hetzner speed-test servers close the connection without an answer when the
|
||||
URL has one, and `no-store` keeps the browser's cache out of the measurement.
|
||||
Each check times out after 80% of the refresh interval (24 seconds at 30
|
||||
seconds) and is then recorded as a timeout, so a round's checks have all
|
||||
finished before the next round is due. When no WAN host answers, a recovery
|
||||
probe checks 4 WAN hosts, picked at random when it starts, every half second,
|
||||
giving up the checks it started half a second before. As soon as one answers, a
|
||||
new round starts at once, as it does after an interval change. A round started
|
||||
early gives up the last round's checks if they are still waiting, and that round
|
||||
records nothing more, so rounds never overlap. The browser chooses between IPv4
|
||||
and IPv6 for each target, as for any request; the local targets are IPv4
|
||||
addresses.
|
||||
|
||||
Each recorded check that fails writes one line to the browser console with
|
||||
`console.error`, and the same line to the debug log: the target's name and URL,
|
||||
the time, whether it timed out, answered over the time limit or hit a network
|
||||
error (with the error the browser gives the page, such as
|
||||
`TypeError: Failed to fetch`, which does not say why), and how long the request
|
||||
took. A target that answers after failed checks writes one `console.info` line
|
||||
with its latency and how many checks in a row had failed.
|
||||
|
||||
### Color coding
|
||||
|
||||
| Latency | Color |
|
||||
| ----------- | ------ |
|
||||
| < 50ms | Green |
|
||||
| < 100ms | Lime |
|
||||
| < 200ms | Yellow |
|
||||
| < 500ms | Orange |
|
||||
| >= 500ms | Red |
|
||||
| Unreachable | Gray |
|
||||
|
||||
### Output structure
|
||||
|
||||
```
|
||||
dist/
|
||||
├── index.html
|
||||
└── assets/
|
||||
├── index-*.css
|
||||
└── index-*.js
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- A round of checks every 3 seconds by default; the interval menu sets 1, 2, 3,
|
||||
5, 10, 15, 30 or 60 seconds and clears the history
|
||||
- Sparklines of each target's last 100 rounds: 300 seconds at 3 seconds
|
||||
- The first round after loading or an interval change is discarded, as DNS and
|
||||
TLS setup inflate its latencies
|
||||
- Health indicator from the WAN hosts' latest results: OFFLINE (red) when more
|
||||
than 10 fail and at most 4 answer, otherwise DEGRADED (orange) when more than
|
||||
4 fail, otherwise SLOW (yellow) when more than 3 take over 1000ms, otherwise
|
||||
HEALTHY (green)
|
||||
- Summary stats across WAN hosts only: how many answered, the min, median,
|
||||
average and max of their latest latencies, the min and max over the whole
|
||||
history, and the number of rounds run (`Checks`)
|
||||
- Fixed chart axes: Y-axis 0–1000ms, higher latencies drawn at the top; X-axis
|
||||
the time the history spans
|
||||
- Color-coded latency figures and sparkline line segments
|
||||
- WAN host rows sorted by latest latency, unreachable last; pinned rows stay on
|
||||
top, in name order
|
||||
- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no
|
||||
false outage)
|
||||
- Debug log panel, behind a checkbox in the footer, with five levels (error,
|
||||
warning, notice, info, debug) and the last 1000 lines
|
||||
- Local and UTC clocks
|
||||
- Clickable service URLs
|
||||
- A footer link to the commit the page was built from
|
||||
- Canvas-based sparkline rendering with devicePixelRatio scaling
|
||||
- Zero runtime dependencies: all resources bundled into build artifacts
|
||||
|
||||
## Deployment
|
||||
|
||||
`make build` writes the page to `dist/`, which any static file host (S3, GCS,
|
||||
Cloudflare Pages, Vercel, Netlify, GitHub Pages) can serve; with no backend
|
||||
there, its reports fail quietly and nothing is stored. Or run the Docker image
|
||||
behind a reverse proxy.
|
||||
|
||||
The Docker image, built from `Dockerfile` by `make docker`, is the whole service
|
||||
in one container: nginx serves the built frontend and passes `/api/`,
|
||||
`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`.
|
||||
`bin/entrypoint.sh` starts it listening only inside the container, on
|
||||
`127.0.0.1:8081`, with `TRUSTED_PROXIES=127.0.0.1/32`, so it takes the client
|
||||
address nginx passes on and no other. The image:
|
||||
|
||||
- Listens on port 8080 by default (override with `PORT` env var)
|
||||
- Takes the client address from `X-Forwarded-For` only on requests from the
|
||||
reverse proxies named in `TRUSTED_PROXIES`, by default from any address in
|
||||
`10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`. `bin/entrypoint.sh` checks
|
||||
each entry with `netwatch-server check-cidr`, which parses it as the backend
|
||||
parses its own `TRUSTED_PROXIES`
|
||||
- Sends access logs to stdout
|
||||
- Sends the page with `Cache-Control: no-cache`, so a browser that keeps a copy
|
||||
checks with the server that it is current before using it again; only the
|
||||
files under `/assets/`, whose names change whenever their contents do, are
|
||||
cached as immutable
|
||||
- Sends the security headers `REPO_POLICIES.md` requires on every response, as
|
||||
`configs/security-headers.conf` sets them, in place of the backend's own
|
||||
- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
|
||||
volume. Before the backend starts, the image creates `DATA_DIR` and gives it
|
||||
and `/data` to user `netwatch` (uid 1000), which the backend runs as, with
|
||||
`netwatch-server prepare-data-dir`, which acts on nothing outside `/data`, so
|
||||
a host directory bind-mounted at `/data` ends up owned by uid 1000
|
||||
- Writes buffered reports to disk on `docker stop`, and exits non-zero if nginx
|
||||
or the backend exits on its own, so the platform restarts it
|
||||
|
||||
## Running under upaas
|
||||
|
||||
What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
||||
|
||||
- **Port:** container port `8080`.
|
||||
- **Volume:** container path `/data`; the reports are kept in `/data/reports`.
|
||||
- **Environment variables:** none is required. An empty one counts as unset,
|
||||
except `TRUSTED_PROXIES`, and one set to a value netwatch cannot use stops the
|
||||
container at start, with the reason in its log.
|
||||
- `PORT`, default `8080`: the container port, from 1 to 65535. `8081` cannot
|
||||
be used: the backend listens on it inside the container
|
||||
- `REPORTS_PER_MINUTE`, default `60`: reports each client address may send a
|
||||
minute
|
||||
- `DATA_DIR_MAX_BYTES`, default `1073741824` (1 GiB): the most room the
|
||||
report files may take; the oldest are deleted to stay under it
|
||||
- `CORS_ALLOWED_ORIGINS`, default empty: other origins whose pages may call
|
||||
the API
|
||||
- `DEBUG`, default empty: debug logging; any non-empty value, `false` and
|
||||
`0` included, turns it on
|
||||
- `DATA_DIR`, default `/data/reports`: the directory the reports are kept
|
||||
in: `/data` or a path below it, with no `.` or `..` part and no extra `/`.
|
||||
The container also stops if the path goes through a symbolic link that
|
||||
leads out of `/data` or is written as a full path
|
||||
- `TRUSTED_PROXIES`, default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`: the
|
||||
addresses the reverse proxy in front of the container connects from, as IP
|
||||
addresses or CIDRs separated by commas. nginx takes the client address
|
||||
from `X-Forwarded-For` only on a request from one of them, and the rate
|
||||
limit counts that address. A list replaces the default. Set and empty,
|
||||
`X-Forwarded-For` is ignored and every client behind the proxy shares the
|
||||
proxy's one allowance of `REPORTS_PER_MINUTE`. Any client that connects
|
||||
from one of these addresses can write its own `X-Forwarded-For`, and
|
||||
through a port Docker publishes, every client may connect from the Docker
|
||||
network's gateway, such as `172.17.0.1`, which the default includes; where
|
||||
that matters, name only the addresses nothing but the proxy connects from.
|
||||
- `METRICS_USERNAME` and `METRICS_PASSWORD`, default empty: with both set,
|
||||
the backend records Prometheus metrics of its requests and serves them at
|
||||
`/metrics` on the container port, to requests with this user name and
|
||||
password as their basic auth credentials. With neither set, there are no
|
||||
metrics and `/metrics` is not found. One set without the other, or a user
|
||||
name containing `:`, stops the container
|
||||
- `SENTRY_DSN`, default empty: set to a Sentry project's DSN, the backend
|
||||
sends its errors to that Sentry project: each request whose handling
|
||||
crashes, which still gets a 500 response. A value Sentry does not accept
|
||||
stops the container. Empty, the backend sends nothing to Sentry
|
||||
- **Health check:** the image's `HEALTHCHECK` requests
|
||||
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
|
||||
both nginx and the backend answer. upaas reads the container's health 60
|
||||
seconds after a deploy and fails the deploy unless it is `healthy`. The
|
||||
container also stops when either process exits.
|
||||
|
||||
## Browser Compatibility
|
||||
|
||||
Requires a modern browser with ES modules, Fetch API, Canvas API, and CSS custom
|
||||
properties.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **CORS**: The checks are cross-origin requests in `no-cors` mode, so the page
|
||||
cannot read the answer, only time it: any answer counts as reachable, an error
|
||||
page included.
|
||||
- **Local targets**: The cable modem at 192.168.100.1 and the detected gateway
|
||||
answer only on a network that has them, and only when NetWatch is served from
|
||||
localhost or a private address (see Monitoring targets).
|
||||
- **Network conditions**: Measurements reflect browser-to-endpoint latency,
|
||||
which includes your local network, ISP, and internet routing.
|
||||
|
||||
## TODO
|
||||
|
||||
|
||||
+63
-91
@@ -13,27 +13,24 @@ set -u
|
||||
# address and start anyway, and reports a bad port without naming
|
||||
# PORT, so a value that is not a usable port stops the container here,
|
||||
# before either process starts.
|
||||
check_port() {
|
||||
export PORT="${PORT:-8080}"
|
||||
case "$PORT" in
|
||||
*[!0-9]*)
|
||||
echo "entrypoint: PORT must be a port number, not '$PORT'" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
# The length is checked first because, for a number too big for it,
|
||||
# the shell's test prints an error and is false, so the range checks
|
||||
# alone would let it through.
|
||||
if [ "${#PORT}" -gt 5 ] || [ "$PORT" -lt 1 ] || [ "$PORT" -gt 65535 ]; then
|
||||
echo "entrypoint: PORT must be from 1 to 65535, not '$PORT'" >&2
|
||||
export PORT="${PORT:-8080}"
|
||||
case "$PORT" in
|
||||
*[!0-9]*)
|
||||
echo "entrypoint: PORT must be a port number, not '$PORT'" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$PORT" -eq 8081 ]; then
|
||||
echo "entrypoint: PORT cannot be 8081, netwatch-server" \
|
||||
"listens there" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
;;
|
||||
esac
|
||||
# The length is checked first because, for a number too big for it,
|
||||
# the shell's test prints an error and is false, so the range checks
|
||||
# alone would let it through.
|
||||
if [ "${#PORT}" -gt 5 ] || [ "$PORT" -lt 1 ] || [ "$PORT" -gt 65535 ]; then
|
||||
echo "entrypoint: PORT must be from 1 to 65535, not '$PORT'" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$PORT" -eq 8081 ]; then
|
||||
echo "entrypoint: PORT cannot be 8081, netwatch-server listens there" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# TRUSTED_PROXIES names the reverse proxies in front of the container,
|
||||
# as IP addresses or CIDRs separated by commas. nginx takes the client
|
||||
@@ -50,23 +47,21 @@ check_port() {
|
||||
# TRUSTED_PROXIES. Its error, naming the CIDR, is dropped for the one
|
||||
# below, naming the entry as written. set -f keeps a * in an entry from
|
||||
# becoming a list of file names.
|
||||
write_trusted_proxies() {
|
||||
TRUSTED_PROXIES="${TRUSTED_PROXIES-10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}"
|
||||
set -f
|
||||
for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
|
||||
case "$proxy" in
|
||||
*/*) cidr="$proxy" ;;
|
||||
*:*) cidr="$proxy/128" ;;
|
||||
*) cidr="$proxy/32" ;;
|
||||
esac
|
||||
if ! netwatch-server check-cidr "$cidr" 2> /dev/null; then
|
||||
echo "entrypoint: TRUSTED_PROXIES must be IP addresses or CIDRs" \
|
||||
"separated by commas; '$proxy' is neither" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "set_real_ip_from $cidr;"
|
||||
done > /etc/nginx/trusted-proxies.conf
|
||||
}
|
||||
TRUSTED_PROXIES="${TRUSTED_PROXIES-10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}"
|
||||
set -f
|
||||
for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
|
||||
case "$proxy" in
|
||||
*/*) cidr="$proxy" ;;
|
||||
*:*) cidr="$proxy/128" ;;
|
||||
*) cidr="$proxy/32" ;;
|
||||
esac
|
||||
if ! netwatch-server check-cidr "$cidr" 2> /dev/null; then
|
||||
echo "entrypoint: TRUSTED_PROXIES must be IP addresses or CIDRs" \
|
||||
"separated by commas; '$proxy' is neither" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "set_real_ip_from $cidr;"
|
||||
done > /etc/nginx/trusted-proxies.conf
|
||||
|
||||
# netwatch-server keeps its report files in DATA_DIR, on the /data
|
||||
# volume, which may be a host directory owned by root or by another
|
||||
@@ -78,10 +73,12 @@ write_trusted_proxies() {
|
||||
# naming DATA_DIR, unless DATA_DIR is /data or a path below it, and it
|
||||
# acts on nothing outside /data, whatever symbolic or hard links it
|
||||
# meets there.
|
||||
prepare_data_dir() {
|
||||
export DATA_DIR="${DATA_DIR:-/data/reports}"
|
||||
netwatch-server prepare-data-dir "$DATA_DIR" || exit 1
|
||||
}
|
||||
export DATA_DIR="${DATA_DIR:-/data/reports}"
|
||||
netwatch-server prepare-data-dir "$DATA_DIR" || exit 1
|
||||
|
||||
# A stop signal is only noted here; the loop below acts on it.
|
||||
stop_requested=""
|
||||
trap 'stop_requested=yes' TERM INT
|
||||
|
||||
# netwatch-server runs as the netwatch user and listens on loopback
|
||||
# only, on a port other than the public one; configs/nginx.conf proxies
|
||||
@@ -90,11 +87,9 @@ prepare_data_dir() {
|
||||
# the container has. The netwatch user has no login shell, hence -s
|
||||
# /bin/sh. busybox su replaces itself with the command instead of
|
||||
# staying on as its parent, so $! is the server's own PID.
|
||||
start_backend() {
|
||||
BIND_ADDRESS=127.0.0.1 PORT=8081 TRUSTED_PROXIES=127.0.0.1/32 \
|
||||
su -s /bin/sh netwatch -c 'exec netwatch-server' &
|
||||
backend=$!
|
||||
}
|
||||
BIND_ADDRESS=127.0.0.1 PORT=8081 TRUSTED_PROXIES=127.0.0.1/32 \
|
||||
su -s /bin/sh netwatch -c 'exec netwatch-server' &
|
||||
backend=$!
|
||||
|
||||
# nginx starts through the nginx image's own entrypoint, which applies
|
||||
# the image's start-up configuration and then replaces itself with
|
||||
@@ -103,11 +98,9 @@ start_backend() {
|
||||
# that rendering to PORT: a variable nginx itself uses, such as $uri,
|
||||
# would otherwise be replaced by an environment variable of the same
|
||||
# name.
|
||||
start_nginx() {
|
||||
NGINX_ENVSUBST_FILTER='^PORT$' \
|
||||
/docker-entrypoint.sh nginx -g 'daemon off;' &
|
||||
nginx=$!
|
||||
}
|
||||
NGINX_ENVSUBST_FILTER='^PORT$' \
|
||||
/docker-entrypoint.sh nginx -g 'daemon off;' &
|
||||
nginx=$!
|
||||
|
||||
running() {
|
||||
kill -0 "$1" 2>/dev/null
|
||||
@@ -116,47 +109,26 @@ running() {
|
||||
# POSIX sh cannot wait for whichever of two children exits first, so
|
||||
# look once a second. The shell collects a child that has exited while
|
||||
# it runs sleep, and running() is false for that child from then on.
|
||||
wait_for_stop_or_exit() {
|
||||
while [ -z "$stop_requested" ] && running "$backend" && running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
while [ -z "$stop_requested" ] && running "$backend" && running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# Stop both, then wait until neither is left.
|
||||
stop_both() {
|
||||
kill -TERM "$backend" "$nginx" 2>/dev/null
|
||||
while running "$backend" || running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
kill -TERM "$backend" "$nginx" 2>/dev/null
|
||||
while running "$backend" || running "$nginx"; do
|
||||
sleep 1
|
||||
done
|
||||
|
||||
wait "$backend"
|
||||
backend_status=$?
|
||||
wait "$nginx"
|
||||
nginx_status=$?
|
||||
echo "entrypoint: netwatch-server exited $backend_status," \
|
||||
"nginx exited $nginx_status"
|
||||
|
||||
# Success is a requested stop that both processes exited cleanly from.
|
||||
exit_with_status() {
|
||||
wait "$backend"
|
||||
backend_status=$?
|
||||
wait "$nginx"
|
||||
nginx_status=$?
|
||||
echo "entrypoint: netwatch-server exited $backend_status," \
|
||||
"nginx exited $nginx_status"
|
||||
if [ -n "$stop_requested" ] && [ "$backend_status" -eq 0 ] &&
|
||||
[ "$nginx_status" -eq 0 ]; then
|
||||
exit 0
|
||||
fi
|
||||
exit 1
|
||||
}
|
||||
|
||||
main() {
|
||||
check_port
|
||||
write_trusted_proxies
|
||||
prepare_data_dir
|
||||
# A stop signal is only noted here; wait_for_stop_or_exit acts on it.
|
||||
stop_requested=""
|
||||
trap 'stop_requested=yes' TERM INT
|
||||
start_backend
|
||||
start_nginx
|
||||
wait_for_stop_or_exit
|
||||
stop_both
|
||||
exit_with_status
|
||||
}
|
||||
|
||||
main "$@"
|
||||
if [ -n "$stop_requested" ] && [ "$backend_status" -eq 0 ] &&
|
||||
[ "$nginx_status" -eq 0 ]; then
|
||||
exit 0
|
||||
fi
|
||||
exit 1
|
||||
|
||||
@@ -15,5 +15,5 @@ var (
|
||||
)
|
||||
|
||||
func main() {
|
||||
os.Exit(cli.Main(os.Stderr, Appname, Version, os.Args[1:], cli.RunServer))
|
||||
os.Exit(cli.Main(Appname, Version, os.Args[1:]))
|
||||
}
|
||||
|
||||
@@ -8,9 +8,10 @@ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" alway
|
||||
# Scripts and styles load only from the page's own origin. Inline ones
|
||||
# are blocked, style attributes in markup included, so style elements
|
||||
# through classes or element.style. data: images are for the favicon
|
||||
# in web/index.html. connect-src is * because the page checks every
|
||||
# target in web/src/main.js. A list of hosts here would have to change
|
||||
# with every change to those targets. It also covers the reports the
|
||||
# in web/index.html. connect-src is * because the browser checks each
|
||||
# probe in web/src/main.js against it, and also every redirect the probe
|
||||
# follows, and several of those hosts redirect to others; a list of
|
||||
# hosts here would block those probes. It also covers the reports the
|
||||
# page sends to its own origin.
|
||||
add_header Content-Security-Policy "default-src 'self'; connect-src *; img-src 'self' data:; object-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'" always;
|
||||
|
||||
|
||||
+13
-39
@@ -1,12 +1,13 @@
|
||||
// Package cli runs netwatch-server from its command line: the
|
||||
// check-cidr and prepare-data-dir subcommands, which bin/entrypoint.sh
|
||||
// runs before it starts the server, and, given no arguments, the
|
||||
// server, with the fx wiring.
|
||||
// runs before it starts the server, and otherwise the server, with the
|
||||
// fx wiring.
|
||||
package cli
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/user"
|
||||
|
||||
"sneak.berlin/go/netwatch/internal/config"
|
||||
@@ -22,32 +23,15 @@ import (
|
||||
"go.uber.org/fx/fxevent"
|
||||
)
|
||||
|
||||
// usageExitCode is what Main returns for a command line it does not
|
||||
// understand, the usual exit code for a usage error.
|
||||
const usageExitCode = 2
|
||||
|
||||
// Main runs netwatch-server with args, its command line after the
|
||||
// program's name, and returns the exit code. With no args, Main runs
|
||||
// the server by calling serve with appname and version, the name and
|
||||
// version the server logs and reports. With "check-cidr CIDR" or
|
||||
// "prepare-data-dir DATA_DIR" it runs that subcommand, which writes its
|
||||
// errors to stderr. Any other args get a one-line usage message on
|
||||
// stderr and exit code 2. main passes os.Stderr and RunServer; the
|
||||
// tests pass a buffer and a function of their own, so that no test
|
||||
// starts the server. When serve is RunServer and the server fails to
|
||||
// start, to listen or to stop, fx's Run exits the process with status 1
|
||||
// and Main never returns.
|
||||
func Main(
|
||||
stderr io.Writer,
|
||||
appname, version string,
|
||||
args []string,
|
||||
serve func(appname, version string),
|
||||
) int {
|
||||
// program's name, and returns the exit code. appname and version are
|
||||
// the name and version the server logs and reports.
|
||||
func Main(appname, version string, args []string) int {
|
||||
// "netwatch-server check-cidr CIDR" exits 1, with the error, if
|
||||
// this server would refuse CIDR in its TRUSTED_PROXIES.
|
||||
// bin/entrypoint.sh runs it on each entry it gives nginx.
|
||||
if len(args) == 2 && args[0] == "check-cidr" {
|
||||
return checkCIDR(stderr, args[1])
|
||||
return checkCIDR(os.Stderr, args[1])
|
||||
}
|
||||
|
||||
// "netwatch-server prepare-data-dir DATA_DIR" gets DATA_DIR ready
|
||||
@@ -55,19 +39,10 @@ func Main(
|
||||
// reportbuf.PrepareDataDir. bin/entrypoint.sh runs it as root
|
||||
// before it starts this server as that user.
|
||||
if len(args) == 2 && args[0] == "prepare-data-dir" {
|
||||
return prepareDataDir(stderr, "/data", args[1], "netwatch")
|
||||
return prepareDataDir(os.Stderr, "/data", args[1], "netwatch")
|
||||
}
|
||||
|
||||
// Any other command line is refused, so that a mistyped one does
|
||||
// not start the server.
|
||||
if len(args) > 0 {
|
||||
_, _ = fmt.Fprintln(stderr,
|
||||
"usage: netwatch-server [check-cidr CIDR | prepare-data-dir DATA_DIR]")
|
||||
|
||||
return usageExitCode
|
||||
}
|
||||
|
||||
serve(appname, version)
|
||||
runServer(appname, version)
|
||||
|
||||
return 0
|
||||
}
|
||||
@@ -108,11 +83,10 @@ func prepareDataDir(stderr io.Writer, volume, dataDir, username string) int {
|
||||
return 0
|
||||
}
|
||||
|
||||
// RunServer runs the server until it is told to stop, with appname and
|
||||
// version as the name and version it logs and reports. If the server
|
||||
// fails to start, to listen or to stop, fx's Run exits the process
|
||||
// with status 1 itself.
|
||||
func RunServer(appname, version string) {
|
||||
// runServer runs the server until it is told to stop. If the server
|
||||
// fails to start or to stop, fx's Run exits the process with status 1
|
||||
// itself.
|
||||
func runServer(appname, version string) {
|
||||
globals.Appname = appname
|
||||
globals.Version = version
|
||||
|
||||
|
||||
@@ -13,109 +13,6 @@ import (
|
||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||
)
|
||||
|
||||
// TestMainRunsCheckCIDR: Main given check-cidr returns 0, writing
|
||||
// nothing, for a CIDR the server takes in its TRUSTED_PROXIES, and 1,
|
||||
// with an error naming TRUSTED_PROXIES and the CIDR, for one it
|
||||
// refuses, and does not start the server.
|
||||
func TestMainRunsCheckCIDR(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var stderr bytes.Buffer
|
||||
|
||||
code := cli.Main(&stderr, "netwatch", "test",
|
||||
[]string{"check-cidr", "10.0.0.0/8"}, failIfServerStarts(t))
|
||||
if code != 0 || stderr.Len() != 0 {
|
||||
t.Errorf("check-cidr 10.0.0.0/8: exit %d, stderr %q; want exit 0 and no output",
|
||||
code, stderr.String())
|
||||
}
|
||||
|
||||
stderr.Reset()
|
||||
|
||||
code = cli.Main(&stderr, "netwatch", "test",
|
||||
[]string{"check-cidr", "1.2.3/32"}, failIfServerStarts(t))
|
||||
|
||||
want := `TRUSTED_PROXIES "1.2.3/32": `
|
||||
if code != 1 || !strings.HasPrefix(stderr.String(), want) {
|
||||
t.Errorf("check-cidr 1.2.3/32: exit %d, stderr %q; want exit 1 and %q...",
|
||||
code, stderr.String(), want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMainRunsPrepareDataDir: Main given prepare-data-dir runs the
|
||||
// prepare-data-dir subcommand for the netwatch user, and does not start
|
||||
// the server. The test image has no netwatch user, so that subcommand
|
||||
// exits 1 with the error from looking the user up;
|
||||
// TestPrepareDataDirRefusesPathOutsideVolume checks the path.
|
||||
func TestMainRunsPrepareDataDir(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var stderr bytes.Buffer
|
||||
|
||||
code := cli.Main(&stderr, "netwatch", "test",
|
||||
[]string{"prepare-data-dir", "/etc"}, failIfServerStarts(t))
|
||||
|
||||
want := user.UnknownUserError("netwatch").Error() + "\n"
|
||||
if code != 1 || stderr.String() != want {
|
||||
t.Errorf("prepare-data-dir /etc: exit %d, stderr %q; want exit 1 and %q",
|
||||
code, stderr.String(), want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMainRunsServerWithNoArguments: Main given no arguments starts the
|
||||
// server with the name and version it was given, writes nothing and
|
||||
// returns 0.
|
||||
func TestMainRunsServerWithNoArguments(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var (
|
||||
stderr bytes.Buffer
|
||||
gotAppname, gotVersion string
|
||||
)
|
||||
|
||||
code := cli.Main(&stderr, "netwatch", "test", []string{},
|
||||
func(appname, version string) {
|
||||
gotAppname, gotVersion = appname, version
|
||||
})
|
||||
if code != 0 || stderr.Len() != 0 {
|
||||
t.Errorf("exit %d, stderr %q; want exit 0 and no output",
|
||||
code, stderr.String())
|
||||
}
|
||||
|
||||
if gotAppname != "netwatch" || gotVersion != "test" {
|
||||
t.Errorf("server started with %q %q; want %q %q",
|
||||
gotAppname, gotVersion, "netwatch", "test")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMainRefusesOtherCommandLines: Main given a subcommand with too
|
||||
// few or too many values, or a misspelt subcommand, writes the usage
|
||||
// message to stderr, returns 2 and does not start the server.
|
||||
func TestMainRefusesOtherCommandLines(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
want := "usage: netwatch-server " +
|
||||
"[check-cidr CIDR | prepare-data-dir DATA_DIR]\n"
|
||||
|
||||
for _, commandLine := range []string{
|
||||
"netwatch-server check-cidr",
|
||||
"netwatch-server check-cidr 192.0.2.0/24 198.51.100.0/24",
|
||||
"netwatch-server prepare-data-dir",
|
||||
"netwatch-server prepare-data-dir /data/reports /data/other",
|
||||
"netwatch-server check-cdir 192.0.2.0/24",
|
||||
} {
|
||||
var stderr bytes.Buffer
|
||||
|
||||
// Main takes the command line after the program's name.
|
||||
args := strings.Fields(commandLine)[1:]
|
||||
|
||||
code := cli.Main(&stderr, "netwatch", "test", args, failIfServerStarts(t))
|
||||
if code != 2 || stderr.String() != want {
|
||||
t.Errorf("%q: exit %d, stderr %q; want exit 2 and %q",
|
||||
commandLine, code, stderr.String(), want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestCheckCIDRTakesCIDR: check-cidr exits 0, writing nothing, for a
|
||||
// CIDR the server takes in its TRUSTED_PROXIES.
|
||||
func TestCheckCIDRTakesCIDR(t *testing.T) {
|
||||
@@ -207,13 +104,3 @@ func currentUsername(t *testing.T) string {
|
||||
|
||||
return current.Username
|
||||
}
|
||||
|
||||
// failIfServerStarts is what the tests give Main in place of
|
||||
// cli.RunServer: it fails the test instead of starting the server.
|
||||
func failIfServerStarts(t *testing.T) func(appname, version string) {
|
||||
t.Helper()
|
||||
|
||||
return func(string, string) {
|
||||
t.Error("Main started the server")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,8 +13,8 @@ import (
|
||||
"go.uber.org/fx"
|
||||
)
|
||||
|
||||
// requireConfigError builds the config as cli.RunServer does and fails
|
||||
// the test unless that fails with an error naming setting. It uses
|
||||
// requireConfigError builds the config as main does and fails the
|
||||
// test unless that fails with an error naming setting. It uses
|
||||
// fx.New, because fxtest.New fails the test itself on an error.
|
||||
func requireConfigError(t *testing.T, setting string) {
|
||||
t.Helper()
|
||||
@@ -31,8 +31,8 @@ func requireConfigError(t *testing.T, setting string) {
|
||||
}
|
||||
}
|
||||
|
||||
// requireTrustedProxies builds the config as cli.RunServer does and
|
||||
// fails the test unless its TrustedProxies are want.
|
||||
// requireTrustedProxies builds the config as main does and fails the
|
||||
// test unless its TrustedProxies are want.
|
||||
func requireTrustedProxies(t *testing.T, want []string) {
|
||||
t.Helper()
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ package globals
|
||||
|
||||
import "go.uber.org/fx"
|
||||
|
||||
//nolint:gochecknoglobals // set by cli.RunServer before fx starts
|
||||
//nolint:gochecknoglobals // set by cli.Main before fx starts
|
||||
var (
|
||||
// Appname is the application name.
|
||||
Appname string
|
||||
|
||||
@@ -3,17 +3,9 @@ package handlers
|
||||
import "net/http"
|
||||
|
||||
// HandleHealthCheck returns a handler for the health check
|
||||
// endpoint. It answers 200 while the status is "ok" and 503 otherwise,
|
||||
// so that the image's HEALTHCHECK fails.
|
||||
// endpoint.
|
||||
func (s *Handlers) HandleHealthCheck() http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
resp := s.hc.Healthcheck()
|
||||
|
||||
code := http.StatusOK
|
||||
if resp.Status != "ok" {
|
||||
code = http.StatusServiceUnavailable
|
||||
}
|
||||
|
||||
s.respondJSON(w, r, resp, code)
|
||||
s.respondJSON(w, r, s.hc.Healthcheck(), http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,19 +9,17 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/netwatch/internal/config"
|
||||
"sneak.berlin/go/netwatch/internal/globals"
|
||||
"sneak.berlin/go/netwatch/internal/handlers"
|
||||
"sneak.berlin/go/netwatch/internal/healthcheck"
|
||||
"sneak.berlin/go/netwatch/internal/logger"
|
||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||
|
||||
"go.uber.org/fx/fxtest"
|
||||
)
|
||||
|
||||
// newStartedHandlers builds Handlers with a real health check for the
|
||||
// server named in g, and a report buffer in a temporary directory, and
|
||||
// starts them, which records the time the uptime counts from.
|
||||
// server named in g, and starts them, which records the time the
|
||||
// uptime counts from.
|
||||
func newStartedHandlers(t *testing.T, g *globals.Globals) *handlers.Handlers {
|
||||
t.Helper()
|
||||
|
||||
@@ -32,23 +30,14 @@ func newStartedHandlers(t *testing.T, g *globals.Globals) *handlers.Handlers {
|
||||
t.Fatalf("logger: %v", err)
|
||||
}
|
||||
|
||||
buf, err := reportbuf.New(lc, reportbuf.Params{
|
||||
Config: &config.Config{DataDir: t.TempDir(), DataDirMaxBytes: 1 << 20},
|
||||
Logger: log,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("report buffer: %v", err)
|
||||
}
|
||||
|
||||
hc, err := healthcheck.New(lc,
|
||||
healthcheck.Params{Buffer: buf, Globals: g, Logger: log})
|
||||
healthcheck.Params{Globals: g, Logger: log})
|
||||
if err != nil {
|
||||
t.Fatalf("health check: %v", err)
|
||||
}
|
||||
|
||||
h, err := handlers.New(lc, handlers.Params{
|
||||
Buffer: buf, Globals: g, Healthcheck: hc, Logger: log,
|
||||
})
|
||||
h, err := handlers.New(lc,
|
||||
handlers.Params{Globals: g, Healthcheck: hc, Logger: log})
|
||||
if err != nil {
|
||||
t.Fatalf("handlers: %v", err)
|
||||
}
|
||||
@@ -59,12 +48,10 @@ func newStartedHandlers(t *testing.T, g *globals.Globals) *handlers.Handlers {
|
||||
return h
|
||||
}
|
||||
|
||||
// TestHandleHealthCheck checks the health check's answer at start, when
|
||||
// no write of a report file has failed: 200, a JSON content type, and a
|
||||
// JSON object with exactly the fields of healthcheck.HealthcheckResponse,
|
||||
// carrying "status":"ok", this server's name and version and an uptime
|
||||
// counted from its start. The answer while writes fail is checked in
|
||||
// internal/reportbuf, whose tests can make a write fail.
|
||||
// TestHandleHealthCheck checks the health check's answer: 200, a JSON
|
||||
// content type, and a JSON object with exactly the fields of
|
||||
// healthcheck.HealthcheckResponse, carrying this server's name and
|
||||
// version and an uptime counted from its start.
|
||||
func TestHandleHealthCheck(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"sneak.berlin/go/netwatch/internal/config"
|
||||
"sneak.berlin/go/netwatch/internal/globals"
|
||||
"sneak.berlin/go/netwatch/internal/logger"
|
||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||
|
||||
"go.uber.org/fx"
|
||||
)
|
||||
@@ -19,7 +18,6 @@ import (
|
||||
type Params struct {
|
||||
fx.In
|
||||
|
||||
Buffer *reportbuf.Buffer
|
||||
Config *config.Config
|
||||
Globals *globals.Globals
|
||||
Logger *logger.Logger
|
||||
@@ -70,18 +68,12 @@ func New(
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// Healthcheck returns the current health status of the application:
|
||||
// "ok", or "error" while report files cannot be written.
|
||||
// Healthcheck returns the current health status of the application.
|
||||
func (s *Healthcheck) Healthcheck() *HealthcheckResponse {
|
||||
status := "ok"
|
||||
if s.params.Buffer.LastWriteFailed() {
|
||||
status = "error"
|
||||
}
|
||||
|
||||
return &HealthcheckResponse{
|
||||
Appname: s.params.Globals.Appname,
|
||||
Now: time.Now().UTC().Format(time.RFC3339Nano),
|
||||
Status: status,
|
||||
Status: "ok",
|
||||
UptimeHuman: s.uptime().String(),
|
||||
UptimeSeconds: int64(s.uptime().Seconds()),
|
||||
Version: s.params.Globals.Version,
|
||||
|
||||
@@ -255,16 +255,6 @@ func (b *Buffer) Append(v any) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// LastWriteFailed reports whether the last write of a report file to
|
||||
// finish failed: false at start until a write fails, and false again
|
||||
// once a write succeeds. While it is true, Append refuses reports.
|
||||
func (b *Buffer) LastWriteFailed() bool {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
|
||||
return b.writeErr != nil
|
||||
}
|
||||
|
||||
// deleteOldestFiles deletes report files, oldest first, until n more
|
||||
// bytes fit under maxBytes. It deletes none when the reports waiting
|
||||
// to be written leave no room for n even with every file gone, since
|
||||
|
||||
@@ -334,9 +334,9 @@ func stopWhileHeld(
|
||||
return <-stopped
|
||||
}
|
||||
|
||||
// startApp starts a Buffer through fx, as cli.RunServer does, with the
|
||||
// DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set, and returns
|
||||
// it with the app, for tests that stop the app themselves.
|
||||
// startApp starts a Buffer through fx, as main does, with the DATA_DIR
|
||||
// and DATA_DIR_MAX_BYTES the calling test has set, and returns it with
|
||||
// the app, for tests that stop the app themselves.
|
||||
func startApp(t *testing.T) (*reportbuf.Buffer, *fxtest.App) {
|
||||
t.Helper()
|
||||
|
||||
@@ -358,8 +358,8 @@ func startApp(t *testing.T) (*reportbuf.Buffer, *fxtest.App) {
|
||||
return buf, app
|
||||
}
|
||||
|
||||
// startBuffer starts a Buffer through fx, as cli.RunServer does, with
|
||||
// the DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set.
|
||||
// startBuffer starts a Buffer through fx, as main does, with the
|
||||
// DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set.
|
||||
func startBuffer(t *testing.T) *reportbuf.Buffer {
|
||||
t.Helper()
|
||||
|
||||
@@ -1074,54 +1074,9 @@ func TestReportsRefusedWhileWritesFail(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestHealthCheckFailsWhileWritesFail asks the health check handler
|
||||
// for its answer around a failed write. It answers 200 with
|
||||
// "status":"ok" at start, 503 with "status":"error" from the failed
|
||||
// write until a write succeeds, and 200 with "status":"ok" again after.
|
||||
func TestHealthCheckFailsWhileWritesFail(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
t.Setenv("DATA_DIR", dir)
|
||||
|
||||
buf, h := startBufferWithHandlers(t)
|
||||
|
||||
checkHealthCheck(t, h, "at start", http.StatusOK, "ok")
|
||||
|
||||
// A flush writes a file only when a report is waiting.
|
||||
err := buf.Append(map[string]string{"id": "kept"})
|
||||
if err != nil {
|
||||
t.Fatalf("append: %v", err)
|
||||
}
|
||||
|
||||
// With the data directory gone, the write has nowhere to go.
|
||||
err = os.Remove(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("remove data dir: %v", err)
|
||||
}
|
||||
|
||||
err = buf.Flush()
|
||||
if err == nil {
|
||||
t.Fatal("flush succeeded with the data directory gone")
|
||||
}
|
||||
|
||||
checkHealthCheck(t, h, "after the failed write",
|
||||
http.StatusServiceUnavailable, "error")
|
||||
|
||||
err = os.Mkdir(dir, 0o750)
|
||||
if err != nil {
|
||||
t.Fatalf("make data dir: %v", err)
|
||||
}
|
||||
|
||||
err = buf.Flush()
|
||||
if err != nil {
|
||||
t.Fatalf("flush with the data directory back: %v", err)
|
||||
}
|
||||
|
||||
checkHealthCheck(t, h, "after a write succeeded", http.StatusOK, "ok")
|
||||
}
|
||||
|
||||
// startBufferWithHandlers starts a Buffer and the Handlers that store
|
||||
// reports in it through fx, as cli.RunServer does, with the DATA_DIR
|
||||
// the calling test has set.
|
||||
// reports in it through fx, as main does, with the DATA_DIR the calling
|
||||
// test has set.
|
||||
func startBufferWithHandlers(
|
||||
t *testing.T,
|
||||
) (*reportbuf.Buffer, *handlers.Handlers) {
|
||||
@@ -1166,39 +1121,6 @@ func postReport(t *testing.T, h *handlers.Handlers, clientID string) int {
|
||||
return rec.Code
|
||||
}
|
||||
|
||||
// checkHealthCheck asks the health check handler for its answer, and
|
||||
// fails the test, saying when it asked, unless the answer has status
|
||||
// code wantCode and a "status" field of wantStatus.
|
||||
func checkHealthCheck(
|
||||
t *testing.T,
|
||||
h *handlers.Handlers,
|
||||
when string,
|
||||
wantCode int,
|
||||
wantStatus string,
|
||||
) {
|
||||
t.Helper()
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequestWithContext(t.Context(),
|
||||
http.MethodGet, "/.well-known/healthcheck", http.NoBody)
|
||||
h.HandleHealthCheck().ServeHTTP(rec, req)
|
||||
|
||||
var body struct {
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
err := json.Unmarshal(rec.Body.Bytes(), &body)
|
||||
if err != nil {
|
||||
t.Fatalf("health check %s: body not JSON: %v (%q)",
|
||||
when, err, rec.Body.String())
|
||||
}
|
||||
|
||||
if rec.Code != wantCode || body.Status != wantStatus {
|
||||
t.Fatalf("health check %s: %d with status %q, want %d with %q",
|
||||
when, rec.Code, body.Status, wantCode, wantStatus)
|
||||
}
|
||||
}
|
||||
|
||||
// TestConcurrentAppendsStopAtCap appends from many goroutines at once
|
||||
// with room for exactly roomFor reports: exactly that many must be
|
||||
// taken, which holds only if Append checks and counts each report
|
||||
|
||||
@@ -22,9 +22,9 @@ import (
|
||||
"go.uber.org/fx/fxtest"
|
||||
)
|
||||
|
||||
// newServer builds a Server from the same constructors as
|
||||
// cli.RunServer, configured from the environment. It is never started,
|
||||
// so nothing listens.
|
||||
// newServer builds a Server from the same constructors as main,
|
||||
// configured from the environment. It is never started, so nothing
|
||||
// listens.
|
||||
func newServer(t *testing.T) *server.Server {
|
||||
t.Helper()
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ func New(
|
||||
s.SetupRoutes()
|
||||
s.httpServer = s.newHTTPServer()
|
||||
|
||||
go func() { s.listenAndServe() }()
|
||||
go s.listenAndServe()
|
||||
|
||||
return nil
|
||||
},
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "netwatch",
|
||||
"private": true,
|
||||
"version": "1.0.0",
|
||||
"description": "Real-time network latency monitor SPA",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -23,8 +23,8 @@ main() {
|
||||
"" | -* | *[[:space:]]*) usage ;;
|
||||
esac
|
||||
cd "$ROOT"
|
||||
"$ROOT/script/yarn" add --dev "$1"
|
||||
"$ROOT/script/yarn" install --frozen-lockfile
|
||||
yarn add --dev "$1"
|
||||
yarn install --frozen-lockfile
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+14
-55
@@ -3,14 +3,11 @@
|
||||
# this repo. Idempotent: every install is guarded by a check so already
|
||||
# installed tools are skipped. Base tooling comes from nix, apt, brew,
|
||||
# or apk (detected in that order); assumes nothing is present. Node is
|
||||
# used directly only if it has the major version of NODE_VERSION and is
|
||||
# at least NODE_MIN_VERSION; otherwise NODE_VERSION is installed via nvm
|
||||
# (installing nvm itself first, from a hash-verified release archive,
|
||||
# never curl | sh). Yarn is used directly only if it is exactly
|
||||
# YARN_VERSION; otherwise that version is prepared through corepack,
|
||||
# or installed with npm where there is no corepack.
|
||||
# Go, with its gofmt, is used directly only if it is exactly GO_VERSION;
|
||||
# otherwise that release is installed from its hash-verified archive.
|
||||
# used directly if it is at least NODE_MIN_VERSION; otherwise it is
|
||||
# installed at a pinned version via nvm (installing nvm itself first,
|
||||
# from a hash-verified release archive, never curl | sh). Go, with its
|
||||
# gofmt, is used directly only if it is exactly GO_VERSION; otherwise
|
||||
# that release is installed from its hash-verified archive.
|
||||
#
|
||||
# What this script installs outside the system package manager lives
|
||||
# under $HOME and is linked into ~/.local/bin, where make and the git
|
||||
@@ -145,63 +142,32 @@ ensure_nvm() {
|
||||
rm -rf "$tmp"
|
||||
}
|
||||
|
||||
# node_ok: the node on PATH has the major version of NODE_VERSION and is
|
||||
# at least NODE_MIN_VERSION. Only the major is compared with the pin,
|
||||
# as REPO_POLICIES.md asks. node itself compares the versions: major,
|
||||
# then minor, then patch. A node that fails does not pass.
|
||||
# script/bootstrap and script/yarn carry this function unchanged.
|
||||
# node_ok: the node on PATH is at least NODE_MIN_VERSION. node itself
|
||||
# compares the two: major, then minor, then patch.
|
||||
node_ok() {
|
||||
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||
if missing node; then return 1; fi
|
||||
node -e '
|
||||
const have = process.versions.node.split(".").map(Number);
|
||||
const want = process.argv[1].split(".").map(Number);
|
||||
const pinned = process.argv[2].split(".").map(Number);
|
||||
if (have[0] !== pinned[0]) process.exit(1);
|
||||
for (let i = 0; i < 3; i++) {
|
||||
if (have[i] !== want[i]) process.exit(have[i] > want[i] ? 0 : 1);
|
||||
}
|
||||
' "$NODE_MIN_VERSION" "$NODE_VERSION"
|
||||
' "$NODE_MIN_VERSION"
|
||||
}
|
||||
|
||||
# ensure_node: unless node_ok, install NODE_VERSION and link its node.
|
||||
# Afterwards node is looked up through PATH again and must pass node_ok.
|
||||
ensure_node() {
|
||||
if node_ok; then
|
||||
echo "bootstrap: using node $(node --version)"
|
||||
return 0
|
||||
fi
|
||||
if node_ok; then return 0; fi
|
||||
ensure_nvm
|
||||
nvm_sh "nvm install $NODE_VERSION"
|
||||
link_bin "$HOME/.nvm/versions/node/v$NODE_VERSION/bin/node" node
|
||||
hash -r
|
||||
if ! node_ok; then
|
||||
echo "bootstrap: after installing node $NODE_VERSION through nvm," >&2
|
||||
echo " the node on PATH, $(command -v node), is not it" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "bootstrap: installed node $(node --version)"
|
||||
}
|
||||
|
||||
# yarn_ok: yarn --version on PATH prints exactly YARN_VERSION, compared
|
||||
# over the whole version. A yarn that fails or prints anything else does
|
||||
# not pass.
|
||||
yarn_ok() {
|
||||
if missing yarn; then return 1; fi
|
||||
version="$(yarn --version 2>/dev/null)" || return 1
|
||||
[ "$version" = "$YARN_VERSION" ]
|
||||
}
|
||||
|
||||
# ensure_yarn: unless yarn_ok, prepare YARN_VERSION and link its yarn.
|
||||
# corepack writes its shims (pnpm and yarnpkg as well as yarn) into
|
||||
# $TOOLCHAIN rather than next to itself, and the npm fallback installs
|
||||
# there too; only yarn is linked. Afterwards yarn is looked up through
|
||||
# PATH again and must pass yarn_ok, so another yarn that hides the link
|
||||
# stops bootstrap.
|
||||
# ensure_yarn: corepack writes its shims (pnpm and yarnpkg as well as
|
||||
# yarn) into $TOOLCHAIN rather than next to itself, and the npm fallback
|
||||
# installs there too; only yarn is linked.
|
||||
ensure_yarn() {
|
||||
if yarn_ok; then
|
||||
echo "bootstrap: using yarn $(yarn --version)"
|
||||
return 0
|
||||
fi
|
||||
if ! missing yarn; then return 0; fi
|
||||
shims="$TOOLCHAIN/corepack-shims"
|
||||
mkdir -p "$shims"
|
||||
if ! missing corepack; then
|
||||
@@ -216,13 +182,6 @@ ensure_yarn() {
|
||||
shims="$TOOLCHAIN/npm-global/bin"
|
||||
fi
|
||||
link_bin "$shims/yarn" yarn
|
||||
hash -r
|
||||
if ! yarn_ok; then
|
||||
echo "bootstrap: after installing yarn $YARN_VERSION in $shims," >&2
|
||||
echo " the yarn on PATH, $(command -v yarn), does not report it" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "bootstrap: installed yarn $(yarn --version)"
|
||||
}
|
||||
|
||||
# go_ok: the go on PATH has its gofmt beside it (a Go release ships the
|
||||
|
||||
+4
-5
@@ -8,12 +8,11 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
"$ROOT/script/yarn" build
|
||||
yarn build
|
||||
|
||||
# VERSION comes from the environment. Unset or empty, it is
|
||||
# `git describe --tags --always --dirty`, as in script/docker, or
|
||||
# "dev" where there is no git or no repository history.
|
||||
version="${VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo dev)}"
|
||||
# VERSION comes from the environment. Unset or empty, it is git
|
||||
# describe, or "dev" where there is no git or no repository history.
|
||||
version="${VERSION:-$(git describe --always --dirty 2>/dev/null || echo dev)}"
|
||||
|
||||
CGO_ENABLED=0 go build -trimpath \
|
||||
-ldflags "-s -w -X main.Version=$version" \
|
||||
|
||||
+1
-1
@@ -7,7 +7,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
"$ROOT/script/yarn" dev
|
||||
yarn dev
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
+1
-1
@@ -10,7 +10,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
"$ROOT/script/yarn" prettier --write '**/*.{js,css,html,md}'
|
||||
yarn prettier --write '**/*.{js,css,html,md}'
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
@@ -7,7 +7,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
"$ROOT/script/yarn" prettier --check '**/*.{js,css,html,md}'
|
||||
yarn prettier --check '**/*.{js,css,html,md}'
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
@@ -14,14 +14,12 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
NODE_OPTIONS=--test-reporter=dot \
|
||||
timeout 90 "$ROOT/script/yarn" --silent run test || {
|
||||
NODE_OPTIONS=--test-reporter=dot timeout 90 yarn --silent run test || {
|
||||
echo "--- Rerunning with every test listed for details ---"
|
||||
NODE_OPTIONS=--test-reporter=spec \
|
||||
timeout 90 "$ROOT/script/yarn" --silent run test
|
||||
NODE_OPTIONS=--test-reporter=spec timeout 90 yarn --silent run test
|
||||
exit 1
|
||||
}
|
||||
timeout 30 "$ROOT/script/yarn" build
|
||||
timeout 30 yarn build
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
||||
-55
@@ -1,55 +0,0 @@
|
||||
#!/bin/sh
|
||||
# script/yarn: run yarn with the given arguments under the node
|
||||
# script/bootstrap would use. Every script that runs yarn on the host
|
||||
# runs it through this one, except script/bootstrap, which picks its
|
||||
# own node and yarn.
|
||||
set -eu
|
||||
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
|
||||
# Must match the pins in script/bootstrap.
|
||||
NODE_VERSION="22.17.0"
|
||||
NODE_MIN_VERSION="22.13.0"
|
||||
|
||||
# node_ok: the node on PATH has the major version of NODE_VERSION and is
|
||||
# at least NODE_MIN_VERSION. Only the major is compared with the pin,
|
||||
# as REPO_POLICIES.md asks. node itself compares the versions: major,
|
||||
# then minor, then patch. A node that fails does not pass.
|
||||
# script/bootstrap and script/yarn carry this function unchanged.
|
||||
node_ok() {
|
||||
if ! command -v node >/dev/null 2>&1; then return 1; fi
|
||||
node -e '
|
||||
const have = process.versions.node.split(".").map(Number);
|
||||
const want = process.argv[1].split(".").map(Number);
|
||||
const pinned = process.argv[2].split(".").map(Number);
|
||||
if (have[0] !== pinned[0]) process.exit(1);
|
||||
for (let i = 0; i < 3; i++) {
|
||||
if (have[i] !== want[i]) process.exit(have[i] > want[i] ? 0 : 1);
|
||||
}
|
||||
' "$NODE_MIN_VERSION" "$NODE_VERSION"
|
||||
}
|
||||
|
||||
# Run the yarn script/bootstrap installed: the yarn on PATH when the
|
||||
# node on PATH passes node_ok, and otherwise yarn under the pinned node
|
||||
# in nvm. nvm is a bash script, hence the subshell.
|
||||
run_yarn() {
|
||||
if node_ok && command -v yarn >/dev/null 2>&1; then
|
||||
exec yarn "$@"
|
||||
fi
|
||||
if node_ok || [ ! -s "$HOME/.nvm/nvm.sh" ]; then
|
||||
echo "script/yarn: no yarn; run script/bootstrap first" >&2
|
||||
exit 1
|
||||
fi
|
||||
exec bash -c '. "$HOME/.nvm/nvm.sh" && nvm use "$1" >/dev/null &&
|
||||
shift && exec yarn "$@"' bash "$NODE_VERSION" "$@"
|
||||
}
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
# script/bootstrap links the node and yarn it installs into
|
||||
# ~/.local/bin, which the shell that called it may not have on PATH.
|
||||
PATH="$HOME/.local/bin:$PATH"
|
||||
run_yarn "$@"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -180,91 +180,6 @@ test("a check fetches the target's URL as written, with no query string added",
|
||||
);
|
||||
});
|
||||
|
||||
test("a check sends no Referer header, wherever the page is served from", async (t) => {
|
||||
mockTargets(t, () => 10);
|
||||
const check = measureLatency("https://fsn1-speed.hetzner.com");
|
||||
t.mock.timers.tick(10);
|
||||
assert.notEqual(await settled(check), "still waiting");
|
||||
assert.equal(
|
||||
fetch.mock.calls[0].arguments[1].referrerPolicy,
|
||||
"no-referrer",
|
||||
);
|
||||
});
|
||||
|
||||
// These targets' home pages answer with a redirect. A check follows it and
|
||||
// times the request to the new URL too, often on another host. The Google
|
||||
// Cloud Console redirects only requests that do not say they come from
|
||||
// another site, such as curl's. Each target is checked at /robots.txt on
|
||||
// its own host, which answers without a redirect. An S3 region answers it
|
||||
// with 403, and any answer counts as reachable.
|
||||
test("a target whose home page redirects is checked at /robots.txt on its own host", () => {
|
||||
const checked = new Map(
|
||||
new AppState([]).wan.map((host) => [host.name, host.url]),
|
||||
);
|
||||
for (const [name, url] of [
|
||||
["AWS Console", "https://console.aws.amazon.com/robots.txt"],
|
||||
["Google Cloud Console", "https://console.cloud.google.com/robots.txt"],
|
||||
["Akamai", "https://www.akamai.com/robots.txt"],
|
||||
["B2", "https://api.backblazeb2.com/robots.txt"],
|
||||
[
|
||||
"S3 af-south-1 (Cape Town)",
|
||||
"https://s3.af-south-1.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 eu-west-2 (London)",
|
||||
"https://s3.eu-west-2.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 me-south-1 (Bahrain)",
|
||||
"https://s3.me-south-1.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 ap-northeast-1 (Tokyo)",
|
||||
"https://s3.ap-northeast-1.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 ap-southeast-1 (Singapore)",
|
||||
"https://s3.ap-southeast-1.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 ap-southeast-2 (Sydney)",
|
||||
"https://s3.ap-southeast-2.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 us-west-2 (Oregon)",
|
||||
"https://s3.us-west-2.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 sa-east-1 (São Paulo)",
|
||||
"https://s3.sa-east-1.amazonaws.com/robots.txt",
|
||||
],
|
||||
]) {
|
||||
assert.equal(checked.get(name), url);
|
||||
}
|
||||
});
|
||||
|
||||
// Chrome keeps the connection to an S3 region open after a HEAD check, but
|
||||
// closes it after most GET checks. The Hetzner servers do not answer HEAD.
|
||||
test("a round checks each S3 region with a HEAD request and every other target with a GET request", async (t) => {
|
||||
const state = new AppState([
|
||||
{ name: "Local CPE", url: "http://192.168.100.1" },
|
||||
]);
|
||||
mockTargets(t, () => 10);
|
||||
const round = tick(state, new AbortController().signal);
|
||||
t.mock.timers.tick(10);
|
||||
assert.notEqual(await settled(round), "still waiting");
|
||||
const methods = new Map(
|
||||
fetch.mock.calls.map((call) => [
|
||||
call.arguments[0],
|
||||
call.arguments[1].method,
|
||||
]),
|
||||
);
|
||||
for (const host of state.allHosts) {
|
||||
const method = host.name.startsWith("S3 ") ? "HEAD" : "GET";
|
||||
assert.equal(methods.get(host.url), method, host.name);
|
||||
}
|
||||
});
|
||||
|
||||
test("at a 30000ms interval, a target answering after 1000ms shows in its row while another target's check is still waiting", async (t) => {
|
||||
CONFIG.updateInterval = 30000;
|
||||
const state = new AppState([
|
||||
|
||||
+91
-123
@@ -1,146 +1,114 @@
|
||||
# The viewport test
|
||||
# Responsive-layout harness
|
||||
|
||||
`make frontend-viewport-test` checks the page's layout at several window widths
|
||||
in a real browser:
|
||||
Automated verification of the responsive layout that landed in #5. Run it with:
|
||||
|
||||
```bash
|
||||
make frontend-viewport-test
|
||||
```
|
||||
|
||||
First it runs the unit tests. Then it builds the page into `dist/`. It serves
|
||||
`dist/` with the nginx image and the `configs/nginx.conf` that the Docker image
|
||||
ships. Before the browser starts, it fetches `/` and `/no-such-page`, which
|
||||
nginx answers with the page. Both must carry `Cache-Control: no-cache` and a
|
||||
`Content-Security-Policy` header. Then it starts a headless Chrome in a
|
||||
container and opens the page in it at each width. For each width it measures the
|
||||
layout inside the page and checks the measurements in Node. The checks decide
|
||||
pass or fail. Screenshots and a `results.json` land in `tmp/viewport/`, for a
|
||||
person to look at when a check fails.
|
||||
It builds `dist/`, serves it from the same digest-pinned `nginx` image and the
|
||||
same `configs/nginx.conf` the shipping container uses, drives a digest-pinned
|
||||
headless Chrome against it over CDP, and asserts on computed layout at every
|
||||
viewport width derived from the app's own CSS. Screenshots land in
|
||||
`tmp/viewport/` alongside a `results.json`; they are artifacts for a human to
|
||||
look at when something fails, not the evidence. The assertions are the evidence.
|
||||
|
||||
The test is not part of `make check`: it takes minutes, and `make test` has to
|
||||
finish in under 60 seconds.
|
||||
The target is deliberately outside `make check`: it takes minutes, and
|
||||
`make test` has to stay under 60 seconds.
|
||||
|
||||
## The widths
|
||||
## How the widths are chosen
|
||||
|
||||
The breakpoints are read from the page's own files. `viewports.js` reads the
|
||||
`min-width` and `max-width` conditions out of the `@media` blocks in
|
||||
`web/src/styles.css`. It also looks in `web/src/main.js` and `web/index.html`
|
||||
for Tailwind's responsive class prefixes, such as `md:`, and takes their
|
||||
breakpoints. Each breakpoint is tested at one pixel below it, exactly on it, and
|
||||
one pixel above it. Today the CSS has one breakpoint, `max-width: 768px`, and no
|
||||
responsive prefix is used.
|
||||
Not from a list of phone models. `viewports.js` parses the `@media` conditions
|
||||
out of `web/src/styles.css` and scans `web/src/main.js` and `web/index.html` for
|
||||
Tailwind responsive prefixes, then tests every breakpoint it finds at one pixel
|
||||
below it, exactly on it, and one pixel above it. A generic 375px "phone" test
|
||||
sails straight past an off-by-one at a media query boundary; `max-width: 768px`
|
||||
matches _at_ 768, and the sweep pins down which side of that line each layout is
|
||||
on.
|
||||
|
||||
A `max-width: 768px` block matches at 768 pixels. Testing both sides of each
|
||||
breakpoint catches a layout that switches one pixel too early or too late. A
|
||||
second media block, or a first `md:` class, is covered without a change to this
|
||||
directory.
|
||||
Nothing hardcodes 768. Add a second media block or start using `md:` classes and
|
||||
the new breakpoint is covered without this directory being touched. The app is
|
||||
desktop-first today (all narrow rules live in `max-width` blocks); a
|
||||
`min-width`-only, mobile-first set is handled as its inverse, and a set that
|
||||
mixes the two makes the run fail loudly rather than test the right widths with
|
||||
the wrong expectation. Four further viewports are fixed anchors, each with a
|
||||
stated reason: a 320px floor, a 1280px desktop baseline, and two phone-landscape
|
||||
sizes straddling the breakpoint for the rotation case.
|
||||
|
||||
The page is desktop-first: the wide layout is the default, and every narrow rule
|
||||
is in a `max-width` block. So a width is expected to show the narrow layout when
|
||||
one of those blocks matches it. A page with only `min-width` blocks would be
|
||||
read the other way round. A page that mixes the two kinds fails the run with a
|
||||
message, because the right expectation for each width can no longer be read from
|
||||
the breakpoints alone.
|
||||
## What it asserts
|
||||
|
||||
Four more widths are fixed, each for a stated reason:
|
||||
- **app-rendered** — enough host rows exist and enough of them show a numeric
|
||||
latency. This one exists so the rest cannot pass vacuously against a blank
|
||||
page.
|
||||
- **no-horizontal-overflow** — `documentElement.scrollWidth` fits the layout
|
||||
viewport, with the widest offending element named.
|
||||
- **nothing-past-viewport-edge** — no visible element's box extends past the
|
||||
viewport edge.
|
||||
- **no-clipped-text** — nothing hides text behind `overflow: hidden`. Deliberate
|
||||
ellipsis truncation (Tailwind's `truncate`, used on host names and URLs) is
|
||||
excluded: it is a design choice, not breakage.
|
||||
- **tap-targets-44px** — every interactive control is at least 44x44 CSS px on
|
||||
touch viewports, _and_ each selector in the control list matched at least the
|
||||
number of visible elements it declares: one of each single control, and one
|
||||
pin button per WAN host row. The second half is what stops the check passing
|
||||
vacuously: with size alone, a renamed class would take its controls out of the
|
||||
measured set and the check would report "all 0 controls are at least 44x44"
|
||||
and pass. See below.
|
||||
- **host-rows-stacked / host-rows-side-by-side** — the rows genuinely reflow.
|
||||
Computed `flex-direction` _and_ the actual geometry are checked, and in the
|
||||
narrow layout the info block and the sparkline must each occupy essentially
|
||||
the full row width. A row that merely shrank its 420px column would fail.
|
||||
- **probing-still-runs / gateway-detection-still-runs** — narrow viewports keep
|
||||
probing and keep detecting the gateway. The mobile early-return path proposed
|
||||
in #8 was rejected; this is what would catch it coming back.
|
||||
|
||||
- 320 by 568 pixels: the narrowest width still in common use
|
||||
- 667 by 375 pixels: a phone in landscape, inside the narrow layout
|
||||
- 844 by 390 pixels: a large phone in landscape, in the wide layout but still a
|
||||
touch device
|
||||
- 1280 by 800 pixels: a desktop window
|
||||
### The tap-target threshold
|
||||
|
||||
Every width but 1280 by 800 is tested as a touch device.
|
||||
44x44 CSS px. That is the figure in Apple's Human Interface Guidelines and in
|
||||
WCAG 2.2 SC 2.5.5 "Target Size (Enhanced)". WCAG 2.2 SC 2.5.8 (level AA) sets a
|
||||
lower 24x24 floor, but that floor comes with a spacing exception these controls
|
||||
do not qualify for — the pin buttons sit directly against the host name they
|
||||
belong to.
|
||||
|
||||
## The checks
|
||||
## Determinism
|
||||
|
||||
A failed check does not stop the others.
|
||||
The browser container runs on an `--internal` docker network and has no route to
|
||||
the internet, so the app's latency probes cannot reach anything real. The
|
||||
harness answers them itself from a fixed delay table, with a deterministic
|
||||
fraction failed outright, so the rows render a realistic spread of one-, two-
|
||||
and three-digit latencies plus some unreachable rows. That spread is what the
|
||||
layout has to survive; a `---` placeholder in every row would not exercise it.
|
||||
|
||||
- `app-rendered`: at least 10 WAN host rows exist, and at least 5 rows show a
|
||||
number as their latency. Without this, the other checks could pass against a
|
||||
blank page.
|
||||
- `no-horizontal-overflow`: the document is no wider than the window. On a
|
||||
failure, the widest content is named.
|
||||
- `nothing-past-viewport-edge`: no visible element extends past the window's
|
||||
edge.
|
||||
- `no-clipped-text`: no element hides its text behind `overflow: hidden` or
|
||||
`overflow: clip`. Text cut with an ellipsis is allowed: that cut is a design
|
||||
choice, not a defect. Tailwind's `truncate` cuts host names and URLs that way.
|
||||
- `tap-targets-44px`, on touch widths: each measured control is at least 44 by
|
||||
44 CSS pixels. The controls are `#pause-btn`, `#interval-select`,
|
||||
`#debug-toggle` and the `.pin-btn` in each WAN host row. The check also
|
||||
demands that each selector matches enough visible elements: one of each single
|
||||
control, and one pin button per WAN host row. A renamed class would otherwise
|
||||
take its controls out of the measured set, and the check would pass on
|
||||
nothing.
|
||||
- `host-rows-stacked` or `host-rows-side-by-side`: each host row shows the
|
||||
layout expected at that width. In the narrow layout, the row's
|
||||
`flex-direction` is `column`, the graph sits below the host's details, and
|
||||
each takes at least 90% of the row's width. In the wide layout, the direction
|
||||
is `row` and the graph sits to the right of the details. A row that only
|
||||
shrank its fixed 420-pixel column fails.
|
||||
- `probing-still-runs`: the page sent at least one check to a target at this
|
||||
width. The narrow layout must not skip the checks.
|
||||
- `gateway-detection-still-runs`: the `Local Gateway` row is present at this
|
||||
width, so the gateway was looked for and found. The narrow layout must not
|
||||
skip that search.
|
||||
## What this cannot verify
|
||||
|
||||
### The tap-target size
|
||||
Real limits, so nobody re-parks this issue as needing hardware:
|
||||
|
||||
44 by 44 CSS pixels is the figure in Apple's Human Interface Guidelines and in
|
||||
WCAG 2.2 success criterion 2.5.5, "Target Size (Enhanced)". WCAG 2.2 criterion
|
||||
2.5.8 sets a lower minimum of 24 by 24 pixels, with an exception that depends on
|
||||
the space around a control. The pin buttons sit against the host name they
|
||||
belong to, so the test keeps 44.
|
||||
- **Non-Chromium engines.** This is Chrome. iOS Safari is WebKit and cannot be
|
||||
emulated by it; Safari-specific bugs (viewport units under a collapsing URL
|
||||
bar, `-webkit-fill-available`, form control metrics) will not show up here.
|
||||
- **Real touch input.** `hasTouch` emulation changes what the page is told, not
|
||||
how a finger behaves. Gesture handling, scroll momentum, double-tap zoom and
|
||||
hover-state fallbacks on touch are out of scope.
|
||||
- **Physical pixel density and rendering.** `deviceScaleFactor` is set, but
|
||||
subpixel antialiasing, OLED colour rendering and actual legibility at a given
|
||||
physical size are not measurable here.
|
||||
- **Fonts.** The container has DejaVu, not the platform's own UI monospace. Text
|
||||
metrics are therefore close to, but not identical to, a real device — a layout
|
||||
that fits here by a few pixels might not there.
|
||||
- **On-device performance.** Canvas sparkline redraw cost, battery, and
|
||||
behaviour on a slow radio are not measured.
|
||||
- **Browser chrome.** The address bar, safe-area insets and notch cutouts are
|
||||
not simulated.
|
||||
|
||||
## The browser's network
|
||||
|
||||
The browser container runs on an internal Docker network with no route to the
|
||||
internet, so the page's checks reach no real host. The test answers them itself
|
||||
from a fixed table of delays: 2, 45, 123, 456 and 780 milliseconds. Which delay
|
||||
a target gets depends on its URL, and about one target in seven fails outright.
|
||||
So the rows show one-, two- and three-digit latencies and some unreachable rows,
|
||||
and the layout is tested against that spread. The request to
|
||||
`http://192.168.1.1` is answered after 5 milliseconds, so the `Local Gateway`
|
||||
row appears.
|
||||
|
||||
At each width, the test sets the interval to 1 second and waits until at least 5
|
||||
rows show a latency before it measures. The page discards its first round, so
|
||||
this reaches a filled page without waiting out two rounds at 3 seconds.
|
||||
|
||||
## What the test cannot check
|
||||
|
||||
- Other browser engines. The test runs Chrome. Safari uses WebKit, and bugs that
|
||||
only WebKit has do not show up here.
|
||||
- Real touch input. Touch emulation changes what the page is told, not how a
|
||||
finger behaves. Gestures, scroll momentum, double-tap zoom and hover states on
|
||||
touch are not tested.
|
||||
- Physical pixels. The device pixel ratio is set, but antialiasing, colour
|
||||
rendering and legibility at a given physical size are not measured.
|
||||
- Fonts. The container has DejaVu, not a phone's own monospace font. Text is
|
||||
about as wide as on a real device, but not exactly. A layout that fits here by
|
||||
a few pixels might not fit there.
|
||||
- Performance. The cost of redrawing the graphs, battery use and behaviour on a
|
||||
slow connection are not measured.
|
||||
- The browser's own frame. The address bar, safe-area insets and notch cutouts
|
||||
are not simulated.
|
||||
|
||||
## The files
|
||||
|
||||
- `viewports.js` derives the widths from the sources.
|
||||
- `facts.js` is run inside the page and measures the layout. It only measures;
|
||||
it decides nothing.
|
||||
- `checks.js` turns the measurements into the pass or fail of each check, with
|
||||
the numbers that decided it.
|
||||
- `harness.js` connects to the browser, runs each width, writes the screenshots
|
||||
and `results.json`, and prints the results. `script/frontend-viewport-test`
|
||||
starts it; every input it needs arrives in the environment.
|
||||
Everything else this issue was actually about — does the layout reflow, does
|
||||
anything overflow, is content clipped, are the controls big enough — is a
|
||||
function of viewport width and CSS, and is covered above.
|
||||
|
||||
## Relation to the unit tests
|
||||
|
||||
The unit tests in `test/unit/` call the functions that `web/src/main.js`
|
||||
exports, with no browser. `make test` runs them with Node's built-in test
|
||||
runner. This test runs the built page in a real browser engine and is the only
|
||||
test here that can see a media query. Neither replaces the other. Checks on
|
||||
computed styles and element geometry belong here; checks on functions belong in
|
||||
`test/unit/`.
|
||||
Complementary layers, not two stacks. The unit tests in `test/unit/`, which
|
||||
`make test` runs with Node's built-in test runner, exercise the functions
|
||||
`web/src/main.js` exports in-process with no browser. This harness exercises
|
||||
rendered layout in a real engine and is the only thing here that can see a media
|
||||
query. Neither replaces the other; assertions about computed styles and element
|
||||
geometry belong here, assertions about functions belong in `test/unit/`.
|
||||
|
||||
+25
-44
@@ -67,66 +67,51 @@ export const CONFIG = {
|
||||
|
||||
// WAN endpoints to monitor. These are used for the aggregate health/stats
|
||||
// display (min/max/avg). Ordered: personal, LLM APIs, big-3 cloud, then
|
||||
// CDN/hosting/other. A check follows a redirect and times the request to
|
||||
// the new URL too. So a target whose home page redirects is checked at
|
||||
// /robots.txt on its own host. That URL answers without a redirect.
|
||||
// CDN/hosting/other.
|
||||
const WAN_HOSTS = [
|
||||
{ name: "datavi.be", url: "https://datavi.be" },
|
||||
{ name: "Anthropic API", url: "https://api.anthropic.com" },
|
||||
{ name: "OpenAI API", url: "https://api.openai.com" },
|
||||
{ name: "AWS Console", url: "https://console.aws.amazon.com/robots.txt" },
|
||||
{
|
||||
name: "Google Cloud Console",
|
||||
url: "https://console.cloud.google.com/robots.txt",
|
||||
},
|
||||
{ name: "AWS Console", url: "https://console.aws.amazon.com" },
|
||||
{ name: "Google Cloud Console", url: "https://console.cloud.google.com" },
|
||||
{ name: "Microsoft Azure", url: "https://portal.azure.com" },
|
||||
{ name: "Cloudflare", url: "https://www.cloudflare.com" },
|
||||
{ name: "Fastly CDN", url: "https://www.fastly.com" },
|
||||
{ name: "Akamai", url: "https://www.akamai.com/robots.txt" },
|
||||
{ name: "Akamai", url: "https://www.akamai.com" },
|
||||
{ name: "Google", url: "https://www.google.com" },
|
||||
{ name: "GitHub", url: "https://github.com" },
|
||||
{ name: "B2", url: "https://api.backblazeb2.com/robots.txt" },
|
||||
// Chrome keeps the connection to an S3 region open after a HEAD check,
|
||||
// but closes it after most GET checks.
|
||||
{ name: "B2", url: "https://api.backblazeb2.com" },
|
||||
{
|
||||
name: "S3 af-south-1 (Cape Town)",
|
||||
url: "https://s3.af-south-1.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.af-south-1.amazonaws.com",
|
||||
},
|
||||
{
|
||||
name: "S3 eu-west-2 (London)",
|
||||
url: "https://s3.eu-west-2.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.eu-west-2.amazonaws.com",
|
||||
},
|
||||
{
|
||||
name: "S3 me-south-1 (Bahrain)",
|
||||
url: "https://s3.me-south-1.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.me-south-1.amazonaws.com",
|
||||
},
|
||||
{
|
||||
name: "S3 ap-northeast-1 (Tokyo)",
|
||||
url: "https://s3.ap-northeast-1.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.ap-northeast-1.amazonaws.com",
|
||||
},
|
||||
{
|
||||
name: "S3 ap-southeast-1 (Singapore)",
|
||||
url: "https://s3.ap-southeast-1.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.ap-southeast-1.amazonaws.com",
|
||||
},
|
||||
{
|
||||
name: "S3 ap-southeast-2 (Sydney)",
|
||||
url: "https://s3.ap-southeast-2.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.ap-southeast-2.amazonaws.com",
|
||||
},
|
||||
{
|
||||
name: "S3 us-west-2 (Oregon)",
|
||||
url: "https://s3.us-west-2.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.us-west-2.amazonaws.com",
|
||||
},
|
||||
{
|
||||
name: "S3 sa-east-1 (São Paulo)",
|
||||
url: "https://s3.sa-east-1.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
url: "https://s3.sa-east-1.amazonaws.com",
|
||||
},
|
||||
// Hetzner regional speed test servers — genuine per-DC endpoints
|
||||
{
|
||||
@@ -286,7 +271,6 @@ export class HostState {
|
||||
constructor(host, pinned = false) {
|
||||
this.name = host.name;
|
||||
this.url = host.url;
|
||||
this.method = host.method;
|
||||
// Each entry is either a check's result, { timestamp, latency,
|
||||
// error }, or a round skipped while paused, { timestamp,
|
||||
// latency: null, paused: true }.
|
||||
@@ -583,18 +567,16 @@ export class Reporter {
|
||||
|
||||
// --- Latency Measurement -----------------------------------------------------
|
||||
|
||||
// Checks one target with the given HTTP method, GET when none is given. The
|
||||
// check times out after CONFIG.requestTimeout; the caller can give it up
|
||||
// sooner through the optional signal, which also ends it as a timeout. A
|
||||
// failed check's reason says what went wrong and after how long; a check
|
||||
// that answered has none.
|
||||
// Checks one target. The check times out after CONFIG.requestTimeout; the
|
||||
// caller can give it up sooner through the optional signal, which also ends
|
||||
// it as a timeout. A failed check's reason says what went wrong and after
|
||||
// how long; a check that answered has none.
|
||||
//
|
||||
// The Hetzner speed-test servers close the connection without an answer
|
||||
// when the URL has a query string or the request has a Referer header. So
|
||||
// the URL is fetched as written, and referrerPolicy: "no-referrer" sends
|
||||
// no Referer header, wherever the page is served from. cache: "no-store"
|
||||
// keeps the browser's cache out of the measurement.
|
||||
export async function measureLatency(url, signal, method = "GET") {
|
||||
// The URL is fetched as written, with nothing added to it: the Hetzner
|
||||
// speed-test servers close the connection without an answer when the URL
|
||||
// has a query string, and cache: "no-store" keeps the browser's cache out
|
||||
// of the measurement.
|
||||
export async function measureLatency(url, signal) {
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
@@ -606,10 +588,9 @@ export async function measureLatency(url, signal, method = "GET") {
|
||||
|
||||
try {
|
||||
await fetch(url, {
|
||||
method,
|
||||
method: "GET",
|
||||
mode: "no-cors",
|
||||
cache: "no-store",
|
||||
referrerPolicy: "no-referrer",
|
||||
signal: controller.signal,
|
||||
});
|
||||
const latency = Math.round(performance.now() - start);
|
||||
@@ -1309,7 +1290,7 @@ export async function tick(state, signal, onOffline) {
|
||||
// the round started.
|
||||
await Promise.all(
|
||||
state.allHosts.map(async (host) => {
|
||||
const r = await measureLatency(host.url, signal, host.method);
|
||||
const r = await measureLatency(host.url, signal);
|
||||
if (state.paused || signal.aborted || state.tickCount === 0) {
|
||||
return;
|
||||
}
|
||||
@@ -1382,7 +1363,7 @@ function startRecoveryProbe(state, startRounds) {
|
||||
const checks = new AbortController();
|
||||
state._recoveryProbeChecks = checks;
|
||||
for (const host of canaries) {
|
||||
measureLatency(host.url, checks.signal, host.method).then((r) => {
|
||||
measureLatency(host.url, checks.signal).then((r) => {
|
||||
if (r.error !== null || checks.signal.aborted) return;
|
||||
log.notice("Recovery probe: connectivity detected");
|
||||
stopRecoveryProbe(state);
|
||||
|
||||
Reference in New Issue
Block a user