diff --git a/README.md b/README.md index 6592f5b..3f51cc2 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,17 @@ -NetWatch is an MIT-licensed JavaScript single-page application by -[@sneak](https://sneak.berlin) that provides real-time network latency -monitoring to common internet hosts, displayed with color-coded figures and -sparkline graphs, served from a static bucket or from its Docker image, where a -small Go backend stores the measurements the page reports. +NetWatch is an MIT-licensed single-page web application by +[@sneak](https://sneak.berlin) that shows the latency from your browser to 26 +well-known internet hosts. The page is JavaScript and does all its measuring in +the browser. A small Go backend, `netwatch-server`, stores the measurements the +page reports. The page and the backend ship together in one Docker image. + +## Rationale + +When a network misbehaves, it helps to see at a glance which hosts answer and +how fast. NetWatch is one page with a latency figure and a small graph for each +host, refreshed every few seconds. The page measures from the browser, so it +shows the path your own traffic takes. It can be served from any static file +host. The backend only stores what the page reports. Without it, the page works +the same and nothing is stored. ## Getting Started @@ -10,10 +19,10 @@ small Go backend stores the measurements the page reports. # Install the dependencies and the git pre-commit hook make setup -# Run the page on the Vite dev server +# Serve the page from the Vite dev server make dev -# Build netwatch-server and run it on port 8080 +# Build the page and netwatch-server, then run the server on port 8080 make run # Run the tests, both linters and the format check @@ -22,593 +31,417 @@ make check # Build the page into dist/ and netwatch-server into the repo root make build -# Build the image and run it +# Build the Docker image and run it make docker docker run -p 8080:8080 netwatch ``` -`make check` and `make docker` need Docker. `make dev` passes `/api` to -`http://127.0.0.1:8080`, where `make run` starts `netwatch-server` with its -defaults, so the reports the page posts are stored in `data/reports`. +`make check` and `make docker` need Docker. `make run` starts `netwatch-server` +on port 8080 with its defaults, so the page's reports are stored in +`data/reports`. `make dev` passes the page's `/api` requests there. -## Entrypoints +## The page -This repository adheres to the -[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) -standard: normalized scripts in `script/` are the entrypoints for the -development workflow, and the Makefile targets are thin shims that call them. -The scripts cover both the frontend and the Go backend, so `make check` fails if -either one is broken. We provide: +The page shows one row per host: the host's name, its URL, its latest latency in +milliseconds and a graph of its last 100 checks. Below the figure are the +minimum, median, average and maximum of the host's answers in that history, or +`timeout` or `unreachable` after a failed check. A round of checks runs every 3 +seconds by default. The header shows the interval, the history's span, and +whether the page is running or paused. -- `script/bootstrap` — install all dependencies (the pinned node via nvm unless - the `node` on `PATH` has the pinned major version and is new enough for the - frontend's dependencies, yarn `YARN_VERSION` via corepack, or npm where there - is no corepack, unless the `yarn` on `PATH` is exactly that version, - `yarn install --frozen-lockfile`, Go `GO_VERSION` unless the `go` on `PATH` is - exactly that version, and the Go modules), linking what it installs itself - into `~/.local/bin`, which has to be on `PATH`. It installs no Go linter and - not Docker: `make test` and `make lint` run in Docker -- `script/setup` — make a fresh clone ready for development: bootstrap plus the - git pre-commit hook -- `script/dev` — run the Vite dev server, which proxies `/api` to a locally - running `netwatch-server` -- `script/build` — build the frontend for production into `dist/`, then the - static `netwatch-server` binary into the repo root with its version stamped - in: `VERSION` from the environment, or when that is unset or empty, - `git describe --tags --always --dirty` inside a git checkout, as - `script/docker` does, then `dev` -- `script/run` — build with `script/build`, then run `netwatch-server` with its - defaults (see [Settings](#settings)) -- `script/clean` — remove what `script/build` writes -- `script/projectname` — print the project name (used for the Docker image tag) -- `script/yarn` — run yarn with the given arguments under the node - `script/bootstrap` would use: with `~/.local/bin` first on `PATH`, the first - `node` on that `PATH` when it has the pinned major version and is new enough - for the frontend's dependencies, and otherwise the pinned node in nvm. Every - script that runs yarn on the host runs it through this one, except - `script/bootstrap`, which picks its own node and yarn -- `script/test` — build the `test` phase of `Dockerfile`, writing no image: the - frontend's unit tests in its `frontend-test` stage, and the backend's Go tests - with the race detector and coverage. The frontend's production build is left - to the image build, in the `frontend-build` stage -- `script/lint` — build the `lint` phase of `Dockerfile`, writing no image: - eslint in its `frontend-lint` stage, and golangci-lint over the Go packages -- `script/fmt` — format all files (writes): prettier over the JavaScript, CSS, - HTML and Markdown, then gofmt over `cmd/` and `internal/`. It runs on the - host, as `script/fmt-check` does, with `~/.local/bin` put on `PATH` -- `script/fmt-check` — check formatting (read-only): prettier, then gofmt -- `script/check` — run test, lint, and fmt-check -- `script/add-dependency` — add a frontend package, or move one to another - version: `make add-dependency PACKAGE=@` runs `yarn add --dev`, - which changes `package.json` and `yarn.lock` together, then - `yarn install --frozen-lockfile` -- `script/tidy` — run `go mod tidy`: to add a Go module, import it and run - `make tidy`; to move one to another version, edit its `require` line in - `go.mod`, then run `make tidy` -- `script/frontend-test` — run the unit tests in `test/unit/` on the host with - Node's built-in test runner, through the `test` script in `package.json`, and - if any fails, run them again listing every test, and fail; then the production - build. Each test run has a 90-second timeout. `make test` runs the same tests - in Docker, without the production build -- `script/frontend-fmt` — format the JavaScript, CSS, HTML and Markdown with - prettier (writes) -- `script/frontend-fmt-check` — check prettier formatting (read-only) -- `script/frontend-viewport-test` — responsive-layout verification of the built - frontend in a containerised headless Chrome (see - [test/viewport/README.md](test/viewport/README.md)). Before the browser - starts, it checks that the page, and a path that falls back to it, are served - with `Cache-Control: no-cache` and `Content-Security-Policy`. Not part of - `script/check`: it takes minutes. -- `script/docker` — build the image from `Dockerfile`, tagged `netwatch` via - `script/projectname` -- `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`, - then builds the image as `script/docker` does -- `script/precommit` — run by the git pre-commit hook; runs `script/check` -- `script/install-precommit` — install the git pre-commit hook +### Targets -`script/test`, `script/lint`, `script/docker` and `script/cibuild` each pass a -new `CACHEBUST` value, so every step in `Dockerfile` that lints, tests or -compiles runs again even when nothing has changed. Docker keeps the system -packages and the Go modules from earlier builds until the base image, the step -that installs them, or `go.mod` and `go.sum` change. The `frontend-build` stage -is the exception: it installs git again on every image build, because it starts -from the `frontend-test` stage, whose steps run again on every build. Go's build -cache and golangci-lint's are kept as well, and each reuses a result only when -everything it was made from is unchanged. Both `go test` runs carry `-count=1`, -so no test result is reused. The yarn packages are installed again on every -build. +The page checks 26 hosts on the internet, the WAN hosts: -## Responsive layout +- `datavi.be`, pinned to the top at start +- the Anthropic API and the OpenAI API +- the AWS Console, the Google Cloud Console and Microsoft Azure +- Cloudflare, Fastly, Akamai, Google, GitHub and Backblaze B2 +- 8 S3 regional endpoints: Cape Town, London, Bahrain, Tokyo, Singapore, Sydney, + Oregon and São Paulo +- 6 Hetzner speed-test servers: Nuremberg, Falkenstein, Helsinki, Ashburn, + Hillsboro and Singapore -The narrow-viewport layout lives in the `max-width: 768px` media block in -`web/src/styles.css`. It is verified automatically by -`make frontend-viewport-test`, which drives a digest-pinned headless Chrome -against the built `dist/` and asserts on computed layout at widths derived from -that CSS — on every breakpoint it declares and one pixel either side of it, plus -a 320px floor, a desktop baseline and two landscape sizes. See -[test/viewport/README.md](test/viewport/README.md) for what it covers and what -it genuinely cannot. +Two local targets follow under the heading "Local Network", left out of the +summary and the health box. `Local CPE` is the cable modem at +`http://192.168.100.1`. `Local Gateway` is the first of `http://192.168.1.1`, +`http://192.168.0.1`, `http://192.168.8.1` and `http://10.0.0.1` to answer +within 1.5 seconds of the page loading; with no answer, it has no row. Browsers +block a page served from a public address from reaching private addresses. So +the local targets answer only when the page is served from `localhost` or from a +private address. -## Rationale +The host list is written in `web/src/main.js`; there is no setting for it. -When debugging network issues, it's useful to have a persistent at-a-glance view -of latency and reachability to multiple well-known internet endpoints. NetWatch -provides this as a single page that does all its measuring in the browser, so it -can be served from anywhere static files are served. The backend in its Docker -image only stores the measurements the page reports; without it, the page works -the same and nothing is stored. +### What a figure measures -## Design +A check is a request to the target's URL: `GET`, or `HEAD` for the S3 regions. +The figure is the time from sending the request until the response headers +arrive, in milliseconds. Unlike ping, it includes the time the server takes to +answer. On a new connection it also includes the DNS lookup, the connection +setup and the TLS handshake. Any answer counts as reachable, an error page +included. The browser follows redirects, so a target whose home page redirects +is checked at `/robots.txt` on the same host. -The page's source is in `web/`, the Go backend's in `cmd/` and `internal/`, and -nginx's configuration in `configs/`. +A check times out after 80% of the interval: 2.4 seconds at the default. A check +that answers after that limit counts as a timeout too. A check that fails with a +network error is recorded as unreachable. The first round after the page loads +is discarded, as its figures include the connection setup. So is the first round +after an interval change. -The page is built with Vite and Tailwind CSS v4. Its code is all in -`web/src/main.js`, with a class-based architecture: +### Health and summary -- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks, - etc.). The interval menu sets `updateInterval`, the one value the page writes - into `CONFIG`; the timeouts, the time the history spans and the x-axis ticks - are computed from it -- **`HostState`**: Per-host state management — history buffer, latency tracking, - status transitions -- **`AppState`**: Top-level state container — WAN hosts, local hosts, pause - state, aggregate stats -- **`SparklineRenderer`**: Canvas 2D sparkline drawing with fixed axes, - color-coded line segments, error regions, and DPR-aware scaling -- **UI functions**: `buildUI()` constructs the DOM, `updateHostRow()` / - `updateSummary()` / `updateHealthBox()` handle incremental updates -- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's - sample and redrawing its row as soon as its check ends, then redraws every - row, the summary and the health box once the last check ends. The first round, - after loading or an interval change, is discarded. The rows are sorted when - the last check ends in round 2, the first one kept, and in rounds 11, 21, 31 - and so on. When paused, pushes blank markers (no probes, no false outage) -- **`Reporter`**: Posts collected samples to the backend +Above the rows, a health box sums up the WAN hosts' latest results. A host +counts as failed when its last check timed out or found it unreachable. -### Reporting +- `OFFLINE` (red): more than 10 hosts failed and at most 4 answered +- `DEGRADED` (orange): otherwise, more than 4 hosts failed +- `SLOW` (yellow): otherwise, more than 3 hosts took over 1000 ms +- `HEALTHY` (green): otherwise -Every `reportInterval` (default 60s) the page POSTs a JSON report to the -same-origin path `/api/v1/reports`: a random per-browser `clientId` kept in -`localStorage`, `geo` sent as null, and each host's unreported, non-paused -samples (timestamp, latency, error). A per-host high-water mark makes every -report a delta, so only new samples are sent; the mark advances only on a -delivered report, and while paused nothing is sent. Changing the interval clears -each host's history on the page; the samples in it not yet reported still go in -the next report. Delivery failure is quiet — one debug-log line per outage, -never blocking probing. The report-building step is a pure function of host -state. +Below it, a summary line covers the WAN hosts only: -A sample in the history that no report has delivered is sent only as long as the -history holds it, so after a refused or failed report the next report carries -the samples the history still has. Samples an interval change cleared from the -history are kept apart, the last 100 per host, until they are delivered. The -history keeps each host's last 100 rounds: 100 seconds at the 1-second interval, -200 seconds at 2 seconds, 5 minutes at the default of 3 seconds and 100 minutes -at 60 seconds. At 1 second, one refused or failed report loses the oldest 20 -seconds of each host's samples, as the next delivered report comes 2 minutes -after the last. At 2 seconds and longer one loses nothing; samples are lost from -the third refused report in a row at 2 seconds, from the fifth at 3 seconds, and -later still at the longer intervals. +- `Reachable`: how many of the 26 answered their last check +- `Now`: the minimum, median, average and maximum of their latest figures +- the history's span, such as `5m`: the minimum and maximum over the whole + history +- `Checks`: the rounds run since the page loaded or the interval changed -No report is sent while the page is paused, but each paused round still adds an -entry to the history, so each report due during a pause counts as a refused one. -A pause longer than the history spans loses the samples in the history recorded -between the last delivered report and the pause: up to a minute's worth for each -host if the report before the pause was delivered. +When no WAN host answers, the page checks 4 of them, picked at random, every +half second. As soon as one answers, a new round starts. -### Backend +### Rows, colors and controls -`netwatch-server` is a small Go HTTP server that stores the reports the page -posts. It keeps them in memory and writes them to `DATA_DIR` as zstd-compressed -files of JSON lines: every minute, whenever 10 MiB are waiting, and when it -stops. Its routes: +A figure and its line in the graph take the color of the legend in the footer, +from green under 50 ms to red from 500 ms up; gray is no answer. The graph's +axis ends at 1000 ms; a higher latency is drawn at the top, and the figure still +shows its real value. -- `POST /api/v1/reports` — takes a report, without credentials; each client - address may send a limited number a minute, and the report files are capped in - size, the oldest deleted first -- `GET /.well-known/healthcheck` — answers 200 with `"status":"ok"`, the - server's version and its uptime; while reports cannot be stored, as the last - write of a report file failed, it answers 503 with `"status":"error"` and the - same fields (see [Report storage](#report-storage)) -- `GET /metrics` — Prometheus metrics behind basic auth, only when - `METRICS_USERNAME` and `METRICS_PASSWORD` are set; each client address may - make a limited number of requests to it a minute +From time to time, the page sorts the WAN rows by their latest latency: fastest +first, failed last. The pin button at the left of a row keeps the row at the +top; pinned rows are sorted by name. `datavi.be` is pinned at start. -It is an `fx`-wired Go application: `cmd/netwatch-server/` starts it, and its -packages are in `internal/`: +`Pause` stops the checks: the graphs keep scrolling with a gap for each paused +round, and the figures turn gray. The `Interval` menu sets the time between +rounds: 1, 2, 3, 5, 10, 15, 30 or 60 seconds. Changing it clears the history and +starts a new round at once. The `Debug log` checkbox in the footer opens a panel +with the page's last 1000 log lines. -- **`cli`**: Runs `netwatch-server` from its command line: the `check-cidr` and - `prepare-data-dir` subcommands, which `bin/entrypoint.sh` runs before it - starts the server, and, given no arguments, the server, with the `fx` wiring. - It refuses any other arguments with a usage message and exit code 2. -- **`config`**: Loads configuration from environment variables and config files - via Viper. -- **`handlers`**: HTTP request handlers for the API (health check, report - ingestion). -- **`reportbuf`**: The in-memory buffer of reports and the files it writes them - to. -- **`server`**: Chi-based HTTP server with middleware wiring and route - registration. -- **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting - infrastructure. +## Reporting -In the image, the `test` phase of `Dockerfile` tests it, the `builder` stage -builds it, and `bin/entrypoint.sh` runs it as user `netwatch` on -`127.0.0.1:8081`, behind nginx (see [Deployment](#deployment)). Outside the -image, `make run` builds it and runs it on port 8080. +Every 60 seconds the page posts its undelivered samples as a report to +`/api/v1/reports` on the origin it was served from. The report is JSON with four +fields: -#### Settings +- `clientId`: a random id, made once per browser and kept in `localStorage`, or + once per session when the browser has no usable storage +- `geo`: always `null` +- `hosts`: one entry per host with new samples: its `name`, `url` and `status`, + and `history`, its samples. A sample has the check's time `t` in Unix + milliseconds, its `latency` (`null` for a failed check) and its `error` + (`"timeout"`, `"unreachable"` or `null`). +- `timestamp`: the time of the report -| Variable | Default | Description | -| ---------------------- | -------------------- | -------------------------------------------------------------------------------------------------------- | -| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface | -| `PORT` | `8080` | HTTP listen port | -| `DATA_DIR` | `./data/reports` | Directory for compressed reports | -| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) | -| `DEBUG` | empty | Debug logging; any non-empty value, `false` and `0` included, turns it on | -| `TRUSTED_PROXIES` | RFC 1918 ranges | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution | -| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) | -| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) | -| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) | -| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) | -| `SENTRY_DSN` | empty | DSN of the Sentry project to send errors to; see [Sentry](#sentry) | +Each report carries only the samples no earlier report has delivered. A sample +waits as long as the history holds it: the last 100 rounds of each host. An +interval change clears the history. The cleared samples are kept, the last 100 +of each host, until a report delivers them. Nothing is sent while the page is +paused. A report with no answer after 30 seconds is abandoned, and its samples +go in the next one. So a server that stored them but answered late gets them +twice. -`TRUSTED_PROXIES` defaults to `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. A list -replaces the default; set and empty, the server trusts no proxy. Loopback is not -in the default, so a reverse proxy on the same host has to be named, as in -`TRUSTED_PROXIES=127.0.0.1/32`. A request whose direct peer is outside this set -has its forwarded headers ignored, and the direct peer is logged and -rate-limited instead. From a direct peer inside the set, the client address is -the right-most `X-Forwarded-For` entry that is not in the set: each proxy -appends the address it took the request from, so the entries further left were -written by the client and are not used. An entry written as an address with a -port, such as `198.51.100.7:5000` or `[2001:db8::1]:443`, is read as that -address. If the search from the right finds only addresses in the set, or meets -an entry that is not an address at all, such as `unknown`, the direct peer is -used. `X-Real-IP` is read only when `X-Forwarded-For` is missing or empty. The -container image does not use this default; see [Deployment](#deployment). - -A variable set to a value the server cannot use, such as `PORT=abc` or a -`BIND_ADDRESS` that is not an IP address, stops it from starting, with an error -naming the variable. An empty variable counts as unset, except -`TRUSTED_PROXIES`. No value of `DEBUG` stops the start. - -#### Report storage - -Reports are written as `reports--.jsonl.zst` files in -`DATA_DIR`. The timestamp is in UTC to the millisecond, so the names sort by -time. The number starts at 1 when the server starts and goes up by one for each -file the server starts to write, so two files written in the same millisecond -still get different names. A failed write uses up its number and leaves a gap in -the numbers: its file, if it was created, is removed. The file stays, counted -toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too. -Files are created with `O_EXCL` to prevent overwrites. - -The reports of a failed write stay in memory, and the next flush, within a -minute, tries again to write them. The server writes one file at a time, so a -file holds its reports in the order they arrived, those of a failed write -included. From the failed write until a write succeeds, `POST /api/v1/reports` -refuses every report with 500 and keeps nothing of it, and the health check -answers 503 with `"status":"error"`. The page sends the refused samples again in -its next report, as long as it still holds them: at the 1-second interval one -refused report already loses some of those in the history (see -[Reporting](#reporting)). Reports already answered 200 are lost only if the -server stops before a write succeeds; the final flush at stop tries them too, -and if it fails, the process exits non-zero. - -#### Report limits - -`POST /api/v1/reports` takes reports from anyone who can reach it, without -credentials, so it is bounded instead. Each refusal below answers with the same -`{"status":"error"}` body as any other error, except nginx's refusal of a body -over the limit in the image. - -- **JSON only.** A report must be sent with `Content-Type: application/json`, - parameters such as `charset` allowed, as the page sends it; any other is - refused with 415, and nothing of it is stored. A page on another origin can - send a `text/plain` POST, or a form, without a CORS preflight; refusing them - keeps any site from making its visitors' browsers store reports (see - [CORS](#cors)). -- **Body limit.** A report body may be at most 1 MiB, the backend's limit for - any request. Run on its own, the backend answers 413 before the route to a - request that declares a longer body. Without a declared length, the route - answers 413 only when the report it reads runs past 1 MiB; nothing after the - end of the report is read, so a longer body whose report ends sooner is - stored, and one that stops being valid JSON sooner is answered 400. In the - image, nginx refuses a body over 1 MiB first, with or without a declared - length, as its `client_max_body_size`, which `configs/nginx.conf` does not - set, is 1 MiB by default. nginx answers 413 with its own HTML error page, and - the backend never sees the request. -- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may - send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with - `Retry-After: 60`. The minute slides: reports from the minute before still - count, fading out over the current one, so an address is sure never to be - refused only while it sends at most half of `REPORTS_PER_MINUTE` in any 60 - seconds. The page sends one report a minute from each open tab, so the default - of 60 refuses nothing from up to 30 tabs behind one address, such as a - household or an office sharing it, however their reports bunch up. Report - responses also carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and - `X-RateLimit-Reset` headers. -- **Size cap.** The report files in `DATA_DIR` may total at most - `DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports - waiting in memory count at their uncompressed size until they are written, - those of a failed write included; the part of its file written is removed. - When a report would take the total past the cap, the oldest report files are - deleted to make room, and each deletion is logged with the file's name and - size; a file still being written is never deleted. A report is refused with - 507, and nothing of it is stored, only when the reports waiting to be written - fill the cap on their own, and then no file is deleted. At start, report files - past the cap, as after lowering it, are deleted the same way. So the cap is - how much of the newest reports is kept: the default of 1 GiB is small enough - for any host; set it to the space you can give `DATA_DIR`. - -At the defaults, one address may send 60 reports of up to 1 MiB a minute, and a -report made up to compress badly takes about as much room in the report files as -its request body, so in about 17 minutes one address can fill the 1 GiB cap with -its own reports, every report stored before them deleted to make room. Lowering -`REPORTS_PER_MINUTE` or raising `DATA_DIR_MAX_BYTES` makes that take longer. - -#### CORS - -The page calls the API from the origin it is served from, so by default the -server sends no CORS headers, and browsers let no other origin's pages call it. -To serve the page from elsewhere, list that origin in `CORS_ALLOWED_ORIGINS` -(for example `https://netwatch.example.com`); pages from a listed origin may -`GET` and `POST` with a `Content-Type` header. Each entry must be a plain -origin, `scheme://host` with an optional `:port`, as browsers send it: no path, -not even a trailing `/`, and no `*`. Any other entry stops the server from -starting, with an error naming `CORS_ALLOWED_ORIGINS`. - -#### Metrics - -With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves -Prometheus metrics at `GET /metrics` to requests with those as their basic auth -credentials, and answers any other with 401. For each request that reaches the -health check or `POST /api/v1/reports`, those the rate limit refuses included, -the metrics record its duration and response size, labelled with its path, -method and status; they also count those requests in progress, and include Go's -runtime and process metrics. No other request is recorded: not those to -`/metrics` itself, and not those answered before they reach either route, such -as a CORS preflight, or a request refused with 404 for a path no route has or -405 for a method its route does not take. Run on its own, the backend also -answers 413 before the route to a request declaring a body length over the 1 MiB -limit, which is not recorded. A body over the limit sent without a declared -length reaches the route and is recorded with the status the route answers, -which is 413 only when the report it reads runs past the limit: nothing after -the end of the report is read. In the image neither reaches the backend: nginx -answers both with 413 itself (see [Report limits](#report-limits)), and nothing -of them is recorded. Clients can make up any number of paths and methods, and -each would add labels to the metrics for as long as the server runs. - -`/metrics` is rate limited, so that its password cannot be guessed quickly: each -client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a -minute, whatever their credentials. Past that it gets 429 with -`Retry-After: 60`, and its credentials are not checked. The minute slides as it -does for reports (see [Report limits](#report-limits)), so a scraper polling -every 2 seconds or less often is never refused. This allowance is apart from the -one for reports. - -#### Sentry - -With `SENTRY_DSN` set, each panic in a handler is reported to that Sentry -project under the release `netwatch-server-` followed by the server's version. -On shutdown the server waits up to 2 seconds for Sentry to finish sending. - -### Monitoring targets - -- **26 WAN hosts**: datavi.be (pinned at start), Anthropic API, OpenAI API, AWS - Console, Google Cloud Console, Microsoft Azure, Cloudflare, Fastly CDN, - Akamai, Google, GitHub, B2, 8 S3 regional endpoints (Cape Town, London, - Bahrain, Tokyo, Singapore, Sydney, Oregon, São Paulo) and 6 Hetzner speed test - servers (Nuremberg, Falkenstein, Helsinki, Ashburn, Hillsboro, Singapore) -- **Local CPE**: Cable modem at 192.168.100.1 (always monitored) -- **Local Gateway**: Auto-detected on startup by probing common default gateway - addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder - wins. Note: modern browsers enforce Private Network Access restrictions that - block public-origin pages from reaching RFC1918 addresses, so local targets - only work when NetWatch is served from localhost or a private address. - -Local hosts are tracked separately from WAN stats. - -### Latency measurement - -Each check is a GET request to the target's URL as written, with -`mode: 'no-cors'`, `cache: 'no-store'` and `referrerPolicy: 'no-referrer'`. Its -figure is the time from sending the request until the answer's headers arrive, -timed with `performance.now()`. Unlike ping, this includes the time the server -takes to answer. It also includes the DNS lookup, connection setup and TLS -whenever the browser opens a new connection. Chrome opens one for every check of -the Hetzner speed-test servers, and for more targets at longer intervals. - -The S3 regions are checked with a HEAD request instead. Chrome keeps the -connection to an S3 region open after a HEAD request, but closes it after most -GET requests. The Hetzner speed-test servers do not answer a HEAD request. - -The browser follows every redirect, and the figure then includes the request to -the new URL too. So a target whose home page answers with a redirect is checked -at `/robots.txt` on the same host, which answers without one. An S3 region -answers it with 403, and any answer counts as reachable. - -The Hetzner speed-test servers close the connection without an answer when the -URL has a query string or the request has a `Referer` header. So no query string -is added, and no `Referer` header is sent, wherever the page is served from. -`no-store` keeps the browser's cache out of the measurement. Each check times -out after 80% of the refresh interval (24 seconds at 30 seconds) and is then -recorded as a timeout, so a round's checks have all finished before the next -round is due. When no WAN host answers, a recovery probe checks 4 WAN hosts, -picked at random when it starts, every half second, giving up the checks it -started half a second before. As soon as one answers, a new round starts at -once, as it does after an interval change. A round started early gives up the -last round's checks if they are still waiting, and that round records nothing -more, so rounds never overlap. The browser chooses between IPv4 and IPv6 for -each target, as for any request; the local targets are IPv4 addresses. - -Each recorded check that fails writes one line to the browser console with -`console.error`, and the same line to the debug log: the target's name and URL, -the time, whether it timed out, answered over the time limit or hit a network -error (with the error the browser gives the page, such as -`TypeError: Failed to fetch`, which does not say why), and how long the request -took. A target that answers after failed checks writes one `console.info` line -with its latency and how many checks in a row had failed. - -### Color coding - -| Latency | Color | -| ----------- | ------ | -| < 50ms | Green | -| < 100ms | Lime | -| < 200ms | Yellow | -| < 500ms | Orange | -| >= 500ms | Red | -| Unreachable | Gray | - -### Output structure - -``` -dist/ -├── index.html -└── assets/ - ├── index-*.css - └── index-*.js -``` - -## Features - -- A round of checks every 3 seconds by default; the interval menu sets 1, 2, 3, - 5, 10, 15, 30 or 60 seconds and clears the history -- Sparklines of each target's last 100 rounds: 300 seconds at 3 seconds -- The first round after loading or an interval change is discarded, as DNS and - TLS setup inflate its latencies -- Health indicator from the WAN hosts' latest results: OFFLINE (red) when more - than 10 fail and at most 4 answer, otherwise DEGRADED (orange) when more than - 4 fail, otherwise SLOW (yellow) when more than 3 take over 1000ms, otherwise - HEALTHY (green) -- Summary stats across WAN hosts only: how many answered, the min, median, - average and max of their latest latencies, the min and max over the whole - history, and the number of rounds run (`Checks`) -- Fixed chart axes: Y-axis 0–1000ms, higher latencies drawn at the top; X-axis - the time the history spans -- Color-coded latency figures and sparkline line segments -- WAN host rows sorted by latest latency, unreachable last; pinned rows stay on - top, in name order -- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no - false outage) -- Debug log panel, behind a checkbox in the footer, with five levels (error, - warning, notice, info, debug) and the last 1000 lines -- Local and UTC clocks -- Clickable service URLs -- A footer link to the commit the page was built from -- Canvas-based sparkline rendering with devicePixelRatio scaling -- Zero runtime dependencies: all resources bundled into build artifacts +A failed delivery is quiet, as on a static file host with no backend: one debug +log line when deliveries start failing, one when they recover. The checks go on. ## Deployment -`make build` writes the page to `dist/`, which any static file host (S3, GCS, -Cloudflare Pages, Vercel, Netlify, GitHub Pages) can serve; with no backend -there, its reports fail quietly and nothing is stored. Or run the Docker image -behind a reverse proxy. +`make docker` builds the image `netwatch` from `Dockerfile`: the whole service +in one container. nginx serves the page and passes `/api/`, +`/.well-known/healthcheck` and `/metrics` to `netwatch-server` in the same +container. The backend listens on `127.0.0.1:8081` only, as user `netwatch` (uid +1000). `bin/entrypoint.sh` starts both and watches them. -The Docker image, built from `Dockerfile` by `make docker`, is the whole service -in one container: nginx serves the built frontend and passes `/api/`, -`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`. -`bin/entrypoint.sh` starts it listening only inside the container, on -`127.0.0.1:8081`, with `TRUSTED_PROXIES=127.0.0.1/32`, so it takes the client -address nginx passes on and no other. The image: +Before the backend starts, `bin/entrypoint.sh` runs two subcommands of +`netwatch-server`. `check-cidr` checks each `TRUSTED_PROXIES` entry (see +[Running under upaas](#running-under-upaas)). `prepare-data-dir` creates the +report directory and gives `/data`, with the directories, regular files and +symbolic links in it, to user `netwatch`. It changes nothing outside `/data`. -- Listens on port 8080 by default (override with `PORT` env var) -- Takes the client address from `X-Forwarded-For` only on requests from the - reverse proxies named in `TRUSTED_PROXIES`, by default from any address in - `10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`. `bin/entrypoint.sh` checks - each entry with `netwatch-server check-cidr`, which parses it as the backend - parses its own `TRUSTED_PROXIES` -- Sends access logs to stdout -- Sends the page with `Cache-Control: no-cache`, so a browser that keeps a copy - checks with the server that it is current before using it again; only the - files under `/assets/`, whose names change whenever their contents do, are - cached as immutable -- Sends the security headers `REPO_POLICIES.md` requires on every response, as - `configs/security-headers.conf` sets them, in place of the backend's own -- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data` - volume. Before the backend starts, the image creates `DATA_DIR` and gives it - and `/data` to user `netwatch` (uid 1000), which the backend runs as, with - `netwatch-server prepare-data-dir`, which acts on nothing outside `/data`, so - a host directory bind-mounted at `/data` ends up owned by uid 1000 -- Writes buffered reports to disk on `docker stop`, and exits non-zero if nginx - or the backend exits on its own, so the platform restarts it +The image also: -## Running under upaas +- takes the client address from `X-Forwarded-For` only on a request from a proxy + named in `TRUSTED_PROXIES`, and passes that one address to the backend. The + backend is started with `TRUSTED_PROXIES=127.0.0.1/32`, so it trusts nginx + alone. +- writes nginx's access log and the backend's JSON log to stdout +- refuses a request body over 1 MiB with nginx's own 413 error page +- requests `/.well-known/healthcheck` through nginx every 30 seconds, as its + `HEALTHCHECK`. The check fails unless both nginx and the backend answer, and + while report files cannot be written. +- on `docker stop`, writes the waiting reports to disk and exits 0, or non-zero + if that write fails. If nginx or the backend exits on its own, the other is + stopped and the container exits non-zero. -What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs: +### Running under upaas -- **Port:** container port `8080`. -- **Volume:** container path `/data`; the reports are kept in `/data/reports`. -- **Environment variables:** none is required. An empty one counts as unset, - except `TRUSTED_PROXIES`, and one set to a value netwatch cannot use stops the - container at start, with the reason in its log. - - `PORT`, default `8080`: the container port, from 1 to 65535. `8081` cannot - be used: the backend listens on it inside the container - - `REPORTS_PER_MINUTE`, default `60`: reports each client address may send a - minute - - `DATA_DIR_MAX_BYTES`, default `1073741824` (1 GiB): the most room the - report files may take; the oldest are deleted to stay under it - - `CORS_ALLOWED_ORIGINS`, default empty: other origins whose pages may call - the API - - `DEBUG`, default empty: debug logging; any non-empty value, `false` and - `0` included, turns it on - - `DATA_DIR`, default `/data/reports`: the directory the reports are kept - in: `/data` or a path below it, with no `.` or `..` part and no extra `/`. - The container also stops if the path goes through a symbolic link that - leads out of `/data` or is written as a full path - - `TRUSTED_PROXIES`, default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`: the - addresses the reverse proxy in front of the container connects from, as IP - addresses or CIDRs separated by commas. nginx takes the client address - from `X-Forwarded-For` only on a request from one of them, and the rate - limit counts that address. A list replaces the default. Set and empty, - `X-Forwarded-For` is ignored and every client behind the proxy shares the - proxy's one allowance of `REPORTS_PER_MINUTE`. Any client that connects - from one of these addresses can write its own `X-Forwarded-For`, and - through a port Docker publishes, every client may connect from the Docker - network's gateway, such as `172.17.0.1`, which the default includes; where - that matters, name only the addresses nothing but the proxy connects from. - - `METRICS_USERNAME` and `METRICS_PASSWORD`, default empty: with both set, - the backend records Prometheus metrics of its requests and serves them at - `/metrics` on the container port, to requests with this user name and - password as their basic auth credentials. With neither set, there are no - metrics and `/metrics` is not found. One set without the other, or a user - name containing `:`, stops the container - - `SENTRY_DSN`, default empty: set to a Sentry project's DSN, the backend - sends its errors to that Sentry project: each request whose handling - crashes, which still gets a 500 response. A value Sentry does not accept - stops the container. Empty, the backend sends nothing to Sentry -- **Health check:** the image's `HEALTHCHECK` requests - `/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless - both nginx and the backend answer. It also fails while reports cannot be - stored, as when the volume is full or `DATA_DIR` cannot be written: the health - check answers 503 from a failed write of a report file until a write succeeds, - and the backend tries the write again every minute. Three failed checks in a - row make the container `unhealthy`, and one that passes makes it `healthy` - again. upaas reads the container's health once, 60 seconds after a deploy, and - fails the deploy unless it is `healthy`. A container that turns `unhealthy` - after that is neither restarted nor reported by upaas, whose `unless-stopped` - restart policy ignores health; only `docker ps` and `docker inspect` show it. - The container also stops when either process exits. +The [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs: -## Browser Compatibility +- Port: container port `8080`. +- Volume: container path `/data`. +- Environment variables: none is required. Every setting in + [Settings](#settings) applies, with four differences in the image: + - `BIND_ADDRESS` has no effect. The backend listens on `127.0.0.1` inside + the container. + - `PORT` is nginx's public port, from 1 to 65535 and not `8081`, where the + backend listens inside the container. + - `DATA_DIR` must be `/data` or a path below it, written in full, with no + `.` or `..` part and no extra `/`. A path through a symbolic link is + refused unless the link is relative and stays inside `/data`. + - `TRUSTED_PROXIES` names the proxies nginx trusts, as IP addresses or CIDRs + separated by commas; the backend's own setting is fixed to `127.0.0.1/32`. + `bin/entrypoint.sh` gives a lone address a `/32` or `/128` and checks each + entry with `netwatch-server check-cidr`, which refuses what the backend + would refuse. Set and empty, nginx trusts no proxy, and every client + behind a proxy then shares the proxy's one allowance of + `REPORTS_PER_MINUTE`. A client that connects from a trusted address can + write its own `X-Forwarded-For`. A port Docker publishes can bring + connections from the Docker network's gateway, such as `172.17.0.1`, which + the default includes. Where that matters, name only the proxy's own + addresses. +- Health: the image's `HEALTHCHECK`, described above. -Requires a modern browser with ES modules, Fetch API, Canvas API, and CSS custom -properties. +The container stops at start when a variable holds a value it cannot use. The +reason is in its log. -## Limitations +## Settings -- **CORS**: The checks are cross-origin requests in `no-cors` mode, so the page - cannot read the answer, only time it: any answer counts as reachable, an error - page included. -- **Local targets**: The cable modem at 192.168.100.1 and the detected gateway - answer only on a network that has them, and only when NetWatch is served from - localhost or a private address (see Monitoring targets). -- **Network conditions**: Measurements reflect browser-to-endpoint latency, - which includes your local network, ISP, and internet routing. +`netwatch-server` reads its settings from environment variables. It also reads a +`.env` file in the directory it starts in, and a YAML file named +`netwatch-server.yaml` in `/etc/netwatch-server/` or +`~/.config/netwatch-server/`, when one exists. A variable in the environment +wins over a file. + +The server does not start when a variable holds a value it cannot use. Its error +names the variable. An empty variable counts as unset, except `TRUSTED_PROXIES`. + +- `BIND_ADDRESS`, default empty: the IP address to listen on. Empty listens on + every interface. The image sets it to `127.0.0.1`. +- `PORT`, default `8080`: the port to listen on, from 1 to 65535. In the image + this is nginx's port; the backend listens on `8081`. +- `DATA_DIR`, default `./data/reports` (`/data/reports` in the image): the + directory for the report files. The server creates it if it is missing, and + stops at start if it cannot create a file in it. +- `DATA_DIR_MAX_BYTES`, default `1073741824` (1 GiB): the most bytes of report + files kept in `DATA_DIR`, oldest deleted first. A positive whole number. See + [Report limits](#report-limits). +- `REPORTS_PER_MINUTE`, default `60`: how many reports each client address may + send a minute. A positive whole number. See [Report limits](#report-limits). +- `TRUSTED_PROXIES`, default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`: the + reverse proxies whose `X-Forwarded-For` and `X-Real-IP` headers are trusted, + as CIDRs separated by commas. See [Trusted proxies](#trusted-proxies). +- `CORS_ALLOWED_ORIGINS`, default empty: other origins whose pages may call the + API, separated by commas. See [CORS](#cors). +- `METRICS_USERNAME` and `METRICS_PASSWORD`, default empty: the basic auth + credentials for `/metrics`. See [Metrics](#metrics). +- `SENTRY_DSN`, default empty: the DSN of a Sentry project. Each request whose + handling crashes is reported there; the request still gets a 500. +- `DEBUG`, default empty: any non-empty value turns debug logging on, `false` + and `0` included. Every value is accepted. + +### Trusted proxies + +Each `TRUSTED_PROXIES` entry must be a CIDR, such as `127.0.0.1/32`; a lone +address is refused. A list replaces the default; set and empty, the server +trusts no proxy. Loopback is not in the default, so a reverse proxy on the same +host has to be named. + +A request from a peer outside this set has its forwarded headers ignored; the +peer's own address is logged and rate limited. From a peer inside the set, the +client address is the right-most `X-Forwarded-For` entry that is not in the set. +The peer's address is used when every entry is in the set, or when the search +from the right meets an entry that is not an address, such as `unknown`. +`X-Real-IP` is read only when `X-Forwarded-For` is missing or empty. + +### Routes + +- `POST /api/v1/reports` takes a report from the page. It needs no credentials; + [Report limits](#report-limits) bound it instead. +- `GET /.well-known/healthcheck` answers 200 with a JSON object: `appname`, + `version`, `now`, `uptime_seconds`, `uptime_human` and `status`, which is + `"ok"`. While report files cannot be written, it answers 503 with the same + fields and `"status":"error"`. +- `GET /metrics` serves Prometheus metrics, only when `METRICS_USERNAME` and + `METRICS_PASSWORD` are set. See [Metrics](#metrics). + +Every request body is capped at 1 MiB, and every request at 60 seconds. + +### Report storage + +The server keeps the reports it takes in memory and writes them to `DATA_DIR` as +files of JSON lines, compressed with zstd. It writes a new file every minute, +whenever 10 MiB of reports are waiting, and when it stops. With none waiting, it +writes no file. The files are named `reports--.jsonl.zst`; +the timestamp is UTC to the millisecond, so the names sort by time. + +The reports of a failed write, as on a full disk, stay in memory for the next +write to try again. Until a write succeeds, `POST /api/v1/reports` refuses every +report with 500 and the health check answers 503. + +### Report limits + +`POST /api/v1/reports` takes reports from anyone who can reach it, so it is +bounded in four ways. Each refusal by the backend answers with the body +`{"status":"error"}`. + +- JSON only. A report must be sent with `Content-Type: application/json`; any + other content type is refused with 415, and malformed JSON with 400. +- Body limit. A report body may be at most 1 MiB. A request that declares a + longer body, or a report that runs past that, is refused with 413. +- Rate limit. Each client address may send `REPORTS_PER_MINUTE` reports a + minute; past that it gets 429 with `Retry-After: 60`. The page sends one + report a minute from each open tab, so the default of 60 refuses nothing from + up to 30 tabs behind one address. +- Size cap. The report files in `DATA_DIR` may total at most + `DATA_DIR_MAX_BYTES`, counting the files already there at start and the + reports waiting in memory. When a report would take the total past the cap, + the oldest files are deleted to make room. A report is refused with 507 when + deleting the oldest files cannot make room. + +At the defaults, one address can fill the 1 GiB cap with its own reports in +about 17 minutes, and every report stored before them is deleted to make room. + +### CORS + +The page needs no entry here: it calls the API from the origin it is served +from. By default the server sends no CORS headers, and browsers let no other +origin's pages call it. `CORS_ALLOWED_ORIGINS` lists other origins, such as +`https://netwatch.example.com`, whose pages may then call the API with `GET` and +`POST`. Each entry must be a plain origin as browsers send it: `scheme://host`, +with an optional `:port`, no path, no trailing `/` and no `*`. + +### Metrics + +With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves +Prometheus metrics at `GET /metrics`. A request with those as its basic auth +credentials gets them; any other gets 401. With neither set, `/metrics` +answers 404. The metrics record each request that reaches the health check or +`POST /api/v1/reports`: its duration and response size, labelled with its path, +method and status. Each client address may make 60 requests to `/metrics` a +minute. Past that it gets 429 with `Retry-After: 60`, and its credentials are +not checked. + +### Logging + +The backend logs JSON to stdout, or plain text when stdout is a terminal. Each +request is logged with its method, URL, status, client address and duration. +Every value taken from a request is cut to 128 bytes before it is logged. + +## Design + +The repository holds the page, the backend and the image: + +- `web/`: the page's source. `index.html` loads `src/main.js` and + `src/styles.css`. +- `cmd/netwatch-server/` and `internal/`: the Go backend. `cmd/` holds only + `main`, which calls into `internal/cli`. +- `configs/`: nginx's configuration for the image, `nginx.conf` and + `security-headers.conf`. +- `bin/entrypoint.sh`: the image's entrypoint. +- `Dockerfile`: the lint and test phases, the two build stages and the image. +- `script/`: the development commands; `Makefile` targets call them. +- `test/unit/`: the page's unit tests. `test/viewport/`: the layout test. + +### The page + +The page is built with Vite and Tailwind CSS v4. All its code is in +`web/src/main.js`: + +- `CONFIG`: the timing and display constants. The interval menu sets its + `updateInterval`; the check timeout and the history span follow from it. +- `WAN_HOSTS`: the WAN targets, each with its URL and, for the S3 entries, the + method `HEAD`. +- `HostState` and `AppState`: one host's history and status, and the whole + page's state. +- `measureLatency`: one check. `tick`: one round, which checks every host in + parallel and draws each row as soon as its check ends. +- `SparklineRenderer`: the graphs. `buildUI`, `updateHostRow`, `updateSummary` + and `updateHealthBox`: the page and its parts. +- `Reporter`: builds and posts the reports. `buildReport` is a pure function of + the hosts' state. + +### The backend + +`netwatch-server` is a Go HTTP server. The `fx` library starts and stops its +parts in dependency order. The backend's packages are in `internal/`: + +- `cli`: the command line: the server when given no arguments, or the + `check-cidr` and `prepare-data-dir` subcommands that `bin/entrypoint.sh` runs. +- `config`: reads the settings and refuses values the server cannot use. +- `server`: the router, the routes and the HTTP server. +- `middleware`: the request log, the security headers, CORS, the body limit, the + rate limit, the client address resolution and the metrics. +- `handlers`: the health check and the report route. +- `reportbuf`: the in-memory buffer of reports and the files it writes them to. + It also holds `PrepareDataDir`, which `prepare-data-dir` runs. +- `healthcheck`, `logger` and `globals`: the health status, the logger, and the + name and version the build stamps in. + +## Development + +### Entrypoints + +This repository adheres to the +[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) +standard. The scripts in `script/` are the entrypoints for the development +workflow, and the `Makefile` targets are thin shims that call them. + +- `script/bootstrap`: install the dependencies. It installs the pinned node, + yarn and Go when the ones on `PATH` do not pass its checks, and links them + into `~/.local/bin`, which has to be on `PATH`. It installs no Go linter and + no Docker. +- `script/setup`: `script/bootstrap`, then the git pre-commit hook. +- `script/dev`: run the Vite dev server. +- `script/build`: build the page into `dist/`, then the static `netwatch-server` + binary into the repo root. Its version is `VERSION` from the environment, else + `git describe --tags --always --dirty`, else `dev`. +- `script/run`: `script/build`, then run `netwatch-server` with its defaults. +- `script/clean`: remove what `script/build` writes. +- `script/projectname`: print `netwatch`, the image tag `script/docker` uses. +- `script/yarn`: run yarn under the node `script/bootstrap` would use. +- `script/test`: build the `test` phase of `Dockerfile`, writing no image: the + Go tests with the race detector and coverage, and the page's unit tests. +- `script/lint`: build the `lint` phase of `Dockerfile`, writing no image: + golangci-lint over the Go packages and eslint over the JavaScript. +- `script/fmt`: run the formatters: prettier over the JavaScript, CSS, HTML and + Markdown, then gofmt over `cmd/` and `internal/`. They write the files in + place. +- `script/fmt-check`: check the formatting (read-only), with the same scope as + `script/fmt`. +- `script/check`: `script/test`, `script/lint` and `script/fmt-check`. +- `script/add-dependency`: add a page dependency, or move one to another + version: `make add-dependency PACKAGE=@`. +- `script/tidy`: run `go mod tidy`. +- `script/frontend-test`: run the unit tests in `test/unit/` on the host, with + Node's built-in test runner, then the production build. +- `script/frontend-fmt` and `script/frontend-fmt-check`: the prettier half of + `script/fmt` and `script/fmt-check`. +- `script/frontend-viewport-test`: check the built page's layout in a + containerised headless Chrome, not as part of `script/check`; see + [test/viewport/README.md](test/viewport/README.md). +- `script/docker`: build the image from `Dockerfile`, tagged `netwatch`. +- `script/cibuild`: `script/bootstrap`, `script/check`, then the image build. + The Gitea workflow `.gitea/workflows/check.yml` runs it on every push. +- `script/precommit`: run by the git pre-commit hook; it runs `script/check`. +- `script/install-precommit`: install the git pre-commit hook. `make hooks` + calls it. + +### Tests + +`make test` runs the backend's Go tests and the page's unit tests in Docker. The +unit tests import `web/src/main.js` in Node and call the functions it exports; +no browser is involved. `make frontend-viewport-test` checks the built page's +layout in a headless Chrome; [test/viewport/README.md](test/viewport/README.md) +says what it checks and what it cannot. ## TODO diff --git a/TODO.md b/TODO.md index 5ab3fb2..8a964eb 100644 --- a/TODO.md +++ b/TODO.md @@ -23,6 +23,10 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of # Completed Steps +- 2026-10-26: `README.md`, `TODO.md` and `test/viewport/README.md` are rewritten + sentence by sentence ([#189](https://git.eeqj.de/sneak/netwatch/issues/189)). + Each sentence reads on the first pass: one idea, under about 30 words, the + verb early. Every statement matches the code on `next`. - 2026-10-26: a target whose home page answers with a redirect is checked at `/robots.txt` on the same host, which answers without one ([#194](https://git.eeqj.de/sneak/netwatch/issues/194)). These are the AWS