1 Commits
Author SHA1 Message Date
sneak 927ee136a9 Serve Prometheus metrics at /metrics behind basic auth (closes #94)
check / check (push) Successful in 3m17s
With METRICS_USERNAME and METRICS_PASSWORD both set, the backend
records request metrics through go-http-metrics in a registry of its
own, with Go's runtime and process metrics, and serves them at
GET /metrics behind basic auth with those credentials; nginx passes
/metrics to it. With neither set there is no such route; one alone
stops the start with an error naming both.

Only requests that reach the health check or POST /api/v1/reports are
recorded, not every request as the conventions show: the labels are
path and method, which clients can make up without end. So
POST /api/v1/reports is registered by its full path, not inside a
route group.

Deviation: go get and go mod tidy ran directly; no entrypoint adds a Go
dependency yet (#45).

Model: opus-5-5
2026-10-04 02:08:45 +00:00
26 changed files with 239 additions and 789 deletions
+1 -2
View File
@@ -1,7 +1,6 @@
backend/
dist/ dist/
node_modules/ node_modules/
tmp/ tmp/
yarn.lock yarn.lock
.claude/ .claude/
# The org standard file, copied verbatim; backend/script/lint checks its sha256.
backend/.golangci.yml
+3 -15
View File
@@ -1,5 +1,5 @@
.PHONY: bootstrap setup dev build test lint fmt fmt-check check \ .PHONY: bootstrap setup dev test lint fmt fmt-check check frontend-check \
add-dependency tidy frontend-check frontend-viewport-test docker hooks frontend-viewport-test docker hooks
# Standard targets are thin shims; the implementations live in script/ # Standard targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section # per the scripts-to-rule-them-all pattern (see the Entrypoints section
@@ -13,11 +13,7 @@ setup:
@script/setup @script/setup
dev: dev:
@script/dev yarn dev
# The frontend only; backend/Makefile's build target builds the Go server.
build:
@script/build
test: test:
@script/test @script/test
@@ -34,14 +30,6 @@ fmt-check:
check: check:
@script/check @script/check
# make add-dependency PACKAGE=<name>@<version>. PACKAGE reaches the
# script through the environment, so the shell never reads it as code.
add-dependency:
@script/add-dependency "$$PACKAGE"
tidy:
@script/tidy
# The frontend tests and format check, for Dockerfile's frontend stage, # The frontend tests and format check, for Dockerfile's frontend stage,
# which has neither Go nor Docker. Use check everywhere else. # which has neither Go nor Docker. Use check everywhere else.
frontend-check: frontend-check:
+60 -126
View File
@@ -1,33 +1,30 @@
NetWatch is an MIT-licensed JavaScript single-page application by NetWatch is an MIT-licensed JavaScript single-page application by
[@sneak](https://sneak.berlin) that provides real-time network latency [@sneak](https://sneak.berlin) that provides real-time network latency
monitoring to common internet hosts, displayed with color-coded figures and monitoring to common internet hosts, displayed with color-coded figures and
sparkline graphs, served from a static bucket or from its Docker image, where a sparkline graphs, served from a static bucket or Docker container.
small Go backend stores the measurements the page reports.
## Getting Started ## Getting Started
```bash ```bash
# Install the dependencies and the git pre-commit hook # Install dependencies
make setup yarn install
# Run the page on the Vite dev server # Development server
make dev yarn dev
# Run the tests, both linters and the format check # Production build
make check yarn build
# Build the page into dist/ # Preview production build
make build yarn preview
# Build the image and run it # Docker
make docker docker build -t netwatch .
docker run -p 8080:8080 netwatch docker run -p 8080:8080 netwatch
``` ```
`make check` and `make docker` need Docker. `make dev` passes `/api` to `yarn dev` proxies `/api` to `http://127.0.0.1:8080`, so a locally running
`http://127.0.0.1:8080`, where `make run` in `backend/` starts `netwatch-server` `netwatch-server` (see `backend/`) receives the reports the page posts.
with its defaults, so the reports the page posts are stored in
`backend/data/reports`.
## Entrypoints ## Entrypoints
@@ -49,10 +46,6 @@ halves, so the root `make check` fails if either one is broken. We provide:
both linters in Docker both linters in Docker
- `script/setup` — make a fresh clone ready for development: bootstrap plus the - `script/setup` — make a fresh clone ready for development: bootstrap plus the
git pre-commit hook git pre-commit hook
- `script/dev` — run the Vite dev server, which proxies `/api` to a locally
running `netwatch-server`
- `script/build` — build the frontend for production into `dist/`;
`backend/script/build` builds the Go server
- `script/projectname` — print the project name (used for the Docker image tag) - `script/projectname` — print the project name (used for the Docker image tag)
- `script/test` — run `script/frontend-test`, then `backend/script/test`, the - `script/test` — run `script/frontend-test`, then `backend/script/test`, the
backend's Go tests with the race detector and coverage backend's Go tests with the race detector and coverage
@@ -61,13 +54,6 @@ halves, so the root `make check` fails if either one is broken. We provide:
- `script/fmt` — format all files (writes): prettier, then gofmt over `backend/` - `script/fmt` — format all files (writes): prettier, then gofmt over `backend/`
- `script/fmt-check` — check formatting (read-only): prettier, then gofmt - `script/fmt-check` — check formatting (read-only): prettier, then gofmt
- `script/check` — run test, lint, and fmt-check - `script/check` — run test, lint, and fmt-check
- `script/add-dependency` — add a frontend package, or move one to another
version: `make add-dependency PACKAGE=<name>@<version>` runs `yarn add --dev`,
which changes `package.json` and `yarn.lock` together, then
`yarn install --frozen-lockfile`
- `script/tidy` — run `go mod tidy` in `backend/`: to add a Go module, import it
and run `make tidy`; to move one to another version, edit its `require` line
in `backend/go.mod`, then run `make tidy`
- `script/frontend-test` — run the unit tests in `test/unit/` with Node's - `script/frontend-test` — run the unit tests in `test/unit/` with Node's
built-in test runner, through the `test` script in `package.json`, and if any built-in test runner, through the `test` script in `package.json`, and if any
fails, run them again listing every test, and fail; then the production build. fails, run them again listing every test, and fail; then the production build.
@@ -75,8 +61,7 @@ halves, so the root `make check` fails if either one is broken. We provide:
- `script/frontend-lint` — run eslint with the rules in `eslint.config.js`; it - `script/frontend-lint` — run eslint with the rules in `eslint.config.js`; it
runs inside the `frontend-lint` stage of `Dockerfile`, which `make lint` runs inside the `frontend-lint` stage of `Dockerfile`, which `make lint`
builds builds
- `script/frontend-fmt` — format everything prettier understands (writes), the - `script/frontend-fmt` — format everything prettier understands (writes)
markdown in `backend/` included
- `script/frontend-fmt-check` — check prettier formatting (read-only) - `script/frontend-fmt-check` — check prettier formatting (read-only)
- `script/frontend-check` — run `script/frontend-test` and - `script/frontend-check` — run `script/frontend-test` and
`script/frontend-fmt-check`, for the frontend stage of `Dockerfile`, which has `script/frontend-fmt-check`, for the frontend stage of `Dockerfile`, which has
@@ -97,30 +82,25 @@ halves, so the root `make check` fails if either one is broken. We provide:
The narrow-viewport layout lives in the `max-width: 768px` media block in The narrow-viewport layout lives in the `max-width: 768px` media block in
`src/styles.css`. It is verified automatically by `make frontend-viewport-test`, `src/styles.css`. It is verified automatically by `make frontend-viewport-test`,
which drives a digest-pinned headless Chrome against the built `dist/` and which drives a digest-pinned headless Chrome against the built `dist/` and
asserts on computed layout at widths derived from that CSS — on every breakpoint asserts on computed layout at widths derived from that CSS — one pixel either
it declares and one pixel either side of it, plus a 320px floor, a desktop side of every breakpoint it declares, plus a 320px floor, a desktop baseline and
baseline and two landscape sizes. See two landscape sizes. See [test/viewport/README.md](test/viewport/README.md) for
[test/viewport/README.md](test/viewport/README.md) for what it covers and what what it covers and what it genuinely cannot.
it genuinely cannot.
## Rationale ## Rationale
When debugging network issues, it's useful to have a persistent at-a-glance view When debugging network issues, it's useful to have a persistent at-a-glance view
of latency and reachability to multiple well-known internet endpoints. NetWatch of latency and reachability to multiple well-known internet endpoints. NetWatch
provides this as a single page that does all its measuring in the browser, so it provides this as a zero-dependency SPA that can be deployed anywhere static
can be served from anywhere static files are served. The backend in its Docker files are served, with no backend required.
image only stores the measurements the page reports; without it, the page works
the same and nothing is stored.
## Design ## Design
The page is built with Vite and Tailwind CSS v4. Its code is all in The application is a single-page app built with Vite and Tailwind CSS v4. All
`src/main.js`, with a class-based architecture: code lives in `src/main.js` with a class-based architecture:
- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks, - **`CONFIG`**: Frozen configuration object (update interval, timeouts, axis
etc.). The interval menu sets `updateInterval`, the one value the page writes ticks, etc.)
into `CONFIG`; the timeouts, the time the history spans and the x-axis ticks
are computed from it
- **`HostState`**: Per-host state management — history buffer, latency tracking, - **`HostState`**: Per-host state management — history buffer, latency tracking,
status transitions status transitions
- **`AppState`**: Top-level state container — WAN hosts, local hosts, pause - **`AppState`**: Top-level state container — WAN hosts, local hosts, pause
@@ -131,10 +111,10 @@ The page is built with Vite and Tailwind CSS v4. Its code is all in
`updateSummary()` / `updateHealthBox()` handle incremental updates `updateSummary()` / `updateHealthBox()` handle incremental updates
- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's - **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's
sample and redrawing its row as soon as its check ends, then redraws every sample and redrawing its row as soon as its check ends, then redraws every
row, the summary and the health box once the last check ends. The first round, row, the summary and the health box once the last check ends. The rows are
after loading or an interval change, is discarded. The rows are sorted when sorted then too, after the first round that is not discarded and every tenth
the last check ends in round 2, the first one kept, and in rounds 11, 21, 31 round after that. When paused, pushes blank markers (no probes, no false
and so on. When paused, pushes blank markers (no probes, no false outage) outage)
- **`Reporter`**: Posts collected samples to the backend - **`Reporter`**: Posts collected samples to the backend
### Reporting ### Reporting
@@ -148,35 +128,12 @@ delivered report, and while paused nothing is sent. Delivery failure is quiet
one debug-log line per outage, retried at the next interval, never blocking one debug-log line per outage, retried at the next interval, never blocking
probing. The report-building step is a pure function of host state. probing. The report-building step is a pure function of host state.
### Backend
`netwatch-server`, in `backend/`, is a small Go HTTP server that stores the
reports the page posts. It keeps them in memory and writes them to `DATA_DIR` as
zstd-compressed files of JSON lines: every minute, whenever 10 MiB are waiting,
and when it stops. Its routes:
- `POST /api/v1/reports` — takes a report, without credentials; each client
address may send a limited number a minute, and the report files are capped in
size, the oldest deleted first
- `GET /.well-known/healthcheck` — answers 200 with `"status":"ok"`, the
server's version and its uptime
- `GET /metrics` — Prometheus metrics behind basic auth, only when
`METRICS_USERNAME` and `METRICS_PASSWORD` are set; each client address may
make a limited number of requests to it a minute
In the image, the `builder` stage of `Dockerfile` tests it and builds it with
`backend/script/build`, and `bin/entrypoint.sh` runs it as user `netwatch` on
`127.0.0.1:8081`, behind nginx. Outside the image, `make run` in `backend/`
builds it and runs it on port 8080. Its settings, report storage and limits are
in [backend/README.md](backend/README.md).
### Monitoring targets ### Monitoring targets
- **26 WAN hosts**: datavi.be (pinned at start), Anthropic API, OpenAI API, AWS - **22 WAN hosts**: datavi.be, Anthropic API, OpenAI API, AWS Console, GCP
Console, Google Cloud Console, Microsoft Azure, Cloudflare, Fastly CDN, Console, Azure, Cloudflare, Fastly, Akamai, GitHub, B2, 7 S3 regional
Akamai, Google, GitHub, B2, 8 S3 regional endpoints (Cape Town, London, endpoints (Cape Town, London, Bahrain, Tokyo, Sydney, Oregon, São Paulo), 4
Bahrain, Tokyo, Singapore, Sydney, Oregon, São Paulo) and 6 Hetzner speed test GCS locational endpoints (Iowa, Belgium, Singapore, Sydney)
servers (Nuremberg, Falkenstein, Helsinki, Ashburn, Hillsboro, Singapore)
- **Local CPE**: Cable modem at 192.168.100.1 (always monitored) - **Local CPE**: Cable modem at 192.168.100.1 (always monitored)
- **Local Gateway**: Auto-detected on startup by probing common default gateway - **Local Gateway**: Auto-detected on startup by probing common default gateway
addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder
@@ -188,17 +145,15 @@ Local hosts are tracked separately from WAN stats.
### Latency measurement ### Latency measurement
GET requests with `mode: 'no-cors'`, `cache: 'no-store'` and a cache-busting HEAD requests with `mode: 'no-cors'` and `cache: 'no-store'`, timed with
query parameter, timed with `performance.now()`. Each check times out after 80% `performance.now()`. Each check times out after 80% of the refresh interval (24
of the refresh interval (24 seconds at 30 seconds) and is then recorded as a seconds at 30 seconds) and is then recorded as a timeout, so a round's checks
timeout, so a round's checks have all finished before the next round is due. have all finished before the next round is due. When no WAN host answers, a
When no WAN host answers, a recovery probe checks 4 WAN hosts, picked at random recovery probe checks 4 random WAN hosts every half second, giving up the checks
when it starts, every half second, giving up the checks it started half a second it started half a second before. As soon as one answers, a new round starts at
before. As soon as one answers, a new round starts at once, as it does after an once, as it does after an interval change. A round started early gives up the
interval change. A round started early gives up the last round's checks if they last round's checks if they are still waiting, and that round records nothing
are still waiting, and that round records nothing more, so rounds never overlap. more, so rounds never overlap. IPv4 only.
The browser chooses between IPv4 and IPv6 for each target, as for any request;
the local targets are IPv4 addresses.
### Color coding ### Color coding
@@ -223,42 +178,25 @@ dist/
## Features ## Features
- A round of checks every 3 seconds by default; the interval menu sets 1, 2, 3, - Real-time monitoring with 2s update interval and 300s history sparklines
5, 10, 15, 30 or 60 seconds and clears the history - Health indicator: green (HEALTHY) or red (DEGRADED) based on WAN reachability
- Sparklines of each target's last 100 rounds: 300 seconds at 3 seconds - Summary stats: reachable count, min/max/avg latency across WAN hosts only
- The first round after loading or an interval change is discarded, as DNS and - Fixed chart axes: Y-axis 0–1000ms, X-axis 0–300s
TLS setup inflate its latencies
- Health indicator from the WAN hosts' latest results: OFFLINE (red) when more
than 10 fail and at most 4 answer, otherwise DEGRADED (orange) when more than
4 fail, otherwise SLOW (yellow) when more than 3 take over 1000ms, otherwise
HEALTHY (green)
- Summary stats across WAN hosts only: how many answered, the min, median,
average and max of their latest latencies, the min and max over the whole
history, and the number of rounds run (`Checks`)
- Fixed chart axes: Y-axis 0–1000ms, higher latencies drawn at the top; X-axis
the time the history spans
- Color-coded latency figures and sparkline line segments - Color-coded latency figures and sparkline line segments
- WAN host rows sorted by latest latency, unreachable last; pinned rows stay on
top, in name order
- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no - Play/pause: pause stops probes but history keeps scrolling (blank gaps, no
false outage) false outage)
- Debug log panel, behind a checkbox in the footer, with five levels (error,
warning, notice, info, debug) and the last 1000 lines
- Local and UTC clocks
- Clickable service URLs - Clickable service URLs
- A footer link to the commit the page was built from
- Canvas-based sparkline rendering with devicePixelRatio scaling - Canvas-based sparkline rendering with devicePixelRatio scaling
- Zero runtime dependencies: all resources bundled into build artifacts - Zero runtime dependencies: all resources bundled into build artifacts
## Deployment ## Deployment
`make build` writes the page to `dist/`, which any static file host (S3, GCS, After running `yarn build`, deploy the contents of the `dist/` directory to any
Cloudflare Pages, Vercel, Netlify, GitHub Pages) can serve; with no backend static file host (S3, GCS, Cloudflare Pages, Vercel, Netlify, GitHub Pages) or
there, its reports fail quietly and nothing is stored. Or run the Docker image use the Docker image behind a reverse proxy.
behind a reverse proxy.
The Docker image, built from `Dockerfile` by `make docker`, is the whole service The Docker image, built from `Dockerfile`, is the whole service in one
in one container: nginx serves the built frontend and passes `/api/`, container: nginx serves the built frontend and passes `/api/`,
`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`, `/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`,
which listens only inside the container, on `127.0.0.1:8081`. The image: which listens only inside the container, on `127.0.0.1:8081`. The image:
@@ -312,12 +250,8 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
the backend records Prometheus metrics of its requests and serves them at the backend records Prometheus metrics of its requests and serves them at
`/metrics` on the container port, to requests with this user name and `/metrics` on the container port, to requests with this user name and
password as their basic auth credentials. With neither set, there are no password as their basic auth credentials. With neither set, there are no
metrics and `/metrics` is not found. One set without the other, or a user metrics and `/metrics` is not found. One set without the other stops the
name containing `:`, stops the container container
- `SENTRY_DSN`, default empty: set to a Sentry project's DSN, the backend
sends its errors to that Sentry project: each request whose handling
crashes, which still gets a 500 response. A value Sentry does not accept
stops the container. Empty, the backend sends nothing to Sentry
- **Health check:** the image's `HEALTHCHECK` requests - **Health check:** the image's `HEALTHCHECK` requests
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless `/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
both nginx and the backend answer. upaas reads the container's health 60 both nginx and the backend answer. upaas reads the container's health 60
@@ -331,19 +265,19 @@ properties.
## Limitations ## Limitations
- **CORS**: The checks are cross-origin requests in `no-cors` mode, so the page - **CORS**: Some hosts may block cross-origin HEAD requests. The app uses
cannot read the answer, only time it: any answer counts as reachable, an error `no-cors` mode which allows the request but provides opaque responses. Latency
page included. is still measurable based on request timing.
- **Local targets**: The cable modem at 192.168.100.1 and the detected gateway - **Local gateway**: The 192.168.100.1 endpoint requires the host to be
answer only on a network that has them, and only when NetWatch is served from accessible from your network.
localhost or a private address (see Monitoring targets).
- **Network conditions**: Measurements reflect browser-to-endpoint latency, - **Network conditions**: Measurements reflect browser-to-endpoint latency,
which includes your local network, ISP, and internet routing. which includes your local network, ISP, and internet routing.
## TODO ## TODO
The to-do list is [TODO.md](TODO.md): where the work stands, the next step, the - Add configurable host list (environment variable or config file)
open work, and what has been done. - Add latency history export (CSV/JSON)
- Add notification/alert when status changes to DEGRADED
## License ## License
+20 -90
View File
@@ -1,108 +1,40 @@
# Workflow # Workflow
- branch from `next` - branch (from `main`)
- do the work in Next Step - do the work in Next Step
- move Next Step to the top of Completed Steps - move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step - move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work) - commit (`TODO.md` changes in the same commit as the work)
- push the branch and open a PR against `next` - merge to `main` if the branch is not protected, otherwise open a PR
- push
# Status # Status
pre-1.0. No git tags. `main` is the stable branch and `next` the development pre-1.0. No git tags. `feat/reportbuf-storage` is merged; the backend, the CI
branch, which every PR targets. The frontend and the Go backend ship as one workflow, and the backend repo standard files are all on `main`. Frontend and
Docker image, and the Gitea workflow `.gitea/workflows/check.yml` runs backend are both functional. Working toward the 1.0.0 milestone by closing the
`script/cibuild` on every push. Working toward 1.0.0. remaining repo-compliance issues on the tracker.
# Next Step # Next Step
Decide whether the repo moves to the layout `REPO_POLICIES.md` gives, with Confirm the `.gitea/workflows/check.yml` run is green (main always green
`backend/` no longer repeating files from the root policy). The workflow file is already on `main`; what is unverified is that its
([#30](https://git.eeqj.de/sneak/netwatch/issues/30)). latest run passes.
# Completed Steps # Completed Steps
- 2026-10-04: the page's footer no longer says "IPv4 only"
([#111](https://git.eeqj.de/sneak/netwatch/issues/111)): each check is a
`fetch`, the browser picks IPv4 or IPv6 for each WAN host, and the local
targets are IPv4 addresses. The rest of the footer is unchanged
- 2026-10-04: `README.md`, `TODO.md` and `test/viewport/README.md` say what the
tree does (issue #24). The README's Getting Started leads with `make` targets;
a new Backend section says what `netwatch-server` stores, its routes and how
the image builds and runs it, and points to `backend/README.md` for its
settings; the checks are GET requests; the 26 WAN hosts, the four health
states, the summary's figures and the features the list lacked are described
as the page has them; and its TODO section points here, as does the one in
`backend/README.md`, whose open items moved to Future Steps. This file's
Workflow branches from `next` and opens the PR against `next`, Status says
where the repo stands, and Next Step and Future Steps hold only open work,
linked to its issue where one exists. The viewport harness README names Node's
test runner, not `vitest`
- 2026-10-04: in `src/main.js` (issue #102), a target's min, max, median and
average latency come from one list of its answers, through the same function
the summary's figures use, so the median is written once. The latency color
limits are one table in `CONFIG`, read by both the figure's and the
sparkline's color. The health thresholds, the debug log's length, the gateway
check's timeout, the recovery probe's number of hosts and interval, how often
the rows are sorted and the delay before the first sparkline resize are
`CONFIG` entries too. A unit test checks the summary's figures. Nothing the
page does or shows changed; the footer's color legend still writes the limits
out as text
- 2026-10-04: password guesses at `/metrics` are rate limited (issue #104): each
client address, resolved through `TRUSTED_PROXIES` as for reports, may make 60
requests to `/metrics` a minute, counted by `go-chi/httprate` apart from its
reports; past that it gets 429 and its basic auth credentials are not checked.
The limit is a constant in `backend/internal/server/routes.go`. A test uses up
one client's allowance on wrong passwords, gets 429 with the right one, and
checks that another client behind the same nginx still gets in
- 2026-10-04: the backend reports errors to Sentry (issue #95). With
`SENTRY_DSN` set, it sets up `sentry-go` with the release `netwatch-server-`
and its version, reports each panic in a handler through `sentryhttp`, the
last of the middleware every request goes through, which panics again so the
request still gets the 500 from the panic recovery, and waits up to 2 seconds
on shutdown for Sentry to finish sending. A DSN Sentry refuses stops the start
with an error naming `SENTRY_DSN`. With it empty, Sentry is not set up and
nothing is sent to it
- 2026-10-04: a target's name and URL and a debug log message show as the
characters they are and are never read as HTML (issue #29): a host row escapes
the name and URL it writes into its markup, and the debug log sets each line
as text. A unit test checks a target whose name and URL hold `<`, `>`, `"`,
`&` and `'`. `README.md` no longer calls `CONFIG` frozen: the interval menu
sets its `updateInterval`, and the values computed from it follow. `AppState`
declares the recovery probe's two properties, the sparkline axis functions
lose the parameters they did not use, and the comment on a target's history
names both kinds of entry it holds. Nothing the page does changed
- 2026-10-04: a dependency can be added without running yarn or go by hand
(issue #45): `make add-dependency PACKAGE=<name>@<version>` shims to the new
`script/add-dependency`, which runs `yarn add --dev`, so `package.json` and
`yarn.lock` change together, then `yarn install --frozen-lockfile`; the same
command moves a package to another version. `make tidy` shims to the new
`script/tidy`, which runs `go mod tidy` in `backend/`: a Go module is added by
importing it, or moved by editing its `require` line, then `make tidy`.
`script/bootstrap` still installs with `--frozen-lockfile`
- 2026-10-04: the backend serves Prometheus metrics (issue #94). With - 2026-10-04: the backend serves Prometheus metrics (issue #94). With
`METRICS_USERNAME` and `METRICS_PASSWORD` both set, it records request `METRICS_USERNAME` and `METRICS_PASSWORD` both set, it records request
duration and response size through `go-http-metrics` and serves them, with duration and response size through `go-http-metrics` and serves them, with
Go's runtime and process metrics, at `GET /metrics` behind basic auth with Go's runtime and process metrics, at `GET /metrics` behind basic auth with
those credentials; nginx passes `/metrics` to it as it does `/api/`. With those credentials; nginx passes `/metrics` to it as it does `/api/`. With
neither set there are no metrics and `/metrics` is 404; one without the other neither set there are no metrics and `/metrics` is 404; one without the other
stops the start with an error naming both, and so does a `METRICS_USERNAME` stops the start with an error naming both. Only requests that reach the health
containing `:`, with an error naming it. Only requests that reach the health
check or `POST /api/v1/reports` are recorded, not `/metrics` itself and not check or `POST /api/v1/reports` are recorded, not `/metrics` itself and not
every request as `GO_HTTP_SERVER_CONVENTIONS.md` shows, because the labels are every request as `GO_HTTP_SERVER_CONVENTIONS.md` shows, because the labels are
the request's path and method, which clients can make up without end. For the request's path and method, which clients can make up without end. For
that, `POST /api/v1/reports` is now registered by its full path instead of that, `POST /api/v1/reports` is now registered by its full path instead of
inside a `/api/v1` route group; it answers as before inside a `/api/v1` route group; it answers as before
- 2026-10-04: `script/` and `Makefile` follow the org models (issue #28):
`make dev` shims to the new `script/dev`, the Vite dev server, and the new
`make build` to `script/build`, the frontend production build.
`.prettierignore` no longer leaves out `backend/`, so `make fmt` and
`make fmt-check` cover `backend/README.md`; it leaves out
`backend/.golangci.yml` by name, the org standard file whose sha256
`backend/script/lint` checks. `script/install-precommit` and the date on
`script/bootstrap`'s pins are the org model again; `script/bootstrap`,
`script/fmt` and `script/fmt-check` each say in a comment why they differ from
it
- 2026-10-04: a frontend build on Node 26 or newer, such as `make test` on a - 2026-10-04: a frontend build on Node 26 or newer, such as `make test` on a
host with Node 26, no longer prints Node's warning that `module.register()` is host with Node 26, no longer prints Node's warning that `module.register()` is
deprecated (issue #32); the build in `Dockerfile` runs on Node 22, which never deprecated (issue #32); the build in `Dockerfile` runs on Node 22, which never
@@ -389,14 +321,12 @@ Decide whether the repo moves to the layout `REPO_POLICIES.md` gives, with
# Future Steps # Future Steps
- Run `make frontend-viewport-test` in CI as its own step; it is not part of - Wire `script/frontend-viewport-test` into CI as its own step (deliberately not
`make check`, as it needs Docker and takes minutes part of `make check` today; the decision has real CI-runtime cost and is
- A backend test that posts a report to `POST /api/v1/reports` and checks the tracked separately)
compressed file it is written to - Compliance top-up as one small commit: add .editorconfig and add the hooks
- A backend route that decompresses the stored reports and answers queries on target to the Makefile
them - After merge, confirm .gitea/workflows/check.yml is on main and CI is green
- Prometheus metrics for the backend's in-memory buffer: its size, the number of (main always green policy)
flushes and the number of reports - Decide what to do with untracked resume.sh: commit it, gitignore it, or delete
- A configurable host list (an environment variable or a config file) it
- Export of the latency history (CSV or JSON)
- A notification when the health status changes to DEGRADED
+23 -44
View File
@@ -32,16 +32,17 @@ pattern as the repo root: the targets in `backend/Makefile` are thin shims over
`test`, `fmt` and `fmt-check`: `test`, `fmt` and `fmt-check`:
- `script/build` — compile the static `netwatch-server` binary with its version - `script/build` — compile the static `netwatch-server` binary with its version
stamped in. The version is `VERSION` from the environment; when that is unset stamped in. The version is `VERSION` from the environment;
or empty, it falls back to `git describe` inside a git checkout, then to `dev` when that is unset or empty, it falls back to `git describe` inside a git
checkout, then to `dev`
- `script/test` — run the Go tests with the race detector and coverage. Go's - `script/test` — run the Go tests with the race detector and coverage. Go's
`-timeout 30s` bounds the tests, not their compile. If they fail, they run `-timeout 30s` bounds the tests, not their compile. If they fail, they run
again with `-v` for the details, and the script fails. The race detector needs again with `-v` for the details, and the script fails. The race detector needs
a C compiler a C compiler
- `script/lint` — check `.golangci.yml` against its pinned sha256, then run - `script/lint` — check `.golangci.yml` against its pinned sha256, then run
golangci-lint. It runs inside the golangci-lint image of the lint stage of the golangci-lint. It runs inside the golangci-lint image of the lint stage of
root `Dockerfile`; from a checkout, run `make lint` at the repo root, which the root `Dockerfile`; from a checkout, run `make lint` at the repo root,
builds that stage which builds that stage
- `script/fmt` — format the Go sources (writes) - `script/fmt` — format the Go sources (writes)
- `script/fmt-check` — check Go formatting (read-only) - `script/fmt-check` — check Go formatting (read-only)
- `script/run` — build and run the server locally - `script/run` — build and run the server locally
@@ -60,9 +61,8 @@ flushes them to compressed files on disk for later analysis.
## Design ## Design
The server is structured as an `fx`-wired Go application under The server is structured as an `fx`-wired Go application under `cmd/netwatch-server/`.
`cmd/netwatch-server/`. Internal packages in `internal/` follow standard Go Internal packages in `internal/` follow standard Go project layout:
project layout:
- **`config`**: Loads configuration from environment variables and config files - **`config`**: Loads configuration from environment variables and config files
via Viper. via Viper.
@@ -90,14 +90,12 @@ project layout:
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) | | `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) | | `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) | | `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
| `SENTRY_DSN` | empty | DSN of the Sentry project to send errors to; see [Sentry](#sentry) |
`TRUSTED_PROXIES` defaults to `TRUSTED_PROXIES` defaults to `127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`.
`127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. The loopback The loopback entries cover a reverse proxy on the same host. A request whose
entries cover a reverse proxy on the same host. A request whose direct peer is direct peer is outside this set has its forwarded headers ignored, and the
outside this set has its forwarded headers ignored, and the direct peer is direct peer is logged and rate-limited instead. The container image does not use
logged and rate-limited instead. The container image does not use this default; this default; see [Container image](#container-image).
see [Container image](#container-image).
A variable set to a value the server cannot use, such as `PORT=abc`, A variable set to a value the server cannot use, such as `PORT=abc`,
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from `DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
@@ -166,7 +164,8 @@ credentials, so it is bounded instead. Both refusals below answer with the same
to be written fill the cap on their own, and then no file is deleted. At to be written fill the cap on their own, and then no file is deleted. At
start, report files past the cap, as after lowering it, are deleted the same start, report files past the cap, as after lowering it, are deleted the same
way. So the cap is how much of the newest reports is kept: the default of 1 way. So the cap is how much of the newest reports is kept: the default of 1
GiB is small enough for any host; set it to the space you can give `DATA_DIR`. GiB is small enough for any host; set it to the space you can give
`DATA_DIR`.
### CORS ### CORS
@@ -190,37 +189,17 @@ method and status; they also count those requests in progress, and include Go's
runtime and process metrics. No other request is recorded: not those to runtime and process metrics. No other request is recorded: not those to
`/metrics` itself, and not those answered before they reach either route, such `/metrics` itself, and not those answered before they reach either route, such
as a CORS preflight, or a request refused with 404 for a path no route has, 405 as a CORS preflight, or a request refused with 404 for a path no route has, 405
for a method its route does not take, or 413 for declaring a body length over for a method its route does not take, or 413 for a body over the 1 MiB limit.
the 1 MiB limit. A report whose body goes over the limit without declaring its Clients can make up any number of paths and methods, and each would add labels
length reaches the route, is answered 413 there, and is recorded with that to the metrics for as long as the server runs. With neither set, nothing is
status. Clients can make up any number of paths and methods, and each would add recorded and `/metrics` answers 404. One without the other stops the server
labels to the metrics for as long as the server runs. With neither set, nothing from starting, with an error naming both.
is recorded and `/metrics` answers 404. One without the other stops the server
from starting, with an error naming both; so does a `METRICS_USERNAME`
containing `:`, which basic auth cannot carry, with an error naming it.
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
minute, whatever their credentials. Past that it gets 429 with
`Retry-After: 60`, and its credentials are not checked. The minute slides as it
does for reports (see [Report limits](#report-limits)), so a scraper polling
every 2 seconds or less often is never refused. This allowance is apart from the
one for reports.
### Sentry
With `SENTRY_DSN` set, the server sends its errors to that Sentry project: each
panic in a handler is reported there, under the release `netwatch-server-`
followed by the server's version, and the request still gets 500 from the
server's panic recovery. On shutdown the server waits up to 2 seconds for Sentry
to finish sending. A DSN Sentry refuses stops the server from starting, with an
error naming `SENTRY_DSN`. With it empty, Sentry is not set up, and nothing is
sent to it.
## TODO ## TODO
The to-do list, this backend's open work included, is [TODO.md](../TODO.md) at - Add integration test that POSTs a report and verifies the compressed output
the repo root. - Add report decompression/query endpoint
- Add metrics (Prometheus) for buffer size, flush count, report count
## License ## License
-22
View File
@@ -115,28 +115,6 @@ func TestMalformedConfigFileStopsTheStart(t *testing.T) {
} }
} }
// TestRefusedSentryDSNStopsTheStart: a SENTRY_DSN that Sentry refuses
// stops the start, and the error, naming SENTRY_DSN, is logged as JSON.
func TestRefusedSentryDSNStopsTheStart(t *testing.T) {
t.Setenv("SENTRY_DSN", "not-a-dsn")
ctx, cancel := context.WithTimeout(t.Context(), childTimeout)
defer cancel()
child, stdout, stderr := startServer(ctx, t, t.TempDir(), freePort(ctx, t))
err := child.Wait()
if child.ProcessState.ExitCode() != 1 {
t.Fatalf("server exit = %v, want exit status 1", err)
}
requireJSONLines(t, stdout, stderr)
if !strings.Contains(stdout.String(), "SENTRY_DSN") {
t.Fatalf("no error naming SENTRY_DSN in stdout:\n%s", stdout)
}
}
// startServer runs main() in a child process listening on // startServer runs main() in a child process listening on
// 127.0.0.1:port, with home as its HOME and working directory and its // 127.0.0.1:port, with home as its HOME and working directory and its
// data directory in home, so it touches nothing outside home. Its // data directory in home, so it touches nothing outside home. Its
-1
View File
@@ -4,7 +4,6 @@ go 1.25.5
require ( require (
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8 github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
github.com/getsentry/sentry-go v0.49.0
github.com/go-chi/chi/v5 v5.2.5 github.com/go-chi/chi/v5 v5.2.5
github.com/go-chi/cors v1.2.2 github.com/go-chi/cors v1.2.2
github.com/go-chi/httprate v0.16.0 github.com/go-chi/httprate v0.16.0
+8 -16
View File
@@ -4,22 +4,18 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0= github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/getsentry/sentry-go v0.49.0 h1:Ehejknu1l023Ub7QoRBVLAI7g3Jnhqku4oWx4B4Sh5s=
github.com/getsentry/sentry-go v0.49.0/go.mod h1:nuMJAoCfe1u0Bts2ocyNI+TW8HT84vRMqwA5Qq/SKUI=
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug= github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0= github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE= github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58= github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8= github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I= github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs= github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -40,12 +36,8 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4= github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU= github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE= github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
@@ -54,8 +46,8 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc= github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI= github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY= github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc= github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik= github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
github.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8= github.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8=
@@ -101,7 +93,7 @@ golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-10
View File
@@ -49,10 +49,6 @@ var (
"METRICS_USERNAME and METRICS_PASSWORD must be set together, " + "METRICS_USERNAME and METRICS_PASSWORD must be set together, " +
"or neither", "or neither",
) )
errMetricsUsernameColon = errors.New(
"METRICS_USERNAME must not contain \":\", " +
"which basic auth cannot carry in a user name",
)
) )
// Params defines the dependencies for Config. // Params defines the dependencies for Config.
@@ -193,12 +189,6 @@ func (s *Config) check() error {
return errMetricsCredentials return errMetricsCredentials
} }
// Basic auth splits the credentials at the first ":", so with one
// in the user name every request to /metrics would get 401.
if strings.Contains(s.MetricsUsername, ":") {
return errMetricsUsernameColon
}
return checkOrigins(s.CORSAllowedOrigins) return checkOrigins(s.CORSAllowedOrigins)
} }
-10
View File
@@ -119,16 +119,6 @@ func TestMetricsCredentialsGoTogether(t *testing.T) {
} }
} }
// TestMetricsUsernameMustNotContainColon: basic auth splits the
// credentials at the first ":", so such a user name would get 401 on
// every request to /metrics.
func TestMetricsUsernameMustNotContainColon(t *testing.T) {
t.Setenv("METRICS_USERNAME", "prom:etheus")
t.Setenv("METRICS_PASSWORD", "secret")
requireConfigError(t, "METRICS_USERNAME")
}
// TestCORSAllowedOriginsMustBeOrigins: "*" would let every origin in, // TestCORSAllowedOriginsMustBeOrigins: "*" would let every origin in,
// and an entry that is not a plain origin would match no page. // and an entry that is not a plain origin would match no page.
func TestCORSAllowedOriginsMustBeOrigins(t *testing.T) { func TestCORSAllowedOriginsMustBeOrigins(t *testing.T) {
-12
View File
@@ -1,21 +1,9 @@
package server package server
import "github.com/go-chi/chi/v5"
// Router exposes the router to the external tests, which add routes
// of their own to it after SetupRoutes.
func (s *Server) Router() *chi.Mux {
return s.router
}
// MaxRequestBodyBytes exposes the router-wide body limit to the // MaxRequestBodyBytes exposes the router-wide body limit to the
// external tests. // external tests.
const MaxRequestBodyBytes = maxRequestBodyBytes const MaxRequestBodyBytes = maxRequestBodyBytes
// MetricsRequestsPerMinute exposes the /metrics rate limit to the
// external tests.
const MetricsRequestsPerMinute = metricsRequestsPerMinute
// ListenAddr exposes the address the server listens on to the // ListenAddr exposes the address the server listens on to the
// external tests. // external tests.
func (s *Server) ListenAddr() string { func (s *Server) ListenAddr() string {
+4 -21
View File
@@ -3,7 +3,6 @@ package server
import ( import (
"time" "time"
sentryhttp "github.com/getsentry/sentry-go/http"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
"github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus"
@@ -18,12 +17,6 @@ const (
// can mount s.mw.MaxBodyBytes with a smaller value to lower // can mount s.mw.MaxBodyBytes with a smaller value to lower
// its bound, but cannot raise it: this cap runs first. // its bound, but cannot raise it: this cap runs first.
maxRequestBodyBytes int64 = 1 << 20 // 1 MiB maxRequestBodyBytes int64 = 1 << 20 // 1 MiB
// metricsRequestsPerMinute is how many requests to /metrics each
// client address may make a minute, whatever their credentials. A
// scraper polling every 2 seconds sends half of it, which httprate
// never refuses.
metricsRequestsPerMinute = 60
) )
// SetupRoutes configures the chi router with middleware and // SetupRoutes configures the chi router with middleware and
@@ -39,12 +32,6 @@ func (s *Server) SetupRoutes() {
s.router.Use(s.mw.MaxBodyBytes(maxRequestBodyBytes)) s.router.Use(s.mw.MaxBodyBytes(maxRequestBodyBytes))
s.router.Use(middleware.Timeout(requestTimeout)) s.router.Use(middleware.Timeout(requestTimeout))
// Sentry reports a panic, then panics again, so that s.mw.Recoverer
// still answers 500.
if s.params.Config.SentryDSN != "" {
s.router.Use(sentryhttp.New(sentryhttp.Options{Repanic: true}).Handle)
}
// The metrics go in a registry of this server's own, not in // The metrics go in a registry of this server's own, not in
// Prometheus' default one, which takes them only once per process. // Prometheus' default one, which takes them only once per process.
registry := prometheus.NewRegistry() registry := prometheus.NewRegistry()
@@ -72,14 +59,10 @@ func (s *Server) SetupRoutes() {
Post("/api/v1/reports", s.h.HandleReport()) Post("/api/v1/reports", s.h.HandleReport())
}) })
// The rate limit comes before the basic auth, so a client past it
// gets 429 and its password is not checked.
if s.params.Config.MetricsUsername != "" { if s.params.Config.MetricsUsername != "" {
s.router.With( s.router.With(s.mw.MetricsAuth()).
s.mw.RateLimit(metricsRequestsPerMinute), Get("/metrics", promhttp.HandlerFor(
s.mw.MetricsAuth(), registry, promhttp.HandlerOpts{},
).Get("/metrics", promhttp.HandlerFor( ).ServeHTTP)
registry, promhttp.HandlerOpts{},
).ServeHTTP)
} }
} }
-111
View File
@@ -1,12 +1,10 @@
package server_test package server_test
import ( import (
"io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"strings" "strings"
"testing" "testing"
"time"
"sneak.berlin/go/netwatch/internal/config" "sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals" "sneak.berlin/go/netwatch/internal/globals"
@@ -17,7 +15,6 @@ import (
"sneak.berlin/go/netwatch/internal/reportbuf" "sneak.berlin/go/netwatch/internal/reportbuf"
"sneak.berlin/go/netwatch/internal/server" "sneak.berlin/go/netwatch/internal/server"
"github.com/getsentry/sentry-go"
"go.uber.org/fx" "go.uber.org/fx"
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
) )
@@ -188,50 +185,6 @@ func TestMetricsBehindBasicAuth(t *testing.T) {
} }
} }
// TestMetricsAreRateLimited: a client that has used up its /metrics
// allowance on wrong passwords gets 429 even with the right one, which
// is then not checked, while another client behind the same nginx
// still gets in.
func TestMetricsAreRateLimited(t *testing.T) {
t.Setenv("METRICS_USERNAME", "prometheus")
t.Setenv("METRICS_PASSWORD", "right")
// As in the container: nginx connects from loopback and names the
// client in X-Forwarded-For.
t.Setenv("TRUSTED_PROXIES", "127.0.0.1/32")
srv := newServer(t)
srv.SetupRoutes()
get := func(client, password string) int {
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodGet, "/metrics", http.NoBody)
req.RemoteAddr = "127.0.0.1:40000"
req.Header.Set("X-Forwarded-For", client)
req.SetBasicAuth("prometheus", password)
srv.ServeHTTP(rec, req)
return rec.Code
}
for i := range server.MetricsRequestsPerMinute {
if code := get("203.0.113.7", "wrong"); code != http.StatusUnauthorized {
t.Fatalf("guess %d: status = %d, want %d",
i+1, code, http.StatusUnauthorized)
}
}
if code := get("203.0.113.7", "right"); code != http.StatusTooManyRequests {
t.Fatalf("right password past the limit: status = %d, want %d",
code, http.StatusTooManyRequests)
}
if code := get("203.0.113.8", "right"); code != http.StatusOK {
t.Fatalf("another client: status = %d, want %d",
code, http.StatusOK)
}
}
// TestMetricsInTwoServers: two servers in one process can both have // TestMetricsInTwoServers: two servers in one process can both have
// metrics on. // metrics on.
func TestMetricsInTwoServers(t *testing.T) { func TestMetricsInTwoServers(t *testing.T) {
@@ -243,70 +196,6 @@ func TestMetricsInTwoServers(t *testing.T) {
} }
} }
// TestSentry: with SENTRY_DSN empty there is no Sentry client. With it
// pointing at a local server standing in for Sentry, a panic in a
// handler reaches that server, and the request still gets the 500 from
// the panic recovery.
func TestSentry(t *testing.T) {
const panicMessage = "handler panic for TestSentry"
// sentry.Init sets the client for the whole process; take it away
// again so that no other test reports to Sentry.
t.Cleanup(func() { sentry.CurrentHub().BindClient(nil) })
t.Setenv("SENTRY_DSN", "")
newServer(t)
if sentry.CurrentHub().Client() != nil {
t.Fatal("a Sentry client exists with SENTRY_DSN empty")
}
// The body of the first request the stand-in for Sentry receives.
received := make(chan string, 1)
sentryServer := httptest.NewServer(http.HandlerFunc(
func(_ http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
select {
case received <- string(body):
default:
}
},
))
defer sentryServer.Close()
t.Setenv("SENTRY_DSN",
"http://key@"+sentryServer.Listener.Addr().String()+"/1")
srv := newServer(t)
srv.SetupRoutes()
srv.Router().Get("/panic", func(http.ResponseWriter, *http.Request) {
panic(panicMessage)
})
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodGet, "/panic", http.NoBody)
srv.ServeHTTP(rec, req)
if rec.Code != http.StatusInternalServerError {
t.Fatalf("status = %d, want %d",
rec.Code, http.StatusInternalServerError)
}
// Sentry sends from a goroutine of its own.
select {
case body := <-received:
if !strings.Contains(body, panicMessage) {
t.Fatalf("the Sentry server received no report of the panic:\n%s",
body)
}
case <-time.After(5 * time.Second):
t.Fatal("nothing reached the Sentry server")
}
}
// TestHealthCheckRejectsOversizeBody sends the health check, which // TestHealthCheckRejectsOversizeBody sends the health check, which
// never reads its body, a body one byte over the limit. Only the // never reads its body, a body one byte over the limit. Only the
// router-wide body limit can reject it. // router-wide body limit can reject it.
+1 -40
View File
@@ -7,10 +7,8 @@ package server
import ( import (
"context" "context"
"fmt"
"log/slog" "log/slog"
"net/http" "net/http"
"time"
"sneak.berlin/go/netwatch/internal/config" "sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals" "sneak.berlin/go/netwatch/internal/globals"
@@ -18,15 +16,10 @@ import (
"sneak.berlin/go/netwatch/internal/logger" "sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/middleware" "sneak.berlin/go/netwatch/internal/middleware"
"github.com/getsentry/sentry-go"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"go.uber.org/fx" "go.uber.org/fx"
) )
// sentryFlushTimeout is how long shutdown waits for Sentry to send
// what it still holds.
const sentryFlushTimeout = 2 * time.Second
// Params defines the dependencies for Server. // Params defines the dependencies for Server.
type Params struct { type Params struct {
fx.In fx.In
@@ -63,11 +56,6 @@ func New(
s.log = params.Logger.Get() s.log = params.Logger.Get()
s.shutdowner = params.Shutdowner s.shutdowner = params.Shutdowner
err := s.enableSentry()
if err != nil {
return nil, err
}
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(_ context.Context) error { OnStart: func(_ context.Context) error {
// Build the router and http.Server synchronously // Build the router and http.Server synchronously
@@ -99,37 +87,10 @@ func (s *Server) ServeHTTP(
s.router.ServeHTTP(w, r) s.router.ServeHTTP(w, r)
} }
// enableSentry sets Sentry up when SENTRY_DSN is set, so that
// SetupRoutes can report panics to it. With SENTRY_DSN empty it does
// nothing. A DSN Sentry refuses stops the start.
func (s *Server) enableSentry() error {
if s.params.Config.SentryDSN == "" {
return nil
}
err := sentry.Init(sentry.ClientOptions{
Dsn: s.params.Config.SentryDSN,
Release: s.params.Globals.Appname + "-" + s.params.Globals.Version,
})
if err != nil {
return fmt.Errorf("SENTRY_DSN: %w", err)
}
s.log.Info("sentry error reporting activated")
return nil
}
// shutdown gracefully stops the HTTP server within the // shutdown gracefully stops the HTTP server within the
// deadline of the context fx provides for OnStop, then gives // deadline of the context fx provides for OnStop.
// Sentry, if set up, time to send what it still holds.
func (s *Server) shutdown(ctx context.Context) error { func (s *Server) shutdown(ctx context.Context) error {
err := s.httpServer.Shutdown(ctx) err := s.httpServer.Shutdown(ctx)
if s.params.Config.SentryDSN != "" {
sentry.Flush(sentryFlushTimeout)
}
if err != nil { if err != nil {
s.log.Error("server clean shutdown failed", "error", err) s.log.Error("server clean shutdown failed", "error", err)
-30
View File
@@ -1,30 +0,0 @@
#!/bin/sh
# script/add-dependency: add a frontend package, or move one to another
# version, with yarn add, which changes package.json and yarn.lock
# together; then install from yarn.lock with --frozen-lockfile, as
# script/bootstrap does, to show it installs as written. --dev because
# no frontend package is needed when the page runs: it ships as the
# built dist/.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
usage() {
echo "usage: make add-dependency PACKAGE=<name>@<version>" >&2
exit 2
}
main() {
# Exactly one package. A value beginning with - would reach yarn as
# an option; yarn would quietly drop all but the first of several
# packages given in one value.
[ "$#" -eq 1 ] || usage
case "$1" in
"" | -* | *[[:space:]]*) usage ;;
esac
cd "$ROOT"
yarn add --dev "$1"
yarn install --frozen-lockfile
}
main "$@"
+1 -3
View File
@@ -17,13 +17,11 @@
# #
# golangci-lint is not installed: make lint runs it in Docker, which # golangci-lint is not installed: make lint runs it in Docker, which
# this script does not install either. # this script does not install either.
#
# Unlike the org model: Go and gcc for backend/, a newer node for eslint.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-06 # Pinned versions, 2026-07-07
NODE_VERSION="22.17.0" NODE_VERSION="22.17.0"
# The oldest node the frontend's dependencies accept: the "engines" # The oldest node the frontend's dependencies accept: the "engines"
# field of eslint 10.12.0, the most demanding of them, asks for 22.13.0 # field of eslint 10.12.0, the most demanding of them, asks for 22.13.0
-13
View File
@@ -1,13 +0,0 @@
#!/bin/sh
# script/build: build the frontend for production into dist/. The Go
# backend is built by backend/script/build.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
yarn build
}
main "$@"
-13
View File
@@ -1,13 +0,0 @@
#!/bin/sh
# script/dev: run the frontend's Vite dev server. It proxies /api to a
# netwatch-server running locally (see vite.config.js).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
yarn dev
}
main "$@"
-1
View File
@@ -1,7 +1,6 @@
#!/bin/sh #!/bin/sh
# script/fmt: format the whole repo (writes): prettier over everything # script/fmt: format the whole repo (writes): prettier over everything
# it understands, then gofmt over the Go backend. # it understands, then gofmt over the Go backend.
# The org model formats only markdown; this repo also has JS and Go.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
-1
View File
@@ -1,7 +1,6 @@
#!/bin/sh #!/bin/sh
# script/fmt-check: check formatting across the whole repo (read-only). # script/fmt-check: check formatting across the whole repo (read-only).
# Same scope as script/fmt, but fails instead of writing. # Same scope as script/fmt, but fails instead of writing.
# The org model checks only markdown; this repo also has JS and Go.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+2 -2
View File
@@ -1,7 +1,7 @@
#!/bin/sh #!/bin/sh
# script/frontend-fmt: format the frontend and every other file prettier # script/frontend-fmt: format the frontend and every other file prettier
# understands, repo-wide (writes), the markdown in backend/ included. # understands, repo-wide (writes). backend/ is in .prettierignore; Go
# Prettier does not read Go; backend/script/fmt formats the Go sources. # sources are formatted by backend/script/fmt.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+2 -2
View File
@@ -8,8 +8,8 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
hook=".git/hooks/pre-commit" hook=".git/hooks/pre-commit"
printf '#!/bin/sh\nset -e\nscript/precommit\n' > .git/hooks/pre-commit printf '#!/bin/sh\nset -e\nscript/precommit\n' > "$hook"
chmod +x .git/hooks/pre-commit chmod +x "$hook"
echo "pre-commit hook installed: runs script/precommit" echo "pre-commit hook installed: runs script/precommit"
} }
-13
View File
@@ -1,13 +0,0 @@
#!/bin/sh
# script/tidy: run go mod tidy in backend/, which adds the modules the
# Go sources import, drops those they no longer do, and updates go.sum.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT/backend"
go mod tidy
}
main "$@"
+97 -130
View File
@@ -8,8 +8,6 @@
// display their real value in the latency figure. The history buffer holds // display their real value in the latency figure. The history buffer holds
// maxHistoryPoints samples (historyDuration / updateInterval). // maxHistoryPoints samples (historyDuration / updateInterval).
// reportInterval is how often collected samples are POSTed to the backend. // reportInterval is how often collected samples are POSTed to the backend.
// The interval menu changes updateInterval while the page runs; the
// getters compute their values from it each time they are read.
export const CONFIG = { export const CONFIG = {
updateInterval: 3000, updateInterval: 3000,
maxHistoryPoints: 100, maxHistoryPoints: 100,
@@ -30,39 +28,6 @@ export const CONFIG = {
return [0, 1, 2, 3, 4, 5].map((i) => Math.round((d * i) / 5)); return [0, 1, 2, 3, 4, 5].map((i) => Math.round((d * i) / 5));
}, },
canvasHeight: 96, canvasHeight: 96,
// A latency figure and its sparkline take the color of the first entry
// whose limit, in ms, the latency is below.
latencyColors: [
{ below: 50, hex: "#22c55e", className: "text-green-500" },
{ below: 100, hex: "#84cc16", className: "text-lime-500" },
{ below: 200, hex: "#eab308", className: "text-yellow-500" },
{ below: 500, hex: "#f97316", className: "text-orange-500" },
{ below: Infinity, hex: "#ef4444", className: "text-red-500" },
],
// The health is offline when more than offlineTimeouts WAN hosts timed
// out or were unreachable and at most offlineReachable answered;
// otherwise degraded when more than degradedTimeouts timed out or were
// unreachable; otherwise slow when more than slowHosts answered after
// more than slowLatency ms.
offlineTimeouts: 10,
offlineReachable: 4,
degradedTimeouts: 4,
slowHosts: 3,
slowLatency: 1000,
// The debug log keeps its last maxLogEntries lines.
maxLogEntries: 1000,
// A gateway candidate that has not answered after gatewayTimeout ms is
// passed over.
gatewayTimeout: 1500,
// When no WAN host answers, the recovery probe checks recoveryProbeHosts
// random ones every recoveryProbeInterval ms.
recoveryProbeHosts: 4,
recoveryProbeInterval: 500,
// The rows are sorted after the first round that is not discarded, then
// every roundsPerSort rounds.
roundsPerSort: 10,
// The sparklines are sized and drawn again resizeDelay ms after start.
resizeDelay: 100,
}; };
// WAN endpoints to monitor. These are used for the aggregate health/stats // WAN endpoints to monitor. These are used for the aggregate health/stats
@@ -147,8 +112,7 @@ const debugLog = [];
const log = (() => { const log = (() => {
function append(level, message) { function append(level, message) {
debugLog.push({ timestamp: new Date(), level, message }); debugLog.push({ timestamp: new Date(), level, message });
if (debugLog.length > CONFIG.maxLogEntries) if (debugLog.length > 1000) debugLog.splice(0, debugLog.length - 1000);
debugLog.splice(0, debugLog.length - CONFIG.maxLogEntries);
const panel = document.getElementById("debug-panel"); const panel = document.getElementById("debug-panel");
if (panel && !panel.classList.contains("hidden")) renderDebugLog(); if (panel && !panel.classList.contains("hidden")) renderDebugLog();
} }
@@ -208,10 +172,7 @@ async function detectGateway() {
const result = await Promise.any( const result = await Promise.any(
GATEWAY_CANDIDATES.map(async (url) => { GATEWAY_CANDIDATES.map(async (url) => {
const controller = new AbortController(); const controller = new AbortController();
const timeoutId = setTimeout( const timeoutId = setTimeout(() => controller.abort(), 1500);
() => controller.abort(),
CONFIG.gatewayTimeout,
);
try { try {
await fetch(url, { await fetch(url, {
method: "GET", method: "GET",
@@ -236,35 +197,11 @@ async function detectGateway() {
// --- App State --------------------------------------------------------------- // --- App State ---------------------------------------------------------------
// The min, max, median and average of latencies, a list of numbers, or all
// null when it is empty. The median of an even count is the mean of the
// middle two; it and the average are rounded.
function latencyStats(latencies) {
if (latencies.length === 0)
return { min: null, max: null, med: null, avg: null };
const sorted = [...latencies].sort((a, b) => a - b);
const mid = Math.floor(sorted.length / 2);
return {
min: sorted[0],
max: sorted[sorted.length - 1],
med:
sorted.length % 2
? sorted[mid]
: Math.round((sorted[mid - 1] + sorted[mid]) / 2),
avg: Math.round(
latencies.reduce((a, b) => a + b, 0) / latencies.length,
),
};
}
export class HostState { export class HostState {
constructor(host, pinned = false) { constructor(host, pinned = false) {
this.name = host.name; this.name = host.name;
this.url = host.url; this.url = host.url;
// Each entry is either a check's result, { timestamp, latency, this.history = []; // { timestamp, latency, paused }
// error }, or a round skipped while paused, { timestamp,
// latency: null, paused: true }.
this.history = [];
this.lastLatency = null; this.lastLatency = null;
this.status = "pending"; // 'online' | 'offline' | 'error' | 'pending' this.status = "pending"; // 'online' | 'offline' | 'error' | 'pending'
this.pinned = pinned; this.pinned = pinned;
@@ -288,16 +225,38 @@ export class HostState {
this._trim(); this._trim();
} }
// The min, max, median and average latency of the checks in the history averageLatency() {
// that got an answer. const valid = this.history.filter((p) => p.latency !== null);
historyStats() { if (valid.length === 0) return null;
return latencyStats( return Math.round(
this.history valid.reduce((s, p) => s + p.latency, 0) / valid.length,
.filter((p) => p.latency !== null)
.map((p) => p.latency),
); );
} }
minLatency() {
const valid = this.history.filter((p) => p.latency !== null);
if (valid.length === 0) return null;
return Math.min(...valid.map((p) => p.latency));
}
maxLatency() {
const valid = this.history.filter((p) => p.latency !== null);
if (valid.length === 0) return null;
return Math.max(...valid.map((p) => p.latency));
}
medianLatency() {
const sorted = this.history
.filter((p) => p.latency !== null)
.map((p) => p.latency)
.sort((a, b) => a - b);
if (sorted.length === 0) return null;
const mid = Math.floor(sorted.length / 2);
return sorted.length % 2
? sorted[mid]
: Math.round((sorted[mid - 1] + sorted[mid]) / 2);
}
_trim() { _trim() {
while (this.history.length > CONFIG.maxHistoryPoints) while (this.history.length > CONFIG.maxHistoryPoints)
this.history.shift(); this.history.shift();
@@ -312,10 +271,6 @@ export class AppState {
this.local = localHosts.map((h) => new HostState(h)); this.local = localHosts.map((h) => new HostState(h));
this.paused = false; this.paused = false;
this.tickCount = 0; this.tickCount = 0;
// The recovery probe's timer, null while it is not running, and the
// checks it started last.
this._recoveryProbeId = null;
this._recoveryProbeChecks = null;
} }
get allHosts() { get allHosts() {
@@ -324,13 +279,33 @@ export class AppState {
/** WAN-only stats from latest sample (excludes local) */ /** WAN-only stats from latest sample (excludes local) */
wanStats() { wanStats() {
const latencies = this.wan const reachable = this.wan.filter((h) => h.lastLatency !== null);
.filter((h) => h.lastLatency !== null) const latencies = reachable.map((h) => h.lastLatency);
.map((h) => h.lastLatency); const total = this.wan.length;
if (latencies.length === 0)
return {
reachable: 0,
total,
min: null,
max: null,
med: null,
avg: null,
};
const sorted = [...latencies].sort((a, b) => a - b);
const mid = Math.floor(sorted.length / 2);
const med =
sorted.length % 2
? sorted[mid]
: Math.round((sorted[mid - 1] + sorted[mid]) / 2);
return { return {
reachable: latencies.length, reachable: latencies.length,
total: this.wan.length, total,
...latencyStats(latencies), min: Math.min(...latencies),
max: Math.max(...latencies),
med,
avg: Math.round(
latencies.reduce((a, b) => a + b, 0) / latencies.length,
),
}; };
} }
@@ -356,16 +331,12 @@ export class AppState {
const timeouts = this.wan.filter( const timeouts = this.wan.filter(
(h) => h.status === "error" || h.status === "offline", (h) => h.status === "error" || h.status === "offline",
).length; ).length;
if ( if (timeouts > 10 && reachable <= 4) return "offline";
timeouts > CONFIG.offlineTimeouts && if (timeouts > 4) return "degraded";
reachable <= CONFIG.offlineReachable
)
return "offline";
if (timeouts > CONFIG.degradedTimeouts) return "degraded";
const slow = this.wan.filter( const slow = this.wan.filter(
(h) => h.lastLatency !== null && h.lastLatency > CONFIG.slowLatency, (h) => h.lastLatency !== null && h.lastLatency > 1000,
).length; ).length;
if (slow > CONFIG.slowHosts) return "slow"; if (slow > 3) return "slow";
return "healthy"; return "healthy";
} }
@@ -577,13 +548,21 @@ export async function measureLatency(url, signal) {
export function latencyHex(latency) { export function latencyHex(latency) {
if (latency === null) return "#6b7280"; if (latency === null) return "#6b7280";
return CONFIG.latencyColors.find((c) => latency < c.below).hex; if (latency < 50) return "#22c55e";
if (latency < 100) return "#84cc16";
if (latency < 200) return "#eab308";
if (latency < 500) return "#f97316";
return "#ef4444";
} }
export function latencyClass(latency, status) { export function latencyClass(latency, status) {
if (status === "offline" || status === "error" || latency === null) if (status === "offline" || status === "error" || latency === null)
return "text-gray-500"; return "text-gray-500";
return CONFIG.latencyColors.find((c) => latency < c.below).className; if (latency < 50) return "text-green-500";
if (latency < 100) return "text-lime-500";
if (latency < 200) return "text-yellow-500";
if (latency < 500) return "text-orange-500";
return "text-red-500";
} }
// --- Sparkline Renderer ------------------------------------------------------ // --- Sparkline Renderer ------------------------------------------------------
@@ -601,8 +580,8 @@ class SparklineRenderer {
const ch = h - m.top - m.bottom; const ch = h - m.top - m.bottom;
ctx.clearRect(0, 0, w, h); ctx.clearRect(0, 0, w, h);
SparklineRenderer._drawYAxis(ctx, w, m, ch); SparklineRenderer._drawYAxis(ctx, w, h, m, ch);
SparklineRenderer._drawXAxis(ctx, h, m, cw); SparklineRenderer._drawXAxis(ctx, w, h, m, cw);
const len = history.length; const len = history.length;
const pw = cw / (CONFIG.maxHistoryPoints - 1); const pw = cw / (CONFIG.maxHistoryPoints - 1);
@@ -618,7 +597,7 @@ class SparklineRenderer {
SparklineRenderer._drawTip(ctx, history, getX, getY); SparklineRenderer._drawTip(ctx, history, getX, getY);
} }
static _drawYAxis(ctx, w, m, ch) { static _drawYAxis(ctx, w, h, m, ch) {
ctx.font = "300 12px monospace"; ctx.font = "300 12px monospace";
ctx.textAlign = "right"; ctx.textAlign = "right";
ctx.textBaseline = "middle"; ctx.textBaseline = "middle";
@@ -635,7 +614,7 @@ class SparklineRenderer {
} }
} }
static _drawXAxis(ctx, h, m, cw) { static _drawXAxis(ctx, w, h, m, cw) {
ctx.textAlign = "center"; ctx.textAlign = "center";
ctx.textBaseline = "top"; ctx.textBaseline = "top";
for (const tick of CONFIG.xAxisTicks) { for (const tick of CONFIG.xAxisTicks) {
@@ -723,18 +702,7 @@ class SparklineRenderer {
// horizontally. // horizontally.
const STATUS_TEXT_CLASS = "status-text text-xs text-right col-span-2 mt-5"; const STATUS_TEXT_CLASS = "status-text text-xs text-right col-span-2 mt-5";
// Escapes text for HTML, so it shows as written inside an element or a function hostRowHTML(host, index, showPin = true) {
// quoted attribute and is never read as markup.
function escapeHTML(text) {
return text
.replaceAll("&", "&amp;")
.replaceAll("<", "&lt;")
.replaceAll(">", "&gt;")
.replaceAll('"', "&quot;")
.replaceAll("'", "&#39;");
}
export function hostRowHTML(host, index, showPin = true) {
const pinColor = host.pinned const pinColor = host.pinned
? "text-blue-500" ? "text-blue-500"
: "text-gray-600 hover:text-gray-400"; : "text-gray-600 hover:text-gray-400";
@@ -753,12 +721,12 @@ export function hostRowHTML(host, index, showPin = true) {
<div class="w-[420px] flex-shrink-0 grid grid-cols-[minmax(0,1fr)_auto] items-center"> <div class="w-[420px] flex-shrink-0 grid grid-cols-[minmax(0,1fr)_auto] items-center">
<div class="flex items-center gap-2 min-w-[200px]"> <div class="flex items-center gap-2 min-w-[200px]">
<div class="w-3 h-3 rounded-full flex-shrink-0 bg-[#6b7280]"></div> <div class="w-3 h-3 rounded-full flex-shrink-0 bg-[#6b7280]"></div>
<span class="font-medium text-white truncate">${escapeHTML(host.name)}</span> <span class="font-medium text-white truncate">${host.name}</span>
</div> </div>
<div class="latency-value text-4xl font-bold tabular-nums text-right mt-3" data-host="${index}"> <div class="latency-value text-4xl font-bold tabular-nums text-right mt-3" data-host="${index}">
<span class="text-gray-500">---</span> <span class="text-gray-500">---</span>
</div> </div>
<a href="${escapeHTML(host.url)}" target="_blank" rel="noopener" class="text-xs text-gray-500 truncate block col-span-2 -mt-2">${escapeHTML(host.url)}</a> <a href="${host.url}" target="_blank" rel="noopener" class="text-xs text-gray-500 truncate block col-span-2 -mt-2">${host.url}</a>
<div class="${STATUS_TEXT_CLASS} text-gray-500" data-host="${index}">waiting...</div> <div class="${STATUS_TEXT_CLASS} text-gray-500" data-host="${index}">waiting...</div>
</div> </div>
<div class="flex-grow sparkline-container rounded overflow-hidden border border-gray-700/30"> <div class="flex-grow sparkline-container rounded overflow-hidden border border-gray-700/30">
@@ -857,7 +825,7 @@ function buildUI(state) {
</div> </div>
<footer class="mt-8 text-center text-gray-600 text-xs"> <footer class="mt-8 text-center text-gray-600 text-xs">
<p>Latency measured via GET requests | CORS restrictions may affect some measurements</p> <p>Latency measured via GET requests | IPv4 only | CORS restrictions may affect some measurements</p>
<p class="mt-2"> <p class="mt-2">
<span class="inline-block w-3 h-3 rounded-full bg-green-500 mr-1 align-middle"></span>&lt;50ms <span class="inline-block w-3 h-3 rounded-full bg-green-500 mr-1 align-middle"></span>&lt;50ms
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span>&lt;100ms <span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span>&lt;100ms
@@ -924,7 +892,10 @@ function updateHostRow(host, index) {
latencyEl.innerHTML = `<span class="text-gray-500">---</span>`; latencyEl.innerHTML = `<span class="text-gray-500">---</span>`;
} }
const { min, med, avg, max } = host.historyStats(); const avg = host.averageLatency();
const med = host.medianLatency();
const min = host.minLatency();
const max = host.maxLatency();
if (host.status === "online" && avg !== null) { if (host.status === "online" && avg !== null) {
statusEl.innerHTML = statusStatsHTML([ statusEl.innerHTML = statusStatsHTML([
["min", min], ["min", min],
@@ -1071,16 +1042,14 @@ function renderDebugLog() {
info: "text-gray-300", info: "text-gray-300",
debug: "text-gray-500", debug: "text-gray-500",
}; };
el.replaceChildren( el.innerHTML = debugLog
...debugLog.map((entry) => { .map((entry) => {
const ts = formatUTCTimestamp(entry.timestamp); const ts = formatUTCTimestamp(entry.timestamp);
const cls = levelColors[entry.level] || "text-gray-400";
const lvl = entry.level.toUpperCase().padEnd(7); const lvl = entry.level.toUpperCase().padEnd(7);
const line = document.createElement("div"); return `<div class="${cls}">${ts} ${lvl} ${entry.message}</div>`;
line.className = levelColors[entry.level] || "text-gray-400"; })
line.textContent = `${ts} ${lvl} ${entry.message}`; .join("");
return line;
}),
);
el.scrollTop = el.scrollHeight; el.scrollTop = el.scrollHeight;
} }
@@ -1191,9 +1160,8 @@ export async function tick(state, signal, onOffline) {
// rows whose check ended before the resume still read "paused" // rows whose check ended before the resume still read "paused"
state.allHosts.forEach((host, i) => updateHostRow(host, i)); state.allHosts.forEach((host, i) => updateHostRow(host, i));
// Sort after the first real check, then every CONFIG.roundsPerSort // Sort after the first real check, then every 10 ticks thereafter
// ticks thereafter if (state.tickCount === 2 || state.tickCount % 10 === 1) {
if (state.tickCount === 2 || state.tickCount % CONFIG.roundsPerSort === 1) {
sortAndRebuildWAN(state); sortAndRebuildWAN(state);
} }
@@ -1216,10 +1184,9 @@ export async function tick(state, signal, onOffline) {
// --- Recovery Probe ---------------------------------------------------------- // --- Recovery Probe ----------------------------------------------------------
// When offline, check CONFIG.recoveryProbeHosts random WAN hosts every // When offline, check 4 random WAN hosts every 500ms, giving up the checks
// CONFIG.recoveryProbeInterval ms, giving up the checks started one interval // started 500ms before, so at most 4 are ever waiting. As soon as one
// before, so at most that many are ever waiting. As soon as one answers, // answers, stop probing and start a new round at once.
// stop probing and start a new round at once.
function startRecoveryProbe(state, startRounds) { function startRecoveryProbe(state, startRounds) {
if (state._recoveryProbeId) return; // already running if (state._recoveryProbeId) return; // already running
const candidates = [...state.wan]; const candidates = [...state.wan];
@@ -1227,7 +1194,7 @@ function startRecoveryProbe(state, startRounds) {
const j = Math.floor(Math.random() * (i + 1)); const j = Math.floor(Math.random() * (i + 1));
[candidates[i], candidates[j]] = [candidates[j], candidates[i]]; [candidates[i], candidates[j]] = [candidates[j], candidates[i]];
} }
const canaries = candidates.slice(0, CONFIG.recoveryProbeHosts); const canaries = candidates.slice(0, 4);
log.notice( log.notice(
`Recovery probe started (${canaries.map((h) => h.name).join(", ")})`, `Recovery probe started (${canaries.map((h) => h.name).join(", ")})`,
); );
@@ -1244,7 +1211,7 @@ function startRecoveryProbe(state, startRounds) {
startRounds(); startRounds();
}); });
} }
}, CONFIG.recoveryProbeInterval); }, 500);
} }
function stopRecoveryProbe(state) { function stopRecoveryProbe(state) {
@@ -1508,7 +1475,7 @@ async function init() {
}); });
window.addEventListener("resize", () => handleResize(state)); window.addEventListener("resize", () => handleResize(state));
setTimeout(() => handleResize(state), CONFIG.resizeDelay); setTimeout(() => handleResize(state), 100);
} }
// Bootstrap only when loaded as the page: a real DOM containing the #app // Bootstrap only when loaded as the page: a real DOM containing the #app
+10 -53
View File
@@ -8,7 +8,6 @@ import {
AppState, AppState,
CONFIG, CONFIG,
greyOutUI, greyOutUI,
hostRowHTML,
HostState, HostState,
humanDuration, humanDuration,
latencyClass, latencyClass,
@@ -231,32 +230,6 @@ test("at a 30000ms interval, after the user pauses and resumes during a round, n
} }
}); });
// The page shows &lt; &gt; &quot; &amp; and &#39; in a row's markup as
// < > " & and '.
test(`a target whose name and URL hold < > " & and ' shows those characters in its row`, () => {
const host = new HostState({
name: `<b>"x" & 'y'</b>`,
url: `https://x.test/<b>?a="x"&b='y'`,
});
const row = hostRowHTML(host, 0);
assert.doesNotMatch(row, /<b>/);
assert.ok(
row.includes(
">&lt;b&gt;&quot;x&quot; &amp; &#39;y&#39;&lt;/b&gt;</span>",
),
);
assert.ok(
row.includes(
'href="https://x.test/&lt;b&gt;?a=&quot;x&quot;&amp;b=&#39;y&#39;"',
),
);
assert.ok(
row.includes(
">https://x.test/&lt;b&gt;?a=&quot;x&quot;&amp;b=&#39;y&#39;</a>",
),
);
});
for (const [seconds, text] of [ for (const [seconds, text] of [
[0, "0s"], [0, "0s"],
[1, "1s"], [1, "1s"],
@@ -347,35 +320,19 @@ for (const { history, latencies, statistics } of [
}, },
]) { ]) {
test(`a target's min, max, average and median latency over ${history}`, () => { test(`a target's min, max, average and median latency over ${history}`, () => {
const { min, max, avg, med } = hostAfter(latencies).historyStats(); const host = hostAfter(latencies);
assert.deepEqual({ min, max, average: avg, median: med }, statistics); assert.deepEqual(
{
min: host.minLatency(),
max: host.maxLatency(),
average: host.averageLatency(),
median: host.medianLatency(),
},
statistics,
);
}); });
} }
// The summary's figures come from each WAN target's last check, by the same
// rules as a target's own: here four answered, one was found unreachable
// and the rest have not been checked yet. The median, 22.5, and the
// average, 21.25, are rounded.
test("the summary's min, max, median and average latency over the WAN targets' last checks", () => {
const state = new AppState([]);
[30, 10, null, 25, 20].forEach((latency, i) =>
state.wan[i].pushSample(
Date.now(),
latency === null
? { latency: null, error: "unreachable" }
: { latency, error: null },
),
);
assert.deepEqual(state.wanStats(), {
reachable: 4,
total: state.wan.length,
min: 10,
max: 30,
med: 23,
avg: 21,
});
});
// An app state in which, of the WAN targets, the first timedOut timed out, // An app state in which, of the WAN targets, the first timedOut timed out,
// the next unreachable were found unreachable, the next answered answered // the next unreachable were found unreachable, the next answered answered
// after latency ms, and the rest have not been checked yet. // after latency ms, and the rest have not been checked yet.
+7 -8
View File
@@ -77,7 +77,7 @@ the internet, so the app's latency probes cannot reach anything real. The
harness answers them itself from a fixed delay table, with a deterministic harness answers them itself from a fixed delay table, with a deterministic
fraction failed outright, so the rows render a realistic spread of one-, two- fraction failed outright, so the rows render a realistic spread of one-, two-
and three-digit latencies plus some unreachable rows. That spread is what the and three-digit latencies plus some unreachable rows. That spread is what the
layout has to survive; a `---` placeholder in every row would not exercise it. layout has to survive; 24 identical `---` placeholders would not exercise it.
## What this cannot verify ## What this cannot verify
@@ -104,11 +104,10 @@ Everything else this issue was actually about — does the layout reflow, does
anything overflow, is content clipped, are the controls big enough — is a anything overflow, is content clipped, are the controls big enough — is a
function of viewport width and CSS, and is covered above. function of viewport width and CSS, and is covered above.
## Relation to the unit tests ## Relation to the unit test framework (#21)
Complementary layers, not two stacks. The unit tests in `test/unit/`, which Complementary layers, not two stacks. `vitest` (#21) will exercise module-level
`make test` runs with Node's built-in test runner, exercise the functions logic in-process with no browser. This harness exercises rendered layout in a
`src/main.js` exports in-process with no browser. This harness exercises real engine and is the only thing here that can see a media query. Neither
rendered layout in a real engine and is the only thing here that can see a media replaces the other; assertions about computed styles and element geometry belong
query. Neither replaces the other; assertions about computed styles and element here, assertions about functions belong in `vitest`.
geometry belong here, assertions about functions belong in `test/unit/`.