Compare commits
5
Commits
2c034b4bf1
..
next
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
186f932eb8 | ||
|
|
64e142c17f | ||
|
|
161f955ae2 | ||
|
|
d40e67d4ab | ||
|
|
dc2d240725 |
@@ -1,30 +1,33 @@
|
|||||||
NetWatch is an MIT-licensed JavaScript single-page application by
|
NetWatch is an MIT-licensed JavaScript single-page application by
|
||||||
[@sneak](https://sneak.berlin) that provides real-time network latency
|
[@sneak](https://sneak.berlin) that provides real-time network latency
|
||||||
monitoring to common internet hosts, displayed with color-coded figures and
|
monitoring to common internet hosts, displayed with color-coded figures and
|
||||||
sparkline graphs, served from a static bucket or Docker container.
|
sparkline graphs, served from a static bucket or from its Docker image, where a
|
||||||
|
small Go backend stores the measurements the page reports.
|
||||||
|
|
||||||
## Getting Started
|
## Getting Started
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Install dependencies
|
# Install the dependencies and the git pre-commit hook
|
||||||
yarn install
|
make setup
|
||||||
|
|
||||||
# Development server
|
# Run the page on the Vite dev server
|
||||||
yarn dev
|
make dev
|
||||||
|
|
||||||
# Production build
|
# Run the tests, both linters and the format check
|
||||||
yarn build
|
make check
|
||||||
|
|
||||||
# Preview production build
|
# Build the page into dist/
|
||||||
yarn preview
|
make build
|
||||||
|
|
||||||
# Docker
|
# Build the image and run it
|
||||||
docker build -t netwatch .
|
make docker
|
||||||
docker run -p 8080:8080 netwatch
|
docker run -p 8080:8080 netwatch
|
||||||
```
|
```
|
||||||
|
|
||||||
`yarn dev` proxies `/api` to `http://127.0.0.1:8080`, so a locally running
|
`make check` and `make docker` need Docker. `make dev` passes `/api` to
|
||||||
`netwatch-server` (see `backend/`) receives the reports the page posts.
|
`http://127.0.0.1:8080`, where `make run` in `backend/` starts `netwatch-server`
|
||||||
|
with its defaults, so the reports the page posts are stored in
|
||||||
|
`backend/data/reports`.
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
|
|
||||||
@@ -94,22 +97,25 @@ halves, so the root `make check` fails if either one is broken. We provide:
|
|||||||
The narrow-viewport layout lives in the `max-width: 768px` media block in
|
The narrow-viewport layout lives in the `max-width: 768px` media block in
|
||||||
`src/styles.css`. It is verified automatically by `make frontend-viewport-test`,
|
`src/styles.css`. It is verified automatically by `make frontend-viewport-test`,
|
||||||
which drives a digest-pinned headless Chrome against the built `dist/` and
|
which drives a digest-pinned headless Chrome against the built `dist/` and
|
||||||
asserts on computed layout at widths derived from that CSS — one pixel either
|
asserts on computed layout at widths derived from that CSS — on every breakpoint
|
||||||
side of every breakpoint it declares, plus a 320px floor, a desktop baseline and
|
it declares and one pixel either side of it, plus a 320px floor, a desktop
|
||||||
two landscape sizes. See [test/viewport/README.md](test/viewport/README.md) for
|
baseline and two landscape sizes. See
|
||||||
what it covers and what it genuinely cannot.
|
[test/viewport/README.md](test/viewport/README.md) for what it covers and what
|
||||||
|
it genuinely cannot.
|
||||||
|
|
||||||
## Rationale
|
## Rationale
|
||||||
|
|
||||||
When debugging network issues, it's useful to have a persistent at-a-glance view
|
When debugging network issues, it's useful to have a persistent at-a-glance view
|
||||||
of latency and reachability to multiple well-known internet endpoints. NetWatch
|
of latency and reachability to multiple well-known internet endpoints. NetWatch
|
||||||
provides this as a zero-dependency SPA that can be deployed anywhere static
|
provides this as a single page that does all its measuring in the browser, so it
|
||||||
files are served, with no backend required.
|
can be served from anywhere static files are served. The backend in its Docker
|
||||||
|
image only stores the measurements the page reports; without it, the page works
|
||||||
|
the same and nothing is stored.
|
||||||
|
|
||||||
## Design
|
## Design
|
||||||
|
|
||||||
The application is a single-page app built with Vite and Tailwind CSS v4. All
|
The page is built with Vite and Tailwind CSS v4. Its code is all in
|
||||||
code lives in `src/main.js` with a class-based architecture:
|
`src/main.js`, with a class-based architecture:
|
||||||
|
|
||||||
- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks,
|
- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks,
|
||||||
etc.). The interval menu sets `updateInterval`, the one value the page writes
|
etc.). The interval menu sets `updateInterval`, the one value the page writes
|
||||||
@@ -125,10 +131,10 @@ code lives in `src/main.js` with a class-based architecture:
|
|||||||
`updateSummary()` / `updateHealthBox()` handle incremental updates
|
`updateSummary()` / `updateHealthBox()` handle incremental updates
|
||||||
- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's
|
- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's
|
||||||
sample and redrawing its row as soon as its check ends, then redraws every
|
sample and redrawing its row as soon as its check ends, then redraws every
|
||||||
row, the summary and the health box once the last check ends. The rows are
|
row, the summary and the health box once the last check ends. The first round,
|
||||||
sorted then too, after the first round that is not discarded and every tenth
|
after loading or an interval change, is discarded. The rows are sorted when
|
||||||
round after that. When paused, pushes blank markers (no probes, no false
|
the last check ends in round 2, the first one kept, and in rounds 11, 21, 31
|
||||||
outage)
|
and so on. When paused, pushes blank markers (no probes, no false outage)
|
||||||
- **`Reporter`**: Posts collected samples to the backend
|
- **`Reporter`**: Posts collected samples to the backend
|
||||||
|
|
||||||
### Reporting
|
### Reporting
|
||||||
@@ -142,12 +148,35 @@ delivered report, and while paused nothing is sent. Delivery failure is quiet
|
|||||||
one debug-log line per outage, retried at the next interval, never blocking
|
one debug-log line per outage, retried at the next interval, never blocking
|
||||||
probing. The report-building step is a pure function of host state.
|
probing. The report-building step is a pure function of host state.
|
||||||
|
|
||||||
|
### Backend
|
||||||
|
|
||||||
|
`netwatch-server`, in `backend/`, is a small Go HTTP server that stores the
|
||||||
|
reports the page posts. It keeps them in memory and writes them to `DATA_DIR` as
|
||||||
|
zstd-compressed files of JSON lines: every minute, whenever 10 MiB are waiting,
|
||||||
|
and when it stops. Its routes:
|
||||||
|
|
||||||
|
- `POST /api/v1/reports` — takes a report, without credentials; each client
|
||||||
|
address may send a limited number a minute, and the report files are capped in
|
||||||
|
size, the oldest deleted first
|
||||||
|
- `GET /.well-known/healthcheck` — answers 200 with `"status":"ok"`, the
|
||||||
|
server's version and its uptime
|
||||||
|
- `GET /metrics` — Prometheus metrics behind basic auth, only when
|
||||||
|
`METRICS_USERNAME` and `METRICS_PASSWORD` are set; each client address may
|
||||||
|
make a limited number of requests to it a minute
|
||||||
|
|
||||||
|
In the image, the `builder` stage of `Dockerfile` tests it and builds it with
|
||||||
|
`backend/script/build`, and `bin/entrypoint.sh` runs it as user `netwatch` on
|
||||||
|
`127.0.0.1:8081`, behind nginx. Outside the image, `make run` in `backend/`
|
||||||
|
builds it and runs it on port 8080. Its settings, report storage and limits are
|
||||||
|
in [backend/README.md](backend/README.md).
|
||||||
|
|
||||||
### Monitoring targets
|
### Monitoring targets
|
||||||
|
|
||||||
- **22 WAN hosts**: datavi.be, Anthropic API, OpenAI API, AWS Console, GCP
|
- **26 WAN hosts**: datavi.be (pinned at start), Anthropic API, OpenAI API, AWS
|
||||||
Console, Azure, Cloudflare, Fastly, Akamai, GitHub, B2, 7 S3 regional
|
Console, Google Cloud Console, Microsoft Azure, Cloudflare, Fastly CDN,
|
||||||
endpoints (Cape Town, London, Bahrain, Tokyo, Sydney, Oregon, São Paulo), 4
|
Akamai, Google, GitHub, B2, 8 S3 regional endpoints (Cape Town, London,
|
||||||
GCS locational endpoints (Iowa, Belgium, Singapore, Sydney)
|
Bahrain, Tokyo, Singapore, Sydney, Oregon, São Paulo) and 6 Hetzner speed test
|
||||||
|
servers (Nuremberg, Falkenstein, Helsinki, Ashburn, Hillsboro, Singapore)
|
||||||
- **Local CPE**: Cable modem at 192.168.100.1 (always monitored)
|
- **Local CPE**: Cable modem at 192.168.100.1 (always monitored)
|
||||||
- **Local Gateway**: Auto-detected on startup by probing common default gateway
|
- **Local Gateway**: Auto-detected on startup by probing common default gateway
|
||||||
addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder
|
addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder
|
||||||
@@ -159,15 +188,17 @@ Local hosts are tracked separately from WAN stats.
|
|||||||
|
|
||||||
### Latency measurement
|
### Latency measurement
|
||||||
|
|
||||||
HEAD requests with `mode: 'no-cors'` and `cache: 'no-store'`, timed with
|
GET requests with `mode: 'no-cors'`, `cache: 'no-store'` and a cache-busting
|
||||||
`performance.now()`. Each check times out after 80% of the refresh interval (24
|
query parameter, timed with `performance.now()`. Each check times out after 80%
|
||||||
seconds at 30 seconds) and is then recorded as a timeout, so a round's checks
|
of the refresh interval (24 seconds at 30 seconds) and is then recorded as a
|
||||||
have all finished before the next round is due. When no WAN host answers, a
|
timeout, so a round's checks have all finished before the next round is due.
|
||||||
recovery probe checks 4 random WAN hosts every half second, giving up the checks
|
When no WAN host answers, a recovery probe checks 4 WAN hosts, picked at random
|
||||||
it started half a second before. As soon as one answers, a new round starts at
|
when it starts, every half second, giving up the checks it started half a second
|
||||||
once, as it does after an interval change. A round started early gives up the
|
before. As soon as one answers, a new round starts at once, as it does after an
|
||||||
last round's checks if they are still waiting, and that round records nothing
|
interval change. A round started early gives up the last round's checks if they
|
||||||
more, so rounds never overlap. IPv4 only.
|
are still waiting, and that round records nothing more, so rounds never overlap.
|
||||||
|
The browser chooses between IPv4 and IPv6 for each target, as for any request;
|
||||||
|
the local targets are IPv4 addresses.
|
||||||
|
|
||||||
### Color coding
|
### Color coding
|
||||||
|
|
||||||
@@ -192,25 +223,42 @@ dist/
|
|||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Real-time monitoring with 2s update interval and 300s history sparklines
|
- A round of checks every 3 seconds by default; the interval menu sets 1, 2, 3,
|
||||||
- Health indicator: green (HEALTHY) or red (DEGRADED) based on WAN reachability
|
5, 10, 15, 30 or 60 seconds and clears the history
|
||||||
- Summary stats: reachable count, min/max/avg latency across WAN hosts only
|
- Sparklines of each target's last 100 rounds: 300 seconds at 3 seconds
|
||||||
- Fixed chart axes: Y-axis 0–1000ms, X-axis 0–300s
|
- The first round after loading or an interval change is discarded, as DNS and
|
||||||
|
TLS setup inflate its latencies
|
||||||
|
- Health indicator from the WAN hosts' latest results: OFFLINE (red) when more
|
||||||
|
than 10 fail and at most 4 answer, otherwise DEGRADED (orange) when more than
|
||||||
|
4 fail, otherwise SLOW (yellow) when more than 3 take over 1000ms, otherwise
|
||||||
|
HEALTHY (green)
|
||||||
|
- Summary stats across WAN hosts only: how many answered, the min, median,
|
||||||
|
average and max of their latest latencies, the min and max over the whole
|
||||||
|
history, and the number of rounds run (`Checks`)
|
||||||
|
- Fixed chart axes: Y-axis 0–1000ms, higher latencies drawn at the top; X-axis
|
||||||
|
the time the history spans
|
||||||
- Color-coded latency figures and sparkline line segments
|
- Color-coded latency figures and sparkline line segments
|
||||||
|
- WAN host rows sorted by latest latency, unreachable last; pinned rows stay on
|
||||||
|
top, in name order
|
||||||
- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no
|
- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no
|
||||||
false outage)
|
false outage)
|
||||||
|
- Debug log panel, behind a checkbox in the footer, with five levels (error,
|
||||||
|
warning, notice, info, debug) and the last 1000 lines
|
||||||
|
- Local and UTC clocks
|
||||||
- Clickable service URLs
|
- Clickable service URLs
|
||||||
|
- A footer link to the commit the page was built from
|
||||||
- Canvas-based sparkline rendering with devicePixelRatio scaling
|
- Canvas-based sparkline rendering with devicePixelRatio scaling
|
||||||
- Zero runtime dependencies: all resources bundled into build artifacts
|
- Zero runtime dependencies: all resources bundled into build artifacts
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
|
|
||||||
After running `yarn build`, deploy the contents of the `dist/` directory to any
|
`make build` writes the page to `dist/`, which any static file host (S3, GCS,
|
||||||
static file host (S3, GCS, Cloudflare Pages, Vercel, Netlify, GitHub Pages) or
|
Cloudflare Pages, Vercel, Netlify, GitHub Pages) can serve; with no backend
|
||||||
use the Docker image behind a reverse proxy.
|
there, its reports fail quietly and nothing is stored. Or run the Docker image
|
||||||
|
behind a reverse proxy.
|
||||||
|
|
||||||
The Docker image, built from `Dockerfile`, is the whole service in one
|
The Docker image, built from `Dockerfile` by `make docker`, is the whole service
|
||||||
container: nginx serves the built frontend and passes `/api/`,
|
in one container: nginx serves the built frontend and passes `/api/`,
|
||||||
`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`,
|
`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`,
|
||||||
which listens only inside the container, on `127.0.0.1:8081`. The image:
|
which listens only inside the container, on `127.0.0.1:8081`. The image:
|
||||||
|
|
||||||
@@ -266,6 +314,10 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
|||||||
password as their basic auth credentials. With neither set, there are no
|
password as their basic auth credentials. With neither set, there are no
|
||||||
metrics and `/metrics` is not found. One set without the other, or a user
|
metrics and `/metrics` is not found. One set without the other, or a user
|
||||||
name containing `:`, stops the container
|
name containing `:`, stops the container
|
||||||
|
- `SENTRY_DSN`, default empty: set to a Sentry project's DSN, the backend
|
||||||
|
sends its errors to that Sentry project: each request whose handling
|
||||||
|
crashes, which still gets a 500 response. A value Sentry does not accept
|
||||||
|
stops the container. Empty, the backend sends nothing to Sentry
|
||||||
- **Health check:** the image's `HEALTHCHECK` requests
|
- **Health check:** the image's `HEALTHCHECK` requests
|
||||||
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
|
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
|
||||||
both nginx and the backend answer. upaas reads the container's health 60
|
both nginx and the backend answer. upaas reads the container's health 60
|
||||||
@@ -279,19 +331,19 @@ properties.
|
|||||||
|
|
||||||
## Limitations
|
## Limitations
|
||||||
|
|
||||||
- **CORS**: Some hosts may block cross-origin HEAD requests. The app uses
|
- **CORS**: The checks are cross-origin requests in `no-cors` mode, so the page
|
||||||
`no-cors` mode which allows the request but provides opaque responses. Latency
|
cannot read the answer, only time it: any answer counts as reachable, an error
|
||||||
is still measurable based on request timing.
|
page included.
|
||||||
- **Local gateway**: The 192.168.100.1 endpoint requires the host to be
|
- **Local targets**: The cable modem at 192.168.100.1 and the detected gateway
|
||||||
accessible from your network.
|
answer only on a network that has them, and only when NetWatch is served from
|
||||||
|
localhost or a private address (see Monitoring targets).
|
||||||
- **Network conditions**: Measurements reflect browser-to-endpoint latency,
|
- **Network conditions**: Measurements reflect browser-to-endpoint latency,
|
||||||
which includes your local network, ISP, and internet routing.
|
which includes your local network, ISP, and internet routing.
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
- Add configurable host list (environment variable or config file)
|
The to-do list is [TODO.md](TODO.md): where the work stands, the next step, the
|
||||||
- Add latency history export (CSV/JSON)
|
open work, and what has been done.
|
||||||
- Add notification/alert when status changes to DEGRADED
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -1,28 +1,43 @@
|
|||||||
# Workflow
|
# Workflow
|
||||||
|
|
||||||
- branch (from `main`)
|
- branch from `next`
|
||||||
- do the work in Next Step
|
- do the work in Next Step
|
||||||
- move Next Step to the top of Completed Steps
|
- move Next Step to the top of Completed Steps
|
||||||
- move the top item of Future Steps into Next Step
|
- move the top item of Future Steps into Next Step
|
||||||
- commit (`TODO.md` changes in the same commit as the work)
|
- commit (`TODO.md` changes in the same commit as the work)
|
||||||
- merge to `main` if the branch is not protected, otherwise open a PR
|
- push the branch and open a PR against `next`
|
||||||
- push
|
|
||||||
|
|
||||||
# Status
|
# Status
|
||||||
|
|
||||||
pre-1.0. No git tags. `feat/reportbuf-storage` is merged; the backend, the CI
|
pre-1.0. No git tags. `main` is the stable branch and `next` the development
|
||||||
workflow, and the backend repo standard files are all on `main`. Frontend and
|
branch, which every PR targets. The frontend and the Go backend ship as one
|
||||||
backend are both functional. Working toward the 1.0.0 milestone by closing the
|
Docker image, and the Gitea workflow `.gitea/workflows/check.yml` runs
|
||||||
remaining repo-compliance issues on the tracker.
|
`script/cibuild` on every push. Working toward 1.0.0.
|
||||||
|
|
||||||
# Next Step
|
# Next Step
|
||||||
|
|
||||||
Confirm the `.gitea/workflows/check.yml` run is green (main always green
|
Decide whether the repo moves to the layout `REPO_POLICIES.md` gives, with
|
||||||
policy). The workflow file is already on `main`; what is unverified is that its
|
`backend/` no longer repeating files from the root
|
||||||
latest run passes.
|
([#30](https://git.eeqj.de/sneak/netwatch/issues/30)).
|
||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-04: the page's footer no longer says "IPv4 only"
|
||||||
|
([#111](https://git.eeqj.de/sneak/netwatch/issues/111)): each check is a
|
||||||
|
`fetch`, the browser picks IPv4 or IPv6 for each WAN host, and the local
|
||||||
|
targets are IPv4 addresses. The rest of the footer is unchanged
|
||||||
|
- 2026-10-04: `README.md`, `TODO.md` and `test/viewport/README.md` say what the
|
||||||
|
tree does (issue #24). The README's Getting Started leads with `make` targets;
|
||||||
|
a new Backend section says what `netwatch-server` stores, its routes and how
|
||||||
|
the image builds and runs it, and points to `backend/README.md` for its
|
||||||
|
settings; the checks are GET requests; the 26 WAN hosts, the four health
|
||||||
|
states, the summary's figures and the features the list lacked are described
|
||||||
|
as the page has them; and its TODO section points here, as does the one in
|
||||||
|
`backend/README.md`, whose open items moved to Future Steps. This file's
|
||||||
|
Workflow branches from `next` and opens the PR against `next`, Status says
|
||||||
|
where the repo stands, and Next Step and Future Steps hold only open work,
|
||||||
|
linked to its issue where one exists. The viewport harness README names Node's
|
||||||
|
test runner, not `vitest`
|
||||||
- 2026-10-04: in `src/main.js` (issue #102), a target's min, max, median and
|
- 2026-10-04: in `src/main.js` (issue #102), a target's min, max, median and
|
||||||
average latency come from one list of its answers, through the same function
|
average latency come from one list of its answers, through the same function
|
||||||
the summary's figures use, so the median is written once. The latency color
|
the summary's figures use, so the median is written once. The latency color
|
||||||
@@ -33,6 +48,21 @@ latest run passes.
|
|||||||
`CONFIG` entries too. A unit test checks the summary's figures. Nothing the
|
`CONFIG` entries too. A unit test checks the summary's figures. Nothing the
|
||||||
page does or shows changed; the footer's color legend still writes the limits
|
page does or shows changed; the footer's color legend still writes the limits
|
||||||
out as text
|
out as text
|
||||||
|
- 2026-10-04: password guesses at `/metrics` are rate limited (issue #104): each
|
||||||
|
client address, resolved through `TRUSTED_PROXIES` as for reports, may make 60
|
||||||
|
requests to `/metrics` a minute, counted by `go-chi/httprate` apart from its
|
||||||
|
reports; past that it gets 429 and its basic auth credentials are not checked.
|
||||||
|
The limit is a constant in `backend/internal/server/routes.go`. A test uses up
|
||||||
|
one client's allowance on wrong passwords, gets 429 with the right one, and
|
||||||
|
checks that another client behind the same nginx still gets in
|
||||||
|
- 2026-10-04: the backend reports errors to Sentry (issue #95). With
|
||||||
|
`SENTRY_DSN` set, it sets up `sentry-go` with the release `netwatch-server-`
|
||||||
|
and its version, reports each panic in a handler through `sentryhttp`, the
|
||||||
|
last of the middleware every request goes through, which panics again so the
|
||||||
|
request still gets the 500 from the panic recovery, and waits up to 2 seconds
|
||||||
|
on shutdown for Sentry to finish sending. A DSN Sentry refuses stops the start
|
||||||
|
with an error naming `SENTRY_DSN`. With it empty, Sentry is not set up and
|
||||||
|
nothing is sent to it
|
||||||
- 2026-10-04: a target's name and URL and a debug log message show as the
|
- 2026-10-04: a target's name and URL and a debug log message show as the
|
||||||
characters they are and are never read as HTML (issue #29): a host row escapes
|
characters they are and are never read as HTML (issue #29): a host row escapes
|
||||||
the name and URL it writes into its markup, and the debug log sets each line
|
the name and URL it writes into its markup, and the debug log sets each line
|
||||||
@@ -359,12 +389,14 @@ latest run passes.
|
|||||||
|
|
||||||
# Future Steps
|
# Future Steps
|
||||||
|
|
||||||
- Wire `script/frontend-viewport-test` into CI as its own step (deliberately not
|
- Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
||||||
part of `make check` today; the decision has real CI-runtime cost and is
|
`make check`, as it needs Docker and takes minutes
|
||||||
tracked separately)
|
- A backend test that posts a report to `POST /api/v1/reports` and checks the
|
||||||
- Compliance top-up as one small commit: add .editorconfig and add the hooks
|
compressed file it is written to
|
||||||
target to the Makefile
|
- A backend route that decompresses the stored reports and answers queries on
|
||||||
- After merge, confirm .gitea/workflows/check.yml is on main and CI is green
|
them
|
||||||
(main always green policy)
|
- Prometheus metrics for the backend's in-memory buffer: its size, the number of
|
||||||
- Decide what to do with untracked resume.sh: commit it, gitignore it, or delete
|
flushes and the number of reports
|
||||||
it
|
- A configurable host list (an environment variable or a config file)
|
||||||
|
- Export of the latency history (CSV or JSON)
|
||||||
|
- A notification when the health status changes to DEGRADED
|
||||||
|
|||||||
+21
-3
@@ -90,6 +90,7 @@ project layout:
|
|||||||
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
||||||
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
|
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
|
||||||
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
|
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
|
||||||
|
| `SENTRY_DSN` | empty | DSN of the Sentry project to send errors to; see [Sentry](#sentry) |
|
||||||
|
|
||||||
`TRUSTED_PROXIES` defaults to
|
`TRUSTED_PROXIES` defaults to
|
||||||
`127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. The loopback
|
`127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. The loopback
|
||||||
@@ -198,11 +199,28 @@ is recorded and `/metrics` answers 404. One without the other stops the server
|
|||||||
from starting, with an error naming both; so does a `METRICS_USERNAME`
|
from starting, with an error naming both; so does a `METRICS_USERNAME`
|
||||||
containing `:`, which basic auth cannot carry, with an error naming it.
|
containing `:`, which basic auth cannot carry, with an error naming it.
|
||||||
|
|
||||||
|
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
||||||
|
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
||||||
|
minute, whatever their credentials. Past that it gets 429 with
|
||||||
|
`Retry-After: 60`, and its credentials are not checked. The minute slides as it
|
||||||
|
does for reports (see [Report limits](#report-limits)), so a scraper polling
|
||||||
|
every 2 seconds or less often is never refused. This allowance is apart from the
|
||||||
|
one for reports.
|
||||||
|
|
||||||
|
### Sentry
|
||||||
|
|
||||||
|
With `SENTRY_DSN` set, the server sends its errors to that Sentry project: each
|
||||||
|
panic in a handler is reported there, under the release `netwatch-server-`
|
||||||
|
followed by the server's version, and the request still gets 500 from the
|
||||||
|
server's panic recovery. On shutdown the server waits up to 2 seconds for Sentry
|
||||||
|
to finish sending. A DSN Sentry refuses stops the server from starting, with an
|
||||||
|
error naming `SENTRY_DSN`. With it empty, Sentry is not set up, and nothing is
|
||||||
|
sent to it.
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
- Add integration test that POSTs a report and verifies the compressed output
|
The to-do list, this backend's open work included, is [TODO.md](../TODO.md) at
|
||||||
- Add report decompression/query endpoint
|
the repo root.
|
||||||
- Add metrics (Prometheus) for buffer size, flush count, report count
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -115,6 +115,28 @@ func TestMalformedConfigFileStopsTheStart(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestRefusedSentryDSNStopsTheStart: a SENTRY_DSN that Sentry refuses
|
||||||
|
// stops the start, and the error, naming SENTRY_DSN, is logged as JSON.
|
||||||
|
func TestRefusedSentryDSNStopsTheStart(t *testing.T) {
|
||||||
|
t.Setenv("SENTRY_DSN", "not-a-dsn")
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(t.Context(), childTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
child, stdout, stderr := startServer(ctx, t, t.TempDir(), freePort(ctx, t))
|
||||||
|
|
||||||
|
err := child.Wait()
|
||||||
|
if child.ProcessState.ExitCode() != 1 {
|
||||||
|
t.Fatalf("server exit = %v, want exit status 1", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
requireJSONLines(t, stdout, stderr)
|
||||||
|
|
||||||
|
if !strings.Contains(stdout.String(), "SENTRY_DSN") {
|
||||||
|
t.Fatalf("no error naming SENTRY_DSN in stdout:\n%s", stdout)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// startServer runs main() in a child process listening on
|
// startServer runs main() in a child process listening on
|
||||||
// 127.0.0.1:port, with home as its HOME and working directory and its
|
// 127.0.0.1:port, with home as its HOME and working directory and its
|
||||||
// data directory in home, so it touches nothing outside home. Its
|
// data directory in home, so it touches nothing outside home. Its
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ go 1.25.5
|
|||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
|
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
|
||||||
|
github.com/getsentry/sentry-go v0.49.0
|
||||||
github.com/go-chi/chi/v5 v5.2.5
|
github.com/go-chi/chi/v5 v5.2.5
|
||||||
github.com/go-chi/cors v1.2.2
|
github.com/go-chi/cors v1.2.2
|
||||||
github.com/go-chi/httprate v0.16.0
|
github.com/go-chi/httprate v0.16.0
|
||||||
|
|||||||
+16
-8
@@ -4,18 +4,22 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
|
|||||||
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
|
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
|
||||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
|
||||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
||||||
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
||||||
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
||||||
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
||||||
|
github.com/getsentry/sentry-go v0.49.0 h1:Ehejknu1l023Ub7QoRBVLAI7g3Jnhqku4oWx4B4Sh5s=
|
||||||
|
github.com/getsentry/sentry-go v0.49.0/go.mod h1:nuMJAoCfe1u0Bts2ocyNI+TW8HT84vRMqwA5Qq/SKUI=
|
||||||
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
|
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
|
||||||
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
|
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
|
||||||
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
|
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
|
||||||
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
|
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
|
||||||
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
|
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
|
||||||
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
|
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
|
||||||
|
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
|
||||||
|
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
||||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||||
@@ -36,8 +40,12 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq
|
|||||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||||
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
|
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
|
||||||
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
|
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
|
||||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
github.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=
|
||||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
github.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=
|
||||||
|
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
||||||
|
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||||
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
||||||
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
|
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
|
||||||
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
|
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
|
||||||
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
|
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
|
||||||
@@ -46,8 +54,8 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
|
|||||||
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
||||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||||
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
|
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
|
||||||
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
|
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
||||||
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
|
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
|
||||||
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
|
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
|
||||||
github.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8=
|
github.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8=
|
||||||
@@ -93,7 +101,7 @@ golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
|
|||||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
|
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
|
||||||
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
|||||||
@@ -1,9 +1,21 @@
|
|||||||
package server
|
package server
|
||||||
|
|
||||||
|
import "github.com/go-chi/chi/v5"
|
||||||
|
|
||||||
|
// Router exposes the router to the external tests, which add routes
|
||||||
|
// of their own to it after SetupRoutes.
|
||||||
|
func (s *Server) Router() *chi.Mux {
|
||||||
|
return s.router
|
||||||
|
}
|
||||||
|
|
||||||
// MaxRequestBodyBytes exposes the router-wide body limit to the
|
// MaxRequestBodyBytes exposes the router-wide body limit to the
|
||||||
// external tests.
|
// external tests.
|
||||||
const MaxRequestBodyBytes = maxRequestBodyBytes
|
const MaxRequestBodyBytes = maxRequestBodyBytes
|
||||||
|
|
||||||
|
// MetricsRequestsPerMinute exposes the /metrics rate limit to the
|
||||||
|
// external tests.
|
||||||
|
const MetricsRequestsPerMinute = metricsRequestsPerMinute
|
||||||
|
|
||||||
// ListenAddr exposes the address the server listens on to the
|
// ListenAddr exposes the address the server listens on to the
|
||||||
// external tests.
|
// external tests.
|
||||||
func (s *Server) ListenAddr() string {
|
func (s *Server) ListenAddr() string {
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package server
|
|||||||
import (
|
import (
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
sentryhttp "github.com/getsentry/sentry-go/http"
|
||||||
"github.com/go-chi/chi/v5"
|
"github.com/go-chi/chi/v5"
|
||||||
"github.com/go-chi/chi/v5/middleware"
|
"github.com/go-chi/chi/v5/middleware"
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
@@ -17,6 +18,12 @@ const (
|
|||||||
// can mount s.mw.MaxBodyBytes with a smaller value to lower
|
// can mount s.mw.MaxBodyBytes with a smaller value to lower
|
||||||
// its bound, but cannot raise it: this cap runs first.
|
// its bound, but cannot raise it: this cap runs first.
|
||||||
maxRequestBodyBytes int64 = 1 << 20 // 1 MiB
|
maxRequestBodyBytes int64 = 1 << 20 // 1 MiB
|
||||||
|
|
||||||
|
// metricsRequestsPerMinute is how many requests to /metrics each
|
||||||
|
// client address may make a minute, whatever their credentials. A
|
||||||
|
// scraper polling every 2 seconds sends half of it, which httprate
|
||||||
|
// never refuses.
|
||||||
|
metricsRequestsPerMinute = 60
|
||||||
)
|
)
|
||||||
|
|
||||||
// SetupRoutes configures the chi router with middleware and
|
// SetupRoutes configures the chi router with middleware and
|
||||||
@@ -32,6 +39,12 @@ func (s *Server) SetupRoutes() {
|
|||||||
s.router.Use(s.mw.MaxBodyBytes(maxRequestBodyBytes))
|
s.router.Use(s.mw.MaxBodyBytes(maxRequestBodyBytes))
|
||||||
s.router.Use(middleware.Timeout(requestTimeout))
|
s.router.Use(middleware.Timeout(requestTimeout))
|
||||||
|
|
||||||
|
// Sentry reports a panic, then panics again, so that s.mw.Recoverer
|
||||||
|
// still answers 500.
|
||||||
|
if s.params.Config.SentryDSN != "" {
|
||||||
|
s.router.Use(sentryhttp.New(sentryhttp.Options{Repanic: true}).Handle)
|
||||||
|
}
|
||||||
|
|
||||||
// The metrics go in a registry of this server's own, not in
|
// The metrics go in a registry of this server's own, not in
|
||||||
// Prometheus' default one, which takes them only once per process.
|
// Prometheus' default one, which takes them only once per process.
|
||||||
registry := prometheus.NewRegistry()
|
registry := prometheus.NewRegistry()
|
||||||
@@ -59,10 +72,14 @@ func (s *Server) SetupRoutes() {
|
|||||||
Post("/api/v1/reports", s.h.HandleReport())
|
Post("/api/v1/reports", s.h.HandleReport())
|
||||||
})
|
})
|
||||||
|
|
||||||
|
// The rate limit comes before the basic auth, so a client past it
|
||||||
|
// gets 429 and its password is not checked.
|
||||||
if s.params.Config.MetricsUsername != "" {
|
if s.params.Config.MetricsUsername != "" {
|
||||||
s.router.With(s.mw.MetricsAuth()).
|
s.router.With(
|
||||||
Get("/metrics", promhttp.HandlerFor(
|
s.mw.RateLimit(metricsRequestsPerMinute),
|
||||||
registry, promhttp.HandlerOpts{},
|
s.mw.MetricsAuth(),
|
||||||
).ServeHTTP)
|
).Get("/metrics", promhttp.HandlerFor(
|
||||||
|
registry, promhttp.HandlerOpts{},
|
||||||
|
).ServeHTTP)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,10 +1,12 @@
|
|||||||
package server_test
|
package server_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/netwatch/internal/config"
|
"sneak.berlin/go/netwatch/internal/config"
|
||||||
"sneak.berlin/go/netwatch/internal/globals"
|
"sneak.berlin/go/netwatch/internal/globals"
|
||||||
@@ -15,6 +17,7 @@ import (
|
|||||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||||
"sneak.berlin/go/netwatch/internal/server"
|
"sneak.berlin/go/netwatch/internal/server"
|
||||||
|
|
||||||
|
"github.com/getsentry/sentry-go"
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"go.uber.org/fx/fxtest"
|
"go.uber.org/fx/fxtest"
|
||||||
)
|
)
|
||||||
@@ -185,6 +188,50 @@ func TestMetricsBehindBasicAuth(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestMetricsAreRateLimited: a client that has used up its /metrics
|
||||||
|
// allowance on wrong passwords gets 429 even with the right one, which
|
||||||
|
// is then not checked, while another client behind the same nginx
|
||||||
|
// still gets in.
|
||||||
|
func TestMetricsAreRateLimited(t *testing.T) {
|
||||||
|
t.Setenv("METRICS_USERNAME", "prometheus")
|
||||||
|
t.Setenv("METRICS_PASSWORD", "right")
|
||||||
|
// As in the container: nginx connects from loopback and names the
|
||||||
|
// client in X-Forwarded-For.
|
||||||
|
t.Setenv("TRUSTED_PROXIES", "127.0.0.1/32")
|
||||||
|
|
||||||
|
srv := newServer(t)
|
||||||
|
srv.SetupRoutes()
|
||||||
|
|
||||||
|
get := func(client, password string) int {
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
req := httptest.NewRequestWithContext(t.Context(),
|
||||||
|
http.MethodGet, "/metrics", http.NoBody)
|
||||||
|
req.RemoteAddr = "127.0.0.1:40000"
|
||||||
|
req.Header.Set("X-Forwarded-For", client)
|
||||||
|
req.SetBasicAuth("prometheus", password)
|
||||||
|
srv.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
return rec.Code
|
||||||
|
}
|
||||||
|
|
||||||
|
for i := range server.MetricsRequestsPerMinute {
|
||||||
|
if code := get("203.0.113.7", "wrong"); code != http.StatusUnauthorized {
|
||||||
|
t.Fatalf("guess %d: status = %d, want %d",
|
||||||
|
i+1, code, http.StatusUnauthorized)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if code := get("203.0.113.7", "right"); code != http.StatusTooManyRequests {
|
||||||
|
t.Fatalf("right password past the limit: status = %d, want %d",
|
||||||
|
code, http.StatusTooManyRequests)
|
||||||
|
}
|
||||||
|
|
||||||
|
if code := get("203.0.113.8", "right"); code != http.StatusOK {
|
||||||
|
t.Fatalf("another client: status = %d, want %d",
|
||||||
|
code, http.StatusOK)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestMetricsInTwoServers: two servers in one process can both have
|
// TestMetricsInTwoServers: two servers in one process can both have
|
||||||
// metrics on.
|
// metrics on.
|
||||||
func TestMetricsInTwoServers(t *testing.T) {
|
func TestMetricsInTwoServers(t *testing.T) {
|
||||||
@@ -196,6 +243,70 @@ func TestMetricsInTwoServers(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestSentry: with SENTRY_DSN empty there is no Sentry client. With it
|
||||||
|
// pointing at a local server standing in for Sentry, a panic in a
|
||||||
|
// handler reaches that server, and the request still gets the 500 from
|
||||||
|
// the panic recovery.
|
||||||
|
func TestSentry(t *testing.T) {
|
||||||
|
const panicMessage = "handler panic for TestSentry"
|
||||||
|
|
||||||
|
// sentry.Init sets the client for the whole process; take it away
|
||||||
|
// again so that no other test reports to Sentry.
|
||||||
|
t.Cleanup(func() { sentry.CurrentHub().BindClient(nil) })
|
||||||
|
|
||||||
|
t.Setenv("SENTRY_DSN", "")
|
||||||
|
newServer(t)
|
||||||
|
|
||||||
|
if sentry.CurrentHub().Client() != nil {
|
||||||
|
t.Fatal("a Sentry client exists with SENTRY_DSN empty")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The body of the first request the stand-in for Sentry receives.
|
||||||
|
received := make(chan string, 1)
|
||||||
|
|
||||||
|
sentryServer := httptest.NewServer(http.HandlerFunc(
|
||||||
|
func(_ http.ResponseWriter, r *http.Request) {
|
||||||
|
body, _ := io.ReadAll(r.Body)
|
||||||
|
|
||||||
|
select {
|
||||||
|
case received <- string(body):
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
},
|
||||||
|
))
|
||||||
|
defer sentryServer.Close()
|
||||||
|
|
||||||
|
t.Setenv("SENTRY_DSN",
|
||||||
|
"http://key@"+sentryServer.Listener.Addr().String()+"/1")
|
||||||
|
|
||||||
|
srv := newServer(t)
|
||||||
|
srv.SetupRoutes()
|
||||||
|
srv.Router().Get("/panic", func(http.ResponseWriter, *http.Request) {
|
||||||
|
panic(panicMessage)
|
||||||
|
})
|
||||||
|
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
req := httptest.NewRequestWithContext(t.Context(),
|
||||||
|
http.MethodGet, "/panic", http.NoBody)
|
||||||
|
srv.ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
if rec.Code != http.StatusInternalServerError {
|
||||||
|
t.Fatalf("status = %d, want %d",
|
||||||
|
rec.Code, http.StatusInternalServerError)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sentry sends from a goroutine of its own.
|
||||||
|
select {
|
||||||
|
case body := <-received:
|
||||||
|
if !strings.Contains(body, panicMessage) {
|
||||||
|
t.Fatalf("the Sentry server received no report of the panic:\n%s",
|
||||||
|
body)
|
||||||
|
}
|
||||||
|
case <-time.After(5 * time.Second):
|
||||||
|
t.Fatal("nothing reached the Sentry server")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// TestHealthCheckRejectsOversizeBody sends the health check, which
|
// TestHealthCheckRejectsOversizeBody sends the health check, which
|
||||||
// never reads its body, a body one byte over the limit. Only the
|
// never reads its body, a body one byte over the limit. Only the
|
||||||
// router-wide body limit can reject it.
|
// router-wide body limit can reject it.
|
||||||
|
|||||||
@@ -7,8 +7,10 @@ package server
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"fmt"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/netwatch/internal/config"
|
"sneak.berlin/go/netwatch/internal/config"
|
||||||
"sneak.berlin/go/netwatch/internal/globals"
|
"sneak.berlin/go/netwatch/internal/globals"
|
||||||
@@ -16,10 +18,15 @@ import (
|
|||||||
"sneak.berlin/go/netwatch/internal/logger"
|
"sneak.berlin/go/netwatch/internal/logger"
|
||||||
"sneak.berlin/go/netwatch/internal/middleware"
|
"sneak.berlin/go/netwatch/internal/middleware"
|
||||||
|
|
||||||
|
"github.com/getsentry/sentry-go"
|
||||||
"github.com/go-chi/chi/v5"
|
"github.com/go-chi/chi/v5"
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// sentryFlushTimeout is how long shutdown waits for Sentry to send
|
||||||
|
// what it still holds.
|
||||||
|
const sentryFlushTimeout = 2 * time.Second
|
||||||
|
|
||||||
// Params defines the dependencies for Server.
|
// Params defines the dependencies for Server.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
fx.In
|
fx.In
|
||||||
@@ -56,6 +63,11 @@ func New(
|
|||||||
s.log = params.Logger.Get()
|
s.log = params.Logger.Get()
|
||||||
s.shutdowner = params.Shutdowner
|
s.shutdowner = params.Shutdowner
|
||||||
|
|
||||||
|
err := s.enableSentry()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
lc.Append(fx.Hook{
|
lc.Append(fx.Hook{
|
||||||
OnStart: func(_ context.Context) error {
|
OnStart: func(_ context.Context) error {
|
||||||
// Build the router and http.Server synchronously
|
// Build the router and http.Server synchronously
|
||||||
@@ -87,10 +99,37 @@ func (s *Server) ServeHTTP(
|
|||||||
s.router.ServeHTTP(w, r)
|
s.router.ServeHTTP(w, r)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// enableSentry sets Sentry up when SENTRY_DSN is set, so that
|
||||||
|
// SetupRoutes can report panics to it. With SENTRY_DSN empty it does
|
||||||
|
// nothing. A DSN Sentry refuses stops the start.
|
||||||
|
func (s *Server) enableSentry() error {
|
||||||
|
if s.params.Config.SentryDSN == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
err := sentry.Init(sentry.ClientOptions{
|
||||||
|
Dsn: s.params.Config.SentryDSN,
|
||||||
|
Release: s.params.Globals.Appname + "-" + s.params.Globals.Version,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("SENTRY_DSN: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
s.log.Info("sentry error reporting activated")
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
// shutdown gracefully stops the HTTP server within the
|
// shutdown gracefully stops the HTTP server within the
|
||||||
// deadline of the context fx provides for OnStop.
|
// deadline of the context fx provides for OnStop, then gives
|
||||||
|
// Sentry, if set up, time to send what it still holds.
|
||||||
func (s *Server) shutdown(ctx context.Context) error {
|
func (s *Server) shutdown(ctx context.Context) error {
|
||||||
err := s.httpServer.Shutdown(ctx)
|
err := s.httpServer.Shutdown(ctx)
|
||||||
|
|
||||||
|
if s.params.Config.SentryDSN != "" {
|
||||||
|
sentry.Flush(sentryFlushTimeout)
|
||||||
|
}
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.log.Error("server clean shutdown failed", "error", err)
|
s.log.Error("server clean shutdown failed", "error", err)
|
||||||
|
|
||||||
|
|||||||
+2
-19
@@ -289,8 +289,7 @@ export class HostState {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// The min, max, median and average latency of the checks in the history
|
// The min, max, median and average latency of the checks in the history
|
||||||
// that got an answer. updateHostRow reads all four from one call; the
|
// that got an answer.
|
||||||
// methods below each return one of them.
|
|
||||||
historyStats() {
|
historyStats() {
|
||||||
return latencyStats(
|
return latencyStats(
|
||||||
this.history
|
this.history
|
||||||
@@ -299,22 +298,6 @@ export class HostState {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
averageLatency() {
|
|
||||||
return this.historyStats().avg;
|
|
||||||
}
|
|
||||||
|
|
||||||
minLatency() {
|
|
||||||
return this.historyStats().min;
|
|
||||||
}
|
|
||||||
|
|
||||||
maxLatency() {
|
|
||||||
return this.historyStats().max;
|
|
||||||
}
|
|
||||||
|
|
||||||
medianLatency() {
|
|
||||||
return this.historyStats().med;
|
|
||||||
}
|
|
||||||
|
|
||||||
_trim() {
|
_trim() {
|
||||||
while (this.history.length > CONFIG.maxHistoryPoints)
|
while (this.history.length > CONFIG.maxHistoryPoints)
|
||||||
this.history.shift();
|
this.history.shift();
|
||||||
@@ -874,7 +857,7 @@ function buildUI(state) {
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<footer class="mt-8 text-center text-gray-600 text-xs">
|
<footer class="mt-8 text-center text-gray-600 text-xs">
|
||||||
<p>Latency measured via GET requests | IPv4 only | CORS restrictions may affect some measurements</p>
|
<p>Latency measured via GET requests | CORS restrictions may affect some measurements</p>
|
||||||
<p class="mt-2">
|
<p class="mt-2">
|
||||||
<span class="inline-block w-3 h-3 rounded-full bg-green-500 mr-1 align-middle"></span><50ms
|
<span class="inline-block w-3 h-3 rounded-full bg-green-500 mr-1 align-middle"></span><50ms
|
||||||
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span><100ms
|
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span><100ms
|
||||||
|
|||||||
+2
-10
@@ -347,16 +347,8 @@ for (const { history, latencies, statistics } of [
|
|||||||
},
|
},
|
||||||
]) {
|
]) {
|
||||||
test(`a target's min, max, average and median latency over ${history}`, () => {
|
test(`a target's min, max, average and median latency over ${history}`, () => {
|
||||||
const host = hostAfter(latencies);
|
const { min, max, avg, med } = hostAfter(latencies).historyStats();
|
||||||
assert.deepEqual(
|
assert.deepEqual({ min, max, average: avg, median: med }, statistics);
|
||||||
{
|
|
||||||
min: host.minLatency(),
|
|
||||||
max: host.maxLatency(),
|
|
||||||
average: host.averageLatency(),
|
|
||||||
median: host.medianLatency(),
|
|
||||||
},
|
|
||||||
statistics,
|
|
||||||
);
|
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -77,7 +77,7 @@ the internet, so the app's latency probes cannot reach anything real. The
|
|||||||
harness answers them itself from a fixed delay table, with a deterministic
|
harness answers them itself from a fixed delay table, with a deterministic
|
||||||
fraction failed outright, so the rows render a realistic spread of one-, two-
|
fraction failed outright, so the rows render a realistic spread of one-, two-
|
||||||
and three-digit latencies plus some unreachable rows. That spread is what the
|
and three-digit latencies plus some unreachable rows. That spread is what the
|
||||||
layout has to survive; 24 identical `---` placeholders would not exercise it.
|
layout has to survive; a `---` placeholder in every row would not exercise it.
|
||||||
|
|
||||||
## What this cannot verify
|
## What this cannot verify
|
||||||
|
|
||||||
@@ -104,10 +104,11 @@ Everything else this issue was actually about — does the layout reflow, does
|
|||||||
anything overflow, is content clipped, are the controls big enough — is a
|
anything overflow, is content clipped, are the controls big enough — is a
|
||||||
function of viewport width and CSS, and is covered above.
|
function of viewport width and CSS, and is covered above.
|
||||||
|
|
||||||
## Relation to the unit test framework (#21)
|
## Relation to the unit tests
|
||||||
|
|
||||||
Complementary layers, not two stacks. `vitest` (#21) will exercise module-level
|
Complementary layers, not two stacks. The unit tests in `test/unit/`, which
|
||||||
logic in-process with no browser. This harness exercises rendered layout in a
|
`make test` runs with Node's built-in test runner, exercise the functions
|
||||||
real engine and is the only thing here that can see a media query. Neither
|
`src/main.js` exports in-process with no browser. This harness exercises
|
||||||
replaces the other; assertions about computed styles and element geometry belong
|
rendered layout in a real engine and is the only thing here that can see a media
|
||||||
here, assertions about functions belong in `vitest`.
|
query. Neither replaces the other; assertions about computed styles and element
|
||||||
|
geometry belong here, assertions about functions belong in `test/unit/`.
|
||||||
|
|||||||
Reference in New Issue
Block a user