Compare commits
8
Commits
62f593f49a
..
next
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dd119a0e46 | ||
|
|
9b08e4ee6a | ||
|
|
7d1f4c6b13 | ||
|
|
cdabf019bb | ||
|
|
d33328ec17 | ||
|
|
0a714bc57f | ||
|
|
93bbd80d57 | ||
|
|
e5db494187 |
No files matched your search
@@ -22,7 +22,7 @@ make setup
|
||||
# Serve the page from the Vite dev server
|
||||
make dev
|
||||
|
||||
# Build the page and netwatch-server, then run the server on port 8080
|
||||
# Build the page and netwatch-server, then run only the server, on port 8080
|
||||
make run
|
||||
|
||||
# Run the tests, both linters and the format check
|
||||
@@ -36,9 +36,16 @@ make docker
|
||||
docker run -p 8080:8080 netwatch
|
||||
```
|
||||
|
||||
`make check` and `make docker` need Docker. `make run` starts `netwatch-server`
|
||||
on port 8080 with its defaults, so the page's reports are stored in
|
||||
`data/reports`. `make dev` passes the page's `/api` requests there.
|
||||
After `make dev`, open the page at `http://localhost:5173/`. From the image, it
|
||||
is at `http://localhost:8080/`.
|
||||
|
||||
`make run` serves no page: `http://localhost:8080/` answers
|
||||
`404 page not found`. It starts `netwatch-server` on port 8080 with its
|
||||
defaults, which stores reports in `data/reports`. `make dev` passes the page's
|
||||
`/api` requests there, so run `make run` in a second terminal to store the
|
||||
page's reports.
|
||||
|
||||
`make check` and `make docker` need Docker.
|
||||
|
||||
## The page
|
||||
|
||||
@@ -57,7 +64,7 @@ The page checks 26 hosts on the internet, the WAN hosts:
|
||||
- the Anthropic API and the OpenAI API
|
||||
- the AWS Console, the Google Cloud Console and Microsoft Azure
|
||||
- Cloudflare, Fastly, Akamai, Google, GitHub and Backblaze B2
|
||||
- 8 S3 regional endpoints: Cape Town, London, Bahrain, Tokyo, Singapore, Sydney,
|
||||
- 8 S3 regional endpoints: Cape Town, London, Mumbai, Tokyo, Singapore, Sydney,
|
||||
Oregon and São Paulo
|
||||
- 6 Hetzner speed-test servers: Nuremberg, Falkenstein, Helsinki, Ashburn,
|
||||
Hillsboro and Singapore
|
||||
@@ -107,25 +114,29 @@ Below it, a summary line covers the WAN hosts only:
|
||||
history
|
||||
- `Checks`: the rounds run since the page loaded or the interval changed
|
||||
|
||||
The health box and the summary take in a WAN host's result as soon as its row
|
||||
shows it.
|
||||
|
||||
When no WAN host answers, the page checks 4 of them, picked at random, every
|
||||
half second. As soon as one answers, a new round starts.
|
||||
|
||||
### Rows, colors and controls
|
||||
|
||||
A figure and its line in the graph take the color of the legend in the footer,
|
||||
from green under 50 ms to red from 500 ms up; gray is no answer. The graph's
|
||||
axis ends at 1000 ms; a higher latency is drawn at the top, and the figure still
|
||||
shows its real value.
|
||||
from green under 50 ms to red from 500 ms up. The dot beside the host's name
|
||||
takes the figure's color. A failed check has a gray figure, and a faint red band
|
||||
in the graph. The graph's axis ends at 1000 ms; a higher latency is drawn at the
|
||||
top, and the figure still shows its real value.
|
||||
|
||||
From time to time, the page sorts the WAN rows by their latest latency: fastest
|
||||
first, failed last. The pin button at the left of a row keeps the row at the
|
||||
top; pinned rows are sorted by name. `datavi.be` is pinned at start.
|
||||
|
||||
`Pause` stops the checks: the graphs keep scrolling with a gap for each paused
|
||||
round, and the figures turn gray. The `Interval` menu sets the time between
|
||||
rounds: 1, 2, 3, 5, 10, 15, 30 or 60 seconds. Changing it clears the history and
|
||||
starts a new round at once. The `Debug log` checkbox in the footer opens a panel
|
||||
with the page's last 1000 log lines.
|
||||
round, and the figures and dots turn gray. The `Interval` menu sets the time
|
||||
between rounds: 1, 2, 3, 5, 10, 15, 30 or 60 seconds. Changing it clears the
|
||||
history and starts a new round at once. The `Debug log` checkbox in the footer
|
||||
opens a panel with the page's last 1000 log lines.
|
||||
|
||||
## Reporting
|
||||
|
||||
@@ -140,7 +151,9 @@ fields:
|
||||
and `history`, its samples. A sample has the check's time `t` in Unix
|
||||
milliseconds, its `latency` (`null` for a failed check) and its `error`
|
||||
(`"timeout"`, `"unreachable"` or `null`).
|
||||
- `timestamp`: the time of the report
|
||||
- `timestamp`: the time of the report, as an ISO 8601 string such as
|
||||
`2026-11-11T20:01:35.685Z`. A report whose `timestamp` is a number is refused
|
||||
with 400.
|
||||
|
||||
Each report carries only the samples no earlier report has delivered. A sample
|
||||
waits as long as the history holds it: the last 100 rounds of each host. An
|
||||
@@ -218,8 +231,9 @@ reason is in its log.
|
||||
`netwatch-server` reads its settings from environment variables. It also reads a
|
||||
`.env` file in the directory it starts in, and a YAML file named
|
||||
`netwatch-server.yaml` in `/etc/netwatch-server/` or
|
||||
`~/.config/netwatch-server/`, when one exists. A variable in the environment
|
||||
wins over a file.
|
||||
`~/.config/netwatch-server/`, when one exists. In the YAML file, each key is a
|
||||
variable's name, in upper or lower case: `PORT: 8090` sets the port. A variable
|
||||
in the environment wins over a file.
|
||||
|
||||
The server does not start when a variable holds a value it cannot use. Its error
|
||||
names the variable. An empty variable counts as unset, except `TRUSTED_PROXIES`.
|
||||
@@ -336,6 +350,10 @@ The backend logs JSON to stdout, or plain text when stdout is a terminal. Each
|
||||
request is logged with its method, URL, status, client address and duration.
|
||||
Every value taken from a request is cut to 128 bytes before it is logged.
|
||||
|
||||
The steps `fx` takes to start and stop the backend are logged at debug level, so
|
||||
they show only with `DEBUG` set. When the backend refuses a setting, its last
|
||||
log line is one error that names the setting and the reason.
|
||||
|
||||
## Design
|
||||
|
||||
The repository holds the page, the backend and the image:
|
||||
@@ -363,7 +381,8 @@ The page is built with Vite and Tailwind CSS v4. All its code is in
|
||||
- `HostState` and `AppState`: one host's history and status, and the whole
|
||||
page's state.
|
||||
- `measureLatency`: one check. `tick`: one round, which checks every host in
|
||||
parallel and draws each row as soon as its check ends.
|
||||
parallel. It draws each row as soon as its check ends, and with a WAN host's
|
||||
row also the summary and the health box.
|
||||
- `SparklineRenderer`: the graphs. `buildUI`, `updateHostRow`, `updateSummary`
|
||||
and `updateHealthBox`: the page and its parts.
|
||||
- `Reporter`: builds and posts the reports. `buildReport` is a pure function of
|
||||
@@ -445,8 +464,8 @@ says what it checks and what it cannot.
|
||||
|
||||
## TODO
|
||||
|
||||
The to-do list is [TODO.md](TODO.md): where the work stands, the next step, the
|
||||
open work, and what has been done.
|
||||
The to-do list is [TODO.md](TODO.md): where the work stands, the next step and
|
||||
the open work.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -2,705 +2,24 @@
|
||||
|
||||
- branch from `next`
|
||||
- do the work in Next Step
|
||||
- move Next Step to the top of Completed Steps
|
||||
- delete Next Step from `TODO.md`
|
||||
- move the top item of Future Steps into Next Step
|
||||
- commit (`TODO.md` changes in the same commit as the work)
|
||||
- push the branch and open a PR against `next`
|
||||
|
||||
# Status
|
||||
|
||||
pre-1.0. Release candidates are tagged on `next`, the first being `v1.0.0-rc1`;
|
||||
`git tag --merged origin/next` lists them. `main` is the stable branch and
|
||||
`next` the development branch, which every PR targets. The frontend and the Go
|
||||
backend ship as one Docker image. The Gitea workflow
|
||||
`.gitea/workflows/check.yml` runs `script/cibuild` on every push, and cancels
|
||||
the older run on the same branch. Working toward 1.0.0.
|
||||
pre-1.0. Release candidates are tagged on `next`; `git tag --merged origin/next`
|
||||
lists them. `main` is the stable branch and `next` the development branch, which
|
||||
every PR targets. The frontend and the Go backend ship as one Docker image. The
|
||||
Gitea workflow `.gitea/workflows/check.yml` runs `script/cibuild` on every push,
|
||||
and cancels the older run on the same branch. Working toward 1.0.0.
|
||||
|
||||
# Next Step
|
||||
|
||||
Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
||||
`make check`, as it needs Docker and takes minutes.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-11-10: the CI time limit of 20 minutes is on the step that runs
|
||||
`script/cibuild` ([#199](https://git.eeqj.de/sneak/netwatch/issues/199)). The
|
||||
runner ignored it on the job. The `Dockerfile`, `script/frontend-test` and
|
||||
`script/frontend-viewport-test` no longer wrap commands in `timeout`, so
|
||||
`make check` has no time limit outside CI.
|
||||
- 2026-10-26: `README.md`, `TODO.md` and `test/viewport/README.md` are rewritten
|
||||
sentence by sentence ([#189](https://git.eeqj.de/sneak/netwatch/issues/189)).
|
||||
Each sentence reads on the first pass: one idea, under about 30 words, the
|
||||
verb early. Every statement matches the code on `next`.
|
||||
- 2026-10-26: a target whose home page answers with a redirect is checked at
|
||||
`/robots.txt` on the same host, which answers without one
|
||||
([#194](https://git.eeqj.de/sneak/netwatch/issues/194)). These are the AWS
|
||||
Console, Akamai, B2 and the 8 S3 regions. The browser followed each redirect,
|
||||
so their figures also timed the page it led to. The Google Cloud Console is
|
||||
checked at `/robots.txt` too, as its home page redirects `curl`'s requests.
|
||||
The S3 regions are checked with a HEAD request, as Chrome closes their
|
||||
connection after most GET requests. `README.md` says what a figure includes.
|
||||
- 2026-10-26: the six Hetzner targets answer wherever the page is served from
|
||||
([#191](https://git.eeqj.de/sneak/netwatch/issues/191)). They close the
|
||||
connection without an answer when the request has a `Referer` header. The
|
||||
browser sent one unless the page came with the image's
|
||||
`Referrer-Policy: no-referrer` header, which a static host and `make dev` do
|
||||
not send. Each check now sets `referrerPolicy: 'no-referrer'`.
|
||||
- 2026-10-19: the server's `OnStart` hook in `internal/server/server.go` starts
|
||||
`listenAndServe` in an anonymous function, as the Go style guide asks
|
||||
([#162](https://git.eeqj.de/sneak/netwatch/issues/162)). It was
|
||||
`go s.listenAndServe()`. Nothing the server does changed.
|
||||
- 2026-10-19: `.gitignore`, `.editorconfig` and `.golangci.yml` are the
|
||||
sneak/prompts copies at `0653579`
|
||||
([#174](https://git.eeqj.de/sneak/netwatch/issues/174)). `.gitignore` is the
|
||||
standard file followed by this repository's own entries. `.editorconfig` is
|
||||
the standard file alone, which carries the `[*.go]` section. `.golangci.yml`
|
||||
disables `canonicalheader`, as the canonical copy does: golangci-lint v2.14.0
|
||||
misses its findings at random.
|
||||
- 2026-10-19: `package.json` has no `version` field and is marked
|
||||
`"private": true` ([#158](https://git.eeqj.de/sneak/netwatch/issues/158)). It
|
||||
said `1.0.0`, a release that has not been made, and nothing read it. The git
|
||||
tag is the one version; the image and `make build` take it from
|
||||
`git describe --tags`. The package is never published.
|
||||
- 2026-10-19: Status names `v1.0.0-rc1`, the first release candidate, and says
|
||||
release candidates are tagged on `next`
|
||||
([#157](https://git.eeqj.de/sneak/netwatch/issues/157)). It said there were no
|
||||
git tags.
|
||||
- 2026-10-19: `netwatch-server` starts the server only when given no arguments
|
||||
([#179](https://git.eeqj.de/sneak/netwatch/issues/179)). `check-cidr CIDR` and
|
||||
`prepare-data-dir DATA_DIR` behave as before. Any other arguments print a
|
||||
one-line usage message naming both forms to stderr and exit 2; they used to
|
||||
start the server. Unit tests cover each case.
|
||||
- 2026-10-19: `main()` in `cmd/netwatch-server/main.go` is one call into the new
|
||||
package `internal/cli`, as the Go style guide asks
|
||||
([#160](https://git.eeqj.de/sneak/netwatch/issues/160)). `internal/cli` runs
|
||||
the `check-cidr` and `prepare-data-dir` subcommands, with the same arguments,
|
||||
messages and exit codes, and otherwise the server with the `fx` wiring.
|
||||
`Appname` and `Version` stay in `main`, where the build sets `Version`. `main`
|
||||
gives `cli.Main` stderr and the function that runs the server; the tests
|
||||
replace both. Unit tests give each subcommand input it takes and input it
|
||||
refuses.
|
||||
- 2026-10-19: when `VERSION` is unset or empty, `script/build` stamps the
|
||||
version from `git describe --tags --always --dirty`, as `script/docker` does
|
||||
([#156](https://git.eeqj.de/sneak/netwatch/issues/156)). Without `--tags` it
|
||||
skipped lightweight tags, so `make run` could report a commit where the image
|
||||
built from the same tree reported the tag.
|
||||
- 2026-10-19: `.dockerignore` keeps Go test binaries and coverage output at the
|
||||
root out of the build context, with `/*.test` and `/*.out` in the repository's
|
||||
own section ([#155](https://git.eeqj.de/sneak/netwatch/issues/155)). Such
|
||||
files were copied into the lint, test and build stages. The entries match only
|
||||
at the root, so the same files in a subdirectory still reach the image.
|
||||
- 2026-10-19: `.gitea/workflows/check.yml` is the sneak/prompts copy at
|
||||
`0653579` ([#153](https://git.eeqj.de/sneak/netwatch/issues/153)). Its
|
||||
checkout step sets `fetch-depth: 0`, so CI clones all history and tags. The
|
||||
image built from a tagged commit then gets the tag as its version; it got the
|
||||
short commit. The step also sets `persist-credentials: false`, so the job's
|
||||
token is not left in `.git/config`. A new push cancels the older run on the
|
||||
same branch, and a run stops after 20 minutes.
|
||||
- 2026-10-19: `script/bootstrap` keeps the `yarn` on `PATH` only when
|
||||
`yarn --version` is exactly `YARN_VERSION`; otherwise it installs that version
|
||||
([#154](https://git.eeqj.de/sneak/netwatch/issues/154)). Any yarn used to be
|
||||
kept. It uses the `node` on `PATH` only when that node has the major version
|
||||
of `NODE_VERSION` and is at least `NODE_MIN_VERSION`; a node 24 was kept
|
||||
before. After installing node or yarn it looks the tool up through `PATH`
|
||||
again and stops if that one fails the same check, as it does for Go. It prints
|
||||
the node and yarn versions it uses. Every script that runs yarn on the host,
|
||||
except `script/bootstrap`, runs it through the new `script/yarn`. That script
|
||||
puts `~/.local/bin` first on `PATH`. It uses the first `node` there when it
|
||||
passes the same test, and otherwise runs yarn under the pinned node in nvm.
|
||||
`make build` ran yarn under whatever node was first on `PATH`.
|
||||
- 2026-10-19: `bin/entrypoint.sh` has each step in a function, and a `main` at
|
||||
the end calls them in order, as the code style guide's shell section asks
|
||||
([#163](https://git.eeqj.de/sneak/netwatch/issues/163)). It is still POSIX sh
|
||||
with no `set -e`, and does what it did, with the same messages and exit codes.
|
||||
- 2026-10-19: while report files cannot be written, the health check answers 503
|
||||
with `"status":"error"` and the same fields, so the image's `HEALTHCHECK`
|
||||
reports the container `unhealthy`
|
||||
([#142](https://git.eeqj.de/sneak/netwatch/issues/142)). It answered 200
|
||||
whatever happened, so the platform saw a healthy container while every report
|
||||
was refused with 500. It answers 200 at start, before any write has failed,
|
||||
and again from the first write that succeeds.
|
||||
- 2026-10-19: report file writes no longer overlap
|
||||
([#150](https://git.eeqj.de/sneak/netwatch/issues/150)), so the reports of
|
||||
failed writes are written in the order they arrived. Two writes that failed at
|
||||
the same time put their reports back in the wrong order when the older failed
|
||||
first. One write runs at a time. Each takes the waiting reports only when it
|
||||
starts, after a failed write has put its own back. The write that `Append`
|
||||
starts at 10 MiB waits for any write under way, and `Append` starts no other
|
||||
while it waits. Files written since the start are deleted to make room in the
|
||||
order they were written, after those found at start.
|
||||
- 2026-10-19: `DEBUG` unset or empty leaves debug logging off, and any other
|
||||
value, `false` and `0` included, turns it on, as the code style guide says
|
||||
([#159](https://git.eeqj.de/sneak/netwatch/issues/159)). No value of it stops
|
||||
the start any more. Before, only `1`, `t`, `T`, `TRUE`, `true`, `True`, `0`,
|
||||
`f`, `F`, `FALSE`, `false` and `False` were accepted. `README.md` no longer
|
||||
gives `false` as its default.
|
||||
- 2026-10-19: check builds keep Go's modules and compiled packages between runs
|
||||
([#118](https://git.eeqj.de/sneak/netwatch/issues/118)), on sneak's rulings in
|
||||
[prompts#113](https://git.eeqj.de/sneak/prompts/issues/113) and
|
||||
[prompts#130](https://git.eeqj.de/sneak/prompts/issues/130).
|
||||
`REPO_POLICIES.md`, `script/test`, `script/lint`, `script/docker` and
|
||||
`script/cibuild` are the sneak/prompts copies at `0653579`. The scripts pass a
|
||||
new `CACHEBUST` value where they passed `--no-cache`, and the lint and test
|
||||
builds write no image. In each stage of `Dockerfile`, `ARG CACHEBUST` comes
|
||||
after the system packages and the Go module download. Every step that lints,
|
||||
tests or compiles comes after it, as does the node stages' `yarn install`.
|
||||
Go's build cache and golangci-lint's are cache mounts, and both `go test` runs
|
||||
carry `-count=1`. The test phase's Go caches used to be in memory, empty on
|
||||
every build.
|
||||
- 2026-10-19: the three unit tests of a discarded check also check that the
|
||||
host's history is still empty, its status still `pending`, and the console
|
||||
untouched ([#151](https://git.eeqj.de/sneak/netwatch/issues/151)). The
|
||||
discarded cases are a round given up, a pause, and the first round. The tests
|
||||
checked only the row. So a change that recorded the result and skipped only
|
||||
the row would have kept them passing.
|
||||
- 2026-10-18: run on its own, the backend stops at start when `DATA_DIR` cannot
|
||||
be created or read, or when no file can be created in it
|
||||
([#148](https://git.eeqj.de/sneak/netwatch/issues/148)). The error names
|
||||
`DATA_DIR` and its value. The first two cases named only the path, and the
|
||||
third was found only at the first write of a report file. At start the server
|
||||
creates a file in `DATA_DIR` and removes it. The report file number is an
|
||||
`atomic.Int64`, as the Go style guide asks for int types
|
||||
([#161](https://git.eeqj.de/sneak/netwatch/issues/161)); file names are
|
||||
unchanged.
|
||||
- 2026-10-18: the image's nginx sends the page with `Cache-Control: no-cache`
|
||||
([#141](https://git.eeqj.de/sneak/netwatch/issues/141)), so a browser that
|
||||
keeps a copy checks with the server that it is current before using it again.
|
||||
The page went out with no `Cache-Control`. So a browser could keep it after a
|
||||
deploy, ask for files under `/assets/` that the new build no longer has, and
|
||||
show an empty page. A path that `location /` answers with the page through its
|
||||
`try_files` fallback gets the same header. `location /` includes the security
|
||||
headers again, as `/assets/` does, since its own `add_header` drops the
|
||||
server's. `/assets/` keeps its immutable caching.
|
||||
`make frontend-viewport-test` fails unless `/` and a path that falls back to
|
||||
the page carry `Cache-Control: no-cache` and `Content-Security-Policy`.
|
||||
- 2026-10-18: `README.md` says what the page loses while reports are refused or
|
||||
the page is paused ([#145](https://git.eeqj.de/sneak/netwatch/issues/145)). It
|
||||
says how fast one address can replace the stored reports
|
||||
([#146](https://git.eeqj.de/sneak/netwatch/issues/146)). It says that in the
|
||||
image nginx answers a body over 1 MiB itself
|
||||
([#147](https://git.eeqj.de/sneak/netwatch/issues/147)). nginx's
|
||||
`client_max_body_size` is 1 MiB by default, so in the image it answers a
|
||||
larger body with 413 and its own HTML page, and the backend never sees it. The
|
||||
reporter, the body limit and nginx are unchanged.
|
||||
- 2026-10-18: a browser that cannot format the header's clocks still probes
|
||||
([#149](https://git.eeqj.de/sneak/netwatch/issues/149)). The clocks name the
|
||||
local time zone through `Intl.DateTimeFormat`'s `shortGeneric` option, which
|
||||
Chrome before 95, Firefox before 91 and Safari before 15.4 reject with an
|
||||
error. It came before the first round started, so such a browser never checked
|
||||
a target. The clocks now start last, after the rounds, and an error formatting
|
||||
them leaves both blank. The footer's color legend reads `>=500ms` for red,
|
||||
where it read `>500ms`
|
||||
([#152](https://git.eeqj.de/sneak/netwatch/issues/152)).
|
||||
- 2026-10-18: `netwatch-server prepare-data-dir` no longer changes the owner of
|
||||
a file outside `/data` through a hard link in the volume
|
||||
([#143](https://git.eeqj.de/sneak/netwatch/issues/143)). A file with a second
|
||||
name (hard link) keeps its owner, and the start logs its path. Owners are set
|
||||
on the opened directory, file or link, never by name, so a hard link swapped
|
||||
in after the check is caught too. A symbolic link is opened without following
|
||||
it, and the link itself is given to the netwatch user. So the server can still
|
||||
follow one in a sticky directory that anyone can write to. Devices, named
|
||||
pipes and sockets keep their owner. Setting the owner on an open file that is
|
||||
not a directory is built only for Linux. `prepare-data-dir` runs only in the
|
||||
Linux image, and macOS has neither `O_PATH` nor `AT_EMPTY_PATH`. On any other
|
||||
system the package still builds, and `prepare-data-dir` fails at the first
|
||||
file that is not a directory.
|
||||
- 2026-10-18: from a trusted proxy, the backend takes the client address from
|
||||
the right-most `X-Forwarded-For` entry that is not in `TRUSTED_PROXIES`
|
||||
([#144](https://git.eeqj.de/sneak/netwatch/issues/144)). It took the left-most
|
||||
entry, which the client writes itself behind a proxy that appends to the
|
||||
header. So a client could choose the address the rate limit counts and the
|
||||
request log records. An entry written as an address with a port is read as
|
||||
that address. The direct peer is used when the header holds only trusted
|
||||
addresses, or when the search meets an entry that is not an address at all,
|
||||
such as `unknown`. `X-Real-IP` is read only without `X-Forwarded-For`, and
|
||||
every `X-Forwarded-For` header line is read, not only the first. The image's
|
||||
nginx, and the `TRUSTED_PROXIES=127.0.0.1/32` it starts the backend with, are
|
||||
unchanged.
|
||||
- 2026-10-18: after an interval change, the summary's Reachable slot reads
|
||||
`--/--` in white, as at page load, until the second round since the change
|
||||
ends ([#140](https://git.eeqj.de/sneak/netwatch/issues/140)). It read 0/26 in
|
||||
red from the change on, as no WAN host had a check recorded, for up to two
|
||||
intervals with the network fine.
|
||||
- 2026-10-14: stopping the server waits for a report file write already under
|
||||
way, then runs the final flush
|
||||
([#122](https://git.eeqj.de/sneak/netwatch/issues/122)). Such a write is the
|
||||
periodic flush's, or the one `Append` starts when 10 MiB are waiting. It used
|
||||
to return at once, and the process exiting cut the write off, losing its
|
||||
reports. A write under way that fails puts its reports back in memory, so the
|
||||
final flush writes them. The stop fails, and the process exits non-zero, only
|
||||
if that flush fails too.
|
||||
- 2026-10-14: `make test` no longer runs the frontend's production build, which
|
||||
the tests do not need ([#118](https://git.eeqj.de/sneak/netwatch/issues/118)).
|
||||
`yarn build` moved out of the stage that runs the frontend's unit tests, now
|
||||
named `frontend-test`, into a new `frontend-build` stage. That stage starts
|
||||
from `frontend-test` and, like the `builder` stage, waits on the lint and test
|
||||
phases, so the image still cannot be built unless they pass. The runtime stage
|
||||
copies `dist/` from it, and the `git` that `vite.config.js` reads the commit
|
||||
with is installed there. No build cache was added.
|
||||
- 2026-10-14: opening the debug log no longer raises the measured latencies
|
||||
([#131](https://git.eeqj.de/sneak/netwatch/issues/131)). With the panel open,
|
||||
each line written drew the panel again with every line before it, up to 1000.
|
||||
A check that ends while the page draws is timed only when the drawing is done.
|
||||
Each line is now added at the end of the open panel, and its first line is
|
||||
dropped once it holds more than 1000. It scrolls to its newest line once a
|
||||
frame rather than once a line. The whole panel is drawn only when it is
|
||||
opened.
|
||||
- 2026-10-14: unset, the backend's own `TRUSTED_PROXIES` trusts `10.0.0.0/8`,
|
||||
`172.16.0.0/12` and `192.168.0.0/16`, and no longer loopback
|
||||
([#129](https://git.eeqj.de/sneak/netwatch/issues/129)). A list replaces that.
|
||||
Set and empty, it trusts no proxy; it used to get the default, as viper reads
|
||||
an empty environment variable as unset. A value it cannot parse still stops
|
||||
the start, and the image still starts the backend with
|
||||
`TRUSTED_PROXIES=127.0.0.1/32`.
|
||||
- 2026-10-14: while report files cannot be written, `POST /api/v1/reports`
|
||||
answers 500, and storing recovers once writing works again, with no restart
|
||||
([#123](https://git.eeqj.de/sneak/netwatch/issues/123)). A failed write puts
|
||||
its reports back in the report buffer, still counted against
|
||||
`DATA_DIR_MAX_BYTES`, where they used to be lost. The next flush, within a
|
||||
minute, tries them again. Until a write succeeds, every report is refused and
|
||||
nothing of it is kept, so the buffer does not grow while writes fail.
|
||||
- 2026-10-14: a request whose handler panics is logged with status 500 and
|
||||
counted with `code` 500 in the metrics, not 200
|
||||
([#125](https://git.eeqj.de/sneak/netwatch/issues/125)). `Recoverer` is still
|
||||
the first middleware. `Write500OnPanic` sits after the request log and after
|
||||
the metrics. It writes the 500 as the panic passes and lets the panic go on to
|
||||
Sentry and `Recoverer`. `Recoverer` now writes a 500 only when no status was
|
||||
written.
|
||||
- 2026-10-14: with `TRUSTED_PROXIES` unset, the container's nginx trusts the
|
||||
three private ranges of RFC 1918, `10.0.0.0/8`, `172.16.0.0/12` and
|
||||
`192.168.0.0/16` ([#119](https://git.eeqj.de/sneak/netwatch/issues/119)), as
|
||||
sneak ruled on 2026-09-22 for every app. It trusted no proxy before. A list
|
||||
replaces them, and set and empty it trusts none. The backend inside the
|
||||
container still trusts only `127.0.0.1/32`.
|
||||
- 2026-10-14: `POST /api/v1/reports` takes only a report sent with
|
||||
`Content-Type: application/json`, parameters such as `charset` allowed
|
||||
([#126](https://git.eeqj.de/sneak/netwatch/issues/126)). It refuses any other
|
||||
with 415 and the usual error body, storing nothing. A page on another origin
|
||||
can send a `text/plain` POST without a CORS preflight, so any site could make
|
||||
its visitors' browsers store reports. A JSON POST waits for the preflight,
|
||||
which only an origin in `CORS_ALLOWED_ORIGINS` passes. The page already sent
|
||||
`application/json`.
|
||||
- 2026-10-14: changing the interval no longer drops the samples not yet reported
|
||||
to the backend ([#124](https://git.eeqj.de/sneak/netwatch/issues/124)). The
|
||||
interval menu resets each host with the new `HostState.reset`, which moves the
|
||||
history's entries to `clearedHistory`, keeping the last 100 of all it has
|
||||
cleared. `buildReport` reads both, so the next report has the samples recorded
|
||||
before the change that no report has delivered. Until its next check, such a
|
||||
host is sent with the status of its last check, not `pending`. The page's
|
||||
history still starts over. `web/src/main.js` exports `Reporter` for the tests.
|
||||
- 2026-10-14: the repo has the layout `REPO_POLICIES.md` gives
|
||||
([#30](https://git.eeqj.de/sneak/netwatch/issues/30)). The Go module is at the
|
||||
root (`go.mod`, `go.sum`, `.golangci.yml`, `cmd/netwatch-server/`,
|
||||
`internal/`), so its path `sneak.berlin/go/netwatch` names the repository.
|
||||
`go.mod` ignores `node_modules/`, where npm packages can ship `.go` files that
|
||||
`make tidy` would otherwise read as this module's packages. The page's source
|
||||
is in `web/` (`index.html`, `src/`), which `vite.config.js` makes Vite's root;
|
||||
the build still goes to `dist/`. `nginx.conf` and `security-headers.conf` are
|
||||
in `configs/`. `backend/` is gone. Its `LICENSE` and `.dockerignore` are
|
||||
deleted, its `.gitignore` entries are in the root one, and its `README.md` is
|
||||
folded into the root `README.md`. Its scripts are in the root `script/`:
|
||||
`script/build` builds the server binary after the frontend, and `script/run`
|
||||
and `script/clean` came from `backend/script/`. `make test`, `make lint`,
|
||||
`make fmt` and `make fmt-check` do what its `test`, `lint`, `fmt` and
|
||||
`fmt-check` did. `script/bootstrap` no longer installs gcc, which only the
|
||||
host Go tests needed. The image and the page are unchanged.
|
||||
- 2026-10-07: the files shared from `sneak/prompts` were its copies at commit
|
||||
`dd4027b` ([#113](https://git.eeqj.de/sneak/netwatch/issues/113)), with this
|
||||
repository's own entries after the shared content in `.gitignore`,
|
||||
`.dockerignore` and `.editorconfig`. `make lint` and `make test` each built
|
||||
one phase of `Dockerfile` without the cache. `lint` ran golangci-lint v2.14.0
|
||||
over `backend/` and, through the `frontend-lint` stage, eslint. `test` ran the
|
||||
Go tests on the Debian Go image and, through the `frontend` stage, the
|
||||
frontend's unit tests and build. The builder stage waits on both. Without a
|
||||
`VERSION` build argument it takes the version from `git describe` on the
|
||||
`.git` in the build context. Neither linter nor the tests run on the host for
|
||||
`make check`. `backend/.golangci.yml` was no longer checked against a sha256.
|
||||
prettier formats only the JavaScript, CSS, HTML and Markdown, so the shared
|
||||
`.golangci.yml` stays as fetched. `script/fmt` and `script/fmt-check` put
|
||||
`~/.local/bin` on `PATH`. `script/frontend-lint` and `script/frontend-check`
|
||||
are gone.
|
||||
- 2026-10-07: the six Hetzner targets answer again, and failed checks are
|
||||
written to the browser console
|
||||
([#114](https://git.eeqj.de/sneak/netwatch/issues/114)). The Hetzner
|
||||
speed-test servers close the connection without an answer when the URL has a
|
||||
query string, and every check added `?_cb=` and the time. Checks now fetch
|
||||
each target's URL as written, with `cache: 'no-store'` as before. Each
|
||||
recorded check that fails writes one `console.error` line, and the same line
|
||||
to the debug log. The line gives the target's name and URL, the time, what
|
||||
failed and how long the request took. A target that answers after failed
|
||||
checks writes one `console.info` line. Checks the page does not record write
|
||||
none, so the debug log no longer lists failures in the first round or in the
|
||||
recovery probe.
|
||||
- 2026-10-04: the page's footer no longer says "IPv4 only"
|
||||
([#111](https://git.eeqj.de/sneak/netwatch/issues/111)). Each check is a
|
||||
`fetch`, the browser picks IPv4 or IPv6 for each WAN host, and the local
|
||||
targets are IPv4 addresses.
|
||||
- 2026-10-04: `README.md`, `TODO.md` and `test/viewport/README.md` say what the
|
||||
tree does ([#24](https://git.eeqj.de/sneak/netwatch/issues/24)). The README's
|
||||
Getting Started leads with `make` targets. A new Backend section says what
|
||||
`netwatch-server` stores, its routes and how the image builds and runs it; it
|
||||
pointed to `backend/README.md` for its settings. The checks are GET requests.
|
||||
The 26 WAN hosts, the four health states, the summary's figures and the
|
||||
features the list lacked are described as the page has them. The README's TODO
|
||||
section points here. This file's Workflow branches from `next` and opens the
|
||||
PR against `next`. Status says where the repo stands, and Next Step and Future
|
||||
Steps hold only open work, linked to its issue where one exists. The viewport
|
||||
test's README names Node's test runner, not `vitest`.
|
||||
- 2026-10-04: in `src/main.js`, a target's min, max, median and average latency
|
||||
come from one list of its answers
|
||||
([#102](https://git.eeqj.de/sneak/netwatch/issues/102)). The summary's figures
|
||||
use the same function, so the median is written once. The latency color limits
|
||||
are one table in `CONFIG`, read by both the figure's and the sparkline's
|
||||
color. The health thresholds, the debug log's length and the gateway check's
|
||||
timeout are `CONFIG` entries too. So are the recovery probe's host count and
|
||||
interval, how often the rows are sorted, and the delay before the first
|
||||
sparkline resize. Nothing the page does or shows changed; the footer's color
|
||||
legend still writes the limits out as text.
|
||||
- 2026-10-04: password guesses at `/metrics` are rate limited
|
||||
([#104](https://git.eeqj.de/sneak/netwatch/issues/104)). Each client address,
|
||||
resolved through `TRUSTED_PROXIES` as for reports, may make 60 requests to
|
||||
`/metrics` a minute, counted by `go-chi/httprate` apart from its reports. Past
|
||||
that it gets 429 and its basic auth credentials are not checked. The limit is
|
||||
a constant in `internal/server/routes.go`.
|
||||
- 2026-10-04: the backend reports errors to Sentry
|
||||
([#95](https://git.eeqj.de/sneak/netwatch/issues/95)). With `SENTRY_DSN` set,
|
||||
it sets up `sentry-go` with the release `netwatch-server-` and its version. It
|
||||
reports each panic in a handler through `sentryhttp`, the last middleware
|
||||
every request goes through. `sentryhttp` panics again, so the request still
|
||||
gets the 500 from the panic recovery. On shutdown it waits up to 2 seconds for
|
||||
Sentry to finish sending. A DSN Sentry refuses stops the start with an error
|
||||
naming `SENTRY_DSN`. With it empty, Sentry is not set up and nothing is sent
|
||||
to it.
|
||||
- 2026-10-04: a target's name and URL and a debug log message show as the
|
||||
characters they are and are never read as HTML
|
||||
([#29](https://git.eeqj.de/sneak/netwatch/issues/29)). A host row escapes the
|
||||
name and URL it writes into its markup, and the debug log sets each line as
|
||||
text. `README.md` no longer calls `CONFIG` frozen: the interval menu sets its
|
||||
`updateInterval`, and the values computed from it follow. `AppState` declares
|
||||
the recovery probe's two properties. The sparkline axis functions lose the
|
||||
parameters they did not use, and the comment on a target's history names both
|
||||
kinds of entry it holds.
|
||||
- 2026-10-04: a dependency can be added without running yarn or go by hand
|
||||
([#45](https://git.eeqj.de/sneak/netwatch/issues/45)).
|
||||
`make add-dependency PACKAGE=<name>@<version>` shims to the new
|
||||
`script/add-dependency`, which runs `yarn add --dev`, so `package.json` and
|
||||
`yarn.lock` change together, then `yarn install --frozen-lockfile`. The same
|
||||
command moves a package to another version. `make tidy` shims to the new
|
||||
`script/tidy`, which runs `go mod tidy`. A Go module is added by importing it,
|
||||
or moved by editing its `require` line, then `make tidy`. `script/bootstrap`
|
||||
still installs with `--frozen-lockfile`.
|
||||
- 2026-10-04: the backend serves Prometheus metrics
|
||||
([#94](https://git.eeqj.de/sneak/netwatch/issues/94)). With `METRICS_USERNAME`
|
||||
and `METRICS_PASSWORD` both set, it records request duration and response size
|
||||
through `go-http-metrics`. It serves them, with Go's runtime and process
|
||||
metrics, at `GET /metrics` behind basic auth with those credentials. nginx
|
||||
passes `/metrics` to it as it does `/api/`. With neither set there are no
|
||||
metrics and `/metrics` is 404. One without the other stops the start with an
|
||||
error naming both, and so does a `METRICS_USERNAME` containing `:`. Only
|
||||
requests that reach the health check or `POST /api/v1/reports` are recorded,
|
||||
not `/metrics` itself and not every request as `GO_HTTP_SERVER_CONVENTIONS.md`
|
||||
shows. The labels are the request's path and method, which clients can make up
|
||||
without end. For that, `POST /api/v1/reports` is registered by its full path
|
||||
instead of inside a `/api/v1` route group.
|
||||
- 2026-10-04: `script/` and `Makefile` follow the org models
|
||||
([#28](https://git.eeqj.de/sneak/netwatch/issues/28)). `make dev` shims to the
|
||||
new `script/dev`, the Vite dev server, and the new `make build` to
|
||||
`script/build`, the frontend production build. `.prettierignore` no longer
|
||||
left out `backend/`, so `make fmt` and `make fmt-check` covered
|
||||
`backend/README.md`; it left out `backend/.golangci.yml` by name.
|
||||
`script/install-precommit` and the date on `script/bootstrap`'s pins are the
|
||||
org model again. `script/bootstrap`, `script/fmt` and `script/fmt-check` each
|
||||
say in a comment why they differ from it.
|
||||
- 2026-10-04: a frontend build on Node 26 or newer no longer prints Node's
|
||||
warning that `module.register()` is deprecated
|
||||
([#32](https://git.eeqj.de/sneak/netwatch/issues/32)). The build in
|
||||
`Dockerfile` runs on Node 22, which never printed it. The call was in
|
||||
`@tailwindcss/node`, which `@tailwindcss/vite` brings in at its own exact
|
||||
version. tailwind 4.3.1 calls `module.registerHooks()` instead where Node has
|
||||
it. `yarn.lock` has `@tailwindcss/vite` and `tailwindcss` at 4.3.3, inside the
|
||||
ranges `package.json` already allowed, and tailwind's own dependencies moved
|
||||
with them. The built CSS changed only in how it is written out, in tailwind's
|
||||
Firefox focus-ring rule, and in tailwind's default sans-serif font list, which
|
||||
the page does not use.
|
||||
- 2026-10-03: the frontend's unit tests cover what the page computes
|
||||
([#21](https://git.eeqj.de/sneak/netwatch/issues/21)). They cover
|
||||
`humanDuration`, and the latency colours of a figure and of a sparkline either
|
||||
side of each boundary. They cover a target's min, max, average and median
|
||||
latency, and each of the four health states either side of its thresholds.
|
||||
`package.json` has a `test` script, so `yarn run test` and `npm run test` run
|
||||
them, and `script/frontend-test` runs it. `src/main.js` exports
|
||||
`humanDuration`, `HostState`, `latencyClass` and `latencyHex` for the tests;
|
||||
nothing it does changed.
|
||||
- 2026-10-03: the backend's logs are one stream
|
||||
([#27](https://git.eeqj.de/sneak/netwatch/issues/27)). fx logs its own steps
|
||||
of starting and stopping through the backend's logger. So off a terminal,
|
||||
every line the backend and fx write is JSON; fx used to write plain text to
|
||||
stderr. A config file that is found but cannot be read stops the start with
|
||||
its error, logged as JSON like a bad setting. It used to end in a Go panic.
|
||||
The backend logs its name, version and architecture once at start. The health
|
||||
check's uptime keys are `uptime_seconds` and `uptime_human`; its path, content
|
||||
type, `"status":"ok"` and 200 are unchanged. `SENTRY_DSN`, `METRICS_USERNAME`
|
||||
and `METRICS_PASSWORD` were still read and unused until
|
||||
[#94](https://git.eeqj.de/sneak/netwatch/issues/94) and
|
||||
[#95](https://git.eeqj.de/sneak/netwatch/issues/95) wired them up.
|
||||
- 2026-10-03: the frontend has a real linter
|
||||
([#47](https://git.eeqj.de/sneak/netwatch/issues/47), and item 2 of
|
||||
[#28](https://git.eeqj.de/sneak/netwatch/issues/28)): `eslint` with its
|
||||
recommended rules, set in `eslint.config.js`, runs in a new `frontend-lint`
|
||||
stage of `Dockerfile`. `script/lint` builds the lint stages and runs no linter
|
||||
on the host. `script/bootstrap` wants node 22.13.0 or newer, as eslint 10
|
||||
does.
|
||||
- 2026-10-03: the tap-target check in `make frontend-viewport-test` expects one
|
||||
visible pin button per WAN host row
|
||||
([#46](https://git.eeqj.de/sneak/netwatch/issues/46)), where it expected at
|
||||
least 10 of the 26. Pin buttons missing from only some rows now fail it. The
|
||||
host row count the test gathers, which the `app-rendered` check also reads,
|
||||
counts only the WAN host rows: the local host rows have no pin button.
|
||||
- 2026-10-03: each target's row shows its result as soon as its check ends
|
||||
([#91](https://git.eeqj.de/sneak/netwatch/issues/91)). Every row waited for
|
||||
the round's slowest check, up to 24 seconds at a 30-second interval. Every row
|
||||
is still redrawn, and sorting, the summary, the health box and offline
|
||||
detection still run, once, when the round's last check ends. So no row reads
|
||||
"paused" after a pause and resume during the round. A check that ends after
|
||||
the user pauses or after its round is given up shows nothing. The first round
|
||||
is still discarded as a whole.
|
||||
- 2026-10-03: root no longer acts outside `/data` when it prepares `DATA_DIR`
|
||||
([#80](https://git.eeqj.de/sneak/netwatch/issues/80)). `bin/entrypoint.sh`
|
||||
runs `netwatch-server prepare-data-dir`, which refuses a `DATA_DIR` that is
|
||||
not `/data` or a path below it written in full. Then it creates `DATA_DIR`,
|
||||
gives `/data` and everything in it to `netwatch` and sets the modes, all
|
||||
through a Go `os.Root` opened on `/data`. That refuses any path leading out of
|
||||
`/data`. So no symbolic link, already there or swapped in by a host process
|
||||
during the start, can make root create or change anything elsewhere.
|
||||
`DATA_DIR=/etc` no longer gives `/etc` to `netwatch`. The `README.md` section
|
||||
"Running under upaas" says which values are accepted.
|
||||
- 2026-10-03: `DATA_DIR_MAX_BYTES` is how much of the report files is kept
|
||||
([#54](https://git.eeqj.de/sneak/netwatch/issues/54)). When a report would
|
||||
take them past it, the oldest report files are deleted to make room, each
|
||||
deletion logged. At start, files already past it are deleted the same way. A
|
||||
file still being written is never deleted. A report is refused with 507 only
|
||||
when the reports waiting to be written fill the cap on their own, and then no
|
||||
file is deleted. A file that cannot be deleted still counts until the next
|
||||
start; one already deleted by hand counts as freed.
|
||||
- 2026-10-03: `backend/script/lint` said what went wrong with its
|
||||
`.golangci.yml` check ([#34](https://git.eeqj.de/sneak/netwatch/issues/34)).
|
||||
On a hash mismatch it said to compare the file with the org standard. If they
|
||||
differed, restore the org standard. If they were the same, the org standard
|
||||
had changed, so update `GOLANGCI_CONFIG_SHA256` in that script. It used to say
|
||||
only to restore the file, which loops once the org standard itself has moved.
|
||||
A missing `.golangci.yml`, and a `sha256sum` that is missing or prints no
|
||||
hash, each got their own message instead of being reported as a mismatch.
|
||||
- 2026-10-03: the Go tests run with the race detector and coverage
|
||||
([#88](https://git.eeqj.de/sneak/netwatch/issues/88)):
|
||||
`go test -timeout 30s -race -cover ./...` and, if that fails, the same again
|
||||
with `-v`, then a failing exit. Go's `-timeout` bounds the tests, not their
|
||||
compile. The race detector needs a C compiler; the binary is still built with
|
||||
`CGO_ENABLED=0`. New tests cover the health check's answer, a valid report's
|
||||
answer, a report file's exact contents, and the flush when the buffer reaches
|
||||
10 MiB.
|
||||
- 2026-10-03: each target check times out after 80% of the refresh interval
|
||||
([#78](https://git.eeqj.de/sneak/netwatch/issues/78)), 24 seconds at 30
|
||||
seconds, where it was capped at 3 seconds. A round started early gives up the
|
||||
last round's checks if they are still waiting, so rounds never overlap. A
|
||||
round starts early after an interval change, or when the recovery probe finds
|
||||
a target answering. The recovery probe gives up its own checks after half a
|
||||
second. The frontend has its first unit tests, run by `script/frontend-test`
|
||||
with Node's built-in test runner. For them, `index.html` links
|
||||
`src/styles.css`, which `src/main.js` used to import.
|
||||
- 2026-09-29: the container sets up its own data directory
|
||||
([#75](https://git.eeqj.de/sneak/netwatch/issues/75)). `bin/entrypoint.sh`,
|
||||
still as root, created `DATA_DIR` if missing and gave it and `/data` to the
|
||||
`netwatch` user with mode 750 before starting the backend as that user. So an
|
||||
empty host directory owned by root, or one holding files from another uid,
|
||||
works with no step on the host. It stopped the start when a symbolic link was
|
||||
on the path to `DATA_DIR`, since root would change whatever the link points
|
||||
to. The `README.md` first-run step that created and chowned the host directory
|
||||
is gone, and the image no longer sets that ownership at build time.
|
||||
- 2026-09-29: CI can no longer pass on checks that did not run
|
||||
([#37](https://git.eeqj.de/sneak/netwatch/issues/37)). `script/cibuild` became
|
||||
the org model, byte for byte. It runs `script/bootstrap` and `script/check`,
|
||||
then builds the image with the version from `git describe` as the `VERSION`
|
||||
build argument. It used to be a plain `docker build .` whose check steps could
|
||||
come from the build cache. The workflow put `~/.local/bin`, where bootstrap
|
||||
links what it installs, on the step's `PATH`. bootstrap installs its pinned
|
||||
node when the installed one is older than the frontend's dependencies need.
|
||||
- 2026-09-29: `backend/.golangci.yml` was re-vendored from `sneak/prompts`
|
||||
([#41](https://git.eeqj.de/sneak/netwatch/issues/41)). `gomodguard`,
|
||||
deprecated in golangci-lint v2.12.0, is disabled and its successor
|
||||
`gomodguard_v2` enabled with the org block list, so lint runs print no
|
||||
deprecation warning. The file also turns `depguard` on with its `test-support`
|
||||
rule, which keeps `net/http/httptest` out of non-test code; netwatch adds no
|
||||
entries of its own to that rule.
|
||||
- 2026-09-29: nginx sends the security headers `REPO_POLICIES.md` requires on
|
||||
every response ([#18](https://git.eeqj.de/sneak/netwatch/issues/18)), errors
|
||||
and `/assets/` included. On what it passes on from the backend, it drops the
|
||||
backend's own copies, so each header goes out once. They live in
|
||||
`security-headers.conf`, which `nginx.conf` includes. The content security
|
||||
policy allows no inline script or style, so the status dot's grey in
|
||||
`src/main.js` is a class. `connect-src` is `*` because probed hosts redirect
|
||||
to others, and the browser checks each redirect against it.
|
||||
- 2026-09-29: the request log is bounded
|
||||
([#60](https://git.eeqj.de/sneak/netwatch/issues/60)). Each field it writes
|
||||
from the request is cut to 128 bytes, the bound the report handler already
|
||||
used. The fields are the method, URL, protocol, `User-Agent`, `Referer`,
|
||||
request ID and client address. chi takes the request ID from the client's
|
||||
`X-Request-Id` header. One request could put about 1 MiB per field into a log
|
||||
line. That bound and its helper live in the `logger` package, shared by both.
|
||||
- 2026-09-29: nginx takes the client address from `X-Forwarded-For` only on
|
||||
requests from the reverse proxies named in the container's `TRUSTED_PROXIES`
|
||||
([#64](https://git.eeqj.de/sneak/netwatch/issues/64)), and at the time by
|
||||
default from none. It trusted every RFC1918 address before, so a client could
|
||||
write a new address on each request and escape the rate limit.
|
||||
`bin/entrypoint.sh` writes one `set_real_ip_from` line per entry into
|
||||
`/etc/nginx/trusted-proxies.conf`, which `nginx.conf` includes. It refuses an
|
||||
entry that is not an IP address or CIDR, as `netwatch-server check-cidr`
|
||||
finds. It starts the backend with `TRUSTED_PROXIES=127.0.0.1/32`, since nginx
|
||||
is its only client.
|
||||
- 2026-09-29: report file names can no longer collide
|
||||
([#61](https://git.eeqj.de/sneak/netwatch/issues/61)). Each is
|
||||
`reports-<timestamp>-<number>.jsonl.zst`, where the number goes up by one for
|
||||
each file the server starts to write. So two flushes in the same millisecond
|
||||
each get a file of their own; the second one used to fail. A flush for size
|
||||
and the final flush at shutdown can fall in the same millisecond. A failed
|
||||
write uses up its number.
|
||||
- 2026-09-29: ready to run under upaas
|
||||
([#59](https://git.eeqj.de/sneak/netwatch/issues/59)). The image has a
|
||||
`HEALTHCHECK` that requests `/.well-known/healthcheck` through nginx on the
|
||||
port from `PORT`. The backend no longer reads a bad `PORT` as 0 or a bad
|
||||
`DEBUG` as false. Those, and a `BIND_ADDRESS` that is not an IP address, stop
|
||||
it from starting with an error naming the variable. The limits,
|
||||
`CORS_ALLOWED_ORIGINS` and, now by name, `TRUSTED_PROXIES` already did.
|
||||
`bin/entrypoint.sh` also refuses a `PORT` outside 1 to 65535, and `8081`,
|
||||
where the backend listens inside the container, naming `PORT`. `README.md` got
|
||||
a "Running under upaas" section.
|
||||
- 2026-09-29: nginx listens on `PORT`
|
||||
([#26](https://git.eeqj.de/sneak/netwatch/issues/26)), 8080 when unset or
|
||||
empty. The nginx image renders `nginx.conf` as a template at container start,
|
||||
filling in `PORT` and no other variable. `bin/entrypoint.sh` refuses to start
|
||||
when `PORT` is not digits only. `server_tokens off` keeps the nginx version
|
||||
out of responses. `script/frontend-viewport-test` renders the template the
|
||||
same way. Gzip and a `50x.html` error page are not added.
|
||||
- 2026-09-29: the report endpoint is bounded
|
||||
([#20](https://git.eeqj.de/sneak/netwatch/issues/20)). `POST /api/v1/reports`
|
||||
still needs no credentials. Each client address, as resolved through
|
||||
`TRUSTED_PROXIES`, may send `REPORTS_PER_MINUTE` (default 60) reports a
|
||||
minute, counted by `go-chi/httprate`, and past that gets 429 with
|
||||
`Retry-After`. The report files in `DATA_DIR`, counted from start with those
|
||||
already there, may total at most `DATA_DIR_MAX_BYTES` (default 1 GiB); past
|
||||
that, reports got 507. The wildcard CORS is gone: no CORS headers unless
|
||||
`CORS_ALLOWED_ORIGINS` lists origins. An entry that is not a plain
|
||||
`scheme://host[:port]` origin, `*` included, stops the server from starting.
|
||||
Deleting report files freed room only at the next start, until
|
||||
[#54](https://git.eeqj.de/sneak/netwatch/issues/54).
|
||||
- 2026-09-28: one container image
|
||||
([#52](https://git.eeqj.de/sneak/netwatch/issues/52)). The root `Dockerfile`
|
||||
builds the only image, and `Dockerfile.backend` is gone. nginx serves the
|
||||
frontend on port 8080 and proxies `/api/` and `/.well-known/healthcheck` to
|
||||
the backend, which listens on `127.0.0.1:8081` in the same container. The new
|
||||
`BIND_ADDRESS` setting sets its listen address. `bin/entrypoint.sh` starts
|
||||
both, passes TERM and INT on to both, and exits non-zero when either exits on
|
||||
its own. The backend runs as user `netwatch` and stores reports on the `/data`
|
||||
volume. `script/docker` is the org model again.
|
||||
- 2026-09-28: one gate ([#16](https://git.eeqj.de/sneak/netwatch/issues/16)).
|
||||
The root `make check` covers the Go backend as well as the frontend, and the
|
||||
pre-commit hook with it. The backend moved onto scripts-to-rule-them-all:
|
||||
`backend/script/*`, `backend/Makefile` as shims, its duplicate hook installer
|
||||
removed. `script/cibuild` built both images and was the workflow's only build
|
||||
step. The root `make lint` ran golangci-lint only in Docker, by building the
|
||||
lint stage of `Dockerfile.backend` without the cache. `script/bootstrap`
|
||||
installed the pinned Go unless the installed one was at least what
|
||||
`backend/go.mod` asked for. It links what it installs into `~/.local/bin`
|
||||
without replacing anything it did not create, and installs no linter. When
|
||||
`VERSION` is unset or empty, the backend binary's version falls back to
|
||||
`git describe` inside a git checkout, then to `dev`.
|
||||
- 2026-09-28: frontend reporting client
|
||||
([#53](https://git.eeqj.de/sneak/netwatch/issues/53)). A `Reporter` class
|
||||
posts collected samples to `/api/v1/reports` every `reportInterval` (default
|
||||
60s) as a per-host delta. The report-building step is a pure exported function
|
||||
of host state. A per-host mark advances only on a delivered POST. At most one
|
||||
report POST is pending at a time, and it is abandoned after half the interval.
|
||||
So a slow POST never overlaps the next report, and a mark never moves
|
||||
backwards. The samples of an abandoned POST are sent again at the next
|
||||
interval, so a backend that stored them but answered late receives them twice.
|
||||
The per-browser client id works in insecure (plain-HTTP) contexts.
|
||||
`vite.config.js` proxies `/api` to the local backend for `yarn dev`.
|
||||
- 2026-09-28: report ingest correctness
|
||||
([#23](https://git.eeqj.de/sneak/netwatch/issues/23)). A storage failure
|
||||
returns 500 instead of a false `ok`. An oversize body returns 413, apart from
|
||||
malformed JSON, which stays 400. A `MaxBodyBytes` middleware caps every route,
|
||||
not just the report route. The raw attacker-controlled `geo` blob is no longer
|
||||
logged, only its length, and `client_id`, `timestamp` and decode error text
|
||||
are length-bounded before logging. A `decodeJSON` handler helper was added.
|
||||
Panic recovery routes the stack through slog instead of chi's plain-text
|
||||
stderr. Writing a report file returns its error, so a failed final flush on
|
||||
shutdown makes the process exit non-zero instead of losing the buffered
|
||||
reports silently.
|
||||
- 2026-09-21: shutdown lifecycle correctness. The process shuts down through fx
|
||||
instead of `os.Exit`, so every component's `OnStop` runs and buffered reports
|
||||
are flushed to disk on `SIGTERM`. Before, a full flush window of telemetry was
|
||||
lost on every restart. The `http.Server` is built before its serving goroutine
|
||||
starts, so shutdown can no longer race or nil-deref it. A listen failure exits
|
||||
non-zero via `fx.Shutdowner`. `reportbuf`'s `OnStop` is idempotent.
|
||||
`writeTimeout` exceeds the chi per-request budget, so that budget is
|
||||
reachable. Dead `startupTime`, `exitCode` and `cancelFunc` fields were
|
||||
removed.
|
||||
- 2026-09-21: backend HTTP hardening
|
||||
([#19](https://git.eeqj.de/sneak/netwatch/issues/19)): `ReadHeaderTimeout` and
|
||||
`IdleTimeout` on the server, and a `SecurityHeaders` middleware (HSTS, tight
|
||||
CSP, frame, sniff, referrer and permissions headers) registered before CORS.
|
||||
`X-Forwarded-For` and `X-Real-IP` are honoured only from a `TRUSTED_PROXIES`
|
||||
allowlist, at the time loopback plus RFC1918 by default.
|
||||
- 2026-08-10: adopted the org-standard `backend/.golangci.yml` verbatim and
|
||||
moved the pinned golangci-lint from v2.7.2 to v2.12.2; the `lint` stage of
|
||||
`Dockerfile.backend` pinned the `golangci/golangci-lint:v2.12.2` image by
|
||||
digest. The previous config declared `version: "2"` but used v1 schema keys,
|
||||
so every threshold in it was inert and its green result meant nothing.
|
||||
`backend/Makefile`'s `lint` target asserted the config's sha256 against the
|
||||
canonical file first, so drift from the org standard failed the build instead
|
||||
of silently degrading to defaults.
|
||||
- 2026-08-10: every interactive control meets the 44x44 CSS px minimum tap
|
||||
target: `.pin-btn`, `#interval-select`, the debug-log label and, on narrow
|
||||
viewports, `#pause-btn`. The pin button's hit area grows via matching negative
|
||||
margins, so its layout footprint and row density are unchanged.
|
||||
- 2026-08-10: the per-host status line wraps below the 768px breakpoint instead
|
||||
of forcing horizontal page scroll at 320px.
|
||||
- 2026-08-09: `Dockerfile.backend` was reworked to the Go multistage lint-stage
|
||||
pattern. It got a separate `lint` stage on the hash-pinned
|
||||
`golangci/golangci-lint` image, and a `COPY --from=lint` stage dependency. The
|
||||
build became a `CGO_ENABLED=0` static build driven by `ARG VERSION`, with no
|
||||
more `COPY .git`.
|
||||
- 2026-08-09: dotfile compliance. `backend/.editorconfig` moved to the repo root
|
||||
so `root = true` covers the frontend too. `.gitignore` became the org model
|
||||
(OS, editor, node, and environment/secrets sections) plus this repo's `dist/`
|
||||
and `*.log`. `.env`, `.env.*`, `*.pem` and `*.key` are ignored repo-wide, not
|
||||
just under `backend/`. Excluding `.git` from `.dockerignore` stayed deferred:
|
||||
both images read git metadata at build time.
|
||||
- 2026-08-09: automated responsive-layout test (`make frontend-viewport-test`):
|
||||
a digest-pinned headless Chrome driven over CDP against the built `dist/`, at
|
||||
viewport widths derived from the breakpoints in `src/styles.css`
|
||||
([#13](https://git.eeqj.de/sneak/netwatch/issues/13)). The tap-target and
|
||||
host-row checks each fail when they measured nothing. The overflow,
|
||||
viewport-edge and clipped-text checks have no such guard of their own and rely
|
||||
on the `app-rendered` check, which fails the run when the app did not render.
|
||||
It found two real layout defects, filed as
|
||||
[#42](https://git.eeqj.de/sneak/netwatch/issues/42) and
|
||||
[#43](https://git.eeqj.de/sneak/netwatch/issues/43).
|
||||
- 2026-07-07: adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
|
||||
shims, README Entrypoints section.
|
||||
- 2026-02-27: backend with buffered zstd-compressed report storage; CI workflow
|
||||
and backend repo standard files; backend Dockerfile fixed (Go 1.25,
|
||||
golangci-lint) and moved to repo root (`feat/reportbuf-storage`).
|
||||
- 2026-02-26: host row layout redesigned with CSS grid; overflow and spacing
|
||||
fixes; nginx config extracted; port hardcoded to 8080.
|
||||
- 2026-02-26: debug log panel, median stats, recovery probe, Docker build fix,
|
||||
S3 Singapore endpoint added.
|
||||
- 2026-02-23: summary box redesign, host pinning, local and UTC clocks, checks
|
||||
counter.
|
||||
- 2026-02-23: hosts sorted by latency; GET instead of HEAD for latency; timeout
|
||||
derived from interval; Hetzner regional endpoints; 3s interval.
|
||||
- 2026-01-29: initial NetWatch network latency monitor.
|
||||
|
||||
# Future Steps
|
||||
|
||||
- A backend test that posts a report to `POST /api/v1/reports` and checks the
|
||||
|
||||
@@ -36,16 +36,103 @@ func TestMain(m *testing.M) {
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
// TestOutputIsJSON: off a terminal, every line the server writes from
|
||||
// start to stop is JSON, fx's own lines included.
|
||||
func TestOutputIsJSON(t *testing.T) {
|
||||
// TestOutputIsJSONWithoutFxLines: off a terminal, every line the server
|
||||
// writes from start to stop is JSON, and without DEBUG none of them is
|
||||
// one of fx's.
|
||||
func TestOutputIsJSONWithoutFxLines(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
stdout, stderr := runServerUntilStopped(t)
|
||||
|
||||
requireJSONLines(t, stdout, stderr)
|
||||
|
||||
if !strings.Contains(stdout.String(), `"msg":"starting"`) {
|
||||
t.Fatalf("no startup line in stdout:\n%s", stdout)
|
||||
}
|
||||
|
||||
if strings.Contains(stdout.String(), `"msg":"started"`) {
|
||||
t.Fatalf("fx line in stdout without DEBUG:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// TestDebugLogsFxLines: with DEBUG set, fx's lines are logged, as
|
||||
// JSON, from the start to the signal that stops the server.
|
||||
func TestDebugLogsFxLines(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
stdout, stderr := runServerUntilStopped(t, "DEBUG=1")
|
||||
|
||||
requireJSONLines(t, stdout, stderr)
|
||||
|
||||
for _, msg := range []string{"started", "received signal"} {
|
||||
if !strings.Contains(stdout.String(), `"msg":"`+msg+`"`) {
|
||||
t.Fatalf("no fx line %q in stdout with DEBUG set:\n%s",
|
||||
msg, stdout)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestRefusedSettingIsLastLine: a setting the server refuses stops the
|
||||
// start, and the one error line logged is the last, naming the setting
|
||||
// and why, without fx's text around it.
|
||||
func TestRefusedSettingIsLastLine(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ctx, cancel := context.WithTimeout(t.Context(), childTimeout)
|
||||
defer cancel()
|
||||
|
||||
child, stdout, stderr := startServer(ctx, t, t.TempDir(),
|
||||
freePort(ctx, t), "REPORTS_PER_MINUTE=0")
|
||||
|
||||
err := child.Wait()
|
||||
if child.ProcessState.ExitCode() != 1 {
|
||||
t.Fatalf("server exit = %v, want exit status 1", err)
|
||||
}
|
||||
|
||||
if stderr.Len() != 0 {
|
||||
t.Fatalf("stderr = %q, want nothing", stderr)
|
||||
}
|
||||
|
||||
lines := strings.Split(strings.TrimSpace(stdout.String()), "\n")
|
||||
|
||||
for _, line := range lines[:len(lines)-1] {
|
||||
if strings.Contains(line, `"level":"ERROR"`) {
|
||||
t.Fatalf("an error line before the last line:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
var last struct {
|
||||
Level string `json:"level"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
|
||||
err = json.Unmarshal([]byte(lines[len(lines)-1]), &last)
|
||||
if err != nil {
|
||||
t.Fatalf("last line is not JSON: %v\n%s", err, stdout)
|
||||
}
|
||||
|
||||
want := `REPORTS_PER_MINUTE "0": must be a positive whole number`
|
||||
if last.Level != "ERROR" || last.Error != want {
|
||||
t.Fatalf("last line has level %q, error %q; want ERROR, %q\n%s",
|
||||
last.Level, last.Error, want, stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// runServerUntilStopped runs main() in a child process with env added
|
||||
// to its environment, waits until it answers its health check, stops it
|
||||
// with SIGTERM and returns its stdout and stderr. It fails the test if
|
||||
// the child does not exit cleanly.
|
||||
func runServerUntilStopped(
|
||||
t *testing.T,
|
||||
env ...string,
|
||||
) (*bytes.Buffer, *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
ctx, cancel := context.WithTimeout(t.Context(), childTimeout)
|
||||
defer cancel()
|
||||
|
||||
port := freePort(ctx, t)
|
||||
child, stdout, stderr := startServer(ctx, t, t.TempDir(), port)
|
||||
child, stdout, stderr := startServer(ctx, t, t.TempDir(), port, env...)
|
||||
|
||||
waitForHealthcheck(ctx, t, port)
|
||||
|
||||
@@ -72,11 +159,7 @@ func TestOutputIsJSON(t *testing.T) {
|
||||
t.Fatalf("server exit = %v, want success", err)
|
||||
}
|
||||
|
||||
requireJSONLines(t, stdout, stderr)
|
||||
|
||||
if !strings.Contains(stdout.String(), `"msg":"starting"`) {
|
||||
t.Fatalf("no startup line in stdout:\n%s", stdout)
|
||||
}
|
||||
return stdout, stderr
|
||||
}
|
||||
|
||||
// TestMalformedConfigFileStopsTheStart: a config file the server finds
|
||||
@@ -137,9 +220,40 @@ func TestRefusedSentryDSNStopsTheStart(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestPortInUseExitsWithStatus1: a server whose port is already taken
|
||||
// cannot listen, and stops with exit status 1.
|
||||
func TestPortInUseExitsWithStatus1(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
ctx, cancel := context.WithTimeout(t.Context(), childTimeout)
|
||||
defer cancel()
|
||||
|
||||
var lc net.ListenConfig
|
||||
|
||||
taken, err := lc.Listen(ctx, "tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
defer func() { _ = taken.Close() }()
|
||||
|
||||
_, port, err := net.SplitHostPort(taken.Addr().String())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
child, stdout, _ := startServer(ctx, t, t.TempDir(), port)
|
||||
|
||||
err = child.Wait()
|
||||
if child.ProcessState.ExitCode() != 1 {
|
||||
t.Fatalf("server exit = %v, want exit status 1\n%s", err, stdout)
|
||||
}
|
||||
}
|
||||
|
||||
// startServer runs main() in a child process listening on
|
||||
// 127.0.0.1:port, with home as its HOME and working directory and its
|
||||
// data directory in home, so it touches nothing outside home. Its
|
||||
// data directory in home, so it touches nothing outside home. env is
|
||||
// added to its environment, as NAME=value strings. Its
|
||||
// stdout and stderr go to the two buffers returned, which hold all of
|
||||
// it once child.Wait returns. The child is killed when ctx ends, and
|
||||
// killed and reaped when the test ends if nothing waited for it.
|
||||
@@ -147,6 +261,7 @@ func startServer(
|
||||
ctx context.Context,
|
||||
t *testing.T,
|
||||
home, port string,
|
||||
env ...string,
|
||||
) (*exec.Cmd, *bytes.Buffer, *bytes.Buffer) {
|
||||
t.Helper()
|
||||
|
||||
@@ -166,6 +281,7 @@ func startServer(
|
||||
"BIND_ADDRESS=127.0.0.1",
|
||||
"PORT="+port,
|
||||
)
|
||||
child.Env = append(child.Env, env...)
|
||||
child.Stdout = &stdout
|
||||
child.Stderr = &stderr
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ require (
|
||||
github.com/prometheus/client_golang v1.24.1
|
||||
github.com/slok/go-http-metrics v0.13.0
|
||||
github.com/spf13/viper v1.21.0
|
||||
go.uber.org/dig v1.19.0
|
||||
go.uber.org/fx v1.24.0
|
||||
golang.org/x/sys v0.47.0
|
||||
)
|
||||
@@ -38,7 +39,6 @@ require (
|
||||
github.com/spf13/pflag v1.0.10 // indirect
|
||||
github.com/subosito/gotenv v1.6.0 // indirect
|
||||
github.com/zeebo/xxh3 v1.0.2 // indirect
|
||||
go.uber.org/dig v1.19.0 // indirect
|
||||
go.uber.org/multierr v1.10.0 // indirect
|
||||
go.uber.org/zap v1.26.0 // indirect
|
||||
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
||||
|
||||
+75
-24
@@ -5,8 +5,11 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"os/user"
|
||||
|
||||
"sneak.berlin/go/netwatch/internal/config"
|
||||
@@ -18,6 +21,7 @@ import (
|
||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||
"sneak.berlin/go/netwatch/internal/server"
|
||||
|
||||
"go.uber.org/dig"
|
||||
"go.uber.org/fx"
|
||||
"go.uber.org/fx/fxevent"
|
||||
)
|
||||
@@ -29,19 +33,17 @@ const usageExitCode = 2
|
||||
// Main runs netwatch-server with args, its command line after the
|
||||
// program's name, and returns the exit code. With no args, Main runs
|
||||
// the server by calling serve with appname and version, the name and
|
||||
// version the server logs and reports. With "check-cidr CIDR" or
|
||||
// "prepare-data-dir DATA_DIR" it runs that subcommand, which writes its
|
||||
// errors to stderr. Any other args get a one-line usage message on
|
||||
// stderr and exit code 2. main passes os.Stderr and RunServer; the
|
||||
// tests pass a buffer and a function of their own, so that no test
|
||||
// starts the server. When serve is RunServer and the server fails to
|
||||
// start, to listen or to stop, fx's Run exits the process with status 1
|
||||
// and Main never returns.
|
||||
// version the server logs and reports, and returns serve's exit code.
|
||||
// With "check-cidr CIDR" or "prepare-data-dir DATA_DIR" it runs that
|
||||
// subcommand, which writes its errors to stderr. Any other args get a
|
||||
// one-line usage message on stderr and exit code 2. main passes
|
||||
// os.Stderr and RunServer; the tests pass a buffer and a function of
|
||||
// their own, so that no test starts the server.
|
||||
func Main(
|
||||
stderr io.Writer,
|
||||
appname, version string,
|
||||
args []string,
|
||||
serve func(appname, version string),
|
||||
serve func(appname, version string) int,
|
||||
) int {
|
||||
// "netwatch-server check-cidr CIDR" exits 1, with the error, if
|
||||
// this server would refuse CIDR in its TRUSTED_PROXIES.
|
||||
@@ -67,9 +69,7 @@ func Main(
|
||||
return usageExitCode
|
||||
}
|
||||
|
||||
serve(appname, version)
|
||||
|
||||
return 0
|
||||
return serve(appname, version)
|
||||
}
|
||||
|
||||
// checkCIDR returns 0 if the server would take cidr in its
|
||||
@@ -109,19 +109,30 @@ func prepareDataDir(stderr io.Writer, volume, dataDir, username string) int {
|
||||
}
|
||||
|
||||
// RunServer runs the server until it is told to stop, with appname and
|
||||
// version as the name and version it logs and reports. If the server
|
||||
// fails to start, to listen or to stop, fx's Run exits the process
|
||||
// with status 1 itself.
|
||||
func RunServer(appname, version string) {
|
||||
// version as the name and version it logs and reports, and returns the
|
||||
// exit code. If the server fails to start, to listen or to stop, it
|
||||
// returns 1. A failed start logs one line with the error's cause, such
|
||||
// as the setting refused and why, as the server's last line.
|
||||
func RunServer(appname, version string) int {
|
||||
globals.Appname = appname
|
||||
globals.Version = version
|
||||
|
||||
fx.New(
|
||||
// fx logs each step of starting and stopping through the
|
||||
// server's own logger, so off a terminal those lines are
|
||||
// JSON like every other line.
|
||||
var (
|
||||
log *logger.Logger
|
||||
fxLog *fxevent.SlogLogger
|
||||
)
|
||||
|
||||
app := fx.New(
|
||||
// fx logs each step of starting and stopping, errors included,
|
||||
// at debug level through the server's own logger: they show
|
||||
// only with DEBUG set, and off a terminal they are JSON like
|
||||
// every other line.
|
||||
fx.WithLogger(func(log *logger.Logger) fxevent.Logger {
|
||||
return &fxevent.SlogLogger{Logger: log.Get()}
|
||||
fxLog = &fxevent.SlogLogger{Logger: log.Get()}
|
||||
fxLog.UseLogLevel(slog.LevelDebug)
|
||||
fxLog.UseErrorLevel(slog.LevelDebug)
|
||||
|
||||
return fxLog
|
||||
}),
|
||||
fx.Provide(
|
||||
config.New,
|
||||
@@ -133,11 +144,51 @@ func RunServer(appname, version string) {
|
||||
reportbuf.New,
|
||||
server.New,
|
||||
),
|
||||
// First, so that log is set for the error below even when a
|
||||
// setting stops the start.
|
||||
fx.Populate(&log),
|
||||
fx.Invoke(
|
||||
// First, so the name and version are logged even when
|
||||
// a setting stops the start.
|
||||
// Before the server, so that the name and version are
|
||||
// logged even when a setting stops the start.
|
||||
func(log *logger.Logger) { log.Identify() },
|
||||
func(*server.Server) {},
|
||||
),
|
||||
).Run()
|
||||
)
|
||||
|
||||
startCtx, cancelStart := context.WithTimeout(
|
||||
context.Background(), app.StartTimeout())
|
||||
defer cancelStart()
|
||||
|
||||
err := app.Start(startCtx)
|
||||
if err != nil {
|
||||
// fx wraps a constructor's error in its own text; the cause
|
||||
// alone names the setting. log is nil only if fx failed
|
||||
// before it built the logger.
|
||||
cause := dig.RootCause(err)
|
||||
if log == nil {
|
||||
_, _ = fmt.Fprintln(os.Stderr, cause)
|
||||
} else {
|
||||
log.Get().Error("cannot start", "error", cause)
|
||||
}
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
// A signal, or the server asking to stop after a listen error.
|
||||
// fx's Run logs the signal here, so this does too.
|
||||
shutdown := <-app.Wait()
|
||||
fxLog.LogEvent(&fxevent.Stopping{Signal: shutdown.Signal})
|
||||
|
||||
stopCtx, cancelStop := context.WithTimeout(
|
||||
context.Background(), app.StopTimeout())
|
||||
defer cancelStop()
|
||||
|
||||
err = app.Stop(stopCtx)
|
||||
if err != nil {
|
||||
log.Get().Error("cannot stop cleanly", "error", err)
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
return shutdown.ExitCode
|
||||
}
|
||||
@@ -63,7 +63,7 @@ func TestMainRunsPrepareDataDir(t *testing.T) {
|
||||
|
||||
// TestMainRunsServerWithNoArguments: Main given no arguments starts the
|
||||
// server with the name and version it was given, writes nothing and
|
||||
// returns 0.
|
||||
// returns the server's exit code.
|
||||
func TestMainRunsServerWithNoArguments(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -73,11 +73,13 @@ func TestMainRunsServerWithNoArguments(t *testing.T) {
|
||||
)
|
||||
|
||||
code := cli.Main(&stderr, "netwatch", "test", []string{},
|
||||
func(appname, version string) {
|
||||
func(appname, version string) int {
|
||||
gotAppname, gotVersion = appname, version
|
||||
|
||||
return 3
|
||||
})
|
||||
if code != 0 || stderr.Len() != 0 {
|
||||
t.Errorf("exit %d, stderr %q; want exit 0 and no output",
|
||||
if code != 3 || stderr.Len() != 0 {
|
||||
t.Errorf("exit %d, stderr %q; want exit 3 and no output",
|
||||
code, stderr.String())
|
||||
}
|
||||
|
||||
@@ -210,10 +212,12 @@ func currentUsername(t *testing.T) string {
|
||||
|
||||
// failIfServerStarts is what the tests give Main in place of
|
||||
// cli.RunServer: it fails the test instead of starting the server.
|
||||
func failIfServerStarts(t *testing.T) func(appname, version string) {
|
||||
func failIfServerStarts(t *testing.T) func(appname, version string) int {
|
||||
t.Helper()
|
||||
|
||||
return func(string, string) {
|
||||
return func(string, string) int {
|
||||
t.Error("Main started the server")
|
||||
|
||||
return 0
|
||||
}
|
||||
}
|
||||
@@ -31,8 +31,8 @@ HARNESS="netwatch-viewport-harness-$RUN_ID"
|
||||
ARTIFACT_DIR="$ROOT/tmp/viewport"
|
||||
|
||||
# Every container is named and removed here, including the harness itself:
|
||||
# an interrupted `docker run` client does not remove the container it
|
||||
# started, and an unnamed survivor keeps the --internal network in use so
|
||||
# a `docker run` client that is killed outright leaves its container
|
||||
# running, and an unnamed survivor keeps the --internal network in use so
|
||||
# `docker network rm` fails too. This host runs many sessions at once and
|
||||
# neither may be left behind.
|
||||
cleanup() {
|
||||
|
||||
+124
-5
@@ -28,8 +28,8 @@ import {
|
||||
// a host's row that tick or greyOutUI draws into is a plain object, made
|
||||
// the first time a test looks it up and kept in elements under its
|
||||
// selector until the next test starts. As on a page, writing its text
|
||||
// replaces its markup; the status dot greyOutUI looks for in it is not
|
||||
// there. Drawing a sparkline does nothing; it looks for the pixel ratio on
|
||||
// replaces its markup; setting a style keeps it in the element's style
|
||||
// object. Drawing a sparkline does nothing; it looks for the pixel ratio on
|
||||
// window and finds none. The commit the page's footer names, which
|
||||
// vite.config.js fills in when it builds the page, is blank. In each test,
|
||||
// console.error and console.info print nothing and keep what they are
|
||||
@@ -63,7 +63,7 @@ globalThis.document = {
|
||||
querySelector: (selector) =>
|
||||
(elements[selector] ??= {
|
||||
getContext: () => canvasContext,
|
||||
querySelector: () => null,
|
||||
style: {},
|
||||
set textContent(text) {
|
||||
this.innerHTML = text;
|
||||
},
|
||||
@@ -83,6 +83,12 @@ function statusText(state, host) {
|
||||
return elements[`.status-text[data-host="${index}"]`]?.innerHTML;
|
||||
}
|
||||
|
||||
// The color last given to the dot beside host's name.
|
||||
function statusDot(state, host) {
|
||||
const index = state.allHosts.indexOf(host);
|
||||
return elements[`.status-dot[data-host="${index}"]`]?.style.backgroundColor;
|
||||
}
|
||||
|
||||
// What this test wrote to the browser console with console[method].
|
||||
function consoleLines(method) {
|
||||
return console[method].mock.calls.map((call) => call.arguments[0]);
|
||||
@@ -215,8 +221,8 @@ test("a target whose home page redirects is checked at /robots.txt on its own ho
|
||||
"https://s3.eu-west-2.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 me-south-1 (Bahrain)",
|
||||
"https://s3.me-south-1.amazonaws.com/robots.txt",
|
||||
"S3 ap-south-1 (Mumbai)",
|
||||
"https://s3.ap-south-1.amazonaws.com/robots.txt",
|
||||
],
|
||||
[
|
||||
"S3 ap-northeast-1 (Tokyo)",
|
||||
@@ -683,6 +689,37 @@ test("a check that timed out or found its target unreachable has a grey figure a
|
||||
assert.equal(latencyHex(null), "#6b7280");
|
||||
});
|
||||
|
||||
test("a check that answers after 488ms turns the dot beside its target's name orange", async (t) => {
|
||||
const { state, target } = stateWithOneTarget();
|
||||
mockTargets(t, () => 488);
|
||||
const round = tick(state, new AbortController().signal);
|
||||
t.mock.timers.tick(488);
|
||||
assert.notEqual(await settled(round), "still waiting");
|
||||
assert.equal(statusDot(state, target), "#f97316");
|
||||
});
|
||||
|
||||
test("a check that fails after one that answered turns the dot beside its target's name gray", async (t) => {
|
||||
const { state, target } = stateWithOneTarget();
|
||||
target.pushSample(Date.now(), { latency: 8, error: null });
|
||||
mockTargets(t, () => 23, target.url);
|
||||
const round = tick(state, new AbortController().signal);
|
||||
t.mock.timers.tick(23);
|
||||
assert.notEqual(await settled(round), "still waiting");
|
||||
assert.equal(statusDot(state, target), "#6b7280");
|
||||
});
|
||||
|
||||
test("pausing turns the dot beside a target's name gray", async (t) => {
|
||||
const { state, target } = stateWithOneTarget();
|
||||
mockTargets(t, () => 8);
|
||||
const round = tick(state, new AbortController().signal);
|
||||
t.mock.timers.tick(8);
|
||||
assert.notEqual(await settled(round), "still waiting");
|
||||
assert.equal(statusDot(state, target), "#22c55e");
|
||||
state.paused = true;
|
||||
greyOutUI(state);
|
||||
assert.equal(statusDot(state, target), "#6b7280");
|
||||
});
|
||||
|
||||
// buildUI draws the page into #app, here a plain object. Each entry in the
|
||||
// footer's color legend is a colored dot and the latencies it stands for,
|
||||
// with < and > written as < and >. A latency takes the color of the
|
||||
@@ -831,6 +868,88 @@ test("once every WAN target's check has failed, the summary's Reachable slot rea
|
||||
assert.equal(elements["#summary-reachable"].className, "text-red-400");
|
||||
});
|
||||
|
||||
// The network has just come back: every WAN target's last check failed, so
|
||||
// the health box reads OFFLINE. In this round every WAN target but one
|
||||
// answers after 10ms, and the one never answers. Each element of the summary
|
||||
// and the health box is a plain object, as in the tests above; the debug
|
||||
// log's panel is not there.
|
||||
test("at a 30000ms interval, the summary and the health box take in each WAN target's answer while another WAN target's check is still waiting", async (t) => {
|
||||
CONFIG.updateInterval = 30000;
|
||||
t.mock.method(globalThis.document, "getElementById", (id) =>
|
||||
id.startsWith("summary-") || id.startsWith("health-")
|
||||
? (elements[`#${id}`] ??= {})
|
||||
: null,
|
||||
);
|
||||
const state = new AppState([]);
|
||||
for (const host of state.wan) {
|
||||
host.pushSample(Date.now(), { latency: null, error: "timeout" });
|
||||
}
|
||||
const silent = state.wan[0];
|
||||
mockTargets(t, (url) => (url === silent.url ? Infinity : 10));
|
||||
// The third tick, as in the tests above.
|
||||
state.tickCount = 2;
|
||||
|
||||
const round = tick(state, new AbortController().signal);
|
||||
t.mock.timers.tick(10);
|
||||
assert.equal(await settled(round), "still waiting");
|
||||
assert.equal(elements["#health-text"].textContent, "HEALTHY");
|
||||
assert.equal(
|
||||
elements["#summary-reachable"].textContent,
|
||||
`${state.wan.length - 1}/${state.wan.length}`,
|
||||
);
|
||||
assert.equal(elements["#summary-max"].textContent, "10");
|
||||
// Checks counts finished rounds only.
|
||||
assert.equal(elements["#summary-checks"].textContent, 2);
|
||||
|
||||
t.mock.timers.tick(CONFIG.requestTimeout - 10);
|
||||
assert.notEqual(await settled(round), "still waiting");
|
||||
assert.equal(elements["#summary-checks"].textContent, 3);
|
||||
});
|
||||
|
||||
// The user pauses during the first recorded round after the page loads.
|
||||
// Every WAN target's check ends during the pause. Silent never answers, and
|
||||
// the user resumes before its check times out. Every element the page looks
|
||||
// up by id is a plain object, except the debug log's panel, which is not
|
||||
// there.
|
||||
test(`at a 30000ms interval, after the user pauses and resumes during a round that records no WAN target's check, the debug log has no "Health:" line`, async (t) => {
|
||||
CONFIG.updateInterval = 30000;
|
||||
t.mock.method(globalThis.document, "getElementById", (id) =>
|
||||
id === "debug-panel" ? null : (elements[`#${id}`] ??= {}),
|
||||
);
|
||||
t.mock.method(log, "notice");
|
||||
t.mock.method(log, "warning");
|
||||
const state = new AppState([
|
||||
{ name: "Silent", url: "https://silent.test" },
|
||||
]);
|
||||
const silent = state.local[0];
|
||||
mockTargets(t, (url) => (url === silent.url ? Infinity : 10));
|
||||
// The second tick: the first that is recorded.
|
||||
state.tickCount = 1;
|
||||
|
||||
const round = tick(state, new AbortController().signal);
|
||||
// The user pauses, which greys out the summary and the health box, as
|
||||
// togglePause does.
|
||||
state.paused = true;
|
||||
greyOutUI(state);
|
||||
t.mock.timers.tick(10);
|
||||
assert.equal(await settled(round), "still waiting");
|
||||
// The user resumes, which leaves the page as it is.
|
||||
state.paused = false;
|
||||
|
||||
t.mock.timers.tick(CONFIG.requestTimeout - 10);
|
||||
assert.notEqual(await settled(round), "still waiting");
|
||||
assert.equal(state.tickCount, 2);
|
||||
const lines = [...log.notice.mock.calls, ...log.warning.mock.calls].map(
|
||||
(call) => call.arguments[0],
|
||||
);
|
||||
assert.deepEqual(
|
||||
lines.filter((line) => line.startsWith("Health:")),
|
||||
[],
|
||||
);
|
||||
assert.equal(elements["#health-text"].textContent, "---");
|
||||
assert.equal(elements["#summary-reachable"].textContent, "--/--");
|
||||
});
|
||||
|
||||
// In the next two tests each of the header's clocks is a plain object, made
|
||||
// the first time updateClocks looks it up and kept in elements under "#" and
|
||||
// its id.
|
||||
|
||||
+32
-24
@@ -30,8 +30,8 @@ export const CONFIG = {
|
||||
return [0, 1, 2, 3, 4, 5].map((i) => Math.round((d * i) / 5));
|
||||
},
|
||||
canvasHeight: 96,
|
||||
// A latency figure and its sparkline take the color of the first entry
|
||||
// whose limit, in ms, the latency is below.
|
||||
// A latency figure, its row's dot and its sparkline take the color of
|
||||
// the first entry whose limit, in ms, the latency is below.
|
||||
latencyColors: [
|
||||
{ below: 50, hex: "#22c55e", className: "text-green-500" },
|
||||
{ below: 100, hex: "#84cc16", className: "text-lime-500" },
|
||||
@@ -99,8 +99,8 @@ const WAN_HOSTS = [
|
||||
method: "HEAD",
|
||||
},
|
||||
{
|
||||
name: "S3 me-south-1 (Bahrain)",
|
||||
url: "https://s3.me-south-1.amazonaws.com/robots.txt",
|
||||
name: "S3 ap-south-1 (Mumbai)",
|
||||
url: "https://s3.ap-south-1.amazonaws.com/robots.txt",
|
||||
method: "HEAD",
|
||||
},
|
||||
{
|
||||
@@ -819,7 +819,7 @@ export function hostRowHTML(host, index, showPin = true) {
|
||||
${pinBtn}
|
||||
<div class="w-[420px] flex-shrink-0 grid grid-cols-[minmax(0,1fr)_auto] items-center">
|
||||
<div class="flex items-center gap-2 min-w-[200px]">
|
||||
<div class="w-3 h-3 rounded-full flex-shrink-0 bg-[#6b7280]"></div>
|
||||
<div class="status-dot w-3 h-3 rounded-full flex-shrink-0 bg-[#6b7280]" data-host="${index}"></div>
|
||||
<span class="font-medium text-white truncate">${escapeHTML(host.name)}</span>
|
||||
</div>
|
||||
<div class="latency-value text-4xl font-bold tabular-nums text-right mt-3" data-host="${index}">
|
||||
@@ -924,7 +924,7 @@ export function buildUI(state) {
|
||||
</div>
|
||||
|
||||
<footer class="mt-8 text-center text-gray-600 text-xs">
|
||||
<p>Latency measured via GET requests | CORS restrictions may affect some measurements</p>
|
||||
<p>Latency: the time until a target's response headers arrive</p>
|
||||
<p class="mt-2">
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-green-500 mr-1 align-middle"></span><50ms
|
||||
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span><100ms
|
||||
@@ -982,8 +982,10 @@ function updateHostRow(host, index) {
|
||||
const canvas = document.querySelector(
|
||||
`.sparkline-canvas[data-host="${index}"]`,
|
||||
);
|
||||
if (!latencyEl || !statusEl || !canvas) return;
|
||||
const dot = document.querySelector(`.status-dot[data-host="${index}"]`);
|
||||
if (!latencyEl || !statusEl || !canvas || !dot) return;
|
||||
|
||||
dot.style.backgroundColor = latencyHex(host.lastLatency);
|
||||
if (host.lastLatency !== null) {
|
||||
const cls = latencyClass(host.lastLatency, host.status);
|
||||
latencyEl.innerHTML = `<span class="${cls}">${host.lastLatency}<span class="text-lg">ms</span></span>`;
|
||||
@@ -1126,11 +1128,6 @@ function updateHealthBox(state) {
|
||||
el.textContent = labels[health];
|
||||
el.className = textColors[health];
|
||||
box.className = `mt-4 p-3 rounded-lg border font-mono text-sm text-center ${boxColors[health]}`;
|
||||
|
||||
if (health === "offline" || health === "degraded")
|
||||
log.warning(`Health: ${labels[health]}`);
|
||||
else if (health === "slow") log.warning(`Health: ${labels[health]}`);
|
||||
else if (health === "healthy") log.notice(`Health: ${labels[health]}`);
|
||||
}
|
||||
|
||||
// Writes the local time, with its time zone's name, and the UTC time into
|
||||
@@ -1301,12 +1298,13 @@ export async function tick(state, signal, onOffline) {
|
||||
|
||||
log.debug(`Tick #${state.tickCount + 1} started`);
|
||||
|
||||
// Each host's row shows its result as soon as its check ends. The
|
||||
// result is discarded if by then the user has paused or the next round
|
||||
// has given up this one's checks, and in the first tick (tickCount is
|
||||
// still 0), which is discarded as a whole below. The row is looked up
|
||||
// when the check ends, as a pin click may have re-sorted the rows since
|
||||
// the round started.
|
||||
// Each host's row shows its result as soon as its check ends, and a WAN
|
||||
// host's result goes into the summary and the health box at the same
|
||||
// time. The result is discarded if by then the user has paused or the
|
||||
// next round has given up this one's checks, and in the first tick
|
||||
// (tickCount is still 0), which is discarded as a whole below. The row
|
||||
// is looked up when the check ends, as a pin click may have re-sorted
|
||||
// the rows since the round started.
|
||||
await Promise.all(
|
||||
state.allHosts.map(async (host) => {
|
||||
const r = await measureLatency(host.url, signal, host.method);
|
||||
@@ -1316,6 +1314,10 @@ export async function tick(state, signal, onOffline) {
|
||||
logCheck(host, r);
|
||||
host.pushSample(ts, r);
|
||||
updateHostRow(host, state.allHosts.indexOf(host));
|
||||
if (state.wan.includes(host)) {
|
||||
updateSummary(state);
|
||||
updateHealthBox(state);
|
||||
}
|
||||
log.debug(`${host.name}: ${r.error ? r.error : r.latency + "ms"}`);
|
||||
}),
|
||||
);
|
||||
@@ -1342,9 +1344,19 @@ export async function tick(state, signal, onOffline) {
|
||||
sortAndRebuildWAN(state);
|
||||
}
|
||||
|
||||
// Redraw the summary and the health box: a pause greys them out too, and
|
||||
// Checks now counts this round.
|
||||
updateSummary(state);
|
||||
updateHealthBox(state);
|
||||
|
||||
// Like the health box, the log gives no health until a WAN host's check
|
||||
// is recorded: a pause may have discarded every WAN check so far.
|
||||
if (state.wan.some((h) => h.status !== "pending")) {
|
||||
const health = state.healthStatus();
|
||||
if (health === "healthy") log.notice("Health: HEALTHY");
|
||||
else log.warning(`Health: ${health.toUpperCase()}`);
|
||||
}
|
||||
|
||||
const reachable = state.allHosts.filter(
|
||||
(h) => h.status === "online",
|
||||
).length;
|
||||
@@ -1420,12 +1432,8 @@ export function greyOutUI(state) {
|
||||
statusEl.textContent = "paused";
|
||||
statusEl.className = `${STATUS_TEXT_CLASS} text-gray-500`;
|
||||
}
|
||||
// Grey out the status dot
|
||||
const row = document.querySelector(`.host-row[data-index="${i}"]`);
|
||||
if (row) {
|
||||
const dot = row.querySelector(".rounded-full");
|
||||
if (dot) dot.style.backgroundColor = "#6b7280";
|
||||
}
|
||||
const dot = document.querySelector(`.status-dot[data-host="${i}"]`);
|
||||
if (dot) dot.style.backgroundColor = latencyHex(null);
|
||||
});
|
||||
|
||||
// Grey out summary current stats
|
||||
|
||||
Reference in new issue
Block a user