48 Commits
Author SHA1 Message Date
sneak 11295f911c Bootstrap keeps yarn at its pinned version and node at its pinned major (closes #154)
check / check (push) Canceled after 0s
script/bootstrap kept any yarn on PATH, so YARN_VERSION applied
only where none was installed. It now keeps the yarn on PATH only when
yarn --version is exactly YARN_VERSION, and otherwise installs that
version. The node on PATH is used only when it has the pinned major
version, as REPO_POLICIES.md asks, and is at least NODE_MIN_VERSION.
After installing either, bootstrap looks the tool up through PATH
again, stops, naming the path, if that one fails the same check, and
prints the version it uses, as for Go. Every other host script that
runs yarn now goes through the new script/yarn, which uses the first
node on PATH, ~/.local/bin first, when it passes the same test, and
otherwise the pinned node in nvm.

Model: opus-5-5
2026-10-19 05:18:28 +00:00
clawbot 0ea169ec46 Entrypoint steps are functions called from main (closes #163)
check / check (push) Canceled after 0s
bin/entrypoint.sh was top-level code with one helper; the shell section
of the code style guide asks for every step in a function and a main
called at the end. The PORT and TRUSTED_PROXIES checks, the data
directory, starting each process, the wait loop, stopping both and the
exit are now functions, which main calls in order, setting the stop
trap before either process starts. It stays POSIX sh with no set -e.
Messages and exit codes are unchanged; the 8081 message is passed to
echo as two arguments so its line fits in 80 columns.

Model: opus-5-5
2026-10-19 07:17:32 +02:00
clawbot 9285591563 Health check answers 503 while report files cannot be written (closes #142)
check / check (push) Canceled after 0s
The health check answered 200 "status":"ok" whatever happened, so a
container whose report files could not be written stayed healthy while
every report was refused with 500. The report buffer now says whether
its last write of a report file failed. While it has, the health
check's status is "error" and the handler answers 503 with the same
fields, which fails the image's HEALTHCHECK. At start no write has
failed, and the first write that succeeds brings back 200. A test
removes DATA_DIR to make a write fail and checks the 503, then the 200
once a write succeeds again.

Model: opus-5-5
2026-10-19 06:55:51 +02:00
clawbot 9677c73e0a Report file writes run one at a time (closes #150)
check / check (push) Canceled after 0s
Two writes that failed at the same time put their reports back in the
wrong order when the older one failed first. A mutex is now held
across each write, from taking the waiting reports out of the buffer
to putting them back if the write fails, so the next write takes them
in order. The write Append starts at the 10 MiB threshold waits for
one under way, and Append starts no other while it waits. Files
written since the start are now deleted to make room in the order they
were written. A test that needed two writes at once is removed; new
tests cover the one waiting write and the order files are deleted in.

Model: opus-5-5
2026-10-19 06:02:11 +02:00
clawbot f5aa81f024 DEBUG turns debug logging on for any non-empty value (closes #159)
check / check (push) Canceled after 0s
DEBUG was read with strconv.ParseBool, so a value such as "yes" stopped
the start. It now follows the code style guide: unset or empty is off,
and any other value, "false" and "0" included, turns debug logging on.
No value of DEBUG stops the start. The test that "yes" fails is replaced
by one covering unset, empty, "false", "0" and "yes". README.md no
longer gives false as the default, and says which values turn it on.

Model: opus-5-5
2026-10-19 05:48:20 +02:00
clawbot 4e3943f02f Check builds keep Go's modules and caches between runs (refs #118)
check / check (push) Canceled after 0s
On sneak's rulings in sneak/prompts#113 and
sneak/prompts#130. REPO_POLICIES.md and
script/test, script/lint, script/docker and script/cibuild are the
sneak/prompts copies at 0653579: the scripts pass a new CACHEBUST value
instead of --no-cache, and the lint and test builds write no image. In
each Dockerfile stage the system packages and the Go module download
come above ARG CACHEBUST; every step that lints, tests or compiles, and
the node stages' yarn install, comes after it. Go's build cache and
golangci-lint's are cache mounts, and both go test runs carry -count=1.
README.md and TODO.md say the same.

Model: opus-5-5
2026-10-19 04:26:25 +02:00
clawbot 648c225919 Discarded-check unit tests check nothing was recorded (closes #151)
check / check (push) Canceled after 0s
The three unit tests of a check whose result is discarded (its round
given up, the user paused, or the first round) checked only that the
row showed nothing. Each now also checks that the host's history is
still empty, its status still pending, and nothing was written to the
console, so a change that recorded the result and skipped only the row
fails them. The console helper moves above its first use.

Model: opus-5-5
2026-10-19 03:44:13 +02:00
clawbot be6dc93eaa Unusable DATA_DIR error names DATA_DIR; report file number is atomic.Int64 (closes #148, closes #161)
check / check (push) Canceled after 0s
When the backend runs on its own, a DATA_DIR that could not be created
or read stopped the start with an error that named the path but not
DATA_DIR, and one where no file could be created let it start, to fail
at the first write of a report file. The start now creates a file in
DATA_DIR and removes it, and all three cases fail with
DATA_DIR "<value>": <reason>, the form config.New uses for the other
settings. Tests start the buffer with DATA_DIR below a regular file and
with DATA_DIR set to /proc.

The report file sequence counter is atomic.Int64 instead of
atomic.Uint64, as the Go style guide asks for int types. File names are
unchanged.

Model: opus-5-5
2026-10-19 03:15:35 +02:00
clawbot 6df743f7fa Page is sent with Cache-Control: no-cache (closes #141)
check / check (push) Canceled after 0s
nginx sent index.html with no Cache-Control, so a browser could keep
it after a deploy and ask for hashed assets the new build no longer
has, and show an empty page. location / now adds Cache-Control:
no-cache, which covers its try_files fallback to /index.html, and
includes security-headers.conf again, since its own add_header drops
the server's. /assets/ and the paths passed to the backend are
unchanged.

script/frontend-viewport-test fails unless / and a path that falls
back to the page carry Cache-Control: no-cache and
Content-Security-Policy, which comes from the same file as the other
security headers.

Model: opus-5-5
2026-10-19 03:10:39 +02:00
clawbot 984047e6ca README states what refused reports, one sender and nginx's 413 do (closes #145, closes #146, closes #147)
check / check (push) Canceled after 0s
The README said the page keeps refused samples and sends them later.
Those in the 100-round history are sent only while it holds them: at
the 1-second interval one refused report loses the oldest 20 seconds of
each host's samples, and a pause longer than the history loses every
sample between the last delivered report and the pause. Samples an
interval change cleared are kept apart, the last 100 per host, until
delivered. Report limits now says one address at the defaults can fill
the 1 GiB cap in about 17 minutes. In the image, nginx answers a body
over 1 MiB itself, with 413 and an HTML page; the backend's own 413
sentences now apply to it run on its own.

Model: opus-5-5
2026-10-19 02:55:29 +02:00
clawbot 007190b541 Clock errors no longer stop probing; legend reads >=500ms (closes #149, closes #152)
check / check (push) Canceled after 0s
The header's clocks start last in init, after the rounds, and an error
formatting them, as in a browser that rejects the "shortGeneric" time
zone name, leaves both clocks blank instead of stopping the page before
its first round. updateClocks moved out of init so a unit test can call
it.

The footer legend's red entry reads >=500ms, as README.md and
CONFIG.latencyColors have it. buildUI is exported so a unit test can draw
the page and check the legend against every entry of
CONFIG.latencyColors.

Model: opus-5-5
2026-10-19 01:34:45 +02:00
clawbot fd32f348b7 prepare-data-dir leaves files with a second name to their owner (closes #143)
check / check (push) Canceled after 0s
A hard link in /data shares its owner with a file that can be anywhere
on the same filesystem, so chowning it gave that file to the netwatch
user. A file with more than one name now keeps its owner and its path
is logged.

Owners are now set on the opened directory, file or link after
checking it, as chmodDir sets modes, so a hard link swapped in
afterwards is caught. Every other file is opened without following it;
a symbolic link is given to the netwatch user itself, and devices,
named pipes and sockets keep their owner. This is built only for
Linux: prepare-data-dir runs only in the Linux image, and macOS has
neither O_PATH nor AT_EMPTY_PATH; elsewhere it returns an error. A
failed owner change names the file.

Model: opus-5-5
2026-10-19 01:11:41 +02:00
clawbot 2532da55e9 Client address is the right-most untrusted X-Forwarded-For entry (closes #144)
check / check (push) Canceled after 0s
From a peer in TRUSTED_PROXIES, clientIP took the left-most
X-Forwarded-For entry, which the client writes itself behind a proxy
that appends to the header, so a client could choose the address the
rate limit counts. It now walks the entries from the right past trusted
proxies and takes the first that is not one. An entry with a port is
read as its address; with only trusted entries, or one that is not an
address at all, it uses the peer. Every X-Forwarded-For header line is
read, and X-Real-IP only when there is no X-Forwarded-For entry.

Model: opus-5-5
2026-10-19 00:28:18 +02:00
clawbot e43a710114 Summary reads --/-- after an interval change, not 0/26 in red (closes #140)
check / check (push) Canceled after 0s
The interval menu resets every host to pending and updates the summary,
which counted the WAN hosts with a latency recorded, none, and showed
0/26 in red until the second round after the change ended. While every
WAN host still waits for its first check, the Reachable slot now reads
--/-- in white, as at page load. updateSummary is exported for a unit
test that resets the hosts as the interval menu does; a second test
checks that the slot still reads 0/26 in red once every check failed.

Model: opus-5-5
2026-10-18 23:00:35 +02:00
clawbot 2861be1c4d Stopping the server waits for report file writes under way (closes #122)
check / check (push) Canceled after 0s
The report buffer's stop now waits, through a sync.WaitGroup, for
flushLoop and for the write Append starts at 10 MiB before its final
flush. It used to return at once, so the process exiting cut such a
write off and its reports were lost.

A write under way that fails puts its reports back in the buffer, so
the final flush writes them; the stop fails only if that flush fails
too.

Tests hold the periodic write and the one at 10 MiB open in a synctest
bubble and find the stop waiting until each is released; a third makes
the held write fail and finds its report written by the final flush.

Model: opus-5-5
2026-10-14 14:47:01 +02:00
clawbot 580d12a934 make test no longer runs the frontend's production build (refs #118)
check / check (push) Canceled after 0s
yarn build moves out of the stage that runs the frontend's unit tests,
now named frontend-test, into a frontend-build stage of its own. It
starts from frontend-test for its yarn packages and source, installs
the git that vite.config.js reads the commit with, and copies from the
lint and test phases as builder does, so it cannot run unless they
passed. The runtime stage copies dist/ from it. The Go compile, most of
make test's time, is unchanged and waits on the ruling asked for in
sneak/prompts#113; no build cache is added.

Model: opus-5-5
2026-10-14 14:30:03 +02:00
clawbot 12f3dd5b07 Debug log adds each line instead of drawing the whole panel (closes #131)
check / check (push) Canceled after 0s
With the debug log open, every line written drew the panel again with
up to 1000 lines, and a check that ends while the page draws is timed
only when the drawing is done, so latencies measured with the panel open
read too high. Each line is now appended to the open panel and its first
line dropped once it holds more than maxLogEntries; the whole panel is
drawn only when it is opened. Scrolling to the newest line reads the
panel's height, which lays out all its lines, so it runs once a frame,
with at most one frame request waiting.

Model: opus-5-5
2026-10-14 14:13:03 +02:00
clawbot 2a47fb9a8e Unset TRUSTED_PROXIES trusts RFC 1918, empty trusts none (closes #129)
check / check (push) Canceled after 0s
The backend's own TRUSTED_PROXIES now follows the ruling for every app:
unset, it trusts 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16, and no
longer loopback; a list replaces that; set and empty, it trusts no
proxy.

viper reads an empty environment variable as unset, so an empty value
got the default. config.New now reads this one setting with
os.LookupEnv first. viper's AllowEmptyEnv would apply to every setting,
so an empty PORT or DEBUG would stop the start instead of counting as
unset, as README.md says it does.

bin/entrypoint.sh still starts the backend with
TRUSTED_PROXIES=127.0.0.1/32.

Model: opus-5-5
2026-10-14 14:01:37 +02:00
clawbot cff7b07bbc Reports refused while report files cannot be written (closes #123)
check / check (push) Canceled after 0s
A failed write of a report file used to drop its reports, and
POST /api/v1/reports kept answering 200 for as long as writes failed.
Now a failed write puts its reports back in the buffer, still counted
against DATA_DIR_MAX_BYTES, and Append refuses every report, which the
handler answers 500, until a write succeeds. The minute flush tries
the held reports again, so storing resumes with no restart once
writing works, and the buffer takes nothing new while writes fail.
Tests also cover a report appended while a write that fails runs, and
the cap once a failed write's reports are written.

Model: opus-5-5
2026-10-14 13:47:14 +02:00
clawbot e628dfe6cf A handler's panic is logged and counted as 500 (closes #125)
check / check (push) Canceled after 0s
Recoverer comes first, so it wrote the 500 only after the panic had
passed the request log and the metrics, which both recorded 200.
Write500OnPanic, placed after the request log and after the metrics,
writes the 500 as the panic passes and panics again, so Sentry and
Recoverer still get it. Recoverer now writes its 500 only when no
status was written, so the server logs no superfluous WriteHeader call.
The test also requires Recoverer's log line for the handler's panic, so
it fails if Write500OnPanic stops passing the panic on.

Model: opus-5-5
2026-10-14 13:30:11 +02:00
clawbot cbcde098dc Unset TRUSTED_PROXIES trusts the RFC 1918 ranges (closes #119)
check / check (push) Canceled after 0s
With TRUSTED_PROXIES unset, bin/entrypoint.sh gives nginx one
set_real_ip_from line for each of 10.0.0.0/8, 172.16.0.0/12 and
192.168.0.0/16, as sneak ruled on 2026-09-22 for every app. A list
replaces them, and set and empty trusts no proxy; each entry is checked
as before. The backend inside the container still trusts only
127.0.0.1/32. README.md, configs/nginx.conf and the comment in
script/frontend-viewport-test say the new default.

Model: opus-5-5
2026-10-14 13:01:34 +02:00
clawbot 73b39e485b Reports are taken only as application/json (closes #126)
check / check (push) Canceled after 0s
A page on another origin can POST text/plain or a form without a CORS
preflight, so its visitors' browsers could store reports although the
server sends no CORS headers. decodeJSON now refuses a request whose
Content-Type is not application/json (parameters such as charset
allowed), and the report handler answers it 415 with the usual error
body, storing nothing. A JSON POST waits for the preflight, which only
an origin in CORS_ALLOWED_ORIGINS passes; the page already sends
application/json.

Model: opus-5-5
2026-10-14 12:43:25 +02:00
clawbot 03bc282871 Interval change keeps unreported samples for the reporter (closes #124)
check / check (push) Canceled after 0s
The interval menu cleared every host's history, and the reporter builds
each report from that history, so the samples recorded since the last
delivered report were never sent. The menu now resets each host with
HostState.reset, which moves the history's entries to clearedHistory,
keeping the last 100 it has cleared, and buildReport reads both. A host
still pending after the reset is sent with the status of its last check.
The per-host marks are unchanged, so a sample a delivered report had is
not sent again, and the page's history still starts over. Reporter is
exported for the new unit tests.

Model: opus-5-5
2026-10-14 12:23:13 +02:00
clawbot 406e073aee Move to the layout REPO_POLICIES.md gives (closes #30)
check / check (push) Canceled after 0s
The Go module moves from backend/ to the root (go.mod, go.sum,
.golangci.yml, cmd/, internal/), the page's source to web/, and nginx's
two files to configs/. backend/ is gone: its LICENSE and .dockerignore
are deleted, and its .gitignore entries, README and scripts are folded
into the root ones. script/build builds the server binary after the
frontend; script/run and script/clean move up from backend/script/;
make test, lint, fmt and fmt-check do what the backend's did. Vite's
root is web/ and it still builds into dist/. go.mod ignores
node_modules/, whose npm packages can ship .go files. script/bootstrap
no longer installs gcc, which only the host Go tests needed.

Model: opus-5-5
2026-10-14 12:01:32 +02:00
clawbot b1eefe0c29 Re-vendor the shared files from sneak/prompts at dd4027b (closes #113)
check / check (push) Canceled after 0s
The shared files are the sneak/prompts copies at dd4027b, plus this
repository's own entries. make lint and make test each build one
Dockerfile phase without the cache, both covering the frontend; the
builder stage waits on both and takes its version from git describe
unless VERSION is given. The test phase keeps Go's module and build
caches in memory, out of the image make test tags. golangci-lint moves
to v2.14.0 with the new .golangci.yml; one test spells X-Request-ID as
canonicalheader asks. prettier formats only JavaScript, CSS, HTML and
Markdown, so .golangci.yml stays as fetched. script/fmt and
script/fmt-check put ~/.local/bin on PATH. script/bootstrap keeps a Go
only if it is exactly GO_VERSION, and re-checks the go on PATH after
installing.

Model: opus-5-5
2026-10-07 13:41:50 +02:00
clawbot dcdee6bfab Hetzner checks answer again; failed checks go to the console (closes #114)
check / check (push) Canceled after 0s
The Hetzner speed-test servers close the connection without an answer
when the URL has a query string, and every check added ?_cb= and the
time, so all six showed unreachable. Checks now fetch each target's URL
as written; cache: "no-store" still keeps the browser's cache out of the
measurement.

Each recorded check that fails writes one console.error line, and the
same line to the debug log: the target's name and URL, the time, what
failed and how long the request took. A target that answers after
failed checks writes one console.info line. Checks the page does not
record write nothing, so the debug log no longer lists failures in the
first round or the recovery probe.

Model: opus-5-5
2026-10-07 12:24:51 +02:00
clawbot 186f932eb8 Footer no longer says IPv4 only (closes #111)
check / check (push) Successful in 2m48s
Each check is a fetch: the browser picks IPv4 or IPv6 for each WAN
host, and the local targets are IPv4 addresses, so "IPv4 only" was
wrong for the page as a whole. The footer's first line drops it and
its separator; the rest of the footer is unchanged. TODO.md moves this
to Completed Steps and the layout decision from Future Steps into Next
Step.

Model: opus-5-5
2026-10-04 07:38:06 +02:00
clawbot 64e142c17f README, TODO and the viewport README say what the tree does (closes #24)
check / check (push) Successful in 2m57s
README.md: Getting Started leads with make targets; a Backend section
gives netwatch-server's routes and how the image builds and runs it;
the checks are GET requests; the WAN host list, health states, summary
figures, sorting and missing features match src/main.js; the TODO
section points to TODO.md, which holds the one to-do list.

backend/README.md: its TODO section points to TODO.md too, whose
Future Steps take its three open items.

TODO.md: Workflow branches from next and opens the PR against next;
Status, Next Step and Future Steps describe the open work, linked to
its issue where one exists.

test/viewport/README.md: the unit tests run on Node's test runner,
not vitest.

Model: opus-5-5
2026-10-04 07:03:05 +02:00
clawbot 161f955ae2 Latency statistics written once, thresholds read from CONFIG (closes #102)
check / check (push) Successful in 2m50s
A target's min, max, median and average latency now come from one list
of its answers, through latencyStats(), which the summary's figures use
too, so the median is written once. latencyHex() and latencyClass() read
one table of color limits in CONFIG. The health thresholds, the debug
log's length, the gateway check's timeout, the recovery probe's number
of hosts and interval, how often the rows are sorted and the delay
before the first sparkline resize are CONFIG entries, read where the
numbers were. HostState's minLatency(), maxLatency(), averageLatency()
and medianLatency() are gone; the statistics test reads historyStats().
Nothing the page does or shows changes. A unit test now covers the
summary's figures.

Model: opus-5-5
2026-10-04 06:51:24 +02:00
clawbot d40e67d4ab Rate limit password attempts on /metrics (closes #104)
check / check (push) Successful in 3m32s
Each client address may make 60 requests to /metrics a minute,
through the same httprate middleware and TRUSTED_PROXIES
resolution the report route uses, with an allowance of its own.
The limit runs before the basic auth, so past it the answer is 429
and the password is not checked. backend/README.md says so; a test
uses up one client's allowance on wrong passwords, gets 429 with
the right one, and checks that another client behind the same
nginx still gets in.

Model: opus-5-5
2026-10-04 06:19:06 +02:00
clawbot dc2d240725 Report handler panics to Sentry when SENTRY_DSN is set (closes #95)
check / check (push) Successful in 3m6s
With SENTRY_DSN set, the server initialises sentry-go with the release
netwatch-server-<version>, adds the sentryhttp middleware with Repanic
as the last router-wide middleware, after the timeout, and flushes
Sentry for 2 seconds on shutdown. A DSN Sentry refuses stops the start
with an error naming SENTRY_DSN. With it empty, nothing is set up.

The metrics middleware stays on the matched routes only, so it runs
inside the Sentry middleware rather than before it.

Model: opus-5-5
2026-10-04 05:57:02 +02:00
clawbot 00c9f8d7d9 Target names, URLs and log lines reach the page as text (closes #29)
check / check (push) Successful in 3m8s
A host row escapes the name and URL it writes into its markup with a new
escapeHTML function, and the debug log builds each line as an element
whose text is set, so neither is read as HTML once targets can be
configured. A unit test builds the row of a target whose name and URL
hold < > " & and ' and checks each comes out escaped; hostRowHTML is
exported for it.

README.md stops calling CONFIG frozen: the interval menu sets
updateInterval, and the timeouts, history span and axis ticks are
computed from it. AppState declares _recoveryProbeId and
_recoveryProbeChecks, the sparkline axis functions drop the parameters
they never used, and HostState's history comment names both entry
shapes.

Model: opus-5-5
2026-10-04 05:35:36 +02:00
clawbot dc11beb6fe Add make add-dependency and make tidy (closes #45)
check / check (push) Successful in 3m29s
No entrypoint could change yarn.lock or go.mod: script/bootstrap
installs with --frozen-lockfile, so adding a package meant running
yarn by hand. make add-dependency PACKAGE=<name>@<version> shims to
the new script/add-dependency: yarn add --dev, then yarn install
--frozen-lockfile. make tidy shims to the new script/tidy, go mod
tidy in backend/; a Go module is added by importing it, or moved by
editing its require line, then make tidy. script/bootstrap is
unchanged.

Model: opus-5-5
2026-10-04 05:17:33 +02:00
clawbot 81d4153e78 Serve Prometheus metrics at /metrics behind basic auth (closes #94)
check / check (push) Successful in 2m53s
With METRICS_USERNAME and METRICS_PASSWORD both set, the backend
records request metrics through go-http-metrics in a registry of its
own, with Go's runtime and process metrics, and serves them at
GET /metrics behind basic auth; nginx passes /metrics to it. With
neither set there is no such route; one alone, or a METRICS_USERNAME
containing ":", stops the start with an error naming the setting.

Only requests that reach the health check or POST /api/v1/reports are
recorded, as the labels are path and method, which clients could
otherwise make up without end; POST /api/v1/reports is registered by
its full path for that.

Deviation: go get and go mod tidy ran directly; no entrypoint added a Go
dependency yet (issue #45).

Model: opus-5-5
2026-10-04 04:58:47 +02:00
clawbot d412815953 Shim make dev and make build, format backend/ markdown (closes #28)
check / check (push) Successful in 4m11s
make dev ran yarn dev inline and there was no make build. Both are now
shims, over script/dev and the new script/build. .prettierignore stops
leaving out backend/, so backend/README.md is formatted and checked;
backend/.golangci.yml is left out by name instead: it is the org
standard file whose sha256 backend/script/lint checks, and prettier
would reindent it. .claude/ stays in .prettierignore, as git does not
ignore it. script/install-precommit and the date on bootstrap's pins go
back to the org model; bootstrap, fmt and fmt-check keep what the Go
backend and eslint need, each with a comment saying so.

Model: opus-5-5
2026-10-04 04:14:59 +02:00
clawbot 9b548da90d Move tailwind off the deprecated module.register() (closes #32)
check / check (push) Successful in 2m40s
Frontend builds on Node 26 or newer, such as make test on a host with
Node 26, printed Node's DEP0205 warning; the build in Dockerfile runs on
Node 22, which never printed it. The trace names @tailwindcss/node,
which @tailwindcss/vite brings in at its own exact version; tailwind
4.3.1 calls module.registerHooks() where Node has it. @tailwindcss/vite
and tailwindcss move to 4.3.3, inside the ranges package.json already
allows, so only yarn.lock changes, every entry still with an integrity
hash. tailwindcss moves too so that one tailwind version is installed.
Deviation: yarn.lock was changed by a raw yarn upgrade; no entrypoint adds a dependency yet (issue #45).

Model: opus-5-5
2026-10-04 03:51:55 +02:00
clawbot 3b1262718d Frontend unit tests cover durations, colours, statistics and health (closes #21)
check / check (push) Successful in 3m38s
New table-driven tests in test/unit/main.test.js: humanDuration, the
figure and sparkline colours either side of each latency boundary, a
target's min, max, average and median over an empty history, an
all-unreachable one, a mixed one, one of only answers and one whose
answers have different numbers of digits, and the four health states
either side of their thresholds, with targets found unreachable counted
as timed out. src/main.js exports the four names they need.

package.json gains a test script, which script/frontend-test runs with
the dot reporter and, if a test fails, again with the spec reporter
before failing. NODE_OPTIONS picks the reporter, since yarn appends its
arguments after the test files.

Model: opus-5-5
2026-10-04 03:25:06 +02:00
clawbot 82bd142429 Log fx through slog, snake_case health check keys (closes #27)
check / check (push) Successful in 4m40s
fx wrote its own steps of starting and stopping as plain text to
stderr. It now logs them with its slog event logger through the
backend's logger, so off a terminal every line the backend's own
logger and fx write is JSON. A malformed config file now makes
config.New return the error instead of panicking. A test runs the
server as a child process and checks that all it writes is JSON, on
a normal start and stop and with such a config file. The backend
logs its name, version and architecture once at start.

The health check's uptime keys are now uptime_seconds and
uptime_human; its type and method take the names
GO_HTTP_SERVER_CONVENTIONS.md gives.
Rules suppressed: revive and tagliatelle on HealthcheckResponse, whose name and keys come from the conventions; gosec where the test starts its own binary.

Model: opus-5-5
2026-10-04 03:11:43 +02:00
clawbot 924527b744 Lint the frontend with eslint in its own Docker stage (closes #47)
check / check (push) Successful in 3m48s
script/lint ran prettier --check, the same check script/fmt-check
runs, so the JavaScript had no linter. eslint now runs with its
recommended rules, set in eslint.config.js, in a new frontend-lint
stage of Dockerfile built from the pinned node image and the lockfile.
The frontend stage copies a file from it, as the builder stage does
from the Go lint stage, so the image cannot build unless eslint passed.
script/lint builds both lint stages with --no-cache and runs no linter
on the host; script/fmt-check keeps prettier on the host, and
script/frontend-check drops its lint step. The viewport harness fixes
the two kinds of finding eslint made. bootstrap wants node 22.13.0, as
eslint 10 does. Also covers item 2 of
#28.

Model: opus-5-5
2026-10-04 01:58:07 +02:00
clawbot 2f0489e3a4 Tap-target check expects a pin button per WAN host row (closes #46)
check / check (push) Successful in 3m50s
The viewport harness's tap-target check required at least 10 visible
pin buttons while 26 render, so pin buttons missing from up to 16 rows
went unnoticed. It now expects one per WAN host row. The host row count
the harness gathers, which the app-rendered check also reads, counts
only the WAN host rows, since the local host rows have no pin button.
Each control's minimum is now worked out from the gathered facts.

TODO.md's harness entry no longer says every check guards itself: the
overflow, viewport-edge and clipped-text checks rely on app-rendered.

Model: opus-5-5
2026-10-04 01:53:02 +02:00
clawbot 91856fa170 Each target's row shows its result as soon as its check ends (closes #91)
check / check (push) Successful in 3m43s
tick drew no row until the round's slowest check ended, up to 24
seconds at a 30-second interval since checks time out at 80% of it.
Each check now pushes its sample and redraws its row as it ends. When
the last check ends, every row is redrawn, so none still reads "paused"
after a pause and resume, and sorting, the summary, the health box and
offline detection run once. A check that ends while paused or after its
round is given up draws nothing, and the first round is still discarded
as a whole. The row is looked up when the check ends, as a pin click can
re-sort the rows mid-round.

Unit tests run tick on the mocked clock against a stand-in page.

Model: opus-5-5
2026-10-04 01:41:04 +02:00
clawbot 39ee6ca839 Entrypoint acts as root on nothing outside /data (closes #80)
check / check (push) Successful in 1m58s
`bin/entrypoint.sh` now runs `netwatch-server prepare-data-dir`, which
refuses a `DATA_DIR` that is not `/data` or a path below it written in
full, then creates `DATA_DIR`, gives `/data` and everything in it to
`netwatch`, and sets mode 750 on `/data` and `DATA_DIR`. Every step goes
through a Go `os.Root` opened on `/data`, and the modes are set on the
opened directories rather than by name, so neither a symbolic link
already there nor one a host process swaps in while the container
starts can make root create or change anything outside `/data`. The
README says which `DATA_DIR` values are accepted.

Model: opus-5-5
2026-10-03 18:24:38 +02:00
clawbot e4df415676 Delete the oldest report files to stay under the size cap (closes #54)
check / check (push) Successful in 1m57s
When a report would take the report files past DATA_DIR_MAX_BYTES,
reportbuf now deletes the oldest report files until it fits, and does
the same at start when files left by an earlier run are already past
it. A file joins the files that may be deleted, at its place by name,
only once it is completely written, so a file still being written is
never deleted. A report is refused with 507 only when the reports
waiting to be written fill the cap on their own, and then no file is
deleted. The reports of a failed write stop counting, and the part of
its file written is removed. A file whose deletion fails keeps
counting; one already deleted by hand counts as freed.

Model: opus-5-5
2026-10-03 17:51:08 +02:00
clawbot 9e4d3fdb54 Lint's .golangci.yml check says which fix applies (closes #34)
check / check (push) Successful in 2m0s
backend/script/lint compared .golangci.yml with its pinned sha256 and,
on any failure, said to restore the file from sneak/prompts. Once the
org standard has legitimately changed, that advice loops: the copied
file is right and GOLANGCI_CONFIG_SHA256 is stale. On a mismatch the
script now says to compare the file with the org standard, restore it
if they differ, and update GOLANGCI_CONFIG_SHA256 if they are the same;
it still prints both hashes. A missing .golangci.yml, and a sha256sum
that is missing or prints no hash, get their own messages instead of
being reported as a mismatch. Each failure still exits 1. .golangci.yml
is unchanged.

Model: opus-5-5
2026-10-03 17:14:25 +02:00
clawbot aa35625d47 Go tests run with -race and -cover under Go's own timeout (closes #88)
check / check (push) Successful in 2m30s
backend/script/test runs go test -timeout 30s -race -cover and, if
that fails, runs it again with -v and fails. The root script/test drops
its one 30-second timeout around both halves: from a cold Go build
cache, compiling the tests with -race used it all up. Each half keeps
its own limit. The race detector needs a C compiler: the Dockerfile
builder stage gains gcc and musl-dev, and script/bootstrap installs gcc,
with the C library headers on apt and apk, when gcc is missing; make
build still sets CGO_ENABLED=0. New tests: the health check's answer, a
valid report's answer, a report file's exact lines, and the flush at the
10 MiB threshold. The handlers TestImport stub is gone.

Model: opus-5-5
2026-10-03 16:26:17 +02:00
clawbot 0ab6418e3d Target check timeout is 80% of the refresh interval (closes #78)
check / check (push) Successful in 1m55s
Each target check now times out after 80% of the refresh interval, 24
seconds at 30 seconds, where it was capped at 3 seconds, so slow, far
targets are recorded with their real time. Rounds never overlap: a
round gives up the last round's checks if they are still waiting, which
happens only when a round starts early, after an interval change or
when the recovery probe finds a target answering. The probe still
checks every half second, giving up its previous checks, so they do not
pile up.

The frontend has its first unit tests, run by script/frontend-test with
Node's built-in test runner on a mocked clock. index.html now links
src/styles.css, which src/main.js imported, since Node cannot import
CSS.

Model: opus-5-5
2026-10-03 15:36:47 +02:00
clawbot 4ce0814b14 Stamp the git tag or short commit in a plain docker build (closes #86)
check / check (push) Successful in 1m36s
The builder stage now has git and takes the version from the VERSION
build argument when one is given, otherwise from `git describe --tags
--always` of the repo's .git, copied to /git so go build does not see
it. A version that still comes out empty, dev or unknown fails the
build. ARG VERSION has no default. .dockerignore keeps .git/config out
of the build context, and the comments in script/docker and
script/cibuild no longer say .git is excluded.

Model: opus-5-5
2026-10-02 04:26:40 +02:00
clawbot e6d6815ecb Remove Buildarch: read the architecture at run time (closes #83)
check / check (push) Successful in 1m37s
The architecture is no longer passed in at build time. The Buildarch
variable and field are gone from main and globals, script/build no
longer stamps it in with -X, and the startup and listen log lines
report runtime.GOARCH under the key "arch". The Dockerfile comment and
backend/README.md no longer describe an architecture being stamped in.

Model: opus-5-5
2026-10-02 01:05:09 +02:00
22 changed files with 1314 additions and 1655 deletions
+1 -4
View File
@@ -73,12 +73,9 @@
# This repository's own entries, after the shared content above: the # This repository's own entries, after the shared content above: the
# frontend build, the viewport test's output, the binary `make build` # frontend build, the viewport test's output, the binary `make build`
# writes and the reports `make run` stores, and Go test binaries, # writes and the reports `make run` stores, and log files at the root.
# coverage output and log files at the root.
/dist /dist
/tmp /tmp
/netwatch-server /netwatch-server
/data /data
/*.test
/*.out
/*.log /*.log
+2 -3
View File
@@ -11,8 +11,7 @@ insert_final_newline = true
[Makefile] [Makefile]
indent_style = tab indent_style = tab
# This repository's own entries, after the shared content above.
[*.go] [*.go]
indent_style = tab indent_style = tab
# This repository's own sections, such as one for another language it
# uses, go below this comment, and a re-vendor keeps them.
-11
View File
@@ -1,20 +1,9 @@
name: check name: check
on: [push] on: [push]
# Free the shared runner: a new push cancels only the same branch's older run.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs: jobs:
check: check:
runs-on: ubuntu-latest runs-on: ubuntu-latest
# Free the shared runner from a hung build.
timeout-minutes: 20
steps: steps:
# actions/checkout v4.2.2, 2026-02-22 # actions/checkout v4.2.2, 2026-02-22
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
# script/cibuild needs no token, so none is left in .git/config.
with:
persist-credentials: false
# All history and tags, so git describe finds the version tag.
fetch-depth: 0
- run: script/cibuild - run: script/cibuild
+2 -4
View File
@@ -27,7 +27,7 @@ node_modules/
# Environment files. `*.env` covers bare `.env` and the `prod.env` # Environment files. `*.env` covers bare `.env` and the `prod.env`
# convention. Only the templates `example.env` and `sample.env` are # convention. Only the templates `example.env` and `sample.env` are
# re-included below. A repository that commits any other template adds # re-included below. A repository that commits any other template adds
# its own negation at the end of this file, for example `!.env.example`. # its own negation after these lines, for example `!.env.example`.
*.[eE][nN][vV] *.[eE][nN][vV]
.[eE][nN][vV].* .[eE][nN][vV].*
.[eE][nN][vV][rR][cC] .[eE][nN][vV][rR][cC]
@@ -46,9 +46,7 @@ node_modules/
[iI][dD]_[eE][dD]25519 [iI][dD]_[eE][dD]25519
[iI][dD]_[eE][dD]25519_[sS][kK] [iI][dD]_[eE][dD]25519_[sS][kK]
# This repository's own entries, such as its build outputs, go below # This repository's own entries, after the shared content above.
# this comment, and a re-vendor keeps them. Anchor a binary built at the
# root: `/myapp`, never `myapp`, which also ignores `cmd/myapp/`.
# Build output # Build output
dist/ dist/
-2
View File
@@ -25,8 +25,6 @@ linters:
# silenced by disabling that name, not by enabling the successor. # silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5 - wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2 - gomodguard # Deprecated, replaced by gomodguard_v2
# Misses findings at random in v2.14.0; back once a pinned release fixes it
- canonicalheader
settings: settings:
lll: lll:
line-length: 88 line-length: 88
+561 -415
View File
File diff suppressed because it is too large Load Diff
+534 -558
View File
File diff suppressed because it is too large Load Diff
+74 -4
View File
@@ -1,11 +1,22 @@
// Package main is the entry point for netwatch-server; package cli // Package main is the entry point for netwatch-server.
// does the work.
package main package main
import ( import (
"fmt"
"os" "os"
"os/user"
"sneak.berlin/go/netwatch/internal/cli" "sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals"
"sneak.berlin/go/netwatch/internal/handlers"
"sneak.berlin/go/netwatch/internal/healthcheck"
"sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/middleware"
"sneak.berlin/go/netwatch/internal/reportbuf"
"sneak.berlin/go/netwatch/internal/server"
"go.uber.org/fx"
"go.uber.org/fx/fxevent"
) )
//nolint:gochecknoglobals // set via ldflags at build time //nolint:gochecknoglobals // set via ldflags at build time
@@ -15,5 +26,64 @@ var (
) )
func main() { func main() {
os.Exit(cli.Main(os.Stderr, Appname, Version, os.Args[1:], cli.RunServer)) // "netwatch-server check-cidr CIDR" exits 1, with the error, if
// this server would refuse CIDR in its TRUSTED_PROXIES.
// bin/entrypoint.sh runs it on each entry it gives nginx.
if len(os.Args) == 3 && os.Args[1] == "check-cidr" {
_, err := middleware.ParseTrustedProxies(os.Args[2:])
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
return
}
// "netwatch-server prepare-data-dir DATA_DIR" gets DATA_DIR ready
// for the netwatch user, or exits 1 with the error; see
// reportbuf.PrepareDataDir. bin/entrypoint.sh runs it as root
// before it starts this server as that user.
if len(os.Args) == 3 && os.Args[1] == "prepare-data-dir" {
netwatch, err := user.Lookup("netwatch")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
err = reportbuf.PrepareDataDir("/data", os.Args[2], netwatch)
if err != nil {
fmt.Fprintf(os.Stderr, "DATA_DIR '%s': %v\n", os.Args[2], err)
os.Exit(1)
}
return
}
globals.Appname = Appname
globals.Version = Version
fx.New(
// fx logs each step of starting and stopping through the
// server's own logger, so off a terminal those lines are
// JSON like every other line.
fx.WithLogger(func(log *logger.Logger) fxevent.Logger {
return &fxevent.SlogLogger{Logger: log.Get()}
}),
fx.Provide(
config.New,
globals.New,
handlers.New,
healthcheck.New,
logger.New,
middleware.New,
reportbuf.New,
server.New,
),
fx.Invoke(
// First, so the name and version are logged even when
// a setting stops the start.
func(log *logger.Logger) { log.Identify() },
func(*server.Server) {},
),
).Run()
} }
+4 -3
View File
@@ -8,9 +8,10 @@ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" alway
# Scripts and styles load only from the page's own origin. Inline ones # Scripts and styles load only from the page's own origin. Inline ones
# are blocked, style attributes in markup included, so style elements # are blocked, style attributes in markup included, so style elements
# through classes or element.style. data: images are for the favicon # through classes or element.style. data: images are for the favicon
# in web/index.html. connect-src is * because the page checks every # in web/index.html. connect-src is * because the browser checks each
# target in web/src/main.js. A list of hosts here would have to change # probe in web/src/main.js against it, and also every redirect the probe
# with every change to those targets. It also covers the reports the # follows, and several of those hosts redirect to others; a list of
# hosts here would block those probes. It also covers the reports the
# page sends to its own origin. # page sends to its own origin.
add_header Content-Security-Policy "default-src 'self'; connect-src *; img-src 'self' data:; object-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'" always; add_header Content-Security-Policy "default-src 'self'; connect-src *; img-src 'self' data:; object-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'" always;
-143
View File
@@ -1,143 +0,0 @@
// Package cli runs netwatch-server from its command line: the
// check-cidr and prepare-data-dir subcommands, which bin/entrypoint.sh
// runs before it starts the server, and, given no arguments, the
// server, with the fx wiring.
package cli
import (
"fmt"
"io"
"os/user"
"sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals"
"sneak.berlin/go/netwatch/internal/handlers"
"sneak.berlin/go/netwatch/internal/healthcheck"
"sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/middleware"
"sneak.berlin/go/netwatch/internal/reportbuf"
"sneak.berlin/go/netwatch/internal/server"
"go.uber.org/fx"
"go.uber.org/fx/fxevent"
)
// usageExitCode is what Main returns for a command line it does not
// understand, the usual exit code for a usage error.
const usageExitCode = 2
// Main runs netwatch-server with args, its command line after the
// program's name, and returns the exit code. With no args, Main runs
// the server by calling serve with appname and version, the name and
// version the server logs and reports. With "check-cidr CIDR" or
// "prepare-data-dir DATA_DIR" it runs that subcommand, which writes its
// errors to stderr. Any other args get a one-line usage message on
// stderr and exit code 2. main passes os.Stderr and RunServer; the
// tests pass a buffer and a function of their own, so that no test
// starts the server. When serve is RunServer and the server fails to
// start, to listen or to stop, fx's Run exits the process with status 1
// and Main never returns.
func Main(
stderr io.Writer,
appname, version string,
args []string,
serve func(appname, version string),
) int {
// "netwatch-server check-cidr CIDR" exits 1, with the error, if
// this server would refuse CIDR in its TRUSTED_PROXIES.
// bin/entrypoint.sh runs it on each entry it gives nginx.
if len(args) == 2 && args[0] == "check-cidr" {
return checkCIDR(stderr, args[1])
}
// "netwatch-server prepare-data-dir DATA_DIR" gets DATA_DIR ready
// for the netwatch user, or exits 1 with the error; see
// reportbuf.PrepareDataDir. bin/entrypoint.sh runs it as root
// before it starts this server as that user.
if len(args) == 2 && args[0] == "prepare-data-dir" {
return prepareDataDir(stderr, "/data", args[1], "netwatch")
}
// Any other command line is refused, so that a mistyped one does
// not start the server.
if len(args) > 0 {
_, _ = fmt.Fprintln(stderr,
"usage: netwatch-server [check-cidr CIDR | prepare-data-dir DATA_DIR]")
return usageExitCode
}
serve(appname, version)
return 0
}
// checkCIDR returns 0 if the server would take cidr in its
// TRUSTED_PROXIES, and otherwise writes the error to stderr and
// returns 1.
func checkCIDR(stderr io.Writer, cidr string) int {
_, err := middleware.ParseTrustedProxies([]string{cidr})
if err != nil {
_, _ = fmt.Fprintln(stderr, err)
return 1
}
return 0
}
// prepareDataDir gets dataDir ready for the user named username with
// reportbuf.PrepareDataDir, volume being where the data volume is
// mounted, and returns 0. Otherwise it writes the error to stderr and
// returns 1.
func prepareDataDir(stderr io.Writer, volume, dataDir, username string) int {
owner, err := user.Lookup(username)
if err != nil {
_, _ = fmt.Fprintln(stderr, err)
return 1
}
err = reportbuf.PrepareDataDir(volume, dataDir, owner)
if err != nil {
_, _ = fmt.Fprintf(stderr, "DATA_DIR '%s': %v\n", dataDir, err)
return 1
}
return 0
}
// RunServer runs the server until it is told to stop, with appname and
// version as the name and version it logs and reports. If the server
// fails to start, to listen or to stop, fx's Run exits the process
// with status 1 itself.
func RunServer(appname, version string) {
globals.Appname = appname
globals.Version = version
fx.New(
// fx logs each step of starting and stopping through the
// server's own logger, so off a terminal those lines are
// JSON like every other line.
fx.WithLogger(func(log *logger.Logger) fxevent.Logger {
return &fxevent.SlogLogger{Logger: log.Get()}
}),
fx.Provide(
config.New,
globals.New,
handlers.New,
healthcheck.New,
logger.New,
middleware.New,
reportbuf.New,
server.New,
),
fx.Invoke(
// First, so the name and version are logged even when
// a setting stops the start.
func(log *logger.Logger) { log.Identify() },
func(*server.Server) {},
),
).Run()
}
-219
View File
@@ -1,219 +0,0 @@
package cli_test
import (
"bytes"
"os"
"os/user"
"path/filepath"
"strconv"
"strings"
"testing"
"sneak.berlin/go/netwatch/internal/cli"
"sneak.berlin/go/netwatch/internal/reportbuf"
)
// TestMainRunsCheckCIDR: Main given check-cidr returns 0, writing
// nothing, for a CIDR the server takes in its TRUSTED_PROXIES, and 1,
// with an error naming TRUSTED_PROXIES and the CIDR, for one it
// refuses, and does not start the server.
func TestMainRunsCheckCIDR(t *testing.T) {
t.Parallel()
var stderr bytes.Buffer
code := cli.Main(&stderr, "netwatch", "test",
[]string{"check-cidr", "10.0.0.0/8"}, failIfServerStarts(t))
if code != 0 || stderr.Len() != 0 {
t.Errorf("check-cidr 10.0.0.0/8: exit %d, stderr %q; want exit 0 and no output",
code, stderr.String())
}
stderr.Reset()
code = cli.Main(&stderr, "netwatch", "test",
[]string{"check-cidr", "1.2.3/32"}, failIfServerStarts(t))
want := `TRUSTED_PROXIES "1.2.3/32": `
if code != 1 || !strings.HasPrefix(stderr.String(), want) {
t.Errorf("check-cidr 1.2.3/32: exit %d, stderr %q; want exit 1 and %q...",
code, stderr.String(), want)
}
}
// TestMainRunsPrepareDataDir: Main given prepare-data-dir runs the
// prepare-data-dir subcommand for the netwatch user, and does not start
// the server. The test image has no netwatch user, so that subcommand
// exits 1 with the error from looking the user up;
// TestPrepareDataDirRefusesPathOutsideVolume checks the path.
func TestMainRunsPrepareDataDir(t *testing.T) {
t.Parallel()
var stderr bytes.Buffer
code := cli.Main(&stderr, "netwatch", "test",
[]string{"prepare-data-dir", "/etc"}, failIfServerStarts(t))
want := user.UnknownUserError("netwatch").Error() + "\n"
if code != 1 || stderr.String() != want {
t.Errorf("prepare-data-dir /etc: exit %d, stderr %q; want exit 1 and %q",
code, stderr.String(), want)
}
}
// TestMainRunsServerWithNoArguments: Main given no arguments starts the
// server with the name and version it was given, writes nothing and
// returns 0.
func TestMainRunsServerWithNoArguments(t *testing.T) {
t.Parallel()
var (
stderr bytes.Buffer
gotAppname, gotVersion string
)
code := cli.Main(&stderr, "netwatch", "test", []string{},
func(appname, version string) {
gotAppname, gotVersion = appname, version
})
if code != 0 || stderr.Len() != 0 {
t.Errorf("exit %d, stderr %q; want exit 0 and no output",
code, stderr.String())
}
if gotAppname != "netwatch" || gotVersion != "test" {
t.Errorf("server started with %q %q; want %q %q",
gotAppname, gotVersion, "netwatch", "test")
}
}
// TestMainRefusesOtherCommandLines: Main given a subcommand with too
// few or too many values, or a misspelt subcommand, writes the usage
// message to stderr, returns 2 and does not start the server.
func TestMainRefusesOtherCommandLines(t *testing.T) {
t.Parallel()
want := "usage: netwatch-server " +
"[check-cidr CIDR | prepare-data-dir DATA_DIR]\n"
for _, commandLine := range []string{
"netwatch-server check-cidr",
"netwatch-server check-cidr 192.0.2.0/24 198.51.100.0/24",
"netwatch-server prepare-data-dir",
"netwatch-server prepare-data-dir /data/reports /data/other",
"netwatch-server check-cdir 192.0.2.0/24",
} {
var stderr bytes.Buffer
// Main takes the command line after the program's name.
args := strings.Fields(commandLine)[1:]
code := cli.Main(&stderr, "netwatch", "test", args, failIfServerStarts(t))
if code != 2 || stderr.String() != want {
t.Errorf("%q: exit %d, stderr %q; want exit 2 and %q",
commandLine, code, stderr.String(), want)
}
}
}
// TestCheckCIDRTakesCIDR: check-cidr exits 0, writing nothing, for a
// CIDR the server takes in its TRUSTED_PROXIES.
func TestCheckCIDRTakesCIDR(t *testing.T) {
t.Parallel()
for _, cidr := range []string{"10.0.0.0/8", "192.0.2.1/32", "2001:db8::/32"} {
var stderr bytes.Buffer
code := cli.CheckCIDR(&stderr, cidr)
if code != 0 || stderr.Len() != 0 {
t.Errorf("%q: exit %d, stderr %q; want exit 0 and no output",
cidr, code, stderr.String())
}
}
}
// TestCheckCIDRRefusesOtherInput: check-cidr exits 1, with an error
// naming TRUSTED_PROXIES and the input, for anything the server would
// refuse in its TRUSTED_PROXIES, such as 1.2.3/32, which nginx would
// look up as a hostname.
func TestCheckCIDRRefusesOtherInput(t *testing.T) {
t.Parallel()
for _, cidr := range []string{
"1.2.3/32", "example.com/32", "10.0.0.0/33", "10.0.0.1", "",
} {
var stderr bytes.Buffer
code := cli.CheckCIDR(&stderr, cidr)
want := "TRUSTED_PROXIES " + strconv.Quote(cidr) + ": "
if code != 1 || !strings.HasPrefix(stderr.String(), want) {
t.Errorf("%q: exit %d, stderr %q; want exit 1 and %q...",
cidr, code, stderr.String(), want)
}
}
}
// TestPrepareDataDirTakesPathInVolume: prepare-data-dir exits 0,
// writing nothing, for a DATA_DIR below the volume, and creates it.
func TestPrepareDataDirTakesPathInVolume(t *testing.T) {
t.Parallel()
volume := t.TempDir()
dataDir := filepath.Join(volume, "reports")
var stderr bytes.Buffer
code := cli.PrepareDataDir(&stderr, volume, dataDir, currentUsername(t))
if code != 0 || stderr.Len() != 0 {
t.Errorf("exit %d, stderr %q; want exit 0 and no output",
code, stderr.String())
}
info, err := os.Stat(dataDir)
if err != nil || !info.IsDir() {
t.Errorf("DATA_DIR not created: %v", err)
}
}
// TestPrepareDataDirRefusesPathOutsideVolume: prepare-data-dir exits
// 1, with an error naming DATA_DIR and its value, for a DATA_DIR
// outside the volume.
func TestPrepareDataDirRefusesPathOutsideVolume(t *testing.T) {
t.Parallel()
volume := t.TempDir()
var stderr bytes.Buffer
code := cli.PrepareDataDir(&stderr, volume, "/etc", currentUsername(t))
want := "DATA_DIR '/etc': " + reportbuf.ErrDataDirOutsideVolume.Error() + "\n"
if code != 1 || stderr.String() != want {
t.Errorf("exit %d, stderr %q; want exit 1 and %q",
code, stderr.String(), want)
}
}
// currentUsername is the name of the user the test runs as, the only
// owner a test not run as root can give files to.
func currentUsername(t *testing.T) string {
t.Helper()
current, err := user.Current()
if err != nil {
t.Fatal(err)
}
return current.Username
}
// failIfServerStarts is what the tests give Main in place of
// cli.RunServer: it fails the test instead of starting the server.
func failIfServerStarts(t *testing.T) func(appname, version string) {
t.Helper()
return func(string, string) {
t.Error("Main started the server")
}
}
-16
View File
@@ -1,16 +0,0 @@
package cli
import "io"
// CheckCIDR exposes the check-cidr subcommand to the external tests,
// which read what it writes to stderr.
func CheckCIDR(stderr io.Writer, cidr string) int {
return checkCIDR(stderr, cidr)
}
// PrepareDataDir exposes the prepare-data-dir subcommand to the
// external tests, which give it a volume and a user of their own in
// place of /data and netwatch.
func PrepareDataDir(stderr io.Writer, volume, dataDir, username string) int {
return prepareDataDir(stderr, volume, dataDir, username)
}
+4 -4
View File
@@ -13,8 +13,8 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
) )
// requireConfigError builds the config as cli.RunServer does and fails // requireConfigError builds the config as main does and fails the
// the test unless that fails with an error naming setting. It uses // test unless that fails with an error naming setting. It uses
// fx.New, because fxtest.New fails the test itself on an error. // fx.New, because fxtest.New fails the test itself on an error.
func requireConfigError(t *testing.T, setting string) { func requireConfigError(t *testing.T, setting string) {
t.Helper() t.Helper()
@@ -31,8 +31,8 @@ func requireConfigError(t *testing.T, setting string) {
} }
} }
// requireTrustedProxies builds the config as cli.RunServer does and // requireTrustedProxies builds the config as main does and fails the
// fails the test unless its TrustedProxies are want. // test unless its TrustedProxies are want.
func requireTrustedProxies(t *testing.T, want []string) { func requireTrustedProxies(t *testing.T, want []string) {
t.Helper() t.Helper()
+1 -1
View File
@@ -4,7 +4,7 @@ package globals
import "go.uber.org/fx" import "go.uber.org/fx"
//nolint:gochecknoglobals // set by cli.RunServer before fx starts //nolint:gochecknoglobals // set from main before fx starts
var ( var (
// Appname is the application name. // Appname is the application name.
Appname string Appname string
+7 -7
View File
@@ -334,9 +334,9 @@ func stopWhileHeld(
return <-stopped return <-stopped
} }
// startApp starts a Buffer through fx, as cli.RunServer does, with the // startApp starts a Buffer through fx, as main does, with the DATA_DIR
// DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set, and returns // and DATA_DIR_MAX_BYTES the calling test has set, and returns it with
// it with the app, for tests that stop the app themselves. // the app, for tests that stop the app themselves.
func startApp(t *testing.T) (*reportbuf.Buffer, *fxtest.App) { func startApp(t *testing.T) (*reportbuf.Buffer, *fxtest.App) {
t.Helper() t.Helper()
@@ -358,8 +358,8 @@ func startApp(t *testing.T) (*reportbuf.Buffer, *fxtest.App) {
return buf, app return buf, app
} }
// startBuffer starts a Buffer through fx, as cli.RunServer does, with // startBuffer starts a Buffer through fx, as main does, with the
// the DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set. // DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set.
func startBuffer(t *testing.T) *reportbuf.Buffer { func startBuffer(t *testing.T) *reportbuf.Buffer {
t.Helper() t.Helper()
@@ -1120,8 +1120,8 @@ func TestHealthCheckFailsWhileWritesFail(t *testing.T) {
} }
// startBufferWithHandlers starts a Buffer and the Handlers that store // startBufferWithHandlers starts a Buffer and the Handlers that store
// reports in it through fx, as cli.RunServer does, with the DATA_DIR // reports in it through fx, as main does, with the DATA_DIR the calling
// the calling test has set. // test has set.
func startBufferWithHandlers( func startBufferWithHandlers(
t *testing.T, t *testing.T,
) (*reportbuf.Buffer, *handlers.Handlers) { ) (*reportbuf.Buffer, *handlers.Handlers) {
+3 -3
View File
@@ -22,9 +22,9 @@ import (
"go.uber.org/fx/fxtest" "go.uber.org/fx/fxtest"
) )
// newServer builds a Server from the same constructors as // newServer builds a Server from the same constructors as main,
// cli.RunServer, configured from the environment. It is never started, // configured from the environment. It is never started, so nothing
// so nothing listens. // listens.
func newServer(t *testing.T) *server.Server { func newServer(t *testing.T) *server.Server {
t.Helper() t.Helper()
+1 -1
View File
@@ -79,7 +79,7 @@ func New(
s.SetupRoutes() s.SetupRoutes()
s.httpServer = s.newHTTPServer() s.httpServer = s.newHTTPServer()
go func() { s.listenAndServe() }() go s.listenAndServe()
return nil return nil
}, },
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "netwatch", "name": "netwatch",
"private": true, "version": "1.0.0",
"description": "Real-time network latency monitor SPA", "description": "Real-time network latency monitor SPA",
"type": "module", "type": "module",
"scripts": { "scripts": {
+3 -4
View File
@@ -10,10 +10,9 @@ main() {
cd "$ROOT" cd "$ROOT"
"$ROOT/script/yarn" build "$ROOT/script/yarn" build
# VERSION comes from the environment. Unset or empty, it is # VERSION comes from the environment. Unset or empty, it is git
# `git describe --tags --always --dirty`, as in script/docker, or # describe, or "dev" where there is no git or no repository history.
# "dev" where there is no git or no repository history. version="${VERSION:-$(git describe --always --dirty 2>/dev/null || echo dev)}"
version="${VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo dev)}"
CGO_ENABLED=0 go build -trimpath \ CGO_ENABLED=0 go build -trimpath \
-ldflags "-s -w -X main.Version=$version" \ -ldflags "-s -w -X main.Version=$version" \
-85
View File
@@ -180,91 +180,6 @@ test("a check fetches the target's URL as written, with no query string added",
); );
}); });
test("a check sends no Referer header, wherever the page is served from", async (t) => {
mockTargets(t, () => 10);
const check = measureLatency("https://fsn1-speed.hetzner.com");
t.mock.timers.tick(10);
assert.notEqual(await settled(check), "still waiting");
assert.equal(
fetch.mock.calls[0].arguments[1].referrerPolicy,
"no-referrer",
);
});
// These targets' home pages answer with a redirect. A check follows it and
// times the request to the new URL too, often on another host. The Google
// Cloud Console redirects only requests that do not say they come from
// another site, such as curl's. Each target is checked at /robots.txt on
// its own host, which answers without a redirect. An S3 region answers it
// with 403, and any answer counts as reachable.
test("a target whose home page redirects is checked at /robots.txt on its own host", () => {
const checked = new Map(
new AppState([]).wan.map((host) => [host.name, host.url]),
);
for (const [name, url] of [
["AWS Console", "https://console.aws.amazon.com/robots.txt"],
["Google Cloud Console", "https://console.cloud.google.com/robots.txt"],
["Akamai", "https://www.akamai.com/robots.txt"],
["B2", "https://api.backblazeb2.com/robots.txt"],
[
"S3 af-south-1 (Cape Town)",
"https://s3.af-south-1.amazonaws.com/robots.txt",
],
[
"S3 eu-west-2 (London)",
"https://s3.eu-west-2.amazonaws.com/robots.txt",
],
[
"S3 me-south-1 (Bahrain)",
"https://s3.me-south-1.amazonaws.com/robots.txt",
],
[
"S3 ap-northeast-1 (Tokyo)",
"https://s3.ap-northeast-1.amazonaws.com/robots.txt",
],
[
"S3 ap-southeast-1 (Singapore)",
"https://s3.ap-southeast-1.amazonaws.com/robots.txt",
],
[
"S3 ap-southeast-2 (Sydney)",
"https://s3.ap-southeast-2.amazonaws.com/robots.txt",
],
[
"S3 us-west-2 (Oregon)",
"https://s3.us-west-2.amazonaws.com/robots.txt",
],
[
"S3 sa-east-1 (São Paulo)",
"https://s3.sa-east-1.amazonaws.com/robots.txt",
],
]) {
assert.equal(checked.get(name), url);
}
});
// Chrome keeps the connection to an S3 region open after a HEAD check, but
// closes it after most GET checks. The Hetzner servers do not answer HEAD.
test("a round checks each S3 region with a HEAD request and every other target with a GET request", async (t) => {
const state = new AppState([
{ name: "Local CPE", url: "http://192.168.100.1" },
]);
mockTargets(t, () => 10);
const round = tick(state, new AbortController().signal);
t.mock.timers.tick(10);
assert.notEqual(await settled(round), "still waiting");
const methods = new Map(
fetch.mock.calls.map((call) => [
call.arguments[0],
call.arguments[1].method,
]),
);
for (const host of state.allHosts) {
const method = host.name.startsWith("S3 ") ? "HEAD" : "GET";
assert.equal(methods.get(host.url), method, host.name);
}
});
test("at a 30000ms interval, a target answering after 1000ms shows in its row while another target's check is still waiting", async (t) => { test("at a 30000ms interval, a target answering after 1000ms shows in its row while another target's check is still waiting", async (t) => {
CONFIG.updateInterval = 30000; CONFIG.updateInterval = 30000;
const state = new AppState([ const state = new AppState([
+91 -123
View File
@@ -1,146 +1,114 @@
# The viewport test # Responsive-layout harness
`make frontend-viewport-test` checks the page's layout at several window widths Automated verification of the responsive layout that landed in #5. Run it with:
in a real browser:
```bash ```bash
make frontend-viewport-test make frontend-viewport-test
``` ```
First it runs the unit tests. Then it builds the page into `dist/`. It serves It builds `dist/`, serves it from the same digest-pinned `nginx` image and the
`dist/` with the nginx image and the `configs/nginx.conf` that the Docker image same `configs/nginx.conf` the shipping container uses, drives a digest-pinned
ships. Before the browser starts, it fetches `/` and `/no-such-page`, which headless Chrome against it over CDP, and asserts on computed layout at every
nginx answers with the page. Both must carry `Cache-Control: no-cache` and a viewport width derived from the app's own CSS. Screenshots land in
`Content-Security-Policy` header. Then it starts a headless Chrome in a `tmp/viewport/` alongside a `results.json`; they are artifacts for a human to
container and opens the page in it at each width. For each width it measures the look at when something fails, not the evidence. The assertions are the evidence.
layout inside the page and checks the measurements in Node. The checks decide
pass or fail. Screenshots and a `results.json` land in `tmp/viewport/`, for a
person to look at when a check fails.
The test is not part of `make check`: it takes minutes, and `make test` has to The target is deliberately outside `make check`: it takes minutes, and
finish in under 60 seconds. `make test` has to stay under 60 seconds.
## The widths ## How the widths are chosen
The breakpoints are read from the page's own files. `viewports.js` reads the Not from a list of phone models. `viewports.js` parses the `@media` conditions
`min-width` and `max-width` conditions out of the `@media` blocks in out of `web/src/styles.css` and scans `web/src/main.js` and `web/index.html` for
`web/src/styles.css`. It also looks in `web/src/main.js` and `web/index.html` Tailwind responsive prefixes, then tests every breakpoint it finds at one pixel
for Tailwind's responsive class prefixes, such as `md:`, and takes their below it, exactly on it, and one pixel above it. A generic 375px "phone" test
breakpoints. Each breakpoint is tested at one pixel below it, exactly on it, and sails straight past an off-by-one at a media query boundary; `max-width: 768px`
one pixel above it. Today the CSS has one breakpoint, `max-width: 768px`, and no matches _at_ 768, and the sweep pins down which side of that line each layout is
responsive prefix is used. on.
A `max-width: 768px` block matches at 768 pixels. Testing both sides of each Nothing hardcodes 768. Add a second media block or start using `md:` classes and
breakpoint catches a layout that switches one pixel too early or too late. A the new breakpoint is covered without this directory being touched. The app is
second media block, or a first `md:` class, is covered without a change to this desktop-first today (all narrow rules live in `max-width` blocks); a
directory. `min-width`-only, mobile-first set is handled as its inverse, and a set that
mixes the two makes the run fail loudly rather than test the right widths with
the wrong expectation. Four further viewports are fixed anchors, each with a
stated reason: a 320px floor, a 1280px desktop baseline, and two phone-landscape
sizes straddling the breakpoint for the rotation case.
The page is desktop-first: the wide layout is the default, and every narrow rule ## What it asserts
is in a `max-width` block. So a width is expected to show the narrow layout when
one of those blocks matches it. A page with only `min-width` blocks would be
read the other way round. A page that mixes the two kinds fails the run with a
message, because the right expectation for each width can no longer be read from
the breakpoints alone.
Four more widths are fixed, each for a stated reason: - **app-rendered** — enough host rows exist and enough of them show a numeric
latency. This one exists so the rest cannot pass vacuously against a blank
page.
- **no-horizontal-overflow** — `documentElement.scrollWidth` fits the layout
viewport, with the widest offending element named.
- **nothing-past-viewport-edge** — no visible element's box extends past the
viewport edge.
- **no-clipped-text** — nothing hides text behind `overflow: hidden`. Deliberate
ellipsis truncation (Tailwind's `truncate`, used on host names and URLs) is
excluded: it is a design choice, not breakage.
- **tap-targets-44px** — every interactive control is at least 44x44 CSS px on
touch viewports, _and_ each selector in the control list matched at least the
number of visible elements it declares: one of each single control, and one
pin button per WAN host row. The second half is what stops the check passing
vacuously: with size alone, a renamed class would take its controls out of the
measured set and the check would report "all 0 controls are at least 44x44"
and pass. See below.
- **host-rows-stacked / host-rows-side-by-side** — the rows genuinely reflow.
Computed `flex-direction` _and_ the actual geometry are checked, and in the
narrow layout the info block and the sparkline must each occupy essentially
the full row width. A row that merely shrank its 420px column would fail.
- **probing-still-runs / gateway-detection-still-runs** — narrow viewports keep
probing and keep detecting the gateway. The mobile early-return path proposed
in #8 was rejected; this is what would catch it coming back.
- 320 by 568 pixels: the narrowest width still in common use ### The tap-target threshold
- 667 by 375 pixels: a phone in landscape, inside the narrow layout
- 844 by 390 pixels: a large phone in landscape, in the wide layout but still a
touch device
- 1280 by 800 pixels: a desktop window
Every width but 1280 by 800 is tested as a touch device. 44x44 CSS px. That is the figure in Apple's Human Interface Guidelines and in
WCAG 2.2 SC 2.5.5 "Target Size (Enhanced)". WCAG 2.2 SC 2.5.8 (level AA) sets a
lower 24x24 floor, but that floor comes with a spacing exception these controls
do not qualify for — the pin buttons sit directly against the host name they
belong to.
## The checks ## Determinism
A failed check does not stop the others. The browser container runs on an `--internal` docker network and has no route to
the internet, so the app's latency probes cannot reach anything real. The
harness answers them itself from a fixed delay table, with a deterministic
fraction failed outright, so the rows render a realistic spread of one-, two-
and three-digit latencies plus some unreachable rows. That spread is what the
layout has to survive; a `---` placeholder in every row would not exercise it.
- `app-rendered`: at least 10 WAN host rows exist, and at least 5 rows show a ## What this cannot verify
number as their latency. Without this, the other checks could pass against a
blank page.
- `no-horizontal-overflow`: the document is no wider than the window. On a
failure, the widest content is named.
- `nothing-past-viewport-edge`: no visible element extends past the window's
edge.
- `no-clipped-text`: no element hides its text behind `overflow: hidden` or
`overflow: clip`. Text cut with an ellipsis is allowed: that cut is a design
choice, not a defect. Tailwind's `truncate` cuts host names and URLs that way.
- `tap-targets-44px`, on touch widths: each measured control is at least 44 by
44 CSS pixels. The controls are `#pause-btn`, `#interval-select`,
`#debug-toggle` and the `.pin-btn` in each WAN host row. The check also
demands that each selector matches enough visible elements: one of each single
control, and one pin button per WAN host row. A renamed class would otherwise
take its controls out of the measured set, and the check would pass on
nothing.
- `host-rows-stacked` or `host-rows-side-by-side`: each host row shows the
layout expected at that width. In the narrow layout, the row's
`flex-direction` is `column`, the graph sits below the host's details, and
each takes at least 90% of the row's width. In the wide layout, the direction
is `row` and the graph sits to the right of the details. A row that only
shrank its fixed 420-pixel column fails.
- `probing-still-runs`: the page sent at least one check to a target at this
width. The narrow layout must not skip the checks.
- `gateway-detection-still-runs`: the `Local Gateway` row is present at this
width, so the gateway was looked for and found. The narrow layout must not
skip that search.
### The tap-target size Real limits, so nobody re-parks this issue as needing hardware:
44 by 44 CSS pixels is the figure in Apple's Human Interface Guidelines and in - **Non-Chromium engines.** This is Chrome. iOS Safari is WebKit and cannot be
WCAG 2.2 success criterion 2.5.5, "Target Size (Enhanced)". WCAG 2.2 criterion emulated by it; Safari-specific bugs (viewport units under a collapsing URL
2.5.8 sets a lower minimum of 24 by 24 pixels, with an exception that depends on bar, `-webkit-fill-available`, form control metrics) will not show up here.
the space around a control. The pin buttons sit against the host name they - **Real touch input.** `hasTouch` emulation changes what the page is told, not
belong to, so the test keeps 44. how a finger behaves. Gesture handling, scroll momentum, double-tap zoom and
hover-state fallbacks on touch are out of scope.
- **Physical pixel density and rendering.** `deviceScaleFactor` is set, but
subpixel antialiasing, OLED colour rendering and actual legibility at a given
physical size are not measurable here.
- **Fonts.** The container has DejaVu, not the platform's own UI monospace. Text
metrics are therefore close to, but not identical to, a real device — a layout
that fits here by a few pixels might not there.
- **On-device performance.** Canvas sparkline redraw cost, battery, and
behaviour on a slow radio are not measured.
- **Browser chrome.** The address bar, safe-area insets and notch cutouts are
not simulated.
## The browser's network Everything else this issue was actually about — does the layout reflow, does
anything overflow, is content clipped, are the controls big enough — is a
The browser container runs on an internal Docker network with no route to the function of viewport width and CSS, and is covered above.
internet, so the page's checks reach no real host. The test answers them itself
from a fixed table of delays: 2, 45, 123, 456 and 780 milliseconds. Which delay
a target gets depends on its URL, and about one target in seven fails outright.
So the rows show one-, two- and three-digit latencies and some unreachable rows,
and the layout is tested against that spread. The request to
`http://192.168.1.1` is answered after 5 milliseconds, so the `Local Gateway`
row appears.
At each width, the test sets the interval to 1 second and waits until at least 5
rows show a latency before it measures. The page discards its first round, so
this reaches a filled page without waiting out two rounds at 3 seconds.
## What the test cannot check
- Other browser engines. The test runs Chrome. Safari uses WebKit, and bugs that
only WebKit has do not show up here.
- Real touch input. Touch emulation changes what the page is told, not how a
finger behaves. Gestures, scroll momentum, double-tap zoom and hover states on
touch are not tested.
- Physical pixels. The device pixel ratio is set, but antialiasing, colour
rendering and legibility at a given physical size are not measured.
- Fonts. The container has DejaVu, not a phone's own monospace font. Text is
about as wide as on a real device, but not exactly. A layout that fits here by
a few pixels might not fit there.
- Performance. The cost of redrawing the graphs, battery use and behaviour on a
slow connection are not measured.
- The browser's own frame. The address bar, safe-area insets and notch cutouts
are not simulated.
## The files
- `viewports.js` derives the widths from the sources.
- `facts.js` is run inside the page and measures the layout. It only measures;
it decides nothing.
- `checks.js` turns the measurements into the pass or fail of each check, with
the numbers that decided it.
- `harness.js` connects to the browser, runs each width, writes the screenshots
and `results.json`, and prints the results. `script/frontend-viewport-test`
starts it; every input it needs arrives in the environment.
## Relation to the unit tests ## Relation to the unit tests
The unit tests in `test/unit/` call the functions that `web/src/main.js` Complementary layers, not two stacks. The unit tests in `test/unit/`, which
exports, with no browser. `make test` runs them with Node's built-in test `make test` runs with Node's built-in test runner, exercise the functions
runner. This test runs the built page in a real browser engine and is the only `web/src/main.js` exports in-process with no browser. This harness exercises
test here that can see a media query. Neither replaces the other. Checks on rendered layout in a real engine and is the only thing here that can see a media
computed styles and element geometry belong here; checks on functions belong in query. Neither replaces the other; assertions about computed styles and element
`test/unit/`. geometry belong here, assertions about functions belong in `test/unit/`.
+25 -44
View File
@@ -67,66 +67,51 @@ export const CONFIG = {
// WAN endpoints to monitor. These are used for the aggregate health/stats // WAN endpoints to monitor. These are used for the aggregate health/stats
// display (min/max/avg). Ordered: personal, LLM APIs, big-3 cloud, then // display (min/max/avg). Ordered: personal, LLM APIs, big-3 cloud, then
// CDN/hosting/other. A check follows a redirect and times the request to // CDN/hosting/other.
// the new URL too. So a target whose home page redirects is checked at
// /robots.txt on its own host. That URL answers without a redirect.
const WAN_HOSTS = [ const WAN_HOSTS = [
{ name: "datavi.be", url: "https://datavi.be" }, { name: "datavi.be", url: "https://datavi.be" },
{ name: "Anthropic API", url: "https://api.anthropic.com" }, { name: "Anthropic API", url: "https://api.anthropic.com" },
{ name: "OpenAI API", url: "https://api.openai.com" }, { name: "OpenAI API", url: "https://api.openai.com" },
{ name: "AWS Console", url: "https://console.aws.amazon.com/robots.txt" }, { name: "AWS Console", url: "https://console.aws.amazon.com" },
{ { name: "Google Cloud Console", url: "https://console.cloud.google.com" },
name: "Google Cloud Console",
url: "https://console.cloud.google.com/robots.txt",
},
{ name: "Microsoft Azure", url: "https://portal.azure.com" }, { name: "Microsoft Azure", url: "https://portal.azure.com" },
{ name: "Cloudflare", url: "https://www.cloudflare.com" }, { name: "Cloudflare", url: "https://www.cloudflare.com" },
{ name: "Fastly CDN", url: "https://www.fastly.com" }, { name: "Fastly CDN", url: "https://www.fastly.com" },
{ name: "Akamai", url: "https://www.akamai.com/robots.txt" }, { name: "Akamai", url: "https://www.akamai.com" },
{ name: "Google", url: "https://www.google.com" }, { name: "Google", url: "https://www.google.com" },
{ name: "GitHub", url: "https://github.com" }, { name: "GitHub", url: "https://github.com" },
{ name: "B2", url: "https://api.backblazeb2.com/robots.txt" }, { name: "B2", url: "https://api.backblazeb2.com" },
// Chrome keeps the connection to an S3 region open after a HEAD check,
// but closes it after most GET checks.
{ {
name: "S3 af-south-1 (Cape Town)", name: "S3 af-south-1 (Cape Town)",
url: "https://s3.af-south-1.amazonaws.com/robots.txt", url: "https://s3.af-south-1.amazonaws.com",
method: "HEAD",
}, },
{ {
name: "S3 eu-west-2 (London)", name: "S3 eu-west-2 (London)",
url: "https://s3.eu-west-2.amazonaws.com/robots.txt", url: "https://s3.eu-west-2.amazonaws.com",
method: "HEAD",
}, },
{ {
name: "S3 me-south-1 (Bahrain)", name: "S3 me-south-1 (Bahrain)",
url: "https://s3.me-south-1.amazonaws.com/robots.txt", url: "https://s3.me-south-1.amazonaws.com",
method: "HEAD",
}, },
{ {
name: "S3 ap-northeast-1 (Tokyo)", name: "S3 ap-northeast-1 (Tokyo)",
url: "https://s3.ap-northeast-1.amazonaws.com/robots.txt", url: "https://s3.ap-northeast-1.amazonaws.com",
method: "HEAD",
}, },
{ {
name: "S3 ap-southeast-1 (Singapore)", name: "S3 ap-southeast-1 (Singapore)",
url: "https://s3.ap-southeast-1.amazonaws.com/robots.txt", url: "https://s3.ap-southeast-1.amazonaws.com",
method: "HEAD",
}, },
{ {
name: "S3 ap-southeast-2 (Sydney)", name: "S3 ap-southeast-2 (Sydney)",
url: "https://s3.ap-southeast-2.amazonaws.com/robots.txt", url: "https://s3.ap-southeast-2.amazonaws.com",
method: "HEAD",
}, },
{ {
name: "S3 us-west-2 (Oregon)", name: "S3 us-west-2 (Oregon)",
url: "https://s3.us-west-2.amazonaws.com/robots.txt", url: "https://s3.us-west-2.amazonaws.com",
method: "HEAD",
}, },
{ {
name: "S3 sa-east-1 (São Paulo)", name: "S3 sa-east-1 (São Paulo)",
url: "https://s3.sa-east-1.amazonaws.com/robots.txt", url: "https://s3.sa-east-1.amazonaws.com",
method: "HEAD",
}, },
// Hetzner regional speed test servers — genuine per-DC endpoints // Hetzner regional speed test servers — genuine per-DC endpoints
{ {
@@ -286,7 +271,6 @@ export class HostState {
constructor(host, pinned = false) { constructor(host, pinned = false) {
this.name = host.name; this.name = host.name;
this.url = host.url; this.url = host.url;
this.method = host.method;
// Each entry is either a check's result, { timestamp, latency, // Each entry is either a check's result, { timestamp, latency,
// error }, or a round skipped while paused, { timestamp, // error }, or a round skipped while paused, { timestamp,
// latency: null, paused: true }. // latency: null, paused: true }.
@@ -583,18 +567,16 @@ export class Reporter {
// --- Latency Measurement ----------------------------------------------------- // --- Latency Measurement -----------------------------------------------------
// Checks one target with the given HTTP method, GET when none is given. The // Checks one target. The check times out after CONFIG.requestTimeout; the
// check times out after CONFIG.requestTimeout; the caller can give it up // caller can give it up sooner through the optional signal, which also ends
// sooner through the optional signal, which also ends it as a timeout. A // it as a timeout. A failed check's reason says what went wrong and after
// failed check's reason says what went wrong and after how long; a check // how long; a check that answered has none.
// that answered has none.
// //
// The Hetzner speed-test servers close the connection without an answer // The URL is fetched as written, with nothing added to it: the Hetzner
// when the URL has a query string or the request has a Referer header. So // speed-test servers close the connection without an answer when the URL
// the URL is fetched as written, and referrerPolicy: "no-referrer" sends // has a query string, and cache: "no-store" keeps the browser's cache out
// no Referer header, wherever the page is served from. cache: "no-store" // of the measurement.
// keeps the browser's cache out of the measurement. export async function measureLatency(url, signal) {
export async function measureLatency(url, signal, method = "GET") {
const controller = new AbortController(); const controller = new AbortController();
const timeoutId = setTimeout( const timeoutId = setTimeout(
() => controller.abort(), () => controller.abort(),
@@ -606,10 +588,9 @@ export async function measureLatency(url, signal, method = "GET") {
try { try {
await fetch(url, { await fetch(url, {
method, method: "GET",
mode: "no-cors", mode: "no-cors",
cache: "no-store", cache: "no-store",
referrerPolicy: "no-referrer",
signal: controller.signal, signal: controller.signal,
}); });
const latency = Math.round(performance.now() - start); const latency = Math.round(performance.now() - start);
@@ -1309,7 +1290,7 @@ export async function tick(state, signal, onOffline) {
// the round started. // the round started.
await Promise.all( await Promise.all(
state.allHosts.map(async (host) => { state.allHosts.map(async (host) => {
const r = await measureLatency(host.url, signal, host.method); const r = await measureLatency(host.url, signal);
if (state.paused || signal.aborted || state.tickCount === 0) { if (state.paused || signal.aborted || state.tickCount === 0) {
return; return;
} }
@@ -1382,7 +1363,7 @@ function startRecoveryProbe(state, startRounds) {
const checks = new AbortController(); const checks = new AbortController();
state._recoveryProbeChecks = checks; state._recoveryProbeChecks = checks;
for (const host of canaries) { for (const host of canaries) {
measureLatency(host.url, checks.signal, host.method).then((r) => { measureLatency(host.url, checks.signal).then((r) => {
if (r.error !== null || checks.signal.aborted) return; if (r.error !== null || checks.signal.aborted) return;
log.notice("Recovery probe: connectivity detected"); log.notice("Recovery probe: connectivity detected");
stopRecoveryProbe(state); stopRecoveryProbe(state);