netwatch/TODO.md

Workflow

• branch from next
• do the work in Next Step
• move Next Step to the top of Completed Steps
• move the top item of Future Steps into Next Step
• commit (TODO.md changes in the same commit as the work)
• push the branch and open a PR against next

Status

pre-1.0. Release candidates are tagged on next, the first being v1.0.0-rc1; git tag --merged origin/next lists them. main is the stable branch and next the development branch, which every PR targets. The frontend and the Go backend ship as one Docker image. The Gitea workflow .gitea/workflows/check.yml runs script/cibuild on every push, and cancels the older run on the same branch. Working toward 1.0.0.

Next Step

Run make frontend-viewport-test in CI as its own step; it is not part of make check, as it needs Docker and takes minutes.

Completed Steps

• 2026-10-26: README.md, TODO.md and test/viewport/README.md are rewritten sentence by sentence (#189). Each sentence reads on the first pass: one idea, under about 30 words, the verb early. Every statement matches the code on next.
• 2026-10-26: a target whose home page answers with a redirect is checked at /robots.txt on the same host, which answers without one (#194). These are the AWS Console, Akamai, B2 and the 8 S3 regions. The browser followed each redirect, so their figures also timed the page it led to. The Google Cloud Console is checked at /robots.txt too, as its home page redirects curl's requests. The S3 regions are checked with a HEAD request, as Chrome closes their connection after most GET requests. README.md says what a figure includes.
• 2026-10-26: the six Hetzner targets answer wherever the page is served from (#191). They close the connection without an answer when the request has a Referer header. The browser sent one unless the page came with the image's Referrer-Policy: no-referrer header, which a static host and make dev do not send. Each check now sets referrerPolicy: 'no-referrer'.
• 2026-10-19: the server's OnStart hook in internal/server/server.go starts listenAndServe in an anonymous function, as the Go style guide asks (#162). It was go s.listenAndServe(). Nothing the server does changed.
• 2026-10-19: .gitignore, .editorconfig and .golangci.yml are the sneak/prompts copies at 0653579 (#174). .gitignore is the standard file followed by this repository's own entries. .editorconfig is the standard file alone, which carries the [*.go] section. .golangci.yml disables canonicalheader, as the canonical copy does: golangci-lint v2.14.0 misses its findings at random.
• 2026-10-19: package.json has no version field and is marked "private": true (#158). It said 1.0.0, a release that has not been made, and nothing read it. The git tag is the one version; the image and make build take it from git describe --tags. The package is never published.
• 2026-10-19: Status names v1.0.0-rc1, the first release candidate, and says release candidates are tagged on next (#157). It said there were no git tags.
• 2026-10-19: netwatch-server starts the server only when given no arguments (#179). check-cidr CIDR and prepare-data-dir DATA_DIR behave as before. Any other arguments print a one-line usage message naming both forms to stderr and exit 2; they used to start the server. Unit tests cover each case.
• 2026-10-19: main() in cmd/netwatch-server/main.go is one call into the new package internal/cli, as the Go style guide asks (#160). internal/cli runs the check-cidr and prepare-data-dir subcommands, with the same arguments, messages and exit codes, and otherwise the server with the fx wiring. Appname and Version stay in main, where the build sets Version. main gives cli.Main stderr and the function that runs the server; the tests replace both. Unit tests give each subcommand input it takes and input it refuses.
• 2026-10-19: when VERSION is unset or empty, script/build stamps the version from git describe --tags --always --dirty, as script/docker does (#156). Without --tags it skipped lightweight tags, so make run could report a commit where the image built from the same tree reported the tag.
• 2026-10-19: .dockerignore keeps Go test binaries and coverage output at the root out of the build context, with /*.test and /*.out in the repository's own section (#155). Such files were copied into the lint, test and build stages. The entries match only at the root, so the same files in a subdirectory still reach the image.
• 2026-10-19: .gitea/workflows/check.yml is the sneak/prompts copy at 0653579 (#153). Its checkout step sets fetch-depth: 0, so CI clones all history and tags. The image built from a tagged commit then gets the tag as its version; it got the short commit. The step also sets persist-credentials: false, so the job's token is not left in .git/config. A new push cancels the older run on the same branch, and a run stops after 20 minutes.
• 2026-10-19: script/bootstrap keeps the yarn on PATH only when yarn --version is exactly YARN_VERSION; otherwise it installs that version (#154). Any yarn used to be kept. It uses the node on PATH only when that node has the major version of NODE_VERSION and is at least NODE_MIN_VERSION; a node 24 was kept before. After installing node or yarn it looks the tool up through PATH again and stops if that one fails the same check, as it does for Go. It prints the node and yarn versions it uses. Every script that runs yarn on the host, except script/bootstrap, runs it through the new script/yarn. That script puts ~/.local/bin first on PATH. It uses the first node there when it passes the same test, and otherwise runs yarn under the pinned node in nvm. make build ran yarn under whatever node was first on PATH.
• 2026-10-19: bin/entrypoint.sh has each step in a function, and a main at the end calls them in order, as the code style guide's shell section asks (#163). It is still POSIX sh with no set -e, and does what it did, with the same messages and exit codes.
• 2026-10-19: while report files cannot be written, the health check answers 503 with "status":"error" and the same fields, so the image's HEALTHCHECK reports the container unhealthy (#142). It answered 200 whatever happened, so the platform saw a healthy container while every report was refused with 500. It answers 200 at start, before any write has failed, and again from the first write that succeeds.
• 2026-10-19: report file writes no longer overlap (#150), so the reports of failed writes are written in the order they arrived. Two writes that failed at the same time put their reports back in the wrong order when the older failed first. One write runs at a time. Each takes the waiting reports only when it starts, after a failed write has put its own back. The write that Append starts at 10 MiB waits for any write under way, and Append starts no other while it waits. Files written since the start are deleted to make room in the order they were written, after those found at start.
• 2026-10-19: DEBUG unset or empty leaves debug logging off, and any other value, false and 0 included, turns it on, as the code style guide says (#159). No value of it stops the start any more. Before, only 1, t, T, TRUE, true, True, 0, f, F, FALSE, false and False were accepted. README.md no longer gives false as its default.
• 2026-10-19: check builds keep Go's modules and compiled packages between runs (#118), on sneak's rulings in prompts#113 and prompts#130. REPO_POLICIES.md, script/test, script/lint, script/docker and script/cibuild are the sneak/prompts copies at 0653579. The scripts pass a new CACHEBUST value where they passed --no-cache, and the lint and test builds write no image. In each stage of Dockerfile, ARG CACHEBUST comes after the system packages and the Go module download. Every step that lints, tests or compiles comes after it, as does the node stages' yarn install. Go's build cache and golangci-lint's are cache mounts, and both go test runs carry -count=1. The test phase's Go caches used to be in memory, empty on every build.
• 2026-10-19: the three unit tests of a discarded check also check that the host's history is still empty, its status still pending, and the console untouched (#151). The discarded cases are a round given up, a pause, and the first round. The tests checked only the row. So a change that recorded the result and skipped only the row would have kept them passing.
• 2026-10-18: run on its own, the backend stops at start when DATA_DIR cannot be created or read, or when no file can be created in it (#148). The error names DATA_DIR and its value. The first two cases named only the path, and the third was found only at the first write of a report file. At start the server creates a file in DATA_DIR and removes it. The report file number is an atomic.Int64, as the Go style guide asks for int types (#161); file names are unchanged.
• 2026-10-18: the image's nginx sends the page with Cache-Control: no-cache (#141), so a browser that keeps a copy checks with the server that it is current before using it again. The page went out with no Cache-Control. So a browser could keep it after a deploy, ask for files under /assets/ that the new build no longer has, and show an empty page. A path that location / answers with the page through its try_files fallback gets the same header. location / includes the security headers again, as /assets/ does, since its own add_header drops the server's. /assets/ keeps its immutable caching. make frontend-viewport-test fails unless / and a path that falls back to the page carry Cache-Control: no-cache and Content-Security-Policy.
• 2026-10-18: README.md says what the page loses while reports are refused or the page is paused (#145). It says how fast one address can replace the stored reports (#146). It says that in the image nginx answers a body over 1 MiB itself (#147). nginx's client_max_body_size is 1 MiB by default, so in the image it answers a larger body with 413 and its own HTML page, and the backend never sees it. The reporter, the body limit and nginx are unchanged.
• 2026-10-18: a browser that cannot format the header's clocks still probes (#149). The clocks name the local time zone through Intl.DateTimeFormat's shortGeneric option, which Chrome before 95, Firefox before 91 and Safari before 15.4 reject with an error. It came before the first round started, so such a browser never checked a target. The clocks now start last, after the rounds, and an error formatting them leaves both blank. The footer's color legend reads >=500ms for red, where it read >500ms (#152).
• 2026-10-18: netwatch-server prepare-data-dir no longer changes the owner of a file outside /data through a hard link in the volume (#143). A file with a second name (hard link) keeps its owner, and the start logs its path. Owners are set on the opened directory, file or link, never by name, so a hard link swapped in after the check is caught too. A symbolic link is opened without following it, and the link itself is given to the netwatch user. So the server can still follow one in a sticky directory that anyone can write to. Devices, named pipes and sockets keep their owner. Setting the owner on an open file that is not a directory is built only for Linux. prepare-data-dir runs only in the Linux image, and macOS has neither O_PATH nor AT_EMPTY_PATH. On any other system the package still builds, and prepare-data-dir fails at the first file that is not a directory.
• 2026-10-18: from a trusted proxy, the backend takes the client address from the right-most X-Forwarded-For entry that is not in TRUSTED_PROXIES (#144). It took the left-most entry, which the client writes itself behind a proxy that appends to the header. So a client could choose the address the rate limit counts and the request log records. An entry written as an address with a port is read as that address. The direct peer is used when the header holds only trusted addresses, or when the search meets an entry that is not an address at all, such as unknown. X-Real-IP is read only without X-Forwarded-For, and every X-Forwarded-For header line is read, not only the first. The image's nginx, and the TRUSTED_PROXIES=127.0.0.1/32 it starts the backend with, are unchanged.
• 2026-10-18: after an interval change, the summary's Reachable slot reads --/-- in white, as at page load, until the second round since the change ends (#140). It read 0/26 in red from the change on, as no WAN host had a check recorded, for up to two intervals with the network fine.
• 2026-10-14: stopping the server waits for a report file write already under way, then runs the final flush (#122). Such a write is the periodic flush's, or the one Append starts when 10 MiB are waiting. It used to return at once, and the process exiting cut the write off, losing its reports. A write under way that fails puts its reports back in memory, so the final flush writes them. The stop fails, and the process exits non-zero, only if that flush fails too.
• 2026-10-14: make test no longer runs the frontend's production build, which the tests do not need (#118). yarn build moved out of the stage that runs the frontend's unit tests, now named frontend-test, into a new frontend-build stage. That stage starts from frontend-test and, like the builder stage, waits on the lint and test phases, so the image still cannot be built unless they pass. The runtime stage copies dist/ from it, and the git that vite.config.js reads the commit with is installed there. No build cache was added.
• 2026-10-14: opening the debug log no longer raises the measured latencies (#131). With the panel open, each line written drew the panel again with every line before it, up to 1000. A check that ends while the page draws is timed only when the drawing is done. Each line is now added at the end of the open panel, and its first line is dropped once it holds more than 1000. It scrolls to its newest line once a frame rather than once a line. The whole panel is drawn only when it is opened.
• 2026-10-14: unset, the backend's own TRUSTED_PROXIES trusts 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16, and no longer loopback (#129). A list replaces that. Set and empty, it trusts no proxy; it used to get the default, as viper reads an empty environment variable as unset. A value it cannot parse still stops the start, and the image still starts the backend with TRUSTED_PROXIES=127.0.0.1/32.
• 2026-10-14: while report files cannot be written, POST /api/v1/reports answers 500, and storing recovers once writing works again, with no restart (#123). A failed write puts its reports back in the report buffer, still counted against DATA_DIR_MAX_BYTES, where they used to be lost. The next flush, within a minute, tries them again. Until a write succeeds, every report is refused and nothing of it is kept, so the buffer does not grow while writes fail.
• 2026-10-14: a request whose handler panics is logged with status 500 and counted with code 500 in the metrics, not 200 (#125). Recoverer is still the first middleware. Write500OnPanic sits after the request log and after the metrics. It writes the 500 as the panic passes and lets the panic go on to Sentry and Recoverer. Recoverer now writes a 500 only when no status was written.
• 2026-10-14: with TRUSTED_PROXIES unset, the container's nginx trusts the three private ranges of RFC 1918, 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16 (#119), as sneak ruled on 2026-09-22 for every app. It trusted no proxy before. A list replaces them, and set and empty it trusts none. The backend inside the container still trusts only 127.0.0.1/32.
• 2026-10-14: POST /api/v1/reports takes only a report sent with Content-Type: application/json, parameters such as charset allowed (#126). It refuses any other with 415 and the usual error body, storing nothing. A page on another origin can send a text/plain POST without a CORS preflight, so any site could make its visitors' browsers store reports. A JSON POST waits for the preflight, which only an origin in CORS_ALLOWED_ORIGINS passes. The page already sent application/json.
• 2026-10-14: changing the interval no longer drops the samples not yet reported to the backend (#124). The interval menu resets each host with the new HostState.reset, which moves the history's entries to clearedHistory, keeping the last 100 of all it has cleared. buildReport reads both, so the next report has the samples recorded before the change that no report has delivered. Until its next check, such a host is sent with the status of its last check, not pending. The page's history still starts over. web/src/main.js exports Reporter for the tests.
• 2026-10-14: the repo has the layout REPO_POLICIES.md gives (#30). The Go module is at the root (go.mod, go.sum, .golangci.yml, cmd/netwatch-server/, internal/), so its path sneak.berlin/go/netwatch names the repository. go.mod ignores node_modules/, where npm packages can ship .go files that make tidy would otherwise read as this module's packages. The page's source is in web/ (index.html, src/), which vite.config.js makes Vite's root; the build still goes to dist/. nginx.conf and security-headers.conf are in configs/. backend/ is gone. Its LICENSE and .dockerignore are deleted, its .gitignore entries are in the root one, and its README.md is folded into the root README.md. Its scripts are in the root script/: script/build builds the server binary after the frontend, and script/run and script/clean came from backend/script/. make test, make lint, make fmt and make fmt-check do what its test, lint, fmt and fmt-check did. script/bootstrap no longer installs gcc, which only the host Go tests needed. The image and the page are unchanged.
• 2026-10-07: the files shared from sneak/prompts were its copies at commit dd4027b (#113), with this repository's own entries after the shared content in .gitignore, .dockerignore and .editorconfig. make lint and make test each built one phase of Dockerfile without the cache. lint ran golangci-lint v2.14.0 over backend/ and, through the frontend-lint stage, eslint. test ran the Go tests on the Debian Go image and, through the frontend stage, the frontend's unit tests and build. The builder stage waits on both. Without a VERSION build argument it takes the version from git describe on the .git in the build context. Neither linter nor the tests run on the host for make check. backend/.golangci.yml was no longer checked against a sha256. prettier formats only the JavaScript, CSS, HTML and Markdown, so the shared .golangci.yml stays as fetched. script/fmt and script/fmt-check put ~/.local/bin on PATH. script/frontend-lint and script/frontend-check are gone.
• 2026-10-07: the six Hetzner targets answer again, and failed checks are written to the browser console (#114). The Hetzner speed-test servers close the connection without an answer when the URL has a query string, and every check added ?_cb= and the time. Checks now fetch each target's URL as written, with cache: 'no-store' as before. Each recorded check that fails writes one console.error line, and the same line to the debug log. The line gives the target's name and URL, the time, what failed and how long the request took. A target that answers after failed checks writes one console.info line. Checks the page does not record write none, so the debug log no longer lists failures in the first round or in the recovery probe.
• 2026-10-04: the page's footer no longer says "IPv4 only" (#111). Each check is a fetch, the browser picks IPv4 or IPv6 for each WAN host, and the local targets are IPv4 addresses.
• 2026-10-04: README.md, TODO.md and test/viewport/README.md say what the tree does (#24). The README's Getting Started leads with make targets. A new Backend section says what netwatch-server stores, its routes and how the image builds and runs it; it pointed to backend/README.md for its settings. The checks are GET requests. The 26 WAN hosts, the four health states, the summary's figures and the features the list lacked are described as the page has them. The README's TODO section points here. This file's Workflow branches from next and opens the PR against next. Status says where the repo stands, and Next Step and Future Steps hold only open work, linked to its issue where one exists. The viewport test's README names Node's test runner, not vitest.
• 2026-10-04: in src/main.js, a target's min, max, median and average latency come from one list of its answers (#102). The summary's figures use the same function, so the median is written once. The latency color limits are one table in CONFIG, read by both the figure's and the sparkline's color. The health thresholds, the debug log's length and the gateway check's timeout are CONFIG entries too. So are the recovery probe's host count and interval, how often the rows are sorted, and the delay before the first sparkline resize. Nothing the page does or shows changed; the footer's color legend still writes the limits out as text.
• 2026-10-04: password guesses at /metrics are rate limited (#104). Each client address, resolved through TRUSTED_PROXIES as for reports, may make 60 requests to /metrics a minute, counted by go-chi/httprate apart from its reports. Past that it gets 429 and its basic auth credentials are not checked. The limit is a constant in internal/server/routes.go.
• 2026-10-04: the backend reports errors to Sentry (#95). With SENTRY_DSN set, it sets up sentry-go with the release netwatch-server- and its version. It reports each panic in a handler through sentryhttp, the last middleware every request goes through. sentryhttp panics again, so the request still gets the 500 from the panic recovery. On shutdown it waits up to 2 seconds for Sentry to finish sending. A DSN Sentry refuses stops the start with an error naming SENTRY_DSN. With it empty, Sentry is not set up and nothing is sent to it.
• 2026-10-04: a target's name and URL and a debug log message show as the characters they are and are never read as HTML (#29). A host row escapes the name and URL it writes into its markup, and the debug log sets each line as text. README.md no longer calls CONFIG frozen: the interval menu sets its updateInterval, and the values computed from it follow. AppState declares the recovery probe's two properties. The sparkline axis functions lose the parameters they did not use, and the comment on a target's history names both kinds of entry it holds.
• 2026-10-04: a dependency can be added without running yarn or go by hand (#45). make add-dependency PACKAGE=<name>@<version> shims to the new script/add-dependency, which runs yarn add --dev, so package.json and yarn.lock change together, then yarn install --frozen-lockfile. The same command moves a package to another version. make tidy shims to the new script/tidy, which runs go mod tidy. A Go module is added by importing it, or moved by editing its require line, then make tidy. script/bootstrap still installs with --frozen-lockfile.
• 2026-10-04: the backend serves Prometheus metrics (#94). With METRICS_USERNAME and METRICS_PASSWORD both set, it records request duration and response size through go-http-metrics. It serves them, with Go's runtime and process metrics, at GET /metrics behind basic auth with those credentials. nginx passes /metrics to it as it does /api/. With neither set there are no metrics and /metrics is 404. One without the other stops the start with an error naming both, and so does a METRICS_USERNAME containing :. Only requests that reach the health check or POST /api/v1/reports are recorded, not /metrics itself and not every request as GO_HTTP_SERVER_CONVENTIONS.md shows. The labels are the request's path and method, which clients can make up without end. For that, POST /api/v1/reports is registered by its full path instead of inside a /api/v1 route group.
• 2026-10-04: script/ and Makefile follow the org models (#28). make dev shims to the new script/dev, the Vite dev server, and the new make build to script/build, the frontend production build. .prettierignore no longer left out backend/, so make fmt and make fmt-check covered backend/README.md; it left out backend/.golangci.yml by name. script/install-precommit and the date on script/bootstrap's pins are the org model again. script/bootstrap, script/fmt and script/fmt-check each say in a comment why they differ from it.
• 2026-10-04: a frontend build on Node 26 or newer no longer prints Node's warning that module.register() is deprecated (#32). The build in Dockerfile runs on Node 22, which never printed it. The call was in @tailwindcss/node, which @tailwindcss/vite brings in at its own exact version. tailwind 4.3.1 calls module.registerHooks() instead where Node has it. yarn.lock has @tailwindcss/vite and tailwindcss at 4.3.3, inside the ranges package.json already allowed, and tailwind's own dependencies moved with them. The built CSS changed only in how it is written out, in tailwind's Firefox focus-ring rule, and in tailwind's default sans-serif font list, which the page does not use.
• 2026-10-03: the frontend's unit tests cover what the page computes (#21). They cover humanDuration, and the latency colours of a figure and of a sparkline either side of each boundary. They cover a target's min, max, average and median latency, and each of the four health states either side of its thresholds. package.json has a test script, so yarn run test and npm run test run them, and script/frontend-test runs it. src/main.js exports humanDuration, HostState, latencyClass and latencyHex for the tests; nothing it does changed.
• 2026-10-03: the backend's logs are one stream (#27). fx logs its own steps of starting and stopping through the backend's logger. So off a terminal, every line the backend and fx write is JSON; fx used to write plain text to stderr. A config file that is found but cannot be read stops the start with its error, logged as JSON like a bad setting. It used to end in a Go panic. The backend logs its name, version and architecture once at start. The health check's uptime keys are uptime_seconds and uptime_human; its path, content type, "status":"ok" and 200 are unchanged. SENTRY_DSN, METRICS_USERNAME and METRICS_PASSWORD were still read and unused until #94 and #95 wired them up.
• 2026-10-03: the frontend has a real linter (#47, and item 2 of #28): eslint with its recommended rules, set in eslint.config.js, runs in a new frontend-lint stage of Dockerfile. script/lint builds the lint stages and runs no linter on the host. script/bootstrap wants node 22.13.0 or newer, as eslint 10 does.
• 2026-10-03: the tap-target check in make frontend-viewport-test expects one visible pin button per WAN host row (#46), where it expected at least 10 of the 26. Pin buttons missing from only some rows now fail it. The host row count the test gathers, which the app-rendered check also reads, counts only the WAN host rows: the local host rows have no pin button.
• 2026-10-03: each target's row shows its result as soon as its check ends (#91). Every row waited for the round's slowest check, up to 24 seconds at a 30-second interval. Every row is still redrawn, and sorting, the summary, the health box and offline detection still run, once, when the round's last check ends. So no row reads "paused" after a pause and resume during the round. A check that ends after the user pauses or after its round is given up shows nothing. The first round is still discarded as a whole.
• 2026-10-03: root no longer acts outside /data when it prepares DATA_DIR (#80). bin/entrypoint.sh runs netwatch-server prepare-data-dir, which refuses a DATA_DIR that is not /data or a path below it written in full. Then it creates DATA_DIR, gives /data and everything in it to netwatch and sets the modes, all through a Go os.Root opened on /data. That refuses any path leading out of /data. So no symbolic link, already there or swapped in by a host process during the start, can make root create or change anything elsewhere. DATA_DIR=/etc no longer gives /etc to netwatch. The README.md section "Running under upaas" says which values are accepted.
• 2026-10-03: DATA_DIR_MAX_BYTES is how much of the report files is kept (#54). When a report would take them past it, the oldest report files are deleted to make room, each deletion logged. At start, files already past it are deleted the same way. A file still being written is never deleted. A report is refused with 507 only when the reports waiting to be written fill the cap on their own, and then no file is deleted. A file that cannot be deleted still counts until the next start; one already deleted by hand counts as freed.
• 2026-10-03: backend/script/lint said what went wrong with its .golangci.yml check (#34). On a hash mismatch it said to compare the file with the org standard. If they differed, restore the org standard. If they were the same, the org standard had changed, so update GOLANGCI_CONFIG_SHA256 in that script. It used to say only to restore the file, which loops once the org standard itself has moved. A missing .golangci.yml, and a sha256sum that is missing or prints no hash, each got their own message instead of being reported as a mismatch.
• 2026-10-03: the Go tests run with the race detector and coverage (#88): go test -timeout 30s -race -cover ./... and, if that fails, the same again with -v, then a failing exit. Go's -timeout bounds the tests, not their compile. The race detector needs a C compiler; the binary is still built with CGO_ENABLED=0. New tests cover the health check's answer, a valid report's answer, a report file's exact contents, and the flush when the buffer reaches 10 MiB.
• 2026-10-03: each target check times out after 80% of the refresh interval (#78), 24 seconds at 30 seconds, where it was capped at 3 seconds. A round started early gives up the last round's checks if they are still waiting, so rounds never overlap. A round starts early after an interval change, or when the recovery probe finds a target answering. The recovery probe gives up its own checks after half a second. The frontend has its first unit tests, run by script/frontend-test with Node's built-in test runner. For them, index.html links src/styles.css, which src/main.js used to import.
• 2026-09-29: the container sets up its own data directory (#75). bin/entrypoint.sh, still as root, created DATA_DIR if missing and gave it and /data to the netwatch user with mode 750 before starting the backend as that user. So an empty host directory owned by root, or one holding files from another uid, works with no step on the host. It stopped the start when a symbolic link was on the path to DATA_DIR, since root would change whatever the link points to. The README.md first-run step that created and chowned the host directory is gone, and the image no longer sets that ownership at build time.
• 2026-09-29: CI can no longer pass on checks that did not run (#37). script/cibuild became the org model, byte for byte. It runs script/bootstrap and script/check, then builds the image with the version from git describe as the VERSION build argument. It used to be a plain docker build . whose check steps could come from the build cache. The workflow put ~/.local/bin, where bootstrap links what it installs, on the step's PATH. bootstrap installs its pinned node when the installed one is older than the frontend's dependencies need.
• 2026-09-29: backend/.golangci.yml was re-vendored from sneak/prompts (#41). gomodguard, deprecated in golangci-lint v2.12.0, is disabled and its successor gomodguard_v2 enabled with the org block list, so lint runs print no deprecation warning. The file also turns depguard on with its test-support rule, which keeps net/http/httptest out of non-test code; netwatch adds no entries of its own to that rule.
• 2026-09-29: nginx sends the security headers REPO_POLICIES.md requires on every response (#18), errors and /assets/ included. On what it passes on from the backend, it drops the backend's own copies, so each header goes out once. They live in security-headers.conf, which nginx.conf includes. The content security policy allows no inline script or style, so the status dot's grey in src/main.js is a class. connect-src is * because probed hosts redirect to others, and the browser checks each redirect against it.
• 2026-09-29: the request log is bounded (#60). Each field it writes from the request is cut to 128 bytes, the bound the report handler already used. The fields are the method, URL, protocol, User-Agent, Referer, request ID and client address. chi takes the request ID from the client's X-Request-Id header. One request could put about 1 MiB per field into a log line. That bound and its helper live in the logger package, shared by both.
• 2026-09-29: nginx takes the client address from X-Forwarded-For only on requests from the reverse proxies named in the container's TRUSTED_PROXIES (#64), and at the time by default from none. It trusted every RFC1918 address before, so a client could write a new address on each request and escape the rate limit. bin/entrypoint.sh writes one set_real_ip_from line per entry into /etc/nginx/trusted-proxies.conf, which nginx.conf includes. It refuses an entry that is not an IP address or CIDR, as netwatch-server check-cidr finds. It starts the backend with TRUSTED_PROXIES=127.0.0.1/32, since nginx is its only client.
• 2026-09-29: report file names can no longer collide (#61). Each is reports-<timestamp>-<number>.jsonl.zst, where the number goes up by one for each file the server starts to write. So two flushes in the same millisecond each get a file of their own; the second one used to fail. A flush for size and the final flush at shutdown can fall in the same millisecond. A failed write uses up its number.
• 2026-09-29: ready to run under upaas (#59). The image has a HEALTHCHECK that requests /.well-known/healthcheck through nginx on the port from PORT. The backend no longer reads a bad PORT as 0 or a bad DEBUG as false. Those, and a BIND_ADDRESS that is not an IP address, stop it from starting with an error naming the variable. The limits, CORS_ALLOWED_ORIGINS and, now by name, TRUSTED_PROXIES already did. bin/entrypoint.sh also refuses a PORT outside 1 to 65535, and 8081, where the backend listens inside the container, naming PORT. README.md got a "Running under upaas" section.
• 2026-09-29: nginx listens on PORT (#26), 8080 when unset or empty. The nginx image renders nginx.conf as a template at container start, filling in PORT and no other variable. bin/entrypoint.sh refuses to start when PORT is not digits only. server_tokens off keeps the nginx version out of responses. script/frontend-viewport-test renders the template the same way. Gzip and a 50x.html error page are not added.
• 2026-09-29: the report endpoint is bounded (#20). POST /api/v1/reports still needs no credentials. Each client address, as resolved through TRUSTED_PROXIES, may send REPORTS_PER_MINUTE (default 60) reports a minute, counted by go-chi/httprate, and past that gets 429 with Retry-After. The report files in DATA_DIR, counted from start with those already there, may total at most DATA_DIR_MAX_BYTES (default 1 GiB); past that, reports got 507. The wildcard CORS is gone: no CORS headers unless CORS_ALLOWED_ORIGINS lists origins. An entry that is not a plain scheme://host[:port] origin, * included, stops the server from starting. Deleting report files freed room only at the next start, until #54.
• 2026-09-28: one container image (#52). The root Dockerfile builds the only image, and Dockerfile.backend is gone. nginx serves the frontend on port 8080 and proxies /api/ and /.well-known/healthcheck to the backend, which listens on 127.0.0.1:8081 in the same container. The new BIND_ADDRESS setting sets its listen address. bin/entrypoint.sh starts both, passes TERM and INT on to both, and exits non-zero when either exits on its own. The backend runs as user netwatch and stores reports on the /data volume. script/docker is the org model again.
• 2026-09-28: one gate (#16). The root make check covers the Go backend as well as the frontend, and the pre-commit hook with it. The backend moved onto scripts-to-rule-them-all: backend/script/*, backend/Makefile as shims, its duplicate hook installer removed. script/cibuild built both images and was the workflow's only build step. The root make lint ran golangci-lint only in Docker, by building the lint stage of Dockerfile.backend without the cache. script/bootstrap installed the pinned Go unless the installed one was at least what backend/go.mod asked for. It links what it installs into ~/.local/bin without replacing anything it did not create, and installs no linter. When VERSION is unset or empty, the backend binary's version falls back to git describe inside a git checkout, then to dev.
• 2026-09-28: frontend reporting client (#53). A Reporter class posts collected samples to /api/v1/reports every reportInterval (default 60s) as a per-host delta. The report-building step is a pure exported function of host state. A per-host mark advances only on a delivered POST. At most one report POST is pending at a time, and it is abandoned after half the interval. So a slow POST never overlaps the next report, and a mark never moves backwards. The samples of an abandoned POST are sent again at the next interval, so a backend that stored them but answered late receives them twice. The per-browser client id works in insecure (plain-HTTP) contexts. vite.config.js proxies /api to the local backend for yarn dev.
• 2026-09-28: report ingest correctness (#23). A storage failure returns 500 instead of a false ok. An oversize body returns 413, apart from malformed JSON, which stays 400. A MaxBodyBytes middleware caps every route, not just the report route. The raw attacker-controlled geo blob is no longer logged, only its length, and client_id, timestamp and decode error text are length-bounded before logging. A decodeJSON handler helper was added. Panic recovery routes the stack through slog instead of chi's plain-text stderr. Writing a report file returns its error, so a failed final flush on shutdown makes the process exit non-zero instead of losing the buffered reports silently.
• 2026-09-21: shutdown lifecycle correctness. The process shuts down through fx instead of os.Exit, so every component's OnStop runs and buffered reports are flushed to disk on SIGTERM. Before, a full flush window of telemetry was lost on every restart. The http.Server is built before its serving goroutine starts, so shutdown can no longer race or nil-deref it. A listen failure exits non-zero via fx.Shutdowner. reportbuf's OnStop is idempotent. writeTimeout exceeds the chi per-request budget, so that budget is reachable. Dead startupTime, exitCode and cancelFunc fields were removed.
• 2026-09-21: backend HTTP hardening (#19): ReadHeaderTimeout and IdleTimeout on the server, and a SecurityHeaders middleware (HSTS, tight CSP, frame, sniff, referrer and permissions headers) registered before CORS. X-Forwarded-For and X-Real-IP are honoured only from a TRUSTED_PROXIES allowlist, at the time loopback plus RFC1918 by default.
• 2026-08-10: adopted the org-standard backend/.golangci.yml verbatim and moved the pinned golangci-lint from v2.7.2 to v2.12.2; the lint stage of Dockerfile.backend pinned the golangci/golangci-lint:v2.12.2 image by digest. The previous config declared version: "2" but used v1 schema keys, so every threshold in it was inert and its green result meant nothing. backend/Makefile's lint target asserted the config's sha256 against the canonical file first, so drift from the org standard failed the build instead of silently degrading to defaults.
• 2026-08-10: every interactive control meets the 44x44 CSS px minimum tap target: .pin-btn, #interval-select, the debug-log label and, on narrow viewports, #pause-btn. The pin button's hit area grows via matching negative margins, so its layout footprint and row density are unchanged.
• 2026-08-10: the per-host status line wraps below the 768px breakpoint instead of forcing horizontal page scroll at 320px.
• 2026-08-09: Dockerfile.backend was reworked to the Go multistage lint-stage pattern. It got a separate lint stage on the hash-pinned golangci/golangci-lint image, and a COPY --from=lint stage dependency. The build became a CGO_ENABLED=0 static build driven by ARG VERSION, with no more COPY .git.
• 2026-08-09: dotfile compliance. backend/.editorconfig moved to the repo root so root = true covers the frontend too. .gitignore became the org model (OS, editor, node, and environment/secrets sections) plus this repo's dist/ and *.log. .env, .env.*, *.pem and *.key are ignored repo-wide, not just under backend/. Excluding .git from .dockerignore stayed deferred: both images read git metadata at build time.
• 2026-08-09: automated responsive-layout test (make frontend-viewport-test): a digest-pinned headless Chrome driven over CDP against the built dist/, at viewport widths derived from the breakpoints in src/styles.css (#13). The tap-target and host-row checks each fail when they measured nothing. The overflow, viewport-edge and clipped-text checks have no such guard of their own and rely on the app-rendered check, which fails the run when the app did not render. It found two real layout defects, filed as #42 and #43.
• 2026-07-07: adopted scripts-to-rule-them-all: script/ entrypoints, Makefile shims, README Entrypoints section.
• 2026-02-27: backend with buffered zstd-compressed report storage; CI workflow and backend repo standard files; backend Dockerfile fixed (Go 1.25, golangci-lint) and moved to repo root (feat/reportbuf-storage).
• 2026-02-26: host row layout redesigned with CSS grid; overflow and spacing fixes; nginx config extracted; port hardcoded to 8080.
• 2026-02-26: debug log panel, median stats, recovery probe, Docker build fix, S3 Singapore endpoint added.
• 2026-02-23: summary box redesign, host pinning, local and UTC clocks, checks counter.
• 2026-02-23: hosts sorted by latency; GET instead of HEAD for latency; timeout derived from interval; Hetzner regional endpoints; 3s interval.
• 2026-01-29: initial NetWatch network latency monitor.

Future Steps

• A backend test that posts a report to POST /api/v1/reports and checks the compressed file it is written to
• A backend route that decompresses the stored reports and answers queries on them
• Prometheus metrics for the backend's in-memory buffer: its size, the number of flushes and the number of reports
• A configurable host list (an environment variable or a config file)
• Export of the latency history (CSV or JSON)
• A notification when the health status changes to DEGRADED