main's tree, with next merged on 26 October, as one commit onto prod, on sneak's order. Model: opus-5-5 Co-authored-by: clawbot <clawbot@noreply.example.org>
This commit was merged in pull request #198.
This commit is contained in:
@@ -1,217 +1,709 @@
|
||||
# Workflow
|
||||
|
||||
- branch (from `main`)
|
||||
- branch from `next`
|
||||
- do the work in Next Step
|
||||
- move Next Step to the top of Completed Steps
|
||||
- move the top item of Future Steps into Next Step
|
||||
- commit (`TODO.md` changes in the same commit as the work)
|
||||
- merge to `main` if the branch is not protected, otherwise open a PR
|
||||
- push
|
||||
- push the branch and open a PR against `next`
|
||||
|
||||
# Status
|
||||
|
||||
pre-1.0. No git tags. `feat/reportbuf-storage` is merged; the backend, the CI
|
||||
workflow, and the backend repo standard files are all on `main`. Frontend and
|
||||
backend are both functional. Working toward the 1.0.0 milestone by closing the
|
||||
remaining repo-compliance issues on the tracker.
|
||||
pre-1.0. Release candidates are tagged on `next`, the first being `v1.0.0-rc1`;
|
||||
`git tag --merged origin/next` lists them. `main` is the stable branch and
|
||||
`next` the development branch, which every PR targets. The frontend and the Go
|
||||
backend ship as one Docker image. The Gitea workflow
|
||||
`.gitea/workflows/check.yml` runs `script/cibuild` on every push, and cancels
|
||||
the older run on the same branch. Working toward 1.0.0.
|
||||
|
||||
# Next Step
|
||||
|
||||
Confirm the `.gitea/workflows/check.yml` run is green (main always green
|
||||
policy). The workflow file is already on `main`; what is unverified is that its
|
||||
latest run passes.
|
||||
Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
||||
`make check`, as it needs Docker and takes minutes.
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-09-29: the container sets up its own data directory (issue #75):
|
||||
`bin/entrypoint.sh`, still as root, creates `DATA_DIR` if missing and gives it
|
||||
and `/data` to the `netwatch` user with mode 750 before starting the backend
|
||||
as that user, so an empty host directory owned by root, or one holding files
|
||||
from another uid, works with no step on the host. It stops the start instead
|
||||
when a symbolic link is on the path to `DATA_DIR`, since root would change
|
||||
whatever the link points to. The `README.md` first-run step that created and
|
||||
chowned the host directory is gone, and the image no longer sets that
|
||||
ownership at build time
|
||||
- 2026-09-29: CI can no longer pass on checks that did not run (issue #37):
|
||||
`script/cibuild` is now the org model, byte for byte. It runs
|
||||
`script/bootstrap` and `script/check`, then builds the image with `--no-cache`
|
||||
and the version from `git describe` as the `VERSION` build argument, where it
|
||||
used to be a plain `docker build .` whose check steps could come from the
|
||||
build cache. The workflow puts `~/.local/bin`, where bootstrap links what it
|
||||
installs, on the step's `PATH`, and bootstrap now installs its pinned node
|
||||
when the installed one is older than the frontend's dependencies need
|
||||
- 2026-09-29: `backend/.golangci.yml` re-vendored from `sneak/prompts` (issue
|
||||
#41): `gomodguard`, deprecated in golangci-lint v2.12.0, is disabled and its
|
||||
successor `gomodguard_v2` enabled with the org block list, so lint runs print
|
||||
no deprecation warning. The new file also turns `depguard` on with its
|
||||
`test-support` rule, which keeps `net/http/httptest` out of non-test code;
|
||||
netwatch adds no entries of its own to that rule. `backend/script/lint` checks
|
||||
the new sha256
|
||||
- 2026-10-26: `README.md`, `TODO.md` and `test/viewport/README.md` are rewritten
|
||||
sentence by sentence ([#189](https://git.eeqj.de/sneak/netwatch/issues/189)).
|
||||
Each sentence reads on the first pass: one idea, under about 30 words, the
|
||||
verb early. Every statement matches the code on `next`.
|
||||
- 2026-10-26: a target whose home page answers with a redirect is checked at
|
||||
`/robots.txt` on the same host, which answers without one
|
||||
([#194](https://git.eeqj.de/sneak/netwatch/issues/194)). These are the AWS
|
||||
Console, Akamai, B2 and the 8 S3 regions. The browser followed each redirect,
|
||||
so their figures also timed the page it led to. The Google Cloud Console is
|
||||
checked at `/robots.txt` too, as its home page redirects `curl`'s requests.
|
||||
The S3 regions are checked with a HEAD request, as Chrome closes their
|
||||
connection after most GET requests. `README.md` says what a figure includes.
|
||||
- 2026-10-26: the six Hetzner targets answer wherever the page is served from
|
||||
([#191](https://git.eeqj.de/sneak/netwatch/issues/191)). They close the
|
||||
connection without an answer when the request has a `Referer` header. The
|
||||
browser sent one unless the page came with the image's
|
||||
`Referrer-Policy: no-referrer` header, which a static host and `make dev` do
|
||||
not send. Each check now sets `referrerPolicy: 'no-referrer'`.
|
||||
- 2026-10-19: the server's `OnStart` hook in `internal/server/server.go` starts
|
||||
`listenAndServe` in an anonymous function, as the Go style guide asks
|
||||
([#162](https://git.eeqj.de/sneak/netwatch/issues/162)). It was
|
||||
`go s.listenAndServe()`. Nothing the server does changed.
|
||||
- 2026-10-19: `.gitignore`, `.editorconfig` and `.golangci.yml` are the
|
||||
sneak/prompts copies at `0653579`
|
||||
([#174](https://git.eeqj.de/sneak/netwatch/issues/174)). `.gitignore` is the
|
||||
standard file followed by this repository's own entries. `.editorconfig` is
|
||||
the standard file alone, which carries the `[*.go]` section. `.golangci.yml`
|
||||
disables `canonicalheader`, as the canonical copy does: golangci-lint v2.14.0
|
||||
misses its findings at random.
|
||||
- 2026-10-19: `package.json` has no `version` field and is marked
|
||||
`"private": true` ([#158](https://git.eeqj.de/sneak/netwatch/issues/158)). It
|
||||
said `1.0.0`, a release that has not been made, and nothing read it. The git
|
||||
tag is the one version; the image and `make build` take it from
|
||||
`git describe --tags`. The package is never published.
|
||||
- 2026-10-19: Status names `v1.0.0-rc1`, the first release candidate, and says
|
||||
release candidates are tagged on `next`
|
||||
([#157](https://git.eeqj.de/sneak/netwatch/issues/157)). It said there were no
|
||||
git tags.
|
||||
- 2026-10-19: `netwatch-server` starts the server only when given no arguments
|
||||
([#179](https://git.eeqj.de/sneak/netwatch/issues/179)). `check-cidr CIDR` and
|
||||
`prepare-data-dir DATA_DIR` behave as before. Any other arguments print a
|
||||
one-line usage message naming both forms to stderr and exit 2; they used to
|
||||
start the server. Unit tests cover each case.
|
||||
- 2026-10-19: `main()` in `cmd/netwatch-server/main.go` is one call into the new
|
||||
package `internal/cli`, as the Go style guide asks
|
||||
([#160](https://git.eeqj.de/sneak/netwatch/issues/160)). `internal/cli` runs
|
||||
the `check-cidr` and `prepare-data-dir` subcommands, with the same arguments,
|
||||
messages and exit codes, and otherwise the server with the `fx` wiring.
|
||||
`Appname` and `Version` stay in `main`, where the build sets `Version`. `main`
|
||||
gives `cli.Main` stderr and the function that runs the server; the tests
|
||||
replace both. Unit tests give each subcommand input it takes and input it
|
||||
refuses.
|
||||
- 2026-10-19: when `VERSION` is unset or empty, `script/build` stamps the
|
||||
version from `git describe --tags --always --dirty`, as `script/docker` does
|
||||
([#156](https://git.eeqj.de/sneak/netwatch/issues/156)). Without `--tags` it
|
||||
skipped lightweight tags, so `make run` could report a commit where the image
|
||||
built from the same tree reported the tag.
|
||||
- 2026-10-19: `.dockerignore` keeps Go test binaries and coverage output at the
|
||||
root out of the build context, with `/*.test` and `/*.out` in the repository's
|
||||
own section ([#155](https://git.eeqj.de/sneak/netwatch/issues/155)). Such
|
||||
files were copied into the lint, test and build stages. The entries match only
|
||||
at the root, so the same files in a subdirectory still reach the image.
|
||||
- 2026-10-19: `.gitea/workflows/check.yml` is the sneak/prompts copy at
|
||||
`0653579` ([#153](https://git.eeqj.de/sneak/netwatch/issues/153)). Its
|
||||
checkout step sets `fetch-depth: 0`, so CI clones all history and tags. The
|
||||
image built from a tagged commit then gets the tag as its version; it got the
|
||||
short commit. The step also sets `persist-credentials: false`, so the job's
|
||||
token is not left in `.git/config`. A new push cancels the older run on the
|
||||
same branch, and a run stops after 20 minutes.
|
||||
- 2026-10-19: `script/bootstrap` keeps the `yarn` on `PATH` only when
|
||||
`yarn --version` is exactly `YARN_VERSION`; otherwise it installs that version
|
||||
([#154](https://git.eeqj.de/sneak/netwatch/issues/154)). Any yarn used to be
|
||||
kept. It uses the `node` on `PATH` only when that node has the major version
|
||||
of `NODE_VERSION` and is at least `NODE_MIN_VERSION`; a node 24 was kept
|
||||
before. After installing node or yarn it looks the tool up through `PATH`
|
||||
again and stops if that one fails the same check, as it does for Go. It prints
|
||||
the node and yarn versions it uses. Every script that runs yarn on the host,
|
||||
except `script/bootstrap`, runs it through the new `script/yarn`. That script
|
||||
puts `~/.local/bin` first on `PATH`. It uses the first `node` there when it
|
||||
passes the same test, and otherwise runs yarn under the pinned node in nvm.
|
||||
`make build` ran yarn under whatever node was first on `PATH`.
|
||||
- 2026-10-19: `bin/entrypoint.sh` has each step in a function, and a `main` at
|
||||
the end calls them in order, as the code style guide's shell section asks
|
||||
([#163](https://git.eeqj.de/sneak/netwatch/issues/163)). It is still POSIX sh
|
||||
with no `set -e`, and does what it did, with the same messages and exit codes.
|
||||
- 2026-10-19: while report files cannot be written, the health check answers 503
|
||||
with `"status":"error"` and the same fields, so the image's `HEALTHCHECK`
|
||||
reports the container `unhealthy`
|
||||
([#142](https://git.eeqj.de/sneak/netwatch/issues/142)). It answered 200
|
||||
whatever happened, so the platform saw a healthy container while every report
|
||||
was refused with 500. It answers 200 at start, before any write has failed,
|
||||
and again from the first write that succeeds.
|
||||
- 2026-10-19: report file writes no longer overlap
|
||||
([#150](https://git.eeqj.de/sneak/netwatch/issues/150)), so the reports of
|
||||
failed writes are written in the order they arrived. Two writes that failed at
|
||||
the same time put their reports back in the wrong order when the older failed
|
||||
first. One write runs at a time. Each takes the waiting reports only when it
|
||||
starts, after a failed write has put its own back. The write that `Append`
|
||||
starts at 10 MiB waits for any write under way, and `Append` starts no other
|
||||
while it waits. Files written since the start are deleted to make room in the
|
||||
order they were written, after those found at start.
|
||||
- 2026-10-19: `DEBUG` unset or empty leaves debug logging off, and any other
|
||||
value, `false` and `0` included, turns it on, as the code style guide says
|
||||
([#159](https://git.eeqj.de/sneak/netwatch/issues/159)). No value of it stops
|
||||
the start any more. Before, only `1`, `t`, `T`, `TRUE`, `true`, `True`, `0`,
|
||||
`f`, `F`, `FALSE`, `false` and `False` were accepted. `README.md` no longer
|
||||
gives `false` as its default.
|
||||
- 2026-10-19: check builds keep Go's modules and compiled packages between runs
|
||||
([#118](https://git.eeqj.de/sneak/netwatch/issues/118)), on sneak's rulings in
|
||||
[prompts#113](https://git.eeqj.de/sneak/prompts/issues/113) and
|
||||
[prompts#130](https://git.eeqj.de/sneak/prompts/issues/130).
|
||||
`REPO_POLICIES.md`, `script/test`, `script/lint`, `script/docker` and
|
||||
`script/cibuild` are the sneak/prompts copies at `0653579`. The scripts pass a
|
||||
new `CACHEBUST` value where they passed `--no-cache`, and the lint and test
|
||||
builds write no image. In each stage of `Dockerfile`, `ARG CACHEBUST` comes
|
||||
after the system packages and the Go module download. Every step that lints,
|
||||
tests or compiles comes after it, as does the node stages' `yarn install`.
|
||||
Go's build cache and golangci-lint's are cache mounts, and both `go test` runs
|
||||
carry `-count=1`. The test phase's Go caches used to be in memory, empty on
|
||||
every build.
|
||||
- 2026-10-19: the three unit tests of a discarded check also check that the
|
||||
host's history is still empty, its status still `pending`, and the console
|
||||
untouched ([#151](https://git.eeqj.de/sneak/netwatch/issues/151)). The
|
||||
discarded cases are a round given up, a pause, and the first round. The tests
|
||||
checked only the row. So a change that recorded the result and skipped only
|
||||
the row would have kept them passing.
|
||||
- 2026-10-18: run on its own, the backend stops at start when `DATA_DIR` cannot
|
||||
be created or read, or when no file can be created in it
|
||||
([#148](https://git.eeqj.de/sneak/netwatch/issues/148)). The error names
|
||||
`DATA_DIR` and its value. The first two cases named only the path, and the
|
||||
third was found only at the first write of a report file. At start the server
|
||||
creates a file in `DATA_DIR` and removes it. The report file number is an
|
||||
`atomic.Int64`, as the Go style guide asks for int types
|
||||
([#161](https://git.eeqj.de/sneak/netwatch/issues/161)); file names are
|
||||
unchanged.
|
||||
- 2026-10-18: the image's nginx sends the page with `Cache-Control: no-cache`
|
||||
([#141](https://git.eeqj.de/sneak/netwatch/issues/141)), so a browser that
|
||||
keeps a copy checks with the server that it is current before using it again.
|
||||
The page went out with no `Cache-Control`. So a browser could keep it after a
|
||||
deploy, ask for files under `/assets/` that the new build no longer has, and
|
||||
show an empty page. A path that `location /` answers with the page through its
|
||||
`try_files` fallback gets the same header. `location /` includes the security
|
||||
headers again, as `/assets/` does, since its own `add_header` drops the
|
||||
server's. `/assets/` keeps its immutable caching.
|
||||
`make frontend-viewport-test` fails unless `/` and a path that falls back to
|
||||
the page carry `Cache-Control: no-cache` and `Content-Security-Policy`.
|
||||
- 2026-10-18: `README.md` says what the page loses while reports are refused or
|
||||
the page is paused ([#145](https://git.eeqj.de/sneak/netwatch/issues/145)). It
|
||||
says how fast one address can replace the stored reports
|
||||
([#146](https://git.eeqj.de/sneak/netwatch/issues/146)). It says that in the
|
||||
image nginx answers a body over 1 MiB itself
|
||||
([#147](https://git.eeqj.de/sneak/netwatch/issues/147)). nginx's
|
||||
`client_max_body_size` is 1 MiB by default, so in the image it answers a
|
||||
larger body with 413 and its own HTML page, and the backend never sees it. The
|
||||
reporter, the body limit and nginx are unchanged.
|
||||
- 2026-10-18: a browser that cannot format the header's clocks still probes
|
||||
([#149](https://git.eeqj.de/sneak/netwatch/issues/149)). The clocks name the
|
||||
local time zone through `Intl.DateTimeFormat`'s `shortGeneric` option, which
|
||||
Chrome before 95, Firefox before 91 and Safari before 15.4 reject with an
|
||||
error. It came before the first round started, so such a browser never checked
|
||||
a target. The clocks now start last, after the rounds, and an error formatting
|
||||
them leaves both blank. The footer's color legend reads `>=500ms` for red,
|
||||
where it read `>500ms`
|
||||
([#152](https://git.eeqj.de/sneak/netwatch/issues/152)).
|
||||
- 2026-10-18: `netwatch-server prepare-data-dir` no longer changes the owner of
|
||||
a file outside `/data` through a hard link in the volume
|
||||
([#143](https://git.eeqj.de/sneak/netwatch/issues/143)). A file with a second
|
||||
name (hard link) keeps its owner, and the start logs its path. Owners are set
|
||||
on the opened directory, file or link, never by name, so a hard link swapped
|
||||
in after the check is caught too. A symbolic link is opened without following
|
||||
it, and the link itself is given to the netwatch user. So the server can still
|
||||
follow one in a sticky directory that anyone can write to. Devices, named
|
||||
pipes and sockets keep their owner. Setting the owner on an open file that is
|
||||
not a directory is built only for Linux. `prepare-data-dir` runs only in the
|
||||
Linux image, and macOS has neither `O_PATH` nor `AT_EMPTY_PATH`. On any other
|
||||
system the package still builds, and `prepare-data-dir` fails at the first
|
||||
file that is not a directory.
|
||||
- 2026-10-18: from a trusted proxy, the backend takes the client address from
|
||||
the right-most `X-Forwarded-For` entry that is not in `TRUSTED_PROXIES`
|
||||
([#144](https://git.eeqj.de/sneak/netwatch/issues/144)). It took the left-most
|
||||
entry, which the client writes itself behind a proxy that appends to the
|
||||
header. So a client could choose the address the rate limit counts and the
|
||||
request log records. An entry written as an address with a port is read as
|
||||
that address. The direct peer is used when the header holds only trusted
|
||||
addresses, or when the search meets an entry that is not an address at all,
|
||||
such as `unknown`. `X-Real-IP` is read only without `X-Forwarded-For`, and
|
||||
every `X-Forwarded-For` header line is read, not only the first. The image's
|
||||
nginx, and the `TRUSTED_PROXIES=127.0.0.1/32` it starts the backend with, are
|
||||
unchanged.
|
||||
- 2026-10-18: after an interval change, the summary's Reachable slot reads
|
||||
`--/--` in white, as at page load, until the second round since the change
|
||||
ends ([#140](https://git.eeqj.de/sneak/netwatch/issues/140)). It read 0/26 in
|
||||
red from the change on, as no WAN host had a check recorded, for up to two
|
||||
intervals with the network fine.
|
||||
- 2026-10-14: stopping the server waits for a report file write already under
|
||||
way, then runs the final flush
|
||||
([#122](https://git.eeqj.de/sneak/netwatch/issues/122)). Such a write is the
|
||||
periodic flush's, or the one `Append` starts when 10 MiB are waiting. It used
|
||||
to return at once, and the process exiting cut the write off, losing its
|
||||
reports. A write under way that fails puts its reports back in memory, so the
|
||||
final flush writes them. The stop fails, and the process exits non-zero, only
|
||||
if that flush fails too.
|
||||
- 2026-10-14: `make test` no longer runs the frontend's production build, which
|
||||
the tests do not need ([#118](https://git.eeqj.de/sneak/netwatch/issues/118)).
|
||||
`yarn build` moved out of the stage that runs the frontend's unit tests, now
|
||||
named `frontend-test`, into a new `frontend-build` stage. That stage starts
|
||||
from `frontend-test` and, like the `builder` stage, waits on the lint and test
|
||||
phases, so the image still cannot be built unless they pass. The runtime stage
|
||||
copies `dist/` from it, and the `git` that `vite.config.js` reads the commit
|
||||
with is installed there. No build cache was added.
|
||||
- 2026-10-14: opening the debug log no longer raises the measured latencies
|
||||
([#131](https://git.eeqj.de/sneak/netwatch/issues/131)). With the panel open,
|
||||
each line written drew the panel again with every line before it, up to 1000.
|
||||
A check that ends while the page draws is timed only when the drawing is done.
|
||||
Each line is now added at the end of the open panel, and its first line is
|
||||
dropped once it holds more than 1000. It scrolls to its newest line once a
|
||||
frame rather than once a line. The whole panel is drawn only when it is
|
||||
opened.
|
||||
- 2026-10-14: unset, the backend's own `TRUSTED_PROXIES` trusts `10.0.0.0/8`,
|
||||
`172.16.0.0/12` and `192.168.0.0/16`, and no longer loopback
|
||||
([#129](https://git.eeqj.de/sneak/netwatch/issues/129)). A list replaces that.
|
||||
Set and empty, it trusts no proxy; it used to get the default, as viper reads
|
||||
an empty environment variable as unset. A value it cannot parse still stops
|
||||
the start, and the image still starts the backend with
|
||||
`TRUSTED_PROXIES=127.0.0.1/32`.
|
||||
- 2026-10-14: while report files cannot be written, `POST /api/v1/reports`
|
||||
answers 500, and storing recovers once writing works again, with no restart
|
||||
([#123](https://git.eeqj.de/sneak/netwatch/issues/123)). A failed write puts
|
||||
its reports back in the report buffer, still counted against
|
||||
`DATA_DIR_MAX_BYTES`, where they used to be lost. The next flush, within a
|
||||
minute, tries them again. Until a write succeeds, every report is refused and
|
||||
nothing of it is kept, so the buffer does not grow while writes fail.
|
||||
- 2026-10-14: a request whose handler panics is logged with status 500 and
|
||||
counted with `code` 500 in the metrics, not 200
|
||||
([#125](https://git.eeqj.de/sneak/netwatch/issues/125)). `Recoverer` is still
|
||||
the first middleware. `Write500OnPanic` sits after the request log and after
|
||||
the metrics. It writes the 500 as the panic passes and lets the panic go on to
|
||||
Sentry and `Recoverer`. `Recoverer` now writes a 500 only when no status was
|
||||
written.
|
||||
- 2026-10-14: with `TRUSTED_PROXIES` unset, the container's nginx trusts the
|
||||
three private ranges of RFC 1918, `10.0.0.0/8`, `172.16.0.0/12` and
|
||||
`192.168.0.0/16` ([#119](https://git.eeqj.de/sneak/netwatch/issues/119)), as
|
||||
sneak ruled on 2026-09-22 for every app. It trusted no proxy before. A list
|
||||
replaces them, and set and empty it trusts none. The backend inside the
|
||||
container still trusts only `127.0.0.1/32`.
|
||||
- 2026-10-14: `POST /api/v1/reports` takes only a report sent with
|
||||
`Content-Type: application/json`, parameters such as `charset` allowed
|
||||
([#126](https://git.eeqj.de/sneak/netwatch/issues/126)). It refuses any other
|
||||
with 415 and the usual error body, storing nothing. A page on another origin
|
||||
can send a `text/plain` POST without a CORS preflight, so any site could make
|
||||
its visitors' browsers store reports. A JSON POST waits for the preflight,
|
||||
which only an origin in `CORS_ALLOWED_ORIGINS` passes. The page already sent
|
||||
`application/json`.
|
||||
- 2026-10-14: changing the interval no longer drops the samples not yet reported
|
||||
to the backend ([#124](https://git.eeqj.de/sneak/netwatch/issues/124)). The
|
||||
interval menu resets each host with the new `HostState.reset`, which moves the
|
||||
history's entries to `clearedHistory`, keeping the last 100 of all it has
|
||||
cleared. `buildReport` reads both, so the next report has the samples recorded
|
||||
before the change that no report has delivered. Until its next check, such a
|
||||
host is sent with the status of its last check, not `pending`. The page's
|
||||
history still starts over. `web/src/main.js` exports `Reporter` for the tests.
|
||||
- 2026-10-14: the repo has the layout `REPO_POLICIES.md` gives
|
||||
([#30](https://git.eeqj.de/sneak/netwatch/issues/30)). The Go module is at the
|
||||
root (`go.mod`, `go.sum`, `.golangci.yml`, `cmd/netwatch-server/`,
|
||||
`internal/`), so its path `sneak.berlin/go/netwatch` names the repository.
|
||||
`go.mod` ignores `node_modules/`, where npm packages can ship `.go` files that
|
||||
`make tidy` would otherwise read as this module's packages. The page's source
|
||||
is in `web/` (`index.html`, `src/`), which `vite.config.js` makes Vite's root;
|
||||
the build still goes to `dist/`. `nginx.conf` and `security-headers.conf` are
|
||||
in `configs/`. `backend/` is gone. Its `LICENSE` and `.dockerignore` are
|
||||
deleted, its `.gitignore` entries are in the root one, and its `README.md` is
|
||||
folded into the root `README.md`. Its scripts are in the root `script/`:
|
||||
`script/build` builds the server binary after the frontend, and `script/run`
|
||||
and `script/clean` came from `backend/script/`. `make test`, `make lint`,
|
||||
`make fmt` and `make fmt-check` do what its `test`, `lint`, `fmt` and
|
||||
`fmt-check` did. `script/bootstrap` no longer installs gcc, which only the
|
||||
host Go tests needed. The image and the page are unchanged.
|
||||
- 2026-10-07: the files shared from `sneak/prompts` were its copies at commit
|
||||
`dd4027b` ([#113](https://git.eeqj.de/sneak/netwatch/issues/113)), with this
|
||||
repository's own entries after the shared content in `.gitignore`,
|
||||
`.dockerignore` and `.editorconfig`. `make lint` and `make test` each built
|
||||
one phase of `Dockerfile` without the cache. `lint` ran golangci-lint v2.14.0
|
||||
over `backend/` and, through the `frontend-lint` stage, eslint. `test` ran the
|
||||
Go tests on the Debian Go image and, through the `frontend` stage, the
|
||||
frontend's unit tests and build. The builder stage waits on both. Without a
|
||||
`VERSION` build argument it takes the version from `git describe` on the
|
||||
`.git` in the build context. Neither linter nor the tests run on the host for
|
||||
`make check`. `backend/.golangci.yml` was no longer checked against a sha256.
|
||||
prettier formats only the JavaScript, CSS, HTML and Markdown, so the shared
|
||||
`.golangci.yml` stays as fetched. `script/fmt` and `script/fmt-check` put
|
||||
`~/.local/bin` on `PATH`. `script/frontend-lint` and `script/frontend-check`
|
||||
are gone.
|
||||
- 2026-10-07: the six Hetzner targets answer again, and failed checks are
|
||||
written to the browser console
|
||||
([#114](https://git.eeqj.de/sneak/netwatch/issues/114)). The Hetzner
|
||||
speed-test servers close the connection without an answer when the URL has a
|
||||
query string, and every check added `?_cb=` and the time. Checks now fetch
|
||||
each target's URL as written, with `cache: 'no-store'` as before. Each
|
||||
recorded check that fails writes one `console.error` line, and the same line
|
||||
to the debug log. The line gives the target's name and URL, the time, what
|
||||
failed and how long the request took. A target that answers after failed
|
||||
checks writes one `console.info` line. Checks the page does not record write
|
||||
none, so the debug log no longer lists failures in the first round or in the
|
||||
recovery probe.
|
||||
- 2026-10-04: the page's footer no longer says "IPv4 only"
|
||||
([#111](https://git.eeqj.de/sneak/netwatch/issues/111)). Each check is a
|
||||
`fetch`, the browser picks IPv4 or IPv6 for each WAN host, and the local
|
||||
targets are IPv4 addresses.
|
||||
- 2026-10-04: `README.md`, `TODO.md` and `test/viewport/README.md` say what the
|
||||
tree does ([#24](https://git.eeqj.de/sneak/netwatch/issues/24)). The README's
|
||||
Getting Started leads with `make` targets. A new Backend section says what
|
||||
`netwatch-server` stores, its routes and how the image builds and runs it; it
|
||||
pointed to `backend/README.md` for its settings. The checks are GET requests.
|
||||
The 26 WAN hosts, the four health states, the summary's figures and the
|
||||
features the list lacked are described as the page has them. The README's TODO
|
||||
section points here. This file's Workflow branches from `next` and opens the
|
||||
PR against `next`. Status says where the repo stands, and Next Step and Future
|
||||
Steps hold only open work, linked to its issue where one exists. The viewport
|
||||
test's README names Node's test runner, not `vitest`.
|
||||
- 2026-10-04: in `src/main.js`, a target's min, max, median and average latency
|
||||
come from one list of its answers
|
||||
([#102](https://git.eeqj.de/sneak/netwatch/issues/102)). The summary's figures
|
||||
use the same function, so the median is written once. The latency color limits
|
||||
are one table in `CONFIG`, read by both the figure's and the sparkline's
|
||||
color. The health thresholds, the debug log's length and the gateway check's
|
||||
timeout are `CONFIG` entries too. So are the recovery probe's host count and
|
||||
interval, how often the rows are sorted, and the delay before the first
|
||||
sparkline resize. Nothing the page does or shows changed; the footer's color
|
||||
legend still writes the limits out as text.
|
||||
- 2026-10-04: password guesses at `/metrics` are rate limited
|
||||
([#104](https://git.eeqj.de/sneak/netwatch/issues/104)). Each client address,
|
||||
resolved through `TRUSTED_PROXIES` as for reports, may make 60 requests to
|
||||
`/metrics` a minute, counted by `go-chi/httprate` apart from its reports. Past
|
||||
that it gets 429 and its basic auth credentials are not checked. The limit is
|
||||
a constant in `internal/server/routes.go`.
|
||||
- 2026-10-04: the backend reports errors to Sentry
|
||||
([#95](https://git.eeqj.de/sneak/netwatch/issues/95)). With `SENTRY_DSN` set,
|
||||
it sets up `sentry-go` with the release `netwatch-server-` and its version. It
|
||||
reports each panic in a handler through `sentryhttp`, the last middleware
|
||||
every request goes through. `sentryhttp` panics again, so the request still
|
||||
gets the 500 from the panic recovery. On shutdown it waits up to 2 seconds for
|
||||
Sentry to finish sending. A DSN Sentry refuses stops the start with an error
|
||||
naming `SENTRY_DSN`. With it empty, Sentry is not set up and nothing is sent
|
||||
to it.
|
||||
- 2026-10-04: a target's name and URL and a debug log message show as the
|
||||
characters they are and are never read as HTML
|
||||
([#29](https://git.eeqj.de/sneak/netwatch/issues/29)). A host row escapes the
|
||||
name and URL it writes into its markup, and the debug log sets each line as
|
||||
text. `README.md` no longer calls `CONFIG` frozen: the interval menu sets its
|
||||
`updateInterval`, and the values computed from it follow. `AppState` declares
|
||||
the recovery probe's two properties. The sparkline axis functions lose the
|
||||
parameters they did not use, and the comment on a target's history names both
|
||||
kinds of entry it holds.
|
||||
- 2026-10-04: a dependency can be added without running yarn or go by hand
|
||||
([#45](https://git.eeqj.de/sneak/netwatch/issues/45)).
|
||||
`make add-dependency PACKAGE=<name>@<version>` shims to the new
|
||||
`script/add-dependency`, which runs `yarn add --dev`, so `package.json` and
|
||||
`yarn.lock` change together, then `yarn install --frozen-lockfile`. The same
|
||||
command moves a package to another version. `make tidy` shims to the new
|
||||
`script/tidy`, which runs `go mod tidy`. A Go module is added by importing it,
|
||||
or moved by editing its `require` line, then `make tidy`. `script/bootstrap`
|
||||
still installs with `--frozen-lockfile`.
|
||||
- 2026-10-04: the backend serves Prometheus metrics
|
||||
([#94](https://git.eeqj.de/sneak/netwatch/issues/94)). With `METRICS_USERNAME`
|
||||
and `METRICS_PASSWORD` both set, it records request duration and response size
|
||||
through `go-http-metrics`. It serves them, with Go's runtime and process
|
||||
metrics, at `GET /metrics` behind basic auth with those credentials. nginx
|
||||
passes `/metrics` to it as it does `/api/`. With neither set there are no
|
||||
metrics and `/metrics` is 404. One without the other stops the start with an
|
||||
error naming both, and so does a `METRICS_USERNAME` containing `:`. Only
|
||||
requests that reach the health check or `POST /api/v1/reports` are recorded,
|
||||
not `/metrics` itself and not every request as `GO_HTTP_SERVER_CONVENTIONS.md`
|
||||
shows. The labels are the request's path and method, which clients can make up
|
||||
without end. For that, `POST /api/v1/reports` is registered by its full path
|
||||
instead of inside a `/api/v1` route group.
|
||||
- 2026-10-04: `script/` and `Makefile` follow the org models
|
||||
([#28](https://git.eeqj.de/sneak/netwatch/issues/28)). `make dev` shims to the
|
||||
new `script/dev`, the Vite dev server, and the new `make build` to
|
||||
`script/build`, the frontend production build. `.prettierignore` no longer
|
||||
left out `backend/`, so `make fmt` and `make fmt-check` covered
|
||||
`backend/README.md`; it left out `backend/.golangci.yml` by name.
|
||||
`script/install-precommit` and the date on `script/bootstrap`'s pins are the
|
||||
org model again. `script/bootstrap`, `script/fmt` and `script/fmt-check` each
|
||||
say in a comment why they differ from it.
|
||||
- 2026-10-04: a frontend build on Node 26 or newer no longer prints Node's
|
||||
warning that `module.register()` is deprecated
|
||||
([#32](https://git.eeqj.de/sneak/netwatch/issues/32)). The build in
|
||||
`Dockerfile` runs on Node 22, which never printed it. The call was in
|
||||
`@tailwindcss/node`, which `@tailwindcss/vite` brings in at its own exact
|
||||
version. tailwind 4.3.1 calls `module.registerHooks()` instead where Node has
|
||||
it. `yarn.lock` has `@tailwindcss/vite` and `tailwindcss` at 4.3.3, inside the
|
||||
ranges `package.json` already allowed, and tailwind's own dependencies moved
|
||||
with them. The built CSS changed only in how it is written out, in tailwind's
|
||||
Firefox focus-ring rule, and in tailwind's default sans-serif font list, which
|
||||
the page does not use.
|
||||
- 2026-10-03: the frontend's unit tests cover what the page computes
|
||||
([#21](https://git.eeqj.de/sneak/netwatch/issues/21)). They cover
|
||||
`humanDuration`, and the latency colours of a figure and of a sparkline either
|
||||
side of each boundary. They cover a target's min, max, average and median
|
||||
latency, and each of the four health states either side of its thresholds.
|
||||
`package.json` has a `test` script, so `yarn run test` and `npm run test` run
|
||||
them, and `script/frontend-test` runs it. `src/main.js` exports
|
||||
`humanDuration`, `HostState`, `latencyClass` and `latencyHex` for the tests;
|
||||
nothing it does changed.
|
||||
- 2026-10-03: the backend's logs are one stream
|
||||
([#27](https://git.eeqj.de/sneak/netwatch/issues/27)). fx logs its own steps
|
||||
of starting and stopping through the backend's logger. So off a terminal,
|
||||
every line the backend and fx write is JSON; fx used to write plain text to
|
||||
stderr. A config file that is found but cannot be read stops the start with
|
||||
its error, logged as JSON like a bad setting. It used to end in a Go panic.
|
||||
The backend logs its name, version and architecture once at start. The health
|
||||
check's uptime keys are `uptime_seconds` and `uptime_human`; its path, content
|
||||
type, `"status":"ok"` and 200 are unchanged. `SENTRY_DSN`, `METRICS_USERNAME`
|
||||
and `METRICS_PASSWORD` were still read and unused until
|
||||
[#94](https://git.eeqj.de/sneak/netwatch/issues/94) and
|
||||
[#95](https://git.eeqj.de/sneak/netwatch/issues/95) wired them up.
|
||||
- 2026-10-03: the frontend has a real linter
|
||||
([#47](https://git.eeqj.de/sneak/netwatch/issues/47), and item 2 of
|
||||
[#28](https://git.eeqj.de/sneak/netwatch/issues/28)): `eslint` with its
|
||||
recommended rules, set in `eslint.config.js`, runs in a new `frontend-lint`
|
||||
stage of `Dockerfile`. `script/lint` builds the lint stages and runs no linter
|
||||
on the host. `script/bootstrap` wants node 22.13.0 or newer, as eslint 10
|
||||
does.
|
||||
- 2026-10-03: the tap-target check in `make frontend-viewport-test` expects one
|
||||
visible pin button per WAN host row
|
||||
([#46](https://git.eeqj.de/sneak/netwatch/issues/46)), where it expected at
|
||||
least 10 of the 26. Pin buttons missing from only some rows now fail it. The
|
||||
host row count the test gathers, which the `app-rendered` check also reads,
|
||||
counts only the WAN host rows: the local host rows have no pin button.
|
||||
- 2026-10-03: each target's row shows its result as soon as its check ends
|
||||
([#91](https://git.eeqj.de/sneak/netwatch/issues/91)). Every row waited for
|
||||
the round's slowest check, up to 24 seconds at a 30-second interval. Every row
|
||||
is still redrawn, and sorting, the summary, the health box and offline
|
||||
detection still run, once, when the round's last check ends. So no row reads
|
||||
"paused" after a pause and resume during the round. A check that ends after
|
||||
the user pauses or after its round is given up shows nothing. The first round
|
||||
is still discarded as a whole.
|
||||
- 2026-10-03: root no longer acts outside `/data` when it prepares `DATA_DIR`
|
||||
([#80](https://git.eeqj.de/sneak/netwatch/issues/80)). `bin/entrypoint.sh`
|
||||
runs `netwatch-server prepare-data-dir`, which refuses a `DATA_DIR` that is
|
||||
not `/data` or a path below it written in full. Then it creates `DATA_DIR`,
|
||||
gives `/data` and everything in it to `netwatch` and sets the modes, all
|
||||
through a Go `os.Root` opened on `/data`. That refuses any path leading out of
|
||||
`/data`. So no symbolic link, already there or swapped in by a host process
|
||||
during the start, can make root create or change anything elsewhere.
|
||||
`DATA_DIR=/etc` no longer gives `/etc` to `netwatch`. The `README.md` section
|
||||
"Running under upaas" says which values are accepted.
|
||||
- 2026-10-03: `DATA_DIR_MAX_BYTES` is how much of the report files is kept
|
||||
([#54](https://git.eeqj.de/sneak/netwatch/issues/54)). When a report would
|
||||
take them past it, the oldest report files are deleted to make room, each
|
||||
deletion logged. At start, files already past it are deleted the same way. A
|
||||
file still being written is never deleted. A report is refused with 507 only
|
||||
when the reports waiting to be written fill the cap on their own, and then no
|
||||
file is deleted. A file that cannot be deleted still counts until the next
|
||||
start; one already deleted by hand counts as freed.
|
||||
- 2026-10-03: `backend/script/lint` said what went wrong with its
|
||||
`.golangci.yml` check ([#34](https://git.eeqj.de/sneak/netwatch/issues/34)).
|
||||
On a hash mismatch it said to compare the file with the org standard. If they
|
||||
differed, restore the org standard. If they were the same, the org standard
|
||||
had changed, so update `GOLANGCI_CONFIG_SHA256` in that script. It used to say
|
||||
only to restore the file, which loops once the org standard itself has moved.
|
||||
A missing `.golangci.yml`, and a `sha256sum` that is missing or prints no
|
||||
hash, each got their own message instead of being reported as a mismatch.
|
||||
- 2026-10-03: the Go tests run with the race detector and coverage
|
||||
([#88](https://git.eeqj.de/sneak/netwatch/issues/88)):
|
||||
`go test -timeout 30s -race -cover ./...` and, if that fails, the same again
|
||||
with `-v`, then a failing exit. Go's `-timeout` bounds the tests, not their
|
||||
compile. The race detector needs a C compiler; the binary is still built with
|
||||
`CGO_ENABLED=0`. New tests cover the health check's answer, a valid report's
|
||||
answer, a report file's exact contents, and the flush when the buffer reaches
|
||||
10 MiB.
|
||||
- 2026-10-03: each target check times out after 80% of the refresh interval
|
||||
([#78](https://git.eeqj.de/sneak/netwatch/issues/78)), 24 seconds at 30
|
||||
seconds, where it was capped at 3 seconds. A round started early gives up the
|
||||
last round's checks if they are still waiting, so rounds never overlap. A
|
||||
round starts early after an interval change, or when the recovery probe finds
|
||||
a target answering. The recovery probe gives up its own checks after half a
|
||||
second. The frontend has its first unit tests, run by `script/frontend-test`
|
||||
with Node's built-in test runner. For them, `index.html` links
|
||||
`src/styles.css`, which `src/main.js` used to import.
|
||||
- 2026-09-29: the container sets up its own data directory
|
||||
([#75](https://git.eeqj.de/sneak/netwatch/issues/75)). `bin/entrypoint.sh`,
|
||||
still as root, created `DATA_DIR` if missing and gave it and `/data` to the
|
||||
`netwatch` user with mode 750 before starting the backend as that user. So an
|
||||
empty host directory owned by root, or one holding files from another uid,
|
||||
works with no step on the host. It stopped the start when a symbolic link was
|
||||
on the path to `DATA_DIR`, since root would change whatever the link points
|
||||
to. The `README.md` first-run step that created and chowned the host directory
|
||||
is gone, and the image no longer sets that ownership at build time.
|
||||
- 2026-09-29: CI can no longer pass on checks that did not run
|
||||
([#37](https://git.eeqj.de/sneak/netwatch/issues/37)). `script/cibuild` became
|
||||
the org model, byte for byte. It runs `script/bootstrap` and `script/check`,
|
||||
then builds the image with the version from `git describe` as the `VERSION`
|
||||
build argument. It used to be a plain `docker build .` whose check steps could
|
||||
come from the build cache. The workflow put `~/.local/bin`, where bootstrap
|
||||
links what it installs, on the step's `PATH`. bootstrap installs its pinned
|
||||
node when the installed one is older than the frontend's dependencies need.
|
||||
- 2026-09-29: `backend/.golangci.yml` was re-vendored from `sneak/prompts`
|
||||
([#41](https://git.eeqj.de/sneak/netwatch/issues/41)). `gomodguard`,
|
||||
deprecated in golangci-lint v2.12.0, is disabled and its successor
|
||||
`gomodguard_v2` enabled with the org block list, so lint runs print no
|
||||
deprecation warning. The file also turns `depguard` on with its `test-support`
|
||||
rule, which keeps `net/http/httptest` out of non-test code; netwatch adds no
|
||||
entries of its own to that rule.
|
||||
- 2026-09-29: nginx sends the security headers `REPO_POLICIES.md` requires on
|
||||
every response (issue #18), including errors, `/assets/` and what it passes on
|
||||
from the backend, whose own copies it drops so each header goes out once. They
|
||||
live in `security-headers.conf`, which `nginx.conf` includes. The content
|
||||
security policy allows no inline script or style, so the status dot's grey in
|
||||
`src/main.js` is now a class; `connect-src` is `*` because probed hosts
|
||||
redirect to others, and the browser checks each redirect against it
|
||||
- 2026-09-29: the request log is bounded (issue #60): the method, URL, protocol,
|
||||
`User-Agent`, `Referer`, request ID (which chi takes from the client's
|
||||
`X-Request-Id` header) and client address it writes are each cut to 128 bytes,
|
||||
the bound the report handler already used, so one request can no longer put
|
||||
about 1 MiB per field into a log line. That bound and its helper now live in
|
||||
the `logger` package, shared by both
|
||||
every response ([#18](https://git.eeqj.de/sneak/netwatch/issues/18)), errors
|
||||
and `/assets/` included. On what it passes on from the backend, it drops the
|
||||
backend's own copies, so each header goes out once. They live in
|
||||
`security-headers.conf`, which `nginx.conf` includes. The content security
|
||||
policy allows no inline script or style, so the status dot's grey in
|
||||
`src/main.js` is a class. `connect-src` is `*` because probed hosts redirect
|
||||
to others, and the browser checks each redirect against it.
|
||||
- 2026-09-29: the request log is bounded
|
||||
([#60](https://git.eeqj.de/sneak/netwatch/issues/60)). Each field it writes
|
||||
from the request is cut to 128 bytes, the bound the report handler already
|
||||
used. The fields are the method, URL, protocol, `User-Agent`, `Referer`,
|
||||
request ID and client address. chi takes the request ID from the client's
|
||||
`X-Request-Id` header. One request could put about 1 MiB per field into a log
|
||||
line. That bound and its helper live in the `logger` package, shared by both.
|
||||
- 2026-09-29: nginx takes the client address from `X-Forwarded-For` only on
|
||||
requests from the reverse proxies named in the container's `TRUSTED_PROXIES`
|
||||
(issue #64), and by default from none, where it trusted every RFC1918 address
|
||||
before, so a client could write a new address on each request and escape the
|
||||
rate limit. `bin/entrypoint.sh` writes one `set_real_ip_from` line per entry
|
||||
into `/etc/nginx/trusted-proxies.conf`, which `nginx.conf` includes, refusing
|
||||
an entry that is not an IP address or CIDR, as `netwatch-server check-cidr`
|
||||
finds; it starts the backend with `TRUSTED_PROXIES=127.0.0.1/32`, since nginx
|
||||
is its only client
|
||||
- 2026-09-29: report file names can no longer collide (issue #61): each is
|
||||
([#64](https://git.eeqj.de/sneak/netwatch/issues/64)), and at the time by
|
||||
default from none. It trusted every RFC1918 address before, so a client could
|
||||
write a new address on each request and escape the rate limit.
|
||||
`bin/entrypoint.sh` writes one `set_real_ip_from` line per entry into
|
||||
`/etc/nginx/trusted-proxies.conf`, which `nginx.conf` includes. It refuses an
|
||||
entry that is not an IP address or CIDR, as `netwatch-server check-cidr`
|
||||
finds. It starts the backend with `TRUSTED_PROXIES=127.0.0.1/32`, since nginx
|
||||
is its only client.
|
||||
- 2026-09-29: report file names can no longer collide
|
||||
([#61](https://git.eeqj.de/sneak/netwatch/issues/61)). Each is
|
||||
`reports-<timestamp>-<number>.jsonl.zst`, where the number goes up by one for
|
||||
each file the server starts to write, so two flushes in the same millisecond,
|
||||
such as a flush for size and the final flush at shutdown, each get a file of
|
||||
their own instead of the second one failing. A failed write uses up its
|
||||
number, leaving a gap if the file could not be created and otherwise a file
|
||||
under that number that may be incomplete.
|
||||
- 2026-09-29: ready to run under upaas (issue #59): the image has a
|
||||
each file the server starts to write. So two flushes in the same millisecond
|
||||
each get a file of their own; the second one used to fail. A flush for size
|
||||
and the final flush at shutdown can fall in the same millisecond. A failed
|
||||
write uses up its number.
|
||||
- 2026-09-29: ready to run under upaas
|
||||
([#59](https://git.eeqj.de/sneak/netwatch/issues/59)). The image has a
|
||||
`HEALTHCHECK` that requests `/.well-known/healthcheck` through nginx on the
|
||||
port from `PORT`. The backend no longer reads a bad `PORT` as 0 or a bad
|
||||
`DEBUG` as false: those, and a `BIND_ADDRESS` that is not an IP address, stop
|
||||
it from starting with an error naming the variable, as the limits,
|
||||
`DEBUG` as false. Those, and a `BIND_ADDRESS` that is not an IP address, stop
|
||||
it from starting with an error naming the variable. The limits,
|
||||
`CORS_ALLOWED_ORIGINS` and, now by name, `TRUSTED_PROXIES` already did.
|
||||
`bin/entrypoint.sh` also refuses a `PORT` outside 1 to 65535, and `8081`,
|
||||
where the backend listens inside the container, naming `PORT`. `README.md` has
|
||||
a "Running under upaas" section, whose first-run steps create the host
|
||||
directory for `/data` owned by uid 1000; the image does not change its owner
|
||||
- 2026-09-29: nginx listens on `PORT` (issue #26), 8080 when unset or empty: the
|
||||
nginx image renders `nginx.conf` as a template at container start, filling in
|
||||
`PORT` and no other variable. `bin/entrypoint.sh` refuses to start when `PORT`
|
||||
is not digits only. `server_tokens off` keeps the nginx version out of
|
||||
responses. `script/frontend-viewport-test` renders the template the same way.
|
||||
Gzip and a `50x.html` error page are not added
|
||||
- 2026-09-29: bounded the report endpoint (issue #20): `POST /api/v1/reports`
|
||||
still needs no credentials, but each client address, as resolved through
|
||||
where the backend listens inside the container, naming `PORT`. `README.md` got
|
||||
a "Running under upaas" section.
|
||||
- 2026-09-29: nginx listens on `PORT`
|
||||
([#26](https://git.eeqj.de/sneak/netwatch/issues/26)), 8080 when unset or
|
||||
empty. The nginx image renders `nginx.conf` as a template at container start,
|
||||
filling in `PORT` and no other variable. `bin/entrypoint.sh` refuses to start
|
||||
when `PORT` is not digits only. `server_tokens off` keeps the nginx version
|
||||
out of responses. `script/frontend-viewport-test` renders the template the
|
||||
same way. Gzip and a `50x.html` error page are not added.
|
||||
- 2026-09-29: the report endpoint is bounded
|
||||
([#20](https://git.eeqj.de/sneak/netwatch/issues/20)). `POST /api/v1/reports`
|
||||
still needs no credentials. Each client address, as resolved through
|
||||
`TRUSTED_PROXIES`, may send `REPORTS_PER_MINUTE` (default 60) reports a
|
||||
minute, counted by `go-chi/httprate`, and past that gets 429 with
|
||||
`Retry-After`; the report files in `DATA_DIR`, counted from start with those
|
||||
already there, may total at most `DATA_DIR_MAX_BYTES` (default 1 GiB), past
|
||||
which reports get 507; and the wildcard CORS is gone: no CORS headers unless
|
||||
`CORS_ALLOWED_ORIGINS` lists origins, and an entry that is not a plain
|
||||
`Retry-After`. The report files in `DATA_DIR`, counted from start with those
|
||||
already there, may total at most `DATA_DIR_MAX_BYTES` (default 1 GiB); past
|
||||
that, reports got 507. The wildcard CORS is gone: no CORS headers unless
|
||||
`CORS_ALLOWED_ORIGINS` lists origins. An entry that is not a plain
|
||||
`scheme://host[:port]` origin, `*` included, stops the server from starting.
|
||||
Deleting report files frees room only at the next start; pruning is issue #54
|
||||
- 2026-09-28: one container image (issue #52): the root `Dockerfile` builds the
|
||||
only image, and `Dockerfile.backend` is gone. nginx serves the frontend on
|
||||
port 8080 and proxies `/api/` and `/.well-known/healthcheck` to the backend,
|
||||
which listens on `127.0.0.1:8081` in the same container; the new
|
||||
Deleting report files freed room only at the next start, until
|
||||
[#54](https://git.eeqj.de/sneak/netwatch/issues/54).
|
||||
- 2026-09-28: one container image
|
||||
([#52](https://git.eeqj.de/sneak/netwatch/issues/52)). The root `Dockerfile`
|
||||
builds the only image, and `Dockerfile.backend` is gone. nginx serves the
|
||||
frontend on port 8080 and proxies `/api/` and `/.well-known/healthcheck` to
|
||||
the backend, which listens on `127.0.0.1:8081` in the same container. The new
|
||||
`BIND_ADDRESS` setting sets its listen address. `bin/entrypoint.sh` starts
|
||||
both, passes TERM and INT on to both, and exits non-zero when either exits on
|
||||
its own. The backend runs as user `netwatch` and stores reports on the `/data`
|
||||
volume. `script/docker` is the org model again
|
||||
- 2026-09-28: unified the gate (issue #16): the root `make check` covers the Go
|
||||
backend as well as the frontend, and the pre-commit hook with it; the backend
|
||||
moved onto scripts-to-rule-them-all (`backend/script/*`, `backend/Makefile` as
|
||||
shims, its duplicate hook installer removed); `script/cibuild` builds both
|
||||
images and is the workflow's only build step. The root `make lint` runs
|
||||
golangci-lint only in Docker, by building the lint stage of
|
||||
`Dockerfile.backend` without the cache. `script/bootstrap` installs the pinned
|
||||
Go unless the installed one is at least what `backend/go.mod` asks for, links
|
||||
what it installs into `~/.local/bin` without replacing anything it did not
|
||||
create, and installs no linter. Root `make test` runs both halves within one
|
||||
30-second timeout. When `VERSION` is unset or empty, the backend binary's
|
||||
version falls back to `git describe` inside a git checkout, then to `dev`
|
||||
- 2026-09-28: frontend reporting client (issue #53): a `Reporter` class posts
|
||||
collected samples to `/api/v1/reports` every `reportInterval` (default 60s) as
|
||||
a per-host delta, with the report-building step a pure exported function of
|
||||
host state; a per-host mark advances only on a delivered POST; at most one
|
||||
report POST is pending at a time and it is abandoned after half the interval,
|
||||
so a slow POST never overlaps the next report and a mark never moves
|
||||
backwards; the samples of an abandoned POST are sent again at the next
|
||||
interval, so a backend that stored them but answered late receives them twice;
|
||||
the per-browser client id works in insecure (plain-HTTP) contexts;
|
||||
`vite.config.js` proxies `/api` to the local backend for `yarn dev`
|
||||
- 2026-09-28: report ingest correctness (issue #23): a storage failure now
|
||||
returns 500 instead of a false `ok`; oversize bodies return 413 (distinguished
|
||||
from malformed JSON, which stays 400); a `MaxBodyBytes` middleware caps every
|
||||
route, not just the report route; the raw attacker-controlled `geo` blob is no
|
||||
longer logged (only its length), and `client_id`, `timestamp` and decode error
|
||||
text are length-bounded before logging; a `decodeJSON` handler helper was
|
||||
added; panic recovery now routes the stack through slog instead of chi's
|
||||
plain-text stderr; and writing a report file now returns its error, so a
|
||||
failed final flush on shutdown makes the process exit non-zero instead of
|
||||
losing the buffered reports silently
|
||||
- 2026-09-21: shutdown lifecycle correctness. The process now shuts down through
|
||||
fx instead of `os.Exit`, so every component's `OnStop` runs and buffered
|
||||
reports are flushed to disk on `SIGTERM` — previously a full flush window of
|
||||
telemetry was silently lost on every restart. The `http.Server` is now built
|
||||
before its serving goroutine starts, so shutdown can no longer race or
|
||||
nil-deref it; a listen failure exits non-zero via `fx.Shutdowner`; `reportbuf`
|
||||
`OnStop` is idempotent; and `writeTimeout` now exceeds the chi per-request
|
||||
budget so that budget is actually reachable. Dead `startupTime`, `exitCode`,
|
||||
and `cancelFunc` fields were removed
|
||||
- 2026-09-21: backend HTTP hardening (issue #19): added `ReadHeaderTimeout` and
|
||||
`IdleTimeout` to the server, a `SecurityHeaders` middleware (HSTS, tight CSP,
|
||||
frame/sniff/referrer/permissions headers) registered before CORS, and
|
||||
trusted-proxy client IP resolution honouring `X-Forwarded-For` / `X-Real-IP`
|
||||
only from a `TRUSTED_PROXIES` allowlist (loopback plus RFC1918 by default)
|
||||
volume. `script/docker` is the org model again.
|
||||
- 2026-09-28: one gate ([#16](https://git.eeqj.de/sneak/netwatch/issues/16)).
|
||||
The root `make check` covers the Go backend as well as the frontend, and the
|
||||
pre-commit hook with it. The backend moved onto scripts-to-rule-them-all:
|
||||
`backend/script/*`, `backend/Makefile` as shims, its duplicate hook installer
|
||||
removed. `script/cibuild` built both images and was the workflow's only build
|
||||
step. The root `make lint` ran golangci-lint only in Docker, by building the
|
||||
lint stage of `Dockerfile.backend` without the cache. `script/bootstrap`
|
||||
installed the pinned Go unless the installed one was at least what
|
||||
`backend/go.mod` asked for. It links what it installs into `~/.local/bin`
|
||||
without replacing anything it did not create, and installs no linter. When
|
||||
`VERSION` is unset or empty, the backend binary's version falls back to
|
||||
`git describe` inside a git checkout, then to `dev`.
|
||||
- 2026-09-28: frontend reporting client
|
||||
([#53](https://git.eeqj.de/sneak/netwatch/issues/53)). A `Reporter` class
|
||||
posts collected samples to `/api/v1/reports` every `reportInterval` (default
|
||||
60s) as a per-host delta. The report-building step is a pure exported function
|
||||
of host state. A per-host mark advances only on a delivered POST. At most one
|
||||
report POST is pending at a time, and it is abandoned after half the interval.
|
||||
So a slow POST never overlaps the next report, and a mark never moves
|
||||
backwards. The samples of an abandoned POST are sent again at the next
|
||||
interval, so a backend that stored them but answered late receives them twice.
|
||||
The per-browser client id works in insecure (plain-HTTP) contexts.
|
||||
`vite.config.js` proxies `/api` to the local backend for `yarn dev`.
|
||||
- 2026-09-28: report ingest correctness
|
||||
([#23](https://git.eeqj.de/sneak/netwatch/issues/23)). A storage failure
|
||||
returns 500 instead of a false `ok`. An oversize body returns 413, apart from
|
||||
malformed JSON, which stays 400. A `MaxBodyBytes` middleware caps every route,
|
||||
not just the report route. The raw attacker-controlled `geo` blob is no longer
|
||||
logged, only its length, and `client_id`, `timestamp` and decode error text
|
||||
are length-bounded before logging. A `decodeJSON` handler helper was added.
|
||||
Panic recovery routes the stack through slog instead of chi's plain-text
|
||||
stderr. Writing a report file returns its error, so a failed final flush on
|
||||
shutdown makes the process exit non-zero instead of losing the buffered
|
||||
reports silently.
|
||||
- 2026-09-21: shutdown lifecycle correctness. The process shuts down through fx
|
||||
instead of `os.Exit`, so every component's `OnStop` runs and buffered reports
|
||||
are flushed to disk on `SIGTERM`. Before, a full flush window of telemetry was
|
||||
lost on every restart. The `http.Server` is built before its serving goroutine
|
||||
starts, so shutdown can no longer race or nil-deref it. A listen failure exits
|
||||
non-zero via `fx.Shutdowner`. `reportbuf`'s `OnStop` is idempotent.
|
||||
`writeTimeout` exceeds the chi per-request budget, so that budget is
|
||||
reachable. Dead `startupTime`, `exitCode` and `cancelFunc` fields were
|
||||
removed.
|
||||
- 2026-09-21: backend HTTP hardening
|
||||
([#19](https://git.eeqj.de/sneak/netwatch/issues/19)): `ReadHeaderTimeout` and
|
||||
`IdleTimeout` on the server, and a `SecurityHeaders` middleware (HSTS, tight
|
||||
CSP, frame, sniff, referrer and permissions headers) registered before CORS.
|
||||
`X-Forwarded-For` and `X-Real-IP` are honoured only from a `TRUSTED_PROXIES`
|
||||
allowlist, at the time loopback plus RFC1918 by default.
|
||||
- 2026-08-10: adopted the org-standard `backend/.golangci.yml` verbatim and
|
||||
moved the pinned golangci-lint from v2.7.2 to v2.12.2 (the `lint` stage of
|
||||
`Dockerfile.backend` now pins the `golangci/golangci-lint:v2.12.2` image by
|
||||
digest); the previous config declared `version: "2"` but used v1 schema keys,
|
||||
so every threshold in it was inert and its green result was meaningless.
|
||||
`backend/Makefile`'s `lint` target now asserts the config's sha256 against the
|
||||
canonical file first, so drift from the org standard fails the build instead
|
||||
of silently degrading to defaults
|
||||
- 2026-08-10: every interactive control now meets the 44x44 CSS px minimum tap
|
||||
target (`.pin-btn`, `#interval-select`, the debug-log label and, on narrow
|
||||
viewports, `#pause-btn`). The pin button's hit area grows via matching
|
||||
negative margins, so its layout footprint and row density are unchanged
|
||||
- 2026-08-10: per-host status line wraps below the 768px breakpoint instead of
|
||||
forcing horizontal page scroll at 320px
|
||||
- 2026-08-09: `Dockerfile.backend` reworked to the mandated Go multistage
|
||||
lint-stage pattern: separate `lint` stage on the hash-pinned
|
||||
`golangci/golangci-lint` image, `COPY --from=lint` stage dependency,
|
||||
`CGO_ENABLED=0` static build driven by `ARG VERSION`, and no more `COPY .git`
|
||||
- 2026-08-09: dotfile compliance — lifted `backend/.editorconfig` to the repo
|
||||
root so `root = true` covers the frontend too, and replaced `.gitignore` with
|
||||
the org model (OS, editor, node, and environment/secrets sections) plus this
|
||||
repo's `dist/` and `*.log`. `.env`, `.env.*`, `*.pem`, and `*.key` are now
|
||||
ignored repo-wide, not just under `backend/`. Excluding `.git` from
|
||||
`.dockerignore` stays deferred: both images read git metadata at build time
|
||||
(`COPY .git` in `Dockerfile.backend`, `git rev-parse` in `vite.config.js`)
|
||||
- 2026-08-09: automated responsive-layout harness
|
||||
(`make frontend-viewport-test`): digest-pinned headless Chrome driven over CDP
|
||||
against the built `dist/`, viewport widths derived from the breakpoints in
|
||||
`src/styles.css` ([#13](https://git.eeqj.de/sneak/netwatch/issues/13)). Every
|
||||
check carries a presence guard so none of them can pass against a page it is
|
||||
not actually measuring. Found two real layout defects, filed as
|
||||
moved the pinned golangci-lint from v2.7.2 to v2.12.2; the `lint` stage of
|
||||
`Dockerfile.backend` pinned the `golangci/golangci-lint:v2.12.2` image by
|
||||
digest. The previous config declared `version: "2"` but used v1 schema keys,
|
||||
so every threshold in it was inert and its green result meant nothing.
|
||||
`backend/Makefile`'s `lint` target asserted the config's sha256 against the
|
||||
canonical file first, so drift from the org standard failed the build instead
|
||||
of silently degrading to defaults.
|
||||
- 2026-08-10: every interactive control meets the 44x44 CSS px minimum tap
|
||||
target: `.pin-btn`, `#interval-select`, the debug-log label and, on narrow
|
||||
viewports, `#pause-btn`. The pin button's hit area grows via matching negative
|
||||
margins, so its layout footprint and row density are unchanged.
|
||||
- 2026-08-10: the per-host status line wraps below the 768px breakpoint instead
|
||||
of forcing horizontal page scroll at 320px.
|
||||
- 2026-08-09: `Dockerfile.backend` was reworked to the Go multistage lint-stage
|
||||
pattern. It got a separate `lint` stage on the hash-pinned
|
||||
`golangci/golangci-lint` image, and a `COPY --from=lint` stage dependency. The
|
||||
build became a `CGO_ENABLED=0` static build driven by `ARG VERSION`, with no
|
||||
more `COPY .git`.
|
||||
- 2026-08-09: dotfile compliance. `backend/.editorconfig` moved to the repo root
|
||||
so `root = true` covers the frontend too. `.gitignore` became the org model
|
||||
(OS, editor, node, and environment/secrets sections) plus this repo's `dist/`
|
||||
and `*.log`. `.env`, `.env.*`, `*.pem` and `*.key` are ignored repo-wide, not
|
||||
just under `backend/`. Excluding `.git` from `.dockerignore` stayed deferred:
|
||||
both images read git metadata at build time.
|
||||
- 2026-08-09: automated responsive-layout test (`make frontend-viewport-test`):
|
||||
a digest-pinned headless Chrome driven over CDP against the built `dist/`, at
|
||||
viewport widths derived from the breakpoints in `src/styles.css`
|
||||
([#13](https://git.eeqj.de/sneak/netwatch/issues/13)). The tap-target and
|
||||
host-row checks each fail when they measured nothing. The overflow,
|
||||
viewport-edge and clipped-text checks have no such guard of their own and rely
|
||||
on the `app-rendered` check, which fails the run when the app did not render.
|
||||
It found two real layout defects, filed as
|
||||
[#42](https://git.eeqj.de/sneak/netwatch/issues/42) and
|
||||
[#43](https://git.eeqj.de/sneak/netwatch/issues/43)
|
||||
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
|
||||
shims, README Entrypoints section
|
||||
[#43](https://git.eeqj.de/sneak/netwatch/issues/43).
|
||||
- 2026-07-07: adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
|
||||
shims, README Entrypoints section.
|
||||
- 2026-02-27: backend with buffered zstd-compressed report storage; CI workflow
|
||||
and backend repo standard files; backend Dockerfile fixed (Go 1.25,
|
||||
golangci-lint) and moved to repo root (feat/reportbuf-storage)
|
||||
golangci-lint) and moved to repo root (`feat/reportbuf-storage`).
|
||||
- 2026-02-26: host row layout redesigned with CSS grid; overflow and spacing
|
||||
fixes; nginx config extracted; port hardcoded to 8080
|
||||
fixes; nginx config extracted; port hardcoded to 8080.
|
||||
- 2026-02-26: debug log panel, median stats, recovery probe, Docker build fix,
|
||||
S3 Singapore endpoint added
|
||||
S3 Singapore endpoint added.
|
||||
- 2026-02-23: summary box redesign, host pinning, local and UTC clocks, checks
|
||||
counter
|
||||
counter.
|
||||
- 2026-02-23: hosts sorted by latency; GET instead of HEAD for latency; timeout
|
||||
derived from interval; Hetzner regional endpoints; 3s interval
|
||||
- 2026-01-29: initial NetWatch network latency monitor
|
||||
derived from interval; Hetzner regional endpoints; 3s interval.
|
||||
- 2026-01-29: initial NetWatch network latency monitor.
|
||||
|
||||
# Future Steps
|
||||
|
||||
- Wire `script/frontend-viewport-test` into CI as its own step (deliberately not
|
||||
part of `make check` today; the decision has real CI-runtime cost and is
|
||||
tracked separately)
|
||||
- Compliance top-up as one small commit: add .editorconfig and add the hooks
|
||||
target to the Makefile
|
||||
- After merge, confirm .gitea/workflows/check.yml is on main and CI is green
|
||||
(main always green policy)
|
||||
- Decide what to do with untracked resume.sh: commit it, gitignore it, or delete
|
||||
it
|
||||
- A backend test that posts a report to `POST /api/v1/reports` and checks the
|
||||
compressed file it is written to
|
||||
- A backend route that decompresses the stored reports and answers queries on
|
||||
them
|
||||
- Prometheus metrics for the backend's in-memory buffer: its size, the number of
|
||||
flushes and the number of reports
|
||||
- A configurable host list (an environment variable or a config file)
|
||||
- Export of the latency history (CSV or JSON)
|
||||
- A notification when the health status changes to DEGRADED
|
||||
|
||||
Reference in New Issue
Block a user