27 Commits
Author SHA1 Message Date
sneak 7dfb3b8c32 Merge branch 'main' into next
check / check (push) Waiting to run
2026-09-29 12:04:10 +02:00
clawbot a5ca73c585 Container sets up its own data directory (closes #75) (#76)
check / check (push) Waiting to run
Closes #75.

`bin/entrypoint.sh`, which already runs as root, now makes the data directory usable before the backend starts: it creates `DATA_DIR` if missing, gives it and `/data` to the `netwatch` user (`chown -R`), and sets mode 750 on both, the mode the backend gives a directory it creates. The backend still runs as `netwatch`. The README "Running under upaas" section loses its first-run step that created and chowned the host directory and names only the path to mount. The Dockerfile's build-time `mkdir` and `chown` of `/data` are gone, since the entrypoint now does this on every start.

What the diff does not show:

- The host directory mounted at `/data` ends up owned by uid 1000 with mode 750, and everything under `DATA_DIR` is chowned to uid 1000 on every start.
- If the directory cannot be created or chowned, the container stops with that tool's error before either process starts.

Recorded runs with `--mount type=bind`: an empty directory owned by root (mode 755, and again mode 700), and one holding a `reports` directory and report file owned by uid 1001 with mode 700. Each time the container turned healthy, `netwatch-server` ran as `netwatch`, and a posted report was written to `DATA_DIR`; a second start on the root-owned and the uid 1001 directories did the same.

Judgement call: `/data` itself is given to `netwatch` as well as `DATA_DIR`, so the backend can reach `DATA_DIR` inside a host directory with mode 700.

Model: opus-5-5
Reviewed-on: #76
Co-authored-by: clawbot <35+clawbot@noreply.example.org>
2026-09-29 12:03:59 +02:00
clawbot f423768975 cibuild: the org model, which runs every check uncached (closes #37)
check / check (push) Waiting to run
script/cibuild was a plain docker build ., so on a tree Docker had
seen before every check step came from the build cache and the build
still passed. It is now the org model from sneak/prompts, byte for
byte: script/bootstrap, script/check, then docker build --no-cache
with the git describe version as the VERSION build argument.

The workflow puts ~/.local/bin, where bootstrap links what it
installs, on the step's PATH. Bootstrap now installs its pinned node
when the installed one is older than 22.12.0, the oldest the
frontend's dependencies accept (puppeteer-core's engines field), as
it already does for Go against backend/go.mod.

Model: opus-5-5
2026-09-29 11:55:52 +02:00
clawbot c226ceee01 chore(backend): re-vendor .golangci.yml with gomodguard_v2 (closes #41)
check / check (push) Successful in 24s
golangci-lint v2.12 deprecates gomodguard, which the org .golangci.yml
reached through "default: all", so every lint run printed a
deprecation warning. backend/.golangci.yml is now the current copy
from sneak/prompts, fetched unedited: gomodguard is disabled and
gomodguard_v2 enabled with the org block list. The new file also
turns depguard on with its test-support rule, which forbids
net/http/httptest outside test code. netwatch has no test-support
packages of its own to add to that rule, so the file is identical to
the canonical one. backend/script/lint checks the new sha256. The
backend raises no findings under the new rules.

Model: opus-5-5
2026-09-29 10:56:00 +02:00
sneak bd08e901ee next into main: netwatch as one container, ready for upaas (#49)
check / check (push) Successful in 12s
Reviewed-on: #49
2026-09-29 10:43:12 +02:00
clawbot d81da05748 nginx: security headers on every response (closes #18)
check / check (push) Successful in 21s
nginx sent none of the security headers REPO_POLICIES.md requires.
security-headers.conf now sets all six with always, included at server
level and again in /assets/, whose own add_header would otherwise drop
them. nginx hides the copies netwatch-server sets, so /api/ and the
health check carry each header once. The content security policy
allows no inline script or style; the host row's status dot took its
grey from a style attribute, now a class. connect-src is * because
several probed hosts redirect to other hosts and the browser checks
every redirect against it. Referrer-Policy is no-referrer, as the
backend already sends.

Model: opus-5-5
2026-09-29 10:22:12 +02:00
clawbot d74d1e311e fix(backend): cut request log fields to the log bound (closes #60)
check / check (push) Successful in 14s
The request log wrote the URL, User-Agent, Referer and other
request-supplied strings with no length limit, and the server accepts
headers up to 1 MiB, so one request could put about 1 MiB per field
into a log line. Every string the request log takes from the request,
including the request ID chi copies from X-Request-Id, is now cut to
the 128-byte bound the report handler already used. That bound and
its helper moved from the handlers package to the logger package so
both use the one copy.

Model: opus-5-5
2026-09-29 09:39:11 +02:00
clawbot 8833603eff nginx: trust X-Forwarded-For only from TRUSTED_PROXIES (closes #64)
check / check (push) Successful in 15s
nginx trusted X-Forwarded-For from every RFC1918 address, so a client
reaching it from one could write a new address on each request and
get a fresh rate-limit allowance. The container's TRUSTED_PROXIES now
names the reverse proxies nginx trusts, none by default.
bin/entrypoint.sh makes each entry a CIDR, checks it with the new
"netwatch-server check-cidr", which runs the server's own
TRUSTED_PROXIES parsing, and writes one set_real_ip_from line per
entry into /etc/nginx/trusted-proxies.conf, which nginx.conf includes.
The backend is started with TRUSTED_PROXIES=127.0.0.1/32, since nginx
is its only client. The viewport test mounts an empty file there.

Model: opus-5-5
2026-09-29 08:55:47 +02:00
clawbot 6022cc8b02 fix(backend): give each report file a name of its own (closes #61)
check / check (push) Successful in 13s
Report files were named by a millisecond timestamp and created with
O_EXCL, so two flushes in the same millisecond, such as a flush for
size and the final flush at shutdown, got the same name and the second
failed, losing its reports. Each name now carries a number after the
timestamp that goes up by one for each file the server starts to
write, so names still sort by time and never repeat within a run. A
failed write uses up its number, leaving a gap if the file could not
be created and otherwise a file under that number that may be
incomplete.

Model: opus-5-5
2026-09-29 08:05:26 +02:00
clawbot d2f219ca19 upaas: health check, settings checked at start, README section (closes #59)
check / check (push) Successful in 15s
The image's HEALTHCHECK requests /.well-known/healthcheck through
nginx on the port from PORT, so it fails unless both processes answer.
The backend reads PORT and DEBUG with strconv instead of viper, which
turned a bad PORT into 0 and a bad DEBUG into false. Those, and a
BIND_ADDRESS that is not an IP address, now stop the start with an
error naming the variable; the TRUSTED_PROXIES error names it too.
bin/entrypoint.sh also refuses a container PORT outside 1 to 65535,
or 8081, where the backend listens, naming PORT. README.md gains
"Running under upaas". Its first-run steps create the host directory
owned by uid 1000, so the image changes no ownership.

Model: opus-5-5
2026-09-29 06:39:10 +02:00
clawbot ced1956b06 nginx: listen on PORT, default 8080; server_tokens off (closes #26)
check / check (push) Successful in 15s
nginx.conf is now a template the nginx image renders into conf.d at
container start. bin/entrypoint.sh sets PORT to 8080 when unset or
empty, and stops with an error before starting anything when PORT is
not digits only: nginx would take a value such as localhost or
unix:/tmp/x.sock as an address and start anyway. NGINX_ENVSUBST_FILTER
limits the rendering to PORT, so $uri, $host and every other nginx
variable pass through unchanged. server_tokens off drops the version
from the Server header and error pages. script/frontend-viewport-test
renders the template the same way. EXPOSE still documents 8080; the
backend stays on 127.0.0.1:8081.

Model: opus-5-5
2026-09-29 04:55:48 +02:00
clawbot ea66caf338 fix(backend): rate-limit and cap report ingest, drop wildcard CORS (closes #20)
check / check (push) Successful in 11s
POST /api/v1/reports stays unauthenticated but is bounded. Each client
address, as the trusted-proxy logic resolves it, may send
REPORTS_PER_MINUTE reports a minute (default 60, counted by
go-chi/httprate over a sliding minute); past that it gets 429 with
Retry-After. reportbuf refuses a report that would take the report
files past DATA_DIR_MAX_BYTES (default 1 GiB), counting the files
already in DATA_DIR and unwritten reports at their uncompressed size;
the handler answers 507. CORS adds nothing unless CORS_ALLOWED_ORIGINS
lists origins. A limit that is not a positive number, or an origin
that is not a plain scheme://host[:port], stops the server from
starting.

Model: opus-5-5
2026-09-29 04:22:19 +02:00
clawbot bbcc7d921d build: one image, nginx in front of the backend on loopback (closes #52)
check / check (push) Successful in 12s
The root Dockerfile builds the only image; Dockerfile.backend is gone.
Its stages: lint, a Go stage that runs the tests and builds
netwatch-server, the node stage, and an nginx runtime. nginx serves
dist/ on 8080 and proxies /api/ and /.well-known/healthcheck to the
backend on 127.0.0.1:8081. bin/entrypoint.sh starts both, turns TERM or
INT into a stop of both, and exits non-zero when either exits on its
own. The backend runs as user netwatch and keeps reports on the /data
volume. New setting BIND_ADDRESS (empty: every interface). STOPSIGNAL is
SIGTERM, since the nginx image's SIGQUIT would miss the entrypoint.
script/docker is the org model verbatim.

Model: opus-5-5
2026-09-29 02:59:33 +02:00
clawbot de4e86c433 build: unify the gate so root make check covers the backend (closes #16)
check / check (push) Successful in 11s
Root make check, and with it the pre-commit hook, now gates the Go
backend too. The backend's Makefile targets are shims over
backend/script/*; script/cibuild builds both images and is the
workflow's only build step. Root make test runs both halves within one
30-second timeout.

Root make lint runs golangci-lint only in Docker, by building the lint
stage of Dockerfile.backend without the cache; the .golangci.yml drift
check moved into backend/script/lint. script/bootstrap installs no
linter: it reuses a Go at least as new as backend/go.mod asks for,
otherwise installs the pinned, hash-verified release, linked into
~/.local/bin without replacing anything it did not create. With VERSION
unset or empty, the backend version falls back to git describe inside a
git checkout, then to dev.

Model: opus-5-5
2026-09-29 01:22:08 +02:00
clawbot 45d2ad21bc feat(frontend): post collected samples to /api/v1/reports (closes #53)
check / check (push) Successful in 9s
A Reporter beside AppState POSTs each host's unreported, non-paused
samples to /api/v1/reports every reportInterval (default 60s).
buildReport is an exported pure function of host state; init() runs only
when #app exists, so a test can import the module. A per-host mark
advances only on a delivered POST. At most one report POST is pending at
a time and it is abandoned after half the interval, so a slow POST never
overlaps the next report and a mark never moves backwards. The samples of
an abandoned POST are sent again at the next interval, so a backend that
stored them but answered late receives them twice. Failure is quiet and
never blocks probing. vite.config.js proxies /api for yarn dev.

Model: opus-5-5
2026-09-28 23:39:11 +02:00
clawbot 503399e020 fix(backend): report ingest correctness: 500 on a refused report, 413 on oversize, global body cap (closes #23)
check / check (push) Successful in 10s
A report the buffer refuses now returns 500 instead of a false `ok`.
Reports reach disk later, so a failed disk write is still answered 200
and shows in the log, and at shutdown as a failed stop with a non-zero
exit. An over-limit body returns 413; malformed JSON stays 400. A
MaxBodyBytes middleware caps every route at 1 MiB; a route group can
only lower that limit. The raw geo blob is no longer logged; client_id,
timestamp and decode error text are cut to 128 bytes before logging.
Panic recovery logs the panic value and stack through slog.

Model: opus-5-5
2026-09-28 20:39:39 +02:00
clawbot 7a1ee6e5a8 lint: adopt org-standard .golangci.yml and golangci-lint v2.12.2 (closes #14)
check / check (push) Failing after 1s
The old backend/.golangci.yml declared version "2" but used v1 schema
keys, so under v2 it never validated and its thresholds were inert: the
linter ran at defaults. Replace it verbatim with the org-standard file,
repin the Dockerfile.backend lint stage to golangci-lint v2.12.2, and
assert the config's sha256 as the first step of the backend lint target
so it cannot silently drift again -- a local hash check, no network.

The standard config surfaces findings only in the tests: the repeated
IP literals in middleware_test.go become named constants (goconst) and
its request switches to NewRequestWithContext (noctx). reportbuf.go's
gosec suppression gains a plain justification comment. The rest of the
backend, including the fx-based server lifecycle, is already clean.
TODO.md updated.

Model: opus-4-8
2026-09-21 19:30:06 +02:00
clawbot d7cf010e00 fix(server): shut down through fx so buffered reports flush (closes #22)
check / check (push) Successful in 1m11s
The server ran os.Exit at the end of its own goroutine, racing fx's
teardown and sometimes killing the process before reportbuf's OnStop
flushed — silently losing a full flush window of telemetry on every
restart, at exit 0. Shutdown now goes through fx.Shutdowner, so every
OnStop runs in order.

The http.Server is built synchronously in OnStart before the serving
goroutine, so shutdown can no longer race or nil-deref it. A listen
failure exits non-zero via fx.ExitCode(1). reportbuf's OnStop is guarded
by sync.Once. writeTimeout now exceeds the chi per-request budget so that
budget is reachable. Dead startupTime, exitCode, and cancelFunc fields
are gone. A new test asserts a buffered report reaches disk after the
lifecycle stops.

Model: opus-4-8
2026-09-21 18:47:12 +02:00
clawbot 14eb376d79 test: automated responsive-layout harness (closes #13)
check / check (push) Failing after 1s
`make frontend-viewport-test` builds `dist/`, serves it from the same
digest-pinned nginx image and nginx.conf the shipping container uses, and
drives a digest-pinned headless Chrome over CDP. Viewport widths are derived
from the app's own @media breakpoints rather than a list of phone models: each
breakpoint is tested one pixel below, on, and above, plus four anchor
viewports. Assertions are on computed layout — horizontal overflow, off-screen
elements, clipped text, 44x44 tap targets, host-row reflow — not screenshots,
and each check declares the minimum elements it must find so a stale selector
fails instead of passing blind against a page it is not measuring. Kept out of
`make check`: it needs Docker and takes minutes. Proven able to fail before
being trusted.

Model: opus-4-8
2026-09-21 18:29:20 +02:00
clawbot f3895789d2 feat(backend): server hardening: timeouts, security headers, trusted-proxy client IP (closes #19)
check / check (push) Failing after 1s
Add ReadHeaderTimeout and IdleTimeout to the http.Server as named constants beside the existing timeouts. Add a SecurityHeaders middleware (HSTS, a JSON-API CSP of default-src 'none'; frame-ancestors 'none', X-Frame-Options DENY, nosniff, Referrer-Policy, Permissions-Policy), registered before CORS so preflight responses carry it. Resolve the client IP from X-Forwarded-For / X-Real-IP only when the direct peer is in the trusted-proxy allowlist (loopback plus RFC1918 by default, configurable via TRUSTED_PROXIES); an untrusted peer's forwarded headers are ignored. Uses net/netip; no new dependency.

Model: opus-4-8 (implementation and review); claude-fable-5 (merge)
2026-09-21 15:05:25 +02:00
clawbot f7c7f92e27 fix(frontend): meet the 44x44 minimum tap target on every control (closes #43)
check / check (push) Successful in 10s
2026-08-10 16:12:00 +02:00
clawbot 852a11eec2 fix: wrap per-host status line so 320px viewport does not scroll (closes #42)
check / check (push) Has been cancelled
2026-08-10 16:07:28 +02:00
clawbot 25a852d35c build: Dockerfile.backend multistage lint stage (closes #17)
check / check (push) Has been cancelled
2026-08-10 16:04:48 +02:00
clawbot a644efe9ff chore: root .editorconfig and hardened .gitignore (closes #15)
check / check (push) Successful in 45s
2026-08-10 15:47:49 +02:00
clawbotandsneak fbfe1df349 frontend: gate the Docker build on make check (closes #11) (#12)
check / check (push) Successful in 27s
Resolves #11. The frontend/root `Dockerfile` ran only `RUN yarn build`, so `script/lint` and `script/fmt-check` (prettier) never gated CI — only a broken build failed it. (`script/cibuild`'s comment even claimed "the Dockerfile runs make check", which was false.) The backend `Dockerfile.backend` already runs `make check`; nothing covered the frontend's lint/fmt-check.

Change (single file, `Dockerfile`):
- `apk add ... git` -> `apk add ... git make` (build stage needs `make`).
- `RUN yarn build` -> `RUN make check` — which runs `script/test` (`yarn build`, producing `dist/`) then `script/lint` + `script/fmt-check`. `dist/` is still produced in one build (no redundant rebuild); the final nginx runtime image is unchanged.

Verified via a fresh clone (a worktree's `.git` pointer breaks `vite`'s `git rev-parse`, so builds must come from a real checkout — as CI's `actions/checkout` provides): positive `docker build` succeeds with in-image `make check` green; a negative test (a prettier-violating but build-valid file) makes the build fail at `make check`, confirming CI now goes red on a check regression, not just a broken build.

Left open for review (not merged).

Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #12
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 17:46:37 +02:00
sneak e45bc578b2 scripts-to-rule-them-all (#10)
check / check (push) Successful in 21s
Reviewed-on: #10
Co-authored-by: sneak <sneak@sneak.berlin>
Co-committed-by: sneak <sneak@sneak.berlin>
2026-07-07 02:14:16 +02:00
sneak 247a3c33fd TODO (#9)
check / check (push) Successful in 22s
Reviewed-on: #9
2026-07-06 21:20:37 +02:00
74 changed files with 5889 additions and 445 deletions
+1
View File
@@ -1,5 +1,6 @@
node_modules node_modules
dist dist
tmp
.DS_Store .DS_Store
*.log *.log
.claude .claude
+4 -2
View File
@@ -6,5 +6,7 @@ jobs:
steps: steps:
# actions/checkout v4.2.2, 2026-02-22 # actions/checkout v4.2.2, 2026-02-22
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- run: docker build . # script/cibuild bootstraps, runs every check and builds the
- run: docker build -f Dockerfile.backend . # image. script/bootstrap links what it installs into
# ~/.local/bin, so that has to be on PATH for the rest.
- run: PATH="$HOME/.local/bin:$PATH" script/cibuild
+26 -2
View File
@@ -1,4 +1,28 @@
node_modules/ # OS
dist/
.DS_Store .DS_Store
Thumbs.db
# Editors
*.swp
*.swo
*~
*.bak
.idea/
.vscode/
*.sublime-*
# Node
node_modules/
# Environment / secrets
.env
.env.*
*.pem
*.key
# Build output
dist/
tmp/
# Logs
*.log *.log
+1
View File
@@ -1,5 +1,6 @@
backend/ backend/
dist/ dist/
node_modules/ node_modules/
tmp/
yarn.lock yarn.lock
.claude/ .claude/
+86 -7
View File
@@ -1,18 +1,97 @@
# The one image netwatch ships: nginx serves the built frontend and
# passes /api/ and /.well-known/healthcheck to netwatch-server, the Go
# backend, which runs in the same container on loopback only.
# bin/entrypoint.sh starts and watches both.
# Lint stage — fast feedback on formatting and lint issues. The
# golangci/golangci-lint image ships Go, gofmt, make and the linter, so
# nothing is installed here. The root make lint builds this stage alone.
# golangci/golangci-lint:v2.12.2 (2026-08-10)
FROM golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
WORKDIR /src
COPY backend/go.mod backend/go.sum ./
RUN go mod download
COPY backend/ .
RUN make fmt-check
RUN make lint
# Backend build stage
# golang:1.25-alpine (2026-02-27)
FROM golang:1.25-alpine@sha256:f6751d823c26342f9506c03797d2527668d095b0a15f1862cddb4d927a7a4ced AS builder
RUN apk add --no-cache make
WORKDIR /src
# Force BuildKit to run the lint stage before proceeding. BuildKit runs
# stages in parallel by default; without this no-op copy a lint failure
# would not gate compilation.
COPY --from=lint /src/go.sum /dev/null
COPY backend/go.mod backend/go.sum ./
RUN go mod download
COPY backend/ .
RUN make test
# make build is a shim around backend/script/build, the one definition
# of the build command:
# CGO_ENABLED=0 go build -trimpath -ldflags "-s -w -X main.Version=... -X main.Buildarch=..."
# That script reads VERSION from the environment, so it is handed over
# there rather than as a make variable.
ARG VERSION=dev
RUN VERSION="${VERSION}" make build
# Frontend stage
# node:22-alpine as of 2026-02-22 # node:22-alpine as of 2026-02-22
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS build FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS frontend
WORKDIR /app WORKDIR /app
COPY package.json yarn.lock ./ COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile RUN yarn install --frozen-lockfile
RUN apk add --no-cache git RUN apk add --no-cache git make
COPY . . COPY . .
RUN yarn build # make frontend-check is the frontend half of make check (test + lint +
# fmt-check); its test step is the production yarn build, so this both
# produces dist/ and gates the image on lint/fmt-check/test regressions.
# This node stage has neither Go nor Docker; the lint and builder stages
# above gate the backend half.
RUN make frontend-check
# Runtime stage
# nginx:stable-alpine as of 2026-02-22 # nginx:stable-alpine as of 2026-02-22
FROM nginx@sha256:15e96e59aa3b0aada3a121296e3bce117721f42d88f5f64217ef4b18f458c6ab FROM nginx@sha256:15e96e59aa3b0aada3a121296e3bce117721f42d88f5f64217ef4b18f458c6ab
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d/netwatch.conf
COPY --from=build /app/dist /usr/share/nginx/html
# netwatch-server runs as this user, which owns the report directory.
# nginx keeps the image's own arrangement: its main process runs as
# root, its worker processes as the nginx user.
RUN addgroup -g 1000 -S netwatch && \
adduser -u 1000 -S netwatch -G netwatch
# At start-up the nginx image renders every template here into
# conf.d; bin/entrypoint.sh says how.
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/templates/netwatch.conf.template
COPY security-headers.conf /etc/nginx/security-headers.conf
COPY --from=frontend /app/dist /usr/share/nginx/html
COPY --from=builder /src/netwatch-server /usr/local/bin/netwatch-server
COPY bin/entrypoint.sh /usr/local/bin/entrypoint.sh
# bin/entrypoint.sh creates DATA_DIR at start and gives it and /data to
# the netwatch user, whatever is mounted there.
ENV DATA_DIR=/data/reports
VOLUME /data
# The default public port; PORT changes it.
EXPOSE 8080 EXPOSE 8080
CMD ["nginx", "-g", "daemon off;"] # Requests the backend's health check through nginx, on the port from
# PORT, so it fails unless both answer. upaas reads the result 60
# seconds after a deploy and fails the deploy unless it is healthy.
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD wget -q -O /dev/null "http://127.0.0.1:${PORT:-8080}/.well-known/healthcheck"
# The nginx image stops its container with SIGQUIT; the entrypoint
# acts on TERM and INT.
STOPSIGNAL SIGTERM
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
-25
View File
@@ -1,25 +0,0 @@
# golang:1.25-alpine (2026-02-27)
FROM golang:1.25-alpine@sha256:f6751d823c26342f9506c03797d2527668d095b0a15f1862cddb4d927a7a4ced AS builder
RUN apk add --no-cache git make gcc musl-dev
# golangci-lint v2.7.2 (2026-02-27)
RUN CGO_ENABLED=0 go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@9f61b0f53f80672872fced07b6874397c3ed197b
WORKDIR /repo/backend
COPY backend/go.mod backend/go.sum ./
RUN go mod download
COPY .git /repo/.git
COPY backend/ .
RUN make check
RUN make build
# alpine:3.23 (2026-02-27)
FROM alpine:3.23@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659
RUN apk add --no-cache ca-certificates
COPY --from=builder /repo/backend/netwatch-server /usr/local/bin/netwatch-server
EXPOSE 8080
ENTRYPOINT ["netwatch-server"]
+34 -7
View File
@@ -1,21 +1,48 @@
.PHONY: dev test lint fmt fmt-check check docker .PHONY: bootstrap setup dev test lint fmt fmt-check check frontend-check \
frontend-viewport-test docker hooks
# Standard targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
# of README.md). test, lint, fmt, fmt-check and check cover the whole
# repo: the frontend here and the Go backend in backend/.
bootstrap:
@script/bootstrap
setup:
@script/setup
dev: dev:
yarn dev yarn dev
test: test:
timeout 30 yarn build @script/test
lint: lint:
yarn prettier --check . @script/lint
fmt: fmt:
yarn prettier --write . @script/fmt
fmt-check: fmt-check:
yarn prettier --check . @script/fmt-check
check: test lint fmt-check check:
@script/check
# The frontend half of check, for Dockerfile's node build stage, which
# has neither Go nor Docker. Use check everywhere else.
frontend-check:
@script/frontend-check
# Responsive-layout verification in a containerised browser. Kept out of
# check: it needs Docker and takes minutes, where make test has to stay
# under 20 seconds.
frontend-viewport-test:
@script/frontend-viewport-test
docker: docker:
timeout 300 docker build -t netwatch . @script/docker
hooks:
@script/install-precommit
+119 -3
View File
@@ -23,6 +23,63 @@ docker build -t netwatch .
docker run -p 8080:8080 netwatch docker run -p 8080:8080 netwatch
``` ```
`yarn dev` proxies `/api` to `http://127.0.0.1:8080`, so a locally running
`netwatch-server` (see `backend/`) receives the reports the page posts.
## Entrypoints
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them.
The Go backend in `backend/` has its own `script/` directory and shim Makefile
(see [backend/README.md](backend/README.md)). The root scripts cover both
halves, so the root `make check` fails if either one is broken. We provide:
- `script/bootstrap` — install all dependencies (the pinned node via nvm unless
one new enough for the frontend's dependencies is installed, yarn via
corepack, `yarn install --frozen-lockfile`, the pinned Go unless one at least
as new as `backend/go.mod` asks for is installed, and the Go modules), linking
what it installs itself into `~/.local/bin`, which has to be on `PATH`. It
installs no Go linter and not Docker: `make lint` runs the linter in Docker
- `script/setup` — make a fresh clone ready for development: bootstrap plus the
git pre-commit hook
- `script/projectname` — print the project name (used for the Docker image tag)
- `script/test` — run `script/frontend-test`, then the backend's Go tests, both
within one 30-second timeout
- `script/lint` — run `script/frontend-lint`, then golangci-lint in Docker, by
building the lint stage of `Dockerfile` without the cache
- `script/fmt` — format all files (writes): prettier, then gofmt over `backend/`
- `script/fmt-check` — check formatting (read-only): prettier, then gofmt
- `script/check` — run test, lint, and fmt-check
- `script/frontend-test` — run the production build as the frontend's test (no
unit tests yet)
- `script/frontend-lint` — run prettier in check mode
- `script/frontend-fmt` — format everything prettier understands (writes)
- `script/frontend-fmt-check` — check prettier formatting (read-only)
- `script/frontend-check` — the frontend half of `script/check`, for
`Dockerfile`, whose node build stage has neither Go nor Docker
- `script/frontend-viewport-test` — responsive-layout verification of the built
frontend in a containerised headless Chrome (see
[test/viewport/README.md](test/viewport/README.md)). Not part of
`script/check`: it needs Docker and takes minutes.
- `script/docker` — build the image from `Dockerfile` without the build cache,
tagged `netwatch` via `script/projectname`
- `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`,
then builds the image as `script/docker` does, without the build cache
- `script/precommit` — run by the git pre-commit hook; runs `script/check`
- `script/install-precommit` — install the git pre-commit hook
## Responsive layout
The narrow-viewport layout lives in the `max-width: 768px` media block in
`src/styles.css`. It is verified automatically by `make frontend-viewport-test`,
which drives a digest-pinned headless Chrome against the built `dist/` and
asserts on computed layout at widths derived from that CSS — one pixel either
side of every breakpoint it declares, plus a 320px floor, a desktop baseline and
two landscape sizes. See [test/viewport/README.md](test/viewport/README.md) for
what it covers and what it genuinely cannot.
## Rationale ## Rationale
When debugging network issues, it's useful to have a persistent at-a-glance view When debugging network issues, it's useful to have a persistent at-a-glance view
@@ -48,6 +105,18 @@ code lives in `src/main.js` with a class-based architecture:
- **`tick()`**: Main loop — measures all hosts in parallel via `Promise.all`, - **`tick()`**: Main loop — measures all hosts in parallel via `Promise.all`,
pushes samples, redraws UI. When paused, pushes blank markers (no probes, no pushes samples, redraws UI. When paused, pushes blank markers (no probes, no
false outage) false outage)
- **`Reporter`**: Posts collected samples to the backend
### Reporting
Every `reportInterval` (default 60s) the page POSTs a JSON report to the
same-origin path `/api/v1/reports`: a random per-browser `clientId` kept in
`localStorage`, `geo` sent as null, and each host's unreported, non-paused
samples (timestamp, latency, error). A per-host high-water mark makes every
report a delta, so only new samples are sent; the mark advances only on a
delivered report, and while paused nothing is sent. Delivery failure is quiet —
one debug-log line per outage, retried at the next interval, never blocking
probing. The report-building step is a pure function of host state.
### Monitoring targets ### Monitoring targets
@@ -110,13 +179,60 @@ After running `yarn build`, deploy the contents of the `dist/` directory to any
static file host (S3, GCS, Cloudflare Pages, Vercel, Netlify, GitHub Pages) or static file host (S3, GCS, Cloudflare Pages, Vercel, Netlify, GitHub Pages) or
use the Docker image behind a reverse proxy. use the Docker image behind a reverse proxy.
The Docker image: The Docker image, built from `Dockerfile`, is the whole service in one
container: nginx serves the built frontend and passes `/api/` and
`/.well-known/healthcheck` to the Go backend, `netwatch-server`, which listens
only inside the container, on `127.0.0.1:8081`. The image:
- Listens on port 8080 by default (override with `PORT` env var) - Listens on port 8080 by default (override with `PORT` env var)
- Trusts `X-Forwarded-For` from RFC1918 reverse proxies (10/8, 172.16/12, - Takes the client address from `X-Forwarded-For` only on requests from the
192.168/16) reverse proxies named in `TRUSTED_PROXIES`, and by default from none
- Sends access logs to stdout - Sends access logs to stdout
- Caches static assets with immutable headers - Caches static assets with immutable headers
- Sends the security headers `REPO_POLICIES.md` requires on every response, as
`security-headers.conf` sets them, in place of the backend's own
- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
volume. Before the backend starts, the image creates `DATA_DIR` and gives it
and `/data` to user `netwatch` (uid 1000), which the backend runs as, so a
host directory bind-mounted at `/data` ends up owned by uid 1000
- Writes buffered reports to disk on `docker stop`, and exits non-zero if nginx
or the backend exits on its own, so the platform restarts it
## Running under upaas
What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
- **Port:** container port `8080`.
- **Volume:** container path `/data`; the reports are kept in `/data/reports`.
- **Environment variables:** none is required. An empty one counts as unset, and
one set to a value netwatch cannot use stops the container at start, with the
reason in its log.
- `PORT`, default `8080`: the container port, from 1 to 65535. `8081` cannot
be used: the backend listens on it inside the container
- `REPORTS_PER_MINUTE`, default `60`: reports each client address may send a
minute
- `DATA_DIR_MAX_BYTES`, default `1073741824` (1 GiB): the most room the
report files may take
- `CORS_ALLOWED_ORIGINS`, default empty: other origins whose pages may call
the API
- `DEBUG`, default `false`: debug logging
- `DATA_DIR`, default `/data/reports`: leave unset; reports kept outside
`/data` do not survive a redeploy
- `TRUSTED_PROXIES`, default empty: set it to the address the reverse proxy
in front of the container connects from, as an IP address or CIDR; several
are separated by commas. nginx takes the client address from
`X-Forwarded-For` only on a request from one of them, and the rate limit
counts that address. Unset, `X-Forwarded-For` is ignored and every client
behind the proxy shares the proxy's one allowance of `REPORTS_PER_MINUTE`.
Name only addresses nothing but the proxy connects from: any client that
connects from one can write its own `X-Forwarded-For`, and through a port
Docker publishes, every client may connect from the Docker network's
gateway, such as `172.17.0.1`.
- **Health check:** the image's `HEALTHCHECK` requests
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
both nginx and the backend answer. upaas reads the container's health 60
seconds after a deploy and fails the deploy unless it is `healthy`. The
container also stops when either process exits.
## Browser Compatibility ## Browser Compatibility
+383 -83
View File
@@ -1,108 +1,408 @@
# Development Policies ---
title: Repository Policies
last_modified: 2026-07-06
---
- Docker image references by tag are server-mutable, therefore using them is an This document covers repository structure, tooling, and workflow standards. Code
RCE vulnerability. All docker image references must use cryptographic hashes style conventions are in separate documents:
to securely specify the exact image that is expected.
- Correspondingly, `go install` commands using things like '@latest' are also - [Code Styleguide](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE.md)
dangerous RCE. Whenever writing scripts or tools, ALWAYS specify go install (general, bash, Docker)
targets using commit hashes which are cryptographically secure. - [Go](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_GO.md)
- [JavaScript](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_JS.md)
- [Python](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/CODE_STYLEGUIDE_PYTHON.md)
- [Go HTTP Server Conventions](https://git.eeqj.de/sneak/prompts/raw/branch/main/prompts/GO_HTTP_SERVER_CONVENTIONS.md)
- Every repo with software in it must have a Makefile in the root. Each such ---
Makefile should support `make test` (runs the project-specific tests),
`make lint`, `make fmt` (writes), `make fmt-check` (readonly), and
`make check` (has `test`, `lint`, and `fmt-check` as prereqs), `make docker`
(builds docker image).
- Every repo should have a Dockerfile. If the repo contains non-server software, - Cross-project documentation (such as this file) must include
the Dockerfile should bring up a development environment and `make check` `last_modified: YYYY-MM-DD` in the YAML front matter so it can be kept in sync
(i.e. the docker build should fail if the branch is not green). with the authoritative source as policies evolve.
- Platform-specific standard formatting should be used. `black` for python, - **ALL external references must be pinned by cryptographic hash.** This
`prettier` for js/css/etc, `go fmt` for go. The only changes to default includes Docker base images, Go modules, npm packages, GitHub Actions, and
settings should be to specify four-space indents where applicable (i.e. anything else fetched from a remote source. Version tags (`@v4`, `@latest`,
everything except `go fmt`). `:3.21`, etc.) are server-mutable and therefore remote code execution
vulnerabilities. The ONLY acceptable way to reference an external dependency
is by its content hash (Docker `@sha256:...`, Go module hash in `go.sum`, npm
integrity hash in lockfile, GitHub Actions `@<commit-sha>`). No exceptions.
This also means never `curl | bash` to install tools like pyenv, nvm, rustup,
etc. Instead, download a specific release archive from GitHub, verify its hash
(hardcoded in the Dockerfile or script), and only then install. Unverified
install scripts are arbitrary remote code execution. This is the single most
important rule in this document. Double-check every external reference in
every file before committing. There are zero exceptions to this rule.
- If local testing is possible (it is not always), `make check` should be a - Every repo with software must have a root `Makefile` with these targets:
pre-commit hook. If it is not possible, `make lint && make fmt-check` should `make bootstrap`, `make setup`, `make test`, `make lint`, `make fmt` (writes),
be a pre-commit hook. `make fmt-check` (read-only), `make check` (runs `test`, `lint`, `fmt-check`),
`make docker`, and `make hooks` (installs pre-commit hook). A model Makefile
is at `https://git.eeqj.de/sneak/prompts/raw/branch/main/Makefile`.
- If a working `make test` takes more than 20 seconds, that's a bug that needs - Repos follow the
fixing. In fact, there should be a timeout specified in the `Makefile` that [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
fails it automatically if it takes >30s. pattern: the implementation of each Makefile target lives in an executable
script in `script/` (`script/bootstrap`, `script/setup`, `script/test`,
`script/lint`, `script/fmt`, `script/fmt-check`, `script/check`,
`script/docker`), and the Makefile targets are thin shims that call them. The
scripts must be POSIX sh (`#!/bin/sh`, `set -eu`, no bashisms) so they run in
minimal containers (e.g. alpine images have no bash); locate the repo root
with `$(cd "$(dirname "$0")/.." && pwd -P)` and `cd` there before acting. From
the standard's canonical set we use `bootstrap`, `setup` (make the repo ready
for development after a fresh clone: runs `bootstrap`, then
`install-precommit`, plus any repo-specific initialization), `test`, and
`cibuild`. `script/bootstrap` installs all dependencies idempotently and
assumes nothing is present: base tools come from nix, apt, brew, or apk
(detected in that order; apt runs noninteractive). For node it uses the
installed node if present; otherwise it installs a PINNED node version via
nvm, first installing nvm itself if missing — from a hash-verified GitHub
release archive (never `curl | sh`), with bash installed as an explicit
prerequisite since nvm requires bash. yarn is then pinned via
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
always exact versions. `script/cibuild` runs the CI build: it changes to the
repo root and runs `docker build .`; the Gitea workflow calls it. Four further
scripts are our own extensions to the standard: `script/check` runs
`script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is
what the git pre-commit hook runs, and it calls `script/check`;
`script/install-precommit` installs the git pre-commit hook (the `make hooks`
target shims to it); and `script/projectname` (literally that filename) simply
outputs the project's name. Scripts that need the name call
`script/projectname` — e.g. `script/docker` assembles its image tag from it —
so those scripts stay byte-identical across all repos. Repo-type-specific
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
`script/precommit`, not in the hook itself. Model scripts are at
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
must document the provided scripts in an **Entrypoints** section (see the
README requirements below).
- Docker builds should time out in 5 minutes or less. - Always use Makefile targets (`make fmt`, `make test`, `make lint`, etc.)
instead of invoking the underlying tools directly. The Makefile is the single
source of truth for how these operations are run.
- The Makefile is authoritative documentation for how the repo is used. Beyond
the required targets above, it should have targets for every common operation:
running a local development server (`make run`, `make dev`), re-initializing
or migrating the database (`make db-reset`, `make migrate`), building
artifacts (`make build`), generating code, seeding data, or anything else a
developer would do regularly. If someone checks out the repo and types
`make<tab>`, they should see every meaningful operation available. A new
contributor should be able to understand the entire development workflow by
reading the Makefile.
- Every repo should have a `Dockerfile`. All Dockerfiles must run `make check`
as a build step so the build fails if the branch is not green. For non-server
repos, the Dockerfile should bring up a development environment and run
`make check`. For server repos, `make check` should run as an early build
stage before the final image is assembled. Dockerfiles install development
prerequisites by running `script/bootstrap` rather than duplicating installs
inline; COPY `script/` and the dependency manifests (`package.json` +
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
layer stays cached until dependencies change.
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
repos use a multistage build where linting runs in an independent stage based
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
`make fmt-check` and `make lint` before the full build begins. The build stage
then declares an explicit dependency on the lint stage via
`COPY --from=lint /src/go.sum /dev/null`, which forces BuildKit to complete
linting before proceeding to compilation and tests. This ensures lint failures
surface in seconds rather than minutes, without blocking on dependency
download or compilation in the build stage.
The standard pattern for a Go repo Dockerfile is:
```dockerfile
# Lint stage — fast feedback on formatting and lint issues
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
FROM golangci/golangci-lint@sha256:... AS lint
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN make fmt-check
RUN make lint
# Build stage
# golang:1.x-alpine, YYYY-MM-DD
FROM golang@sha256:... AS builder
WORKDIR /src
# Force BuildKit to run the lint stage before proceeding
COPY --from=lint /src/go.sum /dev/null
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN make test
ARG VERSION=dev
RUN CGO_ENABLED=0 go build -trimpath \
-ldflags="-s -w -X main.Version=${VERSION}" \
-o /app ./cmd/app/
# Runtime stage
FROM alpine@sha256:...
COPY --from=builder /app /usr/local/bin/app
ENTRYPOINT ["app"]
```
Key points:
- The lint stage uses the `golangci/golangci-lint` image directly (it
includes both Go and the linter), so there is no need to install the
linter separately.
- `COPY --from=lint /src/go.sum /dev/null` is a no-op file copy that creates
a stage dependency. BuildKit runs stages in parallel by default; without
this line, the build stage would not wait for lint to finish and a lint
failure might not fail the overall build.
- If the project uses `//go:embed` directives that reference build artifacts
(e.g. a web frontend compiled in a separate stage), the lint stage must
create placeholder files so the embed directives resolve. Example:
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
The lint stage should not depend on the actual build output — it exists to
fail fast.
- If the project requires CGO or system libraries for linting (e.g.
`vips-dev`), install them in the lint stage with `apk add`.
- The build stage runs `make test` after compilation setup. Tests run in the
build stage, not the lint stage, because they may require compiled
artifacts or heavier dependencies.
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
runs `script/cibuild` (which runs `docker build .`) on push. Since the
Dockerfile already runs `make check`, a successful build implies all checks
pass.
- Use platform-standard formatters: `black` for Python, `prettier` for
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
two exceptions: four-space indents (except Go), and `proseWrap: always` for
Markdown (hard-wrap at 80 columns). Documentation and writing repos (Markdown,
HTML, CSS) should also have `.prettierrc` and `.prettierignore`.
- Pre-commit hook: runs `script/precommit`, which calls `script/check`. If local
testing is not possible in the repo, `script/precommit` may skip `script/test`
and run only `script/lint` and `script/fmt-check`. The hook is installed by
`script/install-precommit`; the Makefile must provide a `make hooks` target
that shims to it.
- All repos with software must have tests that run via the platform-standard
test framework (`go test`, `pytest`, `jest`/`vitest`, etc.). If no meaningful
tests exist yet, add the most minimal test possible — e.g. importing the
module under test to verify it compiles/parses. There is no excuse for
`make test` to be a no-op.
- `make test` must complete in under 20 seconds. Add a 30-second timeout in the
Makefile.
- **`make test` should use the conditional verbose rerun pattern.** Run tests
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
show full output. This keeps CI logs and `docker build` output clean on
success (just package/suite summaries) while providing full diagnostic detail
on failure (every test case, every assertion). The general shell pattern:
```makefile
test:
@<test-command> || \
{ echo "--- Rerunning with -v for details ---"; \
<test-command-with-v>; exit 1; }
```
Go example:
```makefile
test:
@go test -timeout 30s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -timeout 30s -race -v ./...; exit 1; }
```
Python example:
```makefile
test:
@python -m pytest || \
{ echo "--- Rerunning with -v for details ---"; \
python -m pytest -v; exit 1; }
```
The `exit 1` ensures the target always fails after a rerun — the first run
already proved the tests are broken, so the build must not pass even if a
flaky test happens to succeed on the second attempt. The rerun exists solely
for diagnostic output.
- Docker builds must complete in under 5 minutes.
- `make check` must not modify any files in the repo. Tests may use temporary
directories.
- `main` must always pass `make check`, no exceptions. - `main` must always pass `make check`, no exceptions.
- Do all changes on a feature branch. You can do whatever you want on a feature - Never commit secrets. `.env` files, credentials, API keys, and private keys
branch. must be in `.gitignore`. No exceptions.
- We have a standardized `.golangci.yml` which we reuse and is _NEVER_ to be - `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
modified by an agent, only manually by the user. It can be copied from editor files (`.swp`, `*~`), language build artifacts, and `node_modules/`.
`~/dev/upaas/.golangci.yml` if it exists at that location. Fetch the standard `.gitignore` from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
a new repo.
- When specifying images or packages by hash in Dockerfiles or - **No build artifacts in version control.** Code-derived data (compiled
`docker-compose.yml`, put a comment above the line and show the version and bundles, minified output, generated assets) must never be committed to the
date at which it was current. repository if it can be avoided. The build process (e.g. Dockerfile, Makefile)
should generate these at build time. Notable exception: Go protobuf generated
files (`.pb.go`) ARE committed because repos need to work with `go get`, which
downloads code but does not execute code generation.
- For javascript, always use `yarn` over `npm`. - Never use `git add -A` or `git add .`. Always stage files explicitly by name.
- Whenever writing dates, ALWAYS write YYYY-MM-DD (ISO 8601). - Never force-push to `main`.
- Simple projects should be configured with environment variables, as is - Make all changes on a feature branch. You can do whatever you want on a
standard for Dockerized applications. feature branch.
- Dockerized web services should listen on the default HTTP port of 8080 unless - `.golangci.yml` is standardized and must _NEVER_ be modified by an agent, only
overridden with the `PORT` environment variable. manually by the user. Fetch from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml`.
- The `README.md` is a project's primary documentation. It should contain at a - When pinning images or packages by hash, add a comment above the reference
minimum the following sections: with the version and date (YYYY-MM-DD).
- Description
- Include a short and complete description of the functionality and - Use `yarn`, not `npm`.
purpose of the software as the first line in the readme. It must
include: - Write all dates as YYYY-MM-DD (ISO 8601).
- the name
- the purpose - Simple projects should be configured with environment variables.
- the category (web server, SPA, command line tool, etc)
- the license - Dockerized web services listen on port 8080 by default, overridable with
- the author `PORT`.
- eg: "µPaaS is an MIT-licensed Go web application by @sneak that
receives git-frontend webhooks and interacts with a Docker server - **HTTP/web services must be hardened for production internet exposure before
to build and deploy applications in realtime as certain branches tagging 1.0.** This means full compliance with security best practices
are updated." including, without limitation, all of the following:
- Getting Started - **Security headers** on every response:
- a code block with copy-pasteable installation/use sections - `Strict-Transport-Security` (HSTS) with `max-age` of at least one year
- Rationale and `includeSubDomains`.
- why does this exist? - `Content-Security-Policy` (CSP) with a restrictive default policy
- Design (`default-src 'self'` as a baseline, tightened per-resource as
- how is the program structured? needed). Never use `unsafe-inline` or `unsafe-eval` unless
- TODO unavoidable, and document the reason.
- This is your TODO list for the project - update it meticulously, even - `X-Frame-Options: DENY` (or `SAMEORIGIN` if framing is required).
in between commits. Whenever planning, put your todo list in the Prefer the `frame-ancestors` CSP directive as the primary control.
README so that a separate agent with new context can pick up where you - `X-Content-Type-Options: nosniff`.
- `Referrer-Policy: strict-origin-when-cross-origin` (or stricter).
- `Permissions-Policy` restricting access to browser features the
application does not use (camera, microphone, geolocation, etc.).
- **Request and response limits:**
- Maximum request body size enforced on all endpoints (e.g. Go
`http.MaxBytesReader`). Choose a sane default per-route; never accept
unbounded input.
- Maximum response body size where applicable (e.g. paginated APIs).
- `ReadTimeout` and `ReadHeaderTimeout` on the `http.Server` to defend
against slowloris attacks.
- `WriteTimeout` on the `http.Server`.
- `IdleTimeout` on the `http.Server`.
- Per-handler execution time limits via `context.WithTimeout` or
chi/stdlib `middleware.Timeout`.
- **Authentication and session security:**
- Rate limiting on password-based authentication endpoints. API keys are
high-entropy and not susceptible to brute force, so they are exempt.
- CSRF tokens on all state-mutating HTML forms. API endpoints
authenticated via `Authorization` header (Bearer token, API key) are
exempt because the browser does not attach these automatically.
- Passwords stored using bcrypt, scrypt, or argon2 — never plain-text,
MD5, or SHA.
- Session cookies set with `HttpOnly`, `Secure`, and `SameSite=Lax` (or
`Strict`) attributes.
- **Reverse proxy awareness:**
- True client IP detection when behind a reverse proxy
(`X-Forwarded-For`, `X-Real-IP`). The application must accept
forwarded headers only from a configured set of trusted proxy
addresses — never trust `X-Forwarded-For` unconditionally.
- **CORS:**
- Authenticated endpoints must restrict `Access-Control-Allow-Origin` to
an explicit allowlist of known origins. Wildcard (`*`) is acceptable
only for public, unauthenticated read-only APIs.
- **Error handling:**
- Internal errors must never leak stack traces, SQL queries, file paths,
or other implementation details to the client. Return generic error
messages in production; detailed errors only when `DEBUG` is enabled.
- **TLS:**
- Services never terminate TLS directly. They are always deployed behind
a TLS-terminating reverse proxy. The service itself listens on plain
HTTP. However, HSTS headers and `Secure` cookie flags must still be
set by the application so that the browser enforces HTTPS end-to-end.
This list is non-exhaustive. Apply defense-in-depth: if a standard security
hardening measure exists for HTTP services and is not listed here, it is
still expected. When in doubt, harden.
- `README.md` is the primary documentation. Required sections:
- **Description**: First line must include the project name, purpose,
category (web server, SPA, CLI tool, etc.), license, and author. Example:
"µPaaS is an MIT-licensed Go web application by @sneak that receives
git-frontend webhooks and deploys applications via Docker in realtime."
- **Getting Started**: Copy-pasteable install/usage code block.
- **Entrypoints**: Opens by stating that the repo adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard (with that link), then documents each provided `script/`
entrypoint and its purpose.
- **Rationale**: Why does this exist?
- **Design**: How is the program structured?
- **TODO**: Update meticulously, even between commits. When planning, put
the todo list in the README so a new agent can pick up where the last one
left off. left off.
- License - **License**: MIT, GPL, or WTFPL. Ask the user for new projects. Include a
- GPL or MIT or WTFPL - ask the user when beginning a new project and `LICENSE` file in the repo root and a License section in the README.
include a LICENSE file in the root and in a section in the README. - **Author**: [@sneak](https://sneak.berlin).
- Author
- @sneak (link `@sneak` to `https://sneak.berlin`).
- When beginning a new project, initialize a git repo and make the first commit - First commit of a new repo should contain only `README.md`.
simply the first version of the README.md in the root of the repo.
- For Go packages, the module root is `sneak.berlin/go/...`, such as - Go module root: `sneak.berlin/go/<name>`. Always run `go mod tidy` before
`sneak.berlin/go/dnswatcher`. committing.
- We use SemVer always. - Use SemVer.
- If no tag `1.0.0` or greater exists in the repository, modify the existing - Database migrations live in `internal/db/migrations/` and must be embedded in
migrations and assume no installed base or existing databases. If `>=1.0.0`, the binary.
database changes add new migration files. - `000_migration.sql` — contains ONLY the creation of the migrations
tracking table itself. Nothing else.
- `001_schema.sql` — the full application schema.
- **Pre-1.0.0:** never add additional migration files (002, 003, etc.).
There is no installed base to migrate. Edit `001_schema.sql` directly.
- **Post-1.0.0:** add new numbered migration files for each schema change.
Never edit existing migrations after release.
- New repos must have at a minimum the following files: - All repos should have an `.editorconfig` enforcing the project's indentation
- `README.md`, `.git`, `.gitignore` settings.
- `POLICIES.md` (copy from `~/Documents/_PROMPTS/POLICIES.md`)
- Avoid putting files in the repo root unless necessary. Root should contain
only project-level config files (`README.md`, `Makefile`, `Dockerfile`,
`LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and
language-specific config). Everything else goes in a subdirectory. Canonical
subdirectory names:
- `bin/` — executable scripts and tools
- `cmd/` — Go command entrypoints
- `configs/` — configuration templates and examples
- `deploy/` — deployment manifests (k8s, compose, terraform)
- `docs/` — documentation and markdown (README.md stays in root)
- `internal/` — Go internal packages
- `internal/db/migrations/` — database migrations
- `pkg/` — Go library packages
- `share/` — systemd units, data files
- `static/` — static assets (images, fonts, etc.)
- `web/` — web frontend source
- When setting up a new repo, files from the `prompts` repo may be used as
templates. Fetch them from
`https://git.eeqj.de/sneak/prompts/raw/branch/main/<path>`.
- New repos must contain at minimum:
- `README.md`, `.git`, `.gitignore`, `.editorconfig`
- `LICENSE`, `REPO_POLICIES.md` (copy from the `prompts` repo)
- `Makefile`
- `script/` entrypoints (`bootstrap`, `setup`, `projectname`, `test`,
`lint`, `fmt`, `fmt-check`, `check`, `docker`, `cibuild`, `precommit`,
`install-precommit`)
- `Dockerfile`, `.dockerignore` - `Dockerfile`, `.dockerignore`
- for go: `go.mod`, `go.sum`, `.golangci.yml` - `.gitea/workflows/check.yml`
- for js: `package.json` - Go: `go.mod`, `go.sum`, `.golangci.yml`
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
- Python: `pyproject.toml`
+217
View File
@@ -0,0 +1,217 @@
# Workflow
- branch (from `main`)
- do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work)
- merge to `main` if the branch is not protected, otherwise open a PR
- push
# Status
pre-1.0. No git tags. `feat/reportbuf-storage` is merged; the backend, the CI
workflow, and the backend repo standard files are all on `main`. Frontend and
backend are both functional. Working toward the 1.0.0 milestone by closing the
remaining repo-compliance issues on the tracker.
# Next Step
Confirm the `.gitea/workflows/check.yml` run is green (main always green
policy). The workflow file is already on `main`; what is unverified is that its
latest run passes.
# Completed Steps
- 2026-09-29: the container sets up its own data directory (issue #75):
`bin/entrypoint.sh`, still as root, creates `DATA_DIR` if missing and gives it
and `/data` to the `netwatch` user with mode 750 before starting the backend
as that user, so an empty host directory owned by root, or one holding files
from another uid, works with no step on the host. It stops the start instead
when a symbolic link is on the path to `DATA_DIR`, since root would change
whatever the link points to. The `README.md` first-run step that created and
chowned the host directory is gone, and the image no longer sets that
ownership at build time
- 2026-09-29: CI can no longer pass on checks that did not run (issue #37):
`script/cibuild` is now the org model, byte for byte. It runs
`script/bootstrap` and `script/check`, then builds the image with `--no-cache`
and the version from `git describe` as the `VERSION` build argument, where it
used to be a plain `docker build .` whose check steps could come from the
build cache. The workflow puts `~/.local/bin`, where bootstrap links what it
installs, on the step's `PATH`, and bootstrap now installs its pinned node
when the installed one is older than the frontend's dependencies need
- 2026-09-29: `backend/.golangci.yml` re-vendored from `sneak/prompts` (issue
#41): `gomodguard`, deprecated in golangci-lint v2.12.0, is disabled and its
successor `gomodguard_v2` enabled with the org block list, so lint runs print
no deprecation warning. The new file also turns `depguard` on with its
`test-support` rule, which keeps `net/http/httptest` out of non-test code;
netwatch adds no entries of its own to that rule. `backend/script/lint` checks
the new sha256
- 2026-09-29: nginx sends the security headers `REPO_POLICIES.md` requires on
every response (issue #18), including errors, `/assets/` and what it passes on
from the backend, whose own copies it drops so each header goes out once. They
live in `security-headers.conf`, which `nginx.conf` includes. The content
security policy allows no inline script or style, so the status dot's grey in
`src/main.js` is now a class; `connect-src` is `*` because probed hosts
redirect to others, and the browser checks each redirect against it
- 2026-09-29: the request log is bounded (issue #60): the method, URL, protocol,
`User-Agent`, `Referer`, request ID (which chi takes from the client's
`X-Request-Id` header) and client address it writes are each cut to 128 bytes,
the bound the report handler already used, so one request can no longer put
about 1 MiB per field into a log line. That bound and its helper now live in
the `logger` package, shared by both
- 2026-09-29: nginx takes the client address from `X-Forwarded-For` only on
requests from the reverse proxies named in the container's `TRUSTED_PROXIES`
(issue #64), and by default from none, where it trusted every RFC1918 address
before, so a client could write a new address on each request and escape the
rate limit. `bin/entrypoint.sh` writes one `set_real_ip_from` line per entry
into `/etc/nginx/trusted-proxies.conf`, which `nginx.conf` includes, refusing
an entry that is not an IP address or CIDR, as `netwatch-server check-cidr`
finds; it starts the backend with `TRUSTED_PROXIES=127.0.0.1/32`, since nginx
is its only client
- 2026-09-29: report file names can no longer collide (issue #61): each is
`reports-<timestamp>-<number>.jsonl.zst`, where the number goes up by one for
each file the server starts to write, so two flushes in the same millisecond,
such as a flush for size and the final flush at shutdown, each get a file of
their own instead of the second one failing. A failed write uses up its
number, leaving a gap if the file could not be created and otherwise a file
under that number that may be incomplete.
- 2026-09-29: ready to run under upaas (issue #59): the image has a
`HEALTHCHECK` that requests `/.well-known/healthcheck` through nginx on the
port from `PORT`. The backend no longer reads a bad `PORT` as 0 or a bad
`DEBUG` as false: those, and a `BIND_ADDRESS` that is not an IP address, stop
it from starting with an error naming the variable, as the limits,
`CORS_ALLOWED_ORIGINS` and, now by name, `TRUSTED_PROXIES` already did.
`bin/entrypoint.sh` also refuses a `PORT` outside 1 to 65535, and `8081`,
where the backend listens inside the container, naming `PORT`. `README.md` has
a "Running under upaas" section, whose first-run steps create the host
directory for `/data` owned by uid 1000; the image does not change its owner
- 2026-09-29: nginx listens on `PORT` (issue #26), 8080 when unset or empty: the
nginx image renders `nginx.conf` as a template at container start, filling in
`PORT` and no other variable. `bin/entrypoint.sh` refuses to start when `PORT`
is not digits only. `server_tokens off` keeps the nginx version out of
responses. `script/frontend-viewport-test` renders the template the same way.
Gzip and a `50x.html` error page are not added
- 2026-09-29: bounded the report endpoint (issue #20): `POST /api/v1/reports`
still needs no credentials, but each client address, as resolved through
`TRUSTED_PROXIES`, may send `REPORTS_PER_MINUTE` (default 60) reports a
minute, counted by `go-chi/httprate`, and past that gets 429 with
`Retry-After`; the report files in `DATA_DIR`, counted from start with those
already there, may total at most `DATA_DIR_MAX_BYTES` (default 1 GiB), past
which reports get 507; and the wildcard CORS is gone: no CORS headers unless
`CORS_ALLOWED_ORIGINS` lists origins, and an entry that is not a plain
`scheme://host[:port]` origin, `*` included, stops the server from starting.
Deleting report files frees room only at the next start; pruning is issue #54
- 2026-09-28: one container image (issue #52): the root `Dockerfile` builds the
only image, and `Dockerfile.backend` is gone. nginx serves the frontend on
port 8080 and proxies `/api/` and `/.well-known/healthcheck` to the backend,
which listens on `127.0.0.1:8081` in the same container; the new
`BIND_ADDRESS` setting sets its listen address. `bin/entrypoint.sh` starts
both, passes TERM and INT on to both, and exits non-zero when either exits on
its own. The backend runs as user `netwatch` and stores reports on the `/data`
volume. `script/docker` is the org model again
- 2026-09-28: unified the gate (issue #16): the root `make check` covers the Go
backend as well as the frontend, and the pre-commit hook with it; the backend
moved onto scripts-to-rule-them-all (`backend/script/*`, `backend/Makefile` as
shims, its duplicate hook installer removed); `script/cibuild` builds both
images and is the workflow's only build step. The root `make lint` runs
golangci-lint only in Docker, by building the lint stage of
`Dockerfile.backend` without the cache. `script/bootstrap` installs the pinned
Go unless the installed one is at least what `backend/go.mod` asks for, links
what it installs into `~/.local/bin` without replacing anything it did not
create, and installs no linter. Root `make test` runs both halves within one
30-second timeout. When `VERSION` is unset or empty, the backend binary's
version falls back to `git describe` inside a git checkout, then to `dev`
- 2026-09-28: frontend reporting client (issue #53): a `Reporter` class posts
collected samples to `/api/v1/reports` every `reportInterval` (default 60s) as
a per-host delta, with the report-building step a pure exported function of
host state; a per-host mark advances only on a delivered POST; at most one
report POST is pending at a time and it is abandoned after half the interval,
so a slow POST never overlaps the next report and a mark never moves
backwards; the samples of an abandoned POST are sent again at the next
interval, so a backend that stored them but answered late receives them twice;
the per-browser client id works in insecure (plain-HTTP) contexts;
`vite.config.js` proxies `/api` to the local backend for `yarn dev`
- 2026-09-28: report ingest correctness (issue #23): a storage failure now
returns 500 instead of a false `ok`; oversize bodies return 413 (distinguished
from malformed JSON, which stays 400); a `MaxBodyBytes` middleware caps every
route, not just the report route; the raw attacker-controlled `geo` blob is no
longer logged (only its length), and `client_id`, `timestamp` and decode error
text are length-bounded before logging; a `decodeJSON` handler helper was
added; panic recovery now routes the stack through slog instead of chi's
plain-text stderr; and writing a report file now returns its error, so a
failed final flush on shutdown makes the process exit non-zero instead of
losing the buffered reports silently
- 2026-09-21: shutdown lifecycle correctness. The process now shuts down through
fx instead of `os.Exit`, so every component's `OnStop` runs and buffered
reports are flushed to disk on `SIGTERM` — previously a full flush window of
telemetry was silently lost on every restart. The `http.Server` is now built
before its serving goroutine starts, so shutdown can no longer race or
nil-deref it; a listen failure exits non-zero via `fx.Shutdowner`; `reportbuf`
`OnStop` is idempotent; and `writeTimeout` now exceeds the chi per-request
budget so that budget is actually reachable. Dead `startupTime`, `exitCode`,
and `cancelFunc` fields were removed
- 2026-09-21: backend HTTP hardening (issue #19): added `ReadHeaderTimeout` and
`IdleTimeout` to the server, a `SecurityHeaders` middleware (HSTS, tight CSP,
frame/sniff/referrer/permissions headers) registered before CORS, and
trusted-proxy client IP resolution honouring `X-Forwarded-For` / `X-Real-IP`
only from a `TRUSTED_PROXIES` allowlist (loopback plus RFC1918 by default)
- 2026-08-10: adopted the org-standard `backend/.golangci.yml` verbatim and
moved the pinned golangci-lint from v2.7.2 to v2.12.2 (the `lint` stage of
`Dockerfile.backend` now pins the `golangci/golangci-lint:v2.12.2` image by
digest); the previous config declared `version: "2"` but used v1 schema keys,
so every threshold in it was inert and its green result was meaningless.
`backend/Makefile`'s `lint` target now asserts the config's sha256 against the
canonical file first, so drift from the org standard fails the build instead
of silently degrading to defaults
- 2026-08-10: every interactive control now meets the 44x44 CSS px minimum tap
target (`.pin-btn`, `#interval-select`, the debug-log label and, on narrow
viewports, `#pause-btn`). The pin button's hit area grows via matching
negative margins, so its layout footprint and row density are unchanged
- 2026-08-10: per-host status line wraps below the 768px breakpoint instead of
forcing horizontal page scroll at 320px
- 2026-08-09: `Dockerfile.backend` reworked to the mandated Go multistage
lint-stage pattern: separate `lint` stage on the hash-pinned
`golangci/golangci-lint` image, `COPY --from=lint` stage dependency,
`CGO_ENABLED=0` static build driven by `ARG VERSION`, and no more `COPY .git`
- 2026-08-09: dotfile compliance — lifted `backend/.editorconfig` to the repo
root so `root = true` covers the frontend too, and replaced `.gitignore` with
the org model (OS, editor, node, and environment/secrets sections) plus this
repo's `dist/` and `*.log`. `.env`, `.env.*`, `*.pem`, and `*.key` are now
ignored repo-wide, not just under `backend/`. Excluding `.git` from
`.dockerignore` stays deferred: both images read git metadata at build time
(`COPY .git` in `Dockerfile.backend`, `git rev-parse` in `vite.config.js`)
- 2026-08-09: automated responsive-layout harness
(`make frontend-viewport-test`): digest-pinned headless Chrome driven over CDP
against the built `dist/`, viewport widths derived from the breakpoints in
`src/styles.css` ([#13](https://git.eeqj.de/sneak/netwatch/issues/13)). Every
check carries a presence guard so none of them can pass against a page it is
not actually measuring. Found two real layout defects, filed as
[#42](https://git.eeqj.de/sneak/netwatch/issues/42) and
[#43](https://git.eeqj.de/sneak/netwatch/issues/43)
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
shims, README Entrypoints section
- 2026-02-27: backend with buffered zstd-compressed report storage; CI workflow
and backend repo standard files; backend Dockerfile fixed (Go 1.25,
golangci-lint) and moved to repo root (feat/reportbuf-storage)
- 2026-02-26: host row layout redesigned with CSS grid; overflow and spacing
fixes; nginx config extracted; port hardcoded to 8080
- 2026-02-26: debug log panel, median stats, recovery probe, Docker build fix,
S3 Singapore endpoint added
- 2026-02-23: summary box redesign, host pinning, local and UTC clocks, checks
counter
- 2026-02-23: hosts sorted by latency; GET instead of HEAD for latency; timeout
derived from interval; Hetzner regional endpoints; 3s interval
- 2026-01-29: initial NetWatch network latency monitor
# Future Steps
- Wire `script/frontend-viewport-test` into CI as its own step (deliberately not
part of `make check` today; the decision has real CI-runtime cost and is
tracked separately)
- Compliance top-up as one small commit: add .editorconfig and add the hooks
target to the Makefile
- After merge, confirm .gitea/workflows/check.yml is on main and CI is green
(main always green policy)
- Decide what to do with untracked resume.sh: commit it, gitignore it, or delete
it
+71 -5
View File
@@ -1,21 +1,30 @@
version: "2" version: "2"
# Config schema uses the golangci-lint v2 layout (settings live under
# linters.settings, not top-level linters-settings) so that the
# thresholds below are actually applied by golangci-lint >= v2.
run: run:
timeout: 5m timeout: 5m
modules-download-mode: readonly modules-download-mode: readonly
linters: linters:
default: all default: all
enable:
# Successor to the deprecated gomodguard. Named explicitly, rather than
# left to `default: all`, because it carries the module policy below.
- gomodguard_v2
disable: disable:
# Genuinely incompatible with project patterns # Genuinely incompatible with project patterns
- exhaustruct # Requires all struct fields - exhaustruct # Requires all struct fields
- depguard # Dependency allow/block lists
- godot # Requires comments to end with periods - godot # Requires comments to end with periods
- wsl # Deprecated, replaced by wsl_v5
- wrapcheck # Too verbose for internal packages - wrapcheck # Too verbose for internal packages
- varnamelen # Short names like db, id are idiomatic Go - varnamelen # Short names like db, id are idiomatic Go
# Deprecated: the warning is attached to the old name, so it is
linters-settings: # silenced by disabling that name, not by enabling the successor.
- wsl # Deprecated, replaced by wsl_v5
- gomodguard # Deprecated, replaced by gomodguard_v2
settings:
lll: lll:
line-length: 88 line-length: 88
funlen: funlen:
@@ -25,8 +34,65 @@ linters-settings:
max-complexity: 15 max-complexity: 15
dupl: dupl:
threshold: 100 threshold: 100
depguard:
# Test-support code must not be compiled into the shipped binary. A
# test-support package exists to hand a test privileges the program
# itself must never have, so a file that is not a test must not import
# one. Test files, and the files inside a package whose directory name
# ends in `test`, are where that code belongs, and are exempt.
#
# The deny list below is the one part of this file a repository is
# expected to extend, and the only part it may. depguard matches an
# import path against a list of prefixes, so it cannot be told "any path
# whose last segment ends in test"; a repository's own test-support
# packages have to be named here one at a time, by full import path,
# under a module path that differs from repository to repository. Add
# them; change nothing else.
rules:
test-support:
list-mode: lax
files:
- "$all"
- "!$test"
- "!**/*test/**"
deny:
- pkg: net/http/httptest
desc: >-
Test-support code belongs in test files and in packages whose
directory name ends in test, not in the shipped binary.
# Only decisions already recorded in the Go package defaults are
# listed here. Every entry matches the module path exactly.
gomodguard_v2:
blocked:
- module: github.com/rs/zerolog
recommendations:
- log/slog
reason: "Structured logging is stdlib log/slog."
# One entry per pre-fork module path, because the later releases
# are separate paths. A prefix match would be shorter but would
# also reach github.com/go-redis/redismock, the test double for
# the successor these entries recommend.
- module: github.com/go-redis/redis
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/go-redis/redis/v7
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/go-redis/redis/v8
recommendations:
- github.com/redis/go-redis/v9
reason: "Pre-fork module; use the maintained go-redis v9."
- module: github.com/sergi/go-diff
recommendations:
- github.com/aymanbagabas/go-udiff
reason: "No unified diff output; use go-udiff."
- module: github.com/hexops/gotextdiff
recommendations:
- github.com/aymanbagabas/go-udiff
reason: "Unmaintained fork; use go-udiff."
issues: issues:
exclude-use-default: false
max-issues-per-linter: 0 max-issues-per-linter: 0
max-same-issues: 0 max-same-issues: 0
+15 -38
View File
@@ -1,53 +1,30 @@
UNAME_S := $(shell uname -s) # Thin shims; the implementations live in backend/script/ (see the
VERSION := $(shell git describe --always --dirty) # Entrypoints section of README.md). There is no check, hooks or docker
BUILDARCH := $(shell uname -m) # target here: the root Makefile's check covers this directory, its
BINARY := netwatch-server # hooks target installs the repo's only pre-commit hook, and its docker
# target builds the one image, which contains this backend.
GOLDFLAGS += -X main.Version=$(VERSION) .PHONY: all build test lint fmt fmt-check run clean
GOLDFLAGS += -X main.Buildarch=$(BUILDARCH)
ifeq ($(UNAME_S),Darwin)
GOFLAGS := -ldflags "$(GOLDFLAGS)"
else
GOFLAGS = -ldflags "-linkmode external -extldflags -static $(GOLDFLAGS)"
endif
.PHONY: all build test lint fmt fmt-check check docker hooks run clean
all: build all: build
build: ./$(BINARY) build:
@script/build
./$(BINARY): $(shell find . -name '*.go' -type f) go.mod go.sum
go build -o $@ $(GOFLAGS) ./cmd/netwatch-server/
test: test:
timeout 30 go test ./... @script/test
lint: lint:
golangci-lint run ./... @script/lint
fmt: fmt:
go fmt ./... @script/fmt
fmt-check: fmt-check:
@test -z "$$(gofmt -l .)" || \ @script/fmt-check
(echo "Files not formatted:"; gofmt -l .; exit 1)
check: test lint fmt-check run:
@script/run
docker:
timeout 300 docker build -t netwatch-server -f ../Dockerfile.backend ..
hooks:
@printf '#!/bin/sh\ncd backend && make check\n' > \
$$(git rev-parse --show-toplevel)/.git/hooks/pre-commit
@chmod +x \
$$(git rev-parse --show-toplevel)/.git/hooks/pre-commit
@echo "Pre-commit hook installed"
run: build
./$(BINARY)
clean: clean:
rm -f ./$(BINARY) @script/clean
+119 -8
View File
@@ -4,18 +4,51 @@ SPA and persists them as zstd-compressed JSONL files on disk.
## Getting Started ## Getting Started
From this directory:
```bash ```bash
# Build and run locally # Build and run locally
make run make run
```
# Run tests, lint, and format check From the repo root, whose `Dockerfile` builds the one image that ships this
backend behind nginx (see [Container image](#container-image)):
```bash
# Run tests, lint, and format check over the frontend and this backend
make check make check
# Docker # Build the image: nginx, the frontend and this backend
docker build -t netwatch-server . make docker
docker run -p 8080:8080 netwatch-server docker run -p 8080:8080 netwatch
``` ```
## Entrypoints
This directory follows the same
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
pattern as the repo root: the targets in `backend/Makefile` are thin shims over
`backend/script/`. The root `Dockerfile` runs them, and the root scripts call
`test`, `fmt` and `fmt-check`:
- `script/build` — compile the static `netwatch-server` binary with its version
and architecture stamped in. The version is `VERSION` from the environment;
when that is unset or empty, it falls back to `git describe` inside a git
checkout, then to `dev`
- `script/test` — run the Go tests under a 30-second timeout
- `script/lint` — check `.golangci.yml` against its pinned sha256, then run
golangci-lint. It runs inside the golangci-lint image of the lint stage of
the root `Dockerfile`; from a checkout, run `make lint` at the repo root,
which builds that stage
- `script/fmt` — format the Go sources (writes)
- `script/fmt-check` — check Go formatting (read-only)
- `script/run` — build and run the server locally
- `script/clean` — remove build artifacts
There is no `check`, `hooks` or `docker` target here: the root `make check`
covers this directory, the root `make hooks` installs the repo's only pre-commit
hook, and the root `make docker` builds the image that contains this backend.
## Rationale ## Rationale
The NetWatch frontend collects latency measurements from the browser but has no The NetWatch frontend collects latency measurements from the browser but has no
@@ -43,16 +76,94 @@ Internal packages in `internal/` follow standard Go project layout:
### Configuration ### Configuration
| Variable | Default | Description | | Variable | Default | Description |
| ---------- | ------------------ | --------------------------------- | | ---------------------- | -------------------- | -------------------------------------------------------------------------------------------------------- |
| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface |
| `PORT` | `8080` | HTTP listen port | | `PORT` | `8080` | HTTP listen port |
| `DATA_DIR` | `./data/reports` | Directory for compressed reports | | `DATA_DIR` | `./data/reports` | Directory for compressed reports |
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Largest total size of the report files in `DATA_DIR`; see [Report limits](#report-limits) |
| `DEBUG` | `false` | Enable debug logging | | `DEBUG` | `false` | Enable debug logging |
| `TRUSTED_PROXIES` | loopback + RFC1918 | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
`TRUSTED_PROXIES` defaults to `127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`.
The loopback entries cover a reverse proxy on the same host. A request whose
direct peer is outside this set has its forwarded headers ignored, and the
direct peer is logged and rate-limited instead. The container image does not use
this default; see [Container image](#container-image).
A variable set to a value the server cannot use, such as `PORT=abc`,
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
starting, with an error naming the variable. An empty variable counts as unset.
### Container image
The root `Dockerfile` builds one image in which nginx listens on the public port
8080, serves the frontend, and proxies `/api/` and `/.well-known/healthcheck` to
this server. The image's entrypoint, `bin/entrypoint.sh`, starts the server as
user `netwatch` (uid 1000) with `BIND_ADDRESS=127.0.0.1` and `PORT=8081`, so
only nginx reaches it, and with `TRUSTED_PROXIES=127.0.0.1/32`, so it takes the
client address nginx passes on and no other. `DATA_DIR` is `/data/reports`, on
the `/data` volume; the entrypoint creates it and gives it and `/data` to
`netwatch` before starting the server. nginx replaces the security headers
this server sets with those in the root `security-headers.conf`, so those are
what clients of the image see.
The container's own `TRUSTED_PROXIES` goes to nginx instead: IP addresses or
CIDRs, separated by commas, of the reverse proxies in front of the container.
nginx takes the client address from `X-Forwarded-For` only on a request from one
of them. Unset or empty, nginx trusts no proxy, and the client address is the
one each request comes from, so every client behind a proxy shares one rate
limit. An entry that is not an IP address or CIDR, such as a hostname or
`1.2.3`, stops the container at start with an error naming `TRUSTED_PROXIES`:
the entrypoint checks each entry with `netwatch-server check-cidr`, which parses
it as this server parses its own `TRUSTED_PROXIES`.
### Report storage ### Report storage
Reports are written as `reports-<timestamp>.jsonl.zst` files in `DATA_DIR`. Reports are written as `reports-<timestamp>-<number>.jsonl.zst` files in
Each file contains one JSON object per line, compressed with zstd. Files are `DATA_DIR`. The timestamp is in UTC to the millisecond, so the names sort by
created with `O_EXCL` to prevent overwrites. time. The number starts at 1 when the server starts and goes up by one for each
file the server starts to write, so two files written in the same millisecond
still get different names. A failed write uses up its number, leaving a gap in
the numbers if the file could not be created and otherwise a file under that
number that may be incomplete. Each file contains one JSON object per line,
compressed with zstd. Files are created with `O_EXCL` to prevent overwrites.
### Report limits
`POST /api/v1/reports` takes reports from anyone who can reach it, without
credentials, so it is bounded instead. Both refusals below answer with the same
`{"status":"error"}` body as any other error.
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
`Retry-After: 60`. The minute slides: reports from the minute before still
count, fading out over the current one, so an address is sure never to be
refused only while it sends at most half of `REPORTS_PER_MINUTE` in any 60
seconds. The page sends one report a minute from each open tab, so the default
of 60 refuses nothing from up to 30 tabs behind one address, such as a
household or an office sharing it, however their reports bunch up. Report
responses also carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and
`X-RateLimit-Reset` headers.
- **Size cap.** The report files in `DATA_DIR` may total at most
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
waiting in memory count at their uncompressed size until they are written, so
a report that would take the total past the cap is refused with 507, and
nothing of it is stored. Deleting report files frees room only at the next
start, when the files are counted again. The default of 1 GiB is small enough
for any host; set it to the space you can give `DATA_DIR`.
### CORS
The page calls the API from the origin it is served from, so by default the
server sends no CORS headers, and browsers let no other origin's pages call it.
To serve the page from elsewhere, list that origin in `CORS_ALLOWED_ORIGINS`
(for example `https://netwatch.example.com`); pages from a listed origin may
`GET` and `POST` with a `Content-Type` header. Each entry must be a plain
origin, `scheme://host` with an optional `:port`, as browsers send it: no path,
not even a trailing `/`, and no `*`. Any other entry stops the server from
starting, with an error naming `CORS_ALLOWED_ORIGINS`.
## TODO ## TODO
+16
View File
@@ -2,6 +2,9 @@
package main package main
import ( import (
"fmt"
"os"
"sneak.berlin/go/netwatch/internal/config" "sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals" "sneak.berlin/go/netwatch/internal/globals"
"sneak.berlin/go/netwatch/internal/handlers" "sneak.berlin/go/netwatch/internal/handlers"
@@ -22,6 +25,19 @@ var (
) )
func main() { func main() {
// "netwatch-server check-cidr CIDR" exits 1, with the error, if
// this server would refuse CIDR in its TRUSTED_PROXIES.
// bin/entrypoint.sh runs it on each entry it gives nginx.
if len(os.Args) == 3 && os.Args[1] == "check-cidr" {
_, err := middleware.ParseTrustedProxies(os.Args[2:])
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
return
}
globals.Appname = Appname globals.Appname = Appname
globals.Version = Version globals.Version = Version
globals.Buildarch = Buildarch globals.Buildarch = Buildarch
+4 -1
View File
@@ -5,6 +5,7 @@ go 1.25.5
require ( require (
github.com/go-chi/chi/v5 v5.2.5 github.com/go-chi/chi/v5 v5.2.5
github.com/go-chi/cors v1.2.2 github.com/go-chi/cors v1.2.2
github.com/go-chi/httprate v0.16.0
github.com/joho/godotenv v1.5.1 github.com/joho/godotenv v1.5.1
github.com/klauspost/compress v1.18.4 github.com/klauspost/compress v1.18.4
github.com/spf13/viper v1.21.0 github.com/spf13/viper v1.21.0
@@ -14,6 +15,7 @@ require (
require ( require (
github.com/fsnotify/fsnotify v1.9.0 // indirect github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/sagikazarmark/locafero v0.11.0 // indirect github.com/sagikazarmark/locafero v0.11.0 // indirect
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
@@ -21,10 +23,11 @@ require (
github.com/spf13/cast v1.10.0 // indirect github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect github.com/spf13/pflag v1.0.10 // indirect
github.com/subosito/gotenv v1.6.0 // indirect github.com/subosito/gotenv v1.6.0 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.uber.org/dig v1.19.0 // indirect go.uber.org/dig v1.19.0 // indirect
go.uber.org/multierr v1.10.0 // indirect go.uber.org/multierr v1.10.0 // indirect
go.uber.org/zap v1.26.0 // indirect go.uber.org/zap v1.26.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/sys v0.29.0 // indirect golang.org/x/sys v0.30.0 // indirect
golang.org/x/text v0.28.0 // indirect golang.org/x/text v0.28.0 // indirect
) )
+10 -2
View File
@@ -8,6 +8,8 @@ github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0= github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE= github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58= github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs= github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
@@ -16,6 +18,8 @@ github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4= github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c= github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
@@ -42,6 +46,10 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8= github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU= github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4= go.uber.org/dig v1.19.0 h1:BACLhebsYdpQ7IROQ1AGPjrXcP5dF80U3gKoFzbaq/4=
go.uber.org/dig v1.19.0/go.mod h1:Us0rSJiThwCv2GteUN0Q7OKvU7n5J4dxZ9JKUXozFdE= go.uber.org/dig v1.19.0/go.mod h1:Us0rSJiThwCv2GteUN0Q7OKvU7n5J4dxZ9JKUXozFdE=
go.uber.org/fx v1.24.0 h1:wE8mruvpg2kiiL1Vqd0CC+tr0/24XIB10Iwp2lLWzkg= go.uber.org/fx v1.24.0 h1:wE8mruvpg2kiiL1Vqd0CC+tr0/24XIB10Iwp2lLWzkg=
@@ -54,8 +62,8 @@ go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so= go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU= golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng= golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU= golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+133 -3
View File
@@ -4,7 +4,13 @@ package config
import ( import (
"errors" "errors"
"fmt"
"log/slog" "log/slog"
"math"
"net/netip"
"net/url"
"strconv"
"strings"
"sneak.berlin/go/netwatch/internal/globals" "sneak.berlin/go/netwatch/internal/globals"
"sneak.berlin/go/netwatch/internal/logger" "sneak.berlin/go/netwatch/internal/logger"
@@ -14,6 +20,32 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
) )
// defaultTrustedProxies lists the networks whose forwarded
// headers are honoured by default: IPv4 and IPv6 loopback,
// for a reverse proxy on the same host, and the RFC1918
// ranges. The container image does not use it:
// bin/entrypoint.sh gives the server 127.0.0.1/32, since
// nginx is its only client there.
const defaultTrustedProxies = "127.0.0.1/32,::1/128," +
"10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
// Default limits on stored reports; backend/README.md gives the
// reasons for these values.
const (
defaultReportsPerMinute = 60
defaultDataDirMaxBytes = 1 << 30 // 1 GiB
)
var (
errNotPositive = errors.New("must be a positive whole number")
errNotOrigin = errors.New(
"must be an origin, scheme://host with an optional port",
)
errNotPort = errors.New("must be a port number, 1 to 65535")
errNotBool = errors.New("must be true or false")
errNotIP = errors.New("must be an IP address, or empty")
)
// Params defines the dependencies for Config. // Params defines the dependencies for Config.
type Params struct { type Params struct {
fx.In fx.In
@@ -24,18 +56,24 @@ type Params struct {
// Config holds the resolved application configuration. // Config holds the resolved application configuration.
type Config struct { type Config struct {
BindAddress string
CORSAllowedOrigins []string
DataDir string DataDir string
DataDirMaxBytes int64
Debug bool Debug bool
MetricsPassword string MetricsPassword string
MetricsUsername string MetricsUsername string
Port int Port int
ReportsPerMinute int
SentryDSN string SentryDSN string
TrustedProxies []string
log *slog.Logger log *slog.Logger
params *Params params *Params
} }
// New loads configuration from env, .env files, and config // New loads configuration from env, .env files, and config
// files, returning a fully resolved Config. // files, returning a fully resolved Config. It fails, with an error
// naming the setting, on a value the server cannot use.
func New( func New(
_ fx.Lifecycle, _ fx.Lifecycle,
params Params, params Params,
@@ -50,12 +88,19 @@ func New(
viper.AutomaticEnv() viper.AutomaticEnv()
// An empty CORS_ALLOWED_ORIGINS allows no other origin.
viper.SetDefault("CORS_ALLOWED_ORIGINS", "")
viper.SetDefault("DATA_DIR", "./data/reports") viper.SetDefault("DATA_DIR", "./data/reports")
viper.SetDefault("DATA_DIR_MAX_BYTES", defaultDataDirMaxBytes)
viper.SetDefault("DEBUG", "false") viper.SetDefault("DEBUG", "false")
// An empty BIND_ADDRESS listens on every interface.
viper.SetDefault("BIND_ADDRESS", "")
viper.SetDefault("PORT", "8080") viper.SetDefault("PORT", "8080")
viper.SetDefault("REPORTS_PER_MINUTE", defaultReportsPerMinute)
viper.SetDefault("SENTRY_DSN", "") viper.SetDefault("SENTRY_DSN", "")
viper.SetDefault("METRICS_USERNAME", "") viper.SetDefault("METRICS_USERNAME", "")
viper.SetDefault("METRICS_PASSWORD", "") viper.SetDefault("METRICS_PASSWORD", "")
viper.SetDefault("TRUSTED_PROXIES", defaultTrustedProxies)
err := viper.ReadInConfig() err := viper.ReadInConfig()
if err != nil { if err != nil {
@@ -66,17 +111,41 @@ func New(
} }
} }
// Read with strconv: viper's GetInt and GetBool would read a value
// they cannot parse as 0 or false instead of failing.
port, err := strconv.Atoi(viper.GetString("PORT"))
if err != nil || port < 1 || port > math.MaxUint16 {
return nil, fmt.Errorf("PORT %q: %w",
viper.GetString("PORT"), errNotPort)
}
debug, err := strconv.ParseBool(viper.GetString("DEBUG"))
if err != nil {
return nil, fmt.Errorf("DEBUG %q: %w",
viper.GetString("DEBUG"), errNotBool)
}
s := &Config{ s := &Config{
BindAddress: viper.GetString("BIND_ADDRESS"),
CORSAllowedOrigins: splitList(viper.GetString("CORS_ALLOWED_ORIGINS")),
DataDir: viper.GetString("DATA_DIR"), DataDir: viper.GetString("DATA_DIR"),
Debug: viper.GetBool("DEBUG"), DataDirMaxBytes: viper.GetInt64("DATA_DIR_MAX_BYTES"),
Debug: debug,
MetricsPassword: viper.GetString("METRICS_PASSWORD"), MetricsPassword: viper.GetString("METRICS_PASSWORD"),
MetricsUsername: viper.GetString("METRICS_USERNAME"), MetricsUsername: viper.GetString("METRICS_USERNAME"),
Port: viper.GetInt("PORT"), Port: port,
ReportsPerMinute: viper.GetInt("REPORTS_PER_MINUTE"),
SentryDSN: viper.GetString("SENTRY_DSN"), SentryDSN: viper.GetString("SENTRY_DSN"),
TrustedProxies: splitList(viper.GetString("TRUSTED_PROXIES")),
log: log, log: log,
params: &params, params: &params,
} }
err = s.check()
if err != nil {
return nil, err
}
if s.Debug { if s.Debug {
params.Logger.EnableDebugLogging() params.Logger.EnableDebugLogging()
s.log = params.Logger.Get() s.log = params.Logger.Get()
@@ -84,3 +153,64 @@ func New(
return s, nil return s, nil
} }
// check fails with an error naming the first setting here whose value
// the server cannot use. New checks PORT and DEBUG as it reads them,
// and the middleware checks TRUSTED_PROXIES as it parses it.
func (s *Config) check() error {
// viper reads a value that is not a number as 0, so this also
// catches a mistyped setting.
if s.ReportsPerMinute <= 0 {
return fmt.Errorf("REPORTS_PER_MINUTE %q: %w",
viper.GetString("REPORTS_PER_MINUTE"), errNotPositive)
}
if s.DataDirMaxBytes <= 0 {
return fmt.Errorf("DATA_DIR_MAX_BYTES %q: %w",
viper.GetString("DATA_DIR_MAX_BYTES"), errNotPositive)
}
if s.BindAddress != "" {
_, err := netip.ParseAddr(s.BindAddress)
if err != nil {
return fmt.Errorf("BIND_ADDRESS %q: %w", s.BindAddress, errNotIP)
}
}
return checkOrigins(s.CORSAllowedOrigins)
}
// checkOrigins fails on the first CORS_ALLOWED_ORIGINS entry that is
// not a plain origin, scheme://host with an optional port, as browsers
// send it; anything more, such as a trailing "/", would match no page.
// go-chi/cors reads a "*" anywhere in an entry as a wildcard, so no
// entry may contain one.
func checkOrigins(origins []string) error {
for _, origin := range origins {
u, err := url.Parse(origin)
if err != nil || u.Scheme == "" || u.Host == "" ||
strings.Contains(origin, "*") ||
origin != u.Scheme+"://"+u.Host {
return fmt.Errorf("CORS_ALLOWED_ORIGINS %q: %w",
origin, errNotOrigin)
}
}
return nil
}
// splitList turns a comma-separated setting into a trimmed
// slice, dropping empty entries.
func splitList(raw string) []string {
parts := strings.Split(raw, ",")
out := make([]string, 0, len(parts))
for _, p := range parts {
p = strings.TrimSpace(p)
if p != "" {
out = append(out, p)
}
}
return out
}
+121
View File
@@ -0,0 +1,121 @@
package config_test
import (
"strings"
"testing"
"sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals"
"sneak.berlin/go/netwatch/internal/logger"
"go.uber.org/fx"
)
// requireConfigError builds the config as main does and fails the
// test unless that fails with an error naming setting. It uses
// fx.New, because fxtest.New fails the test itself on an error.
func requireConfigError(t *testing.T, setting string) {
t.Helper()
app := fx.New(
fx.NopLogger,
fx.Provide(globals.New, logger.New, config.New),
fx.Invoke(func(*config.Config) {}),
)
err := app.Err()
if err == nil || !strings.Contains(err.Error(), setting) {
t.Fatalf("config error = %v, want one naming %s", err, setting)
}
}
// TestSettingsLoadAsGiven: valid values pass the checks and are used
// as given. bin/entrypoint.sh starts the server with these
// BIND_ADDRESS and PORT values.
func TestSettingsLoadAsGiven(t *testing.T) {
t.Setenv("BIND_ADDRESS", "127.0.0.1")
t.Setenv("PORT", "8081")
t.Setenv("DEBUG", "true")
var cfg *config.Config
app := fx.New(
fx.NopLogger,
fx.Provide(globals.New, logger.New, config.New),
fx.Populate(&cfg),
)
err := app.Err()
if err != nil {
t.Fatalf("config error = %v", err)
}
if cfg.BindAddress != "127.0.0.1" || cfg.Port != 8081 || !cfg.Debug {
t.Fatalf("BindAddress, Port, Debug = %q, %d, %t; "+
"want \"127.0.0.1\", 8081, true",
cfg.BindAddress, cfg.Port, cfg.Debug)
}
}
// TestPortMustBeAPortNumber: viper reads a value that is not a number
// as 0, on which the server would listen on a random port.
func TestPortMustBeAPortNumber(t *testing.T) {
for _, value := range []string{"abc", "0", "65536", "8080.5"} {
t.Run(value, func(t *testing.T) {
t.Setenv("PORT", value)
requireConfigError(t, "PORT")
})
}
}
// TestDebugMustBeTrueOrFalse: viper reads any other value, such as
// "yes", as false.
func TestDebugMustBeTrueOrFalse(t *testing.T) {
t.Setenv("DEBUG", "yes")
requireConfigError(t, "DEBUG")
}
// TestBindAddressMustBeAnIPAddress: a host name would be looked up
// only once the server starts listening, and a mistyped one would stop
// it then with an error that does not name the setting.
func TestBindAddressMustBeAnIPAddress(t *testing.T) {
t.Setenv("BIND_ADDRESS", "localhost")
requireConfigError(t, "BIND_ADDRESS")
}
// TestReportsPerMinuteMustBePositive: unchecked, zero would panic
// when the routes are built, and a negative rate would lift the
// limit.
func TestReportsPerMinuteMustBePositive(t *testing.T) {
t.Setenv("REPORTS_PER_MINUTE", "0")
requireConfigError(t, "REPORTS_PER_MINUTE")
}
// TestDataDirMaxBytesMustBeANumber: viper reads a value that is not
// a number, such as "1GB", as 0, which would refuse every report.
func TestDataDirMaxBytesMustBeANumber(t *testing.T) {
t.Setenv("DATA_DIR_MAX_BYTES", "1GB")
requireConfigError(t, "DATA_DIR_MAX_BYTES")
}
// TestCORSAllowedOriginsMustBeOrigins: "*" would let every origin in,
// and an entry that is not a plain origin would match no page.
func TestCORSAllowedOriginsMustBeOrigins(t *testing.T) {
for _, entry := range []string{
"*",
"https://*.netwatch.example",
"netwatch.example",
"https://netwatch.example/",
} {
t.Run(entry, func(t *testing.T) {
t.Setenv("CORS_ALLOWED_ORIGINS", entry)
requireConfigError(t, "CORS_ALLOWED_ORIGINS")
})
}
}
+10
View File
@@ -0,0 +1,10 @@
package handlers
import "log/slog"
// NewForTest builds a Handlers around a report sink and logger,
// bypassing the fx graph so handler behaviour (including the
// storage failure path) is exercisable in unit tests.
func NewForTest(buf reportAppender, log *slog.Logger) *Handlers {
return &Handlers{buf: buf, log: log}
}
+20 -1
View File
@@ -18,6 +18,13 @@ import (
const jsonContentType = "application/json; charset=utf-8" const jsonContentType = "application/json; charset=utf-8"
// reportAppender is the subset of the report buffer the handlers
// depend on. Defining it here keeps the storage failure path
// exercisable with a stub in tests.
type reportAppender interface {
Append(v any) error
}
// Params defines the dependencies for Handlers. // Params defines the dependencies for Handlers.
type Params struct { type Params struct {
fx.In fx.In
@@ -30,7 +37,7 @@ type Params struct {
// Handlers provides HTTP handler factories for all endpoints. // Handlers provides HTTP handler factories for all endpoints.
type Handlers struct { type Handlers struct {
buf *reportbuf.Buffer buf reportAppender
hc *healthcheck.Healthcheck hc *healthcheck.Healthcheck
log *slog.Logger log *slog.Logger
params *Params params *Params
@@ -72,3 +79,15 @@ func (s *Handlers) respondJSON(
} }
} }
} }
// decodeJSON decodes the request body into v. The body is
// expected to already be bounded by the body-size middleware, so
// a caller can distinguish an over-limit body from malformed
// JSON by testing the returned error for *http.MaxBytesError.
func (s *Handlers) decodeJSON(
_ http.ResponseWriter,
r *http.Request,
v any,
) error {
return json.NewDecoder(r.Body).Decode(v)
}
+65 -27
View File
@@ -2,10 +2,12 @@ package handlers
import ( import (
"encoding/json" "encoding/json"
"errors"
"net/http" "net/http"
)
const maxReportBodyBytes = 1 << 20 // 1 MiB "sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/reportbuf"
)
type reportSample struct { type reportSample struct {
T int64 `json:"t"` T int64 `json:"t"`
@@ -35,48 +37,84 @@ func (s *Handlers) HandleReport() http.HandlerFunc {
} }
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(
w, r.Body, maxReportBodyBytes,
)
var rpt report var rpt report
err := json.NewDecoder(r.Body).Decode(&rpt) err := s.decodeJSON(w, r, &rpt)
if err != nil { if err != nil {
s.log.Error("failed to decode report",
"error", err,
)
s.respondJSON(w, r, s.respondJSON(w, r,
&response{Status: "error"}, &response{Status: "error"},
http.StatusBadRequest, s.decodeErrorStatus(err),
) )
return return
} }
s.logReportReceived(rpt)
err = s.buf.Append(rpt)
if err != nil {
s.respondJSON(w, r,
&response{Status: "error"},
s.appendErrorStatus(err),
)
return
}
s.respondJSON(w, r, &response{Status: "ok"}, http.StatusOK)
}
}
// decodeErrorStatus logs a report decode failure and returns the
// status to send: 413 when the body exceeded the size limit,
// otherwise 400 for malformed JSON.
func (s *Handlers) decodeErrorStatus(err error) int {
var tooLarge *http.MaxBytesError
if errors.As(err, &tooLarge) {
s.log.Warn("report body too large", "limit_bytes", tooLarge.Limit)
return http.StatusRequestEntityTooLarge
}
// The decoder's error text can quote request bytes (a whole
// oversized number, for example), so it is bounded too.
s.log.Error("failed to decode report",
"error", logger.BoundedForLog(err.Error()),
)
return http.StatusBadRequest
}
// appendErrorStatus logs a failure to store a report and returns
// the status to send: 507 when the report files are at their size
// cap, otherwise 500.
func (s *Handlers) appendErrorStatus(err error) int {
if errors.Is(err, reportbuf.ErrFull) {
s.log.Warn("report refused: report files at their size cap")
return http.StatusInsufficientStorage
}
s.log.Error("failed to buffer report", "error", err)
return http.StatusInternalServerError
}
// logReportReceived logs an accepted report. Untrusted fields are
// bounded (client_id, timestamp) or reduced to a length
// (geo_bytes) so the raw attacker-controlled body never reaches
// the log.
func (s *Handlers) logReportReceived(rpt report) {
totalSamples := 0 totalSamples := 0
for _, h := range rpt.Hosts { for _, h := range rpt.Hosts {
totalSamples += len(h.History) totalSamples += len(h.History)
} }
s.log.Info("report received", s.log.Info("report received",
"client_id", rpt.ClientID, "client_id", logger.BoundedForLog(rpt.ClientID),
"timestamp", rpt.Timestamp, "timestamp", logger.BoundedForLog(rpt.Timestamp),
"host_count", len(rpt.Hosts), "host_count", len(rpt.Hosts),
"total_samples", totalSamples, "total_samples", totalSamples,
"geo", string(rpt.Geo), "geo_bytes", len(rpt.Geo),
) )
bufErr := s.buf.Append(rpt)
if bufErr != nil {
s.log.Error("failed to buffer report",
"error", bufErr,
)
}
s.respondJSON(w, r,
&response{Status: "ok"},
http.StatusOK,
)
}
} }
+240
View File
@@ -0,0 +1,240 @@
package handlers_test
import (
"bytes"
"encoding/json"
"errors"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"strings"
"testing"
"sneak.berlin/go/netwatch/internal/handlers"
"sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/middleware"
"sneak.berlin/go/netwatch/internal/reportbuf"
)
var errStorageFailed = errors.New("storage failed")
// stubAppender drives the storage success/failure path without a
// real buffer or disk.
type stubAppender struct {
err error
}
func (s stubAppender) Append(any) error { return s.err }
func newTestHandlers(buf stubAppender, out io.Writer) *handlers.Handlers {
return handlers.NewForTest(buf, slog.New(slog.NewJSONHandler(out, nil)))
}
func decodeStatus(t *testing.T, body []byte) string {
t.Helper()
var resp struct {
Status string `json:"status"`
}
err := json.Unmarshal(body, &resp)
if err != nil {
t.Fatalf("response body not JSON: %v (%q)", err, body)
}
return resp.Status
}
func TestHandleReportStorageFailureIsNon2xx(t *testing.T) {
t.Parallel()
h := newTestHandlers(stubAppender{err: errStorageFailed}, io.Discard)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
)
h.HandleReport().ServeHTTP(rec, req)
if rec.Code < 500 {
t.Fatalf("storage failure status = %d, want a 5xx", rec.Code)
}
if got := decodeStatus(t, rec.Body.Bytes()); got != "error" {
t.Fatalf("status field = %q, want %q", got, "error")
}
}
// TestHandleReportFullIs507 checks the answer when the report files
// are at their size cap: 507 and the usual error body, which tells
// the client nothing more.
func TestHandleReportFullIs507(t *testing.T) {
t.Parallel()
h := newTestHandlers(stubAppender{err: reportbuf.ErrFull}, io.Discard)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
)
h.HandleReport().ServeHTTP(rec, req)
if rec.Code != http.StatusInsufficientStorage {
t.Fatalf("status = %d, want %d",
rec.Code, http.StatusInsufficientStorage)
}
if got := rec.Body.String(); got != "{\"status\":\"error\"}\n" {
t.Errorf("body = %q, want %q", got, "{\"status\":\"error\"}\n")
}
}
func TestHandleReportMalformedJSONIs400(t *testing.T) {
t.Parallel()
h := newTestHandlers(stubAppender{}, io.Discard)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(`{not json`),
)
h.HandleReport().ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("malformed status = %d, want %d",
rec.Code, http.StatusBadRequest)
}
}
func TestHandleReportOversizeIs413(t *testing.T) {
t.Parallel()
const limit = 32
h := newTestHandlers(stubAppender{}, io.Discard)
handler := (&middleware.Middleware{}).MaxBodyBytes(limit)(
h.HandleReport(),
)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(`{"clientId":"`+strings.Repeat("x", 200)+`"}`),
)
// No declared length, so only the middleware's read cap can
// stop this body.
req.ContentLength = -1
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusRequestEntityTooLarge {
t.Fatalf("oversize status = %d, want %d",
rec.Code, http.StatusRequestEntityTooLarge)
}
}
func TestHandleReportDoesNotLogRawGeo(t *testing.T) {
t.Parallel()
const sentinel = "SENSITIVE-GEO-BLOB"
var logbuf bytes.Buffer
h := newTestHandlers(stubAppender{}, &logbuf)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(
`{"clientId":"c1","geo":{"raw":"`+sentinel+`"},"hosts":[]}`,
),
)
h.HandleReport().ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
}
if strings.Contains(logbuf.String(), sentinel) {
t.Fatal("raw geo bytes were written to the log")
}
if !strings.Contains(logbuf.String(), "geo_bytes") {
t.Fatal("expected a bounded geo_bytes field in the log")
}
}
func TestHandleReportLogsClientIDCutToBound(t *testing.T) {
t.Parallel()
long := strings.Repeat("c", 2*logger.MaxLoggedFieldBytes)
var logbuf bytes.Buffer
h := newTestHandlers(stubAppender{}, &logbuf)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(
`{"clientId":"`+long+`","timestamp":"`+long+`","hosts":[]}`,
),
)
h.HandleReport().ServeHTTP(rec, req)
var logged map[string]any
err := json.Unmarshal(logbuf.Bytes(), &logged)
if err != nil {
t.Fatalf("log line not JSON: %v (%q)", err, logbuf.String())
}
want := long[:logger.MaxLoggedFieldBytes]
if logged["client_id"] != want {
t.Fatalf("logged client_id not cut to %d bytes: %q",
logger.MaxLoggedFieldBytes, logged["client_id"])
}
if logged["timestamp"] != want {
t.Fatalf("logged timestamp not cut to %d bytes: %q",
logger.MaxLoggedFieldBytes, logged["timestamp"])
}
}
func TestHandleReportDecodeErrorLogIsBounded(t *testing.T) {
t.Parallel()
// A number too large for its int64 field makes the decoder's
// error text quote the whole number.
huge := strings.Repeat("9", 2*logger.MaxLoggedFieldBytes)
var logbuf bytes.Buffer
h := newTestHandlers(stubAppender{}, &logbuf)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(`{"hosts":[{"history":[{"t":`+huge+`}]}]}`),
)
h.HandleReport().ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
}
if strings.Contains(logbuf.String(), huge) {
t.Fatal("the whole oversized number was written to the log")
}
}
+15
View File
@@ -11,6 +11,21 @@ import (
"go.uber.org/fx" "go.uber.org/fx"
) )
// MaxLoggedFieldBytes bounds untrusted text (request fields,
// header values, decode error text) before it is logged, so a
// caller cannot inflate log volume with an oversized value.
const MaxLoggedFieldBytes = 128
// BoundedForLog truncates an untrusted string to a fixed byte
// bound so an attacker-controlled field cannot dominate the log.
func BoundedForLog(s string) string {
if len(s) > MaxLoggedFieldBytes {
return s[:MaxLoggedFieldBytes]
}
return s
}
// Params defines the dependencies for Logger. // Params defines the dependencies for Logger.
type Params struct { type Params struct {
fx.In fx.In
@@ -0,0 +1,31 @@
package middleware
import (
"log/slog"
"net/http"
"net/netip"
)
// Test-only wrappers exposing unexported helpers to the
// external middleware_test package.
// NewWithLogger builds a Middleware around a logger for tests
// that exercise the logging paths without the fx graph.
func NewWithLogger(log *slog.Logger) *Middleware {
return &Middleware{log: log}
}
// NewWithTrustedProxies builds a Middleware that honours forwarded
// headers from the given networks, for tests of the client address
// paths without the fx graph.
func NewWithTrustedProxies(trusted []netip.Prefix) *Middleware {
return &Middleware{trustedProxies: trusted}
}
func ClientIP(
remoteAddr string,
header http.Header,
trusted []netip.Prefix,
) string {
return clientIP(remoteAddr, header, trusted)
}
+260 -23
View File
@@ -3,9 +3,15 @@
package middleware package middleware
import ( import (
"errors"
"fmt"
"io"
"log/slog" "log/slog"
"net" "net"
"net/http" "net/http"
"net/netip"
"runtime/debug"
"strings"
"time" "time"
"sneak.berlin/go/netwatch/internal/config" "sneak.berlin/go/netwatch/internal/config"
@@ -14,11 +20,29 @@ import (
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
"github.com/go-chi/cors" "github.com/go-chi/cors"
"github.com/go-chi/httprate"
"go.uber.org/fx" "go.uber.org/fx"
) )
const corsMaxAgeSec = 300 const corsMaxAgeSec = 300
// jsonErrorBody is the body written for errors raised inside
// middleware, matching the {"status":"error"} shape the handlers
// return so clients see one error contract across the API.
const (
jsonContentType = "application/json; charset=utf-8"
jsonErrorBody = "{\"status\":\"error\"}\n"
)
// Security header values. The backend is a JSON API with no
// HTML surface, so the CSP forbids every resource type and
// framing outright.
const (
hstsValue = "max-age=31536000; includeSubDomains"
cspValue = "default-src 'none'; frame-ancestors 'none'"
permissionsPolicyValue = "camera=(), microphone=(), geolocation=()"
)
// Params defines the dependencies for Middleware. // Params defines the dependencies for Middleware.
type Params struct { type Params struct {
fx.In fx.In
@@ -32,6 +56,7 @@ type Params struct {
type Middleware struct { type Middleware struct {
log *slog.Logger log *slog.Logger
params *Params params *Params
trustedProxies []netip.Prefix
} }
// New creates a Middleware instance. // New creates a Middleware instance.
@@ -39,13 +64,40 @@ func New(
_ fx.Lifecycle, _ fx.Lifecycle,
params Params, params Params,
) (*Middleware, error) { ) (*Middleware, error) {
trusted, err := ParseTrustedProxies(params.Config.TrustedProxies)
if err != nil {
return nil, err
}
s := new(Middleware) s := new(Middleware)
s.params = &params s.params = &params
s.log = params.Logger.Get() s.log = params.Logger.Get()
s.trustedProxies = trusted
return s, nil return s, nil
} }
// ParseTrustedProxies converts the TRUSTED_PROXIES entries into
// prefixes, failing fast on any malformed entry. Each entry must be
// a CIDR; a lone address is refused. "netwatch-server check-cidr"
// runs it too.
func ParseTrustedProxies(cidrs []string) ([]netip.Prefix, error) {
prefixes := make([]netip.Prefix, 0, len(cidrs))
for _, cidr := range cidrs {
prefix, err := netip.ParsePrefix(cidr)
if err != nil {
return nil, fmt.Errorf(
"TRUSTED_PROXIES %q: %w", cidr, err,
)
}
prefixes = append(prefixes, prefix.Masked())
}
return prefixes, nil
}
type loggingResponseWriter struct { type loggingResponseWriter struct {
http.ResponseWriter http.ResponseWriter
@@ -72,8 +124,75 @@ func ipFromHostPort(hostPort string) string {
return host return host
} }
// clientIP resolves the caller's address. X-Forwarded-For and
// X-Real-IP are honoured only when the direct peer is a
// trusted proxy; otherwise the direct peer is returned so a
// spoofed header cannot forge the logged address.
func clientIP(
remoteAddr string,
header http.Header,
trusted []netip.Prefix,
) string {
peer := ipFromHostPort(remoteAddr)
if !addrInAny(peer, trusted) {
return peer
}
if xff := firstForwardedFor(header.Get("X-Forwarded-For")); xff != "" {
return xff
}
if xr := strings.TrimSpace(header.Get("X-Real-IP")); validIP(xr) {
return xr
}
return peer
}
// firstForwardedFor returns the left-most valid address in an
// X-Forwarded-For list (the original client), or "" if none.
func firstForwardedFor(value string) string {
for part := range strings.SplitSeq(value, ",") {
candidate := strings.TrimSpace(part)
if validIP(candidate) {
return candidate
}
}
return ""
}
func validIP(s string) bool {
_, err := netip.ParseAddr(s)
return err == nil
}
// addrInAny reports whether s parses as an address contained
// in any of the trusted prefixes.
func addrInAny(s string, trusted []netip.Prefix) bool {
addr, err := netip.ParseAddr(s)
if err != nil {
return false
}
addr = addr.Unmap()
for _, prefix := range trusted {
if prefix.Contains(addr) {
return true
}
}
return false
}
// Logging returns middleware that logs each request with // Logging returns middleware that logs each request with
// timing, status code, and client information. // timing, status code, and client information. Every string
// taken from the request is cut to logger.MaxLoggedFieldBytes,
// including the request ID, which chi takes from the client's
// X-Request-Id header when one is sent.
func (s *Middleware) Logging() func(http.Handler) http.Handler { func (s *Middleware) Logging() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler { return func(next http.Handler) http.Handler {
return http.HandlerFunc( return http.HandlerFunc(
@@ -86,17 +205,19 @@ func (s *Middleware) Logging() func(http.Handler) http.Handler {
latency := time.Since(start) latency := time.Since(start)
s.log.InfoContext(ctx, "request", s.log.InfoContext(ctx, "request",
"request_start", start, "request_start", start,
"method", r.Method, "method", logger.BoundedForLog(r.Method),
"url", r.URL.String(), "url", logger.BoundedForLog(r.URL.String()),
"useragent", r.UserAgent(), "useragent", logger.BoundedForLog(r.UserAgent()),
"request_id", "request_id",
ctx.Value( logger.BoundedForLog(middleware.GetReqID(ctx)),
middleware.RequestIDKey, "referer", logger.BoundedForLog(r.Referer()),
), "proto", logger.BoundedForLog(r.Proto),
"referer", r.Referer(),
"proto", r.Proto,
"remote_ip", "remote_ip",
ipFromHostPort(r.RemoteAddr), logger.BoundedForLog(clientIP(
r.RemoteAddr,
r.Header,
s.trustedProxies,
)),
"status", lrw.statusCode, "status", lrw.statusCode,
"latency_ms", "latency_ms",
latency.Milliseconds(), latency.Milliseconds(),
@@ -109,21 +230,137 @@ func (s *Middleware) Logging() func(http.Handler) http.Handler {
} }
} }
// CORS returns middleware that adds permissive CORS headers. // SecurityHeaders returns middleware that sets response
func (s *Middleware) CORS() func(http.Handler) http.Handler { // security headers. It runs before CORS so the headers are
// present on preflight responses the CORS handler writes.
func (s *Middleware) SecurityHeaders() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
h := w.Header()
h.Set("Strict-Transport-Security", hstsValue)
h.Set("Content-Security-Policy", cspValue)
h.Set("X-Frame-Options", "DENY")
h.Set("X-Content-Type-Options", "nosniff")
h.Set("Referrer-Policy", "no-referrer")
h.Set("Permissions-Policy", permissionsPolicyValue)
next.ServeHTTP(w, r)
},
)
}
}
// writeJSONError writes the shared JSON error body with the
// given status. Used where middleware must reject a request
// before it reaches a handler.
func writeJSONError(w http.ResponseWriter, status int) {
w.Header().Set("Content-Type", jsonContentType)
w.WriteHeader(status)
_, _ = io.WriteString(w, jsonErrorBody)
}
// MaxBodyBytes returns middleware that caps the request body at
// limit bytes. A declared Content-Length over the limit is
// rejected immediately with 413. Bodies without a declared
// length (or that understate it) are capped as they are read, so
// a handler that reads the body sees a *http.MaxBytesError it can
// map to 413. Mounted again on a route group, it can only lower
// the limit: a cap applied earlier in the chain still holds.
func (s *Middleware) MaxBodyBytes(
limit int64,
) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if r.ContentLength > limit {
writeJSONError(
w,
http.StatusRequestEntityTooLarge,
)
return
}
r.Body = http.MaxBytesReader(w, r.Body, limit)
next.ServeHTTP(w, r)
},
)
}
}
// Recoverer returns middleware that recovers from a panic in a
// downstream handler, logs the panic and stack trace through
// slog, and responds 500 with no body. http.ErrAbortHandler is
// re-panicked so the server can abort the response as intended.
func (s *Middleware) Recoverer() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
defer func() {
rec := recover()
if rec == nil {
return
}
err, ok := rec.(error)
if ok && errors.Is(err, http.ErrAbortHandler) {
panic(rec)
}
s.log.ErrorContext(r.Context(),
"panic recovered",
"panic", fmt.Sprintf("%v", rec),
"stack", string(debug.Stack()),
)
w.WriteHeader(http.StatusInternalServerError)
}()
next.ServeHTTP(w, r)
},
)
}
}
// CORS returns middleware that lets pages served from the given
// origins call the API. With no origins it adds no CORS headers at
// all, so only same-origin pages can use the API. That case must not
// reach cors.Handler, which treats an empty origin list as "allow
// every origin".
func (s *Middleware) CORS(
origins []string,
) func(http.Handler) http.Handler {
if len(origins) == 0 {
return func(next http.Handler) http.Handler { return next }
}
return cors.Handler(cors.Options{ return cors.Handler(cors.Options{
AllowedOrigins: []string{"*"}, AllowedOrigins: origins,
AllowedMethods: []string{ AllowedMethods: []string{http.MethodGet, http.MethodPost},
"GET", "POST", "PUT", "DELETE", "OPTIONS", AllowedHeaders: []string{"Content-Type"},
},
AllowedHeaders: []string{
"Accept",
"Authorization",
"Content-Type",
"X-CSRF-Token",
},
ExposedHeaders: []string{"Link"},
AllowCredentials: false, AllowCredentials: false,
MaxAge: corsMaxAgeSec, MaxAge: corsMaxAgeSec,
}) })
} }
// RateLimit returns middleware that allows each client address
// perMinute requests a minute and answers the rest with 429, the
// Retry-After header httprate sets, and the usual error body. The
// address is the one clientIP resolves, so clients behind the reverse
// proxy are limited one by one, not together as the proxy.
func (s *Middleware) RateLimit(
perMinute int,
) func(http.Handler) http.Handler {
return httprate.LimitBy(perMinute, time.Minute,
func(r *http.Request) (string, error) {
return clientIP(r.RemoteAddr, r.Header, s.trustedProxies), nil
},
httprate.WithLimitHandler(
func(w http.ResponseWriter, _ *http.Request) {
writeJSONError(w, http.StatusTooManyRequests)
},
),
)
}
@@ -0,0 +1,571 @@
package middleware_test
import (
"bytes"
"encoding/json"
"errors"
"log/slog"
"net/http"
"net/http/httptest"
"net/netip"
"strings"
"testing"
"testing/synctest"
"time"
"sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/middleware"
chimiddleware "github.com/go-chi/chi/v5/middleware"
)
const (
// loopbackPeer is a remote address inside the trusted-proxy allowlist.
loopbackPeer = "127.0.0.1:5000"
// forwardedIP is the client address presented via X-Forwarded-For.
forwardedIP = "203.0.113.7"
// realIP is the client address presented via X-Real-IP.
realIP = "203.0.113.9"
)
func mustPrefixes(t *testing.T, cidrs ...string) []netip.Prefix {
t.Helper()
prefixes, err := middleware.ParseTrustedProxies(cidrs)
if err != nil {
t.Fatalf("ParseTrustedProxies(%v): %v", cidrs, err)
}
return prefixes
}
// TestParseTrustedProxiesRejectsMalformed includes entries nginx would
// read as another address or look up as a hostname, in the CIDR form
// bin/entrypoint.sh gives "netwatch-server check-cidr".
func TestParseTrustedProxiesRejectsMalformed(t *testing.T) {
t.Parallel()
for _, cidr := range []string{
"not-a-cidr", "10.0.0.1", "1.2.3/32", "172.30/32", "10/32",
"cafe/32", "999.1.1.1/32", "10.0.0.0/33", "::1/129",
"fe80::1%eth0/128",
} {
_, err := middleware.ParseTrustedProxies([]string{cidr})
if err == nil || !strings.Contains(err.Error(), "TRUSTED_PROXIES") {
t.Errorf("%q: error = %v, want one naming TRUSTED_PROXIES",
cidr, err)
}
}
}
func TestParseTrustedProxiesAcceptsCIDRs(t *testing.T) {
t.Parallel()
mustPrefixes(t, "172.17.0.1/32", "10.0.0.0/8", "2001:db8::1/128",
"2001:db8::/32", "::ffff:192.0.2.1/128")
}
type clientIPCase struct {
name string
remoteAddr string
xff string
xRealIP string
want string
}
func clientIPCases() []clientIPCase {
return []clientIPCase{
{
name: "trusted proxy uses forwarded-for",
remoteAddr: loopbackPeer,
xff: forwardedIP,
want: forwardedIP,
},
{
name: "trusted proxy uses left-most of chain",
remoteAddr: "10.1.2.3:5000",
xff: forwardedIP + ", 10.1.2.3",
want: forwardedIP,
},
{
name: "trusted proxy falls back to x-real-ip",
remoteAddr: loopbackPeer,
xRealIP: realIP,
want: realIP,
},
{
name: "untrusted peer ignores forwarded-for",
remoteAddr: "198.51.100.4:5000",
xff: forwardedIP,
want: "198.51.100.4",
},
{
name: "untrusted peer ignores x-real-ip",
remoteAddr: "198.51.100.4:5000",
xRealIP: realIP,
want: "198.51.100.4",
},
{
name: "trusted proxy with no headers uses peer",
remoteAddr: "10.1.2.3:5000",
want: "10.1.2.3",
},
{
name: "trusted proxy with garbage header uses peer",
remoteAddr: loopbackPeer,
xff: "not-an-ip",
want: "127.0.0.1",
},
}
}
func TestClientIP(t *testing.T) {
t.Parallel()
trusted := mustPrefixes(t, "127.0.0.1/32", "::1/128", "10.0.0.0/8")
for _, tc := range clientIPCases() {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
header := http.Header{}
if tc.xff != "" {
header.Set("X-Forwarded-For", tc.xff)
}
if tc.xRealIP != "" {
header.Set("X-Real-IP", tc.xRealIP)
}
got := middleware.ClientIP(tc.remoteAddr, header, trusted)
if got != tc.want {
t.Errorf("ClientIP() = %q, want %q", got, tc.want)
}
})
}
}
func TestSecurityHeaders(t *testing.T) {
t.Parallel()
handler := (&middleware.Middleware{}).SecurityHeaders()(
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}),
)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
handler.ServeHTTP(rec, req)
want := map[string]string{
"Strict-Transport-Security": "max-age=31536000; includeSubDomains",
"Content-Security-Policy": "default-src 'none'; frame-ancestors 'none'",
"X-Frame-Options": "DENY",
"X-Content-Type-Options": "nosniff",
"Referrer-Policy": "no-referrer",
"Permissions-Policy": "camera=(), microphone=(), geolocation=()",
}
for name, value := range want {
if got := rec.Header().Get(name); got != value {
t.Errorf("header %s = %q, want %q", name, got, value)
}
}
}
// TestMaxBodyBytesRejectsOversizeOnNonReadingRoute confirms the
// limit is enforced even for a handler that never reads the body
// (for example the health check), via the Content-Length check.
func TestMaxBodyBytesRejectsOversizeOnNonReadingRoute(t *testing.T) {
t.Parallel()
const limit = 16
called := false
handler := (&middleware.Middleware{}).MaxBodyBytes(limit)(
http.HandlerFunc(func(_ http.ResponseWriter, _ *http.Request) {
called = true
}),
)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/.well-known/healthcheck",
strings.NewReader(strings.Repeat("x", limit+1)),
)
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusRequestEntityTooLarge {
t.Fatalf("status = %d, want %d",
rec.Code, http.StatusRequestEntityTooLarge)
}
if called {
t.Fatal("handler ran despite oversize body")
}
if got := rec.Body.String(); got != "{\"status\":\"error\"}\n" {
t.Errorf("body = %q, want %q", got, "{\"status\":\"error\"}\n")
}
got := rec.Header().Get("Content-Type")
if got != "application/json; charset=utf-8" {
t.Errorf("Content-Type = %q, want a JSON content type", got)
}
}
func TestMaxBodyBytesAllowsWithinLimit(t *testing.T) {
t.Parallel()
const limit = 64
handler := (&middleware.Middleware{}).MaxBodyBytes(limit)(
http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}),
)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(`{"clientId":"c1"}`),
)
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
}
}
func TestRecovererReturns500AndLogsThroughSlog(t *testing.T) {
t.Parallel()
var logbuf bytes.Buffer
mw := middleware.NewWithLogger(
slog.New(slog.NewJSONHandler(&logbuf, nil)),
)
handler := mw.Recoverer()(
http.HandlerFunc(func(_ http.ResponseWriter, _ *http.Request) {
panic("boom")
}),
)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusInternalServerError {
t.Fatalf("status = %d, want %d",
rec.Code, http.StatusInternalServerError)
}
var record map[string]any
err := json.Unmarshal(logbuf.Bytes(), &record)
if err != nil {
t.Fatalf("panic log is not one JSON record: %v (%q)", err, logbuf.String())
}
if record["msg"] != "panic recovered" || record["level"] != "ERROR" {
t.Errorf("log record = %v, want msg %q at level ERROR",
record, "panic recovered")
}
if record["panic"] != "boom" {
t.Errorf("panic field = %v, want %q", record["panic"], "boom")
}
stack, _ := record["stack"].(string)
if !strings.HasPrefix(stack, "goroutine ") {
t.Errorf("stack field = %q, want a stack trace", stack)
}
}
// TestRecovererRepanicsOnAbortHandler checks that a handler aborting
// with http.ErrAbortHandler is not treated as a crash: Recoverer
// panics again so the server aborts the response, and logs nothing.
func TestRecovererRepanicsOnAbortHandler(t *testing.T) {
t.Parallel()
var logbuf bytes.Buffer
mw := middleware.NewWithLogger(
slog.New(slog.NewJSONHandler(&logbuf, nil)),
)
handler := mw.Recoverer()(
http.HandlerFunc(func(_ http.ResponseWriter, _ *http.Request) {
panic(http.ErrAbortHandler)
}),
)
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
var recovered any
func() {
defer func() { recovered = recover() }()
handler.ServeHTTP(rec, req)
}()
err, _ := recovered.(error)
if !errors.Is(err, http.ErrAbortHandler) {
t.Errorf("Recoverer panicked with %v, want http.ErrAbortHandler", recovered)
}
if logbuf.Len() != 0 {
t.Errorf("abort was logged: %q", logbuf.String())
}
}
// TestLoggingCutsRequestStringsToBound sends an over-long URL and
// over-long header values, and checks the request log writes each
// one cut to logger.MaxLoggedFieldBytes.
func TestLoggingCutsRequestStringsToBound(t *testing.T) {
t.Parallel()
long := strings.Repeat("a", 2*logger.MaxLoggedFieldBytes)
var logbuf bytes.Buffer
mw := middleware.NewWithLogger(
slog.New(slog.NewJSONHandler(&logbuf, nil)),
)
handler := chimiddleware.RequestID(mw.Logging()(okHandler()))
req := httptest.NewRequestWithContext(t.Context(),
http.MethodGet, "/"+long, http.NoBody)
req.Header.Set("User-Agent", long)
req.Header.Set("Referer", long)
req.Header.Set("X-Request-Id", long)
handler.ServeHTTP(httptest.NewRecorder(), req)
var logged map[string]any
err := json.Unmarshal(logbuf.Bytes(), &logged)
if err != nil {
t.Fatalf("log line not JSON: %v (%q)", err, logbuf.String())
}
want := map[string]string{
"url": ("/" + long)[:logger.MaxLoggedFieldBytes],
"useragent": long[:logger.MaxLoggedFieldBytes],
"referer": long[:logger.MaxLoggedFieldBytes],
"request_id": long[:logger.MaxLoggedFieldBytes],
}
for field, value := range want {
if logged[field] != value {
t.Errorf("logged %s = %q, want it cut to %d bytes",
field, logged[field], logger.MaxLoggedFieldBytes)
}
}
}
// okHandler stands in for the route a middleware guards.
func okHandler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
})
}
// TestRateLimitRefusesPastAllowanceThenResets checks one client
// address: it may use its whole allowance at once, the next request
// is refused with 429, and later it may send again.
func TestRateLimitRefusesPastAllowanceThenResets(t *testing.T) {
t.Parallel()
// synctest runs this on a fake clock: time.Sleep returns at once,
// with the clock moved on.
synctest.Test(t, func(t *testing.T) {
const perMinute = 2
handler := (&middleware.Middleware{}).RateLimit(perMinute)(okHandler())
post := func() *httptest.ResponseRecorder {
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports", http.NoBody)
handler.ServeHTTP(rec, req)
return rec
}
for i := range perMinute {
if code := post().Code; code != http.StatusOK {
t.Fatalf("request %d: status = %d, want %d",
i+1, code, http.StatusOK)
}
}
rec := post()
if rec.Code != http.StatusTooManyRequests {
t.Fatalf("request past the allowance: status = %d, want %d",
rec.Code, http.StatusTooManyRequests)
}
if got := rec.Body.String(); got != "{\"status\":\"error\"}\n" {
t.Errorf("body = %q, want %q", got, "{\"status\":\"error\"}\n")
}
if got := rec.Header().Get("Retry-After"); got != "60" {
t.Fatalf("Retry-After = %q, want %q", got, "60")
}
// httprate also counts the previous minute's requests, fading
// them out over the current one, so two minutes on the whole
// allowance is back.
time.Sleep(2 * time.Minute)
for i := range perMinute {
if code := post().Code; code != http.StatusOK {
t.Fatalf("two minutes later, request %d: status = %d, want %d",
i+1, code, http.StatusOK)
}
}
})
}
// postForwarded sends handler a report from peer that names client in
// X-Forwarded-For, and returns the status.
func postForwarded(
t *testing.T,
handler http.Handler,
peer, client string,
) int {
t.Helper()
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports", http.NoBody)
req.RemoteAddr = peer
req.Header.Set("X-Forwarded-For", client)
handler.ServeHTTP(rec, req)
return rec.Code
}
// TestRateLimitIsPerForwardedClient checks that clients behind a
// trusted proxy each get their own allowance: the limit is keyed on
// the client address clientIP resolves, not on the proxy's.
func TestRateLimitIsPerForwardedClient(t *testing.T) {
t.Parallel()
const otherClient = "203.0.113.8"
mw := middleware.NewWithTrustedProxies(mustPrefixes(t, "127.0.0.1/32"))
handler := mw.RateLimit(1)(okHandler())
code := postForwarded(t, handler, loopbackPeer, forwardedIP)
if code != http.StatusOK {
t.Fatalf("first request: status = %d, want %d", code, http.StatusOK)
}
code = postForwarded(t, handler, loopbackPeer, forwardedIP)
if code != http.StatusTooManyRequests {
t.Fatalf("same client again: status = %d, want %d",
code, http.StatusTooManyRequests)
}
code = postForwarded(t, handler, loopbackPeer, otherClient)
if code != http.StatusOK {
t.Fatalf("other client behind the same proxy: status = %d, want %d",
code, http.StatusOK)
}
}
// TestRateLimitIgnoresForwardedForFromUntrustedPeer checks that a
// peer that is not a trusted proxy cannot get a fresh allowance by
// naming a different client in X-Forwarded-For on each request.
func TestRateLimitIgnoresForwardedForFromUntrustedPeer(t *testing.T) {
t.Parallel()
const untrustedPeer = "198.51.100.4:5000"
mw := middleware.NewWithTrustedProxies(mustPrefixes(t, "127.0.0.1/32"))
handler := mw.RateLimit(1)(okHandler())
code := postForwarded(t, handler, untrustedPeer, "203.0.113.8")
if code != http.StatusOK {
t.Fatalf("first request: status = %d, want %d", code, http.StatusOK)
}
code = postForwarded(t, handler, untrustedPeer, "203.0.113.9")
if code != http.StatusTooManyRequests {
t.Fatalf("same peer naming another client: status = %d, want %d",
code, http.StatusTooManyRequests)
}
}
// preflight sends cors the preflight request a browser makes before
// it POSTs JSON from origin.
func preflight(
t *testing.T,
cors func(http.Handler) http.Handler,
origin string,
) *httptest.ResponseRecorder {
t.Helper()
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodOptions, "/api/v1/reports", http.NoBody)
req.Header.Set("Origin", origin)
req.Header.Set("Access-Control-Request-Method", http.MethodPost)
req.Header.Set("Access-Control-Request-Headers", "content-type")
cors(okHandler()).ServeHTTP(rec, req)
return rec
}
// TestCORSWithoutOriginsAddsNoHeaders checks the default: with no
// origins configured, no origin is given any CORS header.
func TestCORSWithoutOriginsAddsNoHeaders(t *testing.T) {
t.Parallel()
rec := preflight(t,
(&middleware.Middleware{}).CORS(nil), "https://elsewhere.example")
for name := range rec.Header() {
if strings.HasPrefix(name, "Access-Control-") {
t.Errorf("CORS header %s set with no origins configured", name)
}
}
}
func TestCORSAllowsOnlyListedOrigins(t *testing.T) {
t.Parallel()
const listed = "https://netwatch.example"
cors := (&middleware.Middleware{}).CORS([]string{listed})
cases := []struct {
name string
origin string
want string
}{
{name: "listed origin allowed", origin: listed, want: listed},
{name: "other origin refused", origin: "https://elsewhere.example"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
rec := preflight(t, cors, tc.origin)
got := rec.Header().Get("Access-Control-Allow-Origin")
if got != tc.want {
t.Errorf("Access-Control-Allow-Origin = %q, want %q",
got, tc.want)
}
})
}
}
+15
View File
@@ -0,0 +1,15 @@
package reportbuf
import "time"
// Flush writes the buffered reports to a file now, as the periodic
// flush does, so tests need not wait a minute for it.
func (b *Buffer) Flush() error {
return b.flushLocked()
}
// StopClock makes every report file the buffer writes from now on
// carry the timestamp at, as if all were written in one millisecond.
func (b *Buffer) StopClock(at time.Time) {
b.now = func() time.Time { return at }
}
+138 -28
View File
@@ -6,12 +6,15 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io/fs" "io/fs"
"log/slog" "log/slog"
"os" "os"
"path/filepath" "path/filepath"
"strings"
"sync" "sync"
"sync/atomic"
"time" "time"
"sneak.berlin/go/netwatch/internal/config" "sneak.berlin/go/netwatch/internal/config"
@@ -27,8 +30,17 @@ const (
defaultDataDir = "./data/reports" defaultDataDir = "./data/reports"
dirPerms fs.FileMode = 0o750 dirPerms fs.FileMode = 0o750
filePerms fs.FileMode = 0o640 filePerms fs.FileMode = 0o640
// Report files are named filePrefix + timestamp + "-" + number +
// fileSuffix; see writeFile.
filePrefix = "reports-"
fileSuffix = ".jsonl.zst"
) )
// ErrFull is returned by Append when storing the report would
// take the report files past the configured maximum size.
var ErrFull = errors.New("report files at their size cap")
// Params defines the dependencies for Buffer. // Params defines the dependencies for Buffer.
type Params struct { type Params struct {
fx.In fx.In
@@ -44,7 +56,19 @@ type Buffer struct {
dataDir string dataDir string
done chan struct{} done chan struct{}
log *slog.Logger log *slog.Logger
maxBytes int64
mu sync.Mutex mu sync.Mutex
// now is the clock report files are named by: time.Now, except
// in tests that need two flushes to share a timestamp.
now func() time.Time
// seq numbers the report files, so that two named in the same
// millisecond still get different names.
seq atomic.Uint64
stopOnce sync.Once
// usedBytes is what Append checks against maxBytes: the size
// of the report files in dataDir, plus the reports not yet
// written to one at their uncompressed size.
usedBytes int64
} }
// New creates a Buffer and registers lifecycle hooks to // New creates a Buffer and registers lifecycle hooks to
@@ -62,6 +86,8 @@ func New(
dataDir: dir, dataDir: dir,
done: make(chan struct{}), done: make(chan struct{}),
log: params.Logger.Get(), log: params.Logger.Get(),
maxBytes: params.Config.DataDirMaxBytes,
now: time.Now,
} }
lc.Append(fx.Hook{ lc.Append(fx.Hook{
@@ -71,15 +97,30 @@ func New(
return fmt.Errorf("create data dir: %w", err) return fmt.Errorf("create data dir: %w", err)
} }
// Report files left by earlier runs count too.
b.usedBytes, err = reportFilesSize(b.dataDir)
if err != nil {
return err
}
go b.flushLoop() go b.flushLoop()
return nil return nil
}, },
OnStop: func(_ context.Context) error { OnStop: func(_ context.Context) error {
close(b.done) // stopOnce makes OnStop idempotent: a second
b.flushLocked() // invocation must not close an already-closed channel
// (which would panic) or flush again.
var err error
return nil b.stopOnce.Do(func() {
close(b.done)
err = b.flushLocked()
})
// A failed final flush fails the stop, so the process
// exits non-zero.
return err
}, },
}) })
@@ -87,15 +128,27 @@ func New(
} }
// Append marshals v as a single JSON line and appends it to // Append marshals v as a single JSON line and appends it to
// the buffer. If the buffer reaches the size threshold, it is // the buffer. It stores nothing and returns ErrFull if the line
// drained and written to disk asynchronously. // would take usedBytes past maxBytes. If the buffer reaches the
// size threshold, it is drained and written to disk
// asynchronously.
func (b *Buffer) Append(v any) error { func (b *Buffer) Append(v any) error {
line, err := json.Marshal(v) line, err := json.Marshal(v)
if err != nil { if err != nil {
return fmt.Errorf("marshal report: %w", err) return fmt.Errorf("marshal report: %w", err)
} }
lineBytes := int64(len(line)) + 1 // with its newline
b.mu.Lock() b.mu.Lock()
if b.usedBytes+lineBytes > b.maxBytes {
b.mu.Unlock()
return ErrFull
}
b.usedBytes += lineBytes
b.buf.Write(line) b.buf.Write(line)
b.buf.WriteByte('\n') b.buf.WriteByte('\n')
@@ -103,7 +156,12 @@ func (b *Buffer) Append(v any) error {
data := b.drainBuf() data := b.drainBuf()
b.mu.Unlock() b.mu.Unlock()
go b.writeFile(data) go func() {
writeErr := b.writeFile(data)
if writeErr != nil {
b.log.Error("flush reports failed", "error", writeErr)
}
}()
return nil return nil
} }
@@ -122,7 +180,10 @@ func (b *Buffer) flushLoop() {
for { for {
select { select {
case <-ticker.C: case <-ticker.C:
b.flushLocked() err := b.flushLocked()
if err != nil {
b.log.Error("flush reports failed", "error", err)
}
case <-b.done: case <-b.done:
return return
} }
@@ -131,19 +192,19 @@ func (b *Buffer) flushLoop() {
// flushLocked acquires the lock, drains the buffer, and // flushLocked acquires the lock, drains the buffer, and
// writes the data to a compressed file. // writes the data to a compressed file.
func (b *Buffer) flushLocked() { func (b *Buffer) flushLocked() error {
b.mu.Lock() b.mu.Lock()
if b.buf.Len() == 0 { if b.buf.Len() == 0 {
b.mu.Unlock() b.mu.Unlock()
return return nil
} }
data := b.drainBuf() data := b.drainBuf()
b.mu.Unlock() b.mu.Unlock()
b.writeFile(data) return b.writeFile(data)
} }
// drainBuf copies the buffer contents and resets it. // drainBuf copies the buffer contents and resets it.
@@ -158,42 +219,91 @@ func (b *Buffer) drainBuf() []byte {
// writeFile creates a timestamped zstd-compressed JSONL file // writeFile creates a timestamped zstd-compressed JSONL file
// in the data directory. // in the data directory.
func (b *Buffer) writeFile(data []byte) { func (b *Buffer) writeFile(data []byte) error {
ts := time.Now().UTC().Format("2006-01-02T15-04-05.000Z") // The timestamp comes first, so the names sort by time; the number
name := fmt.Sprintf("reports-%s.jsonl.zst", ts) // after it tells apart files named in the same millisecond.
ts := b.now().UTC().Format("2006-01-02T15-04-05.000Z")
name := fmt.Sprintf("%s%s-%d%s", filePrefix, ts, b.seq.Add(1), fileSuffix)
path := filepath.Join(b.dataDir, name) path := filepath.Join(b.dataDir, name)
f, err := os.OpenFile( //nolint:gosec // path built from controlled dataDir + timestamp // path is built from the operator-supplied dataDir plus a
// generated timestamp and number, so it carries no external input.
f, err := os.OpenFile( //nolint:gosec // see comment above
path, path,
os.O_WRONLY|os.O_CREATE|os.O_EXCL, os.O_WRONLY|os.O_CREATE|os.O_EXCL,
filePerms, filePerms,
) )
if err != nil { if err != nil {
b.log.Error("create report file", "error", err) return fmt.Errorf("create report file: %w", err)
return
} }
// Closes the file on the early returns below. The success
// path closes it explicitly to check the error; closing it
// a second time here is harmless.
defer func() { _ = f.Close() }() defer func() { _ = f.Close() }()
enc, err := zstd.NewWriter(f) enc, err := zstd.NewWriter(f)
if err != nil { if err != nil {
b.log.Error("create zstd encoder", "error", err) return fmt.Errorf("create zstd encoder: %w", err)
return
} }
_, writeErr := enc.Write(data) _, err = enc.Write(data)
if writeErr != nil { if err != nil {
b.log.Error("write compressed data", "error", writeErr)
_ = enc.Close() _ = enc.Close()
return return fmt.Errorf("write compressed data: %w", err)
} }
closeErr := enc.Close() err = enc.Close()
if closeErr != nil { if err != nil {
b.log.Error("close zstd encoder", "error", closeErr) return fmt.Errorf("close zstd encoder: %w", err)
} }
info, err := f.Stat()
if err != nil {
return fmt.Errorf("stat report file: %w", err)
}
err = f.Close()
if err != nil {
return fmt.Errorf("close report file: %w", err)
}
// The reports counted at their uncompressed size while they
// waited; now they count as the file. After a failed write they
// stay counted as they were, which errs toward refusing reports
// early rather than letting the files pass the cap.
b.mu.Lock()
b.usedBytes += info.Size() - int64(len(data))
b.mu.Unlock()
return nil
}
// reportFilesSize returns the total size of the report files in
// dir.
func reportFilesSize(dir string) (int64, error) {
entries, err := os.ReadDir(dir)
if err != nil {
return 0, fmt.Errorf("read data dir: %w", err)
}
var total int64
for _, entry := range entries {
name := entry.Name()
if !strings.HasPrefix(name, filePrefix) ||
!strings.HasSuffix(name, fileSuffix) {
continue
}
info, err := entry.Info()
if err != nil {
return 0, fmt.Errorf("stat report file: %w", err)
}
total += info.Size()
}
return total, nil
} }
+433 -5
View File
@@ -1,13 +1,441 @@
package reportbuf_test package reportbuf_test
import ( import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing" "testing"
"time"
_ "sneak.berlin/go/netwatch/internal/reportbuf" "sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals"
"sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/reportbuf"
"github.com/klauspost/compress/zstd"
"go.uber.org/fx"
"go.uber.org/fx/fxtest"
) )
func TestImport(t *testing.T) { // TestFlushOnShutdown proves the flush-on-shutdown path: a
t.Parallel() // report appended after start but before the periodic flush
// Compilation check — verifies the package parses // window must reach disk when the fx lifecycle stops. This is
// and all imports resolve. // the exact case that silent data loss on restart used to
// destroy.
func TestFlushOnShutdown(t *testing.T) {
dir := t.TempDir()
t.Setenv("DATA_DIR", dir)
var buf *reportbuf.Buffer
app := fxtest.New(t,
fx.Provide(
globals.New,
logger.New,
config.New,
reportbuf.New,
),
fx.Populate(&buf),
)
app.RequireStart()
err := buf.Append(map[string]string{"probe": "shutdown"})
if err != nil {
t.Fatalf("append report: %v", err)
}
// RequireStop runs the reportbuf OnStop hook, which is the
// only code path that flushes buffered reports on shutdown.
app.RequireStop()
if !hasReportFile(t, dir) {
t.Fatal("no report file on disk after shutdown; " +
"the buffered report was lost")
}
}
// TestFailedFinalFlushFailsStop proves a final flush that cannot
// write its file makes the stop fail, which makes the process
// exit non-zero instead of dropping the buffered reports silently.
func TestFailedFinalFlushFailsStop(t *testing.T) {
dir := t.TempDir()
t.Setenv("DATA_DIR", dir)
var buf *reportbuf.Buffer
app := fxtest.New(t,
fx.Provide(
globals.New,
logger.New,
config.New,
reportbuf.New,
),
fx.Populate(&buf),
)
app.RequireStart()
err := buf.Append(map[string]string{"probe": "shutdown"})
if err != nil {
t.Fatalf("append report: %v", err)
}
// Removing the data directory leaves the final flush nowhere to
// write. A read-only directory would not do: tests run as root
// in the backend image, and root ignores the read-only bit.
err = os.RemoveAll(dir)
if err != nil {
t.Fatalf("remove data dir: %v", err)
}
err = app.Stop(t.Context())
if !errors.Is(err, fs.ErrNotExist) {
t.Fatalf("stop error = %v, want the final flush's error", err)
}
}
// startBuffer starts a Buffer through fx, as main does, with the
// DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set.
func startBuffer(t *testing.T) *reportbuf.Buffer {
t.Helper()
var buf *reportbuf.Buffer
app := fxtest.New(t,
fx.Provide(
globals.New,
logger.New,
config.New,
reportbuf.New,
),
fx.Populate(&buf),
)
app.RequireStart()
t.Cleanup(app.RequireStop)
return buf
}
// lineBytes is what one report takes in the buffer: its JSON and a
// newline.
func lineBytes(t *testing.T, report any) int {
t.Helper()
line, err := json.Marshal(report)
if err != nil {
t.Fatalf("marshal report: %v", err)
}
return len(line) + 1
}
func TestAppendPastCapIsRefused(t *testing.T) {
report := map[string]string{"id": "cap"}
t.Setenv("DATA_DIR", t.TempDir())
t.Setenv("DATA_DIR_MAX_BYTES", strconv.Itoa(lineBytes(t, report)))
buf := startBuffer(t)
err := buf.Append(report)
if err != nil {
t.Fatalf("report that fills the cap exactly: %v", err)
}
err = buf.Append(report)
if !errors.Is(err, reportbuf.ErrFull) {
t.Fatalf("report past the cap: error = %v, want ErrFull", err)
}
}
// TestCapCountsReportFilesAlreadyInDataDir starts on a data
// directory holding a report file from an earlier run, and a file
// that is not a report, which must not count.
func TestCapCountsReportFilesAlreadyInDataDir(t *testing.T) {
const earlierBytes = 100
report := map[string]string{"id": "cap"}
dir := t.TempDir()
writeBytes(t, filepath.Join(dir, "reports-2026-01-01T00-00-00.000Z.jsonl.zst"),
earlierBytes)
writeBytes(t, filepath.Join(dir, "notes.txt"), 10*earlierBytes)
t.Setenv("DATA_DIR", dir)
t.Setenv("DATA_DIR_MAX_BYTES",
strconv.Itoa(earlierBytes+lineBytes(t, report)))
buf := startBuffer(t)
err := buf.Append(report)
if err != nil {
t.Fatalf("report that fills the cap exactly: %v", err)
}
err = buf.Append(report)
if !errors.Is(err, reportbuf.ErrFull) {
t.Fatalf("report past the cap: error = %v, want ErrFull", err)
}
}
// TestWrittenReportsCountAtFileSize checks that once reports are
// written, they count as their compressed file, not their
// uncompressed size, which frees room under the cap.
func TestWrittenReportsCountAtFileSize(t *testing.T) {
// Repetitive, so its file is far smaller than its JSON.
report := map[string]string{"id": strings.Repeat("a", 1000)}
size := lineBytes(t, report)
t.Setenv("DATA_DIR", t.TempDir())
// Room for the report twice over only if the first one counts
// at its file's size by the time the second arrives.
t.Setenv("DATA_DIR_MAX_BYTES", strconv.Itoa(2*size-1))
buf := startBuffer(t)
err := buf.Append(report)
if err != nil {
t.Fatalf("first report: %v", err)
}
err = buf.Flush()
if err != nil {
t.Fatalf("flush: %v", err)
}
err = buf.Append(report)
if err != nil {
t.Fatalf("second report, after the first was written: %v", err)
}
}
// TestWrittenReportsKeepCounting writes one report file after another
// under a small cap: each report must be taken while the files on disk
// leave room for it, and refused once they do not.
func TestWrittenReportsKeepCounting(t *testing.T) {
const maxBytes = 200
report := map[string]string{"id": "written"}
size := int64(lineBytes(t, report))
dir := t.TempDir()
t.Setenv("DATA_DIR", dir)
t.Setenv("DATA_DIR_MAX_BYTES", strconv.Itoa(maxBytes))
buf := startBuffer(t)
// Every file takes at least a byte, so they fill the cap within
// maxBytes rounds.
for range maxBytes {
used := reportFilesBytes(t, dir)
err := buf.Append(report)
if used+size > maxBytes {
if !errors.Is(err, reportbuf.ErrFull) {
t.Fatalf("with %d bytes of report files: error = %v, "+
"want ErrFull", used, err)
}
return
}
if err != nil {
t.Fatalf("with %d bytes of report files: %v", used, err)
}
err = buf.Flush()
if err != nil {
t.Fatalf("flush: %v", err)
}
}
t.Fatal("the report files never filled the cap")
}
// TestConcurrentAppendsStopAtCap appends from many goroutines at once
// with room for exactly roomFor reports: exactly that many must be
// taken, which holds only if Append checks and counts each report
// under one lock.
func TestConcurrentAppendsStopAtCap(t *testing.T) {
const (
roomFor = 5
senders = 50
)
// Large, so each Append takes long enough for the senders to
// overlap while the cap is reached.
report := map[string]string{"id": strings.Repeat("a", 1_000_000)}
t.Setenv("DATA_DIR", t.TempDir())
t.Setenv("DATA_DIR_MAX_BYTES",
strconv.Itoa(roomFor*lineBytes(t, report)))
buf := startBuffer(t)
var (
taken atomic.Int64
wg sync.WaitGroup
)
start := make(chan struct{})
for range senders {
wg.Go(func() {
<-start
err := buf.Append(report)
if err == nil {
taken.Add(1)
} else if !errors.Is(err, reportbuf.ErrFull) {
t.Errorf("append: %v", err)
}
})
}
close(start)
wg.Wait()
if got := taken.Load(); got != roomFor {
t.Fatalf("%d reports taken, want %d", got, roomFor)
}
}
// TestTwoFlushesInOneMillisecond flushes twice within one millisecond,
// as a flush for size and the final flush at shutdown can: each flush
// must write a file of its own, and the files must hold every report.
func TestTwoFlushesInOneMillisecond(t *testing.T) {
const flushes = 2
dir := t.TempDir()
t.Setenv("DATA_DIR", dir)
buf := startBuffer(t)
buf.StopClock(time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC))
for id := 1; id <= flushes; id++ {
err := buf.Append(map[string]int{"id": id})
if err != nil {
t.Fatalf("append report %d: %v", id, err)
}
err = buf.Flush()
if err != nil {
t.Fatalf("flush %d: %v", id, err)
}
}
files := readReportFiles(t, dir)
if len(files) != flushes {
t.Fatalf("%d report files after %d flushes", len(files), flushes)
}
for id := 1; id <= flushes; id++ {
want := fmt.Sprintf(`{"id":%d}`+"\n", id)
if !slices.Contains(files, want) {
t.Fatalf("no report file holds report %d alone", id)
}
}
}
// reportFilesBytes returns the total size of the report files in dir.
func reportFilesBytes(t *testing.T, dir string) int64 {
t.Helper()
paths, err := filepath.Glob(filepath.Join(dir, "reports-*.jsonl.zst"))
if err != nil {
t.Fatalf("list report files: %v", err)
}
var total int64
for _, path := range paths {
info, statErr := os.Stat(path)
if statErr != nil {
t.Fatalf("stat %s: %v", path, statErr)
}
total += info.Size()
}
return total
}
// readReportFiles returns the decompressed contents of each report
// file in dir.
func readReportFiles(t *testing.T, dir string) []string {
t.Helper()
files := os.DirFS(dir)
names, err := fs.Glob(files, "reports-*.jsonl.zst")
if err != nil {
t.Fatalf("list report files: %v", err)
}
dec, err := zstd.NewReader(nil)
if err != nil {
t.Fatalf("create zstd decoder: %v", err)
}
defer dec.Close()
contents := make([]string, 0, len(names))
for _, name := range names {
compressed, readErr := fs.ReadFile(files, name)
if readErr != nil {
t.Fatalf("read %s: %v", name, readErr)
}
data, decErr := dec.DecodeAll(compressed, nil)
if decErr != nil {
t.Fatalf("decompress %s: %v", name, decErr)
}
contents = append(contents, string(data))
}
return contents
}
func writeBytes(t *testing.T, path string, n int) {
t.Helper()
err := os.WriteFile(path, make([]byte, n), 0o600)
if err != nil {
t.Fatalf("write %s: %v", path, err)
}
}
func hasReportFile(t *testing.T, dir string) bool {
t.Helper()
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("read data dir: %v", err)
}
for _, e := range entries {
if strings.HasSuffix(e.Name(), ".jsonl.zst") {
info, statErr := e.Info()
if statErr != nil {
t.Fatalf("stat %s: %v", e.Name(), statErr)
}
if info.Size() > 0 {
return true
}
}
}
return false
} }
+11
View File
@@ -0,0 +1,11 @@
package server
// MaxRequestBodyBytes exposes the router-wide body limit to the
// external tests.
const MaxRequestBodyBytes = maxRequestBodyBytes
// ListenAddr exposes the address the server listens on to the
// external tests.
func (s *Server) ListenAddr() string {
return s.newHTTPServer().Addr
}
+39 -12
View File
@@ -2,42 +2,69 @@ package server
import ( import (
"errors" "errors"
"fmt" "net"
"net/http" "net/http"
"strconv"
"time" "time"
"go.uber.org/fx"
) )
const ( const (
readTimeout = 10 * time.Second readTimeout = 10 * time.Second
writeTimeout = 10 * time.Second readHeaderTimeout = 5 * time.Second
idleTimeout = 60 * time.Second
maxHeaderBytes = 1 << 20 // 1 MiB maxHeaderBytes = 1 << 20 // 1 MiB
// requestTimeout (routes.go) is the single per-request
// processing budget, enforced by chi's middleware.Timeout.
// writeTimeout must exceed that budget so a handler can write
// its 503 when the chi timeout fires; if it were shorter the
// server would abort the write first and the chi budget would
// be unreachable dead configuration.
writeTimeout = requestTimeout + 5*time.Second
) )
func (s *Server) serveUntilShutdown() { // newHTTPServer constructs the http.Server. It performs no I/O
listenAddr := fmt.Sprintf(":%d", s.params.Config.Port) // and does not start listening.
func (s *Server) newHTTPServer() *http.Server {
listenAddr := net.JoinHostPort(
s.params.Config.BindAddress,
strconv.Itoa(s.params.Config.Port),
)
s.httpServer = &http.Server{ return &http.Server{
Addr: listenAddr, Addr: listenAddr,
Handler: s, Handler: s,
MaxHeaderBytes: maxHeaderBytes, MaxHeaderBytes: maxHeaderBytes,
ReadTimeout: readTimeout, ReadTimeout: readTimeout,
ReadHeaderTimeout: readHeaderTimeout,
WriteTimeout: writeTimeout, WriteTimeout: writeTimeout,
IdleTimeout: idleTimeout,
} }
}
s.SetupRoutes() // listenAndServe runs the listener until the server is shut
// down. A genuine listen failure (not the expected
// ErrServerClosed from a clean shutdown) requests process
// shutdown through fx with a non-zero exit code, so the failure
// is visible to any supervisor.
func (s *Server) listenAndServe() {
s.log.Info("http begin listen", s.log.Info("http begin listen",
"listenaddr", listenAddr, "listenaddr", s.httpServer.Addr,
"version", s.params.Globals.Version, "version", s.params.Globals.Version,
"buildarch", s.params.Globals.Buildarch, "buildarch", s.params.Globals.Buildarch,
) )
err := s.httpServer.ListenAndServe() err := s.httpServer.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) { if err == nil || errors.Is(err, http.ErrServerClosed) {
return
}
s.log.Error("listen error", "error", err) s.log.Error("listen error", "error", err)
if s.cancelFunc != nil { shutdownErr := s.shutdowner.Shutdown(fx.ExitCode(1))
s.cancelFunc() if shutdownErr != nil {
} s.log.Error("request shutdown failed", "error", shutdownErr)
} }
} }
+32
View File
@@ -0,0 +1,32 @@
package server_test
import "testing"
// TestListenAddress checks that the server listens on BIND_ADDRESS
// and PORT, and on port 8080 on every interface when neither is set.
// The container image sets both, to keep the backend on loopback
// behind nginx.
func TestListenAddress(t *testing.T) {
tests := []struct {
bindAddress string
port string
want string
}{
{bindAddress: "", port: "", want: ":8080"},
{bindAddress: "127.0.0.1", port: "8081", want: "127.0.0.1:8081"},
{bindAddress: "::1", port: "8081", want: "[::1]:8081"},
}
for _, tt := range tests {
t.Run(tt.want, func(t *testing.T) {
// t.Setenv rules out t.Parallel.
t.Setenv("BIND_ADDRESS", tt.bindAddress)
t.Setenv("PORT", tt.port)
got := newServer(t).ListenAddr()
if got != tt.want {
t.Errorf("listen address = %q, want %q", got, tt.want)
}
})
}
}
+14 -4
View File
@@ -7,17 +7,26 @@ import (
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
) )
const requestTimeout = 60 * time.Second const (
requestTimeout = 60 * time.Second
// maxRequestBodyBytes caps every request body. A route group
// can mount s.mw.MaxBodyBytes with a smaller value to lower
// its bound, but cannot raise it: this cap runs first.
maxRequestBodyBytes int64 = 1 << 20 // 1 MiB
)
// SetupRoutes configures the chi router with middleware and // SetupRoutes configures the chi router with middleware and
// all application routes. // all application routes.
func (s *Server) SetupRoutes() { func (s *Server) SetupRoutes() {
s.router = chi.NewRouter() s.router = chi.NewRouter()
s.router.Use(middleware.Recoverer) s.router.Use(s.mw.Recoverer())
s.router.Use(middleware.RequestID) s.router.Use(middleware.RequestID)
s.router.Use(s.mw.Logging()) s.router.Use(s.mw.Logging())
s.router.Use(s.mw.CORS()) s.router.Use(s.mw.SecurityHeaders())
s.router.Use(s.mw.CORS(s.params.Config.CORSAllowedOrigins))
s.router.Use(s.mw.MaxBodyBytes(maxRequestBodyBytes))
s.router.Use(middleware.Timeout(requestTimeout)) s.router.Use(middleware.Timeout(requestTimeout))
s.router.Get( s.router.Get(
@@ -26,6 +35,7 @@ func (s *Server) SetupRoutes() {
) )
s.router.Route("/api/v1", func(r chi.Router) { s.router.Route("/api/v1", func(r chi.Router) {
r.Post("/reports", s.h.HandleReport()) r.With(s.mw.RateLimit(s.params.Config.ReportsPerMinute)).
Post("/reports", s.h.HandleReport())
}) })
} }
+142
View File
@@ -0,0 +1,142 @@
package server_test
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals"
"sneak.berlin/go/netwatch/internal/handlers"
"sneak.berlin/go/netwatch/internal/healthcheck"
"sneak.berlin/go/netwatch/internal/logger"
"sneak.berlin/go/netwatch/internal/middleware"
"sneak.berlin/go/netwatch/internal/reportbuf"
"sneak.berlin/go/netwatch/internal/server"
"go.uber.org/fx"
"go.uber.org/fx/fxtest"
)
// newServer builds a Server from the same constructors as main,
// configured from the environment. It is never started, so nothing
// listens.
func newServer(t *testing.T) *server.Server {
t.Helper()
var srv *server.Server
app := fxtest.New(t,
fx.Provide(
config.New,
globals.New,
handlers.New,
healthcheck.New,
logger.New,
middleware.New,
reportbuf.New,
server.New,
),
fx.Populate(&srv),
)
err := app.Err()
if err != nil {
t.Fatalf("build server: %v", err)
}
return srv
}
// TestReportsAreRateLimited checks that POST /api/v1/reports is
// behind the per-address rate limit, set here to two a minute.
func TestReportsAreRateLimited(t *testing.T) {
t.Setenv("REPORTS_PER_MINUTE", "2")
srv := newServer(t)
srv.SetupRoutes()
post := func() int {
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodPost, "/api/v1/reports",
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
)
srv.ServeHTTP(rec, req)
return rec.Code
}
for i := range 2 {
if code := post(); code != http.StatusOK {
t.Fatalf("report %d: status = %d, want %d",
i+1, code, http.StatusOK)
}
}
if code := post(); code != http.StatusTooManyRequests {
t.Fatalf("third report in a minute: status = %d, want %d",
code, http.StatusTooManyRequests)
}
}
// TestCORSAllowedOriginsReachTheRouter checks that an origin listed in
// CORS_ALLOWED_ORIGINS is allowed by the router, not only when handed
// to the CORS middleware directly.
func TestCORSAllowedOriginsReachTheRouter(t *testing.T) {
const origin = "https://netwatch.example:8443"
t.Setenv("CORS_ALLOWED_ORIGINS", origin)
srv := newServer(t)
srv.SetupRoutes()
// The preflight a browser sends before it POSTs JSON from origin.
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodOptions, "/api/v1/reports", http.NoBody)
req.Header.Set("Origin", origin)
req.Header.Set("Access-Control-Request-Method", http.MethodPost)
req.Header.Set("Access-Control-Request-Headers", "content-type")
srv.ServeHTTP(rec, req)
got := rec.Header().Get("Access-Control-Allow-Origin")
if got != origin {
t.Fatalf("Access-Control-Allow-Origin = %q, want %q", got, origin)
}
}
// TestHealthCheckRejectsOversizeBody sends the health check, which
// never reads its body, a body one byte over the limit. Only the
// router-wide body limit can reject it.
func TestHealthCheckRejectsOversizeBody(t *testing.T) {
t.Parallel()
srv := newServer(t)
srv.SetupRoutes()
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(t.Context(),
http.MethodGet, "/.well-known/healthcheck",
strings.NewReader(
strings.Repeat("x", int(server.MaxRequestBodyBytes)+1),
),
)
srv.ServeHTTP(rec, req)
if rec.Code != http.StatusRequestEntityTooLarge {
t.Fatalf("status = %d, want %d",
rec.Code, http.StatusRequestEntityTooLarge)
}
if got := rec.Body.String(); got != "{\"status\":\"error\"}\n" {
t.Errorf("body = %q, want %q", got, "{\"status\":\"error\"}\n")
}
got := rec.Header().Get("Content-Type")
if got != "application/json; charset=utf-8" {
t.Errorf("Content-Type = %q, want a JSON content type", got)
}
}
+27 -71
View File
@@ -1,16 +1,14 @@
// Package server provides the HTTP server lifecycle, // Package server provides the HTTP server lifecycle,
// including startup, routing, signal handling, and graceful // including startup, routing, and graceful shutdown. The
// shutdown. // process lifetime is owned by fx: shutdown is requested
// through fx.Shutdowner so every component's OnStop hook runs
// in dependency order.
package server package server
import ( import (
"context" "context"
"log/slog" "log/slog"
"net/http" "net/http"
"os"
"os/signal"
"syscall"
"time"
"sneak.berlin/go/netwatch/internal/config" "sneak.berlin/go/netwatch/internal/config"
"sneak.berlin/go/netwatch/internal/globals" "sneak.berlin/go/netwatch/internal/globals"
@@ -31,19 +29,18 @@ type Params struct {
Handlers *handlers.Handlers Handlers *handlers.Handlers
Logger *logger.Logger Logger *logger.Logger
Middleware *middleware.Middleware Middleware *middleware.Middleware
Shutdowner fx.Shutdowner
} }
// Server is the top-level HTTP server orchestrator. // Server is the top-level HTTP server orchestrator.
type Server struct { type Server struct {
cancelFunc context.CancelFunc
exitCode int
h *handlers.Handlers h *handlers.Handlers
httpServer *http.Server httpServer *http.Server
log *slog.Logger log *slog.Logger
mw *middleware.Middleware mw *middleware.Middleware
params Params params Params
router *chi.Mux router *chi.Mux
startupTime time.Time shutdowner fx.Shutdowner
} }
// New creates a Server and registers lifecycle hooks for // New creates a Server and registers lifecycle hooks for
@@ -57,23 +54,25 @@ func New(
s.mw = params.Middleware s.mw = params.Middleware
s.h = params.Handlers s.h = params.Handlers
s.log = params.Logger.Get() s.log = params.Logger.Get()
s.shutdowner = params.Shutdowner
lc.Append(fx.Hook{ lc.Append(fx.Hook{
OnStart: func(_ context.Context) error { OnStart: func(_ context.Context) error {
s.startupTime = time.Now().UTC() // Build the router and http.Server synchronously
// here, before spawning the serving goroutine, so
// httpServer is fully constructed by the time OnStop
// (or an early signal) can read it. fx guarantees
// OnStart returns before OnStop runs, so no
// synchronization or nil check is needed at shutdown.
s.SetupRoutes()
s.httpServer = s.newHTTPServer()
go func() { //nolint:contextcheck // fx OnStart ctx is startup-only; run() creates its own go s.listenAndServe()
s.run()
}()
return nil return nil
}, },
OnStop: func(_ context.Context) error { OnStop: func(ctx context.Context) error {
if s.cancelFunc != nil { return s.shutdown(ctx)
s.cancelFunc()
}
return nil
}, },
}) })
@@ -88,60 +87,17 @@ func (s *Server) ServeHTTP(
s.router.ServeHTTP(w, r) s.router.ServeHTTP(w, r)
} }
func (s *Server) run() { // shutdown gracefully stops the HTTP server within the
exitCode := s.serve() // deadline of the context fx provides for OnStop.
os.Exit(exitCode) func (s *Server) shutdown(ctx context.Context) error {
} err := s.httpServer.Shutdown(ctx)
func (s *Server) serve() int {
var ctx context.Context //nolint:wsl // ctx must be declared before multi-assign
ctx, s.cancelFunc = context.WithCancel(
context.Background(),
)
go func() {
c := make(chan os.Signal, 1)
signal.Ignore(syscall.SIGPIPE)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
sig := <-c
s.log.Info("signal received", "signal", sig)
if s.cancelFunc != nil {
s.cancelFunc()
}
}()
go func() {
s.serveUntilShutdown()
}()
<-ctx.Done()
s.cleanShutdown()
return s.exitCode
}
const shutdownTimeout = 5 * time.Second
func (s *Server) cleanShutdown() {
s.exitCode = 0
ctxShutdown, shutdownCancel := context.WithTimeout(
context.Background(),
shutdownTimeout,
)
defer shutdownCancel()
err := s.httpServer.Shutdown(ctxShutdown)
if err != nil { if err != nil {
s.log.Error( s.log.Error("server clean shutdown failed", "error", err)
"server clean shutdown failed",
"error", err, return err
)
} }
s.log.Info("server stopped") s.log.Info("server stopped")
return nil
} }
+21
View File
@@ -0,0 +1,21 @@
#!/bin/sh
# script/build: compile the static netwatch-server binary into the
# backend project root, with its version and architecture stamped in.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
# VERSION comes from the environment (the root Dockerfile passes its
# ARG VERSION in). Unset or empty, it is git describe, or "dev" where
# there is no git or no repository history.
version="${VERSION:-$(git describe --always --dirty 2>/dev/null || echo dev)}"
CGO_ENABLED=0 go build -trimpath \
-ldflags "-s -w -X main.Version=$version -X main.Buildarch=$(uname -m)" \
-o netwatch-server ./cmd/netwatch-server/
}
main "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# script/clean: remove build artifacts.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
rm -f netwatch-server
}
main "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# script/fmt: format the Go sources (writes).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
go fmt ./...
}
main "$@"
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
# script/fmt-check: check Go formatting (read-only). Same scope as
# script/fmt, but fails instead of writing.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
unformatted="$(gofmt -l .)"
if [ -n "$unformatted" ]; then
echo "Files not formatted:" >&2
echo "$unformatted" >&2
exit 1
fi
}
main "$@"
+32
View File
@@ -0,0 +1,32 @@
#!/bin/sh
# script/lint: run golangci-lint over the backend. This runs inside the
# lint stage of the root Dockerfile, whose digest-pinned golangci-lint
# image provides the linter; nothing installs golangci-lint on the host.
# From a checkout, run `make lint` at the repo root, which builds that
# stage.
#
# .golangci.yml is standardized org-wide and must never be edited here
# (REPO_POLICIES.md). Its last silent drift replaced the v2 schema with
# v1 keys, which left every threshold in the file inert while the build
# stayed green. So the file is first checked against the canonical
# copy's sha256: a local comparison, no network, nothing unpinned.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
GOLANGCI_CONFIG_SHA256="a79b63a254602a5318db5d0e9a06bc71b84bf0c1d896305229d8bfed1d1b1776"
main() {
cd "$ROOT"
actual="$(sha256sum .golangci.yml | cut -d' ' -f1)"
if [ "$actual" != "$GOLANGCI_CONFIG_SHA256" ]; then
echo ".golangci.yml has drifted from the org standard." >&2
echo " expected $GOLANGCI_CONFIG_SHA256" >&2
echo " actual $actual" >&2
echo "Restore it verbatim from sneak/prompts; do not edit it." >&2
exit 1
fi
golangci-lint run ./...
}
main "$@"
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
# script/run: build and run netwatch-server locally.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
"$ROOT/script/build"
exec ./netwatch-server "$@"
}
main "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# script/test: run the backend test suite.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
timeout 30 go test ./...
}
main "$@"
+142
View File
@@ -0,0 +1,142 @@
#!/bin/sh
# The container's entrypoint: runs netwatch-server and nginx side by
# side. TERM or INT stops both, and the container exits 0 if both exit
# cleanly. If either exits on its own, the other is stopped too and the
# container exits non-zero, so the platform restarts it instead of
# leaving it half up.
#
# No set -e: kill and wait return non-zero here in normal operation.
set -u
# PORT is the public port nginx listens on, 8080 when unset or empty.
# nginx would take a value such as localhost or unix:/tmp/x.sock as an
# address and start anyway, and reports a bad port without naming
# PORT, so a value that is not a usable port stops the container here,
# before either process starts.
export PORT="${PORT:-8080}"
case "$PORT" in
*[!0-9]*)
echo "entrypoint: PORT must be a port number, not '$PORT'" >&2
exit 1
;;
esac
# The length is checked first because, for a number too big for it,
# the shell's test prints an error and is false, so the range checks
# alone would let it through.
if [ "${#PORT}" -gt 5 ] || [ "$PORT" -lt 1 ] || [ "$PORT" -gt 65535 ]; then
echo "entrypoint: PORT must be from 1 to 65535, not '$PORT'" >&2
exit 1
fi
if [ "$PORT" -eq 8081 ]; then
echo "entrypoint: PORT cannot be 8081, netwatch-server listens there" >&2
exit 1
fi
# TRUSTED_PROXIES names the reverse proxies in front of the container,
# as IP addresses or CIDRs separated by commas. nginx takes the client
# address from X-Forwarded-For only on a request from one of them, so
# unset or empty, it trusts no one. nginx.conf includes the file written
# here, one set_real_ip_from line per entry.
#
# nginx looks up an entry it cannot read as an address as a hostname,
# and trusts what it finds (1.2.3 is found as 1.2.0.3). So each entry
# is made a CIDR, a lone address getting /128 if it is IPv6 and /32 if
# not, and netwatch-server checks it with the parsing it gives its own
# TRUSTED_PROXIES. Its error, naming the CIDR, is dropped for the one
# below, naming the entry as written. set -f keeps a * in an entry from
# becoming a list of file names.
TRUSTED_PROXIES="${TRUSTED_PROXIES:-}"
set -f
for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
case "$proxy" in
*/*) cidr="$proxy" ;;
*:*) cidr="$proxy/128" ;;
*) cidr="$proxy/32" ;;
esac
if ! netwatch-server check-cidr "$cidr" 2> /dev/null; then
echo "entrypoint: TRUSTED_PROXIES must be IP addresses or CIDRs" \
"separated by commas; '$proxy' is neither" >&2
exit 1
fi
echo "set_real_ip_from $cidr;"
done > /etc/nginx/trusted-proxies.conf
# netwatch-server keeps its report files in DATA_DIR, on the /data
# volume, which may be a host directory owned by root or by another
# uid. Both are given to the netwatch user here, with the mode the
# server gives a directory it creates, so the host directory needs no
# preparing.
#
# chown and chmod, run as root, change whatever a symbolic link on the
# path points to, anywhere in the container, and the netwatch user can
# put one in /data. So the start stops unless readlink -f, which
# follows every link on a path, gives /data and DATA_DIR back as they
# are. It also writes a path in full, so a DATA_DIR with '.', '..' or
# an extra '/' in it is refused too.
export DATA_DIR="${DATA_DIR:-/data/reports}"
mkdir -p "$DATA_DIR" || exit 1
if [ "$(readlink -f /data)" != /data ] ||
[ "$(readlink -f "$DATA_DIR")" != "$DATA_DIR" ]; then
echo "entrypoint: DATA_DIR must be a full path with no '.', '..'," \
"extra '/' or symbolic link on it or on /data, not '$DATA_DIR'" >&2
exit 1
fi
chown -R netwatch:netwatch /data "$DATA_DIR" || exit 1
chmod 750 /data "$DATA_DIR" || exit 1
# A stop signal is only noted here; the loop below acts on it.
stop_requested=""
trap 'stop_requested=yes' TERM INT
# netwatch-server runs as the netwatch user and listens on loopback
# only, on a port other than the public one; nginx.conf proxies to this
# address. Its only client is nginx, so it takes the client address
# nginx passes on from 127.0.0.1 alone, whatever TRUSTED_PROXIES the
# container has. The netwatch user has no login shell, hence -s
# /bin/sh. busybox su replaces itself with the command instead of
# staying on as its parent, so $! is the server's own PID.
BIND_ADDRESS=127.0.0.1 PORT=8081 TRUSTED_PROXIES=127.0.0.1/32 \
su -s /bin/sh netwatch -c 'exec netwatch-server' &
backend=$!
# nginx starts through the nginx image's own entrypoint, which applies
# the image's start-up configuration and then replaces itself with
# nginx. Part of that start-up configuration renders nginx.conf into
# conf.d with nginx listening on PORT. NGINX_ENVSUBST_FILTER limits
# that rendering to PORT: a variable nginx itself uses, such as $uri,
# would otherwise be replaced by an environment variable of the same
# name.
NGINX_ENVSUBST_FILTER='^PORT$' \
/docker-entrypoint.sh nginx -g 'daemon off;' &
nginx=$!
running() {
kill -0 "$1" 2>/dev/null
}
# POSIX sh cannot wait for whichever of two children exits first, so
# look once a second. The shell collects a child that has exited while
# it runs sleep, and running() is false for that child from then on.
while [ -z "$stop_requested" ] && running "$backend" && running "$nginx"; do
sleep 1
done
# Stop both, then wait until neither is left.
kill -TERM "$backend" "$nginx" 2>/dev/null
while running "$backend" || running "$nginx"; do
sleep 1
done
wait "$backend"
backend_status=$?
wait "$nginx"
nginx_status=$?
echo "entrypoint: netwatch-server exited $backend_status," \
"nginx exited $nginx_status"
# Success is a requested stop that both processes exited cleanly from.
if [ -n "$stop_requested" ] && [ "$backend_status" -eq 0 ] &&
[ "$nginx_status" -eq 0 ]; then
exit 0
fi
exit 1
+48 -5
View File
@@ -1,14 +1,27 @@
# A template: the nginx image renders it into conf.d at container start,
# filling in PORT and nothing else. bin/entrypoint.sh sets PORT and that
# limit.
server { server {
listen 8080; listen ${PORT};
server_name _; server_name _;
# Keep the nginx version out of the Server header and error pages.
server_tokens off;
# The security headers, on every response. An add_header in a
# location drops every add_header from here, so a location with one
# of its own includes this file again.
include /etc/nginx/security-headers.conf;
root /usr/share/nginx/html; root /usr/share/nginx/html;
index index.html; index index.html;
# Trust RFC1918 reverse proxies for X-Forwarded-For # The client address comes from X-Forwarded-For only on a request
set_real_ip_from 10.0.0.0/8; # from the reverse proxies in TRUSTED_PROXIES: bin/entrypoint.sh
set_real_ip_from 172.16.0.0/12; # writes one set_real_ip_from line for each into this file, and
set_real_ip_from 192.168.0.0/16; # leaves it empty when TRUSTED_PROXIES is unset, so that by default
# the client address is the one each request comes from.
include /etc/nginx/trusted-proxies.conf;
real_ip_header X-Forwarded-For; real_ip_header X-Forwarded-For;
real_ip_recursive on; real_ip_recursive on;
@@ -24,5 +37,35 @@ server {
location /assets/ { location /assets/ {
expires 1y; expires 1y;
add_header Cache-Control "public, immutable"; add_header Cache-Control "public, immutable";
include /etc/nginx/security-headers.conf;
}
# netwatch-server, the Go backend, runs in the same container and
# listens on loopback only: bin/entrypoint.sh starts it on
# 127.0.0.1:8081. These headers go with every request passed to it.
# X-Forwarded-For carries only the client address, as resolved by
# the real IP settings above, and not the chain the request came
# with: the backend takes the first entry, which a client can write.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
# netwatch-server sets the same security headers on its own
# responses. Its copies are dropped so that each header goes out
# once, as security-headers.conf sets it.
proxy_hide_header Strict-Transport-Security;
proxy_hide_header Content-Security-Policy;
proxy_hide_header X-Frame-Options;
proxy_hide_header X-Content-Type-Options;
proxy_hide_header Referrer-Policy;
proxy_hide_header Permissions-Policy;
location /api/ {
proxy_pass http://127.0.0.1:8081;
}
location = /.well-known/healthcheck {
proxy_pass http://127.0.0.1:8081;
} }
} }
+1
View File
@@ -14,6 +14,7 @@
"autoprefixer": "^10.4.23", "autoprefixer": "^10.4.23",
"postcss": "^8.5.6", "postcss": "^8.5.6",
"prettier": "^3.8.1", "prettier": "^3.8.1",
"puppeteer-core": "25.5.0",
"tailwindcss": "^4.1.18", "tailwindcss": "^4.1.18",
"vite": "^7.3.1" "vite": "^7.3.1"
} }
+276
View File
@@ -0,0 +1,276 @@
#!/bin/sh
# script/bootstrap: install all dependencies needed to build and develop
# this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes nothing is present. Node is
# used directly if it is at least NODE_MIN_VERSION; otherwise it is
# installed at a pinned version via nvm (installing nvm itself first,
# from a hash-verified release archive, never curl | sh). Go, with its
# gofmt, is used directly if it is at least the version backend/go.mod
# asks for; otherwise the pinned Go release is installed from its
# hash-verified archive.
#
# What this script installs outside the system package manager lives
# under $HOME and is linked into ~/.local/bin, where make and the git
# hook find it once that directory is on PATH. Nothing in ~/.local/bin
# that this script did not create is ever replaced.
#
# golangci-lint is not installed: make lint runs it in Docker, which
# this script does not install either.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Pinned versions, 2026-07-07
NODE_VERSION="22.17.0"
# The oldest node the frontend's dependencies accept: the "engines"
# field of puppeteer-core 25.5.0, the most demanding of them, asks for
# 22.12.0 or newer, 2026-09-29. An older installed node is not used.
NODE_MIN_VERSION="22.12.0"
NVM_VERSION="0.40.3"
# sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0"
YARN_VERSION="1.22.22"
# The Go inside the golang:1.25-alpine image Dockerfile builds the
# backend with, 2026-08-09. The archive hashes are in ensure_go.
GO_VERSION="1.25.7"
BIN_DIR="$HOME/.local/bin"
TOOLCHAIN="$HOME/.local/share/$("$ROOT/script/projectname")/toolchain"
PKGMGR=""
SUDO=""
APT_UPDATED=""
detect_pkgmgr() {
[ -n "$PKGMGR" ] && return 0
if command -v nix-env >/dev/null 2>&1; then
PKGMGR="nix"
elif command -v apt-get >/dev/null 2>&1; then
PKGMGR="apt"
elif command -v brew >/dev/null 2>&1; then
PKGMGR="brew"
elif command -v apk >/dev/null 2>&1; then
PKGMGR="apk"
else
echo "bootstrap: no supported package manager (nix, apt, brew, apk)" >&2
exit 1
fi
if [ "$PKGMGR" = "apt" ]; then
export DEBIAN_FRONTEND=noninteractive
if [ "$(id -u)" != "0" ]; then
SUDO="sudo"
fi
fi
}
# pkg_install <nix-attr> <apt-pkg> <brew-formula> <apk-pkg>
pkg_install() {
detect_pkgmgr
case "$PKGMGR" in
nix) nix-env -iA "nixpkgs.$1" ;;
apt)
if [ -z "$APT_UPDATED" ]; then
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get update
APT_UPDATED=1
fi
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y "$2"
;;
brew) brew install "$3" ;;
apk) apk add --no-cache "$4" ;;
esac
}
missing() {
! command -v "$1" >/dev/null 2>&1
}
# verify_sha256 <file> <expected-hash>
verify_sha256() {
if command -v sha256sum >/dev/null 2>&1; then
actual="$(sha256sum "$1" | cut -d' ' -f1)"
else
actual="$(shasum -a 256 "$1" | cut -d' ' -f1)"
fi
if [ "$actual" != "$2" ]; then
echo "bootstrap: sha256 mismatch for $1" >&2
echo " expected: $2" >&2
echo " actual: $actual" >&2
exit 1
fi
}
# link_bin <target> <name>: make an installed tool reachable as
# $BIN_DIR/<name>. Only a symlink this script made, one pointing into
# $TOOLCHAIN or ~/.nvm, is ever replaced; if anything else is already
# there, bootstrap stops.
link_bin() {
link="$BIN_DIR/$2"
if [ -L "$link" ] || [ -e "$link" ]; then
case "$(readlink "$link" || true)" in
"$TOOLCHAIN"/* | "$HOME"/.nvm/*) ;;
*)
echo "bootstrap: $link was not created by this script;" >&2
echo " remove or rename it, then re-run bootstrap" >&2
exit 1
;;
esac
fi
mkdir -p "$BIN_DIR"
ln -sf "$1" "$link"
}
# nvm is a bash script; run a command in a bash with nvm loaded
nvm_sh() {
bash -c ". \"\$HOME/.nvm/nvm.sh\" && $*"
}
ensure_nvm() {
[ -s "$HOME/.nvm/nvm.sh" ] && return 0
# nvm prerequisites; nvm itself requires bash
if missing bash; then pkg_install bash bash bash bash; fi
if missing curl; then pkg_install curl curl curl curl; fi
if missing git; then pkg_install git git git git; fi
tmp="$(mktemp -d)"
curl -fsSL -o "$tmp/nvm.tar.gz" \
"https://github.com/nvm-sh/nvm/archive/refs/tags/v${NVM_VERSION}.tar.gz"
verify_sha256 "$tmp/nvm.tar.gz" "$NVM_SHA256"
mkdir -p "$HOME/.nvm"
tar -xzf "$tmp/nvm.tar.gz" -C "$HOME/.nvm" --strip-components=1
rm -rf "$tmp"
}
# node_ok: the node on PATH is at least NODE_MIN_VERSION. node itself
# compares the two: major, then minor, then patch.
node_ok() {
if missing node; then return 1; fi
node -e '
const have = process.versions.node.split(".").map(Number);
const want = process.argv[1].split(".").map(Number);
for (let i = 0; i < 3; i++) {
if (have[i] !== want[i]) process.exit(have[i] > want[i] ? 0 : 1);
}
' "$NODE_MIN_VERSION"
}
# ensure_node: unless node_ok, install NODE_VERSION and link its node.
ensure_node() {
if node_ok; then return 0; fi
ensure_nvm
nvm_sh "nvm install $NODE_VERSION"
link_bin "$HOME/.nvm/versions/node/v$NODE_VERSION/bin/node" node
}
# ensure_yarn: corepack writes its shims (pnpm and yarnpkg as well as
# yarn) into $TOOLCHAIN rather than next to itself, and the npm fallback
# installs there too; only yarn is linked.
ensure_yarn() {
if ! missing yarn; then return 0; fi
shims="$TOOLCHAIN/corepack-shims"
mkdir -p "$shims"
if ! missing corepack; then
corepack enable --install-directory "$shims"
corepack prepare "yarn@$YARN_VERSION" --activate
elif [ -s "$HOME/.nvm/nvm.sh" ]; then
nvm_sh "nvm use $NODE_VERSION >/dev/null && \
corepack enable --install-directory \"$shims\" && \
corepack prepare yarn@$YARN_VERSION --activate"
else
npm install -g --prefix "$TOOLCHAIN/npm-global" "yarn@$YARN_VERSION"
shims="$TOOLCHAIN/npm-global/bin"
fi
link_bin "$shims/yarn" yarn
}
# go_ok: the go on PATH has its gofmt beside it (a Go release ships the
# two together) and is at least the version backend/go.mod asks for.
# GOTOOLCHAIN=local makes an older go fail here instead of fetching a
# newer toolchain for itself.
go_ok() {
if missing go; then return 1; fi
[ -x "$(dirname "$(command -v go)")/gofmt" ] || return 1
(cd "$ROOT/backend" && GOTOOLCHAIN=local go list -m >/dev/null 2>&1)
}
# ensure_go: unless go_ok, install GO_VERSION and link its go and gofmt.
# They are linked on every run that needs them, so a deleted link is put
# back, and the archive is unpacked again if either binary is missing.
ensure_go() {
if go_ok; then return 0; fi
go_dir="$TOOLCHAIN/go-$GO_VERSION"
if [ ! -x "$go_dir/bin/go" ] || [ ! -x "$go_dir/bin/gofmt" ]; then
# sha256 of each archive, from https://go.dev/dl/?mode=json
case "$(uname -s)-$(uname -m)" in
Linux-x86_64)
plat="linux-amd64"
sha="12e6d6a191091ae27dc31f6efc630e3a3b8ba409baf3573d955b196fdf086005"
;;
Linux-aarch64)
plat="linux-arm64"
sha="ba611a53534135a81067240eff9508cd7e256c560edd5d8c2fef54f083c07129"
;;
Darwin-x86_64)
plat="darwin-amd64"
sha="bf5050a2152f4053837b886e8d9640c829dbacbc3370f913351eb0904cb706f5"
;;
Darwin-arm64)
plat="darwin-arm64"
sha="ff18369ffad05c57d5bed888b660b31385f3c913670a83ef557cdfd98ea9ae1b"
;;
*)
echo "bootstrap: no pinned Go release for this platform" >&2
exit 1
;;
esac
if missing curl; then pkg_install curl curl curl curl; fi
mkdir -p "$TOOLCHAIN"
curl -fsSL -o "$go_dir.tar.gz" \
"https://go.dev/dl/go$GO_VERSION.$plat.tar.gz"
verify_sha256 "$go_dir.tar.gz" "$sha"
# Unpacked beside its final place and then moved there, so an
# interrupted run never leaves a partial Go that looks complete.
rm -rf "$go_dir.partial"
mkdir "$go_dir.partial"
tar -xzf "$go_dir.tar.gz" -C "$go_dir.partial" --strip-components=1
rm -rf "$go_dir" "$go_dir.tar.gz"
mv "$go_dir.partial" "$go_dir"
fi
link_bin "$go_dir/bin/go" go
link_bin "$go_dir/bin/gofmt" gofmt
}
main() {
cd "$ROOT"
# Tools linked on an earlier run count as installed, and tools linked
# on this run are found by the steps after it.
path_hint=""
case ":$PATH:" in
*":$BIN_DIR:"*) ;;
*) path_hint=yes ;;
esac
PATH="$BIN_DIR:$PATH"
if missing make; then pkg_install gnumake make make make; fi
if missing git; then pkg_install git git git git; fi
ensure_node
ensure_yarn
yarn install --frozen-lockfile
ensure_go
(cd "$ROOT/backend" && go mod download)
if missing docker; then
echo "bootstrap: docker not found; make lint, and so make check" >&2
echo " and the pre-commit hook, need it to run the Go linter" >&2
fi
if [ -n "$path_hint" ] && [ -d "$BIN_DIR" ]; then
echo "bootstrap: add $BIN_DIR to the front of your PATH, e.g." >&2
echo " export PATH=\"\$HOME/.local/bin:\$PATH\"" >&2
fi
echo "bootstrap complete"
}
main "$@"
Executable
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# script/check: run all checks (test, lint, fmt-check). Our own
# extension to scripts-to-rule-them-all. Must not modify any files.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
main() {
"$SCRIPT_DIR/test"
"$SCRIPT_DIR/lint"
"$SCRIPT_DIR/fmt-check"
}
main "$@"
Executable
+29
View File
@@ -0,0 +1,29 @@
#!/bin/sh
# script/cibuild: run the CI build. It bootstraps first: a CI runner
# checks out and runs this and nothing else, and script/fmt-check runs
# the formatter on the host, which a pristine checkout cannot do.
# --no-cache for the same reason as script/docker: the gate phases the
# final stage depends on are RUN steps, and a cached one is a check that
# did not run.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
"$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/check"
# Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant. VERSION is computed here because .dockerignore
# excludes .git, so `git describe` in a build stage yields an empty
# version without failing.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
--build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" .
}
main "$@"
Executable
+25
View File
@@ -0,0 +1,25 @@
#!/bin/sh
# script/docker: build the Docker image tagged with the project name.
# Identical in all repos; the tag comes from script/projectname.
# --no-cache because the gate phases the final stage depends on are RUN
# steps, and a cached one is a check that did not run.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() {
cd "$ROOT"
# Own line: a failing command substitution inside an argument does
# not trip `set -e`, so the inline form degrades silently to an
# empty constant. VERSION is computed here because .dockerignore
# excludes .git, so `git describe` in a build stage yields an empty
# version without failing.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
docker build --no-cache \
--build-arg VERSION="$version" \
-t "$("$SCRIPT_DIR/projectname")" .
}
main "$@"
Executable
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# script/fmt: format the whole repo (writes): prettier over everything
# it understands, then gofmt over the Go backend.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
"$ROOT/script/frontend-fmt"
"$ROOT/backend/script/fmt"
}
main "$@"
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# script/fmt-check: check formatting across the whole repo (read-only).
# Same scope as script/fmt, but fails instead of writing.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
"$ROOT/script/frontend-fmt-check"
"$ROOT/backend/script/fmt-check"
}
main "$@"
+18
View File
@@ -0,0 +1,18 @@
#!/bin/sh
# script/frontend-check: run the frontend half of the checks only (test,
# lint, fmt-check). This exists for the frontend stage of Dockerfile, a
# node image with neither Go nor Docker; the Dockerfile's lint and
# backend build stages gate the backend half. Everywhere else, use
# script/check, which covers the whole repo. Must not modify any files.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
"$ROOT/script/frontend-test"
"$ROOT/script/frontend-lint"
"$ROOT/script/frontend-fmt-check"
}
main "$@"
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# script/frontend-fmt: format the frontend and every other file prettier
# understands, repo-wide (writes). backend/ is in .prettierignore; Go
# sources are formatted by backend/script/fmt.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
yarn prettier --write .
}
main "$@"
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
# script/frontend-fmt-check: check prettier formatting (read-only). Same
# scope as script/frontend-fmt, but fails instead of writing.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
yarn prettier --check .
}
main "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# script/frontend-lint: run the frontend linter (prettier in check mode).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
yarn prettier --check .
}
main "$@"
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# script/frontend-test: run the frontend test suite. The frontend has no
# unit tests; the production build serves as the test (fails on broken
# code).
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
timeout 30 yarn build
}
main "$@"
+110
View File
@@ -0,0 +1,110 @@
#!/bin/sh
# script/frontend-viewport-test: verify the responsive layout of the built
# frontend in a real browser engine.
#
# Builds dist/, serves it with the same nginx image and the same nginx.conf
# the shipping container uses, points a containerised headless Chrome at it
# over CDP, and asserts on computed layout at every viewport width derived
# from the app's own CSS. See test/viewport/README.md for what this covers
# and what it cannot.
#
# Deliberately not part of script/check: it needs Docker and takes far
# longer than the 20s budget make test has to stay inside.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# chromedp/headless-shell 151.0.7922.109, 2026-08-09
BROWSER_IMAGE="chromedp/headless-shell@sha256:2d349b544a1ea6b5b5fd7c0fe99215ff662339c57407ee2e8c0a11af93516b04"
# nginx:stable-alpine, 2026-02-22 (the digest Dockerfile ships)
SERVER_IMAGE="nginx@sha256:15e96e59aa3b0aada3a121296e3bce117721f42d88f5f64217ef4b18f458c6ab"
# node:22-alpine, 2026-02-22 (the digest Dockerfile builds with)
NODE_IMAGE="node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34"
RUN_ID="$$-$(date +%s)"
NETWORK="netwatch-viewport-$RUN_ID"
SERVER="netwatch-viewport-server-$RUN_ID"
BROWSER="netwatch-viewport-browser-$RUN_ID"
HARNESS="netwatch-viewport-harness-$RUN_ID"
ARTIFACT_DIR="$ROOT/tmp/viewport"
# Every container is named and removed here, including the harness itself:
# `timeout` below kills the `docker run` client, not the container it
# started, and an unnamed survivor keeps the --internal network in use so
# `docker network rm` fails too. This host runs many sessions at once and
# neither may be left behind.
cleanup() {
docker rm -f "$HARNESS" > /dev/null 2>&1 || true
docker rm -f "$BROWSER" > /dev/null 2>&1 || true
docker rm -f "$SERVER" > /dev/null 2>&1 || true
docker network rm "$NETWORK" > /dev/null 2>&1 || true
}
trap cleanup EXIT INT TERM
main() {
cd "$ROOT"
# Test what ships: the production build, not a dev server.
"$ROOT/script/frontend-test"
if [ ! -f "$ROOT/dist/index.html" ]; then
echo "frontend-viewport-test: dist/index.html missing after build" >&2
exit 1
fi
mkdir -p "$ARTIFACT_DIR"
# An --internal network has no route off the host, so the browser
# cannot reach the real internet no matter what the page asks for.
# Latency probes are answered by the harness instead. This also means
# no port can be published from it, which is why the harness itself
# runs as a third container on the same network rather than on the
# host.
docker network create --internal "$NETWORK" > /dev/null
# nginx.conf is a template: the image renders it over its own
# default.conf, with the same port and limit bin/entrypoint.sh uses.
# The empty file it includes trusts no proxy, as bin/entrypoint.sh
# writes it when TRUSTED_PROXIES is unset. nginx.conf also includes
# the security headers, so the page runs under the shipped policy.
docker run -d --rm --name "$SERVER" \
--network "$NETWORK" --network-alias netwatch \
-e PORT=8080 -e NGINX_ENVSUBST_FILTER='^PORT$' \
-v "$ROOT/dist:/usr/share/nginx/html:ro" \
-v "$ROOT/nginx.conf:/etc/nginx/templates/default.conf.template:ro" \
-v /dev/null:/etc/nginx/trusted-proxies.conf:ro \
-v "$ROOT/security-headers.conf:/etc/nginx/security-headers.conf:ro" \
"$SERVER_IMAGE" > /dev/null
# The image's own entrypoint already exposes CDP on 9222 and passes
# --no-sandbox, so only extra flags belong here; re-specifying the
# debugging port collides with it and leaves the endpoint bound to
# loopback only. --hide-scrollbars keeps innerWidth equal to
# clientWidth, so the overflow assertion has no scrollbar-sized slack
# to hide behind, and matches the overlay scrollbars phones use.
docker run -d --rm --name "$BROWSER" --init --shm-size=1g \
--network "$NETWORK" \
"$BROWSER_IMAGE" \
--hide-scrollbars \
> /dev/null
# Chrome refuses DevTools requests whose Host header is neither
# localhost nor an IP address, so dial the container by address rather
# than by its network alias.
browser_ip="$(docker inspect \
-f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' \
"$BROWSER")"
timeout 900 docker run --rm --init --name "$HARNESS" \
--network "$NETWORK" \
--user "$(id -u):$(id -g)" \
-v "$ROOT:/app" \
-w /app \
-e NETWATCH_ROOT=/app \
-e NETWATCH_BASE_URL=http://netwatch:8080 \
-e "NETWATCH_CDP_URL=http://$browser_ip:9222" \
-e NETWATCH_ARTIFACT_DIR=/app/tmp/viewport \
"$NODE_IMAGE" \
node test/viewport/harness.js
}
main "$@"
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
# script/install-precommit: install the git pre-commit hook that runs
# script/precommit. Our own extension to scripts-to-rule-them-all.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
hook=".git/hooks/pre-commit"
printf '#!/bin/sh\nset -e\nscript/precommit\n' > "$hook"
chmod +x "$hook"
echo "pre-commit hook installed: runs script/precommit"
}
main "$@"
Executable
+21
View File
@@ -0,0 +1,21 @@
#!/bin/sh
# script/lint: lint the whole repo: prettier over the frontend, then the
# Go linter over backend/.
#
# The Go linter runs only in Docker: this builds the lint stage of
# Dockerfile, the digest-pinned golangci-lint image, which runs the
# backend's fmt-check and lint targets. --no-cache makes the linter
# really run every time rather than reuse an earlier result, and the
# stage is built for its checks alone, so no image is kept.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
"$ROOT/script/frontend-lint"
timeout 300 docker build --no-cache --target lint \
--output type=cacheonly .
}
main "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# script/precommit: run by the git pre-commit hook; fails the commit if
# checks fail. Our own extension to scripts-to-rule-them-all.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
main() {
"$SCRIPT_DIR/check"
}
main "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# script/projectname: output the name of this project. Our own
# extension to scripts-to-rule-them-all. Other scripts that need the
# name (e.g. script/docker) call this, so they can stay identical
# across all repos.
set -eu
main() {
echo "netwatch"
}
main "$@"
Executable
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
# script/setup: set up the repo for development after a fresh clone:
# installs dependencies and the git pre-commit hook.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
main() {
"$SCRIPT_DIR/bootstrap"
"$SCRIPT_DIR/install-precommit"
}
main "$@"
Executable
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# script/test: run the test suite for the whole repo: the frontend at
# the repo root, then the Go backend in backend/. Both halves together
# get 30 seconds; each also keeps its own limit for the Dockerfiles.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
timeout 30 sh -c 'script/frontend-test && backend/script/test'
}
main "$@"
+24
View File
@@ -0,0 +1,24 @@
# The security headers REPO_POLICIES.md requires on every response.
# nginx.conf includes this file, which Dockerfile copies to
# /etc/nginx/security-headers.conf. always sends each header on error
# responses too.
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# Scripts and styles load only from the page's own origin. Inline ones
# are blocked, style attributes in markup included, so style elements
# through classes or element.style. data: images are for the favicon
# in index.html. connect-src is * because the browser checks each probe in
# src/main.js against it, and also every redirect the probe follows,
# and several of those hosts redirect to others; a list of hosts here
# would block those probes. It also covers the reports the page sends
# to its own origin.
add_header Content-Security-Policy "default-src 'self'; connect-src *; img-src 'self' data:; object-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'" always;
add_header X-Frame-Options DENY always;
add_header X-Content-Type-Options nosniff always;
# The probed hosts are not told where the page is served from.
add_header Referrer-Policy no-referrer always;
add_header Permissions-Policy "accelerometer=(), camera=(), display-capture=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), usb=()" always;
+215 -23
View File
@@ -7,9 +7,11 @@ import "./styles.css";
// graphMaxLatency — values above it pin to the top of the chart but still // graphMaxLatency — values above it pin to the top of the chart but still
// display their real value in the latency figure. The history buffer holds // display their real value in the latency figure. The history buffer holds
// maxHistoryPoints samples (historyDuration / updateInterval). // maxHistoryPoints samples (historyDuration / updateInterval).
// reportInterval is how often collected samples are POSTed to the backend.
const CONFIG = { const CONFIG = {
updateInterval: 3000, updateInterval: 3000,
maxHistoryPoints: 100, maxHistoryPoints: 100,
reportInterval: 60000,
get historyDuration() { get historyDuration() {
return (this.maxHistoryPoints * this.updateInterval) / 1000; return (this.maxHistoryPoints * this.updateInterval) / 1000;
}, },
@@ -346,6 +348,159 @@ class AppState {
} }
} }
// --- Reporting ---------------------------------------------------------------
// A random UUIDv4. `crypto.randomUUID` exists only in secure contexts
// (HTTPS or localhost); over plain HTTP to any other host — the normal LAN
// deployment — it is undefined, so feature-detect it and otherwise build the
// id from `crypto.getRandomValues`, which is available in insecure contexts.
function randomId() {
if (typeof crypto !== "undefined" && crypto.randomUUID) {
return crypto.randomUUID();
}
const bytes = new Uint8Array(16);
crypto.getRandomValues(bytes);
bytes[6] = (bytes[6] & 0x0f) | 0x40; // version 4
bytes[8] = (bytes[8] & 0x3f) | 0x80; // variant 1
const hex = [...bytes].map((b) => b.toString(16).padStart(2, "0"));
return (
hex.slice(0, 4).join("") +
"-" +
hex.slice(4, 6).join("") +
"-" +
hex.slice(6, 8).join("") +
"-" +
hex.slice(8, 10).join("") +
"-" +
hex.slice(10, 16).join("")
);
}
// A random id identifying this browser across reports. Generated once and
// kept in localStorage; if storage is unavailable (e.g. private mode) a
// fresh id is used for this session only.
function getClientId() {
const key = "netwatch-client-id";
try {
let id = localStorage.getItem(key);
if (!id) {
id = randomId();
localStorage.setItem(key, id);
}
return id;
} catch {
return randomId();
}
}
// Build the delta report body the backend decodes, plus the new per-host
// high-water marks. Pure function of the passed state: `hosts` is an array
// of { name, url, status, history }, `since` maps a host url to the Unix-ms
// timestamp of the last sample already reported for it, and `now` is a Date.
// Only non-paused samples newer than the mark are included. Returns null
// when no host has an unreported sample.
export function buildReport(hosts, clientId, now, since) {
const reportHosts = [];
const marks = new Map();
for (const host of hosts) {
const mark = since.get(host.url) ?? 0;
const samples = [];
let high = mark;
for (const p of host.history) {
if (p.paused) continue;
if (p.timestamp <= mark) continue;
samples.push({
t: p.timestamp,
latency: p.latency,
error: p.error ?? null,
});
if (p.timestamp > high) high = p.timestamp;
}
if (samples.length === 0) continue;
reportHosts.push({
name: host.name,
url: host.url,
status: host.status,
history: samples,
});
marks.set(host.url, high);
}
if (reportHosts.length === 0) return null;
return {
body: {
clientId,
geo: null,
hosts: reportHosts,
timestamp: now.toISOString(),
},
marks,
};
}
// Periodically POSTs unreported samples to the same-origin backend. Holds
// the per-host high-water marks so each report is a delta; marks only
// advance on a delivered report, so a failed POST simply re-sends those
// samples next interval (bounded by the history window — whatever has since
// fallen out is dropped). Failure is quiet: one debug line per outage, one
// on recovery, never an alert, never a tight retry loop.
//
// Only one report is ever in flight, and it is abandoned after half the
// interval, so a slow POST can neither overlap the next report (which would
// carry the same samples) nor stall reporting for good.
class Reporter {
constructor(state, clientId, intervalMs) {
this.state = state;
this.clientId = clientId;
this.intervalMs = intervalMs;
this.marks = new Map();
this.failing = false;
this.sending = false;
this.timerId = null;
}
start() {
if (this.timerId) return;
this.timerId = setInterval(() => this.flush(), this.intervalMs);
}
async flush() {
if (this.sending || this.state.paused) return;
const report = buildReport(
this.state.allHosts,
this.clientId,
new Date(),
this.marks,
);
if (!report) return;
this.sending = true;
try {
const resp = await fetch("/api/v1/reports", {
method: "POST",
headers: { "Content-Type": "application/json" },
credentials: "omit",
body: JSON.stringify(report.body),
signal: AbortSignal.timeout(this.intervalMs / 2),
});
if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
for (const [url, t] of report.marks) {
const current = this.marks.get(url) ?? 0;
this.marks.set(url, Math.max(current, t));
}
if (this.failing) {
log.debug("Report delivery recovered");
this.failing = false;
}
} catch (err) {
if (!this.failing) {
log.debug(`Report delivery failed: ${err.message}`);
this.failing = true;
}
} finally {
this.sending = false;
}
}
}
// --- Latency Measurement ----------------------------------------------------- // --- Latency Measurement -----------------------------------------------------
async function measureLatency(url) { async function measureLatency(url) {
@@ -537,6 +692,12 @@ class SparklineRenderer {
// --- UI Renderer ------------------------------------------------------------- // --- UI Renderer -------------------------------------------------------------
// The per-host status line must stay wrappable: its populated content is
// wider than the host column at a 320px viewport, and `whitespace-nowrap`
// here overflows the element and forces the whole document to scroll
// horizontally.
const STATUS_TEXT_CLASS = "status-text text-xs text-right col-span-2 mt-5";
function hostRowHTML(host, index, showPin = true) { function hostRowHTML(host, index, showPin = true) {
const pinColor = host.pinned const pinColor = host.pinned
? "text-blue-500" ? "text-blue-500"
@@ -555,14 +716,14 @@ function hostRowHTML(host, index, showPin = true) {
${pinBtn} ${pinBtn}
<div class="w-[420px] flex-shrink-0 grid grid-cols-[minmax(0,1fr)_auto] items-center"> <div class="w-[420px] flex-shrink-0 grid grid-cols-[minmax(0,1fr)_auto] items-center">
<div class="flex items-center gap-2 min-w-[200px]"> <div class="flex items-center gap-2 min-w-[200px]">
<div class="w-3 h-3 rounded-full flex-shrink-0" style="background-color: ${latencyHex(null)}"></div> <div class="w-3 h-3 rounded-full flex-shrink-0 bg-[#6b7280]"></div>
<span class="font-medium text-white truncate">${host.name}</span> <span class="font-medium text-white truncate">${host.name}</span>
</div> </div>
<div class="latency-value text-4xl font-bold tabular-nums text-right mt-3" data-host="${index}"> <div class="latency-value text-4xl font-bold tabular-nums text-right mt-3" data-host="${index}">
<span class="text-gray-500">---</span> <span class="text-gray-500">---</span>
</div> </div>
<a href="${host.url}" target="_blank" rel="noopener" class="text-xs text-gray-500 truncate block col-span-2 -mt-2">${host.url}</a> <a href="${host.url}" target="_blank" rel="noopener" class="text-xs text-gray-500 truncate block col-span-2 -mt-2">${host.url}</a>
<div class="status-text text-xs text-gray-500 whitespace-nowrap text-right col-span-2 mt-5" data-host="${index}">waiting...</div> <div class="${STATUS_TEXT_CLASS} text-gray-500" data-host="${index}">waiting...</div>
</div> </div>
<div class="flex-grow sparkline-container rounded overflow-hidden border border-gray-700/30"> <div class="flex-grow sparkline-container rounded overflow-hidden border border-gray-700/30">
<canvas class="sparkline-canvas w-full" data-host="${index}" height="${CONFIG.canvasHeight}"></canvas> <canvas class="sparkline-canvas w-full" data-host="${index}" height="${CONFIG.canvasHeight}"></canvas>
@@ -671,7 +832,7 @@ function buildUI(state) {
</p> </p>
<p class="mt-2"><a href="https://git.eeqj.de/sneak/netwatch/commit/${__COMMIT_FULL__}" target="_blank" rel="noopener" class="text-gray-600 hover:text-gray-400">${__COMMIT_HASH__}</a></p> <p class="mt-2"><a href="https://git.eeqj.de/sneak/netwatch/commit/${__COMMIT_FULL__}" target="_blank" rel="noopener" class="text-gray-600 hover:text-gray-400">${__COMMIT_HASH__}</a></p>
<p class="mt-2"> <p class="mt-2">
<label class="cursor-pointer"> <label class="debug-toggle-label cursor-pointer">
<input type="checkbox" id="debug-toggle" class="mr-1"> <input type="checkbox" id="debug-toggle" class="mr-1">
<span>Debug log</span> <span>Debug log</span>
</label> </label>
@@ -688,6 +849,26 @@ function buildUI(state) {
// --- UI Updaters ------------------------------------------------------------- // --- UI Updaters -------------------------------------------------------------
// Renders `min 1ms / med 2ms / avg 3ms / max 4ms`. Each label, value and
// trailing separator is one unbreakable unit, so wrapping only ever happens
// between stats and a wrapped line never starts with a separator.
function statusStatsHTML(stats) {
return stats
.map(([label, value], i) => {
const sep =
i < stats.length - 1
? ` <span class="text-gray-500">/</span>`
: "";
return (
`<span class="whitespace-nowrap">` +
`<span class="text-gray-400">${label} </span>` +
`<span class="${latencyClass(value, "online")}">${value}ms</span>` +
`${sep}</span>`
);
})
.join(" ");
}
function updateHostRow(host, index) { function updateHostRow(host, index) {
const latencyEl = document.querySelector( const latencyEl = document.querySelector(
`.latency-value[data-host="${index}"]`, `.latency-value[data-host="${index}"]`,
@@ -712,28 +893,22 @@ function updateHostRow(host, index) {
const min = host.minLatency(); const min = host.minLatency();
const max = host.maxLatency(); const max = host.maxLatency();
if (host.status === "online" && avg !== null) { if (host.status === "online" && avg !== null) {
statusEl.innerHTML = statusEl.innerHTML = statusStatsHTML([
`<span class="text-gray-400">min </span><span class="${latencyClass(min, "online")}">${min}ms</span>` + ["min", min],
` <span class="text-gray-500">/</span> ` + ["med", med],
`<span class="text-gray-400">med </span><span class="${latencyClass(med, "online")}">${med}ms</span>` + ["avg", avg],
` <span class="text-gray-500">/</span> ` + ["max", max],
`<span class="text-gray-400">avg </span><span class="${latencyClass(avg, "online")}">${avg}ms</span>` + ]);
` <span class="text-gray-500">/</span> ` + statusEl.className = STATUS_TEXT_CLASS;
`<span class="text-gray-400">max </span><span class="${latencyClass(max, "online")}">${max}ms</span>`;
statusEl.className =
"status-text text-xs whitespace-nowrap text-right col-span-2 mt-5";
} else if (host.status === "offline") { } else if (host.status === "offline") {
statusEl.textContent = "unreachable"; statusEl.textContent = "unreachable";
statusEl.className = statusEl.className = `${STATUS_TEXT_CLASS} text-red-400`;
"status-text text-xs text-red-400 whitespace-nowrap text-right col-span-2 mt-5";
} else if (host.status === "error") { } else if (host.status === "error") {
statusEl.textContent = "timeout"; statusEl.textContent = "timeout";
statusEl.className = statusEl.className = `${STATUS_TEXT_CLASS} text-orange-400`;
"status-text text-xs text-orange-400 whitespace-nowrap text-right col-span-2 mt-5";
} else { } else {
statusEl.textContent = "connecting..."; statusEl.textContent = "connecting...";
statusEl.className = statusEl.className = `${STATUS_TEXT_CLASS} text-gray-500`;
"status-text text-xs text-gray-500 whitespace-nowrap text-right col-span-2 mt-5";
} }
SparklineRenderer.draw(canvas, host.history); SparklineRenderer.draw(canvas, host.history);
@@ -1044,8 +1219,7 @@ function greyOutUI(state) {
} }
if (statusEl) { if (statusEl) {
statusEl.textContent = "paused"; statusEl.textContent = "paused";
statusEl.className = statusEl.className = `${STATUS_TEXT_CLASS} text-gray-500`;
"status-text text-xs text-gray-500 whitespace-nowrap text-right col-span-2 mt-5";
} }
// Grey out the status dot // Grey out the status dot
const row = document.querySelector(`.host-row[data-index="${i}"]`); const row = document.querySelector(`.host-row[data-index="${i}"]`);
@@ -1143,6 +1317,19 @@ async function init() {
buildUI(state); buildUI(state);
log.info("UI built, starting tick loop"); log.info("UI built, starting tick loop");
// Reporting is best-effort: any failure setting it up (e.g. no usable
// crypto for the client id) must never stop the monitor from probing.
try {
const reporter = new Reporter(
state,
getClientId(),
CONFIG.reportInterval,
);
reporter.start();
} catch (err) {
log.error(`Reporting disabled: ${err.message}`);
}
document document
.getElementById("pause-btn") .getElementById("pause-btn")
.addEventListener("click", () => togglePause(state)); .addEventListener("click", () => togglePause(state));
@@ -1255,8 +1442,13 @@ async function init() {
setTimeout(() => handleResize(state), 100); setTimeout(() => handleResize(state), 100);
} }
if (document.readyState === "loading") { // Bootstrap only when loaded as the page: a real DOM containing the #app
// mount point this module renders into. Importing the module in a unit test
// (which has no #app) runs nothing, so buildReport can be tested in isolation.
if (typeof document !== "undefined" && document.getElementById("app")) {
if (document.readyState === "loading") {
document.addEventListener("DOMContentLoaded", init); document.addEventListener("DOMContentLoaded", init);
} else { } else {
init(); init();
}
} }
+34 -1
View File
@@ -14,6 +14,38 @@ body {
ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, monospace; ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, monospace;
} }
/* ---- Minimum tap targets ----------------------------------------------
Every interactive control is at least 44x44 CSS px (Apple HIG, WCAG 2.2
SC 2.5.5). Not scoped to a breakpoint or to `pointer: coarse`: a large
phone in landscape is above the 768px breakpoint and still a touch
device. */
/* The button grows to 44x44 while the negative margins keep its layout
footprint at the 16x16 of the icon inside it, so row height and the
icon's position are unchanged. */
.pin-btn {
display: flex;
align-items: center;
justify-content: center;
width: 2.75rem;
height: 2.75rem;
margin: -0.875rem;
}
/* A select paints its own background and border, so it has to actually be
44 tall rather than borrow the trick above. */
#interval-select {
min-height: 2.75rem;
}
/* The tappable target for #debug-toggle is the label wrapping it. */
.debug-toggle-label {
display: inline-flex;
align-items: center;
justify-content: center;
min-height: 2.75rem;
}
.sparkline-container { .sparkline-container {
background: linear-gradient( background: linear-gradient(
to bottom, to bottom,
@@ -38,9 +70,10 @@ body {
gap: 0.75rem; gap: 0.75rem;
} }
/* Pause button: smaller on mobile */ /* Pause button: smaller on mobile, but not below the tap-target floor */
#pause-btn { #pause-btn {
padding: 0.5rem 1rem; padding: 0.5rem 1rem;
min-height: 2.75rem;
} }
#pause-btn svg { #pause-btn svg {
+112
View File
@@ -0,0 +1,112 @@
# Responsive-layout harness
Automated verification of the responsive layout that landed in #5. Run it with:
```bash
make frontend-viewport-test
```
It builds `dist/`, serves it from the same digest-pinned `nginx` image and the
same `nginx.conf` the shipping container uses, drives a digest-pinned headless
Chrome against it over CDP, and asserts on computed layout at every viewport
width derived from the app's own CSS. Screenshots land in `tmp/viewport/`
alongside a `results.json`; they are artifacts for a human to look at when
something fails, not the evidence. The assertions are the evidence.
The target is deliberately outside `make check`: it needs Docker and takes
minutes, and `make test` has to stay under 20 seconds.
## How the widths are chosen
Not from a list of phone models. `viewports.js` parses the `@media` conditions
out of `src/styles.css` and scans `src/main.js` and `index.html` for Tailwind
responsive prefixes, then tests every breakpoint it finds at one pixel below it,
exactly on it, and one pixel above it. A generic 375px "phone" test sails
straight past an off-by-one at a media query boundary; `max-width: 768px`
matches _at_ 768, and the sweep pins down which side of that line each layout is
on.
Nothing hardcodes 768. Add a second media block or start using `md:` classes and
the new breakpoint is covered without this directory being touched. The app is
desktop-first today (all narrow rules live in `max-width` blocks); a
`min-width`-only, mobile-first set is handled as its inverse, and a set that
mixes the two makes the run fail loudly rather than test the right widths with
the wrong expectation. Four further viewports are fixed anchors, each with a
stated reason: a 320px floor, a 1280px desktop baseline, and two phone-landscape
sizes straddling the breakpoint for the rotation case.
## What it asserts
- **app-rendered** — enough host rows exist and enough of them show a numeric
latency. This one exists so the rest cannot pass vacuously against a blank
page.
- **no-horizontal-overflow** — `documentElement.scrollWidth` fits the layout
viewport, with the widest offending element named.
- **nothing-past-viewport-edge** — no visible element's box extends past the
viewport edge.
- **no-clipped-text** — nothing hides text behind `overflow: hidden`. Deliberate
ellipsis truncation (Tailwind's `truncate`, used on host names and URLs) is
excluded: it is a design choice, not breakage.
- **tap-targets-44px** — every interactive control is at least 44x44 CSS px on
touch viewports, _and_ each selector in the control list matched at least the
number of visible elements it declares. The second half is what stops the
check passing vacuously: with size alone, a renamed class would take its
controls out of the measured set and the check would report "all 0 controls
are at least 44x44" and pass. See below.
- **host-rows-stacked / host-rows-side-by-side** — the rows genuinely reflow.
Computed `flex-direction` _and_ the actual geometry are checked, and in the
narrow layout the info block and the sparkline must each occupy essentially
the full row width. A row that merely shrank its 420px column would fail.
- **probing-still-runs / gateway-detection-still-runs** — narrow viewports keep
probing and keep detecting the gateway. The mobile early-return path proposed
in #8 was rejected; this is what would catch it coming back.
### The tap-target threshold
44x44 CSS px. That is the figure in Apple's Human Interface Guidelines and in
WCAG 2.2 SC 2.5.5 "Target Size (Enhanced)". WCAG 2.2 SC 2.5.8 (level AA) sets a
lower 24x24 floor, but that floor comes with a spacing exception these controls
do not qualify for — the pin buttons sit directly against the host name they
belong to.
## Determinism
The browser container runs on an `--internal` docker network and has no route to
the internet, so the app's latency probes cannot reach anything real. The
harness answers them itself from a fixed delay table, with a deterministic
fraction failed outright, so the rows render a realistic spread of one-, two-
and three-digit latencies plus some unreachable rows. That spread is what the
layout has to survive; 24 identical `---` placeholders would not exercise it.
## What this cannot verify
Real limits, so nobody re-parks this issue as needing hardware:
- **Non-Chromium engines.** This is Chrome. iOS Safari is WebKit and cannot be
emulated by it; Safari-specific bugs (viewport units under a collapsing URL
bar, `-webkit-fill-available`, form control metrics) will not show up here.
- **Real touch input.** `hasTouch` emulation changes what the page is told, not
how a finger behaves. Gesture handling, scroll momentum, double-tap zoom and
hover-state fallbacks on touch are out of scope.
- **Physical pixel density and rendering.** `deviceScaleFactor` is set, but
subpixel antialiasing, OLED colour rendering and actual legibility at a given
physical size are not measurable here.
- **Fonts.** The container has DejaVu, not the platform's own UI monospace. Text
metrics are therefore close to, but not identical to, a real device — a layout
that fits here by a few pixels might not there.
- **On-device performance.** Canvas sparkline redraw cost, battery, and
behaviour on a slow radio are not measured.
- **Browser chrome.** The address bar, safe-area insets and notch cutouts are
not simulated.
Everything else this issue was actually about — does the layout reflow, does
anything overflow, is content clipped, are the controls big enough — is a
function of viewport width and CSS, and is covered above.
## Relation to the unit test framework (#21)
Complementary layers, not two stacks. `vitest` (#21) will exercise module-level
logic in-process with no browser. This harness exercises rendered layout in a
real engine and is the only thing here that can see a media query. Neither
replaces the other; assertions about computed styles and element geometry belong
here, assertions about functions belong in `vitest`.
+245
View File
@@ -0,0 +1,245 @@
// Pass/fail decisions for the responsive-layout harness.
//
// Kept in node rather than in the page so that a failure can be reported
// with the measurements that produced it. Every check runs at every
// viewport; none of them short-circuits, so one failure does not hide the
// rest.
// Minimum tap target, in CSS pixels. 44x44 is the figure in Apple's Human
// Interface Guidelines and in WCAG 2.2 SC 2.5.5 "Target Size (Enhanced)".
// WCAG 2.2 SC 2.5.8 (level AA) sets a lower 24x24 floor, but that floor
// comes with a spacing exception these controls do not qualify for: the
// pin buttons sit directly against the host name they belong to. Held at
// 44 deliberately.
export const MIN_TAP_TARGET_PX = 44;
// The controls named in the definition of done, plus the pause button.
// Each carries the smallest number of *visible* instances the page has to
// contain for the tap-target oracle to be measuring anything at all.
//
// Without those floors the check is inert: `undersized` is empty both when
// every control is large enough and when the selectors have gone stale and
// matched nothing, and the pass condition cannot tell those apart. A single
// combined floor would not be enough either — 26 pin buttons would cover
// for all three singleton controls vanishing at once — so the floor is per
// selector, and one stale selector out of four fails the check.
export const INTERACTIVE_CONTROLS = [
{ selector: "#pause-btn", minCount: 1 },
{ selector: "#interval-select", minCount: 1 },
// One per pinnable host row. `app-rendered` already requires at least
// 10 host rows, so a count below that means the pin buttons stopped
// being rendered per row rather than that there were fewer hosts.
{ selector: ".pin-btn", minCount: 10 },
{ selector: "#debug-toggle", minCount: 1 },
];
export const INTERACTIVE_SELECTORS = INTERACTIVE_CONTROLS.map(
(control) => control.selector,
);
// A host row is only "reflowed" if it stacked *and* went full width.
// A row that merely shrank its 420px info column would keep
// flex-direction: row, and a row that stacked but left the info column at
// its fixed width would fail the width test.
const FULL_WIDTH_FRACTION = 0.9;
function summarise(items, format, limit = 3) {
const shown = items.slice(0, limit).map(format).join("; ");
const rest = items.length > limit ? ` (+${items.length - limit} more)` : "";
return shown + rest;
}
// Collapse an overflow report to the elements actually responsible.
// Identical elements (24 host rows all doing the same thing) are counted
// rather than listed, and the deepest ones come first, since every
// ancestor of an overflowing element also reports as overflowing.
function deepestOffenders(entries) {
const byElement = new Map();
for (const entry of entries) {
const reach = entry.reach ?? entry.right;
const existing = byElement.get(entry.el);
if (existing) {
existing.count += 1;
existing.reach = Math.max(existing.reach, reach);
} else {
byElement.set(entry.el, { ...entry, reach, count: 1 });
}
}
return [...byElement.values()].sort(
(a, b) => b.depth - a.depth || b.reach - a.reach,
);
}
function checkRowLayout(row, expectStacked) {
if (expectStacked) {
if (row.flexDirection !== "column") {
return `row ${row.index}: flex-direction is ${row.flexDirection}, expected column`;
}
if (row.sparkline.top < row.info.bottom - 1) {
return `row ${row.index}: sparkline top ${row.sparkline.top} is above info bottom ${row.info.bottom} — still side by side`;
}
const minWidth = row.containerWidth * FULL_WIDTH_FRACTION;
if (row.info.width < minWidth) {
return `row ${row.index}: info block is ${row.info.width}px of ${row.containerWidth}px — shrunk, not reflowed`;
}
if (row.sparkline.width < minWidth) {
return `row ${row.index}: sparkline is ${row.sparkline.width}px of ${row.containerWidth}px — shrunk, not reflowed`;
}
return null;
}
if (row.flexDirection !== "row") {
return `row ${row.index}: flex-direction is ${row.flexDirection}, expected row`;
}
if (row.sparkline.left < row.info.right - 1) {
return `row ${row.index}: sparkline left ${row.sparkline.left} overlaps info right ${row.info.right} — not side by side`;
}
return null;
}
export function evaluateChecks(facts, viewport, probes) {
const checks = [];
const check = (name, ok, detail) => checks.push({ name, ok, detail });
// Guard against the whole harness passing vacuously because the page
// never rendered. Everything below is only meaningful if this holds.
check(
"app-rendered",
facts.rowCount >= 10 && facts.numericLatencies >= 5,
`${facts.rowCount} host rows, ${facts.numericLatencies} showing a numeric latency`,
);
const viewportWidth = Math.min(facts.innerWidth, facts.documentClientWidth);
const culprits = deepestOffenders([
...facts.overflowing,
...facts.contentOverflowing,
]);
check(
"no-horizontal-overflow",
facts.documentScrollWidth <= viewportWidth,
`documentElement.scrollWidth ${facts.documentScrollWidth} vs viewport ${viewportWidth}` +
(culprits.length === 0
? ""
: "; widest content: " +
summarise(
culprits,
(c) =>
`${c.el} reaches ${Math.round(c.reach)}px${c.count > 1 ? ` (x${c.count})` : ""}`,
)),
);
check(
"nothing-past-viewport-edge",
facts.overflowing.length === 0,
facts.overflowing.length === 0
? "no element extends past the viewport"
: `${facts.overflowing.length} element(s) past the edge: ` +
summarise(
facts.overflowing,
(o) => `${o.el} spans ${o.left}..${o.right}`,
),
);
check(
"no-clipped-text",
facts.clipped.length === 0,
facts.clipped.length === 0
? "no element hides text behind overflow (deliberate ellipsis excluded)"
: `${facts.clipped.length} element(s) clipping text: ` +
summarise(
facts.clipped,
(c) =>
`${c.el} scrollWidth ${c.scrollWidth} > clientWidth ${c.clientWidth}`,
),
);
if (viewport.touch) {
// Presence first: a selector that matches nothing contributes no
// undersized targets, so without this the check would report
// "all 0 controls are at least 44x44" and pass.
const seen = new Map();
for (const target of facts.tapTargets) {
seen.set(target.selector, (seen.get(target.selector) ?? 0) + 1);
}
const missing = INTERACTIVE_CONTROLS.filter(
(control) => (seen.get(control.selector) ?? 0) < control.minCount,
);
const undersized = facts.tapTargets.filter(
(t) => t.width < MIN_TAP_TARGET_PX || t.height < MIN_TAP_TARGET_PX,
);
const bySelector = new Map();
for (const target of undersized) {
const existing = bySelector.get(target.selector);
if (!existing || target.width * target.height < existing.area) {
bySelector.set(target.selector, {
...target,
area: target.width * target.height,
count: (existing?.count ?? 0) + 1,
});
} else {
existing.count += 1;
}
}
const detail = [];
if (missing.length > 0) {
detail.push(
"oracle is not measuring the page: " +
summarise(
missing,
(c) =>
`${c.selector} matched ${seen.get(c.selector) ?? 0} visible element(s), expected at least ${c.minCount}`,
4,
),
);
}
detail.push(
undersized.length === 0
? `${facts.tapTargets.length} controls measured, all at least ${MIN_TAP_TARGET_PX}x${MIN_TAP_TARGET_PX}`
: `${undersized.length} of ${facts.tapTargets.length} controls below ${MIN_TAP_TARGET_PX}x${MIN_TAP_TARGET_PX}: ` +
summarise(
[...bySelector.values()],
(t) =>
`${t.selector} ${t.width}x${t.height}${t.count > 1 ? ` (x${t.count})` : ""}`,
4,
),
);
check(
`tap-targets-${MIN_TAP_TARGET_PX}px`,
missing.length === 0 && undersized.length === 0,
detail.join("; "),
);
}
const badRows = facts.rows
.map((row) => checkRowLayout(row, viewport.expectStacked))
.filter(Boolean);
check(
viewport.expectStacked ? "host-rows-stacked" : "host-rows-side-by-side",
facts.rows.length > 0 && badRows.length === 0,
facts.rows.length === 0
? "no host rows were measured"
: badRows.length === 0
? `all ${facts.rows.length} rows laid out as expected`
: `${badRows.length} of ${facts.rows.length} rows wrong: ` +
summarise(badRows, (r) => r),
);
// The mobile early-return path proposed in #8 was rejected: narrow
// viewports must keep probing and keep detecting the gateway, not
// quietly skip work.
check(
"probing-still-runs",
probes.attempted > 0,
`${probes.attempted} outbound probe requests issued`,
);
check(
"gateway-detection-still-runs",
facts.gatewayDetected,
facts.gatewayDetected
? "Local Gateway row present"
: "no Local Gateway row — gateway detection did not run or did not complete",
);
return checks;
}
+184
View File
@@ -0,0 +1,184 @@
// Layout facts collected from inside the page.
//
// This function is serialised and evaluated in the browser, so it must be
// entirely self-contained: no imports, no closures over module scope. It
// only *measures*; every pass/fail decision is made back in node by
// checks.js, so failures can be reported with real numbers attached.
export function collectLayoutFacts(options) {
const describe = (el) => {
const id = el.id ? "#" + el.id : "";
const classes =
typeof el.className === "string" && el.className.trim()
? "." + el.className.trim().split(/\s+/).slice(0, 3).join(".")
: "";
return el.tagName.toLowerCase() + id + classes;
};
const round = (n) => Math.round(n * 10) / 10;
// Overflow propagates up every ancestor, so a single wide element
// reports as body, #app, the row, and so on. Depth lets the report
// name the deepest — that is, the actual — offender.
const depthOf = (el) => {
let depth = 0;
for (let node = el.parentElement; node; node = node.parentElement) {
depth++;
}
return depth;
};
const isVisible = (el) => {
const style = getComputedStyle(el);
if (style.display === "none") return false;
if (style.visibility === "hidden") return false;
const rect = el.getBoundingClientRect();
return rect.width > 0 && rect.height > 0;
};
const innerWidth = window.innerWidth;
const clientWidth = document.documentElement.clientWidth;
// Under mobile emulation Chrome lets window.innerWidth *grow* to the
// width of overflowing content, exactly as a phone zooms out to fit a
// too-wide page. Measuring against it would therefore hide the
// overflow it is supposed to expose: at a 320px device width a page
// that spills to 350 reports innerWidth 350 and looks clean. Every
// comparison below is against the layout viewport instead.
const viewportWidth = Math.min(innerWidth, clientWidth);
const elements = Array.from(document.querySelectorAll("body *"));
// Elements sticking out past the right (or left) edge of the viewport.
// The document-level scrollWidth check says *that* the page overflows;
// this says *what* is doing it.
const overflowing = [];
// Elements clipping their own text. Deliberate ellipsis truncation
// (Tailwind's `truncate`) is opt-in and excluded: it is a design
// choice, not breakage.
const clipped = [];
// Elements whose content spills out of their own box without being
// clipped, past the right edge of the viewport. A block element is
// only ever as wide as its container, so text overflowing it has no
// element rect of its own to catch — but it is exactly what drags
// documentElement.scrollWidth past the viewport width, so without
// this the page-level overflow failure has nothing to point at.
const contentOverflowing = [];
for (const el of elements) {
if (!isVisible(el)) continue;
const rect = el.getBoundingClientRect();
if (rect.right > viewportWidth + 1 || rect.left < -1) {
overflowing.push({
el: describe(el),
depth: depthOf(el),
left: round(rect.left),
right: round(rect.right),
});
}
const style = getComputedStyle(el);
const clips =
style.overflowX === "hidden" || style.overflowX === "clip";
const ellipsis = style.textOverflow === "ellipsis";
const hasText = el.textContent.trim().length > 0;
const spills =
el.clientWidth > 0 && el.scrollWidth > el.clientWidth + 1;
if (clips && !ellipsis && hasText && spills) {
clipped.push({
el: describe(el),
scrollWidth: el.scrollWidth,
clientWidth: el.clientWidth,
});
}
if (
!clips &&
spills &&
rect.left + el.scrollWidth > viewportWidth + 1
) {
contentOverflowing.push({
el: describe(el),
depth: depthOf(el),
scrollWidth: el.scrollWidth,
clientWidth: el.clientWidth,
reach: round(rect.left + el.scrollWidth),
});
}
}
// Interactive controls. The measured target is the nearest thing that
// is genuinely tappable — for a checkbox that is the <label> wrapping
// it, which is larger than the box itself and is what a finger hits.
const tapTargets = [];
for (const selector of options.interactiveSelectors) {
for (const el of document.querySelectorAll(selector)) {
if (!isVisible(el)) continue;
const target = el.closest("button, a, label, select") || el;
const rect = target.getBoundingClientRect();
tapTargets.push({
selector,
el: describe(target),
width: round(rect.width),
height: round(rect.height),
});
}
}
// Host rows. The question is not "did it get narrower" but "did it
// reflow": the info block and the sparkline must end up stacked
// vertically and full width in the narrow layout, and side by side in
// the wide one. Both the computed flex-direction and the actual
// geometry are recorded so a row that claims to be a column but is
// still laid out side by side cannot slip through.
const rows = [];
for (const row of document.querySelectorAll(".host-row")) {
const inner = row.firstElementChild;
if (!inner) continue;
const sparkline = inner.querySelector(".sparkline-container");
const info = sparkline ? sparkline.previousElementSibling : null;
if (!sparkline || !info) continue;
const innerStyle = getComputedStyle(inner);
const innerRect = inner.getBoundingClientRect();
const infoRect = info.getBoundingClientRect();
const sparkRect = sparkline.getBoundingClientRect();
rows.push({
index: row.dataset.index,
flexDirection: innerStyle.flexDirection,
containerWidth: round(innerRect.width),
info: {
left: round(infoRect.left),
right: round(infoRect.right),
bottom: round(infoRect.bottom),
width: round(infoRect.width),
},
sparkline: {
left: round(sparkRect.left),
top: round(sparkRect.top),
width: round(sparkRect.width),
},
});
}
const localRows = Array.from(
document.querySelectorAll("#local-hosts .host-row"),
);
return {
innerWidth,
// innerWidth includes any classic scrollbar, clientWidth does not.
// Reported separately so the overflow check can hold itself to the
// narrower of the two rather than to whichever one is more
// forgiving.
documentClientWidth: document.documentElement.clientWidth,
documentScrollWidth: document.documentElement.scrollWidth,
overflowing,
contentOverflowing,
clipped,
tapTargets,
rows,
rowCount: document.querySelectorAll(".host-row").length,
numericLatencies: Array.from(
document.querySelectorAll(".latency-value"),
).filter((el) => /\d/.test(el.textContent)).length,
gatewayDetected: localRows.some((row) =>
row.textContent.includes("Local Gateway"),
),
};
}
+258
View File
@@ -0,0 +1,258 @@
// Responsive-layout harness.
//
// Drives the built frontend in a real, containerised, digest-pinned Chrome
// over CDP and asserts on computed layout at every viewport width derived
// from the app's own CSS. Screenshots are written alongside as artifacts;
// they are not the evidence, the assertions are.
//
// This is not meant to be run by hand. `make frontend-viewport-test` brings
// up the browser and the web server and then runs this; every input it
// needs arrives in the environment.
import { mkdirSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import puppeteer from "puppeteer-core";
import { collectLayoutFacts } from "./facts.js";
import { evaluateChecks, INTERACTIVE_SELECTORS } from "./checks.js";
import { deriveViewports } from "./viewports.js";
function required(name) {
const value = process.env[name];
if (!value) {
throw new Error(
`${name} is not set; run this via script/frontend-viewport-test`,
);
}
return value;
}
const ROOT = required("NETWATCH_ROOT");
const BASE_URL = required("NETWATCH_BASE_URL");
const CDP_URL = required("NETWATCH_CDP_URL");
const ARTIFACT_DIR = required("NETWATCH_ARTIFACT_DIR");
const BROWSER_TIMEOUT_MS = 60000;
const PAGE_TIMEOUT_MS = 30000;
// Canned responses for the app's outbound latency probes. The browser
// container sits on an --internal docker network and physically cannot
// reach the internet, so nothing here is about blocking traffic; it is
// about determinism. Real probes would render 24 rows of whatever the
// network happened to be doing. These delays make the rows show a
// realistic spread of value widths — one, two and three digit latencies,
// plus some unreachable rows — because that spread is what the layout has
// to survive.
const PROBE_DELAYS_MS = [2, 45, 123, 456, 780];
// One in every UNREACHABLE_MODULUS probes is failed outright so that the
// offline row rendering is exercised too.
const UNREACHABLE_MODULUS = 7;
// The gateway candidate that "answers", so gateway detection succeeds and
// the Local Gateway row renders. Matches GATEWAY_CANDIDATES in src/main.js.
const RESPONSIVE_GATEWAY = "http://192.168.1.1";
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
function stableHash(text) {
let hash = 0;
for (let i = 0; i < text.length; i++) {
hash = (hash * 31 + text.charCodeAt(i)) | 0;
}
return Math.abs(hash);
}
async function connectBrowser() {
const deadline = Date.now() + BROWSER_TIMEOUT_MS;
let lastError;
for (;;) {
try {
const response = await fetch(`${CDP_URL}/json/version`);
const info = await response.json();
// The endpoint advertises whatever Host it was reached on;
// pin it back to the address we actually dialled.
const endpoint = new URL(info.webSocketDebuggerUrl);
endpoint.host = new URL(CDP_URL).host;
const browser = await puppeteer.connect({
browserWSEndpoint: endpoint.toString(),
protocolTimeout: BROWSER_TIMEOUT_MS,
});
return { browser, version: info.Browser };
} catch (error) {
lastError = error;
if (Date.now() > deadline) {
throw new Error(`browser never came up: ${lastError}`);
}
await sleep(250);
}
}
}
function installProbeResponder(page, probes) {
const respond = (request, delayMs) =>
sleep(delayMs).then(() =>
request.respond({
status: 200,
contentType: "text/plain",
body: "",
}),
);
page.on("request", (request) => {
const url = request.url();
const settle = async () => {
if (url.startsWith(BASE_URL) || url.startsWith("data:")) {
return request.continue();
}
probes.attempted++;
if (url.startsWith(RESPONSIVE_GATEWAY)) {
probes.fulfilled++;
return respond(request, 5);
}
const hash = stableHash(url);
if (hash % UNREACHABLE_MODULUS === 0) {
probes.failed++;
return request.abort("connectionfailed");
}
probes.fulfilled++;
return respond(
request,
PROBE_DELAYS_MS[hash % PROBE_DELAYS_MS.length],
);
};
// The page may be torn down while a delayed response is pending;
// that is not a harness failure.
settle().catch(() => {});
});
}
async function runViewport(browser, viewport) {
const page = await browser.newPage();
const probes = { attempted: 0, fulfilled: 0, failed: 0 };
try {
page.setDefaultTimeout(PAGE_TIMEOUT_MS);
await page.setRequestInterception(true);
installProbeResponder(page, probes);
await page.setViewport({
width: viewport.width,
height: viewport.height,
deviceScaleFactor: viewport.deviceScaleFactor,
isMobile: viewport.touch,
hasTouch: viewport.touch,
isLandscape: viewport.width > viewport.height,
});
await page.goto(BASE_URL, { waitUntil: "load" });
await page.waitForSelector(".host-row");
// The app discards its first tick as a cold start, so rows only
// carry real values from the second one. Changing the interval
// restarts the loop at 1s, which reaches a populated UI without
// waiting out two default 3s intervals — and exercises the
// interval dropdown while we are at it.
await page.select("#interval-select", "1000");
await page.waitForFunction(
() =>
Array.from(document.querySelectorAll(".latency-value")).filter(
(el) => /\d/.test(el.textContent),
).length >= 5,
);
// Let the resize/redraw handlers settle before measuring.
await page.evaluate(
() =>
new Promise((resolve) =>
requestAnimationFrame(() => requestAnimationFrame(resolve)),
),
);
const facts = await page.evaluate(collectLayoutFacts, {
interactiveSelectors: INTERACTIVE_SELECTORS,
});
const screenshot = join(
ARTIFACT_DIR,
`${viewport.width}x${viewport.height}-${viewport.name}.png`,
);
await page.screenshot({ path: screenshot, fullPage: true });
return {
viewport,
probes,
facts,
screenshot,
checks: evaluateChecks(facts, viewport, probes),
};
} finally {
await page.close().catch(() => {});
}
}
function report(results, conditions, browserVersion) {
const label = (viewport) =>
`${viewport.width}x${viewport.height}`.padEnd(9) +
" " +
viewport.name.padEnd(24);
console.log(`browser: ${browserVersion}`);
console.log(`served from: ${BASE_URL} (built dist/)`);
console.log(
"breakpoints: " +
conditions
.map((c) => `${c.type}-width ${c.px}px (${c.source})`)
.join(", "),
);
console.log("");
let passed = 0;
let failed = 0;
for (const result of results) {
const bad = result.checks.filter((c) => !c.ok);
passed += result.checks.length - bad.length;
failed += bad.length;
// Passing viewports get one line. Detail is for failures.
console.log(
`${bad.length === 0 ? "PASS" : "FAIL"} ${label(result.viewport)} ` +
`${result.checks.length - bad.length}/${result.checks.length} checks` +
`${result.viewport.expectStacked ? " [narrow layout expected]" : ""}`,
);
for (const check of bad) {
console.log(` ${check.name}: ${check.detail}`);
}
if (bad.length > 0) {
console.log(` why this width: ${result.viewport.why}`);
console.log(` screenshot: ${result.screenshot}`);
}
}
console.log("");
console.log(
`${results.length} viewports, ${passed + failed} checks: ` +
`${passed} passed, ${failed} failed`,
);
console.log(`artifacts: ${ARTIFACT_DIR}`);
return failed;
}
async function main() {
const { conditions, viewports } = deriveViewports(ROOT);
mkdirSync(ARTIFACT_DIR, { recursive: true });
const { browser, version } = await connectBrowser();
const results = [];
try {
for (const viewport of viewports) {
results.push(await runViewport(browser, viewport));
}
} finally {
await browser.disconnect().catch(() => {});
}
writeFileSync(
join(ARTIFACT_DIR, "results.json"),
JSON.stringify({ browser: version, conditions, results }, null, 2) +
"\n",
);
const failed = report(results, conditions, version);
process.exitCode = failed === 0 ? 0 : 1;
}
await main();
+224
View File
@@ -0,0 +1,224 @@
// Viewport derivation for the responsive-layout harness.
//
// The widths tested are read out of the CSS the application actually
// ships, not taken from a list of popular phone models. A generic 375px
// "phone" test sails straight past an off-by-one error at a media query
// boundary, which is the classic way a responsive layout breaks, so
// every breakpoint found in the sources is probed three times: one pixel
// below it, exactly on it, and one pixel above it.
//
// Nothing here hardcodes 768. If someone adds a second media block or
// starts using Tailwind responsive prefixes, that breakpoint starts
// being covered without this file being edited.
import { readFileSync } from "node:fs";
import { join } from "node:path";
// Tailwind CSS v4 default breakpoints, in rem. The app currently uses
// none of these prefixes, so the whole table is inert until someone
// writes an `md:`-prefixed utility class.
const TAILWIND_BREAKPOINT_REM = {
sm: 40,
md: 48,
lg: 64,
xl: 80,
"2xl": 96,
};
// The app does not override the root font size, so rem and em in media
// queries resolve against the browser default.
const ROOT_FONT_SIZE_PX = 16;
// Extract every min-width / max-width condition from the @media blocks in
// a stylesheet. Returns e.g. [{ type: "max", px: 768, source: "..." }].
export function mediaConditionsFromCss(css, source) {
const conditions = [];
for (const block of css.matchAll(/@media([^{]+)\{/g)) {
const features = block[1].matchAll(
/\(\s*(min|max)-width\s*:\s*([\d.]+)(px|rem|em)\s*\)/g,
);
for (const feature of features) {
const scale = feature[3] === "px" ? 1 : ROOT_FONT_SIZE_PX;
conditions.push({
type: feature[1],
px: Math.round(Number(feature[2]) * scale),
source,
});
}
}
return conditions;
}
// Extract the breakpoints implied by Tailwind responsive prefixes used in
// markup. A prefix only counts when it opens a utility class, so `text-sm`
// does not masquerade as the `sm:` breakpoint.
export function mediaConditionsFromMarkup(sources) {
const conditions = [];
for (const { path, text } of sources) {
for (const [name, rem] of Object.entries(TAILWIND_BREAKPOINT_REM)) {
const used = new RegExp(
`(^|["'\\s])${name}:[a-z0-9[\\](),_./%-]+`,
"m",
).test(text);
if (used) {
conditions.push({
type: "min",
px: rem * ROOT_FONT_SIZE_PX,
source: path,
});
}
}
}
return conditions;
}
// Whether a given width should be rendering the app's narrow (stacked)
// layout.
//
// Two breakpoint styles can be answered from the condition list alone:
//
// - Desktop-first, which is what the app ships today: the wide layout is
// unconditional and every narrow rule lives in a `max-width` block, so
// a width is narrow exactly when one of those blocks matches. Note that
// `max-width: 768px` matches *at* 768 — getting this inclusive boundary
// wrong in either direction is what the three-widths-per-breakpoint
// sweep exists to catch.
// - Mobile-first, which is what Tailwind's `sm:`/`md:` prefixes are: the
// stacked layout is the unconditional base and a `min-width` block is
// what widens it, so a width is narrow exactly when it sits below every
// `min-width` breakpoint.
//
// A mix of the two cannot be resolved from the breakpoints alone — which
// block owns the host-row reflow is a property of the rules inside it, not
// of the condition — so this throws rather than guessing. Guessing is how
// the wrong expectation gets applied at the right widths and the whole
// sweep quietly verifies nothing.
export function expectsStackedLayout(width, conditions) {
const kinds = new Set(conditions.map((c) => c.type));
for (const kind of kinds) {
if (kind !== "max" && kind !== "min") {
throw new Error(
`unsupported media condition type "${kind}" in ` +
"expectsStackedLayout (test/viewport/viewports.js)",
);
}
}
if (kinds.has("max") && kinds.has("min")) {
throw new Error(
"the app now mixes max-width and min-width breakpoints (" +
conditions
.map((c) => `${c.type}-width ${c.px}px in ${c.source}`)
.join(", ") +
"), so which layout a width should be showing can no longer " +
"be inferred from the breakpoint list; teach " +
"expectsStackedLayout in test/viewport/viewports.js which " +
"block owns the host-row reflow",
);
}
if (kinds.has("min")) {
return !conditions.some((c) => width >= c.px);
}
return conditions.some((c) => width <= c.px);
}
// Viewports that are not derived from a breakpoint. Each one is here for
// a stated reason; none of them is a stand-in for "a phone".
const ANCHOR_VIEWPORTS = [
{
name: "floor-portrait",
width: 320,
height: 568,
deviceScaleFactor: 2,
touch: true,
why: "320px is the narrowest viewport still in mainstream use; nothing has to work below it",
},
{
name: "phone-landscape-narrow",
width: 667,
height: 375,
deviceScaleFactor: 2,
touch: true,
why: "phone rotated to landscape, still inside the narrow layout",
},
{
name: "phone-landscape-wide",
width: 844,
height: 390,
deviceScaleFactor: 3,
touch: true,
why: "large phone rotated to landscape: crosses into the wide layout while still being a touch device",
},
{
name: "desktop",
width: 1280,
height: 800,
deviceScaleFactor: 1,
touch: false,
why: "desktop baseline",
},
];
export function deriveViewports(root) {
const conditions = [
...mediaConditionsFromCss(
readFileSync(join(root, "src/styles.css"), "utf8"),
"src/styles.css",
),
...mediaConditionsFromMarkup([
{
path: "src/main.js",
text: readFileSync(join(root, "src/main.js"), "utf8"),
},
{
path: "index.html",
text: readFileSync(join(root, "index.html"), "utf8"),
},
]),
];
if (conditions.length === 0) {
throw new Error(
"no responsive breakpoints found in src/styles.css, src/main.js or " +
"index.html — either the responsive layout was deleted or this " +
"derivation has stopped matching the sources",
);
}
const viewports = new Map();
const add = (viewport) => {
const key = `${viewport.width}x${viewport.height}`;
if (!viewports.has(key)) viewports.set(key, viewport);
};
for (const condition of conditions) {
for (const [offset, label] of [
[-1, "below"],
[0, "at"],
[+1, "above"],
]) {
const width = condition.px + offset;
add({
name: `${condition.type}-width-${condition.px}-${label}`,
width,
// Tall enough that the whole app is laid out in one column
// without the viewport height influencing wrapping.
height: 1024,
deviceScaleFactor: 2,
touch: true,
why: `${offset === 0 ? "exactly on" : `1px ${label}`} the ${condition.type}-width: ${condition.px}px breakpoint declared in ${condition.source}`,
});
}
}
for (const anchor of ANCHOR_VIEWPORTS) add(anchor);
return {
conditions,
viewports: [...viewports.values()]
.map((viewport) => ({
...viewport,
expectStacked: expectsStackedLayout(viewport.width, conditions),
}))
.sort((a, b) => a.width - b.width || a.height - b.height),
};
}
+7
View File
@@ -7,6 +7,13 @@ const commitFull = execSync("git rev-parse HEAD").toString().trim();
export default defineConfig({ export default defineConfig({
plugins: [tailwindcss()], plugins: [tailwindcss()],
server: {
// Proxy /api to a locally running netwatch-server so `yarn dev`
// exercises the real report-posting path.
proxy: {
"/api": "http://127.0.0.1:8080",
},
},
define: { define: {
__COMMIT_HASH__: JSON.stringify(commitHash), __COMMIT_HASH__: JSON.stringify(commitHash),
__COMMIT_FULL__: JSON.stringify(commitFull), __COMMIT_FULL__: JSON.stringify(commitFull),
+153 -1
View File
@@ -197,6 +197,14 @@
"@emnapi/runtime" "^1.7.1" "@emnapi/runtime" "^1.7.1"
"@tybys/wasm-util" "^0.10.1" "@tybys/wasm-util" "^0.10.1"
"@puppeteer/browsers@3.1.0":
version "3.1.0"
resolved "https://registry.yarnpkg.com/@puppeteer/browsers/-/browsers-3.1.0.tgz#5728ae0bc649263133ac1f8bd5d360eb75d11748"
integrity sha512-RDLpio3fH/qrj5k4DVY6eyiN8tCS0Zovd/6jW//n605oeqkWcUjn+3k+9ZtZBnbwMpsu0F7xDIiKXvVmG5c5Bw==
dependencies:
modern-tar "^0.7.6"
yargs "^18.0.0"
"@rollup/rollup-android-arm-eabi@4.57.0": "@rollup/rollup-android-arm-eabi@4.57.0":
version "4.57.0" version "4.57.0"
resolved "https://registry.yarnpkg.com/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.57.0.tgz#f762035679a6b168138c94c960fda0b0cdb00d98" resolved "https://registry.yarnpkg.com/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.57.0.tgz#f762035679a6b168138c94c960fda0b0cdb00d98"
@@ -441,6 +449,16 @@
resolved "https://registry.yarnpkg.com/@types/estree/-/estree-1.0.8.tgz#958b91c991b1867ced318bedea0e215ee050726e" resolved "https://registry.yarnpkg.com/@types/estree/-/estree-1.0.8.tgz#958b91c991b1867ced318bedea0e215ee050726e"
integrity sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w== integrity sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==
ansi-regex@^6.2.2:
version "6.2.2"
resolved "https://registry.yarnpkg.com/ansi-regex/-/ansi-regex-6.2.2.tgz#60216eea464d864597ce2832000738a0589650c1"
integrity sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg==
ansi-styles@^6.2.1:
version "6.2.3"
resolved "https://registry.yarnpkg.com/ansi-styles/-/ansi-styles-6.2.3.tgz#c044d5dcc521a076413472597a1acb1f103c4041"
integrity sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg==
autoprefixer@^10.4.23: autoprefixer@^10.4.23:
version "10.4.23" version "10.4.23"
resolved "https://registry.yarnpkg.com/autoprefixer/-/autoprefixer-10.4.23.tgz#c6aa6db8e7376fcd900f9fd79d143ceebad8c4e6" resolved "https://registry.yarnpkg.com/autoprefixer/-/autoprefixer-10.4.23.tgz#c6aa6db8e7376fcd900f9fd79d143ceebad8c4e6"
@@ -473,16 +491,43 @@ caniuse-lite@^1.0.30001759, caniuse-lite@^1.0.30001760:
resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001766.tgz#b6f6b55cb25a2d888d9393104d14751c6a7d6f7a" resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001766.tgz#b6f6b55cb25a2d888d9393104d14751c6a7d6f7a"
integrity sha512-4C0lfJ0/YPjJQHagaE9x2Elb69CIqEPZeG0anQt9SIvIoOH4a4uaRl73IavyO+0qZh6MDLH//DrXThEYKHkmYA== integrity sha512-4C0lfJ0/YPjJQHagaE9x2Elb69CIqEPZeG0anQt9SIvIoOH4a4uaRl73IavyO+0qZh6MDLH//DrXThEYKHkmYA==
chromium-bidi@17.0.2:
version "17.0.2"
resolved "https://registry.yarnpkg.com/chromium-bidi/-/chromium-bidi-17.0.2.tgz#921a586deecd0c2d8b9242c4c1b73c3aa39ff77c"
integrity sha512-5v9GQFhTktFvotn/OFNJBmKLKRAb6n9r0bVCwf7sHgWc3/JryK0bj1nn93L3pHFrfgcsu6Be6EWsDi+1XHTGDg==
dependencies:
mitt "^3.0.1"
zod "^3.24.1"
cliui@^9.0.1:
version "9.0.1"
resolved "https://registry.yarnpkg.com/cliui/-/cliui-9.0.1.tgz#6f7890f386f6f1f79953adc1f78dec46fcc2d291"
integrity sha512-k7ndgKhwoQveBL+/1tqGJYNz097I7WOvwbmmU2AR5+magtbjPWQTS1C5vzGkBC8Ym8UWRzfKUzUUqFLypY4Q+w==
dependencies:
string-width "^7.2.0"
strip-ansi "^7.1.0"
wrap-ansi "^9.0.0"
detect-libc@^2.0.3: detect-libc@^2.0.3:
version "2.1.2" version "2.1.2"
resolved "https://registry.yarnpkg.com/detect-libc/-/detect-libc-2.1.2.tgz#689c5dcdc1900ef5583a4cb9f6d7b473742074ad" resolved "https://registry.yarnpkg.com/detect-libc/-/detect-libc-2.1.2.tgz#689c5dcdc1900ef5583a4cb9f6d7b473742074ad"
integrity sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ== integrity sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==
devtools-protocol@0.0.1653615:
version "0.0.1653615"
resolved "https://registry.yarnpkg.com/devtools-protocol/-/devtools-protocol-0.0.1653615.tgz#c600e0c619612156b2422a66d958ba188d87dbe8"
integrity sha512-pGVkY3T/qXxAp2nFPodwYqOevk6ncNMSmvL8QfRCx5ZWGd6Vor7AFNmyaA8Zs6uJyP1QAfjuLandCgvSix1BNA==
electron-to-chromium@^1.5.263: electron-to-chromium@^1.5.263:
version "1.5.282" version "1.5.282"
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.5.282.tgz#6695816e5b170210d6aa07561546ed7d97347630" resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.5.282.tgz#6695816e5b170210d6aa07561546ed7d97347630"
integrity sha512-FCPkJtpst28UmFzd903iU7PdeVTfY0KAeJy+Lk0GLZRwgwYHn/irRcaCbQQOmr5Vytc/7rcavsYLvTM8RiHYhQ== integrity sha512-FCPkJtpst28UmFzd903iU7PdeVTfY0KAeJy+Lk0GLZRwgwYHn/irRcaCbQQOmr5Vytc/7rcavsYLvTM8RiHYhQ==
emoji-regex@^10.3.0:
version "10.6.0"
resolved "https://registry.yarnpkg.com/emoji-regex/-/emoji-regex-10.6.0.tgz#bf3d6e8f7f8fd22a65d9703475bc0147357a6b0d"
integrity sha512-toUI84YS5YmxW219erniWD0CIVOo46xGKColeNQRgOzDorgBi1v4D71/OFzgD9GO2UGKIv1C3Sp8DAn0+j5w7A==
enhanced-resolve@^5.18.3: enhanced-resolve@^5.18.3:
version "5.18.4" version "5.18.4"
resolved "https://registry.yarnpkg.com/enhanced-resolve/-/enhanced-resolve-5.18.4.tgz#c22d33055f3952035ce6a144ce092447c525f828" resolved "https://registry.yarnpkg.com/enhanced-resolve/-/enhanced-resolve-5.18.4.tgz#c22d33055f3952035ce6a144ce092447c525f828"
@@ -523,7 +568,7 @@ esbuild@^0.27.0:
"@esbuild/win32-ia32" "0.27.2" "@esbuild/win32-ia32" "0.27.2"
"@esbuild/win32-x64" "0.27.2" "@esbuild/win32-x64" "0.27.2"
escalade@^3.2.0: escalade@^3.1.1, escalade@^3.2.0:
version "3.2.0" version "3.2.0"
resolved "https://registry.yarnpkg.com/escalade/-/escalade-3.2.0.tgz#011a3f69856ba189dffa7dc8fcce99d2a87903e5" resolved "https://registry.yarnpkg.com/escalade/-/escalade-3.2.0.tgz#011a3f69856ba189dffa7dc8fcce99d2a87903e5"
integrity sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA== integrity sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==
@@ -543,6 +588,16 @@ fsevents@~2.3.2, fsevents@~2.3.3:
resolved "https://registry.yarnpkg.com/fsevents/-/fsevents-2.3.3.tgz#cac6407785d03675a2a5e1a5305c697b347d90d6" resolved "https://registry.yarnpkg.com/fsevents/-/fsevents-2.3.3.tgz#cac6407785d03675a2a5e1a5305c697b347d90d6"
integrity sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw== integrity sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==
get-caller-file@^2.0.5:
version "2.0.5"
resolved "https://registry.yarnpkg.com/get-caller-file/-/get-caller-file-2.0.5.tgz#4f94412a82db32f36e3b0b9741f8a97feb031f7e"
integrity sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==
get-east-asian-width@^1.0.0, get-east-asian-width@^1.5.0:
version "1.6.0"
resolved "https://registry.yarnpkg.com/get-east-asian-width/-/get-east-asian-width-1.6.0.tgz#216900f91df11a8b2c198c3e1d93d6c035a776b9"
integrity sha512-QRbvDIbx6YklUe6RxeTeleMR0yv3cYH6PsPZHcnVn7xv7zO1BHN8r0XETu8n6Ye3Q+ahtSarc3WgtNWmehIBfA==
graceful-fs@^4.2.4: graceful-fs@^4.2.4:
version "4.2.11" version "4.2.11"
resolved "https://registry.yarnpkg.com/graceful-fs/-/graceful-fs-4.2.11.tgz#4183e4e8bf08bb6e05bbb2f7d2e0c8f712ca40e3" resolved "https://registry.yarnpkg.com/graceful-fs/-/graceful-fs-4.2.11.tgz#4183e4e8bf08bb6e05bbb2f7d2e0c8f712ca40e3"
@@ -634,6 +689,16 @@ magic-string@^0.30.21:
dependencies: dependencies:
"@jridgewell/sourcemap-codec" "^1.5.5" "@jridgewell/sourcemap-codec" "^1.5.5"
mitt@^3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/mitt/-/mitt-3.0.1.tgz#ea36cf0cc30403601ae074c8f77b7092cdab36d1"
integrity sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==
modern-tar@^0.7.6:
version "0.7.7"
resolved "https://registry.yarnpkg.com/modern-tar/-/modern-tar-0.7.7.tgz#ca71d79603630076b10733b0751ccab284bbc1ef"
integrity sha512-t9VmxaqrmANnEOBhpSDI6HD192Ge48k8vmWqQQL7hSFEqHEYwZbbsu49+aKLWZeRvFs3j1pMhXOqqF4kPlvjkQ==
nanoid@^3.3.11: nanoid@^3.3.11:
version "3.3.11" version "3.3.11"
resolved "https://registry.yarnpkg.com/nanoid/-/nanoid-3.3.11.tgz#4f4f112cefbe303202f2199838128936266d185b" resolved "https://registry.yarnpkg.com/nanoid/-/nanoid-3.3.11.tgz#4f4f112cefbe303202f2199838128936266d185b"
@@ -673,6 +738,18 @@ prettier@^3.8.1:
resolved "https://registry.yarnpkg.com/prettier/-/prettier-3.8.1.tgz#edf48977cf991558f4fcbd8a3ba6015ba2a3a173" resolved "https://registry.yarnpkg.com/prettier/-/prettier-3.8.1.tgz#edf48977cf991558f4fcbd8a3ba6015ba2a3a173"
integrity sha512-UOnG6LftzbdaHZcKoPFtOcCKztrQ57WkHDeRD9t/PTQtmT0NHSeWWepj6pS0z/N7+08BHFDQVUrfmfMRcZwbMg== integrity sha512-UOnG6LftzbdaHZcKoPFtOcCKztrQ57WkHDeRD9t/PTQtmT0NHSeWWepj6pS0z/N7+08BHFDQVUrfmfMRcZwbMg==
puppeteer-core@25.5.0:
version "25.5.0"
resolved "https://registry.yarnpkg.com/puppeteer-core/-/puppeteer-core-25.5.0.tgz#a41b14d582056b998e0bc3561ac47c7615d38a53"
integrity sha512-XPNT0dQJtphqQ4I29zxlG4IIPbg1iEHAQKWuQgtMJGXjACV77pZSmJvDi51IIIfd+DTKICcopJwUx4upVQ4XbA==
dependencies:
"@puppeteer/browsers" "3.1.0"
chromium-bidi "17.0.2"
devtools-protocol "0.0.1653615"
typed-query-selector "^2.12.2"
webdriver-bidi-protocol "0.4.2"
ws "^8.21.1"
rollup@^4.43.0: rollup@^4.43.0:
version "4.57.0" version "4.57.0"
resolved "https://registry.yarnpkg.com/rollup/-/rollup-4.57.0.tgz#9fa13c1fb779d480038f45708b5e01b9449b6853" resolved "https://registry.yarnpkg.com/rollup/-/rollup-4.57.0.tgz#9fa13c1fb779d480038f45708b5e01b9449b6853"
@@ -712,6 +789,30 @@ source-map-js@^1.2.1:
resolved "https://registry.yarnpkg.com/source-map-js/-/source-map-js-1.2.1.tgz#1ce5650fddd87abc099eda37dcff024c2667ae46" resolved "https://registry.yarnpkg.com/source-map-js/-/source-map-js-1.2.1.tgz#1ce5650fddd87abc099eda37dcff024c2667ae46"
integrity sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA== integrity sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==
string-width@^7.0.0, string-width@^7.2.0:
version "7.2.0"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-7.2.0.tgz#b5bb8e2165ce275d4d43476dd2700ad9091db6dc"
integrity sha512-tsaTIkKW9b4N+AEj+SVA+WhJzV7/zMhcSu78mLKWSk7cXMOSHsBKFWUs0fWwq8QyK3MgJBQRX6Gbi4kYbdvGkQ==
dependencies:
emoji-regex "^10.3.0"
get-east-asian-width "^1.0.0"
strip-ansi "^7.1.0"
string-width@^8.2.1:
version "8.2.2"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-8.2.2.tgz#7310516493df575742fe98af6fae87d85d5ed0ac"
integrity sha512-GaPUh5gfdrYzqeVNZvUfT23vYYxXzKYidUcnMtJg/3rxRV63EFZy3k6xfKlmfeJD0176lnUV/Usr3XcwSvFzpg==
dependencies:
get-east-asian-width "^1.5.0"
strip-ansi "^7.1.2"
strip-ansi@^7.1.0, strip-ansi@^7.1.2:
version "7.2.0"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-7.2.0.tgz#d22a269522836a627af8d04b5c3fd2c7fa3e32e3"
integrity sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w==
dependencies:
ansi-regex "^6.2.2"
tailwindcss@4.1.18, tailwindcss@^4.1.18: tailwindcss@4.1.18, tailwindcss@^4.1.18:
version "4.1.18" version "4.1.18"
resolved "https://registry.yarnpkg.com/tailwindcss/-/tailwindcss-4.1.18.tgz#f488ba47853abdb5354daf9679d3e7791fc4f4e3" resolved "https://registry.yarnpkg.com/tailwindcss/-/tailwindcss-4.1.18.tgz#f488ba47853abdb5354daf9679d3e7791fc4f4e3"
@@ -735,6 +836,11 @@ tslib@^2.4.0:
resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.8.1.tgz#612efe4ed235d567e8aba5f2a5fab70280ade83f" resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.8.1.tgz#612efe4ed235d567e8aba5f2a5fab70280ade83f"
integrity sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w== integrity sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==
typed-query-selector@^2.12.2:
version "2.12.2"
resolved "https://registry.yarnpkg.com/typed-query-selector/-/typed-query-selector-2.12.2.tgz#65e2462ac6b0aecfae1bfac1a4f3027070dbabaa"
integrity sha512-EOPFbyIub4ngnEdqi2yOcNeDLaX/0jcE1JoAXQDDMIthap7FoN795lc/SHfIq2d416VufXpM8z/lD+WRm2gfOQ==
update-browserslist-db@^1.2.0: update-browserslist-db@^1.2.0:
version "1.2.3" version "1.2.3"
resolved "https://registry.yarnpkg.com/update-browserslist-db/-/update-browserslist-db-1.2.3.tgz#64d76db58713136acbeb4c49114366cc6cc2e80d" resolved "https://registry.yarnpkg.com/update-browserslist-db/-/update-browserslist-db-1.2.3.tgz#64d76db58713136acbeb4c49114366cc6cc2e80d"
@@ -756,3 +862,49 @@ vite@^7.3.1:
tinyglobby "^0.2.15" tinyglobby "^0.2.15"
optionalDependencies: optionalDependencies:
fsevents "~2.3.3" fsevents "~2.3.3"
webdriver-bidi-protocol@0.4.2:
version "0.4.2"
resolved "https://registry.yarnpkg.com/webdriver-bidi-protocol/-/webdriver-bidi-protocol-0.4.2.tgz#f51bb71c2606e90e3d5727607c728b25d617b58b"
integrity sha512-VSV+fzfChirL3e7jay2yUC7B4HQCGtEWEg/MSSQbK+qWbqeGlRLlXTzPpYr3XGUvbpDHumWZBJxgesg4N7dbtA==
wrap-ansi@^9.0.0:
version "9.0.2"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-9.0.2.tgz#956832dea9494306e6d209eb871643bb873d7c98"
integrity sha512-42AtmgqjV+X1VpdOfyTGOYRi0/zsoLqtXQckTmqTeybT+BDIbM/Guxo7x3pE2vtpr1ok6xRqM9OpBe+Jyoqyww==
dependencies:
ansi-styles "^6.2.1"
string-width "^7.0.0"
strip-ansi "^7.1.0"
ws@^8.21.1:
version "8.21.3"
resolved "https://registry.yarnpkg.com/ws/-/ws-8.21.3.tgz#660b4faddb6a3e575c86e078126919961f4de4fc"
integrity sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw==
y18n@^5.0.5:
version "5.0.8"
resolved "https://registry.yarnpkg.com/y18n/-/y18n-5.0.8.tgz#7f4934d0f7ca8c56f95314939ddcd2dd91ce1d55"
integrity sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==
yargs-parser@^22.0.0:
version "22.0.0"
resolved "https://registry.yarnpkg.com/yargs-parser/-/yargs-parser-22.0.0.tgz#87b82094051b0567717346ecd00fd14804b357c8"
integrity sha512-rwu/ClNdSMpkSrUb+d6BRsSkLUq1fmfsY6TOpYzTwvwkg1/NRG85KBy3kq++A8LKQwX6lsu+aWad+2khvuXrqw==
yargs@^18.0.0:
version "18.1.0"
resolved "https://registry.yarnpkg.com/yargs/-/yargs-18.1.0.tgz#cd7e98c703ef51695bbbf062ed58f28e94291b56"
integrity sha512-2rAgRKu54VsHkqI0/tYkmluGXHD4KW7yZoycuqDQ15QOTnc2VVfy0nN/1eMhnQLO00A+dwtK20xuCnc1YGeUyg==
dependencies:
cliui "^9.0.1"
escalade "^3.1.1"
get-caller-file "^2.0.5"
string-width "^8.2.1"
y18n "^5.0.5"
yargs-parser "^22.0.0"
zod@^3.24.1:
version "3.25.76"
resolved "https://registry.yarnpkg.com/zod/-/zod-3.25.76.tgz#26841c3f6fd22a6a2760e7ccb719179768471e34"
integrity sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==