Compare commits
1
Commits
next
..
4091a3b41a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4091a3b41a |
+7
-79
@@ -1,81 +1,9 @@
|
|||||||
# .dockerignore does NOT use .gitignore semantics. Docker matches with
|
node_modules
|
||||||
# moby/patternmatcher: filepath.Match plus `**`, so `*` does not cross
|
dist
|
||||||
# `/` and an unprefixed pattern is anchored at the context root. Every
|
tmp
|
||||||
# depth-independent pattern therefore needs `**/`, or `config/.env` and
|
.DS_Store
|
||||||
# `certs/server.key` still ship while this file reads as solved. Only
|
*.log
|
||||||
# genuinely root-anchored entries go unprefixed. Never transplant these
|
|
||||||
# into .gitignore, where `**/` is wrong.
|
|
||||||
#
|
|
||||||
# Matching is case-sensitive, so secrets use character ranges rather
|
|
||||||
# than an ALL-CAPS twin, which would still miss `Server.Key`.
|
|
||||||
#
|
|
||||||
# Extend with this repo's own host-built artifacts, written anchored:
|
|
||||||
# `/myapp`, never `**/myapp`, which also matches `cmd/myapp/` and
|
|
||||||
# deletes the package directory from the context.
|
|
||||||
|
|
||||||
# .git is sent without its config. Without a VERSION build argument the
|
|
||||||
# stage that compiles runs `git describe --tags --always` on .git, which
|
|
||||||
# does not need .git/config; that file can hold a credential, such as a
|
|
||||||
# password in a remote URL or the token the CI checkout step stores there.
|
|
||||||
# Each submodule keeps a config with the same exposure in its git directory
|
|
||||||
# under .git/modules/, nested again for a submodule's own submodules, or in
|
|
||||||
# its own .git directory when it keeps one.
|
|
||||||
# KNOWN GAP: a submodule whose name has a `config` segment (`config`,
|
|
||||||
# `deploy/config`, `config/lib`) loses its whole git directory, because
|
|
||||||
# `**/.git/modules/**/config` also matches that segment's directory
|
|
||||||
# under .git/modules/. Go's version stamping then fails the build;
|
|
||||||
# nothing leaks. Name such a submodule without that segment:
|
|
||||||
# `git submodule add --name`.
|
|
||||||
**/.git/config
|
|
||||||
**/.git/modules/**/config
|
|
||||||
|
|
||||||
# Agent scratch: one full checkout of the repo per in-flight agent.
|
|
||||||
# Anchored because it occurs once where agents run at the repo root.
|
|
||||||
# KNOWN GAP: a repo running agents in subdirectories still ships
|
|
||||||
# `services/api/.claude/` and must add its own anchored entry.
|
|
||||||
.claude
|
.claude
|
||||||
|
|
||||||
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
# .git is sent so the build can stamp the version, without its config.
|
||||||
# convention. Re-include a committed template with a negation if the
|
.git/config
|
||||||
# build needs one: `!docs/example.env`.
|
|
||||||
**/*.[eE][nN][vV]
|
|
||||||
**/.[eE][nN][vV].*
|
|
||||||
**/.[eE][nN][vV][rR][cC]
|
|
||||||
|
|
||||||
# Private keys and the bundles carrying them. Public certificates
|
|
||||||
# (*.crt, *.cer) are deliberately absent: they are legitimate inputs.
|
|
||||||
**/*.[pP][eE][mM]
|
|
||||||
**/*.[kK][eE][yY]
|
|
||||||
**/*.[pP]12
|
|
||||||
**/*.[pP][fF][xX]
|
|
||||||
**/[iI][dD]_[rR][sS][aA]
|
|
||||||
**/[iI][dD]_[dD][sS][aA]
|
|
||||||
**/[iI][dD]_[eE][cC][dD][sS][aA]
|
|
||||||
**/[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
|
||||||
**/[iI][dD]_[eE][dD]25519
|
|
||||||
**/[iI][dD]_[eE][dD]25519_[sS][kK]
|
|
||||||
|
|
||||||
# Dependencies: restored inside the image, never copied in.
|
|
||||||
**/node_modules
|
|
||||||
|
|
||||||
# OS metadata.
|
|
||||||
**/.DS_Store
|
|
||||||
**/Thumbs.db
|
|
||||||
|
|
||||||
# Editor state: never a build input, and it churns COPY.
|
|
||||||
**/*.swp
|
|
||||||
**/*.swo
|
|
||||||
**/*~
|
|
||||||
**/*.bak
|
|
||||||
**/.idea
|
|
||||||
**/.vscode
|
|
||||||
**/*.sublime-*
|
|
||||||
|
|
||||||
# This repository's own entries, after the shared content above: the
|
|
||||||
# frontend build, the viewport test's output, the binary `make build`
|
|
||||||
# writes and the reports `make run` stores, and log files at the root.
|
|
||||||
/dist
|
|
||||||
/tmp
|
|
||||||
/netwatch-server
|
|
||||||
/data
|
|
||||||
/*.log
|
|
||||||
|
|||||||
@@ -10,8 +10,3 @@ insert_final_newline = true
|
|||||||
|
|
||||||
[Makefile]
|
[Makefile]
|
||||||
indent_style = tab
|
indent_style = tab
|
||||||
|
|
||||||
# This repository's own entries, after the shared content above.
|
|
||||||
|
|
||||||
[*.go]
|
|
||||||
indent_style = tab
|
|
||||||
|
|||||||
@@ -6,4 +6,7 @@ jobs:
|
|||||||
steps:
|
steps:
|
||||||
# actions/checkout v4.2.2, 2026-02-22
|
# actions/checkout v4.2.2, 2026-02-22
|
||||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
|
||||||
- run: script/cibuild
|
# script/cibuild bootstraps, runs every check and builds the
|
||||||
|
# image. script/bootstrap links what it installs into
|
||||||
|
# ~/.local/bin, so that has to be on PATH for the rest.
|
||||||
|
- run: PATH="$HOME/.local/bin:$PATH" script/cibuild
|
||||||
|
|||||||
+5
-41
@@ -11,54 +11,18 @@ Thumbs.db
|
|||||||
.vscode/
|
.vscode/
|
||||||
*.sublime-*
|
*.sublime-*
|
||||||
|
|
||||||
# Agent scratch (worktrees of this repo, created and destroyed by
|
|
||||||
# in-flight tooling). Unanchored: .gitignore patterns already match at
|
|
||||||
# every depth, so no prefix is wanted here. This is not a .dockerignore
|
|
||||||
# entry and must not be given a `**/` prefix on the way into one.
|
|
||||||
.claude/
|
|
||||||
|
|
||||||
# Node
|
# Node
|
||||||
node_modules/
|
node_modules/
|
||||||
|
|
||||||
# Secrets. Unanchored like every entry above, so each matches at every
|
# Environment / secrets
|
||||||
# depth. Matching is case-sensitive on Linux, so names use character
|
.env
|
||||||
# ranges rather than a lowercase form that misses `Server.Key`.
|
.env.*
|
||||||
|
*.pem
|
||||||
# Environment files. `*.env` covers bare `.env` and the `prod.env`
|
*.key
|
||||||
# convention. Only the templates `example.env` and `sample.env` are
|
|
||||||
# re-included below. A repository that commits any other template adds
|
|
||||||
# its own negation after these lines, for example `!.env.example`.
|
|
||||||
*.[eE][nN][vV]
|
|
||||||
.[eE][nN][vV].*
|
|
||||||
.[eE][nN][vV][rR][cC]
|
|
||||||
!example.env
|
|
||||||
!sample.env
|
|
||||||
|
|
||||||
# Private keys and the bundles carrying them.
|
|
||||||
*.[pP][eE][mM]
|
|
||||||
*.[kK][eE][yY]
|
|
||||||
*.[pP]12
|
|
||||||
*.[pP][fF][xX]
|
|
||||||
[iI][dD]_[rR][sS][aA]
|
|
||||||
[iI][dD]_[dD][sS][aA]
|
|
||||||
[iI][dD]_[eE][cC][dD][sS][aA]
|
|
||||||
[iI][dD]_[eE][cC][dD][sS][aA]_[sS][kK]
|
|
||||||
[iI][dD]_[eE][dD]25519
|
|
||||||
[iI][dD]_[eE][dD]25519_[sS][kK]
|
|
||||||
|
|
||||||
# This repository's own entries, after the shared content above.
|
|
||||||
|
|
||||||
# Build output
|
# Build output
|
||||||
dist/
|
dist/
|
||||||
tmp/
|
tmp/
|
||||||
/netwatch-server
|
|
||||||
|
|
||||||
# The reports make run stores, in netwatch-server's default DATA_DIR
|
|
||||||
/data/
|
|
||||||
|
|
||||||
# Go test binaries and coverage output
|
|
||||||
*.test
|
|
||||||
*.out
|
|
||||||
|
|
||||||
# Logs
|
# Logs
|
||||||
*.log
|
*.log
|
||||||
|
|||||||
@@ -1,2 +1,7 @@
|
|||||||
|
dist/
|
||||||
node_modules/
|
node_modules/
|
||||||
|
tmp/
|
||||||
yarn.lock
|
yarn.lock
|
||||||
|
.claude/
|
||||||
|
# The org standard file, copied verbatim; backend/script/lint checks its sha256.
|
||||||
|
backend/.golangci.yml
|
||||||
|
|||||||
+78
-112
@@ -2,129 +2,95 @@
|
|||||||
# passes /api/, /.well-known/healthcheck and /metrics to netwatch-server,
|
# passes /api/, /.well-known/healthcheck and /metrics to netwatch-server,
|
||||||
# the Go backend, which runs in the same container on loopback only.
|
# the Go backend, which runs in the same container on loopback only.
|
||||||
# bin/entrypoint.sh starts and watches both.
|
# bin/entrypoint.sh starts and watches both.
|
||||||
#
|
|
||||||
# The lint and test phases are the gates: `make lint` (script/lint)
|
|
||||||
# builds the lint stage alone and `make test` (script/test) the test
|
|
||||||
# stage alone, and the two build stages, frontend-build and builder,
|
|
||||||
# depend on both, so the image cannot be built unless they pass. Each
|
|
||||||
# phase covers the frontend as well, through a copy from a node stage.
|
|
||||||
# Inside them each tool is invoked directly, never through make or
|
|
||||||
# script/, whose lint and test are themselves docker builds.
|
|
||||||
|
|
||||||
# Frontend lint stage: eslint with the rules in eslint.config.js. The
|
# Lint stage — fast feedback on formatting and lint issues. The
|
||||||
# lint phase below runs it.
|
# golangci/golangci-lint image ships Go, gofmt, make and the linter, so
|
||||||
|
# nothing is installed here. The root make lint builds this stage alone.
|
||||||
|
# golangci/golangci-lint:v2.12.2 (2026-08-10)
|
||||||
|
FROM golangci/golangci-lint@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
COPY backend/go.mod backend/go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
COPY backend/ .
|
||||||
|
RUN make fmt-check
|
||||||
|
RUN make lint
|
||||||
|
|
||||||
|
# Backend build stage
|
||||||
|
# golang:1.25-alpine (2026-02-27)
|
||||||
|
FROM golang:1.25-alpine@sha256:f6751d823c26342f9506c03797d2527668d095b0a15f1862cddb4d927a7a4ced AS builder
|
||||||
|
|
||||||
|
# gcc and musl-dev are for make test: its race detector needs cgo, which
|
||||||
|
# Go turns on by itself once a C compiler is present. make build still
|
||||||
|
# sets CGO_ENABLED=0, so the binary stays static.
|
||||||
|
RUN apk add --no-cache gcc git make musl-dev
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Force BuildKit to run the lint stage before proceeding. BuildKit runs
|
||||||
|
# stages in parallel by default; without this no-op copy a lint failure
|
||||||
|
# would not gate compilation.
|
||||||
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
|
|
||||||
|
COPY backend/go.mod backend/go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
COPY backend/ .
|
||||||
|
|
||||||
|
RUN make test
|
||||||
|
|
||||||
|
# make build is a shim around backend/script/build, the one definition
|
||||||
|
# of the build command:
|
||||||
|
# CGO_ENABLED=0 go build -trimpath -ldflags "-s -w -X main.Version=..."
|
||||||
|
# That script reads VERSION from the environment, so it is handed over
|
||||||
|
# there rather than as a make variable.
|
||||||
|
#
|
||||||
|
# The version is the VERSION build argument when one is given, otherwise
|
||||||
|
# `git describe --tags --always` of the repo's .git: the tag on a tagged
|
||||||
|
# commit, tag-N-gHASH on a commit after one, the short commit when no
|
||||||
|
# tag is reachable. A version that still comes out empty, dev or unknown
|
||||||
|
# fails the build. .git goes to /git, not /src/.git, where go build would
|
||||||
|
# find it and record VCS details of a work tree holding only backend/.
|
||||||
|
COPY .git /git
|
||||||
|
ARG VERSION
|
||||||
|
RUN version="${VERSION:-$(git --git-dir=/git describe --tags --always)}"; \
|
||||||
|
case "$version" in ""|dev|unknown) \
|
||||||
|
echo "version is '$version' although .git is present" >&2; \
|
||||||
|
exit 1 ;; \
|
||||||
|
esac; \
|
||||||
|
VERSION="$version" make build
|
||||||
|
|
||||||
|
# Frontend lint stage — eslint over the JavaScript, as the lint stage
|
||||||
|
# above lints the Go. The root make lint builds this stage alone too.
|
||||||
# node:22-alpine as of 2026-02-22
|
# node:22-alpine as of 2026-02-22
|
||||||
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS frontend-lint
|
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS frontend-lint
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY package.json yarn.lock ./
|
COPY package.json yarn.lock ./
|
||||||
RUN yarn install --frozen-lockfile
|
RUN yarn install --frozen-lockfile
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN yarn eslint .
|
RUN script/frontend-lint
|
||||||
|
|
||||||
# Lint phase: golangci-lint over the Go packages with .golangci.yml,
|
# Frontend stage
|
||||||
# and eslint through the copy from frontend-lint at the end. The
|
# node:22-alpine as of 2026-02-22
|
||||||
# golangci/golangci-lint image ships Go and the linter.
|
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS frontend
|
||||||
# golangci/golangci-lint:v2.14.0, 2026-09-24
|
WORKDIR /app
|
||||||
FROM golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f AS lint
|
|
||||||
WORKDIR /src
|
# Force BuildKit to run the frontend-lint stage before proceeding, as
|
||||||
COPY go.mod go.sum ./
|
# the builder stage does with the lint stage: without this no-op copy an
|
||||||
RUN go mod download
|
# eslint failure would not gate the image.
|
||||||
COPY . .
|
|
||||||
RUN golangci-lint run --config .golangci.yml ./...
|
|
||||||
# Nothing is wanted from frontend-lint; the copy is what makes this
|
|
||||||
# phase run it.
|
|
||||||
COPY --from=frontend-lint /app/yarn.lock /dev/null
|
COPY --from=frontend-lint /app/yarn.lock /dev/null
|
||||||
|
|
||||||
# Frontend test stage: the unit tests in test/unit/. The test phase
|
|
||||||
# below runs it, and the frontend-build stage starts from it. The tests
|
|
||||||
# print a dot each; if any fails, they run again with every test listed,
|
|
||||||
# and the step fails even if that run passes. NODE_OPTIONS chooses the
|
|
||||||
# reporter because yarn adds its arguments after the test files, where
|
|
||||||
# node would take a reporter option for one more file.
|
|
||||||
# node:22-alpine as of 2026-02-22
|
|
||||||
FROM node@sha256:e4bf2a82ad0a4037d28035ae71529873c069b13eb0455466ae0bc13363826e34 AS frontend-test
|
|
||||||
WORKDIR /app
|
|
||||||
COPY package.json yarn.lock ./
|
COPY package.json yarn.lock ./
|
||||||
RUN yarn install --frozen-lockfile
|
RUN yarn install --frozen-lockfile
|
||||||
|
RUN apk add --no-cache git make
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN NODE_OPTIONS=--test-reporter=dot timeout 90 yarn --silent run test || \
|
# make frontend-check runs the frontend tests and format check; its test
|
||||||
{ echo "--- Rerunning with every test listed for details ---"; \
|
# step runs the unit tests, then the production yarn build, so this both
|
||||||
NODE_OPTIONS=--test-reporter=spec timeout 90 yarn --silent run test; \
|
# produces dist/ and gates the image on test and formatting regressions.
|
||||||
exit 1; }
|
# This node stage has neither Go nor Docker; the frontend-lint, lint and
|
||||||
|
# builder stages above gate the rest.
|
||||||
|
RUN make frontend-check
|
||||||
|
|
||||||
# Test phase: the backend's tests with the race detector and coverage,
|
# Runtime stage
|
||||||
# and the frontend's through the copy from the frontend-test stage at
|
|
||||||
# the end. -race needs cgo and so a C compiler, which the Debian Go image
|
|
||||||
# ships and the alpine one does not. -timeout 90s is a backstop above
|
|
||||||
# the 60-second cap on the suite. The rerun with -v only shows details:
|
|
||||||
# the step fails however it ends, because the first run already failed.
|
|
||||||
# Go's module and build caches are in memory (tmpfs) for the go test
|
|
||||||
# step alone, so the image make test tags does not carry them and the
|
|
||||||
# build spends no time writing them into it. They start empty on every
|
|
||||||
# build, so no stored test result can stand in for a run.
|
|
||||||
# golang:1.25.7-trixie, 2026-10-07
|
|
||||||
FROM golang@sha256:2b174ffcf56c7ad0c47d30d2630693265639ddf2a5141149c2da34db921791b4 AS test
|
|
||||||
WORKDIR /src
|
|
||||||
COPY . .
|
|
||||||
RUN --mount=type=tmpfs,target=/go/pkg/mod \
|
|
||||||
--mount=type=tmpfs,target=/root/.cache/go-build \
|
|
||||||
go test -timeout 90s -race -cover ./... || \
|
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
|
||||||
go test -timeout 90s -race -v ./...; exit 1; }
|
|
||||||
# Nothing is wanted from the frontend-test stage; the copy is what makes
|
|
||||||
# this phase run its tests.
|
|
||||||
COPY --from=frontend-test /app/yarn.lock /dev/null
|
|
||||||
|
|
||||||
# Frontend build stage: the production build into dist/, which the
|
|
||||||
# runtime stage serves, on the yarn packages and source of the
|
|
||||||
# frontend-test stage. Nothing is wanted from the two phases; the copies
|
|
||||||
# are what make BuildKit build them first, so this stage cannot run
|
|
||||||
# unless lint and test passed.
|
|
||||||
FROM frontend-test AS frontend-build
|
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
|
||||||
COPY --from=test /src/go.sum /dev/null
|
|
||||||
# vite.config.js reads the commit for the page's footer with git.
|
|
||||||
RUN apk add --no-cache git
|
|
||||||
RUN yarn build
|
|
||||||
|
|
||||||
# Backend build stage. Nothing is wanted from the two phases; the copies
|
|
||||||
# are what make BuildKit build them first, so this stage cannot run
|
|
||||||
# unless lint and test passed.
|
|
||||||
# golang:1.25-alpine (2026-02-27)
|
|
||||||
FROM golang@sha256:f6751d823c26342f9506c03797d2527668d095b0a15f1862cddb4d927a7a4ced AS builder
|
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
|
||||||
COPY --from=test /src/go.sum /dev/null
|
|
||||||
RUN apk add --no-cache git
|
|
||||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
|
||||||
RUN git config --system --add safe.directory /src
|
|
||||||
WORKDIR /src
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
RUN go mod download
|
|
||||||
COPY . .
|
|
||||||
|
|
||||||
# The same go build as in script/build, which builds the binary on the
|
|
||||||
# host.
|
|
||||||
#
|
|
||||||
# The version is the VERSION build argument when one is given, otherwise
|
|
||||||
# `git describe --tags --always` on the .git in the build context: the
|
|
||||||
# tag on a tagged commit, tag-N-gHASH on a commit after one, the short
|
|
||||||
# commit when no tag is reachable. With .git present, a version that is
|
|
||||||
# still empty, dev or unknown fails the build: git is missing or could
|
|
||||||
# not read the checkout.
|
|
||||||
ARG VERSION
|
|
||||||
RUN version="${VERSION:-$(git describe --tags --always)}"; \
|
|
||||||
if [ -e .git ]; then \
|
|
||||||
case "$version" in ""|dev|unknown) \
|
|
||||||
echo "version is '$version' although .git is present" >&2; \
|
|
||||||
exit 1 ;; \
|
|
||||||
esac; \
|
|
||||||
fi; \
|
|
||||||
CGO_ENABLED=0 go build -trimpath \
|
|
||||||
-ldflags "-s -w -X main.Version=$version" \
|
|
||||||
-o netwatch-server ./cmd/netwatch-server/
|
|
||||||
|
|
||||||
# Runtime stage, and the last one: a plain `docker build .` builds it
|
|
||||||
# and the stages it copies from, the two phases included.
|
|
||||||
# nginx:stable-alpine as of 2026-02-22
|
# nginx:stable-alpine as of 2026-02-22
|
||||||
FROM nginx@sha256:15e96e59aa3b0aada3a121296e3bce117721f42d88f5f64217ef4b18f458c6ab
|
FROM nginx@sha256:15e96e59aa3b0aada3a121296e3bce117721f42d88f5f64217ef4b18f458c6ab
|
||||||
|
|
||||||
@@ -137,9 +103,9 @@ RUN addgroup -g 1000 -S netwatch && \
|
|||||||
# At start-up the nginx image renders every template here into
|
# At start-up the nginx image renders every template here into
|
||||||
# conf.d; bin/entrypoint.sh says how.
|
# conf.d; bin/entrypoint.sh says how.
|
||||||
RUN rm /etc/nginx/conf.d/default.conf
|
RUN rm /etc/nginx/conf.d/default.conf
|
||||||
COPY configs/nginx.conf /etc/nginx/templates/netwatch.conf.template
|
COPY nginx.conf /etc/nginx/templates/netwatch.conf.template
|
||||||
COPY configs/security-headers.conf /etc/nginx/security-headers.conf
|
COPY security-headers.conf /etc/nginx/security-headers.conf
|
||||||
COPY --from=frontend-build /app/dist /usr/share/nginx/html
|
COPY --from=frontend /app/dist /usr/share/nginx/html
|
||||||
COPY --from=builder /src/netwatch-server /usr/local/bin/netwatch-server
|
COPY --from=builder /src/netwatch-server /usr/local/bin/netwatch-server
|
||||||
COPY bin/entrypoint.sh /usr/local/bin/entrypoint.sh
|
COPY bin/entrypoint.sh /usr/local/bin/entrypoint.sh
|
||||||
|
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
.PHONY: bootstrap setup dev build run clean test lint fmt fmt-check check \
|
.PHONY: bootstrap setup dev build test lint fmt fmt-check check \
|
||||||
add-dependency tidy frontend-viewport-test docker hooks
|
add-dependency tidy frontend-check frontend-viewport-test docker hooks
|
||||||
|
|
||||||
# Standard targets are thin shims; the implementations live in script/
|
# Standard targets are thin shims; the implementations live in script/
|
||||||
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
||||||
# of README.md). test, lint, fmt, fmt-check and check cover the whole
|
# of README.md). test, lint, fmt, fmt-check and check cover the whole
|
||||||
# repo: the frontend in web/ and the Go backend in cmd/ and internal/.
|
# repo: the frontend here and the Go backend in backend/.
|
||||||
|
|
||||||
bootstrap:
|
bootstrap:
|
||||||
@script/bootstrap
|
@script/bootstrap
|
||||||
@@ -15,17 +15,10 @@ setup:
|
|||||||
dev:
|
dev:
|
||||||
@script/dev
|
@script/dev
|
||||||
|
|
||||||
# The frontend into dist/ and the netwatch-server binary.
|
# The frontend only; backend/Makefile's build target builds the Go server.
|
||||||
build:
|
build:
|
||||||
@script/build
|
@script/build
|
||||||
|
|
||||||
# Builds, then runs netwatch-server on port 8080.
|
|
||||||
run:
|
|
||||||
@script/run
|
|
||||||
|
|
||||||
clean:
|
|
||||||
@script/clean
|
|
||||||
|
|
||||||
test:
|
test:
|
||||||
@script/test
|
@script/test
|
||||||
|
|
||||||
@@ -49,8 +42,14 @@ add-dependency:
|
|||||||
tidy:
|
tidy:
|
||||||
@script/tidy
|
@script/tidy
|
||||||
|
|
||||||
|
# The frontend tests and format check, for Dockerfile's frontend stage,
|
||||||
|
# which has neither Go nor Docker. Use check everywhere else.
|
||||||
|
frontend-check:
|
||||||
|
@script/frontend-check
|
||||||
|
|
||||||
# Responsive-layout verification in a containerised browser. Kept out of
|
# Responsive-layout verification in a containerised browser. Kept out of
|
||||||
# check: it takes minutes.
|
# check: it needs Docker and takes minutes, where make test has to stay
|
||||||
|
# under 20 seconds.
|
||||||
frontend-viewport-test:
|
frontend-viewport-test:
|
||||||
@script/frontend-viewport-test
|
@script/frontend-viewport-test
|
||||||
|
|
||||||
|
|||||||
@@ -1,35 +1,30 @@
|
|||||||
NetWatch is an MIT-licensed JavaScript single-page application by
|
NetWatch is an MIT-licensed JavaScript single-page application by
|
||||||
[@sneak](https://sneak.berlin) that provides real-time network latency
|
[@sneak](https://sneak.berlin) that provides real-time network latency
|
||||||
monitoring to common internet hosts, displayed with color-coded figures and
|
monitoring to common internet hosts, displayed with color-coded figures and
|
||||||
sparkline graphs, served from a static bucket or from its Docker image, where a
|
sparkline graphs, served from a static bucket or Docker container.
|
||||||
small Go backend stores the measurements the page reports.
|
|
||||||
|
|
||||||
## Getting Started
|
## Getting Started
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Install the dependencies and the git pre-commit hook
|
# Install dependencies
|
||||||
make setup
|
yarn install
|
||||||
|
|
||||||
# Run the page on the Vite dev server
|
# Development server
|
||||||
make dev
|
yarn dev
|
||||||
|
|
||||||
# Build netwatch-server and run it on port 8080
|
# Production build
|
||||||
make run
|
yarn build
|
||||||
|
|
||||||
# Run the tests, both linters and the format check
|
# Preview production build
|
||||||
make check
|
yarn preview
|
||||||
|
|
||||||
# Build the page into dist/ and netwatch-server into the repo root
|
# Docker
|
||||||
make build
|
docker build -t netwatch .
|
||||||
|
|
||||||
# Build the image and run it
|
|
||||||
make docker
|
|
||||||
docker run -p 8080:8080 netwatch
|
docker run -p 8080:8080 netwatch
|
||||||
```
|
```
|
||||||
|
|
||||||
`make check` and `make docker` need Docker. `make dev` passes `/api` to
|
`yarn dev` proxies `/api` to `http://127.0.0.1:8080`, so a locally running
|
||||||
`http://127.0.0.1:8080`, where `make run` starts `netwatch-server` with its
|
`netwatch-server` (see `backend/`) receives the reports the page posts.
|
||||||
defaults, so the reports the page posts are stored in `data/reports`.
|
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
|
|
||||||
@@ -37,57 +32,56 @@ This repository adheres to the
|
|||||||
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
||||||
standard: normalized scripts in `script/` are the entrypoints for the
|
standard: normalized scripts in `script/` are the entrypoints for the
|
||||||
development workflow, and the Makefile targets are thin shims that call them.
|
development workflow, and the Makefile targets are thin shims that call them.
|
||||||
The scripts cover both the frontend and the Go backend, so `make check` fails if
|
The Go backend in `backend/` has its own `script/` directory and shim Makefile
|
||||||
either one is broken. We provide:
|
(see [backend/README.md](backend/README.md)). The root scripts cover both
|
||||||
|
halves, so the root `make check` fails if either one is broken. We provide:
|
||||||
|
|
||||||
- `script/bootstrap` — install all dependencies (the pinned node via nvm unless
|
- `script/bootstrap` — install all dependencies (the pinned node via nvm unless
|
||||||
one new enough for the frontend's dependencies is installed, yarn via
|
one new enough for the frontend's dependencies is installed, yarn via
|
||||||
corepack, `yarn install --frozen-lockfile`, Go `GO_VERSION` unless the `go` on
|
corepack, `yarn install --frozen-lockfile`, the pinned Go unless one at least
|
||||||
`PATH` is exactly that version, and the Go modules), linking what it installs
|
as new as `backend/go.mod` asks for is installed, the Go modules, and gcc with
|
||||||
itself into `~/.local/bin`, which has to be on `PATH`. It installs no Go
|
the C library headers unless gcc is installed, for the race detector in
|
||||||
linter and not Docker: `make test` and `make lint` run in Docker
|
`make test`), linking what it installs itself into `~/.local/bin`, which has
|
||||||
|
to be on `PATH`. It installs no Go linter and not Docker: `make lint` runs
|
||||||
|
both linters in Docker
|
||||||
- `script/setup` — make a fresh clone ready for development: bootstrap plus the
|
- `script/setup` — make a fresh clone ready for development: bootstrap plus the
|
||||||
git pre-commit hook
|
git pre-commit hook
|
||||||
- `script/dev` — run the Vite dev server, which proxies `/api` to a locally
|
- `script/dev` — run the Vite dev server, which proxies `/api` to a locally
|
||||||
running `netwatch-server`
|
running `netwatch-server`
|
||||||
- `script/build` — build the frontend for production into `dist/`, then the
|
- `script/build` — build the frontend for production into `dist/`;
|
||||||
static `netwatch-server` binary into the repo root with its version stamped
|
`backend/script/build` builds the Go server
|
||||||
in: `VERSION` from the environment, or when that is unset or empty,
|
|
||||||
`git describe` inside a git checkout, then `dev`
|
|
||||||
- `script/run` — build with `script/build`, then run `netwatch-server` with its
|
|
||||||
defaults (see [Settings](#settings))
|
|
||||||
- `script/clean` — remove what `script/build` writes
|
|
||||||
- `script/projectname` — print the project name (used for the Docker image tag)
|
- `script/projectname` — print the project name (used for the Docker image tag)
|
||||||
- `script/test` — build the `test` phase of `Dockerfile` without the cache: the
|
- `script/test` — run `script/frontend-test`, then `backend/script/test`, the
|
||||||
frontend's unit tests in its `frontend-test` stage, and the backend's Go tests
|
backend's Go tests with the race detector and coverage
|
||||||
with the race detector and coverage. The frontend's production build is left
|
- `script/lint` — run eslint, then golangci-lint, both in Docker, by building
|
||||||
to the image build, in the `frontend-build` stage
|
the `frontend-lint` and `lint` stages of `Dockerfile` without the cache
|
||||||
- `script/lint` — build the `lint` phase of `Dockerfile` without the cache:
|
- `script/fmt` — format all files (writes): prettier, then gofmt over `backend/`
|
||||||
eslint in its `frontend-lint` stage, and golangci-lint over the Go packages
|
|
||||||
- `script/fmt` — format all files (writes): prettier over the JavaScript, CSS,
|
|
||||||
HTML and Markdown, then gofmt over `cmd/` and `internal/`. It runs on the
|
|
||||||
host, as `script/fmt-check` does, with `~/.local/bin` put on `PATH`
|
|
||||||
- `script/fmt-check` — check formatting (read-only): prettier, then gofmt
|
- `script/fmt-check` — check formatting (read-only): prettier, then gofmt
|
||||||
- `script/check` — run test, lint, and fmt-check
|
- `script/check` — run test, lint, and fmt-check
|
||||||
- `script/add-dependency` — add a frontend package, or move one to another
|
- `script/add-dependency` — add a frontend package, or move one to another
|
||||||
version: `make add-dependency PACKAGE=<name>@<version>` runs `yarn add --dev`,
|
version: `make add-dependency PACKAGE=<name>@<version>` runs `yarn add --dev`,
|
||||||
which changes `package.json` and `yarn.lock` together, then
|
which changes `package.json` and `yarn.lock` together, then
|
||||||
`yarn install --frozen-lockfile`
|
`yarn install --frozen-lockfile`
|
||||||
- `script/tidy` — run `go mod tidy`: to add a Go module, import it and run
|
- `script/tidy` — run `go mod tidy` in `backend/`: to add a Go module, import it
|
||||||
`make tidy`; to move one to another version, edit its `require` line in
|
and run `make tidy`; to move one to another version, edit its `require` line
|
||||||
`go.mod`, then run `make tidy`
|
in `backend/go.mod`, then run `make tidy`
|
||||||
- `script/frontend-test` — run the unit tests in `test/unit/` on the host with
|
- `script/frontend-test` — run the unit tests in `test/unit/` with Node's
|
||||||
Node's built-in test runner, through the `test` script in `package.json`, and
|
built-in test runner, through the `test` script in `package.json`, and if any
|
||||||
if any fails, run them again listing every test, and fail; then the production
|
fails, run them again listing every test, and fail; then the production build.
|
||||||
build. Each test run has a 90-second timeout. `make test` runs the same tests
|
Each run has a 30-second timeout
|
||||||
in Docker, without the production build
|
- `script/frontend-lint` — run eslint with the rules in `eslint.config.js`; it
|
||||||
- `script/frontend-fmt` — format the JavaScript, CSS, HTML and Markdown with
|
runs inside the `frontend-lint` stage of `Dockerfile`, which `make lint`
|
||||||
prettier (writes)
|
builds
|
||||||
|
- `script/frontend-fmt` — format everything prettier understands (writes), the
|
||||||
|
markdown in `backend/` included
|
||||||
- `script/frontend-fmt-check` — check prettier formatting (read-only)
|
- `script/frontend-fmt-check` — check prettier formatting (read-only)
|
||||||
|
- `script/frontend-check` — run `script/frontend-test` and
|
||||||
|
`script/frontend-fmt-check`, for the frontend stage of `Dockerfile`, which has
|
||||||
|
neither Go nor Docker
|
||||||
- `script/frontend-viewport-test` — responsive-layout verification of the built
|
- `script/frontend-viewport-test` — responsive-layout verification of the built
|
||||||
frontend in a containerised headless Chrome (see
|
frontend in a containerised headless Chrome (see
|
||||||
[test/viewport/README.md](test/viewport/README.md)). Not part of
|
[test/viewport/README.md](test/viewport/README.md)). Not part of
|
||||||
`script/check`: it takes minutes.
|
`script/check`: it needs Docker and takes minutes.
|
||||||
- `script/docker` — build the image from `Dockerfile` without the build cache,
|
- `script/docker` — build the image from `Dockerfile` without the build cache,
|
||||||
tagged `netwatch` via `script/projectname`
|
tagged `netwatch` via `script/projectname`
|
||||||
- `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`,
|
- `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`,
|
||||||
@@ -98,30 +92,24 @@ either one is broken. We provide:
|
|||||||
## Responsive layout
|
## Responsive layout
|
||||||
|
|
||||||
The narrow-viewport layout lives in the `max-width: 768px` media block in
|
The narrow-viewport layout lives in the `max-width: 768px` media block in
|
||||||
`web/src/styles.css`. It is verified automatically by
|
`src/styles.css`. It is verified automatically by `make frontend-viewport-test`,
|
||||||
`make frontend-viewport-test`, which drives a digest-pinned headless Chrome
|
which drives a digest-pinned headless Chrome against the built `dist/` and
|
||||||
against the built `dist/` and asserts on computed layout at widths derived from
|
asserts on computed layout at widths derived from that CSS — one pixel either
|
||||||
that CSS — on every breakpoint it declares and one pixel either side of it, plus
|
side of every breakpoint it declares, plus a 320px floor, a desktop baseline and
|
||||||
a 320px floor, a desktop baseline and two landscape sizes. See
|
two landscape sizes. See [test/viewport/README.md](test/viewport/README.md) for
|
||||||
[test/viewport/README.md](test/viewport/README.md) for what it covers and what
|
what it covers and what it genuinely cannot.
|
||||||
it genuinely cannot.
|
|
||||||
|
|
||||||
## Rationale
|
## Rationale
|
||||||
|
|
||||||
When debugging network issues, it's useful to have a persistent at-a-glance view
|
When debugging network issues, it's useful to have a persistent at-a-glance view
|
||||||
of latency and reachability to multiple well-known internet endpoints. NetWatch
|
of latency and reachability to multiple well-known internet endpoints. NetWatch
|
||||||
provides this as a single page that does all its measuring in the browser, so it
|
provides this as a zero-dependency SPA that can be deployed anywhere static
|
||||||
can be served from anywhere static files are served. The backend in its Docker
|
files are served, with no backend required.
|
||||||
image only stores the measurements the page reports; without it, the page works
|
|
||||||
the same and nothing is stored.
|
|
||||||
|
|
||||||
## Design
|
## Design
|
||||||
|
|
||||||
The page's source is in `web/`, the Go backend's in `cmd/` and `internal/`, and
|
The application is a single-page app built with Vite and Tailwind CSS v4. All
|
||||||
nginx's configuration in `configs/`.
|
code lives in `src/main.js` with a class-based architecture:
|
||||||
|
|
||||||
The page is built with Vite and Tailwind CSS v4. Its code is all in
|
|
||||||
`web/src/main.js`, with a class-based architecture:
|
|
||||||
|
|
||||||
- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks,
|
- **`CONFIG`**: Configuration object (update interval, timeouts, axis ticks,
|
||||||
etc.). The interval menu sets `updateInterval`, the one value the page writes
|
etc.). The interval menu sets `updateInterval`, the one value the page writes
|
||||||
@@ -137,10 +125,10 @@ The page is built with Vite and Tailwind CSS v4. Its code is all in
|
|||||||
`updateSummary()` / `updateHealthBox()` handle incremental updates
|
`updateSummary()` / `updateHealthBox()` handle incremental updates
|
||||||
- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's
|
- **`tick()`**: Main loop — measures all hosts in parallel, pushing each host's
|
||||||
sample and redrawing its row as soon as its check ends, then redraws every
|
sample and redrawing its row as soon as its check ends, then redraws every
|
||||||
row, the summary and the health box once the last check ends. The first round,
|
row, the summary and the health box once the last check ends. The rows are
|
||||||
after loading or an interval change, is discarded. The rows are sorted when
|
sorted then too, after the first round that is not discarded and every tenth
|
||||||
the last check ends in round 2, the first one kept, and in rounds 11, 21, 31
|
round after that. When paused, pushes blank markers (no probes, no false
|
||||||
and so on. When paused, pushes blank markers (no probes, no false outage)
|
outage)
|
||||||
- **`Reporter`**: Posts collected samples to the backend
|
- **`Reporter`**: Posts collected samples to the backend
|
||||||
|
|
||||||
### Reporting
|
### Reporting
|
||||||
@@ -150,179 +138,16 @@ same-origin path `/api/v1/reports`: a random per-browser `clientId` kept in
|
|||||||
`localStorage`, `geo` sent as null, and each host's unreported, non-paused
|
`localStorage`, `geo` sent as null, and each host's unreported, non-paused
|
||||||
samples (timestamp, latency, error). A per-host high-water mark makes every
|
samples (timestamp, latency, error). A per-host high-water mark makes every
|
||||||
report a delta, so only new samples are sent; the mark advances only on a
|
report a delta, so only new samples are sent; the mark advances only on a
|
||||||
delivered report, and while paused nothing is sent. Changing the interval clears
|
delivered report, and while paused nothing is sent. Delivery failure is quiet —
|
||||||
each host's history on the page; the samples in it not yet reported still go in
|
one debug-log line per outage, retried at the next interval, never blocking
|
||||||
the next report. Delivery failure is quiet — one debug-log line per outage,
|
probing. The report-building step is a pure function of host state.
|
||||||
retried at the next interval, never blocking probing. The report-building step
|
|
||||||
is a pure function of host state.
|
|
||||||
|
|
||||||
### Backend
|
|
||||||
|
|
||||||
`netwatch-server` is a small Go HTTP server that stores the reports the page
|
|
||||||
posts. It keeps them in memory and writes them to `DATA_DIR` as zstd-compressed
|
|
||||||
files of JSON lines: every minute, whenever 10 MiB are waiting, and when it
|
|
||||||
stops. Its routes:
|
|
||||||
|
|
||||||
- `POST /api/v1/reports` — takes a report, without credentials; each client
|
|
||||||
address may send a limited number a minute, and the report files are capped in
|
|
||||||
size, the oldest deleted first
|
|
||||||
- `GET /.well-known/healthcheck` — answers 200 with `"status":"ok"`, the
|
|
||||||
server's version and its uptime
|
|
||||||
- `GET /metrics` — Prometheus metrics behind basic auth, only when
|
|
||||||
`METRICS_USERNAME` and `METRICS_PASSWORD` are set; each client address may
|
|
||||||
make a limited number of requests to it a minute
|
|
||||||
|
|
||||||
It is an `fx`-wired Go application: `cmd/netwatch-server/` starts it, and its
|
|
||||||
packages are in `internal/`:
|
|
||||||
|
|
||||||
- **`config`**: Loads configuration from environment variables and config files
|
|
||||||
via Viper.
|
|
||||||
- **`handlers`**: HTTP request handlers for the API (health check, report
|
|
||||||
ingestion).
|
|
||||||
- **`reportbuf`**: The in-memory buffer of reports and the files it writes them
|
|
||||||
to.
|
|
||||||
- **`server`**: Chi-based HTTP server with middleware wiring and route
|
|
||||||
registration.
|
|
||||||
- **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting
|
|
||||||
infrastructure.
|
|
||||||
|
|
||||||
In the image, the `test` phase of `Dockerfile` tests it, the `builder` stage
|
|
||||||
builds it, and `bin/entrypoint.sh` runs it as user `netwatch` on
|
|
||||||
`127.0.0.1:8081`, behind nginx (see [Deployment](#deployment)). Outside the
|
|
||||||
image, `make run` builds it and runs it on port 8080.
|
|
||||||
|
|
||||||
#### Settings
|
|
||||||
|
|
||||||
| Variable | Default | Description |
|
|
||||||
| ---------------------- | -------------------- | -------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface |
|
|
||||||
| `PORT` | `8080` | HTTP listen port |
|
|
||||||
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
|
|
||||||
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) |
|
|
||||||
| `DEBUG` | `false` | Enable debug logging |
|
|
||||||
| `TRUSTED_PROXIES` | RFC 1918 ranges | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
|
|
||||||
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
|
|
||||||
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
|
||||||
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
|
|
||||||
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
|
|
||||||
| `SENTRY_DSN` | empty | DSN of the Sentry project to send errors to; see [Sentry](#sentry) |
|
|
||||||
|
|
||||||
`TRUSTED_PROXIES` defaults to `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. A list
|
|
||||||
replaces the default; set and empty, the server trusts no proxy. Loopback is not
|
|
||||||
in the default, so a reverse proxy on the same host has to be named, as in
|
|
||||||
`TRUSTED_PROXIES=127.0.0.1/32`. A request whose direct peer is outside this set
|
|
||||||
has its forwarded headers ignored, and the direct peer is logged and
|
|
||||||
rate-limited instead. The container image does not use this default; see
|
|
||||||
[Deployment](#deployment).
|
|
||||||
|
|
||||||
A variable set to a value the server cannot use, such as `PORT=abc`,
|
|
||||||
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
|
|
||||||
starting, with an error naming the variable. An empty variable counts as unset,
|
|
||||||
except `TRUSTED_PROXIES`.
|
|
||||||
|
|
||||||
#### Report storage
|
|
||||||
|
|
||||||
Reports are written as `reports-<timestamp>-<number>.jsonl.zst` files in
|
|
||||||
`DATA_DIR`. The timestamp is in UTC to the millisecond, so the names sort by
|
|
||||||
time. The number starts at 1 when the server starts and goes up by one for each
|
|
||||||
file the server starts to write, so two files written in the same millisecond
|
|
||||||
still get different names. A failed write uses up its number and leaves a gap in
|
|
||||||
the numbers: its file, if it was created, is removed. The file stays, counted
|
|
||||||
toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too.
|
|
||||||
Files are created with `O_EXCL` to prevent overwrites.
|
|
||||||
|
|
||||||
The reports of a failed write stay in memory, and the next flush, within a
|
|
||||||
minute, tries again to write them. From the failed write until a write succeeds,
|
|
||||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
|
||||||
the page keeps its samples and sends them again at its next report. Reports
|
|
||||||
already answered 200 are lost only if the server stops before a write succeeds;
|
|
||||||
the final flush at stop tries them too, and if it fails, the process exits
|
|
||||||
non-zero.
|
|
||||||
|
|
||||||
#### Report limits
|
|
||||||
|
|
||||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
|
||||||
credentials, so it is bounded instead. Each refusal below answers with the same
|
|
||||||
`{"status":"error"}` body as any other error.
|
|
||||||
|
|
||||||
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
|
||||||
parameters such as `charset` allowed, as the page sends it; any other is
|
|
||||||
refused with 415, and nothing of it is stored. A page on another origin can
|
|
||||||
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
|
||||||
keeps any site from making its visitors' browsers store reports (see
|
|
||||||
[CORS](#cors)).
|
|
||||||
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
|
||||||
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
|
||||||
`Retry-After: 60`. The minute slides: reports from the minute before still
|
|
||||||
count, fading out over the current one, so an address is sure never to be
|
|
||||||
refused only while it sends at most half of `REPORTS_PER_MINUTE` in any 60
|
|
||||||
seconds. The page sends one report a minute from each open tab, so the default
|
|
||||||
of 60 refuses nothing from up to 30 tabs behind one address, such as a
|
|
||||||
household or an office sharing it, however their reports bunch up. Report
|
|
||||||
responses also carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and
|
|
||||||
`X-RateLimit-Reset` headers.
|
|
||||||
- **Size cap.** The report files in `DATA_DIR` may total at most
|
|
||||||
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
|
|
||||||
waiting in memory count at their uncompressed size until they are written,
|
|
||||||
those of a failed write included; the part of its file written is removed.
|
|
||||||
When a report would take the total past the cap, the oldest report files are
|
|
||||||
deleted to make room, and each deletion is logged with the file's name and
|
|
||||||
size; a file still being written is never deleted. A report is refused with
|
|
||||||
507, and nothing of it is stored, only when the reports waiting to be written
|
|
||||||
fill the cap on their own, and then no file is deleted. At start, report files
|
|
||||||
past the cap, as after lowering it, are deleted the same way. So the cap is
|
|
||||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
|
||||||
for any host; set it to the space you can give `DATA_DIR`.
|
|
||||||
|
|
||||||
#### CORS
|
|
||||||
|
|
||||||
The page calls the API from the origin it is served from, so by default the
|
|
||||||
server sends no CORS headers, and browsers let no other origin's pages call it.
|
|
||||||
To serve the page from elsewhere, list that origin in `CORS_ALLOWED_ORIGINS`
|
|
||||||
(for example `https://netwatch.example.com`); pages from a listed origin may
|
|
||||||
`GET` and `POST` with a `Content-Type` header. Each entry must be a plain
|
|
||||||
origin, `scheme://host` with an optional `:port`, as browsers send it: no path,
|
|
||||||
not even a trailing `/`, and no `*`. Any other entry stops the server from
|
|
||||||
starting, with an error naming `CORS_ALLOWED_ORIGINS`.
|
|
||||||
|
|
||||||
#### Metrics
|
|
||||||
|
|
||||||
With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves
|
|
||||||
Prometheus metrics at `GET /metrics` to requests with those as their basic auth
|
|
||||||
credentials, and answers any other with 401. For each request that reaches the
|
|
||||||
health check or `POST /api/v1/reports`, those the rate limit refuses included,
|
|
||||||
the metrics record its duration and response size, labelled with its path,
|
|
||||||
method and status; they also count those requests in progress, and include Go's
|
|
||||||
runtime and process metrics. No other request is recorded: not those to
|
|
||||||
`/metrics` itself, and not those answered before they reach either route, such
|
|
||||||
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
|
||||||
for a method its route does not take, or 413 for declaring a body length over
|
|
||||||
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
|
||||||
length reaches the route, is answered 413 there, and is recorded with that
|
|
||||||
status. Clients can make up any number of paths and methods, and each would add
|
|
||||||
labels to the metrics for as long as the server runs.
|
|
||||||
|
|
||||||
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
|
||||||
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
|
||||||
minute, whatever their credentials. Past that it gets 429 with
|
|
||||||
`Retry-After: 60`, and its credentials are not checked. The minute slides as it
|
|
||||||
does for reports (see [Report limits](#report-limits)), so a scraper polling
|
|
||||||
every 2 seconds or less often is never refused. This allowance is apart from the
|
|
||||||
one for reports.
|
|
||||||
|
|
||||||
#### Sentry
|
|
||||||
|
|
||||||
With `SENTRY_DSN` set, each panic in a handler is reported to that Sentry
|
|
||||||
project under the release `netwatch-server-` followed by the server's version.
|
|
||||||
On shutdown the server waits up to 2 seconds for Sentry to finish sending.
|
|
||||||
|
|
||||||
### Monitoring targets
|
### Monitoring targets
|
||||||
|
|
||||||
- **26 WAN hosts**: datavi.be (pinned at start), Anthropic API, OpenAI API, AWS
|
- **22 WAN hosts**: datavi.be, Anthropic API, OpenAI API, AWS Console, GCP
|
||||||
Console, Google Cloud Console, Microsoft Azure, Cloudflare, Fastly CDN,
|
Console, Azure, Cloudflare, Fastly, Akamai, GitHub, B2, 7 S3 regional
|
||||||
Akamai, Google, GitHub, B2, 8 S3 regional endpoints (Cape Town, London,
|
endpoints (Cape Town, London, Bahrain, Tokyo, Sydney, Oregon, São Paulo), 4
|
||||||
Bahrain, Tokyo, Singapore, Sydney, Oregon, São Paulo) and 6 Hetzner speed test
|
GCS locational endpoints (Iowa, Belgium, Singapore, Sydney)
|
||||||
servers (Nuremberg, Falkenstein, Helsinki, Ashburn, Hillsboro, Singapore)
|
|
||||||
- **Local CPE**: Cable modem at 192.168.100.1 (always monitored)
|
- **Local CPE**: Cable modem at 192.168.100.1 (always monitored)
|
||||||
- **Local Gateway**: Auto-detected on startup by probing common default gateway
|
- **Local Gateway**: Auto-detected on startup by probing common default gateway
|
||||||
addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder
|
addresses (192.168.1.1, 192.168.0.1, 192.168.8.1, 10.0.0.1); first responder
|
||||||
@@ -334,28 +159,15 @@ Local hosts are tracked separately from WAN stats.
|
|||||||
|
|
||||||
### Latency measurement
|
### Latency measurement
|
||||||
|
|
||||||
GET requests to each target's URL as written, with `mode: 'no-cors'` and
|
HEAD requests with `mode: 'no-cors'` and `cache: 'no-store'`, timed with
|
||||||
`cache: 'no-store'`, timed with `performance.now()`. No query string is added:
|
`performance.now()`. Each check times out after 80% of the refresh interval (24
|
||||||
the Hetzner speed-test servers close the connection without an answer when the
|
seconds at 30 seconds) and is then recorded as a timeout, so a round's checks
|
||||||
URL has one, and `no-store` keeps the browser's cache out of the measurement.
|
have all finished before the next round is due. When no WAN host answers, a
|
||||||
Each check times out after 80% of the refresh interval (24 seconds at 30
|
recovery probe checks 4 random WAN hosts every half second, giving up the checks
|
||||||
seconds) and is then recorded as a timeout, so a round's checks have all
|
it started half a second before. As soon as one answers, a new round starts at
|
||||||
finished before the next round is due. When no WAN host answers, a recovery
|
once, as it does after an interval change. A round started early gives up the
|
||||||
probe checks 4 WAN hosts, picked at random when it starts, every half second,
|
last round's checks if they are still waiting, and that round records nothing
|
||||||
giving up the checks it started half a second before. As soon as one answers, a
|
more, so rounds never overlap. IPv4 only.
|
||||||
new round starts at once, as it does after an interval change. A round started
|
|
||||||
early gives up the last round's checks if they are still waiting, and that round
|
|
||||||
records nothing more, so rounds never overlap. The browser chooses between IPv4
|
|
||||||
and IPv6 for each target, as for any request; the local targets are IPv4
|
|
||||||
addresses.
|
|
||||||
|
|
||||||
Each recorded check that fails writes one line to the browser console with
|
|
||||||
`console.error`, and the same line to the debug log: the target's name and URL,
|
|
||||||
the time, whether it timed out, answered over the time limit or hit a network
|
|
||||||
error (with the error the browser gives the page, such as
|
|
||||||
`TypeError: Failed to fetch`, which does not say why), and how long the request
|
|
||||||
took. A target that answers after failed checks writes one `console.info` line
|
|
||||||
with its latency and how many checks in a row had failed.
|
|
||||||
|
|
||||||
### Color coding
|
### Color coding
|
||||||
|
|
||||||
@@ -380,62 +192,39 @@ dist/
|
|||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- A round of checks every 3 seconds by default; the interval menu sets 1, 2, 3,
|
- Real-time monitoring with 2s update interval and 300s history sparklines
|
||||||
5, 10, 15, 30 or 60 seconds and clears the history
|
- Health indicator: green (HEALTHY) or red (DEGRADED) based on WAN reachability
|
||||||
- Sparklines of each target's last 100 rounds: 300 seconds at 3 seconds
|
- Summary stats: reachable count, min/max/avg latency across WAN hosts only
|
||||||
- The first round after loading or an interval change is discarded, as DNS and
|
- Fixed chart axes: Y-axis 0–1000ms, X-axis 0–300s
|
||||||
TLS setup inflate its latencies
|
|
||||||
- Health indicator from the WAN hosts' latest results: OFFLINE (red) when more
|
|
||||||
than 10 fail and at most 4 answer, otherwise DEGRADED (orange) when more than
|
|
||||||
4 fail, otherwise SLOW (yellow) when more than 3 take over 1000ms, otherwise
|
|
||||||
HEALTHY (green)
|
|
||||||
- Summary stats across WAN hosts only: how many answered, the min, median,
|
|
||||||
average and max of their latest latencies, the min and max over the whole
|
|
||||||
history, and the number of rounds run (`Checks`)
|
|
||||||
- Fixed chart axes: Y-axis 0–1000ms, higher latencies drawn at the top; X-axis
|
|
||||||
the time the history spans
|
|
||||||
- Color-coded latency figures and sparkline line segments
|
- Color-coded latency figures and sparkline line segments
|
||||||
- WAN host rows sorted by latest latency, unreachable last; pinned rows stay on
|
|
||||||
top, in name order
|
|
||||||
- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no
|
- Play/pause: pause stops probes but history keeps scrolling (blank gaps, no
|
||||||
false outage)
|
false outage)
|
||||||
- Debug log panel, behind a checkbox in the footer, with five levels (error,
|
|
||||||
warning, notice, info, debug) and the last 1000 lines
|
|
||||||
- Local and UTC clocks
|
|
||||||
- Clickable service URLs
|
- Clickable service URLs
|
||||||
- A footer link to the commit the page was built from
|
|
||||||
- Canvas-based sparkline rendering with devicePixelRatio scaling
|
- Canvas-based sparkline rendering with devicePixelRatio scaling
|
||||||
- Zero runtime dependencies: all resources bundled into build artifacts
|
- Zero runtime dependencies: all resources bundled into build artifacts
|
||||||
|
|
||||||
## Deployment
|
## Deployment
|
||||||
|
|
||||||
`make build` writes the page to `dist/`, which any static file host (S3, GCS,
|
After running `yarn build`, deploy the contents of the `dist/` directory to any
|
||||||
Cloudflare Pages, Vercel, Netlify, GitHub Pages) can serve; with no backend
|
static file host (S3, GCS, Cloudflare Pages, Vercel, Netlify, GitHub Pages) or
|
||||||
there, its reports fail quietly and nothing is stored. Or run the Docker image
|
use the Docker image behind a reverse proxy.
|
||||||
behind a reverse proxy.
|
|
||||||
|
|
||||||
The Docker image, built from `Dockerfile` by `make docker`, is the whole service
|
The Docker image, built from `Dockerfile`, is the whole service in one
|
||||||
in one container: nginx serves the built frontend and passes `/api/`,
|
container: nginx serves the built frontend and passes `/api/`,
|
||||||
`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`.
|
`/.well-known/healthcheck` and `/metrics` to the Go backend, `netwatch-server`,
|
||||||
`bin/entrypoint.sh` starts it listening only inside the container, on
|
which listens only inside the container, on `127.0.0.1:8081`. The image:
|
||||||
`127.0.0.1:8081`, with `TRUSTED_PROXIES=127.0.0.1/32`, so it takes the client
|
|
||||||
address nginx passes on and no other. The image:
|
|
||||||
|
|
||||||
- Listens on port 8080 by default (override with `PORT` env var)
|
- Listens on port 8080 by default (override with `PORT` env var)
|
||||||
- Takes the client address from `X-Forwarded-For` only on requests from the
|
- Takes the client address from `X-Forwarded-For` only on requests from the
|
||||||
reverse proxies named in `TRUSTED_PROXIES`, by default from any address in
|
reverse proxies named in `TRUSTED_PROXIES`, and by default from none
|
||||||
`10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`. `bin/entrypoint.sh` checks
|
|
||||||
each entry with `netwatch-server check-cidr`, which parses it as the backend
|
|
||||||
parses its own `TRUSTED_PROXIES`
|
|
||||||
- Sends access logs to stdout
|
- Sends access logs to stdout
|
||||||
- Caches static assets with immutable headers
|
- Caches static assets with immutable headers
|
||||||
- Sends the security headers `REPO_POLICIES.md` requires on every response, as
|
- Sends the security headers `REPO_POLICIES.md` requires on every response, as
|
||||||
`configs/security-headers.conf` sets them, in place of the backend's own
|
`security-headers.conf` sets them, in place of the backend's own
|
||||||
- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
|
- Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
|
||||||
volume. Before the backend starts, the image creates `DATA_DIR` and gives it
|
volume. Before the backend starts, the image creates `DATA_DIR` and gives it
|
||||||
and `/data` to user `netwatch` (uid 1000), which the backend runs as, with
|
and `/data` to user `netwatch` (uid 1000), which the backend runs as, so a
|
||||||
`netwatch-server prepare-data-dir`, which acts on nothing outside `/data`, so
|
host directory bind-mounted at `/data` ends up owned by uid 1000
|
||||||
a host directory bind-mounted at `/data` ends up owned by uid 1000
|
|
||||||
- Writes buffered reports to disk on `docker stop`, and exits non-zero if nginx
|
- Writes buffered reports to disk on `docker stop`, and exits non-zero if nginx
|
||||||
or the backend exits on its own, so the platform restarts it
|
or the backend exits on its own, so the platform restarts it
|
||||||
|
|
||||||
@@ -445,9 +234,9 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
|||||||
|
|
||||||
- **Port:** container port `8080`.
|
- **Port:** container port `8080`.
|
||||||
- **Volume:** container path `/data`; the reports are kept in `/data/reports`.
|
- **Volume:** container path `/data`; the reports are kept in `/data/reports`.
|
||||||
- **Environment variables:** none is required. An empty one counts as unset,
|
- **Environment variables:** none is required. An empty one counts as unset, and
|
||||||
except `TRUSTED_PROXIES`, and one set to a value netwatch cannot use stops the
|
one set to a value netwatch cannot use stops the container at start, with the
|
||||||
container at start, with the reason in its log.
|
reason in its log.
|
||||||
- `PORT`, default `8080`: the container port, from 1 to 65535. `8081` cannot
|
- `PORT`, default `8080`: the container port, from 1 to 65535. `8081` cannot
|
||||||
be used: the backend listens on it inside the container
|
be used: the backend listens on it inside the container
|
||||||
- `REPORTS_PER_MINUTE`, default `60`: reports each client address may send a
|
- `REPORTS_PER_MINUTE`, default `60`: reports each client address may send a
|
||||||
@@ -461,27 +250,22 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
|
|||||||
in: `/data` or a path below it, with no `.` or `..` part and no extra `/`.
|
in: `/data` or a path below it, with no `.` or `..` part and no extra `/`.
|
||||||
The container also stops if the path goes through a symbolic link that
|
The container also stops if the path goes through a symbolic link that
|
||||||
leads out of `/data` or is written as a full path
|
leads out of `/data` or is written as a full path
|
||||||
- `TRUSTED_PROXIES`, default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`: the
|
- `TRUSTED_PROXIES`, default empty: set it to the address the reverse proxy
|
||||||
addresses the reverse proxy in front of the container connects from, as IP
|
in front of the container connects from, as an IP address or CIDR; several
|
||||||
addresses or CIDRs separated by commas. nginx takes the client address
|
are separated by commas. nginx takes the client address from
|
||||||
from `X-Forwarded-For` only on a request from one of them, and the rate
|
`X-Forwarded-For` only on a request from one of them, and the rate limit
|
||||||
limit counts that address. A list replaces the default. Set and empty,
|
counts that address. Unset, `X-Forwarded-For` is ignored and every client
|
||||||
`X-Forwarded-For` is ignored and every client behind the proxy shares the
|
behind the proxy shares the proxy's one allowance of `REPORTS_PER_MINUTE`.
|
||||||
proxy's one allowance of `REPORTS_PER_MINUTE`. Any client that connects
|
Name only addresses nothing but the proxy connects from: any client that
|
||||||
from one of these addresses can write its own `X-Forwarded-For`, and
|
connects from one can write its own `X-Forwarded-For`, and through a port
|
||||||
through a port Docker publishes, every client may connect from the Docker
|
Docker publishes, every client may connect from the Docker network's
|
||||||
network's gateway, such as `172.17.0.1`, which the default includes; where
|
gateway, such as `172.17.0.1`.
|
||||||
that matters, name only the addresses nothing but the proxy connects from.
|
|
||||||
- `METRICS_USERNAME` and `METRICS_PASSWORD`, default empty: with both set,
|
- `METRICS_USERNAME` and `METRICS_PASSWORD`, default empty: with both set,
|
||||||
the backend records Prometheus metrics of its requests and serves them at
|
the backend records Prometheus metrics of its requests and serves them at
|
||||||
`/metrics` on the container port, to requests with this user name and
|
`/metrics` on the container port, to requests with this user name and
|
||||||
password as their basic auth credentials. With neither set, there are no
|
password as their basic auth credentials. With neither set, there are no
|
||||||
metrics and `/metrics` is not found. One set without the other, or a user
|
metrics and `/metrics` is not found. One set without the other, or a user
|
||||||
name containing `:`, stops the container
|
name containing `:`, stops the container
|
||||||
- `SENTRY_DSN`, default empty: set to a Sentry project's DSN, the backend
|
|
||||||
sends its errors to that Sentry project: each request whose handling
|
|
||||||
crashes, which still gets a 500 response. A value Sentry does not accept
|
|
||||||
stops the container. Empty, the backend sends nothing to Sentry
|
|
||||||
- **Health check:** the image's `HEALTHCHECK` requests
|
- **Health check:** the image's `HEALTHCHECK` requests
|
||||||
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
|
`/.well-known/healthcheck` through nginx every 30 seconds, so it fails unless
|
||||||
both nginx and the backend answer. upaas reads the container's health 60
|
both nginx and the backend answer. upaas reads the container's health 60
|
||||||
@@ -495,19 +279,19 @@ properties.
|
|||||||
|
|
||||||
## Limitations
|
## Limitations
|
||||||
|
|
||||||
- **CORS**: The checks are cross-origin requests in `no-cors` mode, so the page
|
- **CORS**: Some hosts may block cross-origin HEAD requests. The app uses
|
||||||
cannot read the answer, only time it: any answer counts as reachable, an error
|
`no-cors` mode which allows the request but provides opaque responses. Latency
|
||||||
page included.
|
is still measurable based on request timing.
|
||||||
- **Local targets**: The cable modem at 192.168.100.1 and the detected gateway
|
- **Local gateway**: The 192.168.100.1 endpoint requires the host to be
|
||||||
answer only on a network that has them, and only when NetWatch is served from
|
accessible from your network.
|
||||||
localhost or a private address (see Monitoring targets).
|
|
||||||
- **Network conditions**: Measurements reflect browser-to-endpoint latency,
|
- **Network conditions**: Measurements reflect browser-to-endpoint latency,
|
||||||
which includes your local network, ISP, and internet routing.
|
which includes your local network, ISP, and internet routing.
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
The to-do list is [TODO.md](TODO.md): where the work stands, the next step, the
|
- Add configurable host list (environment variable or config file)
|
||||||
open work, and what has been done.
|
- Add latency history export (CSV/JSON)
|
||||||
|
- Add notification/alert when status changes to DEGRADED
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
+82
-353
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
title: Repository Policies
|
title: Repository Policies
|
||||||
last_modified: 2026-10-04
|
last_modified: 2026-07-06
|
||||||
---
|
---
|
||||||
|
|
||||||
This document covers repository structure, tooling, and workflow standards. Code
|
This document covers repository structure, tooling, and workflow standards. Code
|
||||||
@@ -60,28 +60,17 @@ style conventions are in separate documents:
|
|||||||
prerequisite since nvm requires bash. yarn is then pinned via
|
prerequisite since nvm requires bash. yarn is then pinned via
|
||||||
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
`corepack prepare yarn@<version> --activate`. Never install "latest" or "lts";
|
||||||
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
always exact versions. `script/cibuild` runs the CI build: it changes to the
|
||||||
repo root, runs `script/bootstrap`, runs `script/check`, and builds the image
|
repo root and runs `docker build .`; the Gitea workflow calls it. Four further
|
||||||
with the version; the Gitea workflow calls it. **`script/cibuild` runs
|
scripts are our own extensions to the standard: `script/check` runs
|
||||||
`script/bootstrap` first**, because the workflow checks out the repo and runs
|
`script/test`, `script/lint`, and `script/fmt-check`; `script/precommit` is
|
||||||
nothing else, while `script/fmt-check` runs the formatter on the host: on a
|
what the git pre-commit hook runs, and it calls `script/check`;
|
||||||
pristine checkout with nothing installed the run dies there, after the
|
`script/install-precommit` installs the git pre-commit hook (the `make hooks`
|
||||||
containerised gates have passed. **The bootstrap alone is not enough**:
|
target shims to it); and `script/projectname` (literally that filename) simply
|
||||||
`script/bootstrap` installs node and yarn under nvm and leaves neither on the
|
outputs the project's name. Scripts that need the name call
|
||||||
`PATH` of the shell that called it, so a bare `yarn` still exits 127. The host
|
`script/projectname` — e.g. `script/docker` assembles its image tag from it —
|
||||||
entrypoints that need yarn — `script/fmt` and `script/fmt-check` — therefore
|
so those scripts stay byte-identical across all repos. Repo-type-specific
|
||||||
source nvm for the pinned node version before invoking it, exactly as
|
pre-commit extras (e.g. `go mod tidy` verification in Go repos) belong in
|
||||||
`script/bootstrap`'s own install step does. A runner carrying nothing but
|
`script/precommit`, not in the hook itself. Model scripts are at
|
||||||
docker and git then gets through `script/check`. Four further scripts are our
|
|
||||||
own extensions to the standard: `script/check` runs `script/test`,
|
|
||||||
`script/lint` and `script/fmt-check`; `script/precommit` is what the git
|
|
||||||
pre-commit hook runs, and it calls `script/check`; `script/install-precommit`
|
|
||||||
installs the git pre-commit hook (the `make hooks` target shims to it); and
|
|
||||||
`script/projectname` (literally that filename) simply outputs the project's
|
|
||||||
name. Scripts that need the name call `script/projectname` — e.g.
|
|
||||||
`script/docker` assembles its image tag from it — so those scripts stay
|
|
||||||
byte-identical across all repos. Repo-type-specific pre-commit extras (e.g.
|
|
||||||
`go mod tidy` verification in Go repos) belong in `script/precommit`, not in
|
|
||||||
the hook itself. Model scripts are at
|
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/script/<name>`. The README
|
||||||
must document the provided scripts in an **Entrypoints** section (see the
|
must document the provided scripts in an **Entrypoints** section (see the
|
||||||
README requirements below).
|
README requirements below).
|
||||||
@@ -100,198 +89,87 @@ style conventions are in separate documents:
|
|||||||
contributor should be able to understand the entire development workflow by
|
contributor should be able to understand the entire development workflow by
|
||||||
reading the Makefile.
|
reading the Makefile.
|
||||||
|
|
||||||
- Every repo should have a `Dockerfile`, and it carries the repo's gates: a
|
- Every repo should have a `Dockerfile`. All Dockerfiles must run `make check`
|
||||||
`lint` phase and a `test` phase, with the final stage depending on both so the
|
as a build step so the build fails if the branch is not green. For non-server
|
||||||
image cannot be built unless they pass. For non-server repos the final stage
|
repos, the Dockerfile should bring up a development environment and run
|
||||||
brings up a development environment; for server repos it is the runtime image.
|
`make check`. For server repos, `make check` should run as an early build
|
||||||
The gate phases and the build stage start from their pinned base images and
|
stage before the final image is assembled. Dockerfiles install development
|
||||||
install what those images lack either inline, as the canonical Go `Dockerfile`
|
prerequisites by running `script/bootstrap` rather than duplicating installs
|
||||||
below does for `git`, or by running `script/bootstrap`, as the `prompts`
|
inline; COPY `script/` and the dependency manifests (`package.json` +
|
||||||
repo's own `Dockerfile` does for its yarn packages. The development
|
`yarn.lock`, `go.mod` + `go.sum`, etc.) before running it so the bootstrap
|
||||||
environment stage installs development prerequisites by running
|
layer stays cached until dependencies change.
|
||||||
`script/bootstrap` rather than duplicating its installs inline. A stage that
|
|
||||||
runs `script/bootstrap` COPYs `script/` and the dependency manifests
|
|
||||||
(`package.json` + `yarn.lock`, `go.mod` + `go.sum`, etc.) before running it.
|
|
||||||
|
|
||||||
- **Linting and testing run in Docker, as phases of the `Dockerfile`.** There is
|
- **Dockerfiles must use a separate lint stage for fail-fast feedback.** Go
|
||||||
no separate lint file. `script/lint` and `script/test` each build one phase
|
repos use a multistage build where linting runs in an independent stage based
|
||||||
and nothing else:
|
on the `golangci/golangci-lint` image (pinned by hash). This stage runs
|
||||||
|
`make fmt-check` and `make lint` before the full build begins. The build stage
|
||||||
|
then declares an explicit dependency on the lint stage via
|
||||||
|
`COPY --from=lint /src/go.sum /dev/null`, which forces BuildKit to complete
|
||||||
|
linting before proceeding to compilation and tests. This ensures lint failures
|
||||||
|
surface in seconds rather than minutes, without blocking on dependency
|
||||||
|
download or compilation in the build stage.
|
||||||
|
|
||||||
```sh
|
The standard pattern for a Go repo Dockerfile is:
|
||||||
docker build --no-cache --target lint -t "$(script/projectname)-lint" .
|
|
||||||
docker build --no-cache --target test -t "$(script/projectname)-test" .
|
|
||||||
```
|
|
||||||
|
|
||||||
**A stage that is not the last one in the file is built only when the final
|
|
||||||
stage's chain depends on it, or when `--target` names it.** That is why the
|
|
||||||
two gates are always invoked by name here, and why the final stage carries a
|
|
||||||
`COPY --from=` of a harmless file from each of them: without that edge a
|
|
||||||
plain `docker build .` builds the last stage alone and exits 0 having linted
|
|
||||||
and tested nothing.
|
|
||||||
|
|
||||||
**Every `docker build` in `script/` is tagged**, here and in
|
|
||||||
`script/cibuild` and `script/docker`. An untagged build leaves a dangling
|
|
||||||
image behind on every invocation, on every developer host and every CI
|
|
||||||
runner; a tagged one replaces the previous image.
|
|
||||||
|
|
||||||
Inside a phase the tool is invoked directly — `golangci-lint`, `go test`,
|
|
||||||
`eslint`, `prettier` — never through `make lint` or `script/test`, which are
|
|
||||||
themselves a `docker build` and would recurse into a daemon that does not
|
|
||||||
exist in a build step. Formatting is the exception and stays on the host:
|
|
||||||
`script/fmt` writes the working tree, and `script/fmt-check` is its
|
|
||||||
read-only twin.
|
|
||||||
|
|
||||||
**No lint verdict may come from a host invocation of the linter.** On a
|
|
||||||
shared host golangci-lint reads a result cache keyed on file content rather
|
|
||||||
than location, so a second checkout of the same content is served the first
|
|
||||||
one's findings, and a host-global lock in `$TMPDIR` makes concurrent runs
|
|
||||||
exit non-zero with `parallel golangci-lint is running` — a status a caller
|
|
||||||
cannot tell from real findings. Both have produced wrong verdicts in this
|
|
||||||
org, in both directions. A container has its own cache, its own `TMPDIR` and
|
|
||||||
a digest-pinned binary, so neither is reachable.
|
|
||||||
|
|
||||||
- **Any build that runs checks is built with `--no-cache`.** Docker invalidates
|
|
||||||
a `COPY` layer only when the copied content changes, so on an unchanged tree
|
|
||||||
the check `RUN` is served from cache, nothing executes, and the build still
|
|
||||||
exits 0. Every `docker build` in `script/` therefore passes `--no-cache`:
|
|
||||||
`script/lint`, `script/test`, `script/cibuild` and `script/docker` are the
|
|
||||||
four, and there is no fifth — `script/check` runs the two gate phases and
|
|
||||||
`script/fmt-check`, and builds no image of its own. A bare `docker build .` is
|
|
||||||
not evidence that anything ran: a sub-second build reporting success is a
|
|
||||||
cache hit, not a result. Never invalidate by pruning — `docker builder prune`
|
|
||||||
and friends destroy a build cache shared with every other build on the host.
|
|
||||||
When a check is added or changed, prove it works by planting a defect it must
|
|
||||||
catch and watching the run fail on it, then revert the defect. A green run
|
|
||||||
alone shows neither that the check ran nor that it covers what it should.
|
|
||||||
|
|
||||||
- **The gate phases are separate stages, and the build stage depends on both.**
|
|
||||||
The lint phase is based on the `golangci/golangci-lint` image (pinned by
|
|
||||||
hash), so lint failures surface in seconds rather than after a full compile,
|
|
||||||
and the test phase is based on the Debian Go image. The canonical Go repo
|
|
||||||
`Dockerfile`:
|
|
||||||
|
|
||||||
```dockerfile
|
```dockerfile
|
||||||
# Lint phase
|
# Lint stage — fast feedback on formatting and lint issues
|
||||||
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
# golangci/golangci-lint:v2.x.x, YYYY-MM-DD
|
||||||
FROM golangci/golangci-lint@sha256:... AS lint
|
FROM golangci/golangci-lint@sha256:... AS lint
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
RUN golangci-lint run --config .golangci.yml ./...
|
RUN make fmt-check
|
||||||
|
RUN make lint
|
||||||
|
|
||||||
# Test phase. -race needs cgo and so a C compiler, which the Debian Go
|
# Build stage
|
||||||
# image ships and the alpine one does not.
|
|
||||||
# golang:1.x, YYYY-MM-DD
|
|
||||||
FROM golang@sha256:... AS test
|
|
||||||
WORKDIR /src
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
RUN go mod download
|
|
||||||
COPY . .
|
|
||||||
RUN go test -timeout 90s -race -cover ./... || \
|
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
|
||||||
go test -timeout 90s -race -v ./...; exit 1; }
|
|
||||||
|
|
||||||
# Build stage. Nothing is wanted from either phase above; the copies
|
|
||||||
# are what make BuildKit build them first, so this stage cannot run
|
|
||||||
# unless lint and test passed.
|
|
||||||
# golang:1.x-alpine, YYYY-MM-DD
|
# golang:1.x-alpine, YYYY-MM-DD
|
||||||
FROM golang@sha256:... AS builder
|
FROM golang@sha256:... AS builder
|
||||||
COPY --from=lint /src/go.sum /dev/null
|
|
||||||
COPY --from=test /src/go.sum /dev/null
|
|
||||||
RUN apk add --no-cache git
|
|
||||||
# A tar-stream context keeps the sender's file owners, which git refuses.
|
|
||||||
RUN git config --system --add safe.directory /src
|
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
|
|
||||||
|
# Force BuildKit to run the lint stage before proceeding
|
||||||
|
COPY --from=lint /src/go.sum /dev/null
|
||||||
|
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN go mod download
|
||||||
COPY . .
|
COPY . .
|
||||||
|
RUN make test
|
||||||
|
|
||||||
# The VERSION build arg when one is given, otherwise
|
ARG VERSION=dev
|
||||||
# `git describe --tags --always` on the .git in the build context. With
|
RUN CGO_ENABLED=0 go build -trimpath \
|
||||||
# .git present, a version that is still empty, dev or unknown fails the
|
|
||||||
# build: git is missing or could not read the checkout.
|
|
||||||
ARG VERSION
|
|
||||||
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
|
|
||||||
if [ -e .git ]; then \
|
|
||||||
case "$VERSION" in ""|dev|unknown) \
|
|
||||||
echo "version is '$VERSION' although .git is present" >&2; \
|
|
||||||
exit 1 ;; \
|
|
||||||
esac; \
|
|
||||||
fi; \
|
|
||||||
CGO_ENABLED=0 go build -trimpath \
|
|
||||||
-ldflags="-s -w -X main.Version=${VERSION}" \
|
-ldflags="-s -w -X main.Version=${VERSION}" \
|
||||||
-o /app ./cmd/app/
|
-o /app ./cmd/app/
|
||||||
|
|
||||||
# Runtime stage, and the last one
|
# Runtime stage
|
||||||
FROM alpine@sha256:...
|
FROM alpine@sha256:...
|
||||||
COPY --from=builder /app /usr/local/bin/app
|
COPY --from=builder /app /usr/local/bin/app
|
||||||
ENTRYPOINT ["app"]
|
ENTRYPOINT ["app"]
|
||||||
```
|
```
|
||||||
|
|
||||||
Key points:
|
Key points:
|
||||||
- The lint phase uses the `golangci/golangci-lint` image directly (it has
|
- The lint stage uses the `golangci/golangci-lint` image directly (it
|
||||||
both Go and the linter), so nothing needs installing.
|
includes both Go and the linter), so there is no need to install the
|
||||||
- `COPY --from=<phase> /src/go.sum /dev/null` is a no-op copy whose only
|
linter separately.
|
||||||
purpose is the ordering edge. BuildKit runs stages in parallel by default,
|
- `COPY --from=lint /src/go.sum /dev/null` is a no-op file copy that creates
|
||||||
and a stage nothing depends on is not built at all, so without these two
|
a stage dependency. BuildKit runs stages in parallel by default; without
|
||||||
lines a red gate would not fail the build.
|
this line, the build stage would not wait for lint to finish and a lint
|
||||||
- Keep the runtime stage last, and if you add a stage after it, give it the
|
failure might not fail the overall build.
|
||||||
same two copies. A plain `docker build .` builds the last stage's chain
|
|
||||||
and nothing else.
|
|
||||||
- If the project uses `//go:embed` directives that reference build artifacts
|
- If the project uses `//go:embed` directives that reference build artifacts
|
||||||
(e.g. a web frontend compiled in a separate stage), the lint phase must
|
(e.g. a web frontend compiled in a separate stage), the lint stage must
|
||||||
create placeholder files so the embed directives resolve. Example:
|
create placeholder files so the embed directives resolve. Example:
|
||||||
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
`RUN mkdir -p web/dist && touch web/dist/index.html web/dist/style.css`.
|
||||||
- If the project requires CGO or system libraries for linting, install them
|
The lint stage should not depend on the actual build output — it exists to
|
||||||
in the lint phase. The `golangci/golangci-lint` image is Debian-based and
|
fail fast.
|
||||||
has no `apk`, so install with `apt-get` under the Debian package name
|
- If the project requires CGO or system libraries for linting (e.g.
|
||||||
(`libvips-dev`, where alpine says `vips-dev`), and delete the package
|
`vips-dev`), install them in the lint stage with `apk add`.
|
||||||
lists in the same `RUN`, so the layer does not keep them:
|
- The build stage runs `make test` after compilation setup. Tests run in the
|
||||||
|
build stage, not the lint stage, because they may require compiled
|
||||||
```dockerfile
|
artifacts or heavier dependencies.
|
||||||
RUN apt-get update \
|
|
||||||
&& apt-get install -y --no-install-recommends libvips-dev \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
```
|
|
||||||
|
|
||||||
- `.dockerignore` lets `.git` into the build context. It keeps out every git
|
|
||||||
`config` at any depth (`**/.git/config`, `**/.git/modules/**/config`): the
|
|
||||||
repository's own, each submodule's under `.git/modules/`, and that of a
|
|
||||||
submodule keeping its own `.git` directory. `git describe` does not need
|
|
||||||
them, and each can hold a credential: a password in a remote URL, or the
|
|
||||||
token the CI checkout step stores there. A submodule whose name has a
|
|
||||||
`config` segment (`config`, `deploy/config`, `config/lib`) loses its whole
|
|
||||||
git directory to `**/.git/modules/**/config`, and Go's version stamping
|
|
||||||
then fails the build: give it a name without that segment
|
|
||||||
(`git submodule add --name`). The stage that compiles has `git` (the
|
|
||||||
Debian Go image has it; an alpine one needs `apk add --no-cache git`) and
|
|
||||||
takes the version from the `VERSION` build argument when one is given,
|
|
||||||
otherwise from `git describe --tags --always`. That gives the tag on a
|
|
||||||
tagged commit; on a later commit, the tag, the number of commits since it
|
|
||||||
and the short commit (`v1.2.3-4-gabc1234`); and the short commit when no
|
|
||||||
tag is reachable. The stage that compiles also marks its working directory
|
|
||||||
safe for git (`git config --system --add safe.directory /src`): a context
|
|
||||||
sent as a tar stream keeps the sender's file owners, and git refuses a
|
|
||||||
checkout owned by another user, so the version would come out empty.
|
|
||||||
`ARG VERSION` has no default, and the build fails if the context carries
|
|
||||||
`.git` and the version still comes out empty, `dev` or `unknown`. A plain
|
|
||||||
`docker build .` with no build arguments must succeed; a Dockerfile that
|
|
||||||
refuses an empty build argument drops that refusal and keeps the argument.
|
|
||||||
|
|
||||||
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
- Every repo should have a Gitea Actions workflow (`.gitea/workflows/`) that
|
||||||
runs `script/cibuild` on push, and checks out the repo as its only other step.
|
runs `script/cibuild` (which runs `docker build .`) on push. Since the
|
||||||
That script bootstraps, runs the gate phases, and then builds the image, so a
|
Dockerfile already runs `make check`, a successful build implies all checks
|
||||||
successful run means every check passed; a bare `docker build .` does not
|
pass.
|
||||||
carry the same guarantee, because its gate phases may come from the cache. The
|
|
||||||
image build is uncached and so runs the gate phases a second time. That is the
|
|
||||||
price of the rule above, and it is worth paying: the image that ships is built
|
|
||||||
from a run of its own gates rather than from a cache entry. A separate
|
|
||||||
workflow limited to `main` by a `branches` list under `on: push` cannot be
|
|
||||||
checked by review: to try a change to it, add the feature branch to that list
|
|
||||||
and push, then remove the branch from the list again before merging. Keep any
|
|
||||||
job in it that publishes behind `if: github.ref_name == 'main'`, so the run
|
|
||||||
from the feature branch publishes nothing.
|
|
||||||
|
|
||||||
- Use platform-standard formatters: `black` for Python, `prettier` for
|
- Use platform-standard formatters: `black` for Python, `prettier` for
|
||||||
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
JS/CSS/Markdown/HTML, `go fmt` for Go. Always use default configuration with
|
||||||
@@ -311,21 +189,14 @@ style conventions are in separate documents:
|
|||||||
module under test to verify it compiles/parses. There is no excuse for
|
module under test to verify it compiles/parses. There is no excuse for
|
||||||
`make test` to be a no-op.
|
`make test` to be a no-op.
|
||||||
|
|
||||||
- `make test` must complete in under 60 seconds. That is the hard cap, and a
|
- `make test` must complete in under 20 seconds. Add a 30-second timeout in the
|
||||||
suite that exceeds it fails. Under 20 seconds is the target. A suite between
|
Makefile.
|
||||||
20 and 60 seconds is still green, but the overage must be filed as an
|
|
||||||
improvement bug against that repo. Add a 90-second timeout to the test
|
|
||||||
invocation (`go test -timeout 90s`). The backstop deliberately sits above the
|
|
||||||
hard cap so that it catches a genuinely hung test rather than a merely slow
|
|
||||||
one.
|
|
||||||
|
|
||||||
- **The test command should use the conditional verbose rerun pattern.** Run
|
- **`make test` should use the conditional verbose rerun pattern.** Run tests
|
||||||
tests without `-v` (verbose) first. If tests fail, automatically rerun with
|
without `-v` (verbose) first. If tests fail, automatically rerun with `-v` to
|
||||||
`-v` to show full output. This keeps CI logs and `docker build` output clean
|
show full output. This keeps CI logs and `docker build` output clean on
|
||||||
on success (just package/suite summaries) while providing full diagnostic
|
success (just package/suite summaries) while providing full diagnostic detail
|
||||||
detail on failure (every test case, every assertion). The command lives in the
|
on failure (every test case, every assertion). The general shell pattern:
|
||||||
`test` phase of the `Dockerfile`, since `script/test` builds that phase; the
|
|
||||||
Makefile form below is the same pattern for any repo-local invocation:
|
|
||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
test:
|
test:
|
||||||
@@ -338,26 +209,11 @@ style conventions are in separate documents:
|
|||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
test:
|
test:
|
||||||
@go test -count=1 -timeout 90s -race -cover ./... || \
|
@go test -timeout 30s -race -cover ./... || \
|
||||||
{ echo "--- Rerunning with -v for details ---"; \
|
{ echo "--- Rerunning with -v for details ---"; \
|
||||||
go test -count=1 -timeout 90s -race -v ./...; exit 1; }
|
go test -timeout 30s -race -v ./...; exit 1; }
|
||||||
```
|
```
|
||||||
|
|
||||||
`-count=1` is required on both invocations: it defeats Go's test _result_
|
|
||||||
cache, so neither run can report a stored pass in place of running the
|
|
||||||
tests. It leaves the build cache alone, so it costs the runtime of the suite
|
|
||||||
and no recompilation.
|
|
||||||
|
|
||||||
That cache is Go's own, separate from Docker's layer cache. Go stores a
|
|
||||||
passing result in its cache directory (`GOCACHE`), and when the same tests
|
|
||||||
run again on unchanged code it prints that result, marked `(cached)`,
|
|
||||||
without running them. That matters on a developer's machine, where this
|
|
||||||
target runs and the directory lasts from one run to the next. The `test`
|
|
||||||
phase of the `Dockerfile` needs no `-count=1`: its base image holds no
|
|
||||||
result for this repo's tests and nothing before its `go test` step runs a
|
|
||||||
test, so there is nothing to replay. `--no-cache` (above) is what makes that
|
|
||||||
step run on an unchanged tree.
|
|
||||||
|
|
||||||
Python example:
|
Python example:
|
||||||
|
|
||||||
```makefile
|
```makefile
|
||||||
@@ -383,84 +239,10 @@ style conventions are in separate documents:
|
|||||||
must be in `.gitignore`. No exceptions.
|
must be in `.gitignore`. No exceptions.
|
||||||
|
|
||||||
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
- `.gitignore` should be comprehensive from the start: OS files (`.DS_Store`),
|
||||||
editor files (`.swp`, `*~`), in-repo agent scratch directories (`.claude/`),
|
editor files (`.swp`, `*~`), language build artifacts, and `node_modules/`.
|
||||||
language build artifacts, and `node_modules/`. Fetch the standard `.gitignore`
|
Fetch the standard `.gitignore` from
|
||||||
from `https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.gitignore` when setting up
|
||||||
setting up a new repo. These patterns are written to `.gitignore`'s own
|
a new repo.
|
||||||
semantics, in which an unanchored pattern already matches at every depth; they
|
|
||||||
are not a `.dockerignore` and must not be transplanted into one unmodified.
|
|
||||||
|
|
||||||
- **`.dockerignore` does not use `.gitignore` semantics, and copying patterns
|
|
||||||
across unmodified leaves secrets in the build context.** Docker matches with
|
|
||||||
`moby/patternmatcher`: `filepath.Match` semantics plus a `**` extension, so
|
|
||||||
`*` does not cross `/` and a pattern without a leading `**/` is anchored at
|
|
||||||
the build-context root. A `.dockerignore` listing `.env`, `*.pem` and `*.key`
|
|
||||||
therefore excludes only the copies at the repository root, while `config/.env`
|
|
||||||
and `certs/server.key` still reach the context and can land in an image layer
|
|
||||||
— which is more dangerous than a short file with no secret patterns at all,
|
|
||||||
because it reads as solved and stops anyone looking. Give every
|
|
||||||
depth-independent pattern the `**/` prefix and leave only genuinely
|
|
||||||
root-anchored entries unprefixed: `.claude`, and the repo's own host-built
|
|
||||||
binary, written `/myapp` and never `**/myapp`, which would also match
|
|
||||||
`cmd/myapp/` and delete the package directory from the context. Matching is
|
|
||||||
case-sensitive, and an ALL-CAPS twin per pattern still misses `Server.Key`, so
|
|
||||||
secret names use character ranges — `**/*.[kK][eE][yY]`, `**/*.[pP][eE][mM]`,
|
|
||||||
and likewise for `.envrc` and the extensionless SSH keys. Where such a pattern
|
|
||||||
also catches something the build needs, re-include it with a negation
|
|
||||||
(`!docs/example.env`); deleting the pattern reopens the exposure for every
|
|
||||||
other file it covers. Fetch the standard `.dockerignore` from
|
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.dockerignore` and extend
|
|
||||||
it with the repo's own artifacts.
|
|
||||||
|
|
||||||
- **In-repo agent scratch belongs in both files, written to each file's own
|
|
||||||
semantics.** `.claude/` holds one worktree per in-flight agent — an entire
|
|
||||||
additional checkout of the repo — so under `COPY . .` the build context
|
|
||||||
inflates by a multiple of the repo and another session's unreviewed work can
|
|
||||||
be copied into an image layer. In `.gitignore` the entry is `.claude/`,
|
|
||||||
unanchored. In `.dockerignore` it is `.claude`, anchored and with **no** `**/`
|
|
||||||
prefix, because the prefixed form would also delete any nested directory of
|
|
||||||
that name from the build. Anchoring carries a known gap that the canonical
|
|
||||||
`.dockerignore` states in its own comment, since consuming repos receive the
|
|
||||||
file and not the tracker: the directory is created in the agent's working
|
|
||||||
directory, so a repo running agents in subdirectories still ships
|
|
||||||
`services/api/.claude/` and must add its own anchored entry there.
|
|
||||||
|
|
||||||
- **A plain `docker build .` of a clone stamps the version that
|
|
||||||
`git describe --tags --always` gives**, derived from the `.git` in the build
|
|
||||||
context as the canonical `Dockerfile` above shows. Without its failure check,
|
|
||||||
a missing `git` or an unreadable checkout would leave `-X main.Version=` empty
|
|
||||||
and the build would still exit 0. `script/docker` and `script/cibuild` pass
|
|
||||||
the version they compute on the host; it takes precedence. They do this
|
|
||||||
byte-identically across repos:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
# Own line: a failing command substitution inside an argument does not
|
|
||||||
# trip `set -e`, so the inline form degrades to an empty constant.
|
|
||||||
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
|
|
||||||
[ -n "$version" ] || version="unknown"
|
|
||||||
docker build --no-cache \
|
|
||||||
--build-arg VERSION="$version" \
|
|
||||||
-t "$(script/projectname)" .
|
|
||||||
```
|
|
||||||
|
|
||||||
`--always` makes an untagged repo yield an abbreviated commit hash rather
|
|
||||||
than failing, and the `[ -n "$version" ]` line is the single place the
|
|
||||||
fallback is applied — a live check that fires on a build from an export with
|
|
||||||
no `.git` and on a repository with no commits yet. Do not fold it into the
|
|
||||||
substitution as `|| echo unknown`, which makes the guard unreachable. The
|
|
||||||
Dockerfile's side is `ARG VERSION` in the stage that compiles, declared
|
|
||||||
there because `ARG` is stage-scoped; passing `VERSION` to a repo whose
|
|
||||||
Dockerfile declares no such `ARG` is ignored and costs nothing, which is why
|
|
||||||
the scripts stay byte-identical. One consequence for CI: the standard
|
|
||||||
checkout action clones shallow and fetches no tags, so a repo that embeds a
|
|
||||||
tag-derived version must set `fetch-depth: 0` on its checkout step.
|
|
||||||
|
|
||||||
- **Verify `.dockerignore` by enumerating the image, not by reading the
|
|
||||||
patterns.** Plant files at the root _and_ at least two directories deep, build
|
|
||||||
a probe image that does `COPY . .`, and list what actually landed
|
|
||||||
(`docker run --rm --entrypoint find IMAGE /app`). The `transferring context`
|
|
||||||
size is not a substitute: a nested secret is a few bytes, and BuildKit
|
|
||||||
transfers only the delta from the previous build.
|
|
||||||
|
|
||||||
- **No build artifacts in version control.** Code-derived data (compiled
|
- **No build artifacts in version control.** Code-derived data (compiled
|
||||||
bundles, minified output, generated assets) must never be committed to the
|
bundles, minified output, generated assets) must never be committed to the
|
||||||
@@ -476,56 +258,9 @@ style conventions are in separate documents:
|
|||||||
- Make all changes on a feature branch. You can do whatever you want on a
|
- Make all changes on a feature branch. You can do whatever you want on a
|
||||||
feature branch.
|
feature branch.
|
||||||
|
|
||||||
- `.golangci.yml` is standardized. The vendored copy in a consuming repo must
|
- `.golangci.yml` is standardized and must _NEVER_ be modified by an agent, only
|
||||||
_NEVER_ be modified by an agent: fetch it from
|
manually by the user. Fetch from
|
||||||
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml` and keep it
|
`https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml`.
|
||||||
byte-identical, so that no repo can quietly loosen its own linting. Linter
|
|
||||||
configuration changes are made to the canonical copy in the `prompts` repo and
|
|
||||||
reach consuming repos by re-vendoring; an agent may open a PR against
|
|
||||||
canonical, which only the user merges. One list is exempt from byte-identity,
|
|
||||||
because it cannot be written once for every repo: the `deny` list of the
|
|
||||||
`test-support` depguard rule, where a repo names its own test-support packages
|
|
||||||
by full import path. A repo adds entries there and changes nothing else, and a
|
|
||||||
re-vendor carries its entries forward. The canonical golangci-lint version is
|
|
||||||
v2.14.0 (released 2026-09-24), pinned as the digest of the lint phase's base
|
|
||||||
image
|
|
||||||
(`golangci/golangci-lint@sha256:ad862ba6b3798cbe0fd9fd7408d498fd74fbd2623a92406b2fd3898faf0bf98f`,
|
|
||||||
which reports `2.14.0 built with go1.27.0 from 114493f9`). A module's `go`
|
|
||||||
directive must not name a newer Go minor version than the one golangci-lint
|
|
||||||
was built with, or golangci-lint refuses to lint it: this release lints
|
|
||||||
`go 1.27.1` but not `go 1.28`. That digest is the only pin, since no repo
|
|
||||||
installs golangci-lint on the host. A repo sets the lint phase digest to the
|
|
||||||
one named here and re-vendors `.golangci.yml` in the same commit, whichever of
|
|
||||||
the two prompted the change: the canonical copy can name linters that an older
|
|
||||||
golangci-lint rejects, and a newer golangci-lint can add linters that
|
|
||||||
`default: all` switches on until the canonical copy disables them.
|
|
||||||
|
|
||||||
- **`script/bootstrap` installs a pinned tool by comparing versions, never by
|
|
||||||
testing presence.** An `if ! command -v <tool>; then install; fi` guard tests
|
|
||||||
`PATH` only, so on an already-provisioned machine the pin is inert and a
|
|
||||||
version bump is a silent no-op — while the Dockerfile, installing into a clean
|
|
||||||
image, gets the pinned version, so a local `make check` and `make docker` can
|
|
||||||
disagree about what the tool even is. The canonical form:
|
|
||||||
- compares the installed version against the pin over the **whole** version
|
|
||||||
token; a parser that stops at the first `-` reports `2.12.2` for a host
|
|
||||||
running `2.12.2-rc1` and skips the install;
|
|
||||||
- treats absent, non-zero, empty or unrecognised `--version` output as a
|
|
||||||
mismatch, so the failure direction is a redundant install and never a
|
|
||||||
skipped one;
|
|
||||||
- after installing, re-resolves the binary the way callers do — `hash -r`,
|
|
||||||
then through `PATH`, not through the directory the installer wrote to —
|
|
||||||
and fails naming the resolved path, since an install that a shadowing
|
|
||||||
binary hides succeeds while changing nothing any caller sees;
|
|
||||||
- is actually called, and prints the version on both success paths: a
|
|
||||||
function defined and never invoked has the same exit status and the same
|
|
||||||
empty output as one that worked.
|
|
||||||
|
|
||||||
Keep it POSIX sh: no arrays, no `[[`, no `grep -P`.
|
|
||||||
|
|
||||||
A Go tool a repo needs on the host is installed with `go install` pinned to
|
|
||||||
a commit hash (`go install <package>@<commit hash>`). It is never tracked as
|
|
||||||
a `go.mod` tool dependency or through a `tools.go` file, either of which
|
|
||||||
pulls the tool's own dependencies into the repo's `go.mod` and `go.sum`.
|
|
||||||
|
|
||||||
- When pinning images or packages by hash, add a comment above the reference
|
- When pinning images or packages by hash, add a comment above the reference
|
||||||
with the version and date (YYYY-MM-DD).
|
with the version and date (YYYY-MM-DD).
|
||||||
@@ -639,14 +374,12 @@ style conventions are in separate documents:
|
|||||||
settings.
|
settings.
|
||||||
|
|
||||||
- Avoid putting files in the repo root unless necessary. Root should contain
|
- Avoid putting files in the repo root unless necessary. Root should contain
|
||||||
only project-level config files (`README.md`, `AGENTS.md`, `Makefile`,
|
only project-level config files (`README.md`, `Makefile`, `Dockerfile`,
|
||||||
`Dockerfile`, `LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`,
|
`LICENSE`, `.gitignore`, `.editorconfig`, `REPO_POLICIES.md`, and
|
||||||
and language-specific config). Everything else goes in a subdirectory.
|
language-specific config). Everything else goes in a subdirectory. Canonical
|
||||||
Canonical subdirectory names:
|
subdirectory names:
|
||||||
- `bin/` — executable scripts and tools
|
- `bin/` — executable scripts and tools
|
||||||
- `cmd/` — Go command entrypoints; thin only: one `main.go` per binary whose
|
- `cmd/` — Go command entrypoints
|
||||||
body is a single call into `internal/` or `pkg/`, no project logic in
|
|
||||||
`cmd/`
|
|
||||||
- `configs/` — configuration templates and examples
|
- `configs/` — configuration templates and examples
|
||||||
- `deploy/` — deployment manifests (k8s, compose, terraform)
|
- `deploy/` — deployment manifests (k8s, compose, terraform)
|
||||||
- `docs/` — documentation and markdown (README.md stays in root)
|
- `docs/` — documentation and markdown (README.md stays in root)
|
||||||
@@ -673,7 +406,3 @@ style conventions are in separate documents:
|
|||||||
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
- Go: `go.mod`, `go.sum`, `.golangci.yml`
|
||||||
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
- JS: `package.json`, `yarn.lock`, `.prettierrc`, `.prettierignore`
|
||||||
- Python: `pyproject.toml`
|
- Python: `pyproject.toml`
|
||||||
|
|
||||||
- Guidance for coding agents lives in one `AGENTS.md` at the repository root. It
|
|
||||||
is never committed under a file or directory named after one agent tool, such
|
|
||||||
as `CLAUDE.md` or `.claude/`, and never split into separate memory files.
|
|
||||||
|
|||||||
@@ -1,193 +1,28 @@
|
|||||||
# Workflow
|
# Workflow
|
||||||
|
|
||||||
- branch from `next`
|
- branch (from `main`)
|
||||||
- do the work in Next Step
|
- do the work in Next Step
|
||||||
- move Next Step to the top of Completed Steps
|
- move Next Step to the top of Completed Steps
|
||||||
- move the top item of Future Steps into Next Step
|
- move the top item of Future Steps into Next Step
|
||||||
- commit (`TODO.md` changes in the same commit as the work)
|
- commit (`TODO.md` changes in the same commit as the work)
|
||||||
- push the branch and open a PR against `next`
|
- merge to `main` if the branch is not protected, otherwise open a PR
|
||||||
|
- push
|
||||||
|
|
||||||
# Status
|
# Status
|
||||||
|
|
||||||
pre-1.0. No git tags. `main` is the stable branch and `next` the development
|
pre-1.0. No git tags. `feat/reportbuf-storage` is merged; the backend, the CI
|
||||||
branch, which every PR targets. The frontend and the Go backend ship as one
|
workflow, and the backend repo standard files are all on `main`. Frontend and
|
||||||
Docker image, and the Gitea workflow `.gitea/workflows/check.yml` runs
|
backend are both functional. Working toward the 1.0.0 milestone by closing the
|
||||||
`script/cibuild` on every push. Working toward 1.0.0.
|
remaining repo-compliance issues on the tracker.
|
||||||
|
|
||||||
# Next Step
|
# Next Step
|
||||||
|
|
||||||
Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
Confirm the `.gitea/workflows/check.yml` run is green (main always green
|
||||||
`make check`, as it needs Docker and takes minutes.
|
policy). The workflow file is already on `main`; what is unverified is that its
|
||||||
|
latest run passes.
|
||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
- 2026-10-14: stopping the server waits for a report file write already under
|
|
||||||
way, the periodic flush's or the one `Append` starts when 10 MiB are waiting,
|
|
||||||
before the final flush
|
|
||||||
([#122](https://git.eeqj.de/sneak/netwatch/issues/122)). It used to return at
|
|
||||||
once, and the process exiting cut the write off, losing its reports. A write
|
|
||||||
under way that fails puts its reports back in memory, so the final flush
|
|
||||||
writes them; the stop fails, and the process exits non-zero, only if that
|
|
||||||
flush fails too. Two tests hold the periodic write and the one at 10 MiB open
|
|
||||||
and find the stop waiting until each is released; a third makes the held write
|
|
||||||
fail and finds its report written by the final flush. They run in a
|
|
||||||
`testing/synctest` bubble, so the periodic flush comes without waiting a
|
|
||||||
minute
|
|
||||||
- 2026-10-14: `make test` no longer runs the frontend's production build, which
|
|
||||||
the tests do not need ([#118](https://git.eeqj.de/sneak/netwatch/issues/118)).
|
|
||||||
`yarn build` moved out of the stage that runs the frontend's unit tests, now
|
|
||||||
named `frontend-test`, into a new `frontend-build` stage, which starts from it
|
|
||||||
and, like the `builder` stage, waits on the lint and test phases, so the image
|
|
||||||
still cannot be built unless they pass. The runtime stage copies `dist/` from
|
|
||||||
it, and the `git` that `vite.config.js` reads the commit with is installed
|
|
||||||
there. No build cache was added
|
|
||||||
- 2026-10-14: opening the debug log no longer raises the measured latencies
|
|
||||||
([#131](https://git.eeqj.de/sneak/netwatch/issues/131)). With the panel open,
|
|
||||||
each line written drew the panel again with every line before it, up to 1000,
|
|
||||||
and a check that ends while the page draws is timed only when the drawing is
|
|
||||||
done. Each line is now added at the end of the open panel, its first line is
|
|
||||||
dropped once it holds more than 1000, and it scrolls to its newest line once a
|
|
||||||
frame rather than once a line; the whole panel is drawn only when it is
|
|
||||||
opened. A unit test writes more lines than the panel keeps and checks that the
|
|
||||||
lines already shown are left as they are
|
|
||||||
- 2026-10-14: unset, the backend's own `TRUSTED_PROXIES` trusts `10.0.0.0/8`,
|
|
||||||
`172.16.0.0/12` and `192.168.0.0/16`, and no longer loopback; a list replaces
|
|
||||||
that; set and empty, it trusts no proxy, where it used to get the default, as
|
|
||||||
viper reads an empty environment variable as unset
|
|
||||||
([#129](https://git.eeqj.de/sneak/netwatch/issues/129)). A value it cannot
|
|
||||||
parse still stops the start, and the image still starts the backend with
|
|
||||||
`TRUSTED_PROXIES=127.0.0.1/32`. Tests load the config with it unset, empty and
|
|
||||||
set to a list
|
|
||||||
- 2026-10-14: while report files cannot be written, `POST /api/v1/reports`
|
|
||||||
answers 500, and storing recovers once writing works again, with no restart
|
|
||||||
([#123](https://git.eeqj.de/sneak/netwatch/issues/123)). A failed write puts
|
|
||||||
its reports back in the report buffer, still counted against
|
|
||||||
`DATA_DIR_MAX_BYTES`, where they used to be lost; the next flush, within a
|
|
||||||
minute, tries them again. Until a write succeeds, every report is refused and
|
|
||||||
nothing of it is kept, so the buffer does not grow while writes fail. A test
|
|
||||||
posts reports through the report handler before, during and after a failure;
|
|
||||||
others check that a report appended while a write runs is kept, behind that
|
|
||||||
write's own, when the write fails, and that the cap still holds exactly once a
|
|
||||||
failed write's reports are written
|
|
||||||
- 2026-10-14: a request whose handler panics is logged with status 500 and
|
|
||||||
counted with `code` 500 in the metrics
|
|
||||||
([#125](https://git.eeqj.de/sneak/netwatch/issues/125)), not 200. `Recoverer`
|
|
||||||
is still the first middleware; `Write500OnPanic`, after the request log and
|
|
||||||
after the metrics, writes the 500 as the panic passes and lets it go on to
|
|
||||||
Sentry and `Recoverer`, which now writes a 500 only when no status was written
|
|
||||||
- 2026-10-14: with `TRUSTED_PROXIES` unset, the container's nginx trusts the
|
|
||||||
three private ranges of RFC 1918, `10.0.0.0/8`, `172.16.0.0/12` and
|
|
||||||
`192.168.0.0/16` ([#119](https://git.eeqj.de/sneak/netwatch/issues/119)), as
|
|
||||||
sneak ruled on 2026-09-22 for every app; it trusted no proxy before. A list
|
|
||||||
replaces them, and set and empty it trusts none. The backend inside the
|
|
||||||
container still trusts only `127.0.0.1/32`
|
|
||||||
- 2026-10-14: `POST /api/v1/reports` takes only a report sent with
|
|
||||||
`Content-Type: application/json`, parameters such as `charset` allowed, and
|
|
||||||
refuses any other with 415 and the usual error body, storing nothing
|
|
||||||
([#126](https://git.eeqj.de/sneak/netwatch/issues/126)). A page on another
|
|
||||||
origin can send a `text/plain` POST without a CORS preflight, so any site
|
|
||||||
could make its visitors' browsers store reports; a JSON POST waits for the
|
|
||||||
preflight, which only an origin in `CORS_ALLOWED_ORIGINS` passes. The page
|
|
||||||
already sent `application/json`. A test posts a report as `text/plain`, as
|
|
||||||
`application/x-www-form-urlencoded`, as `multipart/form-data`, with no
|
|
||||||
`Content-Type`, and as `application/json` with and without a charset, and
|
|
||||||
checks which are stored
|
|
||||||
- 2026-10-14: changing the interval no longer drops the samples not yet reported
|
|
||||||
to the backend ([#124](https://git.eeqj.de/sneak/netwatch/issues/124)). The
|
|
||||||
interval menu resets each host with the new `HostState.reset`, which moves the
|
|
||||||
history's entries to `clearedHistory`, keeping the last 100 of all it has
|
|
||||||
cleared, and `buildReport` reads both, so the next report has the samples
|
|
||||||
recorded before the change that no report has delivered. Until its next check,
|
|
||||||
such a host is sent with the status of its last check, not `pending`. The
|
|
||||||
page's history still starts over, and the per-host marks still leave out of a
|
|
||||||
report the samples a delivered one had. Unit tests reset a host as the
|
|
||||||
interval menu does and find its unreported samples in the next report, and
|
|
||||||
only the last 100 after more were cleared; `web/src/main.js` now exports
|
|
||||||
`Reporter` for them
|
|
||||||
- 2026-10-14: the repo has the layout `REPO_POLICIES.md` gives
|
|
||||||
([#30](https://git.eeqj.de/sneak/netwatch/issues/30)). The Go module is at the
|
|
||||||
root (`go.mod`, `go.sum`, `.golangci.yml`, `cmd/netwatch-server/`,
|
|
||||||
`internal/`), so its path `sneak.berlin/go/netwatch` names the repository.
|
|
||||||
`go.mod` ignores `node_modules/`, where npm packages can ship `.go` files that
|
|
||||||
`make tidy` would otherwise read as this module's packages. The page's source
|
|
||||||
is in `web/` (`index.html`, `src/`), which `vite.config.js` makes Vite's root;
|
|
||||||
the build still goes to `dist/`. `nginx.conf` and `security-headers.conf` are
|
|
||||||
in `configs/`. `backend/` is gone: its `LICENSE` and `.dockerignore` are
|
|
||||||
deleted, its `.gitignore` entries are in the root one, its `README.md` is
|
|
||||||
folded into the root `README.md`, and its scripts into the root `script/`:
|
|
||||||
`script/build` builds the server binary after the frontend, `script/run` and
|
|
||||||
`script/clean` came from `backend/script/`, and `make test`, `make lint`,
|
|
||||||
`make fmt` and `make fmt-check` do what its `test`, `lint`, `fmt` and
|
|
||||||
`fmt-check` did. `script/bootstrap` no longer installs gcc, which only the
|
|
||||||
host Go tests needed. The image and the page are unchanged
|
|
||||||
- 2026-10-07: the files shared from `sneak/prompts` are its copies at commit
|
|
||||||
`dd4027b` ([#113](https://git.eeqj.de/sneak/netwatch/issues/113)), with this
|
|
||||||
repository's own entries after the shared content in `.gitignore`,
|
|
||||||
`.dockerignore` and `.editorconfig`. `make lint` and `make test` each build
|
|
||||||
one phase of `Dockerfile` without the cache: `lint` runs golangci-lint v2.14.0
|
|
||||||
over `backend/` and, through the `frontend-lint` stage, eslint; `test` runs
|
|
||||||
the Go tests on the Debian Go image and, through the `frontend` stage, the
|
|
||||||
frontend's unit tests and build. The builder stage waits on both, and without
|
|
||||||
a `VERSION` build argument takes the version from `git describe` on the `.git`
|
|
||||||
in the build context. Neither linter nor the tests run on the host for
|
|
||||||
`make check`; `backend/script/lint` runs the root one, and
|
|
||||||
`backend/.golangci.yml` is no longer checked against a sha256. prettier
|
|
||||||
formats only the JavaScript, CSS, HTML and Markdown, so the shared
|
|
||||||
`.golangci.yml` stays as fetched. `script/fmt` and `script/fmt-check` put
|
|
||||||
`~/.local/bin` on `PATH`, which the shared workflow no longer does.
|
|
||||||
`script/frontend-lint` and `script/frontend-check` are gone
|
|
||||||
- 2026-10-07: the six Hetzner targets answer again, and failed checks are
|
|
||||||
written to the browser console
|
|
||||||
([#114](https://git.eeqj.de/sneak/netwatch/issues/114)). The Hetzner
|
|
||||||
speed-test servers close the connection without an answer when the URL has a
|
|
||||||
query string, and every check added `?_cb=` and the time; checks now fetch
|
|
||||||
each target's URL as written, with `cache: 'no-store'` as before. Each
|
|
||||||
recorded check that fails writes one `console.error` line, and the same line
|
|
||||||
to the debug log, with the target's name and URL, the time, what failed and
|
|
||||||
how long the request took; a target that answers after failed checks writes
|
|
||||||
one `console.info` line. Checks the page does not record write none, so the
|
|
||||||
debug log no longer lists failures in the first round or in the recovery probe
|
|
||||||
- 2026-10-04: the page's footer no longer says "IPv4 only"
|
|
||||||
([#111](https://git.eeqj.de/sneak/netwatch/issues/111)): each check is a
|
|
||||||
`fetch`, the browser picks IPv4 or IPv6 for each WAN host, and the local
|
|
||||||
targets are IPv4 addresses. The rest of the footer is unchanged
|
|
||||||
- 2026-10-04: `README.md`, `TODO.md` and `test/viewport/README.md` say what the
|
|
||||||
tree does (issue #24). The README's Getting Started leads with `make` targets;
|
|
||||||
a new Backend section says what `netwatch-server` stores, its routes and how
|
|
||||||
the image builds and runs it, and points to `backend/README.md` for its
|
|
||||||
settings; the checks are GET requests; the 26 WAN hosts, the four health
|
|
||||||
states, the summary's figures and the features the list lacked are described
|
|
||||||
as the page has them; and its TODO section points here, as does the one in
|
|
||||||
`backend/README.md`, whose open items moved to Future Steps. This file's
|
|
||||||
Workflow branches from `next` and opens the PR against `next`, Status says
|
|
||||||
where the repo stands, and Next Step and Future Steps hold only open work,
|
|
||||||
linked to its issue where one exists. The viewport harness README names Node's
|
|
||||||
test runner, not `vitest`
|
|
||||||
- 2026-10-04: in `src/main.js` (issue #102), a target's min, max, median and
|
|
||||||
average latency come from one list of its answers, through the same function
|
|
||||||
the summary's figures use, so the median is written once. The latency color
|
|
||||||
limits are one table in `CONFIG`, read by both the figure's and the
|
|
||||||
sparkline's color. The health thresholds, the debug log's length, the gateway
|
|
||||||
check's timeout, the recovery probe's number of hosts and interval, how often
|
|
||||||
the rows are sorted and the delay before the first sparkline resize are
|
|
||||||
`CONFIG` entries too. A unit test checks the summary's figures. Nothing the
|
|
||||||
page does or shows changed; the footer's color legend still writes the limits
|
|
||||||
out as text
|
|
||||||
- 2026-10-04: password guesses at `/metrics` are rate limited (issue #104): each
|
|
||||||
client address, resolved through `TRUSTED_PROXIES` as for reports, may make 60
|
|
||||||
requests to `/metrics` a minute, counted by `go-chi/httprate` apart from its
|
|
||||||
reports; past that it gets 429 and its basic auth credentials are not checked.
|
|
||||||
The limit is a constant in `backend/internal/server/routes.go`. A test uses up
|
|
||||||
one client's allowance on wrong passwords, gets 429 with the right one, and
|
|
||||||
checks that another client behind the same nginx still gets in
|
|
||||||
- 2026-10-04: the backend reports errors to Sentry (issue #95). With
|
|
||||||
`SENTRY_DSN` set, it sets up `sentry-go` with the release `netwatch-server-`
|
|
||||||
and its version, reports each panic in a handler through `sentryhttp`, the
|
|
||||||
last of the middleware every request goes through, which panics again so the
|
|
||||||
request still gets the 500 from the panic recovery, and waits up to 2 seconds
|
|
||||||
on shutdown for Sentry to finish sending. A DSN Sentry refuses stops the start
|
|
||||||
with an error naming `SENTRY_DSN`. With it empty, Sentry is not set up and
|
|
||||||
nothing is sent to it
|
|
||||||
- 2026-10-04: a target's name and URL and a debug log message show as the
|
- 2026-10-04: a target's name and URL and a debug log message show as the
|
||||||
characters they are and are never read as HTML (issue #29): a host row escapes
|
characters they are and are never read as HTML (issue #29): a host row escapes
|
||||||
the name and URL it writes into its markup, and the debug log sets each line
|
the name and URL it writes into its markup, and the debug log sets each line
|
||||||
@@ -514,18 +349,12 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
|||||||
|
|
||||||
# Future Steps
|
# Future Steps
|
||||||
|
|
||||||
- A backend test that posts a report to `POST /api/v1/reports` and checks the
|
- Wire `script/frontend-viewport-test` into CI as its own step (deliberately not
|
||||||
compressed file it is written to
|
part of `make check` today; the decision has real CI-runtime cost and is
|
||||||
- A backend route that decompresses the stored reports and answers queries on
|
tracked separately)
|
||||||
them
|
- Compliance top-up as one small commit: add .editorconfig and add the hooks
|
||||||
- Prometheus metrics for the backend's in-memory buffer: its size, the number of
|
target to the Makefile
|
||||||
flushes and the number of reports
|
- After merge, confirm .gitea/workflows/check.yml is on main and CI is green
|
||||||
- A configurable host list (an environment variable or a config file)
|
(main always green policy)
|
||||||
- Export of the latency history (CSV or JSON)
|
- Decide what to do with untracked resume.sh: commit it, gitignore it, or delete
|
||||||
- A notification when the health status changes to DEGRADED
|
it
|
||||||
- `make test` within the 20-second target
|
|
||||||
([#118](https://git.eeqj.de/sneak/netwatch/issues/118)): every run compiles
|
|
||||||
the Go standard library and every dependency with the race detector from
|
|
||||||
nothing. Whether test builds may keep Go's compiled packages between runs
|
|
||||||
waits on sneak's ruling in
|
|
||||||
[prompts#113](https://git.eeqj.de/sneak/prompts/issues/113)
|
|
||||||
|
|||||||
@@ -0,0 +1,2 @@
|
|||||||
|
node_modules
|
||||||
|
.DS_Store
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
/netwatch-server
|
||||||
|
*.log
|
||||||
|
*.out
|
||||||
|
*.test
|
||||||
|
.env
|
||||||
|
data/
|
||||||
@@ -17,7 +17,6 @@ linters:
|
|||||||
disable:
|
disable:
|
||||||
# Genuinely incompatible with project patterns
|
# Genuinely incompatible with project patterns
|
||||||
- exhaustruct # Requires all struct fields
|
- exhaustruct # Requires all struct fields
|
||||||
- exhaustruct_v5 # Requires all struct fields (successor to exhaustruct)
|
|
||||||
- godot # Requires comments to end with periods
|
- godot # Requires comments to end with periods
|
||||||
- wrapcheck # Too verbose for internal packages
|
- wrapcheck # Too verbose for internal packages
|
||||||
- varnamelen # Short names like db, id are idiomatic Go
|
- varnamelen # Short names like db, id are idiomatic Go
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 @sneak (https://sneak.berlin)
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
# Thin shims; the implementations live in backend/script/ (see the
|
||||||
|
# Entrypoints section of README.md). There is no check, hooks or docker
|
||||||
|
# target here: the root Makefile's check covers this directory, its
|
||||||
|
# hooks target installs the repo's only pre-commit hook, and its docker
|
||||||
|
# target builds the one image, which contains this backend.
|
||||||
|
|
||||||
|
.PHONY: all build test lint fmt fmt-check run clean
|
||||||
|
|
||||||
|
all: build
|
||||||
|
|
||||||
|
build:
|
||||||
|
@script/build
|
||||||
|
|
||||||
|
test:
|
||||||
|
@script/test
|
||||||
|
|
||||||
|
lint:
|
||||||
|
@script/lint
|
||||||
|
|
||||||
|
fmt:
|
||||||
|
@script/fmt
|
||||||
|
|
||||||
|
fmt-check:
|
||||||
|
@script/fmt-check
|
||||||
|
|
||||||
|
run:
|
||||||
|
@script/run
|
||||||
|
|
||||||
|
clean:
|
||||||
|
@script/clean
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
netwatch-server is an MIT-licensed Go HTTP backend by
|
||||||
|
[@sneak](https://sneak.berlin) that receives telemetry reports from the NetWatch
|
||||||
|
SPA and persists them as zstd-compressed JSONL files on disk.
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
From this directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build and run locally
|
||||||
|
make run
|
||||||
|
```
|
||||||
|
|
||||||
|
From the repo root, whose `Dockerfile` builds the one image that ships this
|
||||||
|
backend behind nginx (see [Container image](#container-image)):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run tests, lint, and format check over the frontend and this backend
|
||||||
|
make check
|
||||||
|
|
||||||
|
# Build the image: nginx, the frontend and this backend
|
||||||
|
make docker
|
||||||
|
docker run -p 8080:8080 netwatch
|
||||||
|
```
|
||||||
|
|
||||||
|
## Entrypoints
|
||||||
|
|
||||||
|
This directory follows the same
|
||||||
|
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
||||||
|
pattern as the repo root: the targets in `backend/Makefile` are thin shims over
|
||||||
|
`backend/script/`. The root `Dockerfile` runs them, and the root scripts call
|
||||||
|
`test`, `fmt` and `fmt-check`:
|
||||||
|
|
||||||
|
- `script/build` — compile the static `netwatch-server` binary with its version
|
||||||
|
stamped in. The version is `VERSION` from the environment; when that is unset
|
||||||
|
or empty, it falls back to `git describe` inside a git checkout, then to `dev`
|
||||||
|
- `script/test` — run the Go tests with the race detector and coverage. Go's
|
||||||
|
`-timeout 30s` bounds the tests, not their compile. If they fail, they run
|
||||||
|
again with `-v` for the details, and the script fails. The race detector needs
|
||||||
|
a C compiler
|
||||||
|
- `script/lint` — check `.golangci.yml` against its pinned sha256, then run
|
||||||
|
golangci-lint. It runs inside the golangci-lint image of the lint stage of the
|
||||||
|
root `Dockerfile`; from a checkout, run `make lint` at the repo root, which
|
||||||
|
builds that stage
|
||||||
|
- `script/fmt` — format the Go sources (writes)
|
||||||
|
- `script/fmt-check` — check Go formatting (read-only)
|
||||||
|
- `script/run` — build and run the server locally
|
||||||
|
- `script/clean` — remove build artifacts
|
||||||
|
|
||||||
|
There is no `check`, `hooks` or `docker` target here: the root `make check`
|
||||||
|
covers this directory, the root `make hooks` installs the repo's only pre-commit
|
||||||
|
hook, and the root `make docker` builds the image that contains this backend.
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
The NetWatch frontend collects latency measurements from the browser but has no
|
||||||
|
way to persist or aggregate them. This backend provides a minimal
|
||||||
|
`POST /api/v1/reports` endpoint that buffers incoming reports in memory and
|
||||||
|
flushes them to compressed files on disk for later analysis.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
The server is structured as an `fx`-wired Go application under
|
||||||
|
`cmd/netwatch-server/`. Internal packages in `internal/` follow standard Go
|
||||||
|
project layout:
|
||||||
|
|
||||||
|
- **`config`**: Loads configuration from environment variables and config files
|
||||||
|
via Viper.
|
||||||
|
- **`handlers`**: HTTP request handlers for the API (health check, report
|
||||||
|
ingestion).
|
||||||
|
- **`reportbuf`**: In-memory buffer that accumulates JSONL report lines and
|
||||||
|
flushes to zstd-compressed files when the buffer reaches 10 MiB or every 60
|
||||||
|
seconds.
|
||||||
|
- **`server`**: Chi-based HTTP server with middleware wiring and route
|
||||||
|
registration.
|
||||||
|
- **`healthcheck`**, **`middleware`**, **`logger`**, **`globals`**: Supporting
|
||||||
|
infrastructure.
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
| ---------------------- | -------------------- | -------------------------------------------------------------------------------------------------------- |
|
||||||
|
| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface |
|
||||||
|
| `PORT` | `8080` | HTTP listen port |
|
||||||
|
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
|
||||||
|
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) |
|
||||||
|
| `DEBUG` | `false` | Enable debug logging |
|
||||||
|
| `TRUSTED_PROXIES` | loopback + RFC1918 | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
|
||||||
|
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
|
||||||
|
| `CORS_ALLOWED_ORIGINS` | empty | Comma-separated origins whose pages may call the API; see [CORS](#cors) |
|
||||||
|
| `METRICS_USERNAME` | empty | Basic auth user name for `/metrics`; see [Metrics](#metrics) |
|
||||||
|
| `METRICS_PASSWORD` | empty | Basic auth password for `/metrics`; see [Metrics](#metrics) |
|
||||||
|
|
||||||
|
`TRUSTED_PROXIES` defaults to
|
||||||
|
`127.0.0.1/32,::1/128,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. The loopback
|
||||||
|
entries cover a reverse proxy on the same host. A request whose direct peer is
|
||||||
|
outside this set has its forwarded headers ignored, and the direct peer is
|
||||||
|
logged and rate-limited instead. The container image does not use this default;
|
||||||
|
see [Container image](#container-image).
|
||||||
|
|
||||||
|
A variable set to a value the server cannot use, such as `PORT=abc`,
|
||||||
|
`DEBUG=maybe` or a `BIND_ADDRESS` that is not an IP address, stops it from
|
||||||
|
starting, with an error naming the variable. An empty variable counts as unset.
|
||||||
|
|
||||||
|
### Container image
|
||||||
|
|
||||||
|
The root `Dockerfile` builds one image in which nginx listens on the public port
|
||||||
|
8080, serves the frontend, and proxies `/api/`, `/.well-known/healthcheck` and
|
||||||
|
`/metrics` to this server. The image's entrypoint, `bin/entrypoint.sh`, starts
|
||||||
|
the server as user `netwatch` (uid 1000) with `BIND_ADDRESS=127.0.0.1` and
|
||||||
|
`PORT=8081`, so only nginx reaches it, and with `TRUSTED_PROXIES=127.0.0.1/32`,
|
||||||
|
so it takes the client address nginx passes on and no other. `DATA_DIR` is
|
||||||
|
`/data/reports`, on the `/data` volume; before starting the server, the
|
||||||
|
entrypoint creates it and gives it and `/data` to `netwatch` with
|
||||||
|
`netwatch-server prepare-data-dir`, which acts on nothing outside `/data`. nginx
|
||||||
|
replaces the security headers this server sets with those in the root
|
||||||
|
`security-headers.conf`, so those are what clients of the image see.
|
||||||
|
|
||||||
|
The container's own `TRUSTED_PROXIES` goes to nginx instead: IP addresses or
|
||||||
|
CIDRs, separated by commas, of the reverse proxies in front of the container.
|
||||||
|
nginx takes the client address from `X-Forwarded-For` only on a request from one
|
||||||
|
of them. Unset or empty, nginx trusts no proxy, and the client address is the
|
||||||
|
one each request comes from, so every client behind a proxy shares one rate
|
||||||
|
limit. An entry that is not an IP address or CIDR, such as a hostname or
|
||||||
|
`1.2.3`, stops the container at start with an error naming `TRUSTED_PROXIES`:
|
||||||
|
the entrypoint checks each entry with `netwatch-server check-cidr`, which parses
|
||||||
|
it as this server parses its own `TRUSTED_PROXIES`.
|
||||||
|
|
||||||
|
### Report storage
|
||||||
|
|
||||||
|
Reports are written as `reports-<timestamp>-<number>.jsonl.zst` files in
|
||||||
|
`DATA_DIR`. The timestamp is in UTC to the millisecond, so the names sort by
|
||||||
|
time. The number starts at 1 when the server starts and goes up by one for each
|
||||||
|
file the server starts to write, so two files written in the same millisecond
|
||||||
|
still get different names. A failed write uses up its number and leaves a gap in
|
||||||
|
the numbers: its file, if it was created, is removed. The file stays, counted
|
||||||
|
toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too.
|
||||||
|
Each file contains one JSON object per line, compressed with zstd. Files are
|
||||||
|
created with `O_EXCL` to prevent overwrites.
|
||||||
|
|
||||||
|
### Report limits
|
||||||
|
|
||||||
|
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||||
|
credentials, so it is bounded instead. Both refusals below answer with the same
|
||||||
|
`{"status":"error"}` body as any other error.
|
||||||
|
|
||||||
|
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
||||||
|
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
||||||
|
`Retry-After: 60`. The minute slides: reports from the minute before still
|
||||||
|
count, fading out over the current one, so an address is sure never to be
|
||||||
|
refused only while it sends at most half of `REPORTS_PER_MINUTE` in any 60
|
||||||
|
seconds. The page sends one report a minute from each open tab, so the default
|
||||||
|
of 60 refuses nothing from up to 30 tabs behind one address, such as a
|
||||||
|
household or an office sharing it, however their reports bunch up. Report
|
||||||
|
responses also carry `X-RateLimit-Limit`, `X-RateLimit-Remaining` and
|
||||||
|
`X-RateLimit-Reset` headers.
|
||||||
|
- **Size cap.** The report files in `DATA_DIR` may total at most
|
||||||
|
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
|
||||||
|
waiting in memory count at their uncompressed size until they are written;
|
||||||
|
those lost to a failed write stop counting, and the part of its file written
|
||||||
|
is removed. When a report would take the total past the cap, the oldest report
|
||||||
|
files are deleted to make room, and each deletion is logged with the file's
|
||||||
|
name and size; a file still being written is never deleted. A report is
|
||||||
|
refused with 507, and nothing of it is stored, only when the reports waiting
|
||||||
|
to be written fill the cap on their own, and then no file is deleted. At
|
||||||
|
start, report files past the cap, as after lowering it, are deleted the same
|
||||||
|
way. So the cap is how much of the newest reports is kept: the default of 1
|
||||||
|
GiB is small enough for any host; set it to the space you can give `DATA_DIR`.
|
||||||
|
|
||||||
|
### CORS
|
||||||
|
|
||||||
|
The page calls the API from the origin it is served from, so by default the
|
||||||
|
server sends no CORS headers, and browsers let no other origin's pages call it.
|
||||||
|
To serve the page from elsewhere, list that origin in `CORS_ALLOWED_ORIGINS`
|
||||||
|
(for example `https://netwatch.example.com`); pages from a listed origin may
|
||||||
|
`GET` and `POST` with a `Content-Type` header. Each entry must be a plain
|
||||||
|
origin, `scheme://host` with an optional `:port`, as browsers send it: no path,
|
||||||
|
not even a trailing `/`, and no `*`. Any other entry stops the server from
|
||||||
|
starting, with an error naming `CORS_ALLOWED_ORIGINS`.
|
||||||
|
|
||||||
|
### Metrics
|
||||||
|
|
||||||
|
With both `METRICS_USERNAME` and `METRICS_PASSWORD` set, the server serves
|
||||||
|
Prometheus metrics at `GET /metrics` to requests with those as their basic auth
|
||||||
|
credentials, and answers any other with 401. For each request that reaches the
|
||||||
|
health check or `POST /api/v1/reports`, those the rate limit refuses included,
|
||||||
|
the metrics record its duration and response size, labelled with its path,
|
||||||
|
method and status; they also count those requests in progress, and include Go's
|
||||||
|
runtime and process metrics. No other request is recorded: not those to
|
||||||
|
`/metrics` itself, and not those answered before they reach either route, such
|
||||||
|
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
||||||
|
for a method its route does not take, or 413 for declaring a body length over
|
||||||
|
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
||||||
|
length reaches the route, is answered 413 there, and is recorded with that
|
||||||
|
status. Clients can make up any number of paths and methods, and each would add
|
||||||
|
labels to the metrics for as long as the server runs. With neither set, nothing
|
||||||
|
is recorded and `/metrics` answers 404. One without the other stops the server
|
||||||
|
from starting, with an error naming both; so does a `METRICS_USERNAME`
|
||||||
|
containing `:`, which basic auth cannot carry, with an error naming it.
|
||||||
|
|
||||||
|
## TODO
|
||||||
|
|
||||||
|
- Add integration test that POSTs a report and verifies the compressed output
|
||||||
|
- Add report decompression/query endpoint
|
||||||
|
- Add metrics (Prometheus) for buffer size, flush count, report count
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT. See [LICENSE](LICENSE).
|
||||||
|
|
||||||
|
## Author
|
||||||
|
|
||||||
|
[@sneak](https://sneak.berlin)
|
||||||
@@ -115,28 +115,6 @@ func TestMalformedConfigFileStopsTheStart(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestRefusedSentryDSNStopsTheStart: a SENTRY_DSN that Sentry refuses
|
|
||||||
// stops the start, and the error, naming SENTRY_DSN, is logged as JSON.
|
|
||||||
func TestRefusedSentryDSNStopsTheStart(t *testing.T) {
|
|
||||||
t.Setenv("SENTRY_DSN", "not-a-dsn")
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(t.Context(), childTimeout)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
child, stdout, stderr := startServer(ctx, t, t.TempDir(), freePort(ctx, t))
|
|
||||||
|
|
||||||
err := child.Wait()
|
|
||||||
if child.ProcessState.ExitCode() != 1 {
|
|
||||||
t.Fatalf("server exit = %v, want exit status 1", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
requireJSONLines(t, stdout, stderr)
|
|
||||||
|
|
||||||
if !strings.Contains(stdout.String(), "SENTRY_DSN") {
|
|
||||||
t.Fatalf("no error naming SENTRY_DSN in stdout:\n%s", stdout)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// startServer runs main() in a child process listening on
|
// startServer runs main() in a child process listening on
|
||||||
// 127.0.0.1:port, with home as its HOME and working directory and its
|
// 127.0.0.1:port, with home as its HOME and working directory and its
|
||||||
// data directory in home, so it touches nothing outside home. Its
|
// data directory in home, so it touches nothing outside home. Its
|
||||||
@@ -2,12 +2,8 @@ module sneak.berlin/go/netwatch
|
|||||||
|
|
||||||
go 1.25.5
|
go 1.25.5
|
||||||
|
|
||||||
// npm packages can ship .go files, which ./... and all would include.
|
|
||||||
ignore ./node_modules
|
|
||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
|
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
|
||||||
github.com/getsentry/sentry-go v0.49.0
|
|
||||||
github.com/go-chi/chi/v5 v5.2.5
|
github.com/go-chi/chi/v5 v5.2.5
|
||||||
github.com/go-chi/cors v1.2.2
|
github.com/go-chi/cors v1.2.2
|
||||||
github.com/go-chi/httprate v0.16.0
|
github.com/go-chi/httprate v0.16.0
|
||||||
+8
-16
@@ -4,22 +4,18 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
|
|||||||
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
|
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
|
||||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
||||||
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
||||||
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
||||||
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
||||||
github.com/getsentry/sentry-go v0.49.0 h1:Ehejknu1l023Ub7QoRBVLAI7g3Jnhqku4oWx4B4Sh5s=
|
|
||||||
github.com/getsentry/sentry-go v0.49.0/go.mod h1:nuMJAoCfe1u0Bts2ocyNI+TW8HT84vRMqwA5Qq/SKUI=
|
|
||||||
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
|
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
|
||||||
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
|
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
|
||||||
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
|
github.com/go-chi/cors v1.2.2 h1:Jmey33TE+b+rB7fT8MUy1u0I4L+NARQlK6LhzKPSyQE=
|
||||||
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
|
github.com/go-chi/cors v1.2.2/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
|
||||||
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
|
github.com/go-chi/httprate v0.16.0 h1:8V5DH9j6pSK6UQoBsTpvMyFxycqaKEIToyPKzHJjUa8=
|
||||||
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
|
github.com/go-chi/httprate v0.16.0/go.mod h1:A8lo+qRhk+s9LiuP5saS7XCGDXRXMcrueq0NfIuCa/I=
|
||||||
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
|
|
||||||
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
|
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
||||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||||
@@ -40,12 +36,8 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq
|
|||||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||||
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
|
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
|
||||||
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
|
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
|
||||||
github.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
github.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
|
||||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
|
||||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
|
||||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
|
||||||
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
|
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
|
||||||
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
|
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
|
||||||
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
|
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
|
||||||
@@ -54,8 +46,8 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
|
|||||||
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
||||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||||
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
|
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
|
||||||
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
|
||||||
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
|
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
|
||||||
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
|
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
|
||||||
github.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8=
|
github.com/slok/go-http-metrics v0.13.0 h1:lQDyJJx9wKhmbliyUsZ2l6peGnXRHjsjoqPt5VYzcP8=
|
||||||
@@ -101,7 +93,7 @@ golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
|
|||||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
|
||||||
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
@@ -9,7 +9,6 @@ import (
|
|||||||
"math"
|
"math"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
@@ -22,14 +21,16 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// defaultTrustedProxies lists the networks whose forwarded
|
// defaultTrustedProxies lists the networks whose forwarded
|
||||||
// headers are honoured when TRUSTED_PROXIES is unset: the
|
// headers are honoured by default: IPv4 and IPv6 loopback,
|
||||||
// RFC 1918 ranges. The container image does not use it:
|
// for a reverse proxy on the same host, and the RFC1918
|
||||||
|
// ranges. The container image does not use it:
|
||||||
// bin/entrypoint.sh gives the server 127.0.0.1/32, since
|
// bin/entrypoint.sh gives the server 127.0.0.1/32, since
|
||||||
// nginx is its only client there.
|
// nginx is its only client there.
|
||||||
const defaultTrustedProxies = "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
|
const defaultTrustedProxies = "127.0.0.1/32,::1/128," +
|
||||||
|
"10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
|
||||||
|
|
||||||
// Default limits on stored reports; README.md gives the reasons for
|
// Default limits on stored reports; backend/README.md gives the
|
||||||
// these values.
|
// reasons for these values.
|
||||||
const (
|
const (
|
||||||
defaultReportsPerMinute = 60
|
defaultReportsPerMinute = 60
|
||||||
defaultDataDirMaxBytes = 1 << 30 // 1 GiB
|
defaultDataDirMaxBytes = 1 << 30 // 1 GiB
|
||||||
@@ -134,14 +135,6 @@ func New(
|
|||||||
viper.GetString("DEBUG"), errNotBool)
|
viper.GetString("DEBUG"), errNotBool)
|
||||||
}
|
}
|
||||||
|
|
||||||
// An empty TRUSTED_PROXIES trusts no proxy, but viper reads an
|
|
||||||
// empty environment variable as unset and would give the default,
|
|
||||||
// so the environment is read first.
|
|
||||||
trustedProxies, set := os.LookupEnv("TRUSTED_PROXIES")
|
|
||||||
if !set {
|
|
||||||
trustedProxies = viper.GetString("TRUSTED_PROXIES")
|
|
||||||
}
|
|
||||||
|
|
||||||
s := &Config{
|
s := &Config{
|
||||||
BindAddress: viper.GetString("BIND_ADDRESS"),
|
BindAddress: viper.GetString("BIND_ADDRESS"),
|
||||||
CORSAllowedOrigins: splitList(viper.GetString("CORS_ALLOWED_ORIGINS")),
|
CORSAllowedOrigins: splitList(viper.GetString("CORS_ALLOWED_ORIGINS")),
|
||||||
@@ -153,7 +146,7 @@ func New(
|
|||||||
Port: port,
|
Port: port,
|
||||||
ReportsPerMinute: viper.GetInt("REPORTS_PER_MINUTE"),
|
ReportsPerMinute: viper.GetInt("REPORTS_PER_MINUTE"),
|
||||||
SentryDSN: viper.GetString("SENTRY_DSN"),
|
SentryDSN: viper.GetString("SENTRY_DSN"),
|
||||||
TrustedProxies: splitList(trustedProxies),
|
TrustedProxies: splitList(viper.GetString("TRUSTED_PROXIES")),
|
||||||
log: log,
|
log: log,
|
||||||
params: ¶ms,
|
params: ¶ms,
|
||||||
}
|
}
|
||||||
@@ -1,8 +1,6 @@
|
|||||||
package config_test
|
package config_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"os"
|
|
||||||
"slices"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
@@ -31,62 +29,6 @@ func requireConfigError(t *testing.T, setting string) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// requireTrustedProxies builds the config as main does and fails the
|
|
||||||
// test unless its TrustedProxies are want.
|
|
||||||
func requireTrustedProxies(t *testing.T, want []string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var cfg *config.Config
|
|
||||||
|
|
||||||
app := fx.New(
|
|
||||||
fx.NopLogger,
|
|
||||||
fx.Provide(globals.New, logger.New, config.New),
|
|
||||||
fx.Populate(&cfg),
|
|
||||||
)
|
|
||||||
|
|
||||||
err := app.Err()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("config error = %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !slices.Equal(cfg.TrustedProxies, want) {
|
|
||||||
t.Fatalf("TrustedProxies = %q, want %q", cfg.TrustedProxies, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestTrustedProxiesUnsetTrustsRFC1918: unset, the server trusts the
|
|
||||||
// RFC 1918 ranges and nothing else, loopback included.
|
|
||||||
func TestTrustedProxiesUnsetTrustsRFC1918(t *testing.T) {
|
|
||||||
// t.Setenv first, so the variable is put back as it was after the
|
|
||||||
// test.
|
|
||||||
t.Setenv("TRUSTED_PROXIES", "")
|
|
||||||
|
|
||||||
err := os.Unsetenv("TRUSTED_PROXIES")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
requireTrustedProxies(t,
|
|
||||||
[]string{"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestTrustedProxiesEmptyTrustsNothing: set and empty, the server
|
|
||||||
// trusts no proxy. viper alone would read the empty variable as unset
|
|
||||||
// and give the default.
|
|
||||||
func TestTrustedProxiesEmptyTrustsNothing(t *testing.T) {
|
|
||||||
t.Setenv("TRUSTED_PROXIES", "")
|
|
||||||
|
|
||||||
requireTrustedProxies(t, []string{})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestTrustedProxiesListReplacesDefault: a list is used as given, with
|
|
||||||
// none of the default ranges added.
|
|
||||||
func TestTrustedProxiesListReplacesDefault(t *testing.T) {
|
|
||||||
t.Setenv("TRUSTED_PROXIES", "172.17.0.1/32, 2001:db8::/32")
|
|
||||||
|
|
||||||
requireTrustedProxies(t, []string{"172.17.0.1/32", "2001:db8::/32"})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSettingsLoadAsGiven: valid values pass the checks and are used
|
// TestSettingsLoadAsGiven: valid values pass the checks and are used
|
||||||
// as given. bin/entrypoint.sh starts the server with these
|
// as given. bin/entrypoint.sh starts the server with these
|
||||||
// BIND_ADDRESS and PORT values.
|
// BIND_ADDRESS and PORT values.
|
||||||
@@ -5,9 +5,7 @@ package handlers
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"mime"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
|
|
||||||
"sneak.berlin/go/netwatch/internal/globals"
|
"sneak.berlin/go/netwatch/internal/globals"
|
||||||
@@ -20,10 +18,6 @@ import (
|
|||||||
|
|
||||||
const jsonContentType = "application/json; charset=utf-8"
|
const jsonContentType = "application/json; charset=utf-8"
|
||||||
|
|
||||||
// errNotJSON is what decodeJSON returns for a request whose
|
|
||||||
// Content-Type is not application/json.
|
|
||||||
var errNotJSON = errors.New("content type is not application/json")
|
|
||||||
|
|
||||||
// reportAppender is the subset of the report buffer the handlers
|
// reportAppender is the subset of the report buffer the handlers
|
||||||
// depend on. Defining it here keeps the storage failure path
|
// depend on. Defining it here keeps the storage failure path
|
||||||
// exercisable with a stub in tests.
|
// exercisable with a stub in tests.
|
||||||
@@ -86,25 +80,14 @@ func (s *Handlers) respondJSON(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// decodeJSON decodes the request body into v. It returns errNotJSON,
|
// decodeJSON decodes the request body into v. The body is
|
||||||
// reading nothing, unless the request's Content-Type is
|
// expected to already be bounded by the body-size middleware, so
|
||||||
// application/json, with or without parameters such as charset. The
|
// a caller can distinguish an over-limit body from malformed
|
||||||
// body is expected to already be bounded by the body-size middleware,
|
// JSON by testing the returned error for *http.MaxBytesError.
|
||||||
// so a caller can distinguish an over-limit body from malformed JSON
|
|
||||||
// by testing the returned error for *http.MaxBytesError.
|
|
||||||
func (s *Handlers) decodeJSON(
|
func (s *Handlers) decodeJSON(
|
||||||
_ http.ResponseWriter,
|
_ http.ResponseWriter,
|
||||||
r *http.Request,
|
r *http.Request,
|
||||||
v any,
|
v any,
|
||||||
) error {
|
) error {
|
||||||
// A page on another origin can POST text/plain or a form without
|
|
||||||
// a CORS preflight, so its visitors' browsers would send it
|
|
||||||
// whatever the CORS settings. A JSON POST waits for the
|
|
||||||
// preflight, so only JSON is taken.
|
|
||||||
mediaType, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
|
|
||||||
if err != nil || mediaType != "application/json" {
|
|
||||||
return errNotJSON
|
|
||||||
}
|
|
||||||
|
|
||||||
return json.NewDecoder(r.Body).Decode(v)
|
return json.NewDecoder(r.Body).Decode(v)
|
||||||
}
|
}
|
||||||
@@ -30,7 +30,7 @@ type report struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// HandleReport returns a handler that accepts telemetry
|
// HandleReport returns a handler that accepts telemetry
|
||||||
// reports from NetWatch clients, sent as application/json.
|
// reports from NetWatch clients.
|
||||||
func (s *Handlers) HandleReport() http.HandlerFunc {
|
func (s *Handlers) HandleReport() http.HandlerFunc {
|
||||||
type response struct {
|
type response struct {
|
||||||
Status string `json:"status"`
|
Status string `json:"status"`
|
||||||
@@ -66,15 +66,9 @@ func (s *Handlers) HandleReport() http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// decodeErrorStatus logs a report decode failure and returns the
|
// decodeErrorStatus logs a report decode failure and returns the
|
||||||
// status to send: 415 when the report was not sent as JSON, 413 when
|
// status to send: 413 when the body exceeded the size limit,
|
||||||
// the body exceeded the size limit, otherwise 400 for malformed JSON.
|
// otherwise 400 for malformed JSON.
|
||||||
func (s *Handlers) decodeErrorStatus(err error) int {
|
func (s *Handlers) decodeErrorStatus(err error) int {
|
||||||
if errors.Is(err, errNotJSON) {
|
|
||||||
s.log.Warn("report refused: Content-Type is not application/json")
|
|
||||||
|
|
||||||
return http.StatusUnsupportedMediaType
|
|
||||||
}
|
|
||||||
|
|
||||||
var tooLarge *http.MaxBytesError
|
var tooLarge *http.MaxBytesError
|
||||||
if errors.As(err, &tooLarge) {
|
if errors.As(err, &tooLarge) {
|
||||||
s.log.Warn("report body too large", "limit_bytes", tooLarge.Limit)
|
s.log.Warn("report body too large", "limit_bytes", tooLarge.Limit)
|
||||||
@@ -93,7 +87,7 @@ func (s *Handlers) decodeErrorStatus(err error) int {
|
|||||||
|
|
||||||
// appendErrorStatus logs a failure to store a report and returns
|
// appendErrorStatus logs a failure to store a report and returns
|
||||||
// the status to send: 507 when the reports waiting to be written fill
|
// the status to send: 507 when the reports waiting to be written fill
|
||||||
// the size cap, otherwise 500, as when report files cannot be written.
|
// the size cap, otherwise 500.
|
||||||
func (s *Handlers) appendErrorStatus(err error) int {
|
func (s *Handlers) appendErrorStatus(err error) int {
|
||||||
if errors.Is(err, reportbuf.ErrFull) {
|
if errors.Is(err, reportbuf.ErrFull) {
|
||||||
s.log.Warn("report refused: " +
|
s.log.Warn("report refused: " +
|
||||||
@@ -27,17 +27,6 @@ type stubAppender struct {
|
|||||||
|
|
||||||
func (s stubAppender) Append(any) error { return s.err }
|
func (s stubAppender) Append(any) error { return s.err }
|
||||||
|
|
||||||
// recordingAppender keeps every report the handler stores.
|
|
||||||
type recordingAppender struct {
|
|
||||||
reports []any
|
|
||||||
}
|
|
||||||
|
|
||||||
func (a *recordingAppender) Append(v any) error {
|
|
||||||
a.reports = append(a.reports, v)
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func newTestHandlers(buf stubAppender, out io.Writer) *handlers.Handlers {
|
func newTestHandlers(buf stubAppender, out io.Writer) *handlers.Handlers {
|
||||||
return handlers.NewForTest(buf, slog.New(slog.NewJSONHandler(out, nil)))
|
return handlers.NewForTest(buf, slog.New(slog.NewJSONHandler(out, nil)))
|
||||||
}
|
}
|
||||||
@@ -94,7 +83,6 @@ func TestHandleReportAcceptsValidReports(t *testing.T) {
|
|||||||
http.MethodPost, "/api/v1/reports",
|
http.MethodPost, "/api/v1/reports",
|
||||||
strings.NewReader(tt.body),
|
strings.NewReader(tt.body),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
h.HandleReport().ServeHTTP(rec, req)
|
||||||
|
|
||||||
@@ -115,89 +103,6 @@ func TestHandleReportAcceptsValidReports(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandleReportStoresOnlyJSON checks that a report is stored only
|
|
||||||
// when sent as application/json, as the page sends it. One sent as
|
|
||||||
// text/plain or as a form, which a page on another origin can send
|
|
||||||
// without a CORS preflight, is refused with 415.
|
|
||||||
func TestHandleReportStoresOnlyJSON(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
contentType string
|
|
||||||
wantStatus int
|
|
||||||
wantStored int
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "text/plain",
|
|
||||||
contentType: "text/plain;charset=UTF-8",
|
|
||||||
wantStatus: http.StatusUnsupportedMediaType,
|
|
||||||
wantStored: 0,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "a urlencoded form",
|
|
||||||
contentType: "application/x-www-form-urlencoded",
|
|
||||||
wantStatus: http.StatusUnsupportedMediaType,
|
|
||||||
wantStored: 0,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "a multipart form",
|
|
||||||
contentType: "multipart/form-data; boundary=x",
|
|
||||||
wantStatus: http.StatusUnsupportedMediaType,
|
|
||||||
wantStored: 0,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "no Content-Type",
|
|
||||||
contentType: "",
|
|
||||||
wantStatus: http.StatusUnsupportedMediaType,
|
|
||||||
wantStored: 0,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "application/json",
|
|
||||||
contentType: "application/json",
|
|
||||||
wantStatus: http.StatusOK,
|
|
||||||
wantStored: 1,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "application/json with a charset",
|
|
||||||
contentType: "application/json;charset=UTF-8",
|
|
||||||
wantStatus: http.StatusOK,
|
|
||||||
wantStored: 1,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
var stored recordingAppender
|
|
||||||
|
|
||||||
h := handlers.NewForTest(&stored, slog.New(slog.DiscardHandler))
|
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(),
|
|
||||||
http.MethodPost, "/api/v1/reports",
|
|
||||||
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
|
|
||||||
)
|
|
||||||
|
|
||||||
if tt.contentType != "" {
|
|
||||||
req.Header.Set("Content-Type", tt.contentType)
|
|
||||||
}
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
|
||||||
|
|
||||||
if rec.Code != tt.wantStatus {
|
|
||||||
t.Errorf("status = %d, want %d", rec.Code, tt.wantStatus)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(stored.reports) != tt.wantStored {
|
|
||||||
t.Errorf("reports stored = %d, want %d",
|
|
||||||
len(stored.reports), tt.wantStored)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestHandleReportStorageFailureIsNon2xx(t *testing.T) {
|
func TestHandleReportStorageFailureIsNon2xx(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -208,7 +113,6 @@ func TestHandleReportStorageFailureIsNon2xx(t *testing.T) {
|
|||||||
http.MethodPost, "/api/v1/reports",
|
http.MethodPost, "/api/v1/reports",
|
||||||
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
|
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
h.HandleReport().ServeHTTP(rec, req)
|
||||||
|
|
||||||
@@ -234,7 +138,6 @@ func TestHandleReportFullIs507(t *testing.T) {
|
|||||||
http.MethodPost, "/api/v1/reports",
|
http.MethodPost, "/api/v1/reports",
|
||||||
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
|
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
h.HandleReport().ServeHTTP(rec, req)
|
||||||
|
|
||||||
@@ -258,7 +161,6 @@ func TestHandleReportMalformedJSONIs400(t *testing.T) {
|
|||||||
http.MethodPost, "/api/v1/reports",
|
http.MethodPost, "/api/v1/reports",
|
||||||
strings.NewReader(`{not json`),
|
strings.NewReader(`{not json`),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
h.HandleReport().ServeHTTP(rec, req)
|
||||||
|
|
||||||
@@ -283,8 +185,6 @@ func TestHandleReportOversizeIs413(t *testing.T) {
|
|||||||
http.MethodPost, "/api/v1/reports",
|
http.MethodPost, "/api/v1/reports",
|
||||||
strings.NewReader(`{"clientId":"`+strings.Repeat("x", 200)+`"}`),
|
strings.NewReader(`{"clientId":"`+strings.Repeat("x", 200)+`"}`),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
// No declared length, so only the middleware's read cap can
|
// No declared length, so only the middleware's read cap can
|
||||||
// stop this body.
|
// stop this body.
|
||||||
req.ContentLength = -1
|
req.ContentLength = -1
|
||||||
@@ -313,7 +213,6 @@ func TestHandleReportDoesNotLogRawGeo(t *testing.T) {
|
|||||||
`{"clientId":"c1","geo":{"raw":"`+sentinel+`"},"hosts":[]}`,
|
`{"clientId":"c1","geo":{"raw":"`+sentinel+`"},"hosts":[]}`,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
h.HandleReport().ServeHTTP(rec, req)
|
||||||
|
|
||||||
@@ -346,7 +245,6 @@ func TestHandleReportLogsClientIDCutToBound(t *testing.T) {
|
|||||||
`{"clientId":"`+long+`","timestamp":"`+long+`","hosts":[]}`,
|
`{"clientId":"`+long+`","timestamp":"`+long+`","hosts":[]}`,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
h.HandleReport().ServeHTTP(rec, req)
|
||||||
|
|
||||||
@@ -386,7 +284,6 @@ func TestHandleReportDecodeErrorLogIsBounded(t *testing.T) {
|
|||||||
http.MethodPost, "/api/v1/reports",
|
http.MethodPost, "/api/v1/reports",
|
||||||
strings.NewReader(`{"hosts":[{"history":[{"t":`+huge+`}]}]}`),
|
strings.NewReader(`{"hosts":[{"history":[{"t":`+huge+`}]}]}`),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
h.HandleReport().ServeHTTP(rec, req)
|
||||||
|
|
||||||
@@ -297,16 +297,12 @@ func (s *Middleware) MaxBodyBytes(
|
|||||||
|
|
||||||
// Recoverer returns middleware that recovers from a panic in a
|
// Recoverer returns middleware that recovers from a panic in a
|
||||||
// downstream handler, logs the panic and stack trace through
|
// downstream handler, logs the panic and stack trace through
|
||||||
// slog, and responds 500 with no body, unless a status was already
|
// slog, and responds 500 with no body. http.ErrAbortHandler is
|
||||||
// written, as Write500OnPanic writes one for a handler's panic.
|
// re-panicked so the server can abort the response as intended.
|
||||||
// http.ErrAbortHandler is re-panicked so the server can abort the
|
|
||||||
// response as intended.
|
|
||||||
func (s *Middleware) Recoverer() func(http.Handler) http.Handler {
|
func (s *Middleware) Recoverer() func(http.Handler) http.Handler {
|
||||||
return func(next http.Handler) http.Handler {
|
return func(next http.Handler) http.Handler {
|
||||||
return http.HandlerFunc(
|
return http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
ww := middleware.NewWrapResponseWriter(w, r.ProtoMajor)
|
|
||||||
|
|
||||||
defer func() {
|
defer func() {
|
||||||
rec := recover()
|
rec := recover()
|
||||||
if rec == nil {
|
if rec == nil {
|
||||||
@@ -324,43 +320,10 @@ func (s *Middleware) Recoverer() func(http.Handler) http.Handler {
|
|||||||
"stack", string(debug.Stack()),
|
"stack", string(debug.Stack()),
|
||||||
)
|
)
|
||||||
|
|
||||||
if ww.Status() == 0 {
|
w.WriteHeader(http.StatusInternalServerError)
|
||||||
ww.WriteHeader(http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
}()
|
}()
|
||||||
|
|
||||||
next.ServeHTTP(ww, r)
|
next.ServeHTTP(w, r)
|
||||||
},
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Write500OnPanic returns middleware that writes 500 when a downstream
|
|
||||||
// handler panics before writing a status, and lets the panic go on to
|
|
||||||
// Sentry and Recoverer. Recoverer, which comes first, would write its
|
|
||||||
// 500 only after the request log and the metrics had recorded the
|
|
||||||
// status they saw; placed after each of them, this makes them record
|
|
||||||
// the 500.
|
|
||||||
func (s *Middleware) Write500OnPanic() func(http.Handler) http.Handler {
|
|
||||||
return func(next http.Handler) http.Handler {
|
|
||||||
return http.HandlerFunc(
|
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
ww := middleware.NewWrapResponseWriter(w, r.ProtoMajor)
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
rec := recover()
|
|
||||||
if rec == nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if ww.Status() == 0 {
|
|
||||||
ww.WriteHeader(http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
|
|
||||||
panic(rec)
|
|
||||||
}()
|
|
||||||
|
|
||||||
next.ServeHTTP(ww, r)
|
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
+1
-170
@@ -4,7 +4,6 @@ import (
|
|||||||
"bytes"
|
"bytes"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"log"
|
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
@@ -17,9 +16,7 @@ import (
|
|||||||
"sneak.berlin/go/netwatch/internal/logger"
|
"sneak.berlin/go/netwatch/internal/logger"
|
||||||
"sneak.berlin/go/netwatch/internal/middleware"
|
"sneak.berlin/go/netwatch/internal/middleware"
|
||||||
|
|
||||||
"github.com/go-chi/chi/v5"
|
|
||||||
chimiddleware "github.com/go-chi/chi/v5/middleware"
|
chimiddleware "github.com/go-chi/chi/v5/middleware"
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -326,172 +323,6 @@ func TestRecovererRepanicsOnAbortHandler(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestHandlerPanicIsLoggedAndCountedAs500 sends a request to a handler
|
|
||||||
// that panics through the middleware SetupRoutes puts around a route
|
|
||||||
// with metrics, in its order, leaving out those that write no status
|
|
||||||
// for a panic and Sentry, which TestSentry covers. The request log line
|
|
||||||
// and the metrics must give the 500 the client gets, and the panic must
|
|
||||||
// still reach Recoverer, which logs it.
|
|
||||||
func TestHandlerPanicIsLoggedAndCountedAs500(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
var logbuf bytes.Buffer
|
|
||||||
|
|
||||||
mw := middleware.NewWithLogger(
|
|
||||||
slog.New(slog.NewJSONHandler(&logbuf, nil)),
|
|
||||||
)
|
|
||||||
registry := prometheus.NewRegistry()
|
|
||||||
|
|
||||||
handler := chi.Chain(
|
|
||||||
mw.Recoverer(),
|
|
||||||
chimiddleware.RequestID,
|
|
||||||
mw.Logging(),
|
|
||||||
mw.Write500OnPanic(),
|
|
||||||
mw.Metrics(registry),
|
|
||||||
mw.Write500OnPanic(),
|
|
||||||
).HandlerFunc(func(http.ResponseWriter, *http.Request) {
|
|
||||||
panic("boom")
|
|
||||||
})
|
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
|
||||||
handler.ServeHTTP(rec, req)
|
|
||||||
|
|
||||||
if rec.Code != http.StatusInternalServerError {
|
|
||||||
t.Fatalf("status = %d, want %d",
|
|
||||||
rec.Code, http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
|
|
||||||
request := findLogLine(t, logbuf.Bytes(), "request")
|
|
||||||
if request.Status != http.StatusInternalServerError {
|
|
||||||
t.Errorf("request log status = %d, want %d",
|
|
||||||
request.Status, http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
|
|
||||||
recovered := findLogLine(t, logbuf.Bytes(), "panic recovered")
|
|
||||||
if recovered.Panic != "boom" {
|
|
||||||
t.Errorf("panic recovered line has panic %q, want %q",
|
|
||||||
recovered.Panic, "boom")
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, name := range []string{
|
|
||||||
"http_request_duration_seconds", "http_response_size_bytes",
|
|
||||||
} {
|
|
||||||
codes := metricCodes(t, registry, name)
|
|
||||||
if len(codes) != 1 || codes[0] != "500" {
|
|
||||||
t.Errorf("%s code labels = %v, want [500]", name, codes)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// loggedLine holds the fields the tests read from a JSON log line.
|
|
||||||
type loggedLine struct {
|
|
||||||
Msg string `json:"msg"`
|
|
||||||
Status int `json:"status"`
|
|
||||||
Panic string `json:"panic"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// findLogLine returns the first of the JSON log lines in logged whose
|
|
||||||
// message is msg.
|
|
||||||
func findLogLine(t *testing.T, logged []byte, msg string) loggedLine {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
dec := json.NewDecoder(bytes.NewReader(logged))
|
|
||||||
|
|
||||||
for {
|
|
||||||
var line loggedLine
|
|
||||||
|
|
||||||
err := dec.Decode(&line)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("no %q line in the log: %v (%q)", msg, err, logged)
|
|
||||||
}
|
|
||||||
|
|
||||||
if line.Msg == msg {
|
|
||||||
return line
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// metricCodes returns the code label of each series of the metric
|
|
||||||
// family name in registry.
|
|
||||||
func metricCodes(
|
|
||||||
t *testing.T,
|
|
||||||
registry *prometheus.Registry,
|
|
||||||
name string,
|
|
||||||
) []string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
families, err := registry.Gather()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("gather metrics: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var codes []string
|
|
||||||
|
|
||||||
for _, family := range families {
|
|
||||||
if family.GetName() != name {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, metric := range family.GetMetric() {
|
|
||||||
for _, label := range metric.GetLabel() {
|
|
||||||
if label.GetName() == "code" {
|
|
||||||
codes = append(codes, label.GetValue())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return codes
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestRecovererWritesNoSecondStatus checks, on a real server, that
|
|
||||||
// Recoverer does not write its 500 again after Write500OnPanic has
|
|
||||||
// written one, which the server would log as a superfluous
|
|
||||||
// WriteHeader call.
|
|
||||||
func TestRecovererWritesNoSecondStatus(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
var serverLog bytes.Buffer
|
|
||||||
|
|
||||||
mw := middleware.NewWithLogger(slog.New(slog.DiscardHandler))
|
|
||||||
|
|
||||||
server := httptest.NewUnstartedServer(chi.Chain(
|
|
||||||
mw.Recoverer(),
|
|
||||||
mw.Write500OnPanic(),
|
|
||||||
).HandlerFunc(func(http.ResponseWriter, *http.Request) {
|
|
||||||
panic("boom")
|
|
||||||
}))
|
|
||||||
server.Config.ErrorLog = log.New(&serverLog, "", 0)
|
|
||||||
server.Start()
|
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(t.Context(),
|
|
||||||
http.MethodGet, server.URL, http.NoBody)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("new request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := server.Client().Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_ = resp.Body.Close()
|
|
||||||
|
|
||||||
// Close waits for the handler to return, so whatever the server
|
|
||||||
// logs for this request is in serverLog after it.
|
|
||||||
server.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusInternalServerError {
|
|
||||||
t.Errorf("status = %d, want %d",
|
|
||||||
resp.StatusCode, http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
|
|
||||||
if serverLog.Len() != 0 {
|
|
||||||
t.Errorf("server logged %q", serverLog.String())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestLoggingCutsRequestStringsToBound sends an over-long URL and
|
// TestLoggingCutsRequestStringsToBound sends an over-long URL and
|
||||||
// over-long header values, and checks the request log writes each
|
// over-long header values, and checks the request log writes each
|
||||||
// one cut to logger.MaxLoggedFieldBytes.
|
// one cut to logger.MaxLoggedFieldBytes.
|
||||||
@@ -512,7 +343,7 @@ func TestLoggingCutsRequestStringsToBound(t *testing.T) {
|
|||||||
http.MethodGet, "/"+long, http.NoBody)
|
http.MethodGet, "/"+long, http.NoBody)
|
||||||
req.Header.Set("User-Agent", long)
|
req.Header.Set("User-Agent", long)
|
||||||
req.Header.Set("Referer", long)
|
req.Header.Set("Referer", long)
|
||||||
req.Header.Set("X-Request-ID", long)
|
req.Header.Set("X-Request-Id", long)
|
||||||
|
|
||||||
handler.ServeHTTP(httptest.NewRecorder(), req)
|
handler.ServeHTTP(httptest.NewRecorder(), req)
|
||||||
|
|
||||||
@@ -91,15 +91,6 @@ type Buffer struct {
|
|||||||
// of the report files in dataDir, plus the reports waiting to
|
// of the report files in dataDir, plus the reports waiting to
|
||||||
// be written to one at their uncompressed size.
|
// be written to one at their uncompressed size.
|
||||||
usedBytes int64
|
usedBytes int64
|
||||||
// writeErr is the error of the last write of a report file to
|
|
||||||
// finish, nil if it succeeded or none has. While it is set, Append
|
|
||||||
// refuses reports, and the reports of that write wait in buf for
|
|
||||||
// the next flush.
|
|
||||||
writeErr error
|
|
||||||
// writers are the goroutines that write report files: flushLoop,
|
|
||||||
// and each one Append starts at flushSizeThreshold. OnStop waits
|
|
||||||
// for them before the final flush.
|
|
||||||
writers sync.WaitGroup
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// New creates a Buffer and registers lifecycle hooks to
|
// New creates a Buffer and registers lifecycle hooks to
|
||||||
@@ -151,7 +142,7 @@ func New(
|
|||||||
|
|
||||||
b.mu.Unlock()
|
b.mu.Unlock()
|
||||||
|
|
||||||
b.writers.Go(b.flushLoop)
|
go b.flushLoop()
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
},
|
},
|
||||||
@@ -162,14 +153,7 @@ func New(
|
|||||||
var err error
|
var err error
|
||||||
|
|
||||||
b.stopOnce.Do(func() {
|
b.stopOnce.Do(func() {
|
||||||
// Closing done ends flushLoop after any flush it is
|
|
||||||
// running. Wait for it and for the writes Append
|
|
||||||
// started, which the process exiting would cut off;
|
|
||||||
// one that fails puts its reports back in buf, for the
|
|
||||||
// final flush to write.
|
|
||||||
close(b.done)
|
close(b.done)
|
||||||
b.writers.Wait()
|
|
||||||
|
|
||||||
err = b.flushLocked()
|
err = b.flushLocked()
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -183,12 +167,11 @@ func New(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Append marshals v as a single JSON line and appends it to
|
// Append marshals v as a single JSON line and appends it to
|
||||||
// the buffer. It stores nothing and returns an error if the last
|
// the buffer. If the line would take usedBytes past maxBytes, the
|
||||||
// write of a report file failed, until a write succeeds again. If the
|
// oldest report files are deleted to make room; it stores nothing
|
||||||
// line would take usedBytes past maxBytes, the oldest report files are
|
// and returns ErrFull if that cannot make room. If the buffer
|
||||||
// deleted to make room; it stores nothing and returns ErrFull if that
|
// reaches the size threshold, it is drained and written to disk
|
||||||
// cannot make room. If the buffer reaches the size threshold, it is
|
// asynchronously.
|
||||||
// drained and written to disk asynchronously.
|
|
||||||
func (b *Buffer) Append(v any) error {
|
func (b *Buffer) Append(v any) error {
|
||||||
line, err := json.Marshal(v)
|
line, err := json.Marshal(v)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -199,13 +182,6 @@ func (b *Buffer) Append(v any) error {
|
|||||||
|
|
||||||
b.mu.Lock()
|
b.mu.Lock()
|
||||||
|
|
||||||
if b.writeErr != nil {
|
|
||||||
err = fmt.Errorf("report files cannot be written: %w", b.writeErr)
|
|
||||||
b.mu.Unlock()
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
b.deleteOldestFiles(lineBytes)
|
b.deleteOldestFiles(lineBytes)
|
||||||
|
|
||||||
if b.usedBytes+lineBytes > b.maxBytes {
|
if b.usedBytes+lineBytes > b.maxBytes {
|
||||||
@@ -222,12 +198,12 @@ func (b *Buffer) Append(v any) error {
|
|||||||
data := b.drainBuf()
|
data := b.drainBuf()
|
||||||
b.mu.Unlock()
|
b.mu.Unlock()
|
||||||
|
|
||||||
b.writers.Go(func() {
|
go func() {
|
||||||
writeErr := b.writeFile(data)
|
writeErr := b.writeFile(data)
|
||||||
if writeErr != nil {
|
if writeErr != nil {
|
||||||
b.log.Error("flush reports failed", "error", writeErr)
|
b.log.Error("flush reports failed", "error", writeErr)
|
||||||
}
|
}
|
||||||
})
|
}()
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -315,8 +291,7 @@ func (b *Buffer) drainBuf() []byte {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// writeFile writes data, reports drained from the buffer, to a new
|
// writeFile writes data, reports drained from the buffer, to a new
|
||||||
// timestamped zstd-compressed JSONL file in the data directory. If the
|
// timestamped zstd-compressed JSONL file in the data directory.
|
||||||
// write fails, it puts the reports back in the buffer.
|
|
||||||
func (b *Buffer) writeFile(data []byte) error {
|
func (b *Buffer) writeFile(data []byte) error {
|
||||||
// The timestamp comes first, so the names sort by time; the number
|
// The timestamp comes first, so the names sort by time; the number
|
||||||
// after it tells apart files named in the same millisecond.
|
// after it tells apart files named in the same millisecond.
|
||||||
@@ -325,27 +300,19 @@ func (b *Buffer) writeFile(data []byte) error {
|
|||||||
|
|
||||||
size, err := b.createFile(filepath.Join(b.dataDir, name), data)
|
size, err := b.createFile(filepath.Join(b.dataDir, name), data)
|
||||||
|
|
||||||
|
// The reports no longer wait to be written, so they stop counting
|
||||||
|
// at their uncompressed size. If the write failed they are lost;
|
||||||
|
// otherwise they count as the file, which from here on may be
|
||||||
|
// deleted to make room.
|
||||||
b.mu.Lock()
|
b.mu.Lock()
|
||||||
defer b.mu.Unlock()
|
defer b.mu.Unlock()
|
||||||
|
|
||||||
b.writeErr = err
|
b.usedBytes -= int64(len(data))
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
// The reports go back in front of any appended since, for the
|
|
||||||
// next flush to write, and still count at their uncompressed
|
|
||||||
// size.
|
|
||||||
appended := bytes.Clone(b.buf.Bytes())
|
|
||||||
b.buf.Reset()
|
|
||||||
b.buf.Write(data)
|
|
||||||
b.buf.Write(appended)
|
|
||||||
|
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
// The reports no longer wait to be written, so they stop counting
|
|
||||||
// at their uncompressed size and count as the file, which from here
|
|
||||||
// on may be deleted to make room.
|
|
||||||
b.usedBytes -= int64(len(data))
|
|
||||||
b.usedBytes += size
|
b.usedBytes += size
|
||||||
b.filesBytes += size
|
b.filesBytes += size
|
||||||
|
|
||||||
+26
-424
@@ -5,8 +5,6 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io/fs"
|
"io/fs"
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"slices"
|
"slices"
|
||||||
@@ -15,13 +13,10 @@ import (
|
|||||||
"sync"
|
"sync"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"testing"
|
"testing"
|
||||||
"testing/synctest"
|
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/netwatch/internal/config"
|
"sneak.berlin/go/netwatch/internal/config"
|
||||||
"sneak.berlin/go/netwatch/internal/globals"
|
"sneak.berlin/go/netwatch/internal/globals"
|
||||||
"sneak.berlin/go/netwatch/internal/handlers"
|
|
||||||
"sneak.berlin/go/netwatch/internal/healthcheck"
|
|
||||||
"sneak.berlin/go/netwatch/internal/logger"
|
"sneak.berlin/go/netwatch/internal/logger"
|
||||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||||
|
|
||||||
@@ -108,174 +103,9 @@ func TestFailedFinalFlushFailsStop(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestStopWaitsForPeriodicWrite holds open the write the periodic flush
|
// startBuffer starts a Buffer through fx, as main does, with the
|
||||||
// starts, then stops the app. The stop must wait until the write is
|
// DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set.
|
||||||
// released, and the report must then be in its file. The test runs in
|
func startBuffer(t *testing.T) *reportbuf.Buffer {
|
||||||
// a synctest bubble, whose clock moves on at once when every goroutine
|
|
||||||
// in it is waiting, so the periodic flush does not take a minute.
|
|
||||||
func TestStopWaitsForPeriodicWrite(t *testing.T) {
|
|
||||||
dir := t.TempDir()
|
|
||||||
t.Setenv("DATA_DIR", dir)
|
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
|
||||||
buf, app := startApp(t)
|
|
||||||
|
|
||||||
created := make(chan struct{})
|
|
||||||
release := make(chan struct{})
|
|
||||||
|
|
||||||
buf.OnFileCreated(func(*os.File) {
|
|
||||||
created <- struct{}{}
|
|
||||||
|
|
||||||
<-release
|
|
||||||
})
|
|
||||||
|
|
||||||
err := buf.Append(map[string]string{"id": "periodic"})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("append report: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
<-created
|
|
||||||
|
|
||||||
err = stopWhileHeld(t, app, release)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("stop: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
files, err := readReportFiles(dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read report files: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
want := `{"id":"periodic"}` + "\n"
|
|
||||||
if len(files) != 1 || files[0] != want {
|
|
||||||
t.Fatalf("report files = %q, want one holding %q", files, want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestStopWaitsForWriteAtSizeThreshold holds open the write Append
|
|
||||||
// starts when the buffer reaches FlushSizeThreshold, then stops the
|
|
||||||
// app. The stop must wait until the write is released, and the reports
|
|
||||||
// must then be in its file.
|
|
||||||
func TestStopWaitsForWriteAtSizeThreshold(t *testing.T) {
|
|
||||||
dir := t.TempDir()
|
|
||||||
t.Setenv("DATA_DIR", dir)
|
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
|
||||||
buf, app := startApp(t)
|
|
||||||
|
|
||||||
created := make(chan struct{})
|
|
||||||
release := make(chan struct{})
|
|
||||||
|
|
||||||
buf.OnFileCreated(func(*os.File) {
|
|
||||||
created <- struct{}{}
|
|
||||||
|
|
||||||
<-release
|
|
||||||
})
|
|
||||||
|
|
||||||
appended := appendToThreshold(t, buf)
|
|
||||||
|
|
||||||
<-created
|
|
||||||
|
|
||||||
err := stopWhileHeld(t, app, release)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("stop: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
files, err := readReportFiles(dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read report files: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(files) != 1 || files[0] != appended {
|
|
||||||
t.Fatalf("%d report files, want one holding the %d bytes "+
|
|
||||||
"appended", len(files), len(appended))
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFinalFlushWritesReportsOfFailedWrite holds open the write the
|
|
||||||
// periodic flush starts, makes it fail, then stops the app. The stop
|
|
||||||
// waits for the write, which puts its report back in the buffer, and
|
|
||||||
// the final flush writes it, so the stop succeeds with nothing lost.
|
|
||||||
func TestFinalFlushWritesReportsOfFailedWrite(t *testing.T) {
|
|
||||||
dir := t.TempDir()
|
|
||||||
t.Setenv("DATA_DIR", dir)
|
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
|
||||||
buf, app := startApp(t)
|
|
||||||
|
|
||||||
created := make(chan *os.File)
|
|
||||||
release := make(chan struct{})
|
|
||||||
|
|
||||||
buf.OnFileCreated(func(f *os.File) {
|
|
||||||
created <- f
|
|
||||||
|
|
||||||
<-release
|
|
||||||
})
|
|
||||||
|
|
||||||
err := buf.Append(map[string]string{"id": "failed"})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("append report: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
held := <-created
|
|
||||||
|
|
||||||
// Closing the file under the write makes the write fail.
|
|
||||||
_ = held.Close()
|
|
||||||
|
|
||||||
// The final flush's write goes through.
|
|
||||||
buf.OnFileCreated(func(*os.File) {})
|
|
||||||
|
|
||||||
err = stopWhileHeld(t, app, release)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("stop: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
files, err := readReportFiles(dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read report files: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
want := `{"id":"failed"}` + "\n"
|
|
||||||
if len(files) != 1 || files[0] != want {
|
|
||||||
t.Fatalf("report files = %q, want one holding %q", files, want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// stopWhileHeld stops app while a report file write is held open until
|
|
||||||
// release is closed, and fails the test if the stop returns before
|
|
||||||
// that. It then closes release and returns the stop's error. It must
|
|
||||||
// be called in a synctest bubble.
|
|
||||||
func stopWhileHeld(
|
|
||||||
t *testing.T,
|
|
||||||
app *fxtest.App,
|
|
||||||
release chan<- struct{},
|
|
||||||
) error {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
stopped := make(chan error, 1)
|
|
||||||
|
|
||||||
go func() { stopped <- app.Stop(t.Context()) }()
|
|
||||||
|
|
||||||
// Wait returns once every other goroutine in the bubble is waiting:
|
|
||||||
// the stop has got as far as it can while the write is held.
|
|
||||||
synctest.Wait()
|
|
||||||
|
|
||||||
if len(stopped) != 0 {
|
|
||||||
t.Error("stop returned while a report file write was held open")
|
|
||||||
}
|
|
||||||
|
|
||||||
close(release)
|
|
||||||
|
|
||||||
return <-stopped
|
|
||||||
}
|
|
||||||
|
|
||||||
// startApp starts a Buffer through fx, as main does, with the DATA_DIR
|
|
||||||
// and DATA_DIR_MAX_BYTES the calling test has set, and returns it with
|
|
||||||
// the app, for tests that stop the app themselves.
|
|
||||||
func startApp(t *testing.T) (*reportbuf.Buffer, *fxtest.App) {
|
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
var buf *reportbuf.Buffer
|
var buf *reportbuf.Buffer
|
||||||
@@ -293,16 +123,6 @@ func startApp(t *testing.T) (*reportbuf.Buffer, *fxtest.App) {
|
|||||||
app.RequireStart()
|
app.RequireStart()
|
||||||
t.Cleanup(app.RequireStop)
|
t.Cleanup(app.RequireStop)
|
||||||
|
|
||||||
return buf, app
|
|
||||||
}
|
|
||||||
|
|
||||||
// startBuffer starts a Buffer through fx, as main does, with the
|
|
||||||
// DATA_DIR and DATA_DIR_MAX_BYTES the calling test has set.
|
|
||||||
func startBuffer(t *testing.T) *reportbuf.Buffer {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
buf, _ := startApp(t)
|
|
||||||
|
|
||||||
return buf
|
return buf
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -687,17 +507,13 @@ func TestFileBeingWrittenIsNeverDeleted(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestFailedWriteKeepsItsReports makes a write fail once its file is
|
// TestFailedWriteStopsCounting makes a write fail once its file is
|
||||||
// created, under a cap with room for one report. The part of the file
|
// created. Its reports are lost, so they stop counting, and the part of
|
||||||
// written is removed, so it takes no room, and the report waits in
|
// the file written is removed, so it takes no room.
|
||||||
// memory for the next flush, which writes it. Once written, it counts
|
func TestFailedWriteStopsCounting(t *testing.T) {
|
||||||
// as its file and no longer as a report waiting, so the cap still holds
|
report := map[string]string{"id": "failed"}
|
||||||
// exactly: one more report is taken and the one after it is refused.
|
|
||||||
func TestFailedWriteKeepsItsReports(t *testing.T) {
|
|
||||||
report := map[string]string{"id": "kept"}
|
|
||||||
dir := t.TempDir()
|
|
||||||
|
|
||||||
t.Setenv("DATA_DIR", dir)
|
t.Setenv("DATA_DIR", t.TempDir())
|
||||||
t.Setenv("DATA_DIR_MAX_BYTES", strconv.Itoa(lineBytes(t, report)))
|
t.Setenv("DATA_DIR_MAX_BYTES", strconv.Itoa(lineBytes(t, report)))
|
||||||
|
|
||||||
buf := startBuffer(t)
|
buf := startBuffer(t)
|
||||||
@@ -728,213 +544,10 @@ func TestFailedWriteKeepsItsReports(t *testing.T) {
|
|||||||
// Writes from here on, the final one at stop included, succeed.
|
// Writes from here on, the final one at stop included, succeed.
|
||||||
buf.OnFileCreated(func(*os.File) {})
|
buf.OnFileCreated(func(*os.File) {})
|
||||||
|
|
||||||
err = buf.Flush()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("flush after the failed write: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
files, err := readReportFiles(dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read report files: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
want := `{"id":"kept"}` + "\n"
|
|
||||||
if len(files) != 1 || files[0] != want {
|
|
||||||
t.Fatalf("report files = %q, want one holding %q", files, want)
|
|
||||||
}
|
|
||||||
|
|
||||||
// The file is deleted to make room if need be.
|
|
||||||
err = buf.Append(report)
|
err = buf.Append(report)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("report after the written one: %v", err)
|
t.Fatalf("report after the failed write: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
err = buf.Append(report)
|
|
||||||
if !errors.Is(err, reportbuf.ErrFull) {
|
|
||||||
t.Fatalf("report past the cap: error = %v, want ErrFull", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFailedWriteKeepsReportsAppendedDuringIt holds a write open,
|
|
||||||
// appends a report while it is held, then makes the held write fail.
|
|
||||||
// Both reports are kept, and the next flush writes them in the order
|
|
||||||
// they were appended, the held write's first.
|
|
||||||
func TestFailedWriteKeepsReportsAppendedDuringIt(t *testing.T) {
|
|
||||||
dir := t.TempDir()
|
|
||||||
t.Setenv("DATA_DIR", dir)
|
|
||||||
|
|
||||||
buf := startBuffer(t)
|
|
||||||
|
|
||||||
created := make(chan *os.File)
|
|
||||||
release := make(chan struct{})
|
|
||||||
|
|
||||||
buf.OnFileCreated(func(f *os.File) {
|
|
||||||
created <- f
|
|
||||||
|
|
||||||
<-release
|
|
||||||
})
|
|
||||||
|
|
||||||
err := buf.Append(map[string]string{"id": "held"})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("report of the held write: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
flushed := make(chan error)
|
|
||||||
|
|
||||||
go func() { flushed <- buf.Flush() }()
|
|
||||||
|
|
||||||
held := <-created
|
|
||||||
|
|
||||||
// Errorf, not Fatalf, until the held write is released, so that a
|
|
||||||
// failure here does not leave it held.
|
|
||||||
err = buf.Append(map[string]string{"id": "appended"})
|
|
||||||
if err != nil {
|
|
||||||
t.Errorf("report appended while the write was held: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Closing the file under the write makes the write fail.
|
|
||||||
_ = held.Close()
|
|
||||||
|
|
||||||
close(release)
|
|
||||||
|
|
||||||
err = <-flushed
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("held write succeeded, though its file was closed under it")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Writes from here on, the final one at stop included, succeed.
|
|
||||||
buf.OnFileCreated(func(*os.File) {})
|
|
||||||
|
|
||||||
err = buf.Flush()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("flush after the failed write: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
files, err := readReportFiles(dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read report files: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
want := `{"id":"held"}` + "\n" + `{"id":"appended"}` + "\n"
|
|
||||||
if len(files) != 1 || files[0] != want {
|
|
||||||
t.Fatalf("report files = %q, want one holding %q", files, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestReportsRefusedWhileWritesFail posts reports through the report
|
|
||||||
// handler, as the page does. Once a write has failed, a report is
|
|
||||||
// refused with 500 until a write succeeds again; the reports taken
|
|
||||||
// before and after are stored, and the one refused is not.
|
|
||||||
func TestReportsRefusedWhileWritesFail(t *testing.T) {
|
|
||||||
dir := t.TempDir()
|
|
||||||
t.Setenv("DATA_DIR", dir)
|
|
||||||
|
|
||||||
buf, h := startBufferWithHandlers(t)
|
|
||||||
|
|
||||||
if code := postReport(t, h, "before"); code != http.StatusOK {
|
|
||||||
t.Fatalf("report before the failed write: status = %d, want %d",
|
|
||||||
code, http.StatusOK)
|
|
||||||
}
|
|
||||||
|
|
||||||
// With the data directory gone, the write has nowhere to go.
|
|
||||||
err := os.Remove(dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("remove data dir: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = buf.Flush()
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("flush succeeded with the data directory gone")
|
|
||||||
}
|
|
||||||
|
|
||||||
if code := postReport(t, h, "refused"); code != http.StatusInternalServerError {
|
|
||||||
t.Fatalf("report after the failed write: status = %d, want %d",
|
|
||||||
code, http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = os.Mkdir(dir, 0o750)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("make data dir: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = buf.Flush()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("flush with the data directory back: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if code := postReport(t, h, "after"); code != http.StatusOK {
|
|
||||||
t.Fatalf("report after a write succeeded: status = %d, want %d",
|
|
||||||
code, http.StatusOK)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = buf.Flush()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("flush of the report after: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
files, err := readReportFiles(dir)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read report files: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
stored := strings.Join(files, "")
|
|
||||||
|
|
||||||
for _, clientID := range []string{"before", "after"} {
|
|
||||||
if !strings.Contains(stored, `"clientId":"`+clientID+`"`) {
|
|
||||||
t.Errorf("report %q not stored", clientID)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if strings.Contains(stored, `"clientId":"refused"`) {
|
|
||||||
t.Error("refused report stored")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// startBufferWithHandlers starts a Buffer and the Handlers that store
|
|
||||||
// reports in it through fx, as main does, with the DATA_DIR the calling
|
|
||||||
// test has set.
|
|
||||||
func startBufferWithHandlers(
|
|
||||||
t *testing.T,
|
|
||||||
) (*reportbuf.Buffer, *handlers.Handlers) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var (
|
|
||||||
buf *reportbuf.Buffer
|
|
||||||
h *handlers.Handlers
|
|
||||||
)
|
|
||||||
|
|
||||||
app := fxtest.New(t,
|
|
||||||
fx.Provide(
|
|
||||||
config.New,
|
|
||||||
globals.New,
|
|
||||||
handlers.New,
|
|
||||||
healthcheck.New,
|
|
||||||
logger.New,
|
|
||||||
reportbuf.New,
|
|
||||||
),
|
|
||||||
fx.Populate(&buf, &h),
|
|
||||||
)
|
|
||||||
|
|
||||||
app.RequireStart()
|
|
||||||
t.Cleanup(app.RequireStop)
|
|
||||||
|
|
||||||
return buf, h
|
|
||||||
}
|
|
||||||
|
|
||||||
// postReport posts a report from clientID to the report handler, as the
|
|
||||||
// page does, and returns the status of the answer.
|
|
||||||
func postReport(t *testing.T, h *handlers.Handlers, clientID string) int {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(),
|
|
||||||
http.MethodPost, "/api/v1/reports",
|
|
||||||
strings.NewReader(`{"clientId":"`+clientID+`","hosts":[]}`),
|
|
||||||
)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
h.HandleReport().ServeHTTP(rec, req)
|
|
||||||
|
|
||||||
return rec.Code
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestConcurrentAppendsStopAtCap appends from many goroutines at once
|
// TestConcurrentAppendsStopAtCap appends from many goroutines at once
|
||||||
@@ -1068,32 +681,6 @@ func TestFlushAtSizeThreshold(t *testing.T) {
|
|||||||
|
|
||||||
buf := startBuffer(t)
|
buf := startBuffer(t)
|
||||||
|
|
||||||
appended := appendToThreshold(t, buf)
|
|
||||||
|
|
||||||
// Append writes the file in the background, so wait for it.
|
|
||||||
deadline := time.Now().Add(10 * time.Second)
|
|
||||||
|
|
||||||
for {
|
|
||||||
files, err := readReportFiles(dir)
|
|
||||||
if err == nil && len(files) == 1 && files[0] == appended {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if time.Now().After(deadline) {
|
|
||||||
t.Fatalf("%d report files (error: %v), want one holding the "+
|
|
||||||
"%d bytes appended", len(files), err, len(appended))
|
|
||||||
}
|
|
||||||
|
|
||||||
time.Sleep(10 * time.Millisecond)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// appendToThreshold appends reports to buf until it holds
|
|
||||||
// FlushSizeThreshold bytes, so that the last append starts writing them
|
|
||||||
// to a report file, and returns the lines appended.
|
|
||||||
func appendToThreshold(t *testing.T, buf *reportbuf.Buffer) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
// Large reports, so the threshold takes a few hundred appends.
|
// Large reports, so the threshold takes a few hundred appends.
|
||||||
pad := strings.Repeat("a", 64<<10)
|
pad := strings.Repeat("a", 64<<10)
|
||||||
|
|
||||||
@@ -1116,7 +703,22 @@ func appendToThreshold(t *testing.T, buf *reportbuf.Buffer) string {
|
|||||||
appended.WriteByte('\n')
|
appended.WriteByte('\n')
|
||||||
}
|
}
|
||||||
|
|
||||||
return appended.String()
|
// Append writes the file in the background, so wait for it.
|
||||||
|
deadline := time.Now().Add(10 * time.Second)
|
||||||
|
|
||||||
|
for {
|
||||||
|
files, err := readReportFiles(dir)
|
||||||
|
if err == nil && len(files) == 1 && files[0] == appended.String() {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
t.Fatalf("%d report files (error: %v), want one holding the "+
|
||||||
|
"%d bytes appended", len(files), err, appended.Len())
|
||||||
|
}
|
||||||
|
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// reportFilesBytes returns the total size of the report files in dir.
|
// reportFilesBytes returns the total size of the report files in dir.
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
package server
|
||||||
|
|
||||||
|
// MaxRequestBodyBytes exposes the router-wide body limit to the
|
||||||
|
// external tests.
|
||||||
|
const MaxRequestBodyBytes = maxRequestBodyBytes
|
||||||
|
|
||||||
|
// ListenAddr exposes the address the server listens on to the
|
||||||
|
// external tests.
|
||||||
|
func (s *Server) ListenAddr() string {
|
||||||
|
return s.newHTTPServer().Addr
|
||||||
|
}
|
||||||
@@ -3,7 +3,6 @@ package server
|
|||||||
import (
|
import (
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
sentryhttp "github.com/getsentry/sentry-go/http"
|
|
||||||
"github.com/go-chi/chi/v5"
|
"github.com/go-chi/chi/v5"
|
||||||
"github.com/go-chi/chi/v5/middleware"
|
"github.com/go-chi/chi/v5/middleware"
|
||||||
"github.com/prometheus/client_golang/prometheus"
|
"github.com/prometheus/client_golang/prometheus"
|
||||||
@@ -18,12 +17,6 @@ const (
|
|||||||
// can mount s.mw.MaxBodyBytes with a smaller value to lower
|
// can mount s.mw.MaxBodyBytes with a smaller value to lower
|
||||||
// its bound, but cannot raise it: this cap runs first.
|
// its bound, but cannot raise it: this cap runs first.
|
||||||
maxRequestBodyBytes int64 = 1 << 20 // 1 MiB
|
maxRequestBodyBytes int64 = 1 << 20 // 1 MiB
|
||||||
|
|
||||||
// metricsRequestsPerMinute is how many requests to /metrics each
|
|
||||||
// client address may make a minute, whatever their credentials. A
|
|
||||||
// scraper polling every 2 seconds sends half of it, which httprate
|
|
||||||
// never refuses.
|
|
||||||
metricsRequestsPerMinute = 60
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// SetupRoutes configures the chi router with middleware and
|
// SetupRoutes configures the chi router with middleware and
|
||||||
@@ -34,20 +27,11 @@ func (s *Server) SetupRoutes() {
|
|||||||
s.router.Use(s.mw.Recoverer())
|
s.router.Use(s.mw.Recoverer())
|
||||||
s.router.Use(middleware.RequestID)
|
s.router.Use(middleware.RequestID)
|
||||||
s.router.Use(s.mw.Logging())
|
s.router.Use(s.mw.Logging())
|
||||||
// s.mw.Recoverer would write a panic's 500 only after the panic has
|
|
||||||
// passed the request log, so it is written here, where the log sees it.
|
|
||||||
s.router.Use(s.mw.Write500OnPanic())
|
|
||||||
s.router.Use(s.mw.SecurityHeaders())
|
s.router.Use(s.mw.SecurityHeaders())
|
||||||
s.router.Use(s.mw.CORS(s.params.Config.CORSAllowedOrigins))
|
s.router.Use(s.mw.CORS(s.params.Config.CORSAllowedOrigins))
|
||||||
s.router.Use(s.mw.MaxBodyBytes(maxRequestBodyBytes))
|
s.router.Use(s.mw.MaxBodyBytes(maxRequestBodyBytes))
|
||||||
s.router.Use(middleware.Timeout(requestTimeout))
|
s.router.Use(middleware.Timeout(requestTimeout))
|
||||||
|
|
||||||
// Sentry reports a panic, then panics again, so that it still
|
|
||||||
// reaches s.mw.Recoverer.
|
|
||||||
if s.params.Config.SentryDSN != "" {
|
|
||||||
s.router.Use(sentryhttp.New(sentryhttp.Options{Repanic: true}).Handle)
|
|
||||||
}
|
|
||||||
|
|
||||||
// The metrics go in a registry of this server's own, not in
|
// The metrics go in a registry of this server's own, not in
|
||||||
// Prometheus' default one, which takes them only once per process.
|
// Prometheus' default one, which takes them only once per process.
|
||||||
registry := prometheus.NewRegistry()
|
registry := prometheus.NewRegistry()
|
||||||
@@ -67,8 +51,6 @@ func (s *Server) SetupRoutes() {
|
|||||||
// other.
|
// other.
|
||||||
if s.params.Config.MetricsUsername != "" {
|
if s.params.Config.MetricsUsername != "" {
|
||||||
r.Use(s.mw.Metrics(registry))
|
r.Use(s.mw.Metrics(registry))
|
||||||
// Likewise inside the metrics.
|
|
||||||
r.Use(s.mw.Write500OnPanic())
|
|
||||||
}
|
}
|
||||||
|
|
||||||
r.Get("/.well-known/healthcheck", s.h.HandleHealthCheck())
|
r.Get("/.well-known/healthcheck", s.h.HandleHealthCheck())
|
||||||
@@ -77,13 +59,9 @@ func (s *Server) SetupRoutes() {
|
|||||||
Post("/api/v1/reports", s.h.HandleReport())
|
Post("/api/v1/reports", s.h.HandleReport())
|
||||||
})
|
})
|
||||||
|
|
||||||
// The rate limit comes before the basic auth, so a client past it
|
|
||||||
// gets 429 and its password is not checked.
|
|
||||||
if s.params.Config.MetricsUsername != "" {
|
if s.params.Config.MetricsUsername != "" {
|
||||||
s.router.With(
|
s.router.With(s.mw.MetricsAuth()).
|
||||||
s.mw.RateLimit(metricsRequestsPerMinute),
|
Get("/metrics", promhttp.HandlerFor(
|
||||||
s.mw.MetricsAuth(),
|
|
||||||
).Get("/metrics", promhttp.HandlerFor(
|
|
||||||
registry, promhttp.HandlerOpts{},
|
registry, promhttp.HandlerOpts{},
|
||||||
).ServeHTTP)
|
).ServeHTTP)
|
||||||
}
|
}
|
||||||
@@ -1,12 +1,10 @@
|
|||||||
package server_test
|
package server_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"io"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
|
||||||
|
|
||||||
"sneak.berlin/go/netwatch/internal/config"
|
"sneak.berlin/go/netwatch/internal/config"
|
||||||
"sneak.berlin/go/netwatch/internal/globals"
|
"sneak.berlin/go/netwatch/internal/globals"
|
||||||
@@ -17,7 +15,6 @@ import (
|
|||||||
"sneak.berlin/go/netwatch/internal/reportbuf"
|
"sneak.berlin/go/netwatch/internal/reportbuf"
|
||||||
"sneak.berlin/go/netwatch/internal/server"
|
"sneak.berlin/go/netwatch/internal/server"
|
||||||
|
|
||||||
"github.com/getsentry/sentry-go"
|
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
"go.uber.org/fx/fxtest"
|
"go.uber.org/fx/fxtest"
|
||||||
)
|
)
|
||||||
@@ -66,7 +63,6 @@ func TestReportsAreRateLimited(t *testing.T) {
|
|||||||
http.MethodPost, "/api/v1/reports",
|
http.MethodPost, "/api/v1/reports",
|
||||||
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
|
strings.NewReader(`{"clientId":"c1","hosts":[]}`),
|
||||||
)
|
)
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
srv.ServeHTTP(rec, req)
|
srv.ServeHTTP(rec, req)
|
||||||
|
|
||||||
return rec.Code
|
return rec.Code
|
||||||
@@ -189,50 +185,6 @@ func TestMetricsBehindBasicAuth(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestMetricsAreRateLimited: a client that has used up its /metrics
|
|
||||||
// allowance on wrong passwords gets 429 even with the right one, which
|
|
||||||
// is then not checked, while another client behind the same nginx
|
|
||||||
// still gets in.
|
|
||||||
func TestMetricsAreRateLimited(t *testing.T) {
|
|
||||||
t.Setenv("METRICS_USERNAME", "prometheus")
|
|
||||||
t.Setenv("METRICS_PASSWORD", "right")
|
|
||||||
// As in the container: nginx connects from loopback and names the
|
|
||||||
// client in X-Forwarded-For.
|
|
||||||
t.Setenv("TRUSTED_PROXIES", "127.0.0.1/32")
|
|
||||||
|
|
||||||
srv := newServer(t)
|
|
||||||
srv.SetupRoutes()
|
|
||||||
|
|
||||||
get := func(client, password string) int {
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(),
|
|
||||||
http.MethodGet, "/metrics", http.NoBody)
|
|
||||||
req.RemoteAddr = "127.0.0.1:40000"
|
|
||||||
req.Header.Set("X-Forwarded-For", client)
|
|
||||||
req.SetBasicAuth("prometheus", password)
|
|
||||||
srv.ServeHTTP(rec, req)
|
|
||||||
|
|
||||||
return rec.Code
|
|
||||||
}
|
|
||||||
|
|
||||||
for i := range server.MetricsRequestsPerMinute {
|
|
||||||
if code := get("203.0.113.7", "wrong"); code != http.StatusUnauthorized {
|
|
||||||
t.Fatalf("guess %d: status = %d, want %d",
|
|
||||||
i+1, code, http.StatusUnauthorized)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if code := get("203.0.113.7", "right"); code != http.StatusTooManyRequests {
|
|
||||||
t.Fatalf("right password past the limit: status = %d, want %d",
|
|
||||||
code, http.StatusTooManyRequests)
|
|
||||||
}
|
|
||||||
|
|
||||||
if code := get("203.0.113.8", "right"); code != http.StatusOK {
|
|
||||||
t.Fatalf("another client: status = %d, want %d",
|
|
||||||
code, http.StatusOK)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestMetricsInTwoServers: two servers in one process can both have
|
// TestMetricsInTwoServers: two servers in one process can both have
|
||||||
// metrics on.
|
// metrics on.
|
||||||
func TestMetricsInTwoServers(t *testing.T) {
|
func TestMetricsInTwoServers(t *testing.T) {
|
||||||
@@ -244,70 +196,6 @@ func TestMetricsInTwoServers(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestSentry: with SENTRY_DSN empty there is no Sentry client. With it
|
|
||||||
// pointing at a local server standing in for Sentry, a panic in a
|
|
||||||
// handler reaches that server, and the request still gets the 500 from
|
|
||||||
// the panic recovery.
|
|
||||||
func TestSentry(t *testing.T) {
|
|
||||||
const panicMessage = "handler panic for TestSentry"
|
|
||||||
|
|
||||||
// sentry.Init sets the client for the whole process; take it away
|
|
||||||
// again so that no other test reports to Sentry.
|
|
||||||
t.Cleanup(func() { sentry.CurrentHub().BindClient(nil) })
|
|
||||||
|
|
||||||
t.Setenv("SENTRY_DSN", "")
|
|
||||||
newServer(t)
|
|
||||||
|
|
||||||
if sentry.CurrentHub().Client() != nil {
|
|
||||||
t.Fatal("a Sentry client exists with SENTRY_DSN empty")
|
|
||||||
}
|
|
||||||
|
|
||||||
// The body of the first request the stand-in for Sentry receives.
|
|
||||||
received := make(chan string, 1)
|
|
||||||
|
|
||||||
sentryServer := httptest.NewServer(http.HandlerFunc(
|
|
||||||
func(_ http.ResponseWriter, r *http.Request) {
|
|
||||||
body, _ := io.ReadAll(r.Body)
|
|
||||||
|
|
||||||
select {
|
|
||||||
case received <- string(body):
|
|
||||||
default:
|
|
||||||
}
|
|
||||||
},
|
|
||||||
))
|
|
||||||
defer sentryServer.Close()
|
|
||||||
|
|
||||||
t.Setenv("SENTRY_DSN",
|
|
||||||
"http://key@"+sentryServer.Listener.Addr().String()+"/1")
|
|
||||||
|
|
||||||
srv := newServer(t)
|
|
||||||
srv.SetupRoutes()
|
|
||||||
srv.Router().Get("/panic", func(http.ResponseWriter, *http.Request) {
|
|
||||||
panic(panicMessage)
|
|
||||||
})
|
|
||||||
|
|
||||||
rec := httptest.NewRecorder()
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(),
|
|
||||||
http.MethodGet, "/panic", http.NoBody)
|
|
||||||
srv.ServeHTTP(rec, req)
|
|
||||||
|
|
||||||
if rec.Code != http.StatusInternalServerError {
|
|
||||||
t.Fatalf("status = %d, want %d",
|
|
||||||
rec.Code, http.StatusInternalServerError)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sentry sends from a goroutine of its own.
|
|
||||||
select {
|
|
||||||
case body := <-received:
|
|
||||||
if !strings.Contains(body, panicMessage) {
|
|
||||||
t.Fatalf("the Sentry server received no report of the panic:\n%s",
|
|
||||||
body)
|
|
||||||
}
|
|
||||||
case <-time.After(5 * time.Second):
|
|
||||||
t.Fatal("nothing reached the Sentry server")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHealthCheckRejectsOversizeBody sends the health check, which
|
// TestHealthCheckRejectsOversizeBody sends the health check, which
|
||||||
// never reads its body, a body one byte over the limit. Only the
|
// never reads its body, a body one byte over the limit. Only the
|
||||||
// router-wide body limit can reject it.
|
// router-wide body limit can reject it.
|
||||||
@@ -7,10 +7,8 @@ package server
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"time"
|
|
||||||
|
|
||||||
"sneak.berlin/go/netwatch/internal/config"
|
"sneak.berlin/go/netwatch/internal/config"
|
||||||
"sneak.berlin/go/netwatch/internal/globals"
|
"sneak.berlin/go/netwatch/internal/globals"
|
||||||
@@ -18,15 +16,10 @@ import (
|
|||||||
"sneak.berlin/go/netwatch/internal/logger"
|
"sneak.berlin/go/netwatch/internal/logger"
|
||||||
"sneak.berlin/go/netwatch/internal/middleware"
|
"sneak.berlin/go/netwatch/internal/middleware"
|
||||||
|
|
||||||
"github.com/getsentry/sentry-go"
|
|
||||||
"github.com/go-chi/chi/v5"
|
"github.com/go-chi/chi/v5"
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
)
|
)
|
||||||
|
|
||||||
// sentryFlushTimeout is how long shutdown waits for Sentry to send
|
|
||||||
// what it still holds.
|
|
||||||
const sentryFlushTimeout = 2 * time.Second
|
|
||||||
|
|
||||||
// Params defines the dependencies for Server.
|
// Params defines the dependencies for Server.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
fx.In
|
fx.In
|
||||||
@@ -63,11 +56,6 @@ func New(
|
|||||||
s.log = params.Logger.Get()
|
s.log = params.Logger.Get()
|
||||||
s.shutdowner = params.Shutdowner
|
s.shutdowner = params.Shutdowner
|
||||||
|
|
||||||
err := s.enableSentry()
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
lc.Append(fx.Hook{
|
lc.Append(fx.Hook{
|
||||||
OnStart: func(_ context.Context) error {
|
OnStart: func(_ context.Context) error {
|
||||||
// Build the router and http.Server synchronously
|
// Build the router and http.Server synchronously
|
||||||
@@ -99,37 +87,10 @@ func (s *Server) ServeHTTP(
|
|||||||
s.router.ServeHTTP(w, r)
|
s.router.ServeHTTP(w, r)
|
||||||
}
|
}
|
||||||
|
|
||||||
// enableSentry sets Sentry up when SENTRY_DSN is set, so that
|
|
||||||
// SetupRoutes can report panics to it. With SENTRY_DSN empty it does
|
|
||||||
// nothing. A DSN Sentry refuses stops the start.
|
|
||||||
func (s *Server) enableSentry() error {
|
|
||||||
if s.params.Config.SentryDSN == "" {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
err := sentry.Init(sentry.ClientOptions{
|
|
||||||
Dsn: s.params.Config.SentryDSN,
|
|
||||||
Release: s.params.Globals.Appname + "-" + s.params.Globals.Version,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("SENTRY_DSN: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
s.log.Info("sentry error reporting activated")
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// shutdown gracefully stops the HTTP server within the
|
// shutdown gracefully stops the HTTP server within the
|
||||||
// deadline of the context fx provides for OnStop, then gives
|
// deadline of the context fx provides for OnStop.
|
||||||
// Sentry, if set up, time to send what it still holds.
|
|
||||||
func (s *Server) shutdown(ctx context.Context) error {
|
func (s *Server) shutdown(ctx context.Context) error {
|
||||||
err := s.httpServer.Shutdown(ctx)
|
err := s.httpServer.Shutdown(ctx)
|
||||||
|
|
||||||
if s.params.Config.SentryDSN != "" {
|
|
||||||
sentry.Flush(sentryFlushTimeout)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s.log.Error("server clean shutdown failed", "error", err)
|
s.log.Error("server clean shutdown failed", "error", err)
|
||||||
|
|
||||||
Executable
+21
@@ -0,0 +1,21 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/build: compile the static netwatch-server binary into the
|
||||||
|
# backend project root, with its version stamped in.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
|
||||||
|
# VERSION comes from the environment (the root Dockerfile passes its
|
||||||
|
# ARG VERSION in). Unset or empty, it is git describe, or "dev" where
|
||||||
|
# there is no git or no repository history.
|
||||||
|
version="${VERSION:-$(git describe --always --dirty 2>/dev/null || echo dev)}"
|
||||||
|
|
||||||
|
CGO_ENABLED=0 go build -trimpath \
|
||||||
|
-ldflags "-s -w -X main.Version=$version" \
|
||||||
|
-o netwatch-server ./cmd/netwatch-server/
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Executable
+12
@@ -0,0 +1,12 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/clean: remove build artifacts.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
rm -f netwatch-server
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Executable
+12
@@ -0,0 +1,12 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/fmt: format the Go sources (writes).
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
go fmt ./...
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Executable
+18
@@ -0,0 +1,18 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/fmt-check: check Go formatting (read-only). Same scope as
|
||||||
|
# script/fmt, but fails instead of writing.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
unformatted="$(gofmt -l .)"
|
||||||
|
if [ -n "$unformatted" ]; then
|
||||||
|
echo "Files not formatted:" >&2
|
||||||
|
echo "$unformatted" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Executable
+50
@@ -0,0 +1,50 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/lint: run golangci-lint over the backend. This runs inside the
|
||||||
|
# lint stage of the root Dockerfile, whose digest-pinned golangci-lint
|
||||||
|
# image provides the linter; nothing installs golangci-lint on the host.
|
||||||
|
# From a checkout, run `make lint` at the repo root, which builds that
|
||||||
|
# stage.
|
||||||
|
#
|
||||||
|
# .golangci.yml is standardized org-wide and must never be edited here
|
||||||
|
# (REPO_POLICIES.md). Its last silent drift replaced the v2 schema with
|
||||||
|
# v1 keys, which left every threshold in the file inert while the build
|
||||||
|
# stayed green. So the file is first checked against the canonical
|
||||||
|
# copy's sha256: a local comparison, no network, nothing unpinned.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
# The sha256 of the org standard .golangci.yml. When that file changes in
|
||||||
|
# sneak/prompts and is copied here again, this changes with it.
|
||||||
|
GOLANGCI_CONFIG_SHA256="a79b63a254602a5318db5d0e9a06bc71b84bf0c1d896305229d8bfed1d1b1776"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
if [ ! -f .golangci.yml ]; then
|
||||||
|
echo "backend/.golangci.yml is missing. Copy the org standard verbatim" >&2
|
||||||
|
echo "from https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
actual="$(sha256sum .golangci.yml | cut -d' ' -f1)"
|
||||||
|
if [ -z "$actual" ]; then
|
||||||
|
echo "sha256sum is missing or printed no hash, so" >&2
|
||||||
|
echo "backend/.golangci.yml could not be checked." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ "$actual" != "$GOLANGCI_CONFIG_SHA256" ]; then
|
||||||
|
echo "backend/.golangci.yml does not match GOLANGCI_CONFIG_SHA256" >&2
|
||||||
|
echo "in backend/script/lint." >&2
|
||||||
|
echo " expected $GOLANGCI_CONFIG_SHA256" >&2
|
||||||
|
echo " actual $actual" >&2
|
||||||
|
echo "Compare it with the org standard," >&2
|
||||||
|
echo "https://git.eeqj.de/sneak/prompts/raw/branch/main/.golangci.yml" >&2
|
||||||
|
echo "- If they differ, it was edited here: restore the org standard" >&2
|
||||||
|
echo " verbatim. Do not edit it." >&2
|
||||||
|
echo "- If they are the same, the org standard changed: set" >&2
|
||||||
|
echo " GOLANGCI_CONFIG_SHA256 in backend/script/lint to the actual hash." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
golangci-lint run ./...
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
@@ -1,6 +1,5 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/run: build with script/build, then run netwatch-server locally
|
# script/run: build and run netwatch-server locally.
|
||||||
# with its defaults: port 8080, reports stored in data/reports.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
Executable
+20
@@ -0,0 +1,20 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/test: run the backend test suite with the race detector and
|
||||||
|
# coverage. Go's own -timeout bounds the tests and not their compile,
|
||||||
|
# so a cold build cache cannot fail it. The race detector needs cgo,
|
||||||
|
# and so a C compiler. If the tests fail, they run again with -v for
|
||||||
|
# the details, and the script fails even if that run passes.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
go test -timeout 30s -race -cover ./... || {
|
||||||
|
echo "--- Rerunning with -v for details ---"
|
||||||
|
go test -timeout 30s -race -v ./...
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
+10
-12
@@ -34,11 +34,9 @@ fi
|
|||||||
|
|
||||||
# TRUSTED_PROXIES names the reverse proxies in front of the container,
|
# TRUSTED_PROXIES names the reverse proxies in front of the container,
|
||||||
# as IP addresses or CIDRs separated by commas. nginx takes the client
|
# as IP addresses or CIDRs separated by commas. nginx takes the client
|
||||||
# address from X-Forwarded-For only on a request from one of them.
|
# address from X-Forwarded-For only on a request from one of them, so
|
||||||
# Unset, they are the three private ranges of RFC 1918; set and empty,
|
# unset or empty, it trusts no one. nginx.conf includes the file written
|
||||||
# nginx trusts no one, which is why the default below follows - and not
|
# here, one set_real_ip_from line per entry.
|
||||||
# :-. configs/nginx.conf includes the file written here, one
|
|
||||||
# set_real_ip_from line per entry.
|
|
||||||
#
|
#
|
||||||
# nginx looks up an entry it cannot read as an address as a hostname,
|
# nginx looks up an entry it cannot read as an address as a hostname,
|
||||||
# and trusts what it finds (1.2.3 is found as 1.2.0.3). So each entry
|
# and trusts what it finds (1.2.3 is found as 1.2.0.3). So each entry
|
||||||
@@ -47,7 +45,7 @@ fi
|
|||||||
# TRUSTED_PROXIES. Its error, naming the CIDR, is dropped for the one
|
# TRUSTED_PROXIES. Its error, naming the CIDR, is dropped for the one
|
||||||
# below, naming the entry as written. set -f keeps a * in an entry from
|
# below, naming the entry as written. set -f keeps a * in an entry from
|
||||||
# becoming a list of file names.
|
# becoming a list of file names.
|
||||||
TRUSTED_PROXIES="${TRUSTED_PROXIES-10.0.0.0/8,172.16.0.0/12,192.168.0.0/16}"
|
TRUSTED_PROXIES="${TRUSTED_PROXIES:-}"
|
||||||
set -f
|
set -f
|
||||||
for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
|
for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
|
||||||
case "$proxy" in
|
case "$proxy" in
|
||||||
@@ -78,10 +76,10 @@ stop_requested=""
|
|||||||
trap 'stop_requested=yes' TERM INT
|
trap 'stop_requested=yes' TERM INT
|
||||||
|
|
||||||
# netwatch-server runs as the netwatch user and listens on loopback
|
# netwatch-server runs as the netwatch user and listens on loopback
|
||||||
# only, on a port other than the public one; configs/nginx.conf proxies
|
# only, on a port other than the public one; nginx.conf proxies to this
|
||||||
# to this address. Its only client is nginx, so it takes the client
|
# address. Its only client is nginx, so it takes the client address
|
||||||
# address nginx passes on from 127.0.0.1 alone, whatever TRUSTED_PROXIES
|
# nginx passes on from 127.0.0.1 alone, whatever TRUSTED_PROXIES the
|
||||||
# the container has. The netwatch user has no login shell, hence -s
|
# container has. The netwatch user has no login shell, hence -s
|
||||||
# /bin/sh. busybox su replaces itself with the command instead of
|
# /bin/sh. busybox su replaces itself with the command instead of
|
||||||
# staying on as its parent, so $! is the server's own PID.
|
# staying on as its parent, so $! is the server's own PID.
|
||||||
BIND_ADDRESS=127.0.0.1 PORT=8081 TRUSTED_PROXIES=127.0.0.1/32 \
|
BIND_ADDRESS=127.0.0.1 PORT=8081 TRUSTED_PROXIES=127.0.0.1/32 \
|
||||||
@@ -90,8 +88,8 @@ backend=$!
|
|||||||
|
|
||||||
# nginx starts through the nginx image's own entrypoint, which applies
|
# nginx starts through the nginx image's own entrypoint, which applies
|
||||||
# the image's start-up configuration and then replaces itself with
|
# the image's start-up configuration and then replaces itself with
|
||||||
# nginx. Part of that start-up configuration renders configs/nginx.conf
|
# nginx. Part of that start-up configuration renders nginx.conf into
|
||||||
# into conf.d with nginx listening on PORT. NGINX_ENVSUBST_FILTER limits
|
# conf.d with nginx listening on PORT. NGINX_ENVSUBST_FILTER limits
|
||||||
# that rendering to PORT: a variable nginx itself uses, such as $uri,
|
# that rendering to PORT: a variable nginx itself uses, such as $uri,
|
||||||
# would otherwise be replaced by an environment variable of the same
|
# would otherwise be replaced by an environment variable of the same
|
||||||
# name.
|
# name.
|
||||||
|
|||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
// eslint's recommended rules over every JavaScript file in the repo.
|
// eslint's recommended rules over every JavaScript file in the repo.
|
||||||
// make lint runs it, in the frontend-lint stage of Dockerfile.
|
// script/frontend-lint runs it, in the frontend-lint stage of Dockerfile.
|
||||||
import js from "@eslint/js";
|
import js from "@eslint/js";
|
||||||
import globals from "globals";
|
import globals from "globals";
|
||||||
import { defineConfig } from "eslint/config";
|
import { defineConfig } from "eslint/config";
|
||||||
@@ -8,7 +8,7 @@ export default defineConfig([
|
|||||||
js.configs.recommended,
|
js.configs.recommended,
|
||||||
{
|
{
|
||||||
// The page, and facts.js, which the viewport test runs inside it.
|
// The page, and facts.js, which the viewport test runs inside it.
|
||||||
files: ["web/src/**/*.js", "test/viewport/facts.js"],
|
files: ["src/**/*.js", "test/viewport/facts.js"],
|
||||||
languageOptions: {
|
languageOptions: {
|
||||||
globals: {
|
globals: {
|
||||||
...globals.browser,
|
...globals.browser,
|
||||||
|
|||||||
@@ -9,7 +9,7 @@
|
|||||||
type="image/svg+xml"
|
type="image/svg+xml"
|
||||||
href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>📡</text></svg>"
|
href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>📡</text></svg>"
|
||||||
/>
|
/>
|
||||||
<!-- Linked here, not imported by web/src/main.js, so the unit tests can
|
<!-- Linked here, not imported by src/main.js, so the unit tests can
|
||||||
import that module in Node, which cannot import CSS. -->
|
import that module in Node, which cannot import CSS. -->
|
||||||
<link rel="stylesheet" href="/src/styles.css" />
|
<link rel="stylesheet" href="/src/styles.css" />
|
||||||
</head>
|
</head>
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
package server
|
|
||||||
|
|
||||||
import "github.com/go-chi/chi/v5"
|
|
||||||
|
|
||||||
// Router exposes the router to the external tests, which add routes
|
|
||||||
// of their own to it after SetupRoutes.
|
|
||||||
func (s *Server) Router() *chi.Mux {
|
|
||||||
return s.router
|
|
||||||
}
|
|
||||||
|
|
||||||
// MaxRequestBodyBytes exposes the router-wide body limit to the
|
|
||||||
// external tests.
|
|
||||||
const MaxRequestBodyBytes = maxRequestBodyBytes
|
|
||||||
|
|
||||||
// MetricsRequestsPerMinute exposes the /metrics rate limit to the
|
|
||||||
// external tests.
|
|
||||||
const MetricsRequestsPerMinute = metricsRequestsPerMinute
|
|
||||||
|
|
||||||
// ListenAddr exposes the address the server listens on to the
|
|
||||||
// external tests.
|
|
||||||
func (s *Server) ListenAddr() string {
|
|
||||||
return s.newHTTPServer().Addr
|
|
||||||
}
|
|
||||||
@@ -18,10 +18,9 @@ server {
|
|||||||
|
|
||||||
# The client address comes from X-Forwarded-For only on a request
|
# The client address comes from X-Forwarded-For only on a request
|
||||||
# from the reverse proxies in TRUSTED_PROXIES: bin/entrypoint.sh
|
# from the reverse proxies in TRUSTED_PROXIES: bin/entrypoint.sh
|
||||||
# writes one set_real_ip_from line for each into this file. Unset,
|
# writes one set_real_ip_from line for each into this file, and
|
||||||
# TRUSTED_PROXIES is the three private ranges of RFC 1918; set and
|
# leaves it empty when TRUSTED_PROXIES is unset, so that by default
|
||||||
# empty, the file is empty and the client address is the one each
|
# the client address is the one each request comes from.
|
||||||
# request comes from.
|
|
||||||
include /etc/nginx/trusted-proxies.conf;
|
include /etc/nginx/trusted-proxies.conf;
|
||||||
real_ip_header X-Forwarded-For;
|
real_ip_header X-Forwarded-For;
|
||||||
real_ip_recursive on;
|
real_ip_recursive on;
|
||||||
@@ -54,7 +53,7 @@ server {
|
|||||||
|
|
||||||
# netwatch-server sets the same security headers on its own
|
# netwatch-server sets the same security headers on its own
|
||||||
# responses. Its copies are dropped so that each header goes out
|
# responses. Its copies are dropped so that each header goes out
|
||||||
# once, as configs/security-headers.conf sets it.
|
# once, as security-headers.conf sets it.
|
||||||
proxy_hide_header Strict-Transport-Security;
|
proxy_hide_header Strict-Transport-Security;
|
||||||
proxy_hide_header Content-Security-Policy;
|
proxy_hide_header Content-Security-Policy;
|
||||||
proxy_hide_header X-Frame-Options;
|
proxy_hide_header X-Frame-Options;
|
||||||
+20
-30
@@ -6,18 +6,19 @@
|
|||||||
# used directly if it is at least NODE_MIN_VERSION; otherwise it is
|
# used directly if it is at least NODE_MIN_VERSION; otherwise it is
|
||||||
# installed at a pinned version via nvm (installing nvm itself first,
|
# installed at a pinned version via nvm (installing nvm itself first,
|
||||||
# from a hash-verified release archive, never curl | sh). Go, with its
|
# from a hash-verified release archive, never curl | sh). Go, with its
|
||||||
# gofmt, is used directly only if it is exactly GO_VERSION; otherwise
|
# gofmt, is used directly if it is at least the version backend/go.mod
|
||||||
# that release is installed from its hash-verified archive.
|
# asks for; otherwise the pinned Go release is installed from its
|
||||||
|
# hash-verified archive.
|
||||||
#
|
#
|
||||||
# What this script installs outside the system package manager lives
|
# What this script installs outside the system package manager lives
|
||||||
# under $HOME and is linked into ~/.local/bin, where make and the git
|
# under $HOME and is linked into ~/.local/bin, where make and the git
|
||||||
# hook find it once that directory is on PATH. Nothing in ~/.local/bin
|
# hook find it once that directory is on PATH. Nothing in ~/.local/bin
|
||||||
# that this script did not create is ever replaced.
|
# that this script did not create is ever replaced.
|
||||||
#
|
#
|
||||||
# golangci-lint is not installed: make lint runs it in Docker, as make
|
# golangci-lint is not installed: make lint runs it in Docker, which
|
||||||
# test runs the tests, and this script does not install Docker either.
|
# this script does not install either.
|
||||||
#
|
#
|
||||||
# Unlike the org model: Go for the backend, a newer node for eslint.
|
# Unlike the org model: Go and gcc for backend/, a newer node for eslint.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
@@ -185,30 +186,20 @@ ensure_yarn() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# go_ok: the go on PATH has its gofmt beside it (a Go release ships the
|
# go_ok: the go on PATH has its gofmt beside it (a Go release ships the
|
||||||
# two together) and go version reports exactly GO_VERSION, compared over
|
# two together) and is at least the version backend/go.mod asks for.
|
||||||
# the whole version, not a prefix of it. A go that fails or prints
|
# GOTOOLCHAIN=local makes an older go fail here instead of fetching a
|
||||||
# anything else does not pass. GOTOOLCHAIN=local makes go report itself
|
# newer toolchain for itself.
|
||||||
# rather than a toolchain it would fetch.
|
|
||||||
go_ok() {
|
go_ok() {
|
||||||
if missing go; then return 1; fi
|
if missing go; then return 1; fi
|
||||||
[ -x "$(dirname "$(command -v go)")/gofmt" ] || return 1
|
[ -x "$(dirname "$(command -v go)")/gofmt" ] || return 1
|
||||||
version="$(GOTOOLCHAIN=local go version 2>/dev/null)" || return 1
|
(cd "$ROOT/backend" && GOTOOLCHAIN=local go list -m >/dev/null 2>&1)
|
||||||
case "$version" in
|
|
||||||
"go version go$GO_VERSION "*) return 0 ;;
|
|
||||||
*) return 1 ;;
|
|
||||||
esac
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# ensure_go: unless go_ok, install GO_VERSION and link its go and gofmt.
|
# ensure_go: unless go_ok, install GO_VERSION and link its go and gofmt.
|
||||||
# They are linked on every run that needs them, so a deleted link is put
|
# They are linked on every run that needs them, so a deleted link is put
|
||||||
# back, and the archive is unpacked again if either binary is missing.
|
# back, and the archive is unpacked again if either binary is missing.
|
||||||
# Afterwards go is looked up through PATH again and must pass go_ok, so
|
|
||||||
# another go that hides the link stops bootstrap.
|
|
||||||
ensure_go() {
|
ensure_go() {
|
||||||
if go_ok; then
|
if go_ok; then return 0; fi
|
||||||
echo "bootstrap: using $(GOTOOLCHAIN=local go version)"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
go_dir="$TOOLCHAIN/go-$GO_VERSION"
|
go_dir="$TOOLCHAIN/go-$GO_VERSION"
|
||||||
if [ ! -x "$go_dir/bin/go" ] || [ ! -x "$go_dir/bin/gofmt" ]; then
|
if [ ! -x "$go_dir/bin/go" ] || [ ! -x "$go_dir/bin/gofmt" ]; then
|
||||||
# sha256 of each archive, from https://go.dev/dl/?mode=json
|
# sha256 of each archive, from https://go.dev/dl/?mode=json
|
||||||
@@ -249,13 +240,6 @@ ensure_go() {
|
|||||||
fi
|
fi
|
||||||
link_bin "$go_dir/bin/go" go
|
link_bin "$go_dir/bin/go" go
|
||||||
link_bin "$go_dir/bin/gofmt" gofmt
|
link_bin "$go_dir/bin/gofmt" gofmt
|
||||||
hash -r
|
|
||||||
if ! go_ok; then
|
|
||||||
echo "bootstrap: after installing go $GO_VERSION in $go_dir," >&2
|
|
||||||
echo " the go on PATH, $(command -v go), is not it or has no gofmt" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
echo "bootstrap: installed $(GOTOOLCHAIN=local go version)"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
@@ -272,17 +256,23 @@ main() {
|
|||||||
|
|
||||||
if missing make; then pkg_install gnumake make make make; fi
|
if missing make; then pkg_install gnumake make make make; fi
|
||||||
if missing git; then pkg_install git git git git; fi
|
if missing git; then pkg_install git git git git; fi
|
||||||
|
# The race detector in make test needs cgo, which Go turns on only
|
||||||
|
# when it finds its C compiler, gcc on Linux. apt and apk ship the C
|
||||||
|
# library headers apart from gcc.
|
||||||
|
if missing gcc; then
|
||||||
|
pkg_install gcc "gcc libc6-dev" gcc "gcc musl-dev"
|
||||||
|
fi
|
||||||
|
|
||||||
ensure_node
|
ensure_node
|
||||||
ensure_yarn
|
ensure_yarn
|
||||||
yarn install --frozen-lockfile
|
yarn install --frozen-lockfile
|
||||||
|
|
||||||
ensure_go
|
ensure_go
|
||||||
go mod download
|
(cd "$ROOT/backend" && go mod download)
|
||||||
|
|
||||||
if missing docker; then
|
if missing docker; then
|
||||||
echo "bootstrap: docker not found; make test and make lint, and so" >&2
|
echo "bootstrap: docker not found; make lint, and so make check" >&2
|
||||||
echo " make check and the pre-commit hook, need it" >&2
|
echo " and the pre-commit hook, need it to run the linters" >&2
|
||||||
fi
|
fi
|
||||||
if [ -n "$path_hint" ] && [ -d "$BIN_DIR" ]; then
|
if [ -n "$path_hint" ] && [ -d "$BIN_DIR" ]; then
|
||||||
echo "bootstrap: add $BIN_DIR to the front of your PATH, e.g." >&2
|
echo "bootstrap: add $BIN_DIR to the front of your PATH, e.g." >&2
|
||||||
|
|||||||
+2
-11
@@ -1,7 +1,6 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/build: build the frontend for production into dist/, then the
|
# script/build: build the frontend for production into dist/. The Go
|
||||||
# static netwatch-server binary into the repo root, with its version
|
# backend is built by backend/script/build.
|
||||||
# stamped in.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
@@ -9,14 +8,6 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
yarn build
|
yarn build
|
||||||
|
|
||||||
# VERSION comes from the environment. Unset or empty, it is git
|
|
||||||
# describe, or "dev" where there is no git or no repository history.
|
|
||||||
version="${VERSION:-$(git describe --always --dirty 2>/dev/null || echo dev)}"
|
|
||||||
|
|
||||||
CGO_ENABLED=0 go build -trimpath \
|
|
||||||
-ldflags "-s -w -X main.Version=$version" \
|
|
||||||
-o netwatch-server ./cmd/netwatch-server/
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+1
-3
@@ -1,8 +1,6 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/check: run all checks (test, lint, fmt-check). Our own
|
# script/check: run all checks (test, lint, fmt-check). Our own
|
||||||
# extension to scripts-to-rule-them-all. test and lint are Docker
|
# extension to scripts-to-rule-them-all. Must not modify any files.
|
||||||
# phases; fmt-check is native, because a formatter writes the working
|
|
||||||
# tree. Must not modify any files.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
|
|||||||
@@ -1,13 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
# script/clean: remove what script/build writes: dist/ and the
|
|
||||||
# netwatch-server binary.
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|
||||||
|
|
||||||
main() {
|
|
||||||
cd "$ROOT"
|
|
||||||
rm -rf dist netwatch-server
|
|
||||||
}
|
|
||||||
|
|
||||||
main "$@"
|
|
||||||
+4
-9
@@ -1,20 +1,15 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/fmt: format the whole repo (writes): prettier over the
|
# script/fmt: format the whole repo (writes): prettier over everything
|
||||||
# JavaScript, CSS, HTML and Markdown, then gofmt over the Go sources in
|
# it understands, then gofmt over the Go backend.
|
||||||
# cmd/ and internal/; not over the whole root, as node_modules/ holds
|
# The org model formats only markdown; this repo also has JS and Go.
|
||||||
# .go files of its own. The org model formats only markdown; this repo
|
|
||||||
# also has JS and Go.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
# script/bootstrap links the node, yarn and gofmt it installs into
|
|
||||||
# ~/.local/bin, which the shell that called it may not have on PATH.
|
|
||||||
PATH="$HOME/.local/bin:$PATH"
|
|
||||||
"$ROOT/script/frontend-fmt"
|
"$ROOT/script/frontend-fmt"
|
||||||
gofmt -l -w cmd internal
|
"$ROOT/backend/script/fmt"
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+1
-9
@@ -8,16 +8,8 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
# script/bootstrap links the node, yarn and gofmt it installs into
|
|
||||||
# ~/.local/bin, which the shell that called it may not have on PATH.
|
|
||||||
PATH="$HOME/.local/bin:$PATH"
|
|
||||||
"$ROOT/script/frontend-fmt-check"
|
"$ROOT/script/frontend-fmt-check"
|
||||||
unformatted="$(gofmt -l cmd internal)"
|
"$ROOT/backend/script/fmt-check"
|
||||||
if [ -n "$unformatted" ]; then
|
|
||||||
echo "Go files not formatted:" >&2
|
|
||||||
echo "$unformatted" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
Executable
+18
@@ -0,0 +1,18 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/frontend-check: run the frontend tests and format check only.
|
||||||
|
# This exists for the frontend stage of Dockerfile, a node image with
|
||||||
|
# neither Go nor Docker; the Dockerfile's frontend-lint stage runs the
|
||||||
|
# frontend linter, and its lint and builder stages gate the backend.
|
||||||
|
# Everywhere else, use script/check, which covers the whole repo. Must
|
||||||
|
# not modify any files.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
"$ROOT/script/frontend-test"
|
||||||
|
"$ROOT/script/frontend-fmt-check"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
+4
-6
@@ -1,16 +1,14 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/frontend-fmt: format the JavaScript, CSS, HTML and Markdown,
|
# script/frontend-fmt: format the frontend and every other file prettier
|
||||||
# repo-wide (writes). Those are the languages REPO_POLICIES.md gives
|
# understands, repo-wide (writes), the markdown in backend/ included.
|
||||||
# prettier; the YAML is left alone, as .golangci.yml must stay the org
|
# Prettier does not read Go; backend/script/fmt formats the Go sources.
|
||||||
# standard byte for byte. Prettier does not read Go; script/fmt runs
|
|
||||||
# gofmt over the Go sources.
|
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
yarn prettier --write '**/*.{js,css,html,md}'
|
yarn prettier --write .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
yarn prettier --check '**/*.{js,css,html,md}'
|
yarn prettier --check .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
Executable
+15
@@ -0,0 +1,15 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/frontend-lint: run eslint over the frontend. This runs inside
|
||||||
|
# the frontend-lint stage of Dockerfile, the digest-pinned node image
|
||||||
|
# with the packages from yarn.lock. From a checkout, run `make lint`,
|
||||||
|
# which builds that stage.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
yarn eslint .
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
@@ -1,11 +1,9 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/frontend-test: run the frontend test suite on the host: the
|
# script/frontend-test: run the frontend test suite: the unit tests in
|
||||||
# unit tests in test/unit/, through the test script in package.json,
|
# test/unit/, through the test script in package.json, then the
|
||||||
# then the production build, which fails on broken code. make test runs
|
# production build, which fails on broken code. The tests print a dot
|
||||||
# the same tests in the frontend-test stage of Dockerfile, and the image
|
# each; if any fails, they run again with every test listed, and the
|
||||||
# build runs the production build in its frontend-build stage. The tests
|
# script fails even if that run passes. NODE_OPTIONS chooses the
|
||||||
# print a dot each; if any fails, they run again with every test listed,
|
|
||||||
# and the script fails even if that run passes. NODE_OPTIONS chooses the
|
|
||||||
# reporter because yarn adds its arguments after the test files, where
|
# reporter because yarn adds its arguments after the test files, where
|
||||||
# node would take a reporter option for one more file.
|
# node would take a reporter option for one more file.
|
||||||
set -eu
|
set -eu
|
||||||
@@ -14,9 +12,9 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
NODE_OPTIONS=--test-reporter=dot timeout 90 yarn --silent run test || {
|
NODE_OPTIONS=--test-reporter=dot timeout 30 yarn --silent run test || {
|
||||||
echo "--- Rerunning with every test listed for details ---"
|
echo "--- Rerunning with every test listed for details ---"
|
||||||
NODE_OPTIONS=--test-reporter=spec timeout 90 yarn --silent run test
|
NODE_OPTIONS=--test-reporter=spec timeout 30 yarn --silent run test
|
||||||
exit 1
|
exit 1
|
||||||
}
|
}
|
||||||
timeout 30 yarn build
|
timeout 30 yarn build
|
||||||
|
|||||||
@@ -2,11 +2,11 @@
|
|||||||
# script/frontend-viewport-test: verify the responsive layout of the built
|
# script/frontend-viewport-test: verify the responsive layout of the built
|
||||||
# frontend in a real browser engine.
|
# frontend in a real browser engine.
|
||||||
#
|
#
|
||||||
# Builds dist/, serves it with the same nginx image and the same
|
# Builds dist/, serves it with the same nginx image and the same nginx.conf
|
||||||
# configs/nginx.conf the shipping container uses, points a containerised
|
# the shipping container uses, points a containerised headless Chrome at it
|
||||||
# headless Chrome at it over CDP, and asserts on computed layout at every
|
# over CDP, and asserts on computed layout at every viewport width derived
|
||||||
# viewport width derived from the app's own CSS. See
|
# from the app's own CSS. See test/viewport/README.md for what this covers
|
||||||
# test/viewport/README.md for what this covers and what it cannot.
|
# and what it cannot.
|
||||||
#
|
#
|
||||||
# Deliberately not part of script/check: it needs Docker and takes far
|
# Deliberately not part of script/check: it needs Docker and takes far
|
||||||
# longer than the 20s budget make test has to stay inside.
|
# longer than the 20s budget make test has to stay inside.
|
||||||
@@ -61,18 +61,18 @@ main() {
|
|||||||
# host.
|
# host.
|
||||||
docker network create --internal "$NETWORK" > /dev/null
|
docker network create --internal "$NETWORK" > /dev/null
|
||||||
|
|
||||||
# configs/nginx.conf is a template: the image renders it over its own
|
# nginx.conf is a template: the image renders it over its own
|
||||||
# default.conf, with the same port and limit bin/entrypoint.sh uses.
|
# default.conf, with the same port and limit bin/entrypoint.sh uses.
|
||||||
# The empty file it includes trusts no proxy, as bin/entrypoint.sh
|
# The empty file it includes trusts no proxy, as bin/entrypoint.sh
|
||||||
# writes it when TRUSTED_PROXIES is set and empty. It also includes
|
# writes it when TRUSTED_PROXIES is unset. nginx.conf also includes
|
||||||
# the security headers, so the page runs under the shipped policy.
|
# the security headers, so the page runs under the shipped policy.
|
||||||
docker run -d --rm --name "$SERVER" \
|
docker run -d --rm --name "$SERVER" \
|
||||||
--network "$NETWORK" --network-alias netwatch \
|
--network "$NETWORK" --network-alias netwatch \
|
||||||
-e PORT=8080 -e NGINX_ENVSUBST_FILTER='^PORT$' \
|
-e PORT=8080 -e NGINX_ENVSUBST_FILTER='^PORT$' \
|
||||||
-v "$ROOT/dist:/usr/share/nginx/html:ro" \
|
-v "$ROOT/dist:/usr/share/nginx/html:ro" \
|
||||||
-v "$ROOT/configs/nginx.conf:/etc/nginx/templates/default.conf.template:ro" \
|
-v "$ROOT/nginx.conf:/etc/nginx/templates/default.conf.template:ro" \
|
||||||
-v /dev/null:/etc/nginx/trusted-proxies.conf:ro \
|
-v /dev/null:/etc/nginx/trusted-proxies.conf:ro \
|
||||||
-v "$ROOT/configs/security-headers.conf:/etc/nginx/security-headers.conf:ro" \
|
-v "$ROOT/security-headers.conf:/etc/nginx/security-headers.conf:ro" \
|
||||||
"$SERVER_IMAGE" > /dev/null
|
"$SERVER_IMAGE" > /dev/null
|
||||||
|
|
||||||
# The image's own entrypoint already exposes CDP on 9222 and passes
|
# The image's own entrypoint already exposes CDP on 9222 and passes
|
||||||
|
|||||||
+13
-13
@@ -1,23 +1,23 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/lint: run the linter. Linting is a phase of the Dockerfile and
|
# script/lint: lint the whole repo: eslint over the frontend, then the Go
|
||||||
# this builds that phase alone; the linter is never installed or run on
|
# linter over backend/.
|
||||||
# a developer host, where a shared result cache and a host-global lock
|
|
||||||
# make its answer untrustworthy.
|
|
||||||
#
|
#
|
||||||
# The phase is not the last stage in the file, so it is built only when
|
# No linter runs on the host: this builds the frontend-lint and lint
|
||||||
# --target names it. --no-cache because a cached lint layer is a lint
|
# stages of Dockerfile, the digest-pinned node and golangci-lint images.
|
||||||
# that did not run. The tag makes each build replace the previous image
|
# The first runs eslint; the second runs the backend's fmt-check and
|
||||||
# instead of leaving a dangling one behind.
|
# lint targets. --no-cache makes each linter really run every time
|
||||||
|
# rather than reuse an earlier result, and each stage is built for its
|
||||||
|
# checks alone, so no image is kept.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache \
|
timeout 300 docker build --no-cache --target frontend-lint \
|
||||||
--target lint \
|
--output type=cacheonly .
|
||||||
-t "$("$SCRIPT_DIR/projectname")-lint" .
|
timeout 300 docker build --no-cache --target lint \
|
||||||
|
--output type=cacheonly .
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+8
-10
@@ -1,19 +1,17 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/test: run the test suite. Testing is a phase of the Dockerfile
|
# script/test: run the test suite for the whole repo: the frontend at
|
||||||
# and this builds that phase alone, on the same terms as script/lint:
|
# the repo root, then the Go backend in backend/. Each half has its own
|
||||||
# --target because a phase that is not the last stage is built only when
|
# 30-second limit, and there is none around both: from a cold Go build
|
||||||
# named, --no-cache because a cached test layer is a test that did not
|
# cache, compiling the backend's tests with the race detector can take
|
||||||
# run, and a tag so each build replaces the previous image.
|
# 30 seconds on its own, and Go's -timeout leaves the compile out.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
docker build --no-cache \
|
script/frontend-test
|
||||||
--target test \
|
backend/script/test
|
||||||
-t "$("$SCRIPT_DIR/projectname")-test" .
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+3
-3
@@ -1,12 +1,12 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# script/tidy: run go mod tidy, which adds the modules the Go sources
|
# script/tidy: run go mod tidy in backend/, which adds the modules the
|
||||||
# import, drops those they no longer do, and updates go.sum.
|
# Go sources import, drops those they no longer do, and updates go.sum.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT/backend"
|
||||||
go mod tidy
|
go mod tidy
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
# The security headers REPO_POLICIES.md requires on every response.
|
# The security headers REPO_POLICIES.md requires on every response.
|
||||||
# configs/nginx.conf includes this file, which Dockerfile copies to
|
# nginx.conf includes this file, which Dockerfile copies to
|
||||||
# /etc/nginx/security-headers.conf. always sends each header on error
|
# /etc/nginx/security-headers.conf. always sends each header on error
|
||||||
# responses too.
|
# responses too.
|
||||||
|
|
||||||
@@ -8,11 +8,11 @@ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" alway
|
|||||||
# Scripts and styles load only from the page's own origin. Inline ones
|
# Scripts and styles load only from the page's own origin. Inline ones
|
||||||
# are blocked, style attributes in markup included, so style elements
|
# are blocked, style attributes in markup included, so style elements
|
||||||
# through classes or element.style. data: images are for the favicon
|
# through classes or element.style. data: images are for the favicon
|
||||||
# in web/index.html. connect-src is * because the browser checks each
|
# in index.html. connect-src is * because the browser checks each probe in
|
||||||
# probe in web/src/main.js against it, and also every redirect the probe
|
# src/main.js against it, and also every redirect the probe follows,
|
||||||
# follows, and several of those hosts redirect to others; a list of
|
# and several of those hosts redirect to others; a list of hosts here
|
||||||
# hosts here would block those probes. It also covers the reports the
|
# would block those probes. It also covers the reports the page sends
|
||||||
# page sends to its own origin.
|
# to its own origin.
|
||||||
add_header Content-Security-Policy "default-src 'self'; connect-src *; img-src 'self' data:; object-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'" always;
|
add_header Content-Security-Policy "default-src 'self'; connect-src *; img-src 'self' data:; object-src 'none'; base-uri 'none'; form-action 'none'; frame-ancestors 'none'" always;
|
||||||
|
|
||||||
add_header X-Frame-Options DENY always;
|
add_header X-Frame-Options DENY always;
|
||||||
+114
-215
@@ -30,39 +30,6 @@ export const CONFIG = {
|
|||||||
return [0, 1, 2, 3, 4, 5].map((i) => Math.round((d * i) / 5));
|
return [0, 1, 2, 3, 4, 5].map((i) => Math.round((d * i) / 5));
|
||||||
},
|
},
|
||||||
canvasHeight: 96,
|
canvasHeight: 96,
|
||||||
// A latency figure and its sparkline take the color of the first entry
|
|
||||||
// whose limit, in ms, the latency is below.
|
|
||||||
latencyColors: [
|
|
||||||
{ below: 50, hex: "#22c55e", className: "text-green-500" },
|
|
||||||
{ below: 100, hex: "#84cc16", className: "text-lime-500" },
|
|
||||||
{ below: 200, hex: "#eab308", className: "text-yellow-500" },
|
|
||||||
{ below: 500, hex: "#f97316", className: "text-orange-500" },
|
|
||||||
{ below: Infinity, hex: "#ef4444", className: "text-red-500" },
|
|
||||||
],
|
|
||||||
// The health is offline when more than offlineTimeouts WAN hosts timed
|
|
||||||
// out or were unreachable and at most offlineReachable answered;
|
|
||||||
// otherwise degraded when more than degradedTimeouts timed out or were
|
|
||||||
// unreachable; otherwise slow when more than slowHosts answered after
|
|
||||||
// more than slowLatency ms.
|
|
||||||
offlineTimeouts: 10,
|
|
||||||
offlineReachable: 4,
|
|
||||||
degradedTimeouts: 4,
|
|
||||||
slowHosts: 3,
|
|
||||||
slowLatency: 1000,
|
|
||||||
// The debug log keeps its last maxLogEntries lines.
|
|
||||||
maxLogEntries: 1000,
|
|
||||||
// A gateway candidate that has not answered after gatewayTimeout ms is
|
|
||||||
// passed over.
|
|
||||||
gatewayTimeout: 1500,
|
|
||||||
// When no WAN host answers, the recovery probe checks recoveryProbeHosts
|
|
||||||
// random ones every recoveryProbeInterval ms.
|
|
||||||
recoveryProbeHosts: 4,
|
|
||||||
recoveryProbeInterval: 500,
|
|
||||||
// The rows are sorted after the first round that is not discarded, then
|
|
||||||
// every roundsPerSort rounds.
|
|
||||||
roundsPerSort: 10,
|
|
||||||
// The sparklines are sized and drawn again resizeDelay ms after start.
|
|
||||||
resizeDelay: 100,
|
|
||||||
};
|
};
|
||||||
|
|
||||||
// WAN endpoints to monitor. These are used for the aggregate health/stats
|
// WAN endpoints to monitor. These are used for the aggregate health/stats
|
||||||
@@ -144,15 +111,12 @@ const WAN_HOSTS = [
|
|||||||
|
|
||||||
const debugLog = [];
|
const debugLog = [];
|
||||||
|
|
||||||
export const log = (() => {
|
const log = (() => {
|
||||||
function append(level, message) {
|
function append(level, message) {
|
||||||
const entry = { timestamp: new Date(), level, message };
|
debugLog.push({ timestamp: new Date(), level, message });
|
||||||
debugLog.push(entry);
|
if (debugLog.length > 1000) debugLog.splice(0, debugLog.length - 1000);
|
||||||
if (debugLog.length > CONFIG.maxLogEntries)
|
|
||||||
debugLog.splice(0, debugLog.length - CONFIG.maxLogEntries);
|
|
||||||
const panel = document.getElementById("debug-panel");
|
const panel = document.getElementById("debug-panel");
|
||||||
if (panel && !panel.classList.contains("hidden"))
|
if (panel && !panel.classList.contains("hidden")) renderDebugLog();
|
||||||
appendDebugLogLine(entry);
|
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
error: (msg) => append("error", msg),
|
error: (msg) => append("error", msg),
|
||||||
@@ -210,10 +174,7 @@ async function detectGateway() {
|
|||||||
const result = await Promise.any(
|
const result = await Promise.any(
|
||||||
GATEWAY_CANDIDATES.map(async (url) => {
|
GATEWAY_CANDIDATES.map(async (url) => {
|
||||||
const controller = new AbortController();
|
const controller = new AbortController();
|
||||||
const timeoutId = setTimeout(
|
const timeoutId = setTimeout(() => controller.abort(), 1500);
|
||||||
() => controller.abort(),
|
|
||||||
CONFIG.gatewayTimeout,
|
|
||||||
);
|
|
||||||
try {
|
try {
|
||||||
await fetch(url, {
|
await fetch(url, {
|
||||||
method: "GET",
|
method: "GET",
|
||||||
@@ -238,35 +199,6 @@ async function detectGateway() {
|
|||||||
|
|
||||||
// --- App State ---------------------------------------------------------------
|
// --- App State ---------------------------------------------------------------
|
||||||
|
|
||||||
// The min, max, median and average of latencies, a list of numbers, or all
|
|
||||||
// null when it is empty. The median of an even count is the mean of the
|
|
||||||
// middle two; it and the average are rounded.
|
|
||||||
function latencyStats(latencies) {
|
|
||||||
if (latencies.length === 0)
|
|
||||||
return { min: null, max: null, med: null, avg: null };
|
|
||||||
const sorted = [...latencies].sort((a, b) => a - b);
|
|
||||||
const mid = Math.floor(sorted.length / 2);
|
|
||||||
return {
|
|
||||||
min: sorted[0],
|
|
||||||
max: sorted[sorted.length - 1],
|
|
||||||
med:
|
|
||||||
sorted.length % 2
|
|
||||||
? sorted[mid]
|
|
||||||
: Math.round((sorted[mid - 1] + sorted[mid]) / 2),
|
|
||||||
avg: Math.round(
|
|
||||||
latencies.reduce((a, b) => a + b, 0) / latencies.length,
|
|
||||||
),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// The status a check's error gives its host: "error" for a timeout,
|
|
||||||
// "offline" for any other error, and "online" for none.
|
|
||||||
function statusFromError(error) {
|
|
||||||
if (error === "timeout") return "error";
|
|
||||||
if (error) return "offline";
|
|
||||||
return "online";
|
|
||||||
}
|
|
||||||
|
|
||||||
export class HostState {
|
export class HostState {
|
||||||
constructor(host, pinned = false) {
|
constructor(host, pinned = false) {
|
||||||
this.name = host.name;
|
this.name = host.name;
|
||||||
@@ -275,15 +207,9 @@ export class HostState {
|
|||||||
// error }, or a round skipped while paused, { timestamp,
|
// error }, or a round skipped while paused, { timestamp,
|
||||||
// latency: null, paused: true }.
|
// latency: null, paused: true }.
|
||||||
this.history = [];
|
this.history = [];
|
||||||
// The last 100 (CONFIG.maxHistoryPoints) entries the interval menu
|
|
||||||
// has cleared from the history, which the reporter may not have
|
|
||||||
// sent yet.
|
|
||||||
this.clearedHistory = [];
|
|
||||||
this.lastLatency = null;
|
this.lastLatency = null;
|
||||||
this.status = "pending"; // 'online' | 'offline' | 'error' | 'pending'
|
this.status = "pending"; // 'online' | 'offline' | 'error' | 'pending'
|
||||||
this.pinned = pinned;
|
this.pinned = pinned;
|
||||||
// How many recorded checks in a row have failed, up to the last one.
|
|
||||||
this.consecutiveFailures = 0;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
pushSample(timestamp, result) {
|
pushSample(timestamp, result) {
|
||||||
@@ -294,10 +220,9 @@ export class HostState {
|
|||||||
});
|
});
|
||||||
this._trim();
|
this._trim();
|
||||||
this.lastLatency = result.latency;
|
this.lastLatency = result.latency;
|
||||||
this.status = statusFromError(result.error);
|
if (result.error === "timeout") this.status = "error";
|
||||||
this.consecutiveFailures = result.error
|
else if (result.error) this.status = "offline";
|
||||||
? this.consecutiveFailures + 1
|
else this.status = "online";
|
||||||
: 0;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
pushPaused(timestamp) {
|
pushPaused(timestamp) {
|
||||||
@@ -305,26 +230,36 @@ export class HostState {
|
|||||||
this._trim();
|
this._trim();
|
||||||
}
|
}
|
||||||
|
|
||||||
// What the interval menu does to each host: the history starts over and
|
averageLatency() {
|
||||||
// the host waits for its next check. The history's entries move to
|
const valid = this.history.filter((p) => p.latency !== null);
|
||||||
// clearedHistory, so the reporter still sends those it has not.
|
if (valid.length === 0) return null;
|
||||||
reset() {
|
return Math.round(
|
||||||
this.clearedHistory = [...this.clearedHistory, ...this.history].slice(
|
valid.reduce((s, p) => s + p.latency, 0) / valid.length,
|
||||||
-CONFIG.maxHistoryPoints,
|
|
||||||
);
|
);
|
||||||
this.history = [];
|
|
||||||
this.lastLatency = null;
|
|
||||||
this.status = "pending";
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// The min, max, median and average latency of the checks in the history
|
minLatency() {
|
||||||
// that got an answer.
|
const valid = this.history.filter((p) => p.latency !== null);
|
||||||
historyStats() {
|
if (valid.length === 0) return null;
|
||||||
return latencyStats(
|
return Math.min(...valid.map((p) => p.latency));
|
||||||
this.history
|
}
|
||||||
|
|
||||||
|
maxLatency() {
|
||||||
|
const valid = this.history.filter((p) => p.latency !== null);
|
||||||
|
if (valid.length === 0) return null;
|
||||||
|
return Math.max(...valid.map((p) => p.latency));
|
||||||
|
}
|
||||||
|
|
||||||
|
medianLatency() {
|
||||||
|
const sorted = this.history
|
||||||
.filter((p) => p.latency !== null)
|
.filter((p) => p.latency !== null)
|
||||||
.map((p) => p.latency),
|
.map((p) => p.latency)
|
||||||
);
|
.sort((a, b) => a - b);
|
||||||
|
if (sorted.length === 0) return null;
|
||||||
|
const mid = Math.floor(sorted.length / 2);
|
||||||
|
return sorted.length % 2
|
||||||
|
? sorted[mid]
|
||||||
|
: Math.round((sorted[mid - 1] + sorted[mid]) / 2);
|
||||||
}
|
}
|
||||||
|
|
||||||
_trim() {
|
_trim() {
|
||||||
@@ -353,13 +288,33 @@ export class AppState {
|
|||||||
|
|
||||||
/** WAN-only stats from latest sample (excludes local) */
|
/** WAN-only stats from latest sample (excludes local) */
|
||||||
wanStats() {
|
wanStats() {
|
||||||
const latencies = this.wan
|
const reachable = this.wan.filter((h) => h.lastLatency !== null);
|
||||||
.filter((h) => h.lastLatency !== null)
|
const latencies = reachable.map((h) => h.lastLatency);
|
||||||
.map((h) => h.lastLatency);
|
const total = this.wan.length;
|
||||||
|
if (latencies.length === 0)
|
||||||
|
return {
|
||||||
|
reachable: 0,
|
||||||
|
total,
|
||||||
|
min: null,
|
||||||
|
max: null,
|
||||||
|
med: null,
|
||||||
|
avg: null,
|
||||||
|
};
|
||||||
|
const sorted = [...latencies].sort((a, b) => a - b);
|
||||||
|
const mid = Math.floor(sorted.length / 2);
|
||||||
|
const med =
|
||||||
|
sorted.length % 2
|
||||||
|
? sorted[mid]
|
||||||
|
: Math.round((sorted[mid - 1] + sorted[mid]) / 2);
|
||||||
return {
|
return {
|
||||||
reachable: latencies.length,
|
reachable: latencies.length,
|
||||||
total: this.wan.length,
|
total,
|
||||||
...latencyStats(latencies),
|
min: Math.min(...latencies),
|
||||||
|
max: Math.max(...latencies),
|
||||||
|
med,
|
||||||
|
avg: Math.round(
|
||||||
|
latencies.reduce((a, b) => a + b, 0) / latencies.length,
|
||||||
|
),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -385,16 +340,12 @@ export class AppState {
|
|||||||
const timeouts = this.wan.filter(
|
const timeouts = this.wan.filter(
|
||||||
(h) => h.status === "error" || h.status === "offline",
|
(h) => h.status === "error" || h.status === "offline",
|
||||||
).length;
|
).length;
|
||||||
if (
|
if (timeouts > 10 && reachable <= 4) return "offline";
|
||||||
timeouts > CONFIG.offlineTimeouts &&
|
if (timeouts > 4) return "degraded";
|
||||||
reachable <= CONFIG.offlineReachable
|
|
||||||
)
|
|
||||||
return "offline";
|
|
||||||
if (timeouts > CONFIG.degradedTimeouts) return "degraded";
|
|
||||||
const slow = this.wan.filter(
|
const slow = this.wan.filter(
|
||||||
(h) => h.lastLatency !== null && h.lastLatency > CONFIG.slowLatency,
|
(h) => h.lastLatency !== null && h.lastLatency > 1000,
|
||||||
).length;
|
).length;
|
||||||
if (slow > CONFIG.slowHosts) return "slow";
|
if (slow > 3) return "slow";
|
||||||
return "healthy";
|
return "healthy";
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -453,12 +404,9 @@ function getClientId() {
|
|||||||
|
|
||||||
// Build the delta report body the backend decodes, plus the new per-host
|
// Build the delta report body the backend decodes, plus the new per-host
|
||||||
// high-water marks. Pure function of the passed state: `hosts` is an array
|
// high-water marks. Pure function of the passed state: `hosts` is an array
|
||||||
// of { name, url, status, clearedHistory, history }, `since` maps a host url
|
// of { name, url, status, history }, `since` maps a host url to the Unix-ms
|
||||||
// to the Unix-ms timestamp of the last sample already reported for it, and
|
// timestamp of the last sample already reported for it, and `now` is a Date.
|
||||||
// `now` is a Date. Only non-paused samples newer than the mark are included,
|
// Only non-paused samples newer than the mark are included. Returns null
|
||||||
// from clearedHistory as well as history. A host whose status is pending,
|
|
||||||
// as the interval menu leaves it until its next check, is sent with the
|
|
||||||
// status of its last check, which is the last sample sent. Returns null
|
|
||||||
// when no host has an unreported sample.
|
// when no host has an unreported sample.
|
||||||
export function buildReport(hosts, clientId, now, since) {
|
export function buildReport(hosts, clientId, now, since) {
|
||||||
const reportHosts = [];
|
const reportHosts = [];
|
||||||
@@ -467,7 +415,7 @@ export function buildReport(hosts, clientId, now, since) {
|
|||||||
const mark = since.get(host.url) ?? 0;
|
const mark = since.get(host.url) ?? 0;
|
||||||
const samples = [];
|
const samples = [];
|
||||||
let high = mark;
|
let high = mark;
|
||||||
for (const p of [...host.clearedHistory, ...host.history]) {
|
for (const p of host.history) {
|
||||||
if (p.paused) continue;
|
if (p.paused) continue;
|
||||||
if (p.timestamp <= mark) continue;
|
if (p.timestamp <= mark) continue;
|
||||||
samples.push({
|
samples.push({
|
||||||
@@ -481,10 +429,7 @@ export function buildReport(hosts, clientId, now, since) {
|
|||||||
reportHosts.push({
|
reportHosts.push({
|
||||||
name: host.name,
|
name: host.name,
|
||||||
url: host.url,
|
url: host.url,
|
||||||
status:
|
status: host.status,
|
||||||
host.status === "pending"
|
|
||||||
? statusFromError(samples.at(-1).error)
|
|
||||||
: host.status,
|
|
||||||
history: samples,
|
history: samples,
|
||||||
});
|
});
|
||||||
marks.set(host.url, high);
|
marks.set(host.url, high);
|
||||||
@@ -511,7 +456,7 @@ export function buildReport(hosts, clientId, now, since) {
|
|||||||
// Only one report is ever in flight, and it is abandoned after half the
|
// Only one report is ever in flight, and it is abandoned after half the
|
||||||
// interval, so a slow POST can neither overlap the next report (which would
|
// interval, so a slow POST can neither overlap the next report (which would
|
||||||
// carry the same samples) nor stall reporting for good.
|
// carry the same samples) nor stall reporting for good.
|
||||||
export class Reporter {
|
class Reporter {
|
||||||
constructor(state, clientId, intervalMs) {
|
constructor(state, clientId, intervalMs) {
|
||||||
this.state = state;
|
this.state = state;
|
||||||
this.clientId = clientId;
|
this.clientId = clientId;
|
||||||
@@ -569,13 +514,7 @@ export class Reporter {
|
|||||||
|
|
||||||
// Checks one target. The check times out after CONFIG.requestTimeout; the
|
// Checks one target. The check times out after CONFIG.requestTimeout; the
|
||||||
// caller can give it up sooner through the optional signal, which also ends
|
// caller can give it up sooner through the optional signal, which also ends
|
||||||
// it as a timeout. A failed check's reason says what went wrong and after
|
// it as a timeout.
|
||||||
// how long; a check that answered has none.
|
|
||||||
//
|
|
||||||
// The URL is fetched as written, with nothing added to it: the Hetzner
|
|
||||||
// speed-test servers close the connection without an answer when the URL
|
|
||||||
// has a query string, and cache: "no-store" keeps the browser's cache out
|
|
||||||
// of the measurement.
|
|
||||||
export async function measureLatency(url, signal) {
|
export async function measureLatency(url, signal) {
|
||||||
const controller = new AbortController();
|
const controller = new AbortController();
|
||||||
const timeoutId = setTimeout(
|
const timeoutId = setTimeout(
|
||||||
@@ -584,10 +523,13 @@ export async function measureLatency(url, signal) {
|
|||||||
);
|
);
|
||||||
signal?.addEventListener("abort", () => controller.abort());
|
signal?.addEventListener("abort", () => controller.abort());
|
||||||
|
|
||||||
|
const targetUrl = new URL(url);
|
||||||
|
targetUrl.searchParams.set("_cb", Date.now().toString());
|
||||||
|
|
||||||
const start = performance.now();
|
const start = performance.now();
|
||||||
|
|
||||||
try {
|
try {
|
||||||
await fetch(url, {
|
await fetch(targetUrl.toString(), {
|
||||||
method: "GET",
|
method: "GET",
|
||||||
mode: "no-cors",
|
mode: "no-cors",
|
||||||
cache: "no-store",
|
cache: "no-store",
|
||||||
@@ -596,28 +538,18 @@ export async function measureLatency(url, signal) {
|
|||||||
const latency = Math.round(performance.now() - start);
|
const latency = Math.round(performance.now() - start);
|
||||||
clearTimeout(timeoutId);
|
clearTimeout(timeoutId);
|
||||||
if (latency > CONFIG.maxLatency) {
|
if (latency > CONFIG.maxLatency) {
|
||||||
return {
|
log.error(`${url} timeout (${latency}ms > ${CONFIG.maxLatency}ms)`);
|
||||||
latency: null,
|
return { latency: null, error: "timeout" };
|
||||||
error: "timeout",
|
|
||||||
reason: `answered after ${latency} ms, over the ${CONFIG.maxLatency} ms limit`,
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
return { latency, error: null, reason: null };
|
return { latency, error: null };
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const took = Math.round(performance.now() - start);
|
|
||||||
clearTimeout(timeoutId);
|
clearTimeout(timeoutId);
|
||||||
if (err.name === "AbortError") {
|
if (err.name === "AbortError") {
|
||||||
return {
|
log.error(`${url} timeout (aborted)`);
|
||||||
latency: null,
|
return { latency: null, error: "timeout" };
|
||||||
error: "timeout",
|
|
||||||
reason: `timed out after ${took} ms (limit ${CONFIG.requestTimeout} ms)`,
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
return {
|
log.error(`${url} unreachable`);
|
||||||
latency: null,
|
return { latency: null, error: "unreachable" };
|
||||||
error: "unreachable",
|
|
||||||
reason: `network error (${err.name}: ${err.message}) after ${took} ms`,
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -625,13 +557,21 @@ export async function measureLatency(url, signal) {
|
|||||||
|
|
||||||
export function latencyHex(latency) {
|
export function latencyHex(latency) {
|
||||||
if (latency === null) return "#6b7280";
|
if (latency === null) return "#6b7280";
|
||||||
return CONFIG.latencyColors.find((c) => latency < c.below).hex;
|
if (latency < 50) return "#22c55e";
|
||||||
|
if (latency < 100) return "#84cc16";
|
||||||
|
if (latency < 200) return "#eab308";
|
||||||
|
if (latency < 500) return "#f97316";
|
||||||
|
return "#ef4444";
|
||||||
}
|
}
|
||||||
|
|
||||||
export function latencyClass(latency, status) {
|
export function latencyClass(latency, status) {
|
||||||
if (status === "offline" || status === "error" || latency === null)
|
if (status === "offline" || status === "error" || latency === null)
|
||||||
return "text-gray-500";
|
return "text-gray-500";
|
||||||
return CONFIG.latencyColors.find((c) => latency < c.below).className;
|
if (latency < 50) return "text-green-500";
|
||||||
|
if (latency < 100) return "text-lime-500";
|
||||||
|
if (latency < 200) return "text-yellow-500";
|
||||||
|
if (latency < 500) return "text-orange-500";
|
||||||
|
return "text-red-500";
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Sparkline Renderer ------------------------------------------------------
|
// --- Sparkline Renderer ------------------------------------------------------
|
||||||
@@ -905,7 +845,7 @@ function buildUI(state) {
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<footer class="mt-8 text-center text-gray-600 text-xs">
|
<footer class="mt-8 text-center text-gray-600 text-xs">
|
||||||
<p>Latency measured via GET requests | CORS restrictions may affect some measurements</p>
|
<p>Latency measured via GET requests | IPv4 only | CORS restrictions may affect some measurements</p>
|
||||||
<p class="mt-2">
|
<p class="mt-2">
|
||||||
<span class="inline-block w-3 h-3 rounded-full bg-green-500 mr-1 align-middle"></span><50ms
|
<span class="inline-block w-3 h-3 rounded-full bg-green-500 mr-1 align-middle"></span><50ms
|
||||||
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span><100ms
|
<span class="inline-block w-3 h-3 rounded-full bg-lime-500 mr-1 ml-3 align-middle"></span><100ms
|
||||||
@@ -972,7 +912,10 @@ function updateHostRow(host, index) {
|
|||||||
latencyEl.innerHTML = `<span class="text-gray-500">---</span>`;
|
latencyEl.innerHTML = `<span class="text-gray-500">---</span>`;
|
||||||
}
|
}
|
||||||
|
|
||||||
const { min, med, avg, max } = host.historyStats();
|
const avg = host.averageLatency();
|
||||||
|
const med = host.medianLatency();
|
||||||
|
const min = host.minLatency();
|
||||||
|
const max = host.maxLatency();
|
||||||
if (host.status === "online" && avg !== null) {
|
if (host.status === "online" && avg !== null) {
|
||||||
statusEl.innerHTML = statusStatsHTML([
|
statusEl.innerHTML = statusStatsHTML([
|
||||||
["min", min],
|
["min", min],
|
||||||
@@ -1109,8 +1052,9 @@ function updateHealthBox(state) {
|
|||||||
|
|
||||||
// --- Debug Log Renderer ------------------------------------------------------
|
// --- Debug Log Renderer ------------------------------------------------------
|
||||||
|
|
||||||
// One line of the debug log panel, its message set as text.
|
function renderDebugLog() {
|
||||||
function debugLogLine(entry) {
|
const el = document.getElementById("debug-log");
|
||||||
|
if (!el) return;
|
||||||
const levelColors = {
|
const levelColors = {
|
||||||
error: "text-red-400",
|
error: "text-red-400",
|
||||||
warning: "text-orange-400",
|
warning: "text-orange-400",
|
||||||
@@ -1118,43 +1062,19 @@ function debugLogLine(entry) {
|
|||||||
info: "text-gray-300",
|
info: "text-gray-300",
|
||||||
debug: "text-gray-500",
|
debug: "text-gray-500",
|
||||||
};
|
};
|
||||||
|
el.replaceChildren(
|
||||||
|
...debugLog.map((entry) => {
|
||||||
const ts = formatUTCTimestamp(entry.timestamp);
|
const ts = formatUTCTimestamp(entry.timestamp);
|
||||||
const lvl = entry.level.toUpperCase().padEnd(7);
|
const lvl = entry.level.toUpperCase().padEnd(7);
|
||||||
const line = document.createElement("div");
|
const line = document.createElement("div");
|
||||||
line.className = levelColors[entry.level] || "text-gray-400";
|
line.className = levelColors[entry.level] || "text-gray-400";
|
||||||
line.textContent = `${ts} ${lvl} ${entry.message}`;
|
line.textContent = `${ts} ${lvl} ${entry.message}`;
|
||||||
return line;
|
return line;
|
||||||
}
|
}),
|
||||||
|
);
|
||||||
// Draws every line of the debug log into the panel, when it is opened.
|
|
||||||
function renderDebugLog() {
|
|
||||||
const el = document.getElementById("debug-log");
|
|
||||||
if (!el) return;
|
|
||||||
el.replaceChildren(...debugLog.map((entry) => debugLogLine(entry)));
|
|
||||||
el.scrollTop = el.scrollHeight;
|
el.scrollTop = el.scrollHeight;
|
||||||
}
|
}
|
||||||
|
|
||||||
// The request that scrolls the open panel to its newest line in the next
|
|
||||||
// frame. One waits at most, also in a hidden tab, where frames do not run.
|
|
||||||
let debugLogScrollFrame = 0;
|
|
||||||
|
|
||||||
// Adds entry to the open panel as its last line, and drops the first line
|
|
||||||
// once the panel holds more than maxLogEntries. A check that ends while the
|
|
||||||
// page is busy is timed only when the page is done, so the lines already
|
|
||||||
// shown are left as they are, and the panel is scrolled once a frame
|
|
||||||
// rather than once a line: scrolling it lays out all of its lines again.
|
|
||||||
function appendDebugLogLine(entry) {
|
|
||||||
const el = document.getElementById("debug-log");
|
|
||||||
if (!el) return;
|
|
||||||
el.append(debugLogLine(entry));
|
|
||||||
if (el.children.length > CONFIG.maxLogEntries)
|
|
||||||
el.removeChild(el.firstElementChild);
|
|
||||||
cancelAnimationFrame(debugLogScrollFrame);
|
|
||||||
debugLogScrollFrame = requestAnimationFrame(() => {
|
|
||||||
el.scrollTop = el.scrollHeight;
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
// --- Sorting -----------------------------------------------------------------
|
// --- Sorting -----------------------------------------------------------------
|
||||||
|
|
||||||
// Sort WAN hosts: pinned hosts first (alphabetically), then unpinned hosts
|
// Sort WAN hosts: pinned hosts first (alphabetically), then unpinned hosts
|
||||||
@@ -1207,26 +1127,6 @@ function sortAndRebuildWAN(state) {
|
|||||||
|
|
||||||
// --- Main Loop ---------------------------------------------------------------
|
// --- Main Loop ---------------------------------------------------------------
|
||||||
|
|
||||||
// Writes one line to the browser console, and the same line to the debug
|
|
||||||
// log, for a check the page is about to record: for every check that
|
|
||||||
// failed, and for a check that answered after checks that failed. Called
|
|
||||||
// before host.pushSample, while host.consecutiveFailures still counts the
|
|
||||||
// checks before this one.
|
|
||||||
function logCheck(host, result) {
|
|
||||||
const target = `${host.name} ${host.url} at ${new Date().toISOString()}`;
|
|
||||||
const failed = host.consecutiveFailures;
|
|
||||||
if (result.error) {
|
|
||||||
const line = `netwatch: check failed: ${target}: ${result.reason}`;
|
|
||||||
console.error(line);
|
|
||||||
log.error(line);
|
|
||||||
} else if (failed > 0) {
|
|
||||||
const checks = failed === 1 ? "check" : "checks";
|
|
||||||
const line = `netwatch: target recovered: ${target}: answered after ${result.latency} ms, following ${failed} failed ${checks} in a row`;
|
|
||||||
console.info(line);
|
|
||||||
log.info(line);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function tick(state, signal, onOffline) {
|
export async function tick(state, signal, onOffline) {
|
||||||
const ts = Date.now();
|
const ts = Date.now();
|
||||||
|
|
||||||
@@ -1260,7 +1160,6 @@ export async function tick(state, signal, onOffline) {
|
|||||||
if (state.paused || signal.aborted || state.tickCount === 0) {
|
if (state.paused || signal.aborted || state.tickCount === 0) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
logCheck(host, r);
|
|
||||||
host.pushSample(ts, r);
|
host.pushSample(ts, r);
|
||||||
updateHostRow(host, state.allHosts.indexOf(host));
|
updateHostRow(host, state.allHosts.indexOf(host));
|
||||||
log.debug(`${host.name}: ${r.error ? r.error : r.latency + "ms"}`);
|
log.debug(`${host.name}: ${r.error ? r.error : r.latency + "ms"}`);
|
||||||
@@ -1283,9 +1182,8 @@ export async function tick(state, signal, onOffline) {
|
|||||||
// rows whose check ended before the resume still read "paused"
|
// rows whose check ended before the resume still read "paused"
|
||||||
state.allHosts.forEach((host, i) => updateHostRow(host, i));
|
state.allHosts.forEach((host, i) => updateHostRow(host, i));
|
||||||
|
|
||||||
// Sort after the first real check, then every CONFIG.roundsPerSort
|
// Sort after the first real check, then every 10 ticks thereafter
|
||||||
// ticks thereafter
|
if (state.tickCount === 2 || state.tickCount % 10 === 1) {
|
||||||
if (state.tickCount === 2 || state.tickCount % CONFIG.roundsPerSort === 1) {
|
|
||||||
sortAndRebuildWAN(state);
|
sortAndRebuildWAN(state);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1308,10 +1206,9 @@ export async function tick(state, signal, onOffline) {
|
|||||||
|
|
||||||
// --- Recovery Probe ----------------------------------------------------------
|
// --- Recovery Probe ----------------------------------------------------------
|
||||||
|
|
||||||
// When offline, check CONFIG.recoveryProbeHosts random WAN hosts every
|
// When offline, check 4 random WAN hosts every 500ms, giving up the checks
|
||||||
// CONFIG.recoveryProbeInterval ms, giving up the checks started one interval
|
// started 500ms before, so at most 4 are ever waiting. As soon as one
|
||||||
// before, so at most that many are ever waiting. As soon as one answers,
|
// answers, stop probing and start a new round at once.
|
||||||
// stop probing and start a new round at once.
|
|
||||||
function startRecoveryProbe(state, startRounds) {
|
function startRecoveryProbe(state, startRounds) {
|
||||||
if (state._recoveryProbeId) return; // already running
|
if (state._recoveryProbeId) return; // already running
|
||||||
const candidates = [...state.wan];
|
const candidates = [...state.wan];
|
||||||
@@ -1319,7 +1216,7 @@ function startRecoveryProbe(state, startRounds) {
|
|||||||
const j = Math.floor(Math.random() * (i + 1));
|
const j = Math.floor(Math.random() * (i + 1));
|
||||||
[candidates[i], candidates[j]] = [candidates[j], candidates[i]];
|
[candidates[i], candidates[j]] = [candidates[j], candidates[i]];
|
||||||
}
|
}
|
||||||
const canaries = candidates.slice(0, CONFIG.recoveryProbeHosts);
|
const canaries = candidates.slice(0, 4);
|
||||||
log.notice(
|
log.notice(
|
||||||
`Recovery probe started (${canaries.map((h) => h.name).join(", ")})`,
|
`Recovery probe started (${canaries.map((h) => h.name).join(", ")})`,
|
||||||
);
|
);
|
||||||
@@ -1336,7 +1233,7 @@ function startRecoveryProbe(state, startRounds) {
|
|||||||
startRounds();
|
startRounds();
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}, CONFIG.recoveryProbeInterval);
|
}, 500);
|
||||||
}
|
}
|
||||||
|
|
||||||
function stopRecoveryProbe(state) {
|
function stopRecoveryProbe(state) {
|
||||||
@@ -1555,7 +1452,9 @@ async function init() {
|
|||||||
|
|
||||||
// Clear all host history and reset state
|
// Clear all host history and reset state
|
||||||
for (const host of state.allHosts) {
|
for (const host of state.allHosts) {
|
||||||
host.reset();
|
host.history = [];
|
||||||
|
host.lastLatency = null;
|
||||||
|
host.status = "pending";
|
||||||
}
|
}
|
||||||
state.tickCount = 0;
|
state.tickCount = 0;
|
||||||
|
|
||||||
@@ -1598,7 +1497,7 @@ async function init() {
|
|||||||
});
|
});
|
||||||
|
|
||||||
window.addEventListener("resize", () => handleResize(state));
|
window.addEventListener("resize", () => handleResize(state));
|
||||||
setTimeout(() => handleResize(state), CONFIG.resizeDelay);
|
setTimeout(() => handleResize(state), 100);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Bootstrap only when loaded as the page: a real DOM containing the #app
|
// Bootstrap only when loaded as the page: a real DOM containing the #app
|
||||||
+28
-322
@@ -1,4 +1,4 @@
|
|||||||
// Unit tests for web/src/main.js, run with Node's built-in test runner by the
|
// Unit tests for src/main.js, run with Node's built-in test runner by the
|
||||||
// test script in package.json. Importing the module does not start the
|
// test script in package.json. Importing the module does not start the
|
||||||
// page.
|
// page.
|
||||||
|
|
||||||
@@ -13,29 +13,22 @@ import {
|
|||||||
humanDuration,
|
humanDuration,
|
||||||
latencyClass,
|
latencyClass,
|
||||||
latencyHex,
|
latencyHex,
|
||||||
log,
|
|
||||||
measureLatency,
|
measureLatency,
|
||||||
Reporter,
|
|
||||||
tick,
|
tick,
|
||||||
} from "../../web/src/main.js";
|
} from "../../src/main.js";
|
||||||
|
|
||||||
// There is no page here, so the tests stand in for it. The debug log looks
|
// There is no page here, so the tests stand in for it. The debug log looks
|
||||||
// for its panel by id and finds none, an element the page creates is a
|
// for its panel by id and finds none. Each element of a host's row that
|
||||||
// plain object, and no frame the page asks for ever comes. Each element of
|
// tick or greyOutUI draws into is a plain object, made the first time a
|
||||||
// a host's row that tick or greyOutUI draws into is a plain object, made
|
// test looks it up and kept in elements under its selector until the next
|
||||||
// the first time a test looks it up and kept in elements under its
|
// test starts. As on a page, writing its text replaces its markup; the
|
||||||
// selector until the next test starts. As on a page, writing its text
|
// status dot greyOutUI looks for in it is not there. Drawing a sparkline
|
||||||
// replaces its markup; the status dot greyOutUI looks for in it is not
|
// does nothing; it looks for the pixel ratio on window and finds none.
|
||||||
// there. Drawing a sparkline does nothing; it looks for the pixel ratio on
|
|
||||||
// window and finds none. In each test, console.error and console.info
|
|
||||||
// print nothing and keep what they are given.
|
|
||||||
const doNothing = () => {};
|
|
||||||
let elements;
|
let elements;
|
||||||
beforeEach((t) => {
|
beforeEach(() => {
|
||||||
elements = {};
|
elements = {};
|
||||||
t.mock.method(console, "error", doNothing);
|
|
||||||
t.mock.method(console, "info", doNothing);
|
|
||||||
});
|
});
|
||||||
|
const doNothing = () => {};
|
||||||
const canvasContext = {
|
const canvasContext = {
|
||||||
clearRect: doNothing,
|
clearRect: doNothing,
|
||||||
beginPath: doNothing,
|
beginPath: doNothing,
|
||||||
@@ -48,11 +41,8 @@ const canvasContext = {
|
|||||||
arc: doNothing,
|
arc: doNothing,
|
||||||
};
|
};
|
||||||
globalThis.window = {};
|
globalThis.window = {};
|
||||||
globalThis.requestAnimationFrame = doNothing;
|
|
||||||
globalThis.cancelAnimationFrame = doNothing;
|
|
||||||
globalThis.document = {
|
globalThis.document = {
|
||||||
getElementById: () => null,
|
getElementById: () => null,
|
||||||
createElement: () => ({}),
|
|
||||||
querySelector: (selector) =>
|
querySelector: (selector) =>
|
||||||
(elements[selector] ??= {
|
(elements[selector] ??= {
|
||||||
getContext: () => canvasContext,
|
getContext: () => canvasContext,
|
||||||
@@ -79,10 +69,8 @@ function statusText(state, host) {
|
|||||||
// Mocks the clock for test t, so that a check lasting seconds takes no real
|
// Mocks the clock for test t, so that a check lasting seconds takes no real
|
||||||
// time, and replaces fetch with targets that each answer after
|
// time, and replaces fetch with targets that each answer after
|
||||||
// answerAfter(url) milliseconds of that clock, or never when that is
|
// answerAfter(url) milliseconds of that clock, or never when that is
|
||||||
// Infinity. The target at unreachableUrl, if one is given, does not
|
// Infinity. Both are restored when the test ends.
|
||||||
// answer: after that time its fetch fails with the error a browser gives
|
function mockTargets(t, answerAfter) {
|
||||||
// the page for a network error. Both are restored when the test ends.
|
|
||||||
function mockTargets(t, answerAfter, unreachableUrl) {
|
|
||||||
t.mock.timers.enable({ apis: ["setTimeout", "Date"] });
|
t.mock.timers.enable({ apis: ["setTimeout", "Date"] });
|
||||||
t.mock.method(performance, "now", () => Date.now());
|
t.mock.method(performance, "now", () => Date.now());
|
||||||
t.mock.method(
|
t.mock.method(
|
||||||
@@ -90,12 +78,8 @@ function mockTargets(t, answerAfter, unreachableUrl) {
|
|||||||
"fetch",
|
"fetch",
|
||||||
(url, { signal }) =>
|
(url, { signal }) =>
|
||||||
new Promise((resolve, reject) => {
|
new Promise((resolve, reject) => {
|
||||||
const answer =
|
|
||||||
url === unreachableUrl
|
|
||||||
? () => reject(new TypeError("Failed to fetch"))
|
|
||||||
: resolve;
|
|
||||||
if (answerAfter(url) !== Infinity) {
|
if (answerAfter(url) !== Infinity) {
|
||||||
setTimeout(answer, answerAfter(url));
|
setTimeout(resolve, answerAfter(url));
|
||||||
}
|
}
|
||||||
signal.addEventListener("abort", () => reject(signal.reason));
|
signal.addEventListener("abort", () => reject(signal.reason));
|
||||||
}),
|
}),
|
||||||
@@ -123,7 +107,6 @@ for (const interval of [10000, 30000]) {
|
|||||||
assert.deepEqual(await settled(check), {
|
assert.deepEqual(await settled(check), {
|
||||||
latency: slowAnswer,
|
latency: slowAnswer,
|
||||||
error: null,
|
error: null,
|
||||||
reason: null,
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -137,37 +120,10 @@ for (const interval of [10000, 30000]) {
|
|||||||
assert.deepEqual(await settled(check), {
|
assert.deepEqual(await settled(check), {
|
||||||
latency: null,
|
latency: null,
|
||||||
error: "timeout",
|
error: "timeout",
|
||||||
reason: `timed out after ${timeout} ms (limit ${timeout} ms)`,
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// A browser can run a timer late, on a busy page or in a background tab.
|
|
||||||
// Moving the mocked clock on 30000ms at once runs the 24000ms timeout with
|
|
||||||
// the clock already at 30000ms.
|
|
||||||
test("at a 30000ms interval, a check whose timeout runs late gives how long the request took and the time limit", async (t) => {
|
|
||||||
CONFIG.updateInterval = 30000;
|
|
||||||
mockTargets(t, () => Infinity);
|
|
||||||
const check = measureLatency("https://target.test");
|
|
||||||
t.mock.timers.tick(30000);
|
|
||||||
assert.deepEqual(await settled(check), {
|
|
||||||
latency: null,
|
|
||||||
error: "timeout",
|
|
||||||
reason: "timed out after 30000 ms (limit 24000 ms)",
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
test("a check fetches the target's URL as written, with no query string added", async (t) => {
|
|
||||||
mockTargets(t, () => 10);
|
|
||||||
const check = measureLatency("https://fsn1-speed.hetzner.com");
|
|
||||||
t.mock.timers.tick(10);
|
|
||||||
assert.notEqual(await settled(check), "still waiting");
|
|
||||||
assert.equal(
|
|
||||||
fetch.mock.calls[0].arguments[0],
|
|
||||||
"https://fsn1-speed.hetzner.com",
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("at a 30000ms interval, a target answering after 1000ms shows in its row while another target's check is still waiting", async (t) => {
|
test("at a 30000ms interval, a target answering after 1000ms shows in its row while another target's check is still waiting", async (t) => {
|
||||||
CONFIG.updateInterval = 30000;
|
CONFIG.updateInterval = 30000;
|
||||||
const state = new AppState([
|
const state = new AppState([
|
||||||
@@ -176,7 +132,7 @@ test("at a 30000ms interval, a target answering after 1000ms shows in its row wh
|
|||||||
const answering = state.local[0];
|
const answering = state.local[0];
|
||||||
const waiting = state.wan[0];
|
const waiting = state.wan[0];
|
||||||
// No target but the answering one ever answers.
|
// No target but the answering one ever answers.
|
||||||
mockTargets(t, (url) => (url === answering.url ? 1000 : Infinity));
|
mockTargets(t, (url) => (url.startsWith(answering.url) ? 1000 : Infinity));
|
||||||
// The third tick: the first is discarded as a whole, and the second ends
|
// The third tick: the first is discarded as a whole, and the second ends
|
||||||
// by sorting the rows, which rebuilds a page that is not here.
|
// by sorting the rows, which rebuilds a page that is not here.
|
||||||
state.tickCount = 2;
|
state.tickCount = 2;
|
||||||
@@ -203,7 +159,7 @@ test("at a 30000ms interval, a check still waiting when its round is given up do
|
|||||||
{ name: "Answering", url: "https://answering.test" },
|
{ name: "Answering", url: "https://answering.test" },
|
||||||
]);
|
]);
|
||||||
const answering = state.local[0];
|
const answering = state.local[0];
|
||||||
mockTargets(t, (url) => (url === answering.url ? 1000 : Infinity));
|
mockTargets(t, (url) => (url.startsWith(answering.url) ? 1000 : Infinity));
|
||||||
state.tickCount = 2;
|
state.tickCount = 2;
|
||||||
|
|
||||||
const roundChecks = new AbortController();
|
const roundChecks = new AbortController();
|
||||||
@@ -221,7 +177,7 @@ test("at a 30000ms interval, a check still waiting when the user pauses does not
|
|||||||
{ name: "Answering", url: "https://answering.test" },
|
{ name: "Answering", url: "https://answering.test" },
|
||||||
]);
|
]);
|
||||||
const answering = state.local[0];
|
const answering = state.local[0];
|
||||||
mockTargets(t, (url) => (url === answering.url ? 1000 : Infinity));
|
mockTargets(t, (url) => (url.startsWith(answering.url) ? 1000 : Infinity));
|
||||||
state.tickCount = 2;
|
state.tickCount = 2;
|
||||||
|
|
||||||
const round = tick(state, new AbortController().signal);
|
const round = tick(state, new AbortController().signal);
|
||||||
@@ -238,7 +194,7 @@ test("at a 30000ms interval, a check in the first round does not show in its row
|
|||||||
{ name: "Answering", url: "https://answering.test" },
|
{ name: "Answering", url: "https://answering.test" },
|
||||||
]);
|
]);
|
||||||
const answering = state.local[0];
|
const answering = state.local[0];
|
||||||
mockTargets(t, (url) => (url === answering.url ? 1000 : Infinity));
|
mockTargets(t, (url) => (url.startsWith(answering.url) ? 1000 : Infinity));
|
||||||
|
|
||||||
const round = tick(state, new AbortController().signal);
|
const round = tick(state, new AbortController().signal);
|
||||||
t.mock.timers.tick(1000);
|
t.mock.timers.tick(1000);
|
||||||
@@ -252,7 +208,7 @@ test("at a 30000ms interval, after the user pauses and resumes during a round, n
|
|||||||
{ name: "Answering", url: "https://answering.test" },
|
{ name: "Answering", url: "https://answering.test" },
|
||||||
]);
|
]);
|
||||||
const answering = state.local[0];
|
const answering = state.local[0];
|
||||||
mockTargets(t, (url) => (url === answering.url ? 1000 : Infinity));
|
mockTargets(t, (url) => (url.startsWith(answering.url) ? 1000 : Infinity));
|
||||||
state.tickCount = 2;
|
state.tickCount = 2;
|
||||||
|
|
||||||
const round = tick(state, new AbortController().signal);
|
const round = tick(state, new AbortController().signal);
|
||||||
@@ -275,240 +231,6 @@ test("at a 30000ms interval, after the user pauses and resumes during a round, n
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// In the next tests a round checks one target, Target, at a 30000ms
|
|
||||||
// interval, so a check times out after 24000ms. The mocked clock starts at
|
|
||||||
// 1970-01-01T00:00:00.000Z.
|
|
||||||
|
|
||||||
// An app state with Target and no WAN targets, so its rounds check only
|
|
||||||
// Target, and whose next round is recorded: it is the third, as the first
|
|
||||||
// is discarded and the second ends by sorting the rows, which rebuilds a
|
|
||||||
// page that is not here. Returns it and Target.
|
|
||||||
function stateWithOneTarget() {
|
|
||||||
CONFIG.updateInterval = 30000;
|
|
||||||
const state = new AppState([
|
|
||||||
{ name: "Target", url: "https://target.test" },
|
|
||||||
]);
|
|
||||||
state.wan = [];
|
|
||||||
state.tickCount = 2;
|
|
||||||
return { state, target: state.local[0] };
|
|
||||||
}
|
|
||||||
|
|
||||||
// What this test wrote to the browser console with console[method].
|
|
||||||
function consoleLines(method) {
|
|
||||||
return console[method].mock.calls.map((call) => call.arguments[0]);
|
|
||||||
}
|
|
||||||
|
|
||||||
test("a check that fails with a network error writes one console line with the target, the time, the error and how long the request took", async (t) => {
|
|
||||||
const { state, target } = stateWithOneTarget();
|
|
||||||
mockTargets(t, () => 23, target.url);
|
|
||||||
const round = tick(state, new AbortController().signal);
|
|
||||||
t.mock.timers.tick(23);
|
|
||||||
assert.notEqual(await settled(round), "still waiting");
|
|
||||||
assert.deepEqual(consoleLines("error"), [
|
|
||||||
"netwatch: check failed: Target https://target.test at 1970-01-01T00:00:00.023Z: network error (TypeError: Failed to fetch) after 23 ms",
|
|
||||||
]);
|
|
||||||
assert.deepEqual(consoleLines("info"), []);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("a check that times out writes one console line with the target, the time, how long the request took and the time limit", async (t) => {
|
|
||||||
const { state } = stateWithOneTarget();
|
|
||||||
mockTargets(t, () => Infinity);
|
|
||||||
const round = tick(state, new AbortController().signal);
|
|
||||||
t.mock.timers.tick(24000);
|
|
||||||
assert.notEqual(await settled(round), "still waiting");
|
|
||||||
assert.deepEqual(consoleLines("error"), [
|
|
||||||
"netwatch: check failed: Target https://target.test at 1970-01-01T00:00:24.000Z: timed out after 24000 ms (limit 24000 ms)",
|
|
||||||
]);
|
|
||||||
assert.deepEqual(consoleLines("info"), []);
|
|
||||||
});
|
|
||||||
|
|
||||||
// An answer that took longer than CONFIG.maxLatency is recorded as a
|
|
||||||
// timeout. The limit is the check's own timeout, so such an answer is one
|
|
||||||
// that came in just as the check timed out; here it is lowered to 500ms.
|
|
||||||
test("a check answered over the time limit writes one console line with the target, the time, how long the answer took and the limit", async (t) => {
|
|
||||||
const { state } = stateWithOneTarget();
|
|
||||||
t.mock.getter(CONFIG, "maxLatency", () => 500);
|
|
||||||
mockTargets(t, () => 1000);
|
|
||||||
const round = tick(state, new AbortController().signal);
|
|
||||||
t.mock.timers.tick(1000);
|
|
||||||
assert.notEqual(await settled(round), "still waiting");
|
|
||||||
assert.deepEqual(consoleLines("error"), [
|
|
||||||
"netwatch: check failed: Target https://target.test at 1970-01-01T00:00:01.000Z: answered after 1000 ms, over the 500 ms limit",
|
|
||||||
]);
|
|
||||||
assert.deepEqual(consoleLines("info"), []);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("a check that answers writes nothing to the console", async (t) => {
|
|
||||||
const { state } = stateWithOneTarget();
|
|
||||||
mockTargets(t, () => 30);
|
|
||||||
const round = tick(state, new AbortController().signal);
|
|
||||||
t.mock.timers.tick(30);
|
|
||||||
assert.notEqual(await settled(round), "still waiting");
|
|
||||||
assert.deepEqual(consoleLines("error"), []);
|
|
||||||
assert.deepEqual(consoleLines("info"), []);
|
|
||||||
});
|
|
||||||
|
|
||||||
for (const [failed, checks] of [
|
|
||||||
[1, "1 failed check"],
|
|
||||||
[3, "3 failed checks"],
|
|
||||||
]) {
|
|
||||||
test(`a check that answers after ${checks} writes one console line saying the target recovered, and the next writes nothing`, async (t) => {
|
|
||||||
const { state, target } = stateWithOneTarget();
|
|
||||||
for (let i = 0; i < failed; i++) {
|
|
||||||
target.pushSample(Date.now(), {
|
|
||||||
latency: null,
|
|
||||||
error: "unreachable",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
mockTargets(t, () => 40);
|
|
||||||
for (let i = 0; i < 2; i++) {
|
|
||||||
const round = tick(state, new AbortController().signal);
|
|
||||||
t.mock.timers.tick(40);
|
|
||||||
assert.notEqual(await settled(round), "still waiting");
|
|
||||||
}
|
|
||||||
assert.deepEqual(consoleLines("info"), [
|
|
||||||
`netwatch: target recovered: Target https://target.test at 1970-01-01T00:00:00.040Z: answered after 40 ms, following ${checks} in a row`,
|
|
||||||
]);
|
|
||||||
assert.deepEqual(consoleLines("error"), []);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
test("a check that fails in the first round, which is discarded, writes nothing to the console", async (t) => {
|
|
||||||
const { state, target } = stateWithOneTarget();
|
|
||||||
state.tickCount = 0;
|
|
||||||
mockTargets(t, () => 23, target.url);
|
|
||||||
const round = tick(state, new AbortController().signal);
|
|
||||||
t.mock.timers.tick(23);
|
|
||||||
assert.notEqual(await settled(round), "still waiting");
|
|
||||||
assert.deepEqual(consoleLines("error"), []);
|
|
||||||
assert.deepEqual(consoleLines("info"), []);
|
|
||||||
});
|
|
||||||
|
|
||||||
// The reports Reporter sent in this test, each as the hosts it sent.
|
|
||||||
function reportedHosts() {
|
|
||||||
return fetch.mock.calls.map(
|
|
||||||
(call) => JSON.parse(call.arguments[1].body).hosts,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
// In the next two tests every report is delivered, and only Target has
|
|
||||||
// checks to report. target.reset() does to Target what the interval menu
|
|
||||||
// does to each target.
|
|
||||||
|
|
||||||
// The interval changes after the first report. The second report is sent
|
|
||||||
// before Target's first check since then, and the third after it. A fourth
|
|
||||||
// would hold no check the first three did not, so it is not sent.
|
|
||||||
test("checks recorded before an interval change and not yet reported go in the next report, with the status of the last check, and no check is reported twice", async (t) => {
|
|
||||||
const state = new AppState([
|
|
||||||
{ name: "Target", url: "https://target.test" },
|
|
||||||
]);
|
|
||||||
const target = state.local[0];
|
|
||||||
const reporter = new Reporter(state, "client", CONFIG.reportInterval);
|
|
||||||
t.mock.method(globalThis, "fetch", async () => ({ ok: true }));
|
|
||||||
|
|
||||||
target.pushSample(1000, { latency: 10, error: null });
|
|
||||||
await reporter.flush();
|
|
||||||
target.pushSample(2000, { latency: 20, error: null });
|
|
||||||
target.pushSample(3000, { latency: null, error: "timeout" });
|
|
||||||
target.reset();
|
|
||||||
await reporter.flush();
|
|
||||||
target.pushSample(4000, { latency: 40, error: null });
|
|
||||||
await reporter.flush();
|
|
||||||
await reporter.flush();
|
|
||||||
|
|
||||||
const name = "Target";
|
|
||||||
const url = "https://target.test";
|
|
||||||
assert.deepEqual(reportedHosts(), [
|
|
||||||
[
|
|
||||||
{
|
|
||||||
name,
|
|
||||||
url,
|
|
||||||
status: "online",
|
|
||||||
history: [{ t: 1000, latency: 10, error: null }],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
[
|
|
||||||
{
|
|
||||||
name,
|
|
||||||
url,
|
|
||||||
status: "error",
|
|
||||||
history: [
|
|
||||||
{ t: 2000, latency: 20, error: null },
|
|
||||||
{ t: 3000, latency: null, error: "timeout" },
|
|
||||||
],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
[
|
|
||||||
{
|
|
||||||
name,
|
|
||||||
url,
|
|
||||||
status: "online",
|
|
||||||
history: [{ t: 4000, latency: 40, error: null }],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
]);
|
|
||||||
// The page's history starts over.
|
|
||||||
assert.deepEqual(target.history, [
|
|
||||||
{ timestamp: 4000, latency: 40, error: null },
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
// The interval changes twice, after 60 checks each time, before the first
|
|
||||||
// report. Of the 120 checks the interval menu cleared, the report has the
|
|
||||||
// last 100 (CONFIG.maxHistoryPoints).
|
|
||||||
test("of the checks interval changes cleared before they were reported, the next report has the last 100", async (t) => {
|
|
||||||
const state = new AppState([
|
|
||||||
{ name: "Target", url: "https://target.test" },
|
|
||||||
]);
|
|
||||||
const target = state.local[0];
|
|
||||||
const reporter = new Reporter(state, "client", CONFIG.reportInterval);
|
|
||||||
t.mock.method(globalThis, "fetch", async () => ({ ok: true }));
|
|
||||||
|
|
||||||
for (let n = 1; n <= 120; n++) {
|
|
||||||
target.pushSample(n * 1000, { latency: 10, error: null });
|
|
||||||
if (n % 60 === 0) target.reset();
|
|
||||||
}
|
|
||||||
await reporter.flush();
|
|
||||||
|
|
||||||
const sent = reportedHosts()[0][0].history;
|
|
||||||
assert.equal(sent.length, 100);
|
|
||||||
assert.equal(sent[0].t, 21000);
|
|
||||||
assert.equal(sent.at(-1).t, 120000);
|
|
||||||
});
|
|
||||||
|
|
||||||
// panel stands in for the open debug log panel, and for the element in it
|
|
||||||
// that holds its lines, whichever the page looks up by id; the lines are
|
|
||||||
// kept in order in lines. Drawing the whole panel, as opening it does,
|
|
||||||
// replaces every line with a new one.
|
|
||||||
test("with the debug log open, writing more lines than it keeps adds each to the panel and leaves the lines already shown as they are", (t) => {
|
|
||||||
const lines = [];
|
|
||||||
const panel = {
|
|
||||||
classList: { contains: () => false },
|
|
||||||
children: lines,
|
|
||||||
get firstElementChild() {
|
|
||||||
return lines[0];
|
|
||||||
},
|
|
||||||
append: (line) => lines.push(line),
|
|
||||||
removeChild: (line) => lines.splice(lines.indexOf(line), 1),
|
|
||||||
replaceChildren: (...newLines) =>
|
|
||||||
lines.splice(0, lines.length, ...newLines),
|
|
||||||
};
|
|
||||||
t.mock.method(globalThis.document, "getElementById", () => panel);
|
|
||||||
const kept = CONFIG.maxLogEntries;
|
|
||||||
|
|
||||||
for (let i = 1; i <= kept; i++) log.info(`line ${i}`);
|
|
||||||
const shown = [...lines];
|
|
||||||
for (let i = kept + 1; i <= kept + 5; i++) log.info(`line ${i}`);
|
|
||||||
|
|
||||||
assert.deepEqual(
|
|
||||||
lines.map((line) => line.textContent.match(/line \d+$/)?.[0]),
|
|
||||||
Array.from({ length: kept }, (_, i) => `line ${i + 6}`),
|
|
||||||
);
|
|
||||||
// The first five lines shown were dropped, and the rest are the same
|
|
||||||
// lines, not drawn again.
|
|
||||||
shown.slice(5).forEach((line, i) => assert.equal(lines[i], line));
|
|
||||||
});
|
|
||||||
|
|
||||||
// The page shows < > " & and ' in a row's markup as
|
// The page shows < > " & and ' in a row's markup as
|
||||||
// < > " & and '.
|
// < > " & and '.
|
||||||
test(`a target whose name and URL hold < > " & and ' shows those characters in its row`, () => {
|
test(`a target whose name and URL hold < > " & and ' shows those characters in its row`, () => {
|
||||||
@@ -625,35 +347,19 @@ for (const { history, latencies, statistics } of [
|
|||||||
},
|
},
|
||||||
]) {
|
]) {
|
||||||
test(`a target's min, max, average and median latency over ${history}`, () => {
|
test(`a target's min, max, average and median latency over ${history}`, () => {
|
||||||
const { min, max, avg, med } = hostAfter(latencies).historyStats();
|
const host = hostAfter(latencies);
|
||||||
assert.deepEqual({ min, max, average: avg, median: med }, statistics);
|
assert.deepEqual(
|
||||||
|
{
|
||||||
|
min: host.minLatency(),
|
||||||
|
max: host.maxLatency(),
|
||||||
|
average: host.averageLatency(),
|
||||||
|
median: host.medianLatency(),
|
||||||
|
},
|
||||||
|
statistics,
|
||||||
|
);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// The summary's figures come from each WAN target's last check, by the same
|
|
||||||
// rules as a target's own: here four answered, one was found unreachable
|
|
||||||
// and the rest have not been checked yet. The median, 22.5, and the
|
|
||||||
// average, 21.25, are rounded.
|
|
||||||
test("the summary's min, max, median and average latency over the WAN targets' last checks", () => {
|
|
||||||
const state = new AppState([]);
|
|
||||||
[30, 10, null, 25, 20].forEach((latency, i) =>
|
|
||||||
state.wan[i].pushSample(
|
|
||||||
Date.now(),
|
|
||||||
latency === null
|
|
||||||
? { latency: null, error: "unreachable" }
|
|
||||||
: { latency, error: null },
|
|
||||||
),
|
|
||||||
);
|
|
||||||
assert.deepEqual(state.wanStats(), {
|
|
||||||
reachable: 4,
|
|
||||||
total: state.wan.length,
|
|
||||||
min: 10,
|
|
||||||
max: 30,
|
|
||||||
med: 23,
|
|
||||||
avg: 21,
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// An app state in which, of the WAN targets, the first timedOut timed out,
|
// An app state in which, of the WAN targets, the first timedOut timed out,
|
||||||
// the next unreachable were found unreachable, the next answered answered
|
// the next unreachable were found unreachable, the next answered answered
|
||||||
// after latency ms, and the rest have not been checked yet.
|
// after latency ms, and the rest have not been checked yet.
|
||||||
|
|||||||
+18
-19
@@ -7,22 +7,22 @@ make frontend-viewport-test
|
|||||||
```
|
```
|
||||||
|
|
||||||
It builds `dist/`, serves it from the same digest-pinned `nginx` image and the
|
It builds `dist/`, serves it from the same digest-pinned `nginx` image and the
|
||||||
same `configs/nginx.conf` the shipping container uses, drives a digest-pinned
|
same `nginx.conf` the shipping container uses, drives a digest-pinned headless
|
||||||
headless Chrome against it over CDP, and asserts on computed layout at every
|
Chrome against it over CDP, and asserts on computed layout at every viewport
|
||||||
viewport width derived from the app's own CSS. Screenshots land in
|
width derived from the app's own CSS. Screenshots land in `tmp/viewport/`
|
||||||
`tmp/viewport/` alongside a `results.json`; they are artifacts for a human to
|
alongside a `results.json`; they are artifacts for a human to look at when
|
||||||
look at when something fails, not the evidence. The assertions are the evidence.
|
something fails, not the evidence. The assertions are the evidence.
|
||||||
|
|
||||||
The target is deliberately outside `make check`: it takes minutes, and
|
The target is deliberately outside `make check`: it needs Docker and takes
|
||||||
`make test` has to stay under 60 seconds.
|
minutes, and `make test` has to stay under 20 seconds.
|
||||||
|
|
||||||
## How the widths are chosen
|
## How the widths are chosen
|
||||||
|
|
||||||
Not from a list of phone models. `viewports.js` parses the `@media` conditions
|
Not from a list of phone models. `viewports.js` parses the `@media` conditions
|
||||||
out of `web/src/styles.css` and scans `web/src/main.js` and `web/index.html` for
|
out of `src/styles.css` and scans `src/main.js` and `index.html` for Tailwind
|
||||||
Tailwind responsive prefixes, then tests every breakpoint it finds at one pixel
|
responsive prefixes, then tests every breakpoint it finds at one pixel below it,
|
||||||
below it, exactly on it, and one pixel above it. A generic 375px "phone" test
|
exactly on it, and one pixel above it. A generic 375px "phone" test sails
|
||||||
sails straight past an off-by-one at a media query boundary; `max-width: 768px`
|
straight past an off-by-one at a media query boundary; `max-width: 768px`
|
||||||
matches _at_ 768, and the sweep pins down which side of that line each layout is
|
matches _at_ 768, and the sweep pins down which side of that line each layout is
|
||||||
on.
|
on.
|
||||||
|
|
||||||
@@ -77,7 +77,7 @@ the internet, so the app's latency probes cannot reach anything real. The
|
|||||||
harness answers them itself from a fixed delay table, with a deterministic
|
harness answers them itself from a fixed delay table, with a deterministic
|
||||||
fraction failed outright, so the rows render a realistic spread of one-, two-
|
fraction failed outright, so the rows render a realistic spread of one-, two-
|
||||||
and three-digit latencies plus some unreachable rows. That spread is what the
|
and three-digit latencies plus some unreachable rows. That spread is what the
|
||||||
layout has to survive; a `---` placeholder in every row would not exercise it.
|
layout has to survive; 24 identical `---` placeholders would not exercise it.
|
||||||
|
|
||||||
## What this cannot verify
|
## What this cannot verify
|
||||||
|
|
||||||
@@ -104,11 +104,10 @@ Everything else this issue was actually about — does the layout reflow, does
|
|||||||
anything overflow, is content clipped, are the controls big enough — is a
|
anything overflow, is content clipped, are the controls big enough — is a
|
||||||
function of viewport width and CSS, and is covered above.
|
function of viewport width and CSS, and is covered above.
|
||||||
|
|
||||||
## Relation to the unit tests
|
## Relation to the unit test framework (#21)
|
||||||
|
|
||||||
Complementary layers, not two stacks. The unit tests in `test/unit/`, which
|
Complementary layers, not two stacks. `vitest` (#21) will exercise module-level
|
||||||
`make test` runs with Node's built-in test runner, exercise the functions
|
logic in-process with no browser. This harness exercises rendered layout in a
|
||||||
`web/src/main.js` exports in-process with no browser. This harness exercises
|
real engine and is the only thing here that can see a media query. Neither
|
||||||
rendered layout in a real engine and is the only thing here that can see a media
|
replaces the other; assertions about computed styles and element geometry belong
|
||||||
query. Neither replaces the other; assertions about computed styles and element
|
here, assertions about functions belong in `vitest`.
|
||||||
geometry belong here, assertions about functions belong in `test/unit/`.
|
|
||||||
|
|||||||
@@ -51,8 +51,7 @@ const PROBE_DELAYS_MS = [2, 45, 123, 456, 780];
|
|||||||
// offline row rendering is exercised too.
|
// offline row rendering is exercised too.
|
||||||
const UNREACHABLE_MODULUS = 7;
|
const UNREACHABLE_MODULUS = 7;
|
||||||
// The gateway candidate that "answers", so gateway detection succeeds and
|
// The gateway candidate that "answers", so gateway detection succeeds and
|
||||||
// the Local Gateway row renders. Matches GATEWAY_CANDIDATES in
|
// the Local Gateway row renders. Matches GATEWAY_CANDIDATES in src/main.js.
|
||||||
// web/src/main.js.
|
|
||||||
const RESPONSIVE_GATEWAY = "http://192.168.1.1";
|
const RESPONSIVE_GATEWAY = "http://192.168.1.1";
|
||||||
|
|
||||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||||
|
|||||||
@@ -161,27 +161,26 @@ const ANCHOR_VIEWPORTS = [
|
|||||||
export function deriveViewports(root) {
|
export function deriveViewports(root) {
|
||||||
const conditions = [
|
const conditions = [
|
||||||
...mediaConditionsFromCss(
|
...mediaConditionsFromCss(
|
||||||
readFileSync(join(root, "web/src/styles.css"), "utf8"),
|
readFileSync(join(root, "src/styles.css"), "utf8"),
|
||||||
"web/src/styles.css",
|
"src/styles.css",
|
||||||
),
|
),
|
||||||
...mediaConditionsFromMarkup([
|
...mediaConditionsFromMarkup([
|
||||||
{
|
{
|
||||||
path: "web/src/main.js",
|
path: "src/main.js",
|
||||||
text: readFileSync(join(root, "web/src/main.js"), "utf8"),
|
text: readFileSync(join(root, "src/main.js"), "utf8"),
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
path: "web/index.html",
|
path: "index.html",
|
||||||
text: readFileSync(join(root, "web/index.html"), "utf8"),
|
text: readFileSync(join(root, "index.html"), "utf8"),
|
||||||
},
|
},
|
||||||
]),
|
]),
|
||||||
];
|
];
|
||||||
|
|
||||||
if (conditions.length === 0) {
|
if (conditions.length === 0) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
"no responsive breakpoints found in web/src/styles.css, " +
|
"no responsive breakpoints found in src/styles.css, src/main.js or " +
|
||||||
"web/src/main.js or web/index.html — either the responsive " +
|
"index.html — either the responsive layout was deleted or this " +
|
||||||
"layout was deleted or this derivation has stopped matching " +
|
"derivation has stopped matching the sources",
|
||||||
"the sources",
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -6,10 +6,6 @@ const commitHash = execSync("git rev-parse --short HEAD").toString().trim();
|
|||||||
const commitFull = execSync("git rev-parse HEAD").toString().trim();
|
const commitFull = execSync("git rev-parse HEAD").toString().trim();
|
||||||
|
|
||||||
export default defineConfig({
|
export default defineConfig({
|
||||||
// The page's source is in web/; the build still goes to dist/ at the
|
|
||||||
// repo root, which Dockerfile and the viewport test serve. Vite
|
|
||||||
// empties only an outDir inside root unless told to.
|
|
||||||
root: "web",
|
|
||||||
plugins: [tailwindcss()],
|
plugins: [tailwindcss()],
|
||||||
server: {
|
server: {
|
||||||
// Proxy /api to a locally running netwatch-server so `yarn dev`
|
// Proxy /api to a locally running netwatch-server so `yarn dev`
|
||||||
@@ -23,8 +19,6 @@ export default defineConfig({
|
|||||||
__COMMIT_FULL__: JSON.stringify(commitFull),
|
__COMMIT_FULL__: JSON.stringify(commitFull),
|
||||||
},
|
},
|
||||||
build: {
|
build: {
|
||||||
outDir: "../dist",
|
|
||||||
emptyOutDir: true,
|
|
||||||
target: "esnext",
|
target: "esnext",
|
||||||
minify: "esbuild",
|
minify: "esbuild",
|
||||||
cssMinify: true,
|
cssMinify: true,
|
||||||
|
|||||||
Reference in New Issue
Block a user