README.md: Getting Started leads with make targets; a Backend section
gives netwatch-server's routes and how the image builds and runs it;
the checks are GET requests; the WAN host list, health states, summary
figures, sorting and missing features match src/main.js; the TODO
section points to TODO.md, which holds the one to-do list.
backend/README.md: its TODO section points to TODO.md too, whose
Future Steps take its three open items.
TODO.md: Workflow branches from next and opens the PR against next;
Status, Next Step and Future Steps describe the open work, linked to
its issue where one exists.
test/viewport/README.md: the unit tests run on Node's test runner,
not vitest.
Model: opus-5-5
Each client address may make 60 requests to /metrics a minute,
through the same httprate middleware and TRUSTED_PROXIES
resolution the report route uses, with an allowance of its own.
The limit runs before the basic auth, so past it the answer is 429
and the password is not checked. backend/README.md says so; a test
uses up one client's allowance on wrong passwords, gets 429 with
the right one, and checks that another client behind the same
nginx still gets in.
Model: opus-5-5
With SENTRY_DSN set, the server initialises sentry-go with the release
netwatch-server-<version>, adds the sentryhttp middleware with Repanic
as the last router-wide middleware, after the timeout, and flushes
Sentry for 2 seconds on shutdown. A DSN Sentry refuses stops the start
with an error naming SENTRY_DSN. With it empty, nothing is set up.
The metrics middleware stays on the matched routes only, so it runs
inside the Sentry middleware rather than before it.
Model: opus-5-5
With METRICS_USERNAME and METRICS_PASSWORD both set, the backend
records request metrics through go-http-metrics in a registry of its
own, with Go's runtime and process metrics, and serves them at
GET /metrics behind basic auth; nginx passes /metrics to it. With
neither set there is no such route; one alone, or a METRICS_USERNAME
containing ":", stops the start with an error naming the setting.
Only requests that reach the health check or POST /api/v1/reports are
recorded, as the labels are path and method, which clients could
otherwise make up without end; POST /api/v1/reports is registered by
its full path for that.
Deviation: go get and go mod tidy ran directly; no entrypoint added a Go
dependency yet (issue #45).
Model: opus-5-5
make dev ran yarn dev inline and there was no make build. Both are now
shims, over script/dev and the new script/build. .prettierignore stops
leaving out backend/, so backend/README.md is formatted and checked;
backend/.golangci.yml is left out by name instead: it is the org
standard file whose sha256 backend/script/lint checks, and prettier
would reindent it. .claude/ stays in .prettierignore, as git does not
ignore it. script/install-precommit and the date on bootstrap's pins go
back to the org model; bootstrap, fmt and fmt-check keep what the Go
backend and eslint need, each with a comment saying so.
Model: opus-5-5
`bin/entrypoint.sh` now runs `netwatch-server prepare-data-dir`, which
refuses a `DATA_DIR` that is not `/data` or a path below it written in
full, then creates `DATA_DIR`, gives `/data` and everything in it to
`netwatch`, and sets mode 750 on `/data` and `DATA_DIR`. Every step goes
through a Go `os.Root` opened on `/data`, and the modes are set on the
opened directories rather than by name, so neither a symbolic link
already there nor one a host process swaps in while the container
starts can make root create or change anything outside `/data`. The
README says which `DATA_DIR` values are accepted.
Model: opus-5-5
When a report would take the report files past DATA_DIR_MAX_BYTES,
reportbuf now deletes the oldest report files until it fits, and does
the same at start when files left by an earlier run are already past
it. A file joins the files that may be deleted, at its place by name,
only once it is completely written, so a file still being written is
never deleted. A report is refused with 507 only when the reports
waiting to be written fill the cap on their own, and then no file is
deleted. The reports of a failed write stop counting, and the part of
its file written is removed. A file whose deletion fails keeps
counting; one already deleted by hand counts as freed.
Model: opus-5-5
backend/script/test runs go test -timeout 30s -race -cover and, if
that fails, runs it again with -v and fails. The root script/test drops
its one 30-second timeout around both halves: from a cold Go build
cache, compiling the tests with -race used it all up. Each half keeps
its own limit. The race detector needs a C compiler: the Dockerfile
builder stage gains gcc and musl-dev, and script/bootstrap installs gcc,
with the C library headers on apt and apk, when gcc is missing; make
build still sets CGO_ENABLED=0. New tests: the health check's answer, a
valid report's answer, a report file's exact lines, and the flush at the
10 MiB threshold. The handlers TestImport stub is gone.
Model: opus-5-5
The architecture is no longer passed in at build time. The Buildarch
variable and field are gone from main and globals, script/build no
longer stamps it in with -X, and the startup and listen log lines
report runtime.GOARCH under the key "arch". The Dockerfile comment and
backend/README.md no longer describe an architecture being stamped in.
Model: opus-5-5
Closes #75.
`bin/entrypoint.sh`, which already runs as root, now makes the data directory usable before the backend starts: it creates `DATA_DIR` if missing, gives it and `/data` to the `netwatch` user (`chown -R`), and sets mode 750 on both, the mode the backend gives a directory it creates. The backend still runs as `netwatch`. The README "Running under upaas" section loses its first-run step that created and chowned the host directory and names only the path to mount. The Dockerfile's build-time `mkdir` and `chown` of `/data` are gone, since the entrypoint now does this on every start.
What the diff does not show:
- The host directory mounted at `/data` ends up owned by uid 1000 with mode 750, and everything under `DATA_DIR` is chowned to uid 1000 on every start.
- If the directory cannot be created or chowned, the container stops with that tool's error before either process starts.
Recorded runs with `--mount type=bind`: an empty directory owned by root (mode 755, and again mode 700), and one holding a `reports` directory and report file owned by uid 1001 with mode 700. Each time the container turned healthy, `netwatch-server` ran as `netwatch`, and a posted report was written to `DATA_DIR`; a second start on the root-owned and the uid 1001 directories did the same.
Judgement call: `/data` itself is given to `netwatch` as well as `DATA_DIR`, so the backend can reach `DATA_DIR` inside a host directory with mode 700.
Model: opus-5-5
Reviewed-on: #76
Co-authored-by: clawbot <35+clawbot@noreply.example.org>
nginx sent none of the security headers REPO_POLICIES.md requires.
security-headers.conf now sets all six with always, included at server
level and again in /assets/, whose own add_header would otherwise drop
them. nginx hides the copies netwatch-server sets, so /api/ and the
health check carry each header once. The content security policy
allows no inline script or style; the host row's status dot took its
grey from a style attribute, now a class. connect-src is * because
several probed hosts redirect to other hosts and the browser checks
every redirect against it. Referrer-Policy is no-referrer, as the
backend already sends.
Model: opus-5-5
nginx trusted X-Forwarded-For from every RFC1918 address, so a client
reaching it from one could write a new address on each request and
get a fresh rate-limit allowance. The container's TRUSTED_PROXIES now
names the reverse proxies nginx trusts, none by default.
bin/entrypoint.sh makes each entry a CIDR, checks it with the new
"netwatch-server check-cidr", which runs the server's own
TRUSTED_PROXIES parsing, and writes one set_real_ip_from line per
entry into /etc/nginx/trusted-proxies.conf, which nginx.conf includes.
The backend is started with TRUSTED_PROXIES=127.0.0.1/32, since nginx
is its only client. The viewport test mounts an empty file there.
Model: opus-5-5
Report files were named by a millisecond timestamp and created with
O_EXCL, so two flushes in the same millisecond, such as a flush for
size and the final flush at shutdown, got the same name and the second
failed, losing its reports. Each name now carries a number after the
timestamp that goes up by one for each file the server starts to
write, so names still sort by time and never repeat within a run. A
failed write uses up its number, leaving a gap if the file could not
be created and otherwise a file under that number that may be
incomplete.
Model: opus-5-5
The image's HEALTHCHECK requests /.well-known/healthcheck through
nginx on the port from PORT, so it fails unless both processes answer.
The backend reads PORT and DEBUG with strconv instead of viper, which
turned a bad PORT into 0 and a bad DEBUG into false. Those, and a
BIND_ADDRESS that is not an IP address, now stop the start with an
error naming the variable; the TRUSTED_PROXIES error names it too.
bin/entrypoint.sh also refuses a container PORT outside 1 to 65535,
or 8081, where the backend listens, naming PORT. README.md gains
"Running under upaas". Its first-run steps create the host directory
owned by uid 1000, so the image changes no ownership.
Model: opus-5-5
POST /api/v1/reports stays unauthenticated but is bounded. Each client
address, as the trusted-proxy logic resolves it, may send
REPORTS_PER_MINUTE reports a minute (default 60, counted by
go-chi/httprate over a sliding minute); past that it gets 429 with
Retry-After. reportbuf refuses a report that would take the report
files past DATA_DIR_MAX_BYTES (default 1 GiB), counting the files
already in DATA_DIR and unwritten reports at their uncompressed size;
the handler answers 507. CORS adds nothing unless CORS_ALLOWED_ORIGINS
lists origins. A limit that is not a positive number, or an origin
that is not a plain scheme://host[:port], stops the server from
starting.
Model: opus-5-5
The root Dockerfile builds the only image; Dockerfile.backend is gone.
Its stages: lint, a Go stage that runs the tests and builds
netwatch-server, the node stage, and an nginx runtime. nginx serves
dist/ on 8080 and proxies /api/ and /.well-known/healthcheck to the
backend on 127.0.0.1:8081. bin/entrypoint.sh starts both, turns TERM or
INT into a stop of both, and exits non-zero when either exits on its
own. The backend runs as user netwatch and keeps reports on the /data
volume. New setting BIND_ADDRESS (empty: every interface). STOPSIGNAL is
SIGTERM, since the nginx image's SIGQUIT would miss the entrypoint.
script/docker is the org model verbatim.
Model: opus-5-5
Root make check, and with it the pre-commit hook, now gates the Go
backend too. The backend's Makefile targets are shims over
backend/script/*; script/cibuild builds both images and is the
workflow's only build step. Root make test runs both halves within one
30-second timeout.
Root make lint runs golangci-lint only in Docker, by building the lint
stage of Dockerfile.backend without the cache; the .golangci.yml drift
check moved into backend/script/lint. script/bootstrap installs no
linter: it reuses a Go at least as new as backend/go.mod asks for,
otherwise installs the pinned, hash-verified release, linked into
~/.local/bin without replacing anything it did not create. With VERSION
unset or empty, the backend version falls back to git describe inside a
git checkout, then to dev.
Model: opus-5-5
Add ReadHeaderTimeout and IdleTimeout to the http.Server as named constants beside the existing timeouts. Add a SecurityHeaders middleware (HSTS, a JSON-API CSP of default-src 'none'; frame-ancestors 'none', X-Frame-Options DENY, nosniff, Referrer-Policy, Permissions-Policy), registered before CORS so preflight responses carry it. Resolve the client IP from X-Forwarded-For / X-Real-IP only when the direct peer is in the trusted-proxy allowlist (loopback plus RFC1918 by default, configurable via TRUSTED_PROXIES); an untrusted peer's forwarded headers are ignored. Uses net/netip; no new dependency.
Model: opus-4-8 (implementation and review); claude-fable-5 (merge)
Add .gitea/workflows/check.yml that builds both the root and
backend Docker images on push. Add LICENSE and README.md to the
backend subproject to match repo standards.
sneak
clawbot