netwatch/TODO.md

# Workflow

- branch (from `main`)
- do the work in Next Step
- move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work)
- merge to `main` if the branch is not protected, otherwise open a PR
- push

# Status

pre-1.0. No git tags. `feat/reportbuf-storage` is merged; the backend, the CI
workflow, and the backend repo standard files are all on `main`. Frontend and
backend are both functional. Working toward the 1.0.0 milestone by closing the
remaining repo-compliance issues on the tracker.

# Next Step

Confirm the `.gitea/workflows/check.yml` run is green (main always green
policy). The workflow file is already on `main`; what is unverified is that its
latest run passes.

# Completed Steps

- 2026-09-29: CI can no longer pass on checks that did not run (issue #37):
  `script/cibuild` is now the org model, byte for byte. It runs
  `script/bootstrap` and `script/check`, then builds the image with `--no-cache`
  and the version from `git describe` as the `VERSION` build argument, where it
  used to be a plain `docker build .` whose check steps could come from the
  build cache. The workflow puts `~/.local/bin`, where bootstrap links what it
  installs, on the step's `PATH`, and bootstrap now installs its pinned node
  when the installed one is older than the frontend's dependencies need
- 2026-09-29: `backend/.golangci.yml` re-vendored from `sneak/prompts` (issue
  #41): `gomodguard`, deprecated in golangci-lint v2.12.0, is disabled and its
  successor `gomodguard_v2` enabled with the org block list, so lint runs print
  no deprecation warning. The new file also turns `depguard` on with its
  `test-support` rule, which keeps `net/http/httptest` out of non-test code;
  netwatch adds no entries of its own to that rule. `backend/script/lint` checks
  the new sha256
- 2026-09-29: nginx sends the security headers `REPO_POLICIES.md` requires on
  every response (issue #18), including errors, `/assets/` and what it passes on
  from the backend, whose own copies it drops so each header goes out once. They
  live in `security-headers.conf`, which `nginx.conf` includes. The content
  security policy allows no inline script or style, so the status dot's grey in
  `src/main.js` is now a class; `connect-src` is `*` because probed hosts
  redirect to others, and the browser checks each redirect against it
- 2026-09-29: the request log is bounded (issue #60): the method, URL, protocol,
  `User-Agent`, `Referer`, request ID (which chi takes from the client's
  `X-Request-Id` header) and client address it writes are each cut to 128 bytes,
  the bound the report handler already used, so one request can no longer put
  about 1 MiB per field into a log line. That bound and its helper now live in
  the `logger` package, shared by both
- 2026-09-29: nginx takes the client address from `X-Forwarded-For` only on
  requests from the reverse proxies named in the container's `TRUSTED_PROXIES`
  (issue #64), and by default from none, where it trusted every RFC1918 address
  before, so a client could write a new address on each request and escape the
  rate limit. `bin/entrypoint.sh` writes one `set_real_ip_from` line per entry
  into `/etc/nginx/trusted-proxies.conf`, which `nginx.conf` includes, refusing
  an entry that is not an IP address or CIDR, as `netwatch-server check-cidr`
  finds; it starts the backend with `TRUSTED_PROXIES=127.0.0.1/32`, since nginx
  is its only client
- 2026-09-29: report file names can no longer collide (issue #61): each is
  `reports-<timestamp>-<number>.jsonl.zst`, where the number goes up by one for
  each file the server starts to write, so two flushes in the same millisecond,
  such as a flush for size and the final flush at shutdown, each get a file of
  their own instead of the second one failing. A failed write uses up its
  number, leaving a gap if the file could not be created and otherwise a file
  under that number that may be incomplete.
- 2026-09-29: ready to run under upaas (issue #59): the image has a
  `HEALTHCHECK` that requests `/.well-known/healthcheck` through nginx on the
  port from `PORT`. The backend no longer reads a bad `PORT` as 0 or a bad
  `DEBUG` as false: those, and a `BIND_ADDRESS` that is not an IP address, stop
  it from starting with an error naming the variable, as the limits,
  `CORS_ALLOWED_ORIGINS` and, now by name, `TRUSTED_PROXIES` already did.
  `bin/entrypoint.sh` also refuses a `PORT` outside 1 to 65535, and `8081`,
  where the backend listens inside the container, naming `PORT`. `README.md` has
  a "Running under upaas" section, whose first-run steps create the host
  directory for `/data` owned by uid 1000; the image does not change its owner
- 2026-09-29: nginx listens on `PORT` (issue #26), 8080 when unset or empty: the
  nginx image renders `nginx.conf` as a template at container start, filling in
  `PORT` and no other variable. `bin/entrypoint.sh` refuses to start when `PORT`
  is not digits only. `server_tokens off` keeps the nginx version out of
  responses. `script/frontend-viewport-test` renders the template the same way.
  Gzip and a `50x.html` error page are not added
- 2026-09-29: bounded the report endpoint (issue #20): `POST /api/v1/reports`
  still needs no credentials, but each client address, as resolved through
  `TRUSTED_PROXIES`, may send `REPORTS_PER_MINUTE` (default 60) reports a
  minute, counted by `go-chi/httprate`, and past that gets 429 with
  `Retry-After`; the report files in `DATA_DIR`, counted from start with those
  already there, may total at most `DATA_DIR_MAX_BYTES` (default 1 GiB), past
  which reports get 507; and the wildcard CORS is gone: no CORS headers unless
  `CORS_ALLOWED_ORIGINS` lists origins, and an entry that is not a plain
  `scheme://host[:port]` origin, `*` included, stops the server from starting.
  Deleting report files frees room only at the next start; pruning is issue #54
- 2026-09-28: one container image (issue #52): the root `Dockerfile` builds the
  only image, and `Dockerfile.backend` is gone. nginx serves the frontend on
  port 8080 and proxies `/api/` and `/.well-known/healthcheck` to the backend,
  which listens on `127.0.0.1:8081` in the same container; the new
  `BIND_ADDRESS` setting sets its listen address. `bin/entrypoint.sh` starts
  both, passes TERM and INT on to both, and exits non-zero when either exits on
  its own. The backend runs as user `netwatch` and stores reports on the `/data`
  volume. `script/docker` is the org model again
- 2026-09-28: unified the gate (issue #16): the root `make check` covers the Go
  backend as well as the frontend, and the pre-commit hook with it; the backend
  moved onto scripts-to-rule-them-all (`backend/script/*`, `backend/Makefile` as
  shims, its duplicate hook installer removed); `script/cibuild` builds both
  images and is the workflow's only build step. The root `make lint` runs
  golangci-lint only in Docker, by building the lint stage of
  `Dockerfile.backend` without the cache. `script/bootstrap` installs the pinned
  Go unless the installed one is at least what `backend/go.mod` asks for, links
  what it installs into `~/.local/bin` without replacing anything it did not
  create, and installs no linter. Root `make test` runs both halves within one
  30-second timeout. When `VERSION` is unset or empty, the backend binary's
  version falls back to `git describe` inside a git checkout, then to `dev`
- 2026-09-28: frontend reporting client (issue #53): a `Reporter` class posts
  collected samples to `/api/v1/reports` every `reportInterval` (default 60s) as
  a per-host delta, with the report-building step a pure exported function of
  host state; a per-host mark advances only on a delivered POST; at most one
  report POST is pending at a time and it is abandoned after half the interval,
  so a slow POST never overlaps the next report and a mark never moves
  backwards; the samples of an abandoned POST are sent again at the next
  interval, so a backend that stored them but answered late receives them twice;
  the per-browser client id works in insecure (plain-HTTP) contexts;
  `vite.config.js` proxies `/api` to the local backend for `yarn dev`
- 2026-09-28: report ingest correctness (issue #23): a storage failure now
  returns 500 instead of a false `ok`; oversize bodies return 413 (distinguished
  from malformed JSON, which stays 400); a `MaxBodyBytes` middleware caps every
  route, not just the report route; the raw attacker-controlled `geo` blob is no
  longer logged (only its length), and `client_id`, `timestamp` and decode error
  text are length-bounded before logging; a `decodeJSON` handler helper was
  added; panic recovery now routes the stack through slog instead of chi's
  plain-text stderr; and writing a report file now returns its error, so a
  failed final flush on shutdown makes the process exit non-zero instead of
  losing the buffered reports silently
- 2026-09-21: shutdown lifecycle correctness. The process now shuts down through
  fx instead of `os.Exit`, so every component's `OnStop` runs and buffered
  reports are flushed to disk on `SIGTERM` — previously a full flush window of
  telemetry was silently lost on every restart. The `http.Server` is now built
  before its serving goroutine starts, so shutdown can no longer race or
  nil-deref it; a listen failure exits non-zero via `fx.Shutdowner`; `reportbuf`
  `OnStop` is idempotent; and `writeTimeout` now exceeds the chi per-request
  budget so that budget is actually reachable. Dead `startupTime`, `exitCode`,
  and `cancelFunc` fields were removed
- 2026-09-21: backend HTTP hardening (issue #19): added `ReadHeaderTimeout` and
  `IdleTimeout` to the server, a `SecurityHeaders` middleware (HSTS, tight CSP,
  frame/sniff/referrer/permissions headers) registered before CORS, and
  trusted-proxy client IP resolution honouring `X-Forwarded-For` / `X-Real-IP`
  only from a `TRUSTED_PROXIES` allowlist (loopback plus RFC1918 by default)
- 2026-08-10: adopted the org-standard `backend/.golangci.yml` verbatim and
  moved the pinned golangci-lint from v2.7.2 to v2.12.2 (the `lint` stage of
  `Dockerfile.backend` now pins the `golangci/golangci-lint:v2.12.2` image by
  digest); the previous config declared `version: "2"` but used v1 schema keys,
  so every threshold in it was inert and its green result was meaningless.
  `backend/Makefile`'s `lint` target now asserts the config's sha256 against the
  canonical file first, so drift from the org standard fails the build instead
  of silently degrading to defaults
- 2026-08-10: every interactive control now meets the 44x44 CSS px minimum tap
  target (`.pin-btn`, `#interval-select`, the debug-log label and, on narrow
  viewports, `#pause-btn`). The pin button's hit area grows via matching
  negative margins, so its layout footprint and row density are unchanged
- 2026-08-10: per-host status line wraps below the 768px breakpoint instead of
  forcing horizontal page scroll at 320px
- 2026-08-09: `Dockerfile.backend` reworked to the mandated Go multistage
  lint-stage pattern: separate `lint` stage on the hash-pinned
  `golangci/golangci-lint` image, `COPY --from=lint` stage dependency,
  `CGO_ENABLED=0` static build driven by `ARG VERSION`, and no more `COPY .git`
- 2026-08-09: dotfile compliance — lifted `backend/.editorconfig` to the repo
  root so `root = true` covers the frontend too, and replaced `.gitignore` with
  the org model (OS, editor, node, and environment/secrets sections) plus this
  repo's `dist/` and `*.log`. `.env`, `.env.*`, `*.pem`, and `*.key` are now
  ignored repo-wide, not just under `backend/`. Excluding `.git` from
  `.dockerignore` stays deferred: both images read git metadata at build time
  (`COPY .git` in `Dockerfile.backend`, `git rev-parse` in `vite.config.js`)
- 2026-08-09: automated responsive-layout harness
  (`make frontend-viewport-test`): digest-pinned headless Chrome driven over CDP
  against the built `dist/`, viewport widths derived from the breakpoints in
  `src/styles.css` ([#13](https://git.eeqj.de/sneak/netwatch/issues/13)). Every
  check carries a presence guard so none of them can pass against a page it is
  not actually measuring. Found two real layout defects, filed as
  [#42](https://git.eeqj.de/sneak/netwatch/issues/42) and
  [#43](https://git.eeqj.de/sneak/netwatch/issues/43)
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
  shims, README Entrypoints section
- 2026-02-27: backend with buffered zstd-compressed report storage; CI workflow
  and backend repo standard files; backend Dockerfile fixed (Go 1.25,
  golangci-lint) and moved to repo root (feat/reportbuf-storage)
- 2026-02-26: host row layout redesigned with CSS grid; overflow and spacing
  fixes; nginx config extracted; port hardcoded to 8080
- 2026-02-26: debug log panel, median stats, recovery probe, Docker build fix,
  S3 Singapore endpoint added
- 2026-02-23: summary box redesign, host pinning, local and UTC clocks, checks
  counter
- 2026-02-23: hosts sorted by latency; GET instead of HEAD for latency; timeout
  derived from interval; Hetzner regional endpoints; 3s interval
- 2026-01-29: initial NetWatch network latency monitor

# Future Steps

- Wire `script/frontend-viewport-test` into CI as its own step (deliberately not
  part of `make check` today; the decision has real CI-runtime cost and is
  tracked separately)
- Compliance top-up as one small commit: add .editorconfig and add the hooks
  target to the Makefile
- After merge, confirm .gitea/workflows/check.yml is on main and CI is green
  (main always green policy)
- Decide what to do with untracked resume.sh: commit it, gitignore it, or delete
  it