next into main (#73)

Reviewed-on: #73

.gitea/workflows/check.yml

@@ -6,5 +6,7 @@ jobs:
         steps:
             # actions/checkout v4.2.2, 2026-02-22
             - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
-            # script/cibuild builds the image, whose stages run every check.
+            # script/cibuild bootstraps, runs every check and builds the
-            - run: script/cibuild
+            # image. script/bootstrap links what it installs into
+            # ~/.local/bin, so that has to be on PATH for the rest.
+            - run: PATH="$HOME/.local/bin:$PATH" script/cibuild

Dockerfile

@@ -77,8 +77,9 @@ COPY --from=frontend /app/dist /usr/share/nginx/html
 COPY --from=builder /src/netwatch-server /usr/local/bin/netwatch-server
 COPY bin/entrypoint.sh /usr/local/bin/entrypoint.sh
 
+# bin/entrypoint.sh creates DATA_DIR at start and gives it and /data to
+# the netwatch user, whatever is mounted there.
 ENV DATA_DIR=/data/reports
-RUN mkdir -p /data/reports && chown -R netwatch:netwatch /data
 VOLUME /data
 
 # The default public port; PORT changes it.

README.md

@@ -36,12 +36,12 @@ The Go backend in `backend/` has its own `script/` directory and shim Makefile
 (see [backend/README.md](backend/README.md)). The root scripts cover both
 halves, so the root `make check` fails if either one is broken. We provide:
 
-- `script/bootstrap` — install all dependencies (pinned node via nvm if needed,
+- `script/bootstrap` — install all dependencies (the pinned node via nvm unless
-  yarn via corepack, `yarn install --frozen-lockfile`, the pinned Go unless one
+  one new enough for the frontend's dependencies is installed, yarn via
-  at least as new as `backend/go.mod` asks for is installed, and the Go
+  corepack, `yarn install --frozen-lockfile`, the pinned Go unless one at least
-  modules), linking what it installs itself into `~/.local/bin`, which has to be
+  as new as `backend/go.mod` asks for is installed, and the Go modules), linking
-  on `PATH`. It installs no Go linter and not Docker: `make lint` runs the
+  what it installs itself into `~/.local/bin`, which has to be on `PATH`. It
-  linter in Docker
+  installs no Go linter and not Docker: `make lint` runs the linter in Docker
 - `script/setup` — make a fresh clone ready for development: bootstrap plus the
   git pre-commit hook
 - `script/projectname` — print the project name (used for the Docker image tag)
@@ -65,7 +65,8 @@ halves, so the root `make check` fails if either one is broken. We provide:
   `script/check`: it needs Docker and takes minutes.
 - `script/docker` — build the image from `Dockerfile` without the build cache,
   tagged `netwatch` via `script/projectname`
-- `script/cibuild` — CI entrypoint: builds the image
+- `script/cibuild` — CI entrypoint: runs `script/bootstrap` and `script/check`,
+  then builds the image as `script/docker` does, without the build cache
 - `script/precommit` — run by the git pre-commit hook; runs `script/check`
 - `script/install-precommit` — install the git pre-commit hook
 
@@ -191,8 +192,9 @@ only inside the container, on `127.0.0.1:8081`. The image:
 - Sends the security headers `REPO_POLICIES.md` requires on every response, as
   `security-headers.conf` sets them, in place of the backend's own
 - Stores reports in `DATA_DIR`, `/data/reports` by default, on the `/data`
-  volume. The backend runs as user `netwatch` (uid 1000), so a directory
+  volume. Before the backend starts, the image creates `DATA_DIR` and gives it
-  bind-mounted at `/data` must be writable by uid 1000
+  and `/data` to user `netwatch` (uid 1000), which the backend runs as, so a
+  host directory bind-mounted at `/data` ends up owned by uid 1000
 - Writes buffered reports to disk on `docker stop`, and exits non-zero if nginx
   or the backend exits on its own, so the platform restarts it
 
@@ -202,16 +204,6 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for netwatch needs:
 
 - **Port:** container port `8080`.
 - **Volume:** container path `/data`; the reports are kept in `/data/reports`.
-- **First run:** upaas bind-mounts the host directory it is given and does not
-  create it, and the backend, which runs as uid 1000, does not start unless it
-  can write there. Create the directory, owned by uid 1000, before the first
-  deploy:
-
-    ```bash
-    mkdir -p /path/to/data
-    chown 1000:1000 /path/to/data
-    ```
-
 - **Environment variables:** none is required. An empty one counts as unset, and
   one set to a value netwatch cannot use stops the container at start, with the
   reason in its log.

TODO.md

@@ -23,6 +23,23 @@ latest run passes.
 
 # Completed Steps
 
+- 2026-09-29: the container sets up its own data directory (issue #75):
+  `bin/entrypoint.sh`, still as root, creates `DATA_DIR` if missing and gives it
+  and `/data` to the `netwatch` user with mode 750 before starting the backend
+  as that user, so an empty host directory owned by root, or one holding files
+  from another uid, works with no step on the host. It stops the start instead
+  when a symbolic link is on the path to `DATA_DIR`, since root would change
+  whatever the link points to. The `README.md` first-run step that created and
+  chowned the host directory is gone, and the image no longer sets that
+  ownership at build time
+- 2026-09-29: CI can no longer pass on checks that did not run (issue #37):
+  `script/cibuild` is now the org model, byte for byte. It runs
+  `script/bootstrap` and `script/check`, then builds the image with `--no-cache`
+  and the version from `git describe` as the `VERSION` build argument, where it
+  used to be a plain `docker build .` whose check steps could come from the
+  build cache. The workflow puts `~/.local/bin`, where bootstrap links what it
+  installs, on the step's `PATH`, and bootstrap now installs its pinned node
+  when the installed one is older than the frontend's dependencies need
 - 2026-09-29: `backend/.golangci.yml` re-vendored from `sneak/prompts` (issue
   #41): `gomodguard`, deprecated in golangci-lint v2.12.0, is disabled and its
   successor `gomodguard_v2` enabled with the org block list, so lint runs print

backend/README.md

@@ -104,7 +104,8 @@ this server. The image's entrypoint, `bin/entrypoint.sh`, starts the server as
 user `netwatch` (uid 1000) with `BIND_ADDRESS=127.0.0.1` and `PORT=8081`, so
 only nginx reaches it, and with `TRUSTED_PROXIES=127.0.0.1/32`, so it takes the
 client address nginx passes on and no other. `DATA_DIR` is `/data/reports`, on
-the `/data` volume, which `netwatch` owns. nginx replaces the security headers
+the `/data` volume; the entrypoint creates it and gives it and `/data` to
+`netwatch` before starting the server. nginx replaces the security headers
 this server sets with those in the root `security-headers.conf`, so those are
 what clients of the image see.
 

bin/entrypoint.sh

@@ -61,6 +61,29 @@ for proxy in $(printf '%s' "$TRUSTED_PROXIES" | tr ',' ' '); do
     echo "set_real_ip_from $cidr;"
 done > /etc/nginx/trusted-proxies.conf
 
+# netwatch-server keeps its report files in DATA_DIR, on the /data
+# volume, which may be a host directory owned by root or by another
+# uid. Both are given to the netwatch user here, with the mode the
+# server gives a directory it creates, so the host directory needs no
+# preparing.
+#
+# chown and chmod, run as root, change whatever a symbolic link on the
+# path points to, anywhere in the container, and the netwatch user can
+# put one in /data. So the start stops unless readlink -f, which
+# follows every link on a path, gives /data and DATA_DIR back as they
+# are. It also writes a path in full, so a DATA_DIR with '.', '..' or
+# an extra '/' in it is refused too.
+export DATA_DIR="${DATA_DIR:-/data/reports}"
+mkdir -p "$DATA_DIR" || exit 1
+if [ "$(readlink -f /data)" != /data ] ||
+    [ "$(readlink -f "$DATA_DIR")" != "$DATA_DIR" ]; then
+    echo "entrypoint: DATA_DIR must be a full path with no '.', '..'," \
+        "extra '/' or symbolic link on it or on /data, not '$DATA_DIR'" >&2
+    exit 1
+fi
+chown -R netwatch:netwatch /data "$DATA_DIR" || exit 1
+chmod 750 /data "$DATA_DIR" || exit 1
+
 # A stop signal is only noted here; the loop below acts on it.
 stop_requested=""
 trap 'stop_requested=yes' TERM INT

script/bootstrap

@@ -3,12 +3,12 @@
 # this repo. Idempotent: every install is guarded by a check so already
 # installed tools are skipped. Base tooling comes from nix, apt, brew,
 # or apk (detected in that order); assumes nothing is present. Node is
-# used directly if installed; otherwise it is installed at a pinned
+# used directly if it is at least NODE_MIN_VERSION; otherwise it is
-# version via nvm (installing nvm itself first, from a hash-verified
+# installed at a pinned version via nvm (installing nvm itself first,
-# release archive, never curl | sh). Go, with its gofmt, is used
+# from a hash-verified release archive, never curl | sh). Go, with its
-# directly if it is at least the version backend/go.mod asks for;
+# gofmt, is used directly if it is at least the version backend/go.mod
-# otherwise the pinned Go release is installed from its hash-verified
+# asks for; otherwise the pinned Go release is installed from its
-# archive.
+# hash-verified archive.
 #
 # What this script installs outside the system package manager lives
 # under $HOME and is linked into ~/.local/bin, where make and the git
@@ -23,6 +23,10 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
 
 # Pinned versions, 2026-07-07
 NODE_VERSION="22.17.0"
+# The oldest node the frontend's dependencies accept: the "engines"
+# field of puppeteer-core 25.5.0, the most demanding of them, asks for
+# 22.12.0 or newer, 2026-09-29. An older installed node is not used.
+NODE_MIN_VERSION="22.12.0"
 NVM_VERSION="0.40.3"
 # sha256 of https://github.com/nvm-sh/nvm/archive/refs/tags/v0.40.3.tar.gz
 NVM_SHA256="5f4d6aaa04a177dc93c985e31dbc411ab6b8c6e1e21d8015dbc1372625fcd1d0"
@@ -136,8 +140,22 @@ ensure_nvm() {
     rm -rf "$tmp"
 }
 
+# node_ok: the node on PATH is at least NODE_MIN_VERSION. node itself
+# compares the two: major, then minor, then patch.
+node_ok() {
+    if missing node; then return 1; fi
+    node -e '
+        const have = process.versions.node.split(".").map(Number);
+        const want = process.argv[1].split(".").map(Number);
+        for (let i = 0; i < 3; i++) {
+            if (have[i] !== want[i]) process.exit(have[i] > want[i] ? 0 : 1);
+        }
+    ' "$NODE_MIN_VERSION"
+}
+
+# ensure_node: unless node_ok, install NODE_VERSION and link its node.
 ensure_node() {
-    if ! missing node; then return 0; fi
+    if node_ok; then return 0; fi
     ensure_nvm
     nvm_sh "nvm install $NODE_VERSION"
     link_bin "$HOME/.nvm/versions/node/v$NODE_VERSION/bin/node" node

script/cibuild

@@ -1,15 +1,29 @@
 #!/bin/sh
-# script/cibuild: run the CI build: build the one image from Dockerfile,
+# script/cibuild: run the CI build. It bootstraps first: a CI runner
-# whose stages run the checks as build steps (the backend's fmt-check,
+# checks out and runs this and nothing else, and script/fmt-check runs
-# lint and tests, and the frontend's test, lint and fmt-check). This is
+# the formatter on the host, which a pristine checkout cannot do.
-# the only build step the Gitea workflow runs.
+# --no-cache for the same reason as script/docker: the gate phases the
+# final stage depends on are RUN steps, and a cached one is a check that
+# did not run.
 set -eu
 
-ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
 
 main() {
     cd "$ROOT"
-    timeout 300 docker build .
+    "$SCRIPT_DIR/bootstrap"
+    "$SCRIPT_DIR/check"
+    # Own line: a failing command substitution inside an argument does
+    # not trip `set -e`, so the inline form degrades silently to an
+    # empty constant. VERSION is computed here because .dockerignore
+    # excludes .git, so `git describe` in a build stage yields an empty
+    # version without failing.
+    version="$(git describe --tags --always --dirty 2>/dev/null || true)"
+    [ -n "$version" ] || version="unknown"
+    docker build --no-cache \
+        --build-arg VERSION="$version" \
+        -t "$("$SCRIPT_DIR/projectname")" .
 }
 
 main "$@"