build: run the browser e2e suites in CI (closes #259)

Nothing automatic ran either e2e suite, so every browser-level guarantee in this
wallet -- WebAssembly under the shipped CSP, the recovery-phrase and private-key
DOM wipes, the ConfirmTx spend gate, the dApp approval round trips -- held only
when a human or an agent remembered to run it by hand.

.gitea/workflows/e2e.yml adds two jobs, e2e-chrome and e2e-firefox, one per
browser so a Chrome failure cannot hide the Firefox result. They are separate
from the check workflow: REPO_POLICIES.md caps make test at 20 seconds and
script/cibuild is a docker build whose Dockerfile runs make check, so neither the
cap nor the local fast path is touched. make check is byte-for-byte unchanged.

Neither suite could run on the runner as it stood, and the reason is not
docker-in-docker. The runner executes a job inside a container against the HOST's
docker daemon, and the job's checkout lives on a docker volume rather than a host
path, so `docker run -v "$PWD:/work"` is resolved by the host, silently succeeds
and mounts an empty directory -- measured on this runner. The runner image's node
is also too old to install this repo's dependencies. Both suites therefore ship
the repo to the daemon as a build context and build the extension inside the
pinned image, which leaves docker as the only prerequisite on a runner or a
laptop. The suites themselves are unchanged; only how the repo reaches the
container is.

Both scripts now build with --iidfile and run the image by ID rather than by tag,
so two clones running a suite at once on the same host cannot swap it under each
other.

The jobs report, they do not gate. Whether a check blocks a merge is Gitea branch
protection, which this repo does not configure, so a failure is a red mark a
reviewer must account for. Nothing can pass vacuously: no continue-on-error, no
`|| true`, and both scripts exit non-zero when docker is missing, when the image
build fails and when the browser fails to start.

Wiring this up measured something that has to be said rather than absorbed: the
Chrome suite is flaky under load. Two of six runs of unmutated code on a loaded
machine lost the approval popup out from under the dApp signing wait. It is
filed as #287 and not papered over here -- no retry wrapper, no longer timeout,
no weakened assertion -- and it is the reason e2e-chrome cannot become a
required check yet. README and the workflow say so where a reader meets them.

.gitea/workflows/e2e.yml

@@ -0,0 +1,49 @@
+name: e2e
+on: [push]
+
+# The browser end-to-end suites, one job per browser, deliberately kept out
+# of the check workflow: REPO_POLICIES.md caps make test at 20 seconds and
+# script/cibuild is a plain `docker build .` whose Dockerfile runs
+# make check, so folding a browser suite into either would blow that cap
+# and slow the local fast path. Before this workflow every browser-level
+# guarantee in this repo held only when a human remembered to run it.
+#
+# One job per browser rather than two steps in one job, so a Chrome failure
+# does not hide the Firefox result.
+#
+# Each job is one script and nothing else. Both scripts need docker and
+# nothing else — they deliver the repo to the daemon as a build context and
+# build the extension inside the pinned image — which is what makes them
+# runnable here at all: the runner executes the job in a container against
+# the host's docker socket, so a `-v "$PWD:/work"` source path is resolved
+# by the host daemon and mounts an empty directory, and the runner image's
+# node is too old to install this repo's dependencies.
+#
+# These jobs REPORT, they do not gate. Whether a check blocks a merge is
+# Gitea branch protection, which this repo does not configure, so a failure
+# here is a red mark a reviewer has to account for rather than a hard
+# block. Making e2e-chrome a required check is blocked on the measured
+# flake in the dApp signing wait -- two of six runs of unmutated code on a
+# loaded machine -- tracked as
+# https://git.eeqj.de/sneak/AutistMask/issues/287. A gate that fails at
+# random teaches people to merge past red.
+#
+# Nothing here may pass vacuously. There is no continue-on-error and no
+# `|| true`. Both scripts exit non-zero when docker is missing, when the
+# image build fails, and when the browser fails to start; the Chrome
+# harness aborts the suite outright if its network interception is not in
+# effect.
+jobs:
+    e2e-chrome:
+        runs-on: ubuntu-latest
+        steps:
+            # actions/checkout v4.2.2, 2026-02-22
+            - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+            - run: script/test-e2e
+
+    e2e-firefox:
+        runs-on: ubuntu-latest
+        steps:
+            # actions/checkout v4.2.2, 2026-02-22
+            - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+            - run: script/test-e2e-firefox

README.md

@@ -83,10 +83,11 @@ provide:
   git pre-commit hook
 - `script/projectname` — print the project name (used for the Docker image tag)
 - `script/test` — run the test suite (jest)
-- `script/test-e2e` — run the Chrome browser end-to-end suite (docker required;
-  see [End-to-End Tests](#end-to-end-tests))
-- `script/test-e2e-firefox` — run the Firefox browser end-to-end suite (docker
-  required; builds its own pinned image, see
+- `script/test-e2e` — run the Chrome browser end-to-end suite (docker is the
+  only prerequisite: it builds a pinned image that carries the repo and a fresh
+  extension build, see [End-to-End Tests](#end-to-end-tests))
+- `script/test-e2e-firefox` — run the Firefox browser end-to-end suite (same,
+  against an image with a pinned Firefox and geckodriver, see
   [End-to-End Tests](#end-to-end-tests))
 - `script/lint` — run the linter
 - `script/fmt` — format all files (writes)
@@ -136,11 +137,12 @@ are outside `make check`.
 
 `make test-e2e` builds `dist/chrome/` and drives the **real popup in a real
 Chrome**, loaded as an unpacked MV3 extension inside a pinned
-`mcr.microsoft.com/playwright` container (pinned by digest in `script/test-e2e`;
-docker is required and the suite fails loudly rather than skipping if it is
-unavailable). The suite lives in `tests/e2e/` and is driven by
-`playwright-core`, whose version must stay matched to the container's Playwright
-version — the browsers ship inside the image.
+`mcr.microsoft.com/playwright` container (pinned by digest in
+`tests/e2e/Dockerfile`, which is also where the extension is built; docker is
+required and the suite fails loudly rather than skipping if it is unavailable).
+The suite lives in `tests/e2e/` and is driven by `playwright-core`, whose
+version must stay matched to the container's Playwright version — the browsers
+ship inside the image.
 
 It covers popup load, WebAssembly compilation under the shipped CSP (see
 [Content Security Policy](#content-security-policy)), wallet creation through
@@ -320,9 +322,46 @@ Two limits are worth knowing, both real differences from the Chrome suite:
 Neither `make test-e2e` nor `make test-e2e-firefox` is part of `make check` or
 `make test`. `REPO_POLICIES.md` caps `make test` at 20 seconds and a browser
 suite does not fit; nothing in `tests/e2e/` is named `*.test.js`, so jest cannot
-pick it up either. Neither is wired into the Gitea workflow yet —
-docker-in-docker in CI is a separate question. Run them locally before changing
-anything under `src/popup/views/`.
+pick it up either. Run them locally before changing anything under
+`src/popup/views/`.
+
+### In CI
+
+`.gitea/workflows/e2e.yml` runs both suites on every push, as two jobs —
+`e2e-chrome` and `e2e-firefox` — separate from the `check` workflow, so the
+20-second `make test` cap and the local fast path are untouched. Each job is a
+checkout and the matching `script/` entrypoint, nothing else.
+
+Docker is the only thing either job needs from the runner, and that is not an
+accident. The runner executes a job inside a container against the **host's**
+docker daemon, so a `docker run -v "$PWD:/work"` source path is resolved by the
+host and mounts an empty directory, and the runner image's node is too old to
+install this repo's dependencies. Both suites therefore ship the repo to the
+daemon as a build context and build the extension inside the image, which works
+identically on a laptop.
+
+The jobs **report, they do not gate.** A failure is a red mark against the
+commit that a reviewer has to account for, not a hard block: whether a check
+blocks a merge is Gitea branch protection, which this repo does not configure.
+
+That is not only a statement about configuration. The Chrome suite is
+**measurably flaky under load** — two of six runs of unmutated code on a busy
+machine lost the approval popup out from under the dApp signing wait, always in
+the `#183` section, tracked as
+[#287](https://git.eeqj.de/sneak/AutistMask/issues/287). So a red `e2e-chrome`
+has to be read before it is believed, and that flake is the blocker to ever
+making this a required check. Do not answer it with a retry wrapper: a suite
+that reruns until it is green stops being evidence.
+
+Nothing in either job can pass vacuously. There is no `continue-on-error` and no
+`|| true`; both scripts exit non-zero when docker is missing, when the image
+build fails, and when the browser fails to start; the Chrome harness aborts the
+suite outright if its network interception is not in effect.
+
+Measured on this repo's runner: `e2e-chrome` about 1m55s cold, almost all of it
+the one-time pull of the pinned ~800MB Playwright layer, and well under a minute
+once that layer is cached. `e2e-firefox` about 1m05s cold, and it caches its
+Firefox and geckodriver downloads the same way.
 
 ## Rationale
 

TODO.md

@@ -33,7 +33,8 @@ The backlog lives on the
 authoritative; this file does not duplicate it. Full policy file set present.
 Real-browser end-to-end suites (`make test-e2e` for Chrome,
 `make test-e2e-firefox` for Firefox) now sit alongside `make check`, which
-cannot see a runtime `ReferenceError` in a popup view.
+cannot see a runtime `ReferenceError` in a popup view, and
+`.gitea/workflows/e2e.yml` runs both of them on every push.
 
 # Next Step
 
@@ -45,6 +46,24 @@ undefined identifiers, which is how
 
 # Completed Steps
 
+- 2026-08-14: CI runs the browser end-to-end suites. `.gitea/workflows/e2e.yml`
+  runs `script/test-e2e` and `script/test-e2e-firefox` as two jobs on every
+  push, separate from `check`, so `make check` and its 20-second `make test` cap
+  are untouched. Every browser-level guarantee in this repo — the WASM-under-CSP
+  check, the recovery-phrase and private-key DOM wipes, the ConfirmTx spend
+  gate, the dApp approval round trips — was enforced only when a human
+  remembered to run it by hand. The suites could not run on the runner as they
+  stood: the runner executes a job in a container against the host's docker
+  daemon, so `docker run -v "$PWD:/work"` mounts an empty directory (measured),
+  and the runner image's node cannot install this repo's dependencies. Both
+  suites now ship the repo to the daemon as a build context and build the
+  extension inside the pinned image, so docker is the only prerequisite on a
+  runner or a laptop, and both run the image by ID rather than by tag so
+  concurrent clones cannot swap it. The jobs report rather than gate — this repo
+  configures no branch protection, and the Chrome suite is measurably flaky
+  under load, filed as [#287](https://git.eeqj.de/sneak/AutistMask/issues/287)
+  rather than papered over
+  ([#259](https://git.eeqj.de/sneak/AutistMask/issues/259)).
 - 2026-08-12: EIP-1193 error codes now reach the page. `src/content/inpage.js`
   rebuilt every failure as `new Error(error.message)`, so the code the
   background produced and the content script relayed intact was dropped in the
@@ -368,9 +387,5 @@ tracker.
 - Pre-1.0 security review of the extension (key handling, DEBUG mode policy, RPC
   input validation) before any 1.0rc tag. Individual filed issues are parts of
   it, but the review is broader than any of them.
-- Decide whether docker-in-docker makes `make test-e2e` and
-  `make test-e2e-firefox` runnable in the Gitea workflow. Extending the Chrome
-  suite itself is tracked as
-  [#183](https://git.eeqj.de/sneak/AutistMask/issues/183).
 - Cut 1.0.0 once the milestone is empty, then continue tagging as milestones
   land.

script/test-e2e

@@ -7,17 +7,29 @@
 # caps make test at 20 seconds and a browser suite does not fit. Run it
 # yourself before touching popup views; it is the only check that can see
 # a used-but-not-imported identifier blow up at runtime.
+# .gitea/workflows/e2e.yml also runs it on every push, in a job separate
+# from check so that cap and the local fast path both stay intact.
+#
+# Docker is the only prerequisite. The repo reaches the container as a
+# build context and the extension is built inside it (see
+# tests/e2e/Dockerfile), so nothing here depends on the node, yarn or make
+# on the machine that starts the run. That is not a convenience: a bind
+# mount cannot work under Gitea Actions, and the runner image's node is too
+# old to install this repo's dependencies.
 set -eu
 
-ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
 
-# mcr.microsoft.com/playwright:v1.56.0-noble, 2026-08-09
-#
-# The playwright-core devDependency is pinned to the matching Playwright
-# version (1.56.0) and the two must be bumped together: the browsers ship
-# inside this image, and playwright-core looks for the exact browser
-# revision its own version expects. A mismatch fails at launch.
-IMAGE="mcr.microsoft.com/playwright@sha256:35246d87a7c88ea9b771c65d33171b2611b02a8253b4b12ce6f94376c55f99f2"
+IMAGE="$("$SCRIPT_DIR/projectname")-e2e-chrome"
+
+IIDFILE=""
+
+cleanup() {
+    if [ -n "$IIDFILE" ]; then
+        rm -f "$IIDFILE"
+    fi
+}
 
 main() {
     cd "$ROOT"
@@ -27,14 +39,23 @@ main() {
         exit 1
     fi
 
-    echo "Building extension for e2e..."
-    yarn run build 2>&1
+    IIDFILE="$(mktemp)"
+    trap cleanup EXIT
+    trap 'cleanup; exit 130' INT TERM
+
+    echo "Building the Chrome e2e image (extension included)..."
+    docker build --iidfile "$IIDFILE" -t "$IMAGE" -f tests/e2e/Dockerfile .
 
     echo "Running e2e suite in the pinned Playwright container..."
+    # The image is run by ID, not by tag: where two clones of this repo run
+    # the suite at once, the other build can move the tag between this
+    # build and this run, and the suite would then silently test the other
+    # checkout.
+    #
     # --ipc=host: Chromium's shared-memory needs more than the default
     #   64MB /dev/shm or renderers crash.
-    # --user: keep files the suite touches owned by the caller, not root.
-    # HOME=/tmp: the mapped uid has no home directory in the image.
+    # HOME=/tmp: the image's root home is not a reliable place for the
+    #   browser profile.
     # PW_EXPERIMENTAL_SERVICE_WORKER_NETWORK_EVENTS=1: without it,
     #   ctx.route() intercepts page requests only, and every fetch made by
     #   the MV3 background service worker — including the phishing
@@ -51,13 +72,10 @@ main() {
     #   on a deliberate bump.
     docker run --rm \
         --ipc=host \
-        --user "$(id -u):$(id -g)" \
         -e HOME=/tmp \
         -e PW_EXPERIMENTAL_SERVICE_WORKER_NETWORK_EVENTS=1 \
         -e "E2E_TRACE_NETWORK=${E2E_TRACE_NETWORK:-0}" \
-        -v "$ROOT:/work" \
-        -w /work \
-        "$IMAGE" \
+        "$(cat "$IIDFILE")" \
         node tests/e2e/run.js
 }
 

script/test-e2e-firefox

@@ -5,12 +5,17 @@
 #
 # Deliberately NOT called by script/check or script/test, for the same
 # reason as the Chrome suite: REPO_POLICIES.md caps make test at 20 seconds
-# and a browser suite does not fit.
+# and a browser suite does not fit. .gitea/workflows/e2e.yml also runs it
+# on every push, in a job separate from check.
 #
-# Unlike script/test-e2e this builds its image locally, because no
+# Unlike script/test-e2e this builds its base image locally, because no
 # published image carries both a pinned Firefox and a matching geckodriver.
 # All three external artifacts are pinned by digest inside the Dockerfile;
-# see tests/e2e/firefox/Dockerfile.
+# see tests/e2e/firefox/Dockerfile, which also explains why the repo and
+# the extension build are baked into the image rather than mounted.
+#
+# Docker is the only prerequisite: nothing here depends on the node, yarn
+# or make on the machine that starts the run.
 set -eu
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -18,6 +23,14 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
 
 IMAGE="$("$SCRIPT_DIR/projectname")-e2e-firefox"
 
+IIDFILE=""
+
+cleanup() {
+    if [ -n "$IIDFILE" ]; then
+        rm -f "$IIDFILE"
+    fi
+}
+
 main() {
     cd "$ROOT"
 
@@ -26,16 +39,20 @@ main() {
         exit 1
     fi
 
-    echo "Building extension for e2e..."
-    yarn run build 2>&1
+    IIDFILE="$(mktemp)"
+    trap cleanup EXIT
+    trap 'cleanup; exit 130' INT TERM
 
-    # The build context is tests/e2e/firefox/ and holds nothing but the
-    # Dockerfile: the harness itself arrives over the bind mount below, so
-    # editing it never invalidates an image layer.
-    echo "Building the pinned Firefox e2e image..."
-    docker build -t "$IMAGE" "$ROOT/tests/e2e/firefox"
+    echo "Building the pinned Firefox e2e image (extension included)..."
+    docker build --iidfile "$IIDFILE" -t "$IMAGE" \
+        -f tests/e2e/firefox/Dockerfile .
 
     echo "Running the Firefox e2e suite..."
+    # The image is run by ID, not by tag: where two clones of this repo run
+    # the suite at once, the other build can move the tag between this
+    # build and this run, and the suite would then silently test the other
+    # checkout.
+    #
     # --shm-size=1g: Firefox needs more than the default 64MB /dev/shm.
     # --network none: the suite stubs nothing, so this is what keeps the
     #   run offline and deterministic. The extension swallows its own
@@ -43,8 +60,8 @@ main() {
     #   network note in README.md. Weaker than the Chrome suite's
     #   fixture interception, and honestly so — it proves no request
     #   escaped, but it cannot report which ones were attempted.
-    # --user: keep files the suite touches owned by the caller, not root.
-    # HOME=/tmp: the mapped uid has no home directory in the image.
+    # HOME=/tmp: the image's root home is not a reliable place for the
+    #   browser profile.
     #
     # No --privileged. Firefox's sandbox logs
     # "CanCreateUserNamespace() clone() failure: EPERM" on startup here;
@@ -52,11 +69,8 @@ main() {
     docker run --rm \
         --shm-size=1g \
         --network none \
-        --user "$(id -u):$(id -g)" \
         -e HOME=/tmp \
-        -v "$ROOT:/work" \
-        -w /work \
-        "$IMAGE" \
+        "$(cat "$IIDFILE")" \
         node tests/e2e/firefox/run.js dist/firefox
 }
 

tests/e2e/Dockerfile

@@ -0,0 +1,34 @@
+# Chrome end-to-end image: the pinned Playwright image with this repo and a
+# freshly built extension inside it, built by script/test-e2e. The suite is
+# still started with `docker run`, so every runtime flag the harness needs
+# (--ipc=host in particular) applies as before.
+#
+# The repo is baked in rather than bind-mounted because a bind mount does
+# not resolve under Gitea Actions: the runner runs the job in a container
+# against the HOST's docker socket, so the source side of a -v is resolved
+# by the host daemon while the job's checkout lives on a docker volume that
+# is not a host path -- the mount silently succeeds and /work is empty. A
+# build context is streamed to the daemon and so works from anywhere.
+# Building the extension here too means the machine starting a run needs
+# docker and nothing else.
+
+# mcr.microsoft.com/playwright:v1.56.0-noble, 2026-08-09
+#
+# The playwright-core devDependency is pinned to the matching Playwright
+# version (1.56.0) and the two must be bumped together: the browsers ship
+# inside this image, and playwright-core looks for the exact browser
+# revision its own version expects. A mismatch fails at launch.
+FROM mcr.microsoft.com/playwright@sha256:35246d87a7c88ea9b771c65d33171b2611b02a8253b4b12ce6f94376c55f99f2
+
+WORKDIR /work
+
+# Same layering as the root Dockerfile: script/bootstrap installs the
+# prerequisites and the dependencies, and the manifests are copied first so
+# that layer is cached until they change.
+COPY script/ script/
+COPY package.json yarn.lock ./
+RUN script/bootstrap
+
+COPY . .
+
+RUN make build

tests/e2e/firefox/Dockerfile

@@ -1,10 +1,24 @@
 # Firefox end-to-end image: stock Firefox plus geckodriver on a node base,
-# built by script/test-e2e-firefox. The repo is bind-mounted at /work; the
-# harness itself has no dependencies, so nothing is installed for it.
+# with this repo and a freshly built extension inside it, built by
+# script/test-e2e-firefox. The harness itself has no dependencies, so
+# nothing is installed for it.
 #
-# All three external artifacts are pinned by digest. The Firefox version in
-# particular must not float: -remote-allow-system-access is mandatory on 153
-# and was not on 142, so the flag the harness passes is version-coupled.
+# The build context is the repo root. The repo is baked in rather than
+# bind-mounted because a bind mount does not resolve under Gitea Actions:
+# the runner runs the job in a container against the HOST's docker socket,
+# so the source side of a -v is resolved by the host daemon while the job's
+# checkout lives on a docker volume that is not a host path -- the mount
+# silently succeeds and /work is empty. Baking the build in is also the
+# only way this suite can have both a built extension and the
+# `--network none` it runs under, since a container with no network cannot
+# install anything.
+#
+# All three external artifacts are pinned by digest, and are fetched in
+# layers above the repo copy, so editing the harness or any source file
+# re-runs only the two cheap layers at the bottom. The Firefox version in
+# particular must not float: -remote-allow-system-access is mandatory on
+# 153 and was not on 142, so the flag the harness passes is
+# version-coupled.
 
 # node:22-bookworm-slim, 2026-08-12
 FROM node@sha256:d649c27dae7ba0137b3cef5dd75baa422c08dc3d9e3fc0c23dfb172dc3cc6436
@@ -48,4 +62,16 @@ ENV FIREFOX_BIN=/opt/firefox/firefox
 ENV GECKODRIVER=/usr/local/bin/geckodriver
 
 WORKDIR /work
+
+# Same layering as the root Dockerfile: script/bootstrap installs the
+# prerequisites and the dependencies, and the manifests are copied first so
+# that layer is cached until they change.
+COPY script/ script/
+COPY package.json yarn.lock ./
+RUN script/bootstrap
+
+COPY . .
+
+RUN make build
+
 CMD ["node", "tests/e2e/firefox/run.js", "dist/firefox"]