build: run the browser e2e suites in CI (closes #259)
All checks were successful
check / check (push) Successful in 33s
e2e / e2e-chrome (push) Successful in 51s
e2e / e2e-firefox (push) Successful in 20s

Nothing automatic ran either e2e suite, so every browser-level guarantee in this
wallet -- WebAssembly under the shipped CSP, the recovery-phrase and private-key
DOM wipes, the ConfirmTx spend gate, the dApp approval round trips -- held only
when a human or an agent remembered to run it by hand.

.gitea/workflows/e2e.yml adds two jobs, e2e-chrome and e2e-firefox, one per
browser so a Chrome failure cannot hide the Firefox result. They are separate
from the check workflow: REPO_POLICIES.md caps make test at 20 seconds and
script/cibuild is a docker build whose Dockerfile runs make check, so neither the
cap nor the local fast path is touched. make check is byte-for-byte unchanged.

Neither suite could run on the runner as it stood, and the reason is not
docker-in-docker. The runner executes a job inside a container against the HOST's
docker daemon, and the job's checkout lives on a docker volume rather than a host
path, so `docker run -v "$PWD:/work"` is resolved by the host, silently succeeds
and mounts an empty directory -- measured on this runner. The runner image's node
is also too old to install this repo's dependencies. Both suites therefore ship
the repo to the daemon as a build context and build the extension inside the
pinned image, which leaves docker as the only prerequisite on a runner or a
laptop. The suites themselves are unchanged; only how the repo reaches the
container is.

Both scripts now build with --iidfile and run the image by ID rather than by tag,
so two clones running a suite at once on the same host cannot swap it under each
other.

The jobs report, they do not gate. Whether a check blocks a merge is Gitea branch
protection, which this repo does not configure, so a failure is a red mark a
reviewer must account for. Nothing can pass vacuously: no continue-on-error, no
`|| true`, and both scripts exit non-zero when docker is missing, when the image
build fails and when the browser fails to start.

Wiring this up measured something that has to be said rather than absorbed: the
Chrome suite is flaky under load. Two of six runs of unmutated code on a loaded
machine lost the approval popup out from under the dApp signing wait. It is
filed as #287 and not papered over here -- no retry wrapper, no longer timeout,
no weakened assertion -- and it is the reason e2e-chrome cannot become a
required check yet. README and the workflow say so where a reader meets them.
This commit is contained in:
2026-08-14 04:12:23 +00:00
parent 0be20d7270
commit 329f3e1558
7 changed files with 249 additions and 54 deletions

View File

@@ -83,10 +83,11 @@ provide:
git pre-commit hook
- `script/projectname` — print the project name (used for the Docker image tag)
- `script/test` — run the test suite (jest)
- `script/test-e2e` — run the Chrome browser end-to-end suite (docker required;
see [End-to-End Tests](#end-to-end-tests))
- `script/test-e2e-firefox` — run the Firefox browser end-to-end suite (docker
required; builds its own pinned image, see
- `script/test-e2e` — run the Chrome browser end-to-end suite (docker is the
only prerequisite: it builds a pinned image that carries the repo and a fresh
extension build, see [End-to-End Tests](#end-to-end-tests))
- `script/test-e2e-firefox` — run the Firefox browser end-to-end suite (same,
against an image with a pinned Firefox and geckodriver, see
[End-to-End Tests](#end-to-end-tests))
- `script/lint` — run the linter
- `script/fmt` — format all files (writes)
@@ -136,11 +137,12 @@ are outside `make check`.
`make test-e2e` builds `dist/chrome/` and drives the **real popup in a real
Chrome**, loaded as an unpacked MV3 extension inside a pinned
`mcr.microsoft.com/playwright` container (pinned by digest in `script/test-e2e`;
docker is required and the suite fails loudly rather than skipping if it is
unavailable). The suite lives in `tests/e2e/` and is driven by
`playwright-core`, whose version must stay matched to the container's Playwright
version — the browsers ship inside the image.
`mcr.microsoft.com/playwright` container (pinned by digest in
`tests/e2e/Dockerfile`, which is also where the extension is built; docker is
required and the suite fails loudly rather than skipping if it is unavailable).
The suite lives in `tests/e2e/` and is driven by `playwright-core`, whose
version must stay matched to the container's Playwright version — the browsers
ship inside the image.
It covers popup load, WebAssembly compilation under the shipped CSP (see
[Content Security Policy](#content-security-policy)), wallet creation through
@@ -320,9 +322,46 @@ Two limits are worth knowing, both real differences from the Chrome suite:
Neither `make test-e2e` nor `make test-e2e-firefox` is part of `make check` or
`make test`. `REPO_POLICIES.md` caps `make test` at 20 seconds and a browser
suite does not fit; nothing in `tests/e2e/` is named `*.test.js`, so jest cannot
pick it up either. Neither is wired into the Gitea workflow yet —
docker-in-docker in CI is a separate question. Run them locally before changing
anything under `src/popup/views/`.
pick it up either. Run them locally before changing anything under
`src/popup/views/`.
### In CI
`.gitea/workflows/e2e.yml` runs both suites on every push, as two jobs —
`e2e-chrome` and `e2e-firefox` — separate from the `check` workflow, so the
20-second `make test` cap and the local fast path are untouched. Each job is a
checkout and the matching `script/` entrypoint, nothing else.
Docker is the only thing either job needs from the runner, and that is not an
accident. The runner executes a job inside a container against the **host's**
docker daemon, so a `docker run -v "$PWD:/work"` source path is resolved by the
host and mounts an empty directory, and the runner image's node is too old to
install this repo's dependencies. Both suites therefore ship the repo to the
daemon as a build context and build the extension inside the image, which works
identically on a laptop.
The jobs **report, they do not gate.** A failure is a red mark against the
commit that a reviewer has to account for, not a hard block: whether a check
blocks a merge is Gitea branch protection, which this repo does not configure.
That is not only a statement about configuration. The Chrome suite is
**measurably flaky under load** — two of six runs of unmutated code on a busy
machine lost the approval popup out from under the dApp signing wait, always in
the `#183` section, tracked as
[#287](https://git.eeqj.de/sneak/AutistMask/issues/287). So a red `e2e-chrome`
has to be read before it is believed, and that flake is the blocker to ever
making this a required check. Do not answer it with a retry wrapper: a suite
that reruns until it is green stops being evidence.
Nothing in either job can pass vacuously. There is no `continue-on-error` and no
`|| true`; both scripts exit non-zero when docker is missing, when the image
build fails, and when the browser fails to start; the Chrome harness aborts the
suite outright if its network interception is not in effect.
Measured on this repo's runner: `e2e-chrome` about 1m55s cold, almost all of it
the one-time pull of the pinned ~800MB Playwright layer, and well under a minute
once that layer is cached. `e2e-firefox` about 1m05s cold, and it caches its
Firefox and geckodriver downloads the same way.
## Rationale