build: run the browser e2e suites in CI (closes #259)
Nothing automatic ran either e2e suite, so every browser-level guarantee in this wallet -- WebAssembly under the shipped CSP, the recovery-phrase and private-key DOM wipes, the ConfirmTx spend gate, the dApp approval round trips -- held only when a human or an agent remembered to run it by hand. .gitea/workflows/e2e.yml adds two jobs, e2e-chrome and e2e-firefox, one per browser so a Chrome failure cannot hide the Firefox result. They are separate from the check workflow: REPO_POLICIES.md caps make test at 20 seconds and script/cibuild is a docker build whose Dockerfile runs make check, so neither the cap nor the local fast path is touched. make check is byte-for-byte unchanged. Neither suite could run on the runner as it stood, and the reason is not docker-in-docker. The runner executes a job inside a container against the HOST's docker daemon, and the job's checkout lives on a docker volume rather than a host path, so `docker run -v "$PWD:/work"` is resolved by the host, silently succeeds and mounts an empty directory -- measured on this runner. The runner image's node is also too old to install this repo's dependencies. Both suites therefore ship the repo to the daemon as a build context and build the extension inside the pinned image, which leaves docker as the only prerequisite on a runner or a laptop. The suites themselves are unchanged; only how the repo reaches the container is. Both scripts now build with --iidfile and run the image by ID rather than by tag, so two clones running a suite at once on the same host cannot swap it under each other. The jobs report, they do not gate. Whether a check blocks a merge is Gitea branch protection, which this repo does not configure, so a failure is a red mark a reviewer must account for. Nothing can pass vacuously: no continue-on-error, no `|| true`, and both scripts exit non-zero when docker is missing, when the image build fails and when the browser fails to start.
This commit is contained in:
54
README.md
54
README.md
@@ -83,10 +83,11 @@ provide:
|
||||
git pre-commit hook
|
||||
- `script/projectname` — print the project name (used for the Docker image tag)
|
||||
- `script/test` — run the test suite (jest)
|
||||
- `script/test-e2e` — run the Chrome browser end-to-end suite (docker required;
|
||||
see [End-to-End Tests](#end-to-end-tests))
|
||||
- `script/test-e2e-firefox` — run the Firefox browser end-to-end suite (docker
|
||||
required; builds its own pinned image, see
|
||||
- `script/test-e2e` — run the Chrome browser end-to-end suite (docker is the
|
||||
only prerequisite: it builds a pinned image that carries the repo and a fresh
|
||||
extension build, see [End-to-End Tests](#end-to-end-tests))
|
||||
- `script/test-e2e-firefox` — run the Firefox browser end-to-end suite (same,
|
||||
against an image with a pinned Firefox and geckodriver, see
|
||||
[End-to-End Tests](#end-to-end-tests))
|
||||
- `script/lint` — run the linter
|
||||
- `script/fmt` — format all files (writes)
|
||||
@@ -136,11 +137,12 @@ are outside `make check`.
|
||||
|
||||
`make test-e2e` builds `dist/chrome/` and drives the **real popup in a real
|
||||
Chrome**, loaded as an unpacked MV3 extension inside a pinned
|
||||
`mcr.microsoft.com/playwright` container (pinned by digest in `script/test-e2e`;
|
||||
docker is required and the suite fails loudly rather than skipping if it is
|
||||
unavailable). The suite lives in `tests/e2e/` and is driven by
|
||||
`playwright-core`, whose version must stay matched to the container's Playwright
|
||||
version — the browsers ship inside the image.
|
||||
`mcr.microsoft.com/playwright` container (pinned by digest in
|
||||
`tests/e2e/Dockerfile`, which is also where the extension is built; docker is
|
||||
required and the suite fails loudly rather than skipping if it is unavailable).
|
||||
The suite lives in `tests/e2e/` and is driven by `playwright-core`, whose
|
||||
version must stay matched to the container's Playwright version — the browsers
|
||||
ship inside the image.
|
||||
|
||||
It covers popup load, WebAssembly compilation under the shipped CSP (see
|
||||
[Content Security Policy](#content-security-policy)), wallet creation through
|
||||
@@ -320,9 +322,37 @@ Two limits are worth knowing, both real differences from the Chrome suite:
|
||||
Neither `make test-e2e` nor `make test-e2e-firefox` is part of `make check` or
|
||||
`make test`. `REPO_POLICIES.md` caps `make test` at 20 seconds and a browser
|
||||
suite does not fit; nothing in `tests/e2e/` is named `*.test.js`, so jest cannot
|
||||
pick it up either. Neither is wired into the Gitea workflow yet —
|
||||
docker-in-docker in CI is a separate question. Run them locally before changing
|
||||
anything under `src/popup/views/`.
|
||||
pick it up either. Run them locally before changing anything under
|
||||
`src/popup/views/`.
|
||||
|
||||
### In CI
|
||||
|
||||
`.gitea/workflows/e2e.yml` runs both suites on every push, as two jobs —
|
||||
`e2e-chrome` and `e2e-firefox` — separate from the `check` workflow, so the
|
||||
20-second `make test` cap and the local fast path are untouched. Each job is a
|
||||
checkout and the matching `script/` entrypoint, nothing else.
|
||||
|
||||
Docker is the only thing either job needs from the runner, and that is not an
|
||||
accident. The runner executes a job inside a container against the **host's**
|
||||
docker daemon, so a `docker run -v "$PWD:/work"` source path is resolved by the
|
||||
host and mounts an empty directory, and the runner image's node is too old to
|
||||
install this repo's dependencies. Both suites therefore ship the repo to the
|
||||
daemon as a build context and build the extension inside the image, which works
|
||||
identically on a laptop.
|
||||
|
||||
The jobs **report, they do not gate.** A failure is a red mark against the
|
||||
commit that a reviewer has to account for, not a hard block: whether a check
|
||||
blocks a merge is Gitea branch protection, which this repo does not configure.
|
||||
|
||||
Nothing in either job can pass vacuously. There is no `continue-on-error` and no
|
||||
`|| true`; both scripts exit non-zero when docker is missing, when the image
|
||||
build fails, and when the browser fails to start; the Chrome harness aborts the
|
||||
suite outright if its network interception is not in effect.
|
||||
|
||||
Measured on this repo's runner: `e2e-chrome` about 1m55s cold, almost all of it
|
||||
the one-time pull of the pinned ~800MB Playwright layer, and well under a minute
|
||||
once that layer is cached. `e2e-firefox` about 1m05s cold, and it caches its
|
||||
Firefox and geckodriver downloads the same way.
|
||||
|
||||
## Rationale
|
||||
|
||||
|
||||
Reference in New Issue
Block a user