test: containerized Firefox end-to-end harness (closes #184)
All checks were successful
check / check (push) Successful in 46s

Drives the real popup in a real Firefox with dist/firefox/ installed as an
unpacked MV2 temporary add-on, via geckodriver. Covers popup load, wallet
creation through the UI, and the Add Token screen. Outside make check, like
the Chrome suite.

Zero npm dependencies: tests/e2e/firefox/driver.js is a WebDriver client
over global fetch and child_process against geckodriver's HTTP API. The
Dockerfile pins the node base image, the Firefox 153.0.3 tarball and
geckodriver 0.36.0 by digest.

Errors are read from the privileged nsIConsoleService in Marionette's chrome
context, filtered to non-warning entries whose sourceName is the extension
origin. BiDi log.entryAdded delivers nothing at all for extension pages, so
a Playwright-BiDi or Puppeteer-BiDi harness would see nothing and report
success; the code says so where someone would be tempted to simplify it.
This also captures background-page and content-script errors, verified by
probe rather than assumed.

No driver layer is shared with the Chrome suite and the three UI steps are
written twice deliberately: the two backends have no common substrate, and
three steps do not pay for a shim.

Two limits are documented rather than papered over. Error capture is
poll-based, so an error is attributed to a step and not to a moment within
it. Nothing is stubbed; the container runs with --network none instead,
which proves no request escaped but cannot report which were attempted.
This commit is contained in:
2026-08-12 08:42:22 +00:00
parent bd4bdcafc7
commit 288c5560de
7 changed files with 911 additions and 16 deletions

View File

@@ -83,7 +83,10 @@ provide:
git pre-commit hook
- `script/projectname` — print the project name (used for the Docker image tag)
- `script/test` — run the test suite (jest)
- `script/test-e2e` — run the browser end-to-end suite (docker required; see
- `script/test-e2e` — run the Chrome browser end-to-end suite (docker required;
see [End-to-End Tests](#end-to-end-tests))
- `script/test-e2e-firefox` — run the Firefox browser end-to-end suite (docker
required; builds its own pinned image, see
[End-to-End Tests](#end-to-end-tests))
- `script/lint` — run the linter
- `script/fmt` — format all files (writes)
@@ -115,6 +118,14 @@ The Makefile shims to those. It also carries a few targets that have no
## End-to-End Tests
There are two suites, one per browser, and they share no code. Chrome runs on
Playwright; Firefox has its own WebDriver client, because Playwright cannot
observe errors on a Firefox extension page at all — see
[Firefox](#firefox-make-test-e2e-firefox) below. Both require docker, and both
are outside `make check`.
### Chrome (`make test-e2e`)
`make test-e2e` builds `dist/chrome/` and drives the **real popup in a real
Chrome**, loaded as an unpacked MV3 extension inside a pinned
`mcr.microsoft.com/playwright` container (pinned by digest in `script/test-e2e`;
@@ -172,12 +183,63 @@ a `ReferenceError` from a used-but-not-imported identifier is invisible to
`make check` (`script/lint` is only `prettier --check`) but fatal in a browser,
and this suite exists because exactly that class of bug shipped twice.
`make test-e2e` is deliberately **not** part of `make check` or `make test`.
`REPO_POLICIES.md` caps `make test` at 20 seconds and a browser suite does not
fit; nothing in `tests/e2e/` is named `*.test.js`, so jest cannot pick it up
either. It is also not wired into the Gitea workflow yet — docker-in-docker in
CI is a separate question. Run it locally before changing anything under
`src/popup/views/`.
### Firefox (`make test-e2e-firefox`)
`make test-e2e-firefox` builds `dist/firefox/` and drives the **real popup in a
real Firefox**, installed as an unpacked MV2 temporary add-on via geckodriver.
It covers popup load, wallet creation through the UI, and the Add Token screen.
The suite lives in `tests/e2e/firefox/` and has **no npm dependencies at all**:
it is a small WebDriver client built on global `fetch` and `child_process`
against geckodriver's HTTP API.
Unlike the Chrome suite it builds its own container image rather than pulling a
published one, because no published image carries both a pinned Firefox and a
matching geckodriver. `tests/e2e/firefox/Dockerfile` pins all three external
artifacts by digest — the `node` base image, the Firefox 153.0.3 tarball, and
geckodriver 0.36.0 — and the Firefox version in particular must not float:
`-remote-allow-system-access` is **mandatory** on 153 and was not on 142.
Without that flag, both navigating to `moz-extension://` and running
chrome-context script fail with `unsupported operation`. The flag grants the
driver full chrome privileges over that browser, which is acceptable only
because it is a throwaway container.
The popup's `moz-extension://` uuid is **pinned, not discovered**: the profile
pref `extensions.webextensions.uuids` maps the extension id that
`manifest/firefox.json` already declares to a fixed uuid, so the popup URL is
deterministic. Navigation uses **classic** WebDriver `POST /session/{id}/url`,
because BiDi's `browsingContext.navigate` refuses `moz-extension://` outright.
**Any uncaught error from a `moz-extension://` source fails the run**, including
errors from the background page and content scripts, which the suite never
navigates to. Errors are read from the privileged `nsIConsoleService` in
Marionette's chrome context and filtered to non-warning entries whose
`sourceName` is the extension origin. That mechanism is not a stylistic choice.
WebDriver BiDi's `log.entryAdded` delivers **nothing** for extension pages: on a
plain `http://` page it reports uncaught errors with stack traces, and on the
`moz-extension://` popup it reports zero events, because Firefox's remote agent
excludes extension browsing contexts from BiDi observation. Any harness built on
Playwright-BiDi or Puppeteer-BiDi would therefore see nothing and report
success, which is exactly the vacuous check this repo has already shipped twice.
Do not migrate this suite to BiDi.
Two limits are worth knowing, both real differences from the Chrome suite:
- **Error capture is poll-based, not event-streamed.** The console is drained at
each step boundary, so an error is attributed to the step it was drained
after. Nothing is lost, but an error cannot be located within a step the way
the Chrome suite's `pageerror` events can.
- **Nothing is stubbed.** There is no fixture layer; the container runs with
`--network none` instead, so the run is offline and deterministic and no
request can escape. The extension swallows its own fetch failures, so the
flows are unaffected. This is weaker than the Chrome suite's interception: it
proves nothing got out, but it cannot report which requests were attempted.
Neither `make test-e2e` nor `make test-e2e-firefox` is part of `make check` or
`make test`. `REPO_POLICIES.md` caps `make test` at 20 seconds and a browser
suite does not fit; nothing in `tests/e2e/` is named `*.test.js`, so jest cannot
pick it up either. Neither is wired into the Gitea workflow yet —
docker-in-docker in CI is a separate question. Run them locally before changing
anything under `src/popup/views/`.
## Rationale