test: containerized Firefox end-to-end harness (closes #184)
All checks were successful
check / check (push) Successful in 34s

Drives the real popup in a real Firefox with dist/firefox/ installed as an
unpacked MV2 temporary add-on via geckodriver. make test-e2e-firefox, outside
make check like the Chrome suite. Zero npm dependencies: plain fetch and
child_process against geckodriver's HTTP API. Base image, Firefox tarball and
geckodriver are each pinned by digest and verified at build time.

Error capture reads the privileged console service through Marionette's chrome
context, not WebDriver BiDi. BiDi delivers nothing at all for extension pages,
so a BiDi-based harness would observe zero events and report success -- the
vacuous-check shape this repo has shipped twice. Both the driver and the README
say so where someone would be tempted to simplify.

Demonstrated to discriminate: a background page that throws at the top of the
file, a missing import, and an async throw where every UI assertion still
passes each fail the run.

Three limits are measured and documented rather than papered over: capture is
poll-based so an error is attributed to a step, not a moment; the console ring
buffer holds 250 messages and evicts the oldest, measured against a clean-run
peak of 4; and the drained window ends roughly 1.5s after the last step, with
observed jitter rather than a hard boundary. Content-script capture is marked
unverified because --network none leaves no page to inject into, and that same
choice inverts coverage of network-dependent code.
This commit was merged in pull request #256.
This commit is contained in:
2026-08-12 12:20:14 +02:00
parent 918e581ef3
commit 52c7c1b060
7 changed files with 993 additions and 16 deletions

33
TODO.md
View File

@@ -30,9 +30,10 @@ compiled off.
The backlog lives on the
[Gitea tracker](https://git.eeqj.de/sneak/AutistMask/issues), which is
authoritative; this file does not duplicate it. Full policy file set present. A
real-browser end-to-end suite (`make test-e2e`) now sits alongside `make check`,
which cannot see a runtime `ReferenceError` in a popup view.
authoritative; this file does not duplicate it. Full policy file set present.
Real-browser end-to-end suites (`make test-e2e` for Chrome,
`make test-e2e-firefox` for Firefox) now sit alongside `make check`, which
cannot see a runtime `ReferenceError` in a popup view.
# Next Step
@@ -44,6 +45,24 @@ undefined identifiers, which is how
# Completed Steps
- 2026-08-12: A containerized Firefox end-to-end harness
(`make test-e2e-firefox`) drives the real popup in a real Firefox with the MV2
build installed as a temporary add-on. Zero npm dependencies — a WebDriver
client over `fetch` against geckodriver — with `node`, Firefox 153.0.3 and
geckodriver 0.36.0 all pinned by digest. Uncaught errors are read from the
privileged console service in Marionette's chrome context, because BiDi
`log.entryAdded` reports nothing at all for extension pages; each drain reads
and clears the console in one chrome round trip, so no error is destroyed
unread by the drain itself, and errors logged during add-on install and
background startup are folded into step 1 instead of being cleared. The two
measured limits are documented rather than claimed away: the console ring
buffer holds 250 messages (a clean run peaks at 4), and the drained window
ends ≈1.5s after the last step returns. Demonstrated discriminating by exiting
1 on a `throw` at the top of `src/background/index.js`, on a build with one
import removed, on a `setTimeout` throw whose UI assertions all pass, on an
unhandled `Promise.reject` and on an undefined identifier in `home.js`, and 0
on the branch as it stands
([#184](https://git.eeqj.de/sneak/AutistMask/issues/184)).
- 2026-08-12: The transaction a dApp asks for is now populated in the background
before the approval window opens, so the object the user is shown is the
object the signed artifact is verified against — nonce, gas limit and every
@@ -279,9 +298,9 @@ tracker.
- Pre-1.0 security review of the extension (key handling, DEBUG mode policy, RPC
input validation) before any 1.0rc tag. Individual filed issues are parts of
it, but the review is broader than any of them.
- Decide whether docker-in-docker makes `make test-e2e` runnable in the Gitea
workflow. Extending the suite itself is tracked as
[#183](https://git.eeqj.de/sneak/AutistMask/issues/183) and
[#184](https://git.eeqj.de/sneak/AutistMask/issues/184).
- Decide whether docker-in-docker makes `make test-e2e` and
`make test-e2e-firefox` runnable in the Gitea workflow. Extending the Chrome
suite itself is tracked as
[#183](https://git.eeqj.de/sneak/AutistMask/issues/183).
- Cut 1.0.0 once the milestone is empty, then continue tagging as milestones
land.