test: containerized Firefox end-to-end harness (closes #184)
All checks were successful
check / check (push) Successful in 34s
All checks were successful
check / check (push) Successful in 34s
Drives the real popup in a real Firefox with dist/firefox/ installed as an unpacked MV2 temporary add-on via geckodriver. make test-e2e-firefox, outside make check like the Chrome suite. Zero npm dependencies: plain fetch and child_process against geckodriver's HTTP API. Base image, Firefox tarball and geckodriver are each pinned by digest and verified at build time. Error capture reads the privileged console service through Marionette's chrome context, not WebDriver BiDi. BiDi delivers nothing at all for extension pages, so a BiDi-based harness would observe zero events and report success -- the vacuous-check shape this repo has shipped twice. Both the driver and the README say so where someone would be tempted to simplify. Demonstrated to discriminate: a background page that throws at the top of the file, a missing import, and an async throw where every UI assertion still passes each fail the run. Three limits are measured and documented rather than papered over: capture is poll-based so an error is attributed to a step, not a moment; the console ring buffer holds 250 messages and evicts the oldest, measured against a clean-run peak of 4; and the drained window ends roughly 1.5s after the last step, with observed jitter rather than a hard boundary. Content-script capture is marked unverified because --network none leaves no page to inject into, and that same choice inverts coverage of network-dependent code.
This commit was merged in pull request #256.
This commit is contained in:
63
script/test-e2e-firefox
Executable file
63
script/test-e2e-firefox
Executable file
@@ -0,0 +1,63 @@
|
||||
#!/bin/sh
|
||||
# script/test-e2e-firefox: build the extension and drive the real popup in
|
||||
# a real Firefox inside a pinned container. The Firefox counterpart to
|
||||
# script/test-e2e. Our own extension to scripts-to-rule-them-all.
|
||||
#
|
||||
# Deliberately NOT called by script/check or script/test, for the same
|
||||
# reason as the Chrome suite: REPO_POLICIES.md caps make test at 20 seconds
|
||||
# and a browser suite does not fit.
|
||||
#
|
||||
# Unlike script/test-e2e this builds its image locally, because no
|
||||
# published image carries both a pinned Firefox and a matching geckodriver.
|
||||
# All three external artifacts are pinned by digest inside the Dockerfile;
|
||||
# see tests/e2e/firefox/Dockerfile.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
IMAGE="$("$SCRIPT_DIR/projectname")-e2e-firefox"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
echo "test-e2e-firefox: docker is required to run the e2e suite" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Building extension for e2e..."
|
||||
yarn run build 2>&1
|
||||
|
||||
# The build context is tests/e2e/firefox/ and holds nothing but the
|
||||
# Dockerfile: the harness itself arrives over the bind mount below, so
|
||||
# editing it never invalidates an image layer.
|
||||
echo "Building the pinned Firefox e2e image..."
|
||||
docker build -t "$IMAGE" "$ROOT/tests/e2e/firefox"
|
||||
|
||||
echo "Running the Firefox e2e suite..."
|
||||
# --shm-size=1g: Firefox needs more than the default 64MB /dev/shm.
|
||||
# --network none: the suite stubs nothing, so this is what keeps the
|
||||
# run offline and deterministic. The extension swallows its own
|
||||
# fetch failures, so the popup flows work unchanged; see the
|
||||
# network note in README.md. Weaker than the Chrome suite's
|
||||
# fixture interception, and honestly so — it proves no request
|
||||
# escaped, but it cannot report which ones were attempted.
|
||||
# --user: keep files the suite touches owned by the caller, not root.
|
||||
# HOME=/tmp: the mapped uid has no home directory in the image.
|
||||
#
|
||||
# No --privileged. Firefox's sandbox logs
|
||||
# "CanCreateUserNamespace() clone() failure: EPERM" on startup here;
|
||||
# it is cosmetic and headless Firefox runs fine without it.
|
||||
docker run --rm \
|
||||
--shm-size=1g \
|
||||
--network none \
|
||||
--user "$(id -u):$(id -g)" \
|
||||
-e HOME=/tmp \
|
||||
-v "$ROOT:/work" \
|
||||
-w /work \
|
||||
"$IMAGE" \
|
||||
node tests/e2e/firefox/run.js dist/firefox
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user