Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
189bd3f4ba |
@@ -339,9 +339,7 @@ There are two suites, one per browser, and they share no code. Chrome runs on
|
|||||||
Playwright; Firefox has its own WebDriver client, because Playwright cannot
|
Playwright; Firefox has its own WebDriver client, because Playwright cannot
|
||||||
observe errors on a Firefox extension page at all — see
|
observe errors on a Firefox extension page at all — see
|
||||||
[Firefox](#firefox-make-test-e2e-firefox) below. Both require docker, and both
|
[Firefox](#firefox-make-test-e2e-firefox) below. Both require docker, and both
|
||||||
are outside `make check`. Neither opens the popup from the toolbar button: both
|
are outside `make check`.
|
||||||
load its page in an ordinary tab, so what the toolbar popup itself adds, its
|
|
||||||
size and its closing when it loses focus, is covered by neither.
|
|
||||||
|
|
||||||
### Chrome (`make test-e2e`)
|
### Chrome (`make test-e2e`)
|
||||||
|
|
||||||
@@ -427,36 +425,14 @@ required to be present, so that check cannot pass by observing nothing. That
|
|||||||
last one is the standing floor under
|
last one is the standing floor under
|
||||||
[#157](https://git.eeqj.de/sneak/AutistMask/issues/157).
|
[#157](https://git.eeqj.de/sneak/AutistMask/issues/157).
|
||||||
|
|
||||||
The limits of that coverage and of the rest of the Chrome suite, none of them
|
Two limits of that coverage, neither of them papered over. The RPC is stubbed
|
||||||
papered over:
|
throughout, so this is **not** a real dApp against a real network with real
|
||||||
|
funds; that remains a human pass before 1.0.0. The site-connection prompt is
|
||||||
- The RPC is stubbed throughout, so this is **not** a real dApp against a real
|
raised through `chrome.action.openPopup()`, and headless Chromium's
|
||||||
network with real funds; that remains a human pass before 1.0.0.
|
browser-action popup is not a page Playwright can see or click, so that one
|
||||||
- The site-connection prompt is raised through `chrome.action.openPopup()`, and
|
prompt is driven at the URL the extension itself puts on the action — the same
|
||||||
headless Chromium's browser-action popup is not a page Playwright can see or
|
page and the same approval id, but whether a real toolbar click shows it is not
|
||||||
click, so that one prompt is driven at the URL the extension itself puts on
|
observable here.
|
||||||
the action — the same page and the same approval id, but whether a real
|
|
||||||
toolbar click shows it is not observable here.
|
|
||||||
- Each site-connection request is made 1.5 seconds after the tab it will be
|
|
||||||
driven in is opened (`APPROVAL_TAB_SETTLE_MS` in `tests/e2e/run.js`), because
|
|
||||||
opening that tab closes the previous prompt's toolbar popup; a request made
|
|
||||||
just after that popup closes is not covered.
|
|
||||||
- In the tests that approve a signature or a transaction, the approval window
|
|
||||||
runs with `chrome.runtime.sendMessage` wrapped to record what it sends, and in
|
|
||||||
the tests that reject a site connection the prompt gets a click listener that
|
|
||||||
records that Reject was pressed; neither changes what the window does.
|
|
||||||
- The tap to copy test first grants clipboard permission to every page in the
|
|
||||||
browser, which the manifest does not ask for, so whether a real popup may
|
|
||||||
write to the clipboard on a click alone is not covered.
|
|
||||||
- The layout tests for an over-long flash message and for the password error
|
|
||||||
lines write the text straight into the page instead of letting the popup's
|
|
||||||
code put it there, and the second brings each screen up by toggling its
|
|
||||||
`hidden` class rather than navigating to it; they cover the layout, not the
|
|
||||||
code that fills it.
|
|
||||||
- Leaving the recovery phrase or private key screen while its decrypt runs is
|
|
||||||
forced by clicking Reveal and the settings gear in one page task, which a
|
|
||||||
person cannot do; the case a person can hit, the first decrypt after the popup
|
|
||||||
opens while libsodium is still loading, is not driven.
|
|
||||||
|
|
||||||
Any test that drives a failure path on purpose declares the `console.error` it
|
Any test that drives a failure path on purpose declares the `console.error` it
|
||||||
is about to provoke, via `errors.expect()`. That is not a mute: the declaration
|
is about to provoke, via `errors.expect()`. That is not a mute: the declaration
|
||||||
@@ -470,8 +446,8 @@ collecting for a fixed grace period after the last test returns
|
|||||||
the context. A request whose _first_ dispatch falls after that window is never
|
the context. A request whose _first_ dispatch falls after that window is never
|
||||||
seen at all and cannot fail the run. In practice a request a test fires without
|
seen at all and cannot fail the run. In practice a request a test fires without
|
||||||
awaiting reaches the route handler about 10ms later, and anything on a repeating
|
awaiting reaches the route handler about 10ms later, and anything on a repeating
|
||||||
timer gets observed on an earlier tick during the suite — but a one-shot call
|
timer gets observed on an earlier tick during the ~20s suite — but a one-shot
|
||||||
deliberately deferred past the window will escape.
|
call deliberately deferred past the window will escape.
|
||||||
|
|
||||||
That interception covers the MV3 background service worker as well as the popup
|
That interception covers the MV3 background service worker as well as the popup
|
||||||
page, which it does not by default — `script/test-e2e` sets
|
page, which it does not by default — `script/test-e2e` sets
|
||||||
@@ -603,26 +579,25 @@ without an `await`. Demonstrated, not assumed: a `throw` placed past the first
|
|||||||
and on Chrome (`pageerror`), with the rest of the run unaffected because the
|
and on Chrome (`pageerror`), with the rest of the run unaffected because the
|
||||||
approval view had already rendered.
|
approval view had already rendered.
|
||||||
|
|
||||||
One error is still tolerated rather than fatal, listed in `ALLOWED_ERRORS` in
|
One error is tolerated rather than fatal, listed in `ALLOWED_ERRORS` in
|
||||||
`tests/e2e/firefox/run.js` and printed on every occurrence so the concession
|
`tests/e2e/firefox/run.js` with the issue that will delete it, and printed on
|
||||||
stays visible in the run output: Firefox reporting an extension promise that
|
every occurrence so the concession stays visible in the run output. It is
|
||||||
settled after its page unloaded, from anywhere in the popup. Its cause was the
|
Firefox reporting the site-approval popup's unawaited `sendMessage` settling
|
||||||
site-approval popup's unawaited `sendMessage` before `window.close()`, which
|
after `window.close()` unloaded the context — the same teardown ordering as
|
||||||
[#275](https://git.eeqj.de/sneak/AutistMask/issues/275) removed; Firefox runs
|
[#275](https://git.eeqj.de/sneak/AutistMask/issues/275), and unsuppressable from
|
||||||
since have not printed it, and
|
the calling code, because `BaseContext.wrapPromise` reports it whether or not a
|
||||||
[#487](https://git.eeqj.de/sneak/AutistMask/issues/487) removes the entry.
|
handler is attached. Errors are read from the privileged `nsIConsoleService` in
|
||||||
Errors are read from the privileged `nsIConsoleService` in Marionette's chrome
|
Marionette's chrome context and filtered to non-warning entries whose
|
||||||
context and filtered to non-warning entries whose `sourceName` is the extension
|
`sourceName` is the extension origin. That mechanism is not a stylistic choice.
|
||||||
origin. That mechanism is not a stylistic choice. WebDriver BiDi's
|
WebDriver BiDi's `log.entryAdded` delivers **nothing** for extension pages: on a
|
||||||
`log.entryAdded` delivers **nothing** for extension pages: on a plain `http://`
|
plain `http://` page it reports uncaught errors with stack traces, and on the
|
||||||
page it reports uncaught errors with stack traces, and on the `moz-extension://`
|
`moz-extension://` popup it reports zero events, because Firefox's remote agent
|
||||||
popup it reports zero events, because Firefox's remote agent excludes extension
|
excludes extension browsing contexts from BiDi observation. Any harness built on
|
||||||
browsing contexts from BiDi observation. Any harness built on Playwright-BiDi or
|
Playwright-BiDi or Puppeteer-BiDi would therefore see nothing and report
|
||||||
Puppeteer-BiDi would therefore see nothing and report success, which is exactly
|
success, which is exactly the vacuous check this repo has already shipped twice.
|
||||||
the vacuous check this repo has already shipped twice. Do not migrate this suite
|
Do not migrate this suite to BiDi.
|
||||||
to BiDi.
|
|
||||||
|
|
||||||
Three limits are worth knowing, all real differences from the Chrome suite:
|
Two limits are worth knowing, both real differences from the Chrome suite:
|
||||||
|
|
||||||
- **Error capture is poll-based, not event-streamed.** The console is drained at
|
- **Error capture is poll-based, not event-streamed.** The console is drained at
|
||||||
each step boundary, so an error is attributed to the step it was drained
|
each step boundary, so an error is attributed to the step it was drained
|
||||||
@@ -639,30 +614,24 @@ Three limits are worth knowing, all real differences from the Chrome suite:
|
|||||||
and silently evicts the oldest, so more than 250 console messages between two
|
and silently evicts the oldest, so more than 250 console messages between two
|
||||||
drains destroys the excess unread. 400 throws inside one step are reported as
|
drains destroys the excess unread. 400 throws inside one step are reported as
|
||||||
exactly the newest 250, three runs running. That buffer is shared with
|
exactly the newest 250, three runs running. That buffer is shared with
|
||||||
Firefox's own console noise; a clean run, measured when the suite had three
|
Firefox's own console noise; a clean run peaks at 4 of 250 at the install
|
||||||
steps (popup load, wallet creation and Add Token), peaked at 4 of 250 at the
|
drain and 0 at every later drain, so the three steps here have wide headroom,
|
||||||
install drain and 0 at every later drain, but a step that logs heavily could
|
but a step that logs heavily could evict unread errors. What poll-based costs
|
||||||
evict unread errors. What poll-based costs is location, not coverage: an error
|
is location, not coverage: an error cannot be placed within a step the way the
|
||||||
cannot be placed within a step the way the Chrome suite's `pageerror` events
|
Chrome suite's `pageerror` events place it.
|
||||||
place it.
|
|
||||||
- **Almost nothing is stubbed, which inverts the coverage of network-dependent
|
- **Almost nothing is stubbed, which inverts the coverage of network-dependent
|
||||||
code.** The container still runs with `--network none`, so the run is offline
|
code.** The container still runs with `--network none`, so the run is offline
|
||||||
and no request can escape. The one thing it can reach is the loopback fixture
|
and no request can escape. The one thing it can reach is the loopback fixture
|
||||||
in `tests/e2e/firefox/dapp.js`, which serves the dApp page and a JSON-RPC node
|
in `tests/e2e/firefox/dapp.js`, which serves the dApp page and a JSON-RPC node
|
||||||
and which the extension's `rpcUrl` is pointed at for the dApp steps; a
|
and which the extension's `rpcUrl` is pointed at for the dApp steps; a
|
||||||
JSON-RPC method that fixture does not model fails the run rather than
|
JSON-RPC method that fixture does not model fails the run rather than
|
||||||
answering `null`. Everything else, Blockscout and the price feed among it, has
|
answering `null`. Everything else — Blockscout, the price feed, the phishing
|
||||||
no fixture and simply fails, and the extension swallows its own fetch
|
blocklist — has no fixture and simply fails, and the extension swallows its
|
||||||
failures, so only the _failure_ branches of that code are ever executed. A
|
own fetch failures, so only the _failure_ branches of that code are ever
|
||||||
`ReferenceError` in the success path of `renderTransactions`, or of price
|
executed. A `ReferenceError` in the success path of `renderTransactions`, or
|
||||||
rendering, passes this suite green. The offline run is also weaker than the
|
of price rendering, passes this suite green. The offline run is also weaker
|
||||||
Chrome suite's interception for those calls: it proves nothing got out, but it
|
than the Chrome suite's interception for those calls: it proves nothing got
|
||||||
cannot report which requests were attempted.
|
out, but it cannot report which requests were attempted.
|
||||||
- **The site-connection prompt always opens in a window of its own.** The
|
|
||||||
profile turns off `extensions.openPopupWithoutUserGesture.enabled`, so
|
|
||||||
`src/background/index.js` falls back from the toolbar popup, which WebDriver
|
|
||||||
cannot see, to `windows.create()`; the toolbar popup path is not covered on
|
|
||||||
Firefox.
|
|
||||||
|
|
||||||
Neither `make test-e2e` nor `make test-e2e-firefox` is part of `make check` or
|
Neither `make test-e2e` nor `make test-e2e-firefox` is part of `make check` or
|
||||||
`make test`. `REPO_POLICIES.md` caps `make test` at 60 seconds and a browser
|
`make test`. `REPO_POLICIES.md` caps `make test` at 60 seconds and a browser
|
||||||
|
|||||||
@@ -55,20 +55,6 @@ but the review is broader than any of them.
|
|||||||
zlib that made the committed files did. `build.js` now copies each manifest
|
zlib that made the committed files did. `build.js` now copies each manifest
|
||||||
from the same path `copyIcons()` reads its icons from.
|
from the same path `copyIcons()` reads its icons from.
|
||||||
|
|
||||||
- 2026-10-07: The README's end-to-end limits now match what the two browser
|
|
||||||
suites do to the extension
|
|
||||||
([#293](https://git.eeqj.de/sneak/AutistMask/issues/293)). The `window.close`
|
|
||||||
override the issue named went with
|
|
||||||
[#275](https://git.eeqj.de/sneak/AutistMask/issues/275), so it needs no entry.
|
|
||||||
Added: both suites load the popup in a tab rather than from the toolbar;
|
|
||||||
Chrome waits 1.5 seconds before each site-connection request, records what
|
|
||||||
approval windows send and click, grants clipboard permission, writes text
|
|
||||||
straight into the page in two layout tests, and forces the leave during a
|
|
||||||
decrypt; Firefox forces the site-connection prompt into a window. Corrected:
|
|
||||||
Firefox's one tolerated error, whose cause that fix removed (the entry goes in
|
|
||||||
[#487](https://git.eeqj.de/sneak/AutistMask/issues/487)), the phishing
|
|
||||||
blocklist the extension no longer fetches, and two stale figures.
|
|
||||||
|
|
||||||
- 2026-10-07: Back from Settings no longer shows Settings again after the
|
- 2026-10-07: Back from Settings no longer shows Settings again after the
|
||||||
settings gear was pressed on the recovery phrase or a delete wallet screen
|
settings gear was pressed on the recovery phrase or a delete wallet screen
|
||||||
opened from Settings, with or without a reopen in between
|
opened from Settings, with or without a reopen in between
|
||||||
|
|||||||
Reference in New Issue
Block a user