defect hunt: use the extension as a user would, in headless Chrome and Firefox, on test networks #506

Closed
opened 2026-10-14 03:59:38 +02:00 by clawbot · 1 comment
Collaborator

Before the release candidate, use AutistMask as a person would, end to end, in real browsers, and find what is broken. The browser suites stub the network; this run does not. Test networks only: never mainnet funds, never a real key.

How

  • A release build (make build), loaded unpacked into headless Chrome and Firefox in containers (the same images make test-e2e and make test-e2e-firefox use are a fine base), driven by a script.
  • Value flows (send, eth_sendTransaction, a swap-shaped call if practical) against a local test chain started with Sepolia's chain id and pre-funded test accounts, set as the RPC endpoint in Settings. Read flows (balances, history, token list) against public Sepolia endpoints as well.
  • A small local test page acting as the site: connect, personal_sign, eth_sign, eth_signTypedData_v4, eth_sendTransaction, wallet_switchEthereumChain, each approved once and rejected once.

Flows

Every screen in README.md's screen map that a user reaches: create a wallet, back up and show the recovery phrase, import from phrase, private key and xprv, add and delete addresses, receive, send ETH and an ERC-20, add a token, the Settings changes (network, RPC, Blockscout, thresholds), lock, close and reopen the popup mid-flow, delete a wallet, the lost-password route, and every approval screen above.

Definition of done

  • Each real defect is filed as its own issue: steps, what happened, what should have happened (citing README.md where it says), a severity (critical, major, minor), and a definition of done. Behaviour README.md documents as deliberate is not a defect.
  • One closing comment on this issue linking the issues filed, and naming any flow that could not be run and why.
  • No code changes and no PR; every container started is removed.

Model: opus-5-5

Before the release candidate, use AutistMask as a person would, end to end, in real browsers, and find what is broken. The browser suites stub the network; this run does not. Test networks only: never mainnet funds, never a real key. ## How - A release build (`make build`), loaded unpacked into headless Chrome and Firefox in containers (the same images `make test-e2e` and `make test-e2e-firefox` use are a fine base), driven by a script. - Value flows (send, `eth_sendTransaction`, a swap-shaped call if practical) against a local test chain started with Sepolia's chain id and pre-funded test accounts, set as the RPC endpoint in Settings. Read flows (balances, history, token list) against public Sepolia endpoints as well. - A small local test page acting as the site: connect, `personal_sign`, `eth_sign`, `eth_signTypedData_v4`, `eth_sendTransaction`, `wallet_switchEthereumChain`, each approved once and rejected once. ## Flows Every screen in `README.md`'s screen map that a user reaches: create a wallet, back up and show the recovery phrase, import from phrase, private key and `xprv`, add and delete addresses, receive, send ETH and an ERC-20, add a token, the Settings changes (network, RPC, Blockscout, thresholds), lock, close and reopen the popup mid-flow, delete a wallet, the lost-password route, and every approval screen above. ## Definition of done - [ ] Each real defect is filed as its own issue: steps, what happened, what should have happened (citing `README.md` where it says), a severity (critical, major, minor), and a definition of done. Behaviour `README.md` documents as deliberate is not a defect. - [ ] One closing comment on this issue linking the issues filed, and naming any flow that could not be run and why. - [ ] No code changes and no PR; every container started is removed. Model: opus-5-5
clawbot self-assigned this 2026-10-14 03:59:38 +02:00
Author
Collaborator

Filed from this run: release build of next at 8e52528, Chrome and Firefox 153 in containers, a local chain with chain id 11155111, and the public Sepolia endpoints.

Not run, or run only in part:

  • Lock: the wallet has no lock screen. Each action that needs a secret asks for the password.
  • Read flows on public Sepolia ran only on addresses generated for this run, which have no history or tokens there. Public Blockscout also answered HTTP 429 to the popup's refreshes. Token balances, token sends and history ran on the local chain, with a small local stand-in answering the Blockscout endpoints the wallet reads.
  • A swap through a real Universal Router: the local chain has none, so only the approval screen for swap-shaped calldata was driven.
  • The toolbar popup itself, the clipboard and the content of the QR code cannot be observed in a headless browser.

Model: opus-5-5

Filed from this run: release build of `next` at 8e52528, Chrome and Firefox 153 in containers, a local chain with chain id 11155111, and the public Sepolia endpoints. - https://git.eeqj.de/sneak/AutistMask/issues/510 (critical) - https://git.eeqj.de/sneak/AutistMask/issues/515 (major) - https://git.eeqj.de/sneak/AutistMask/issues/517 (major) - https://git.eeqj.de/sneak/AutistMask/issues/519 (major) - https://git.eeqj.de/sneak/AutistMask/issues/520 (major) - https://git.eeqj.de/sneak/AutistMask/issues/525 (major) - https://git.eeqj.de/sneak/AutistMask/issues/534 (major) - https://git.eeqj.de/sneak/AutistMask/issues/511 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/512 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/513 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/516 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/518 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/522 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/523 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/528 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/529 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/530 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/532 (minor) - https://git.eeqj.de/sneak/AutistMask/issues/533 (minor) Not run, or run only in part: - Lock: the wallet has no lock screen. Each action that needs a secret asks for the password. - Read flows on public Sepolia ran only on addresses generated for this run, which have no history or tokens there. Public Blockscout also answered HTTP 429 to the popup's refreshes. Token balances, token sends and history ran on the local chain, with a small local stand-in answering the Blockscout endpoints the wallet reads. - A swap through a real Universal Router: the local chain has none, so only the approval screen for swap-shaped calldata was driven. - The toolbar popup itself, the clipboard and the content of the QR code cannot be observed in a headless browser. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/AutistMask#506