release: package the extension, pin the Chrome extension id, and prove the wallet survives a reinstall (closes #310)

There was no packaging target anywhere, no artifact, and no `key` in
`manifest/chrome.json` — so an unpacked Chrome load derived its extension
id, and therefore its `chrome.storage.local` partition, from the absolute
checkout path. Moving or re-cloning the checkout presented an empty
wallet, with no error and nothing in the UI to say so.

`manifest/chrome.json` now carries a fixed `key`: the public half of an
RSA keypair, which pins the extension id to
`gipbhkogfopeahplcjhipkgpcimdpkip`. The private half is a credential and
is not in this repo; no target generates one into the working tree, and
`tests/extensionId.test.js` fails if a `.pem` is ever committed. Changing
`key` changes the id and orphans every wallet stored under the old one.

`make package` (script/package) runs `make build` — the only audited path
to a release build — and writes one self-contained, versioned archive per
browser into `release/`, plus `SHA256SUMS`. The archives are deterministic:
entries sorted, timestamps fixed, compression level fixed, so two builds
of one commit are byte-identical. Self-containment is checked rather than
assumed: every path the manifests and the popup HTML reference is resolved
and required to be inside the archive, a reference that climbs out of the
extension root is a hard failure, and files left at the `dist/` root —
`dist/styles.css`, which build.js copies into each browser directory — are
reported as deliberately not shipped rather than dropped by a glob. The
archive is then read back off disk and compared member by member against
the directory it was built from. The zip writer and reader are stdlib zlib
in `script/lib/zip.js`; no new dependency, and nothing unpinned.

One version, enforced rather than generated. `script/lib/version.js`
requires `package.json`, `manifest/chrome.json` and `manifest/firefox.json`
to agree and fails the build naming each file and what it said, instead of
reading from one of the three. `BUILD_COMMIT` now carries `-dirty` when the
working tree does not match `HEAD`, and `-unknown` when git cannot say;
the full hash behind the About screen's commit link stays clean so the link
still resolves.

Two real-browser observations, both run through the pinned harnesses:

- `tests/e2e/storagePartition.js` loads the build from two different paths
  in one Chrome profile. With `key`: same id, and the second load reads the
  first load's storage. Without `key`: different ids, and the second load
  sees an empty partition. Loading both keyed copies at once yields one id,
  not two.
- `tests/e2e/firefox/reinstall.js` installs the packaged XPI in a real
  Firefox, creates a wallet, quits the browser, restarts on the same
  profile, adds the add-on again, and decrypts the vault back to the
  original recovery phrase. It then observes that an explicit uninstall
  DESTROYS that storage — correct browser behaviour, but for a wallet it
  means Remove is irreversible except from the recovery phrase, so
  README.md says so.

Firefox ships an UNSIGNED XPI. README.md states plainly that release
Firefox and ESR will refuse it, that Developer Edition, Nightly or an
Unbranded build is required, and that a temporary add-on does not survive
a browser restart. AMO signing, CRX packing, tagging and any upload are
deliberately out of scope.

tests/e2e/firefox/Dockerfile

@@ -72,6 +72,11 @@ RUN script/bootstrap
 
 COPY . .
 
-RUN make build
+# make package builds (and verifies) dist/ and then writes the release
+# artifacts, so the image carries both: run.js installs the unpacked
+# dist/firefox and reinstall.js installs the packaged .xpi, which is how the
+# artifact that would actually be handed to someone gets exercised in a real
+# Firefox rather than only being produced.
+RUN make package
 
 CMD ["node", "tests/e2e/firefox/run.js", "dist/firefox"]

tests/e2e/firefox/driver.js

@@ -103,7 +103,13 @@ class Driver {
 
     // ------------------------------------------------------------ setup
 
-    async newSession() {
+    // `profileDir` reuses an existing profile directory in place instead of
+    // letting geckodriver make a throwaway one. That is the only way to ask
+    // what survives a browser RESTART, which for a Firefox add-on that can
+    // only be installed temporarily is the question that decides whether the
+    // extension is usable at all: a temporary add-on is unloaded when Firefox
+    // exits, so every session begins by adding it again.
+    async newSession(profileDir) {
         const prefs = {
             // See EXTENSION_UUID above. The pref is a string pref whose
             // value is itself JSON.
@@ -132,26 +138,27 @@ class Driver {
             "extensions.openPopupWithoutUserGesture.enabled": false,
         };
 
+        const args = [
+            "-headless",
+            // Mandatory on Firefox 153: without it, navigating to
+            // moz-extension:// and running chrome-context script both
+            // fail with "unsupported operation".
+            //
+            // It grants the driver FULL CHROME PRIVILEGES over this
+            // browser. Acceptable only because the browser is a
+            // throwaway in a CI container; never point a session with
+            // this flag at anything you care about.
+            "-remote-allow-system-access",
+        ];
+        if (profileDir) args.push("-profile", profileDir);
+
         const value = await this.send("POST", "/session", {
             capabilities: {
                 alwaysMatch: {
                     browserName: "firefox",
                     "moz:firefoxOptions": {
                         binary: FIREFOX_BIN,
-                        args: [
-                            "-headless",
-                            // Mandatory on Firefox 153: without it,
-                            // navigating to moz-extension:// and running
-                            // chrome-context script both fail with
-                            // "unsupported operation".
-                            //
-                            // It grants the driver FULL CHROME PRIVILEGES
-                            // over this browser. Acceptable only because
-                            // the browser is a throwaway in a CI
-                            // container; never point a session with this
-                            // flag at anything you care about.
-                            "-remote-allow-system-access",
-                        ],
+                        args,
                         prefs,
                     },
                 },
@@ -162,16 +169,30 @@ class Driver {
         return value;
     }
 
-    // Installs the unpacked MV2 build straight from a directory.
-    // temporary:true bypasses signature checks, so no XPI and no signing
-    // are involved, and the add-on dies with the profile.
-    async installAddon(dir) {
+    // Installs the MV2 build, either from an unpacked directory or from an
+    // XPI file. temporary:true bypasses signature checks — which is the only
+    // way an UNSIGNED xpi installs at all, and the reason README.md says
+    // release Firefox will refuse the artifact this repo produces — and the
+    // add-on dies with the profile.
+    //
+    // Returns the add-on id Firefox assigned, which is
+    // browser_specific_settings.gecko.id from the manifest and is what
+    // uninstallAddon() takes.
+    async installAddon(pathToAddon) {
         return this.session("POST", "/moz/addon/install", {
-            path: dir,
+            path: pathToAddon,
             temporary: true,
         });
     }
 
+    // Removes an installed add-on, the way clicking Remove in about:addons
+    // does. tests/e2e/firefox/reinstall.js uses it to ask the one question
+    // that decides whether this extension can be used at all on Firefox: does
+    // the vault survive being removed and added again.
+    async uninstallAddon(id) {
+        return this.session("POST", "/moz/addon/uninstall", { id });
+    }
+
     // Classic navigation on purpose. BiDi's browsingContext.navigate
     // refuses moz-extension:// URLs outright.
     async navigate(url) {

tests/e2e/firefox/reinstall.js

@@ -0,0 +1,438 @@
+// Does the wallet survive being installed again on Firefox?
+//
+// This is the question behind
+// https://git.eeqj.de/sneak/AutistMask/issues/310, and it is the one property
+// that has to hold before real money goes into this extension. The only route
+// that works on release Firefox is a TEMPORARY add-on, which is unloaded when
+// the browser exits: daily use means adding it again from about:debugging on
+// every browser start. If extension storage did not survive that, every start
+// would present an empty wallet and the recovery phrase would be the only copy
+// of the money.
+//
+// manifest/firefox.json declares a fixed browser_specific_settings.gecko.id,
+// which is the right SHAPE for storage to survive — Firefox keys the storage
+// area by add-on id — but shape is not observation, and nothing asserted it.
+//
+// Two different things are asked here, because they have different answers and
+// conflating them would be the whole mistake:
+//
+//   RESTART      one profile, two browser runs, the add-on added temporarily
+//                in each. This is what a user does every day, and the vault
+//                has to survive it.
+//   REMOVAL      an explicit uninstall, the way about:addons "Remove" works,
+//                inside one browser run. Firefox destroys an add-on's storage
+//                when it is uninstalled, and the observed result is recorded
+//                here rather than wished away — for a wallet it means Remove
+//                is irreversible except from the recovery phrase.
+//
+// The vault is not merely compared as bytes in the restart case. It is
+// DECRYPTED with the original password through the real Show Recovery Phrase
+// screen and the phrase is compared against the one wallet creation produced,
+// because "the ciphertext is still in storage" and "the wallet still works"
+// are different claims and only the second one is worth anything.
+//
+// Run through script/test-e2e-firefox, which builds the artifact and the
+// pinned container. The one argument is what to install — an unpacked
+// directory or an .xpi. With none, the packaged XPI in release/ is used, so
+// this doubles as the check that the release artifact installs in a real
+// Firefox.
+//
+//   node tests/e2e/firefox/reinstall.js release/autistmask-firefox-0.1.0.xpi
+//
+// A separate program from run.js rather than another step in it: every step
+// there shares one browser session with one installed add-on, and this needs
+// two browsers and three installs.
+
+"use strict";
+
+const fs = require("fs");
+const os = require("os");
+const path = require("path");
+
+const { EXTENSION_ID, EXTENSION_UUID, start } = require("./driver");
+
+const REPO_ROOT = path.resolve(__dirname, "..", "..", "..");
+const PASSWORD = "e2e-harness-password";
+
+const STEP_TIMEOUT_MS = 120000;
+
+const checks = [];
+let failed = 0;
+
+function check(name, cond, detail) {
+    checks.push(name);
+    if (cond) {
+        console.log("ok " + checks.length + " - " + name);
+    } else {
+        failed += 1;
+        console.log("not ok " + checks.length + " - " + name);
+        if (detail) console.log("  " + detail);
+    }
+}
+
+// The moz-extension:// uuid Firefox currently serves this add-on from, read
+// out of the pref that holds the mapping. Privileged scope, because that pref
+// is not reachable from content.
+//
+// It has to be read after every install and never assumed. driver.js pins a
+// uuid through extensions.webextensions.uuids at each session start, but an
+// uninstall inside a running session drops that mapping and the next install
+// mints a fresh one — and navigating to the stale origin does not fail, it
+// HANGS until the session times out, which is how this was found.
+async function extensionUuid(d) {
+    const raw = await d.executeChrome(
+        `return Services.prefs.getStringPref(
+             "extensions.webextensions.uuids", "{}");`,
+    );
+    let map;
+    try {
+        map = JSON.parse(raw);
+    } catch (e) {
+        throw new Error(
+            "extensions.webextensions.uuids is not JSON (" +
+                e.message +
+                "): " +
+                raw,
+        );
+    }
+    return map[EXTENSION_ID] || null;
+}
+
+async function openPopup(d) {
+    const uuid = await extensionUuid(d);
+    if (!uuid) {
+        throw new Error(
+            "no uuid for " +
+                EXTENSION_ID +
+                " in extensions.webextensions.uuids, so the popup has no " +
+                "origin to be served from",
+        );
+    }
+    await d.navigate("moz-extension://" + uuid + "/src/popup/index.html");
+    // Whichever screen it lands on, wait for one of the two it can land on.
+    // Waiting for #view-main directly would time out rather than say what
+    // happened, and an empty storage partition is exactly the case where it
+    // lands on the other one.
+    await d.waitFor(
+        "the popup to finish restoring",
+        `const w = document.getElementById("view-welcome");
+         const m = document.getElementById("view-main");
+         if (!w || !m) return false;
+         const shown = (el) => {
+             const r = el.getBoundingClientRect();
+             return r.width > 0 && r.height > 0;
+         };
+         return shown(w) || shown(m);`,
+        [],
+        STEP_TIMEOUT_MS,
+    );
+    return uuid;
+}
+
+// The persisted record, straight out of extension storage, read from the popup
+// page — the one moz-extension:// document this program opens and therefore
+// the only place the storage API is reachable from.
+async function readVault(d) {
+    const outcome = await d.executeAsync(
+        `const done = arguments[arguments.length - 1];
+         const api = typeof browser !== "undefined" ? browser : chrome;
+         Promise.resolve(api.storage.local.get("autistmask"))
+             .then((r) => {
+                 const s = r.autistmask;
+                 if (!s) return done({ present: false });
+                 const w = (s.wallets || [])[0];
+                 if (!w) return done({ present: false });
+                 done({
+                     present: true,
+                     wallets: s.wallets.length,
+                     name: w.name,
+                     xpub: w.xpub,
+                     address: (w.addresses || []).map((a) => a.address)[0],
+                     vault: JSON.stringify(w.encryptedSecret),
+                 });
+             })
+             .catch((e) => done({ error: String((e && e.message) || e) }));`,
+    );
+    if (outcome && outcome.error) {
+        throw new Error("could not read extension storage: " + outcome.error);
+    }
+    return outcome;
+}
+
+async function createWallet(d) {
+    await d.click("#btn-welcome-add");
+    await d.waitVisible("#view-add-wallet");
+    await d.click("#btn-generate-phrase");
+    await d.waitFor(
+        "a generated recovery phrase of at least 12 words",
+        `const el = document.getElementById("wallet-mnemonic");
+         return !!el && el.value.trim().split(/\\s+/).length >= 12;`,
+    );
+    const phrase = (await d.value("#wallet-mnemonic")).trim();
+    await d.fill("#add-wallet-password", PASSWORD);
+    await d.fill("#add-wallet-password-confirm", PASSWORD);
+    await d.click("#btn-add-wallet-confirm");
+    // Argon2id under libsodium, for real.
+    await d.waitVisible("#view-main", STEP_TIMEOUT_MS);
+    return phrase;
+}
+
+// The phrase the vault decrypts to, obtained the way a user would: Settings,
+// Show Recovery Phrase, the original password.
+async function revealPhrase(d) {
+    if (!(await d.isVisible("#view-settings"))) {
+        await d.click("#btn-settings");
+    }
+    await d.waitVisible("#view-settings");
+    await d.click("#settings-wallet-list .btn-show-phrase");
+    await d.waitVisible("#view-show-phrase");
+    await d.fill("#show-phrase-password", PASSWORD);
+    await d.click("#btn-show-phrase-reveal");
+    await d.waitVisible("#show-phrase-result", STEP_TIMEOUT_MS);
+    return (await d.text("#show-phrase-value")).trim();
+}
+
+// What to install. An explicit argument wins; with none, the packaged XPI in
+// release/ is used, and there is deliberately no fallback to dist/firefox: the
+// point of running this against the artifact is that the artifact is what gets
+// installed, and quietly testing something else instead would leave the XPI
+// unexercised while the run stayed green.
+function resolveArtifact() {
+    if (process.argv[2]) return path.resolve(REPO_ROOT, process.argv[2]);
+
+    const releaseDir = path.join(REPO_ROOT, "release");
+    const xpis = fs.existsSync(releaseDir)
+        ? fs.readdirSync(releaseDir).filter((f) => f.endsWith(".xpi"))
+        : [];
+    if (xpis.length !== 1) {
+        throw new Error(
+            "expected exactly one .xpi in release/, found " +
+                xpis.length +
+                " (" +
+                xpis.join(", ") +
+                "). Run make package, or name the artifact as the argument.",
+        );
+    }
+    return path.join(releaseDir, xpis[0]);
+}
+
+async function main() {
+    let artifact;
+    try {
+        artifact = resolveArtifact();
+    } catch (e) {
+        console.error("e2e-firefox-reinstall: " + e.message);
+        process.exitCode = 1;
+        return;
+    }
+    if (!fs.existsSync(artifact)) {
+        console.error(
+            "e2e-firefox-reinstall: nothing to install at " + artifact,
+        );
+        process.exitCode = 1;
+        return;
+    }
+    console.log("# installing: " + artifact);
+
+    // One profile, reused by both browser runs below. geckodriver would
+    // otherwise make a throwaway one per session, and "survives a restart"
+    // cannot be asked of a profile that does not.
+    const profile = fs.mkdtempSync(
+        path.join(os.tmpdir(), "autistmask-reinstall-profile-"),
+    );
+
+    let phrase = null;
+    let before = null;
+    let first;
+
+    // --- run one: install, create a wallet, close the browser --------------
+    try {
+        first = await start();
+        await first.newSession(profile);
+    } catch (e) {
+        // A browser we cannot start is a failure of this program, never an
+        // absent one.
+        console.error("e2e-firefox-reinstall: cannot run: " + e.message);
+        if (first) await first.quit().catch(() => {});
+        process.exitCode = 1;
+        return;
+    }
+
+    try {
+        const firstId = await first.installAddon(artifact);
+        check(
+            "the artifact installs and reports the manifest's gecko id",
+            firstId === EXTENSION_ID,
+            "installed add-on id is " +
+                JSON.stringify(firstId) +
+                ", expected " +
+                JSON.stringify(EXTENSION_ID) +
+                ". Without a stable id Firefox has no key to hang the " +
+                "storage area on, and nothing below can hold.",
+        );
+
+        const firstUuid = await openPopup(first);
+        await first.waitVisible("#view-welcome", STEP_TIMEOUT_MS);
+        phrase = await createWallet(first);
+        before = await readVault(first);
+        check(
+            "a wallet created through the UI is in extension storage",
+            before.present && before.wallets === 1 && !!before.vault,
+            JSON.stringify(before),
+        );
+        console.log(
+            "# run 1 moz-extension uuid: " +
+                firstUuid +
+                (firstUuid === EXTENSION_UUID ? " (the pinned one)" : ""),
+        );
+    } catch (e) {
+        failed += 1;
+        console.log("# ERROR in run 1: " + (e && e.stack ? e.stack : e));
+    } finally {
+        await first.quit().catch(() => {});
+    }
+
+    // --- run two: same profile, add-on added again -------------------------
+    //
+    // This is the restart. The temporary add-on died with the previous
+    // browser; the profile, and whatever Firefox kept in it, did not.
+    let second;
+    try {
+        second = await start();
+        await second.newSession(profile);
+    } catch (e) {
+        console.error(
+            "e2e-firefox-reinstall: cannot restart the browser: " + e.message,
+        );
+        if (second) await second.quit().catch(() => {});
+        process.exitCode = 1;
+        return;
+    }
+
+    try {
+        const secondId = await second.installAddon(artifact);
+        check(
+            "the add-on installs again after a browser restart with the " +
+                "same id",
+            secondId === EXTENSION_ID,
+            "re-installed id is " + JSON.stringify(secondId),
+        );
+
+        const secondUuid = await openPopup(second);
+        // The origin the popup is served from is not the thing that carries
+        // the wallet, and the uuid is read live for exactly that reason. In
+        // this run it comes back as the pinned one, because driver.js writes
+        // extensions.webextensions.uuids into the profile at every session
+        // start; an uninstall inside a running session drops the mapping and
+        // the next install mints a fresh uuid instead. Either way the storage
+        // area is keyed on the add-on id, never on this.
+        console.log(
+            "# run 2 moz-extension uuid: " +
+                secondUuid +
+                (secondUuid === EXTENSION_UUID
+                    ? " (the pinned one, re-applied at session start)"
+                    : " (freshly minted)"),
+        );
+
+        const onWelcome = await second.isVisible("#view-welcome");
+        check(
+            "after a restart and re-add, the extension does not come up as " +
+                "a fresh install",
+            !onWelcome,
+            "the popup shows the welcome screen, which is what an empty " +
+                "storage partition looks like: the wallet is gone and only " +
+                "the recovery phrase would get it back.",
+        );
+
+        const after = await readVault(second);
+        check(
+            "the vault, xpub and first address survive the restart unchanged",
+            after.present &&
+                after.wallets === before.wallets &&
+                after.vault === before.vault &&
+                after.xpub === before.xpub &&
+                after.address === before.address,
+            "before: " +
+                JSON.stringify(before) +
+                " after: " +
+                JSON.stringify(after),
+        );
+
+        if (after.present) {
+            const revealed = await revealPhrase(second);
+            check(
+                "the vault still decrypts with the original password to the " +
+                    "original recovery phrase",
+                revealed === phrase,
+                "the recovery phrase read back after the restart is not the " +
+                    "one the wallet was created with",
+            );
+        } else {
+            check(
+                "the vault still decrypts with the original password to the " +
+                    "original recovery phrase",
+                false,
+                "there was no vault left to decrypt",
+            );
+        }
+
+        // --- an explicit removal, in the same browser run ------------------
+        //
+        // Leave the popup FIRST. Removing the add-on destroys every document
+        // it serves, and the popup is this session's only window: uninstalling
+        // while it is on screen discards the browsing context, and every
+        // subsequent WebDriver command fails with "no such window" rather than
+        // with anything about the add-on. Observed, not anticipated.
+        await second.navigate("about:blank");
+        await second.uninstallAddon(EXTENSION_ID);
+        await second.installAddon(artifact);
+        await openPopup(second);
+
+        const afterRemoval = await readVault(second);
+        console.log(
+            "# after an explicit uninstall: " + JSON.stringify(afterRemoval),
+        );
+        // Recorded as observed behaviour, not as something this repo wants.
+        // Firefox destroys an add-on's storage when it is uninstalled, and
+        // that is correct of a browser — it is stated here, and in README.md,
+        // because for a WALLET it means about:addons "Remove" is irreversible
+        // except from the recovery phrase. If a Firefox ever stops doing it
+        // this fails and the claim gets rewritten from a new observation.
+        check(
+            "an explicit uninstall DESTROYS the vault (observed Firefox " +
+                "behaviour: Remove is irreversible, unlike a restart)",
+            afterRemoval.present === false,
+            "the vault survived an explicit uninstall: " +
+                JSON.stringify(afterRemoval),
+        );
+    } catch (e) {
+        failed += 1;
+        console.log("# ERROR in run 2: " + (e && e.stack ? e.stack : e));
+    } finally {
+        await second.quit().catch(() => {});
+        fs.rmSync(profile, { recursive: true, force: true });
+    }
+
+    console.log("1.." + checks.length);
+    if (checks.length === 0) {
+        console.log("# FAILED: this program asserted nothing");
+        process.exitCode = 1;
+        return;
+    }
+    console.log(
+        "# " +
+            (checks.length - failed) +
+            "/" +
+            checks.length +
+            " checks passed",
+    );
+    if (failed > 0) {
+        console.log("# FAILED");
+        process.exitCode = 1;
+    }
+}
+
+main().catch((e) => {
+    console.error("e2e-firefox-reinstall: " + (e && e.stack ? e.stack : e));
+    process.exitCode = 1;
+});