feat: vendor and censor the phishing blocklist at build time (closes #219)
The blocklist URL in shipped code named a competitor and pointed at a moving ref, and the extension re-fetched from it every 24 hours, which also meant a third party decided what this wallet warns about. All of that is gone. script/vendor-blocklist fetches upstream at a pinned commit, verifies the sha256 of the bytes that commit serves, and writes src/shared/phishingBlocklist.json. It is build-time tooling, never shipped, and the one place in the repo that names the upstream project; a source reference nobody can verify is not a source reference. The artifact stores truncated sha256 digests rather than domain names. That is what censors it: the previous file contained the competitor's name 6,475 times, as phishing domains impersonating them, and not one of those domains is dropped. It also makes lookups a binary search over a fixed-width string, so nothing is built at module load — which matters on MV3, where the worker re-evaluates the module on every wake — and takes the file from 8.7 MB to 1.7 MB. script/check-censored enforces the rest: it reads the name out of the vendoring script rather than repeating it, and fails on any occurrence in the working tree or under dist/ that is not one of the three literals shipped code cannot avoid — two provider-shim identifiers in src/content/inpage.js and one ERC-20's on-chain name in src/shared/tokenList.js. Each is permitted only at the path that carries it, and at the emitted paths that path is bundled into, so a literal appearing anywhere else fails like any other occurrence. It runs in make check, which inspects dist/ when there is one and says loudly when there is not, and again with --require-dist at the end of every make build. Removing the runtime fetch retires the delta, the extension-storage persistence and the 24-hour alarm from #158. A retired alarm is now cleared rather than left waking the worker forever on installs that already have it. The e2e suite drives the warning end to end from a real blocklisted origin served as a real http(s) site, with a control asserting the banner stays hidden for one that is not listed. Its service-worker interception canary needed a new anchor, since the startup fetch it used to watch for no longer happens: it now wakes the worker with a message and asks it for one throwaway fetch. LICENSE no longer cites a repository that returns 404. eslint.config.js gains one block: script/lib/ holds node programs the shell entrypoints call, and without it they lint with no globals at all.
This commit is contained in:
147
script/lib/build-blocklist.js
Normal file
147
script/lib/build-blocklist.js
Normal file
@@ -0,0 +1,147 @@
|
||||
// The transform half of script/vendor-blocklist: upstream's config.json in,
|
||||
// src/shared/phishingBlocklist.json out. Build-time repo tooling; nothing here
|
||||
// is shipped to users.
|
||||
//
|
||||
// Usage: node script/lib/build-blocklist.js <source.json> <output.json>
|
||||
//
|
||||
// What it does, and why each step is here:
|
||||
//
|
||||
// - only the blacklist is carried over. The extension matches a hostname and
|
||||
// its parent domains against that one list; upstream's whitelist, fuzzylist
|
||||
// and version metadata are read by nothing here, so shipping them would add
|
||||
// megabytes of dead weight to every install.
|
||||
// - entries are lowercased and de-duplicated, because that is the form
|
||||
// isPhishingDomain() compares against.
|
||||
// - entries that cannot be a hostname are dropped and counted. Upstream
|
||||
// carries the odd URL-shaped entry (a path, a scheme); hostname matching can
|
||||
// never match one, and once the artifact is hashes nobody can see that it is
|
||||
// in there, so it is reported at vendoring time instead.
|
||||
// - entries are hashed (see src/shared/domainHash.js) and sorted, and the
|
||||
// digests are concatenated into one fixed-width string. Sorted is what makes
|
||||
// the runtime lookup a binary search over that string, with no set to build
|
||||
// on every service-worker wake; one string rather than an array of 100k+ is
|
||||
// what keeps the file, the bundle and the JSON parse small.
|
||||
//
|
||||
// Deterministic by construction: same input bytes, same output bytes.
|
||||
|
||||
"use strict";
|
||||
|
||||
const fs = require("fs");
|
||||
|
||||
const {
|
||||
HASH_ALGORITHM,
|
||||
HASH_HEX_CHARS,
|
||||
hashDomain,
|
||||
} = require("../../src/shared/domainHash");
|
||||
|
||||
// A blocklist that has collapsed to a handful of entries is a broken fetch or a
|
||||
// changed upstream shape, not a quiet day in phishing. Vendoring it would
|
||||
// disarm the feature, so it fails instead and a human decides.
|
||||
const MIN_ENTRIES = 10000;
|
||||
|
||||
function fail(message) {
|
||||
process.stderr.write("build-blocklist: " + message + "\n");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// A hostname, as the matcher understands one: dot-separated labels of letters,
|
||||
// digits, hyphens and underscores. Anything else — a path, a scheme, a space,
|
||||
// an empty string, a non-ASCII label a browser would have punycoded before it
|
||||
// ever reached isPhishingDomain() — cannot be produced by the hostname variants
|
||||
// the extension looks up, so it could only ever sit in the artifact unused.
|
||||
//
|
||||
// Underscores are deliberate. They are not legal in a hostname per RFC 1123,
|
||||
// but DNS carries them and browsers resolve them, and upstream lists 141 entries
|
||||
// that use one — real phishing sites on shared subdomain hosts. A stricter
|
||||
// pattern silently drops every one of them.
|
||||
const HOSTNAME_RE =
|
||||
/^[a-z0-9_]([a-z0-9_-]*[a-z0-9_])?(\.[a-z0-9_]([a-z0-9_-]*[a-z0-9_])?)+$/;
|
||||
|
||||
function main(argv) {
|
||||
const [source, output] = argv;
|
||||
if (!source || !output) {
|
||||
fail("usage: build-blocklist.js <source.json> <output.json>");
|
||||
}
|
||||
|
||||
let config;
|
||||
try {
|
||||
config = JSON.parse(fs.readFileSync(source, "utf8"));
|
||||
} catch (e) {
|
||||
fail("could not read " + source + " as JSON: " + e.message);
|
||||
}
|
||||
|
||||
if (!Array.isArray(config.blacklist)) {
|
||||
fail(
|
||||
"the source has no blacklist array, so its shape is not the one " +
|
||||
"this transform understands. Refusing to write an artifact.",
|
||||
);
|
||||
}
|
||||
|
||||
const seen = new Set();
|
||||
let dropped = 0;
|
||||
for (const raw of config.blacklist) {
|
||||
if (typeof raw !== "string") {
|
||||
dropped++;
|
||||
continue;
|
||||
}
|
||||
const domain = raw.trim().toLowerCase();
|
||||
if (!HOSTNAME_RE.test(domain)) {
|
||||
dropped++;
|
||||
continue;
|
||||
}
|
||||
seen.add(domain);
|
||||
}
|
||||
|
||||
if (seen.size < MIN_ENTRIES) {
|
||||
fail(
|
||||
"the source yielded " +
|
||||
seen.size +
|
||||
" usable entries, below the " +
|
||||
MIN_ENTRIES +
|
||||
" floor. That is a broken source or a changed upstream " +
|
||||
"shape, and vendoring it would disarm phishing detection. " +
|
||||
"Refusing to write an artifact.",
|
||||
);
|
||||
}
|
||||
|
||||
const hashes = [];
|
||||
for (const domain of seen) hashes.push(hashDomain(domain));
|
||||
hashes.sort();
|
||||
|
||||
// Truncation makes collisions possible; they are harmless (both entries are
|
||||
// blocked either way) but they must not inflate the count the artifact
|
||||
// claims, which the runtime cross-checks against the string length.
|
||||
const unique = [];
|
||||
for (const hash of hashes) {
|
||||
if (unique.length === 0 || unique[unique.length - 1] !== hash) {
|
||||
unique.push(hash);
|
||||
}
|
||||
}
|
||||
|
||||
const artifact = {
|
||||
algorithm: HASH_ALGORITHM,
|
||||
hashHexChars: HASH_HEX_CHARS,
|
||||
count: unique.length,
|
||||
hashes: unique.join(""),
|
||||
};
|
||||
|
||||
// Four-space JSON with a trailing newline: what prettier emits for this
|
||||
// shape, so a vendored artifact passes make fmt-check untouched.
|
||||
fs.writeFileSync(output, JSON.stringify(artifact, null, 4) + "\n");
|
||||
|
||||
process.stdout.write(
|
||||
"build-blocklist: " +
|
||||
config.blacklist.length +
|
||||
" source entries -> " +
|
||||
seen.size +
|
||||
" usable domains -> " +
|
||||
unique.length +
|
||||
" digests (" +
|
||||
dropped +
|
||||
" not hostnames, " +
|
||||
(seen.size - unique.length) +
|
||||
" digest collisions)\n",
|
||||
);
|
||||
}
|
||||
|
||||
main(process.argv.slice(2));
|
||||
Reference in New Issue
Block a user