sneak/AutistMask
1
0
Fork 0
Code Issues 34 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat: vendor and censor the phishing blocklist at build time (closes #219)
Some checks failed
check / check (push) Has been cancelled
Details
e2e / e2e-chrome (push) Has been cancelled
Details
e2e / e2e-firefox (push) Has been cancelled
Details

Browse Source 
The blocklist URL in shipped code named a competitor and pointed at a moving
ref, and the extension re-fetched from it every 24 hours, which also meant a
third party decided what this wallet warns about. All of that is gone.

script/vendor-blocklist fetches upstream at a pinned commit, verifies the
sha256 of the bytes that commit serves, and writes
src/shared/phishingBlocklist.json. It is build-time tooling, never shipped, and
the one place in the repo that names the upstream project; a source reference
nobody can verify is not a source reference.

The artifact stores truncated sha256 digests rather than domain names. That is
what censors it: the previous file contained the competitor's name 6,475 times,
as phishing domains impersonating them, and not one of those domains is
dropped. It also makes lookups a binary search over a fixed-width string, so
nothing is built at module load — which matters on MV3, where the worker
re-evaluates the module on every wake — and takes the file from 8.7 MB to
1.7 MB.

script/check-censored enforces the rest: it reads the name out of the vendoring
script rather than repeating it, and fails on any occurrence in the working
tree or under dist/ that is not one of the two literals shipped code cannot
avoid. It runs in make check, which inspects dist/ when there is one and says
loudly when there is not, and again with --require-dist at the end of every
make build.

Removing the runtime fetch retires the delta, the extension-storage persistence
and the 24-hour alarm from #158. A retired alarm is now cleared rather than
left waking the worker forever on installs that already have it.

The e2e suite drives the warning end to end from a real blocklisted origin
served as a real http(s) site, with a control asserting the banner stays hidden
for one that is not listed. Its service-worker interception canary needed a new
anchor, since the startup fetch it used to watch for no longer happens: it now
wakes the worker with a message and asks it for one throwaway fetch.

LICENSE no longer cites a repository that returns 404.

eslint.config.js gains one block: script/lib/ holds node programs the shell
entrypoints call, and without it they lint with no globals at all.
This commit is contained in:
Jeffrey Paul
2026-08-17 07:07:52 +00:00
parent a60c4a616a
commit f3a18d2154
24 changed files with 1316 additions and 232496 deletions
Download Patch File Download Diff File Expand all files Collapse all files

375
src/shared/phishingDomains.js
View File

@@ -1,158 +1,109 @@
// Domain-based phishing detection using a vendored blocklist with delta updates.
// Domain-based phishing detection against a blocklist vendored at build time.
//
// A community-maintained phishing domain blocklist is vendored in
// phishingBlocklist.json and bundled at build time. At runtime, we fetch
// the live list periodically and keep only the delta (new entries not in
// the vendored list) in memory. This keeps runtime memory usage small.
// The list is produced by script/vendor-blocklist from a hash-pinned upstream
// commit, committed as phishingBlocklist.json, and bundled. There is no runtime
// fetch: the extension asks nobody anything to answer this question, so no third
// party learns which sites a user connects to, and no third party decides what
// this wallet warns about. The cost is staleness — the shipped list is exactly
// as fresh as the last vendoring run that was released — and the refresh path is
// re-running that script and shipping the diff.
//
// The domain-checker checks the in-memory delta first (fresh/recent scam
// sites), then falls back to the vendored list.
// The artifact holds digests, not domains: sha256 truncated to 64 bits, one
// entry per 16 hex characters, concatenated in sorted order into a single
// string (see domainHash.js). Three things follow from that shape, and all
// three are the reason for it:
//
// If the delta and its fetch timestamp fit in 256 KiB they are persisted to
// extension storage, so they survive termination of the MV3 service worker.
// Extension storage, not localStorage: localStorage does not exist in a
// service worker, so the previous persistence never ran on Chrome at all.
// The stored timestamps are what keep a restarted worker from re-fetching on
// every wake while still noticing an overdue update. Those guards apply to the
// startup path only; the 24-hour alarm tick bypasses them, or it would veto
// its own refresh — see updatePhishingList().
// - the extension ships no plaintext list of anyone's domain names, which is
// what makes a blocklist assembled elsewhere shippable here at all.
// - a lookup is a binary search over that string. Nothing is built at module
// load, which matters because the MV3 service worker is torn down when idle
// and re-evaluates this file on every wake.
// - the file is 1.7 MB rather than 8.7 MB.
//
// Nothing here is async: callers answer an approval prompt with the result.
const vendoredConfig = require("./phishingBlocklist.json");
const { storageLocal } = require("./browserApi");
const vendored = require("./phishingBlocklist.json");
const { HASH_ALGORITHM, HASH_HEX_CHARS, hashDomain } = require("./domainHash");
const BLOCKLIST_URL =
"https://raw.githubusercontent.com/MetaMask/eth-phishing-detect/main/src/config.json";
// The artifact is generated, so a shape it does not have is a build fault, not
// a runtime condition. It is checked anyway, and loudly, because every way of
// getting it wrong — a stale format, a truncated file, a different digest —
// produces a blocklist that matches nothing at all while looking perfectly
// healthy. A phishing check that silently answers "no" to everything is the one
// failure this module must not have.
function checkArtifact(a) {
const bad = (why) =>
new Error(
"phishingBlocklist.json " +
why +
". It is generated by script/vendor-blocklist; re-run that " +
"rather than editing it.",
);
const CACHE_TTL_MS = 24 * 60 * 60 * 1000; // 24 hours
// Floor on how often an unscheduled path may hit the network. The worker is
// revived every ~30 seconds while the browser is busy, and every revival runs
// the startup path; without a persisted record of the last attempt, any state
// that leaves lastFetchTime unset — a fetch that failed, or a delta too large
// to store — would download the full list on every single wake.
const MIN_FETCH_ATTEMPT_INTERVAL_MS = 60 * 60 * 1000; // 1 hour
const DELTA_STORAGE_KEY = "phishing-delta";
const MAX_DELTA_BYTES = 256 * 1024; // 256 KiB
// Vendored set — built once from the bundled JSON.
const vendoredBlacklist = new Set(
(vendoredConfig.blacklist || []).map((d) => d.toLowerCase()),
);
// Delta set — only entries from live list that are NOT in vendored.
let deltaBlacklist = new Set();
let lastFetchTime = 0;
let lastAttemptTime = 0;
let fetchPromise = null;
let loadPromise = null;
// storageLocal() resolves on use rather than at module load, so a test can
// install a stub after requiring this module, and it returns null where the
// API is absent — which is why the popup, with no reason to touch the delta,
// loads fine without it.
/**
* Sanitise a timestamp read back from storage.
*
* A value in the future is permanent poison: every guard here measures elapsed
* time as `Date.now() - stamp` and tests only the lower bound, so a stamp a
* year ahead suppresses updates for a year with no path that ever clears it.
* Clock skew and a restored profile backup both produce one. Since these
* timestamps only ever gate work, discarding an impossible one is safe: it
* costs at most a single extra fetch and restores a sane value immediately.
*
* @param {unknown} value
* @returns {number} the timestamp, or 0 if it is unusable.
*/
function sanitizeTimestamp(value) {
if (typeof value !== "number" || !Number.isFinite(value)) return 0;
if (value <= 0 || value > Date.now()) return 0;
return value;
}
/**
* Load the persisted delta and its timestamps from extension storage.
* Runs once per worker lifetime; every entry point funnels through
* ensureDeltaLoaded() so a wake from termination restores state exactly once.
*
* @returns {Promise<void>}
*/
async function loadDeltaFromStorage() {
const storage = storageLocal();
if (!storage) return;
try {
const result = await storage.get(DELTA_STORAGE_KEY);
const data = result && result[DELTA_STORAGE_KEY];
if (!data) return;
if (Array.isArray(data.blacklist)) {
deltaBlacklist = new Set(
data.blacklist.map((d) => d.toLowerCase()),
);
}
lastFetchTime = sanitizeTimestamp(data.lastFetchTime);
lastAttemptTime = sanitizeTimestamp(data.lastAttemptTime);
} catch {
// Storage unavailable or corrupt — start empty and re-fetch.
if (!a || typeof a !== "object") throw bad("is not an object");
if (a.algorithm !== HASH_ALGORITHM) {
throw bad(
"declares algorithm " +
JSON.stringify(a.algorithm) +
", but this build hashes with " +
HASH_ALGORITHM,
);
}
if (a.hashHexChars !== HASH_HEX_CHARS) {
throw bad(
"declares " +
JSON.stringify(a.hashHexChars) +
" hex characters per entry, but this build produces " +
HASH_HEX_CHARS,
);
}
if (typeof a.hashes !== "string") throw bad("has no hashes string");
if (!Number.isInteger(a.count) || a.count < 1) {
throw bad("declares no usable entry count");
}
if (a.hashes.length !== a.count * HASH_HEX_CHARS) {
throw bad(
"holds " +
a.hashes.length +
" hex characters, which is not the " +
a.count * HASH_HEX_CHARS +
" its count of " +
a.count +
" entries requires",
);
}
}
function ensureDeltaLoaded() {
if (!loadPromise) loadPromise = loadDeltaFromStorage();
return loadPromise;
}
checkArtifact(vendored);
const HASHES = vendored.hashes;
const COUNT = vendored.count;
/**
* Persist the delta and its timestamps if they fit within MAX_DELTA_BYTES.
* Is this digest one of the vendored entries?
*
* The 256 KiB cap covers the delta and its freshness claim: when the delta is
* too large to keep, lastFetchTime goes with it, so the next start re-fetches
* rather than trusting a freshness claim for a delta it no longer holds.
* lastAttemptTime is written either way — it records that the network was
* contacted, which stays true whatever became of the response, and it is what
* stops a permanently oversized list from downloading on every worker wake.
* Binary search over fixed-width records. The digests are lowercase hex of one
* width, so lexicographic order is numeric order and the artifact is written
* sorted; tests assert that ordering against the committed file, because an
* unsorted artifact would fail lookups silently rather than loudly.
*
* @returns {Promise<void>}
* @param {string} hash
* @returns {boolean}
*/
async function saveDeltaToStorage() {
const storage = storageLocal();
if (!storage) return;
try {
const data = {
blacklist: Array.from(deltaBlacklist),
lastFetchTime,
lastAttemptTime,
};
const json = JSON.stringify(data);
if (json.length < MAX_DELTA_BYTES) {
await storage.set({ [DELTA_STORAGE_KEY]: data });
} else if (lastAttemptTime > 0) {
await storage.set({ [DELTA_STORAGE_KEY]: { lastAttemptTime } });
} else {
await storage.remove(DELTA_STORAGE_KEY);
}
} catch {
// Storage unavailable — skip silently
function hashListed(hash) {
let lo = 0;
let hi = COUNT - 1;
while (lo <= hi) {
const mid = (lo + hi) >> 1;
const at = HASHES.slice(
mid * HASH_HEX_CHARS,
(mid + 1) * HASH_HEX_CHARS,
);
if (at === hash) return true;
if (at < hash) lo = mid + 1;
else hi = mid - 1;
}
}
/**
* Load a pre-parsed config and compute the delta against the vendored list.
* Used for both live fetches and testing.
*
* @param {{ blacklist?: string[] }} config
* @returns {Promise<void>} resolves once the delta has been persisted.
*/
function loadConfig(config) {
const liveBlacklist = (config.blacklist || []).map((d) => d.toLowerCase());
// Delta = entries in the live list that are NOT in the vendored list
deltaBlacklist = new Set(
liveBlacklist.filter((d) => !vendoredBlacklist.has(d)),
);
lastFetchTime = Date.now();
return saveDeltaToStorage();
return false;
}
/**
@@ -175,161 +126,33 @@ function hostnameVariants(hostname) {
/**
* Check if a hostname is on the phishing blocklist.
* Checks delta first (fresh/recent scam sites), then vendored list.
*
* Synchronous by design — callers answer an approval prompt with it. On a
* worker that has just woken, the persisted delta may still be loading; the
* vendored list, which is bundled and always present, carries the check until
* it lands.
*
* @param {string} hostname - The hostname to check.
* @returns {boolean}
*/
function isPhishingDomain(hostname) {
if (!hostname) return false;
const variants = hostnameVariants(hostname);
// Check delta blacklist first (fresh/recent scam sites), then vendored
for (const v of variants) {
if (deltaBlacklist.has(v) || vendoredBlacklist.has(v)) return true;
for (const variant of hostnameVariants(hostname)) {
if (hashListed(hashDomain(variant))) return true;
}
return false;
}
/**
* Fetch the latest blocklist and compute delta against vendored data.
* De-duplicates concurrent fetches. Results are cached for CACHE_TTL_MS,
* counted from the persisted timestamp so the cache outlives the worker.
*
* `force` is what makes the 24-hour alarm actually refresh every 24 hours.
* The alarm fires one period after the previous alarm, but lastFetchTime is
* stamped when that fetch *completed*, so an unforced tick lands one fetch
* latency inside its own TTL, skips, and turns the real cadence into 48 hours.
* Shortening the TTL instead would not fix it: the worker wakes every ~30
* seconds and the startup path re-checks the TTL each time, so a shortened TTL
* simply becomes the real cadence. The TTL is there to stop redundant fetches
* on wake, and the scheduled tick is not redundant, so it bypasses it.
*
* @param {{force?: boolean}} [opts] force: fetch unless one is already in
* flight, ignoring both the freshness and the retry guard. For the scheduled
* alarm tick only.
* @returns {Promise<void>}
*/
async function updatePhishingList({ force = false } = {}) {
// A worker that has just been revived knows nothing until the persisted
// record is back in memory; without this the freshness check below would
// always see 0 and re-fetch on every wake.
await ensureDeltaLoaded();
if (!force) {
const now = Date.now();
// Skip if recently fetched.
if (lastFetchTime > 0 && now - lastFetchTime < CACHE_TTL_MS) return;
// Skip if the network was contacted recently and the result was not
// usable — a failed fetch or an oversized delta leaves lastFetchTime
// unset, and without this every wake would retry.
if (
lastAttemptTime > 0 &&
now - lastAttemptTime < MIN_FETCH_ATTEMPT_INTERVAL_MS
) {
return;
}
}
// De-duplicate concurrent calls
if (fetchPromise) return fetchPromise;
fetchPromise = (async () => {
lastAttemptTime = Date.now();
try {
const resp = await fetch(BLOCKLIST_URL);
if (!resp.ok) throw new Error("HTTP " + resp.status);
const config = await resp.json();
await loadConfig(config);
} catch {
// Silently fail — vendored list still provides coverage. Persist
// the attempt so a persistently failing fetch is retried on the
// schedule rather than on every wake.
await saveDeltaToStorage();
} finally {
fetchPromise = null;
}
})();
return fetchPromise;
}
/**
* Restore persisted state and fetch if the list is overdue.
*
* Called from the background script every time it starts — a fresh install,
* a browser start, and every revival of a terminated service worker all land
* here. The recurring 24-hour schedule itself is an alarm (see
* shared/alarms.js), not a timer, because timers die with the worker.
*
* @returns {Promise<void>}
*/
async function initPhishingList() {
await ensureDeltaLoaded();
return updatePhishingList();
}
/**
* The 24-hour alarm tick. Separate from initPhishingList() because this is the
* scheduled refresh and must not be vetoed by the guards that exist to keep
* the unscheduled startup path off the network.
*
* @returns {Promise<void>}
*/
async function refreshPhishingListOnSchedule() {
return updatePhishingList({ force: true });
}
/**
* Return the total blocklist size (vendored + delta) for diagnostics.
* Return the blocklist size for diagnostics.
*
* @returns {number}
*/
function getBlocklistSize() {
return vendoredBlacklist.size + deltaBlacklist.size;
}
/**
* Return the delta blocklist size for diagnostics.
*
* @returns {number}
*/
function getDeltaSize() {
return deltaBlacklist.size;
}
/**
* Reset internal state (for testing).
*/
function _reset() {
deltaBlacklist = new Set();
lastFetchTime = 0;
lastAttemptTime = 0;
fetchPromise = null;
loadPromise = null;
return COUNT;
}
module.exports = {
isPhishingDomain,
updatePhishingList,
refreshPhishingListOnSchedule,
initPhishingList,
loadDeltaFromStorage,
loadConfig,
CACHE_TTL_MS,
MIN_FETCH_ATTEMPT_INTERVAL_MS,
DELTA_STORAGE_KEY,
MAX_DELTA_BYTES,
getBlocklistSize,
getDeltaSize,
hostnameVariants,
_reset,
// Exposed for testing only
_getVendoredBlacklistSize: () => vendoredBlacklist.size,
_getDeltaBlacklist: () => deltaBlacklist,
// Exposed for testing only: the ends of the search range are where an
// off-by-one hides, and reaching them through isPhishingDomain() would mean
// knowing which domain hashes to the first or last entry.
_hashListed: hashListed,
};