chore: bring the token-list enrichment tooling into script/lib (closes #495)
check / check (push) Waiting to run
e2e / e2e-chrome (push) Waiting to run
e2e / e2e-firefox (push) Waiting to run

The two node programs and the data file from f7a2437 regenerate the name and
url of each token in src/shared/tokenList.js. They are now CommonJS under
script/lib/, find their files from their own location, and read
ethereum-lists at a pinned commit. The apply step also rewrites an entry that
already has a name and url, so it runs on the current list; on the committed
data it leaves the list unchanged. The data file carries the MUSD token's
on-chain name, so script/check-censored allows that one literal there too.
README.md names both programs in its Entrypoints section.

Model: opus-5-5
This commit is contained in:
2026-10-14 02:17:23 +00:00
parent 8e52528f8b
commit d53b98e315
5 changed files with 2369 additions and 8 deletions
+21 -6
View File
@@ -246,11 +246,13 @@ provide:
nowhere in the working tree or under `dist/` outside its documented nowhere in the working tree or under `dist/` outside its documented
exceptions: the pinned source reference in `script/vendor-blocklist`, the two exceptions: the pinned source reference in `script/vendor-blocklist`, the two
provider-shim identifiers in `src/content/inpage.js`, and one ERC-20's provider-shim identifiers in `src/content/inpage.js`, and one ERC-20's
on-chain name in `src/shared/tokenList.js`. Each is scoped to that path and on-chain name in `src/shared/tokenList.js` and in
fails anywhere else. Run by the `lint` phase, so part of `make check`; it `script/lib/token-enrichment.json`, the fetched data that file's names are
inspects `dist/` when there is one and says loudly when there is not, and the written from. Each is scoped to the paths named and fails anywhere else. Run
build context of that phase never has one. `make build` re-runs it with by the `lint` phase, so part of `make check`; it inspects `dist/` when there
`--require-dist`, so a build artifact is always covered is one and says loudly when there is not, and the build context of that phase
never has one. `make build` re-runs it with `--require-dist`, so a build
artifact is always covered
- `script/package` — produce the release artifacts: `make build` first, so the - `script/package` — produce the release artifacts: `make build` first, so the
archives can only ever be made from a `dist/` that has been verified against archives can only ever be made from a `dist/` that has been verified against
that build's own receipt as a RELEASE build, then one self-contained, that build's own receipt as a RELEASE build, then one self-contained,
@@ -333,6 +335,19 @@ The Makefile shims to those. It also carries a few targets that have no
change anywhere else, `package.json` and `build.js` included, starts no build, change anywhere else, `package.json` and `build.js` included, starts no build,
and a directory created while it runs is not watched; restart it after either and a directory created while it runs is not watched; restart it after either
Two node programs regenerate the `name` and `url` of each token in
`src/shared/tokenList.js`. They are run by hand, never by a build or by
`make check`, and they do not change which tokens are on the list:
- `node script/lib/enrich-tokens.js` — fetch each token's name from CoinGecko's
coin list and its website from the `ethereum-lists/tokens` repository, at a
pinned commit, into `script/lib/token-enrichment.json`. CoinGecko's list is
live and has no fixed version, so a later run can return different names
- `node script/lib/apply-enrichment.js` — write the names and URLs in
`script/lib/token-enrichment.json` into `src/shared/tokenList.js`, taking a
URL from the table of project websites in the script where the fetched data
has none
## End-to-End Tests ## End-to-End Tests
There are two suites, one per browser, and they share no code. Chrome runs on There are two suites, one per browser, and they share no code. Chrome runs on
@@ -2762,7 +2777,7 @@ exact URL, the commit it is pinned to and the sha256 of the bytes that commit
serves, because a source reference nobody can verify is not a source reference. serves, because a source reference nobody can verify is not a source reference.
`script/check-censored` reads the name back out of that one file and fails the `script/check-censored` reads the name back out of that one file and fails the
build wherever else it appears, save for three shipped-code literals it cannot build wherever else it appears, save for three shipped-code literals it cannot
avoid — each permitted only at the one path that carries it, and listed in that avoid — each permitted only at the paths that carry it, and listed in that
script's header. script's header.
## Author ## Author
+4 -2
View File
@@ -18,6 +18,8 @@
# (src/shared/balances.js), and this contract's on-chain name is that # (src/shared/balances.js), and this contract's on-chain name is that
# string, so censoring the repo cannot stop the wallet displaying it. # string, so censoring the repo cannot stop the wallet displaying it.
# Dropping the entry instead would cost the user MUSD spoof detection. # Dropping the entry instead would cost the user MUSD spoof detection.
# script/lib/token-enrichment.json, the fetched data tokenList.js's names
# are written from, carries the same name for the same reason.
# #
# Everything else fails, in the working tree and under dist/. The last two are # Everything else fails, in the working tree and under dist/. The last two are
# literals rather than whole files, so they are enforced by counting, and each # literals rather than whole files, so they are enforced by counting, and each
@@ -102,8 +104,8 @@ allowed_literals_for() {
src/content/inpage.js | dist/*/src/content/inpage.js) src/content/inpage.js | dist/*/src/content/inpage.js)
printf 'is%s\n_%s\n' "$NAME" "$NAME" >"$ALLOWED_LITERALS_FILE" printf 'is%s\n_%s\n' "$NAME" "$NAME" >"$ALLOWED_LITERALS_FILE"
;; ;;
src/shared/tokenList.js | dist/*/src/background/index.js | \ src/shared/tokenList.js | script/lib/token-enrichment.json | \
dist/*/src/popup/index.js) dist/*/src/background/index.js | dist/*/src/popup/index.js)
printf '%s USD\n' "$NAME" >"$ALLOWED_LITERALS_FILE" printf '%s USD\n' "$NAME" >"$ALLOWED_LITERALS_FILE"
;; ;;
esac esac
+177
View File
@@ -0,0 +1,177 @@
// Writes the names and URLs in script/lib/token-enrichment.json, which
// script/lib/enrich-tokens.js fetches, into the matching entries of
// src/shared/tokenList.js: the name and url an entry already has are replaced,
// and an entry without them gets them after its decimals. Where the fetched
// data has no URL for a token, EXTRA_URLS below supplies one if it lists the
// token. An entry the data does not mention is left as it is. Run by hand; no
// build runs it.
//
// Usage: node script/lib/apply-enrichment.js
"use strict";
const fs = require("fs");
const path = require("path");
const TOKEN_LIST = path.join(__dirname, "..", "..", "src/shared/tokenList.js");
const enrichment = JSON.parse(
fs.readFileSync(path.join(__dirname, "token-enrichment.json"), "utf8"),
);
// Well-known URLs for major tokens not in ethereum-lists.
// These are official project websites verified from public sources.
const EXTRA_URLS = {
"0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2": "https://weth.io",
"0xdAC17F958D2ee523a2206206994597C13D831ec7": "https://tether.to",
"0xB8c77482e45F1F44dE1745F52C74426C631bDD52": "https://www.bnbchain.org",
"0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48": "https://www.circle.com/usdc",
"0xdC035D45d973E3EC169d2276DDab16f1e407384F": "https://sky.money",
"0x2AF5D2aD76741191D15Dfe7bF6aC92d4Bd912Ca3": "https://www.bitfinex.com",
"0x514910771AF9Ca656af840dff83E8264EcF986CA": "https://chain.link",
"0x4c9EDD5852cd905f086C759E8383e09bff1E68B3": "https://ethena.fi",
"0x8d0D000Ee44948FC98c9B98A4FA4921476f08B0d":
"https://www.worldlibertyfinancial.com",
"0x6B175474E89094C44Da98b954EedeAC495271d0F": "https://makerdao.com",
"0x6c3ea9036406852006290770BEdFcAbA0e23A0e8":
"https://www.paypal.com/us/digital-wallet/manage-money/crypto/pyusd",
"0x95aD61b0a150d79219dCF64E1E6Cc01f0B64C4cE": "https://shibatoken.com",
"0xdA5e1988097297dCdc1f90D4dFE7909e847CBeF6":
"https://www.worldlibertyfinancial.com",
"0xA0b73E1Ff0B80914AB6fe0444E65848C4C34450b": "https://crypto.com",
"0x582d872A1B094FC48F5DE31D3B73F2D9bE47def1": "https://ton.org",
"0x68749665FF8D2d112Fa859AA293F07A622782F38": "https://gold.tether.to",
"0x196C20DA81Fbc324EcdF55501e95Ce9f0bD84d14": "https://polkadot.network",
"0x1f9840a85d5aF5bf1D1762F925BDADdC4201F984": "https://uniswap.org",
"0x45804880De22913dAFE09f4980848ECE6EcbAf78":
"https://www.paxos.com/paxgold",
"0x3c3a81e81dc49A522A592e7622A7E711c06bf354": "https://www.mantle.xyz",
"0x7Fc66500c84A76Ad7e9c93437bFc5Ac33E2DDaE9": "https://aave.com",
"0x75231F58b43240C9718Dd58B4967c5114342a86c": "https://www.okx.com",
"0x6982508145454Ce325dDbE47a25d4ec3d2311933": "https://www.pepe.vip",
"0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599": "https://wbtc.network",
"0xae7ab96520DE3A18E5e111B5EaAb095312D7fE84": "https://lido.fi",
"0x5A98FcBEA516Cf06857215779Fd812CA3beF1B32": "https://lido.fi",
"0xC18360217D8F7Ab5e7c516566761Ea12Ce7F9D72": "https://ens.domains",
"0x7D1AfA7B718fb893dB30A3aBc0Cfc608AaCfeBB0": "https://polygon.technology",
"0x9f8F72aA9304c8B593d555F12eF6589cC3A579A2": "https://makerdao.com",
"0xD533a949740bb3306d119CC777fa900bA034cd52": "https://curve.fi",
"0x111111111117dC0aa78b770fA6A738034120C302": "https://1inch.io",
"0x163f8C2467924be0ae7B5347228CABF260318753": "https://worldcoin.org",
"0xba100000625a3754423978a60c9317c58a424e3D": "https://balancer.fi",
"0xC011a73ee8576Fb46F5E1c5751cA3B9Fe0af2a6F": "https://synthetix.io",
"0x1a7e4e63778B4f12a199C062f3eFdD288afCBce8": "https://www.euler.finance",
"0xBBbbCA6A901c926F240b89EacB641d8Aec7AEafD": "https://loopring.org",
"0xC669928185DbCE49d2230CC9B0979BE6DC797957": "https://bitstable.finance",
"0x7f39C581F595B53c5cb19bD0b3f8dA6c935E2Ca0": "https://lido.fi",
"0xBe9895146f7AF43049ca1c1AE358B0541Ea49704":
"https://www.coinbase.com/cbeth",
"0xfAbA6f8e4a5E8Ab82F62fe7C39859FA577269BE3": "https://ondo.finance",
"0x0C10bF8FcB7Bf5412187A595ab97a3609160b5c6": "https://decentraland.org",
"0x6De037ef9aD2725EB40118Bb1702EBb27e4Aeb24": "https://www.render.com",
"0x4a220E6096B25EADb88358cb44068A3248254675": "https://www.quant.network",
"0xba5BDe662c17e2aDFF1075610382B9B691296350": "https://superrare.com",
"0x15D4c048F83bd7e37d49eA4C83a07267Ec4203dA": "https://galxe.com",
"0xc00e94Cb662C3520282E6f5717214004A7f26888": "https://compound.finance",
"0xE41d2489571d322189246DaFA5ebDe1F4699F498": "https://0x.org",
"0x0bc529c00C6401aEF6D220BE8C6Ea1667F6Ad93e": "https://yearn.fi",
"0x6fB3e0A217407EFFf7Ca062D46c26E5d60a14d69": "https://iotex.io",
"0xD31a59c85aE9D8edEFec411D448f90841571b89c": "https://solana.com",
"0xa1faa113cbE53436Df28FF0aEe54275c13B40975": "https://alphafinance.io",
"0xF57e7e7C23978C3cAEC3C3548E3D615c346e79fF": "https://immutable.com",
"0x0f5D2fB29fb7d3CFeE444a200298f468908cC942": "https://decentraland.org",
"0x3845badAde8e6dFF049820680d1F14bD3903a5d0": "https://www.sandbox.game",
"0x4d224452801ACEd8B2F0aebE155379bb5D594381": "https://apecoin.com",
"0x6810e776880C02933D47DB1b9fc05908e5386b96": "https://gnosis.io",
"0x5f98805A4E8be255a32880FDeC7F6728C6568bA0": "https://www.liquity.org",
"0xd26114cd6EE289AccF82350c8d8487fedB8A0C07": "https://omg.network",
"0x0D8775F648430679A709E98d2b0Cb6250d2887EF":
"https://basicattentiontoken.org",
"0xc944E90C64B2c07662A292be6244BDf05Cda44a7": "https://thegraph.com",
"0x1985365e9f78359a9B6AD760e32412f4a445E862": "https://augur.net",
"0x4E15361FD6b4BB609Fa63C81A2be19d873717870": "https://fantom.foundation",
"0x853d955aCEf822Db058eb8505911ED77F175b99e": "https://frax.finance",
"0x0000000000085d4780B73119b644AE5ecd22b376": "https://www.trueusd.com",
"0x0cEC1A9154Ff802e7934Fc916Ed7Ca50bDE6844e": "https://polymesh.network",
"0x956F47F50A910163D8BF957Cf5846D573E7f87CA": "https://fei.money",
"0x3432B6A60D23Ca0dFCa7761B7ab56459D9C964D0": "https://frax.finance",
"0xf939E0A03FB07F59A73314E73794Be0E57ac1b4E": "https://curve.fi",
"0x0100546F2cD4C9D97f798fFC9755E47865FF7Ee6": "https://alchemix.fi",
"0x808507121B80c02388fAd14726482e061B8da827": "https://pendle.finance",
"0xCa14007Eff0dB1f8135f4C25B34De49AB0d42766": "https://starknet.io",
"0x8457CA5040ad67fdebbCC8EdCE889A335Bc0fbFB": "https://alt.layer",
"0x4C19596f5aAfF459fA38B0f7eD92F11AE6543784": "https://www.truflation.com",
"0xB50721BCf8d664c30412Cfbc6cf7a15145234ad1": "https://arbitrum.io",
"0x912CE59144191C1204E64559FE8253a0e49E6548": "https://arbitrum.io",
"0x6985884C4392D348587B19cb9eAAf157F13271cd": "https://layerzero.network",
"0xFe0c30065B384F05761f15d0CC899D4F9F9Cc0eB": "https://ethfi.org",
"0x18084fbA666a33d37592fA2633fD49a74DD93a88": "https://tbtc.network",
"0x9D39A5DE30e57443BfF2A8307A4256c8797A3497": "https://suilend.fi",
"0x5283D291DBCF85356A21bA090E6db59121208b44": "https://blur.io",
"0x43Dfc4159D86F3A37A5A4B3D4580b888ad7d4DDd": "https://dydx.exchange",
"0xCf0C122c6b73ff809C693DB761e7BaeBe62b6a2E": "https://floki.com",
"0xe53EC727dbDEB9E2d5456c3be40cFF031AB40A55": "https://superfarm.com",
"0x32353A6C91143bfd6C7d363B546e62a9A2489A20": "https://www.agora.finance",
"0x1aBaEA1f7C830bD89Acc67eC4af516284b1bC33c": "https://www.euro-coin.com",
"0x1a03eF3d1a80b5f214e468ACD97F55714E049393": "https://gemspad.io",
"0x7122985656e38BDC0302Db86685bb972b145bD3C": "https://stone.fi",
"0xa35923162C49cF95e6BF26623385eb431ad920D3": "https://www.turbo-eth.com",
"0xdBdb4d16EdA451D0503b854CF79D55697F90c8DF": "https://alchemix.fi",
"0x08c32b0726C5684024ea6e141C50aDe9690bBdcc": "https://stratis.io",
"0x65Ef703f5594D2573eb71Aaf55BC0CB548492df4": "https://multichain.org",
"0x1151CB3d861920e07a38e03eEAd12C32178567F6": "https://bonk.com",
"0x090185f2135308BaD17527004364eBcC2D37e5F6": "https://spell.club",
"0xAf5191B0De278C7286d6C7CC6ab6BB8A73bA2Cd6": "https://stargate.finance",
};
// Apply extra URLs where enrichment has none
for (const [addr, url] of Object.entries(EXTRA_URLS)) {
const e = enrichment[addr];
if (e && !e.url) {
e.url = url;
}
}
// Read original tokenList.js
const original = fs.readFileSync(TOKEN_LIST, "utf8");
// Replace each token block: name and url after decimals
let result = original;
let replacements = 0;
for (const [addr, data] of Object.entries(enrichment)) {
// Escape special regex characters in the address
const escaped = addr.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
// Match the token block: { address: "...", symbol: "...", decimals: N, }
// and the name: "...", url: "..." after decimals when it has them.
// The closing }, may have trailing whitespace
const blockRe = new RegExp(
`(\\{\\s*address:\\s*"${escaped}",\\s*symbol:\\s*"[^"]+",\\s*decimals:\\s*\\d+)` +
`(?:,\\s*name:\\s*"(?:[^"\\\\]|\\\\.)*",\\s*url:\\s*"(?:[^"\\\\]|\\\\.)*")?` +
`,?(\\s*\\})`,
);
const match = result.match(blockRe);
if (!match) {
console.error(`WARNING: Could not find block for ${addr}`);
continue;
}
// Build replacement: original fields + name + url
const nameEscaped = data.name.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
const urlEscaped = data.url.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
const replacement =
match[1] +
`,\n name: "${nameEscaped}",` +
`\n url: "${urlEscaped}",` +
match[2];
// A function, so a "$" in a name is not read as a replacement pattern.
result = result.replace(blockRe, () => replacement);
replacements++;
}
console.error(`Applied ${replacements} enrichments to tokenList.js`);
fs.writeFileSync(TOKEN_LIST, result);
console.error("Done!");
+117
View File
@@ -0,0 +1,117 @@
// Fetches a name and a website for every token in src/shared/tokenList.js and
// writes them, keyed by address, to script/lib/token-enrichment.json, which
// script/lib/apply-enrichment.js then writes into tokenList.js. Run by hand;
// no build runs it and nothing here is shipped.
//
// Usage: node script/lib/enrich-tokens.js
//
// Names come from CoinGecko's coin list, one request for every coin, matched on
// the token's Ethereum contract address. A token CoinGecko does not list gets
// its symbol as its name. The list is live data with no fixed version, so a
// later run can return different names.
//
// Websites come from the ethereum-lists/tokens repository, one file per token,
// read at a pinned commit. Most tokens have no file there and get an empty URL.
// A file that is missing is not an error; any other failed request stops the
// run, so a network fault cannot quietly empty a URL.
"use strict";
const fs = require("fs");
const path = require("path");
const TOKEN_LIST = path.join(__dirname, "..", "..", "src/shared/tokenList.js");
const OUTPUT = path.join(__dirname, "token-enrichment.json");
const COINGECKO_LIST =
"https://api.coingecko.com/api/v3/coins/list?include_platform=true";
// ethereum-lists/tokens at its master of 2026-02-27, when the committed
// token-enrichment.json was made.
const ETHEREUM_LISTS =
"https://raw.githubusercontent.com/ethereum-lists/tokens/c3560d499a380a38e94caa7c4968d59aeffde59a/tokens/eth";
const CONCURRENT = 10;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function main() {
const src = fs.readFileSync(TOKEN_LIST, "utf8");
const tokens = [];
const re =
/\{\s*address:\s*"(0x[a-fA-F0-9]{40})",\s*symbol:\s*"([^"]+)",\s*decimals:\s*(\d+)/g;
let m;
while ((m = re.exec(src)) !== null) {
tokens.push({ address: m[1], symbol: m[2], decimals: parseInt(m[3]) });
}
console.error(`Parsed ${tokens.length} tokens`);
// --- Names from CoinGecko ---
console.error("Fetching CoinGecko coins list...");
const listResp = await fetch(COINGECKO_LIST);
if (!listResp.ok) throw new Error(`CoinGecko list HTTP ${listResp.status}`);
const coins = await listResp.json();
const addrMap = new Map();
for (const coin of coins) {
const eth = (coin.platforms?.ethereum || "").toLowerCase().trim();
if (eth && eth.startsWith("0x")) {
addrMap.set(eth, { name: coin.name, id: coin.id });
}
}
console.error(`CoinGecko map: ${addrMap.size} Ethereum tokens`);
const enriched = new Map();
let namesMatched = 0;
for (const t of tokens) {
const info = addrMap.get(t.address.toLowerCase());
if (info) namesMatched++;
enriched.set(t.address, { name: info?.name || t.symbol, url: "" });
}
console.error(`Matched ${namesMatched}/${tokens.length} names`);
// --- Websites from ethereum-lists ---
console.error("Fetching URLs from ethereum-lists GitHub...");
let urlsFound = 0;
for (let i = 0; i < tokens.length; i += CONCURRENT) {
const batch = tokens.slice(i, i + CONCURRENT);
await Promise.all(
batch.map(async (t) => {
const r = await fetch(`${ETHEREUM_LISTS}/${t.address}.json`);
if (r.status === 404) return;
if (!r.ok) {
throw new Error(
`ethereum-lists HTTP ${r.status} for ${t.address}`,
);
}
const data = await r.json();
const website = (data.website || "").trim();
if (website && website !== "https://") {
enriched.get(t.address).url = website;
urlsFound++;
}
}),
);
if (i % 100 === 0) {
console.error(
` GitHub: ${Math.min(i + CONCURRENT, tokens.length)}/${tokens.length}, ${urlsFound} URLs`,
);
}
await sleep(200);
}
console.error(`Found ${urlsFound} URLs in ethereum-lists`);
// --- Output, in the layout prettier gives it ---
const output = {};
for (const [addr, e] of enriched) {
output[addr] = { name: e.name, url: e.url };
}
fs.writeFileSync(OUTPUT, JSON.stringify(output, null, 4) + "\n");
console.error(`Done! ${tokens.length} names, ${urlsFound} URLs.`);
}
main().catch((err) => {
console.error(`enrich-tokens: ${err.message}`);
process.exit(1);
});
File diff suppressed because it is too large Load Diff