chore: bring the token-list enrichment tooling into script/lib (closes #495)
check / check (push) Waiting to run
e2e / e2e-chrome (push) Waiting to run
e2e / e2e-firefox (push) Waiting to run

The two node programs and the data file from f7a2437 regenerate the name and
url of each token in src/shared/tokenList.js. They are now CommonJS under
script/lib/, find their files from their own location, and read
ethereum-lists at a pinned commit. The apply step also rewrites an entry that
already has a name and url, so it runs on the current list; on the committed
data it leaves the list unchanged. make enrich-tokens and make
apply-enrichment run them, and README.md names both targets. The data file
carries the MUSD token's on-chain name, so script/check-censored allows that
one literal there too.

Model: opus-5-5
This commit is contained in:
2026-10-14 04:40:26 +00:00
parent adff1e0b9d
commit 33ba37c8da
6 changed files with 2376 additions and 9 deletions
+12 -1
View File
@@ -1,4 +1,4 @@
.PHONY: bootstrap setup install test test-e2e test-e2e-firefox lint fmt fmt-check check check-censored docker hooks build build-debug package vendor-blocklist icons clean dev .PHONY: bootstrap setup install test test-e2e test-e2e-firefox lint fmt fmt-check check check-censored docker hooks build build-debug package vendor-blocklist icons enrich-tokens apply-enrichment clean dev
# Standard targets are thin shims; the implementations live in script/ # Standard targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section # per the scripts-to-rule-them-all pattern (see the Entrypoints section
@@ -111,6 +111,17 @@ vendor-blocklist:
icons: icons:
@node script/lib/icons.js @node script/lib/icons.js
# Fetch a name and a website for every token in src/shared/tokenList.js into
# script/lib/token-enrichment.json. Run deliberately, land the diff: the names
# come from live data, so a later run can return different ones.
enrich-tokens:
@node script/lib/enrich-tokens.js
# Write the names and websites in script/lib/token-enrichment.json into
# src/shared/tokenList.js. Commit the result with the data file.
apply-enrichment:
@node script/lib/apply-enrichment.js
clean: clean:
@rm -rf dist/ release/ @rm -rf dist/ release/
+19 -6
View File
@@ -246,11 +246,13 @@ provide:
nowhere in the working tree or under `dist/` outside its documented nowhere in the working tree or under `dist/` outside its documented
exceptions: the pinned source reference in `script/vendor-blocklist`, the two exceptions: the pinned source reference in `script/vendor-blocklist`, the two
provider-shim identifiers in `src/content/inpage.js`, and one ERC-20's provider-shim identifiers in `src/content/inpage.js`, and one ERC-20's
on-chain name in `src/shared/tokenList.js`. Each is scoped to that path and on-chain name in `src/shared/tokenList.js` and in
fails anywhere else. Run by the `lint` phase, so part of `make check`; it `script/lib/token-enrichment.json`, the fetched data that file's names are
inspects `dist/` when there is one and says loudly when there is not, and the written from. Each is scoped to the paths named and fails anywhere else. Run
build context of that phase never has one. `make build` re-runs it with by the `lint` phase, so part of `make check`; it inspects `dist/` when there
`--require-dist`, so a build artifact is always covered is one and says loudly when there is not, and the build context of that phase
never has one. `make build` re-runs it with `--require-dist`, so a build
artifact is always covered
- `script/package` — produce the release artifacts: `make build` first, so the - `script/package` — produce the release artifacts: `make build` first, so the
archives can only ever be made from a `dist/` that has been verified against archives can only ever be made from a `dist/` that has been verified against
that build's own receipt as a RELEASE build, then one self-contained, that build's own receipt as a RELEASE build, then one self-contained,
@@ -322,6 +324,17 @@ The Makefile shims to those. It also carries a few targets that have no
that already holds the drawn image, the same IHDR and every pixel, is left that already holds the drawn image, the same IHDR and every pixel, is left
untouched. Commit the files with the drawing: `make check` fails while their untouched. Commit the files with the drawing: `make check` fails while their
image differs from what it draws image differs from what it draws
- `make enrich-tokens` — fetch the name of each token in
`src/shared/tokenList.js` from CoinGecko's coin list and its website from the
`ethereum-lists/tokens` repository, at a pinned commit, into
`script/lib/token-enrichment.json`, by running `script/lib/enrich-tokens.js`.
CoinGecko's list is live and has no fixed version, so a later run can return
different names
- `make apply-enrichment` — write the names and URLs in
`script/lib/token-enrichment.json` into `src/shared/tokenList.js`, by running
`script/lib/apply-enrichment.js`, taking a URL from the table of project
websites in that script only where the fetched data has none. Neither target
changes which tokens are on the list, and no build or `make check` runs either
- `make clean` — remove `dist/` and `release/` - `make clean` — remove `dist/` and `release/`
- `make dev` — run the build `make build` runs, without the checks that follow - `make dev` — run the build `make build` runs, without the checks that follow
it, then run it again after every change to a file under `src/`, `manifest/` it, then run it again after every change to a file under `src/`, `manifest/`
@@ -2762,7 +2775,7 @@ exact URL, the commit it is pinned to and the sha256 of the bytes that commit
serves, because a source reference nobody can verify is not a source reference. serves, because a source reference nobody can verify is not a source reference.
`script/check-censored` reads the name back out of that one file and fails the `script/check-censored` reads the name back out of that one file and fails the
build wherever else it appears, save for three shipped-code literals it cannot build wherever else it appears, save for three shipped-code literals it cannot
avoid — each permitted only at the one path that carries it, and listed in that avoid — each permitted only at the paths that carry it, and listed in that
script's header. script's header.
## Author ## Author
+4 -2
View File
@@ -18,6 +18,8 @@
# (src/shared/balances.js), and this contract's on-chain name is that # (src/shared/balances.js), and this contract's on-chain name is that
# string, so censoring the repo cannot stop the wallet displaying it. # string, so censoring the repo cannot stop the wallet displaying it.
# Dropping the entry instead would cost the user MUSD spoof detection. # Dropping the entry instead would cost the user MUSD spoof detection.
# script/lib/token-enrichment.json, the fetched data tokenList.js's names
# are written from, carries the same name for the same reason.
# #
# Everything else fails, in the working tree and under dist/. The last two are # Everything else fails, in the working tree and under dist/. The last two are
# literals rather than whole files, so they are enforced by counting, and each # literals rather than whole files, so they are enforced by counting, and each
@@ -102,8 +104,8 @@ allowed_literals_for() {
src/content/inpage.js | dist/*/src/content/inpage.js) src/content/inpage.js | dist/*/src/content/inpage.js)
printf 'is%s\n_%s\n' "$NAME" "$NAME" >"$ALLOWED_LITERALS_FILE" printf 'is%s\n_%s\n' "$NAME" "$NAME" >"$ALLOWED_LITERALS_FILE"
;; ;;
src/shared/tokenList.js | dist/*/src/background/index.js | \ src/shared/tokenList.js | script/lib/token-enrichment.json | \
dist/*/src/popup/index.js) dist/*/src/background/index.js | dist/*/src/popup/index.js)
printf '%s USD\n' "$NAME" >"$ALLOWED_LITERALS_FILE" printf '%s USD\n' "$NAME" >"$ALLOWED_LITERALS_FILE"
;; ;;
esac esac
+176
View File
@@ -0,0 +1,176 @@
// Writes the names and URLs in script/lib/token-enrichment.json, which
// script/lib/enrich-tokens.js fetches, into the matching entries of
// src/shared/tokenList.js: the name and url an entry already has are replaced,
// and an entry without them gets them after its decimals. Where the fetched
// data has no URL for a token, EXTRA_URLS below supplies one if it lists the
// token. An entry the data does not mention is left as it is. Run by
// `make apply-enrichment`; no build runs it.
"use strict";
const fs = require("fs");
const path = require("path");
const TOKEN_LIST = path.join(__dirname, "..", "..", "src/shared/tokenList.js");
const enrichment = JSON.parse(
fs.readFileSync(path.join(__dirname, "token-enrichment.json"), "utf8"),
);
// Project websites by token address. A token gets its URL here only where the
// fetched data has none; an address the data has a URL for, or that is not on
// the token list, is never used.
const EXTRA_URLS = {
"0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2": "https://weth.io",
"0xdAC17F958D2ee523a2206206994597C13D831ec7": "https://tether.to",
"0xB8c77482e45F1F44dE1745F52C74426C631bDD52": "https://www.bnbchain.org",
"0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48": "https://www.circle.com/usdc",
"0xdC035D45d973E3EC169d2276DDab16f1e407384F": "https://sky.money",
"0x2AF5D2aD76741191D15Dfe7bF6aC92d4Bd912Ca3": "https://www.bitfinex.com",
"0x514910771AF9Ca656af840dff83E8264EcF986CA": "https://chain.link",
"0x4c9EDD5852cd905f086C759E8383e09bff1E68B3": "https://ethena.fi",
"0x8d0D000Ee44948FC98c9B98A4FA4921476f08B0d":
"https://www.worldlibertyfinancial.com",
"0x6B175474E89094C44Da98b954EedeAC495271d0F": "https://makerdao.com",
"0x6c3ea9036406852006290770BEdFcAbA0e23A0e8":
"https://www.paypal.com/us/digital-wallet/manage-money/crypto/pyusd",
"0x95aD61b0a150d79219dCF64E1E6Cc01f0B64C4cE": "https://shibatoken.com",
"0xdA5e1988097297dCdc1f90D4dFE7909e847CBeF6":
"https://www.worldlibertyfinancial.com",
"0xA0b73E1Ff0B80914AB6fe0444E65848C4C34450b": "https://crypto.com",
"0x582d872A1B094FC48F5DE31D3B73F2D9bE47def1": "https://ton.org",
"0x68749665FF8D2d112Fa859AA293F07A622782F38": "https://gold.tether.to",
"0x196C20DA81Fbc324EcdF55501e95Ce9f0bD84d14": "https://polkadot.network",
"0x1f9840a85d5aF5bf1D1762F925BDADdC4201F984": "https://uniswap.org",
"0x45804880De22913dAFE09f4980848ECE6EcbAf78":
"https://www.paxos.com/paxgold",
"0x3c3a81e81dc49A522A592e7622A7E711c06bf354": "https://www.mantle.xyz",
"0x7Fc66500c84A76Ad7e9c93437bFc5Ac33E2DDaE9": "https://aave.com",
"0x75231F58b43240C9718Dd58B4967c5114342a86c": "https://www.okx.com",
"0x6982508145454Ce325dDbE47a25d4ec3d2311933": "https://www.pepe.vip",
"0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599": "https://wbtc.network",
"0xae7ab96520DE3A18E5e111B5EaAb095312D7fE84": "https://lido.fi",
"0x5A98FcBEA516Cf06857215779Fd812CA3beF1B32": "https://lido.fi",
"0xC18360217D8F7Ab5e7c516566761Ea12Ce7F9D72": "https://ens.domains",
"0x7D1AfA7B718fb893dB30A3aBc0Cfc608AaCfeBB0": "https://polygon.technology",
"0x9f8F72aA9304c8B593d555F12eF6589cC3A579A2": "https://makerdao.com",
"0xD533a949740bb3306d119CC777fa900bA034cd52": "https://curve.fi",
"0x111111111117dC0aa78b770fA6A738034120C302": "https://1inch.io",
"0x163f8C2467924be0ae7B5347228CABF260318753": "https://worldcoin.org",
"0xba100000625a3754423978a60c9317c58a424e3D": "https://balancer.fi",
"0xC011a73ee8576Fb46F5E1c5751cA3B9Fe0af2a6F": "https://synthetix.io",
"0x1a7e4e63778B4f12a199C062f3eFdD288afCBce8": "https://www.euler.finance",
"0xBBbbCA6A901c926F240b89EacB641d8Aec7AEafD": "https://loopring.org",
"0xC669928185DbCE49d2230CC9B0979BE6DC797957": "https://bitstable.finance",
"0x7f39C581F595B53c5cb19bD0b3f8dA6c935E2Ca0": "https://lido.fi",
"0xBe9895146f7AF43049ca1c1AE358B0541Ea49704":
"https://www.coinbase.com/cbeth",
"0xfAbA6f8e4a5E8Ab82F62fe7C39859FA577269BE3": "https://ondo.finance",
"0x0C10bF8FcB7Bf5412187A595ab97a3609160b5c6": "https://decentraland.org",
"0x6De037ef9aD2725EB40118Bb1702EBb27e4Aeb24": "https://www.render.com",
"0x4a220E6096B25EADb88358cb44068A3248254675": "https://www.quant.network",
"0xba5BDe662c17e2aDFF1075610382B9B691296350": "https://superrare.com",
"0x15D4c048F83bd7e37d49eA4C83a07267Ec4203dA": "https://galxe.com",
"0xc00e94Cb662C3520282E6f5717214004A7f26888": "https://compound.finance",
"0xE41d2489571d322189246DaFA5ebDe1F4699F498": "https://0x.org",
"0x0bc529c00C6401aEF6D220BE8C6Ea1667F6Ad93e": "https://yearn.fi",
"0x6fB3e0A217407EFFf7Ca062D46c26E5d60a14d69": "https://iotex.io",
"0xD31a59c85aE9D8edEFec411D448f90841571b89c": "https://solana.com",
"0xa1faa113cbE53436Df28FF0aEe54275c13B40975": "https://alphafinance.io",
"0xF57e7e7C23978C3cAEC3C3548E3D615c346e79fF": "https://immutable.com",
"0x0f5D2fB29fb7d3CFeE444a200298f468908cC942": "https://decentraland.org",
"0x3845badAde8e6dFF049820680d1F14bD3903a5d0": "https://www.sandbox.game",
"0x4d224452801ACEd8B2F0aebE155379bb5D594381": "https://apecoin.com",
"0x6810e776880C02933D47DB1b9fc05908e5386b96": "https://gnosis.io",
"0x5f98805A4E8be255a32880FDeC7F6728C6568bA0": "https://www.liquity.org",
"0xd26114cd6EE289AccF82350c8d8487fedB8A0C07": "https://omg.network",
"0x0D8775F648430679A709E98d2b0Cb6250d2887EF":
"https://basicattentiontoken.org",
"0xc944E90C64B2c07662A292be6244BDf05Cda44a7": "https://thegraph.com",
"0x1985365e9f78359a9B6AD760e32412f4a445E862": "https://augur.net",
"0x4E15361FD6b4BB609Fa63C81A2be19d873717870": "https://fantom.foundation",
"0x853d955aCEf822Db058eb8505911ED77F175b99e": "https://frax.finance",
"0x0000000000085d4780B73119b644AE5ecd22b376": "https://www.trueusd.com",
"0x0cEC1A9154Ff802e7934Fc916Ed7Ca50bDE6844e": "https://polymesh.network",
"0x956F47F50A910163D8BF957Cf5846D573E7f87CA": "https://fei.money",
"0x3432B6A60D23Ca0dFCa7761B7ab56459D9C964D0": "https://frax.finance",
"0xf939E0A03FB07F59A73314E73794Be0E57ac1b4E": "https://curve.fi",
"0x0100546F2cD4C9D97f798fFC9755E47865FF7Ee6": "https://alchemix.fi",
"0x808507121B80c02388fAd14726482e061B8da827": "https://pendle.finance",
"0xCa14007Eff0dB1f8135f4C25B34De49AB0d42766": "https://starknet.io",
"0x8457CA5040ad67fdebbCC8EdCE889A335Bc0fbFB": "https://alt.layer",
"0x4C19596f5aAfF459fA38B0f7eD92F11AE6543784": "https://www.truflation.com",
"0xB50721BCf8d664c30412Cfbc6cf7a15145234ad1": "https://arbitrum.io",
"0x912CE59144191C1204E64559FE8253a0e49E6548": "https://arbitrum.io",
"0x6985884C4392D348587B19cb9eAAf157F13271cd": "https://layerzero.network",
"0xFe0c30065B384F05761f15d0CC899D4F9F9Cc0eB": "https://ethfi.org",
"0x18084fbA666a33d37592fA2633fD49a74DD93a88": "https://tbtc.network",
"0x9D39A5DE30e57443BfF2A8307A4256c8797A3497": "https://suilend.fi",
"0x5283D291DBCF85356A21bA090E6db59121208b44": "https://blur.io",
"0x43Dfc4159D86F3A37A5A4B3D4580b888ad7d4DDd": "https://dydx.exchange",
"0xCf0C122c6b73ff809C693DB761e7BaeBe62b6a2E": "https://floki.com",
"0xe53EC727dbDEB9E2d5456c3be40cFF031AB40A55": "https://superfarm.com",
"0x32353A6C91143bfd6C7d363B546e62a9A2489A20": "https://www.agora.finance",
"0x1aBaEA1f7C830bD89Acc67eC4af516284b1bC33c": "https://www.euro-coin.com",
"0x1a03eF3d1a80b5f214e468ACD97F55714E049393": "https://gemspad.io",
"0x7122985656e38BDC0302Db86685bb972b145bD3C": "https://stone.fi",
"0xa35923162C49cF95e6BF26623385eb431ad920D3": "https://www.turbo-eth.com",
"0xdBdb4d16EdA451D0503b854CF79D55697F90c8DF": "https://alchemix.fi",
"0x08c32b0726C5684024ea6e141C50aDe9690bBdcc": "https://stratis.io",
"0x65Ef703f5594D2573eb71Aaf55BC0CB548492df4": "https://multichain.org",
"0x1151CB3d861920e07a38e03eEAd12C32178567F6": "https://bonk.com",
"0x090185f2135308BaD17527004364eBcC2D37e5F6": "https://spell.club",
"0xAf5191B0De278C7286d6C7CC6ab6BB8A73bA2Cd6": "https://stargate.finance",
};
// Apply extra URLs where enrichment has none
for (const [addr, url] of Object.entries(EXTRA_URLS)) {
const e = enrichment[addr];
if (e && !e.url) {
e.url = url;
}
}
// Read original tokenList.js
const original = fs.readFileSync(TOKEN_LIST, "utf8");
// Replace each token block: name and url after decimals
let result = original;
let replacements = 0;
for (const [addr, data] of Object.entries(enrichment)) {
// Escape special regex characters in the address
const escaped = addr.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
// Match the token block: { address: "...", symbol: "...", decimals: N, }
// and the name: "...", url: "..." after decimals when it has them.
// The closing }, may have trailing whitespace
const blockRe = new RegExp(
`(\\{\\s*address:\\s*"${escaped}",\\s*symbol:\\s*"[^"]+",\\s*decimals:\\s*\\d+)` +
`(?:,\\s*name:\\s*"(?:[^"\\\\]|\\\\.)*",\\s*url:\\s*"(?:[^"\\\\]|\\\\.)*")?` +
`,?(\\s*\\})`,
);
const match = result.match(blockRe);
if (!match) {
console.error(`WARNING: Could not find block for ${addr}`);
continue;
}
// Build replacement: original fields + name + url
const nameEscaped = data.name.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
const urlEscaped = data.url.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
const replacement =
match[1] +
`,\n name: "${nameEscaped}",` +
`\n url: "${urlEscaped}",` +
match[2];
// A function, so a "$" in a name is not read as a replacement pattern.
result = result.replace(blockRe, () => replacement);
replacements++;
}
console.error(`Applied ${replacements} enrichments to tokenList.js`);
fs.writeFileSync(TOKEN_LIST, result);
console.error("Done!");
+115
View File
@@ -0,0 +1,115 @@
// Fetches a name and a website for every token in src/shared/tokenList.js and
// writes them, keyed by address, to script/lib/token-enrichment.json, which
// script/lib/apply-enrichment.js then writes into tokenList.js. Run by
// `make enrich-tokens`; no build runs it and nothing here is shipped.
//
// Names come from CoinGecko's coin list, one request for every coin, matched on
// the token's Ethereum contract address. A token CoinGecko does not list gets
// its symbol as its name. The list is live data with no fixed version, so a
// later run can return different names.
//
// Websites come from the ethereum-lists/tokens repository, one file per token,
// read at a pinned commit. Most tokens have no file there and get an empty URL.
// A file that is missing is not an error; any other failed request stops the
// run, so a network fault cannot quietly empty a URL.
"use strict";
const fs = require("fs");
const path = require("path");
const TOKEN_LIST = path.join(__dirname, "..", "..", "src/shared/tokenList.js");
const OUTPUT = path.join(__dirname, "token-enrichment.json");
const COINGECKO_LIST =
"https://api.coingecko.com/api/v3/coins/list?include_platform=true";
// ethereum-lists/tokens at its master of 2026-02-27, when the committed
// token-enrichment.json was made.
const ETHEREUM_LISTS =
"https://raw.githubusercontent.com/ethereum-lists/tokens/c3560d499a380a38e94caa7c4968d59aeffde59a/tokens/eth";
const CONCURRENT = 10;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
async function main() {
const src = fs.readFileSync(TOKEN_LIST, "utf8");
const tokens = [];
const re =
/\{\s*address:\s*"(0x[a-fA-F0-9]{40})",\s*symbol:\s*"([^"]+)",\s*decimals:\s*(\d+)/g;
let m;
while ((m = re.exec(src)) !== null) {
tokens.push({ address: m[1], symbol: m[2], decimals: parseInt(m[3]) });
}
console.error(`Parsed ${tokens.length} tokens`);
// --- Names from CoinGecko ---
console.error("Fetching CoinGecko coins list...");
const listResp = await fetch(COINGECKO_LIST);
if (!listResp.ok) throw new Error(`CoinGecko list HTTP ${listResp.status}`);
const coins = await listResp.json();
const addrMap = new Map();
for (const coin of coins) {
const eth = (coin.platforms?.ethereum || "").toLowerCase().trim();
if (eth && eth.startsWith("0x")) {
addrMap.set(eth, { name: coin.name, id: coin.id });
}
}
console.error(`CoinGecko map: ${addrMap.size} Ethereum tokens`);
const enriched = new Map();
let namesMatched = 0;
for (const t of tokens) {
const info = addrMap.get(t.address.toLowerCase());
if (info) namesMatched++;
enriched.set(t.address, { name: info?.name || t.symbol, url: "" });
}
console.error(`Matched ${namesMatched}/${tokens.length} names`);
// --- Websites from ethereum-lists ---
console.error("Fetching URLs from ethereum-lists GitHub...");
let urlsFound = 0;
for (let i = 0; i < tokens.length; i += CONCURRENT) {
const batch = tokens.slice(i, i + CONCURRENT);
await Promise.all(
batch.map(async (t) => {
const r = await fetch(`${ETHEREUM_LISTS}/${t.address}.json`);
if (r.status === 404) return;
if (!r.ok) {
throw new Error(
`ethereum-lists HTTP ${r.status} for ${t.address}`,
);
}
const data = await r.json();
const website = (data.website || "").trim();
if (website && website !== "https://") {
enriched.get(t.address).url = website;
urlsFound++;
}
}),
);
if (i % 100 === 0) {
console.error(
` GitHub: ${Math.min(i + CONCURRENT, tokens.length)}/${tokens.length}, ${urlsFound} URLs`,
);
}
await sleep(200);
}
console.error(`Found ${urlsFound} URLs in ethereum-lists`);
// --- Output, in the layout prettier gives it ---
const output = {};
for (const [addr, e] of enriched) {
output[addr] = { name: e.name, url: e.url };
}
fs.writeFileSync(OUTPUT, JSON.stringify(output, null, 4) + "\n");
console.error(`Done! ${tokens.length} names, ${urlsFound} URLs.`);
}
main().catch((err) => {
console.error(`enrich-tokens: ${err.message}`);
process.exit(1);
});
File diff suppressed because it is too large Load Diff