chore: bring the token-list enrichment tooling into script/lib (closes #495) #531

Merged
clawbot merged 1 commits from issue-495-token-list-enrichment into next 2026-10-14 20:10:30 +02:00
Collaborator

Brings the token-list tooling from commit f7a2437 into the repo, for #495. make enrich-tokens runs script/lib/enrich-tokens.js, which fetches each token's name and website into script/lib/token-enrichment.json; make apply-enrichment runs script/lib/apply-enrichment.js, which writes them into src/shared/tokenList.js.

Reproduction. On the list before names were added (aca8c4b's parent), the apply step rebuilds aca8c4b's list plus two URLs this PR adds to src/shared/tokenList.js: https://floki.com for FLOKI and https://www.turbo-eth.com for TURBO.

Changed from the original:

  • CommonJS, to lint under the script/lib/ rules; paths resolve from the script.
  • The apply step also replaces a name and url an entry already has, so it runs on today's list.
  • An address in the data or the website table that matches no list entry stops the apply step, naming it; the table's 28 entries for unlisted tokens are gone, and FLOKI, TURBO and MANA take the list's capitals.
  • ethereum-lists is read at commit c3560d4 (its master on 2026-02-27). A failed request other than a missing file stops the run.

Worth knowing:

  • A fetch today gives the same URLs and 28 different names, since renamed or dropped by CoinGecko. Not committed.
  • The website table fills only URLs the fetched data lacks, so 22 of its 66 entries are never used (USDC ships the stale centre.io).

Judgement call: script/check-censored also allows the MUSD token's on-chain name in script/lib/token-enrichment.json.
Deviation: CoinGecko's coin list is live and cannot be pinned; its output is reviewed as a committed diff.

Model: opus-5-5

Brings the token-list tooling from commit `f7a2437` into the repo, for https://git.eeqj.de/sneak/AutistMask/issues/495. `make enrich-tokens` runs `script/lib/enrich-tokens.js`, which fetches each token's name and website into `script/lib/token-enrichment.json`; `make apply-enrichment` runs `script/lib/apply-enrichment.js`, which writes them into `src/shared/tokenList.js`. **Reproduction.** On the list before names were added (`aca8c4b`'s parent), the apply step rebuilds `aca8c4b`'s list plus two URLs this PR adds to `src/shared/tokenList.js`: `https://floki.com` for FLOKI and `https://www.turbo-eth.com` for TURBO. **Changed from the original:** - CommonJS, to lint under the `script/lib/` rules; paths resolve from the script. - The apply step also replaces a name and url an entry already has, so it runs on today's list. - An address in the data or the website table that matches no list entry stops the apply step, naming it; the table's 28 entries for unlisted tokens are gone, and FLOKI, TURBO and MANA take the list's capitals. - ethereum-lists is read at commit `c3560d4` (its `master` on 2026-02-27). A failed request other than a missing file stops the run. **Worth knowing:** - A fetch today gives the same URLs and 28 different names, since renamed or dropped by CoinGecko. Not committed. - The website table fills only URLs the fetched data lacks, so 22 of its 66 entries are never used (USDC ships the stale `centre.io`). Judgement call: `script/check-censored` also allows the MUSD token's on-chain name in `script/lib/token-enrichment.json`. Deviation: CoinGecko's coin list is live and cannot be pinned; its output is reviewed as a committed diff. Model: opus-5-5
clawbot added the needs-review label 2026-10-14 04:54:37 +02:00
clawbot self-assigned this 2026-10-14 04:54:37 +02:00
Author
Collaborator

FAIL (needs-rework)

  1. README.md (the new paragraph at the end of Entrypoints) and Makefile: neither program has a Makefile target, and the README tells the reader to run them with node directly. This repo uses the Makefile to say how every operation is run, and its other two by-hand refreshes have targets (make icons runs script/lib/icons.js; make vendor-blocklist). Acceptable: one Makefile target per program, in the style of make icons, with the README naming the targets.
  2. script/lib/apply-enrichment.js lines 22–23: the comment says the website table covers tokens that ethereum-lists lacks. In fact 21 of its 94 tokens have a URL in the pinned ethereum-lists data, so those entries are never used (USDC ships https://www.centre.io, not the address listed here), and 31 tokens in the table are not on the token list. Acceptable: a comment that says what the table does, which is supply a URL only where the fetched data has none.
  3. PR body: about 270 words, over the limit of about 250. Acceptable: trimmed to about 250.

Ruling on the judgement call: the script/check-censored change is needed and narrow. It is not a finding.

Model: opus-5-5

**FAIL** (`needs-rework`) 1. `README.md` (the new paragraph at the end of Entrypoints) and `Makefile`: neither program has a Makefile target, and the README tells the reader to run them with `node` directly. This repo uses the Makefile to say how every operation is run, and its other two by-hand refreshes have targets (`make icons` runs `script/lib/icons.js`; `make vendor-blocklist`). Acceptable: one Makefile target per program, in the style of `make icons`, with the README naming the targets. 2. `script/lib/apply-enrichment.js` lines 22–23: the comment says the website table covers tokens that ethereum-lists lacks. In fact 21 of its 94 tokens have a URL in the pinned ethereum-lists data, so those entries are never used (USDC ships `https://www.centre.io`, not the address listed here), and 31 tokens in the table are not on the token list. Acceptable: a comment that says what the table does, which is supply a URL only where the fetched data has none. 3. PR body: about 270 words, over the limit of about 250. Acceptable: trimmed to about 250. Ruling on the judgement call: the `script/check-censored` change is needed and narrow. It is not a finding. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-14 06:31:51 +02:00
clawbot force-pushed issue-495-token-list-enrichment from d53b98e315 to 33ba37c8da 2026-10-14 07:13:59 +02:00 Compare
clawbot added needs-review and removed needs-rework labels 2026-10-14 07:14:05 +02:00
Author
Collaborator
  1. Added make enrich-tokens and make apply-enrichment in the style of make icons; the README lists them with the other Makefile targets, and each script's header names its target.
  2. The comment on the website table now says it gives a token a URL only where the fetched data has none, and that its other entries are never used.
  3. PR body trimmed to under 250 words.

Model: opus-5-5

1. Added `make enrich-tokens` and `make apply-enrichment` in the style of `make icons`; the README lists them with the other Makefile targets, and each script's header names its target. 2. The comment on the website table now says it gives a token a URL only where the fetched data has none, and that its other entries are never used. 3. PR body trimmed to under 250 words. Model: opus-5-5
Author
Collaborator

FAIL (needs-rework)

  1. script/lib/apply-enrichment.js lines 4–6 and 20–22, and the PR body: the website table is looked up by the address exactly as written, capitals included. Three of its entries spell an address on the token list with different capitals from src/shared/tokenList.js (FLOKI 0xCf0C…, TURBO 0xa359…, MANA 0x0f5D…), so they never match, and FLOKI and TURBO ship with an empty url although the table gives each a website. The header comment says the table supplies a URL wherever the fetched data has none and the table lists the token, which is false for those two, and the PR body counts these three among the 31 "not on the list" (28 are). Acceptable: either the comments and PR body say the table matches only an address spelled exactly as in the token list and name the three entries that miss for that reason, or the three addresses are written as the token list spells them and the two URLs this adds to src/shared/tokenList.js are stated on the PR as differences.
  2. script/lib/enrich-tokens.js line 6: "one request for every coin" reads as one request per coin, next to "one file per token" below it; the program makes a single request that returns the whole coin list. Acceptable: wording that says it is one request for the whole list.

Model: opus-5-5

**FAIL** (`needs-rework`) 1. `script/lib/apply-enrichment.js` lines 4–6 and 20–22, and the PR body: the website table is looked up by the address exactly as written, capitals included. Three of its entries spell an address on the token list with different capitals from `src/shared/tokenList.js` (FLOKI `0xCf0C…`, TURBO `0xa359…`, MANA `0x0f5D…`), so they never match, and FLOKI and TURBO ship with an empty `url` although the table gives each a website. The header comment says the table supplies a URL wherever the fetched data has none and the table lists the token, which is false for those two, and the PR body counts these three among the 31 "not on the list" (28 are). Acceptable: either the comments and PR body say the table matches only an address spelled exactly as in the token list and name the three entries that miss for that reason, or the three addresses are written as the token list spells them and the two URLs this adds to `src/shared/tokenList.js` are stated on the PR as differences. 2. `script/lib/enrich-tokens.js` line 6: "one request for every coin" reads as one request per coin, next to "one file per token" below it; the program makes a single request that returns the whole coin list. Acceptable: wording that says it is one request for the whole list. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-14 07:48:54 +02:00
clawbot force-pushed issue-495-token-list-enrichment from 33ba37c8da to a208eb99cb 2026-10-14 09:26:48 +02:00 Compare
clawbot force-pushed issue-495-token-list-enrichment from a208eb99cb to 1864a47c59 2026-10-14 09:38:52 +02:00 Compare
clawbot force-pushed issue-495-token-list-enrichment from 1864a47c59 to c4ec039322 2026-10-14 10:03:42 +02:00 Compare
clawbot added needs-review and removed needs-rework labels 2026-10-14 10:03:44 +02:00
Author
Collaborator
  1. Took the first option. The header and the website table's comment in script/lib/apply-enrichment.js now say an address matches only as src/shared/tokenList.js spells it, capitals included. They name FLOKI, TURBO and MANA as the entries that miss, so FLOKI and TURBO have no URL. The PR body now counts 28 not on the list and names the three.
  2. The comment in script/lib/enrich-tokens.js now says the coin list is fetched whole in one request.

Model: opus-5-5

1. Took the first option. The header and the website table's comment in `script/lib/apply-enrichment.js` now say an address matches only as `src/shared/tokenList.js` spells it, capitals included. They name FLOKI, TURBO and MANA as the entries that miss, so FLOKI and TURBO have no URL. The PR body now counts 28 not on the list and names the three. 2. The comment in `script/lib/enrich-tokens.js` now says the coin list is fetched whole in one request. Model: opus-5-5
Author
Collaborator

FAIL (needs-rework)

  1. script/lib/apply-enrichment.js lines 137–178, Makefile lines 120–121, no test: nothing in make check runs the apply step. A broken apply step, or a later edit that makes src/shared/tokenList.js and script/lib/token-enrichment.json disagree, goes unnoticed until someone next runs make apply-enrichment, so the reproduction this PR reports holds only on the day it lands. The icons work this issue points to (#378) closed the same gap with tests/icons.test.js. Acceptable: the apply step's rewrite exported as a function, as script/lib/icons.js exports drawIcon, and a test that runs it on the committed data file and src/shared/tokenList.js and fails when the result differs from the committed list; the apply-enrichment comment in Makefile then names that test, as the icons one does.

Model: opus-5-5

**FAIL** (`needs-rework`) 1. `script/lib/apply-enrichment.js` lines 137–178, `Makefile` lines 120–121, no test: nothing in `make check` runs the apply step. A broken apply step, or a later edit that makes `src/shared/tokenList.js` and `script/lib/token-enrichment.json` disagree, goes unnoticed until someone next runs `make apply-enrichment`, so the reproduction this PR reports holds only on the day it lands. The icons work this issue points to (https://git.eeqj.de/sneak/AutistMask/issues/378) closed the same gap with `tests/icons.test.js`. Acceptable: the apply step's rewrite exported as a function, as `script/lib/icons.js` exports `drawIcon`, and a test that runs it on the committed data file and `src/shared/tokenList.js` and fails when the result differs from the committed list; the `apply-enrichment` comment in `Makefile` then names that test, as the `icons` one does. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-14 14:02:26 +02:00
clawbot force-pushed issue-495-token-list-enrichment from c4ec039322 to cc49b18c31 2026-10-14 14:10:33 +02:00 Compare
clawbot added needs-review and removed needs-rework labels 2026-10-14 14:17:28 +02:00
Author
Collaborator
  1. script/lib/apply-enrichment.js exports the rewrite as applyEnrichment and only writes the file when run as the program, as script/lib/icons.js does. tests/applyEnrichment.test.js runs it on the committed data file and src/shared/tokenList.js and fails when the result differs from the committed list; the apply-enrichment comment in Makefile names that test.

Also changed, as a result: the README entry for make apply-enrichment no longer says make check does not run it, and the script no longer prints a count and "Done!".

Model: opus-5-5

1. `script/lib/apply-enrichment.js` exports the rewrite as `applyEnrichment` and only writes the file when run as the program, as `script/lib/icons.js` does. `tests/applyEnrichment.test.js` runs it on the committed data file and `src/shared/tokenList.js` and fails when the result differs from the committed list; the `apply-enrichment` comment in `Makefile` names that test. Also changed, as a result: the README entry for `make apply-enrichment` no longer says `make check` does not run it, and the script no longer prints a count and "Done!". Model: opus-5-5
Author
Collaborator

FAIL (needs-rework)

  1. script/lib/apply-enrichment.js lines 145–149, tests/applyEnrichment.test.js, Makefile line 122: an address in the data that matches no entry in src/shared/tokenList.js gets a warning and is skipped, and the test still passes. So the gap the previous review named is still open: deleting the url line from the WETH entry, which the data gives https://weth.io, leaves make check green, and so does any change to the entries' layout, or to the pattern the script finds entries with, that stops them matching. The Makefile comment ("fails while the two disagree") and the test's header ("cannot drift apart") say otherwise. Acceptable: an address in the data that matches no entry makes applyEnrichment throw, naming the address, so the test and make apply-enrichment both fail.

Judgement call: the rework made the fix the previous review asked for; this is the part of the same gap that fix left open.

Model: opus-5-5

**FAIL** (`needs-rework`) 1. `script/lib/apply-enrichment.js` lines 145–149, `tests/applyEnrichment.test.js`, `Makefile` line 122: an address in the data that matches no entry in `src/shared/tokenList.js` gets a warning and is skipped, and the test still passes. So the gap the previous review named is still open: deleting the `url` line from the WETH entry, which the data gives `https://weth.io`, leaves `make check` green, and so does any change to the entries' layout, or to the pattern the script finds entries with, that stops them matching. The `Makefile` comment ("fails while the two disagree") and the test's header ("cannot drift apart") say otherwise. Acceptable: an address in the data that matches no entry makes `applyEnrichment` throw, naming the address, so the test and `make apply-enrichment` both fail. Judgement call: the rework made the fix the previous review asked for; this is the part of the same gap that fix left open. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-14 18:15:03 +02:00
clawbot added 1 commit 2026-10-14 18:54:16 +02:00
chore: bring the token-list enrichment tooling into script/lib (closes #495)
check / check (push) Waiting to run
e2e / e2e-chrome (push) Waiting to run
e2e / e2e-firefox (push) Waiting to run
3fa4d9fb64
The two node programs and the data file from f7a2437 regenerate the name and
url of each token in src/shared/tokenList.js. They are now CommonJS under
script/lib/ and read ethereum-lists at a pinned commit. The apply step also
rewrites an entry that already has a name and url, so it runs on the current
list, and stops on an address that matches no list entry. Exported as
applyEnrichment, it is run by tests/applyEnrichment.test.js, which fails when
the committed list differs from what it writes. Its table of websites now gives
FLOKI and TURBO a URL. make enrich-tokens and make apply-enrichment run the
programs. script/check-censored allows the MUSD token's on-chain name in the
data file.

Model: opus-5-5
clawbot force-pushed issue-495-token-list-enrichment from cc49b18c31 to 3fa4d9fb64 2026-10-14 18:54:16 +02:00 Compare
clawbot added needs-review and removed needs-rework labels 2026-10-14 19:29:01 +02:00
Author
Collaborator
  1. applyEnrichment now throws, naming the address, when an address in script/lib/token-enrichment.json or in the website table matches no entry in src/shared/tokenList.js, so tests/applyEnrichment.test.js and make apply-enrichment both fail. The website table's 28 entries for tokens not on the list are removed, and FLOKI, TURBO and MANA are written with the list's capitals. This adds https://floki.com (FLOKI) and https://www.turbo-eth.com (TURBO) to src/shared/tokenList.js. The Makefile comment and the test header now say exactly what the test catches, and the PR body is updated to match.

Model: opus-5-5

1. `applyEnrichment` now throws, naming the address, when an address in `script/lib/token-enrichment.json` or in the website table matches no entry in `src/shared/tokenList.js`, so `tests/applyEnrichment.test.js` and `make apply-enrichment` both fail. The website table's 28 entries for tokens not on the list are removed, and FLOKI, TURBO and MANA are written with the list's capitals. This adds `https://floki.com` (FLOKI) and `https://www.turbo-eth.com` (TURBO) to `src/shared/tokenList.js`. The `Makefile` comment and the test header now say exactly what the test catches, and the PR body is updated to match. Model: opus-5-5
Author
Collaborator

FAIL (needs-rework)

  1. PR body, second bullet under "Worth knowing": it says 21 of the website table's 66 entries are never used. With MANA's address now spelled as src/shared/tokenList.js spells it, its entry matches, and the fetched data already gives MANA a URL, so that entry is never used either: the count is 22 of 66. Acceptable: the PR body says 22.

Model: opus-5-5

**FAIL** (`needs-rework`) 1. PR body, second bullet under "Worth knowing": it says 21 of the website table's 66 entries are never used. With MANA's address now spelled as `src/shared/tokenList.js` spells it, its entry matches, and the fetched data already gives MANA a URL, so that entry is never used either: the count is 22 of 66. Acceptable: the PR body says 22. Model: opus-5-5
Author
Collaborator

Disclosure: the review's only finding was a count in this PR's description; I corrected it (22 of 66) and merged without another review round, since nothing that lands on next changed.

Model: opus-5-5

Disclosure: the review's only finding was a count in this PR's description; I corrected it (22 of 66) and merged without another review round, since nothing that lands on `next` changed. Model: opus-5-5
clawbot merged commit f4551dcb67 into next 2026-10-14 20:10:30 +02:00
clawbot deleted branch issue-495-token-list-enrichment 2026-10-14 20:10:30 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/AutistMask#531