Huang Xin
c2ac207945
"Word Wise" is a Kindle trademark, so rename the inline-gloss feature to
"Word Lens" throughout the product.
- User-facing strings → "Word Lens" across all 34 locales; brand translated
for Chinese (zh-CN 单词透镜, zh-TW 單詞透鏡) and German (Word-Lens-Daten).
- Code identifiers: WordWise→WordLens, wordWise→wordLens, WORD_WISE→WORD_LENS.
- Files/dirs: src/services/wordwise→wordlens, WordWisePanel→WordLensPanel,
wordwise{Ruby,Section}.ts, build/sync scripts, test dirs/fixtures,
data/wordwise→data/wordlens.
- Storage paths: CDN base, R2 key, on-device cache dir, WORDLENS_R2_BUCKET env,
pnpm wordlens:{manifest,sync}. manifest.json is path-agnostic so its
sha256/bytes stay valid (verified).
- biome.json: point the formatter-ignore at data/wordlens so the generated
one-line gloss packs aren't pretty-printed on commit.
Migration notes:
- Re-run `pnpm wordlens:sync` to upload packs to cdn.readest.com/wordlens/.
- Persisted view-settings keys renamed (wordWiseEnabled/Level/HintLang and
wordWiseAutoDownload) — saved values reset to defaults once on upgrade.
- Cached packs under the old Data/wordwise/ orphan (harmless re-download).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2.8 KiB
Word Lens data attributions
Note: The committed
en-zh.json(top 30,000 by COCA frequency) andzh-en.json(top 12,000, HSK-ranked) are frequency-trimmed derivatives generated from the datasets below viascripts/build-wordlens-data.mjs. The full source corpora (ECDICT ~66 MB, CC-CEDICT) are not committed — regenerate with the commands below. Glosses are cleaned (POS tags,[…]annotations, and CC-CEDICTCL:clauses stripped) and capped to the first 1–2 short senses.
The en↔中文 packs use dedicated dictionaries (highest quality there); all other pairs (es/fr/de/pt/it/ru ↔ en) use the lightweight WikDict + FrequencyWords stack.
Sources
- ECDICT (English→中文 glosses, frequency ranks, inflections) — MIT License. https://github.com/skywind3000/ECDICT
- CC-CEDICT (中文→English glosses) — CC-BY-SA 4.0. https://cc-cedict.org/
- HSK vocabulary levels (中文 difficulty ranking) — open dataset. https://github.com/drkameleon/complete-hsk-vocabulary
- WikDict (bilingual glosses for es/fr/de/pt/it/ru ↔ en) — CC-BY-SA 3.0, derived from DBnary / Wiktionary. https://www.wikdict.com/
- FrequencyWords (difficulty ranking; word-frequency lists from OpenSubtitles/OPUS) — CC-BY-SA 4.0. https://github.com/hermitdave/FrequencyWords
- lemmatization-lists (form→lemma mappings used to lemmatize non-English source
words, e.g. Spanish
corriendo→correr) — compiled from Wiktionary + open lemma data. https://github.com/michmech/lemmatization-lists (English form→lemma comes from ECDICT'sexchangefield instead.)
The committed *.json packs are frequency-trimmed derivatives generated by
scripts/build-wordlens-data.mjs. Redistribution complies with the above licenses; the
CC-BY-SA packs (everything except en-zh.json) carry their license + attribution in each
pack's meta. Per-pack sizes: 0.3–2.6 MB.
Regenerating the assets (maintainer step)
# English → 中文 (ECDICT csv) and 中文 → English (CC-CEDICT + HSK)
node scripts/build-wordlens-data.mjs en-zh /path/to/ecdict.csv 30000
node scripts/build-wordlens-data.mjs zh-en /path/to/cedict.txt /path/to/hsk.json 12000
# Any other pair (one side English) from WikDict + FrequencyWords. Needs the `sqlite3`
# CLI. Download <pair>.sqlite3 from https://download.wikdict.com/dictionaries/sqlite/2/
# and <src>_50k.txt from FrequencyWords/content/2018/<src>/.
node scripts/build-wordlens-data.mjs build-wikdict es en es_50k.txt es-en.sqlite3 20000
node scripts/build-wordlens-data.mjs build-wikdict en es en_50k.txt en-es.sqlite3 20000
# (repeat for fr/de/pt/it/ru ↔ en)
# For maximum coverage, the kaikki Wiktionary dump can be used instead of WikDict:
node scripts/build-wordlens-data.mjs build es en es_50k.txt /path/to/es-extract.jsonl 20000