The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languages
Open accessAbstract
In this paper, we propose a model-agnostic cost-effective approach to developing bilingual base large language models (LLMs) to support English and any target language. The method includes vocabulary expansion, initialization of new embeddings, model training and evaluation. We performed our experiments with three languages, each using a non-Latin script - Ukrainian, Arabic, and Georgian. Our approach demonstrates improved language performance while reducing computational costs. It mitigates the disproportionate penalization of underrepresented languages, promoting fairness and minimizing adverse phenomena such as code-switching and broken grammar. Additionally, we introduce new metrics to evaluate language quality, revealing that vocabulary size significantly impacts the quality of generated text.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.07082 · cited 2×
“In particular, we have used XML-based Ukrainian NER model 4 , SpaCy (Honnibal et al., 2020) uk_core_news_lg 5 model, and Stanza (Qi et al., 2020) Ukrainian model.”
Cited by
Sorted by citations
Writing-System-Level Tokenizer Adaptation for Byte-Level BPE
Didenko, B. (2026) · arXiv · doi:10.48550/arxiv.2608.00582
“For SentencePiece-based models, Kiulian et al. (2025) demonstrate fixed-vocabulary reallocation for Ukrainian, Arabic, and Georgian via score-reassigned merging.”
Metadata
- Authors
- Kiulian, A., Polishko, A., Khandoga, M., Kostiuk, Y., Gabrielli, G., Gagała, Ł., Zaraket, F., Obaida, Q. A., Garud, H., Mak, W. W. Y., Chaplynskyi, D., Amor, S. B., & Peradze, G.
- Affiliations
- University of Dundee (GB)
- Doha Institute for Graduate Studies (QA)
- Google (United States) (US)
- Ukrainian Catholic University (UA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 1-13
- arXiv
- 2410.18836
- OpenAlex
- W4412944228
- Cited by
- 2
- References
- 1
- Citations by year
- 2026: 1 · 2025: 1
- FWCI
- 1.90
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Ukrainian Natural Language Processing Workshop (UNLP) (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- large language models, underrepresented languages, low-resource languages, bilingual language models, multilingual NLP
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
