Cited by 221Natural Language Processing TechniquesTopic ModelingSpeech Recognition and Synthesis

The DOI matches its Crossref record; 56 of the 62 references it cites were found in a registry or an index, and 6 are not checked yet.

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Open access

Rust, Phillip, Pfeiffer, Jonas, Vulić, Ivan, Ruder, Sebastian, Gurevych, Iryna·2021·Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

DOI 10.18653/v1/2021.acl-long.243

Open this paper in CoChat — free account Open PDF

Abstract

Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.18653/v1/2021.acl-long.243

Cites

56 of 62 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 1×
“Exponential smoothing was used when creating the subword vocabulary based on WordPieces (Wu et al., 2016) and a pretraining corpus.”
Method

Show all 57 references

Research works printed but not linked: 10

Erkin Demirtas and Mykola Pechenizkiy. 2013. Cross- lingual polarity detection with machine translation. In Proceedings of the Second International Work- shop on Issues of Sentiment Discovery and Opinion Mining (WISDOM '13), pages 9:1-8, Chicago, USA. Association for Computing Machinery. · DOI
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Gra- ham Neubig, Orhan Firat, and Melvin Johnson. 2020. XTREME: A massively multilingual multi- task benchmark for evaluating cross-lingual gener- alisation. In Proceedings of the 37th International Conference on Machine Learning, pages 4411-4421, Virtual.
PMLR.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analy- sis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Hu- man Language Technologies, pages 142-150, Port- land, Oregon, USA. Association for Computational Linguistics.
Joakim Nivre, Marie-Catherine de Marneffe, Filip Gin- ter, Yoav Goldberg, Jan Hajič, Christopher D. Man- ning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth In- ternational Conference on Language Resources and Evaluation (LREC'16), pages 1659-1666, Portorož, Slovenia. European Language Resources Associa- tion (ELRA).
Debora Nozza, Federico Bianchi, and Dirk Hovy. 2020. What the [MASK]? Making sense of language- specific BERT models. arXiv preprint.
Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Se- bastian Ruder. 2020c. UNKs Everywhere: Adapt- ing Multilingual Language Models to New Scripts. arXiv preprint.
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Re- search, 21(140):1-67.
Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, and Ayu Purwarianti. 2020. IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Lan- guage Processing, pages 843-857, Suzhou, China. Association for Computational Linguistics.
Yian Zhang, Alex Warstadt, Haau-Sing Li, and Samuel R. Bowman. 2020. When do you need bil- lions of words of pretraining data? arXiv preprint.

Sorted by citations

Items 101–200 of 221

Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations
Berend, G. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.176 · Cited by 6
Cross-lingual dependency parsing for a language with a unique script
Zhou, H., Dakota, D., & Kübler, S. (2024) · Natural language processing. · doi:10.1017/nlp.2024.21 · Cited by 5
Advancing Large Language Models for Low-Resource Languages: A Systematic Review of Pretraining, Adaptation, and Ethical Challenges
Hossain, I., Banik, M., Farid, F. A., Uddin, J., & Karim, H. B. A. (2026) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2026.075507 · Cited by 5
A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systems
Hu, S., Zhou, H., Yuan, M., Gritta, M., Zhang, G., Iacobacci, I., Korhonen, A., & Vulić, I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.422 · Cited by 7
Self-citation
Improving Large Language Model Russian Adaptation with Preliminary Vocabulary Optimization
Tikhomirov, M. M., & Chernyshev, D. (2024) · Lobachevskii Journal of Mathematics · doi:10.1134/s1995080224604120 · Cited by 5
Punctuation Restoration for Singaporean Spoken Languages: English, Malay, and Mandarin
Rao, A., Ho, T.-N., & Siong, C. E. (2022) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9980338 · Cited by 5
Stop filtering: Multi-view attribute-enhanced dialogue learning
Li, Y., Sun, B., Feng, S., & Li, K. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110833 · Cited by 5
Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Training
Marchisio, K., Lewis, P., Chen, Y., & Artetxe, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.338 · Cited by 11
Multilingual Event Extraction from Historical Newspaper Adverts
Borenstein, N., Perez, N. D. S., & Augenstein, I. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.574 · Cited by 5
You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Models
Limisiewicz, T., Malkin, D., & Stanovsky, G. (2023) · Workshop on Research in Computational Linguistic Typology and Multilingual NLP (SIGTYP) · doi:10.18653/v1/2023.sigtyp-1.1 · Cited by 5
A Multi-dimensional Evaluation of Tokenizer-free Multilingual Pretrained Models
Sun, J., Fernandes, P., Wang, X., & Neubig, G. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.128 · Cited by 7
MEDVOC: Vocabulary Adaptation for Fine-tuning Pre-trained Language Models on Medical Text Summarization
Balde, G., Roy, S., Mondal, M., & Ganguly, N. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/683 · Cited by 5
On the ability of monolingual models to learn language-agnostic representationsPreprint
de Souza, L. R., Nogueira, R. F., & Lotufo, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.01942 · Cited by 6
Domain and Language adaptive pre-training of BERT models for Korean-English bilingual clinical text analysis
Jo, E., Cho, E., Lee, Y., Song, S., & Joo, H. J. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03262-7 · Cited by 4
Valuation of AI Companies: Leveraging Large-Language Models and Deep Learning Approaches
Chen, Z. (2024) · International Conference on Computer Technology Applications (ICCTA) · doi:10.1145/3674558.3674601 · Cited by 4
Local Byte Fusion for Neural Machine Translation
Sreedhar, M. N., Wan, X., Cheng, Y., & Hu, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.397 · Cited by 7
Hints on the data for language modeling of synthetic languages with transformers
Zevallos, R., & Bel, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.699 · Cited by 4
MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling
Godey, N., Castagné, R., de la Clergerie, É. V., & Sagot, B. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.207 · Cited by 4
CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models
Minixhofer, B., Pfeiffer, J., & Vulić, I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.24 · Cited by 6
Self-citation
Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing
(2022) · doi:10.18653/v1/2022.deeplo-1 · Cited by 4
Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization
Xu, N., Zhang, Q., Ye, J., Zhang, M., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.931 · Cited by 5
Tokenizers for African Languages
Ndomba, G. E., Mswahili, M. E., & Jeong, Y.-S. (2024) · IEEE Access · doi:10.1109/access.2024.3522285 · Cited by 4
On Generalist and Domain-Specific Music Classification Models and Their Impacts on Brazilian Music Genre Recognition
Silva, D. F., da Silva, A. C. M., Ortolan, L. F., & Marcacini, R. M. (2021) · Simpósio Brasileiro de Computação Musical (SBCM) · doi:10.5753/sbcm.2021.19427 · Cited by 3
Towards robust complexity indices in linguistic typology
Oh, Y. M., & Pellegrino, F. (2022) · Studies in Language · doi:10.1075/sl.22034.oh · Cited by 5
Gradual Language Model Adaptation Using Fine-Grained Typology
Fekete, M. R., & Bjerva, J. (2023) · Workshop on Research in Computational Linguistic Typology and Multilingual NLP (SIGTYP) · doi:10.18653/v1/2023.sigtyp-1.19 · Cited by 3
The Batch Primary Components Transformer and Auto-Plasticity Learning Linear Units Architecture: Synthetic Image Generation Case
Selitskiy, S., Inoue, C., Schetinin, V., & Jakaite, L. (2023) · International Conference on Social Networks Analysis, Management and Security (SNAMS) · doi:10.1109/snams60348.2023.10375471 · Cited by 4
Proceedings of the 5th Workshop on Research in Computational Linguistic Typology and Multilingual NLP
(2023) · doi:10.18653/v1/2023.sigtyp-1 · Cited by 3
From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languages
Kiulian, A., Polishko, A., Khandoga, M., Kostiuk, Y., Gabrielli, G., Gagała, Ł., Zaraket, F., Abu Obaida, Q., Garud, H., Wing Yee Mak, W., et al. (2025) · Proceedings of the Fourth Ukrainian Natural Language Processing Workshop (UNLP 2025) · doi:10.18653/v1/2025.unlp-1.1 · Cited by 2
Test-driving information theory-based compositional distributional semantics: A case study on Spanish song lyrics
Ghajari, A., Benito-Santos, A., Ros, S., Fresno, V., & González‐Blanco, E. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113549 · Cited by 2
Pretraining and evaluation of BERT models for climate research
Poleksić, A., & Martinčić-Ipšić, S. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07740-5 · Cited by 2
Language-Specific Tokenization for Assamese: Efficiency and Downstream Integration with LLMs
Nath, B., & Tamang, S. (2025) · Impact in Computics · doi:10.65500/computics-2025-004 · Cited by 2
Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment
Vemula, S. R., Dandapat, S., Sharma, D. M., & Krishnamurthy, P. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.ijcnlp-srw.20 · Cited by 3
Multilingual Large Language Models for Low-Resource Languages: Challenges and Opportunities
Khaldy, M. A. A., Gheraibia, Y., & Shaheen, A. (2026) · International Conference on Computational Intelligence Approaches and Applications (ICCIAA) · doi:10.1109/icciaa68481.2026.11543773 · Cited by 2
Federated Learning-Based Tokenizer for Domain-Specific Language Models in Finance
Damoun, F., Seba, H., & State, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78538-2_1 · Cited by 3
A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models
Zhou, Y. X., Camacho-Collados, J., & Bollegala, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.683 · Cited by 5
You Are What You Write: Author re-identification privacy attacks in the era of pre-trained language models
Plant, R., Giuffrida, V., & Gkatzia, D. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101746 · Cited by 2
Resource-Aware Arabic LLM Creation: Model Adaptation, Integration, and Multi-domain Testing
Aryan, P. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-83793-7_27 · Cited by 2
From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences
Kodali, P., Goel, A., Asapu, L., Bonagiri, V. K., Govil, A., Choudhury, M., Kumaraguru, P., & Shrivastava, M. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3748312 · Cited by 7
Infusing clinical knowledge into language models by subword optimisation and embedding initialisation
Hasan, A., Wu, J., Nguyen, Q., Andres, S., Guellil, I., Zhang, H., Casey, A., Alex, B., Guthrie, B., & Wu, H. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110747 · Cited by 2
Exploring the Impact of Transliteration on NLP Performance: Treating Maltese as an Arabic Dialect
Micallef, K., Eryani, F., Habash, N. Y., Bouamor, H., & Borg, C. (2023) · doi:10.18653/v1/2023.cawl-1.4 · Cited by 2
Benchmarking State of the Art Website Embedding Methods for Effective Processing and Analysis in the Public SectorPreprint
Gerber, J., Saxer, J., Kreiner, B., & Weiler, A. (2025) · Research Square · doi:10.21203/rs.3.rs-5664280/v1 · Cited by 2
Counterfactually Probing Language Identity in Multilingual Models
Srinivasan, A., Govindarajan, V. S., & Mahowald, K. (2023) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2023.mrl-1.3 · Cited by 2
A Comparative Evaluation of Transformer-Based Models for Abstractive Turkish News Summarization
Gülsoy, M., & Akbulut, A. (2025) · International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap68205.2025.11222238 · Cited by 1
Morpheme-Aware Hybrid Sentiment Analysis for Azerbaijani: A Lexicon–Embedding Fusion With Domain Adaptation
Aykaç, Y. E., Ozkan, M., & Samet, R. (2026) · IEEE Access · doi:10.1109/access.2026.3659042 · Cited by 1
Impact of Query Language on the Structure and Guideline Alignment of AI-Generated Rehabilitation Programs in Chronic Kidney Disease
Bezruk, V. V., Ivanov, D. D., Ivanchuk, M., Shkrobanets, І. D., Makarova, O. V., Rynzhuk, L. V., Bulyk, T. S., Maksymiv, О. О., & Ivanova, M. D. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15114218 · Cited by 1
Information Parity: Measuring and Predicting the Multilingual Capabilities of Language Models
Tsvetkov, A., & Kipnis, A. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.468 · Cited by 2
Evaluation of Local Explainability Methods in Turkish Text Classification Tasks
Porsuk, O., Yıldırım, S., & Başar, A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-92602-0_30 · Cited by 1
CrammedBERTurk: Pretraining/Finetuning a New Language Model for Turkish Question Answering on Limited Budget
Sarıhan, T., & Ergün, C. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3780096 · Cited by 1
The Capabilities of Large Language Models in Extracting Unstructured Data From Histopathology Reports
Adamson, S., Berry, C., Adler, N. R., Christian, T., Librata, W., & Mar, V. J. (2025) · Australasian Journal of Dermatology · doi:10.1111/ajd.70034 · Cited by 1
Exploring Cross-Lingual Knowledge Transfer via Transliteration-Based MLM Fine-Tuning for Critically Low-resource Chakma Language
Khisa, A., Lia, N. J., Nafis, T. M., Masud, Z., Pial, T., Rayana, S., & Kabir, A. (2025) · Workshop on Bangla Language Processing (BLP) · doi:10.18653/v1/2025.banglalp-1.23 · Cited by 1
Beyond the Tokenization Bottleneck: A Conceptual Framework for Efficiency-Plasticity Trade-Offs in ASEAN Large Language Model Adaptation
Sheng, C. Y., Xin, C. Y., & Jie, T. G. (2026) · Cureus Journal of Computer Science. · doi:10.7759/s44389-026-00047-5 · Cited by 1
Understanding bias in language models: comparing the cultural distributions of basic colour terms in English, German, Japanese, and Russian
Stillman, M., Kruspe, A., Kirjakovski, A., & Marutschke, D. M. (2026) · Journal of Cultural Cognitive Science · doi:10.1007/s41809-026-00198-w · Cited by 1
Text-to-SQL for Korean Language based on Multilingual BERT
Yoon, H., Heo, J.-H., Kim, J. S., & Kang, P. (2022) · Journal of Korean Institute of Industrial Engineers · doi:10.7232/jkiie.2022.48.1.091 · Cited by 1
Analyzing the Impact of Tokenization on Multilingual Epidemic Surveillance in Low-Resource Languages
Mutuvi, S., Boroş, E., Doucet, A., Lejeune, G., Jatowt, A., & Odeo, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41682-8_2 · Cited by 1
To Split or Not to Split: Composing Compounds in Contextual Vector Spaces
Jenkins, C., Miletić, F., & Walde, S. S. I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1002 · Cited by 1
Creating, anonymizing and evaluating the first medical language model pre-trained on Dutch Electronic Health Records: MedRoBERTa.nl
Verkijk, S., & Vossen, P. T. J. M. (2025) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2025.103148 · Cited by 2
On Language Spaces, Scales and Cross-Lingual Transfer of UD Parsers
Samardżić, T., Gutierrez-Vasques, X., van der Goot, R., Müller-Eberstein, M., Pelloni, O., & Plank, B. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.18 · Cited by 1
The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations
Soler, A. G., Labeau, M., & Clavel, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00647 · Cited by 6
MergeDistill: Merging Pre-trained Language Models using DistillationPreprint
Khanuja, S., Johnson, M., & Talukdar, P. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02834 · Cited by 1
Wine is Not v i n. -- On the Compatibility of Tokenizations Across LanguagesPreprint
Moens (Ed.), M.-F., Huang (Ed.), X., Specia (Ed.), L., & Yih (Ed.), S. W.-T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05772 · Cited by 1
Whither developmental psycholinguistics?Preprint
Gomes, V. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/zm57t · Cited by 1
Epidemic Event Extraction in Multilingual and Low-resource Settings
Mutuvi, S. (2022) · theses.fr (ABES) · doi:10.70675/82e91446zf499z4438zaaa7z66548ffb98b0 · Cited by 1
Writing-System-Level Tokenizer Adaptation for Byte-Level BPE
Didenko, B. (2026) · arXiv · doi:10.48550/arxiv.2608.00582
“…fertility (tokens per word) in underrepresented languages: the unmodified Nemotron/GPT-OSS tokenizers require 2.71/2.60 tokens per Ukrainian word versus 1.32/1.26 for English (Table 6), increasing inference cost and compressing effective context windows (Rust et al., 2021; Petrov et al., 2024).”
Influential
MDAPT: Multilingual Domain Adaptive Pretraining in a Single ModelPreprint
Jørgensen, R. K., Hartmann, M., Dai, X., & Elliott, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06605
Medical BigBERTa: An Optimized Transformer for Long Medical DocumentPreprint
Mugisha, C., & Paik, I. (2022) · doi:10.36227/techrxiv.21673598.v1
Medical BigBERTa: An Optimized Transformer for Long Medical DocumentPreprint
Mugisha, C., & Paik, I. (2022) · doi:10.36227/techrxiv.21673598
Medical BigBERTa: An Optimized Transformer for Long Medical DocumentPreprint
Mugisha, C., & Paik, I. (2022) · doi:10.36227/techrxiv.21673598.v2
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks
Geigle, G., Liu, C., Pfeiffer, J., & Gurevych, I. (2023) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2023.repl4nlp-1.9
Self-citation
Revealing Weaknesses of Vietnamese Language Models Through Unanswerable Questions in Machine Reading Comprehension
Tran, S. Q., Nguyen-Thuan, P., Nguyen, K. V., & Nguyen, N. L.-T. (2023) · Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (EACL SRW) · doi:10.18653/v1/2023.eacl-srw.1
Bridging the Gap between Subword and Character Segmentation in Pretrained Language Models
Kiyono, S., Takase, S., Li, S., & Sato, T. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_062
Text Classification of German Mental Healthcare Data: A Hands-on Tutorial on Natural Language Processing Techniques.Preprint
Hornstein, S., Hilbert, K., Ferizaj, D., Zantvoort, K., Lueken, U., & Funk, B. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/5xch6
Towards Developing an Automatic Punctuation Prediction Model for Bangla Language: A Pre-trained Mono-lingual Transformer-based Approach
Iktidar, A., & Murad, H. (2024) · International Conference on Electrical Engineering and Information Communication Technology · doi:10.1109/iceeict62016.2024.10534556
BTVD-BERT: A Bilingual Domain-Adaptation Pre-Trained Model for Textural Vulnerability Descriptions
Wang, Z., Liang, X., Du, R., & Zhou, X. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831823
Optimal word order for non-causal text generation with Large Language Models: The Spanish case
Busto-Castiñeira, A., García-Méndez, S., de Arriba-Pérez, F., & González-Castaño, F. J. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.02.010
A Turkish Word Frequency Tool: LexiTR Frequency
Sezer, T., & Karadağ, Ö. (2025) · Ana Dili Eğitimi Dergisi · doi:10.16916/aded.1636416
Beyond Dataset Watermarking: Model-Level Copyright Protection for Code Summarization Models
Zhang, J., Li, H., Wu, D., Sun, X., Lu, Q., & Long, G. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714641
Multilingual Tokenization Efficiency in Large Language Models: A Study on Indian LanguagesPreprint
Mohamed, A., & Dhamodharan, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5201887 · Cited by 1
Don’t stop pretraining! Efficiently building specialised language models in resource-constrained settings.
Najem-Meyer, S., Kaplan, F., & Romanello, M. (2025) · Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL) · doi:10.18653/v1/2025.latechclfl-1.22 · Cited by 1
Understanding and Simplifying Text with Generative AI: A Performance Comparison
Shoeibi, N., García‐Holgado, A., García‐Peñalvo, F. J., & Therón, R. (2025) · Lecture notes in educational technology · doi:10.1007/978-981-96-5658-5_19
Tokenization Efficiency in Code-Switched Text: Comparing SentencePiece and Byte-Pair Encoding on TaglishPreprint
Salve, H. N., & Tubil, P. N. T. (2025) · doi:10.36227/techrxiv.175756338.86755792/v1 · Cited by 1
Improving Information Retrieval in Turkish: A ColBERT-Based Approach
Saoud, A., Kazerooni, P., Amasyalı, M. F., & Kesgin, H. T. (2025) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-031-88999-8_7
PERFORMANCE ANALYSIS OF MULTILINGUAL AND MONOLINGUAL MODELS IN PREDICTING INDONESIAN LANGUAGE EMOTION USING TWITTER DATASET
Paramarta, M. M. A., Dwiyansaputra, R., & Rassy, R. P. (2025) · Jurnal Teknologi Informasi Komputer dan Aplikasinya (JTIKA ) · doi:10.29303/jtika.v7i2.482 · Cited by 1
Learning Rate Sensitivity of Multilingual Transformer Based Large Language Models in Turkish Binary Classification
Alav, S., Daş, B., & Benli, K. S. (2025) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk67458.2025.11207074
Large Language Models in Drug Discovery
Gangwal, A., & Lavecchia, A. (2026) · Springer eBooks · doi:10.1007/978-3-031-98022-0_14
Judge the Votes: a System to Classify Bug Reports and Give Suggestions
Dinç, E., & Tüzün, E. (2025) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1109/aiware69974.2025.00025
Continual Pretraining of a Small Language Model on Cuban Spanish Corpora
Estevanell-Valladares, E. L., Estévez-Velarde, S., Piad-Morffis, A., Almeida-Cruz, Y., Varela, A. B., Valera, C. S. P., Pérez, D. A. V., Rodríguez, D. F., Horta, E. R., Rodríguez, G. H., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-11358-0_11
BEAST: BYTE-WISE EMBEDDING ARCHITECTURE FOR SEMANTIC TRANSFERPreprint
Palomo-Alonso, A., Jiménez-Fernández, S., Portilla-Figueras, A., & Salcedo-Sanz, S. (2026) · doi:10.36227/techrxiv.177220385.59586322/v1
HelBERT: A BERT-Based Pretraining Model for Public Procurement Tasks in Portuguese
Lima, W. E. M., Silva, V. R. D., da Silva, J. C., Rabêlo, R. A. L., & de Paiva, A. C. (2026) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2026.5511
Encoding Fidelity and Coherent Misalignment in Non-English Clinical AIPreprint
M, L. M. (2026) · Preprints.org · doi:10.20944/preprints202604.0061.v1
Neural methods for spoken dialogue understanding
Chapuis, É. (2021) · theses.fr (ABES) · doi:10.70675/1fc9d449zab9az41c9zaa43z5f3ca7ec8ba4
Improving Representations for Language Modeling
Godey, N. (2024) · theses.fr (ABES) · doi:10.70675/b6a15d0ez15bez43e5za967z16a069e0a8bb
Understanding and Evaluating Unsupervised Cross-lingual Embeddings in the General and in the Clinical Domains
Gaschi, F. (2023) · theses.fr (ABES) · doi:10.70675/134784a8z1491z4726za4a2z2e367ad63c88
Small is Beautiful : addressing resource scarcity, language variation, and transfer challenges for automatic detection of Harmful language
Riabi, A. (2025) · theses.fr (ABES) · doi:10.70675/5a649ef5za082z43d8z906azd8051be91d15
LLMic: Building a Romanian Foundation Language Model
Bădoiu, V.-A., Gherghescu, A. M., Agache, A., Dumitru, M.-V., & Raiciu, C. (2025) · International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc69064.2025.00031
Consumer chatbots gave similarly empathic answers whether safe or unsafe: a physician-rated evaluation in six languagesPreprint
Ariel, D., Grumberg, L. R., Supakul, S., Wannasri, S., Mitchnik, I. Y., Lev, A., Ariyamethanon, W., Agbarieh, M., Miari, S., Laban, G., et al. (2026) · medRxiv · doi:10.64898/2026.05.09.26352813
Global vs. Regional: A Comparative Measurement of Token Efficiency in State-of-the-Art Large Language Models
Zhang, W., Du, Q., Feng, Z., Fan, Z., Tian, F., Zhou, Y., & Tao, Y. (2026) · International Conference on Generative Artificial Intelligence and Information Security (GAIIS) · doi:10.1109/gaiis69281.2026.11519207
Mathematical Foundations of Cross-Lingual Vulnerabilities in LLMs: Latent Space Entanglement and Token Fragmentation
Hasan, U., & Nayeem, M. A. (2026) · Mathematics · doi:10.3390/math14111849
The analysis of customized tokenizer for the development of isiNdebele language model
Malatji, P. T., & Modipa, T. I. (2026) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai68641.2026.11536371
Extending a Vision–Language Model with Audio Understanding: Introducing Qolda-AVL for the Kazakh Language
Arystanbekov, B., Maxutov, A., Nurimanov, A., & Varol, H. A. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10060192
Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet
Miletić, M., Kallini, J., & Shutova, E. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1872

Authors: Rust, P., Pfeiffer, J., Vulić, I., Ruder, S., & Gurevych, I.
Affiliations: Technische Universität Darmstadt (DE)
University of Cambridge (GB)
ORCID: Rust, Phillip
Pfeiffer, Jonas
Vulić, Ivan
Gurevych, Iryna
Venue: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Publisher: Association for Computational Linguistics
Published: 2021
Pages: 3118-3135
DOI: 10.18653/v1/2021.acl-long.243
OpenAlex: W3120253119
Cited by: 202
References: 57
Citations by year: 2026: 32 · 2025: 32 · 2024: 22 · 2023: 50 · 2022: 35 · 2021: 31
FWCI: 15.94
Type: conference paper
Language: en
Open access: gold
doi.org
Licence: cc-by
Locations: arXiv (Cornell University) (submittedVersion)
TUbilio (Technical University of Darmstadt) (submittedVersion, cc-by-nc-sa)
arXiv (Cornell University)
Funders: H2020 European Research Council (648909)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Keywords: tokenization, multilingual language models, multilingual tokenization, language model evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks