The DOI matches its Crossref record; 56 of the 62 references it cites were found in a registry or an index, and 6 are not checked yet.
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models
Open accessAbstract
Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“increasing inference cost and compressing effective context windows (Rust et al., 2021; Petrov et al., 2024).”
From Writing-System-Level Tokenizer Adaptation for Byte-Level BPE (2026) · cites this paper as Rust et al., 2021
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 1×
“Exponential smoothing was used when creating the subword vocabulary based on WordPieces (Wu et al., 2016) and a pretraining corpus.”
Method
Research works printed but not linked: 10
- Erkin Demirtas and Mykola Pechenizkiy. 2013. Cross- lingual polarity detection with machine translation. In Proceedings of the Second International Work- shop on Issues of Sentiment Discovery and Opinion Mining (WISDOM '13), pages 9:1-8, Chicago, USA. Association for Computing Machinery. · DOI
- Junjie Hu, Sebastian Ruder, Aditya Siddhant, Gra- ham Neubig, Orhan Firat, and Melvin Johnson. 2020. XTREME: A massively multilingual multi- task benchmark for evaluating cross-lingual gener- alisation. In Proceedings of the 37th International Conference on Machine Learning, pages 4411-4421, Virtual.
- PMLR.
- Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analy- sis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Hu- man Language Technologies, pages 142-150, Port- land, Oregon, USA. Association for Computational Linguistics.
- Joakim Nivre, Marie-Catherine de Marneffe, Filip Gin- ter, Yoav Goldberg, Jan Hajič, Christopher D. Man- ning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal Dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth In- ternational Conference on Language Resources and Evaluation (LREC'16), pages 1659-1666, Portorož, Slovenia. European Language Resources Associa- tion (ELRA).
- Debora Nozza, Federico Bianchi, and Dirk Hovy. 2020. What the [MASK]? Making sense of language- specific BERT models. arXiv preprint.
- Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Se- bastian Ruder. 2020c. UNKs Everywhere: Adapt- ing Multilingual Language Models to New Scripts. arXiv preprint.
- Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Re- search, 21(140):1-67.
- Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, and Ayu Purwarianti. 2020. IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Lan- guage Processing, pages 843-857, Suzhou, China. Association for Computational Linguistics.
- Yian Zhang, Alex Warstadt, Haau-Sing Li, and Samuel R. Bowman. 2020. When do you need bil- lions of words of pretraining data? arXiv preprint.
Cited by
Sorted by citations
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language ProcessingPreprint
Kalyan, K. S., Rajasekharan, A., & Sangeetha, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.05542 · Cited by 357
MEGA: Multilingual Evaluation of Generative AI
Ahuja, K., Diddee, H., Hada, R., Ochieng, M., Ramesh, K., Jain, P., Nambi, A. U., Ganu, T., Segal, S., Ahmed, M., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.258 · Cited by 321
The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language ModelsPreprint
Inoue, G., Alhafni, B., Baimukan, N., Bouamor, H., & Habash, N. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06678 · Cited by 224
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPPreprint
Mielke, S. J., Alyafeai, Z., Salesky, E., Raffel, C., Dey, M., Gallé, M., Raja, A., Si, C., Lee, W. Y., Sagot, B., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.10508 · Cited by 328
KLUE: Korean Language Understanding EvaluationPreprint
Park, S., Moon, J., Kim, S., Cho, W. I., Han, J., Park, J., Song, C., Kim, J., Song, Y., Oh, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09680 · Cited by 193
(2021) · doi:10.1145/3462757.3466088 · Cited by 151
mGPT: Few-Shot Learners Go Multilingual
Shliazhko, O., Fenogenova, A., Tikhonova, M., Kozlova, A., Mikhailov, V., & Shavrina, T. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00633 · Cited by 183
Zhang, X., Malkov, Y. A., Florez, O. U., Park, S., McWilliams, B., Han, J., & El-Kishky, A. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599921 · Cited by 116
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts
Pfeiffer, J., Vulić, I., Gurevych, I., & Ruder, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.800 · Cited by 107
Self-citationManias, G., Mavrogiorgou, A., Kiourtis, A., Symvoulidis, C., & Kyriazis, D. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08629-3 · Cited by 94
What to Pre-Train on? Efficient Intermediate Task Selection
Poth, C., Pfeiffer, J., Rücklé, A., & Gurevych, I. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.827 · Cited by 92
Self-citationLifting the Curse of Multilinguality by Pre-training Modular Transformers
Pfeiffer, J., Goyal, N., Lin, X., Li, X., Cross, J., Riedel, S., & Artetxe, M. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.255 · Cited by 110
Self-citationDeep lexical hypothesis: Identifying personality structure in natural language.
Cutler, A., & Condon, D. M. (2022) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000443 · Cited by 71
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D. R., Smith, N. A., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.614 · Cited by 145
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
Ansell, A., Ponti, E. M., Pfeiffer, J., Ruder, S., Glavaš, G., Vulić, I., & Korhonen, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.410 · Cited by 68
Self-citationmedBERT.de: A comprehensive German BERT model for the medical domain
Bressem, K. K., Papaioannou, J.-M., Grundmann, P., Borchert, F., Adams, L. C., Liu, L., Busch, F., Xu, L., Loyen, J. P., Niehues, S. M., et al. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121598 · Cited by 77
Razumovskaia, E., Glavaš, G., Majewska, O., Ponti, E. M., Korhonen, A., & Vulić, I. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13083 · Cited by 65
Self-citationXLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models
Liang, D., Gonen, H., Mao, Y., Hou, R., Goyal, N., Ghazvininejad, M., Zettlemoyer, L., & Khabsa, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.813 · Cited by 89
xGQA: Cross-Lingual Visual Question Answering
Pfeiffer, J., Geigle, G., Kamath, A., Steitz, J.-M. O., Roth, S., Vulić, I., & Gurevych, I. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.196 · Cited by 67
Self-citationCLIPPO: Image-and-Language Understanding from Pixels Only
Tschannen, M., Mustafa, B., & Houlsby, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01059 · Cited by 65
A Current Review of Generative AI in Medicine: Core Concepts, Applications, and Current Limitations
Rouzrokh, P., Khosravi, B., Faghani, S., Moassefi, M., Shariatnia, M. M., Rouzrokh, P., & Erickson, B. J. (2025) · Current Reviews in Musculoskeletal Medicine · doi:10.1007/s12178-025-09961-y · Cited by 58
Zhao, J., Huang, S., & Cole, J. M. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01259 · Cited by 56
Şahin, G. G. (2021) · Computational Linguistics · doi:10.1162/coli_a_00425 · Cited by 50
Smadi, B. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107921 · Cited by 50
A Representative Study on Human Detection of Artificially Generated Media Across Countries
Frank, J., Herbert, F., Ricker, J., Schönherr, L., Eisenhofer, T., Fischer, A., Dürmuth, M., & Holz, T. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00159 · Cited by 66
Minixhofer, B., Paischer, F., & Rekabsaz, N. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.293 · Cited by 69
Introduction to Neural Transfer Learning With Transformers for Social Science Text Analysis
Wankmüller, S. (2022) · Sociological Methods & Research · doi:10.1177/00491241221134527 · Cited by 45
MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs
Jumelet, J., Weissweiler, L., Nivre, J., & Bisazza, A. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.600 · Cited by 66
Training-free Lexical Backdoor Attacks on Language Models
Huang, Y., Zhuo, T. Y., Xu, Q., Hu, H., Yuan, X., & Chen, C. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583348 · Cited by 34
XLM-T: A Multilingual Language Model Toolkit for Twitter.Preprint
Barbieri, F., Espinosa-Anke, L., & Camacho-Collados, J. (2021) · arXiv (Cornell University) · Cited by 31
Turkish abstractive text summarization using pretrained sequence-to-sequence models
Baykara, B., & Güngör, T. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000195 · Cited by 34
Language Modelling with PixelsPreprint
Rust, P., Lotz, J. F., Bugliarello, E., Salesky, E., de Lhoneux, M., & Elliott, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.06991 · Cited by 58
Self-citationPractical Transformer-based Multilingual Text Classification
Wang, C., & Banko, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-industry.16 · Cited by 29
Improving Low-Resource Languages in Pre-Trained Multilingual Language Models
Hangya, V., Saadi, H. S., & Fraser, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.822 · Cited by 32
AVocaDo: Strategy for Adapting Vocabulary to Downstream Domain
Hong, J., Kim, T., Lim, H., & Choo, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.385 · Cited by 26
Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold
Ruder, S., Vulić, I., & Søgaard, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.184 · Cited by 29
Self-citationExploring electric vehicle adoption in Indonesia using zero-shot aspect-based sentiment analysis
Adi, N. S., & Widianto, S. (2024) · Sustainable Operations and Computers · doi:10.1016/j.susoc.2024.08.002 · Cited by 26
A Comprehensive Analysis of Various Tokenizers for Arabic Large Language Models
Qarah, F., & Alsanoosy, T. (2024) · Applied Sciences · doi:10.3390/app14135696 · Cited by 26
Pre-trained transformer-based language models for Sundanese
Wongso, W., Lucky, H., & Suhartono, D. (2022) · Journal Of Big Data · doi:10.1186/s40537-022-00590-7 · Cited by 26
Large-Language-Model-Enabled Text Semantic Communication Systems
Wang, Z., Zou, L., Wei, S., Li, K., Liao, F., Mi, H., & Lai, R. (2025) · Applied Sciences · doi:10.3390/app15137227 · Cited by 35
LexFit: Lexical Fine-Tuning of Pretrained Language Models
Vulić, I., Ponti, E. M., Korhonen, A., & Glavaš, G. (2021) · doi:10.18653/v1/2021.acl-long.410 · Cited by 27
Self-citationBERT for Natural Language Processing in Bahasa Indonesia
Sebastián, D., Purnomo, H. D., & Sembiring, I. (2022) · International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA) · doi:10.1109/icicyta57421.2022.10038230 · Cited by 25
Tan, Y., Li, M., Zhou, Z., Tan, P., Yu, H., Fan, G., & Liang, H. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00884-3 · Cited by 22
Facilitating Large Language Model Russian Adaptation with Learned Embedding Propagation
Tikhomirov, M. M., & Chernyshev, D. (2024) · Journal of language and Education · doi:10.17323/jle.2024.22224 · Cited by 22
Deep Lexical Hypothesis: Identifying personality structure in natural languagePreprint
Cutler, A., & Condon, D. M. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/gdm5v · Cited by 19
Thin, D. V., Hao, D. N., & Nguyen, N. L.-T. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3589131 · Cited by 21
Philippy, F., Guo, S., & Haddadan, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.323 · Cited by 36
Comparing Biases and the Impact of Multilingual Training across Multiple Languages
Levy, S., John, N. A., Liu, L., Vyas, Y., Ma, J., Fujinuma, Y., Ballesteros, M., Castelli, V., & Roth, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.634 · Cited by 18
SPOK: tokenizing geographic space for enhanced spatial reasoning in GeoAI
Fei, H., Lv, J., Li, G., & Yue, Y. (2025) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2025.2497810 · Cited by 19
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models
Ahuja, K., Kumar, S., Dandapat, S., & Choudhury, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.374 · Cited by 22
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages
Patil, V., Talukdar, P. P., & Sarawagi, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.18 · Cited by 21
Code-switched inspired losses for spoken dialog representations
Colombo, P., Chapuis, É., Labeau, M., & Clavel, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.656 · Cited by 20
The impact of tokenizer selection in genomic language models
Lindsey, L. M., Pershing, N. L., Habib, A. K. M. R. R., Dufault‐Thompson, K., Stephens, W. Z., Blaschke, A. J., Jiang, X., & Sundar, H. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf456 · Cited by 20
What do tokens know about their characters and how do they know it?
Kaushal, A., & Mahowald, K. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.179 · Cited by 16
Introducing UberText 2.0: A Corpus of Modern Ukrainian at Scale
Chaplynskyi, D. (2023) · Ukrainian Natural Language Processing Workshop (UNLP) · doi:10.18653/v1/2023.unlp-1.1 · Cited by 16
Impact of Tokenization on LLaMa Russian Adaptation
Tikhomirov, M., & Chernyshev, D. (2023) · Ivannikov ISPRAS Open Conference (ISPRAS) · doi:10.1109/ispras60948.2023.10508177 · Cited by 17
Specializing Multilingual Language Models: An Empirical Study
Chau, E. C., & Smith, N. A. (2021) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2021.mrl-1.5 · Cited by 18
Languages Through the Looking Glass of BPE Compression
Gutierrez-Vasques, X., Bentz, C., & Samardżić, T. (2023) · Computational Linguistics · doi:10.1162/coli_a_00489 · Cited by 22
Where’s the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation
Minixhofer, B., Pfeiffer, J., & Vulić, I. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.398 · Cited by 17
Self-citationModelling Latent Translations for Cross-Lingual TransferPreprint
Ponti, E. M., Kreutzer, J., Vulić, I., & Reddy, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.11353 · Cited by 14
Self-citationChang, T. A., Tu, Z., & Bergen, B. K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00708 · Cited by 13
Micallef, K., Albert Gatt, A., Tanti, M., & van der Plas, L. (2022) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/2022.deeplo-1.10 · Cited by 13
Compositional Evaluation on Japanese Textual Entailment and Similarity
Yanaka, H., & Mineshima, K. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00518 · Cited by 16
mDAPT: Multilingual Domain Adaptive Pretraining in a Single Model
Jørgensen, R. K., Hartmann, M., Dai, X., & Elliott, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.290 · Cited by 13
A survey on text classification: Practical perspectives on the Italian language
Gasparetto, A., Zangari, A., Marcuzzo, M., & Albarelli, A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0270904 · Cited by 13
Bhattacharjee, A., Hasan, T., Samin, K., Islam, S., Rahman, M. S., Iqbal, A., & Shahriyar, R. (2021) · arXiv (Cornell University) · Cited by 12
Named Entity Recognition for Low-Resource Languages - Profiting from Language Families
Torge, S., Politov, A., Lehmann, C. U., Saffar, B., & Tao, Z. (2023) · Workshop on Slavic Natural Language Processing (SlavicNLP) · doi:10.18653/v1/2023.bsnlp-1.1 · Cited by 13
Hu, S., Zhou, H., Hergul, M., Gritta, M., Zhang, G., Iacobacci, I., Vulić, I., & Korhonen, A. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00609 · Cited by 13
Self-citationUnveiling Challenges for LLMs in Enterprise Data Engineering
Bodensohn, J.-M., Brackmann, U., Vogel, L., Sanghi, A., & Binnig, C. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3773749.3773758 · Cited by 11
Limisiewicz, T., Balhar, J., & Mareček, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.350 · Cited by 24
LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation
Guo, H., Liu, J., Huang, H., Yang, J., Li, Z., Zhang, D., & Cui, Z. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.184 · Cited by 12
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies
Wu, Z., Tamkin, A., & Papadimitriou, I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.198 · Cited by 17
MultiFin: A Dataset for Multilingual Financial NLP
Jørgensen, R. L., Brandt, O., Hartmann, M., Dai, X., Igel, C., & Elliott, D. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.66 · Cited by 10
Gao, S., Zhao, X., Chen, L., Yu, J., Tian, S., Zhou, H., Chen, J., Long, S., He, Q., & Feng, X. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02139-3 · Cited by 10
Ahuja, K., Dandapat, S., Sitaram, S., & Choudhury, M. (2022) · doi:10.18653/v1/2022.nlppower-1.7 · Cited by 13
Fujii, T., Shibata, K., Yamaguchi, A., Morishita, T., & Sogawa, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.5 · Cited by 17
Xu, N., Gui, T., Ma, R., Zhang, Q., Ye, J., Zhang, M., & Huang, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.552 · Cited by 14
Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis
Goldfarb-Tarrant, S., Roß, B., & Lopez, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.346 · Cited by 13
Wine is not v i n. On the Compatibility of Tokenizations across Languages
Maronikolakis, A., Dufter, P., & Schütze, H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.205 · Cited by 9
Cross-lingual Transfer of Monolingual Models
Gogoulou, E., Ekgren, A., Isbister, T., & Sahlgren, M. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5635surro6fy · Cited by 11
Scene-Text Aware Image and Text Retrieval with Dual-Encoder
Miyawaki, S., Hasegawa, T., Nishida, K., Kato, T., & Suzuki, J. (2022) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2022.acl-srw.34 · Cited by 9
FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models
Dobler, K., & de Melo, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.829 · Cited by 8
gaBERT — an Irish Language Model
Barry, J., Wagner, J., Cassidy, L., Cowap, A., Lynn, T., Walsh, A., Meachair, M. J. Ó., & Foster, J. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2c485c2jd2yz · Cited by 8
Causal and Masked Language Modeling of Javanese Language using Transformer-based Architectures
Wongso, W., Setiawan, D. S., & Suhartono, D. (2021) · International Conference on Advanced Computer Science and Information Systems · doi:10.1109/icacsis53237.2021.9631331 · Cited by 10
MergeDistill: Merging Language Models using Pre-trained Distillation
Khanuja, S., Johnson, M., & Talukdar, P. P. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.254 · Cited by 10
Energy-Aware Multilingual Evaluation of Large Language Models
de Zarzà, I., Liz, M., de Curtò, J., & Calafate, C. T. (2026) · Electronics · doi:10.3390/electronics15071395 · Cited by 7
Predictability and Causality in Spanish and English Natural Language Generation
Busto-Castiñeira, A., González-Castaño, F. J., García-Méndez, S., & de Arriba-Pérez, F. (2024) · IEEE Access · doi:10.1109/access.2024.3420710 · Cited by 7
Benchmarking LLaMA-3 on Arabic Language Generation Tasks
Khondaker, M. T. I., Naeem, N., Khan, F., Elmadany, A., & Abdul-Mageed, M. (2024) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/2024.arabicnlp-1.24 · Cited by 7
Schleifer, A. G., Klebanov, B. B., & Alexandron, G. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00485-7 · Cited by 7
Incorporating Context into Subword Vocabularies
Yehezkel, S., & Pinter, Y. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.45 · Cited by 7
Does Transliteration Help Multilingual Language Modeling?
Moosa, I. M., Akhter, M. E., & Habib, A. B. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.50 · Cited by 11
Romanization-based Large-scale Adaptation of Multilingual Language Models
Purkayastha, S., Ruder, S., Pfeiffer, J., Gurevych, I., & Vulić, I. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.538 · Cited by 7
Self-citationBalde, G., Roy, S., Mondal, M., & Ganguly, N. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.863 · Cited by 6
A Study of Vietnamese Sentiment Classification with Ensemble Pre-Trained Language Models
Thin, D. V., Hao, D. N., & Nguyen, N. L.-T. (2023) · Vietnam Journal of Computer Science · doi:10.1142/s2196888823500173 · Cited by 6
He, R., Grodzki, R., Altay, N., Aston, B., Alonso-Sánchez, M. F., Homan, P., Sommer, I. E., Palaniyappan, L. K., & Hinzen, W. (2026) · Scientific Reports · doi:10.1038/s41598-026-39025-1 · Cited by 6
BERT2D: Two Dimensional Positional Embeddings for Efficient Turkish NLP
Kaya, Y. B., & Tantug, A. C. (2024) · IEEE Access · doi:10.1109/access.2024.3407983 · Cited by 6
Subword Mapping and Anchoring across Languages
Vernikos, G., & Popescu-Belis, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.224 · Cited by 8
Are Multilingual Sentiment Models Equally Right for the Right Reasons?
Jørgensen, R. L., Caccavale, F., Igel, C., & Søgaard, A. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.11 · Cited by 6
Analyzing Cognitive Plausibility of Subword Tokenization
Beinborn, L., & Pinter, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.272 · Cited by 6
Metadata
- Authors
- Rust, P., Pfeiffer, J., Vulić, I., Ruder, S., & Gurevych, I.
- Affiliations
- Technische Universität Darmstadt (DE)
- University of Cambridge (GB)
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 3118-3135
- OpenAlex
- W3120253119
- Cited by
- 202
- References
- 57
- Citations by year
- 2026: 32 · 2025: 32 · 2024: 22 · 2023: 50 · 2022: 35 · 2021: 31
- FWCI
- 15.94
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- arXiv (Cornell University) (submittedVersion)
- TUbilio (Technical University of Darmstadt) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- H2020 European Research Council (648909)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- tokenization, multilingual language models, multilingual tokenization, language model evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
