Crosslingual Generalization through Multitask Finetuning
Open accessAbstract
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The scientific documentation of BloomZ includes multiple peer-reviewed papers, while no peer-reviewed papers providing scientific documentation or evaluation of Llama are known currently.
“The scientific documentation of BloomZ also includes multiple peer-reviewed papers , from a scientific description of the multitask fine-tuning procedures [41] to an estimation of the carbon footprint [36] currently one of the very few scientifically vetted sources of data on the energy footprint of training large language models. No peerreviewed papers providing scientific documentation or eval”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [41]
Machine-translated data can cause translationese or possible name bias, gender bias, or cultural bias.
“We try to avoid using machine-translated data to finetune the model in order to prevent translationese (Bizzoni et al., 2020; Muennighoff et al., 2023) or possible name bias (Wang et al., 2022a), gender bias (Savoldi et al., 2021), or cultural bias (Ji et al., 2023).”
From The Llama 3 Herd of Models (2024) · cites this paper as Muennighoff et al., 2023
Muennighoff et al. is cited for the translationese concern; the other biases have separate citations. The copy lists machine-translated prompts (Appendix M) and discusses an unnatural prompting format (Section 6, page unavailable), but neither passage says machine-translated data can cause translationese. Searches for “translationese” and translated-data artifacts found no passage addressing that claim. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 10,387 · see citing works
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 2,523 · see citing works
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.41 · Cited by 1,629 · see citing works
Self-citationMultilingual Denoising Pre-training for Neural Machine Translation
Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., & Zettlemoyer, L. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00343 · Cited by 1,608 · see citing works
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · see citing works
The Flan Collection: Designing Data and Methods for Effective Instruction TuningPreprint
Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H. W., Tay, Y., Zhou, D., Le, Q. V., Zoph, B., Wei, J., et al. (2023) · International Conference on Machine Learning · doi:10.48550/arxiv.2301.13688 · Cited by 1,029 · see citing works
Self-citationTraining Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932 · see citing works
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Ben Zaken, E., Goldberg, Y., & Ravfogel, S. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.1 · Cited by 765 · see citing works
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., & Raffel, C. (2022) · neural information processing systems · doi:10.52202/068431-0142 · Cited by 445 · see citing works
Self-citationMLQA: Evaluating Cross-lingual Extractive Question Answering
Lewis, P., Oğuz, B., Rinott, R., Riedel, S., & Schwenk, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.653 · Cited by 424 · see citing works
GLM-130B: An Open Bilingual Pre-trained Model
Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.02414 · Cited by 281 · see citing works
StarCoder: may the source be with you!
Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.06161 · Cited by 261 · see citing works
Self-citationPAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
Yang, Y., Zhang, Y. E., Tar, C., & Baldridge, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1382 · Cited by 231 · see citing works
PAWS: Paraphrase Adversaries from Word ScramblingPreprint
Zhang, Y., Baldridge, J., & He, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01130 · Cited by 160 · see citing works
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor
Honovich, O., Scialom, T., Levy, O., & Schick, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.806 · Cited by 160 · see citing works
Measuring Coding Challenge Competence With APPS
Hendrycks, D., Basart, S., Kadavath, S., Mazeika, M., Arora, A., Guo, E., Burns, C., Puranik, S., He, H., Song, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.09938 · Cited by 142 · see citing works
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
Logan IV, R., Balazevic, I., Wallace, E., Petroni, F., Singh, S., & Riedel, S. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.222 · Cited by 133 · see citing works
Global Relational Models of Source Code
Hellendoorn, V. J., Sutton, C. A., Singh, R., & Maniatis, P. (2020) · International Conference on Learning Representations · Cited by 128 · see citing works
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Iyer, S., Lin, X. V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., Koura, P. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.12017 · Cited by 81 · see citing works
Self-citationFew-shot Learning with Multilingual Language ModelsPreprint
Lin, X. V., Mihaylov, T., Artetxe, M., Wang, T., Chen, S., Simig, D., Ott, M., Goyal, N., Bhosale, S., Du, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.10668 · Cited by 79 · see citing works
Self-citationLongpre, S., Yauney, G., Reif, E., Lee, K., Roberts, A., Zoph, B., Zhou, D., Wei, J., Robinson, K., Mimno, D., et al. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.179 · Cited by 75 · see citing works
Self-citationThe BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Laurençon, H., Saulnier, L., Wang, T., Akiki, C., del Moral, A. V., Scao, T. L., Von Werra, L., Mou, C., Ponferrada, E. G., Nguyen, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03915 · Cited by 61 · see citing works
Self-citationMetaICL: Learning to Learn In Context
Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H. (2021) · arXiv · doi:10.48550/arxiv.2110.15943 · Cited by 61 · see citing works
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
Yong, Z. X., Schoelkopf, H., Muennighoff, N., Aji, A. F., Adelani, D. I., Almubarak, K., Bari, M. S., Sutawika, L., Kasai, J., Baruwa, A., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.653 · Cited by 30 · see citing works
Self-citationNeural Code Search Evaluation DatasetPreprint
Li, H.-Y., Kim, S., & Chandra, S. (2019) · arXiv.org · doi:10.48550/arxiv.1908.09804 · Cited by 29 · see citing works
Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation
Da, Y., Liu, X., Yin, F., Zhong, M., Bansal, H., Han, J., & Chang, K.-W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.245 · Cited by 26 · see citing works
BiSECT: Learning to Split and Rephrase Sentences with Bitexts
Kim, J., Maddela, M., Kriz, R., Xu, W. H., & Callison-Burch, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.500 · Cited by 16 · see citing works
Sparse Upcycling: Training Mixture-of-Experts from Dense CheckpointsPreprint
Komatsuzaki, A., Puigcerver, J., Lee-Thorp, J., Riquelme, C., Mustafa, B., Ainslie, J., Tay, Y., Dehghani, M., & Houlsby, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.05055 · Cited by 15 · see citing works
From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction TuningPreprint
Liu, Q., Zhou, F., Jiang, Z., Dou, L., & Lin, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.07995 · Cited by 12 · see citing works
Meta-tuning Language Models to Answer Prompts Better.Preprint
Zhong, R., Lee, K., Zhang, Z., & Klein, D. (2021) · arXiv (Cornell University) · Cited by 8 · see citing works
Packing: Towards 2x NLP BERT AccelerationPreprint
Kosec, M., Fu, S., & Krell, M. M. (2021) · arXiv (Cornell University) · Cited by 4 · see citing works
Adapting BigScience Multilingual Model to Unseen Languages
Yong, Z.-X., & Nikoulina, V. (2022) · arXiv · doi:10.48550/arxiv.2204.04873 · Cited by 3 · see citing works
Self-citationOPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257 · see citing works
Instruction Tuning for Large Language Models: A Survey
Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wang, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3777411 · Cited by 640 · see citing works
MTEB: Massive Text Embedding Benchmark
Muennighoff, N., Tazi, N., Magne, L., & Reimers, N. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.148 · Cited by 548 · see citing works
Self-citationC-Pack: Packed Resources For General Chinese Embeddings
Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657878 · Cited by 523 · see citing works
Self-citationPower Hungry Processing: Watts Driving the Cost of AI Deployment?
Luccioni, S., Jernite, Y., & Strubell, E. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658542 · Cited by 392 · see citing works
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models
Hu, Z., Wang, L., Lan, Y., Xu, W., Lim, E., Bing, L., Xu, X., Poria, S., & Lee, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.319 · Cited by 349 · see citing works
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335 · see citing works
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, V. N. (2024) · Applied Sciences · doi:10.3390/app14052074 · Cited by 315 · see citing works
How Can Recommender Systems Benefit from Large Language Models: A Survey
Lin, J., Dai, X., Xi, Y., Liu, W., Chen, B., Zhang, H., Liu, Y., Wu, C., Li, X., Zhu, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3678004 · Cited by 242 · see citing works
MEGA: Multilingual Evaluation of Generative AI
Ahuja, K., Hada, R., Ochieng, M., Jain, P., Diddee, H., Ramesh, K., Maina, S., Ganu, T., Segal, S., Axmed, M., et al. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.258 · Cited by 470 · see citing works
Larger and more instructable language models become less reliable
Zhou, L., Schellaert, W., Martínez-Plumed, F., Moros-Daval, Y., Ferri, C., & Hernández-Orallo, J. (2024) · Nature · doi:10.1038/s41586-024-07930-y · Cited by 203 · see citing works
A survey of multilingual large language models
Qin, L., Chen, Q., Zhou, Y., Zhi, C., Li, Y., Liao, L., Li, M., Che, W., & Yu, P. S. (2025) · Patterns · doi:10.1016/j.patter.2024.101118 · Cited by 197 · see citing works
Parameter-efficient fine-tuning in large language models: a survey of methodologies
Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11236-4 · Cited by 186 · see citing works
Evaluating large language models on a highly-specialized topic, radiation oncology physics
Holmes, J., Liu, Z., Zhang, L., Ding, Y., Sio, T. T. W., McGee, L. A., Ashman, J. B., Li, X., Liu, T., Shen, J., et al. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1219326 · Cited by 166 · see citing works
Datasets for large language models: a comprehensive survey
Liu, Y., Cao, J., Liu, C., Ding, K., & Jin, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11403-7 · Cited by 159 · see citing works
A Survey of Techniques for Optimizing Transformer Inference
Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.102990 · Cited by 154 · see citing works
A large-scale audit of dataset licensing and attribution in AI
Longpre, S., Mahari, R., Chen, A., Obeng-Marnu, N., Sileo, D., Brannon, W., Muennighoff, N., Khazam, N., Kabbara, J., Perisetla, K., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00878-8 · Cited by 150 · see citing works
Self-citationGrammatical Error Correction: A Survey of the State of the Art
Bryant, C., Yuan, Z., Qorib, M. R., Cao, H., Ng, H. T., & Briscoe, T. (2023) · Computational Linguistics · doi:10.1162/coli_a_00478 · Cited by 147 · see citing works
Yang, S., Zhao, H., Zhu, S., Zhou, G., Hongfei, X., Jia, Y., & Zan, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29907 · Cited by 145 · see citing works
Aligning Large Multimodal Models with Factually Augmented RLHF
Sun, Z., Shen, S., Cao, S., Liu, H., Li, C., Shen, Y., Gan, C., Gui, L.-Y., Wang, Y.-X., Yang, Y., et al. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.775 · Cited by 146 · see citing works
Self-citationAn Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslpro.2025.3606231 · Cited by 136 · see citing works
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.08747 · Cited by 135 · see citing works
Large Language Models for Therapy Recommendations Across 3 Clinical Specialties: Comparative Study
Wilhelm, T. I., Roos, J., & Kaczmarczyk, R. (2023) · Journal of Medical Internet Research · doi:10.2196/49324 · Cited by 133 · see citing works
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
Laskar, M. T. R., Bari, M. S., Rahman, M., Bhuiyan, M. A. H., Joty, S., & Huang, J. X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.29 · Cited by 117 · see citing works
Self-citationA Watermark for Large Language Models
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023) · arXiv · doi:10.48550/arxiv.2301.10226 · Cited by 194 · see citing works
“Consider the following process for creating the lists.”
Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models
Chen, Z., Xu, L., Zheng, H., Chen, L., Tolba, A. K. R., Zhao, L., Yu, K., & Feng, H. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.052618 · Cited by 112 · see citing works
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3604316 · Cited by 111 · see citing works
History, development, and principles of large language models: an introductory survey
Wang, Z., Chu, Z., Doan, T. V., Ni, S., Yang, M., & Zhang, W. (2025) · AI and Ethics · doi:10.1007/s43681-024-00583-7 · Cited by 109 · see citing works
Generative language models exhibit social identity biases
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00741-1 · Cited by 100 · see citing works
“5 (text-davinci-003) [34], BLOOMZ [35], OPT-IML [36], Flan-T5 [37], Dolly 2.0 [38], Jurassic-2 Jumbo Instruct [39], Alpaca [40], and PaLM 2 (text-bison@001) [41]).”
InfluentialLarge language models in food science: Innovations, applications, and future
Ma, P., Tsai, S., He, Y., Jia, X., Zhen, D., Yu, N., Wang, Q., Ahuja, J. K. C., & Wei, C. (2024) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2024.104488 · Cited by 97 · see citing works
When brain-inspired AI meets AGI
Zhao, L., Zhang, L., Wu, Z., Chen, Y., Dai, H., Yu, X., Liu, Z., Zhang, T., Hu, X., Jiang, X., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100005 · Cited by 96 · see citing works
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2024) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 92 · see citing works
Taiyi: a bilingual fine-tuned large language model for diverse biomedical tasks
Luo, L., Ning, J., Zhao, Y., Wang, Z., Ding, Z., Chen, P., Fu, W., Han, Q., Xu, G., Qiu, Y., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae037 · Cited by 89 · see citing works
Designing Heterogeneous LLM Agents for Financial Sentiment Analysis
Xing, F. (2025) · ACM Transactions on Management Information Systems · doi:10.1145/3688399 · Cited by 88 · see citing works
Wei, X., Zhang, Y., Li, Y., Zhang, X., Gong, R., Guo, J., & Liu, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.102 · Cited by 86 · see citing works
Sabiá: Portuguese Large Language Models
Pires, R., Abonizio, H., Almeida, T. S., & Nogueira, R. F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45392-2_15 · Cited by 85 · see citing works
Knowledge Editing for Large Language Models: A Survey
Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., & Li, J. (2025) · ACM Computing Surveys · doi:10.1145/3698590 · Cited by 79 · see citing works
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP
Khondaker, M. T. I., Waheed, A., Nagoudi, E. M. B., & Abdul-Mageed, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.16 · Cited by 79 · see citing works
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance
Whitehouse, C., Choudhury, M., & Aji, A. F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.44 · Cited by 79 · see citing works
Self-citationAceGPT, Localizing Large Language Models in Arabic
Huang, H., Yu, F., Zhu, J., Sun, X., Cheng, H., Dingjie, S., Chen, Z., Alharthi, M., An, B., He, J., et al. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.450 · Cited by 78 · see citing works
Wandelt, S., Zheng, C., Wang, S., Liu, Y., & Sun, X. (2024) · Applied Sciences · doi:10.3390/app14177455 · Cited by 78 · see citing works
Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative StudyPreprint
Keraghel, I., Morbieu, S., & Nadif, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.10825 · Cited by 78 · see citing works
RenAIssance: A Survey Into AI Text-to-Image Generation in the Era of Large Model
Bie, F., Yang, Y., Zhou, Z., Ghanem, A., Zhang, M., Yao, Z., Wu, X., Holmes, C., Golnari, P. A., Clifton, D. A., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3522305 · Cited by 76 · see citing works
Yuan, A., Colato, E. G., Pescosolido, B. A., Song, H., & Samtani, S. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3701041 · Cited by 74 · see citing works
Neural natural language processing for long texts: A survey on classification and summarization
Tsirmpas, D., Gkionis, I., Papadopoulos, G. T., & Mademlis, I. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108231 · Cited by 73 · see citing works
A survey on large language models with multilingualism: recent advances and new frontiers
Huang, K., Mo, F., Zhang, X., Li, H., Li, Y., Zhang, Y., yi, W., Mao, Y., Liu, J., Xu, Y., et al. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11534-5 · Cited by 73 · see citing works
Kim, S., Joo, S., Kim, D., Jang, J., Ye, S., Shin, J., & Seo, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.782 · Cited by 69 · see citing works
Multitask Vision-Language Prompt Tuning
Shen, S., Yang, S., Zhang, T., Zhai, B., Gonzalez, J. E., Keutzer, K., & Darrell, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00556 · Cited by 69 · see citing works
Self-citationRethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 67 · see citing works
A Literature Survey on Open Source Large Language Models
Kukreja, S., Kumar, T., Purohit, A., Dasgupta, A., & Guha, D. (2024) · International Conference on Computers in Management and Business (ICCMB) · doi:10.1145/3647782.3647803 · Cited by 65 · see citing works
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D. R., Smith, N. A., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.614 · Cited by 63 · see citing works
Zero-Shot Cross-Lingual Summarization via Large Language Models
Wang, J., Liang, Y., Meng, F., Zou, B., Li, Z., Qu, J., & Zhou, J. (2023) · Workshop on New Frontiers in Summarization (NewSum) · doi:10.18653/v1/2023.newsum-1.2 · Cited by 62 · see citing works
Li, J., Zhou, H., Huang, S., Cheng, S., & Chen, J. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00655 · Cited by 61 · see citing works
Qibo: A Large Language Model for traditional Chinese medicine
Jia, Y., Ji, X., Wang, X., Zhang, H., Meng, Z., Zhang, J., Chen, Z., Zhuang, P., Xu, D., Guo, W., et al. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127672 · Cited by 60 · see citing works
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges
Winata, G. I., Aji, A. F., Yong, Z., & Solorio, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.185 · Cited by 56 · see citing works
Self-citationBenchmarking Vision Language Models for Cultural Understanding
Nayak, S., Jain, K., Awal, R., Reddy, S., van Steenkiste, S., Hendricks, L. A., Stańczak, K., & Agrawal, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.329 · Cited by 56 · see citing works
Petals: Collaborative Inference and Fine-tuning of Large Models
Borzunov, A., Baranchuk, D., Dettmers, T., Riabinin, M., Belkada, Y., Chumachenko, A., Samygin, P., & Raffel, C. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · doi:10.18653/v1/2023.acl-demo.54 · Cited by 52 · see citing works
Self-citationA Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · doi:10.31219/osf.io/ac4sp · Cited by 51 · see citing works
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
Qin, Y., Cai, Z., Jin, D., Yan, L., Liang, S., Zhu, K., Lin, Y., Han, X., Ding, N., Wang, H., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.499 · Cited by 50 · see citing works
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
Qin, L., Chen, Q., Wei, F., Huang, S., & Che, W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.163 · Cited by 50 · see citing works
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 50 · see citing works
Federated and edge learning for large language models
Piccialli, F., Chiaro, D., Qi, P., Bellandi, V., & Damiani, E. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102840 · Cited by 49 · see citing works
Privacy Preserving Prompt Engineering: A Survey
Edemacu, K., & Wu, X. (2025) · ACM Computing Surveys · doi:10.1145/3729219 · Cited by 48 · see citing works
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, A. V. (2024) · doi:10.20944/preprints202402.0357.v1 · Cited by 296 · see citing works
Zhang, X., Rajabi, N., Duh, K., & Koehn, P. (2023) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2023.wmt-1.43 · Cited by 47 · see citing works
Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech
Plaza-del-Arco, F. M., Nozza, D., & Hovy, D. (2023) · doi:10.18653/v1/2023.woah-1.6 · Cited by 44 · see citing works
A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction
Shui, R., Cao, Y., Wang, X., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.490 · Cited by 43 · see citing works
LMEye: An Interactive Perception Network for Large Language Models
Li, Y., Hu, B., Chen, X., Ma, L., Xu, Y. D., & Zhang, M. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3428317 · Cited by 43 · see citing works
MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
Cai, Y., Wang, L., Wang, Y., De Melo, G., Zhang, Y., Wang, Y., & He, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29723 · Cited by 42 · see citing works
Proposed Framework for Human-like Language Processing of ChatGPT in Academic Writing
Mahyoob, M., Algaraady, J., & Alblwi, A. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i14.41725 · Cited by 42 · see citing works
Exploring the Capabilities of LLMs for Code-Change-Related Tasks
Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., & Li, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709358 · Cited by 41 · see citing works
“For example, the Bloom model [58] contains 176B parameters, 789 times more than CCT5 (223M), is pre-trained on 363B natural language tokens, and significantly outperforms all the models with less than 1B parameters on natural language processing tasks.”
Best Practices for Large Language Models in Radiology
Blüthgen, C., Veen, D. V., Zakka, C., Link, K. E., Fanous, A. H., Daneshjou, R., Frauenfelder, T., Langlotz, C. P., Gatidis, S., & Chaudhari, A. S. (2025) · Radiology · doi:10.1148/radiol.240528 · Cited by 41 · see citing works
CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs
Bhattacharjee, A., Hasan, T., Ahmad, W. U., Li, Y.-F., Kang, Y., & Shahriyar, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.143 · Cited by 41 · see citing works
Several Categories of Large Language Models (LLMs): A Short Survey
Pahune, S. A., & Chandrasekharan, M. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.54677 · Cited by 40 · see citing works
TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties
Kadaoui, K., Magdy, S. M., Waheed, A., Khondaker, M. T. I., El-Shangiti, A. O., Nagoudi, E. M. B., & Abdul-Mageed, M. (2023) · Arabic Natural Language Processing Conference (ArabicNLP) · doi:10.18653/v1/2023.arabicnlp-1.6 · Cited by 39 · see citing works
LLMs-based machine translation for E-commerce
Gao, D., Chen, K., Chen, B., Dai, H.-F., Jin, L., Jiang, W., Ning, W., Yu, S., Xuan, Q., Cai, X., et al. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125087 · Cited by 39 · see citing works
Nguyen, X.-P., Aljunied, M., Joty, S., & Bing, L. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.192 · Cited by 39 · see citing works
Choi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.699 · Cited by 38 · see citing works
EcomGPT: Instruction-Tuning Large Language Models with Chain-of-Task Tasks for E-commerce
Li, Y., Ma, S., Wang, X., Huang, S., Jiang, C., Zheng, H.-T., Xie, P., Huang, F., & Jiang, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29820 · Cited by 38 · see citing works
Chen, L., Zuo, H., Cai, Z., Yin, Y., Zhang, Y. E., Sun, L., Childs, P., & Wang, B. (2024) · Journal of Mechanical Design · doi:10.1115/1.4065562 · Cited by 38 · see citing works
Wu, Z., Zhang, L., Cao, C., Yu, X., Liu, Z., Zhao, L., Li, Y.-W., Dai, H.-X., Ma, C., Li, G., et al. (2023) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3536928 · Cited by 52 · see citing works
Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon
Koto, F., Beck, T., Talat, Z., Gurevych, I., & Baldwin, T. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.18 · Cited by 36 · see citing works
NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task
Abdul-Mageed, M., Elmadany, A., Zhang, C., Nagoudi, E. M. B., Bouamor, H., & Habash, N. Y. (2023) · Arabic Natural Language Processing Conference (ArabicNLP) · doi:10.18653/v1/2023.arabicnlp-1.62 · Cited by 35 · see citing works
Jain, R., Sojitra, D., Acharya, A., Saha, S., Jatowt, A., & Dandapat, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.418 · Cited by 34 · see citing works
BLESS: Benchmarking Large Language Models on Sentence Simplification
Kew, T., Chi, A., Vásquez-Rodríguez, L., Agrawal, S., Aumiller, D., Alva-Manchego, F. E., & Shardlow, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.821 · Cited by 34 · see citing works
SLM: Bridge the Thin Gap Between Speech and Text Foundation Models
Wang, M., Han, W., Izhak Shafran, J., Wu, Z., Chiu, C., Cao, Y. J., Chen, N., Zhang, Y., Soltau, H., Rubenstein, P. K., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389703 · Cited by 34 · see citing works
Beyond Words: A Comparative Analysis of LLM Embeddings for Effective Clustering
Keraghel, I., Morbieu, S., & Nadif, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-58547-0_17 · Cited by 34 · see citing works
BiMediX: Bilingual Medical Mixture of Experts LLM
Pieri, S., Mullappilly, S. S., Khan, F. S., Anwer, R. M., Khan, S., Baldwin, T., & Cholakkal, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.989 · Cited by 34 · see citing works
Artificial general intelligence for radiation oncology
Liu, C., Liu, Z., Holmes, J., Zhang, L., Zhang, L., Ding, Y., Shu, P., Wu, Z., Dai, H., Li, Y., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100045 · Cited by 34 · see citing works
Yong, Z., Zhang, R., Forde, J. Z., Wang, S., Subramonian, A., Lovenia, H., Cahyawijaya, S., Winata, G. I., Sutawika, L., Cruz, J. C. B., et al. (2023) · Workshop on Computational Approaches to Code Switching · doi:10.18653/v1/2023.calcs-1.5 · Cited by 34 · see citing works
Self-citationEvaluating Diverse Large Language Models for Automatic and General Bug Reproduction
Kang, S., Yoon, J., Askarbekkyzy, N., & Yoo, S. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3450837 · Cited by 33 · see citing works
MasakhaNEWS: News Topic Classification for African languages
Adelani, D. I., Masiak, M., Azime, I. A., Alabi, J. O., Tonja, A. L., Mwase, C., Ogundepo, O., Dossou, B. F. P., Oladipo, A., Nixdorf, D., et al. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.10 · Cited by 33 · see citing works
Lai, V. D., Nguyen, C., Ngo, N., Nguyen, T., Dernoncourt, F., Rossi, R. A., & Nguyen, T. H. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.28 · Cited by 32 · see citing works
TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rules
Manas, K., Zwicklbauer, S., & Paschke, A. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588650 · Cited by 31 · see citing works
To What Extent Have LLMs Reshaped the Legal Domain So Far? A Scoping Literature Review
Padiu, B., Iacob, R. C. A., Rebedea, T. E., & Dascălu, M. (2024) · Information · doi:10.3390/info15110662 · Cited by 30 · see citing works
Multilingual Large Language Models Are Not (Yet) Code-Switchers
Zhang, R., Cahyawijaya, S., Cruz, J. C. B., Winata, G. I., & Aji, A. F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.774 · Cited by 30 · see citing works
Self-citationXCOT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning
Chai, L., Yang, J. J., Sun, T., Guo, H., Liu, J., Wang, B., Liang, X., Bai, J., Li, T., Peng, Q., et al. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34524 · Cited by 30 · see citing works
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Qi, J., Fernández, R., & Bisazza, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.658 · Cited by 29 · see citing works
Enhancing Zero-Shot Crypto Sentiment With Fine-Tuned Language Model and Prompt Engineering
Wahidur, R. S. M., Tashdeed, I., Kaur, M., & Lee, H.-N. (2024) · IEEE Access · doi:10.1109/access.2024.3350638 · Cited by 28 · see citing works
Cultural Adaptation of Recipes
Cao, Y., Kementchedjhieva, Y., Cui, R., Karamolegkou, A., Zhou, L., Dare, M., Donatelli, L. E., & Hershcovich, D. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00634 · Cited by 28 · see citing works
- Authors
- Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M. S., Shen, S., Yong, Z. X., Schoelkopf, H., Tang, X., Radev, D., Aji, A. F., Almubarak, K., Albanie, S., Alyafeai, Z., Webson, A., Raff, E., & Raffel, C.
- Affiliations
- University of Cambridge (GB)
- Google (United States) (US)
- Booz Allen Hamilton (United States) (US)
- University of California, Berkeley (US)
- Brown University (US)
- King Fahd University of Petroleum and Minerals (SA)
- Corresponding author
- Muennighoff, Niklas
- ORCID
- Venue
- Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2023
- Pages
- 15991-16111
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4385571124
- Cited by
- 361
- References
- 33
- Citations by year
- 2026: 57 · 2025: 76 · 2024: 112 · 2023: 116
- FWCI
- 63.96
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- Centre National de la Recherche Scientifique
- Grand Équipement National De Calcul Intensif
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- cross-lingual generalization, multilingual language models, transfer learning, low-resource languages
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
