Cited by 540Topic ModelingDomain Adaptation and Few-Shot LearningNatural Language Processing Techniques

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Crosslingual Generalization through Multitask Finetuning

Open access

Muennighoff, Niklas, Wang, Thomas, Sutawika, Lintang, Roberts, Adam, Biderman, Stella, Le Scao, Teven, Bari, M Saiful, Shen, Sheng, Yong, Zheng Xin, Schoelkopf, Hailey, Tang, Xiangru, Radev, Dragomir, Aji, Alham Fikri, Almubarak, Khalid, Albanie, Samuel, Alyafeai, Zaid, Webson, Albert, Raff, Edward, Raffel, Colin·2023·Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

DOI 10.18653/v1/2023.acl-long.891

Open this paper in CoChat — free account Open PDF

Abstract

Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.18653/v1/2023.acl-long.891

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 10,387
Multilingual Denoising Pre-training for Neural Machine Translation
Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., & Zettlemoyer, L. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00343 · Cited by 1,762
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.41 · Cited by 1,629
Self-citation
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Ben Zaken, E., Goldberg, Y., & Ravfogel, S. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.1 · Cited by 765
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., & Raffel, C. (2022) · neural information processing systems · doi:10.52202/068431-0142 · Cited by 634
Self-citation
The Flan Collection: Designing Data and Methods for Effective Instruction TuningPreprint
Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H. W., Tay, Y., Zhou, D., Le, Q. V., Zoph, B., Wei, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.13688 · Cited by 553
Self-citation
MLQA: Evaluating Cross-lingual Extractive Question Answering
Lewis, P., Oğuz, B., Rinott, R., Riedel, S., & Schwenk, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.653 · Cited by 509

Show all 33 references

Sorted by citations

A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
Instruction Tuning for Large Language Models: A Survey
Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wang, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3777411 · Cited by 1,271
MTEB: Massive Text Embedding Benchmark
Muennighoff, N., Tazi, N., Magne, L., & Reimers, N. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.148 · Cited by 548
Self-citation
C-Pack: Packed Resources For General Chinese Embeddings
Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657878 · Cited by 523
Self-citation
Power Hungry Processing: Watts Driving the Cost of AI Deployment?
Luccioni, S., Jernite, Y., & Strubell, E. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658542 · Cited by 392
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models
Hu, Z., Wang, L., Lan, Y., Xu, W., Lim, E., Bing, L., Xu, X., Poria, S., & Lee, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.319 · Cited by 581
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, V. N. (2024) · Applied Sciences · doi:10.3390/app14052074 · Cited by 355
How Can Recommender Systems Benefit from Large Language Models: A Survey
Lin, J., Dai, X., Xi, Y., Liu, W., Chen, B., Zhang, H., Liu, Y., Wu, C., Li, X., Zhu, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3678004 · Cited by 242
MEGA: Multilingual Evaluation of Generative AI
Ahuja, K., Diddee, H., Hada, R., Ochieng, M., Ramesh, K., Jain, P., Nambi, A. U., Ganu, T., Segal, S., Ahmed, M., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.258 · Cited by 321
Larger and more instructable language models become less reliable
Zhou, L., Schellaert, W., Martínez-Plumed, F., Moros-Daval, Y., Ferri, C., & Hernández-Orallo, J. (2024) · Nature · doi:10.1038/s41586-024-07930-y · Cited by 203
A survey of multilingual large language models
Qin, L., Chen, Q., Zhou, Y., Zhi, C., Li, Y., Liao, L., Li, M., Che, W., & Yu, P. S. (2025) · Patterns · doi:10.1016/j.patter.2024.101118 · Cited by 282
Parameter-efficient fine-tuning in large language models: a survey of methodologies
Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11236-4 · Cited by 186
Evaluating large language models on a highly-specialized topic, radiation oncology physics
Holmes, J., Liu, Z., Zhang, L., Ding, Y., Sio, T. T. W., McGee, L. A., Ashman, J. B., Li, X., Liu, T., Shen, J., et al. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1219326 · Cited by 198
Datasets for large language models: a comprehensive survey
Liu, Y., Cao, J., Liu, C., Ding, K., & Jin, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11403-7 · Cited by 268
A Survey of Techniques for Optimizing Transformer Inference
Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.102990 · Cited by 154
A large-scale audit of dataset licensing and attribution in AI
Longpre, S., Mahari, R., Chen, A., Obeng-Marnu, N., Sileo, D., Brannon, W., Muennighoff, N., Khazam, N., Kabbara, J., Perisetla, K., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00878-8 · Cited by 248
Self-citation
Grammatical Error Correction: A Survey of the State of the Art
Bryant, C., Yuan, Z., Qorib, M. R., Cao, H., Ng, H. T., & Briscoe, T. (2023) · Computational Linguistics · doi:10.1162/coli_a_00478 · Cited by 173
Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-World Multi-Turn Dialogue
Yang, S., Zhao, H., Zhu, S., Zhou, G., Hongfei, X., Jia, Y., & Zan, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29907 · Cited by 179
Aligning Large Multimodal Models with Factually Augmented RLHF
Sun, Z., Shen, S., Cao, S., Liu, H., Li, C., Shen, Y., Gan, C., Gui, L.-Y., Wang, Y.-X., Yang, Y., et al. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.775 · Cited by 185
Self-citation
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslpro.2025.3606231 · Cited by 136
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.08747 · Cited by 135
Large Language Models for Therapy Recommendations Across 3 Clinical Specialties: Comparative Study
Wilhelm, T. I., Roos, J., & Kaczmarczyk, R. (2023) · Journal of Medical Internet Research · doi:10.2196/49324 · Cited by 133
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
Laskar, M. T. R., Bari, M. S., Rahman, M., Bhuiyan, M. A. H., Joty, S., & Huang, J. X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.29 · Cited by 160
Self-citation
A Watermark for Large Language Models
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023) · arXiv · doi:10.48550/arxiv.2301.10226 · Cited by 194
“Consider the following process for creating the lists.”
Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models
Chen, Z., Xu, L., Zheng, H., Chen, L., Tolba, A. K. R., Zhao, L., Yu, K., & Feng, H. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.052618 · Cited by 126
Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3604316 · Cited by 111
History, development, and principles of large language models: an introductory survey
Wang, Z., Chu, Z., Doan, T. V., Ni, S., Yang, M., & Zhang, W. (2025) · AI and Ethics · doi:10.1007/s43681-024-00583-7 · Cited by 109
Generative language models exhibit social identity biases
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00741-1 · Cited by 100
“5 (text-davinci-003) [34], BLOOMZ [35], OPT-IML [36], Flan-T5 [37], Dolly 2.0 [38], Jurassic-2 Jumbo Instruct [39], Alpaca [40], and PaLM 2 (text-bison@001) [41]).”
Influential
Large language models in food science: Innovations, applications, and future
Ma, P., Tsai, S., He, Y., Jia, X., Zhen, D., Yu, N., Wang, Q., Ahuja, J. K. C., & Wei, C. (2024) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2024.104488 · Cited by 99
When brain-inspired AI meets AGI
Zhao, L., Zhang, L., Wu, Z., Chen, Y., Dai, H., Yu, X., Liu, Z., Zhang, T., Hu, X., Jiang, X., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100005 · Cited by 116
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2024) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 133
Taiyi: a bilingual fine-tuned large language model for diverse biomedical tasks
Luo, L., Ning, J., Zhao, Y., Wang, Z., Ding, Z., Chen, P., Fu, W., Han, Q., Xu, G., Qiu, Y., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae037 · Cited by 100
Designing Heterogeneous LLM Agents for Financial Sentiment Analysis
Xing, F. (2025) · ACM Transactions on Management Information Systems · doi:10.1145/3688399 · Cited by 88
Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling
Wei, X., Zhang, Y., Li, Y., Zhang, X., Gong, R., Guo, J., & Liu, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.102 · Cited by 178
Sabiá: Portuguese Large Language Models
Pires, R., Abonizio, H., Almeida, T. S., & Nogueira, R. F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45392-2_15 · Cited by 104
Knowledge Editing for Large Language Models: A Survey
Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., & Li, J. (2025) · ACM Computing Surveys · doi:10.1145/3698590 · Cited by 79
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP
Khondaker, M. T. I., Waheed, A., Nagoudi, E. M. B., & Abdul-Mageed, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.16 · Cited by 89
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance
Whitehouse, C., Choudhury, M., & Aji, A. F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.44 · Cited by 113
Self-citation
AceGPT, Localizing Large Language Models in Arabic
Huang, H., Yu, F., Zhu, J., Sun, X., Cheng, H., Dingjie, S., Chen, Z., Alharthi, M., An, B., He, J., et al. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.450 · Cited by 78
Large Language Models for Intelligent Transportation: A Review of the State of the Art and Challenges
Wandelt, S., Zheng, C., Wang, S., Liu, Y., & Sun, X. (2024) · Applied Sciences · doi:10.3390/app14177455 · Cited by 86
Recent Advances in Named Entity Recognition: A Comprehensive Survey and Comparative StudyPreprint
Keraghel, I., Morbieu, S., & Nadif, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.10825 · Cited by 144
RenAIssance: A Survey Into AI Text-to-Image Generation in the Era of Large Model
Bie, F., Yang, Y., Zhou, Z., Ghanem, A., Zhang, M., Yao, Z., Wu, X., Holmes, C., Golnari, P. A., Clifton, D. A., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3522305 · Cited by 96
Improving Workplace Well-being in Modern Organizations: A Review of Large Language Model-based Mental Health Chatbots
Yuan, A., Colato, E. G., Pescosolido, B. A., Song, H., & Samtani, S. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3701041 · Cited by 84
Neural natural language processing for long texts: A survey on classification and summarization
Tsirmpas, D., Gkionis, I., Papadopoulos, G. T., & Mademlis, I. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108231 · Cited by 77
A survey on large language models with multilingualism: recent advances and new frontiers
Huang, K., Mo, F., Zhang, X., Li, H., Li, Y., Zhang, Y., yi, W., Mao, Y., Liu, J., Xu, Y., et al. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11534-5 · Cited by 131
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning
Kim, S., Joo, S., Kim, D., Jang, J., Ye, S., Shin, J., & Seo, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.782 · Cited by 139
Multitask Vision-Language Prompt Tuning
Shen, S., Yang, S., Zhang, T., Zhai, B., Gonzalez, J. E., Keutzer, K., & Darrell, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00556 · Cited by 85
Self-citation
Rethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 123
A Literature Survey on Open Source Large Language Models
Kukreja, S., Kumar, T., Purohit, A., Dasgupta, A., & Guha, D. (2024) · International Conference on Computers in Management and Business (ICCMB) · doi:10.1145/3647782.3647803 · Cited by 74
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D. R., Smith, N. A., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.614 · Cited by 145
Zero-Shot Cross-Lingual Summarization via Large Language Models
Wang, J., Liang, Y., Meng, F., Zou, B., Li, Z., Qu, J., & Zhou, J. (2023) · Workshop on New Frontiers in Summarization (NewSum) · doi:10.18653/v1/2023.newsum-1.2 · Cited by 81
Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions
Li, J., Zhou, H., Huang, S., Cheng, S., & Chen, J. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00655 · Cited by 78
Qibo: A Large Language Model for traditional Chinese medicine
Jia, Y., Ji, X., Wang, X., Zhang, H., Meng, Z., Zhang, J., Chen, Z., Zhuang, P., Xu, D., Guo, W., et al. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127672 · Cited by 68
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges
Winata, G. I., Aji, A. F., Yong, Z., & Solorio, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.185 · Cited by 75
Self-citation
Benchmarking Vision Language Models for Cultural Understanding
Nayak, S., Jain, K., Awal, R., Reddy, S., van Steenkiste, S., Hendricks, L. A., Stańczak, K., & Agrawal, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.329 · Cited by 56
Petals: Collaborative Inference and Fine-tuning of Large Models
Borzunov, A., Baranchuk, D., Dettmers, T., Riabinin, M., Belkada, Y., Chumachenko, A., Samygin, P., & Raffel, C. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · doi:10.18653/v1/2023.acl-demo.54 · Cited by 52
Self-citation
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 143
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
Qin, Y., Cai, Z., Jin, D., Yan, L., Liang, S., Zhu, K., Lin, Y., Han, X., Ding, N., Wang, H., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.499 · Cited by 78
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
Qin, L., Chen, Q., Wei, F., Huang, S., & Che, W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.163 · Cited by 89
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 56
Federated and edge learning for large language models
Piccialli, F., Chiaro, D., Qi, P., Bellandi, V., & Damiani, E. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102840 · Cited by 56
Privacy Preserving Prompt Engineering: A Survey
Edemacu, K., & Wu, X. (2025) · ACM Computing Surveys · doi:10.1145/3729219 · Cited by 72
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, A. V. (2024) · doi:10.20944/preprints202402.0357.v1 · Cited by 296
Machine Translation with Large Language Models: Prompting, Few-shot Learning, and Fine-tuning with QLoRA
Zhang, X., Rajabi, N., Duh, K., & Koehn, P. (2023) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2023.wmt-1.43 · Cited by 56
Scaling Data-Constrained Language Models
Muennighoff, N., Rush, A., Barak, B., Scao, T. L., Tazi, N., Piktus, A., Pyysalo, S., Wolf, T., & Raffel, C. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2191 · Cited by 47
Self-citation
Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech
Plaza-del-Arco, F. M., Nozza, D., & Hovy, D. (2023) · doi:10.18653/v1/2023.woah-1.6 · Cited by 47
A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction
Shui, R., Cao, Y., Wang, X., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.490 · Cited by 58
LMEye: An Interactive Perception Network for Large Language Models
Li, Y., Hu, B., Chen, X., Ma, L., Xu, Y. D., & Zhang, M. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3428317 · Cited by 58
MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
Cai, Y., Wang, L., Wang, Y., De Melo, G., Zhang, Y., Wang, Y., & He, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29723 · Cited by 42
Proposed Framework for Human-like Language Processing of ChatGPT in Academic Writing
Mahyoob, M., Algaraady, J., & Alblwi, A. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i14.41725 · Cited by 48
Exploring the Capabilities of LLMs for Code-Change-Related Tasks
Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., & Li, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709358 · Cited by 41
“For example, the Bloom model [58] contains 176B parameters, 789 times more than CCT5 (223M), is pre-trained on 363B natural language tokens, and significantly outperforms all the models with less than 1B parameters on natural language processing tasks.”
Best Practices for Large Language Models in Radiology
Blüthgen, C., Veen, D. V., Zakka, C., Link, K. E., Fanous, A. H., Daneshjou, R., Frauenfelder, T., Langlotz, C. P., Gatidis, S., & Chaudhari, A. S. (2025) · Radiology · doi:10.1148/radiol.240528 · Cited by 48
CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs
Bhattacharjee, A., Hasan, T., Ahmad, W. U., Li, Y.-F., Kang, Y., & Shahriyar, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.143 · Cited by 44
Several Categories of Large Language Models (LLMs): A Short Survey
Pahune, S. A., & Chandrasekharan, M. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.54677 · Cited by 54
TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties
Kadaoui, K., Magdy, S. M., Waheed, A., Khondaker, M. T. I., El-Shangiti, A. O., Nagoudi, E. M. B., & Abdul-Mageed, M. (2023) · Arabic Natural Language Processing Conference (ArabicNLP) · doi:10.18653/v1/2023.arabicnlp-1.6 · Cited by 46
LLMs-based machine translation for E-commerce
Gao, D., Chen, K., Chen, B., Dai, H.-F., Jin, L., Jiang, W., Ning, W., Yu, S., Xuan, Q., Cai, X., et al. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125087 · Cited by 44
Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts
Nguyen, X.-P., Aljunied, M., Joty, S., & Bing, L. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.192 · Cited by 61
Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark
Choi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.699 · Cited by 38
EcomGPT: Instruction-Tuning Large Language Models with Chain-of-Task Tasks for E-commerce
Li, Y., Ma, S., Wang, X., Huang, S., Jiang, C., Zheng, H.-T., Xie, P., Huang, F., & Jiang, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29820 · Cited by 58
Toward Controllable Generative Design: A Conceptual Design Generation Approach Leveraging the Function–Behavior–Structure Ontology and Large Language Models
Chen, L., Zuo, H., Cai, Z., Yin, Y., Zhang, Y. E., Sun, L., Childs, P., & Wang, B. (2024) · Journal of Mechanical Design · doi:10.1115/1.4065562 · Cited by 41
Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
Wu, Z., Zhang, L., Cao, C., Yu, X., Liu, Z., Zhao, L., Li, Y., Dai, H., Ma, C., Li, G., et al. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3536928 · Cited by 56
Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon
Koto, F., Beck, T., Talat, Z., Gurevych, I., & Baldwin, T. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.18 · Cited by 36
NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task
Abdul-Mageed, M., Elmadany, A., Zhang, C., Nagoudi, E. M. B., Bouamor, H., & Habash, N. Y. (2023) · Arabic Natural Language Processing Conference (ArabicNLP) · doi:10.18653/v1/2023.arabicnlp-1.62 · Cited by 35
Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models
Jain, R., Sojitra, D., Acharya, A., Saha, S., Jatowt, A., & Dandapat, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.418 · Cited by 39
BLESS: Benchmarking Large Language Models on Sentence Simplification
Kew, T., Chi, A., Vásquez-Rodríguez, L., Agrawal, S., Aumiller, D., Alva-Manchego, F. E., & Shardlow, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.821 · Cited by 50
SLM: Bridge the Thin Gap Between Speech and Text Foundation Models
Wang, M., Han, W., Izhak Shafran, J., Wu, Z., Chiu, C., Cao, Y. J., Chen, N., Zhang, Y., Soltau, H., Rubenstein, P. K., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389703 · Cited by 47
Beyond Words: A Comparative Analysis of LLM Embeddings for Effective Clustering
Keraghel, I., Morbieu, S., & Nadif, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-58547-0_17 · Cited by 40
BiMediX: Bilingual Medical Mixture of Experts LLM
Pieri, S., Mullappilly, S. S., Khan, F. S., Anwer, R. M., Khan, S., Baldwin, T., & Cholakkal, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.989 · Cited by 52
Artificial general intelligence for radiation oncology
Liu, C., Liu, Z., Holmes, J., Zhang, L., Zhang, L., Ding, Y., Shu, P., Wu, Z., Dai, H., Li, Y., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100045 · Cited by 39
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
Yong, Z., Zhang, R., Forde, J. Z., Wang, S., Subramonian, A., Lovenia, H., Cahyawijaya, S., Winata, G. I., Sutawika, L., Cruz, J. C. B., et al. (2023) · Workshop on Computational Approaches to Code Switching · doi:10.18653/v1/2023.calcs-1.5 · Cited by 48
Self-citation
Evaluating Diverse Large Language Models for Automatic and General Bug Reproduction
Kang, S., Yoon, J., Askarbekkyzy, N., & Yoo, S. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3450837 · Cited by 40
MasakhaNEWS: News Topic Classification for African languages
Adelani, D. I., Masiak, M., Azime, I. A., Alabi, J. O., Tonja, A. L., Mwase, C., Ogundepo, O., Dossou, B. F. P., Oladipo, A., Nixdorf, D., et al. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.10 · Cited by 46
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback
Lai, V. D., Nguyen, C., Ngo, N., Nguyen, T., Dernoncourt, F., Rossi, R. A., & Nguyen, T. H. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.28 · Cited by 51
TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rules
Manas, K., Zwicklbauer, S., & Paschke, A. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588650 · Cited by 34
To What Extent Have LLMs Reshaped the Legal Domain So Far? A Scoping Literature Review
Padiu, B., Iacob, R. C. A., Rebedea, T. E., & Dascălu, M. (2024) · Information · doi:10.3390/info15110662 · Cited by 38
Multilingual Large Language Models Are Not (Yet) Code-Switchers
Zhang, R., Cahyawijaya, S., Cruz, J. C. B., Winata, G. I., & Aji, A. F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.774 · Cited by 55
Self-citation
XCOT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning
Chai, L., Yang, J. J., Sun, T., Guo, H., Liu, J., Wang, B., Liang, X., Bai, J., Li, T., Peng, Q., et al. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34524 · Cited by 30
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Qi, J., Fernández, R., & Bisazza, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.658 · Cited by 59
Enhancing Zero-Shot Crypto Sentiment With Fine-Tuned Language Model and Prompt Engineering
Wahidur, R. S. M., Tashdeed, I., Kaur, M., & Lee, H.-N. (2024) · IEEE Access · doi:10.1109/access.2024.3350638 · Cited by 30

Authors: Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M. S., Shen, S., Yong, Z. X., Schoelkopf, H., Tang, X., Radev, D., Aji, A. F., Almubarak, K., Albanie, S., Alyafeai, Z., Webson, A., Raff, E., & Raffel, C.
Affiliations: University of Cambridge (GB)
Google (United States) (US)
Booz Allen Hamilton (United States) (US)
University of California, Berkeley (US)
Brown University (US)
King Fahd University of Petroleum and Minerals (SA)
Corresponding author: Muennighoff, Niklas
ORCID: Muennighoff, Niklas
Biderman, Stella
Shen, Sheng
Tang, Xiangru
Radev, Dragomir
Aji, Alham Fikri
Almubarak, Khalid
Albanie, Samuel
Alyafeai, Zaid
Raff, Edward
Raffel, Colin
Venue: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Publisher: Association for Computational Linguistics
Published: 2023
Pages: 15991-16111
ISSN: 0736-587X (ISSN-L)
DOI: 10.18653/v1/2023.acl-long.891
OpenAlex: W4385571124
Cited by: 361
References: 33
Citations by year: 2026: 57 · 2025: 76 · 2024: 112 · 2023: 116
FWCI: 63.96
Type: conference paper
Language: en
Open access: hybrid
doi.org
Licence: cc-by
Locations: Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
Funders: Centre National de la Recherche Scientifique
Grand Équipement National De Calcul Intensif
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords: cross-lingual generalization, multilingual language models, transfer learning, low-resource languages

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Crosslingual Generalization through Multitask Finetuning

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks