The DOI matches its DataCite record; 3 of the 13 references it cites were found in a registry or an index, and 10 are not checked yet.
Scaling Instruction-Finetuned Language Models
Open accessAbstract
Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, the latest generation of language models is already able to perform above the human level in the majority of tasks from standard benchmark datasets (11, 12), making purely performance-based evaluation less meaningful as time progresses.”
From Using cognitive psychology to understand GPT-3 (2023) · cites this paper as [12]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Classical and instruction-following summarizers such as BART (Lewis et al., 2019), FLAN-T5 (Chung et al., 2022), and compression-oriented methods like LLMLingua (Jiang et al., 2023) generate concise summaries but often remove structural dependencies required for multi-step reasoning.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Chung et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, by drawing on medical texts present within their training datasets, general-purpose models such as Flan-PaLM or ChatGPT can accurately answer medical questions, achieving passing scores on the United States Medical Licensing Exam 10,52,53 .”
From Foundation models for generalist medical artificial intelligence (2023) · cites this paper as [52]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Instruction fine-tuning (IFT) (Chung et al., 2022; Wang et al., 2022) is another well-known strategy to make language models truthful.”
From Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023) · cites this paper as Chung et al., 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Furthermore, recent studies [60, 129] have confirmed the effectiveness of supervised fine-tuning to achieve exceptional performance on unseen tasks, showcasing their remarkable generalization abilities.”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 78
- Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. Explanations for CommonsenseQA: New Dataset and Models. In ACL , 2021. URL https: //aclanthology.org/2021.acl-long.238 .
- Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. Muppet: Massive multi-task representations with pre-finetuning. In EMNLP , 2021. URL https:// aclanthology.org/2021.emnlp-main.468 .
- Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, and Srinivas Chappidi. Open-domain question answering goes conversational via question rewriting. In NAACL , 2021. URL https://aclanthology.org/2021.naacl-main.44/ .
- Ankur Bapna, Isaac Caswell, Julia Kreutzer, Orhan Firat, Daan van Esch, Aditya Siddhant, Mengmeng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, Theresa Breiner, Vera Axelrod, Jason Riesa, Yuan Cao, Mia Xu Chen, Klaus Macherey, Maxim Krikun, Pidong Wang, Alexander Gutkin, Apurva Shah, Yanping Huang, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. Building machine translation systems for the next thousand languages, 2022. URL https://arxiv.org/abs/2205.03983 . · arXiv
- Shaily Bhatt, Sunipa Dev, Partha Talukdar, Shachi Dave, and Vinodkumar Prabhakaran. Re-contextualizing fairness in nlp: The case of india, 2022. URL https://arxiv.org/abs/2209.12226 . · arXiv
- Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 , 2021. · arXiv
- Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification, 2019a. URL https://arxiv.org/abs/ 1903.04561 .
- Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification, 2019b. URL https://arxiv.org/abs/ 1903.04561 .
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. NeurIPS , 2020. URL https://proceedings.neurips.cc/paper/2020/file/ 1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf .
- Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. e-SNLI: Natural language inference with natural language explanations. NeurIPS , 2018. URL https://arxiv.org/pdf/1812.01193. pdf . · arXiv
- Oana-Maria Camburu, Brendan Shillingford, Pasquale Minervini, Thomas Lukasiewicz, and Phil Blunsom. Make up your mind! Adversarial generation of inconsistent natural language explanations. In ACL , 2020. URL https://aclanthology.org/2020.acl-main.382 .
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 , 2021. URL https://arxiv.org/abs/2107.03374 . · arXiv
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, et al. PaLM: Scaling language modeling with Pathways. arXiv preprint arXiv:2204.02311 , 2022. URL https://arxiv.org/abs/2204.02311 . · arXiv
- Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. TyDiQA: A benchmark for information-seeking question answering in typologically diverse languages. TACL , 2020. URL https://aclanthology.org/2020.tacl-1.30 .
- Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021. URL https://arxiv.org/abs/2110.14168 . · arXiv
- Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu. Dialog inpainting: Turning documents into dialogs. In ICML , 2022. URL https: //proceedings.mlr.press/v162/dai22a.html .
- Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, and Kai-Wei Chang. On measures of biases and harms in nlp, 2021b. URL https://arxiv.org/abs/2108.03362 . · arXiv
- Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V Le, Yonghui Wu, Zhifeng Chen, and Claire Cui. GLaM: Efficient Scaling of Language Models with Mixture-of-Experts. ICML , 2022. URL https://arxiv.org/abs/2112.06905 . · arXiv
- Tanmay Garg, Sarah Masud, Tharun Suresh, and Tanmoy Chakraborty. Handling bias in toxic speech detection: A survey, 2022. URL https://arxiv.org/abs/2202.00126 . · arXiv
- Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. TACL , 2021. doi: 10.1162/ tacl\_a\_00370. URL https://aclanthology.org/2021.tacl-1.21 .
- Nitesh Goyal, Ian Kivlichan, Rachel Rosen, and Lucy Vasserman. Is your toxicity my toxicity? exploring the impact of rater identity on toxicity annotation, 2022. URL https://arxiv.org/abs/2205.00501 . · arXiv
- Saga Hansson, Konstantinos Mavromatakis, Yvonne Adesam, Gerlof Bouma, and Dana Dannélls. The Swedish Winogender dataset. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa) , pages 452-459, Reykjavik, Iceland (Online), May 31-2 June 2021. Linköping University Electronic Press, Sweden. URL https://aclanthology.org/2021.nodalida-main.52 .
- Ali Hasan, Shea Brown, Jovana Davidovic, Benjamin Lange, and Mitt Regan. Algorithmic bias and risk assessments: Lessons from practice. Digital Society , 1(1):1-15, 2022.
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. ICLR , 2020. URL https://openreview.net/ forum?id=d7KBjmI3GmQ .
- Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 , 2022. · arXiv
- Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. In arxiv , 2022. URL https://openreview.net/forum?id=NiEtU7blzN .
- Norman P Jouppi, Doe Hyun Yoon, George Kurian, Sheng Li, Nishant Patil, James Laudon, Cliff Young, and David Patterson. A domain-specific supercomputer for training deep neural networks. Communications of the ACM , 63(7):67-78, 2020.
- Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. UnifiedQA: Crossing format boundaries with a single QA system. In Findings of the Association for Computational Linguistics: EMNLP 2020 , 2020. URL https://aclanthology.org/2020.findings-emnlp. 171 .
- Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. QASC: A dataset for question answering via sentence composition. In AAAI , 2020. URL https://arxiv.org/abs/1910.11473 . · arXiv
- Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. NeurIPS , 2022. URL https://arxiv.org/abs/2205.11916 . · arXiv
- Matthew Lamm, Jennimaria Palomaki, Chris Alberti, Daniel Andor, Eunsol Choi, Livio Baldini Soares, and Michael Collins. QED: A framework and dataset for explanations in question answering. TACL , 9:790-806, 2021. URL https://arxiv.org/abs/2009.06354 . · arXiv
- Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 , 2019. URL https://arxiv. org/abs/1910.13461 . · arXiv
- Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858 , 2022. URL https://arxiv.org/abs/ 2206.14858 . · arXiv
- Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode. arXiv preprint arXiv:2203.07814 , 2022. URL http://go/arxiv/2203.07814 . · arXiv
- Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. ACL , 2017. URL https://aclanthology.org/ P17-1015 .
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019. URL https://arxiv.org/abs/1907.11692 . · arXiv
- Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. MetaICL: Learning to learn in context. In NAACL , 2022. URL https://aclanthology.org/2022.naacl-main.201 .
- Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. ACL , 2021. URL https://arxiv.org/abs/2104.08773 . · arXiv
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency , pages 220-229, 2019.
- Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114 , 2021. URL http: //arxiv.org/abs/2112.00114 . · arXiv
- Yasumasa Onoe, Michael JQ Zhang, Eunsol Choi, and Greg Durrett. CREAK: A dataset for commonsense reasoning over entity knowledge. In NeurIPS Datasets and Benchmarks Track (Round 2) , 2021. URL https: //arxiv.org/abs/2109.01653 .
- Vishakh Padmakumar, Leonard Lausen, Miguel Ballesteros, Sheng Zha, He He, and George Karypis. Exploring the role of task transferability in large-scale multi-task learning. arXiv preprint arXiv:2204.11117 , 2022. URL https://aclanthology.org/2022.naacl-main.183/ . · arXiv
- Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar, and Chitta Baral. How many data samples is an additional instruction worth? arXiv preprint arXiv:2203.09161 , 2022. URL https://arxiv.org/abs/2203. 09161 . · arXiv
- Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. Data cards: Purposeful and transparent dataset documentation for responsible ai. In Data-centric AI Workshop at NeurIPS , 2021. URL https: //arxiv.org/abs/2204.01075 .
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019. URL https://d4mucfpksywv.cloudfront. net/better-language-models/language\_models\_are\_unsupervised\_multitask\_learners.pdf .
- Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training Gopher. arXiv preprint arXiv:2112.11446 , 2021a. URL https://arxiv.org/abs/2112. 11446 . · arXiv
- Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant M. Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew Johnson, Blake A. Hechtman, Laura Weidinger, Iason Gabriel, William S. Isaac, Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, and Geoffrey Irving. Scaling language models: Methods, analysis & insights from training gopher. CoRR , abs/2112.11446, 2021b. URL https://arxiv.org/abs/2112.11446 . · arXiv
- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21:1-67, 2020. URL https://arxiv.org/abs/1910.10683 . · arXiv
- AdamRoberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, and Andrea Gesmundo. Scaling up models and data with t5x and seqio . arXiv preprint arXiv:2203.17189 , 2022. URL https://arxiv.org/ abs/2203.17189 . · arXiv
- Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. ICLR 2022 , 2021. URL https://arxiv.org/abs/2110.08207 . · arXiv
- Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection, 2021. URL https: //arxiv.org/abs/2111.07997 .
- Timo Schick, Sahana Udupa, and Hinrich Schütze. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP. CoRR , abs/2103.00453, 2021. URL https://arxiv.org/abs/2103.00453 . · arXiv
- Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. Continual-T0: Progressively instructing 50+ tasks to language models without forgetting. arXiv preprint arXiv:2205.12393 , 2022. URL https: //arxiv.org/abs/2205.12393 . · arXiv
- Nikil Roashan Selvam, Sunipa Dev, Daniel Khashabi, Tushar Khot, and Kai-Wei Chang. The tail wagging the dog: Dataset construction biases of social bias benchmarks, 2022. URL https://arxiv.org/abs/2210. 10040 .
- Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In ICML , 2018. URL https://arxiv.org/abs/1804.04235 . · arXiv
- Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N'Mah Yilla, Jess Gallegos, Andrew Smart, Emilio Garcia, and Gurleen Virk. Sociotechnical harms: Scoping a taxonomy for harm reduction, 2022. URL https://arxiv.org/abs/2210.05791 . · arXiv
- Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057 , 2022. URL https://arxiv.org/abs/2210. 03057 . · arXiv
- Eric Michael Smith, Melissa Hall, Melanie Kambadur, Eleonora Presani, and Adina Williams. "i'm sorry to hear that": Finding new biases in language models with a holistic descriptor dataset, 2022. URL https://arxiv.org/abs/2205.09209 . · arXiv
- David So, Wojciech Mańke, Hanxiao Liu, Zihang Dai, Noam Shazeer, and Quoc V. Le. Searching for efficient transformers for language modeling. Advances in Neural Information Processing Systems , 2021. URL https://arxiv.org/abs/2109.08668 . · arXiv
- Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615 , 2022. URL https://arxiv.org/abs/2206.04615 . · arXiv
- Mirac Suzgun, Nathan Scales, Nathaneal Scharli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny ZHou, and Jason Wei. Challenging BIG-Bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261 , 2022. URL https://arxiv.org/abs/ 2210.09261 . · arXiv
- Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge. NeurIPS , 2020. URL https://arxiv. org/abs/2006.06609 .
- Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, and Donald Metzler. Unifying language learning paradigms. arXiv preprint arXiv:2205.05131 , 2022a. URL https://arxiv.org/abs/2205.05131 . · arXiv
- Yi Tay, Jason Wei, Hyung Won Chung, David R. So, Siamak Shakeri, Xavier Garcia, Vinh Q. Tran, Hauixiu Steven Zheng, Jinfeng Rao, Denny Zhou, Donald Metzler, Neil Houlsby, Quoc V. Le, and Mostafa Dehghani. Transcending scaling laws with 0.1% extra compute. In arxiv , 2022b.
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. SuperGlue: A stickier benchmark for general-purpose language understanding systems. NeurIPS , 2019a. URL http://go/arxiv/1905.00537 .
- Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao. Does it make sense? And why? A pilot study for sense making and explanation. In ACL , 2019b. URL https://arxiv.org/abs/1906.00363 . · arXiv
- Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, and Colin Raffel. What language model architecture and pretraining objective work best for zero-shot generalization? ICML , 2022a. URL https://arxiv.org/abs/2204.05832 . · arXiv
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 , 2022b. URL https: //arxiv.org/abs/2203.11171 . · arXiv
- Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv preprint arXiv:2204.07705 , 2022c. URL https://arxiv.org/abs/2204.07705 . · arXiv
- Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. ICLR 2022 , 2021. URL https: //openreview.net/forum?id=gEZrGCozdqR .
- Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. TMLR , 2022a. URL https://openreview.net/forum?id=yzkSU5zdwD .
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. NeurIPS , 2022b. URL https://arxiv.org/abs/2201.11903 . · arXiv
- Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. Detoxifying language models risks marginalizing minority voices, 2021. URL https://arxiv.org/abs/2104.06390 . · arXiv
- Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. ByT5: Towards a token-free future with pre-trained byte-to-byte models. TACL , 2022. URL https://aclanthology.org/2022.tacl-1.17 .
- Michihiro Yasunaga and Percy Liang. Graph-based, self-supervised program repair from diagnostic feedback. In ICML , 2020. URL http://go/arxiv/2005.10636 .
- Qinyuan Ye, Bill Yuchen Lin, and Xiang Ren. Crossfit: A few-shot learning challenge for cross-task generalization in NLP. In EMNLP , 2021. URL https://arxiv.org/abs/2104.08835 . · arXiv
- Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, and Sebastian Gehrmann. Synthbio: A case study in human-ai collaborative curation of text datasets, 2021. URL https://arxiv. org/abs/2111.06467 .
- Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections. EMNLP Findings , 2021. URL https://aclanthology. org/2021.findings-emnlp.244/ .
Cited by
Sorted by citations
Items 301–400 of 1,083
Dr.ICL: Demonstration-Retrieved In-context Learning
Luo, M., Xu, X., Dai, Z., Pasupat, P., Kazemi, M., Baral, C. R., Imbrasaitė, V., & Zhao, V. Y. (2024) · Data Intelligence · doi:10.3724/2096-7004.di.2024.0012 · Cited by 86
Self-citationA survey on occupancy perception for autonomous driving: The information fusion perspective
Xu, H., Chen, J., Meng, S., Wang, Y., & Chau, L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102671 · Cited by 64
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 143
Self-citationVisualWebArena: evaluating multimodal agents on realistic visual web tasks
Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., & Fried, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.50 · Cited by 50
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Sorensen, T., Jiang, L., Hwang, J. D., Levine, S., Pyatkin, V., West, P., Dziri, N., Lu, X., Rao, K., Bhagavatula, C., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29970 · Cited by 101
nach0: multimodal natural and chemical languages foundation model
Livne, M., Miftahutdinov, Z., Tutubalina, E. V., Kuznetsov, M., Polykovskiy, D. A., Brundyn, A., Jhunjhunwala, A., Costa, A. B., Aliper, A., Aspuru‐Guzik, A., et al. (2024) · Chemical Science · doi:10.1039/d4sc00966e · Cited by 61
IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologues
Yang, D., Rao, J., Chen, K., Guo, X., Zhang, Y., Yang, J., & Zhang, Y. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657760 · Cited by 61
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 56
Impacts of generative artificial intelligence on the future of labor market: a systematic review
Salari, N., Beiromvand, M., Hosseinian-Far, A., Habibi, J., Babajani, F., & Mohammadi, M. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100652 · Cited by 49
“Large Language Models (LLMs), that are deep learning algorithms, have demonstrated remarkable performance in undertaking NLP tasks (Chowdhery et al., 2022; Chung et al., 2022; Brown et al., 2020).”
MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records
Fleming, S. L., Lozano, A., Haberkorn, W. J., Jindal, J. A., Reis, E., Thapa, R., Blankemeier, L., Genkins, J. Z., Steinberg, E., Nayak, A., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30205 · Cited by 66
Janus: A Unified Distributed Training Framework for Sparse Mixture-of-Experts Models
Liu, J., Wang, J. H., & Jiang, Y. (2023) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3603269.3604869 · Cited by 54
ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning
Masry, A., Shahmohammadi, M., Parvez, M. R., Hoque, E., & Joty, S. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.619 · Cited by 81
Federated and edge learning for large language models
Piccialli, F., Chiaro, D., Qi, P., Bellandi, V., & Damiani, E. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102840 · Cited by 56
Query Performance Prediction Using Relevance Judgments Generated by Large Language Models
Meng, C., Arabzadeh, N., Askari, A., Aliannejadi, M., & de Rijke, M. (2025) · ACM Transactions on Information Systems · doi:10.1145/3736402 · Cited by 61
Deng, C., Duan, Y., Jin, X., Chang, H., Tian, Y., Liu, H., Wang, Y., Gao, K., Zou, H. P., Jin, Y., et al. (2025) · AI and Ethics · doi:10.1007/s43681-025-00797-3 · Cited by 73
The False Promise of Imitating Proprietary LLMs
Gudibande, A., Wallace, E., Snell, C., Geng, X., Liu, H., Abbeel, P., Levine, S., & Song, D. (2023) · arXiv · doi:10.48550/arxiv.2305.15717 · Cited by 48
“At the same time, open-source LMs are becoming increasingly accurate, with models like LLaMA and FLAN-T5 providing many of the same basic capabilities as their commercial counterparts, albeit at a lower level of performance (Touvron et al., 2023; Chung et al., 2022).”
MethodArtificial intelligence generated content (AIGC) in medicine: A narrative review
Shao, L., Chen, B., Zhang, Z., Zhang, Z., & Chen, X. (2024) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2024073 · Cited by 51
Self-citationZhang, H., Cheah, Y., Alyasiri, O. M., & An, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10633-x · Cited by 51
Liu, S., Yu, Z., Huang, F., Bulbulia, Y., Bergen, A., & Liut, M. (2024) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3649217.3653554 · Cited by 58
Improving accuracy of GPT-3/4 results on biomedical data using a retrieval-augmented language model
Soong, D., Sridhar, S., Si, H., Wagner, J.-S., Sá, A. C. C., Yu, C. Y., Karagoz, K., Guan, M., Kumar, S., Hamadeh, H. K., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000568 · Cited by 59
Mask Grounding for Referring Image Segmentation
Chng, Y. X., Zheng, H. W., Han, Y., Qiu, X., & Huang, G. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02509 · Cited by 62
GenTron: Diffusion Transformers for Image and Video Generation
Chen, S., Xu, M., Ren, J., Cong, Y., He, S., Xie, Y., Sinha, A., Luo, P., Xiang, T., & Pérez-Rúa, J.-M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00616 · Cited by 84
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, A. V. (2024) · doi:10.20944/preprints202402.0357.v1 · Cited by 296
Scaling Data-Constrained Language Models
Muennighoff, N., Rush, A., Barak, B., Scao, T. L., Tazi, N., Piktus, A., Pyysalo, S., Wolf, T., & Raffel, C. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2191 · Cited by 47
Diffusion Models for Audio Semantic Communication
Grassucci, E., Marinoni, C., Rodriguez, A., & Comminiello, D. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447612 · Cited by 47
EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation
Zhou, W., Dou, Z., Cao, Z., Liao, Z., Wang, J., Wang, W., Liu, Y., Komura, T., Wang, W., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72627-9_2 · Cited by 71
AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception
Huang, Y., Sheng, X., Yang, Z., Yuan, Q., Duan, Z., Chen, P., Li, L., Lin, W., & Shi, G. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680649 · Cited by 61
Self-citationFan, J., Yin, Y., Wang, T., Dong, W., Zheng, P., & Wang, L. (2025) · Frontiers of Engineering Management · doi:10.1007/s42524-025-4136-9 · Cited by 53
Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review
Li, M., Enkhtur, A., Yamamoto, B. A., Cheng, F., & Chen, L. (2025) · Open Praxis · doi:10.55982/openpraxis.17.1.750 · Cited by 57
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v2 · Cited by 54
Applications of Generative AI in Healthcare: algorithmic, ethical, legal and societal considerationsPreprint
Okonji, O. R., Yunusov, K., & Gordon, B. (2024) · doi:10.36227/techrxiv.171527587.75649430/v1 · Cited by 51
Task-aware Retrieval with Instructions
Asai, A., Schick, T., Lewis, P., Chen, X., Izacard, G., Riedel, S., Hajishirzi, H., & Yih, W.-T. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.225 · Cited by 46
Self-citationThe Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
Self-citationChuang, Y.-N., Tang, R., Jiang, X., & Hu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104606 · Cited by 46
Agrawal, G., Pal, K. K., Deng, Y., Liu, H., & Chen, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30362 · Cited by 53
Retrieval-Augmented Code Generation for Universal Information Extraction
Guo, Y., Li, Z., Jin, X., Liu, Y., Zeng, Y., Liu, W., Li, X., Pan, Y., Bai, L., Guo, J., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9434-8_3 · Cited by 66
PUB: A Pragmatics Understanding Benchmark for Assessing LLMs’ Pragmatics Capabilities
Sravanthi, S., Doshi, M., Tankala, P., Murthy, R., Dabre, R., & Bhattacharyya, P. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.719 · Cited by 46
Bhattarai, K., Oh, I. Y., Sierra, J. M., Tang, J., Payne, P. R. O., Abrams, Z., & Lai, A. M. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae060 · Cited by 45
Language models are not naysayers: an analysis of language models on negation benchmarks
Truong, T. H., Baldwin, T., Verspoor, K. M., & Cohn, T. (2023) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2023.starsem-1.10 · Cited by 69
Brain-Inspired Remote Sensing Foundation Models and Open Problems: A Comprehensive Survey
Jiao, L., Huang, Z., Lu, X., Liu, X., Yang, Y., Zhao, J., Zhang, J., Hou, B., Yang, S., Liu, F., et al. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3316302 · Cited by 49
Self-citationSymbol tuning improves in-context learning in language models
Wei, J., Hou, L., Lampinen, A. K., Chen, X., Huang, D. Y., Tay, Y., Chen, X., Lu, Y., Zhou, D., Ma, T., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.61 · Cited by 80
Self-citationOn Context Utilization in Summarization with Large Language Models
Ravaut, M., Sun, A., Chen, N. F., & Joty, S. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.153 · Cited by 65
UniIR: Training and Benchmarking Universal Multimodal Information Retrievers
Wei, C., Chen, Y., Chen, H., Hu, H., Zhang, G., Fu, J., Ritter, A., & Chen, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73021-4_23 · Cited by 95
KoSEL: Knowledge subgraph enhanced large language model for medical question answering
Zeng, Z., Cheng, Q., Hu, X., Zhuang, Y., Liu, X., He, K., & Liu, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112837 · Cited by 46
Liusie, A., Manakul, P., & Gales, M. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.8 · Cited by 78
Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
Xiong, K., Ding, X., Cao, Y., Liu, T., & Qin, B. (2023) · arXiv · doi:10.48550/arxiv.2305.11595 · Cited by 45
Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
Shapira, N., Levy, M., Alavi, S. H., Zhou, X., Choi, Y., Goldberg, Y., Sap, M., & Shwartz, V. (2024) · Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.eacl-long.138 · Cited by 44
Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech
Plaza-del-Arco, F. M., Nozza, D., & Hovy, D. (2023) · doi:10.18653/v1/2023.woah-1.6 · Cited by 47
Prompt Tuning in Code Intelligence: An Experimental Evaluation
Wang, C., Yang, Y., Gao, C., Peng, Y., Zhang, H., & Lyu, M. R.-T. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3313881 · Cited by 48
Mačina, J., Daheim, N., Chowdhury, S. P., Sinha, T., Kapur, M., Gurevych, I., & Sachan, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.372 · Cited by 62
Jiang, G., & Chen, J. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106223 · Cited by 45
Exploring Universal Intrinsic Task Subspace for Few-Shot Learning via Prompt Tuning
Qin, Y., Wang, X., Su, Y., Lin, Y., Ding, N., Yi, J., Chen, W., Liu, Z., Li, J., Hou, L., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3430545 · Cited by 72
Self-citationBack to the Future: Towards Explainable Temporal Reasoning with Large Language Models
Yuan, C., Xie, Q., Huang, J., & Ananiadou, S. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645376 · Cited by 62
LMEye: An Interactive Perception Network for Large Language Models
Li, Y., Hu, B., Chen, X., Ma, L., Xu, Y. D., & Zhang, M. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3428317 · Cited by 58
Self-citationEvaluating and Improving Compositional Text-to-Visual Generation
Li, B., Lin, Z., Pathak, D., Li, J., Fei, Y., Wu, K., Xia, X., Zhang, P., Neubig, G., & Ramanan, D. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00538 · Cited by 43
CoEdIT: Text Editing by Task-Specific Instruction Tuning
Raheja, V., Kumar, D., Koo, R., & Kang, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.350 · Cited by 62
The language of prompting: What linguistic properties make a prompt successful?
Leidinger, A., van Rooij, R., & Shutova, E. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.618 · Cited by 68
CLIPSyntel: CLIP and LLM Synergy for Multimodal Question Summarization in Healthcare
Ghosh, A., Acharya, A., Jain, R., Saha, S., Chadha, A. R., & Sinha, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30206 · Cited by 50
VIGC: Visual Instruction Generation and Correction
Wang, B., Wu, F., Han, X. H., Peng, J., Zhong, H., Zhang, P., Dong, X., Li, W., Li, W., Wang, J., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28338 · Cited by 65
Improving Automatic VQA Evaluation Using Large Language Models
Mañas, O., Krojer, B., & Agrawal, A. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28212 · Cited by 42
Cybersecurity in the generative artificial intelligence era
Teo, Z. L., Quek, C. W. N., Wong, J. L. Y., & Ting, D. S. W. (2024) · Asia-Pacific Journal of Ophthalmology · doi:10.1016/j.apjo.2024.100091 · Cited by 50
Hashmi, E., Yayilgan, S. Y., Hameed, I. A., Yamin, M. M., Ullah, M., & Abomhara, M. (2024) · IEEE Access · doi:10.1109/access.2024.3452987 · Cited by 45
Diaz-Garcia, J. A., & Carvalho, J. P. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70029 · Cited by 51
Active Learning Principles for In-Context Learning with Large Language Models
Margatina, K., Schick, T., Αλέτρας, Ν., & Dwivedi-Yu, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.334 · Cited by 72
Knowledge Graph-augmented Language Models for Complex Question Answering
Sen, P., Mavadia, S., & Saffari, A. (2023) · doi:10.18653/v1/2023.nlrse-1.1 · Cited by 41
Opportunities for large language models and discourse in engineering design
Göpfert, J., Weinand, J. M., Kuckertz, P., & Stolten, D. (2024) · Energy and AI · doi:10.1016/j.egyai.2024.100383 · Cited by 52
Abductive Ego-View Accident Video Understanding for Safe Driving Perception
Fang, J., Li, L., Zhou, J., Xiao, J., Yu, H., Lv, C., Xue, J., & Chua, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02080 · Cited by 46
Self-citationLongAlign: A Recipe for Long Context Alignment of Large Language Models
Bai, Y., Lv, X., Zhang, J., He, Y., Qi, J., Hou, L., Tang, J., Dong, Y., & Li, J. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.74 · Cited by 134
Self-citationA multimodal conversational agent for DNA, RNA and protein tasks
de Almeida, B. P., Richard, G., Dalla-Torre, H., Blum, C. F., Hexemer, L., Pandey, P., Laurent, S., Rajesh, C., Lopez, M., Laterre, A., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01047-1 · Cited by 50
Transcending Scaling Laws with 0.1% Extra Compute
Tay, Y., Wei, J., Chung, H. W., Tran, V., So, D. R., Shakeri, S., Garcia, X., Zheng, S., Rao, J., Chowdhery, A., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.91 · Cited by 95
Self-citationUnveiling the Implicit Toxicity in Large Language Models
Wen, J., Ke, P., Sun, H., Zhang, Z., Li, C., Bai, J., & Huang, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.84 · Cited by 76
RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting
Shu, L., Luo, L., Hoskere, J., Zhu, Y., Liu, Y., Tong, S., Chen, J., & Meng, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29863 · Cited by 59
ArcheType: A Novel Framework for Open-Source Column Type Annotation Using Large Language Models
Feuer, B., Liu, Y., Hegde, C., & Freire, J. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3665844.3665857 · Cited by 50
Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation
Xue, J., Deng, Y., Gao, Y., & Li, Y. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3485485 · Cited by 71
Self-citationShapeGPT: 3D Shape Generation With a Unified Multi-Modal Language Model
Yin, F., Chen, X., Zhang, C., Jiang, B., Zhao, Z., Liu, W., Yu, G., & Chen, T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535389 · Cited by 63
Self-citationIs LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment
Raina, V., Liusie, A., & Gales, M. (2024) · arXiv · doi:10.48550/arxiv.2402.14016 · Cited by 40
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking
Zhuang, S., Liu, B., Koopman, B., & Zuccon, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.590 · Cited by 65
SayCanPay: Heuristic Planning with Large Language Models Using Learnable Domain Knowledge
Hazra, R., Martires, P. Z. D., & Raedt, L. D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29991 · Cited by 51
A Review of Advancements and Applications of Pre-Trained Language Models in Cybersecurity
Liu, Z. (2024) · International Symposium on Digital Forensics and Security (ISDFS) · doi:10.1109/isdfs60797.2024.10527236 · Cited by 43
SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning
Wang, B., Liu, Z., Huang, X., Jiao, F., Ding, Y., Aw, A. T., & Chen, N. F. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.22 · Cited by 60
The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks
Chen, X., Tang, S., Zhu, R., Yan, S., Jin, L., Wang, Z., Su, L., Zhang, Z., Wang, X., & Tang, H. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690325 · Cited by 52
Self-citationTian, Y., Han, Y., Chen, X., Wang, W. W., & Chawla, N. V. (2025) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3701551.3703577 · Cited by 63
Self-citationChoi, M., Pei, J., Kumar, S., Shu, C., & Jurgens, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.699 · Cited by 38
Towards Explainable Conversational Recommender Systems
Guo, S., Zhang, S., Sun, W., Ren, P., Chen, Z., & Ren, Z. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591884 · Cited by 41
LENS: A Learnable Evaluation Metric for Text Simplification
Maddela, M., Dou, Y., Heineman, D., & Xu, W. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.905 · Cited by 52
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers
Saad-Falcon, J., Khattab, O., Santhanam, K., Florian, R., Franz, M., Roukos, S., Sil, A., Sultan, M. A., & Potts, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.693 · Cited by 53
Chen, L., Zuo, H., Cai, Z., Yin, Y., Zhang, Y. E., Sun, L., Childs, P., & Wang, B. (2024) · Journal of Mechanical Design · doi:10.1115/1.4065562 · Cited by 41
AlAfnan, M. A. (2024) · Journal of Artificial Intelligence and Technology · doi:10.37965/jait.2024.0549 · Cited by 45
Fine-Tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review
Yu, Y., Rong, G., Shen, H., Zhang, H., Shao, D., Wang, M., Wei, Z., Xu, Y., & Wang, J. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695993 · Cited by 47
Guo, J., Wang, M., Yin, H., Song, B., Chi, Y., Yu, F. R., & Yuen, C. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3496491 · Cited by 42
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
Wang, P., Zhang, D., Li, L., Tan, C., Wang, X., Zhang, M., Ren, K. L., Jiang, B., & Qiu, X. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.585 · Cited by 38
Self-citationMELTing Point: Mobile Evaluation of Language Transformers
Laskaridis, S., Katevas, K., Minto, L., & Haddadi, H. (2024) · Proceedings of the 30th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3636534.3690668 · Cited by 37
Clinical insights: A comprehensive review of language models in medicine
Neveditsin, N., Lingras, P., & Mago, V. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000800 · Cited by 37
Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images
Bitton-Guetta, N., Bitton, Y., Hessel, J., Schmidt, L., Elovici, Y., Stanovsky, G., & Schwartz, R. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00247 · Cited by 37
SemEval-2023 Task 7: Multi-Evidence Natural Language Inference for Clinical Trial Data
Jullien, M., Valentino, M., Frost, H., O’Regan, P., Landers, D., & Freitas, A. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.307 · Cited by 46
Revisiting Automated Topic Model Evaluation with Large Language Models
Stammbach, D., Zouhar, V., Hoyle, A., Sachan, M., & Ash, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.581 · Cited by 52
Serapio, A., Chaudhari, G., Savage, C. H., Lee, Y. J., Vella, M., Sridhar, S., Schroeder, J., Liu, J., Yala, A., & Sohn, J. H. (2024) · BMC Medical Imaging · doi:10.1186/s12880-024-01435-w · Cited by 39
LLMs4OM: Matching Ontologies with Large Language Models
Giglou, H. B., D’Souza, J., Engel, F., & Auer, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-78952-6_3 · Cited by 62
A comprehensive overview of Generative AI (GAI): Technologies, applications, and challenges
Zhang, Z., Zhang, J., Zhang, X., & Mai, W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129645 · Cited by 45
Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following
Ye, S., Hwang, H., Yang, S., Yun, H., Kim, Y., & Seo, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29909 · Cited by 59
Metadata
- Authors
- Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Castro-Ros, A., Pellat, M., Robinson, K., Valter, D., Narang, S., Mishra, G., Yu, A., Zhao, V., Huang, Y., Dai, A., Yu, H., Petrov, S., Chi, E. H., Dean, J., Devlin, J., Roberts, A., Zhou, D., Le, Q. V., & Wei, J.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2210.11416
- OpenAlex
- W4307079201
- Cited by
- 1,441
- References
- 0
- Citations by year
- 2026: 29 · 2025: 395 · 2024: 489 · 2023: 524 · 2022: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- instruction tuning, task scaling, model size scaling, Flan-T5, zero-shot prompting, few-shot prompting, open-ended generation, MMLU, TyDiQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
