The DOI matches its DataCite record; 3 of the 13 references it cites were found in a registry or an index, and 10 are not checked yet.
Scaling Instruction-Finetuned Language Models
Open accessAbstract
Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, the latest generation of language models is already able to perform above the human level in the majority of tasks from standard benchmark datasets (11, 12), making purely performance-based evaluation less meaningful as time progresses.”
From Using cognitive psychology to understand GPT-3 (2023) · cites this paper as [12]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Classical and instruction-following summarizers such as BART (Lewis et al., 2019), FLAN-T5 (Chung et al., 2022), and compression-oriented methods like LLMLingua (Jiang et al., 2023) generate concise summaries but often remove structural dependencies required for multi-step reasoning.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Chung et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, by drawing on medical texts present within their training datasets, general-purpose models such as Flan-PaLM or ChatGPT can accurately answer medical questions, achieving passing scores on the United States Medical Licensing Exam 10,52,53 .”
From Foundation models for generalist medical artificial intelligence (2023) · cites this paper as [52]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Instruction fine-tuning (IFT) (Chung et al., 2022; Wang et al., 2022) is another well-known strategy to make language models truthful.”
From Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023) · cites this paper as Chung et al., 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Furthermore, recent studies [60, 129] have confirmed the effectiveness of supervised fine-tuning to achieve exceptional performance on unseen tasks, showcasing their remarkable generalization abilities.”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 78
- Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. Explanations for CommonsenseQA: New Dataset and Models. In ACL , 2021. URL https: //aclanthology.org/2021.acl-long.238 .
- Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. Muppet: Massive multi-task representations with pre-finetuning. In EMNLP , 2021. URL https:// aclanthology.org/2021.emnlp-main.468 .
- Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, and Srinivas Chappidi. Open-domain question answering goes conversational via question rewriting. In NAACL , 2021. URL https://aclanthology.org/2021.naacl-main.44/ .
- Ankur Bapna, Isaac Caswell, Julia Kreutzer, Orhan Firat, Daan van Esch, Aditya Siddhant, Mengmeng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, Theresa Breiner, Vera Axelrod, Jason Riesa, Yuan Cao, Mia Xu Chen, Klaus Macherey, Maxim Krikun, Pidong Wang, Alexander Gutkin, Apurva Shah, Yanping Huang, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. Building machine translation systems for the next thousand languages, 2022. URL https://arxiv.org/abs/2205.03983 . · arXiv
- Shaily Bhatt, Sunipa Dev, Partha Talukdar, Shachi Dave, and Vinodkumar Prabhakaran. Re-contextualizing fairness in nlp: The case of india, 2022. URL https://arxiv.org/abs/2209.12226 . · arXiv
- Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 , 2021. · arXiv
- Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification, 2019a. URL https://arxiv.org/abs/ 1903.04561 .
- Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification, 2019b. URL https://arxiv.org/abs/ 1903.04561 .
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. NeurIPS , 2020. URL https://proceedings.neurips.cc/paper/2020/file/ 1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf .
- Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. e-SNLI: Natural language inference with natural language explanations. NeurIPS , 2018. URL https://arxiv.org/pdf/1812.01193. pdf . · arXiv
- Oana-Maria Camburu, Brendan Shillingford, Pasquale Minervini, Thomas Lukasiewicz, and Phil Blunsom. Make up your mind! Adversarial generation of inconsistent natural language explanations. In ACL , 2020. URL https://aclanthology.org/2020.acl-main.382 .
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 , 2021. URL https://arxiv.org/abs/2107.03374 . · arXiv
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, et al. PaLM: Scaling language modeling with Pathways. arXiv preprint arXiv:2204.02311 , 2022. URL https://arxiv.org/abs/2204.02311 . · arXiv
- Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. TyDiQA: A benchmark for information-seeking question answering in typologically diverse languages. TACL , 2020. URL https://aclanthology.org/2020.tacl-1.30 .
- Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021. URL https://arxiv.org/abs/2110.14168 . · arXiv
- Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu. Dialog inpainting: Turning documents into dialogs. In ICML , 2022. URL https: //proceedings.mlr.press/v162/dai22a.html .
- Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, and Kai-Wei Chang. On measures of biases and harms in nlp, 2021b. URL https://arxiv.org/abs/2108.03362 . · arXiv
- Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V Le, Yonghui Wu, Zhifeng Chen, and Claire Cui. GLaM: Efficient Scaling of Language Models with Mixture-of-Experts. ICML , 2022. URL https://arxiv.org/abs/2112.06905 . · arXiv
- Tanmay Garg, Sarah Masud, Tharun Suresh, and Tanmoy Chakraborty. Handling bias in toxic speech detection: A survey, 2022. URL https://arxiv.org/abs/2202.00126 . · arXiv
- Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. TACL , 2021. doi: 10.1162/ tacl\_a\_00370. URL https://aclanthology.org/2021.tacl-1.21 .
- Nitesh Goyal, Ian Kivlichan, Rachel Rosen, and Lucy Vasserman. Is your toxicity my toxicity? exploring the impact of rater identity on toxicity annotation, 2022. URL https://arxiv.org/abs/2205.00501 . · arXiv
- Saga Hansson, Konstantinos Mavromatakis, Yvonne Adesam, Gerlof Bouma, and Dana Dannélls. The Swedish Winogender dataset. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa) , pages 452-459, Reykjavik, Iceland (Online), May 31-2 June 2021. Linköping University Electronic Press, Sweden. URL https://aclanthology.org/2021.nodalida-main.52 .
- Ali Hasan, Shea Brown, Jovana Davidovic, Benjamin Lange, and Mitt Regan. Algorithmic bias and risk assessments: Lessons from practice. Digital Society , 1(1):1-15, 2022.
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. ICLR , 2020. URL https://openreview.net/ forum?id=d7KBjmI3GmQ .
- Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 , 2022. · arXiv
- Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. In arxiv , 2022. URL https://openreview.net/forum?id=NiEtU7blzN .
- Norman P Jouppi, Doe Hyun Yoon, George Kurian, Sheng Li, Nishant Patil, James Laudon, Cliff Young, and David Patterson. A domain-specific supercomputer for training deep neural networks. Communications of the ACM , 63(7):67-78, 2020.
- Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. UnifiedQA: Crossing format boundaries with a single QA system. In Findings of the Association for Computational Linguistics: EMNLP 2020 , 2020. URL https://aclanthology.org/2020.findings-emnlp. 171 .
- Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. QASC: A dataset for question answering via sentence composition. In AAAI , 2020. URL https://arxiv.org/abs/1910.11473 . · arXiv
- Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. NeurIPS , 2022. URL https://arxiv.org/abs/2205.11916 . · arXiv
- Matthew Lamm, Jennimaria Palomaki, Chris Alberti, Daniel Andor, Eunsol Choi, Livio Baldini Soares, and Michael Collins. QED: A framework and dataset for explanations in question answering. TACL , 9:790-806, 2021. URL https://arxiv.org/abs/2009.06354 . · arXiv
- Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 , 2019. URL https://arxiv. org/abs/1910.13461 . · arXiv
- Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858 , 2022. URL https://arxiv.org/abs/ 2206.14858 . · arXiv
- Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode. arXiv preprint arXiv:2203.07814 , 2022. URL http://go/arxiv/2203.07814 . · arXiv
- Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. ACL , 2017. URL https://aclanthology.org/ P17-1015 .
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019. URL https://arxiv.org/abs/1907.11692 . · arXiv
- Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. MetaICL: Learning to learn in context. In NAACL , 2022. URL https://aclanthology.org/2022.naacl-main.201 .
- Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. ACL , 2021. URL https://arxiv.org/abs/2104.08773 . · arXiv
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency , pages 220-229, 2019.
- Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114 , 2021. URL http: //arxiv.org/abs/2112.00114 . · arXiv
- Yasumasa Onoe, Michael JQ Zhang, Eunsol Choi, and Greg Durrett. CREAK: A dataset for commonsense reasoning over entity knowledge. In NeurIPS Datasets and Benchmarks Track (Round 2) , 2021. URL https: //arxiv.org/abs/2109.01653 .
- Vishakh Padmakumar, Leonard Lausen, Miguel Ballesteros, Sheng Zha, He He, and George Karypis. Exploring the role of task transferability in large-scale multi-task learning. arXiv preprint arXiv:2204.11117 , 2022. URL https://aclanthology.org/2022.naacl-main.183/ . · arXiv
- Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar, and Chitta Baral. How many data samples is an additional instruction worth? arXiv preprint arXiv:2203.09161 , 2022. URL https://arxiv.org/abs/2203. 09161 . · arXiv
- Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. Data cards: Purposeful and transparent dataset documentation for responsible ai. In Data-centric AI Workshop at NeurIPS , 2021. URL https: //arxiv.org/abs/2204.01075 .
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019. URL https://d4mucfpksywv.cloudfront. net/better-language-models/language\_models\_are\_unsupervised\_multitask\_learners.pdf .
- Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training Gopher. arXiv preprint arXiv:2112.11446 , 2021a. URL https://arxiv.org/abs/2112. 11446 . · arXiv
- Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant M. Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d'Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew Johnson, Blake A. Hechtman, Laura Weidinger, Iason Gabriel, William S. Isaac, Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, and Geoffrey Irving. Scaling language models: Methods, analysis & insights from training gopher. CoRR , abs/2112.11446, 2021b. URL https://arxiv.org/abs/2112.11446 . · arXiv
- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21:1-67, 2020. URL https://arxiv.org/abs/1910.10683 . · arXiv
- AdamRoberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, and Andrea Gesmundo. Scaling up models and data with t5x and seqio . arXiv preprint arXiv:2203.17189 , 2022. URL https://arxiv.org/ abs/2203.17189 . · arXiv
- Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. ICLR 2022 , 2021. URL https://arxiv.org/abs/2110.08207 . · arXiv
- Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection, 2021. URL https: //arxiv.org/abs/2111.07997 .
- Timo Schick, Sahana Udupa, and Hinrich Schütze. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP. CoRR , abs/2103.00453, 2021. URL https://arxiv.org/abs/2103.00453 . · arXiv
- Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. Continual-T0: Progressively instructing 50+ tasks to language models without forgetting. arXiv preprint arXiv:2205.12393 , 2022. URL https: //arxiv.org/abs/2205.12393 . · arXiv
- Nikil Roashan Selvam, Sunipa Dev, Daniel Khashabi, Tushar Khot, and Kai-Wei Chang. The tail wagging the dog: Dataset construction biases of social bias benchmarks, 2022. URL https://arxiv.org/abs/2210. 10040 .
- Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In ICML , 2018. URL https://arxiv.org/abs/1804.04235 . · arXiv
- Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N'Mah Yilla, Jess Gallegos, Andrew Smart, Emilio Garcia, and Gurleen Virk. Sociotechnical harms: Scoping a taxonomy for harm reduction, 2022. URL https://arxiv.org/abs/2210.05791 . · arXiv
- Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057 , 2022. URL https://arxiv.org/abs/2210. 03057 . · arXiv
- Eric Michael Smith, Melissa Hall, Melanie Kambadur, Eleonora Presani, and Adina Williams. "i'm sorry to hear that": Finding new biases in language models with a holistic descriptor dataset, 2022. URL https://arxiv.org/abs/2205.09209 . · arXiv
- David So, Wojciech Mańke, Hanxiao Liu, Zihang Dai, Noam Shazeer, and Quoc V. Le. Searching for efficient transformers for language modeling. Advances in Neural Information Processing Systems , 2021. URL https://arxiv.org/abs/2109.08668 . · arXiv
- Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615 , 2022. URL https://arxiv.org/abs/2206.04615 . · arXiv
- Mirac Suzgun, Nathan Scales, Nathaneal Scharli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny ZHou, and Jason Wei. Challenging BIG-Bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261 , 2022. URL https://arxiv.org/abs/ 2210.09261 . · arXiv
- Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge. NeurIPS , 2020. URL https://arxiv. org/abs/2006.06609 .
- Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, and Donald Metzler. Unifying language learning paradigms. arXiv preprint arXiv:2205.05131 , 2022a. URL https://arxiv.org/abs/2205.05131 . · arXiv
- Yi Tay, Jason Wei, Hyung Won Chung, David R. So, Siamak Shakeri, Xavier Garcia, Vinh Q. Tran, Hauixiu Steven Zheng, Jinfeng Rao, Denny Zhou, Donald Metzler, Neil Houlsby, Quoc V. Le, and Mostafa Dehghani. Transcending scaling laws with 0.1% extra compute. In arxiv , 2022b.
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. SuperGlue: A stickier benchmark for general-purpose language understanding systems. NeurIPS , 2019a. URL http://go/arxiv/1905.00537 .
- Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao. Does it make sense? And why? A pilot study for sense making and explanation. In ACL , 2019b. URL https://arxiv.org/abs/1906.00363 . · arXiv
- Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, and Colin Raffel. What language model architecture and pretraining objective work best for zero-shot generalization? ICML , 2022a. URL https://arxiv.org/abs/2204.05832 . · arXiv
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 , 2022b. URL https: //arxiv.org/abs/2203.11171 . · arXiv
- Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv preprint arXiv:2204.07705 , 2022c. URL https://arxiv.org/abs/2204.07705 . · arXiv
- Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. ICLR 2022 , 2021. URL https: //openreview.net/forum?id=gEZrGCozdqR .
- Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. TMLR , 2022a. URL https://openreview.net/forum?id=yzkSU5zdwD .
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. NeurIPS , 2022b. URL https://arxiv.org/abs/2201.11903 . · arXiv
- Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. Detoxifying language models risks marginalizing minority voices, 2021. URL https://arxiv.org/abs/2104.06390 . · arXiv
- Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. ByT5: Towards a token-free future with pre-trained byte-to-byte models. TACL , 2022. URL https://aclanthology.org/2022.tacl-1.17 .
- Michihiro Yasunaga and Percy Liang. Graph-based, self-supervised program repair from diagnostic feedback. In ICML , 2020. URL http://go/arxiv/2005.10636 .
- Qinyuan Ye, Bill Yuchen Lin, and Xiang Ren. Crossfit: A few-shot learning challenge for cross-task generalization in NLP. In EMNLP , 2021. URL https://arxiv.org/abs/2104.08835 . · arXiv
- Ann Yuan, Daphne Ippolito, Vitaly Nikolaev, Chris Callison-Burch, Andy Coenen, and Sebastian Gehrmann. Synthbio: A case study in human-ai collaborative curation of text datasets, 2021. URL https://arxiv. org/abs/2111.06467 .
- Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections. EMNLP Findings , 2021. URL https://aclanthology. org/2021.findings-emnlp.244/ .
Cited by
Sorted by citations
Items 401–500 of 1,083
INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
Chia, Y. K., Hong, P., Bing, L., & Poria, S. (2023) · arXiv · doi:10.48550/arxiv.2306.04757 · Cited by 36
Ding, J.-E., Thao, P. N. M., Peng, W.-C., Wang, J.-Z., Chug, C.-C., Hsieh, M.-C., Tseng, Y.-C., Chen, L., Luo, D., Wu, C., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-71020-2 · Cited by 36
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve
Li, X., & Qiu, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.392 · Cited by 53
Learning Concise and Descriptive Attributes for Visual Recognition
Yan, A., Wang, Y., Zhong, Y., Dong, C., He, Z., Lu, Y., Wang, W. Y., Shang, J., & McAuley, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00287 · Cited by 43
Divide-and-Conquer: Generating UI Code from Screenshots
Wan, Y., Wang, C., Dong, Y., Wang, W., Li, S., Huo, Y., & Lyu, M. R.-T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729364 · Cited by 36
Language Models Hallucinate, but May Excel at Fact Verification
Guan, J., Dodge, J., Wadden, D., Huang, M., & Peng, H. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.62 · Cited by 49
Yi, Z., Xiao, T., & Albert, M. V. (2025) · Information · doi:10.3390/info16020136 · Cited by 43
Mitigating Label Biases for In-context Learning
Fei, Y., Hou, Y., Chen, Z., & Bosselut, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.783 · Cited by 65
Hadi, M. U., Tashi, Q. A., Qureshi, R., Shah, A., Muneer, A., Irfan, M. I., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2025) · doi:10.36227/techrxiv.23589741.v8 · Cited by 49
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Yu, Z., Xiong, C., Shi, Y., & Liu, Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.136 · Cited by 65
Encyclopedic VQA: Visual questions about detailed properties of fine-grained categories
Mensink, T., Uijlings, J., Castrejón, L., Goel, A., Cadar, F., Zhou, H., Sha, F., Araújo, A. D. M., & Ferrari, V. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00289 · Cited by 61
Pelaez, S., Verma, G., Ribeiro, B., & Philip Shapira, A. (2024) · Quantitative Science Studies · doi:10.1162/qss_a_00285 · Cited by 37
Can Language Models Employ the Socratic Method? Experiments with Code Debugging
Al-Hossami, E., Bunescu, R., Smith, J., & Teehan, R. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630799 · Cited by 46
Sun, Q., Huang, K., Yang, X., Tong, R., Zhang, K., & Poria, S. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645678 · Cited by 42
Luo, S. X., Chen, W., Tian, W., Liu, R., Hou, L., Zhang, X., Shen, H., Wu, R., Geng, S., Zhou, Y., et al. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3406372 · Cited by 40
Self-citationOne-Shot Learning as Instruction Data Prospector for Large Language Models
Li, Y., Hui, B., Xia, X., Yang, J., Yang, M., Zhang, L., Si, S., Chen, L.-H., Liu, J., Liu, T., et al. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.252 · Cited by 73
Self-citationWahidur, R. S. M., Kim, S., Choi, H., Bhatti, D. S., & Lee, H.-N. (2025) · IEEE Access · doi:10.1109/access.2025.3542125 · Cited by 40
Inseq: An Interpretability Toolkit for Sequence Generation Models
Sarti, G., Feldhus, N., Sickert, L., & van der Wal, O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.40 · Cited by 56
Prompted LLMs as Chatbot Modules for Long Open-domain Conversation
Lee, G., Hartmann, V., Park, J., Papailiopoulos, D., & Lee, K. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.277 · Cited by 34
Large language models for automated scholarly paper review: A survey
Zhuang, Z., Chen, J., Xu, H., Jiang, Y., & Lin, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103332 · Cited by 34
Video ReCap: Recursive Captioning of Hour-Long Videos
Islam, M. M., Ho, N., Yang, X., Nagarajan, T., Torresani, L., & Bertasius, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01723 · Cited by 34
Language models are not naysayers: an analysis of language models on negation benchmarks
Truong, T. H., Baldwin, T., Verspoor, K., & Cohn, T. (2023) · arXiv · doi:10.48550/arxiv.2306.08189 · Cited by 34
Language Models Meet World Models: Embodied Experiences Enhance Language Models
Xiang, J., Tao, T., Gu, Y., Shu, T., Wang, Z., Yang, Z., & Hu, Z. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3295 · Cited by 34
Exploring Zero and Few-shot Techniques for Intent Classification
Parikh, S., Tiwari, M., Tumbade, P., & Vohra, Q. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.71 · Cited by 46
Jain, R., Sojitra, D., Acharya, A., Saha, S., Jatowt, A., & Dandapat, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.418 · Cited by 39
AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications
Radharapu, B., Robinson, K., Aroyo, L. M., & Lahoti, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.37 · Cited by 61
Self-citationBLESS: Benchmarking Large Language Models on Sentence Simplification
Kew, T., Chi, A., Vásquez-Rodríguez, L., Agrawal, S., Aumiller, D., Alva-Manchego, F. E., & Shardlow, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.821 · Cited by 50
SLM: Bridge the Thin Gap Between Speech and Text Foundation Models
Wang, M., Han, W., Izhak Shafran, J., Wu, Z., Chiu, C., Cao, Y. J., Chen, N., Zhang, Y., Soltau, H., Rubenstein, P. K., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389703 · Cited by 47
The Evolution of Large Language Model: Models, Applications and Challenges
Sindhu, B., Prathamesh, R. P., Sameera, M. B., & Kumaraswamy, S. (2024) · International Conference on Current Trends in Advanced Computing (ICCTAC) · doi:10.1109/icctac61556.2024.10581180 · Cited by 41
FOFO: A Benchmark to Evaluate LLMs’ Format-Following Capability
Xia, C., Xing, C., Du, J., Yang, X., Feng, Y., Xu, R., Yin, W., & Xiong, C. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.40 · Cited by 81
Can LLMs Learn from Previous Mistakes? Investigating LLMs’ Errors to Boost for Reasoning
Tong, Y., Li, D., Wang, S., Wang, Y., Teng, F., & Shang, J. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.169 · Cited by 73
One Model for All: Large Language Models Are Domain-Agnostic Recommendation Systems
Tang, Z., Huan, Z., Li, Z., Zhang, X., Hu, J., Fu, C., Zhou, J., Zou, L., & Li, C. (2024) · ACM Transactions on Information Systems · doi:10.1145/3705727 · Cited by 46
ControlLLM: Augment Language Models with Tools by Searching on Graphs
Liu, Z., Lai, Z., Gao, Z., Cui, E., Li, Z., Zhu, X., Lu, L., Chen, Q., Qiao, Y., Dai, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73254-6_6 · Cited by 60
Huang, K.-H., Chan, H. P., Fung, Y. R., Qiu, H., Zhou, M., Joty, S., Chang, S., & Ji, H. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3513320 · Cited by 56
Prompting and Fine-Tuning Open Source Large Language Models for Stance Classification
Cruickshank, I. J., & Ng, L. H. X. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3725816 · Cited by 51
Davidson, T., & Karell, D. (2025) · Sociological Methods & Research · doi:10.1177/00491241251339184 · Cited by 40
Alleviating Hallucinations of Large Language Models through Induced Hallucinations
Zhang, Y., Cui, L., W., V., & Shi, S. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.459 · Cited by 78
Yong, Z., Zhang, R., Forde, J. Z., Wang, S., Subramonian, A., Lovenia, H., Cahyawijaya, S., Winata, G. I., Sutawika, L., Cruz, J. C. B., et al. (2023) · Workshop on Computational Approaches to Code Switching · doi:10.18653/v1/2023.calcs-1.5 · Cited by 48
“According to . . . ”: Prompting Language Models Improves Quoting from Pre-Training Data
Weller, O., Marone, M., Weir, N., Lawrie, D., Khashabi, D., & Durme, B. V. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.140 · Cited by 52
TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models
Gekhman, Z., Herzig, J., Aharoni, R., Elkind, C., & Szpektor, I. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.127 · Cited by 33
Pre-Training to Learn in Context
Gu, Y., Dong, L., Wei, F., & Huang, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.267 · Cited by 51
Audio Generation with Multiple Conditional Diffusion Model
Guo, Z., Mao, J., Tao, R. M., Yan, L., Ouchi, K., Liu, H., & Wang, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29773 · Cited by 36
Self-citationUncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs
Imran, M. M., Chatterjee, P., & Damevski, K. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639223 · Cited by 39
Fu, X.-Y., Laskar, M. T. R., Khasanova, E., Chen, C., & Shashi Bhushan, T. N. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-industry.33 · Cited by 47
Zhou, J., & Ma, Z. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106151 · Cited by 35
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
Weller, O., Chang, B., MacAvaney, S., Lo, K., Cohan, A., Durme, B. V., Lawrie, D., & Soldaini, L. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.597 · Cited by 76
Reasoning beyond limits: Advances and open problems for LLMs
Ferrag, M. A., Tihanyi, N., & Debbah, M. A. (2025) · ICT Express · doi:10.1016/j.icte.2025.09.003 · Cited by 56
A multi-agentic framework for real-time, autonomous freeform metasurface design
Lupoiu, R. C., Shao, Y., Dai, T., Mao, C., Edée, K., & Fan, J. A. (2025) · Science Advances · doi:10.1126/sciadv.adx8006 · Cited by 40
Autoregressive Structured Prediction with Language Models
Liu, T., Jiang, Y. E., Monath, N., Cotterell, R., & Sachan, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.70 · Cited by 38
Li, W., Hacid, H., Almazrouei, E., & Debbah, M. A. (2023) · AI · doi:10.3390/ai4030039 · Cited by 37
MasakhaNEWS: News Topic Classification for African languages
Adelani, D. I., Masiak, M., Azime, I. A., Alabi, J. O., Tonja, A. L., Mwase, C., Ogundepo, O., Dossou, B. F. P., Oladipo, A., Nixdorf, D., et al. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.10 · Cited by 46
GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot LearningPreprint
Zhao, H., Liu, S., Ma, C., Xu, H., Fu, J., Deng, Z., Kong, L., & Liu, Q. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.05.30.542904 · Cited by 35
Lampinen, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00525 · Cited by 32
Scaling Data-Constrained Language Models
Muennighoff, N., Rush, A. M., Barak, B., Scao, T. L., Piktus, A., Tazi, N., Pyysalo, S., Wolf, T., & Raffel, C. (2023) · arXiv · doi:10.48550/arxiv.2305.16264 · Cited by 49
Silver Lining in the Fake News Cloud: Can Large Language Models Help Detect Misinformation?
Kumar, R., Goddu, B., Saha, S., & Jatowt, A. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3440248 · Cited by 32
Lai, V. D., Nguyen, C., Ngo, N., Nguyen, T., Dernoncourt, F., Rossi, R. A., & Nguyen, T. H. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.28 · Cited by 51
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions
Qin, L., Pan, W., Chen, Q., Liao, L., Zhou, Y., Zhang, Y., Che, W., & Li, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.363 · Cited by 39
RAG-Ex: A Generic Framework for Explaining Retrieval Augmented Generation
Sudhi, V., Bhat, S. R., Rudat, M., & Teucher, R. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657660 · Cited by 39
Wang, H., Zhao, S., Qiang, Z., Li, Z., Liu, C., Xi, N., Du, Y., Qin, B., & Liu, T. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3686807 · Cited by 42
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning
Yang, Z., Pang, T., Feng, H., Wang, H., Chen, W., Zhu, M., & Liu, Q. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.58 · Cited by 32
Salemi, A., & Zamani, H. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744595 · Cited by 32
Teaching Large Language Models to Translate with Comparison
Zeng, J., Meng, F., Yin, Y., & Zhou, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29920 · Cited by 52
Ghadban, Y. A., Lu, H. Y., Adavi, U., Sharma, A., Gara, S., Das, N., Kumar, B., John, R., Praveen, D., & Hirst, J. E. (2023) · medRxiv · doi:10.1101/2023.12.15.23300009 · Cited by 36
Zhao, Q., Wang, R., Cen, Y., Zha, D., Tan, S., Dong, Y., & Tang, J. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1259 · Cited by 32
The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models
Qi, J., Xu, Z., Shen, Y., Liu, M., Jin, D., Wang, Q., & Huang, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.255 · Cited by 48
Wang, L., He, J., Li, S., Liu, N., & Lim, E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53302-0_3 · Cited by 38
PreLog: A Pre-trained Model for Log Analytics
Le, V.-H., & Zhang, H. (2024) · Proceedings of the ACM on Management of Data · doi:10.1145/3654966 · Cited by 36
WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning
Yu, Z., Zhang, X., Shang, N., Huang, Y., Xu, C., Zhao, Y., Hu, W., & Yin, Q. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.280 · Cited by 66
Self-citationGiorgi, J., Toma, A., Xie, R., Chen, S., An, K., Zheng, G., & Wang, B. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.36 · Cited by 31
Goal-Driven Explainable Clustering via Language Descriptions
Wang, Z., Shang, J., & Zhong, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.657 · Cited by 31
Towards LLM-driven Dialogue State Tracking
Feng, Y., Lu, Z., Liu, B., Zhan, L., & Wu, X.-M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.48 · Cited by 30
ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models
Chen, Z., Zhou, K., Zhang, B., Gong, Z., Zhao, X., & Wen, J.-R. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.985 · Cited by 30
A psychometric framework for evaluating and shaping personality traits in large language models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01115-6 · Cited by 30
Large language model trained on clinical oncology data predicts cancer progression
Zhu, M., Lin, H., Jiang, J., Jinia, A. J., Jee, J., Pichotta, K., Waters, M., Rose, D., Schultz, N., Chalise, S., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01780-2 · Cited by 30
Wang, Y., Liu, Y., Shi, C., Li, H., Chen, C., Lu, H., & Yang, Y. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.37 · Cited by 47
Lee, N., Jung, C., Myung, J., Jin, J., Camacho-Collados, J., Kim, J., & Oh, A. H. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.236 · Cited by 30
Šuster, S., Baldwin, T., & Verspoor, K. M. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1749 · Cited by 31
Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
Chen, Z., Liu, K., Wang, Q., Zhang, W., Liu, J., Lin, D., Chen, K., & Zhao, F. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.557 · Cited by 128
AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyond
Zhou, Z., Wan, Y., & Wang, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00135 · Cited by 49
CityLLaVA: Efficient Fine-Tuning for VLMs in City Scenario
Duan, Z., Cheng, H., Xu, D., Wu, X., Zhang, X., Ye, X., & Xie, Z. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00713 · Cited by 36
Visual Question Answering: A Survey of Methods, Datasets, Evaluation, and Challenges
Kim, B. S., Kim, J., Lee, D., & Jang, B. (2025) · ACM Computing Surveys · doi:10.1145/3728635 · Cited by 38
Ke, W., Zheng, Y., Li, Y., Xu, H., Nie, D., Wang, P., & He, Y. (2025) · ACM Transactions on Information Systems · doi:10.1145/3768156 · Cited by 36
Self-citationSay What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge
Chen, J., Shi, W., Fu, Z., Cheng, S., Li, L., & Xiao, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.550 · Cited by 35
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
Yong, Z. X., Schoelkopf, H., Muennighoff, N., Aji, A. F., Adelani, D. I., Almubarak, K., Bari, M. S., Sutawika, L., Kasai, J., Baruwa, A., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.653 · Cited by 30
Multilingual Large Language Models Are Not (Yet) Code-Switchers
Zhang, R., Cahyawijaya, S., Cruz, J. C. B., Winata, G. I., & Aji, A. F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.774 · Cited by 55
Benchmark Evaluations, Applications, and Challenges of Large Vision Language Models: A SurveyPreprint
Li, Z., Wu, X., Du, H., Nghiem, H. T., & Shi, G. (2025) · Qeios · doi:10.32388/gxr68q · Cited by 33
The language of prompting: What linguistic properties make a prompt successful?
Leidinger, A., van Rooij, R., & Shutova, E. (2023) · arXiv · doi:10.48550/arxiv.2311.01967 · Cited by 29
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
Yu, J., Wang, X., Tu, S., Cao, S., Zhang-Li, D., Lv, X., Peng, H., Yao, Z., Zhang, X., Li, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.09296 · Cited by 29
“…T0++ (11B) (Bang et al., 2023), LLaMa (65B) (Touvron et al., 2023), GLM (130B) (Zeng et al., 2022), UL2 (20B) (Tay et al., 2022), FLAN-T5 (11B) (Chung et al., 2022), FLAN-UL2 (20B) (Research, 2022), Alpaca (7B) (Taori et al., 2023), ChatGLM (6B) (Du et al., 2022), Dolly-v2 (12B) (Conover et…”
Self-citationZVQAF: Zero-shot visual question answering with feedback from large language models
Liu, C., Wang, C., Peng, Y., & Li, Z. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127505 · Cited by 30
Torbarina, L., Ferkovic, T., Roguski, Ł., Mihelčić, V., Šarlija, B., & Kraljević, Ž. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100076 · Cited by 29
Wang, G., Ye, J., Cheng, J., Li, T., Chen, Z., Cai, J., He, J., & Zhuang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72114-4_53 · Cited by 29
LLMs4OL 2024 Overview: The 1st Large Language Models for Ontology Learning Challenge
Giglou, H. B., D’Souza, J., & Auer, S. (2024) · Open Conference Proceedings · doi:10.52825/ocp.v4i.2473 · Cited by 39
Luo, M., Fei, H., Li, B., Wu, S., Liu, Q., Poria, S., Cambria, E., Lee, M. L., & Hsu, W. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680705 · Cited by 38
BRAVE: Broadening the Visual Encoding of Vision-Language Models
Kar, O. F., Tonioni, A., Poklukar, P., Kulshrestha, A., Zamir, A., & Tombari, F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_7 · Cited by 61
Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfer
Zhao, Z., Cao, Y., Gong, S., & Patras, I. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00089 · Cited by 29
LLM4DSR: Leveraging Large Language Model for Denoising Sequential Recommendation
Wang, B., Liu, F., Zhang, C., Chen, J., Wu, Y., Zhou, S., Lou, X., Wang, J., Feng, Y., Chen, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3762182 · Cited by 46
Mañas, O., Rodríguez, P., Ahmadi, S., Nematzadeh, A., Goyal, Y., & Agrawal, A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.185 · Cited by 45
Hu, Z., Feng, Y., Tuan, L. A., Hooi, B., & Lipani, A. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615220 · Cited by 29
Evaluation of African American Language Bias in Natural Language Generation
Deas, N., Grieser, J. A., Kleiner, S., Patton, D. U., Turcan, E., & McKeown, K. R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.421 · Cited by 41
Evaluating Large Language Models on Controlled Generation Tasks
Sun, J., Tian, Y., Zhou, W., Xu, N., Hu, Q., Gupta, R., Wieting, J., Peng, N., & Ma, X. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.190 · Cited by 29
Metadata
- Authors
- Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Castro-Ros, A., Pellat, M., Robinson, K., Valter, D., Narang, S., Mishra, G., Yu, A., Zhao, V., Huang, Y., Dai, A., Yu, H., Petrov, S., Chi, E. H., Dean, J., Devlin, J., Roberts, A., Zhou, D., Le, Q. V., & Wei, J.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2210.11416
- OpenAlex
- W4307079201
- Cited by
- 1,441
- References
- 0
- Citations by year
- 2026: 29 · 2025: 395 · 2024: 489 · 2023: 524 · 2022: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- instruction tuning, task scaling, model size scaling, Flan-T5, zero-shot prompting, few-shot prompting, open-ended generation, MMLU, TyDiQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
