The DOI matches its DataCite record; 5 of the 5 references it cites were found in a registry or an index.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Open accessAbstract
AbstractThere is a failure mode in large language models that we do not have a good name for, and thatwe therefore tend not to treat seriously enough. It is not hallucination — the model is not assertingsomething false. It is not refusal — the model answers at length. It is the production of responses thatcarry the complete outward form of careful reasoning while the cognitive work that reasoning issupposed to represent has not, in any meaningful sense, occurred. We call this theatrical compliance,and we argue that it is, in practical terms, more dangerous than either of the failure modes thatcurrently dominate alignment research. This paper identifies the phenomenon, characterizes its fiveprincipal forms, explains the asymmetry that makes it particularly costly in high-stakes settings, andoutlines the design requirements for systems intended to resist it. We do not describe such a systemin detail here. Our goal is to establish theatrical compliance as a research problem in its own rightand to argue that addressing it requires instruments operating at a fundamentally different level ofabstraction than task-level prompting frameworks.Keywords: theatrical compliance, large language models, AI reasoning quality, cognitiveprocess evaluation, prompt engineering, metacognitive systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, while recent agent frameworks adopt reasoning heuristics like Chain-of-Thought (CoT) [163,193] and ReAct [136] to simulate deliberative processes, these approaches remain shallow in semantic comprehension.”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [163]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, they also report limitations such as sensitivity to surface form variation, degraded performance under increased compositionality, and poor generalization”
From From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs (2026) · cites this paper as Wei et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In most cases, this enhancement can improve model performance39,40.”
From Enhancing diagnostic capability with multi-agents conversational large language models (2025) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interestingly, a comparatively simple chain-of-thought (CoT) baseline demonstrates competitive accuracy and efficiency.”
From Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs (2025) · cites this paper as [61]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This agentic loop acts as a powerful test-time scaling mechanism [25, 17, 32], utilizing the reasoning capabilities [31] of LLMs at each turn while grounding them through environment responses.”
From The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management (2025) · cites this paper as [31]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The other approach to provide LLMs with userdefined runtime rails is to use chain-of-thought (CoT) (Wei et al., 2022).”
From NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails (2023) · cites this paper as Wei et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The scaling up of network and training data lead to the leap-forward of large language models' capability.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This not only helps to improve the model's transparency, persuasiveness, and reliability, but also facilitates the identification and rectification of potential errors through insightful explanations.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, the Chain of Thought proposed by Jason [38] could trigger LLM to reason and engage in step-by-step thinking, which benefits the tool-using capability.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These components are fundamental for task decomposition, logical inference, action execution, and iterative improvement, following established patterns in agentic systems literature.”
From Understanding and Optimizing Agentic Workflows via Shapley value (2025) · cites this paper as Wei et al. 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“which has been shown to improve the quality of reasoning and future actions71,72”
From Evaluation and mitigation of the limitations of large language models in clinical decision-making (2024) · cites this paper as [71]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Unlike raw finetuning, this approach does scale with model size [4].”
From Exploring Length Generalization in Large Language Models (2022) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, CoT prompting (Wei et al., 2022;Wang et al., 2022) has been proposed to bolster LLM reasoning capabilities, demonstrating improved performance across various reasoning and natural language processing tasks.”
From Large language models as tool makers (2023) · cites this paper as Wei et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Perhaps the most well-known work of using LLMs for reasoning is Chain-of-Thought (CoT) (Wei et al., 2022), which reveals the ability of LLMs to formulate their own 'thinking procedure' for problem solving.”
From ReAct: Synergizing Reasoning and Acting in Language Models (2022) · cites this paper as Wei et al., 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Agentic systems embed iterative reasoning capabilities using frameworks such as ReAct [136], Chain-of-Thought (CoT) prompting [163], and Tree of Thoughts [164].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [163]
The cited source says what the claim attributes to it.
In the citing paper
“For many of our tasks, it is likely that performance could be improved by more advanced prompting techniques; e.g., Wei et al.’s chain-of-thought approach (26) substantially improved performance on last-letter concatenation, which is similar to our acronym task.”
From Embers of autoregression show how large language models are shaped by the problem they are trained to solve (2024) · cites this paper as [26]
The cited source says what the claim attributes to it.
In the citing paper
“In previous work (Wei et al., 2022), chain-of-thought (CoT) has been proposed to improve the ability of LLMs to perform reasoning and derive the final answer by introducing a series of intermediate reasoning steps.”
From HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023) · cites this paper as Wei et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“CoT prompting is a powerful method by breaking down complex tasks into a series of intermediate steps [17].”
From Enhancing hepatopathy clinical trial efficiency: a secure, large language model-powered pre-screening pipeline (2025) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“Chain-of-Thought (CoT) prompting elicits explicit reasoning steps from LLMs, improving performance on multi-step tasks (Wei et al., 2022).”
From VeReaFine: Iterative Verification Reasoning Refinement RAG for Hallucination-Resistant on Open-Ended Clinical QA (2025) · cites this paper as Wei et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“Conversely, techniques such as Chain-of-Thought (CoT) enable models to generate more extensive outputs detailing their reasoning processes.”
From Enhancing diagnostic capability with multi-agents conversational large language models (2025) · cites this paper as [39]
The cited source says what the claim attributes to it.
In the citing paper
“Prompt engineering demonstrated that model behavior could be substantially altered without touching the weights: few-shot examples, role descriptions, chain-of-thought decomposition, and self-consistency traces all changed how the same model performed on the same underlying task [Wei et al., 2022, Wang et al., 2023b, Kojima et al., 2022].”
From Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (2026) · cites this paper as Wei et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“By providing the models with the 'chain of thoughts' [38], such as prompting with 'let us think about it step by step' , the large language models exhibit emergent abilities for reasoning and can arrive at conclusions or judgments according to the evidence or logics.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“Some recent developments [38, 156] involve utilizing chains of thought to prompt reasoning for improved interpretability; however, the opacity of the reasoning process of each step remains a concern, and [157] has questioned the possible unfaithful explanations of chain-of-thought prompting.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“It helps guide the model to output the final answer step by step (Wei et al., 2022;Peng et al., 2023;Kojima et al., 2022).”
From CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation (2023) · cites this paper as Wei et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“Wei et al. [4], Nye et al. [15] and Lewkowycz et al. [5] showed that combining prompting (i.e. in-context learning) with scratchpad strategies present a powerful combination.”
From Exploring Length Generalization in Large Language Models (2022) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“Language models can generate reasoning chains to answer complex questions (Wei et al., 2022).”
From RARR: Researching and revising what language models say, using language models (2023) · cites this paper as Wei et al., 2022
In the citing paper
“GPT is known to have difficulty with answering questions that require multiple hops of reasoning [48].”
From Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses (2023) · cites this paper as [48]
In the citing paper
“This generalization gap can be partially bridged by “chain of thought” approaches, whereby a model is prompted to generate intermediate computation steps before arriving at an answer [147].”
From Dissociating language and thought in large language models (2024) · cites this paper as [147]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
Decentralizing Privacy: Using Blockchain to Protect Personal Data
Zyskind, G., Nathan, O., & Pentland, A. '. (2015) · 2015 IEEE Security and Privacy Workshops · doi:10.1109/spw.2015.27 · Cited by 2,617
On Microtargeting Socially Divisive Ads
Ribeiro, F. N., Saha, K., Babaei, M., Henrique, L., Messias, J., Benevenuto, F., Goga, O., Gummadi, K. P., & Redmiles, E. M. (2019) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3287560.3287580 · Cited by 125
Ji, Y., & Gao, S. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 10
Research works printed but not linked: 85
- Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. 2022. Do as I can, not as I say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691 . · arXiv
- Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. MathQA: Towards interpretable math word problem solving with operationbased formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , Minneapolis, Minnesota. Association for Computational Linguistics.
- Daniel Andor, Luheng He, Kenton Lee, and Emily Pitler. 2019. Giving BERT a calculator: Finding operations and arguments with reading comprehension. EMNLP .
- Jacob Andreas, Dan Klein, and Sergey Levine. 2018. Learning with latent language. NAACL .
- Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 . · arXiv
- Kaj Bostrom, Xinyu Zhao, Swarat Chaudhuri, and Greg Durrett. 2021. Flexible generation of natural language deductions. EMNLP .
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. NeurIPS .
- Jonathon Cai, Richard Shin, and Dawn Song. 2017. Making neural programming architectures generalize via recursion. ICLR .
- Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-SNLI: Natural language inference with natural language explanations. NeurIPS .
- Howard Chen, Jacqueline He, Karthik Narasimhan, and Danqi Chen. 2022. Can rationalization improve robustness? NAACL .
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 . · arXiv
- Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, and Quoc V. Le. 2019. Neural symbolic reader: Scalable integration of distributed and symbolic representations for reading comprehension. ICLR .
- Ting-Rui Chiang and Yun-Nung Chen. 2019. Semantically-aligned equation generation for solving and reasoning math word problems. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 2656-2668, Minneapolis, Minnesota. Association for Computational Linguistics.
- Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020. Transformers as soft reasoners over language. IJCAI .
- Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 . · arXiv
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL .
- Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, and Denny Zhou. 2019. Neural logic machines. ICLR .
- Dheeru Dua, Sameer Singh, and Matt Gardner. 2020. Benefits of intermediate annotations in reading comprehension. ACL .
- Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. TACL .
- Yuling Gu, Bhavana Dalvi Mishra, and Peter Clark. 2022. DREAM: Uncovering mental models behind language models. NAACL .
- Braden Hancock, Paroma Varma, Stephanie Wang, Martin Bringmann, Percy Liang, and Christopher Ré. 2018. Training classifiers with natural language explanations. ACL .
- Peter Hase and Mohit Bansal. 2022. When can models learn from explanations? a formal framework for understanding the roles of explanation data. ACL . https://arxiv.org/abs/2102.02201 · arXiv
- Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 . · arXiv
- Mohammad Javad Hosseini, Hannaneh Hajishirzi, Oren Etzioni, and Nate Kushman. 2014. Learning to solve arithmetic word problems with verb categorization. EMNLP .
- Zhanming Jie, Jierui Li, and Wei Lu. 2022. Learning to reason deductively: Math word problem solving as complex relation extraction. arXiv preprint arXiv:2203.10316 . · arXiv
- Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 . · arXiv
- Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016. MAWPS: A math word problem repository. NAACL .
- Andrew K. Lampinen, Ishita Dasgupta, Stephanie C.Y. Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James L. McClelland, Jane X. Wang, and Felix Hill. 2022. Can language models learn from explanations in context? arXiv preprint arXiv:2204.02329 . · arXiv
- Yihuai Lan, Lei Wang, Qiyuan Zhang, Yunshi Lan, Bing Tian Dai, Yan Wang, Dongxiang Zhang, and Ee-Peng Lim. 2021. MWPToolkit: An open-source framework for deep learning-based math word problem solvers. arXiv preprint arXiv:2109.00799 . · arXiv
- Teven Le Scao and Alexander Rush. 2021. How many data points is a prompt worth? NAACL .
- Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. EMNLP .
- Iddo Lev, Bill MacCartney, Christopher Manning, and Roger Levy. 2004. Solving logic puzzles: From robust processing to precise semantics. Proceedings of the 2nd Workshop on Text Meaning and Interpretation .
- Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. ACL .
- Zhengzhong Liang, Steven Bethard, and Mihai Surdeanu. 2021. Explainable multi-hop verbal reasoning through internal monologue. NAACL .
- Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. ACL . https://doi.org/10.18653/v1/P17-1015 · DOI · Paper page
- Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 . · arXiv
- Bodhisattwa Prasad Majumder, Oana-Maria Camburu, Thomas Lukasiewicz, and Julian McAuley. 2021. Rationale-inspired natural language explanations with commonsense. arXiv preprint arXiv:2106.13876 . · arXiv
- Ana Marasovi´ c, Iz Beltagy, Doug Downey, and Matthew E Peters. 2022. Few-shot self-rationalization with natural language prompts. NAACL Findings .
- Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In ACL .
- Shen Yun Miao, Chao Chun Liang, and Keh Yih Su. 2020. A diverse corpus for evaluating and developing English math word problem solvers. ACL .
- Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837 . · arXiv
- Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020. WT5?! Training text-to-text models to explain their predictions. arXiv preprint arXiv:2004.14546 . · arXiv
- Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2021. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114 . · arXiv
- Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 . · arXiv
- Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are NLP models really able to solve simple math word problems? NAACL .
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. NAACL .
- Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Yan Gao, Qiang Fu, Jian-Guang Lou, and Weizhu Chen. 2022. Reasoning like program executors. arXiv preprint arXiv:2201.11473 . · arXiv
- Piotr Pi˛ ekos, Mateusz Malinowski, and Henryk Michalewski. 2021. Measuring and improving BERT's mathematical abilities by predicting the order of reasoning. ACL .
- Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training Gopher. arXiv preprint arXiv:2112.11446 . · arXiv
- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21:1-67.
- Dheeraj Rajagopal, Vidhisha Balachandran, Eduard H. Hovy, and Yulia Tsvetkov. 2021. SelfExplain: A self-explaining architecture for neural text classifiers. EMNLP .
- Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! Leveraging language models for commonsense reasoning. ACL .
- Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, and Zhiyuan Liu. 2019. NumNet: Machine reading comprehension with numerical reasoning. EMNLP .
- Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. 2021. Measuring attribution in natural language generation models. arXiv preprint arXiv:2112.12870 . · arXiv
- Gabriel Recchia. 2021. Teaching autoregressive language models complex tasks by demonstration. arXiv preprint arXiv:2109.02102 . · arXiv
- Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison-Burch, and Jason Wei. 2022. A recipe for arbitrary text style transfer with large language models. ACL .
- Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems .
- Subhro Roy and Dan Roth. 2015. Solving general arithmetic word problems. EMNLP .
- Subhro Roy, Tim Vieira, and Dan Roth. 2015. Reasoning about Quantities in Natural Language. TACL .
- Mohammed Saeed, Naser Ahmadi, Preslav Nakov, and Paolo Papotti. 2021. RuleBERT: Teaching soft rules to pre-trained language models. EMNLP .
- Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2022. Multitask prompted training enables zero-shot task generalization. ICLR .
- Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, and Qun Liu. 2021. Generate & rank: A multi-task framework for math word problems. In Findings of the Association for Computational Linguistics: EMNLP 2021 .
- Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. NAACL .
- Alon Talmor, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. 2020. Leap-ofthought: Teaching pre-trained models to systematically reason over implicit knowledge. NeurIPS .
- Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, and Jonathan Berant. 2021. CommonsenseQA 2.0: Exposing the limits of ai through gamification. NeurIPS Track on Datasets and Benchmarks .
- Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, and Donald Metzler. 2022. Unifying language learning paradigms. arXiv preprint arXiv:2205.05131 . · arXiv
- Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. LaMDA: Language models for dialog applications. arXiv preprint arXiv:2201.08239 . · arXiv
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022a. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 . · arXiv
- Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022b. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv preprint arXiv:2204.07705 . · arXiv
- Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022a. Finetuned language models are zero-shot learners. ICLR .
- Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022b. Emergent abilities of large language models. Transactions on Machine Learning Research .
- Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2022. Reframing human-AI collaboration for generating free-text explanations. NAACL .
- Sarah Wiegreffe and Ana Marasovi´ c. 2021. Teach me to explain: A review of datasets for explainable NLP. NeurIPS .
- Sarah Wiegreffe, Ana Marasovi´ c, and Noah A. Smith. 2021. Measuring association between labels and free-text rationales. EMNLP .
- Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022a. PromptChainer: Chaining large language model prompts through visual programming. CHI Extended Abstracts .
- Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022b. AI chains: Transparent and controllable human-AI interaction by chaining large language model prompts. CHI . https://dl.acm.org/doi/abs/10.1145/3491102.3517582 · DOI
- Yujun Yan, Kevin Swersky, Danai Koutra, Parthasarathy Ranganathan, and Milad Hashemi. 2020. Neural execution engines: Learning to execute subroutines. NeurIPS .
- Huihan Yao, Ying Chen, Qinyuan Ye, Xisen Jin, and Xiang Ren. 2021. Refining language models with compositional explanations. NeurIPS .
- Xi Ye and Greg Durrett. 2022. The unreliability of explanations in few-shot in-context learning. arXiv preprint arXiv:2205.03401 . · arXiv
- Yordan Yordanov, Vid Kocijan, Thomas Lukasiewicz, and Oana-Maria Camburu. 2021. Few-shot out-of-domain transfer learning of natural language explanations. arXiv preprint arXiv:2112.06204 . · arXiv
- Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using 'annotator rationales' to improve machine learning for text categorization. NAACL .
- Wojciech Zaremba and Ilya Sutskever. 2014. Learning to execute. arXiv preprint arXiv:1410.4615 . · arXiv
- Eric Zelikman, Yuhuai Wu, and Noah D. Goodman. 2022. STaR: Bootstrapping reasoning with reasoning. arXiv preprint arXiv:2203.14465 . · arXiv
- Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. ICML .
- Wangchunshu Zhou, Jinyi Hu, Hanlin Zhang, Xiaodan Liang, Maosong Sun, Chenyan Xiong, and Jian Tang. 2020. Towards interpretable natural language understanding with explanations as latent variables. NeurIPS .
Cited by
Sorted by citations
Items 301–341 of 341
Shahriar, T., & Matsuda, N. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-64299-9_2 · Cited by 4
Experience-guided multi-agent interpretable framework for radiology report summarization
Li, J., Zhou, T., Zhou, Z., Wei, X., Song, H., Wang, Z., Chen, Y., & Lv, H. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.109078 · Cited by 4
Goli (2023) · doi:10.2139/ssrn.4437617 · Cited by 3
Freeman, J., Millikin, R. J., Xu, L., Sharma, I., Moore, B., Lock, C., Shine George, K., Bal, A., Mohanty, C., & Stewart, R. (2025) · BMC Bioinformatics · doi:10.1186/s12859-025-06350-7 · Cited by 4
CyberNER-LLM: Cyber Threat Intelligence Named Entity Recognition With Large Language Model
Liu, X., Lin, W., & Ding, Z. (2026) · Lecture Notes in Computer Science · doi:10.1007/978-981-95-3543-9_28 · Cited by 4
Xiao, B., Yin, Z., & Shan, Z. (2023) · arXiv · doi:10.48550/arxiv.2311.06957 · Cited by 4
“Recent advances like chain-of-thought prompting have improved LLM reasoning abilities [46], enabling applications like GABM which embeds GAs into epidemic models [47].”
BackgroundOASIS: Open Agent Social Interaction Simulations with One Million Agents
Yang, Z., Zhang, Z., Zheng, Z., Jiang, Y., Gan, Z., Wang, Z., Ling, Z., Chen, J., Ma, M., Dong, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.11581 · Cited by 3
“Chain-of-Thought (CoT, Wei et al. (2022)) reasoning is incorporated,”
Bogdanov, I., Lung, C.-H., Kunz, T., Gao, J., Taylor, A., & Zaman, M. (2026) · Proceedings of the ACM Conference on AI and Agentic Systems · doi:10.1145/3786335.3813149 · Cited by 3
Thought Space Explorer: Navigating and Expanding Thought Space for Large Language Model Reasoning
Zhang, J., & Liu, K. (2024) · 2024 IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825638 · Cited by 2
Structured reasoning failures compromise LLM interpretation of clinical oncology notes
Kenaston, M. W., Ayub, U., Parmar, M., Kumar, P., Anjum, M. U., Naqvi, S. A. A., Chaudhuri, A. A., Zakharia, Y., Heath, E. I., Bekaii-Saab, T. S., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02951-5 · Cited by 2
Understanding and Optimizing Agentic Workflows via Shapley value
Yang, Y., Huang, B., Qi, S., Feng, C., Hu, H., Zhu, Y., Hu, J., Zhao, H., He, Z., Liu, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.00510 · Cited by 2
Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
Zelikman, E., Lorch, E., Mackey, L., & Kalai, A. T. (2023) · arXiv · doi:10.48550/arxiv.2310.02304 · Cited by 4
Intuitive physics understanding emerges from self-supervised pretraining on natural videos
Garrido, Q., Ballas, N., Assran, M., Bardes, A., Najman, L., Rabbat, M., Dupoux, E., & LeCun, Y. (2025) · arXiv · doi:10.48550/arxiv.2502.11831 · Cited by 3
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
Kang, H., Zhang, Q., Kundu, S., Jeong, G., Liu, Z., Krishna, T., & Zhao, T. (2024) · arXiv · doi:10.48550/arxiv.2403.05527 · Cited by 2
“However, a stark contrast emerges when applying these methods to complex generative tasks that require models to generate longer responses or involve reasoning, such as mathematical problem-solving (Cobbe et al., 2021) and chain-of-thought (CoT) reasoning (Wei et al., 2023).”
BackgroundMemento: Fine-tuning LLM Agents without Fine-tuning LLMs
Zhou, H., Chen, Y., Guo, S., Yan, X., Lee, K. H., Wang, Z., Lee, K. Y., Zhang, G., Shao, K., Yang, L., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.16153 · Cited by 2
“They are increasingly deployed as autonomous problem solvers (Choudhary et al., 2021, Wei et al., 2022, Yao et al., 2023) spanning various domains.”
ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memory
Ge, Z., Li, H., Wang, Y., Hu, N., Zhang, C. J., & Li, Q. (2026) · Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · doi:10.1145/3770855.3818931 · Cited by 2
Generative Agents Navigating Digital Libraries
Zerhoudi, S., & Granitzer, M. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-0865-2_14 · Cited by 1
SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQL
Lee, J., Baek, I., Kim, B., Bae, H., & Lee, H. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.962 · Cited by 3
Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Ahmed, T., Pai, K. S., Devanbu, P., & Barr, E. T. (2023) · arXiv · Cited by 112
Search Arena: Analyzing Search-Augmented LLMs
Miroyan, M., Wu, T.-H., King, L., Li, T., Pan, J., Hu, X., Chiang, W.-L., Angelopoulos, A. N., Darrell, T., Norouzi, N., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.05334 · Cited by 1
From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
Ahn, J., & Kim, M. (2026) · arXiv · doi:10.48550/arxiv.2607.08028 · Cited by 1
HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
He, Y., Wu, Y., Jia, Y., Mihalcea, R., Chen, Y., & Deng, N. (2023) · arXiv · doi:10.48550/arxiv.2310.16755 · Cited by 1
“Furthermore, we test chain-of-thought prompting (Wei et al., 2022) and conduct a thorough analysis.”
Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling
Wunderlich, F. V., Kaesberg, L. B., Wahle, J. P., Ruas, T., & Gipp, B. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) · doi:10.18653/v1/2026.acl-srw.1 · Cited by 5
Lindenbauer, T., Slinko, I., Felder, L., Bogomolov, E., & Zharov, Y. (2025) · arXiv · doi:10.48550/arxiv.2508.21433
From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs
Wang, H., Liang, W., Fu, Z., Zheng, N., Zhang, Y., Tong, Y., Zhu, T., Jiang, H., Li, C., Wu, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2509.23196
Hindsight: Structured Agent Memory that Retains, Recalls, and Reflects
Latimer, C., Boschi, N., Neeser, A., Bartholomew, C., Srivastava, G., Wang, X., & Ramakrishnan, N. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · doi:10.18653/v1/2026.acl-demo.27 · Cited by 5
Self-citationAn, J., Yang, L., Wang, M., Lu, L., An, W., & Yang, E. (2026) · arXiv · doi:10.48550/arxiv.2603.26323
Phasook, P., Pitijaroonpong, R., Kinchagawat, J., Chinkamol, A., Saengja, T., Udomlapsakul, K., Sawatphol, J., & Ittichaiwong, P. (2025) · Proceedings of the 24th Workshop on Biomedical Language Processing (Shared Tasks) · doi:10.18653/v1/2025.bionlp-share.34
LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
Zhang, G., Meng, F., Wan, G., Li, Z., Wang, K., Yin, Z., Bai, L., & Yan, S. (2025) · arXiv · doi:10.48550/arxiv.2509.24771
From Tasks to Traits: Can LLMs Infer Personality From Brief Assessment Center Behavior?
Härtel, T. M. (2026) · International Journal of Selection and Assessment · doi:10.1111/ijsa.70074
Self-Harness: Harnesses That Improve Themselves
Zhang, H., Zhang, S., Li, K., Zhang, C., Chen, Y., Zhang, Y., Bai, L., & Hu, S. (2026) · arXiv · doi:10.48550/arxiv.2606.09498
Taylor, E. (2026) · Ethics & Human Research · doi:10.1002/eahr.70024
Weatherhead, S., Salim, F., & Belbasis, A. (2025) · arXiv · doi:10.48550/arxiv.2510.18254 · Cited by 1
Avrumova, F., Cesar, M. D., Loggia, G., Strebel, K., Altorfer, F. C., Castro, R. I., Zhu, J., Abjornson, C., Bono, C. M., & Lebl, D. R. (2026) · The Spine Journal · doi:10.1016/j.spinee.2026.08.002
A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
Khan, S., Madhavan, S., & Natarajan, K. (2025) · arXiv · doi:10.48550/arxiv.2506.18957
Post-Thinking in NPC Dialogue: A Paradigm for Reflective Character Models
Carey, K., & Wang, H. (2026) · Proceedings of the 21st International Conference on the Foundations of Digital Games · doi:10.1145/3815598.3815681
InferCept: Efficient Intercept Support for Augmented Large Language Model Inference
Abhyankar, R., He, Z., Srivatsa, V., Zhang, H., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.01869
“For example, Chain-of-Thought (Wei et al., 2023; Zhang et al., 2023) breaks one request into a chain of “thought” steps, each being a new prompt to an LLM that continues the history of thoughts.”
BackgroundWhen Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMs
Mehdizadeh, A., & Hilbert, M. (2025) · arXiv · doi:10.48550/arxiv.2510.19107
From Illusion to Insight: A Taxonomic Survey of Hallucination Mitigation Techniques in LLMs
Kazlaris, I., Antoniou, E., Diamantaras, K., & Bratsas, C. (2025) · doi:10.20944/preprints202508.1942.v1 · Cited by 13
Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
Tie, G., Yuan, Z., Zhao, Z., Hu, C., Gu, T., Zhang, R., Zhang, S., Wu, J., Tu, X., Jin, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.16062 · Cited by 16
Model evaluation for extreme risks
Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., Kokotajlo, D., Marchal, N., Anderljung, M., Kolt, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.15324 · Cited by 66
“For example, after GPT-3 had already existed for many months, researchers demonstrated that chain-of-thought prompting could significantly increase performance (Wei et al., 2022c).”
Background
Metadata
- Authors
- Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- OpenAlex
- W4221143046
- Cited by
- 4,263
- References
- 5
- Citations by year
- 2026: 212 · 2025: 1547 · 2024: 1537 · 2023: 900 · 2022: 66
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, prompt engineering, AI alignment, failure modes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
