LLMs cannot find reasoning errors, but can correct them given the error location
Open accessAbstract
While self-correction has shown promise in improving LLM outputs in terms of style and quality (e.g. Chen et al., 2023b; Madaan et al., 2023), recent attempts to self-correct logical or reasoning errors often cause correct answers to become incorrect, resulting in worse performances overall (Huang et al., 2023). In this paper, we show that poor self-correction performance stems from LLMs' inability to find logical mistakes, rather than their ability to correct a known mistake. Firstly, we benchmark several state-of-the-art LLMs on their mistake-finding ability and demonstrate that they generally struggle with the task, even in highly objective, unambiguous cases. Secondly, we test the correction abilities of LLMs -- separately from mistake finding -- using a backtracking setup that feeds ground truth mistake location information to the model. We show that this boosts downstream task performance across our 5 reasoning tasks, indicating that LLMs' correction abilities are robust. Finally, we show that it is possible to obtain mistake location information without ground truth labels or in-domain training data. We train a small classifier with out-of-domain data, which exhibits stronger mistake-finding performance than prompting a large model. We release our dataset of LLM-generated logical mistakes, BIG-Bench Mistake, to enable further research into locating LLM reasoning mistakes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior works have shown self-verification is not particularly effective for reasoning tasks and is often noisy and ill-calibrated.
“prior works have shown selfverification is not particularly effective for reasoning tasks and is often noisy and ill-calibrated [Tyen et al., 2023, Huang et al., 2023b]”
From AutoMix: Automatically Mixing Language Models (2023) · cites this paper as Tyen et al., 2023
Section 3.3 concludes that prompting models to find mistakes is a poor way to determine whether a reasoning answer is correct, and Section 3.2 describes models identifying mistakes in traces that contain none. These findings support ineffective and noisy self-verification, but the paper does not establish that it is ill-calibrated. (LLM, conf=0.84)
Experimental — accuracy not yet measuredEven state-of-the-art LLMs struggle to reliably detect their own mistakes across a variety of tasks.
“recent studies have shown that even state-of-the-art LLMs struggle to reliably detect their own mistakes across a variety of tasks [58, 28]”
From Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs (2025) · cites this paper as [58]
Section 3.1 reports that five state-of-the-art LLMs struggled to find mistakes across five reasoning tasks. But the evaluated traces were generated by PaLM 2, so most models were detecting another model’s mistakes; the authors say the difference between cross-model and self-evaluation needs further study. The claim’s emphasis on detecting their own mistakes is therefore not fully supported. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning
Miao, N., Teh, Y. W., & Rainforth, T. (2023) · arXiv · doi:10.48550/arxiv.2308.00436 · Cited by 18 · cited 2× · see citing works
“• Unlike Shinn et al. (2023), Miao et al. (2023), and many others, backtracking does not rely on any specific prompt text or phrasing, thereby reducing associated idiosyncrasies.”
Background
Flagged first, then by citations
Research works printed but not linked: 29
- Rohan Anil, Andrew M Dai, Orhan Firat, Melvin John- son, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403. · arXiv
- Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022a. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. · arXiv
- Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022b. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073. · arXiv
- Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2023a. Codet: Code generation with generated tests. In The Eleventh International Conference on Learning Rep- resentations.
- Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Ken- neth Heafield. 2023b. Iterative translation refine- ment with large language models. arXiv preprint arXiv:2306.03856. · arXiv
- Bhavana Dalvi Mishra, Oyvind Tafjord, and Peter Clark. 2022. Towards teachable reasoning systems: Using a dynamic memory of user feedback for con- tinual system improvement. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages 9465-9480, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics. · DOI
- Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Ols- son, Danny Hernandez, et al. 2023. The capacity for moral self-correction in large language models. arXiv preprint arXiv:2302.07459. · arXiv
- Andrew F Hayes and Klaus Krippendorff. 2007. An- swering the call for a standard reliability measure for coding data. Communication methods and mea- sures, 1(1):77-89.
- Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xiny- ing Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798. · arXiv
- Geunwoo Kim, Pierre Baldi, and Stephen McAleer. 2023. Language models can solve computer tasks. arXiv preprint arXiv:2303.17491. · arXiv
- Harrison Lee, Samrat Phatale, Hassan Mansoor, Kel- lie Lu, Thomas Mesnard, Colton Bishop, Victor Car- bune, and Abhinav Rastogi. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267. · arXiv
- Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint arXiv:2305.20050. · arXiv
- Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. · arXiv
- Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Pe- ter Clark, Yiming Yang, and Eduard Hovy. 2021. Think about it! improving defeasible reasoning by first modeling the question scenario. In Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6291-6310, Online and Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics. · DOI
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural In- formation Processing Systems, 35:27730-27744.
- Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2023. Automatically correcting large lan- guage models: Surveying the landscape of diverse self-correction strategies. arXiv preprint arXiv:2308.03188. · arXiv
- Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beat- riz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2023. Refiner: Reasoning feedback on intermediate representations. arXiv preprint arXiv:2304.01904. · arXiv
- William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. Self-critiquing models for assisting human evaluators. arXiv preprint arXiv:2206.05802. · arXiv
- Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. · arXiv
- Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2023. Beyond the im- itation game: Quantifying and extrapolating the ca- pabilities of language models. Transactions on Ma- chine Learning Research.
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowd- hery, and Denny Zhou. 2023. Self-consistency im- proves chain of thought reasoning in language mod- els. In ICLR 2023.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824-24837.
- Sean Welleck, Ximing Lu, Peter West, Faeze Brah- man, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2023. Generating sequences by learning to self-correct. In ICLR 2023.
- Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2023, pages 2550-2575, Singapore. Association for Com- putational Linguistics. · DOI · Paper page
- Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min-Yen Kan, Junxian He, and Qizhe Xie. 2023. Self-evaluation guided beam search for reasoning. In Thirty-seventh Conference on Neural Information Processing Systems.
- Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. · arXiv
- Michihiro Yasunaga and Percy Liang. 2020. Graph- based, self-supervised program repair from diagnos- tic feedback. In International Conference on Ma- chine Learning, pages 10799-10808.
- PMLR.
- Weizhe Yuan, Kyunghyun Cho, and Jason Weston. 2024. System-level natural language feedback. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 2773-2789, St. Julian's, Malta. Association for Computational Linguistics.
Cited by
Sorted by citations
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., & Liu, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.05579 · Cited by 40 · see citing works
AutoMix: Automatically Mixing Language Models
Aggarwal, P., Madaan, A., Anand, A., Potharaju, S. P., Mishra, S., Zhou, P., Gupta, A., Rajagopal, D., Kappaganthu, K., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.12963 · Cited by 19 · see citing works
MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning
Chen, J., Prasad, A., Saha, S., Stengel-Eskin, E., & Bansal, M. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.1660 · Cited by 3 · see citing works
DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines
Singhvi, A., Shetty, M., Tan, S., Potts, C., Sen, K., Zaharia, M., & Khattab, O. (2023) · arXiv · doi:10.48550/arxiv.2312.13382 · Cited by 2 · see citing works
TopoRefine: Iterative Refinement with Reasoning Topology as High-Level Feedback
Liao, H., Hu, S., Zhu, Z., He, H., & Jin, Y. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10888346
Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
Tie, G., Yuan, Z., Zhao, Z., Hu, C., Gu, T., Zhang, R., Zhang, S., Wu, J., Tu, X., Jin, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.16062
“Despite significant progress, recent studies have shown that even state-of-the-art LLMs struggle to reliably detect their own mistakes across a variety of tasks [58, 28].”
- Authors
- Tyen, G., Mansoor, H., Cărbune, V., Chen, P., & Mak, T.
- Affiliations
- University of Cambridge (GB)
- Google (United States) (US)
- ORCID
- Tyen, Gladys
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 13894-13908
- arXiv
- 2311.08516
- OpenAlex
- W4402670356
- Cited by
- 32
- References
- 1
- Citations by year
- 2026: 15 · 2025: 10 · 2024: 7
- FWCI
- 9.44
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Keywords
- error location, backtracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
