CoChat
Sign Up
PreprintCited by 89Topic ModelingAI-based Problem Solving and PlanningNatural Language Processing Techniques

Self-Reflection in LLM Agents: Effects on Problem-Solving Performance

Open access
Renze, Matthew, Guven, Erhan·2024·arXiv
DOI 10.48550/arxiv.2405.06682arXiv 2405.06682
Open this paper in CoChat — free accountOpen PDF

Abstract

In this study, we investigated the effects of self-reflection in large language models (LLMs) on problem-solving performance. We instructed nine popular LLMs to answer a series of multiple-choice questions to provide a performance baseline. For each incorrectly answered question, we instructed eight types of self-reflecting LLM agents to reflect on their mistakes and provide themselves with guidance to improve problem-solving. Then, using this guidance, each self-reflecting agent attempted to re-answer the same questions. Our results indicate that LLM agents are able to significantly improve their problem-solving performance through self-reflection ($p < 0.001$). In addition, we compared the various types of self-reflection to determine their individual contribution to performance. All code and data are available on GitHub at https://github.com/matthewrenze/self-reflection

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2405.06682

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Surveys and targeted studies suggest that many gains attributed to reflection depend on external signals-tests, retrieval, or explicit error flags-with limited success at autonomous corrective reasoning when those signals are weak.

    “Surveys and targeted studies, however, suggest that many gains attributed to reflection depend on external signals-tests, retrieval, or explicit error flags-with limited success at autonomous corrective reasoning when those signals are weak[3, 12-14].”

    From Illusions of reflection: open-ended task reveals systematic failures in Large Language Models' reflective reasoning (2025) · cites this paper as [3]

    Section 2.4 (Process; page unavailable) says the agents received the correct answer as an external feedback signal before reflecting, and section 4.1 (Interpretation; page unavailable) reports that even knowing a previous answer was wrong improved performance. Section 1.2 (Prior Literature; page unavailable) reports studies finding that LLMs struggle to identify reasoning errors without external feedback. But this study did not test weak signals or establish that reflection gains generally depend on tests or retrieval; section 4.4 proposes tool access and retrieval as future research. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Self-reflection empowers an agent to learn from its past experiences, allowing it to reason about and optimize decisions.

    “Self-reflection empowers an agent i to learn from its past experiences, allowing it to reason about and optimize decisions (Moreno & Mayer, 2005;Renze & Guven, 2024;Shinn et al., 2024;Qu et al., 2024).”

    From From Imitation to Introspection: Probing Self-Consciousness in Language Models (2025) · cites this paper as Renze & Guven, 2024

  • While reflection helps LLMs in aggregate, the practical effect is far less pronounced than in [3]'s closed-ended benchmarks.

    “While reflection helps LLMs in aggregate, the practical effect is far less pronounced than in [3]'s closed-ended benchmarks.”

    From Illusions of reflection: open-ended task reveals systematic failures in Large Language Models' reflective reasoning (2025) · cites this paper as [3]

  • Notably, in our open-ended setting, a simple Retry was statistically indistinguishable from Instructions and Keywords after multiplicity control figure 6, whereas prior reports on closed-ended benchmarks found clearer gains for more 'active' reflection styles [3].

    “Notably, in our open-ended setting, a simple Retry was statistically indistinguishable from Instructions and Keywords after multiplicity control figure 6, whereas prior reports on closed-ended benchmarks found clearer gains for more 'active' reflection styles [3].”

    From Illusions of reflection: open-ended task reveals systematic failures in Large Language Models' reflective reasoning (2025) · cites this paper as [3]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 39

  1. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, "Large language models are zero-shot reasoners," in Advances in Neural Information Processing Systems, vol. 35, 5 2022, pp. 22 199-22 213. [Online]. Available: https://arxiv.org/abs/2205.11916 · arXiv
  2. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, "Chain-of-thought prompting elicits reasoning in large language models," arXiv, 1 2022. [Online]. Available: https://arxiv.org/abs/2201.11903 · arXiv
  3. Y. Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba, "Large language models are human-level prompt engineers," The Eleventh International Conference on Learning Representations, 11 2023. [Online]. Available: https://arxiv.org/abs/2211.01910 · arXiv
  4. Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. Bang, D. Chen, H. S. Chan, W. Dai, A. Madotto, and P. Fung, "Survey of hallucination in natural language generation," ACM Computing Surveys, vol. 55, 2 2022. [Online]. Available: http://dx.doi.org/10.1145/3571730 · DOI · Paper page
  5. M. U. Hadi, qasem al tashi, R. Qureshi, A. Shah, amgad muneer, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, S. Mirjalili, Q. Al-Tashi, and A. Muneer, "A survey on large language models: Applications, challenges, limitations, and practical usage," Authorea Preprints, 10 2023. [Online]. Available: https://doi.org/10.36227/techrxiv.23589741.v1 · DOI
  6. A. Payandeh, D. Pluth, J. Hosier, X. Xiao, and V. K. Gurbani, "How susceptible are llms to logical fallacies?" arXiv, 8 2023. [Online]. Available: https://arxiv.org/abs/2308.09853v1 · arXiv
  7. J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. W. Yu, X. Song, and D. Zhou, "Large language models cannot self-correct reasoning yet," arXiv, 10 2023. [Online]. Available: https://arxiv.org/abs/2310.01798 · arXiv
  8. Z. Ji, T. Yu, Y. Xu, N. Lee, E. Ishii, and P. Fung, "Towards mitigating hallucination in large language models via self-reflection," arXiv, 10 2023. [Online]. Available: https://arxiv.org/abs/2310.06271 · arXiv
  9. S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao, "Large language models: A survey," arXiv, 2 2024. [Online]. Available: https://arxiv.org/abs/2402.06196 · arXiv
  10. N. Shinn, F. Cassano, E. Berman, A. Gopinath, K. Narasimhan, and S. Yao, "Reflexion: Language agents with verbal reinforcement learning," arXiv, 3 2023. [Online]. Available: https://arxiv.org/abs/2303.11366 · arXiv
  11. L. Pan, M. Saxon, W. Xu, D. Nathani, X. Wang, and W. Y. Wang, "Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies," arXiv, 8 2023. [Online]. Available: https://arxiv.org/abs/2308.03188 · arXiv
  12. A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark, "Self-refine: Iterative refinement with self-feedback," arXiv, 3 2023. [Online]. Available: https://arxiv.org/abs/2303.17651v2 · arXiv
  13. J. Toy, J. MacAdam, and P. Tabor, "Metacognition is all you need? using introspection in generative agents to improve goal-directed behavior," arXiv, 1 2024. [Online]. Available: https://arxiv.org/abs/2401.10910 · arXiv
  14. Y. Wang and Y. Zhao, "Metacognitive prompting improves understanding in large language models," arXiv, 8 2023. [Online]. Available: https://arxiv.org/abs/2308.05342 · arXiv
  15. A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, "Self-rag: Learning to retrieve, generate, and critique through self-reflection," arXiv, 10 2023. [Online]. Available: https://arxiv.org/abs/2310.11511v1 · arXiv
  16. G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, A. Anandkumar, U. Austin, and U. Madison, "Voyager: An open-ended embodied agent with large language models," arXiv, 5 2023. [Online]. Available: https://arxiv.org/abs/2305.16291v2 · arXiv
  17. Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, R. Zheng, X. Fan, X. Wang, L. Xiong, Y. Zhou, W. Wang, C. Jiang, Y. Zou, X. Liu, Z. Yin, S. Dou, R. Weng, W. Cheng, Q. Zhang, W. Qin, Y. Zheng, X. Qiu, X. Huang, and T. Gui, "The rise and potential of large language model based agents: A survey," arXiv, 9 2023. [Online]. Available: https://arxiv.org/abs/2309.07864v3 · arXiv
  18. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, "React: Synergizing reasoning and acting in language models," arXiv, 10 2022. [Online]. Available: https://arxiv.org/abs/2210.03629 · arXiv
  19. N. Miao, Y. W. Teh, and T. Rainforth, "Selfcheck: Using llms to zero-shot check their own step-by-step reasoning," arXiv, 8 2023. [Online]. Available: https://arxiv.org/abs/2308.00436v3 · arXiv
  20. R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, X. Jiang, K. Cobbe, T. Eloundou, G. Krueger, K. Button, M. Knight, B. Chess, and J. S. Openai, "Webgpt: Browser-assisted question-answering with human feedback," arXiv, 12 2021. [Online]. Available: https://arxiv.org/abs/2112.09332v3 · arXiv
  21. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom, "Toolformer: Language models can teach themselves to use tools," arXiv, 2 2023. [Online]. Available: https://arxiv.org/abs/2302.04761v1 · arXiv
  22. P. Lewis and et al., "Retrieval-augmented generation for knowledge-intensive nlp tasks," arXiv, 5 2020. [Online]. Available: https://arxiv.org/abs/2005.11401 · arXiv
  23. Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang, "Retrieval-augmented generation for large language models: A survey," arXiv, 12 2023. [Online]. Available: https://arxiv.org/abs/2312.10997v5 · arXiv
  24. W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang, "Memorybank: Enhancing large language models with long-term memory," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 19 724-19 731, 5 2023. [Online]. Available: https://arxiv.org/abs/2305.10250v3 · arXiv
  25. Z. Wang, A. Liu, H. Lin, J. Li, X. Ma, and Y. Liang, "Rat: Retrieval augmented thoughts elicit context-aware reasoning in long-horizon generation," arXiv, 3 2024. [Online]. Available: https://arxiv.org/abs/2403.05313 · arXiv
  26. G. Tyen, H. Mansoor, V. Cȃrbune, P. Chen, and T. Mak, "Llms cannot find reasoning errors, but can correct them!" arXiv, 11 2023. [Online]. Available: https://arxiv.org/abs/2311.08516v2 · arXiv
  27. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, "Think you have solved question answering? try arc, the ai2 reasoning challenge," ArXiv, 3 2018. [Online]. Available: https://arxiv.org/abs/1803.05457 · arXiv
  28. R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi, "Hellaswag: Can a machine really finish your sentence?" in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. [Online]. Available: https://arxiv.org/abs/1905.07830 · arXiv
  29. A. Pal, L. K. Umapathi, and M. Sankarasubbu, "Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering," in Proceedings of the Conference on Health, Inference, and Learning. PMLR, 2022, pp. 248-260. [Online]. Available: https://proceedings.mlr.press/v174/pal22a.html
  30. W. Zhong, R. Cui, Y. Guo, Y. Liang, S. Lu, Y. Wang, A. Saied, W. Chen, and N. Duan, "Agieval: A human-centric benchmark for evaluating foundation models," ArXiv, 4 2023. [Online]. Available: https://arxiv.org/abs/2304.06364 · arXiv
  31. J. Liu, L. Cui, H. Liu, D. Huang, Y. Wang, and Y. Zhang, "Logiqa: A challenge dataset for machine reading comprehension with logical reasoning," in International Joint Conference on Artificial Intelligence, 2020. [Online]. Available: https://arxiv.org/abs/2007.08124 · arXiv
  32. S. Wang, Z. Liu, W. Zhong, M. Zhou, Z. Wei, Z. Chen, and N. Duan, "From lsat: The progress and challenges of complex reasoning," IEEE/ACM Transactions on Audio, Speech and Language Processing, vol. 30, pp. 2201-2216, 8 2021. [Online]. Available: https://doi.org/10.1109/TASLP.2022.3164218 · DOI
  33. Gemini-Team, "Gemini: A family of highly capable multimodal models," arXiv, 12 2023.
  34. Gemini-Team, "Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context," 2024. [Online]. Available: https://arxiv.org/abs/2403.05530 · arXiv
  35. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, "Llama 2: Open foundation and fine-tuned chat models," arXiv, 7 2023. [Online]. Available: https://arxiv.org/abs/2307.09288 · arXiv
  36. S. M. Bsharat, A. Myrzakhan, and Z. Shen, "Principled instructions are all you need for questioning llama-1/2, gpt-3.5/4," arXiv, 12 2023. [Online]. Available: https://arxiv.org/abs/2312.16171 · arXiv
  37. M. Renze and E. Guven, "The benefits of a concise chain of thought on problem-solving in large language models," arXiv, 1 2024. [Online]. Available: https://arxiv.org/abs/2401.05618v1 · arXiv
  38. --, "The effect of sampling temperature on problem solving in large language models," arXiv, 2 2024. [Online]. Available: https://arxiv.org/abs/2402.05201v1 · arXiv
  39. Q. McNemar, "Note on the sampling error of the difference between correlated proportions or percentages," Psychometrika, vol. 12, pp. 153-157, 6 1947. [Online]. Available: https://doi.org/10.1007/ · DOI · Paper page

Cited by

Sorted by citations

Authors
Renze, M., & Guven, E.
Affiliations
Johns Hopkins University (US)
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2405.06682
arXiv
2405.06682
OpenAlex
W4396912858
Cited by
89
References
0
Citations by year
2026: 34 · 2025: 48 · 2024: 7
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
self-reflection, large language models, LLM agents, problem-solving performance, human development, multiple-choice questions, meta-learning, self-directed learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review