Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 101–200 of 3,537
Antonesi, G., Cioara, T., Anghel, I. M., Michalakopoulos, V., Sarmas, E., & Toderean, L. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.126670 · Cited by 142
Mathis, W. S., Zhao, S., Pratt, N., Weleff, J., & Paoli, S. D. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108356 · Cited by 131
Guan, L., Valmeekam, K., Sreedharan, S., & Subbarao Kambhampati, B. (2023) · neural information processing systems · doi:10.52202/075280-3459 · Cited by 52
Embodied LLM Agents Learn to Cooperate in Organized Teams
Guo, X., Huang, K., Liu, J., Fan, W., Vélez, N., Wu, Q., Wang, H., Griffiths, T. L., & Wang, M. (2026) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3637527 · Cited by 123
“Popular prompting techniques for doing so include ReAct [63] and Reflexion [48].”
MethodHave LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models
Arora, D., Singh, H., & Mausam (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.468 · Cited by 51
Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey
Ni, B., Liu, Z., Lei, Y., Wang, L., Zhao, Y., Cheng, X., Zeng, Q., Dong, L., Xia, Y., Kenthapadi, K., et al. (2026) · ACM Computing Surveys · doi:10.1145/3837074 · Cited by 137
“…retrieval introduces additional trustworthiness challenges beyond those of dense retrieval, such as error propagation across multi-step reasoning chains and the difficulty of attributing failures to specific retrieval decisions, and we identify these as important open directions for future work.”
Zhang, Z., Gao, J., Dhaliwal, R. S., & Li, T. J.-J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606800 · Cited by 108
Calibration and Correctness of Language Models for Code
Spieß, C. K., Gros, D., Pai, K. S., Pradel, M., Rabin, M. R. I., Alipour, M. A., Jha, S., Dévanbu, P., & Ahmed, T. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00040 · Cited by 122
“However, LLMs do have very high capacity, and a demonstrated ability to usefully reflect [50], [52] on their generated text.”
Avalon's Game of Thoughts: Battle Against Deception through Recursive ContemplationPreprint
Wang, S.-Z., Liu, C., Zheng, Z., Qi, S., Chen, S., Yang, Q.-S., Zhao, A., Wang, C., Song, S.-J., & Huang, G. (2023) · arXiv.org · doi:10.48550/arxiv.2310.01320 · Cited by 111
Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models
Arora, D., Singh, H. G., & Mausam (2023) · arXiv · doi:10.48550/arxiv.2305.15074 · Cited by 107
Text2BIM: Generating Building Models Using a Large Language Model-Based Multiagent Framework
Du, C., Esser, S., Nousias, S., & Borrmann, A. (2025) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-6386 · Cited by 143
Evaluation of large language models for discovery of gene set function
Hu, M., Alkhairy, S., Lee, I., Pillich, R. T., Fong, D., Smith, K., Bachelder, R., Ideker, T., & Pratt, D. (2025) · Nature Methods · doi:10.1038/s41592-024-02525-x · Cited by 104
“Acute-Phase Response (48) Acute Phase Response (56%) 0.”
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon GenerationPreprint
Wang, Z., Liu, A., Lin, H., Li, J., Ma, X., & Liang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.05313 · Cited by 137
LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing
Wang, B., Li, Y., Lv, Z., Xia, H., Xu, Y., & Sodhi, R. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645143 · Cited by 63
EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms
Yuan, S., Song, K., Chen, J., Tan, X., Li, D., & Yang, D. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.315 · Cited by 162
RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models
Wang, Z., Liu, Z., Zhang, Y., Zhong, A., Wang, J., Yin, F., Fan, L., Wu, L., & Wen, Q. (2024) · Proceedings of the 33rd ACM International Conference on Information and Knowledge Management · doi:10.1145/3627673.3680016 · Cited by 99
Xu, S., Kee, K. F., Li, W., Yamamoto, M., & Riggs, R. E. (2024) · Communication Research · doi:10.1177/00936502231191832 · Cited by 69
Kourani, H., Berti, A., Schuster, D., & van der Aalst, W. M. P. (2025) · Software & Systems Modeling · doi:10.1007/s10270-025-01318-w · Cited by 105
“Based on this feedback, the LLM is prompted to revise the model, ensuring continual improvement.”
Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents
Yang, W., Bi, X., Lin, Y., Chen, S., Zhou, J., & Sun, X. (2024) · neural information processing systems · doi:10.52202/079017-3201 · Cited by 227
CORE: Resolving Code Quality Issues using LLMs
Wadhwa, N., Pradhan, J., Sonwane, A., Sahu, S. P., Natarajan, N., Kanade, A., Parthasarathy, S., & Rajamani, S. K. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643762 · Cited by 118
“For this, we plan to draw upon the traditional static and dynamic analysis techniques for automated feedback generation and use the recent advances based on reinforcement learning and human or tool feedback [19, 47, 52, 54].”
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Fernandes, P., Madaan, A., Liu, E., Farinhas, A., Martins, P. H., Bertsch, A., de Souza, J. G. C., Zhou, S., Wu, T., Neubig, G., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00626 · Cited by 44
Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tPreprint
Dang, Q.-A., & Ngo, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.16219 · Cited by 107
“More recently, techniques such as intermediate self-reflection have been proposed to enable models to iteratively refine their reasoning processes (Shinn et al. 2023; Madaan et al. 2023).”
LLMs Will Always Hallucinate, and We Need to Live with This
Banerjee, S., Agarwal, A., & Singla, S. (2025) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-031-99965-9_39 · Cited by 93
Yuan, M., Bao, P., Yuan, J., Shen, Y., Chen, Z., Xie, Y., Zhao, J., Li, Q., Chen, Y., Zhang, L., et al. (2024) · Medicine Plus · doi:10.1016/j.medp.2024.100030 · Cited by 70
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92
Self-citationWu, M., Xu, J., Yuan, Y., Haffari, G., Wan, L., Luo, W., & Zhang, K. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.25 · Cited by 123
Can Large Language Models Really Improve by Self-critiquing Their Own Plans?
Valmeekam, K., Marquez, M., & Kambhampati, S. (2023) · arXiv · doi:10.48550/arxiv.2310.08118 · Cited by 90
Agent-R: Training Language Model Agents to Reflect via Iterative Self-TrainingPreprint
Yuan, S., Chen, Z., Xi, Z., Ye, J., Du, Z., & Chen, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.11425 · Cited by 102
A Survey of Robot Intelligence with Large Language Models
Jeong, H., Lee, H., Kim, C., & Shin, S. (2024) · Applied Sciences · doi:10.3390/app14198868 · Cited by 121
A platform for the biomedical application of large language models
Lobentanzer, S., Feng, S., Bruderer, N., Maier, A., Díaz, A., Strange, A., Ismail, A., Kulaga, A. Y., Dugourd, A. J. F., Zdrazil, B., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-024-02534-3 · Cited by 94
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages
Qin, L., Chen, Q., Wei, F., Huang, S., & Che, W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.163 · Cited by 190
Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations
Huang, X., Lian, J., Lei, Y., Yao, J., Lian, D., & Xie, X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3731446 · Cited by 89
“Thus, we propose Candidate Bus, which is a separate memory to store the current item candidates, eliminating the need to append them to prompt inputs.”
Self-Reflection in LLM Agents: Effects on Problem-Solving Performance
Renze, M., & Guven, E. (2024) · arXiv · doi:10.48550/arxiv.2405.06682 · Cited by 89
“However, to create effective LLM solutions, prompt engineering is often necessary.”
LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendations
Gao, J., Chen, B., Zhao, X., Liu, W., Li, X., Wang, Y., Wang, W., Guo, H., & Tang, R. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714922 · Cited by 135
Wang, X., Feng, M., Qiu, J., Gu, J., & Zhao, J. (2024) · neural information processing systems · doi:10.52202/079017-1853 · Cited by 240
Planning In Natural Language Improves LLM Search For Code Generation
Wang, E., Cassano, F., Wu, C., Bai, Y., Song, W., Nath, V., Han, Z., Hendryx, S., Yue, S., & Zhang, H. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2409.03733 · Cited by 113
Self-citationAutoKaggle: A Multi-Agent Framework for Autonomous Data Science CompetitionsPreprint
Li, Z., Zang, Q., Ma, D., Guo, J., Zheng, T., Liu, M., Niu, X., Wang, Y., Yang, J., Liu, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.20424 · Cited by 103
ProAgent: Building Proactive Cooperative Agents with Large Language Models
Zhang, C., Yang, K., Hu, S., Wang, Z., Li, G., Sun, Y., Zhang, C., Zhang, Z., Liu, A., Zhu, S.-C., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29710 · Cited by 88
“Recent endeavors have translated LLMs’ reasoning capability into planning by constructing a monologue with feedback (Welleck et al. 2023; Shinn et al. 2023; Paul et al. 2023) to facilitate the reasoning and planning process.”
BackgroundMethodA Survey on (M)LLM-Based GUI AgentsPreprint
Tang, F., Xu, H., Zhang, H., Chen, S., Wu, X., Shen, Y., Zhang, W., Hou, G., Tan, Z., Yan, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.13865 · Cited by 102
“Additionally, Reflexion [60] conducts a more comprehensive evaluation by generating detailed textual outputs based on the trajectories generated by the agent.”
InfluentialZhang, L., Ford, V., Chen, Z., & Chen, J. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.115116 · Cited by 100
Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
Tseng, Y.-M., Huang, Y.-C., Hsiao, T.-Y., Chen, W.-L., Huang, C.-W., Meng, Y., & Chen, Y.-N. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.969 · Cited by 87
Chen, J., Yuan, S., Ye, R., Majumder, B. P., & Richardson, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.05746 · Cited by 103
“Self-reflection or refinement mechanisms [20, 25] enable the model to learn from feedback and improve.”
BackgroundPan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00660 · Cited by 86
“Reflexion (Shinn et al., 2023) extends the method, adding a ‘‘long-term memory’’ to recall past errors and integrating diverse feedback forms.”
BackgroundMethodNatural-Language Agent HarnessesPreprint
Pan, L., Zou, L., Guo, S., Ni, J., & Zheng, H.-T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.25723 · Cited by 89
“Core agent control patterns include reason–act loops (Yao et al., 2023), retrieval augmentation (Lewis et al., 2021), and reflection/self-feedback (Shinn et al., 2023).”
Agentic AI: The age of reasoning—A review
Nisa, U., Shirazi, M., Saip, M. A., & Pozi, M. S. M. (2025) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2025.08.003 · Cited by 141
Strategic Reasoning with Language ModelsPreprint
Gandhi, K., Sadigh, D., & Goodman, N. D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.19165 · Cited by 104
ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM AgentPreprint
Aksitov, R., Miryoosefi, S., Li, Z., Li, D., Babayan, S., Kopparapu, K., Fisher, Z., Guo, R., Prakash, S., Srinivasan, P., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.10003 · Cited by 116
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · arXiv · doi:10.48550/arxiv.2305.13281 · Cited by 85
Generating Automatic Feedback on UI Mockups with Large Language Models
Duan, P., Warner, J., Li, Y., & Hartmann, B. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642782 · Cited by 83
“To accomplish this, the feedback must include these three things: 1) the expected standard, 2) the gap between the quality of work and the standard, and 3) what needs to be done to close this gap.”
BackgroundAgainst The Achilles' Heel: A Survey on Red Teaming for Generative Models
Lin, L., Mu, H., Zhai, Z., Wang, M., Wang, Y., Wang, R., Gao, J., Zhang, Y., Che, W., Baldwin, T., et al. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.17654 · Cited by 110
CodeMind: Evaluating Large Language Models for Implicit and Explicit Code Execution Reasoning
Liu, C., Chen, Y., & Jabbarvand, R. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3691250 · Cited by 89
“The design of heuristics in these techniques is problem-specific rather than task-specific, making their automated generation a separate research problem and out of the scope of this paper [38].”
Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation
Ding, R., Zhang, C., Wang, L., Xu, Y., Ma, M., Wei, Z., Qin, S., Rajmohan, S., Lin, Q., & Zhang, D. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.95 · Cited by 111
AgentBench: Evaluating LLMs as Agents
Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.03688 · Cited by 79
NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
Fan, L., Hua, W., Li, L., Ling, H., & Zhang, Y. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.225 · Cited by 122
Test-Driven Development and LLM-based Code Generation
Mathews, N. S., & Nagappan, M. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695527 · Cited by 140
From screens to scenes: A survey of embodied AI in healthcare
Liu, Y., Cao, X., Chen, T., Jiang, Y., You, J., Wu, M., Wang, X., Feng, M., Jin, Y., & Chen, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103033 · Cited by 91
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment
Guo, Y. F., Wang, Y., Zha, L., & Chen, J. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802284 · Cited by 138
“Works in the instruction-following benchmark [27], [28] like ReAct [29], and Reflexion [30], incorporate feedback into LLM prompts to help planning after each step of the plan is finished.”
TDAG: A multi-agent framework based on dynamic Task Decomposition and Agent Generation
Wang, Y., Wu, Z., Yao, J., & Su, J. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107200 · Cited by 103
(Ir)rationality and cognitive biases in large language models
Macmillan-Scott, O., & Musolesi, M. (2024) · Royal Society Open Science · doi:10.1098/rsos.240255 · Cited by 47
“Others have focused on specific qualities of human decision-making that are not possessed by LLMs, namely the ability to reflect and learn from mistakes, and propose an approach using verbal reinforcement to address this limitation [27].”
BackgroundAssistGUI: Task-Oriented PC Graphical User Interface Automation
Gao, D., Ji, L., Bai, Z., Ouyang, M., Li, P., Mao, D., Wu, Q., Zhang, W., Wang, P., Guo, X., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01262 · Cited by 115
“Some works [23, 27, 27, 33, 40, 50, 50], such as CoT, and ReAct, enhance the model’s capability for better conversation by logical reasoning.”
BackgroundFIMO: A Challenge Formal Dataset for Automated Theorem ProvingPreprint
Liu, C., Shen, J., Xin, H., Liu, Z., Yuan, Y., Wang, H., Ju, W., Zheng, C., Yin, Y., Li, L., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.04295 · Cited by 98
“To ameliorate this issue, we draw inspiration from the concept expounded in (Shinn et al. 2023), wherein verbal reinforcement is employed to facilitate the LLMs’ learning from previous inadequacies.”
BackgroundDo Large Language Models Know What They Don’t Know?
Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.551 · Cited by 75
Agentic Large Language Models, a Survey
Plaat, A., Van Duijn, M., Van Stein, N., Preuss, M., Van der Putten, P., & Batenburg, K. J. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.18675 · Cited by 75
Do Large Language Models Know What They Don't Know?
Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., & Huang, X. (2023) · arXiv · doi:10.48550/arxiv.2305.18153 · Cited by 75
AIGC-empowered smart manufacturing: Prospects and challenges
Leng, J., Zheng, K., Li, R. C., Chen, C., Wang, B., Liu, Q., Chen, X., & Shen, W. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103076 · Cited by 84
Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F., & Ji, H. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.15 · Cited by 74
InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and ReflectionPreprint
Liu, Y., Li, P., Wei, Z., Xie, C., Hu, X., Xu, X., Zhang, S., Han, X., Yang, H., & Wu, F. Q. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.04575 · Cited by 83
“…& Chang, 2023; Zhang et al., 2024b; Huang et al., 2024), and (2) Expectation-reflection reasoning, which fosters adaptive self-correction and reflection (Shinn et al., 2023; Yao et al., 2023; Hu et al., 2024a), enabling agents to learn from past actions and improve decision-making consistency.”
CAPE: Corrective Actions from Precondition Errors using Large Language Models
Raman, S. S., Cohen, V., Idrees, I., Rosen, E., Mooney, R. J., Tellex, S., & Paulius, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611376 · Cited by 112
“Re-flexion [31] converts scalar feedback (from heuristic-based evaluators) into structured linguistic feedback with long-term memory to improve decision making via trial-and-error; in contrast, CAPE does not enable multiple trials nor access retrospective feedback to re-plan from initialization.”
BackgroundMethodIn-Context Learning with Iterative Demonstration Selection
Qin, C., Zhang, A., Chen, C., Dagar, A., & Ye, W. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.438 · Cited by 118
ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search
Zhuang, Y., Chen, X., Yu, T., Mitra, S., Bursztyn, V. S., Rossi, R. A., Sarkhel, S., & Zhang, C. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.13227 · Cited by 98
Large Language Models are Zero Shot Hypothesis ProposersPreprint
Qi, B., Zhang, K., Li, H., Tian, K., Zeng, S., Chen, Z.-R., & Zhou, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.05965 · Cited by 95
“…[25, 29], decision-making [37, 14, 5], and long-term memory capabilities [43, 12], and they are constructed upon the foundation of LLMs. LAAs can autonomously plan sub-goals for complex tasks, execute actions, obtain feedback from the environment, and adjust their behaviors to adapt [40, 36, 26].”
BackgroundCognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 798
“B : Language agents (Ahn et al., 2022; Huang et al., 2022c) place the LLM in a direct feedback loop with the external environment by transforming observations into text and using the LLM to choose actions.”
BackgroundMethodInfluentialSelf-citationQin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16789 · Cited by 71
“Our DFSDT extends Reflexion to a more general method by allowing LLMs to assess different reasoning paths and select the most promising one.”
BackgroundLarge Language Model based Multi-Agents: A Survey of Progress and Challenges
Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., & Zhang, X. (2024) · arXiv · doi:10.48550/arxiv.2402.01680 · Cited by 1,713
“Hence, LLM-based agent has been studied and rapidly developed to understand and generate human-like instructions, facilitating sophisticated interactions and decision-making in a wide range of contexts [Yao et al. , 2023; Shinn et al. , 2023; Li et al. , 2023d].”
BackgroundNie, T., Sun, J., & Ma, W. (2025) · Artificial Intelligence for Transportation · doi:10.1016/j.ait.2025.100003 · Cited by 118
SAIL: Search-Augmented Instruction LearningPreprint
Luo, H., Chuang, Y.-S., Gong, Y., Zhang, T., Kim, Y., Wu, X., Fox, D., Meng, H. M. L., & Glass, J. R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15225 · Cited by 85
A review of LLMs and their applications in the architecture, engineering and construction industry
Kampelopoulos, D., Tsanousa, A., Vrochidis, S., & Kompatsiaris, I. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11241-7 · Cited by 69
“LLM agents demonstrated promising per - formance on a variety of tasks, including code generation (Shinn et al. 2023 ), autopilot systems (Jin et al. 2023 ), complex tasks in gaming environment (Wang et al. 2023 ) and more.”
Stance Detection with Collaborative Role-Infused LLM-Based Agents
Lan, X., Gao, C., Jin, D., & Li, Y. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31360 · Cited by 69
MedCoT: Medical Chain of Thought via Hierarchical Expert
Liu, J., Wang, Y., Du, J., Zhou, J. T., & Liu, Z. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.962 · Cited by 111
Enhancing AI Systems with Agentic Workflows Patterns in Large Language Model
Singh, A., Ehtesham, A., Kumar, S., & Khoei, T. T. (2024) · 2024 IEEE World AI IoT Congress (AIIoT) · doi:10.1109/aiiot61789.2024.10578990 · Cited by 68
“Similarly, Shinn et al. proposed Reflexion, a framework that uses linguistic feedback for language agents to learn from trial-and-error efficiently, achieving notable results in coding and reasoning tasks [4] as illustrated in Fig.”
BackgroundMoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations
Yao, H., Song, Z., Zhou, Y., Ao, T., Chen, B., & Liu, L. (2024) · ACM Transactions on Graphics · doi:10.1145/3658137 · Cited by 103
Proactive Conversational AI: A Comprehensive Survey of Advancements and Opportunities
Deng, Y., Liao, L., Lei, W., Yang, G. H., Lam, W. P., & Chua, T. (2025) · ACM Transactions on Information Systems · doi:10.1145/3715097 · Cited by 120
S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning
Ma, R., Wang, P., Liu, C., Liu, X., Chen, J., Zhang, B., Zhou, X., Du, N., & Li, J. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1104 · Cited by 101
ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction
Wang, H., POSKITT, C. M., Wei, J., & Sun, J. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837413 · Cited by 83
The Hitchhiker's Guide to Program Analysis: A Journey with Large Language ModelsPreprint
Li, H., Hao, Y., Zhai, Y., & Qian, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.00245 · Cited by 86
“We believe our effort only scratched the surface of the vast design space, and hope our work will inspire future research in this exciting direction.”
BackgroundNNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the WildPreprint
Murty, S., Zhu, H., Bahdanau, D., & Manning, C. D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.02907 · Cited by 87
Kim, J., Shin, B., Chung, J.-W., & Rhu, M. (2025) · International Symposium on High-Performance Computer Architecture · doi:10.1109/hpca68181.2026.11408569 · Cited by 64
MaScQA: investigating materials science knowledge of large language models
Zaki, M. F. M., Jayadeva, Mausam, M., & Krishnan, N. M. A. (2024) · Digital Discovery · doi:10.1039/d3dd00188a · Cited by 101
“42,43 However, in this work, we choose the basic prompting methodologies like zero-shot and CoT to investigate existing knowledge in LLMs and present baselines on the newly introduced dataset of MaScQA.”
MethodGu, X., & Krenn, M. (2026) · Proceedings of the National Academy of Sciences · doi:10.48550/arxiv.2405.17044 · Cited by 66
“…instructed the model to follow a self-reflective critique-and-revision procedure inspired by iterative self-feedback and verbal reflection methods [40, 41]: it first briefly explained the guiding concepts, then generated candidate scientific contexts in which they might be combined, criticized…”
LLM-Based Multi-Agent Decision-Making: Challenges and Future Directions
Sun, C.-N., Huang, S., & Pompili, D. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3562371 · Cited by 78
StructuredRAG: JSON Response Formatting with Large Language ModelsPreprint
Shorten, C., Pierse, C., Smith, T. B., Cardenas, E., Sharma, A., Trengrove, J., & van Luijt, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.11061 · Cited by 87
“This paradigm of LLM computing is also being pioneered by works such as DSPy Assertions [22] or SPADE [23] that further tackle verifying and correcting outputs in Compound AI Systems [24].”
MethodLarge language models as tax attorneys: a case study in legal capabilities emergence
Nay, J. J., Karamardian, D., Lawsky, S. B., Tao, W., Bhat, M., Jain, R., Lee, A. T., Choi, J. H., & Kasai, J. (2024) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2023.0159 · Cited by 40
LightVA: Lightweight Visual Analytics With LLM Agent-Based Task Planning and Execution
Zhao, Y., Wang, J., Xiang, L., Zhang, X., Guo, Z., Turkay, Ç., Zhang, Y., & Chen, S. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3496112 · Cited by 92
“During the exploration process, users can ask various questions and discuss them with LLM in the dialog box, such as the problem analysis method, understanding of the answer, and explanation for the recommendation.”
Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills
Liu, D., Li, Z., Du, H., Wu, X., Gui, S., Kuang, Y., & Sun, L. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.2614 · Cited by 70
Watkins, R. (2023) · AI and Ethics · doi:10.1007/s43681-023-00294-5 · Cited by 65
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
Chen, J., Saha, S., & Bansal, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.381 · Cited by 64
NetSafe: Exploring the Topological Safety of Multi-agent NetworksPreprint
Yu, M., Wang, S., Zhang, G., Mao, J., Yin, C., Liu, Q., Wen, Q., Wang, K., & Wang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15686 · Cited by 72
Human-AI collaboration to identify literature for evidence synthesis
Spillias, S., Tuohy, P., Andreotta, M., Annand-Jones, R., Boschetti, F., Cvitanovic, C., Duggan, J., Fulton, E. A., Karcher, D. B., Paris, C., et al. (2024) · Cell Reports Sustainability · doi:10.1016/j.crsus.2024.100132 · Cited by 52
From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging
Shi, Y., Wang, S., Wan, C., Wang, M., & Gu, X. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773255 · Cited by 67
“• We introduce MGDebugger, a novel hierarchical debugging • We propose an LLM-simulated execution approach that enables”
Savage, C. H., Kanhere, A., Parekh, V. S., Langlotz, C. P., Joshi, A., Huang, H., & Doo, F. X. (2025) · Radiology · doi:10.1148/radiol.241073 · Cited by 73
“Each leaderboard uses different inclusion criteria and metrics to generate rankings (17,18,20,22,23).”
Influential
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.