Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 201–300 of 3,537
MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPs
Wang, Q., Wang, T., Tang, Z., Li, Q., Chen, N., Liang, J., & He, B. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.259 · Cited by 112
Dunivin, Z. O. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.15170 · Cited by 85
CYCLE: Learning to Self-Refine the Code Generation
Ding, Y., Min, M. J., Kaiser, G. E., & Ray, B. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3649825 · Cited by 94
“Further, could be applied to top-k generation to further improve the overall performance of code generation.”
BackgroundMethodSuspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4Preprint
Guo, J., Yang, B., Yoo, P., Lin, Y., Iwasawa, Y., & Matsuo, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.17277 · Cited by 79
RedAgent: An Autonomous Agent for Context-Aware Red Teaming of LLM Jailbreaks
Xu, H., Zhang, W., Wang, Z., Xiao, F., Zheng, R., Ba, Z., & Ren, K. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3665230 · Cited by 76
Liu, Y., Chen, S., Cheng, H., Yu, M., Ran, X., Mo, A., Tang, Y., & Huang, Y. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642698 · Cited by 61
Fagbohun, O., Harrison, R. M., & Dereventsov, A. (2023) · Journal of Artificial Intelligence Machine Learning and Data Science · doi:10.51219/jaimld/oluwole-fagbohun/15 · Cited by 70
Future of ADHD Care: Evaluating the Efficacy of ChatGPT in Therapy Enhancement
Berrezueta-Guzman, S., Kandil, M., Martín-Ruiz, M.-L., Pau de la Cruz, I., & Krusche, S. (2024) · Healthcare · doi:10.3390/healthcare12060683 · Cited by 51
“This method involves language agents reflecting on their performance, storing these reflections in memory, and using this accumulated knowledge to improve decision making in subsequent attempts [42].”
MethodCan large language models explore in-context?
Krishnamurthy, A., Harris, K., Foster, D. J., Zhang, C., & Slivkins, A. (2024) · neural information processing systems · doi:10.52202/079017-3818 · Cited by 120
Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in LLMs
Hu, Z., Liu, C., Feng, X., Zhao, Y., Ng, S.-K., Liu, C., He, J., Koh, P. W., & Hooi, B. (2024) · neural information processing systems · doi:10.52202/079017-0762 · Cited by 135
SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models
Zhang, D., Du, Z., Zhoubian, S., Du, Z., Yang, K., Wang, Z., Yue, Y., Dong, Y., & Tang, J. (2024) · neural information processing systems · doi:10.52202/079017-0046 · Cited by 94
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60
Cognitive Mirage: A Review of Hallucinations in Large Language Models
Ye, H., Liu, T., Zhang, A., Hua, W., & Jia, W. (2023) · arXiv · doi:10.48550/arxiv.2309.06794 · Cited by 61
Wang, W. Y., Zhang, S., Li, G., Lu, J., Ren, Y. S., Wang, X., Gao, X., Su, Y., Song, H., & Li, J. (2024) · Materials Genome Engineering Advances · doi:10.1002/mgea.56 · Cited by 82
Wickramasekara, A., Breitinger, F., & Scanlon, M. (2025) · Forensic Science International Digital Investigation · doi:10.1016/j.fsidi.2024.301859 · Cited by 89
“These MLLMs adhere to a shared task set, encompassing visual question answering, visual captioning, visual common-sense reasoning, visual generation, multimodal affective computing, visual retrieval, vision-language navigation, multimodal machine translation, visual question generation, and visual…”
Language models and psychological sciences
Sartori, G., & Orrù, G. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1279317 · Cited by 91
Xu, L., Hu, Z., Zhou, D., Ren, H., Dong, Z., Keutzer, K., Ng, S. K., & Feng, J. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.416 · Cited by 124
A Comprehensive Review of Humanoid Robots
Sheng, Q., Zhou, Z., Li, J., Mi, X., Xiang, P., Chen, Z., Xu, H., Jia, S., Wu, X., Cui, Y., et al. (2025) · SmartBot · doi:10.1002/smb2.12008 · Cited by 71
A Survey of Test-time Compute: From Intuitive Inference to Deliberate Reasoning
Ji, Y., Li, J., Xiang, Y., Ye, H., Wu, K., Kai, Y., Xu, J., Mo, L., & Zhang, M. (2026) · Computational Linguistics · doi:10.1162/coli.a.629 · Cited by 67
“Self-correction is a sequential test-time compute method that enables LLMs to iteratively revise and refine generated results based on external or internal feedback [263].”
Towards Optimizing with Large Language ModelsPreprint
Guo, P.-F., Chen, Y., Tsai, Y.-D., & Lin, S.-D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.05204 · Cited by 58
“Reflexion [5] Reflexion introduces a novel framework for training language agents that rely on linguistic feedback rather than traditional reinforcement learning.”
BackgroundOS Agents: A Survey on MLLM-Based Agents for General Computing Devices UsePreprint
Hu, X., Tao, X., Yi, B., Wei, Z., Xiao, R., Chen, Y., Ye, J., Tao, M., Zhou, X., Zhao, Z., et al. (2024) · Preprints.org · doi:10.20944/preprints202412.2294.v1 · Cited by 79
Robot learning in the era of foundation models: a survey
Xiao, X., Liu, J., Wang, Z., Zhou, Y., Qi, Y., Jiang, S., He, B., & Cheng, Q. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129963 · Cited by 91
LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agents
Chen, B., Yue, Z., Chen, S., Wang, Z., Liu, Y., Li, P., & Wang, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01882 · Cited by 93
“Agent-based methods [1, 7, 17, 23, 41, 50, 54] have the ability to decompose complex tasks.”
Reasoning beyond limits: Advances and open problems for LLMs
Ferrag, M. A., Tihanyi, N., & Debbah, M. A. (2025) · ICT Express · doi:10.1016/j.icte.2025.09.003 · Cited by 66
“Researchers have explored breaking down solutions into step-by-step reasoning paths—using methods like Chain-of-Thought [9], Self-Refine [10], and Reflexion—to enhance problem-solving [11], although these approaches sometimes face challenges related to feedback inefficiencies and local optima.”
Clinical Prompt Learning With Frozen Language Models
Taylor, N., Zhang, Y., Joyce, D. W., Gao, Z., Kormilitzin, A., & Nevado‐Holgado, A. J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3294633 · Cited by 68
Liu, Z., Hu, H., Zhang, S., Guo, H., Ke, S., Liu, B., & Wang, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.17382 · Cited by 63
“, 2022) and Reflexion (Shinn et al., 2023) agents, we allow them to reflect on the environment’s feedback but require them to generate a plan immediately without sophisticated reasoning.”
BackgroundInfluentialLemur: Harmonizing Natural Language and Code for Language Agents
Xu, Y., Su, H., Chen, X., Mi, B., Liu, Q., Shi, W., Hui, B., Zhou, F., Liu, Y., Xie, T., et al. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.06830 · Cited by 73
“…Large Language Models (LLMs) (Brown et al., 2020; Chen et al., 2021; Chowdhery et al., 2022; OpenAI, 2023) as the principal controllers for these agents (Yao et al., 2022b; Chase, 2022; Grav-itas, 2023; Shinn et al., 2023; Wang et al., 2023a; Xu et al., 2023a; Lin et al., 2023; Yao et al., 2023).”
BackgroundEmpowering Private Tutoring by Chaining Large Language Models
Chen, Y., Ding, N., Zheng, H.-T., Liu, Z., Sun, M., & Zhou, B. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679665 · Cited by 78
“A number of works have exploited LLMs in decomposing tasks (Wei et al. 2023), refining answers (Madaan et al. 2023; Shinn et al. 2023), and using external tools (Qin et al. 2023).”
BackgroundTowards conversational artificial intelligence for disease management
Liévin, V., Palepu, A. K., Weng, W., Saab, K. K., Stutz, D., Cheng, Y., Kulkarni, K., Mahdavi, S. S., Barral, J. K., Webster, D. R., et al. (2026) · Nature · doi:10.1038/s41586-026-10764-5 · Cited by 75
StateFlow: Enhancing LLM Task-Solving through State-Driven WorkflowsPreprint
Wu, Y., Yue, T., Zhang, S., Wang, C., & Wu, Q. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.11322 · Cited by 86
Octopus v2: On-device language model for super agentPreprint
Chen, W., & Li, Z. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.01744 · Cited by 64
ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress ConditionsPreprint
Gupta, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.06112 · Cited by 73
PaperQA: Retrieval-Augmented Generative Agent for Scientific Research
Lála, J., O'Donoghue, O., Shtedritski, A., Cox, S., Rodriques, S. G., & White, A. D. (2023) · arXiv · doi:10.48550/arxiv.2312.07559 · Cited by 55
Franceschelli, G., & Musolesi, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15278 · Cited by 76
“Using AI itself to provide feedback is also an option; notably, AI-based feedback is also used outside the RL paradigm, e.g., to provide verbal feedback to be appended to prompts (Shinn et al., 2023) or to collaborate with other LLMs at inference time (Dong et al., 2023; Du et al., 2023).”
BackgroundMethodMetacognitive Retrieval-Augmented Large Language Models
Zhou, Y., Liu, Z., Jin, J., Nie, J.-Y., & Dou, Z. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645481 · Cited by 38
“Other innovative approaches [29, 31] embed introspective mechanisms that iteratively refine the model’s outputs.”
BackgroundUsing Large Language Models for the Interpretation of Building Regulations
Fuchs, S., Witbrock, M., Dimyadi, J., & Amor, R. (2024) · Journal of Engineering Project and Production Management · doi:10.32738/jeppm-2024-0035 · Cited by 83
ChatLLM network: More brains, more intelligence
Hao, R., Hu, L., Qi, W., Wu, Q., Zhang, Y., & Nie, L. (2025) · AI Open · doi:10.1016/j.aiopen.2025.01.001 · Cited by 65
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement LearningPreprint
Wu, J., Yang, S., Lu, Z., Zhang, F., Shen, Y., Feng, L., Luo, H., Lian, Z., Zhang, S., Wen, Z., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.14777 · Cited by 61
“…where completed experience can be reinterpreted to improve learning under sparse feedback (Andrychowicz et al., 2017), and to language-agent methods based on verbal reflection, episodic memory, or experience summaries (Shinn et al., 2023; Zhao et al., 2024; Wang et al., 2023; Madaan et al., 2023).”
LLMatic: Neural Architecture Search Via Large Language Models And Quality Diversity Optimization
Nasir, M. U., Earle, S., Togelius, J., James, S., & Cleghorn, C. (2024) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3638529.3654017 · Cited by 52
Opportunities for large language models and discourse in engineering design
Göpfert, J., Weinand, J. M., Kuckertz, P., & Stolten, D. (2024) · Energy and AI · doi:10.1016/j.egyai.2024.100383 · Cited by 54
Ma, K., Grandi, D., McComb, C., & Goucher-Lambert, K. (2024) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4067332 · Cited by 56
Issue Localization via LLM-Driven Iterative Code Graph Searching
Jiang, Z., Ren, X., Yan, M. L., Jiang, W., Li, Y., & Liu, Z. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00249 · Cited by 68
“The rationale behind not utilizing the entire module call graph is that the repository-structure tree already provides a comprehensive view of the repository.”
ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning
Xiao, Y., Peng, B., Vajipey, V., Cheng, H., Galley, M., Gao, J., & Yu, Z. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.02052 · Cited by 64
“…improve agent’s decision making process, such as: iteratively prompting the model to improve its own output (Madaan et al., 2023; Yu et al., 2024; Shinn et al., 2023); and augmenting agent’s decision process using search algorithms such as BFS/DFS (Yao et al., 2023a), best-first search (Koh et…”
MethodThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing TasksPreprint
Shabbir, A., Munir, M. A., Dudhane, A. A., Sheikh, M., Khan, M. H., Fraccaro, P., Moreno, J. B., Khan, F. S., & Khan, S. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.23752 · Cited by 66
Zhang, Z., Yao, Y., Zhang, A., Tang, X., Ma, X., He, Z., Wang, Y., Gerstein, M., Wang, R., Liu, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3719341 · Cited by 51
“The gap between reasoning and action is bridged by combining interleaving thought, action, and observation [50, 126, 174].”
Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld
Yang, Y., Zhou, T., Li, K., Tao, D., Li, L., Shen, L., He, X., Jiang, J., & Shi, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02482 · Cited by 131
“For a fair comparison, we follow the same setting as prior work [41, 54–56, Baselines.”
BackgroundMethodResultInfluentialAMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
Zhao, Y., Yuan, B., Huang, J., Yuan, H., Yu, Z., Xu, H., Hu, L., Shankarampeta, A., Huang, Z., Ni, W., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.22769 · Cited by 51
“We consider LLM agents operating within the reason and act paradigm (Yao et al., 2022; Shinn et al., 2023; Wang et al., 2023), where sequential decision-making is formulated as a Partially Observable Markov Decision Process (POMDP) defined by the tuple M = ( S , A , O , P, r ) (See Fig.”
Liu, Y., Lo, S. K., Lu, Q., Zhu, L., Zhao, D., Xu, X., Harrer, S., & Whittle, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112278 · Cited by 51
Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains
Cheng, Y., Chen, J., Huang, Q., Xing, Z., Xu, X., & Lu, Q. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3638247 · Cited by 33
PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
Sun, S., Liu, Y., Wang, S., Iter, D., Zhu, C., & Iyyer, M. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.29 · Cited by 88
An In-depth Survey of Large Language Model-based Artificial Intelligence AgentsPreprint
Zhao, P., Jin, Z., & Cheng, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.14365 · Cited by 64
SPRING: Studying the Paper and Reasoning to Play GamesPreprint
Wu, Y., Prabhumoye, S., Min, S. Y., Bisk, Y., Salakhutdinov, R., Azaria, A., Mitchell, T. M., & Li, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15486 · Cited by 50
Quality-Diversity through AI Feedback
Bradley, H., Dai, A. M., Teufel, H., Zhang, J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K. O., Schott, G., & Lehman, J. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.13032 · Cited by 74
LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent EcosystemPreprint
Ge, Y., Ren, Y., Hua, W., Xu, S., Tan, J., & Zhang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.03815 · Cited by 68
“Consequently, two primary challenges emerge: 1) the reduction of computational costs associated with processing long contexts, and 2) the development of a flexible position encoding mechanism suitable for extended contexts.”
BackgroundCodeTree: Agent-guided Tree Search for Code Generation with Large Language Models
Li, J., Le, H., Zhou, Y., Xiong, C., Savarese, S., & Sahoo, D. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.189 · Cited by 77
Jin, W., Wang, N., Zhang, L., Tian, X., Shi, B., & Zhao, B. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.067857 · Cited by 78
Shi, W., Xu, R., Zhuang, Y., Yu, Y., Zhang, J., Wu, H., Zhu, Y., Ho, J. C., Yang, C., & Wang, M. D. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1245 · Cited by 54
“We will explore stricter evaluation metrics to assess the cases of misinformation that could pose a risk within clinical settings in our future work.”
MethodA Holistic View of AI-driven Network Incident Management
Hamadanian, P., Arzani, B., Fouladi, S., Kakarla, S. K. R., Fonseca, R., Billor, D., Cheema, A., Nkposong, E., & Chandra, R. (2023) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3626111.3628176 · Cited by 68
Tan, W., Zhang, W., Liu, S., Zheng, L., Wang, X., & An, B. (2024) · International Journal of Artificial Intelligence and Robotics Research · doi:10.1142/s2972335324500042 · Cited by 51
LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path PlanningPreprint
Meng, S., Wang, Y., Yang, C., Peng, N., & Chang, K.-W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.02511 · Cited by 53
Large Language Models Are Neurosymbolic Reasoners
Fang, M., Deng, S., Zhang, Y., Shi, Z., Chen, L., Pechenizkiy, M., & Wang, Y.-X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29754 · Cited by 85
ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow DataPreprint
Shen, J.-H., Jain, A., Xiao, Z., Amlekar, I., Hadji, M., Podolny, A., & Talwalkar, A. (2024) · Qeios · doi:10.32388/8vog0o · Cited by 54
Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models
Koa, K. J. L., Ma, Y., Ng, R., & Chua, T.-S. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645611 · Cited by 47
“For the incorrect samples, we then prompt a LLM M to generate a verbal feedback r 𝑠𝑡 for each iteration 𝑖 , given its previous inputs and outputs, which we refer to as its short-term memory [59].”
BackgroundMethodInfluentialLearning to Ask: When LLM Agents Meet Unclear Instruction
Wang, W., Shi, J., Ling, Z., Chan, Y.-K., Wang, C., Lee, C. Q. E., Yuan, Y., Huang, J.-T., Jiao, W., & Lyu, M. R.-T. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1104 · Cited by 81
AgentLens: Visual Analysis for Agent Behaviors in LLM-Based Autonomous Systems
Lu, J., Pan, B., Chen, J., Feng, Y., Hu, J., Peng, Y., & Chen, W. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3394053 · Cited by 66
“Our primary goal is to create a system that enhances users’ comprehension of LLMAS.”
Autonomous Industrial Control using an Agentic Framework with Large Language Models
Vyas, J., & Mercangöz, M. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.07.170 · Cited by 58
“REACT (Yao et al. (2022)) enables agents to process thoughts before taking actions, while REFLECTION (Shinn et al. (2023)) allows agents to interact with the system, reflect on actions, and store the interaction history for iterative learning.”
BackgroundBenchmarking Mobile Device Control Agents across Diverse ConfigurationsPreprint
Lee, J., Min, T., An, M., Hahm, D., Lee, H., Kim, C., & Lee, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.16660 · Cited by 54
Multi-Agent Causal Discovery Using Large Language ModelsPreprint
Le, H. D., Xia, X., Xu, H., & Zhang, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.15073 · Cited by 57
“Planning employs techniques like Chain of Thought and Tree of Thoughts, and for complex tasks, feedback mechanisms like Re-Act (Yao et al., 2023c) and Reflexion (Shinn et al., 2023) refine plans based on past actions and observations.”
Huang, J., Li, X., Li, X., Gao, L., & Teng, Y. (2026) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2025.3650586 · Cited by 57
“The integration of LLMs with evolutionary algorithms has proven effective for various tasks, such as prompt optimization [34] and algorithm self-improvement [35], and has shown particular promise in addressing combinatorial optimization problems, including the traveling salesman problem (TSP) [35],…”
Exploring and Characterizing Large Language Models for Embedded System Development and Debugging
Englhardt, Z., Li, R. C., Nissanka, D., Zhang, Z., Narayanswamy, G., Breda, J., Liu, X., Patel, S., & Iyer, V. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650764 · Cited by 67
USimAgent: Large Language Models for Simulating Search Users
Zhang, E., Wang, X., Gong, P., Lin, Y., & Mao, J. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657963 · Cited by 78
“Numerous studies aim to enhance LLMs’ capabilities inexecuting complex tasks[15,23,31,36, 38].”
BackgroundVideoGameBench: Can Vision-Language Models complete popular video games?Preprint
Zhang, A. L., Griffiths, T. L., Narasimhan, K., & Press, O. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.18134 · Cited by 60
“Our VG-Agent uses ReAct (Yao et al., 2023) with the ability to store information in context (Shinn et al., 2023), and has basic information on the controls and objectives for each game.”
Self-citationA Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and ApplicationsPreprint
Zhou, Y., Shu, W., Su, Y., Du, W., Fang, Y., & Lin, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.07358 · Cited by 50
“As task complexity grows, that orchestration burden still falls back on the LLM at inference time and becomes a major source of brittleness [4], [19].”
InfluentialAuto-GPT for Online Decision Making: Benchmarks and Additional Opinions
Yang, H., Yue, S., & He, Y. (2023) · arXiv · doi:10.48550/arxiv.2306.02224 · Cited by 44
Deductive Verification of Chain-of-Thought Reasoning
Ling, Z., Fang, Y., Li, X., Huang, Z., Lee, M., Memisevic, R., & Su, H. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1580 · Cited by 44
Gao, L., Lu, J., Shao, Z., Lin, Z., Yue, S., Ieong, C., Sun, Y., Zauner, R. J., Wei, Z., & Chen, S. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456145 · Cited by 56
Cognitive Agents Powered by Large Language Models for Agile Software Project Management
Cinkusz, K., Chudziak, J. A., & Niewiadomska‐Szynkiewicz, E. (2024) · Electronics · doi:10.3390/electronics14010087 · Cited by 64
“Quality Checker Agents ensure that deliverables meet predefined quality standards [54] by conducting code reviews, performing testing, and validating outputs against requirements.”
V-D roid : Advancing Mobile GUI Agent Through Generative Verifiers
Dai, G., Jiang, S., Cao, T., Li, Y., Yang, Y., Tan, R., Li, M., & Qiu, L. (2026) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3795866.3796681 · Cited by 49
Thinking Machines: A Survey of LLM based Reasoning StrategiesPreprint
Bandyopadhyay, D., Bhattacharjee, S., & Ekbal, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.10814 · Cited by 52
“ReAct (Yao et al., 2023b) ; Reflexion (Shinn et al., 2023) Reward Based Outcome Supervision DeepSeek-R1 (DeepSeek-AI et al., 2025) ; Chu et al. (2025) ; GLoRe (Havrilla et al., 2024) Process Supervision”
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Fernandes, P., Madaan, A., Liu, E., Farinhas, A., Martins, P. H., Bertsch, A., de Souza, J. G. C., Zhou, S., Wu, T., Neubig, G., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.00955 · Cited by 44
Wang, Z., Moriyama, S., Wang, W.-Y., Gangopadhyay, B., & Takamatsu, S. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3676025 · Cited by 51
“Agent-Specific Memory : While some prior works such as LangGraph [39] and GPTSwarm [21] utilize a shared memory pool accessible by all agents, we follow the approach in [40] and adopt agent-specific memory.”
Improving the Validity of Automatically Generated Feedback via Reinforcement Learning
Scarlatos, A., Smith, D., Woodhead, S., & Lan, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-64302-6_20 · Cited by 73
“We note that it may be possible to resolve these issues using additional prompt engineering or tools such as self-reflection [28] and code execution to evaluate math expressions [5,31].”
OAgents: An Empirical Study of Building Effective Agents
Zhu, H., Qin, T., Zhu, K., Huang, H., Guan, Y., Xia, J., Li, H., Yao, Y., Wang, N., Liu, P., et al. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.720 · Cited by 55
Pre-Trained Large Language Models for Industrial ControlPreprint
Song, L., Zhang, C., Zhao, L., & Bian, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.03028 · Cited by 62
PokeLLMon: A Human-Parity Agent for Pokemon Battles with Large Language ModelsPreprint
Hu, S., Huang, T., & Liu, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.01118 · Cited by 52
MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device ControlPreprint
Lee, J.-Y., Hahm, D., Choi, J. S., Knox, W. B., & Lee, K. (2024) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.48550/arxiv.2410.17520 · Cited by 51
Sparks of Science: Hypothesis Generation Using Structured Paper DataPreprint
O'Neill, C., Ghosal, T., Răileanu, R., Walmsley, M., Bui, T., Schawinski, K., & Ciucă, I. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.12976 · Cited by 46
“Reflexive approaches such as Shinn et al. (2023) and Lindes & Peter (2023) explicitly incorporate iterative self-correction of hypothesized actions.”
Exploring the Boundaries of GPT-4 in Radiology
Liu, Q., Hyland, S. L., Bannur, S., Bouzid, K., Castro, D. C., Wetscherek, M. T., Tinn, R., Sharma, H., Pérez‐García, F., Schwaighofer, A., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.891 · Cited by 59
GestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents
Zeng, X., Wang, X., Zhang, T., Yu, C., Zhao, S., & Chen, Y. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3698145 · Cited by 48
DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential RecommendationPreprint
Wang, Y., Liu, Z., Zhang, J., Yao, W., Heinecke, S., & Yu, P. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.11336 · Cited by 51
Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositories
Yildiz, A., Teo, S. G., Lou, Y., Feng, Y., Wang, C., & Divakaran, D. M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1490 · Cited by 43
Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models
Yu, J., He, R., & Ying, R. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.03965 · Cited by 55
Yang, J., Liu, X., Lv, W., Deng, K., Guo, S., Jing, L., Li, Y., Liu, S., Luo, X., Luo, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.18538 · Cited by 50
Guardian: A Runtime Framework for LLM-Based UI Exploration
Ran, D., Wang, H., Song, Z., Wu, M. S., Cao, Y., Zhang, Y., Yang, W. M., & Xie, T. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680334 · Cited by 68
“While LLM-based approaches have been successfully applied to sequential planning problems [52, 70, 74, 75], our evaluation in Section 4.2 shows that ReAct [75], Reflexion [52], and Droidbot-GPT [70], three state-of-the-art LLM-based approaches, all exhibit low effectiveness when applied to…”
InfluentialSoft Self-Consistency Improves Language Models Agents
Wang, H., Prasad, A., Stengel-Eskin, E., & Bansal, M. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-short.28 · Cited by 76
AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories
Song, Y., Xiong, W., Zhao, X., Zhu, D., Wu, W., Wang, K., Li, C., Peng, W., & Li, S. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.116 · Cited by 77
Safety case template for frontier AI: A cyber inability argumentPreprint
Goemans, A., Buhl, M. D., Schuett, J., Korbak, T., Wang, J. J., Hilton, B., & Irving, G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.08088 · Cited by 52
“…prompt engineering, SOTA elicitation attempts will typically involve enhancing the model with tools (such as a web browser) and agentic scaffoldings as well as fine-tuning the model on specific datasets—all of which can dramatically improve a model’s performance on a task [19, 57, 67, 75, 76].”
Karim, M. M., Khan, S., Van, D. H., Liu, X. L., Wang, C., & Qu, Q. (2025) · Future Internet · doi:10.3390/fi17080353 · Cited by 50
“Another example is the Reflexion agent framework [96], in which an LLM agent maintains an internal memory of feedback and “reflections” on its performance.”
AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
Chen, W., Su, Y., Zuo, J., Yang, C., Yuan, C., Chan, C.-M., Yu, H., Lu, Y., Hung, Y.-H., Qian, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.10848 · Cited by 39
“These include self-reflection and output critique [51][31], effective use of external tools and APIs [50][45][56], complex task planning and decomposition [57][55], and multi-agent collaboration through structured dialogues [11][49].”
Buehler, M. J. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00003-z · Cited by 48
“Other work has focused on RL to enhance reasoning and decision-making 30 .”
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.