Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 901–1,000 of 3,537
Feng, Y., Sun, J., Yang, Z., Ai, J., Li, C., Li, Z., Zhang, F., He, K., Ma, R., Lin, J., et al. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1497 · Cited by 18
SeLaR: Selective Latent Reasoning in Large Language Models
Fu, R., & Luo, G. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.320 · Cited by 13
Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistance
Ma, Q., Guo, C., Tian, Z., Wang, S., Xiao, J., Yue, Y., & Zhang, Z. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2140 · Cited by 10
Pinto, R., Lagorio, A., Ciceri, C., Mangano, G., Zenezini, G., & Rafele, C. (2024) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2024.09.198 · Cited by 8
CritiQ: Mining Data Quality Criteria from Human Preferences
Guo, H., Lv, K., Guo, Q., Liang, T., Xi, Z., Song, D., Zhang, Q., Sun, Y., Chen, K., Qiu, X., et al. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.792 · Cited by 10
“Reflection is a common technique to improve the performance of LLMs through iterative critiquing and refinement (Shinn et al., 2023; Madaan et al., 2023; Saunders et al., 2022; Xi et al., 2024).”
Can Graph Descriptive Order Affect Solving Graph Problems with LLMs?
Ge, Y., Liu, S., Bi, B., Wang, Y., Mei, L., Feng, W., Chen, L., & Cheng, X. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.321 · Cited by 15
DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer
Wang, R., Wu, J., Xia, Y., Yu, T., Rossi, R. A., McAuley, J., & Yao, L. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818028 · Cited by 7
“Agentic Framework Agentic frameworks empower LLMs to solve long-horizon tasks by enabling observation reading, natural-language planning, and tool use [57, 31, 33, 63, 21, 60, 25].”
InfluentialJin, K., & Zhuo, H. H. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3729236 · Cited by 12
ByteComposer: a Human-like Melody Composition Method based on Language Model AgentPreprint
Liang, X., Du, X., Lin, J., Zou, P., Wan, Y., & Zhu, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.17785 · Cited by 13
“Besides forward reasoning, (Shinn et al., 2023; Madaan et al., 2023; Paul et al., 2023) introduced the “self-reflection” mechanism, where LMs provide feed-back to their generation candidates to improve LMs’ performance on specific tasks.”
BackgroundSelf-correction is Not An Innate Capability in Language ModelsPreprint
Liu, G., Qi, Z., Zhang, X., Cheng, L., & Johnson, K. M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.20513 · Cited by 8
“Further studies asked for external tools or knowledge for better self-correction (Shinn et al., 2023; Chen et al., 2023b; Gou et al., 2024; Gao et al., 2023).”
Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm DetectionPreprint
Lee, J. A., Fong, W., Le, A., Shah, S., Han, K., & Zhu, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.04509 · Cited by 16
“Our approach incorporates linguistic principles to mimic how humans reason through emotionally complex text as well as reflection strategies commonly found in LLM reasoning and planning agents (Shinn et al., 2023).”
Xu, Z., Gu, W., Yu, C., Wu, Y., & Wang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.04686 · Cited by 10
AI Agents for Web Testing: A Case Study in the WildPreprint
Ye, N., Yu, X., Xu, R., Peng, T., & Yu, Z. (2025) · Preprints.org · doi:10.20944/preprints202508.1890.v1 · Cited by 8
Advancing Retrieval-Augmented Generation for Structured Enterprise and Internal DataPreprint
Cheerla, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.12425 · Cited by 9
“We envision extending this framework towards agentic RAG systems [4, 16], where intelligent agents autonomously select retrieval strategies, adaptively reformulate queries, and integrate multimodal data sources—including images, scanned documents, and audio—thereby broadening the landscape of…”
Gumaan, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.08744 · Cited by 9
BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
Zhang, Z., Bajaj, A. P., Handa, D., Liu, S., Raj, A. S., Chen, H., Wang, H., Liu, Y., Basque, Z. L., Nath, S., et al. (2026) · Transactions on machine learning research · doi:10.48550/arxiv.2509.25248 · Cited by 7
“As such, LLM-based AI agents (Yao et al., 2023; Shinn et al., 2023), which are autonomous systems that use LLMs to iteratively plan, reason, and act, are increasingly used to automate and facilitate complex software engineering workflows (Wang et al., 2024).”
SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language ReasoningPreprint
Guo, X., Zhou, R., Wang, Y., Zhang, Q., Zhang, C., Jegelka, S., Wang, X., Chai, J., Yin, G., Wei, L., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.16416 · Cited by 8
“Self-Instruct [51] bootstraps fine-tuning data through synthetic instructions, while STaR [57] and Reflexion [48] improve reasoning via self-generated rationales and feedback.”
LANPO: Bootstrapping Language and Numerical Feedback for Reinforcement Learning in LLMsPreprint
Li, A., Wang, Y., Yuan, Z., Jegelka, S., & Wang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.16552 · Cited by 7
“This includes inference-time correction [13, 14], generating self-critiques [15–17], and maintaining reflections across episodes [18, 19].”
Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic WorkflowsPreprint
Ming, Y., Ke, Z., Nguyen, X.-P., Wang, J., & Joty, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.03332 · Cited by 7
Cross-Task Experiential Learning on LLM-based Multi-Agent CollaborationPreprint
Li, Y., Qian, C., Yu, X., Shi, R., Dang, Y., Xie, Z., You, Z., Chen, W., Cheng, Y., Liu, W., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.23187 · Cited by 8
“Building directly on these foundational advances, there has been a surge of interest in autonomous agents that harness the reasoning and generation strengths of LLMs [Zhou et al., 2023, Wang et al., 2023a, Park et al., 2023, Wang et al., 2023b, Richards, 2023, Wang et al., 2024, Shinn et al., 2024].”
Surfer 2: The Next Generation of Cross-Platform Computer Use AgentsPreprint
Andreux, M., Bakler, M., Barbier, Y., Benchekroun, H., Biré, E., Bonnet, A., Bordie, R., Bout, N., Brunel, M., Cambray, A., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.19949 · Cited by 7
“Our system’s self-correction and validation loop relates to recent work on autonomous evaluation and refinement of agent behavior [46, 10], particularly in methods that reinforce agents through linguistic feedback and reflective episodic memory rather than weight updates [47].”
Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space
Hong, J., Liu, K., Ling, Z., Chen, J., & Levine, S. (2025) · arXiv · doi:10.48550/arxiv.2512.04601 · Cited by 7
“First, we compare against a variation of SDPO [Hubotter et al., 2026] that we call Self-Distill, where the rollouts are directly fed to the refinement policy in-context, similar to as in Shinn et al. [2023].”
How can we assess human-agent interactions? Case studies in software agent designPreprint
Chen, V., Malhotra, R., Wang, X., Michelini, J., Zhou, X., Soni, A. B., Tran, H. H., Smith, C., Talwalkar, A., & Neubig, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.09801 · Cited by 8
“However, with LLM-powered agents, recent approaches have largely adopted language reasoning as a medium for planning as opposed to symbolic operators and explicit transition models [49, 37].”
Graph-Enhanced Policy Optimization in LLM Agent TrainingPreprint
Yuan, J., Gong, Z., Hang, J., Bai, Z., & Zhao, W. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.26270 · Cited by 7
“GEPOisbenchmarked againsttwo classesofmethods:prompting-based techniques (e.g., ReAct [43]) and reinforcement learning algorithms, including Reflection[33], PPO[30], RLOO[2], GRPO [31], and the current state-of-the-art, GiGPO [9].”
VIGIL: A Reflective Runtime for Self-Healing AgentsPreprint
Cruz, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.07094 · Cited by 7
“More recently, large language model (LLM) agents have adopted reflection strategies, often leveraging self-evaluation or critique [2, 1, 4].”
DLLM Agent: See Farther, Run FasterPreprint
Zhen, H., Lin, W., Liu, R., Han, K., Li, Y., Tian, Y., Chen, H., Li, X., Li, X., Chen, C., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.07451 · Cited by 7
“…line of work studies internal model dynamics (e.g., attention patterns and information flow) to explain how models coordinate decisions over multi-step interactions, though such analyses are often token-centric or single-generation-centric rather than aligned to explicit workflow stages [1,26,33].”
Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM AgentsPreprint
Yang, R., Ye, F., We, X., Zhao, R., Luo, K., Xu, X., Zhao, B., Ma, R., Wang, S., Tu, Z., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.12662 · Cited by 7
“Additional details are provided in Appendix D. Baselines We compare our method against several baselines (see Appendix G for implementation details): • Prompt-based methods , including ReAct (Yao et al., 2023b) and Reflexion (Shinn et al., 2023), a self-refinement method that summarizes each trial to guide future deci…”
The Auton Agentic AI FrameworkPreprint
Cao, S., Chang, Z., Li, C., Li, H., Fu, L., & Tang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.23720 · Cited by 9
“At this level, the agent adapts without any modification to its model weights [25, 26].”
Wang, Y., & Luan, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.05258 · Cited by 7
“A parallel literature on multi-agent collaboration [30, 31, 32, 33] provides the building blocks but does not target end-to-end research itself.”
Formal Skill: Programmable Runtime Skills for Efficient and Accurate LLM AgentsPreprint
Zhang, X., Gao, M., Zhao, Y., Tan, X., Yao, Y., Wang, F., Wang, Y., Dingsiyi, & Yang, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.19604 · Cited by 7
“Reflexion, Self-Refine, and Voyager study feedback and reusable executable skills [7, 8, 9]; SWE-agent and AutoCodeRover show that repository navigation, editing, testing, search, and repair workflows are shaped by the agent-computer interface [10, 11].”
Agentic and Multi-agent Systems: A Systematic Review of Tool Use, Benchmarks, and GovernancePreprint
Rah, A. (2026) · Research Square · doi:10.21203/rs.3.rs-9839978/v1 · Cited by 9
Qu, B., & Chen, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.29771 · Cited by 7
“…§2.4 made executable — perception (IA), comprehension (MSS), projection-and-decision (MAPC), action (COE), and reflection (ALE) — and the role decomposition follows the multi-agent financial-trading line [10] while the reflect-and-learn stage instantiates the ReAct [15] / Reflexion [16] paradigm.”
InfluentialWhat to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and AgentsPreprint
Colaco, A. G., & Lahjouji, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.08032 · Cited by 7
“Reflexion specializes this to failure, writing a verbal self-critique after a failed attempt and prepending it on the next trial [101], and ExpeL distills cross-task experience by contrasting successful and failed trajectories into natural-language insights [143].”
Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0Preprint
Wang, W., Kattakinda, P., & Feizi, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.14004 · Cited by 9
“Reflexion attaches verbal, LLM-generated self-critique to a fixed agent architecture, using linguistic feedback in place of a scalar reward signal (Shinn et al., 2023).”
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement LearningPreprint
Wang, J.-Z., Zheng, K.-W., Liu, Y., Cao, Y., Zhang, Q.-L., Chen, J.-L., Yang, H., & Chen, Q.-L. (2026) · arXiv.org · doi:10.48550/arxiv.2607.26873 · Cited by 6
“Other methods improve a frozen model through test-time sampling or search (Wang et al. 2023; Yao et al. 2023; Bay and Yearick 2026), iterative self-feedback (Madaan et al. 2023; Shinn et al. 2023), or accumulated external state.”
EmbodiedEvo: Self-Evolving Coding Agent For Physical IntelligencePreprint
Li, W., Chi, Y., Zheng, Y., Liu, C., Wang, X., Gao, M., Jiang, Z., Hong, D., Xiao, C., Yang, K., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.18867 · Cited by 10
“This trace covers every accepted execution in the Astra/xhigh self-harness run of stack_blocks_by_language , public layout 37.”
Who Improves the Robot? Toward Physical Recursive Self-ImprovementPreprint
Hai, Z., Yang, F., Chen, Z., Zhang, L., Fu, C., Li, J., Zhu, J., Liu, X., Yang, Y., Meng, L., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.18842 · Cited by 10
Mao, L., Gu, Z., Long, R., Song, T., Huang, H., & Ma, L. (2026) · International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA) · doi:10.1109/caibda70336.2026.11621567 · Cited by 6
SRDrone: LLM-Driven Self-Refinement for Embodied Drone Task Planning
Zhang, D., Zhang, X., Zhang, J., Li, J., Long, T., Dai, X., Fu, Y., Ren, J., & Zhang, Y. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3664806 · Cited by 7
Ying, T., Wang, L., Hu, P., & Ding, T. (2026) · International Conference on Electronic Technology and Information Science (ICETIS) · doi:10.1109/icetis70504.2026.11633437 · Cited by 7
“Such a formulation provides the basis for subsequent memory update and strategy adjustment, while also preserving a clear separation between present evidence and accumulated historical knowledge[37-43].”
Taherkhani, H., DaghighFarsoodeh, A., Chowdhury, M., Pham, H. V., & Hemmati, H. (2026) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst69053.2026.00039 · Cited by 8
Unraveling the emergence of collective behavior in networks of cognitive agents
Zomer, N., & Domenico, M. D. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00091-5 · Cited by 13
Agentic AI: From Autonomous Agents to Enterprise Agentic AI Systems
Li, M. M., Reinhard, P., & Leimeister, J. M. (2026) · Business & Information Systems Engineering · doi:10.1007/s12599-026-01009-w · Cited by 9
“…by reasoning strategies, for example Chain-of-Thought (Wei et al. 2022), ReAct (Yao et al. 2023a), Tree-of-Thought (Yao et al. 2023b), or Reflexion (Shinn et al. 2023), which structure how agents decompose goals, select and orchestrate tools, draw on memory, and determine their action space.”
Cao, J., Nguyen, H., Yavuz, S., Yu, B., Wang, S., Bharaj, P. K., & Francis, D. C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29770-9_23 · Cited by 7
Agentic AI for Medicare Annual Wellness Visits: Improving Preventive Care
Viradia, V., Mishra, V. K., & -, S. P. (2025) · International Conference on Bio-engineering for Smart Technologies (BioSMART) · doi:10.1109/biosmart66413.2025.11046123 · Cited by 6
Watkins, R. (2023) · doi:10.31219/osf.io/6uh8p · Cited by 6
Implementing Financial Regulations Using Large Language ModelsPreprint
Fazlija, B., Ibraimi, M., Forouzandeh, A., & Fazlija, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5010694 · Cited by 9
UsmleGPT: An AI application for developing MCQs via multi-agent system
Jiang, Z., & Feng, S. (2025) · Software Impacts · doi:10.1016/j.simpa.2025.100742 · Cited by 9
Collaborative intelligence in AI: Evaluating the performance of a council of AIs on the USMLE
Shaikh, Y., Jeelani-Shaikh, Z. A., Jeelani, M., Javaid, A., Mahmud, T., Gaglani, S. M., Gibbons, M., Cheema, M., Cross, A., Livingston, D., et al. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000787 · Cited by 7
“This aligns with recent studies on self-correction mechanisms in LLMs. [36–38] This phenomenon, where multiple AI instances engage in a dialogue and reconcile their differences to achieve a consensus, reflects a simulation of collective intelligence, [39] enabling a group of AIs to ‘change their…”
Abdelkarim, S., Lu, D., Flores, D., Jaeggi, S. M., & Baldi, P. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100170 · Cited by 7
“We built an LLM agent that consisted of three components that interact with each other, an actor, a critic, and a self-reflector, similar to the setups in Chen, Lin, Schärli, and Zhou (2024), Madaan et al. (2023), Shinn, Cassano, Gopinath, Narasimhan, and Yao (2023).”
Leong, H. Y., & Wu, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5276004 · Cited by 14
Mao, K., Hu, B., Lin, R., Li, Z., Lu, G., & Zhang, Z. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1660912 · Cited by 10
MeshAgent: enabling reliable network management with large language models
Zhou, Y., Hsieh, K., Mani, S. K., Kandula, S., & Liu, Z. (2025) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3771567 · Cited by 7
SmartPlay: A Benchmark for LLMs as Intelligent Agents
Wu, Y., Tang, X., Mitchell, T. M., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.01557 · Cited by 6
Fang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6
ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph
Liu, X., Peng, Z., Yi, X., Xie, X., Xiang, L., Liu, Y., & Xu, D. (2024) · arXiv · doi:10.48550/arxiv.2403.00839 · Cited by 6
Casper, S., Bailey, L., Hunter, R., Ezell, C., Cabalé, E., Gerovitch, M., Slocum, S., Wei, K., Jurkovic, N., Khan, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.01635 · Cited by 6
“Planning of complex series of actions is typically facilitated through chain-of-thought-based reasoning processes (Wei et al., 2022; Yao et al., 2022c; 2023; Shinn et al., 2023; OpenAI, 2024).”
On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models
Verma, M., Bhambri, S., & Kambhampati, S. (2024) · arXiv · doi:10.48550/arxiv.2405.13966 · Cited by 6
AI Agentic workflows and Enterprise APIs: Adapting API architectures for the age of AI agents
Tupe, V., & Thube, S. (2025) · doi:10.36227/techrxiv.173895544.45005813/v1 · Cited by 6
RoboCleaner: Robotic Tabletop Cleaning via VLM-Powered Multi-Agent Collaboration
Li, H., Zhang, S., & Guo, D. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3594680 · Cited by 8
“These systems achieve significant success by decomposing complex tasks into subtasks and assigning them to specialized agents with distinct roles [13], [14], enabling real-time interaction and iterative feedback [15], [16], [17].”
Guiding Large Language Models in Modeling Optimization Problems via Question Partitioning
Pan, X., Fang, J., Wu, F., Zhang, S., Hu, Y., Li, S., & Li, X. (2025) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2025/296 · Cited by 10
“…the modeling, verification, and validation of optimization problems, building on methods like Chain-of-Thought (CoT) [Wei et al. , 2022], Tree of Thoughts [Yao et al. , 2024], and Reflexion [Shinn et al. , 2024], achieving notable progress [Xiao et al. , 2023; AhmadiTeshnizi et al. , 2024].”
Han, B., Mathrani, A., Wanniarachchi, V. U., & Sušnjak, T. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10110 · Cited by 9
LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity Recognition
Bai, F., Hassanzadeh, H. R., Saeedi, A., & Dredze, M. H. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1441 · Cited by 10
“Recent research has explored this capability across a wide range of tasks, like code generation (Madaan et al., 2023) and planning (Shinn et al., 2023).”
Liu, C., Lu, Y., Ye, F., Li, J., Chen, X., Ren, F., Tu, Z., & Li, X. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1389 · Cited by 14
Tan, W., Jiang, C., Duan, Y., Lei, M., Li, J., Hong, Y., Wang, X., & An, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37132-4_14 · Cited by 6
InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection
Liu, Y., Li, P., Wei, Z., Xie, C., Hu, X., Xu, X., Zhang, S., Han, X., Yang, H., & Wu, F. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.45 · Cited by 12
Yuan, C., Wei, T., Li, C., Yi, X., Liu, S., Zhang, Z., Cai, Y., & Du, X. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00708-4 · Cited by 10
KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
Yuan, Y., Chowdhury, M., & Talati, N. (2027) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3845814.3850050 · Cited by 8
Willful Disobedience: Automatically Detecting Failures in Agentic Traces
Sharma, R. K., Barke, S., & Zorn, B. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813153 · Cited by 9
“A natural next step is to use these signals to guide prompt or tool updates [32], or to generate training data for improving agent behavior.”
optimize_anything: Unified Text Optimization can Outperform Specialized Systems
Agrawal, L. A., Lee, D., Tan, S., Ma, W., Elmaaroufi, K., Sandadi, R., Seshia, S. A., Sen, K., Klein, D., Stoica, I. G., et al. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813167 · Cited by 7
Khapre, S., Mersha, M. A., Olalekan, Z., Shakil, H., Iskandar, S., Moustafa, A., & Kalita, J. K. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134049 · Cited by 12
MemFail: Stress-Testing Failure Modes of LLM Memory Systems
Garg, I., Kolhe, N., Song, D., & Zhao, X. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.779 · Cited by 6
BacktestBench: Benchmarking Large Language Models for Automated Quantitative Strategy Backtesting
Wang, Z., Yang, W., Wu, Q., Ma, L., Zhang, Y., & Jia, W. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817460 · Cited by 7
Agentic search for BIM information extraction: Systematic evaluation
Hellin, S., Jang, S., Fuchs, S., Nousias, S., & Borrmann, A. (2026) · Automation in Construction · doi:10.1016/j.autcon.2026.107260 · Cited by 7
NSync: Automated Cloud Infrastructure-as-Code Reconciliation with AI Agents
Yang, Z., Guan, H., Nicolet, V., Paulsen, B., Dodds, J., Kroening, D., & Chen, A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832121 · Cited by 10
“The learning process is agent-driven : the agent invokes the tools knowledge_update and knowledge_retrieval when it judges that the current context is complete and accurate enough to warrant persistence for reuse, as opposed to always logging every intermediate action.”
Boosting LLM Agents with Recursive Contemplation for Effective Deception Handling
Wang, S., Liu, C., Zheng, Z., Qi, S., Chen, S., Yang, Q., Zhao, A., Wang, C., Song, S., & Huang, G. M. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.591 · Cited by 27
“Thought Methods of LLMs Various thought mechanisms enhance LLMs’ reasoning and decision-making (Yao et al., 2022; Ma et al., 2023; Shinn et al., 2023; Madaan et al., 2023; Wei et al., 2022).”
BackgroundPRACT: Optimizing Principled Reasoning and Acting of LLM Agent
Liu, Z., Yao, W., Zhang, J., Liu, Z., Yang, L., Rithesh, R. N., Lan, T., Zhu, M., Tan, J., Kokane, S., et al. (2024) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2024.conll-1.33 · Cited by 10
Visual Environment-Interactive Planning for Embodied Complex-Question Answering
Lan, N., Ou, B., Xie, X., & Shi, G. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3538860 · Cited by 7
“Reflexion [38] generates self-reflections through linguistic feedback to reduce LLMs hallucination.”
Enhancing long-form question answering via reflection with question decomposition
Xiao, J.-J., Wu, W., Zhao, J.-X., Fang, M., & Wang, J. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104274 · Cited by 6
Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting
Chen, W., Zhang, J., Zhu, H., Xu, B., Hao, Z., Zhang, K., Ye, J., & Cai, R. (2025) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2025/478 · Cited by 11
Self-Critique and Refinement for Faithful Natural Language Explanations
Wang, Y., & Atanasova, P. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.427 · Cited by 14
Accelerating training speed of tiny recursive models with curriculum guided adaptive recursion
Qasim, K. U., Zhang, J., Alharith, R., & Shaheen, M. K. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109720 · Cited by 7
Exogenous and Endogenous Data Augmentation for Low-Resource Complex Named Entity Recognition
Zhang, X., Chen, G., Cui, S., Sheng, J., Liu, T., & Xu, H. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657754 · Cited by 6
WebWISE: Unlocking Web Interface Control for LLMs via Sequential Exploration
Tao, H., Sethuraman, T. V., Shlapentokh-Rothman, M., Gupta, T., Ji, H., & Hoiem, D. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.234 · Cited by 10
PodAgent: A Comprehensive Framework for Podcast Generation
Xiao, Y., He, L., Guo, H., Xie, F.-L., & Lee, T. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1226 · Cited by 12
Applying Cognitive Design Patterns to General LLM Agents
Wray, R. E., Kirk, J. R., & Laird, J. E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00800-8_28 · Cited by 9
“Reflexion [39], a system designed to improve its problem-solving policy via LLM-mediated reflection, caches its reflections in a memory to be used in subsequent trials.”
InfluentialA short survey on small reasoning models: training, inference, applications, and research directions
Wang, C., Zhang, T., Hong, R., & Huang, J. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-50990-0 · Cited by 10
Large Language Models (LLMs) for Natural Language Processing (NLP) of Oil and Gas Drilling Data
Kumar, P., & Kathuria, S. (2023) · doi:10.2118/215167-ms · Cited by 6
Bridging Intelligence and Instinct: A New Control Paradigm for Autonomous RobotsPreprint
Zhang, S., & Lu, Q. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.10690 · Cited by 6
Self-Convinced Prompting: Few-Shot Question Answering with Repeated IntrospectionPreprint
Zhang, H., Cai, M., Zhang, X., Zhang, C., Mao, R., & Wu, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.05035 · Cited by 9
“Notably, this iterative refinement framework does not require supervised training data or reinforcement learning, and it operates solely with a single LLM. Reflexion (Shinn et al., 2023) represents a significant advancement over previous approaches such as ReAct and DEPS.”
MethodLeer, C., Trost, V., & Voruganti, V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.06983 · Cited by 6
“Other papers looking to achieve similar types of outcomes, without needing to update model weights, rely on in-context methods entirely [23, 28].”
BackgroundKwaiYiiMath: Technical ReportPreprint
Fu, J., Lin, L., Gao, X., Liu, P., Chen, Z., Yang, Z., Zhang, S., Zheng, X., Li, Y., Liu, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.07488 · Cited by 8
TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking
Wang, D., Ma, J., Fang, F., & Li, L. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.01952 · Cited by 10
AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information AssistantPreprint
Zhou, Y., Liu, Z., & Dou, Z. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.06805 · Cited by 6
“Innovative methods used by Flare [17] and Self-Ask [6] determine optimal times for information retrieval, while Reflexion [31] endows LLMs with introspective mechanisms that continually refine their outputs.”
MethodCreating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIsPreprint
Tzachristas, I. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.13233 · Cited by 6
Fine-tuning with RAG for Improving LLM Learning of New SkillsPreprint
Ibrahim, H., Rozanov, N., & Rei, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.01375 · Cited by 7
“Reflexion (Shinn et al., 2023) generates self-critiques after failures and accumulates lessons across multiple attempts on the same tasks, reaching 91% on HumanEval through iterative refinement.”
Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?Preprint
GX-Chen, A., Lin, D., Samiei, M., Precup, D., Richards, B. A., Fergus, R., & Marino, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.09614 · Cited by 11
MemOrb: A Plug-and-Play Verbal-Reinforcement Memory Layer for E-Commerce Customer ServicePreprint
Huang, Y., Liu, Y., Zhao, R., Zhong, X., Yue, X., & Jiang, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.18713 · Cited by 8
“MemOrb builds on the reflexion paradigm (Shinn et al., 2023), which generates structured reflections from completed tasks and reuses them for future decision-making.”
Contemporary Agent Technology: LLM-Driven Advancements vs Classic Multi-Agent SystemsPreprint
Bădică, C., Bădică, A., Ganzha, M., Ivanović, M. D., Paprzycki, M., Selişteanu, D., & Wrona, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.02515 · Cited by 7
Agentomics-ML: Autonomous Machine Learning Experimentation Agent for Genomic and Transcriptomic DataPreprint
Martinek, V., Gariboldi, A., Tzimotoudis, D., Escudero‐Contreras, A., Blake, E., Čechák, D., Cassar, L., Balestrucci, A., & Alexiou, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.05542 · Cited by 7
“Due to the effectiveness for error-correcting capability in complex tasks (Huang et al., 2024b; Shinn et al., 2023), we use a reflection step.”
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.