Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 1,801–1,900 of 3,534
Natural Language Navigation Task Allocation for Robot Based on Agents
Li, Y., Liu, Y., Ren, J., & Dong, Y. (2024) · International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS) · doi:10.1109/iciibms62405.2024.10792847 · Cited by 1
“[6] proposed the framework of Reflexion, in which the agent will use language to think for itself about the task and record the reflection into memory to guide subsequent decisions.”
Generative AI and Its Potential to Augment NDE
Garg, V., & Singh, R. R. (2025) · Springer eBooks · doi:10.1007/978-3-031-84477-5_76 · Cited by 1
Self-Reflective Retrieval-Augmented Framework for Reliable Pharmacological Recommendations
Zhang, Z., Qiao, Y., & Lu, P. (2024) · Journal of Computational Methods in Engineering Applications · doi:10.62836/jcmea.v4i1.040108 · Cited by 1
“Works like ReAct [1] and Reflexion [2] have highlighted the importance of feedback loops in enhancing the reasoning and acting capabilities of LLMs.”
Cai, G., & Zhu, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-2725-0_13 · Cited by 1
Arigela, S., & Virwal, G. (2025) · International Journal of Advanced Information Technology · doi:10.5121/ijait.2025.15501 · Cited by 1
Mistake Notebook Learning: Batch-Clustered Failures for Training-Free Agent Adaptation
Su, X., Zhang, Y., Luo, H., Liu, X., & Huang, L. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.719 · Cited by 2
“Memory-based approaches store instance-level experience (Shinn et al., 2023; Zhao et al., 2024; Zhang et al., 2024) to correct errors locally.”
The Embodied World Model Based on LLM with Visual Information and Prediction-Oriented PromptsPreprint
Haijima, W., Nakakubo, K., Suzuki, M., & Matsuo, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.00765 · Cited by 1
“5 and GPT-4, acquired 3.3 times more items, traveled 2.3 times longer distances, and unlocked key tech tree milestones up to 15.3 times faster compared to other LLM-based methods (e.g., ReAct (Yao, 2022), Reflexion (Shinn, 2023) in MineDojo (Fan, 2022)).”
MethodInfluentialGame On: Towards Language Models as RL ExperimentersPreprint
Zhang, J., Lampe, T., Abdolmaleki, A., Springenberg, J. T., & Riedmiller, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.03402 · Cited by 1
EnvBridge: Bridging Diverse Environments with Cross-Environment Knowledge Transfer for Embodied AIPreprint
Kagaya, T., Lou, Y., Yuan, T. J., Lakshmi, S., Karlekar, J., Pranata, S., Murakami, N., Kinose, A., Oguri, K., Wick, F., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.16919 · Cited by 2
“As the method for Self-Reflection, we used a novel conventional approach (Shinn et al. (2023)) to generate a plan for the next trial.”
MethodQuery-Efficient Planning with Language ModelsPreprint
Gonzalez-Pumariega, G., Chen, W., Kedia, K., & Choudhury, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.06162 · Cited by 1
Burgess, P., & Shareghi, E. (2025) · Oxford University Press eBooks · doi:10.1093/9780198945215.003.0093 · Cited by 1
Self-Programming AI: Code-Learning Agents for Autonomous Refactoring and Architectural EvolutionPreprint
Khemani, K. (2025) · Research Square · doi:10.21203/rs.3.rs-6688473/v1 · Cited by 2
Mission Impossible: Feedback-Guided Dynamic Interactive Planning for Improving Reasoning on LLMsPreprint
Yan, D., Wu, G., & Zhou, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.05577 · Cited by 1
“Studies have explored adaptive learning techniques, which are crucial for agents to reflect on past interactions, learn from experiences, and improve future performance (Shinn et al., 2023; Madaan et al., 2023).”
InfluentialPan, J., Duan, Z., Tu, J., Cheng, A., & Wang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.06240 · Cited by 1
“Each agent can adjust its reasoning strategy through self-reflection to improve overall task performance [17, 18].”
Frog Soup: Zero-Shot, In-Context, and Sample-Efficient Frogger AgentsPreprint
Li, X. R., Hao, Y., & Fulop, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.03947 · Cited by 1
“Inspired by the Reflexion framework (Shinn et al., 2023) which allows LLM agents to verbally reflect on task feed-back signals, store reflection in memory and improve decision-making in subsequent trials, we design a simplified Reflexion LLM agent for playing Frogger.”
Enhancing Text Classification with a Novel Multi-Agent Collaboration Framework Leveraging BERTPreprint
Baban, H., Pidapar, S. A., Nema, A., & Lu, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.18653 · Cited by 1
“• Contextual Agent : Simulated contextual analysis based on historical data, considering the last 5 interactions per user, and employed diverse architectures such as BERT, RoBERTa, and XLNet [40, 42], integrating reinforcement techniques [35].”
AgentX: Towards Orchestrating Robust Agentic Workflow Patterns with FaaS-hosted MCP ServicesPreprint
Tokal, S. S. K. A., Jha, V., Eswaran, A., Jayachandran, P., & Simmhan, Y. L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.07595 · Cited by 2
“Many agentic design patterns like ReAct [41], Reflection [17], Reflexion [25], Plan and Exe-cute [35], and Magentic-One [9] have distinct strategies.”
AI Agents with Human-Like Collaborative Tools: Adaptive Strategies for Enhanced Problem-SolvingPreprint
Reed, H., Sugimura, M., & Zangari, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13547 · Cited by 1
“Chain-of-thought prompting enhances individual reasoning through prescribed intermediate steps [9], while Reflexion maintains episodic memory within single interactions [8].”
Multi-Agent LLM Systems: From Emergent Collaboration to Structured Collective IntelligencePreprint
Chen, F. (2025) · Preprints.org · doi:10.20944/preprints202511.1370.v1 · Cited by 4
Kobaladze, Z., Arnania, A., & Sanikidze, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.00013 · Cited by 2
DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMsPreprint
Li, Y., Liu, M., Wang, H., Zhang, Y., Ma, Y., & Tan, W. (2025) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.48550/arxiv.2511.20468 · Cited by 1
Subgoal Graph-Augmented Planning for LLM-Guided Open-World Reinforcement LearningPreprint
Fan, S., Zhang, B., Xu, Z., Teng, Y., Dai, S., Cheng, L., & Fan, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.20993 · Cited by 1
Toral, L., & Lazebnik, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.08329 · Cited by 2
“The first direction, using RL to improve LLMs, typically involves training LLMs to optimize specific objectives, such as generating more coherent text [73, 74], reducing toxicity [75, 76], or improving task completion rates [77].”
TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-makingPreprint
Jiao, K., Fang, Z., Liu, J., Li, B., Wang, Q., Liu, X., Ruan, J., Qiao, Z., Zhu, Y., Xu, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.08500 · Cited by 1
“…SayCan with a closed-loop principle, also applied in works like (Yao et al., 2023; Huang et al., 2022b; Kim et al., 2024a; Singh et al., 2023; Liang et al., 2023; Shinn et al., 2023; Wang et al., 2023b) to refine plans based on environment feedback for tasks such as automation and Minecraft.”
MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific CodingPreprint
Raghavan, S., & Mallick, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.08804 · Cited by 2
“Self-reflection and reinforcement learning methods, such as Reflexion [25], show particular promise in an iterative trial and error method that mimics the experimental cycle.”
Pather, K., Hadjigeorgiou, E., Krasniqi, A., Schmit, C., Rusu, I., Pons, M., & Khan, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.01412 · Cited by 1
Verbal Process Supervision Elicits Better Coding AgentsPreprint
Chen, H.-Y., Huang, C.-P., & Yao, J.-M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.18494 · Cited by 1
“Traditional approaches rely on static datasets, limiting the model’s capacity to refine its outputs dynamically(Shinn et al., 2023a).”
LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity RecognitionPreprint
Bai, F., Hassanzadeh, H., Saeedi, A., & Dredze, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.23722 · Cited by 4
“Recent research has explored this capability across a wide range of tasks, like code generation (Madaan et al., 2023) and planning (Shinn et al., 2023).”
CoMet: Metaphor-Driven Covert Communication for Multi-Agent Language GamesPreprint
Xu, S., & Zhong, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.18218 · Cited by 1
Cognitive Duality for Adaptive Web AgentsPreprint
Liu, J., Zhang, C., & Hu, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.05081 · Cited by 2
“To address this, works like Shinn et al. [2023] extend the policy to π 2 ( a t , h t +1 | s t , h t , g ) , where h t is no longer restricted to a simple sequence of past actions, but instead encodes rich representations of the agent’s episodic experiences.”
Okutomi, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.14925 · Cited by 1
“Prompt-based critique-and-revision (CnR) methods such as Self-Refine, Reflexion, CRITIC, and related approaches (e.g., [28, 29, 30, 31]) introduce an explicit feedback loop: an initial answer is critiqued and then revised, often with an auxiliary LLM-as-judge module or multi-agent debate.”
An Adaptive Multi-Agent Bitcoin Trading SystemPreprint
Singhi, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5580590 · Cited by 1
Lin, J., Pan, Z., Zhu, Y., Song, R., & Yang, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.23436 · Cited by 1
“A I] 28 N ov 2025 tories—initial samples that fail certain conditions progressively reflects and refine until all conditions are satisfied [23, 31].”
Monadic Context EngineeringPreprint
Zhang, Y., Yuan, Y., Wang, M., & Yao, A. C.-C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.22431 · Cited by 1
“The vanguard of artificial intelligence research increasingly focuses on building autonomous agents: systems that reason, plan, and act to accomplish goals by interacting with environments (Yao et al., 2022; Shinn et al., 2023).”
Adam, A. M. I. (2026) · Preprints.org · doi:10.20944/preprints202601.0195.v1 · Cited by 1
WebCryptoAgent: Agentic Crypto Trading with Web InformaticsPreprint
Kurban, A., Luo, W., Zuo, L., Zhang, Z., Han, R., Kang, Z., Tang, H., & Zhao, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.04687 · Cited by 2
“Our self-improvement process is inspired by the Re-flexion framework (Shinn et al., 2023) and extended through Contextual Experience Replay (CER) (Liu et al., 2025).”
Jeong, C. (2026) · Research Square · doi:10.21203/rs.3.rs-8139402/v2 · Cited by 1
Tiwari, A., & Gupta, V. (2026) · Research Square · doi:10.21203/rs.3.rs-8589677/v1 · Cited by 1
A Single Revision Step Improves Token-Efficient LLM ReasoningPreprint
Zhang, Y., Ma, T., Zhong, W., & Ma, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.02828 · Cited by 1
“…the answer, repeatedly (Madaan et al., 2023); Reflexion uses verbal feedback stored in memory to improve subsequent trials in agentic settings (Shinn et al., 2023); and multi-agent debate frameworks exchange arguments over multiple rounds to improve factuality and reasoning (Du et al., 2023;…”
PieArena: Ranking and Profiling Language Agents in Realistic Negotiation ScenariosPreprint
Zhu, C. Z. G., Cui, S., Dufallo, W. S., Jin, R., Xu, Z., Zhang, L., & Cain, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.05302 · Cited by 1
MemCoRe: Recovering Evidence from Progressively Compressed Factual Knowledge for Agent MemoryPreprint
Zhang, Z., Jia, X., Yang, Z., Song, Z., Xue, W., Han, S., & Guo, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.07885 · Cited by 1
The Potential of CoT for Reasoning: A Closer Look at Trace DynamicsPreprint
Bachmann, G., Jiang, Y., Dezfooli, S. M. M., & Nabi, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.14903 · Cited by 4
“…is that LLMs similarly benefit from spelling out bigger computations more slowly, using techniques such as backtracking and verification to explore several avenues before finally arriving at the best answer (Zhou et al., 2023; Shinn et al., 2023; Madaan et al., 2023b; Press et al., 2023).”
Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution EnvironmentsPreprint
Mukherjee, K., & Mukherjee, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.19450 · Cited by 1
“A further complication is that LLMs are increasingly embedded into agentic tool-using pipelines rather than used only for Q&A, including reasoning-and-acting designs that interleave natural-language deliberation with tool calls and external observations [13], [14], [15].”
Asking the Right Questions: Improving Reasoning with Generated Stepping StonesPreprint
Hu, H., Fu, T., Jiang, M., Miller, A. H., Bachrach, Y., & Foerster, J. N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.19069 · Cited by 3
SARE: Sample-wise Adaptive Reasoning for Training-free Fine-grained Visual RecognitionPreprint
Yang, J., He, D., Pan, M., Yao, K., Su, G., Zhang, W., Hu, Y., Li, T., Li, Y., & Zhang, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.17729 · Cited by 1
“Finally, these methods are typically stateless, preventing accumulation of reusable experience and leading to repeated failures in similar scenarios (Shinn et al., 2023).”
Farmer, M., Kochar, A., & Lee, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.19262 · Cited by 1
“…in LLMs. Recent work has explored various mechanisms for LLM self-correction, including chain-of-thought prompting (Wei et al., 2022), self-refinement (Madaan et al., 2023), verbal reinforcement learning (Shinn et al., 2023), and multi-agent debate (Du et al., 2023; Liang et al., 2023).”
RVLM: Recursive Vision-Language Models with Adaptive DepthPreprint
Mayumu, N., Khan, Z., Stephens, M., Mukala, P., & Oroumchian, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.24224 · Cited by 1
Early Discoveries of Algorithmist I: Promise of Provable Algorithm Synthesis at ScalePreprint
Kulkarni, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.22363 · Cited by 3
A Visionary Look at Vibe ResearchingPreprint
Feng, Y., & Liu, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.00945 · Cited by 2
“Reflexion [Shinn et al., 2023] adds a verbal self-critique loop: after an attempt fails or produces a suboptimal result, the agent generates a reflection on what went wrong and uses it to guide the next attempt.”
How does Chain of Thought decompose complex tasks?Preprint
Nadgir, A., Balasubramanian, V., & Chaudhari, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.08872 · Cited by 1
“These include generating parallel reasoning traces (Wang et al., 2023; Snell et al., 2024), making the model self-reflect or check its work (Shinn et al., 2023; Muennighoff et al., 2025), navigating a tree of potential completions (Yao et al., 2023) and other more sophisticated strategies (Besta et…”
Shi, J., Tao, S., Yin, X., Huang, C., Lei, W., & Ng, S.-K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.09566 · Cited by 2
Preregistered Belief Revision ContractsPreprint
Alqithami, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.15558 · Cited by 1
Harnessing Pre-Resolution Signals for Future Prediction AgentsPreprint
Wei, C., Gao, M., Han, Z., Chen, K., Zhuang, Y., Guan, H., Zhang, Y., Cheng, Y., Zhou, X., Chen, H., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.15719 · Cited by 1
“Reflexion stores verbal reflections in episodic memory, ExpeL distills reusable lessons, Voyager builds a reusable skill library, and GEPA evolves prompts through reflective search [Shinn et al., 2023, Zhao et al., 2024, Wang et al., 2023, Agrawal et al., 2025].”
HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMsPreprint
Kachroo, D., Anbazhagan, A. P., Caraeni, A., Lagasse, B., & Zhu, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.20140 · Cited by 1
“Shinn et al. [12] demonstrated how models can learn from their mistakes through structured self-reflection.”
Shanto, M. H., Asaduzzaman, M., & Ngom, A. (2026) · arXiv.org · doi:10.48550/arxiv.2604.22217 · Cited by 3
Operating-Layer Controls for Onchain Language-Model Agents Under Real CapitalPreprint
Barton, T. J., Constantakis, C., Hauseman, P., Mous, A., Hoffman, A., Bergeron, B., & Goodreau, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.26091 · Cited by 3
“Agent work often benefits from explicit reasoning and trajectory context, from ReAct-style thought/action/observation traces to memory streams and self-reflection buffers (Yao et al., 2023; Park et al., 2023; Shinn et al., 2023).”
Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language ModelsPreprint
Liu, X.-Z., Ram, D., Zhang, Y., Zhang, Z., Xia, W., & Soatto, S. (2026) · arXiv.org · doi:10.48550/arxiv.2605.07244 · Cited by 1
Huang, Z., Wu, J., Yu, T., Yan, Q., Surana, R., Bhattacharya, U., Yao, L., Wang, X. E., & McAuley, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.08526 · Cited by 1
“In these settings, the same task can yield markedly different action sequences across trials, even when the underlying policy is unchanged (Shinn et al., 2023; Xia et al., 2025).”
OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search SpacesPreprint
Li, X., Chen, J., Fang, X., Ding, S., Duan, H., Liu, Q., & Chen, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.08904 · Cited by 1
“LLM Agents Recent frameworks have empowered LLMs with tool use (ReAct (Yao et al., 2022a), RDAgent (Yang et al., 2025), Toolformer (Schick et al., 2023)) and feedback loops (Reflexion (Shinn et al., 2023)).”
Wang, X., Yu, K., Liang, X., Wang, L., & Han, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.14237 · Cited by 1
forge-harness: Engineering Methods for Robust AI Collaboration HarnessesPreprint
Kwon, S. (2026) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.20397565 · Cited by 1
Experiments in Agentic AI for SciencePreprint
Fox, J., & Fox, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.26305 · Cited by 2
“Furthermore, if an LLM endpoint hangs or hits a rate limit (HTTP 429), a thread-safe "guillotine" timeout catches it, applies a penalty strike, and seamlessly cascades the query to a fallback model—employing principles of verbal reinforcement and self-correction [37].”
Rah, A. (2026) · Research Square · doi:10.21203/rs.3.rs-9839527/v1 · Cited by 2
CoBG: Learning from Player Reviews to Design, Critique, and Revise Board Game RulebooksPreprint
Li, Z., Sun, J., Wang, Y., Li, C., Feng, Y., Zhang, F., Huang, Y., Sun, M., & Zhang, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.01976 · Cited by 1
Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping AttacksPreprint
Barker, M., Lakha, B., Serra, E., & Gullo, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.03606 · Cited by 1
“Other approaches focus on iterative improvement: Self-Refine and Reflex-ion use critique, feedback, or memory from earlier attempts to revise an initial solution and improve performance over repeated trials [40, 41].”
Self-Reflective APIs: Structure Beats Verbosity for AI Agent RecoveryPreprint
Canedo, A., & Chethan, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.05037 · Cited by 1
“Chain-of-Thought [9], ReAct [10], and Reflexion [11] improve how the agent thinks.”
Agents All the Way Down; A Methodology for Building Custom AI Agents from Substrate to ProductionPreprint
Forment, M. A., Pereira, J., García‐Peñalvo, F. J., & Guerrero, M. J. C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.11869 · Cited by 1
“The published Reflexion line [Shinn et al. 2023] prototypes; the methodology cycles (see §4).”
Duraj, J., Yetukuri, J., Zhou, S., Varma, D., Kong, R., Khan, I., & Zhou, Q. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.12924 · Cited by 1
StarOR: Synergizing Tree Search and Test-Time Reinforcement Learning for Optimization ModelingPreprint
Li, J., Ding, Y., Guan, S., Hou, R., & Wang, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.15197 · Cited by 1
“Reflexion [Shinn et al., 2023] is implemented as an iterative generate-execute-refine loop.”
Teaching agentic AI to generalize expert diagnostic reasoning in rare diseasesPreprint
Nguyen, M.-H., Gray, E., Schuler, B. A., Byram, K., Yang, C.-T., Ma, F., Xu, H., Su, W.-C., Yan, C., Wei, W.-Q., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.16149 · Cited by 1
“…analogue of reinforcement learning with human feedback (RLHF) — feedback optimizes a policy document, not model weights — and it places LiteOdyssey among weight-free, in-context methods 15,16,17,18,19,20 , where performance improves by changing the text in the context rather than the parameters.”
Intelligence Entropy Principle and the ADE Stability Engineering FrameworkPreprint
Liu, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.18065 · Cited by 1
Dixon, M. (2026) · arXiv.org · doi:10.48550/arxiv.2606.17383
“Modern frontier models are increasingly used as reasoning engines embedded within larger workflows that incorporate retrieval, planning, tool use, memory, and iterative decision making [9, 26, 38, 31, 37, 35].”
REVES: REvision and VErification--Augmented Training for Test-Time ScalingPreprint
Liu, Y., Zhou, R., Zhao, X., Sharaf, A., Lin, H., Biswas, A., Ghavamzadeh, M., Wang, Z., & Hong, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.18910 · Cited by 1
“A growing family of test-time scaling (TTS) algorithms formalizes this pattern, including sequential revision (Madaan et al., 2023; Shinn et al., 2023), tree search (Inoue et al., 2025), evolutionary refinement (Lee et al., 2025b), and best-of-N with a verifier (Chow et al., 2025).”
Marginal Advantage Accumulation for Memory-Driven Agent Self-EvolutionPreprint
Yang, M., Zheng, K., Cheng, C., Liu, Y., Lu, X., Jiang, F., & Zheng, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.20475 · Cited by 2
PaperClaw: Harnessing Agents for Autonomous Research and Human-in-the-Loop RefinementPreprint
Ye, W., Liu, H., Li, D., & Jiang, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.22610 · Cited by 2
“The core of the cycle is a propose → test → reflect loop that grows a hypothesis map only from measured verdicts, building on inference-time reasoning and self-correction (Wei et al., 2022b; Yao et al., 2023a; Shinn et al., 2023; Madaan et al., 2023).”
Self-Evolution for Multi-Turn Tool-Calling Agents via Divergence-Point Preference LearningPreprint
Tang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.23112 · Cited by 1
Goel, S., Vaidhyanathan, P., Schorling, L., Ares, N., & Osborne, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.20638 · Cited by 1
“Reflexion [30] showed that language agents can improve across trials by verbally reflecting on feedback and storing those reflections in episodic memory, avoiding weight updates while improving decision-making, coding, and reasoning.”
World-Model Collapse as a Phase TransitionPreprint
Song, X., & Cai, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.31399 · Cited by 1
State-Aware Runtime for Long-Horizon LLM Agents: A Conceptual Framework and Research AgendaPreprint
Chen, X. (2026) · doi:10.33774/coe-2026-vt9t2-v2 · Cited by 1
Secret Scanner Agent: Extracting Secrets and Access Context from Unstructured DocumentsPreprint
Chen, Z., Wakabayashi, M., & Siska, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.09011 · Cited by 1
“Reflexion and Self-Refine use critique and revision to improve outputs without additional training (Shinn et al., 2023; Madaan et al., 2023), while multi-agent debate lets separate agents propose and challenge answers (Du et al., 2024).”
Heterogeneous Agent Cohorts for Safe Open-Ended Exploration with Runtime Constraint MemoryPreprint
Liu, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.11226 · Cited by 1
Replicating Belief, Not Bits: Epistemic State Replication for Agentic SystemsPreprint
He, J., & Yu, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.09748 · Cited by 1
“ReAct interleaves reasoning traces with tool actions [10]; Reflex-ion stores linguistic feedback from prior attempts [11]; Generative Agents use observation, reflection, and retrieval to synthesize social behavior over persistent memories [12]; Voyager accumulates executable skills and environment…”
Automatic Model-Hardware Co-Adaptation for Heterogeneous AI AcceleratorsPreprint
Chen, T., Mi, M., Wang, P., Hu, C., & Zhang, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.12875 · Cited by 1
“Reflexion studies language agents that improve from verbal feedback [17].”
Cura 1T: Healthcare Foundation Model via Recursive Self-ImprovementPreprint
Chen, H., :, Brown, S., Metelski, D., Xia, T., Lee, J., Wang, Q., Riley, K., Wang, Q., & Yao, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.15314 · Cited by 1
“…critique or task feedback to improve subsequent attempts, while OPRO, DSPy, and TextGrad treat prompts, programs, or pipeline components as objects to optimize against a metric [Madaan et al., 2023, Shinn et al., 2023, Yang et al., 2023, Khattab et al., 2023, Yuksekgonul et al., 2024].”
Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context RestorationPreprint
Sheverev, M., Finkelstein, D., & Nikolenko, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.16848 · Cited by 1
“Reflexion [37] replaces parameter updates with verbal self-critique: an actor produces a trajectory, an evaluator scores it, and a self-reflection module writes a natural-language post-mortem that is appended to the next attempt’s prompt.”
Supra Cognitive Modes: A Routed Architecture for Agent MemoryPreprint
Tobkin, J., & Yang, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.19096 · Cited by 1
“Generative Agents stores observations and synthesized reflections for later planning [18]; CoALA organizes language agents around modular memory, actions, and decision processes [22]; Reflexion reuses verbal self-critique [21]; and Voyager accumulates a reusable skill library [26].”
Engineering Trustworthy Agentic AI for Critical SystemsPreprint
Al-Refai, O., Shahbaz, I., Husseinat, A. A., Mandulak, M., Kim, J., & Hammad, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.18548 · Cited by 1
“Self-reflective agents, such as Reflexion, add an explicit feedback step in which the agent converts failures into natural language reflections and reuses them in future attempts [3].”
InfluentialGeneric Constraints Projection: Four-Dimensional Type Inference for Dynamic LanguagesPreprint
Zhang, Q. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.19693 · Cited by 1
Lekkala, S. D., Appadurai, J. P., Bellibatlu, R. R., & Singh, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.20499 · Cited by 2
“Reflexion [10] introduces verbal self-critique with episode memory; Self-Debug [16] prompts the same model to inspect and fix its own program with execution traces.”
When AI Reviews Itself: Zero Answer Changes Across 72 Self-Correction RoundsPreprint
Hamidavi, A. (2026) · Research Square · doi:10.21203/rs.3.rs-10463884/v1 · Cited by 1
Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent TeamsPreprint
Wang, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.22917 · Cited by 1
Efficiency Matters in Autonomous ResearchPreprint
Yang, H., & Cao, Y. (2026) · arXiv.org · doi:10.48550/arxiv.2607.24647
DualityCert: Verifier-Gated Language-Model Repair of Broken Duality Claims in Quantum Field TheoryPreprint
Yu, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.23614 · Cited by 1
“Iterative refinement with feedback is studied in [15, 16], process-level verifiers are studied in [17], and program search filtered by a programmatic evaluator has produced new mathematical constructions [18].”
What makes prompts a graph: necessary and sufficient conditions for prompt graph engineeringPreprint
Macedo, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.27578 · Cited by 1
“Verification: which properties of a prompt graph can be checked statically, type compatibility along edges, termination of cycles with exit conditions, cost and latency bounds, and which require semantic judgment of text, where verification itself may need model-bearing nodes [33, 42]?”
Mirzaei, I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.28576 · Cited by 1
“The first version is Reflexion as Shinn et al. [23] describe it, and it is the only one of the two that supports any claim about their method.”
InfluentialRitchie, L., Mehta, S., Panavas, L., & Chen, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.01604 · Cited by 1
“…agents has concentrated on how to build them: ReAct interleaves reasoning with actions [Yao et al., 2023b], Reflexion adds verbal self-reflection [Shinn et al., 2023], search wraps inference-time exploration around the model [Yao et al., 2023a, Zhou et al., 2024], and decomposition-based…”
Cognitive Demand Steering for Adaptive Meta-Reasoning in Large Language ModelsPreprint
Scoville, J., Chen, S., Bang, Y., Winzeck, S., & Schwarz, J. R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.01319 · Cited by 1
“Self-Refine [11] generates feedback on completed drafts and revises them in a loop; Reflexion [14] leverages verbal feedback across prior episodes to avoid repeating past mistakes.”
Tiny Enough to Break In: Agentic Remote Access Trojans Powered by Small Language ModelsPreprint
You, Y., Adavelly, S., Lovelace, V., Berryman, C., Sadler, J., & Graham, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.03009 · Cited by 2
“Reflexion takes a related approach: agents reflect on feedback from earlier attempts and store it in memory, improving later decisions without updating model weights [10].”
Argus: A General-Purpose Agentic Reasoning Runtime for Long-Horizon TasksPreprint
Li, B., Wen, Z., Fan, Y., Wen, C., Yang, F., Guo, H., Wu, J., Zhang, J., Lei, J., Li, M., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.05144 · Cited by 1
“Generative Agents combine stored observations with reflection and retrieval (Park et al., 2023); Reflexion retains feedback across attempts (Shinn et al., 2023); Voyager accumulates executable skills (Wang et al., 2023); and MemGPT manages multiple memory tiers (Packer et al., 2023).”
From Trajectories to Evidence: Auditable Experimental Records for Industrial Research AgentsPreprint
Zhuang, Z., Lao, C., Xu, P., Xu, H., Yang, R., He, Y., Zhang, P., Cao, J., Huang, Y., Mu, G., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.05235 · Cited by 1
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.