Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 2,501–2,600 of 3,537
SoK: The Attack Surface of Agentic AI - Tools and Autonomy
Dehghantanha, A., & Homayoun, S. (2026) · arXiv · doi:10.48550/arxiv.2603.22928
PAACE: A Plan-Aware Automated Agent Context Engineering Framework
Yuksel, K. A. (2025) · arXiv · doi:10.48550/arxiv.2512.16970
“AI researchers have used this taxonomy to design agent memory stacks (Park et al., 2023; Shinn et al., 2023).”
Deshpande, D., Gangal, V., Mehta, H., Kannappan, A., Qian, R., & Wang, P. (2025) · arXiv · doi:10.48550/arxiv.2510.01353
“While acquisition of memory is relatively less standardized, works such as Reflexion [43] and MemInsight [44] rely on LLM self reflection capabilities to decide which memories are relevant and useful.”
LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
Zhang, G., Meng, F., Wan, G., Li, Z., Wang, K., Yin, Z., Bai, L., & Yan, S. (2025) · arXiv · doi:10.48550/arxiv.2509.24771
“For example, in verbal reinforcement learning approaches such as Reflexion (Shinn et al., 2023) and Mind Evolution (Lee et al., 2025), successful reflective strategies are instance-specific and are not transferred to subsequent tasks.”
MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning
Zhang, X., Zheng, Y., Xu, Z., Zhou, K., Shen, B., Ou, H., Zhang, T., & Lam, K.-Y. (2026) · arXiv · doi:10.48550/arxiv.2605.26154
“This component has been widely adopted by modern agent frameworks to stabilize behavior and improve decisions via stored experience, state, and reflection (Park et al., 2023; Shinn et al., 2023).”
Finding the Right Fit: Model-Harness Interactions across Agent Tasks
Li, Y., Zhou, Y., Teng, Y. L., Yang, F., Deng, Y., Lyu, Z., Dong, X., Chen, F., & An, B. (2026) · arXiv
“Language agents can improve by reflecting on feedback from earlier attempts [21], but self-correction without external feedback is unreliable [6], and models often fail to verify their own outputs [3, 5].”
MEMAUDIT: An Exact Package-Oracle Evaluation Protocol for Budgeted Long-Term LLM Memory Writing
Bhargava, N., & Barrento, R. S. (2026) · arXiv · doi:10.48550/arxiv.2605.02199
“MemGPT, Reflexion, MemoryBank, Mem0, and A-Mem propose memory architectures or policies for long-term agents [9, 10, 15, 3, 13].”
Lodhiya, H., McManus, A., & Walker, R. (2026) · arXiv · doi:10.48550/arxiv.2609.27844
MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution
Wang, Z., Yan, M., Bi, J., Yan, S., Tresp, V., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2607.05297
VADAOrchestra: Neurosymbolic Orchestration of Adaptive Reasoning Workflows
Baldazzi, T., Bellomarini, L., Coletta, A., Iezzi, M., Maple, C., Pesare, A., & Sallinger, E. (2026) · arXiv · doi:10.48550/arxiv.2606.22485
Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis
Goo, T., Kim, J., Han, K., Jo, G., Kim, J.-S., & Kim, T.-H. (2026) · arXiv · doi:10.48550/arxiv.2607.20527
MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision
Ke, Z., Xu, A., Ming, Y., Nguyen, X.-P., Chin, R., Xiong, C., & Joty, S. (2025) · arXiv · doi:10.48550/arxiv.2505.14996
“1 These agents often assume distinct roles , such as generator or verifier [43], engage in debates offering varied perspectives [38, 46], and perform assigned sub-tasks [27].”
Oracle Agent Memory as an Enterprise Memory Substrate for Long-Horizon AI Agents
Alake, R., Bernardis, C., Cayet, P., Engel, L., Hilloulin, D., Hong, S., Hosler, A., Kavantzas, N., Kossyk, I., Le, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.13157
Shared Selective Persistent Memory for Agentic LLM Systems
Pedada, S., Dhavala, A., & Patil, N. (2026) · arXiv · doi:10.48550/arxiv.2607.09493 · Cited by 1
EvoFlow: Evolving Diverse Agentic Workflows On The Fly
Zhang, G., Chen, K., Wan, G., Chang, H., Cheng, H., Wang, K., Hu, S., & Bai, L. (2025) · arXiv · doi:10.48550/arxiv.2502.07373
Multi-agent Architecture Search via Agentic Supernet
Zhang, G., Niu, L., Fang, J., Wang, K., Bai, L., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2502.04180
AutoHarness: improving LLM agents by automatically synthesizing a code harness
Lou, X., Lázaro-Gredilla, M., Dedieu, A., Wendelken, C., Lehrach, W., & Murphy, K. P. (2026) · arXiv · doi:10.48550/arxiv.2603.03329
“Reflexion (Shinn et al., 2023) introduced a verbal reinforcement learning loop where agents reflect on failure logs.”
VERSE: Verified Self-Evolving Optimizer for Agent Harnesses
Wang, Z., Ge, Y., Wang, Z., Wang, H., Xu, Y., Frandsen, J., Fu, S., Wilson, A. C., & Reddy, C. K. (2026) · arXiv · doi:10.48550/arxiv.2610.02616
“…(Nie et al., 2026), refines prompt-level harnesses from pairwise comparisons (Lee et al., 2026a), optimizes prompts and programs with LLM feedback (Shinn et al., 2023; Yang et al., 2024; Khattab et al., 2024), updates model weights (Hebbar et al., 2026), or distills skills from trajectories (Ni…”
Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement
Yan, H., Su, M.-L., Zhang, H., Li, Z., Zhang, C., Zhang, S., Chen, Y., Bai, L., & Hu, S. (2026) · arXiv · doi:10.48550/arxiv.2609.01481 · Cited by 5
Can LLMs Self-Correct Table Reasoning Errors?
Shaikh, F. (2026) · Proceedings of the First Workshop on Structured Understanding, Retrieval, and Generation in the LLM Era (SURGeLLM 2026) · doi:10.18653/v1/2026.surgellm-1.19
Barbaste, P., Darrigol, T., Vu, G., & Wiltberger, T. (2026) · arXiv · doi:10.48550/arxiv.2609.00006
Transferable End-to-End Optimization for Indirect Long-Term Memory Poisoning in LLM Agents
Zang, C., Wang, J., Chen, W., Meng, X., Wang, L., Gao, X., Li, Z., & Guo, S. (2026) · arXiv
Li, H., Feng, L., & Wang, D. (2026) · Electronics · doi:10.3390/electronics15102011
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Zhao, H., Yu, Z., He, Z., Fang, M., Tutunov, R., Bou-Ammar, H., Luo, W., & Wang, J. (2026) · arXiv · doi:10.48550/arxiv.2610.11794
“Reflexion stores linguistic reflections on task feedback to improve later decisions [36].”
ADIAS: Automated Design of Interactive Agentic Systems
Jiang, L., Tang, B., Goetz, S., & Guo, Y. (2026) · arXiv · doi:10.48550/arxiv.2608.06410
“Reflection and experiential-learning methods retain lessons or heuristics derived from previous trajectories, while systems such as ACE and SkillOpt maintain editable playbooks or skill documents (Shinn et al., 2023; Zhao et al., 2024; Zhang et al., 2026c; Yang et al., 2026).”
When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agents
Hu, Q., Long, Q., & Wang, W. (2026) · arXiv · doi:10.48550/arxiv.2604.27003 · Cited by 19
Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
Suzgun, M., Yuksekgonul, M., Bianchi, F., Jurafsky, D., & Zou, J. (2025) · arXiv · doi:10.48550/arxiv.2504.07952
“In the broader test-time learning literature, reflexive, compositional, and iterative refinement approaches like Re-flexion (Shinn et al., 2023), Self-Refine (Madaan et al., 2023), (Self-)Critic (Gou et al., 2023), Chameleon (Lu et al., 2023), Meta-Prompting (Suzgun & Kalai, 2024), and Self-RAG…”
The AI Hippocampus: How Far are We From Human Memory?
Jia, Z., Li, J., Kang, Y., Wang, Y., Wu, T., Wang, Q., Wang, X., Zhang, S., Shen, J., Li, Q., et al. (2026) · arXiv · doi:10.48550/arxiv.2601.09113
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Shah, D., Badhe, S., Kathrotia, N., & Tiwari, P. (2026) · arXiv
Ning, X., Tieu, K., Fu, D., Wei, T., Li, Z., Bei, Y., Zou, J., Ai, M., Liu, Z., Li, T.-W., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.18747
Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity
Zhang, H., Cui, Z., Chen, J., Wang, X., Zhang, Q., Wang, Z., Wu, D., & Hu, S. (2025) · arXiv · doi:10.48550/arxiv.2502.08788 · Cited by 84
“These results suggest that it is valuable to explore integrating more powerful single-agent inference approaches (Yao et al., 2022; Shinn et al., 2024), or advanced models with inherently strong reasoning capabilities (Guo et al., 2025; Liu et al., 2024; OpenAI, 2024b), to further optimize…”
A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052 · Cited by 70
At Equal Inference Cost, Multi-Agent Structure Does Not Beat a Single Frozen Agent
Dylan, D., Brennan, A., Murphy, C., O'Sullivan, N., Kelly, C., & Walsh, S. (2026) · arXiv · doi:10.48550/arxiv.2609.04217 · Cited by 3
“Called on a stall (a repeated or null-effect observation), it inspects the last action and observation and emits a one-line correction in the manner of verbal self-correction [7], injected into the Executor’s preamble for the next step.”
Nam, Y., Jeong, J., & Kim, N. (2026) · arXiv · doi:10.48550/arxiv.2606.09500 · Cited by 4
Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents
Wu, Y., Zhang, L., Zhou, Y., Wang, M., Peng, B., Li, S., Fan, X., & Zhao, Z. (2026) · arXiv · doi:10.48550/arxiv.2607.08716 · Cited by 20
“…improves model outputs using self-generated feedback (Madaan et al., 2023); Reflexion stores verbal reflections from prior attempts in an episodic memory buffer (Shinn et al., 2023); and Tree of Thoughts expands inference-time reasoning through search over intermediate thoughts (Yao et al., 2023a).”
TopoRefine: Iterative Refinement with Reasoning Topology as High-Level Feedback
Liao, H., Hu, S., Zhu, Z., He, H., & Jin, Y. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10888346
“External signals typically include tools [19], [20], experts feedback [21], [22], and verifier models [23], [24].”
Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents
Wei, T., Shi, Z., Lin, M., He, B., Liu, Z., Sang, Y., Bei, Y., Ning, X., Zou, J., Li, T.-W., et al. (2026) · arXiv · doi:10.48550/arxiv.2608.15071
ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization
Wang, Y., Yang, L., Li, G., Wang, M., & Aragam, B. (2025) · arXiv · doi:10.48550/arxiv.2502.04306
Tran, D., & Kiela, D. (2026) · arXiv · doi:10.48550/arxiv.2604.02460 · Cited by 47
“Prior work has proposed many ways to structure collaboration between LLMs, including debate, role specialization, ensemble/self-consistency, and reflection (Du et al., 2024; Zhang et al., 2023; Li et al., 2024; Shinn et al., 2023).”
Tsui, K. (2025) · arXiv · doi:10.48550/arxiv.2507.02778 · Cited by 29
“Recent work has highlighted intrinsic self-correction, where LLMs generate feedback on their own outputs (Shinn et al., 2023; Madaan et al., 2023; Kim et al., 2023; Kamoi et al., 2024a), as a way to improve performance, but critical limitations persist.”
Are Retrials All You Need? Enhancing Large Language Model Reasoning Without Verbalized Feedback
Potamitis, N., & Arora, A. (2025) · arXiv · doi:10.48550/arxiv.2504.12951 · Cited by 2
Beyond the Capability Boundary: Zeroth-Order Optimization for Self-Evolving LLM Agents
Liu, B., Fan, X., Wu, Y., Gao, Z., Gao, M., Li, C., & Jia, Y. (2026) · arXiv
Jiang, J., Shen, J., Kim, S., Yoo, K. M., Kim, J., & Kim, S. (2026) · arXiv · doi:10.48550/arxiv.2603.05863 · Cited by 8
Useful Memories Become Faulty When Continuously Updated by LLMs
Zhang, D., Lin, Y., Wu, Z., Sun, Y., Li, B., Li, D., & Peng, H. (2026) · arXiv · doi:10.48550/arxiv.2605.12978 · Cited by 35
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
Chen, Z., Sun, Z., Shi, Y., Lo, D., & Jiang, L. (2026) · arXiv · doi:10.48550/arxiv.2607.01211 · Cited by 9
“In practice, current agent systems often use multi-agent workflows rather than relying on a single run [24], [29]–[31].”
ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
Lange, R. T., Imajuku, Y., & Cetin, E. (2025) · arXiv · doi:10.48550/arxiv.2509.19349
Recursive Self-Evolving Agents via Held-Out Selection
Nguyen, M., Nguyen, Q., & Vuong, P. (2026) · arXiv · doi:10.48550/arxiv.2606.28374
Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies
Zhou, H., Wan, X., Sun, R., Palangi, H., Iqbal, S., Vulić, I., Korhonen, A., & Arık, S. Ö. (2025) · arXiv · doi:10.48550/arxiv.2502.02533
“The serial form aims to advance the exploitation of a task via a chain of agents, where LLMs can serve as reflective agents to self-justify and refine former predictions (Madaan et al., 2024; Shinn et al., 2024).”
InfluentialGitSwarm: Decentralized Compounding Inference
Shah, V., Samanta, A., Dahal, P., Plekhanov, M., Wu, C.-J., Yih, S., Munos, R., Fergus, R., Foerster, J., Salakhutdinov, R., et al. (2026) · arXiv · doi:10.48550/arxiv.2610.04862
Role-Augmented Intent-Driven Generative Search Engine Optimization
Chen, X., Bao, J., Wu, H., Chen, Z., Pan, L., & Liao, Y. (2025) · arXiv · doi:10.48550/arxiv.2508.11158
“Reflection strategies have demonstrated efficacy in improving LLM performance across a variety of tasks [8], [23], especially in multi-perspective reflection paradigms [28], [29].”
ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents
Xue, S., Zhong, J., Nan, Z., Li, W., Yu, Z., Ding, J., Gao, Q., Zhan, P., Zhang, Y., Cheng, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2609.17523
“Reflection and harness optimization revise reusable instructions and execution procedures [2, 11, 16, 27]; interaction-driven adaptation and rubric-based reinforcement learning provide mechanisms for model improvement [7, 19, 28].”
SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories
Bogin, B., Yang, K., Gupta, S., Richardson, K., Bransom, E., Clark, P., Sabharwal, A., & Khot, T. (2024) · arXiv · doi:10.48550/arxiv.2409.07440
“Reflexion Agent Prompt You will be given the history of a past experience in which you were placed in an environment and given a task to complete.”
BackgroundMethodInfluentialTwin: Playing an Unknown Game with a Test-Time Digital Twin
Skoutnev, A., Acharya, K., Longhitano, G., Udell, M., Ellis, K., & Drori, I. (2026) · arXiv · doi:10.48550/arxiv.2608.14490
Tiwari, S., Chugh, T., Rickert, N., Peter, S., Mahajan, R., & Shen, H. (2026) · arXiv · doi:10.48550/arxiv.2606.16824
“We observe that requests triggered by tool completion (i.e., model-generated tool calls) [33, 41] are 20 × more frequent than user-initiated requests at the median, indicating that the request generation process is predominantly closed-loop.”
Courtis, D., Li, W., & Sanner, S. (2026) · arXiv · doi:10.48550/arxiv.2607.01531
“For our factored-dynamics interpretation refer to Appendix B.2.”
Agentic Memory Management Is the Next Frontier of Information Retrieval
Ai, Q. (2026) · arXiv · doi:10.48550/arxiv.2610.32464
“It can accumulate experience : Reflexion stores verbal reflections on failed attempts and conditions later attempts on them [56].”
EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms
Yuan, S., Song, K., Chen, J., Tan, X., Li, D., & Yang, D. (2024) · arXiv · doi:10.48550/arxiv.2406.14228
EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
Wu, R., Wang, X., Mei, J., Cai, P., Fu, D., Yang, C., Wen, L., Yang, X., Shen, Y., Wang, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.16079
“LLM agents have been widely explored through frameworks such as ReAct, which interleaves reasoning and actions, and Reflecion, which improves task performance via self-reflection [4, 17].”
Inducing Programmatic Skills for Agentic Tasks
Wang, Z. Z., Gandhi, A., Neubig, G., & Fried, D. (2025) · arXiv · doi:10.48550/arxiv.2504.06821
“Meanwhile, non-parametric approaches can directly augment reflections (Shinn et al., 2023), raw past experiences (Wang et al., 2023; Zheng et al., 2023), or further induced reusable workflows (Wang et al., 2024b).”
GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators
Guo, J., Yang, L., Chen, P., Xiao, Q., Wang, Y., Juan, X., Qiu, J., Shen, K., & Wang, M. (2025) · arXiv · doi:10.48550/arxiv.2512.19682
Li, X. (2024) · arXiv · doi:10.48550/arxiv.2406.05804
“…context of this survey, LLM-based agents are defined by their ability to interact actively with external tools ( tool use ) or environments (Yao et al., 2023b) and are designed to function as integral components of agency for planning (Yao et al., 2023a) and feedback learning (Shinn et al., 2023).”
InfluentialLLM-as-a-Verifier: A General-Purpose Verification Framework
Kwok, J., Li, S., Atreya, P., Liu, Y., Jiang, Y., Finn, C., Pavone, M., Stoica, I., & Mirhoseini, A. (2026) · arXiv · doi:10.48550/arxiv.2607.05391
Critique Ability of Large Language Models
Luo, L., Lin, Z., Liu, Y., Shu, L., Zhu, Y., Shang, J., & Meng, L. (2023) · arXiv · doi:10.48550/arxiv.2310.04815
Recursive self-improvement of AI research agents
Srikanth, D., Zhao, B., Xu, D., Wu, Y., & Jiang, Z. (2026) · arXiv · doi:10.48550/arxiv.2609.26457
AI for Auto-Research: Roadmap & User Guide
Kong, L., Sun, X., Chow, W., Li, L., Lin, K. Q., Zhang, X. B., Wang, S., Li, R., Wu, Q., Gao, W., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.18661
Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey
Zhu, J., Zhu, M., Rui, R., Shan, R., Zheng, C., Chen, B., Xi, Y., Lin, J., Liu, W., Tang, R., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.11102
“Representative examples include web search agents [7], API-calling agents for online tasks [8], and code agents that iteratively debug based on execution feedback [9].”
From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
Xia, S., Xu, Z., Chai, J., Fan, W., Song, Y., Wang, X., Yin, G., Lin, W., Zhang, H., & Wang, J. (2025) · arXiv · doi:10.48550/arxiv.2511.07800
Mishra, S., Niroula, S., Yadav, U., Thakur, D., Gyawali, S., & Gaire, S. (2026) · arXiv · doi:10.48550/arxiv.2603.07379
“We adopt four classes: Chain-of-Thought, ReAct-style interleaving, reflection-based reasoning, and tree-based exploration [12], [13], [23], [30].”
InfluentialNex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
AGI Team, Cai, Y., Chen, L., Chen, Q., Ding, Y., Fan, L., Fu, W., Gao, Y., Guo, H., Guo, P., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.04987
“Unlike biological systems that learn through interaction, LLMs typically fail to perform robust error recovery or self-correction when actions fail (Shinn et al., 2023; Yao et al., 2023).”
Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Traces
Yu, S., Chong, D., Nandi, A., Soylu, D., Sun, J., Manning, C. D., & Shi, W. (2026) · arXiv · doi:10.48550/arxiv.2605.10913
“Other lines refine an agent’s context and long-term retrieval through evolutionary search [25, 38] or memory-augmented architectures [50, 21, 51, 42, 39, 30].”
Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses
Yu, Z., Wu, Y., Yin, Z., Chen, K., Zhao, Z., Wang, M., Yan, S., & Yang, L. (2026) · arXiv · doi:10.48550/arxiv.2608.24876
“Although these methods differ in what they rewrite, they agree on how a rewrite is admitted: by the model’s own judgment of usefulness (Shinn et al., 2023), or by an improvement in a single aggregate number (Agrawal et al., 2026; Khattab et al., 2024; Liu et al., 2026b; Yang et al., 2026).”
Putting It All into Context: Simplifying Agents with LCLMs
Jiang, M., Ruan, Y., Lastras, L., Kapanipathi, P., & Hashimoto, T. (2025) · arXiv · doi:10.48550/arxiv.2505.08120
“LM agents for software engineering With the rapid advancement of autonomous LM-based agents (Yao et al., 2023; Shinn et al., 2023; Schick et al., 2023), there has been growing research interest in developing LM agents for solving repository-level software engineering tasks on benchmarks like…”
CLI-Anything: Towards Agent-Native Computer Use
Yang, Y., Fan, T., & Huang, C. (2026) · arXiv · doi:10.48550/arxiv.2606.03854
“API-Bank and Gorilla focus more directly on API-call coverage and correctness [8, 7]; reflection work shows how agents can use feedback from failed attempts [26].”
Huang, J., Hsia, J., Sun, J., Shi, F., Huang, W., & White, I. H. (2026) · arXiv · doi:10.48550/arxiv.2607.14890
“The point is methodological: a system that gates its author is forced to either fix a real bug or visibly refuse to weaken its own Self-reflection and iterative refinement [32, 19, 11, 26] show that language agents can improve through tool feedback, reflection, retry, and skill reuse.”
N-Critics: Self-Refinement of Large Language Models with Ensemble of Critics
Mousavi, S., Gutiérrez, R. L., Rengarajan, D., Gundecha, V., Babu, A. R., Naug, A., Guillen, A., & Sarkar, S. (2023) · arXiv · doi:10.48550/arxiv.2310.18679
“…Li et al., 2019; Jauregi Unanue et al., 2021; Zelikman et al., 2022; Huang et al., 2022) to post output generation refinement (Madaan et al., 2023; Shinn et al., 2023; Zhang et al., 2023; Pan et al., 2023; Du et al., 2022; Yu et al., 2023; Gou et al., 2023; Paul et al., 2023; Le et al., 2022;…”
MethodWhen Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents
Torres, G., Shrestha, S., & Misra, S. (2026) · arXiv · doi:10.48550/arxiv.2607.06595
“The works discussed thus far have focused on conversational agents that engaged in conversation with a single user; a separate line of work involves action-planning agents such as Reflexion [23], which uses repeated self-reflections on past attempts to improve its current action plan.”
Are You Still the Agent I Authorized? Earned Authority under a Fixed Ceiling for Evolving Agents
Zhang, Z., & Zhang, X. (2026) · arXiv · doi:10.48550/arxiv.2607.23586
“Reflexion stores linguistic feedback in episodic memory, ExpeL distills experience across tasks, and Voyager accumulates executable skills through interaction [3]–[5].”
Mendel Gödel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution
Liu, C., Liu, Y., Yan, S., Tresp, V., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2608.07645
Generative Agents for Multi-Agent Autoformalization of Interaction Scenarios
Mensfelt, A., Stathis, K., & Trencsenyi, V. (2025) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia251256
“Moreover, this formal representation can be iteratively refined using automatic reflection and refinement mechanisms [21, 29, 32, 13], enabling continuous improvement via feedback loops.”
AutoAgent: Evolving Cognition and Elastic Memory Orchestration for Adaptive Agents
Wang, X., Liao, N., Wei, S., Tang, C., & Xiong, F. (2026) · arXiv · doi:10.48550/arxiv.2603.09716
“Reflexion shows that language agents can improve by reflecting on failures and incorporating verbal feedback into subsequent trials [17].”
PreAct: Computer-Using Agents that Get Faster on Repeated Tasks
Li, B. (2026) · arXiv · doi:10.48550/arxiv.2606.17929
“Almost all of these agents follow the observe–reason–act paradigm popularized by ReAct [59], layered on chain-of-thought and search-style reasoning [52, 58, 50], tool use [38], self-reflection [39], and autonomous task loops [40, 41].”
Agensh: Scaling Organizational Intelligence to 1,024 Agents
Zhan, Z., Song, T., Dong, L., Huang, S., Lian, J., Xia, Y., & Wei, F. (2026) · arXiv · doi:10.48550/arxiv.2609.26781
Learning the ARTS of Search for Automated Discovery
Juneja, G., Jain, A. K., Nathani, D., Wang, W. Y., & Wang, X. E. (2026) · arXiv · doi:10.48550/arxiv.2606.21891
Super-additive Cooperation in Language Model Agents
Tonini, F., & Galke, L. (2025) · arXiv · doi:10.48550/arxiv.2508.15510
“…either mechanism alone: The average one-shot cooperation rate µ osc under the combined (super-additive) condition exceeds that under either mechanism alone: With the popularity of LLM agents, various frameworks have emerged to enhance agent coherence, reasoning, and overall performance [20] [17].”
NormCode: A Semi-Formal Language for Auditable AI Planning
Guan, X., Li, Y., Wu, Z., & Zhang, R. (2025) · arXiv · doi:10.48550/arxiv.2512.10563
“Reflex-ion (Shinn et al., 2023) extends this with self-evaluation: after each trial, the agent generates linguistic feedback stored as episodic memory, enabling iterative self-correction without parameter updates.”
NeuroProlog: Multi-Task Fine-Tuning for Neurosymbolic Mathematical Reasoning via the Cocktail Effect
Zunjare, P., & Hsiao, M. (2026) · arXiv · doi:10.48550/arxiv.2603.02504
Chen, Z., Zhu, D., & Zheng, L. N. (2026) · arXiv
“Agents can also preserve reflections, skills, and experience across tasks [40–42].”
Past as a Guide: Leveraging Retrospective Learning for Python Code CompletionPreprint
Shin, S., Chang, S., & Choi, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.07635
“On a different tangent, "Reflexion," as developed by Shinn et al. [17], grants agents dynamic memory and the ability for self-reflection to improve reasoning abilities.”
BackgroundInfluentialLDM$^2$: A Large Decision Model Imitating Human Cognition with Dynamic Memory EnhancementPreprint
Wang, X., Li, L., & Zeng, D. D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.08402
Plan of Thoughts: Heuristic-Guided Problem Solving with Large Language ModelsPreprint
Liu, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.19055
Towards Reliable and Efficient Long-Term Recommendation with Large Foundation Models
Shi, W. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651258
CERET: Cost-Effective Extrinsic Refinement for Text Generation
Cai, J. C., Su, H., Sunkara, M., Shalyminov, I., & Mansour, S. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.409 · Cited by 2
Wu, Y., & Henriksson, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70352-2_12
A Summary of Advances in Document Summarization from 2023-2024
Sandia National Labs., A., Link, H., Administration, U. A. S. F. H. R. A., & Hooker, A. (2024) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/2462994
“[58] • Devil’s advocate evaluation [53] and reinforcement learning from human feedback [64] to ensure internal scoring of actions is well correlated with human scores.”
CELI: Controller-Embedded Language Model InteractionsPreprint
Wagner, J.-S., DeCaprio, D., Raja, A. C. M., Holman, J. M., Brady, L. K., Cheung, S. C., Barzekar, H., Yang, E., Martinez, M. A., Soong, D., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.14627
“Subsequent research introduced more advanced strategies such as Chain-of-Thought (Wei et al., 2022), Tree of Thoughts (Yao et al., 2023a), ReAct (Yao et al., 2023b), and Re-flexion (Shinn et al., 2023), each enhancing LMs’ problem-solving and decision-making abilities.”
MethodPredictive Indicators of the Performance of Large Language Models
Sandia National Laboratories (SNL-NM), A., Link, H., Administration, U. A. S. F. H. R. A., Berman, B., Cooper, R., Dodge, H., Fluke, N., & VanCleave, R. (2024) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/2480207
Leveraging feature engineering in AI-generated content for enhanced virtual learning experiences
Wang, X., Wang, H., & Cheng, J. (2025) · International Conference on Big Data, Computational Intelligence, and Applications (BDCIA) · doi:10.1117/12.3059003
“Define the feature weight vector for learner 𝑖 at time 𝑡 as 𝐰 𝑖 (𝑡) = {𝑤 𝑖1 (𝑡), 𝑤 𝑖2 (𝑡), … , 𝑤 𝑖𝑚 (𝑡)} , where 𝑤 𝑖𝑗 (𝑡) represents the weight of feature 𝑥 𝑖𝑗 at time 𝑡 [9].”
Optimizing English Lexical Databases with BERT and Reinforcement Learning
Saranya, V., Murthy, G. R. K., Alapati, P. R., Mythili, M., Swarnamughi, K., & Bala, B. K. (2025) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect63952.2025.10958921
“Thhese studies advocate for the integration of domain knowledge in the model of NLP. Shinn [10] describe a new framework called Reflexion for improving language agents using language feedbacks.”
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.