Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 3,001–3,100 of 3,537
Benchmarking General Mobile Assistants in Challenging Real-World ScenariosPreprint
Zhu, Y., Gao, F., Fan, J., Zeng, J., Wu, M., Li, C., Xu, H., Li, P., Yan, M., & Liu, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.27477
“Reflexion (Shinn et al., 2023) further augments agents with linguistic feedback, self-reflection, and episodic memory to improve behavior across repeated attempts.”
EVAR: Evidence-Validated Hypothesis Admission for Budget-Aware Narrative Reasoning
Liu, P., Ji, Z., & Ping, J. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.577
Policy-TD: Teacher-Distilled Runtime Control for Frozen Small Language ModelsPreprint
Avvaru, S. T., Pokala, K. S., & Dodda, V. K. R. (2026) · Research Square · doi:10.21203/rs.3.rs-10846457/v1
Self-Evolving Skills via Surrogate-Guided Solve-and-ReproducePreprint
Liu, J., Ren, P., Xia, Y., Wang, H., Zhao, Z., & Dong, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.28638
“Madaan et al. (2023) show that language feedback alone can iteratively improve generation quality, and Shinn et al. (2023) extend this with verbal reinforcement from environment traces.”
Mehta, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.28795
“CodeAgent [7] and the self-correction literature [8] propose systems in which running code and reading the results improve outcomes, with Reflexion [9] and Self-Debugging [10] the canonical demonstrations that a model can improve its own output from execution feedback.”
Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOpsPreprint
Sakhinana, S. S., & Runkana, V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.29615
“Graph engineering [Zhuge et al., 2024, Zhang et al., 2024] structures the end-to-end cloud MLOps workflow and its transitions, loop engineering [Shinn et al., 2023, Gou et al., 2023] provides bounded correction and adaptation, and agent harness engineering [Zhong and Zhu, 2026, Lin et al., 2026]…”
Tambe, W. M. A., Gaur, N., Kaur, J., & Kaur, G. (2026) · Research Square · doi:10.21203/rs.3.rs-10875799/v1
Dudaka, M. (2026) · Acadlore Transactions on AI and Machine Learning · doi:10.56578/ataiml050305
YesTrack: Referring Multi-object Tracking via MLLM-Based Yes/No Verification
Hu, Q., Sun, Q., Dai, Q., Ding, J., Zhang, W., Zhou, X., & Zou, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37020-4_4
“Such discriminative formulations have been increasingly adopted as lightweight decision mechanisms, where large models are prompted to produce explicit Yes/No judgments for evaluation, task verification, and iterative control [29,30,32].”
Guyot, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.01702
Implementace AI agentů do rezervačního systému
Heralt, J. (2026) · DSpace VŠB-TUO (VŠB-TUO)
Counterexamples as Feedback for Agent Self-CorrectionPreprint
Badrinarayan, S., & Parthasarathy, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.02892
A scalable AI framework for automated TNM staging of 51,242 patients with nasopharyngeal carcinoma
Zhen, Z.-C., Qi, Y.-X., Wang, Y.-Q., Zhang, W.-W., Xu, J.-M., Liu, S.-T., Zeng, X.-Y., Zhou, G., Feng, C., Jiao, Z.-T., et al. (2026) · npj Precision Oncology · doi:10.1038/s41698-026-01675-9
Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meeting
Khan, M., Kirstein, F., Ruas, T., & Gipp, B. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1377
Mirage of Machine CertaintyPreprint
ABDULLAH, U. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7411838
A Scoping Review of Agentic AI: Applications, Emerging Trends, Risks & Future Directions
Yadav, A., Nnadi, L. C., Isiwu, C. P., Okechukwu, I. S., & Watanobe, Y. (2026) · Electronics · doi:10.3390/electronics15215024
Substrate-Aware AI Agents: Execution Context as a First-Class InputPreprint
Agrawal, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.05232
“Execution-feedback approaches such as Reflexion and Self-Refine show how later feedback can improve subsequent generations [8, 9].”
Inventory-Grounded Policy-Level Optimization for Training-Free AI Search
Zhou, W., Wu, T., Ding, J., Fan, Z., & Cao, Y. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-industry.7
Shanmugavadivelu, P. (2026) · International Journal of Artificial Intelligence and Machine Learning · doi:10.51483/ijaiml.6.9s.2026.1500-1512
MARBO: Relational Belief Grounding for LLM Agents in Social Deduction Games
Yechan, H., Bae, S., Jeongmo, K., Bang, S., & Seungyul, H. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1376
The impact of GenAI on the future of requirements engineering
Breaux, T. D., & Singhal, A. (2026) · Requirements Engineering · doi:10.1007/s00766-026-00477-6
Ponduru, P. S., Nandanavanam, P. P. V., & Ponduru, S. K. K. (2026) · Engineering and Technology Journal · doi:10.47191/etj/v11i09.05
Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial ObservabilityPreprint
Chattopadhayay, A., & Halder, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.10036
CA-WM: Learning Cellular Automaton World Models via LLM Rule Induction and Online Correction
Zavattari, C., Tommasi, A., & Prencipe, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-40774-0_51
The Computational Primitives of AdaptationPreprint
Page, J. W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.11989
“Moreover, some of the leading agent architectures provide this continuity through interleaved reasoning-and-acting traces (Yao et al., 2023), external verbal memory and self-reflection (Shinn et al., 2023), or engineered memory streams with periodic reflection (Park et al., 2023).”
AI-Research Agents in the Wild. From GitHub and arXiv to Regularities and GapsPreprint
Komissarov, A., & Ustyuzhanin, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.11975
SEBGen: A Self-Evolving Benchmark Generation Framework for Software Engineering Agents
Huang, J., Feng, Z., Xiong, D., Ren, C., & Geng, Y. (2026) · Electronics · doi:10.3390/electronics15204663
ECAS: An Edge-Controlled Agentic System for Validation-Gated Scientific Application ExecutionPreprint
Sun, B., Xia, M., & Zheng, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.14211
“A line of work lets agents improve by accumulating reusable experience: Voyager maintains a growing skill library of executable programs [9], ExpeL extracts natural-language insights from past trials [10], and Reflexion converts verbal feedback into self-improvement signals [11].”
Grynets, O., Kaskun, O., Seletska, A., Tukalo, D., & Lyashkevych, V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.14413 · Cited by 1
“LLMCompiler targets parallel function calling [22], while Reflexion uses feedback and episodic memory for iterative improvement [23].”
AI Analysts and the Changing BI Labor Market: A Review of Database Agents and Analyst WorkflowsPreprint
Rao, I., Soriano, B., George, K., Lim, M., Matias, A., Kapoor, D., Thomas, N., Mendoza, S., Menon, R., Manalo, T., et al. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7433959
CounterPersona: Append-Only Defense Against Unauthorized Persona Skill DistillationPreprint
Wang, P., Wang, Z., Cao, H., Zhao, Q., Li, H., & Xu, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.15097
“Such skill artifacts can be updated independently of model weights, allowing agents to accumulate and reuse experience without repeated model fine-tuning [36, 39].”
Emadi, S. M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.14500
“ReAct interleaves reasoning and action, Toolformer learns tool-use opportunities, and Reflexion incorporates feedback through verbal memory (Yao et al., 2023b; Schick et al., 2023; Shinn et al., 2023).”
Vibe Patenting: Evaluating LLM Judges for Professional Patent-Drafting AgentsPreprint
Koike–Akino, T., Blaykhman, V., Wang, Y., Liu, J., & Vinokur, G. V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.13422
“Self-Refine [19] repeatedly generates feedback on an LLM’s own output and uses that feedback for revision, while Reflexion [23] uses linguistic feedback and reflective memory to improve subsequent agent behavior.”
LLM-Enhanced Experience Replay for Reinforcement Learning-based Path Planning
Long, H., Zhou, J., & Wang, J. (2026) · International Conference on Machine Learning and Neural Networks · doi:10.1145/3821725.3821812
“Currently, research on the fusion of Large Language Models and Reinforcement Learning mainly focuses on three aspects: (1) Using LLM as a state encoder to transform unstructured environmental information into vector representations suitable for RL processing [7].”
Liang, Y., Gan, C., Ying, R., Wei, H., Cui, Z., & Ni, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.12012
“AlphaCodium organizes code generation into a flow, and Reflexion carries verbal feedback across attempts [59]–[61].”
Gaikwad, M. (n.d.) · PhilPapers (PhilPapers Foundation)
Memory-Skill Isomorphism: One Skill Carrier, Two Native UsesPreprint
Ruiyuan, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.16669
The Immutable Past: Formalizing State Mutability and Conflict Resolution in Mutable RAGPreprint
HaddadPajouh, H., & AmiriTabat, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.16073
Model-Driven Autonomous Voice Agents for Regulated Healthcare Operations
Kalicheti, B. (2026) · International Journal of Artificial Intelligence and Machine Learning · doi:10.51483/ijaiml.6.10s.2026.449-457
Pinheiro, J. H. N. C. D. M. (2026) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
LLM-Driven NPCs for Realistic Behavior and Collaboration in Games
Liu, S. (2026) · ResearchWorks at the University of Washington (University of Washington)
Wang, R. (2026) · International Conference on Image, Algorithms, and Artificial Intelligence (ICIAAI) · doi:10.4108/eai.22-5-2026.2365227
“The generated output of the model can be checked and evaluated at certain nodes, and this reflection mechanism can help in quick detection of possible errors and initiating corrective or re-reasoning feedback mechanisms [3, 4, 5].”
Wu, X., Ying, Z., Wu, H., Zheng, X., Cheng, H., Shi, X., & Guo, J. (2026) · arXiv (Cornell University)
LearnActCoder: Role-Aware Error Memory for Adaptive Clinical Coding AgentsPreprint
Ghaffari, M., Sen, B., Sabetpour, N., Fatehi, N., Agarwal, A., & Morato, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.19721
“Reflexion (Shinn et al., 2023) and Self-Refine (Madaan et al., 2023) use language feedback to revise reasoning, while ExpeL (Zhao et al., 2023) accumulates experience across tasks.”
LLM-as-an-Improver: Turning Verification into Better CandidatesPreprint
Tomihari, A., & Ichikawa, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.19515
“Self-Refine and Reflexion use natural-language feed-back to improve model outputs iteratively (Madaan et al., 2023; Shinn et al., 2023).”
Scaffolding students-AI dialogue for safe educational interactions
Muss, O., Leisten, L. M., & Bardyn, C. E. (2026) · Scientific Reports · doi:10.1038/s41598-026-69820-9
Проблематика консистентності великих мовних моделей при навчанні з підкріпленням
Кулик, Л. Р., Мокін, О. Б., & Kulyk, L. (2025) · Repository of Vinnytsia National Technical University (Vinnytsia National Technical University)
LEGIT: Credentialing Protocol for Trustworthy AI Agent MarketplacesPreprint
Drew, S., & Zhou, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.21325
“Reflexion stores feedback as verbal reflections for later attempts [34].”
Vedurupaka, N. K. (2026) · International Journal of Computer Information Systems and Industrial Management Applications · doi:10.70917/ijcisim-2026-5709
Babu, M. A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.23512
“Reflexion explicitly uses verbal reinforcement and reflective retry [3]; AgentBetta used substantially fewer resources on the paired scoreable tasks, but the success difference was unresolved and the repetition coverage was incomplete.”
StepKV: Step-Aware KV Cache Compression for LLM AgentsPreprint
Feng, B., Liu, J., Li, Y., Yu, W., Qiu, Z., Sun, Y., Shen, M., Li, X., Dai, Q., & King, I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.22158
“…frameworks such as ReAct interleave reasoning and tool use through repeated Thought, Action, and Observation steps [37], while related systems leverage external tools, web interaction, reflection, or self-improvement mechanisms to enhance task completion over multiple interactions [19, 26, 27].”
LEVERAGING LARGE LANGUAGE MODELS TO ACCELERATE SYSTEMATIC REVIEWS IN ENVIRONMENTAL HEALTHPreprint
Khider, D., & Tanghal, R. B. (2026) · doi:10.22541/essoar.15009303/v1
Spectra: A Rules-Driven LLM Pipeline for Automated KYC Document ProcessingPreprint
Wahib, M., Tran, E., Mourad, R., Muti, M., & Dvornik, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.25474
“This approach builds on recent work demonstrating that LLMs perform more reliably when reasoning and acting are interleaved in structured loops [1], when models can invoke external tools to ground their outputs [2], and when verbal feedback enables iterative refinement [3].”
Reinforcement Learning in Action: From Foundations to Frontier AI
Kamath, U., & Vajre, V. (2026) · CRC Press eBooks · doi:10.1201/9781003668268
Offline Routing for Health Language Systems with Clarification and Escalation
Cao, Z., & Gao, J. (2025) · Journal of Technology Informatics and Engineering · doi:10.51903/jtie.v4i2.591
Pirouzan, A. H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.26830
AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building modelsPreprint
Pantoja-Rosero, B. G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.28061 · Cited by 2
“In an agentic setting, these capabilities can support dynamic task decomposition, execution of intermediate operations, and iterative evaluation and refinement of their outputs [19].”
Fu, H., Chen, K., Jia, X., Wang, J., Yin, Y., & 胡玉彪 (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.27417
“Reflexion translates task feedback into verbal reflections for later action (Shinn et al., 2023).”
Reachable Global Optimization in AI Systems: How Global Is Global?Preprint
Shu, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.27855
“Verifier-guided reasoning and LLM tool use make the issue sharper: systems often improve by generating candidates, evaluating them, and revising through feedback, not by enumerating a formal space (Yao et al., 2023; Shinn et al., 2023; Chen et al., 2021).”
Embodied Agentic Harnesses for Physical Intelligence A SurveyPreprint
Huang, Y., Chen, Y., Yang, Z., & Ding, G. (2026) · Engineering Archive (engrXiv) · doi:10.31224/8297 · Cited by 1
HEXIS: Compiling Agent Skills into Extended Finite State MachinesPreprint
Minghao, L. I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.30123
“Reflexion (Shinn et al., 2023) supplies feedback from earlier attempts, while Agent Workflow Memory (Wang et al., 2025) provides work-flow descriptions extracted from experience.”
SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World RepositoriesPreprint
Wu, J., Sun, L., Tan, Z., Liu, Y., Zhao, Z., Zhang, D., Li, S., Quan, S., Wu, Y., Quan, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.29465 · Cited by 1
“Empirical studies of flaky tests show that instability in the test infrastructure can itself contaminate engineering conclusions [42].”
REAT: A Reflective Experience-Augmented Tutoring Framework for Multi-turn Mathematical Instruction
Zhou, J., Zhang, C., 魏兴军, Zhou, X., Fortino, G., Fan, X., Wang, Y., Wen, Q., & Li, H. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.605
Khudhair, K. M., & Khudhair, B. M. (2026) · Algorithms · doi:10.3390/a19090805
Towards Adaptive Reverse Thinking in Large Language Models: Quantifying and Optimizing
Fang, A., & 朝, 乐. (2026) · Data Intelligence · doi:10.3724/2096-7004.di.2026.2092
Memory as Middleware for Self-Improving AI AgentsPreprint
Jayaram, K. R., Isahagian, V., Muthusamy, V., Thomas, G., Oum, P., Fang, G., & Aravindan, A. V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32091
“A large body of work makes agentsimprovefromtheirownexecutionwithoutweightupdates:re-flectiononfailures[41],simulatedexperienceandepisodicrecall[38],…”
LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual PuzzlesPreprint
Zhang, B., Lu, H., Li, Z., Li, G., Zhang, A. Y., & Wang, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34769 · Cited by 2
“Studies of recovery examine verbal feedback across trials (Shinn et al., 2023), intrinsic self-correction (Huang et al., 2024), and policy-induced GUI errors (Bu et al., 2026), but not failures that must be inferred from accumulated visual consequences.”
Evo2Team: When Do Evolved Skills Transfer? From Selection to DeploymentPreprint
Wang, R., & Cui, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34135
“Reflexion stores verbal feedback, while Voyager and Agent Workflow Memory retain routines that can be used again [Shinn et al., 2023, Wang et al., 2023, 2025b].”
Direct Self-Evolving Optimization: Evolving LLMs without Challenger TrainingPreprint
Deng, Y., Wang, Y., & Wang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34279
Parashar, J., Douglass, E., Bastian, W. C., & Bhandarkar, S. M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33822 · Cited by 2
“This design builds on trace-driven harness revision (Lee et al., 2026; Lin et al., 2026; Chen et al., 2026a), reflective lessons (Shinn et al., 2023; Zhao et al., 2024), failure attribution and taxonomy-guided search (Cemri et al., 2025; Zhang et al., 2025c; Cemri et al., 2026), and reflective…”
ARISE: Adapting to Evolving Capability Gaps in Agentic Reinforcement LearningPreprint
Feng, K., Fang, Y., Tan, Y., Gu, S., Zhao, Y., Liu, Y., Lu, X., Ma, L., & Ren, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.35532
“These long-horizon tasks require the coordination of capabilities such as planning, execution, and verification (Yao et al., 2023; Shinn et al., 2023).”
Learning Native Reflection in Unified Models with Interleaved Reinforcement LearningPreprint
Fan, Y., Huang, Z., Cai, Z., Li, Y., Wen, Z., Yin, W., Diao, H., & Liu, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.35767
“Self-Refine and Reflexion let a language model critique and revise its own draft by prompting alone [21, 29], yet without external feedback such intrinsic self-correction can lower accuracy [14].”
SGG-ReflAct: Sub-Goal Guided ReflAct with Structured Planning for Reliable Long-Horizon ReasoningPreprint
Jung, J., & Jung, S. H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34548
“The third employs reflection on top of a reasoning backbone, revisiting failed episodes to inform subsequent attempts (Shinn et al., 2023).”
InfluentialFrom Migration to Calibration: Preserving Agent Capabilities across Models, Jurisdictions, and ScalePreprint
Liu, Y.-X., Liu, P., Liu, Y.-W., Guan, Y.-H., & Song, J.-B. (2026) · arXiv.org · doi:10.48550/arxiv.2609.35149
Agentsensus: Consensus-Compressed Shared Memory for Multi-Agent Story WorldsPreprint
Pan, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32297
“Variants change what is stored or how it is maintained rather than who owns it: MemGPT (Packer et al., 2023) pages context like an operating system, MemoryBank (Zhong et al., 2024) decays it on a forgetting curve, Reflexion (Shinn et al., 2023) persists verbal self-feedback, and A-MEM (Xu et al., 2025) links an agent’…”
Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement LearningPreprint
Ren, Y., Zhu, H., Yang, L., Xu, S., Lai, P., & Zhang, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.35082
Afoud, A., & Wu, J. J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.31937
Chen, W. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7531901
Liability for rogue behaviors of agentic AI
Papyshev, G., Chan, K. J. D., & Migliorini, S. (2026) · Telecommunications Policy · doi:10.1016/j.telpol.2026.103339
LoRo-Mark:Provably Lossless and Robust Agent WatermarkingPreprint
Zou, H., Wang, Y., Yao, J., Zhang, W., & Fang, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34080
“…how an agent plans and executes complex multi-step tasks (Xi et al., 2025; Yao et al., 2023b; Schick et al., 2023; Park et al., 2023; Liu et al., 2024a; Shinn et al., 2023; Yao et al., 2023a; Wei et al., 2022; Nakano et al., 2021; Qin et al., 2023; Patil et al., 2023; Karpas et al., 2022).”
Self-Designed Evaluators and Warm Memory for Long-Horizon AgentsPreprint
Asgari, S., Kıcıman, E., Nunes, L. D. O., & Chandra, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33717
“Reflexion [57] stores verbal self-reflections between trials; ExpeL [88] extracts insights from successful and failed trajectories; Voyager [67] and Agent Workflow Memory [71] accumulate reusable skills and workflows; generative agents [53], MemGPT [50], MemoryBank [91], A-MEM [74] and Mem0 [13]…”
CORTEX: A Verified Experience Layer for Generalist AgentsPreprint
Keerthana, G., & Gupta, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33260
“Today’s systems can reason with tools, coordinate roles, retrieve external knowledge, and reflect on mistakes (Schick et al., 2023; Yao et al., 2022, 2023; Wu et al., 2023; Li et al., 2023; Hong et al., 2023; Shinn et al., 2023; Madaan et al., 2023).”
Planner-as-Router: Joint Plan-Time Model Routing for Cost-Efficient Multi-Agent WorkflowsPreprint
Singh, V. K., Priyam, P., & Bhowmick, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32917
Shockingly Simple Self-retrospection Improves Agentic Models Without RLPreprint
Light, J., Cui, C., Kim, J., Castanyer, R. C., Penaloza, E., Shi, Z., Sordoni, A., Côté, M.-A., Yuan, X., & Kim, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.35741
After the Fix: Transfer of Corrected Agent ExperiencePreprint
Zhang, Y., & Lin, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34603
“Self-Refine and Reflexion improve attempts through feedback (Madaan et al., 2023; Shinn et al., 2023); human–AI collaboration distinguishes correction, remembering, and adaptation (Amershi et al., 2019).”
Can Generative AI Automate Data Extraction for Meta-Analysis? A Case Study on Intercropping Research
Lu, Z., Xiong, X., van der Werf, W., van der Plas, T. L., & Athanasiadis, I. N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.35089
“Reflexion showed a similar problem: even when previous error patterns were provided, the system did not reliably avoid them.”
From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace DiagnosisPreprint
Yang, S.-X., Wang, Y., Feng, Z., Wen, B., Gui, J., Yang, D., Yu, W., Zhang, H., Tang, J., & Wang, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32514
“…LLM frameworks, including ReAct-style tool use, self-refinement, reflection, and multi-agent collaboration (Yao et al., 2023; Madaan et al., 2023; Shinn et al., 2023), but specializes them for structured causal trace diagnosis, producing visualizable CEGs with repair-value estimates for…”
Park, C., Yoon, J., Han, S., Moon, H., & Lim, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34428
Same Winners, Different Success Rates: Evaluating How LLM Agents Recover from FailuresPreprint
Xu, D., Yang, Z., Wu, J., Zhang, S., Zhu, Z., Li, J., Zhang, J., & Ji, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34215
“ReAct (Yao et al., 2023), Reflexion (Shinn et al., 2023), and AutoGen (Wu et al., 2023) provide agent architectures for reasoning, self-improvement, and multi-agent coordination.”
Liu, P., Herath, U., & Petersen, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32868
“Broader systems such as The AI Scientist and its successor target research generation [19, 20], while ReAct and Reflexion provide foundations for tool interaction and feedback memory [21, 22].”
Environmental Impact of Generative and Agentic AI: An in-Depth Analysis and Green SolutionsPreprint
Bahi, A., Ourici, A., & Gasmi, I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.32960
“…answers each request with a single model invocation, agentic systems engage in iterative reasoning-action loops, with ReAct [53], Reflexion [54], and Tree of Thoughts [55] as archetypal patterns, in which the model calls tools, retrieves information, evaluates intermediate results, and…”
TRACE: Governing Memory Validity in Evolving Multi-Agent SystemsPreprint
Xiong, W., Zhang, S., Gu, S., Tang, B., Li, Z., Xiong, F., Wen, Y., & Wen, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33517
“Persistent memory and retrieval systems are designed to reuse historical information (Park et al., 2023; Shinn et al., 2023; Lewis et al., 2020; Packer et al., 2023; Chhikara et al., 2025; Ma-harana et al., 2024; Wu et al., 2025).”
Opera: A Verbal Critic Framework for Long-horizon Coding AgentsPreprint
Mei, K., Hu, Z., Dai, Y., Tan, J., Zhang, Y., Song, D., Metaxas, D. N., Savarese, S., Xu, R., & Chen, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33987
“Prior work provides strong foundations for feedback and reflection, including Self-Refine (Madaan et al., 2023), Reflexion (Shinn et al., 2023), and CRITIC (Gou et al., 2023), and for coding agents, SWE-PRM (Gandhi et al., 2025), SWE-Search (Antoniades et al., 2024), and Agentic Rubrics…”
InfluentialThinking Before Thinking: Scaling Agentic Inference Through Meta-ReasoningPreprint
Dahal, P., Bakhtin, A., Cohen, T., Chen, Z., Wu, C.-J., Fergus, R., Yih, S., Synnaeve, G., Salakhutdinov, R., Arora, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.38147
“Existing agents often interleave these decisions with the object-level work, taking each control decision in a single step conditioned on their accumulating history (Yao et al., 2023b; Shinn et al., 2023; Shen et al., 2023; Packer et al., 2023; Zhang et al., 2025a).”
Neurosymbolic Routing for Reliable Reasoning on Resource-Constrained Edge DevicesPreprint
Sadhu, A., Velasquez, A., & Chen, L. (2026) · arXiv.org · doi:10.48550/arxiv.2609.35833
“Reflexion [18] adds verbal self-reflection across multiple attempts, and CodeAct [19] frames actions as executable code, which places it close to Program-of-Thought.”
Query Normalization and Error-Aware Feedback for Aeronautical Information Manual Question Answering
Je, J., & Kim, B. W. (2026) · Journal of information and communication convergence engineering · doi:10.56977/jicce.2026.24.3.219
A Proactive, Interoperable, and Multi-Agent Virtual Assistant System
Karandikar, S. P., Merchant, K. A., & Suryawanshi, A. L. (2026) · International Conference on Software and Computer Applications (ICSCA) · doi:10.1145/3801619.3801624
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.