Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 2,401–2,500 of 3,537
Effective Context Management for LLM Agents via Hybrid Chat History Optimization
Songa, S., Bayyavarapu, S. R. K. V., Gulkotwar, N., Pasupuleti, V., Allala, S. R., & Tyagi, S. (2026) · IEEE World Conference on Applied Intelligence and Computing (AIC) · doi:10.1109/aic70896.2026.11717333
Agents for Rule-Intensive Decision Making in Virtual Power Plant Operations
Pengfei, Y., Lin, W., & Yang, Z. (2026) · Power System and Green Energy Conference (PSGEC) · doi:10.1109/psgec70194.2026.11719196
“LLM-based agents have shown potential in text understanding, retrieval-augmented generation, reasoning, tool use, multi-agent collaboration, and human-machine interaction [9]–[15].”
Comparing large language models and prompting techniques to control an embodied persuasive coach
Corrò, C., & Chittaro, L. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2742725
Cognitive evolution enables de novo design of untethered millimeter-scale robotsPreprint
Li, H., Zhao, W., Tao, Z., Li, P., Wang, X., Wang, Q., Wang, B., Zhai, Y., Wei, W., Wang, S., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-10770716/v1
Evaluating Low-Shot Procedural Skill Transfer in Language-Agent Debugging
Sun, J. (2026) · International Workshop on Agentic AI for Next-Generation Software Development · doi:10.1145/3843282.3844421
“Language-agent memory ranges from verbal reflection to distilled guidelines, workflows, and procedural repositories [2, 5, 6, 8].”
From model trust to software topology: a Perspective on structurally governed agentic AI systems
Morin, M., & Sufi, F. K. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1919950
Lops, A., Ferrara, A., Narducci, F., Ragone, A., & Trizio, M. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834507
Constrained Automated Requirements Formalization for Rigorous Analysis with Alloy
Koscinski, V., & Mirakhorli, M. T. (2026) · doi:10.1145/3843777.3844526
Optimization-Free Repair of Code LLMs: A Closed-Form Fisher-Guided Bayesian Approach
Duan, Z., Bian, C., Zhang, Y., & Yang, S. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834442
Sanjay, K., Sudharsan, A., & Dantuluri, V. (2026) · International Journal Of Engineering And Computer Science · doi:10.18535/ijecs.v15i10.5761
Xu, J., Wu, Z., Peng, Y., Chong, C. Y., Meng, L., Liu, K., & Jia, X. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834522
Li, L. (2026) · Research Square · doi:10.21203/rs.3.rs-11095379/v1
SE-Blackboard: A Shared-State Architecture for Multi-Agent Software Engineering Pipelines
Liu, E., Zhu, Q., & Dong, X. (2026) · IEEE Access · doi:10.1109/access.2026.3695435
Plan for Constructing DataDiscoveryLab: Creating DataBases for Well-Rounded SearchesPreprint
Keskinoglu, E. (2023) · Preprints.org · doi:10.20944/preprints202304.1074.v2
MENTAL SIMULATION SELF-REFLECTION TASKS FOR INCREASED LEARNING OF ENGLISH
Tarasiuk, N. (2023) · Polonia University Scientific Journal · doi:10.23856/5918
Lu, M., Gao, F., Tang, X., & Chen, L. (2023) · ChemRxiv · doi:10.26434/chemrxiv-2023-gxmxr
Research with Subsymbolic AI: Preliminary Theses
Sudmann, A., Echterhölter, A., Ramsauer, M., Retkowski, F., & Schröter, J. (2023) · AI critique · doi:10.1515/9783839467664-003
On Sample-Efficient Code Generation
Han, H., Kim, Y. J., Kim, B., Lee, Y., Lee, K., Lee, K. C., Lee, M., Bae, K., & Hwang, S.-W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.73 · Cited by 1
Wit and Wisdom: Using Computational Humour to Communicate about EconomicsPreprint
Postavaru, I., Bunea, E., Pungulescu, C., & Stolin, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4818220
Understanding How ChatGPT Memorizes Chinese Tang Poetry
Chang, C.-Y. (2024) · IEEE International Conference on Big Data Security on Cloud, IEEE International Conference on High Performance and Smart Computing, and IEEE International Conference on Intelligent Data and Security (BigDataSecurity/HPSC/IDS) · doi:10.1109/ids62739.2024.00008
“I also notice that using sophisticated prompts can significantly improve the ability of the LLMs as pointed in [13].”
BackgroundStudying the Knowledge and Behavior of ChatGPT via Journey to the West
Zhou, Z. (2024) · IEEE International Conference on Big Data Security on Cloud, IEEE International Conference on High Performance and Smart Computing, and IEEE International Conference on Intelligent Data and Security (BigDataSecurity/HPSC/IDS) · doi:10.1109/hpsc62738.2024.00012
“Another different approach is to use strategies of reflextion [14].”
MethodPersonal Manifold: Management of Personal Data in the Age of Large Language Models
Halevy, A. Y., Li, Y., & Tan, W.-C. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00440 · Cited by 4
“Since the result obtained from the LLM may come with constraint violations, the planner can continue with a self-reflection cycle [19] taking response from the symbolic planner (i.e., the “checker”) as feedback to further refine the plan (Step 3).”
Method(2024) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets60636
Wit and wisdom: using computational humor to communicate about economics
Postavaru, I., Bunea, E., Pungulescu, C., & Stolin, D. (2024) · Prometheus · doi:10.13169/prometheus.40.1.0020
“To mitigate potential errors while preserving the autonomy of the process, we draw on the methodological contributions of Shinn et al. (2023) and Nair et al. (2023) and add an iterative ‘self-reflection’ feature to the pipeline: the model will self-evaluate and improve through several iterations.”
MethodLLM-AS: A Self-Improve LLM Reasoning Framework Integrated with A* Heuristics Algorithm
Meng, X., Niu, K., & Chen, X. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia241464 · Cited by 1
Kh.Kh., N. (2024) · Emergent Journal of Educational Discoveries and Lifelong Learning (EJEDL) · doi:10.47134/emergent.v3i4.53
Collaborative intelligence in AI: Evaluating the performance of a council of AIs on the USMLEPreprint
Shaikh, Y., Asiya, Z., Jeelani, M. M., et al. (2025) · medRxiv · doi:10.1101/2025.02.17.25322388
BERT4Anno: An annotation misuse detection method for Java
Yang, J., Ji, X., Wu, W., Liao, X., Zhang, K., Dong, L., Xiang, N., & Ren, J. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107763
Decoding Greenwashing: Llm Insights into Corporate NarrativePreprint
Shimamura, T., Tanaka, Y., & Managi, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5253231
MODELOS DE RACIOCÍNIO E ARQUITETURA EXTENSÍVEL PARA PESQUISA INTELIGENTE
Júnior, H. C. P. (2025) · Congresso Internacional IDEA: Inovação, Diálogo e Experiências na Aprendizagem (IDEA) · doi:10.29327/9786527213314.1086647
Madrid-García, A., & Barbancho, B. M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5256592
Fan, Z., Zhang, B., He, M. X., Teng, Z., & Fan, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5269808
Number of Players, Zero: Using Multi-Agent Collaboration to Evaluate Unsolvable Collaborative Items
Wang, Y., & Bergner, Y. (2025) · Computer-supported collaborative learning/The Computer-Supported Collaborative Learning Conference · doi:10.22318/cscl2025.325936
Yin, R., & Wu, Y. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251352156
LLM-Assisted Plan Execution for Robots in Dynamic Environments
Peña‐Narvaez, J. D., Manzanares, J. C., Moreno, F. M., Pérez-Rodríguez, R., & Martín, F. (2026) · Robotics · doi:10.3390/robotics15040080
Enhancing Flight Deck Decision Support with Distributed GenAI: A Multi-Agent Approach
Prado, J. P. D. A., Zammit-Mangion, D., Zammit, B., Gauci, J., Muscat, A., Mizzi, S., & Manicolo, A. (2025) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero63441.2025.11068478 · Cited by 1
“The structure proposed by Shinn et al. [49] allows the agent to reflect on episodic memory to generate new semantic inferences.”
Zhijie, Z., Deng, Y., Qi, H., & Wang, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5351223
Self-Improvement: Large Language Models Can Self-Improve Generation with External Tools
Liu, D. Y., Ma, C., & Feng, Y. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9994-0_2
Civitas: A Reflexive Cognitive Architecture for Ethically Governed Causal InferencePreprint
Mazzocchetti, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5362693
Large Language Models for Social ProductivityPreprint
Li, E. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5365332
Reasoning Distillation and Structural Alignment for Improved Code GenerationPreprint
Jalilifard, A., Raimundo, M. M., & Rocha, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5403116 · Cited by 1
Sharma, A., Kumar, P., & Dhamija, S. (2025) · Wiley eBooks · doi:10.1002/9781394287499.ch10
Machine Reasoning Framework for Large Language Models
Georgiev, K., GEGOV, A. E., Khaleghzadeh, H., Adda, M., & Ichtev, A. (2025) · 2025 International Conference Automatics, Robotics and Artificial Intelligence (ICARAI) · doi:10.1109/icarai67046.2025.11137913
“Recent AI research has begun to explore self-reflective language models capable of recursively verifying and refining their outputs [45] in addition to what we have covered in this paper.”
Chatbots integrated in the metaverse: a conceptual model based on interaction ritual chain theory
Yan, Y., & Fan, M. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.3235 · Cited by 1
Rouam, A., & Buvet, P.-A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-07989-3_9
KI-unterstützter Filmschnitt – Chancen und Herausforderungen
Döbert, A. (2025) · Springer eBooks · doi:10.1007/978-3-658-47898-8_1-1
Evaluating Agentic AI Through a Geopolitical Simulation Sandbox
Dessureault, J.-S., Manzi, A. T. I., Ismaili, S. A., Barisoa, A. M., Djioke, D. Y., Bergeron, A., Yared-Amare, A., Lalancette, M., & Bélanger, É. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-15632-7_11 · Cited by 2
Mokssit, Y., Ravi, K., Nie, M., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-8746188/v1
Chintalapati, J., Akhil Rajeev, P., Choudhury, B., Bedare, M., & Kumar, Y. (2025) · Journal of the Indian Institute of Science · doi:10.1007/s41745-026-00501-2
“Such architectures also allow for meta learning frameworks where AI agents can iteratively and recursively learn and adapt to their new environments 37 .”
A survey of large language models: techniques, applications, and challenges
Ji, J., Zhao, X., & Shi, X. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02288-9
“…into executable sub-tasks [ 99 ], hierarchical memory systems maintaining both short-term context and long-term experi - ence [ 100 ], tool-use interfaces enabling API calls and envi - ronment interaction [ 101 ], and reflection mechanisms that evaluate and verify intermediate outcomes [ 102 ].”
Jo, S., Kim, J., & Lee, S. (2026) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2026.24.4.11
Ichimura, E., Khan, M. F. F., & Sakamura, K. (2026) · Studies in computational intelligence · doi:10.1007/978-3-032-16157-4_1
Kátai, Z., & Iclănzan, D. (2026) · Acta Universitatis Sapientiae Informatica · doi:10.1007/s44427-026-00033-8
“…ben-efit from explicit uncertainty estimates for EAII, sensitivity analyses with respect to the median and peak, alternative normalizations when sample sizes permit, controlled language-channel experiments, and ablations on feedback limits to disentangle iteration effects from core capability [22].”
InfluentialMartins, A., Lapão, L. V., Nunes, I. L., & Londral, A. R. M. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-28939-1_11
LLM-Driven Diagnostic Reasoning for Distributed Hydrological Model CalibrationPreprint
Zhao, Y., Li, Z., Zhu, L., Jin, J., & Zhang, J. B. (2026) · Research Square · doi:10.21203/rs.3.rs-10106384/v1
Liao, C., Chou, T.-Y., Liu, T.-W., Wu, G.-Y., & Lai, I. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-29788-4_73
Kashiwada, Y., Sakurai, R., Yokokawa, Y., Ando, K., Gocho, C., Murata, C., Tohda, K., Tanioka, M., Sonobe, S., Harada, T., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-10026525/v1
Feng, Y., Yang, S., & Xing, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3441-7_50
Son, B., Jung, M., & Lee, S. (2026) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2026.24.7.81
Agents in the Wild: Where Research Meets Deployment
Yang, G. H., Venkit, P. N., Sedghamiz, H., Santus, E., Dibia, V. C., & Baldini, I. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3816449
EvoEdge: A Traceable Agentic Workflow for IoT and Edge Security Review
Wang, X., Liao, L., Wang, F., & Ye, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36493-7_7
Vien, P. T. (2026) · International Conference on Intelligent Information Technology (ICIIT) · doi:10.1145/3805862.3805918
Ferrán, C. (2026) · AI and Ethics · doi:10.1007/s43681-026-01320-y
“The Reflexion and Self-Refine frameworks demonstrate that this generative capacity is not static; AI outputs are responsive to structured critique, distinguishing the technology from a conventional information retrieval system [ 17, 46, 70 ].”
Cost-Quality Trade-Offs in Reasoning Architectures for Agentic Software Engineering Deliberation
Harbar, Y., & Chudziak, J. A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36590-3_16
Jirapatpongthada, C., Sawang, S., Klinhomutis, T., & Siriborvornratanakul, T. (2026) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-026-05114-2
“Self-Refine demonstrates iterative genera - tion, feedback, and revision using self-feedback (Madaan et al. 2023 ), while Reflexion uses verbal feedback to guide later trials in language-agent settings (Shinn et al. 2023).”
Qian, X., Xu, Y., Ma, C., & Chan, C. K. (2026) · Research Square · doi:10.21203/rs.3.rs-10522439/v1
Rahmani, A. S. E. I., Yahiaoui, Y., & Bouziane, A. (2026) · SN Computer Science · doi:10.1007/s42979-026-05333-6
“…of prompting-based self-improvement mechanisms that did not require parameter updates, including verbal reinforcement through self-reflection [ 3], iterative self-feedback [4], and interleaved reasoning-action loops [5], alongside broader surveys charting the rapid capability growth of the…”
Synth: Code as the Control Plane for Long-Horizon Agents
Gandhi, S., Jain, S., Agarwal, G., Singh, T., & Prajwala, T. R. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-35078-7_18
Liu, Z. (2026) · International Conference on Digital Society and Artificial Intelligence (DSAI) · doi:10.1145/3833541.3833805
“Shinn et al. proposed Reflexion, which enables agents to revise previous behavior through language feedback [16].”
Qu, X., Wang, H., Jiang, Z., Zhu, J., & Li, H. (2026) · International Conference on Software Engineering and Computer Applications · doi:10.1145/3829641.3829689
“Reflexion [3] even uses verbal reinforcement to help models learn from past mistakes.”
AI Employees: A Role Programming Framework for Supervising Autonomous Systems
Zheng, S. (2026) · International Conference on Automation Control, Algorithm and Intelligent Bionics (ACAIB) · doi:10.1145/3828836.3828919
“Extending the framework to support multi-agent role coordination would significantly increase applicability [13, 16, 17, 21].”
Focus on Evidence: Relational Structure Enhances LLM Effectiveness in TableQA
Yang, Z., Du, Z., Zhang, M., Du, W., Chen, J., Qian, F., & Zhao, S. G. (2026) · ACM Transactions on Information Systems · doi:10.1145/3848125
Evaluating Symbolic Feedback Granularity for LLM-Based Program Repair
Guo, T. (2026) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1145/3843776.3844675
“This verifier-in-the-loop paradigm is closely related to recent work on self-correction, tool-assisted critique, and iterative program repair [5, 7, 13, 18].”
Kerbel, P., Kerbel, M., Abramov, V., & Abramov, L. (2026) · International Workshop on Agentic AI for Next-Generation Software Development · doi:10.1145/3843282.3844425
“A rich research line gives agents persistent memory: Reflexion [12] stores verbal self-reflections on failures and improves within a task series; Voyager [13] accumulates an ever-growing library of validated skills; MemGPT [10] pages long-term memory in and out of the context window; generative…”
Enhancing Robotic Simulations in Webots Using AI-Driven Decision Making
Gorbounov, Y., Peychinov, P., Ivanov, N., & Astakova, D. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-33123-6_25
RepoTune: Co-evolving Vulnerability Detection with a Repository’s Genome
Sun, Y., Yang, R., & Li, Y. (2026) · doi:10.1145/3844178.3844643
Multi-agent Collaboration with State Management
Liu, M., Chen, T., Xu, Z., Jiang, X., & Dong, Y. (2026) · arXiv
“Reflexion and Generative Agents use reflection and episodic memory to improve later decisions (Shinn et al., 2023; Park et al., 2023).”
VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation
Miculicich, L., Parmar, M., Palangi, H., Dvijotham, K. D., Montanari, M., Pfister, T., & Le, L. T. (2025) · arXiv · doi:10.48550/arxiv.2510.05156 · Cited by 2
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
Li, Y., Lin, Z., Deng, A., Zhang, X., He, Y., Ji, S., Cao, T., & Hooi, B. (2026) · arXiv · doi:10.48550/arxiv.2601.18510
“…and Generative Agents (Packer et al., 2023; Park et al., 2023) utilize hierarchical memory to store historical interactions, while frameworks like Voyager and Reflexion (Wang et al., 2023; Shinn et al., 2023) retrieve textual descriptions of past skills or failures to improve future performance.”
Influential"Theater of Mind" for LLMs: A Cognitive Architecture Based on Global Workspace Theory
Shang, W. (2026) · arXiv · doi:10.48550/arxiv.2604.08206
“Furthermore, self-correction mechanisms endowed individual agents with iterative refinement capabilities based on environmental feedback [8].”
Human-Inspired Memory Architecture for LLM Agents
Kerestecioglu, D., Robsky, A., Vasters, C., Sharma, A., & Kesselman, Y. (2026) · arXiv · doi:10.48550/arxiv.2605.08538
“Reflexion Shinn et al. [2023] enables agents to reflect on failures, but stores reflections as unstructured text without lifecycle management.”
Lindenbauer, T., Slinko, I., Felder, L., Bogomolov, E., & Zharov, Y. (2025) · arXiv · doi:10.48550/arxiv.2508.21433
“This is akin to providing reflections within a single rollout, instead of across multiple rollouts [24].”
PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
Woo, S., Kim, H., Shim, S., Jo, M., Jeong, H., Lee, J., Kim, J., Lee, S., Ryu, J., Park, B., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.12029
“Building on these advances, recent systems increasingly explore agentic work-flows, where models interact with tools, intermediate memory states, and external resources to solve complex problems (Yao et al., 2022; Schick et al., 2023; Shinn et al., 2023).”
Weatherhead, S., Salim, F., & Belbasis, A. (2025) · arXiv · doi:10.48550/arxiv.2510.18254 · Cited by 1
“Step-by-step prompting ‘chain-of-thought’) can boost performance[1], and recent work encourages models to ‘reflect’ via self-critiques, reflection loops (routing outputs back into a new attempt), and tool-augmented checks[2–4].”
ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification
Lee, H., Oh, S., Kim, J., Shin, J., & Tack, J. (2025) · arXiv · doi:10.48550/arxiv.2502.14565 · Cited by 26
“Ensuring the reliability of reasoning approaches such as Re-flexion (Shinn et al., 2024) and Self-Refine (Madaan et al., 2023) introduce iterative feedback loops, while verification techniques like step-by-step validation (Lightman et al., 2023) help maintain consistency and reduce errors.”
MAPLE-Guard: Memory-Aware Link Enforcement Against Memory-Link Poisoning in Multi-Agent Systems
Xiong, W., Zhou, Y., Wang, J., Gu, S., Tang, B., Li, Z., Xiong, F., Wen, Y., & Wen, M. (2026) · arXiv · doi:10.48550/arxiv.2608.00426 · Cited by 2
“The memory manager identifies utility with the Q -value, ρ n ( m ) ≡ Q n ( m ) , on the underlying memory substrate [37, 39, 46, 58] and applies a single-step update from the task outcome, with provenance trust updated separately after each task, Agent distrust a 1 − C ( τ agent ) from stored…”
Process Supervision via Verbal Critique Improves Reasoning in Large Language Models
Chen, H.-Y. (2026) · arXiv · doi:10.48550/arxiv.2604.21611 · Cited by 1
“Reflexion Shinn et al. [2023] occupies the outcome-level end of the same axis.”
MemGym: a Long-Horizon Memory Environment for LLM Agents
Xu, W., Wang, Y., Mei, K., Liang, K., Wang, Z., Jin, M., Zhang, H., Zhang, S.-X., Hua, W., Sahu, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.20833 · Cited by 11
Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
Wei, T., Sachdeva, N., Coleman, B., He, Z., Bei, Y., Ning, X., Ai, M., Li, Y., He, J., Chi, E. H., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.20857 · Cited by 1
Continual Learning, Not Training: Online Adaptation For Agents
Jaglan, A., & Barnes, J. (2025) · arXiv · doi:10.48550/arxiv.2511.01093
“Approaches such as Reflexion (Shinn et al., 2023), LATS (Zhou et al., 2024), Voyager (Wang et al., 2023), and MemGPT (Packer et al., 2023) maintain logs of past actions, outcomes, and textual reflections that agents can reference during subsequent tasks.”
AgentSHAP: Interpreting LLM Agent Tool Importance with Monte Carlo Shapley Value Estimation
Horovicz, M. (2025) · arXiv · doi:10.48550/arxiv.2512.12597
“Recent work on autonomous agents [15] shows agents can learn from feedback.”
Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
Chen, M., Wang, L., & Qu, B. (2026) · arXiv
“…behaviors now has a machine counterpart in this corpus: the error notebook is experience and strategy memory (Reflexion’s verbal episodic memory [17], ISM’s self-refined bank of schemas learned from failed episodes [151], experience graphs that persist the branch-fail-repair structure of…”
Self-Harness: Harnesses That Improve Themselves
Zhang, H., Zhang, S., Li, K., Zhang, C., Chen, Y., Zhang, Y., Bai, L., & Hu, S. (2026) · arXiv · doi:10.48550/arxiv.2606.09498
“A growing line of work studies systems that adapt their inputs, memories, contexts, or workflows over time [23, 32, 34, 31].”
MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
Zhou, Z., Qu, A., Wu, Z., Kim, S., Prakash, A., Rus, D., Zhao, J., Low, B. K. H., & Liang, P. P. (2025) · arXiv · doi:10.48550/arxiv.2506.15841 · Cited by 218
“Building on this reasoning-acting prompting paradigm, subsequent works have explored ways to improve agent performance through natural language feedback, enabling iterative refinement [45, 32].”
DAEDALUS: Bootstrapping Agent Memory from Self-Generated Tasks
Edy, A., Conti, M., Xing, V., Allard, M.-A., Benhamdane, N., & Viaud, G. (2026) · arXiv · doi:10.48550/arxiv.2610.08048
“When an attempt fails, the Extractor writes a heuristic from the instruction and the failed trajectory, or revises the heuristic written after an earlier failure, and the Solver retries with it in context (Shinn et al., 2023).”
Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control
Rezazadeh, A., Li, Z., Lou, A., Zhao, Y., Wei, W., & Bao, Y. (2025) · arXiv · doi:10.48550/arxiv.2505.18279 · Cited by 94
“…abstraction for managing contextual memory over time [13]; MemTree employs a hierarchical memory structure for organizing and retrieving information dynamically [14]; and GraphRAG constructs a graph-based memory of entities and relations to enable structured knowledge access [3, 17].”
Accurate and Efficient Long-Term Memory for LLM Agents
Zhao, Z., Guo, X., Lv, L., Wang, M., Li, M., & Li, S. (2026) · arXiv · doi:10.48550/arxiv.2607.16211
“Re-flexion [Shinn et al., 2023] adds verbal reinforcement learning through self-reflective feedback loops.”
Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents
Zhou, Y., Li, Y., Liu, Z. L., Yavuz, S., & Joty, S. (2026) · arXiv · doi:10.48550/arxiv.2609.27334 · Cited by 1
“Systems differ in what they distill: verbal reflections (Shinn et al., 2023), extracted insights (Zhao et al., 2024), memory streams with periodic summarization (Park et al., 2023), executable skills (Wang et al., 2023), induced workflows (Wang et al., 2024b), memory items at multiple granularities…”
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.