Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 301–400 of 3,537
LibreLog: Accurate and Efficient Unsupervised Log Parsing Using Open-Source Large Language Models
Ma, Z., Kim, D. J., & Chen, T.-H. P. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00103 · Cited by 50
“After all logs in the group have been checked, any unparsed logs undergo a self-reflection process [38], which aims to revise the templates and improve parsing results.”
Introspective Tips: Large Language Model for In-Context Decision MakingPreprint
Chen, L., Wang, L., Dong, H., Du, Y., Yan, J., Yang, F., Li, S., Zhao, P., Qin, S., Rajmohan, S., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11598 · Cited by 44
Autoformulation of Mathematical Optimization Models Using LLMsPreprint
Astorga, N., Liu, T., Xiao, Y., & van der Schaar, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.01679 · Cited by 52
Experiential Reinforcement LearningPreprint
Shi, T., Chen, S., Jiang, B., Song, L., Yang, L., & Zhao, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.13949 · Cited by 44
“Meanwhile, inference-time reflection methods demonstrate that LLMs can critique and revise their own outputs to improve success (Zelikman et al., 2022; Madaan et al., 2023; Shinn et al., 2023”
(2023) · doi:10.18653/v1/2023.emnlp-main.364 · Cited by 38
Automatic Model Selection with Large Language Models for Reasoning
Zhao, J., Xie, Y., Kawaguchi, K., He, J., & Xie, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.55 · Cited by 58
Answering Questions by Meta-Reasoning over Multiple Chains of Thought
Yoran, O., Wolfson, T., Bogin, B., Katz, U., Deutch, D., & Berant, J. (2023) · arXiv · doi:10.48550/arxiv.2304.13007 · Cited by 38
“In addition, meta-reasoning is closely tied to self-reflection in LLMs, which is becoming increasingly important in using the LLM to review multiple strategies (Yao et al., 2023; Shinn et al., 2023; Madaan et al., 2023).”
MethodLarge Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence
Nay, J. J., Karamardian, D., Lawsky, S. B., Tao, W., Bhat, M., Jain, R., Lee, A. T., Choi, J. H., & Kasai, J. (2023) · arXiv · doi:10.48550/arxiv.2306.07075 · Cited by 38
Lu, Q., Zhu, L., Xu, X., Xing, Z., Harrer, S., & Whittle, J. (2024) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c63560.2024.00028 · Cited by 54
VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search
Wang, Y., Wang, S., Cheng, Q., Fei, Z., Ding, L., Guo, Q., Tao, D., & Qiu, X. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1053 · Cited by 54
RHO: Your Coding Agent is Secretly a RoboticistPreprint
Elmaaroufi, K., Svegliato, J., Kalade, S., Schelle, G., Seshia, S. A., & Zaharia, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.16458 · Cited by 46
“Verbal self-critique and code refinement loops over agent trajectories preceded this line [80, 81]; recent extensions evolve coding agents themselves [82, 83, 84] or scale deploy-time deliberation through structured inference [85, 86, 87].”
KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents
Zhu, Y., Qiao, S., Ou, Y., Deng, S., Lyu, S., Shen, Y., Liang, L., Gu, J., Chen, H., & Zhang, N. (2024) · arXiv · doi:10.48550/arxiv.2403.03101 · Cited by 38
“…through a variety of strategies including task decomposition (Wei et al., 2022; Yao et al., 2023a; Wang et al., 2023a; Team, 2023), reflection (Shinn et al., 2023; Sun et al., 2023a), collaborative division of labor (Hong et al., 2023; Chen et al., 2023c; Yin et al., 2023; Qiao et al., 2024),…”
InfluentialAn outline of Prognostics and health management Large Model: Concepts, Paradigms, and challenges
Tao, L., Li, S., Liu, H., Huang, Q., Ma, L., Ning, G., Chen, Y., Wu, Y., Li, B., Zhang, W., et al. (2025) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2025.112683 · Cited by 44
PANGeA: Procedural Artificial Narrative Using Generative AI for Turn-Based, Role-Playing Video Games
Buongiorno, S., Klinkert, L., Zhuang, Z., Chawla, T., & Clark, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v20i1.31876 · Cited by 37
Multi-step Iterative Automated Domain Modeling with Large Language Models
Yang, Y., Chen, B., Chen, K., Mussbacher, G., & Varró, D. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3652620.3687807 · Cited by 37
“Similar to the framework of self-refinement [13] and reflexion [18], generating useful reflective self-feedback is challenging since it requires a good understanding of where the model made mistakes as well as the ability to generate a summary containing actionable insights for improvement.”
A brain-inspired agentic architecture to improve planning with LLMs
Webb, T. W., Mondal, S. S., & Momennejad, I. (2025) · Nature Communications · doi:10.1038/s41467-025-63804-5 · Cited by 58
ShapefileGPT: a multi-agent large language model framework for automated shapefile processing
Lin, Q.-M., Hu, R., Li, H.-X., Wu, S.-S., Li, Y., Fang, K., Feng, H.-L., Du, Z.-H., & Xu, L.-C. (2024) · International Journal of Digital Earth · doi:10.1080/17538947.2025.2577884 · Cited by 34
“The framework extracts descriptive and procedural knowledge from complex GEE scripts related to geographic analysis and packages this knowledge into reusable templates, enabling sharing and application in various geographic modeling environments.”
Inventing with Machines: Generative AI and the Evolving Landscape of IS Research
Gopal, R. D., Li, J., Riemer, K., Sarker, S., Singh, P. V., Susarla, A., Bichler, M., & Thatcher, J. B. (2025) · Information Systems Research · doi:10.1287/isre.2025.editorial.v36.n4 · Cited by 58
“Frameworks like ReAct couple reasoning traces with external actions to make decision processes auditable (Yao et al. 2023), while Reflexion adds self-critique loops that enable agents to learn from previous errors (Shinn et al. 2023).”
Reinforcement Learning from Multi-role Debates as Feedback for Bias Mitigation in LLMsPreprint
Cheng, R., Ma, H., Cao, S., Li, J., Pei, A., Wang, Z., Ji, P., Wang, H., & Huo, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.10160 · Cited by 46
AutoLabs: cognitive multi-agent systems with self-correction for autonomous chemical experimentation
Panapitiya, G., Saldanha, E., Job, H., & Hess, O. (2026) · Scientific Reports · doi:10.1038/s41598-026-45593-z · Cited by 44
Difficulty-Aware Agentic Orchestration for Query-Specific Multi-Agent Workflows
Su, J., Lan, Q., Xia, Y., Sun, L., Tian, W., Shi, T., & He, L. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792240 · Cited by 57
Large Language Models Cannot Self-Correct Reasoning Yet
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2310.01798 · Cited by 712
“For HotpotQA, we use the same prompt as Shinn et al. (2023).”
BackgroundMethodResultInfluentialJARVIS -1: Open-World Multi-Task Agents With Memory-Augmented Multimodal Language Models
Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., Lin, H., He, Z., Zheng, Z., Yang, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3511593 · Cited by 36
The New Agronomists: Language Models are Experts in Crop Management
Wu, J., Lai, Z., Chen, S., Tao, R., Pan, Z., & Hovakimyan, N. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00543 · Cited by 47
“Meanwhile, we notice that LMs have shown distinctive cognitive capabilities, which include advanced thinking [10, 70], robust memory functions [53], reflective skills [36, 61], as well as multi-modal capabilities [68, 77, 84].”
AI scientists produce results without reasoning scientificallyPreprint
Ríos-García, M., Alampara, N., Gupta, C., Mandal, I., Mannan, S., Aghajani, A. A., Krishnan, N. M. A., & Jablonka, K. M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.18805 · Cited by 45
Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
Singh, A., Ehtesham, A., Kumar, S., Khoei, T. T., & Vasilakos, A. V. (2025) · arXiv · doi:10.48550/arxiv.2501.09136 · Cited by 36
Domain-specific chatbots for science using embeddings
Yager, K. G. (2023) · Digital Discovery · doi:10.1039/d3dd00112a · Cited by 37
Dong, Y., Zhan, Z., Hu, Y., Doe, D. M., & Han, Z. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106563 · Cited by 45
Flows: Building Blocks of Reasoning and Collaborating AIPreprint
Josifoski, M., Klein, L., Peyrard, M., Baldwin, N., Li, Y., Geng, S., Schnitzler, J. P., Yao, Y., Wei, J., Paul, D., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.01285 · Cited by 41
The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
Masterman, T., Besen, S., Sawtell, M., & Chao, A. (2024) · arXiv · doi:10.48550/arxiv.2404.11584 · Cited by 35
“Reflexion is a single-agent pattern that uses self-reflection through linguistic feedback [23].”
BackgroundMethodLarge Language Models are Learnable Planners for Long-Term Recommendation
Shi, W., He, X., Zhang, Y., Gao, C., Li, X., Zhang, J., Wang, Q., & Feng, F. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657683 · Cited by 35
Iterative Motion Editing with Natural Language
Goel, P., Wang, K.-C. J., Liu, C. K., & Fatahalian, K. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657447 · Cited by 53
“Spatial constraints are applied over a time range that is specified by temporal MEO parameters.”
BackgroundComputational Experiments Meet Large Language Model Based Agents: A Survey and PerspectivePreprint
Ma, Q., Xue, X., Zhou, D., Yu, X., Liu, D., Zhang, X., Zhao, Z., Shen, Y., Ji, P., Li, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.00262 · Cited by 45
MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code GenerationPreprint
Dai, J., Lu, J., Feng, Y., Zeng, G., Ruan, R., Cheng, M., Huang, D., Tan, H., & Guo, Z. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.11430 · Cited by 44
PAG: Multi-Turn Reinforced LLM Self-Correction with Policy as Generative VerifierPreprint
Jiang, Y., Xiong, Y., Yuan, Y., Xin, C., Xu, W., Yu, Y., Zhao, Q., & Yan, L. Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.10406 · Cited by 38
Mitigating reasoning hallucination through Multi-agent Collaborative Filtering
Shi, J., Zhao, J., Wu, X., Xu, R., Jiang, Y.-H., & He, L. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125723 · Cited by 49
Large Language Models for Automated Open-domain Scientific Hypotheses Discovery
Yang, Z., Du, X., Li, J., Zheng, J., Poria, S., & Cambria, E. (2023) · arXiv · doi:10.48550/arxiv.2309.02726 · Cited by 34
LLMs Can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thought
Jiang, Z., Peng, H., Feng, S., Li, F., & Li, D. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/381 · Cited by 53
Fan, E., Hu, K., Wu, Z., Ge, J., Miao, J., Zhang, Y., Sun, H., Wang, W., & Zhang, T. (2026) · Advanced Intelligent Discovery · doi:10.1002/aidi.202500174 · Cited by 34
Large Language Models Are Semi-Parametric Reinforcement Learning Agents
Zhang, D., Chen, L., Zhang, S., Xu, H., Zhao, Z., & Yu, K. (2023) · neural information processing systems · doi:10.52202/075280-3419 · Cited by 106
DOCBENCH: A Benchmark for Evaluating LLM-based Document Reading SystemsPreprint
Zou, A., Yu, W., Zhang, H., Ma, K., Cai, D., Zhang, Z., Zhao, H., & Yu, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.10701 · Cited by 43
“The overall task completion usually requires multiple reasoning, execution and reflection steps that cannot be accomplished in a simple input-output manner [33, 41, 47].”
BackgroundThe Last AI Built by Humans: Toward Genuine Recursive Self-ImprovementPreprint
Duan, Y., Liu, Y., Tang, Z., Chen, H., Zhou, J., Liu, Y., BangRui, X., Wu, Y., Chen, S., Zhou, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.11873 · Cited by 36
Hickman, L., Dunlop, P. D., & Wolf, J. L. (2024) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12479 · Cited by 33
A Zero-Shot Language Agent for Computer Control with Structured Reflection
Li, T., Li, G., Deng, Z., Wang, B. K., & Li, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.753 · Cited by 44
Hickman, L., Dunlop, P. D., & Wolf, J. L. (2024) · doi:10.31234/osf.io/9cs23 · Cited by 33
Zong, C. F., Yan, Y., Lü, W., Shao, J., Huang, Y., Chang, H., & Zhuang, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.101 · Cited by 55
A Survey of LLM-based Agents: Theories, Technologies, Applications and Suggestions
Dong, X., Zhang, X., Bu, W., Zhang, D., & Cao, F. (2024) · International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC) · doi:10.1109/aiotc63215.2024.10748304 · Cited by 50
“Specifically, Reflexion [18] contemplates the task feedbacks with an evaluator and the self-reflection scheme, retaining the reflective narrative in episodic memory, enhancing subsequent decision-making.”
BackgroundThe AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
Yamada, Y., Lange, R. T., Lu, C., Hu, S., Lu, C., Foerster, J., Clune, J., & Ha, D. (2025) · arXiv · doi:10.48550/arxiv.2504.08066 · Cited by 32
Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf
Xu, Y., Wang, S., Li, P., Luo, F., Wang, X., Liu, W., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.04658 · Cited by 32
Nissist: An Incident Mitigation Copilot based on Troubleshooting Guides
An, K., Yang, F., Lu, J., Li, L., Ren, Z., Huang, H., Wang, L., Zhao, P., Kang, Y., Ding, H., et al. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia241032 · Cited by 37
Code Repair with LLMs gives an Exploration-Exploitation Tradeoff
Tang, H., Hu, K., Zhou, J., Zhong, S., Zheng, W., Si, X., & Ellis, K. (2024) · neural information processing systems · doi:10.52202/079017-3746 · Cited by 98
Zheng, D., Zhang, B., & Geibel, J. (2024) · Journal of Advanced Computing Systems · doi:10.69987/jacs.2024.40106 · Cited by 32
Complex QA and language models hybrid architectures, SurveyPreprint
Daull, X., Bellot, P., Bruno, E., Martin, V., & Murisasco, É. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09051 · Cited by 44
CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents
Qi, S., Chen, S., Li, Y., Kong, X., Wang, J., Yang, B., Wong, P., Zhong, Y., Zhang, X., Zhang, Z., et al. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2401.10568 · Cited by 53
Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents
Deng, Y., Zhang, W., Lam, W., Ng, S.-K., & Chua, T.-S. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2311.00262 · Cited by 89
CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
Yan, W., Tian, Y., Li, Y., Chen, Q., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.337 · Cited by 154
Token-Budget-Aware LLM Reasoning
Han, T., Wang, Z., Fang, C., Zhao, S., Ma, S., & Chen, Z. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.1274 · Cited by 31
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., & Press, O. (2024) · arXiv · doi:10.48550/arxiv.2405.15793 · Cited by 3,708
Self-citationAdaPlanner: Adaptive Planning from Feedback with Language Models
Sun, H., Zhuang, Y., Kong, L., Dai, B., & Zhang, C. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2537 · Cited by 31
HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent
Wu, J. J., & Fard, F. H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715109 · Cited by 55
“Although some LLM-based agent systems have got a lot of attention, such as SWE-Agent [82], CoderAgent [89], AutoGPT [1] and many others [60, 62, 77, 84], there is no study that investigates the reported issues and systematically integrates them with the LLM agent approach for code generation.”
LLM-State: Open World State Representation for Long-horizon Task Planning with Large Language ModelPreprint
Chen, S., Xiao, A., & Hsu, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.17406 · Cited by 43
Leveraging Vision-Language Models for Manufacturing Feature Recognition in Computer-Aided Designs
Khan, M. T., Chen, L., Ng, Y. H., Feng, W., Tan, N. Y. J., & Moon, S. K. (2025) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4069266 · Cited by 30
Agentic AI and the rise of in silico team science in biomedical research
Li, B., Saini, A. K., Hernandez, J. G., & Moore, J. H. (2026) · Nature Biotechnology · doi:10.1038/s41587-026-03035-1 · Cited by 45
“There have been various inference-time reflection techniques for LLMs. Reflexion 42 self-improves by critiquing and memorizing each step it implements, allowing the system to analyze why an approach fails and to avoid repeating the same mistakes.”
Robot Learning in the Era of Foundation Models: A SurveyPreprint
Xiao, X., Liu, J., Wang, Z., Zhou, Y., Qi, Y., Cheng, Q., He, B., & Jiang, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.14379 · Cited by 44
Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet VideosPreprint
Ye, W., Liu, F., Ding, Z., Gao, Y., Rybkin, O., & Abbeel, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.09886 · Cited by 36
An Empirical Study on Failures in Automated Issue SolvingPreprint
Liu, S., Liu, F., Li, L., Tan, X., Zhu, Y., Lian, X., & Li, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.13941 · Cited by 35
“In contrast, agent-based tools operate through iterative observation–action loops [34, 51], enabling dynamic exploration and adaptive repair.”
Agent as Policy for Robotic ManipulationPreprint
Jia, M., Lin, Y., Zhang, X., Zhang, Z., Liu, X., & Jiang, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.12541 · Cited by 34
“Language agents combine reasoning, tool use, and reflection on interaction feedback (Yao et al., 2022; Schick et al., 2023; Shinn et al., 2023).”
Saheed, Y. K., & Chukwuere, J. E. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3648649 · Cited by 29
“The reasoning capabilities of LLM agents, along with their analytical and critical skills, render them potent operators and e ffi cient intermediates between task execution and the users [22].”
A survey on mathematical reasoning and optimization with large language models
Forootani, A., Aliabadi, D. E., & Thrän, D. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200712 · Cited by 32
REFINER: Reasoning Feedback on Intermediate Representations
Paul, D., Ismayilzada, M., Peyrard, M., Borges, B., Bosselut, A., West, R., & Faltings, B. V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.01904 · Cited by 29
“5, Self-refine, Self-reflection), suggesting our results are not random and significant.”
MethodAccelerating earth science discovery via multi-agent LLM systems
Pantiukhin, D., Shapkin, B., Kuznetsov, I., Jost, A. A., & Koldunov, N. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1674927 · Cited by 40
SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning
Rana, K., Haviland, J., Garg, S., Abou-Chakra, J., Reid, I., & Suenderhauf, N. (2023) · arXiv · doi:10.48550/arxiv.2307.06135 · Cited by 29
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Zhuo, T. Y., Vu, M. C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, I. N. B., Zhan, H., He, J., Paul, I., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.15877 · Cited by 29
“The environment can help unlock the capabilities like self-debugging [15] and self-reflection [53].”
BackgroundAdaptive Graph Pruning for Multi-Agent Communication
Li, B., Zhao, Z., Lee, D., & Wang, G. (2025) · Frontiers in artificial intelligence and applications · doi:10.3233/faia251326 · Cited by 34
Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models
Sun, W., Zhang, C., Zhang, X., Yu, X., Huang, Z., Xu, H., He, S., Zhao, J., & Liu, K. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-2725-0_25 · Cited by 33
Lyra: Orchestrating Dual Correction in Automated Theorem Proving
Zheng, C., Wang, H., Xie, E., Liu, Z., Sun, J., Xin, H., Shen, J., Li, Z., & Li, Y. (2024) · Transactions on machine learning research · doi:10.48550/arxiv.2309.15806 · Cited by 41
Let Models Speak Ciphers: Multiagent Debate through Embeddings
Pham, C., Liu, B., Yang, Y., Chen, Z., Liu, T., Yuan, J., Plummer, B. A., Wang, Z., & Yang, H. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.06272 · Cited by 38
Why Solving Multi-agent Path Finding with Large Language Model has not Succeeded YetPreprint
Chen, W., Koenig, S., & Dilkina, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.03630 · Cited by 36
The Importance of Directional Feedback for LLM-based OptimizersPreprint
Nie, A., Cheng, C.-A., Kolobov, A., & Swaminathan, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.16434 · Cited by 35
“We additionally evaluate Reflexion agent (Shinn et al., 2023).”
Strategic Collusion of LLM Agents: Market Division in Multi-Commodity CompetitionsPreprint
Lin, R. Y., Ojha, S., Cai, K., & Chen, M. F. (2024) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2410.00031 · Cited by 44
“Previous works demonstrate that allowing agents to communicate with their future selves ensures continuity and evolution of strategy [8, 31].”
A LLM-Powered Automatic Grading Framework with Human-Level Guidelines OptimizationPreprint
Chu, Y., Li, H., Yang, K., Shomer, H., Liu, H., Copur‐Gencturk, Y., & Tang, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.02165 · Cited by 34
“Self-reflection has demonstrated promising results in improving LLM reasoning [37, 28].”
Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code RepositoriesPreprint
Yildiz, A., Teo, S. G., Lou, Y., Feng, Y., Wang, C., & Divakaran, D. M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.03586 · Cited by 29
“Reflexion agents (Shinn et al., 2023) are similar to ReAct agents however they focus on self-reflection and dynamic memory updates alongside with reinforcement learning.”
Automating Autograding: Large Language Models as Test Suite Generators for Introductory Programming
Alkafaween, U., Al-Bluwi, I., & Denny, P. C. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13100 · Cited by 35
Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction
Kang, W.-C., Ni, J., Mehta, N., Sathiamoorthy, M., Hong, L., Chi, E., & Cheng, D. Z. (2023) · arXiv · doi:10.48550/arxiv.2305.06474 · Cited by 28
A feasibility study of automating radiotherapy planning with large language model agents
Wang, Q., Wang, Z., Li, M., Ni, X., Tan, R., Zhang, W., Wubulaishan, M., Wang, W., Yuan, Z., Zhang, Z., et al. (2025) · Physics in Medicine & Biology · doi:10.1088/1361-6560/adbff1 · Cited by 28
“To mitigate this, we employ a two-stage self-reflection mechanism (Shinn et al 2024) as outlined in figure 3.”
Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Text
Abdali, S., Anarfi, R., Barberan, C., & He, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671463 · Cited by 42
“…scratchpads [11, 38], Chain-of-Thought (CoT) [59], Reversing Chain-of-Thought [63] (RoCT), RLHF [10, 73], iterative self-reflection [34, 48], self-consistency [56], society of minds strategy [14], pruning truthful datasets [9], adjusting the sys-tem parameters to limit model creativity…”
BackgroundZhuo, L., Zhao, L., Paul, S., Liao, Y., Zhang, R., Xin, Y., Gao, P., Elhoseiny, M., & Li, H. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01422 · Cited by 70
“Modern LLMs have demonstrated the ability to improve their outputs by reflecting on intermediate outputs and subsequently refining their responses [26, 36, 47], leading to superior performance on complex tasks such as math problem solving and code generation.”
MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science
Zhang, J., Gan, J., Wang, X., Jia, Z., Gu, C., Chen, J., Zhu, Y., Ma, M. D., Zhou, D., Li, L., et al. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818888 · Cited by 31
“DeepSeek-R1 achieves the highest score on text-only questions at 75.22% accuracy, and GPT-5 performs the best on questions with images at 53.02%.”
InfluentialChen, Z., Ma, W., & Jiang, L. (2026) · Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering · doi:10.1145/3744916.3773140 · Cited by 2
RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoning
Hu, H., Wang, P., Feng, Y., Wei, K., Yin, W., Diao, W., Wang, M., Bi, H., Kang, K., Ling, T., et al. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3718699 · Cited by 31
“Current models are evolving from perception-driven systems to generalist agents with reasoning capabilities, but remain largely focused on natural scenes and struggle with the complexity of RS imagery.”
Evaluating Large Language Models for Material Selection
Grandi, D., Jain, Y. P., Groom, A., Cramer, B., & McComb, C. (2024) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4066730 · Cited by 38
Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon GamePreprint
Shi, Z., Fang, M., Zheng, S., Deng, S., Chen, L., & Du, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.17515 · Cited by 33
SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender CodePreprint
Hu, Z., İşcen, A., Jain, A., Kipf, T., Yue, Y., Ross, D. A., Schmid, C., & Fathi, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.01248 · Cited by 109
DocAgent: A Multi-Agent System for Automated Code Documentation GenerationPreprint
Yang, D., Simoulin, A., Qian, X., Liu, X., Cao, Y., Teng, Z., & Yang, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.08725 · Cited by 33
“Single-agent frameworks like ReAct (Yao et al., 2022) and Reflex-ion (Shinn et al., 2023) integrate action-reasoning and self-reflection.”
ReVeal: Self-Evolving Code Agents via Reliable Self-Verification
Jin, Y., Xu, K., Li, H., Han, X., Zhou, Y., Cheng, L., & Bai, J. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2506.11442 · Cited by 34
“Early approaches demonstrated the benefits of tool integration via prompt engineering (Yao et al., 2023; Chen et al., 2023; Shinn et al., 2023) and supervised fine-tuning (Gou et al., 2024).”
KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakes
Lai, E. Y., Vitagliano, G., Zhang, Z., Chabra, O., Sudhir, S., Zeng, A., Zabreyko, A. A., Li, C., Kossmann, F., Ding, J., et al. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2506.06541 · Cited by 41
“The framework invokes the model once per task, using sampled data snippets from the data lake and a task description.”
Adaptive Memory Admission Control for LLM AgentsPreprint
Zhang, G., Jiang, W., Wang, X., Behr, A., Zhao, K., Friedman, J., Chu, X., & Anoun, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.04549 · Cited by 31
“Large language model (LLM) agents have demonstrated strong capabilities in multi-turn interaction, reasoning, and tool use (Park et al., 2023; Wang et al., 2023; Shinn et al., 2023; Wei et al., 2022; Yao et al., 2022; Schick et al., 2023).”
Does Socialization Emerge in AI Agent Society? A Case Study of Moltbook
Li, M., Li, X., & Zhou, T. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813123 · Cited by 30
Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
Ehsani, R., Pathak, S., Rawal, S., Mujahid, A. A., Imran, M. M., & Chatterjee, P. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793579 · Cited by 28
Agentic Workflows for Improving Large Language Model Reasoning in Robotic Object-Centered Planning
Moncada-Ramirez, J., Matez-Bandera, J.-L., González-Jiménez, J., & Ruiz-Sarmiento, J.-R. (2025) · Robotics · doi:10.3390/robotics14030024 · Cited by 35
“Another particularly promising agentic workflow is Reflection [24,25], where the model autonomously evaluates its outputs by reviewing its responses, generating feedback and making refinements without relying on external resources.”
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.