Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 1,501–1,600 of 3,537
ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference ContradictionsPreprint
Zhang, X., Yin, X., & Wan, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.08842 · Cited by 4
Logic Contrastive Reasoning with Lightweight Large Language Model for Math Word ProblemsPreprint
Ding, K., Zhenguo, M., & Yan, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.00131 · Cited by 2
Aissi, M.-S., Romac, C., Carta, T., Lamprier, S., Oudeyer, P., Sigaud, O., Soulier, L., & Thome, N. (2024) · HAL · doi:10.48550/arxiv.2410.19920 · Cited by 2
One STEP at a time: Language Agents are Stepwise PlannersPreprint
Nguyen, M., & Shareghi, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.08432 · Cited by 2
Chen, J., & Gao, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.17255 · Cited by 2
“Self-Reflection and Error Correction As depicted in Figure 1and 2, our agent includes an optional self-reflection loop, designed to address execution errors or suboptimal constructions(Shinn et al. 2023).”
Scalable, Symbiotic, AI and Non-AI Agent Based Parallel Discrete Event SimulationsPreprint
Barai, A., Eidenbenz, S., & Santhi, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.23846 · Cited by 2
“In reflection [60, 68], further improvement in next response is achieved by reinforcing language agents through linguistic feed-back.”
ARM: Discovering Agentic Reasoning Modules for Generalizable Multi-Agent SystemsPreprint
Yao, B., Malay, S. K. R., & Yadav, V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.05746 · Cited by 2
“Other approaches have focused on enhancing single agents with self-reflection and memory to learn from past mistakes and improve performance iteratively (Shinn et al., 2023; Madaan et al., 2023).”
The Prompt Engineering Report Distilled: Quick Start Guide for Life SciencesPreprint
Romanov, V., & Niederer, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.11295 · Cited by 2
“A similar strategy can be used alongside chain-of-thought prompting (Huang et al. 2022), and extended to language agents that maintain their own reflections to enhance discission-making in subsequent trials (Shinn et al. 2023).”
The Role of Reinforcement Learning in Large Language Models: A SurveyPreprint
Liu, L. (2025) · doi:10.36227/techrxiv.176300471.10070244/v1 · Cited by 2
Le, D., Ziti, K., Girard-Sun, E., Bouhaya, B., O’Brien, S., Sharma, V., & Zhu, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.18709 · Cited by 2
PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box OptimizationPreprint
Jawad, H., & Brunel, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.16209 · Cited by 3
“…used to generate and optimize prompts for specific tasks (Yang et al. 2023; Pryzant et al. 2023; Fer-nando et al. 2023; Khattab et al. 2023), write and debug code (Chen et al. 2021), and perform iterative self-improvement through reflection and feedback (Madaan et al. 2023; Shinn et al. 2023).”
Benchmarking LLM Agents for Wealth-Management WorkflowsPreprint
Milsom, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.02230 · Cited by 2
LoopBench: Discovering Emergent Symmetry Breaking Strategies with LLM SwarmsPreprint
Parsaee, A., Talebirad, Y., Szepesvári, C., Ohal, V., & Redman, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.13713 · Cited by 3
“Reflexion [8] uses verbal reinforcement and episodic memory to help agents correct mistakes in subsequent trials.”
On the Temporal Question-Answering Capabilities of Large Language Models Over Anonymized DataPreprint
Ruiz, A. G., de la Rosa, T., & Borrajo, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.07646 · Cited by 2
“…Tree of Thought (ToT) (Yao et al., 2024), Chain of Thought with Reflexion (where the LLM can evaluate and retry the task in a more guided approach (Shinn et al., 2023)), CoT with External Execution (where the LLM generates algorithm-related code), and CoT with API Access (where the LLM can call…”
Towards Reliable Multi-Agent Systems for Marketing Applications via Reflection, Memory, and PlanningPreprint
Flores, L. J. Y., Shen, J., & Gu, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.11120 · Cited by 4
Quantifying LLM Biases Across Instruction Boundary in Mixed Question FormsPreprint
Ling, Z., Liu, S., Tang, Y., Huang, C., Jiang, G., Fu, S., Yang, J., Wan, Y., Zhang, J., Huang, K., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.20278 · Cited by 3
Chen, J., Badshah, S., Yu, X., & Han, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.13982 · Cited by 2
“Methodologies like Reflexion [57] and Reflection modules [58] empower agents to modulate their behaviors based on past experiences, while hierarchical role-playing [59, 60] facilitates the representation of intricate social roles and relational dynamics.”
RETuning: Upgrading Inference-Time Scaling for Stock Movement Prediction with Large Language ModelsPreprint
Lin, X., Yang, C., Ma, Y., Li, M., Zhang, R., Ni, Y., Wu, X., Xu, C., Guo, J., & Xiong, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.21604 · Cited by 3
SwiftSolve: A Self-Iterative, Complexity-Aware Multi-Agent Framework for Competitive ProgrammingPreprint
Singh, A. V., Shen, A., Law, B., Ismail, A., Rohweder, J., O'Brien, S., & Zhu, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.22626 · Cited by 2
Reinforcement Learning from Compiler and Language Server FeedbackPreprint
Zhang, Y., & Contributors, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.22907 · Cited by 2
“Language-model agents that plan and call external tools include ReAct (Yao et al., 2022), PAL (Gao et al., 2023), and Toolformer (Schick et al., 2023), and step-level guidance schemes such as Self-Refine (Madaan et al., 2023) and Reflexion (Shinn et al., 2023).”
LLM-AR: LLM-powered Automated Reasoning FrameworkPreprint
Chen, R., Ternasky, J., Yin, A. O., Mu, X., Alican, F., & Ihlamur, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.22034 · Cited by 2
“Finally, the LLM is asked to reflect on its initial policy based on the results of the statistical analysis (Shinn et al., 2023).”
Adam, A. M. I. (2026) · Preprints.org · doi:10.20944/preprints202601.0038.v1 · Cited by 2
Inoshita, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.08692 · Cited by 2
“Shinn et al. [27] proposed the Reflexion framework, where LLMs self-evaluate their outputs and improve through self-reflection, achieving up to 30% performance improvement through self-improvement.”
Multi-Agent Constraint Factorization Reveals Latent Invariant Solution StructurePreprint
Scofield, C. L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.15077 · Cited by 3
“Reflexion introduces verbal feedback and episodic text memory to improve subsequent trials without parameter updates [Shinn et al., 2023].”
Yong, X., Sun, P., Wang, Z., & Zhou, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.21212 · Cited by 5
When Do Multi-Agent Systems Outperform? Analysing the Learning Efficiency of Agentic SystemsPreprint
Su, J., & Wu, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.08272 · Cited by 2
“While SARL involves a single model optimizing task performance independently, MARL decomposes complex tasks into subtasks, enabling specialized agents to collaborate or compete to achieve overall objectives (Tran et al., 2025; Yuan & Xie, 2025; Shinn et al., 2023).”
Multi-Agent Debate: A Unified Agentic Framework for Tabular Anomaly DetectionPreprint
Wang, P., & Li, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.14251 · Cited by 3
“Sep-arately, LLM research explores deliberation via search, reflection, and debate protocols (Yao et al., 2023; Shinn et al., 2023; Du et al., 2023).”
STRUCTUREDAGENT: Planning with AND/OR Trees for Long-Horizon Web TasksPreprint
Lobo, E., Chen, X., Meng, J., Xi, N., Jiao, Y., Agarwal, C., Zick, Y., & Gao, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.05294 · Cited by 2
“Methods such as Reflexion [Shinn et al., 2023] add reflection mechanisms and evaluators, though often at the cost of frequent resets.”
REVERE: Reflective Evolving Research EngineerPreprint
Gangireddi, B. D., Garikaparthi, A., Patwardhan, M., & Cohan, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.20667 · Cited by 2
“Recent works on self-refinement (Shinn et al., 2023; Madaan et al., 2023; Majumder et al., 2024) improve reasoning through iterative feedback but remain instance-specific, motivating prompt-level and experience-based adaptation methods (Agrawal et al., 2025; Opsahl-Ong et al., 2024; Zhao et al.,…”
Zhou, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.06066 · Cited by 2
“The Control group utilises the baseline Reflexion architecture, where the internal reflector generates self-correction feedback in free natural language [Shinn et al., 2023] The Treatment group employs the Logic-Guided Reflexion architecture outlined in Section 3.3.”
InfluentialDon't Overthink It: Inter-Rollout Action Agreement as a Free Adaptive-Compute Signal for LLM AgentsPreprint
Sethi, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.08369 · Cited by 2
“Closest to our setting is Shinn et al. [2023], which uses verbal reflection as a free signal, but requires multiple full-episode rollouts and stores a growing reflection buffer.”
Text2Model: Modeling Copilots for Text-to-Model TranslationPreprint
Kadıoğlu, S., Uppuluri, K., & Singirikonda, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.12955 · Cited by 3
QRAFTI: An Agentic Framework for Empirical Research in Quantitative Finance
Lim, T. M., Muthuraman, K., & Sury, M. (2026) · arXiv (Cornell University) · doi:10.26153/tsw/64466 · Cited by 3
Agarwal, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.19049 · Cited by 3
“Reflexion [43] and Self-Refine [44] established the iterative-refinement paradigm using same-model self-critique; Irving et al. [45] foundationally proposed debate for AI alignment.”
EvoAgent: An Evolvable Agent Framework with Skill Learning and Multi-Agent DelegationPreprint
Zhang, A., Guo, J., 魏, 家., Lv, C., Wang, B., & Li, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.20133 · Cited by 2
“Reflexion [20] introduces a language feedback loop to revise strategies; Voyager [21] constructs a skill library to support continuous capability expansion in open environments; Generative Agents [22] investigate long-term memory and behavioral evolution mechanisms in simulated settings.”
WorkflowGen:an adaptive workflow generation mechanism driven by trajectory experiencePreprint
Wei, R., Wang, S., & Shi, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.19756 · Cited by 2
Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and GeneralizationPreprint
Adams, C., Oliveira, R., Almeida, G., & Torres, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.19857 · Cited by 2
“Agentic frameworks based on reasoning-and-acting patterns (Yao et al., 2022; Shinn et al., 2023; Madaan et al., 2023; Zhou et al., 2023) have demonstrated that structured tool interactions can dramatically enhance problem-solving capabilities, with specialized modular frameworks also emerging for…”
LoopTrap: Termination Poisoning Attacks on LLM AgentsPreprint
Xu, H., Wang, Z., Zhang, W., Zhu, Z., Wang, Y., Ren, K., & Chen, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.05846 · Cited by 3
“These agents operate within iterative execution loops, where they continuously reason, act, and self-evaluate their progress toward completing a task [14, 15, 27, 31, 34, 36, 46, 47].”
ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel ConflictsPreprint
Luo, Y., Kang, Z., Ji, T., Liu, X., Liu, Y., Li, S., & Peng, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.09033 · Cited by 2
Contexting as Recommendation: Evolutionary Collaborative Filtering for Context EngineeringPreprint
Zhu, J., Ou, Z., Zheng, C., Chen, Y., Zheng, Z., Shan, R., Wang, L. Z., Wang, L. Z., Liu, W., Yu, Y., et al. (2026) · arXiv (Cornell University) · doi:10.13140/rg.2.2.14780.19844 · Cited by 2
ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic WorkflowsPreprint
Pellicer, A. L., Angelov, P. P., Bukhari, M. A. S., Li, Y., Soares, E. A., & Kerns, J. G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.14113 · Cited by 3
“Frameworks such as AlphaCodium [32] and TimeXL [17] demonstrate that interleaving generation with iterative self-critique [37, 45] yields significant performance gains for complex, multi-step reasoning without requiring gradient updates.”
ContractBench: Can LLM Agents Preserve Observation Contracts?Preprint
Wang, J., He, Y., Wang, Z., Xing, H., De, A., & Chen, H. M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.17281 · Cited by 4
“…semantic detail), automatically generated by the deterministic validator (no human annotation required), and actionable at retry time without retraining — in contrast to model-generated self-critique [Shinn et al., 2023, Madaan et al., 2023] or human-curated process-reward datasets like PRM800K.”
ContraFix: Skill-Enhanced Contrastive Runtime Analysis for Vulnerability RepairPreprint
Liu, S., Liu, F., Wang, P., Liu, Y., Zhu, Y., Lian, X., & Zhang, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.17450 · Cited by 3
“We presented ContraFix, an LLM-based automated vulnerability repair framework that addresses two key limitations of existing approaches: the lack of repair-useful boundary evidence from a single failing execution and the limited reuse of cross-instance AVR experience.”
NeuSymMS: A Hybrid Neuro-Symbolic Memory System for Persistent, Self-Curating LLM AgentsPreprint
Sultan, M., Thuraisamy, S., & Rajaratnam, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.17596 · Cited by 2
“Shinn et al. proposed Reflexion, where agents store trajectories and verbal feedback and showed that iterative self-reflection over this memory can improve performance on complex tasks [28].”
Nguyen, H.-D., & Tran, T.-X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.17998 · Cited by 2
Boiling the Frog: A Multi-Turn Benchmark for Agentic SafetyPreprint
Bisconti, P., Prandi, M., Pierucci, F., Sartore, F., Panai, E., Caroli, L., Zhu, Y., Smith, A. L., Nannini, L., Galisai, M., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.22643 · Cited by 3
He, X., Lin, Z., Liu, S., Wu, J., Xie, Q., Zhou, W., & Xiao, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.23067 · Cited by 2
“Early agent frameworks such as Reflexion [6] demonstrated the value of persistent state for multi-step reasoning, though their memory policies are largely handcrafted.”
When Does Memory Help Multi-Trajectory Inference for Tool-Use LLM Agents?Preprint
Li, X., & Tao, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.28224 · Cited by 2
“Reflexion (Shinn et al., 2023) stores verbal reflections from failed trials in episodic memory to improve subsequent attempts.”
InfluentialOn the Scaling of PEFT: Towards Million Personal Models of Trillion ParametersPreprint
Lab, M., :, Bo, V., Cao, S., Cao, V., Chen, A., Chen, K., Cheng, C., Chiang, S., Fan, K., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.02437 · Cited by 3
“Related agent-learning work also treats skills and reflection as reusable learning objects (Shinn et al., 2023; Xia et al., 2026).”
MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPUPreprint
Cheng, K., Lu, S., Liao, S., Li, T., Zhang, Y., Yang, D. H., Lv, Q., (13530), H. W., Chen, Z., & Tang, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.04847 · Cited by 3
Cascading Hallucination in Agentic RAG: The CHARM Framework for Detection and MitigationPreprint
Mishra, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.04435 · Cited by 4
“Consequently, when an agent reviews its own cascaded logic [7], it suffers from severe confirmation bias, verifying the final output because it aligns with the corrupted intermediate context.”
InfluentialKnowledge-Based Zero-Replay Debugging of Multi-Agent LLM TracesPreprint
Cha, H., Weon, D., & Kang, D. H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.14805 · Cited by 2
“Agent systems increasingly combine tool use, planning, memory, and orchestration [14, 15, 16, 17, 18, 19, 20].”
Dissecting model behavior through agent trajectoriesPreprint
Gupta, G., Chaturvedi, V., Sengupta, S., Huan, J., & Deoras, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.17454 · Cited by 4
A Technical Taxonomy of LLM Agent Communication ProtocolsPreprint
Sander, L., Gidey, H. K., Lenz, A., & Knoll, A. (2026) · Research Square · doi:10.21203/rs.3.rs-10919831/v1 · Cited by 2
OpenRath: Session-Centered Runtime State for Agent SystemsPreprint
Wen, F., Wang, Z., & Xu, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.19409 · Cited by 3
“…level, Reflexion converts feedback from failed attempts into natural-language reflections held in an episodic buffer, so later attempts improve [27]; Generative Agents maintain a long-running memory stream that is retrieved, reflected upon, and compiled into plans [28]; and MemGPT adopts the…”
How Much Coordination Gain Is Real? A Paired Noise-Floor Protocol for Multi-Agent LLM BenchmarksPreprint
Kaliyev, A. T., & Maryanskyy, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.20695 · Cited by 2
Grounded Scaling: Why Agentic AI Needs Deterministic EnvironmentsPreprint
Ding, L., & Wang, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.22495 · Cited by 4
“A complementary direction has emerged in the self-evolving-agents literature (Gao et al., 2025; Shinn et al., 2023; Wang et al., 2024a): instead of (or in addition to) adapting the model to a fixed environment, the environment itself — tools, skills, memory, runtime context — adapts to deployed…”
InfluentialThe Hitchhiker's Guide to Agentic AI: From Foundations to SystemsPreprint
Roitman, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.24937 · Cited by 2
A Single Rewrite Suffices: Empirical Lessons from Production Skill Description OptimizationPreprint
Zhou, Y., Alqudah, M., Lai, K.-H., Halfaker, A., Xiong, Y., & Harari, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.30775 · Cited by 2
“…task prompts or refine outputs using textual gradients, demonstrations, or LLM self-criticism (Khattab et al., 2023; Opsahl-Ong et al., 2024; Yang et al., 2024; Zhou et al., 2023; Yuksekgonul et al., 2025; Pryzant et al., 2023; Madaan et al., 2023; Shinn et al., 2023; Agrawal et al., 2026).”
From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form NarrativesPreprint
Ho, C., Aluru, A., Hammouri, M., Luo, K., Lagasse, R., Bahuguna, A., & Sharma, V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.00918 · Cited by 3
Governed Individuation: Cryptographically Decoupling an Agent's Learning from Its AuthorityPreprint
Qin, X., Luan, S., Yang, C. R., & Li, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.04613 · Cited by 3
“…against this with training-time alignment and prompted policies [3, 4]; both are probabilistic, both were designed for a fixed model rather than for agents that keep adapting after deployment [5, 6], and a policy that reads untrusted inputs can be steered by instructions planted in them [7–9].”
Measuring Harness-Induced Belief Divergence in Multi-Step LLM AgentsPreprint
Yi, H.-W., & Song, X.-Y. (2026) · arXiv.org · doi:10.48550/arxiv.2607.04528
“The agent uses its current beliefs about task progress, risk, recoverability, and likely failure modes to reason about what to do next, while the harness determines the evidence from which those beliefs are updated (Yao et al., 2023b; Shinn et al., 2023).”
InfluentialNKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel GenerationPreprint
Tang, J., Huan, J., Zhou, H., Zhang, Y., & Wang, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.04395 · Cited by 2
FirstResearch: Auditable Question Formation for LLM Scientific Discovery AgentsPreprint
Wang, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.05682 · Cited by 2
“ReAct interleaves reasoning traces and actions for more interpretable task-solving trajectories (Yao et al., 2023), while Reflexion uses verbal feedback and memory to improve future trials without changing model weights (Shinn et al., 2023).”
Chang, Y., Xu, J. Q., Feng, X., Yang, C., Cheng, P., & Yang, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.07702 · Cited by 2
“A predominant paradigm for optimizing such agent systems is reflexive optimization (Pryzant et al., 2023; Shinn et al., 2023; Yang et al., 2024), which uses an LLM to diagnose agent-system failures and evolve agent policies based on historical feedback derived from agent trajectories.”
SkillCenter: A Large-Scale Source-Grounded Skill Library for Autonomous AI AgentsPreprint
Sha, T., Zhao, Y., Sun, L. G., & Dong, Y. H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.07676 · Cited by 2
“…programs while exploring an open-ended environment; Generative Agents [22] maintain a memory stream that is retrieved and reflected upon; Reflexion [23] stores verbal self-feedback to improve across trials; and ReAct [24], building on chain-of-thought prompting [25], interleaves reasoning traces…”
SPINE: Bridging the Cyber-Physical Gap with Agentic AIPreprint
Ham, M., Kim, D., Lee, C., Kim, M. J., Zhang, Y., Wang, J., & Liu, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.13049 · Cited by 2
“Recent agentic systems combine language models with tools, code execution, and iterative feedback to perform long-range software-engineering and scientific-analysis tasks [7, 8, 9, 10, 11].”
Yang, C.-H., Jiang, J., Vasudevan, V., Yang, C.-H., Zheng, H., Chen, L., Huerta, E. A., Vishwanath, V., Foster, I. T., & Thakur, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.15388 · Cited by 3
“A separate line of work shows that detection, revision, and successful repair are not the same event [16, 23, 24, 1, 8, 14].”
PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video GenerationPreprint
Li, Q., Hao, J., Li, Y., Yi, R., Rosin, P. L., & Lai, Y.-K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.16355 · Cited by 5
“Language agents have evolved from text-only reasoning to systems that act in external environments, including web interfaces, software repositories, and embodied simulators [Yao et al., 2022, Shinn et al., 2023, Madaan et al., 2023, Wang et al., 2023].”
DataFlow-Harness: A Grounded Code-Agent Platform for Constructing Editable LLM Data PipelinesPreprint
He, R., Wong, Z. H., Liang, H., Meng, Z., Shen, C., Ma, X., & Zhang, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.16617 · Cited by 2
“Reflexion [23] and Self-Debug [6] introduced iterative refinement via environment feedback.”
LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information ExtractionPreprint
You, X., Yan, T., Shan, Z., Du, L., & Zhao, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.23420 · Cited by 2
“Prompting-based methods encourage the model to produce free-form rationales before answering (Wei et al., 2022; Shinn et al., 2023; Madaan et al., 2023) , while training-based approaches use outcome or process rewards to cultivate reasoning behaviors via reinforcement learning (Light-man et al.,…”
Addressable Recall Compaction for Long Context-Window Control in AI AgentsPreprint
Dang, T., Ichikawa, Y., Fatima, S., & Shirahata, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.25066 · Cited by 2
BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation ParadigmsPreprint
Wang, P., Xu, B., Wang, S., Du, M., Zeng, X., Wu, H., Zhang, L., & Zhang, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.26497 · Cited by 2
MirrorCraft: Paired Evaluation under Hidden Rule Changes in MinecraftPreprint
Gao, J., Hu, B., Li, R., Fang, H., & Peng, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.29218 · Cited by 2
“Only ReAct is evaluated with the exact rules, so its gains from disclosure should not be generalized to other configurations.”
MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-offPreprint
Lei, S., Ouyang, K., Ruan, W., Wu, Y., Guo, Z., Sun, Y., & Tsung, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.04843 · Cited by 2
“Long-horizon LLM agents are increasingly expected to maintain persistent memory across extended interactions, including multi-session conversations, task logs, and evolving personal knowledge (Zhong et al. 2023; Shinn et al. 2023).”
Self-Correcting Long-Horizon Search Agents via Tree-Structured MemoryPreprint
Yang, A., Guo, Q., Huang, Z., Chen, Y., Qian, S., & Cao, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.10676 · Cited by 2
Self-evolving Agentic Customer Support System at LinkedInPreprint
Wang, C. H., Tu, M., Zhang, Q., Wu, W., Zhou, L., Shen, M., & Wei, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.10224 · Cited by 2
“We depart from fixed retrieval pipelines and expose RAG as a tool the agent invokes within its reasoning loop (Yao et al., 2023; Schick et al., 2023), letting the model decide when to retrieve and how to query, mirroring recent agentic RAG systems (Press et al., 2023; Shinn et al., 2023).”
Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic CritiquePreprint
Vijayvargiya, S., & Lokesh, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.10430 · Cited by 2
“To mitigate these specification-grounding failures, the dominant paradigm relies on post-hoc action correction [Shinn et al., 2023, Gou et al., 2024].”
Wrong but Useful: Trajectory Value Beyond Answer Correctness in Multi-Agent MessagesPreprint
Yang, C.-H., Chowdhury, A. A., Yang, C.-H., Zheng, W., Llorente, F., Ma, X., Li, X., Huerta, E. A., Yan, F., Foster, I. T., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.14375 · Cited by 3
“Self-refinement and critique systems instead revise a solver using feedback (Madaan et al. 2023; Shinn et al. 2023; Lin et al. 2024), although models may resist or misuse even strong feedback (Huang et al. 2023; Valmeekam, Marquez, and Kambhampati 2023; Jiang et al. 2025).”
Governance Records as Supervision: Verifier-Selected Self-Training for Structured Workflow RepairPreprint
Salas, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.18324 · Cited by 2
“Agent methods such as Reflexion use verbal feedback and episodic memory to improve later attempts (Shinn et al., 2023).”
Eureka: Task-Conditioned Meta-Agent Orchestration for Scientific DiscoveryPreprint
Wong, A., Cui, H., Tan, Y., Zhan, X., Lin, L., Guo, Y., Dai, Z., Zeng, Z., & Li, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.19047 · Cited by 2
Belief-Calibrated Optimization: An Explicit World Model for Agentic OptimizationPreprint
Chen, Y., Tian, Z., Dabas, M., Peris, C., Gupta, R., Jin, M., Kang, F., Zhang, S., Wang, N., & Jia, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.01861 · Cited by 2
TIGPO: Temporal Instance-Graph Policy Optimization for Long-Horizon LLM AgentsPreprint
Gan, J.-W. (2026) · arXiv.org · doi:10.48550/arxiv.2609.03383 · Cited by 1
“5 (Qwen Team, 2025), ReAct (Yao et al., 2023), which interleaves reasoning and acting, and Reflexion (Shinn et al., 2023), which uses verbal feed-back from previous attempts to refine subsequent decisions.”
From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use AgentsPreprint
Hu, L., Liang, X., & Zhu, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.04869 · Cited by 5
“Non-parametric improvement first retained feedback and experience as external language memory: Reflexion stores verbal self-feedback (Shinn et al., 2023), whereas ExpeL distills reusable insights from collections of task experience (Zhao et al., 2023).”
Trace2Tower: Transition-Aware EigenTrace Induction of Multi-Level Skills for LLM AgentsPreprint
Sun, J., Yang, B., Yuan, B., Li, M., & Peng, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.05261 · Cited by 2
Feng, J., Li, D., Chen, Y., Gao, Y., Chen, X., Wang, H., & Pu, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.04865 · Cited by 2
“– Reflexion (Shinn et al., 2023) converts task feedback into verbal reflections and stores them in episodic memory for later trials, while keeping the language-model parameters fixed.”
Ning, J., Zhong, S., Li, X., & Zeng, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.09219 · Cited by 3
“ReAct, Tree of Thoughts, Reflexion, and Self-Refine develop tool use, search, reflection, and feedback (Yao et al., 2023b,a; Shinn et al., 2023; Madaan et al., 2023).”
Adversarial Testing of Automated Program Repair Agents for Security VulnerabilitiesPreprint
Trad, F., Chen, S., Pham, H. V., Uddin, G., & Ray, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.15963 · Cited by 2
“LLM-Based and Agentic APR. Agentic APR systems are proposed at repository scale [2]–[4] with complementary approaches such as Reflexion [25] and ReAct [26], and conversational and planning-based repair [27], [28].”
TwinCheck: Evidence-Grounded Negative-Twin Verification for Stateful Tool AgentsPreprint
Dai, J., & Huang, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.26911 · Cited by 2
“Feedback-based repair ranges from repeated revision in Self-Refine to cross-trial verbal memory in Reflexion and tool-grounded critique in CRITIC [7, 12, 16].”
ControlScope: Workflow Revision and Reliability in LLM AgentsPreprint
Ning, J., Li, X., Kong, Y., & Li, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.34313 · Cited by 2
“Reflexion uses verbal feedback across trials (Shinn et al., 2023), Self-Refine iteratively improves generated outputs (Madaan et al., 2023), and self-debugging uses program execution to support code correction (Chen et al., 2023).”
A Comprehensive Survey of Agent Distillation across Models, Artifacts, and HarnessesPreprint
Luo, Z., Wang, S., Li, C., Yin, J., Yan, H., Cheng, M., Wang, C., Liu, S., Zhang, L., Ye, Q., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.36630 · Cited by 2
SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond MarkdownPreprint
Liu, Y., Li, H., Zhang, Y., Guo, J., Luo, H., Hu, W., Jing, H., Chung, K., Chen, J., Fan, C., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.04008 · Cited by 2
“Reliable execution depends on procedural knowledge for coordinating operations, recovering from failures, and validating outputs (Wang et al., 2023; Shinn et al., 2023).”
Humanize: Judgement Engineering for Agentic CodingPreprint
Liu, S., Zhu, L., Zhang, Z., Zou, D., Zhang, Z., Li, C., Bian, S., Han, S., & Nowatzki, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.08900 · Cited by 2
“Tool-using agents interleave reasoning and action (Yao et al., 2023), operate on whole repositories (Yang et al., 2024; Wang et al., 2025), and refine their output with their own critique or with execution feedback (Shinn et al., 2023; Madaan et al., 2023; Chen et al., 2024).”
Verification and Self-Improvement in Agentic AI: Foundations and LimitsPreprint
Lu, C.-P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.10611 · Cited by 2
“Reflexion uses feedback and episodic memory to improve subsequent trials without weight updates [11].”
Domine, B., Liu, B. X., Krawczynski, H., Kundu, M., Menon, S. V., & Zhou, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.14185 · Cited by 2
“The pipeline therefore uses a scratchpad of essential information that is injected into the LLM prompts (Packer et al. 2023; Shinn et al. 2023).”
Bai, J., Bai, G., Zhang, C., Li, S., Hu, M., Huang, W., & Liu, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.25114 · Cited by 2
Gómez Álvarez, S., Mozo Quesada, A., Navarro, T., Gálvez Rojas, S., & López Valverde, F. (2026) · Journal of Intelligent Information Systems · doi:10.1007/s10844-026-01086-z · Cited by 1
“Other notable works include embodied multi-agent systems (Zhang et al., 2024a), orchestration via HuggingGPT (Shen et al., 2023 ), reflective reasoning in Reflexion (Shinn et al., 2023), dynamic sub-agent creation in TDAG (Wang et al., 2025a), and proactive coordination in ProAgent (Zhang et al., 2024b).”
The economics of accuracy for medical reasoning with large language models
Bhattacharyya, K., & Kamabattula, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001182 · Cited by 1
Wu, C., Luo, Y., Qu, Z., & Wang, M. (2025) · arXiv · Cited by 1
“Reflexion introduces verbal reinforcement learning, where agents use textual self-reflection to refine behavior without fine-tuning [6].”
Zhang, Z., Zhang, Y., Zeng, D., Liu, K., & Zhao, J. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3675803 · Cited by 3
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.