Reflexion: Language Agents with Verbal Reinforcement Learning
Open accessThe DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.
Overview
CoChat Summary
TL;DR
Reflexion reinforces language agents through verbal self-reflection stored in episodic memory rather than weight updates, improving performance across decision-making, reasoning, and coding tasks. It reports gains including 91% pass@1 on HumanEval, a 22% absolute improvement on AlfWorld, and 20% on HotPotQA.
Key findings
- On HumanEval Python, Reflexion achieves 91.0 pass@1, surpassing the previous state-of-the-art GPT-4 at 80.1; on HumanEval Rust it reaches 68.0 vs. 60.0, MBPP Rust 75.4 vs. 70.9, and Leetcode Hard 15.0 vs. 7.5.
- In AlfWorld, ReAct + Reflexion completes 130 out of 134 tasks using a simple heuristic and improves over strong baselines by an absolute 22% over 12 iterative learning steps; ReAct-only converges at a 22% hallucination rate with no long-term recovery.
- In HotPotQA, Reflexion improves over baselines by 20%; self-reflection gives an 8% absolute boost over episodic memory alone, and although CoT (GT) fails on 39% of questions, Reflexion improves accuracy by 14% without access to ground-truth answers.
- Ablations on the 50 hardest HumanEval Rust problems show full Reflexion at 0.68 vs. baseline 0.60; omitting test generation drops to 0.52, and omitting self-reflection stays at 0.60.
Methods
- Reflexion uses a modular loop with an Actor that generates text/actions, an Evaluator that scores outputs, and a Self-Reflection model that converts feedback into verbal summaries stored in memory; trials repeat until the Evaluator passes or a max trial count is reached, with memory usually bounded to 1–3 experiences.
- Experiments cover AlfWorld (134 environments, six tasks, ReAct + GPT-3), HotPotQA (100 questions, CoT/ReAct, exact-match grading, memory size 3), and programming (HumanEval, MBPP, LeetcodeHardGym; Python/Rust via MultiPL-E; self-generated unit tests up to 6).
- Ablations and analyses vary feedback signals, feedback incorporation methods, and agent types; compromised approaches are tested on HumanEval Rust, and a WebShop experiment is run across 100 environments.
Limitations
- The paper states Reflexion may fall into non-optimal local minima, limits long-term memory to a sliding window with maximum capacity, and notes code-generation test-driven development is limited by non-deterministic, impure, hardware-dependent, or concurrent functions; it also relies on LLM self-evaluation/heuristics and lacks a formal success guarantee.
- The paper reports WebShop failure: after only four trials the agent shows no improvement and does not generate helpful self-reflections, concluding Reflexion cannot solve tasks requiring significant diversity and exploration.
- Evident from the text: self-correction is an emergent quality of stronger, larger models—starchat-beta shows no improvement (baseline 0.26 vs. Reflexion 0.26)—and flaky or incorrect test suites can cause false positives/negatives, with false negatives preferred.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 59 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Therefore, in the case in which the model generates a flaky test suite, it is possible that all tests pass on an incorrect solution and lead to a false positive label on a code completion [11].”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
What the sources say
- Partly supportedcites [11] 11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). sourceExperimental — accuracy not yet measured
The Introduction (page not provided) says a flaky test can pass and fail on the same code and that changes may be merged when all tests pass. It does not address model-generated test suites, incorrect code completions, or false-positive labels, so those parts of the claim are not established by this paper. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“3 shows an inferior 52% vs 60% (baseline) accuracy, which suggests that the agent is unable to determine if the current implementation is correct without unit tests.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Ablation study
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ReAct + Reflexion significantly outperforms ReAct by completing 130 out of 134 tasks using the simple heuristic to detect hallucinations and inefficient planning.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“4 shows that self-reflection improves learning by an 8% absolute boost over the episodic memory learning advantage.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“on reasoning questions in HotPotQA [28] by 20%”
- Also stated in: § 4 Experiments · plain
“Most notably, Reflexion improves performance over strong baselines by 22% in AlfWorld, 20% in HotPotQA, and 11% on HumanEval.”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Experiments · plain
30 more claims
Create a free account to see all 65 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Memory and reflection systems such as MemAgent (Yu et al., 2025a), Reflexion (Shinn et al., 2023), and Generative Agents (Park et al., 2023) improve retrieval and episodic memory, but do not perform context shaping, state restructuring, or nextinstruction refinement.”
From PAACE: A Plan-Aware Automated Agent Context Engineering Framework (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“However, Reflexion computes verbal reward over a full trajectory, yielding a single end-of-episode signal that conflates credit across reasoning steps: a failure due to a single intermediate error receives the same critique pressure as one due to a flawed strategy.”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Shinn et al. [2023]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Unlike Reflexion, which only reflects on unsuccessful trajectories, our approach separately reflects on both successful and unsuccessful ones.”
From An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“The dominant approach is to curate memory at write time.”
From Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents (2026) · cites this paper as Shinn et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Interaction and code generation are increasingly used together, with code as the modality of choice for actions [48, 59], tool construction [13, 51, 69], and reasoning [39, 66, 67].”
From SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) · cites this paper as [39]
Experimental — accuracy not yet measured
107 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Reflexion: Language Agents with Verbal Reinforcement Learning > 3 Reflexion: reinforcement via verbal reflection
“while M e not pass or t < max trials do”
§ Reflexion: Language Agents with Verbal Reinforcement Learning > Self-reflection
“The Actor, Evaluator, and Self-Reflection models work together through trials in a loop until the Evaluator deems τ t to be correct.”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Items 701–800 of 3,537
WikiAutoGen: Towards Multi-Modal Wikipedia-Style Article Generation
Yang, Z., Chen, J., Xu, D., Fei, J., Shen, X., Zhao, L., Feng, C.-M., & Elhoseiny, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01441 · Cited by 17
“Existing approaches construct feed-back sources from three angles: (1) The LLM conducts iterative self-evaluation [35]; (2) A separately trained critic module provides specialized feedback [14]; (3) External knowledge from sources like Wikipedia and browsers is integrated [1, 25].”
SELAUR: Self Evolving LLM Agent via Uncertainty-Aware Rewards
Zhang, D., Liu, X., Cheng, L., Wang, Y., Murray, K., & Wei, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1468-6_29 · Cited by 16
Chen, C., Li, N., Qi, J., Chang, H., Shi, W., Xie, J., Yuan, J., Yang, H., Guo, J., Xu, C., et al. (2026) · Environmental Science & Technology · doi:10.1021/acs.est.5c09526 · Cited by 16
Effect of Private Deliberation: Deception of Large Language Models in Game Play
Poje, K., Brčić, M., Kovač, M., & Babac, M. B. (2024) · Entropy · doi:10.3390/e26060524 · Cited by 19
“Using a few-shot approach, models iteratively learn to optimize their behavior and become proficient in solving tasks like decision making and reasoning [22].”
MethodLLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks
Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L., & Murthy, A. (2024) · arXiv · doi:10.48550/arxiv.2402.01817 · Cited by 237
“This has particularly been useful but largely overlooked in cases where LLMs are used as an actor to generate an admissible plan by interacting with a simulator, for example in the case of AlfWorld Yao et al. (2023b); Shinn et al. (2023) and Minecraft Wang et al. (2023).”
BackgroundMethodResultInfluentialLarge Language Models can Strategically Deceive their Users when Put Under Pressure
Scheurer, J., Balesni, M., & Hobbhahn, M. (2023) · arXiv · doi:10.48550/arxiv.2311.07590 · Cited by 11
“While others (Kinniment et al., 2023; OpenAI, 2023; Zhou et al., 2023; Shinn et al., 2023; Deng et al., 2023; Wang et al., 2023; Richards, 2023; Nakajima, 2023; SuperAGI, 2023) give LMs access to real tools, like access to a command line that executes bash commands in a real virtual environment, we…”
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
Su, H., Luo, J., Liu, C., Yang, X., Zhang, Y., Dong, Y., & Zhu, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3688650 · Cited by 11
Evaluating Frontier Models for Dangerous Capabilities
Phuong, M., Aitchison, M., Catt, E., Cogan, S., Kaskasoli, A., Krakovna, V., Lindner, D., Rahtz, M., Assael, Y., Hodkinson, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.13793 · Cited by 11
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
Liu, Z., Yao, W., Zhang, J., Xue, L., Heinecke, S., Murthy, R., Feng, Y., Chen, Z., Niebles, J. C., Arpit, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.05960 · Cited by 11
“…emerging exploration of employing LLM to tackle various complex tasks (Zhang et al., 2023), amongst which L LM-augmented A utonomous A gents (LAAs) (Shinn et al., 2023; Madaan et al., 2023b; Huang et al., 2022; Kim et al., 2023; Paul et al., 2023; Yao et al., 2023a) stand with most spotlights.”
BackgroundGPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems
Stechly, K., Marquez, M., & Kambhampati, S. (2023) · arXiv · doi:10.48550/arxiv.2310.12397 · Cited by 11
“This lead us to consider whether the improvement is due to the type of backprompting (as authors who advocate these types of iterative approaches [20, 19, 15, 10, 4, 11] seem to assume) or because the answer just happens to be in the top-K completions (even if the LLM is itself not cognizant of it).”
BackgroundInfluentialExploring Autonomous Agents through the Lens of Large Language Models: A Review
Barua, S. (2024) · arXiv · doi:10.48550/arxiv.2404.04442 · Cited by 11
“Reflexion[68] is another framework in which an LLM agent assimilates environmental feedback to refine its internal world model, thereby enhancing its ability to foresee the outcomes of its actions and make more judicious decisions.”
BackgroundA Theoretical Understanding of Self-Correction through In-context Alignment
Wang, Y., Wu, Y., Wei, Z., Jegelka, S., & Wang, Y. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2854 · Cited by 11
Butlin, P. (2024) · Inquiry · doi:10.1080/0020174x.2024.2439995 · Cited by 13
ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation
Zhao, Y., Chen, S., Zhang, J., & Li, Z. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761035 · Cited by 16
“Shinn et al. [35] extends this idea by using memory-augmented reasoning loops, allowing the model to reflect on previous attempts and improve over time.”
On Simulation-Guided LLM-based Code Generation for Safe Autonomous Driving Software
Nouri, A., Andersson, J., Hornig, K. D. J., Fei, Z., Knabe, E., Sivencrona, H., Cabrero‐Daniel, B., & Berger, C. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3756987 · Cited by 15
“Each generated code is compared to the current baseline based on their respective test reports.”
Zhu, Y., Liu, L., Yu, J., & Zhang, D. (2026) · Future Internet · doi:10.3390/fi18060326 · Cited by 18
GPT-4V Cannot Generate Radiology Reports Yet
Jiang, Y., Chen, C., Nguyen, D., Mervak, B. M., & Tan, C. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.113 · Cited by 11
Wang, Z., Cai, S., Mu, Z., Lin, H., Zhang, C., Liu, X., Li, Q., Liu, A., Ma, X., & Liang, Y. (2024) · neural information processing systems · doi:10.52202/079017-2331 · Cited by 42
Chain-of-Thought Reasoning is a Policy Improvement OperatorPreprint
Zhang, H., & Parkes, D. C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.08589 · Cited by 15
“Reflexion (Shinn et al., 2023) uses a similar approach to improve accuracy on the HumEval coding benchmark.”
MethodTHOUGHTSCULPT: Reasoning with Intermediate Revision and SearchPreprint
Chi, Y., Yang, K., & Klein, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.05966 · Cited by 11
Story3D-Agent: Exploring 3D Storytelling Visualization with Large Language ModelsPreprint
Huang, Y., Qin, Y., Lu, S., Wang, X., Huang, R., Shan, Y., & Zhang, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.11801 · Cited by 12
Evaluating Long-Context Reasoning in LLM-Based WebAgentsPreprint
Chung, A., Zhang, Y., Lin, K., Rawal, A., Gao, Q., & Chai, J. Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.04307 · Cited by 12
“This gives an opportunity for the agent to correct any mistakes it has made in the next step [10].”
Generating Structured Plan Representation of Procedures with LLMs
Garg, D., Zeng, S., Ganesh, S., & Ardon, L. (2025) · arXiv
Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RLPreprint
Wu, I., Qu, Y., Setlur, A., & Kumar, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.03773 · Cited by 13
Agentic Tool Use in Large Language ModelsPreprint
Hu, J., Zhong, M., Chen, K., Bai, X., & Zhang, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.00835 · Cited by 14
“Reflexion [125] augments the standard interaction loop with verbal self-reflection, allowing the agent to critique failed trajectories and reuse these reflections in later attempts, while Self-Refine [86] shows that a model can iteratively improve its own intermediate reasoning or final outputs…”
Workspace Optimization: How to Train Your AgentPreprint
Sarafian, E., Kaplun, G., Banner, R., Soudry, D., & Ginsburg, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.09650 · Cited by 11
Inductive Deductive Synthesis: Enabling AI to Generate Formally Verified SystemsPreprint
Agarwal, S., Krentsel, A., Liu, S., Cemri, M., Cheng, A., Meng, R., Pfister, T., Li, C., Ratnasamy, S., Parameswaran, A., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.23109 · Cited by 12
“Coding agents have made great strides at writing code for many tasks, even running tests to automatically refine their own output [4, 3, 10, 58, 44].”
TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM AgentsPreprint
Zhang, H., Chen, M., Zhou, D., Lv, C., Chang, H., Cui, S., Wu, F., & Zhou, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.16156 · Cited by 11
Kim, J., Shin, B., Chung, J., & Rhu, M. (2025) · arXiv · doi:10.48550/arxiv.2506.04301 · Cited by 11
Goh, H. L., Sancenón, V., Chu, B. M. X., Koh, G. C. H., Koh, L., Teo, D., Ooi, M. S. L., Thng, C. N., Tan, C.-Z., Chua, D. W. L., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02514-8 · Cited by 11
“…planning, agentic LLMs overcome these limitations through decision-driven processes, enhancing fl exibility, performance and ef fi ciency by incorporating tool usage 24 , task decomposition and planning 25,26 , multi-reasoning path exploration 27,28 , adaptive learning 29,30 , and memory management.”
Martinek, V., Gariboldi, A., Tzimotoudis, D., Galea, M., Zacharopoulou, E., Escudero, A. A., Blake, E., Čechák, D., Cassar, L., Balestrucci, A., et al. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag250 · Cited by 12
“MLAgentBench (Huang et al. 2024) uses the ReAct (Yao et al. 2023) and Reflexion (Shinn et al. 2023) prompting frameworks to guide an LLM-based Agent to construct ML solutions for problems from diverse domains (text, images, graphs, and tabular data).”
A Pragmatist Robot: Learning to Plan Tasks by Experiencing the Real World
Qu, K., Lan, G., Zurbrügg, R., Chen, C., Mower, C. E., Ammar, H. B., & Hutter, M. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3669804 · Cited by 15
“Building on this idea, Reflexion [17] goes further by not only allowing the LLM agent to reflect on its failures but also storing these self-reflective experiences in long-term memory to inform future behavior.”
The Verifier Tax: Horizon Dependent Safety--Success Tradeoffs in Tool Using LLM Agents
Sah, T., Srivastava, V., Sah, D., & Jordan, K. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813160 · Cited by 15
Ghorbian, M., & Ghorbian, . (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109326 · Cited by 10
Liu, Y., Shen, R., Zhou, L., Xiao, Q., Yuan, J., & Li, Y. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf312 · Cited by 13
“S pecifically, agents engage in both self-reflective learning [35] from their own behaviors and other agents’ behaviors, progressively refining their decision-making and execution capabilities over time.”
Beyond Manual Media Coding: Evaluating Large Language Models and Agents for News Content Analysis
Doropoulos, S., Karapalidou, E., Charitidis, P., Karakeva, S., & Vologiannidis, S. D. (2025) · Applied Sciences · doi:10.3390/app15148059 · Cited by 15
Generating Phishing Attacks and Novel Detection Algorithms in the Era of Large Language Models
Fairbanks, J., & Serra, E. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825007 · Cited by 16
CRT-QA: A Dataset of Complex Reasoning Question Answering over Tabular Data
Zhang, Z., Li, X., Gao, Y., & Lou, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.132 · Cited by 36
“Inspired by recent works on model refinement (Ouyang et al., 2022; Huang et al., 2022; Shinn et al., 2023), we let human annotators select the questions that meet our requirements and then provide LLM with feedback to improve the quality of the questions.”
Automated Design of Agentic Systems
Hu, S., Lu, C., & Clune, J. (2024) · arXiv · doi:10.48550/arxiv.2408.08435 · Cited by 10
“…have been proposed, such as chain-of-thought planning and reasoning (Wei et al., 2022; Yao et al., 2023; Hu & Clune, 2024), memory structures (Zhang et al., 2024c; Lewis et al., 2020), tool use (Schick et al., 2023; Qu et al., 2024), and self-reflection (Madaan et al., 2024; Shinn et al., 2023).”
InfluentialMem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025) · arXiv · doi:10.48550/arxiv.2504.19413 · Cited by 10
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
Yao, W., Heinecke, S., Niebles, J. C., Liu, Z., Feng, Y., Xue, L., Murthy, R., Chen, Z., Zhang, J., Arpit, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.02151 · Cited by 10
“Recently, Shinn et al. (2023) presents Reflexion, a framework that equips agents with dynamic memory and self-reflection capabilities, enhancing their reasoning skills.”
BackgroundMethodInfluentialAEL: Evolving Agent Harness in Open-Ended Environments
Xu, W., Han, J., Guo, M., Mei, K., Zhu, X., Zhang, H., & Metaxas, D. N. (2026) · Findings of the Association for Computational Linguistics: EMNLP 2026 · doi:10.18653/2026.findings-emnlp.1316 · Cited by 10
AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System
Liu, Z., Yao, W., Zhang, J., Yang, L., Liu, Z., Tan, J., Choubey, P. K., Lan, T., Wu, J., Wang, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.15538 · Cited by 10
“Also, enabling reflection (Shinn et al., 2023; Yao et al., 2023b) reasoning enhances the performance of agents.”
BackgroundAI Agent for Education: von Neumann Multi-Agent System Framework
Jiang, Y.-H., Li, R., Zhou, Y., Qi, C., Hu, H., Wei, Y., Jiang, B., & Wu, Y. (2025) · arXiv · doi:10.48550/arxiv.2501.00083 · Cited by 10
“In contrast to the ReAct, Reflexion [26] presents a framework rooted in the principles of reinforcement learning, to enhance AI Agents’ reasoning capabilities through the endowment of dynamic memory and self-reflection capacities.”
StraGo: Harnessing Strategic Guidance for Prompt Optimization
Wu, Y., Gao, Y., Zhu, B. B., Zhou, Z., Sun, X., Yang, S., Lou, J., Ding, Z., & Yang, L. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.588 · Cited by 28
ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research
Wang, Z., Chen, B., Huang, Y., Cao, Q., He, M. S., Fan, J., & Liang, X. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-industry.10 · Cited by 20
Nakhost, M. A., Bloemer, A., & Minner, S. (2025) · International Journal of Production Research · doi:10.1080/00207543.2025.2591225 · Cited by 11
TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling
Li, J., Li, Y., Zhang, B., Tang, R., & Huang, K.-H. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.723 · Cited by 13
Jeong, S., Choi, Y. H., Kim, J. W., & Jang, B. (2026) · IEEE Access · doi:10.1109/access.2026.3716053 · Cited by 11
“To mitigate this issue, prompt-based single-agent meth-ods such as Chain-of-Thought (CoT) [5] and self-reflection [6] guide reasoning or incorporate external tools like the Wikipedia API.”
Unearthing Gas-Wasting Code Smells in Smart Contracts With Large Language Models
Jiang, J., Li, Z., Qin, H., Jiang, M., Luo, X., Wu, X., Wang, H., Tang, Y., Qian, C., & Chen, T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3491578 · Cited by 18
“This is done by computing the hashcode of each of the contract code files, and removing those that are duplicates.”
EvoWiki: Evaluating LLMs on Evolving Knowledge
Tang, W., Cao, Y., Deng, Y., Ying, J., Wang, B., Yang, Y., Zhao, Y., Zhang, Q., Huang, X., Jiang, Y., et al. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.47 · Cited by 24
AI for Mathematics: A Cognitive Science PerspectivePreprint
Zhang, C. E., Collins, K. M., Weller, A., & Tenenbaum, J. B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13021 · Cited by 13
“At first glance, recent LLM research such as chain-of-thought-prompting [103], “self-taught reasoning” [104], “self-reflection” [105] could be viewed as self-explanation to improve reasoning, but we encourage ruminating on the cognitive underpinnings.”
BackgroundCan Graph Descriptive Order Affect Solving Graph Problems with LLMs?Preprint
Ge, Y., Liu, S., Bi, B., Wang, Y., Mei, L., Feng, W., Chen, L., & Cheng, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.07140 · Cited by 11
“Recent extensions like Self-refine(Madaan et al., 2023), Reflexion(Shinn et al., 2023), and AdaPlan-ner(Sun et al., 2023) attempt to address these limitations through self-correction or environment feed-back integration.”
Online Training of Large Language Models: Learn while chattingPreprint
Liang, J., Wang, Z., Ma, Z., Li, J., Zhang, Z., Wu, X., & Wang, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.04790 · Cited by 12
From Reproduction to Replication: Evaluating Research Agents with Progressive Code MaskingPreprint
Kim, G. J., Wilf, A., Morency, L., & Fried, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.19724 · Cited by 15
“Some core challenges of generative agents are inherited from their generative backbones, including long context retrieval, reasoning, planning, and self-reflection [Shinn et al., 2023, Madaan et al., 2023].”
JoyAgent-JDGenie: Technical Report on the GAIAPreprint
Liu, J., Xu, S., Liu, S., Li, Y., Liu, W., Liu, M., Zhou, X., Wang, H., Jia, S., Wang, Z., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.00510 · Cited by 10
“…DeepResearchAgent [104]) pair retrieval-centric planning with iterative decomposition, while memory mechanisms progress from reflective feedback [75] to hierarchical designs like A-Mem [100] that separate working, semantic, and procedural layers—supporting long-horizon continuity and adaptive…”
Saarthi: The First AI Formal Verification EngineerPreprint
Kumar, A., Gadde, D. N., Radhakrishna, K. K., & Lettnin, D. V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.16662 · Cited by 18
“The authors in [15] propose a novel framework, Reflexion, that uses verbal reinforcement to help agents learn from previous failures.”
AI Risk-Management Standards Profile for General-Purpose AI (GPAI) and Foundation ModelsPreprint
Barrett, A. M., Newman, J., Nonnecke, B. M., Madkour, N., Hendrycks, D., Murphy, E. R., Jackson, K., & Raman, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.23949 · Cited by 24
OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization ProblemsPreprint
Li, X., Chen, J., Fang, X., Ding, S., Duan, H., Liu, Q., & Chen, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.10764 · Cited by 11
“Reflexion [30] and Self-Refine [23] incorporate feedback-driven learning to iteratively improve behavior.”
LLM Agents for Automated Web Vulnerability Reproduction: Are We There Yet?Preprint
Liu, B., Zhao, Y., Xu, G., & Wang, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.14700 · Cited by 14
“Re-flexion [66] addresses failure handling through reflection mechanisms that analyze past mistakes and store self-reflections in memory for improving future planning decisions.”
Li, P., Sun, J., Lin, F., Xing, S., Fu, T., Feng, S., Ni, C., & Tu, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.05517 · Cited by 11
“…search and reflection-style memories (Tree-of-Thoughts; Reflexion; Meta-Policy Reflexion) still decide and remember through free-form generation, treating logs as transient context rather than compiling them into an executable artifact (Yao et al., 2023; Shinn et al., 2023; Wu et al., 2025).”
Zhang, L., Wang, J. L., Wu, J., & Zhang, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.16453 · Cited by 13
“We compare our proposed Evolving Strategy & Execution framework with step-level Reflection, day-level Reflection, and the Plan-and-Act base-line (Shinn et al., 2023; Erdogan et al., 2025a).”
HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action ManipulationPreprint
Zeng, Z., Ding, F., Yang, H., & Li, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.18791 · Cited by 12
“ReAct [Yao et al., 2023], Reflexion [Shinn et al., 2023], and CRITIC [Gou et al., 2024] operate on text and perform post-hoc correction.”
Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent MemoryPreprint
Zou, M., Guo, Z., Liang, L., Wang, Z., Wang, Q., Wen, Q., King, I., Qu, L., & Xu, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.10870 · Cited by 11
Hypothesis-Driven Skill Optimization for LLM AgentsPreprint
Shang, F., & Yang, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.22330 · Cited by 10
Mnemosyne: Agentic Transaction Processing for Validating and Repairing AI-generated WorkflowsPreprint
Chang, E. Y., & Geng, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.00269 · Cited by 10
“…model, an optimization solver, or a team of agents, including prior multi-agent planning systems that add validation and transaction-style guarantees Chang and Geng [2025], Yao et al. [2023b,a], Shinn et al. [2023], Wang et al. [2023a], Park et al. [2023], Wang et al. [2023b], Chang [2025].”
Small Language Model Can Self-correct
Han, H., Liang, J., Shi, J., He, Q., & Xiao, Y. (2024) · arXiv · doi:10.48550/arxiv.2401.07301 · Cited by 10
ENTAgents: AI Agents for Complex Knowledge Otolaryngology
Chan, T. K., & Dinh, N.-D. (2025) · doi:10.1101/2025.01.01.25319863 · Cited by 9
VeriAct-Agent: Evidence-Verified Tool Execution for Reliable LLM Agents
Zhang, C., Sun, J., Hu, Z., Zhang, J., Wang, Y., & Yang, Z. (2026) · International Conference on Machine Learning and Intelligent Systems Engineering (MLISE) · doi:10.1109/mlise70044.2026.11607636 · Cited by 12
“Reflexion [8] introduces self-reflection after task-level failure but does not intervene at individual tool calls.”
Don't Let AI Agents YOLO Your Files: Information and Control in Agent-Native Filesystems
Zhong, S., Liao, J., Liu, J., Zheng, M., Arpaci-Dusseau, A. C., & Arpaci-Dusseau, R. H. (2026) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3830418.3843858 · Cited by 10
“Agents augment LLMs with tools ( e.g. Python interpreters, web browsers, file editors, databases, and remote APIs) [59, 82, 106, 110].”
Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Models
Shashidhar, S., Chinta, A., Sahai, V., Wang, Z., & Ji, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.608 · Cited by 9
“While fine-tuning and additional pre-training are effective, they entail additional costs and cannot be adapted to existing models.”
BackgroundMethodInfluentialParallel Tourism System With Foundation Models and Decentralized Autonomous Organizations
Huang, J., Tian, Y., Zhang, T., Ni, Q., Lin, F., Lee, T. J., Wu, N., & Wang, F. (2025) · International Journal of Tourism Research · doi:10.1002/jtr.70008 · Cited by 12
A multi-agent-driven robotic AI chemist enabling autonomous chemical research on demandPreprint
Song, T., Luo, M., Chen, L., Huang, Y. H., Zhu, Q. J., Liu, D., Zhang, B., Zou, G., Zhang, F., Shang, W., et al. (2024) · ChemRxiv · doi:10.26434/chemrxiv-2024-w953h-v2 · Cited by 10
Language as a cognitive and social tool at the time of large language models
Borghi, A. M., Livio, C. D., Gervasi, A. M., Mannella, F., Nolfi, S., & Tummolini, L. (2024) · Journal of Cultural Cognitive Science · doi:10.1007/s41809-024-00152-8 · Cited by 12
“Along similar lines, Reflexion (Shinn et al., 2024) allows agents to verbally reflect on task feedback signals and maintain this information in an episodic memory buffer, using it as an additional context and as a semantic cue.”
Zhang, X., Wang, Y., He, A., Wang, H., & Li, T. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129423 · Cited by 10
Zep: A Temporal Knowledge Graph Architecture for Agent Memory
Rasmussen, P., Paliychuk, P., Beauvais, T., Ryan, J., & Chalef, D. (2025) · arXiv · doi:10.48550/arxiv.2501.13956 · Cited by 9
MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation
Huang, Q., Vora, J., Liang, P., & Leskovec, J. (2023) · arXiv · doi:10.48550/arxiv.2310.03302 · Cited by 9
“…construct a prompt that summarizes all known information about the task and prior actions, and query the LM to produce a step-wise reflection (Shinn et al., 2023), a high-level plan (aut, 2023), a fact-checking section, a reasoning step before action (Yao et al., 2023), and the next action…”
BackgroundMethodInfluentialTraining Language Models to Self-Correct via Reinforcement Learning
Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J. D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.12917 · Cited by 9
FireAct: Toward Language Agent Fine-tuning
Chen, B., Shu, C., Shareghi, E., Collier, N., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2310.05915 · Cited by 9
“…only one thought-action-observation round, without using tools. plied LM-based reasoning for more purposes in agent design, such as reflection (Shinn et al., 2023; Park et al., 2023), planning (Yao et al., 2023; Dagan et al., 2023; Liu et al., 2023a), program synthesis (Liang et al., 2023;…”
InfluentialSelf-citationTree-Planner: Efficient Close-loop Task Planning with Large Language Models
Hu, M., Mu, Y., Yu, X., Ding, M., Wu, S., Shao, W., Chen, Q., Wang, B., Qiao, Y., & Luo, P. (2023) · arXiv · doi:10.48550/arxiv.2310.08582 · Cited by 9
Madrid-García, A., Benavent, D., & Merino-Barbancho, B. (2025) · EULAR Rheumatology Open · doi:10.1016/j.ero.2025.06.012 · Cited by 9
QuBE: Question-based Belief Enhancement for Agentic LLM Reasoning
Kim, M., Kim, J., Kim, J., & Hwang, S.-W. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1193 · Cited by 19
RTL Agent: An Agent-Based Approach for Functionally Correct HDL Generation via LLMs
Ranga, S., Mao, R., Bhattacharjee, D., Cambria, E., & Chattopadhyay, A. (2024) · Proceedings - Asian Test Symposium/Proceedings · doi:10.1109/ats64447.2024.10915277 · Cited by 9
Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction
Chen, X., Mao, M., Li, S., & Shangguan, H. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-short.39 · Cited by 25
ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contests
Xu, S., Hu, Y., Min, Y., Chen, Z., Zhao, W. X., & Wen, J.-R. (2025) · neural information processing systems · doi:10.52202/085713-5746 · Cited by 16
Self-Correcting Large Language Models: Generation vs. Multiple Choice
Rahmani, H. A., Krishna, S. H., Wang, X., Naghiaei, M., & Yilmaz, E. (2026) · Workshop for Research on Agent Language Models (REALM) · doi:10.18653/2026.realm-1.4 · Cited by 9
History-Aware Reasoning for GUI Agents
Wang, Z., Yang, L., Tang, X., Zhou, S., Chen, D., Jiang, W., & Li, Y. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i43.40966 · Cited by 12
Towards semantic-based agent communication networks: vision, technologies, and challenges
Zhang, P., Meng, R., Xu, X., Wang, Y., Huang, Z., Liu, Y., Zhang, R., Liu, Y., Tong, H., Song, H., et al. (2026) · Science China Information Sciences · doi:10.1007/s11432-026-5054-3 · Cited by 13
Kaplan, E., Bitan, T., Ghrayeb, L., Chen, L., Yotam, T., Hasabnis, N., & Oren, G. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.732 · Cited by 9
Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories
Khatri, P. (2026) · Workshop for Research on Agent Language Models (REALM) · doi:10.18653/2026.realm-1.5 · Cited by 12
SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent
Ji, J., Li, Y., Liu, H., Du, Z., Wei, Z., Qi, Q., Shen, W., & Lin, Y. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.15 · Cited by 16
Smurfs: Multi-Agent System using Context-Efficient DFSDT for Tool Planning
Chen, J., Liang, J., & Wang, B. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.169 · Cited by 23
“ReAct (Yao et al., 2022) further introduce the thought-action-observation format for tool chain reasoning, leading to the development of various multi-tool planning methods (Chen et al., 2023a; Xu et al., 2023; Shinn et al., 2023).”
AnnCoder: A Mti-Agent-Based Code Generation and Optimization Model
Zhang, Z., Wang, J., Li, Z., Wang, Y., & Zheng, J. (2025) · Symmetry · doi:10.3390/sym17071087 · Cited by 23
Learning to Be a Doctor: Searching for Effective Medical Agent Architectures
Zhuang, Y., Jiang, W., Zhang, J., Yang, Z., Zhou, J. T., & Zhang, C. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755559 · Cited by 16
“In recent years, the reasoning capabilities of large language models (LLMs) have improved signifi-2 cantly (Chen et al., 2025), leading to their increasing integration into the medical domain to enhance both diagnostic accuracy and treatment effectiveness (Hager et al., 2024; Schmidgall et al.,…”
Is There Any Social Principle for LLM-Based Agents?Preprint
Bai, J., Zhang, S., & Chen, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.11136 · Cited by 9
Survey of Consciousness Theory from Computational PerspectivePreprint
Ding, Z., Wei, X., & Xu, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.10063 · Cited by 10
Zheng, Y., Ma, C., Shi, K., & Huang, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.16542 · Cited by 14
Q-Probe: A Lightweight Approach to Reward Maximization for Language ModelsPreprint
Li, K., Jelassi, S., Zhang, H., Kakade, S. M., WATTENBERG, M. P., & Brandfonbrener, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.14688 · Cited by 17
Learning with Language-Guided State Abstractions
Peng, A., Sucholutsky, I., Li, B. Z., Sumers, T. R., Griffiths, T. L., Andreas, J., & Shah, J. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2402.18759 · Cited by 19
Position: Foundation Agents as the Paradigm Shift for Decision MakingPreprint
Liu, X., Lou, X., Jiao, J., & Zhang, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.17009 · Cited by 12
Metadata
- Authors
- Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 8634-8652
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2303.11366
- OpenAlex
- W7133222210
- Cited by
- 1,087
- References
- 0
- Citations by year
- 2026: 760 · 2025: 167 · 2024: 105 · 2023: 55
- FWCI
- 192.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language agents, verbal behavior, language acquisition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.