The DOI matches its Crossref record; 196 of the 196 references it cites were found in a registry or an index; 159 of the 182 sources its claims cite were checked for support.
A survey on large language model based autonomous agents
Open accessAbstract
Abstract Autonomous agents have long been a research focus in academic and industry communities. Previous research often focuses on training agents with limited knowledge within isolated environments, which diverges significantly from human learning processes, and makes the agents hard to achieve human-like decisions. Recently, through the acquisition of vast amounts of Web knowledge, large language models (LLMs) have shown potential in human-level intelligence, leading to a surge in research on LLM-based autonomous agents. In this paper, we present a comprehensive survey of these studies, delivering a systematic review of LLM-based autonomous agents from a holistic perspective. We first discuss the construction of LLM-based autonomous agents, proposing a unified framework that encompasses much of previous work. Then, we present a overview of the diverse applications of LLM-based autonomous agents in social science, natural science, and engineering. Finally, we delve into the evaluation strategies commonly used for LLM-based autonomous agents. Based on the previous studies, we also present several challenges and future directions in this field.
Overview
CoChat Summary
## TL;DR
This paper is a survey of LLM-based autonomous agents, organizing the literature around three pillars: construction, application, and evaluation. It proposes a unified framework for agent architecture, reviews application areas across social science, natural science, and engineering, and summarizes common evaluation strategies plus open challenges.
## Key findings
- The authors state they compiled 100 relevant works on LLM-based agents and synthesized them into a taxonomy covering construction, application, and evaluation.
- For agent construction, they propose a unified architecture with four modules: profiling, memory, planning, and action; they also divide capability acquisition into fine-tuning and non-fine-tuning strategies.
- For evaluation, they distinguish subjective and objective methods; objective evaluation is further broken into metrics, protocols, and benchmarks such as task success, human similarity, efficiency, real-world simulation, social evaluation, multi-task evaluation, and software testing.
- The paper identifies major challenges including role-playing capability, prompt robustness, hallucination, knowledge boundary, and efficiency.
## Methods
- This is a survey/review paper, not an experimental study; it systematically organizes prior work on LLM-based autonomous agents.
- The survey is structured around three aspects: agent construction, application, and evaluation; construction is further split into architecture design and capability acquisition.
- The authors use taxonomies and representative examples from prior papers to synthesize methods, applications, benchmarks, and challenges.
## Limitations
- The paper does not report original experiments or new quantitative results; this is implied by the survey nature of the work.
- The text states that the field is “still at its initial stage”, so the synthesis is limited by the maturity of the underlying literature.
- The authors note that some content is “not explicitly discussed in the paper” in their tables/figures, indicating incomplete coverage of all possible agent designs and evaluation modes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
90 Valid · 12 Not valid · 9 Not checked · 90 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Building upon this capability, there has been a growing research area that employs LLMs as central controllers to construct autonomous agents to obtain human-like decisionmaking capabilities [11][12][13][14][15][16][17].”
§ A survey on large language model based autonomous agents > 1 Introduction
What the sources say
- Could not verifycites [11] Chen X, Li S, Li H, Jiang S, Qi Y, Song L. Generative adversarial user model for reinforcement learning based recommendation system. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 1052–1061 sourceExperimental — accuracy not yet measured
Our copy of Chen et al. (2018) is entirely about a model-based RL framework for recommendation systems: a GAN user-behaviour model plus a Cascading DQN policy (Abstract; Introduction, pp.1-2; Sections 5-6.3). No passage mentions large language models, LLMs as controllers, or autonomous agents with human-like decision-making; searches for 'large language model as central controller autonomous agent', 'LLM agent human-like decision-making capabilities' and 'language model transformer pre-trained controller agent' returned only semantically nearest passages on model-based RL, robotics sample efficiency (Introduction; Related Work) and the MovieLens/Taobao datasets (Section 6). The reference list (References, items 1-24) likewise stops at 2019 works (Shi et al., 2019) with no LLM-related entry, so the claim's point is not addressed by this source at all — neither supported nor contradicted. (LLM, conf=0.72)
- Supportedcites [12] Shinn N, Cassano F, Gopinath A, rasimhan K, Yao S. Reflexion: language agents with verbal reinforcement learning. NaIn: Proceedings of the 37th Conference on Neural Information Processing Systems. 2023, 36 source
The Introduction (page not specified) describes recent research on autonomous decision-making agents built around an LLM core, with LLMs generating actions executed in an environment. This supports the citing sentence’s broad description of the research area; it does not establish that the agents’ decisions match human decisions. (LLM, conf=0.88)
- Partly supportedcites [13] Shen Y, Song K, Tan X, Li D, Lu W, Zhuang Y. HuggingGPT: solving AI tasks with chatGPT and its friends in hugging face. In: Proceedings of the 37th Conference on Neural Information Processing Systems. 2023, 36 sourceExperimental — accuracy not yet measured
The “LLM as Controller” section describes an LLM directing expert models and serving as the brain for planning and decisions. “B More Discussion about Related Works” identifies HuggingGPT and other LLM-based systems as autonomous agents. These passages support the controller-and-agent research claim, but they do not establish that the agents attain human-like decision-making capabilities. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Autonomous agents typically perform tasks by assuming specific roles, such as coders, teachers and domain experts [18,19].”
§ A survey on large language model based autonomous agents > 2.1.1 Profiling module
What the sources say
- Partly supportedcites [18] Qian C, Cong X, Liu W, Yang C, Chen W, Su Y, Dang Y, Li J, Xu J, Li S, Liu Z, Sun M. Communicative agents for software development. 2023, arXiv preprint arXiv: 2307.07924 sourceExperimental — accuracy not yet measured
The Introduction describes autonomous agents with specialized roles cooperating on software-development tasks, and the Ablation Study discusses programmer and reviewer roles. The paper supports the role-based approach and the coder example, but the cited passages do not establish teachers or domain experts as agent roles, or that agents typically use roles across tasks generally. (LLM, conf=0.84)
- Supportedcites [19] Chen W, Su Y, Zuo J, Yang C, Yuan C, Chan C, Yu H, Lu Y, Hung Y, Qian C, Qin Y, Cong X, Xie R, Liu Z, Sun M, Zhou, J. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10848. source
Section 2.1, “Expert Recruitment” (page not provided), says that current methods predominantly assign role descriptions to autonomous agents. The paper’s role prompts illustrate domain experts and coding-related roles, while its Evaluator prompt assigns an agent the role of a mathematics teacher. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“More specifically, in Generative Agent [20], the agent has the capability to summarize its past experiences stored in memory into broader and more abstract insights.”
§ A survey on large language model based autonomous agents > Memory operations:
What the sources say
- Supportedcites [20] Park J S, O’Brien J, Cai C J, Morris M R, Liang P, Bernstein M S. Generative agents: interactive simulacra of human behavior. In: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 2023, 2 source
The Introduction describes a memory stream that records agents’ experiences and a reflection component that synthesizes those memories into higher-level inferences. Section 4 likewise says memory records are recursively synthesized into higher-level reflections. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Usually, there three commonly used criteria for information extraction, that is, the recency, relevance, and importance [20].”
§ A survey on large language model based autonomous agents > Memory operations:
What the sources say
- Partly supportedcites [20] Park J S, O’Brien J, Cai C J, Morris M R, Liang P, Bernstein M S. Generative agents: interactive simulacra of human behavior. In: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 2023, 2 sourceExperimental — accuracy not yet measured
The Introduction says the paper’s memory-retrieval model combines recency, relevance, and importance to select records. It does not establish that these are commonly used criteria for information extraction generally; the claim extends the paper’s specific design into a broader practice. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Agents can directly alter environment states by actions, such as moving their positions, collecting items, and constructing buildings.”
§ A survey on large language model based autonomous agents > Action goal:
What the sources say
- Supportedcites [16] Zhu X, Chen Y, Tian H, Tao C, Su W, Yang C, Huang G, Li B, Lu L, Wang X, Qiao Y, Zhang Z, Dai J. Ghost in the minecraft: generally capable agents for open-world environments via large language models with text-based knowledge and memory. 2023, arXiv preprint arXiv: 2305.17144 source
The action descriptions on p. 14 say the agent walks toward objects, collects dropped items after mining or attacking, and places blocks according to a blueprint when building. These support the examples of changing position, collecting items, and constructing buildings. (LLM, conf=0.93)
- Partly supportedcites [38] Wang G, Xie Y, Jiang Y, Mandlekar A, Xiao C, Zhu Y, Fan L, Anandkumar A. Voyager: an open-ended embodied agent with large language models. 2023, arXiv preprint arXiv: 2305.16291 sourceExperimental — accuracy not yet measured
The Introduction (p. 2) describes exploration and gathering resources, and the Full Prompt (p. 27) provides actions for moving to a position and placing a block. But the automatic-curriculum prompt (p. 22) explicitly excludes building tasks, so the paper does not establish constructing buildings as an activity of these agents. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
41 more claims · 22 with their cited sources checked
Create a free account to see all 201 claims and to verify the 5 references still open.
- It may state two separate claims as one. 79
- Probably not a substantial claim. 112
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The existing literature reaches neither goal. Benchmark surveys report scores (typically SWEBench [1]) without touching architecture; conceptual taxonomies [5, 6] describe patterns without implementations; the one concurrent source-level taxonomy [34] excludes the provider-native systems that define the field's frontier.”
From Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent studies have converged on a recurring set of functional components underlying such systems, often defined by 4 core components: planning, action, reflection, and memory (PARM) [11].”
From Medical AI Agents for Clinical Decision Support: Viewpoint Using the Planning, Action, Reflection, and Memory (PARM) Analytical Lens (2026) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“MAS, in which multiple autonomous agents collaborate as a team, are emerging as a powerful paradigm for biomedical and clinical data analysis [1,2].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLM-based autonomous agents can achieve greater flexibility and robustness by dividing labor among sub-agents [1,3].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“collaborative agent teams have produced superior results on complex bioinformatics analyses and clinical decision support challenges where no single model excels [1,4].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [1]
Experimental — accuracy not yet measured
20 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 1×
“In earlier paradigms, the policy functions that dictated the agent’s actions were conceived through heuristic methodologies and subsequently refined through environmental engagements [101, 86, 120, 55, 9, 116].”
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 2×
“Hallucination can lead to serious consequences such as incorrect or misleading code, security risks, and ethical issues [74].”
BackgroundLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Research works printed but not linked: 5
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI Blog, 2019, 1(8): 9
- Raman S S, Cohen V, Rosen E, Idrees I, Paulius D, Tellex S. Planning with large language models via corrective re-prompting. In: Proceedings of Foundation Models for Decision Making Workshop at Neural Information Processing Systems. 2022
- Weng X, Gu Y, Zheng B, Chen S, Stevens S, Wang B, Sun H, Su Y. Mind2Web: towards a generalist agent for the Web. In: Proceedings of the 37th Conference on Neural Information Processing Systems. 2023, 36
- Aher G V, Arriaga R I, Kalai A T. Using large language models to simulate multiple humans and replicate human subject studies. In: Proceedings of the 40th International Conference on Machine Learning. 2023, 337–371
- Nottingham K, Ammanabrolu P, Suhr A, Choi Y, Hajishirzi H, Singh S, Fox R. Do embodied agents dream of pixelated sheep: embodied decision making using language guided world modelling. In: Proceedings of the 40th International Conference on Machine Learning. 2023, 26311–26325
Cited by
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Self-citationThe Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
A Survey on the Memory Mechanism of Large Language Model based Agents
Zhang, Z., Bo, X., Ma, C., Li, R., Chen, X., Dai, Q., Zhu, J., Dong, Z., & Wen, J.-R. (2024) · arXiv · doi:10.48550/arxiv.2404.13501 · Cited by 149
Self-citationA survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024) · Vicinagearth · doi:10.1007/s44336-024-00009-2 · Cited by 404
“This mechanism is vividly demonstrated in complex task planning and robotic simulations within dynamic environmental scenarios [17, 132, 268].”
Influential
Metadata
- Authors
- Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., & Wen, J.
- Affiliations
- Renmin University of China (CN)
- Corresponding author
- Chen, Xu
- Lin, Yankai
- ORCID
- Venue
- Frontiers of Computer Science
- Published
- 2024-03-22
- Volume
- 18
- Issue
- 6
- ISSN
- 2095-2228 (ISSN-L)
- 2095-2236
- OpenAlex
- W4393065402
- Semantic Scholar
- 28c6ac721f54544162865f41c5692e70d61bccab
- Cited by
- 1,941
- References
- 61
- Citations by year
- 2026: 1060 · 2025: 623 · 2024: 249
- FWCI
- 284.94
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Frontiers of Computer Science (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Natural Science Foundation of China (62102420)
- Renmin University of China
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, autonomous agents, LLM agents, agent applications, assessment methods, human-like decision making, social science applications, engineering applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A survey on large language model based autonomous agents. https://app.cochat.ai/papers/10.1007/s11704-024-40231-1
- The paper
- Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., & Wen, J. (2024). A survey on large language model based autonomous agents. Frontiers of Computer Science. https://doi.org/10.1007/s11704-024-40231-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.