From language to action: a review of large language models as autonomous agents and tool users
Open accessThe DOI matches its Crossref record; 130 of the 130 references it cites were found in a registry or an index.
Abstract
The pursuit of human-level artificial intelligence (AI) has significantly advanced the development of autonomous agents and Large Language Models (LLMs). LLMs are now widely utilized as decision-making agents for their ability to interpret instructions, manage sequential tasks, and adapt through feedback. This review examines recent developments in employing LLMs as autonomous agents and tool users and comprises seven research questions. We only used the papers published between 2023 and 2025 in conferences of the A* and A-ranked and Q1 journals. A structured analysis of the LLM agents’ architectural design principles, dividing their applications into single-agent and multi-agent systems, and strategies for integrating external tools is presented. In addition, the cognitive mechanisms of LLMs, including reasoning, planning, and memory, and the impact of prompting methods and fine-tuning procedures on agent performance are also investigated. Furthermore, we have evaluated current benchmarks and assessment protocols and provided an analysis of 68 publicly available datasets to assess the performance of LLM-based agents in various tasks. In conducting this review, we have identified critical findings on verifiable reasoning of LLMs, the capacity for self-improvement, and the personalization of LLM-based agents. Finally, we have discussed ten future research directions to overcome these gaps.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These findings suggest that model scale alone does not guarantee robust biological learning [26, 39].”
From The next paradigm in bioinformatics: a review of multi-agent systems and foundational models for end-to-end scientific discovery (2026) · cites this paper as [39]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
Next-generation characterization of the Cancer Cell Line Encyclopedia
Ghandi, M., Huang, F. W., Jané-Valbuena, J., Kryukov, G. V., Lo, C. C., McDonald, E. R., Barretina, J., Gelfand, E. T., Bielski, C. M., Li, H., et al. (2019) · Nature · doi:10.1038/s41586-019-1186-3 · Cited by 3,394
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 2×
“…workflows where they perform planning, decision making, and tool interaction in various real-world applications, including research assistance [6], software development [7], drug discoveries [8], multi-robot systems [9], clinical support [10], game simulation [11] and scientific simulations [12].”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
WIDER FACE: A Face Detection Benchmark
Yang, S., Luo, P., Loy, C. C., & Tang, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.596 · Cited by 1,830
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804
Large Language Model based Multi-Agents: A Survey of Progress and Challenges
Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., & Zhang, X. (2024) · arXiv · doi:10.48550/arxiv.2402.01680 · Cited by 1,713 · cited 2×
“Guo et al. [31] examined LLM-based multi-agent systems, classified popular architectures and communication strategies, tools integration, and evaluated agent interaction through benchmarks.”
FEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187
Autonomous chemical research with large language models
Boiko, D. A., MacKnight, R., Kline, B., & Gomes, G. (2023) · Nature · doi:10.1038/s41586-023-06792-0 · Cited by 1,050 · cited 6×
“For instance, ChemCrow and Coscientist are agent systems evaluated on their ability to plan and carry out complex chemical syntheses using expert-designed chemistry tools [54, 67].”
InfluentialShen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.03825 · Cited by 861
Augmenting large language models with chemistry tools
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00832-8 · Cited by 794
The NarrativeQA Reading Comprehension Challenge
Kočiský, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K. M., Melis, G., & Grefenstette, E. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00023 · Cited by 596
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W., Mottaghi, R., Zettlemoyer, L., & Fox, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01075 · Cited by 540
Shen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670388 · Cited by 441 · cited 1×
“Beyond interface risks, agent architectures suffer from jailbreaking [217], and environmental signal-triggered backdoor attacks [89].”
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435 · cited 10×
“Environments like Overcooked-AI, AL-FRED, and WebShop are established settings for testing zero-shot coordination, instruction following, and web navigation, respectively [25,35,114].”
InfluentialLiao, F., Hu, Y., & Ye, S. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-023-02237-1 · Cited by 430
Large language models (LLMs): survey, technical frameworks, and future challenges
Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395 · cited 1×
“Large language models (LLMs) have become central in artificial intelligence (AI) research due to their strong human-like ability to understand, generate, and reason in natural language [1,2].”
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference
Ji, J., Hong, D., Zhang, B., Chen, B., Dai, J., Zheng, B., Qiu, T. A., Zhou, J., Wang, K., Li, B., et al. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1544 · Cited by 394
Jankauskaitė, J., Jiménez-García, B., Dapkūnas, J., Fernández-Recio, J., & Moal, I. H. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty635 · Cited by 389
EgoGesture: A New Dataset and Benchmark for Egocentric Hand Gesture Recognition
Zhang, Y., Cao, C., Cheng, J., & Lu, H. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2808769 · Cited by 372
StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models
Guo, Z., Cheng, S., Wang, H., Liang, S., Qin, Y., Li, P., Liu, Z., Sun, M., & Liu, Y. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.664 · Cited by 345
TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Chen, H., Suhr, A., Misra, D., Snavely, N., & Artzi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01282 · Cited by 343
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
Qi, Y., Wu, Q., Anderson, P., Wang, X., Wang, W. Y., Shen, C., & van den Hengel, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01000 · Cited by 331
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
Li, G., Hammoud, H. A. A. K., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · arXiv · doi:10.48550/arxiv.2303.17760 · Cited by 315
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N., & Lewis, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.378 · Cited by 303
Measuring and Narrowing the Compositionality Gap in Language Models
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., & Lewis, M. (2022) · arXiv · doi:10.48550/arxiv.2210.03350 · Cited by 294
Large language models empowered agent-based modeling and simulation: a survey and perspectives
Gao, C., Lan, X., Li, N., Yuan, Y., Ding, J., Zhou, Z., Xu, F., & Li, Y. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03611-3 · Cited by 294
♫ MuSiQue: Multihop Questions via Single-hop Question Composition
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00475 · Cited by 283
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274
Self-collaboration code generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672459 · Cited by 256 · cited 1×
“The profiling module defines an agent’s operational persona (e.g., developer, advisor, or task-specific role), conditioning its behavior and role policy through static profiles defined by experts or dynamic generative mechanisms [21, 138].”
Self-collaboration Code Generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2023) · arXiv · doi:10.48550/arxiv.2304.07590 · Cited by 256
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242
Analysing Mathematical Reasoning Abilities of Neural Models
Saxton, D., Grefenstette, E., Hill, F., & Kohli, P. (2019) · arXiv · doi:10.48550/arxiv.1904.01557 · Cited by 241
Zhang, R., Du, H., Liu, Y., Niyato, D., Kang, J., Xiong, Z., Jamalipour, A., & In Kim, D. (2024) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2024.3459037 · Cited by 233 · cited 8×
“…offering researchers and developers powerful alternatives to proprietary models [44,84] [56,58,71,83,124], comprising models with 7B [41,43,50,55,68,70,84,89,98,125], 13B [43,98,109], and 70B [44,69,109] parameters, remains integral to the field, frequently used for zero-shot…”
InfluentialEASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction
Yuan, S., Song, K., Chen, J., Tan, X., Shen, Y., Ren, K., Li, D., & Yang, D. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.44 · Cited by 232
Learning to mine aligned code and natural language pairs from stack overflow
Yin, P., Deng, B., Chen, E., Vasilescu, B., & Neubig, G. (2018) · Proceedings of the 15th International Conference on Mining Software Repositories · doi:10.1145/3196398.3196408 · Cited by 218
LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent
Yang, J., Chen, X., Qian, S., Madaan, N., Iyengar, M., Fouhey, D. F., & Chai, J. Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610443 · Cited by 212 · cited 6×
“Cognitive agents using task stack planning [45] achieve structured goal decomposition, particularly useful in household or task-intensive domains.”
InfluentialCogAgent: A Visual Language Model for GUI Agents
Hong, W., Wang, W., Lv, Q., Xu, J., Yu, W., Ji, J., Wang, Y., Wang, Z., Dong, Y., Ding, M., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01354 · Cited by 202 · cited 7×
“This reasoning technique outperforms ReACT by approximately 81% in average pass rate.”
InfluentialEditable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents
Wei, Y., Wang, Z., Lu, Y., Xu, C., Liu, C., Zhao, H., Chen, S., & Wang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01428 · Cited by 190 · cited 4×
“The flexibility of prompting makes it the primary way to set goals, integrate tools, and simulate complex social interactions.”
InfluentialSMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models
Kannan, S. S., Venkatesh, V. L. N., & Min, B.-C. (2024) · 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802322 · Cited by 185 · cited 6×
“These emerging strategies emphasize whether an agent can succeed and how it reasons, interacts, adapts, and safeguards users along the way.”
Influential(2024) · doi:10.1145/3704435 · Cited by 184 · cited 1×
“In terms of reactivity, LLMs can now handle multimodal inputs and interact with their environment using embodiment and tool integration, despite the latency caused by the textual reasoning stages [14,135–137].”
Tool Learning with Foundation Models
Qin, Y., Hu, S., Lin, Y., Chen, W., Ding, N., Cui, G., Zeng, Z., Huang, Y., Xiao, C., Han, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.08354 · Cited by 184 · cited 1×
“In terms of reactivity, LLMs can now handle multimodal inputs and interact with their environment using embodiment and tool integration, despite the latency caused by the textual reasoning stages [14,135–137].”
QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
Zhong, M., Yin, D., Yu, T., Zaidi, A., Mutuma, M., Jha, R., Awadallah, A. H., Celikyilmaz, A., Liu, Y., Qiu, X., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.472 · Cited by 178
AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning
Qiao, S., Zhang, N., Fang, R., Luo, Y., Zhou, W., Jiang, Y., Lv, C., & Chen, H. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.165 · Cited by 178 · cited 1×
“Examples such as MetaGPT [21], CAMEL [22], AgentBoard [23], AutoAct [24], and ProAgent [25] showcase how cooperative agents execute role-specific instructions and coordinate plans, while Generative Agents [12] simulate human-like behaviors in interactive environments.”
Ferber, D., El Nahhas, O. S. M., Wölflein, G., Wiest, I. C., Clusmann, J., Leßmann, M.-E., Foersch, S., Lammert, J., Tschochohei, M., Jäger, D., et al. (2025) · Nature Cancer · doi:10.1038/s43018-025-00991-6 · Cited by 175 · cited 5×
“Other studies assess agents in their ability to execute clinical calculations using established medical calculators or make complex oncology decisions based on multimodal patient data [53,73].”
InfluentialJo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581503 · Cited by 174 · cited 7×
“Furthermore, these benchmarks often overlook critical real-world constraints, such as temporal dynamics, which are essential for assessing an agent’s ability to plan and multitask efficiently [87].”
InfluentialWhen Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment
Xu, M., Niyato, D. T., Kang, J., Xiong, Z., Mao, S., Han, Z., Kim, D. I., & Letaief, K. B. (2024) · IEEE Wireless Communications · doi:10.1109/mwc.005.2400019 · Cited by 174 · cited 4×
“By addressing the critical issue of coordinating overlapping tasks with limited resources, it enables the evaluation and improvement of language agents’ multitasking and temporal reasoning capabilities.”
InfluentialWikiHow: A Large Scale Text Summarization Dataset
Koupaee, M., & Wang, W. Y. (2018) · arXiv · doi:10.48550/arxiv.1810.09305 · Cited by 166
Penedo, G., Malartic, Q., Hesslow, D., Cojocaru, R., Cappelli, A., Alobeidli, H., Pannier, B., Almazrouei, E., & Launay, J. (2023) · arXiv · doi:10.48550/arxiv.2306.01116 · Cited by 164
Reflective Multi-Agent Collaboration based on Large Language Models
Bo, X., Zhang, Z., Dai, Q., Feng, X., Wang, L., Li, R., Chen, X., & Wen, J.-R. (2024) · neural information processing systems · doi:10.52202/079017-4397 · Cited by 158 · cited 9×
“Similarly, Bo et al. [107] fine-tuned a shared "reflector" agent within their COPPER framework to enhance collaboration among multiple agents.”
InfluentialAccelerating scientific discovery with Co-Scientist
Gottweis, J., Weng, W.-H., Daryin, A., Tu, T., Sirkovic, P., Myaskovsky, A., Glowaty, G., Weissenberger, F., Orlandi, A., Popovici, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.18864 · Cited by 158 · cited 1×
“…workflows where they perform planning, decision making, and tool interaction in various real-world applications, including research assistance [6], software development [7], drug discoveries [8], multi-robot systems [9], clinical support [10], game simulation [11] and scientific simulations [12].”
ChatEDA: A Large Language Model Powered Autonomous Agent for EDA
Wu, H., He, Z., Zhang, X., Yao, X., Zheng, S., Zheng, H., & Yu, B. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3383347 · Cited by 152 · cited 9×
“While prompting provides dynamic control, fine-tuning offers a method for permanently embedding specialized knowledge and complex skills in an agent.”
InfluentialEnhancing diagnostic capability with multi-agents conversational large language models
Chen, X., Yi, H., You, M., Liu, W., Wang, L., Li, H., Zhang, X., Guo, Y., Fan, L., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01550-0 · Cited by 147 · cited 8×
“Applications also include triage decision support [37], scenario-based medical training [80], and co-dialogue agents for clinical support [74].”
InfluentialSOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents
Wang, R., Yu, H., Zhang, W., Qi, Z., Sap, M., Bisk, Y., Neubig, G., & Zhu, H. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.698 · Cited by 147
Efficient Attentions for Long Document Summarization
Huang, L., Cao, S., Parulian, N., Ji, H., & Wang, L. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.112 · Cited by 146
Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
Qian, C., He, B., Zhuang, Z., Deng, J., Qin, Y., Cong, X., Zhang, Z., Zhou, J., Lin, Y., Liu, Z., et al. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.61 · Cited by 139
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
Dasigi, P., Lo, K., Beltagy, I., Cohan, A., Smith, N. A., & Gardner, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.365 · Cited by 138
Xia, Y., Xiao, Z., Jazdi, N., & Weyrich, M. (2024) · IEEE Access · doi:10.1109/access.2024.3415470 · Cited by 129 · cited 3×
“These interactive benchmarks are essential for fostering the growth of more capable, reliable, and aligned LLM agents.”
ChemCrow: Augmenting large-language models with chemistry tools
Bran, A. M., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2023) · arXiv · doi:10.48550/arxiv.2304.05376 · Cited by 122
Yang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120 · cited 3×
“Future research should enhance LLM agents’ scalability, adaptability, and real-time operational capacity, for deployment in multimodal and domain-specific environments [72,77,85,102].”
WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Luo, Z., Xu, C., Zhao, P., Sun, Q., Geng, X., Hu, W., Tao, C., Ma, J., Lin, Q., & Jiang, D. (2023) · arXiv · doi:10.48550/arxiv.2306.08568 · Cited by 120
PrivacyAsst: Safeguarding User Privacy in Tool-Using Large Language Model Agents
Zhang, X., Xu, H., Ba, Z., Wang, Z., Hong, Y., Liu, J., Qin, Z., & Ren, K. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3372777 · Cited by 116 · cited 10×
“The Claude model family, developed by Anthropic, has emerged as a key player in the proprietary LLM domain and is often considered as a primary competitor to OpenAI models [23, 40, 59, 90].”
InfluentialOn the creativity of large language models
Franceschelli, G., & Musolesi, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02127-3 · Cited by 113 · cited 1×
“First, they demonstrate autonomy by performing tasks without granular instructions, adapting responses to input, and generating creative content independently [132–134].”
The benefits and dangers of anthropomorphic conversational agents
Peter, S., Riemer, K., & West, J. D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415898122 · Cited by 110 · cited 1×
“Their humanlike interactions create deception and encourage vulnerable populations, such as children or the elderly, to disclose sensitive information [215].”
Ghafarollahi, A., & Buehler, M. J. (2024) · Digital Discovery · doi:10.1039/d4dd00013g · Cited by 106
LLM-Based Multi-Agent Decision-Making: Challenges and Future Directions
Sun, C., Huang, S., & Pompili, D. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3562371 · Cited by 102 · cited 1×
“However, they often struggle in dynamic environments that require simultaneous context tracking, ex-ternal memory integration, and adaptive tool usage [16,17].”
RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models
Wang, Z., Liu, Z., Zhang, Y., Zhong, A., Wang, J., Yin, F., Fan, L., Wu, L., & Wen, Q. (2024) · Proceedings of the 33rd ACM International Conference on Information and Knowledge Management · doi:10.1145/3627673.3680016 · Cited by 99 · cited 5×
“Conversation-centric agents often adopt Vicuna-13B or the larger Wizard-Vicuna-Uncensored 30B [52,87,94,106,124,129,130], while the instruction-tuned WizardLM 70B serves as a high-capacity but fully open chat baseline [69].”
InfluentialLarge Language Model-based Human-Agent Collaboration for Complex Task Solving
Feng, X., Chen, Z., Qin, Y., Lin, Y., Chen, X., Liu, Z., & Wen, J.-R. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.72 · Cited by 98
Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations
Huang, X., Lian, J., Lei, Y., Yao, J., Lian, D., & Xie, X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3731446 · Cited by 89 · cited 7×
“Agents simulate realistic user behavior for testing [102] and provide interactive recommendation dialogues [84], capturing evolving user intent with minimal supervision.”
InfluentialRecommender AI Agent: Integrating Large Language Models for Interactive Recommendations
Huang, X., Lian, J., Lei, Y., Yao, J., Lian, D., & Xie, X. (2023) · arXiv · doi:10.48550/arxiv.2308.16505 · Cited by 89
BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization
Kryscinski, W., Rajani, N., Agarwal, D., Xiong, C., & Radev, D. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.488 · Cited by 88
Benchmarking Data Science Agents
Zhang, Y., Jiang, Q., XingyuHan, X., Chen, N., Yang, Y., & Ren, K. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.308 · Cited by 82
Theory of Mind for Multi-Agent Collaboration via Large Language Models
Li, H., Chong, Y. Q., Stepputtis, S., Campbell, J., Hughes, D., Lewis, M., & Sycara, K. (2023) · arXiv · doi:10.48550/arxiv.2310.10701 · Cited by 82
ChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.07924 · Cited by 81
Theory of Mind for Multi-Agent Collaboration via Large Language Models
Li, H., Chong, Y., Stepputtis, S., Campbell, J., Hughes, D., Lewis, C., & Sycara, K. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.13 · Cited by 79 · cited 9×
“The evaluation of LLM-based agents is quickly changing from basic accuracy checks to more complex methods that look at task performance, reasoning quality, and interaction dynamics [23,93,100].”
InfluentialLarge‐Language‐Model‐Based AI Agent for Organic Semiconductor Device Research
Zhang, Q., Hu, Y., Yan, J., Zhang, H., Xie, X., Zhu, J., Li, H., Niu, X., Li, L., Sun, Y., et al. (2024) · Advanced Materials · doi:10.1002/adma.202405163 · Cited by 72 · cited 1×
“Agents can benchmark themselves against human researchers in AI tasks [86], conduct chemistry experiments autonomously [67], or aid in semiconductor and biological sequence analysis [130, 144].”
Meng, L., Ho, B., Ren, D., & Wang, X. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.329 · Cited by 72 · cited 8×
“Applications also include triage decision support [37], scenario-based medical training [80], and co-dialogue agents for clinical support [74].”
InfluentialEmergent autonomous scientific research capabilities of large language models
Boiko, D. A., MacKnight, R., & Gomes, G. (2023) · arXiv · doi:10.48550/arxiv.2304.05332 · Cited by 72
Personal Large Language Model Agents: A Case Study on Tailored Travel Planning
Singh, H., Verma, N., Wang, Y., Bharadwaj, M., Fashandi, H., Ferreira, K., & Chul, L. E. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-industry.37 · Cited by 71 · cited 6×
“Robotics and navigation datasets, for instance Street View [55] and AndroidArena [109] often lack diversity in user behavior, scene variation, and environmental dynamics, resulting in biased models with limited robustness [64, 113].”
InfluentialFrering, L., Steinbauer-Wagner, G., & Holzinger, A. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109771 · Cited by 69 · cited 7×
“Integrating BDI architectures [99], visual grounding [56,99], and few-shot learning for robot task planning [114], these agents demonstrate their ability to translate high-level instructions into actions in physical or simulated environments.”
InfluentialEvaluating large language models and agents in healthcare: key challenges in clinical applications
Chen, X., Xiang, J., Lu, S., Liu, Y., He, M., & Shi, D. (2025) · Intelligent Medicine · doi:10.1016/j.imed.2025.03.002 · Cited by 68 · cited 1×
“…workflows where they perform planning, decision making, and tool interaction in various real-world applications, including research assistance [6], software development [7], drug discoveries [8], multi-robot systems [9], clinical support [10], game simulation [11] and scientific simulations [12].”
Xue, X., Lu, Z., Huang, D., Wang, Z., Ouyang, W., & Bai, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02292 · Cited by 65 · cited 2×
“By addressing the critical issue of coordinating overlapping tasks with limited resources, it enables the evaluation and improvement of language agents’ multitasking and temporal reasoning capabilities.”
From prompt to platform: an agentic AI workflow for healthcare simulation scenario design
Barra, F. L., Rodella, G., Costa, A., Scalogna, A., Carenzo, L., Monzani, A., & Corte, F. D. (2025) · Advances in Simulation · doi:10.1186/s41077-025-00357-z · Cited by 64 · cited 10×
“Applications also include triage decision support [37], scenario-based medical training [80], and co-dialogue agents for clinical support [74].”
InfluentialLiu, W., Li, J., Tang, Y., Zhao, Y., Liu, C., Song, M., Ju, Z., Kumar, S. V., Lu, Y., Akbani, R., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-57430-4 · Cited by 63
LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations
Tony, C., Mutas, M., Ferreyra, N. E. D., & Scandariato, R. (2023) · 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr59073.2023.00084 · Cited by 61
Monaco SUMO Traffic (MoST) Scenario: A 3D Mobility Scenario for Cooperative ITS
Codecà, L., & Härri, J. (2018) · EPiC series in engineering · doi:10.29007/1zt5 · Cited by 61
EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities
Li, N., Gao, C., Li, M., Li, Y., & Liao, Q. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.829 · Cited by 59 · cited 3×
“Methods also explore emerging social behaviors, such as conformity and reaching consensus in agent societies, or the ability to simulate human trust behaviors in economic games [19,81].”
Agent Laboratory: Using LLM Agents as Research Assistants
Schmidgall, S., Su, Y., Wang, Z., Sun, X., Wu, J., Yu, X., Liu, J., Moor, M., Liu, Z., & Barsoum, E. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.320 · Cited by 59 · cited 1×
“…workflows where they perform planning, decision making, and tool interaction in various real-world applications, including research assistance [6], software development [7], drug discoveries [8], multi-robot systems [9], clinical support [10], game simulation [11] and scientific simulations…”
Wang, Z., Cai, S., Chen, G., Liu, A., Ma, X., & Liang, Y. (2023) · arXiv · doi:10.48550/arxiv.2302.01560 · Cited by 55
Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View
Zhang, J., Xu, X., Zhang, N., Liu, R., Hooi, B., & Deng, S. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.782 · Cited by 53 · cited 10×
“In addition, collaborative approaches such as Multi-Agent Collaborative Reasoning [19, 37, 66, 74] allow interactions between multiple agents to debate and refine conclusions.”
InfluentialConversational health agents: a personalized large language model-powered agent framework
Abbasian, M., Azimi, I., Rahmani, A. M., & Jain, R. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf067 · Cited by 52 · cited 2×
“These methods help developers shape the agent’s behavior, give it expertise in specific areas, and prepare it for complex long-term tasks [66,75,79].”
A-Mem: Agentic Memory for LLM Agents
Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., & Zhang, Y. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-0593 · Cited by 51 · cited 1×
“The profiling module defines an agent’s operational persona (e.g., developer, advisor, or task-specific role), conditioning its behavior and role policy through static profiles defined by experts or dynamic generative mechanisms [21, 138].”
Towards Autonomous Testing Agents via Conversational Large Language Models
Feldt, R., Kang, S., Yoon, J., & Yoo, S. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00148 · Cited by 49 · cited 1×
“They improve code generation [39], perform automated software testing [115], and diagnose complex failures in cloud systems through the use of integrated tools [52].”
Large language model agents can use tools to perform clinical calculations
Goodell, A. J., Chu, S. N., Rouholiman, D., & Chu, L. F. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01475-8 · Cited by 48 · cited 7×
“Other studies assess agents in their ability to execute clinical calculations using established medical calculators or make complex oncology decisions based on multimodal patient data [53,73].”
InfluentialGestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents
Zeng, X., Wang, X., Zhang, T., Yu, C., Zhao, S., & Chen, Y. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3698145 · Cited by 48 · cited 4×
“Future research should enhance LLM agents’ scalability, adaptability, and real-time operational capacity, for deployment in multimodal and domain-specific environments [72,77,85,102].”
InfluentialChain of Agents: Large Language Models Collaborating on Long-Context Tasks
Zhang, Y., Sun, R., Chen, Y., Pfister, T., Zhang, R., & Arik, S. Ö. (2024) · arXiv · doi:10.48550/arxiv.2406.02818 · Cited by 47 · cited 9×
“The Claude model family, developed by Anthropic, has emerged as a key player in the proprietary LLM domain and is often considered as a primary competitor to OpenAI models [23, 40, 59, 90].”
InfluentialA Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health
Behari, N., Zhang, E., Zhao, Y., Taneja, A., Raj, A., & Tambe, M. (2024) · neural information processing systems · doi:10.52202/079017-0130 · Cited by 44
Language Agents: Foundations, Prospects, and Risks
Su, Y., Yang, D., Yao, S., & Yu, T. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-tutorials.3 · Cited by 42 · cited 2×
“The architecture and functionality of an LLM agent are fundamentally related to its underlying language model [10, 32, 123].”
Muthusamy, V., Rizk, Y., Kate, K., Venkateswaran, P., Isahagian, V., Gulati, A., & Dube, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.461 · Cited by 40 · cited 2×
“…behavior elicitation ultra-long context, safety-first alignment, strong instruction following Google DeepMind [38,40,80,87,90, 97,101,103,105,121] Gemini 1.0 Pro, Gemini 1.5 Pro, Gem-ini Pro, PaLM 2, FLAN-T5 Agent development, distraction reasoning, comparative evaluation, enterprise agent…”
InfluentialAgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
Chen, W., Su, Y., Zuo, J., Yang, C., Yuan, C., Chan, C.-M., Yu, H., Lu, Y., Hung, Y.-H., Qian, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.10848 · Cited by 39
Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models
Li, L., Wang, Y., Xu, R., Wang, P., Feng, X., Kong, L., & Liu, Q. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.775 · Cited by 36
Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data Entry
Cuadra, A., Breuch, J., Estrada, S., Ihim, D., Hung, I., Askaryar, D., Hassanien, M., Fessele, K. L., & Landay, J. A. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3659624 · Cited by 35 · cited 2×
“Healthcare dialogue benefits from memory buffers [110], while VLN-Imagine augments vision transformer memory with generated images [147].”
Jin, A., Ye, Y., Lee, B., & Qiao, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3481641 · Cited by 33 · cited 7×
“Temporal memory tools include prompt context tracking in ToolEmu [106] and compressed Research Logs in MLA-gentBench [112].”
InfluentialLLM Multi-Agent Systems: Challenges and Open Problems
Han, S., Zhang, Q., Jin, W., & Xu, Z. (2024) · arXiv · doi:10.48550/arxiv.2402.03578 · Cited by 33 · cited 1×
“Through structured communication, reflective reasoning, and explicit role assignments in simulated settings, multi-agent LLMs exhibit capabilities such as consensus building, uncertainty-aware planning, and autonomous tool interaction [18–20].”
CochlScene: Acquisition of acoustic scene data using crowdsourcing
Jeong, I.-Y., & Park, J. (2022) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9979822 · Cited by 30
TimeArena: Shaping Efficient Multitasking Language Agents in a Time-Aware Simulation
Zhang, Y., Yuan, S., Hu, C., Richardson, K., Xiao, Y., & Chen, J. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.215 · Cited by 27
Huang, H., Shi, X., Lei, H., Hu, F., & Cai, Y. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01345 · Cited by 24 · cited 4×
“Specific systems apply short-term memory optimization techniques for immediate reactivity tasks to focus on rapid access to updated data while minimizing overhead [70,75,78,107].”
InfluentialTravelPlanner: A Benchmark for Real-World Planning with Language Agents
Xie, J., Zhang, K., Chen, J., Zhu, T., Lou, R., Tian, Y., Xiao, Y., & Su, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.01622 · Cited by 22
Large Language Model-based Test Case Generation for GP Agents
Jorgensen, S., Nadizar, G., Pietropolli, G., Manzoni, L., Medvet, E., O’Reilly, U.-M., & Hemberg, E. (2024) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3638529.3654056 · Cited by 21 · cited 3×
“Interactive environments used in agentic research include virtual platforms like game worlds such as Minecraft (via MineDojo) [56, 118], the MarioAI platform [46], and text-based simulations such as ALFWorld [13].”
InfluentialDo Visual Imaginations Improve Vision-and-Language Navigation Agents?
Perincherry, A., Krantz, J., & Lee, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00364 · Cited by 21 · cited 2×
“A range of reasoning strategies has been designed for single-agent settings where an agent acts, plans, and self-assesses autonomously.”
FG-PRM: Fine-grained Hallucination Detection and Mitigation in Language Model Mathematical Reasoning
Li, R., Luo, Z., & Du, X. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.228 · Cited by 21
Understanding the Weakness of Large Language Model Agents within a Complex Android Environment
Xing, M., Zhang, R., Xue, H., Chen, Q., Yang, F., & Xiao, Z. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671650 · Cited by 19 · cited 9×
“These agents operate autonomously in mobile and distributed environments, even as the limitations in edge cases are explored [109], showcasing their adaptability and evolving maturity.”
InfluentialWang, X., Jiang, H., Yu, Y., Yu, J., Lin, Y., Yi, P., Wang, Y., Qiao, Y., Li, L., & Wang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11222-w · Cited by 18 · cited 1×
“Large language models (LLMs) have become central in artificial intelligence (AI) research due to their strong human-like ability to understand, generate, and reason in natural language [1,2].”
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Yin, D., Brahman, F., Ravichander, A., Chandu, K., Chang, K.-W., Choi, Y., & Lin, B. Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.670 · Cited by 17 · cited 8×
“5 in 23 studies, establishing it as a performance benchmark despite cost and access restrictions.”
InfluentialAbdaljalil, S., Kurban, H., Qaraqe, K. A., & Serpedin, E. (2025) · Workshop on Towards Knowledgeable Foundation Models (KnowFM) · doi:10.18653/v1/2025.knowllm-1.10 · Cited by 16
ChatNT: A Multimodal Conversational Agent for DNA, RNA and Protein TasksPreprint
Richard, G., de Almeida, B. P., Dalla-Torre, H., Blum, C. F., Hexemer, L., Pandey, P., Laurent, S., Lopez, M., Laterre, A., Lang, M., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.04.30.591835 · Cited by 14
GAIA: a benchmark for General AI Assistants
Mialon, G., Fourrier, C., Swift, C., Wolf, T., LeCun, Y., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2311.12983 · Cited by 12
Agent-in-the-loop to distill expert knowledge into artificial intelligence models: a survey
Gao, J., Zhang, Y., Chen, Y., Dong, Y., Chen, Y., Song, S., Tang, B., & Gu, Y. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11255-1 · Cited by 10 · cited 1×
“However, they often struggle in dynamic environments that require simultaneous context tracking, ex-ternal memory integration, and adaptive tool usage [16,17].”
WildChat: 1M ChatGPT Interaction Logs in the Wild
Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., & Deng, Y. (2024) · arXiv · doi:10.48550/arxiv.2405.01470 · Cited by 10
Balancing Minds and Data: The Privacy Dilemma of LLMs and Anthropomorphism in LLMs
Meier, R. L. (2025) · Journal of Social Computing · doi:10.23919/jsc.2025.0014 · Cited by 8 · cited 1×
“Since LLM-based agents lack moral agency, attributing accountability is conceptually flawed [216].”
Can a Large Language Model Keep My Secrets? A Study on LLM-Controlled Agents
Hemken, N., Koneru, S., Jacob, F., Hartenstein, H., & Niehues, J. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-srw.49 · Cited by 6 · cited 2×
“With the advent of LLaMA 3 [64, 76, 83, 126], this foundation has grown significantly: its 8B [23,48,51,64] and 70B [51,53,91] instruction-tuned models are now integral to planning, tool-use, and multi-agent benchmarks.”
EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction
Yuan, S., Song, K., Chen, J., Tan, X., Shen, Y., Kan, R., Li, D., & Yang, D. (2024) · arXiv · doi:10.48550/arxiv.2401.06201 · Cited by 4
Self-alignment of large language models via monopolylogue-based social scene simulation
Pang, X., Tang, S., Ye, R., Xiong, Y., Zhang, B., Wang, Y., & Chen, S. (2024) · arXiv · doi:10.48550/arxiv.2402.05699 · Cited by 4
George, S., Sypherd, C., & Cashman, D. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.905 · Cited by 3 · cited 7×
“…R&D comparison, behavior elicitation ultra-long context, safety-first alignment, strong instruction following Google DeepMind [38,40,80,87,90, 97,101,103,105,121] Gemini 1.0 Pro, Gemini 1.5 Pro, Gem-ini Pro, PaLM 2, FLAN-T5 Agent development, distraction reasoning, comparative evaluation,…”
InfluentialALYMPICS: LLM Agents Meet Game Theory -- Exploring Strategic Decision-Making with AI Agents
Mao, S., Cai, Y., Xia, Y., Wu, W., Wang, X., Wang, F., Ge, T., & Wei, F. (2023) · arXiv · doi:10.48550/arxiv.2311.03220 · Cited by 3
Yim, W.-W., Fu, Y., Ben Abacha, A., Snider, N., lin, T., & Yetisgen, M. (2023) · figshare · doi:10.6084/m9.figshare.22494601 · Cited by 2
Saluki: Predicting mRNA half-life from mRNA sequence
Agarwal, V., & Kelley, D. R. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.6326408 · Cited by 2
On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents
Huang, J.-T., Zhou, J., Jin, T., Zhou, X., Chen, Z., Wang, W., Yuan, Y., Lyu, M. R., & Sap, M. (2024) · arXiv · doi:10.48550/arxiv.2408.00989 · Cited by 1 · cited 8×
“This shift has expanded new frontiers in diverse domains where collective intelligence offers advantages in robustness, scalability, and adaptability [25,34,35,41,91].”
Influential
Research works printed but not linked: 93
- Arumugam D, Griffiths TL (2025) Toward efficient exploration by large language model agents. In: The exploration in AI today workshop at ICML 2025
- Austin J, Odena A, Nye M, Bosma M, Michalewski H et al (2021) Program synthesis with large language models. arXiv preprint arXiv:2108.07732 · arXiv
- Bai Y, Jones A, Ndousse K, Askell A, Chen A et al (2022) Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 · arXiv
- Bai X, Huang S, Wei C, Wang R (2025) Collaboration between intelligent agents and large language models: a novel approach for enhancing code generation capability. Expert Syst Appl 269:126357
- Bhardwaj R, Poria S (2023) Red-teaming large language models using chain of utterances for safety-alignment. arXiv preprint arXiv:2308.09662 · arXiv
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
- Cai Y, Li X, Goswami M, Wiliński M, Welter G, Dubrawski A (2025) LLM agents struggle at time series machine learning engineering. In: 1st ICML Workshop on foundation models for structured data
- Chang M, Zhang J, Zhu Z, Yang C, Yang Y, Jin Y, Lan Z, Kong L, He J (2024) Agentboard: An analytical evaluation board of multi-turn llm agents. Adv Neural Inf Process Syst 37:74325–74362
- Chen M, Tworek J, Jun H, Yuan Q et al (2021) Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 · arXiv
- Chen J, Yu C, Zhou X, Xu T, Mu Y, et al (2025c) Emos: Embodiment-aware heterogeneous multi-robot operating system with llm agents. arXiv preprint arXiv:2410.22662v2 · arXiv
- Cheng J, Chin P (2024) Sociodojo: Building lifelong analytical agents with real-world text and time series. In: The twelfth international conference on learning representations
- Cheng Y, Zhang C, Zhang Z, Meng X, Hong S, et al (2024) Exploring large language model based intelligent agents: definitions, methods, and prospects. arXiv preprint arXiv:2401.03428 · arXiv
- Cobbe K, Kosaraju V, Bavarian M, Chen M, Jun H et al (2021) Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 · arXiv
- Dasari S, Ebert F, Tian S, Nair S, Bucher B et al (2019) Robonet: Large-scale multi-robot learning. arXiv preprint arXiv:1910.11215 · arXiv
- Deng X, Gu Y, Zheng B, Chen S, Stevens S et al (2023) Mind2web: Towards a generalist agent for the web. Adv Neural Inf Process Syst 36:28091–28114
- Driess D, Xia F, Sajjadi MSM, Lynch C, Chowdhery A et al (2023) Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378 · arXiv
- Ferrag MA, Tihanyi N, Debbah M (2025) From llm reasoning to autonomous ai agents: a comprehensive review. arXiv preprint arXiv:2504.19678 · DOI · Paper page
- Fu Y, Kim D-K, Kim J, Sohn S, Logeswaran L et al (2024) Autoguide: Automated generation and selection of context-aware guidelines for large language model agents. Adv Neural Inf Process Syst 37:119919–119948
- Hendrycks D, Burns C, Basart S, Zou A, Mazeika M et al (2021) Measuring massive multitask language understanding. In: Proceedings of the international conference on learning representations (ICLR)
- Hong S, Zhuge M, Chen J, Zheng X, Cheng Y et al (2023) Metagpt: Meta programming for a multi-agent collaborative framework. In: The twelfth international conference on learning representations
- Huang Q, Vora J, Liang P, Leskovec J (2023) Benchmarking large language models as ai research agents. In: NeurIPS 2023 foundation models for decision making workshop
- Ji J, Liu M, Dai J, Pan X, Zhang C et al (2023) Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Adv Neural Inf Process Syst 36:24678–24704
- Jin Y, Ma J (2024) Large language model as parking planning agent in the context of mixed period of autonomous vehicles and human-driven vehicles. Sustain Cities Soc 117:105940
- Kang J, Laroche R, Yuan X, Trischler A, Liu X, Fu J (2024) Think before you act: Decision transformers with working memory. arXiv preprint arXiv:2305.16338v3 · arXiv
- Kapitanov A, Kvanchiani K, xNagaev K, Kraynov R, Makhliarchuk A (2024) Hagrid – hand gesture recognition image dataset. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision (WACV), pp 4572–4581
- Khan MA, Amani M, Das S, Ghosh B, Wu Q et al (2025) In agents we trust, but who do agents trust? latent source preferences steer LLM generations. In: ICML 2025 workshop on reliable and responsible foundation models
- Kim Y, Park C, Jeong H, Chan YS, Xu X et al (2024) Mdagents: An adaptive collaboration of llms for medical decision-making. Adv Neural Inf Process Syst 37:79410–79452
- Klein LH, Potamitis N, Aydin R, West R, Gulcehre C, Arora A (2025) Fleet of agents: coordinated problem solving with large language models. In: Forty-second international conference on machine learning
- Köpf A, Kilcher Y, Von Rütte D, Anagnostidis S, Tam ZR et al (2023) Openassistant conversations-democratizing large language model alignment. Adv Neural Inf Process Syst 36:47669–47681
- Lampinen A, Chan S, Dasgupta I, Nam A, Wang J (2023) Passive learning of active causal strategies in agents and language models. Adv Neural Inf Process Syst 36:1283–1297
- Li X (2025) A review of prominent paradigms for llm-based agents: tool use, planning (including rag), and feedback learning. In: Proceedings of the 31st international conference on computational linguistics, pp 9760–9779
- Li XL, Chowdhury N, Johnson DD, Hashimoto T, Liang P, et al (2025a) Eliciting language model behaviors with investigator agents. In: Forty-second international conference on machine learning
- Li Z, Zhang R, Wang Z, Xie Z, Song Y (2025b) Llm-guided decision-making toolkit for multi-agent reinforcement learning. Neurocomputing 638:130105
- Liu T, Xu C, McAuley J (2023) Repobench: Benchmarking repository-level code auto-completion systems. arXiv preprint arXiv:2306.03091 · arXiv
- Liu Z, Hu H, Zhang S, Guo H, Ke S, et al (2024a) Reason for future, act for now: a principled architecture for autonomous LLM agents. In: Proceedings of the 41st international conference on machine learning, ser. Proceedings of Machine Learning Research, vol. 235. PMLR, pp 31 186–31 261
- Liu R, Yang R, Jia C, Zhang G, Yang D, Vosoughi S (2024b) Training socially aligned language models on simulated social interactions. In: The twelfth international conference on learning representations
- Liu X, Yu H, Zhang H, Xu Y, Lei X et al (2024c) Agentbench: Evaluating LLMs as agents. In: The twelfth international conference on learning representations
- Liu X, ZHANG J, Shang H, Guo S, Chengxu Y, Zhu Q (2025b) Exploring prosocial irrationality for LLM agents: a social cognition view. In: The thirteenth international conference on learning representations
- Lu P, Qiu L, Chang KW, Wu YN, Zhu SC et al (2023) Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. In: International conference on learning representations (ICLR)
- Luo J, Zhang W, Yuan Y, Zhao Y, Yang J et al (2025) Large language model agent: a survey on methodology, applications and challenges. arXiv preprint arXiv:2503.21460 · arXiv
- Ma H, Hu T, Pu Z, Boyin L, Ai X et al (2024) Coevolving with the other you: fine-tuning LLM with sequential cooperative multi-agent reinforcement learning. Adv Neural Inf Process Syst 37:15497–15525
- Ni B, Buehler MJ (2024) Mechagents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledge. Extreme Mech Lett 67:102131
- Ning Y, Liu H (2024) Urbankgent: A unified large language model agent framework for urban knowledge graph construction. Adv Neural Inf Process Syst 37:123127–123154
- Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C et al (2022) Training language models to follow instructions with human feedback. Adv Neural Inf Process Syst 35:27730–27744
- Pang J-C, Yang S-H, Li K, Zhang J, Chen X-H et al (2024a) Kalm: Knowledgeable agents by offline reinforcement learning from large language model rollouts. Adv Neural Inf Process Syst 37:126620–126652
- Penedo G, Kydlíček H, allal LB, Lozhkov A, Mitchell M et al (2024) The fineweb datasets: Decanting the web for the finest text data at scale. In: The thirty-eight conference on neural information processing systems datasets and benchmarks track
- Qiao S, Fang R, Zhang N, Zhu Y, Chen X et al (2024a) Agent planning with world knowledge model. Adv Neural Inf Process Syst 37:114843–114871
- Qin Y, Liang S, Ye Y, Zhu K, Yan L et al (2024b) ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In: The twelfth international conference on learning representations
- Qu Y, Zhang T, Garg N, Kumar A (2024) Recursive introspection: teaching language model agents how to self-improve. Adv Neural Inf Process Syst 37:55249–55285
- Ruan Y, Dong H, Wang A, Pitis S, Zhou Y et al (2024) Identifying the risks of LM agents with an LM-emulated sandbox. In: The twelfth international conference on learning representations
- Schick T, Dwivedi-Yu J, Dessì R, Raileanu R, Lomeli M et al (2023) Toolformer: Language models can teach themselves to use tools. Adv Neural Inf Process Syst 36:68539–68551
- Schumann R, Zhu W, Feng W, Fu T-J, Riezler S, Wang WY (2024) Velma: Verbalization embodiment of llm agents for vision and language navigation in street view. Proc AAAI Conf Artif Intell 38(17):18924–18933
- Shinn N, Cassano F, Gopinath A, Narasimhan K, Yao S (2023) Reflexion: Language agents with verbal reinforcement learning. Adv Neural Inf Process Syst 36:8634–8652
- Shi F, Suzgun M, Freitag M, Wang X, Srivats S et al (2022) Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057 · arXiv
- Shridhar M, Yuan X, Côté MA, Bisk Y, Trischler A, Hausknecht M (2021) ALFWorld: aligning text and embodied environments for interactive learning. In: Proceedings of the international conference on learning representations (ICLR)
- Talebirad Y, Nadiri A (2023) Multi-agent collaboration: Harnessing the power of intelligent llm agents. arXiv preprint arXiv:2306.03314 · arXiv
- Tennant E, Hailes S, Musolesi M (2025) Moral alignment for LLM agents. In: The thirteenth international conference on learning representations
- Valmeekam K, Marquez M, Olmo A, Sreedharan S, Kambhampati S (2023) Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Adv Neural Inf Process Syst 36:38975–38987
- Walke HR, Black K, Zhao TZ, Vuong Q, Zheng C et al (2023) Bridgedata v2: A dataset for robot learning at scale. In: Conference on robot learning. PMLR, pp 1723–1736
- Wang P, Li L, Shao Z, Xu R, Dai D et al (2023b) Math-shepherd: Verify and reinforce llms step-by-step without human annotations. arXiv preprint arXiv:2312.08935v3 · arXiv
- Wang G, Xie Y, Jiang Y, Mandlekar A, Xiao C et al (2023c) Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 · arXiv
- Wang Z, Zhang G, Yang K, Shi N, Zhou W et al (2023d) Interactive natural language processing. arXiv preprint arXiv:2305.13246 · arXiv
- Wang Z, Cai S, Mu Z, Lin H, Zhang C et al (2024a) Omnijarvis: Unified vision-language-action tokenization enables open-world instruction following agents. Adv Neural Inf Process Syst 37:73278–73308
- Wang L, Ma C, Feng X et al (2024b) A survey on large language model based autonomous agents. Front Comp Sci 18:186345
- Wang L, Zhang J, Yang H, Chen Z-Y, Tang J et al (2025b) User behavior simulation with large language model-based agents. ACM Trans Inform Syst 43(2):1–37
- Wang Z, Zhu Y, Zhao H, Zheng X, Sui D et al (2025c) Colacare: Enhancing electronic health record modeling through large language model-driven multi-agent collaboration. Proc ACM Web Conf 2025:2250–2261
- Weber M, Fu DY, Anthony Q, Oren Y, Adams S et al (2024) Redpajama: an open dataset for training large language models. NeurIPS Datasets and Benchmarks Track
- Wei J, Wang X, Schuurmans D, Bosma M, Ichter B et al (2022) Chain-of-thought prompting elicits reasoning in large language models. In: Proceedings of the 36th international conference on neural information processing systems, ser. NIPS ’22. Red Hook, NY, USA: Curran Associates Inc
- Wijk H, Lin T, Becker J, Jawhar S, Parikh N et al (2025) RE-bench: Evaluating frontier AI r &d capabilities of language model agents against human experts. In: Forty-second international conference on machine learning
- Wu S, Zhao S, Huang Q, Huang K, Yasunaga M et al (2024b) Avatar: Optimizing llm agents for tool usage via contrastive reasoning. Adv Neural Inf Process Syst 37:25981–26010
- Wu Q, Bansal G, Zhang J, Wu Y, Li B et al (2024c) Autogen: Enabling next-gen LLM applications via multi-agent conversations. In: First conference on language modeling
- Wu S, Zhao S, Yasunaga M, Huang K, Cao K et al (2024d) Stark: Benchmarking llm retrieval on textual and relational knowledge bases. In: NeurIPS datasets and benchmarks track
- Xi Z, Chen W, Guo X et al (2025) The rise and potential of large language model based agents: a survey. Sci China Inf Sci 68:121101
- Xia Y, Shen W, Wang Y, Liu JK, Sun H et al (2025) Leetcodedataset: A temporal dataset for robust evaluation and efficient training of code llms. arXiv preprint arXiv:2504.14655 · arXiv
- Xiang Z, Zheng L, Li Y, Hong J, Li Q et al (2025) Guardagent: safeguard LLM agents via knowledge-enabled reasoning. In: ICML 2025 workshop on computer use agents
- Xie C, Chen C, Jia F, Ye Z, Lai S et al (2024a) Can large language model agents simulate human trust behavior? Adv Neural Inf Process Syst 37:15674–15729
- Xu Y, Su H, Xing C, Mi B, Liu Q et al (2024b) Lemur: Harmonizing natural language and code for language agents. In: The twelfth international conference on learning representations
- Yang J, Prabhakar A, Narasimhan K, Yao S (2023) Intercode: Standardizing and benchmarking interactive coding with execution feedback. Adv Neural Inf Process Syst 36:23826–23854
- Yang W, Bi X, Lin Y, Chen S, Zhou J, Sun X (2024a) Watch out for your agents! investigating backdoor threats to llm-based agents. Adv Neural Inf Process Syst 37:100938–100964
- Yang Y, Zhou T, Li K, Tao D, Li L et al (2024d) Embodied multi-modal agent trained by an llm from a parallel textworld. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 26 275–26 285
- Yao S, Yu D, Zhao J, Shafran I, Griffiths T et al (2023a) Tree of thoughts: deliberate problem solving with large language models. Adv Neural Inf Process Syst 36:11809–11822
- Yao S, Zhao J, Yu D, Du N, Shafran I et al (2023b) React: Synergizing reasoning and acting in language models. In: International conference on learning representations (ICLR)
- Zhai S, Bai H, Lin Z, Pan J, Tong P et al (2024) Fine-tuning large vision-language models as decision-making agents via reinforcement learning. Adv Neural Inf Process Syst 37:110935–110971
- Zhang D, Chen L, Zhang S, Xu H, Zhao Z, Yu K (2023) Large language models are semi-parametric reinforcement learning agents. Adv Neural Inf Process Syst 36:78227–78239
- Zhang C, Yang K, Hu S, Wang Z, Li G et al (2024d) Proagent: building proactive cooperative agents with large language models. Proceedings AAAI Conf Artif Intell 38(16):17591–17599
- Zhang S, Dong Y, Zhang Y, Payne TR, Zhang J (2024e) Large language model assissted multi-agent dialogue for ontology alignment. In: Proceedings of the 23rd international conference on autonomous agents and multiagent systems, pp 2594–2596 · doi:10.65109/pxec2992
- Zhang H, Du W, Shan J, Zhou Q, Du Y et al (2024g) Building cooperative embodied agents modularly with large language models. In: The Twelfth international conference on learning representations
- Zhang S, Zhang J, Liu J, Song L, Wang C et al (2024k) Offline training of language model agents with functions as learnable weights. In: Proceedings of the 41st international conference on machine learning, ser. Proceedings of Machine Learning Research, vol. 235. PMLR, 21–27, pp 60315–60335
- Zhao Q, Wang J, Zhang Y, Jin Y, Zhu K, et al (2024b) CompeteAI: Understanding the competition dynamics of large language model-based agents. In: Proceedings of the 41st international conference on machine learning, ser. Proceedings of Machine Learning Research, vol. 235. PMLR, pp 61 092–61 107
- Zheng S, jiazheng liu, Feng Y, Lu Z (2024) Steve-eye: Equipping LLM-based embodied agents with visual perception in open worlds. In: The Twelfth international conference on learning representations
- Zhou H, Tang Y, Qin H, Yang Y, Jin R et al (2024) Star-agents: Automatic data optimization with llm agents for instruction tuning. Adv Neural Inf Process Syst 37:4575–4597
- Zhu F, Simmons R (2024) Bootstrapping cognitive agents with a large language model. Proc AAAI Conf Artif Intell 38(1):655–663
- Zou A, Wang Z, Carlini N, Nasr M, Kolter JZ, Fredrikson M (2023) Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 · arXiv
Cited by
Hallucination to truth: a review of fact-checking and factuality evaluation in large language models
Rahman, S. S., Islam, M. A., Alam, M. M., Zeba, M., Rahman, M. A., Chowa, S. S., Raiaan, M. A. K., & Azam, S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11454-w · Cited by 39
Self-citationIbrahim, A., Alsukkar, M., Xu, Y., Hermaputi, R. L., Yin, S., & Xiao, Y. (2026) · Energy Sustainable Development/Energy for sustainable development · doi:10.1016/j.esd.2026.102104 · Cited by 11
Toward Sustainable Agentic AI Systems: A Survey of Architectures and Methodologies
Lee, Y., & Park, E. (2026) · Sustainable Development · doi:10.1002/sd.70942 · Cited by 13
Tian, Y., Jiang, Y., Jiang, Y., & Mrzljak, V. (2026) · Buildings · doi:10.3390/buildings16132569 · Cited by 6
A Holistic Review of Agentic AI Frameworks, Applications, and Research Trajectories
Peykani, P., Ghanidel, S., Javadi-Sisi, I., Snåšel, V., & Mirjalili, S. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10675-8 · Cited by 7
Metadata
- Authors
- Chowa, S. S., Alvi, R., Rahman, S. S., Rahman, M. A., Raiaan, M. A. K., Islam, M. R., Hussain, M., & Azam, S.
- Affiliations
- Daffodil International University (BD)
- United International University (BD)
- Monash University (AU)
- Charles Darwin University (AU)
- Corresponding author
- Raiaan, Mohaimenul Azam Khan
- Azam, Sami
- ORCID
- Venue
- Artificial Intelligence Review
- Published
- 2026-01-06
- Volume
- 59
- Issue
- 2
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W7118468219
- Cited by
- 50
- References
- 99
- Citations by year
- 2026: 50
- FWCI
- 169.75
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor, 14 d)
- Locations
- Artificial Intelligence Review (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Charles Darwin University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, autonomous agents, tool use, multi-agent systems, LLM agents, LLM reasoning, LLM memory, prompting, fine-tuning, benchmarking, human development, personalization, verifiable reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.