LLM-Based Agents for Tool Learning: A Survey
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Human beings capable of making and using tools can accomplish tasks far beyond their innate abilities, and this paradigm of integration with tools may not be limited to humans themselves. Recently, the large language model (LLM) has demonstrated immense potential across various fields with its unique planning and reasoning abilities. However, there are still many challenges beyond its capabilities due to deficiencies in its training data and inherent illusions. Thus, integrating LLMs and tools into tool learning agents has become a new emerging research direction. To this end, we present a systematic investigation and comprehensive review of tool-learning agents in this paper. We start by introducing the definition of the tool learning task for Agents and then illustrating the typical architecture of the tool-learning models. Since these tools are all defined by users, LLM does not know what tools there are and what their functions are. Thus, LLMs should first find appropriate tools and split the tool retrieval methods into two categories: training-based and non-training-based. To accurately complete the user task, it is important to decompose the task into several sub-tasks and execute them in the correct order. Following that, we introduce the tool planning methods and organize these works by whether they rely on the model’s inherent reasoning capabilities for planning or utilize external reasoning tools. Due to the rapid development of this field, we also introduce an emerging frontier direction: using multimodal tools for LLM. In addition, we compile current open-source benchmarks and evaluation metrics, focusing on their scale, composition, calculation methods, and assessment dimensions. Next, we introduce several application scenarios for the LLM-based tool learning methods. Finally, we discuss the safety and ethical issues involved in tool learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 1×
“Upon receiving feedback from the environment, LLM processes the information, formalizes it into PDDL language, and then employs a BFS [60] solver to generate a plan.”
A Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · cited 1×
“Building (9) P = p 1 , p 2 , ⋯ , p n = plan( LLM , g ; Θ , T ) ; on this, ToT employs the MCTS algorithm [11] to identify the optimal planning path.”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 1×
“Some people also evaluate the finetuned models on other non-tool-specific general datasets, which require multi-step reasoning, such as mathematical reasoning [23, 87, 97], and Wikipedia(WikiQA) [65, 142].”
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863 · cited 1×
“Specifically, it adopts ImageBind [38] as the primary multimodal encoder, enabling the system to align and interpret inputs across various modalities effectively.”
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556
Research works printed but not linked: 126
- Achiam J, Adler S, Agarwal S et al (2023) Gpt-4 technical report. ArXiv preprint arXiv:2303.08774 · arXiv
- Agarwal A, Chan A, Chandel S et al (2024) Copilot evaluation harness: evaluating llm-guided software programming. ArXiv preprint arXiv:2402.14261 · arXiv
- Ambrose SH (2001) Paleolithic technology and human evolution. Science 291(5509):1748–1753 · DOI · Paper page
- Anantha R, Bandyopadhyay B, Kashi A et al (2023) Protip: progressive tool retrieval improves planning. ArXiv preprint arXiv:2312.10332 · arXiv
- Barwise J (1977) An introduction to first-order logic. In: Studies in logic and the foundations of mathematics, vol 90. Elsevier, pp 5–46 · doi:10.1016/s0049-237x(08)71097-8
- Bhat V, Kaypak AU, Krishnamurthy P et al (2024) Grounding llms for robot task planning using closed-loop state feedback. CoRR · DOI
- Birr T, Pohl C, Younes A, et al (2024) Autogpt+ p: affordance-based task planning with large language models. ArXiv preprint arXiv:2402.10778 · DOI
- Cai T, Wang X, Ma T et al (2024) Large language models as tool makers. In: ICLR
- Chaslot GMJBC (2010) Monte-Carlo tree search. Doctoral Thesis
- Chen L, Shang S (2019) Region-based message exploration over spatio-temporal data streams. In: Proceedings of the AAAI conference on artificial intelligence, pp 873–880 · DOI
- Chen L, Shang S, Jensen CS, et al (2019) Effective and efficient reuse of past travel behavior for route recommendation. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 488–498 · DOI
- Chen L, Lu K, Rajeswaran A et al (2021) Decision transformer: reinforcement learning via sequence modeling. In: Ranzato M, Yann N, Dauphin AB, Liang P et al (eds) Advances in neural information processing systems 34: annual conference on neural information processing systems 2021, NeurIPS 2021, December 6–14, 2021, virtual, pp 15084–15097. https://proceedings.neurips.cc/paper/2021/hash/7f489f642a0ddb10272b5c31057f0663-Abstract.html
- Chen W, Wang X, Wang WY (2023) A dataset for answering time-sensitive questions. In: Thirty-fifth conference on neural information processing systems datasets and benchmarks track (round 2)
- Chen Y, Liu Z, Li J et al (2022) Intent contrastive learning for sequential recommendation. IN: Proceedings of the ACM web conference, pp 2172–2182 · DOI · Paper page
- Chen Z, Du W, Zhang W et al (2023) T-eval: Evaluating the tool utilization capability step by step. ArXiv preprint arXiv:2312.14033 · arXiv
- Chen Z, Zhang D, Feng S et al (2024) Kgts: contrastive trajectory similarity learning over prompt knowledge graph embedding. In: Proceedings of the AAAI conference on artificial intelligence, pp 8311–8319 · DOI
- Cheong M, Abedin E, Ferreira M et al (2024) Investigating gender and racial biases in dall-e mini images. ACM J Responsib Comput 6:66
- Cobbe K, Kosaraju V, Bavarian M et al (2021) Training verifiers to solve math word problems. ArXiv preprint arXiv:2110.14168 · arXiv
- Dagan G, Keller F, Lascarides A (2023) Dynamic planning with a llm. ArXiv preprint arXiv:2308.06391 · arXiv
- Deng S, Xu W, Sun H et al (2024) Mobile-bench: an evaluation benchmark for llm-based mobile agents. In: Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: long papers), pp 8813–8831 · DOI
- Ding T (2024) Mobileagent: enhancing mobile control via human-machine interaction and sop integration. ArXiv preprint arXiv:2401.04124 · arXiv
- Du C, Li Y, Qiu Z et al (2024) Stable diffusion is unstable. Adv Neural Inf Process Syst 36:66
- Du Y, Wei F, Zhang H (2024) Anytool: Self-reflective, hierarchical agents for large-scale api calls. In: International conference on machine learning (PMLR), pp 11812–11829
- Fan T, Kang Y, Ma G et al (2023) Fate-llm: a industrial grade federated learning framework for large language models. arXiv preprint arXiv:2310.10049 · arXiv
- Farn N, Shin R (2023) Tooltalk: evaluating tool-usage in a conversational setting. ArXiv preprint arXiv:2311.10775 · arXiv
- Fu D, Huang J, Lu S et al (2025) Preact: Prediction enhances agent’s planning ability. In: Proceedings of the 31st international conference on computational linguistics, pp 1–16
- Gao S, Shi Z, Zhu M et al (2024) Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum. In: Proceedings of the AAAI conference on artificial intelligence, pp 18030–18038 · DOI
- Gao S, Dwivedi-Yu J, Yu P et al (2025) Efficient tool use with chain-of-abstraction reasoning. In: COLING
- Gao Z, Du Y, Zhang X et al (2024b) Clova: a closed-loop visual assistant with tool usage and update. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13258–13268 · DOI
- Ge Y, Hua W, Mei K et al (2024) Openagi: when llm meets domain experts. Adv Neural Inf Process Syst 36:66
- Gibson KR, Gibson KR, Ingold T (1993) Tools, language and cognition in human evolution. Cambridge University Press, Cambridge
- Gou Z, Shao Z, Gong Y et al (2023) Tora: a tool-integrated reasoning agent for mathematical problem solving. CoRR
- Grande V, Kiesler N, Francisco RMA (2024) Student perspectives on using a large language model (llm) for an assignment on professional ethics. In: Proceedings of the 2024 on innovation and technology in computer science education, vol 1, pp 478–484 · DOI
- Gui A, Li J, Dai Y et al (2024) Look before you leap: towards decision-aware and generalizable tool-usage for large language models. ArXiv preprint arXiv:2402.16696 · arXiv
- Han P, Yang P, Zhao P et al (2019) Gcn-mf: disease-gene association identification by graph convolutional networks and matrix factorization. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 705–713 · DOI
- Hao S, Liu T, Wang Z et al (2023) Toolkengpt: augmenting frozen language models with massive tools via tool embeddings. Adv Neural Inf Process Syst 36:45870–45894
- He J, Chen J, He X et al (2016) Deep reinforcement learning with a natural language action space. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers). Association for Computational Linguistics, Berlin, Germany, pp 1621–1630. https://doi.org/10.18653/v1/P16-1153, https://aclanthology.org/P16-1153 · DOI · Paper page
- Hori K, Suzuki K, Ogata T (2024) Interactively robot action planning with uncertainty analysis and active questioning by large language model. In: 2024 IEEE/SICE international symposium on system integration (SII). IEEE, pp 85–91 · DOI
- Hsieh CY, Chen SA, Li CL et al (2023) Tool documentation enables zero-shot tool-usage with large language models. ArXiv preprint arXiv:2308.00675 · arXiv
- Hu EJ, Shen Y, Wallis P et al (2022) Lora: low-rank adaptation of large language models. In: The tenth international conference on learning representations, ICLR 2022, Virtual Event, April 25–29, 2022. OpenReview.net. https://openreview.net/forum?id=nZeVKeeFYf9
- Huang S, Zhong W, Lu J et al (2024) Planning, creation, usage: benchmarking llms for comprehensive tool utilization in real-world complex scenarios. Find Assoc Comput Linguist ACL 2024:4363–4400
- Huang X, Liu W, Chen X et al (2024) Understanding the planning of llm agents: a survey. CoRR
- Huang Y, Shi J, Li Y et al (2024) Metatool benchmark for large language models: deciding whether to use tools and which to use. In: ICLR
- Imrie F, Rauba P, van der Schaar M (2023) Redefining digital health interfaces with large language models. ArXiv preprint arXiv:2310.03560 · arXiv
- Jiang F, Peng Y, Dong L et al (2024) Large language model enhanced multi-agent systems for 6g communications. IEEE Wirel Commun 6:66
- Jin Q, Wang Z, Yang Y et al (2024) Agentmd: empowering language agents for risk prediction with large-scale clinical tool learning. ArXiv preprint arXiv:2402.13225 · DOI
- Kapania S, Wang R, Li TJJ et al (2024) “ i’m categorizing llm as a productivity tool”: examining ethics of llm use in hci research practices. arXiv preprint arXiv:2403.19876 · DOI
- Kumar A, Singh S, Murty SV et al (2024) The ethics of interaction: Mitigating security threats in llms. arXiv preprint arXiv:2401.12273 · arXiv
- Kumar SS, Jain D, Agarwal E et al (2024) Swissnyf: tool grounded llm agents for black box setting. ArXiv preprint arXiv:2402.10051 · arXiv
- Li J, Ye D, Shang S (2019) Adversarial transfer for named entity boundary detection with pointer networks. In: IJCAI, pp 5053–5059 · DOI
- Liu O, Fu D, Yogatama D et al (2024) Dellma: a framework for decision making under uncertainty with large language models. ArXiv preprint arXiv:2402.02392 · arXiv
- Liu S, Biswal A, Cheng A et al (2024) Optimizing llm queries in relational workloads. CoRR
- Liu S, Cheng H, Liu H et al (2024) Llava-plus: learning to use tools for creating multimodal agents. In: European conference on computer vision. Springer, Berlin, pp 126–142 · DOI
- Liu Y, Deng G, Xu Z et al (2023) Jailbreaking chatgpt via prompt engineering: an empirical study. arXiv preprint arXiv:2305.13860 · arXiv
- Liu Y, Peng X, Zhang Y et al (2024) Tool-planner: dynamic solution tree planning for large language model with tool clustering. arXiv preprint arXiv:2406.03807 · arXiv
- Liu Y, Yuan Y, Wang C et al (2024) From summary to action: Enhancing large language models for complex tasks with open world apis. ArXiv preprint arXiv:2402.18157 · arXiv
- Liu Z, Lai Z, Gao Z et al (2024) Controlllm: augment language models with tools by searching on graphs. In: European conference on computer vision. Springer, Berlin, pp 89–105 · DOI
- Long J (2023) Large language model guided tree-of-thought. ArXiv preprint arXiv:2305.08291 · arXiv
- Lu P, Peng B, Cheng H et al (2024) Chameleon: plug-and-play compositional reasoning with large language models. Adv Neural Inf Process Syst 36:66
- Lykov A, Konenkov M, Gbagbe KF et al (2024) Cognitiveos: large multimodal model based system to endow any type of robot with generative ai. ArXiv preprint arXiv:2401.16205 · DOI
- Lyu B, Cong X, Yu H et al (2023) Gitagent: Facilitating autonomous agent with github by tool extension. ArXiv preprint arXiv:2312.17294 · arXiv
- Ma X, Zhang Z, Zhao H (2024) Coco-agent: a comprehensive cognitive mllm agent for smartphone gui automation. Find Assoc Comput Linguist ACL 2024:9097–9110
- Mekala D, Weston J, Lanchantin J et al (2024) Toolverifier: generalization to new tools via self-verification. Find Assoc Comput Linguist EMNLP 2024:5026–5041
- Michael K, Abbas R, Roussos G et al (2020) Ethics in ai and autonomous system applications design. IEEE Trans Technol Soc 1(3):114–127 · DOI
- Moreira C, Calado P, Martins B (2015) Learning to rank academic experts in the dblp dataset. Expert Syst 32(4):477–493 · DOI
- Nakano R, Hilton J, Balaji S et al (2021) Webgpt: Browser-assisted question-answering with human feedback. ArXiv preprint arXiv:2112.09332 · arXiv
- Patil SG, Zhang T, Wang X et al (2024) Gorilla: large language model connected with massive apis. Adv Neural Inf Process Syst 37:126544–126565
- Pedro R, Castro D, Carreira P et al (2023) From prompt injections to sql injection attacks: How protected is your llm-integrated web application? ArXiv preprint arXiv:2308.01990 · arXiv
- Qian C, Xiong C, Liu Z et al (2024) Toolink: Linking toolkit creation and using through chain-of-solving on open-source model. In: Proceedings of the 2024 conference of the North American Chapter of the Association for Computational Linguistics: human language technologies (volume 1: long papers), pp 831–854 · DOI
- Qin Y, Liang S, Ye Y et al (2024) Toolllm: facilitating large language models to master 16000+ real-world apis. In: ICLR
- Rawles C, Li A, Rodriguez D et al (2023) Androidinthewild: a large-scale dataset for android device control. Adv Neural Inf Process Syst 36:59708–59728
- Roy D, Zhang X, Bhave R et al (2024) Exploring llm-based agents for root cause analysis. In: Companion proceedings of the 32nd ACM international conference on the foundations of software engineering, pp 208–219 · DOI · Paper page
- Ruan J, Chen Y, Zhang B et al (2023) Tptu: task planning and tool usage of large language model-based ai agents. In: NeurIPS 2023 foundation models for decision making workshop
- Ruan Y, Dong H, Wang A et al (2023) Identifying the risks of lm agents with an lm-emulated sandbox. In: NeurIPS 2023 foundation models for decision making workshop
- Schick T, Dwivedi-Yu J, Dessì R et al (2023) Toolformer: language models can teach themselves to use tools. Adv Neural Inf Process Syst 36:68539–68551
- Schick T, Dwivedi-Yu J, Dessì R et al (2024) Toolformer: language models can teach themselves to use tools. Adv Neural Inf Process Syst 36:66
- Shang S, Chen L, Zheng K et al (2018) Parallel trajectory-to-location join. IEEE Trans Knowl Data Eng 31(6):1194–1207 · DOI
- Shen W, Li C, Chen H et al (2024) Small llms are weak tool learners: a multi-llm agent. In: Proceedings of the 2024 conference on empirical methods in natural language processing, pp 16658–16680 · DOI
- Shen Y, Song K, Tan X et al (2024) Hugginggpt: solving ai tasks with chatgpt and its friends in hugging face. Adv Neural Inf Process Syst 36:66
- Shi Z, Gao S, Chen X et al (2024) Learning to use tools via cooperative and interactive agents. Find Assoc Comput Linguist EMNLP 2024:10642–10657
- Silver T, Hariprasad V, Shuttleworth RS et al (2022) Pddl planning with pretrained large language models. In: NeurIPS 2022 foundation models for decision making workshop
- Silver T, Dan S, Srinivas K et al (2024) Generalized planning in pddl domains with pretrained large language models. In: Proceedings of the AAAI conference on artificial intelligence, pp 20256–20264 · DOI
- Skrynnik A, Andreychuk A, Borzilov A et al (2024) Pogema: A benchmark platform for cooperative multi-agent navigation. arXiv preprint arXiv:2407.14931 · arXiv
- Song Y, Xiong W, Zhu D et al (2023) Restgpt: connecting large language models with real-world restful apis. ArXiv preprint arXiv:2306.06624 · arXiv
- Theuma A, Shareghi E (2024) Equipping language models with tool use capability for tabular data analysis in finance. In: Proceedings of the 18th conference of the european chapter of the Association for Computational Linguistics (volume 2: short papers), pp 90–103 · DOI
- Toubal IE, Avinash A, Alldrin NG et al (2024) Modeling collaborator: Enabling subjective vision classification with minimal human effort via llm tool-use. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 17553–17563 · DOI
- Toyama D, Hamel P, Gergely A et al (2021) Androidenv: a reinforcement learning platform for android. ArXiv preprint arXiv:2105.13231 · arXiv
- Wang B, Fang H, Eisner J et al (2024) Llms in the imaginarium: tool learning through simulated trial and error. In: Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: long papers), pp 10583–10604 · DOI
- Wang C, Luo W, Chen Q et al (2024) Tool-lmm: a large multi-modal model for tool agent learning. ArXiv preprint arXiv:2401.10727 · arXiv
- Wang H, Wang H, Wang L et al (2023) Tpe: towards better compositional reasoning over conceptual tools with multi-persona collaboration. ArXiv preprint arXiv:2309.16090 · DOI
- Wang H, Feng S, Chen L et al (2024) Simulating individual infection risk over big trajectory data. In: International conference on database systems for advanced applications. Springer, Berlin, pp 136–151 · DOI
- Wang J, Shi E, Hu H et al (2024) Large language models for robotics: opportunities, challenges, and perspectives. J Autom Intell 6:66
- Wang J, Xu H, Ye J et al (2024) Mobile-agent: autonomous multi-modal mobile device agent with visual perception. In: ICLR 2024 workshop on large language model (LLM) agents
- Wang W, Shi J, Wang C et al (2024) Learning to ask: when llms meet unclear instruction. arXiv:2409.00557 · DOI
- Wang X, Wei J, Schuurmans D et al (2022) Self-consistency improves chain of thought reasoning in language models. ArXiv preprint arXiv:2203.11171 · arXiv
- Wang Y, Wu Z, Yao J et al (2025) Tdag: a multi-agent framework based on dynamic task decomposition and agent generation. Neural Netw 66:107200 · DOI
- Wei J, Wang X, Schuurmans D et al (2022) Chain-of-thought prompting elicits reasoning in large language models. Adv Neural Inf Process Syst 35:24824–24837
- Wei Y, Su Y, Ma H et al (2023) Menatqa: a new dataset for testing the temporal comprehension and reasoning abilities of large language models. In: Findings of the Association for Computational Linguistics: EMNLP 2023, pp 1434–1447 · DOI
- Wu Q, Xu W, Liu W et al (2024) Mobilevlm: a vision-language model for better intra-and inter-ui understanding. arXiv preprint arXiv:2409.14818 · DOI
- Xie J, Chen Z, Zhang R et al (2024) Large multimodal agents: a survey. ArXiv preprint arXiv:2402.15116 · arXiv
- Xu Q, Hong F, Li B et al (2023) On the tool manipulation capability of open-sourced large language models. In: NeurIPS 2023 foundation models for decision making workshop
- Yang C, Chen L, Shang S et al (2019) Toward efficient navigation of massive-scale geo-textual streams. In: IJCAI, pp 4838–4845 · doi:10.24963/ijcai.2019/672
- Yang R, Song L, Li Y et al (2024) Gpt4tools: teaching large language model to use tools via self-instruction. Adv Neural Inf Process Syst 36:66
- Yang Z, Liu J, Han Y et al (2023) Appagent: Multimodal agents as smartphone users. ArXiv preprint arXiv:2312.13771 · arXiv
- Yao S, Yu D, Zhao J et al (2023) Tree of thoughts: deliberate problem solving with large language models. Adv Neural Inf Process Syst 36:11809–11822
- Yao S, Zhao J, Yu D et al (2023) React: synergizing reasoning and acting in language models. In: International conference on learning representations (ICLR)
- Ye J, Li S, Li G et al (2024) Toolsword: Unveiling safety issues of large language models in tool learning across three stages. In: Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: long papers), pp 2181–2211 · DOI
- Ye J, Wu Y, Gao S et al (2024) Rotbench: a multi-level benchmark for evaluating the robustness of large language models in tool learning. In: Proceedings of the 2024 conference on empirical methods in natural language processing, pp 313–333 · DOI
- Ye J, Li G, Gao S et al (2025) Tooleyes: fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios. In: Proceedings of the 31st international conference on computational linguistics, pp 156–187
- Ye X, Chen Q, Dillig I et al (2024) Satlm: satisfiability-aided language models using declarative prompting. Adv Neural Inf Process Syst 36:66
- Yuan S, Song K, Chen J et al (2024) Easytool: enhancing llm-based agents with concise tool instruction. In: ICLR 2024 workshop on large language model (LLM) agents
- Zhang D, Chen L, Yu K (2023) Mobile-env: a universal platform for training and evaluation of mobile interaction. CoRR
- Zhang J (2023) Graph-toolformer: to empower llms with graph reasoning ability via prompt augmented by chatgpt. ArXiv preprint arXiv:2304.11116 · arXiv
- Zhang Z, Zhang A (2024) You only look at screens: Multimodal chain-of-action agents. In: Findings of the Association for Computational Linguistics ACL 2024, pp 3132–3149 · DOI · Paper page
- Zhao H, Liu Z, Wu Z et al (2024) Revolutionizing finance with llms: an overview of applications and insights. arXiv preprint arXiv:2401.11641 · arXiv
- Zhao L, Yang Y, Zhang K et al (2024) Diffagent: fast and accurate text-to-image api selection with large language model. In: 2024 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE Computer Society, pp 6390–6399 · DOI
- Zhao Q, Zhou X, Wu J et al (2024) Biotreasury: a community-based repository enabling indexing and rating of bioinformatics tools. Sci China Life Sci 67(2):221–229 · DOI
- Zhao Y, Xia J, Liu G et al (2019) Preference-aware task assignment in spatial crowdsourcing. In: Proceedings of the AAAI conference on artificial intelligence, pp 2629–2636 · doi:10.1609/aaai.v33i01.33012629
- Zheng B, Gou B, Kil J et al (2024) Gpt-4v (ision) is a generalist web agent, if grounded. In: International conference on machine learning, PMLR, pp 61349–61385
- Zheng Y, Li P, Liu W et al (2024) Toolrerank: adaptive and hierarchy-aware reranking for tool retrieval. In: Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024), pp 16263–16273
- Zheng Y, Li P, Yan M et al (2024) Budget-constrained tool learning with planning. In: Findings of the Association for Computational Linguistics ACL 2024, pp 9039–9052 · DOI
- Zhou X, Zhao X, Li G (2024) Llm-enhanced data management. ArXiv preprint arXiv:2402.02643 · arXiv
- Zhu JY, Cano CG, Bermudez DV et al (2024) Incoro: in-context learning for robotics control with feedback loops. ArXiv preprint arXiv:2402.05188 · arXiv
- Zhuang Y, Yu Y, Wang K et al (2023) Toolqa: a dataset for llm question answering with external tools. In: Proceedings of the 37th international conference on neural information processing systems, pp 50117–50143
- Zhuang Y, Chen X, Yu T et al (2024) Toolchain*: efficient action space navigation in large language models with a* search. In: The twelfth international conference on learning representations
- Zou A, Wang Z, Carlini N et al (2023) Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 · arXiv
Cited by
From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents
Yue, L., Bhandari, K. R., Ko, C.-Y., Patel, D., Lin, S., Zhou, N., Gao, J., Chen, P.-Y., & Pan, S. (2026) · arXiv · doi:10.48550/arxiv.2603.22386 · Cited by 1
“Tool-learning surveys focus on retrieving, selecting, and invoking tools (Xu et al., 2025b).”
Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration
Xu, J., Wu, X., Zhang, Z., Xu, Y., Yang, Y., Zhu, J., Xiao, L., Wu, W., & He, L. (2026) · arXiv · doi:10.48550/arxiv.2606.18259
“Failure detection must occur in real time as early mistakes compound [73, 74].”
MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning
Zhang, X., Zheng, Y., Xu, Z., Zhou, K., Shen, B., Ou, H., Zhang, T., & Lam, K.-Y. (2026) · arXiv · doi:10.48550/arxiv.2605.26154
“In such tool-augmented setting, agent’s performance highly depends on making correct action-routing decisions, including planing multi-step behaviors, de-composing tasks, and selecting an appropriate tool at each step (Xu et al., 2025).”
Metadata
- Authors
- Xu, W., Huang, C., Gao, S., & Shang, S.
- Venue
- Data Science and Engineering
- Published
- 2025-06-26
- Volume
- 10
- Issue
- 4
- Pages
- 533-563
- ISSN
- 2364-1185
- 2364-1541
- Cited by
- 74
- References
- 52
- Language
- en
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- Natural Science Foundation of China (62406061)
- National Natural Science Foundation of China (62432002)
- Natural Science Foundation of Shandong Province (ZR2023QF159)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: LLM-Based Agents for Tool Learning: A Survey. https://app.cochat.ai/papers/10.1007/s41019-025-00296-9
- The paper
- Xu, W., Huang, C., Gao, S., & Shang, S. (2025). LLM-Based Agents for Tool Learning: A Survey. Data Science and Engineering. https://doi.org/10.1007/s41019-025-00296-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.