Tool and Agent Selection for Large Language Model Agents in Production: A Survey
Open accessAbstract
Large Language Model (LLM) agents have demonstrated remarkable abilities to interact with external tools, functions, Model Context Protocol (MCP) servers, and agents. Due to the fast-paced nature of the industry, existing literature does not accurately represent the current state of tool and agent selection. This work provides the first detailed examination of tool selection from a production perspective, distinguishing between the frontend layer where users interact with agents through buttons, slash commands, or natural language, and the backend layer where retrieval, execution, orchestration, and memory enable scalable reasoning. The paper contributes a unified taxonomy of modern tool and agent selection approaches, a production-oriented framework covering dynamic tool retrieval, context engineering, execution methods, memory, and evaluation, and identifies challenges and promising avenues for research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Commercial deep research systems such as Perplexity AI, ChatGPT with web search, and Google Gemini with grounding have made citation-generating LLMs widely accessible.
“Commercial deep research systems such as Perplexity AI, ChatGPT with web search, and Google Gemini with grounding have made citation-generating LLMs widely accessible (Perplexity AI, 2024; OpenAI, 2024; Lumer et al., 2026; 2025d).”
From Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents (2026) · cites this paper as Lumer et al., 2026
The Introduction discusses LLM agents using external tools, and the References list Perplexity tools and a Gemini API document, but neither passage addresses citation-generating deep research systems or their accessibility. Searches for web search, grounding, citations, and deep research found no passage in our copy supporting the attributed claim. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Part of this paper’s reference list could not be split into single entries.
Research works printed but not linked: 27
- Lumer, E.; Subbiah, V.K.; Burke, J.A.; Basavaraju, P.H.; Huber, A. Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases, 2024, [arXiv:cs.CL/2410.14594]. Also published in ICAART 2025.
- Lumer, E.; Gulati, A.; Subbiah, V.K.; Basavaraju, P.H.; Burke, J.A. ScaleMCP: Dynamic and Auto- Synchronizing Model Context Protocol Tools for LLM Agents, 2025, [arXiv:cs.CL/2505.06416].
- Qin, Y.; Liang, S.; Ye, Y.; et al. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-World APIs. arXiv preprint 2023, [2307.16789].
- Du, Y.; Wei, F.; Zhang, H. AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls. arXiv preprint 2024, [2402.04253].
- Frank, K.; Gulati, A.; Lumer, E.; Campagna, S.; Subbiah, V.K. Jackal: A Real-World Execution-Based Benchmark Evaluating Large Language Models on Text-to-JQL Tasks, 2025, [arXiv:cs.CL/2509.23579].
- Murali, V.; Gulati, A.; Lumer, E.; Frank, K.; Campagna, S.; Subbiah, V.K. Agentic Jackal: Live Execution and Semantic Value Grounding for Text-to-JQL, 2026, [arXiv:cs.CL/2604.09470].
- Mo, G.; Zhong, W.; Chen, J.; Chen, X.; Lu, Y.; Lin, H.; He, B.; Han, X.; Sun, L. LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?, 2025, [arXiv:cs.AI/2508.01780].
- Zheng, Y.; Li, P.; Liu, W.; et al. ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval. arXiv preprint 2024, [2403.06551].
- Sarmah, P.; et al. Hybrid RAG: Integrating Knowledge Graphs with Retrieval-Augmented Generation. arXiv preprint arXiv:2408.XXXX 2024.
- Ouyang, S.; Yan, J.; Hsu, I.H.; Chen, Y.; Jiang, K.; Wang, Z.; Han, R.; Le, L.T.; Daruki, S.; Tang, X.; et al. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. arXiv preprint 2025, [2509.25140].
- Liu, M.M.; Garcia, D.; Parllaku, F.; Upadhyay, V.; Shah, S.F.A.; Roth, D. ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering, 2025, [arXiv:cs.CL/2510.20036].
- Krishnan, N. AI Agents: Evolution, Architecture, and Real-World Applications, 2025, [arXiv:cs.AI/2503.12687].
- Li, M.; Zhao, Y.; Yu, B.; et al. API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs. arXiv preprint 2023, [2304.08244].
- Wang, Z.; Cheng, Z.; Zhu, H.; Fried, D.; Neubig, G. What Are Tools Anyway? A Survey from the Language Model Perspective, 2024, [arXiv:cs.CL/2403.15452].
- Qu, C.; Dai, S.; Wei, X.; Cai, H.; Wang, S.; Yin, D.; Xu, J.; Wen, J.r. Tool Learning with Large Language Models: A Survey. Frontiers of Computer Science 2025, 19. https://doi.org/10.1007/s11704-024-40678-2. · DOI
- Patil, S.G.; Zhang, T.; Wang, X.; Gonzalez, J.E. Gorilla: Large Language Model Connected with Massive APIs. arXiv preprint 2023, [2305.15334].
- Huang, T.; Jung, D.; Chen, M. Planning and Editing What You Retrieve for Enhanced Tool Learning. In Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2024, 2024. Accessed 2025-11-10.
- Radford, A.; Kim, J.W.; Xu, T.; Brockman, G.; McLeavey, C.; Sutskever, I. Robust Speech Recognition via Large-Scale Weak Supervision. arXiv preprint arXiv:2212.04356 2022. · arXiv
- Baevski, A.; Zhou, H.; Mohamed, A.; Auli, M. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. arXiv preprint arXiv:2006.11477 2020. · arXiv
- Zhang, Y.; et al. USM: Scaling Automatic Speech Recognition Beyond 100 Languages. arXiv preprint arXiv:2303.01037 2023. · arXiv
- Liu, H.; Li, C.; Wu, Q.; Lee, Y.J. Visual Instruction Tuning. arXiv preprint arXiv:2304.08485 2023. · arXiv
- Lumer, E.; Cardenas, A.; Melich, M.; Mason, M.; Dieter, S.; Subbiah, V.K.; Basavaraju, P.H.; Hernandez, R. Comparison of Text-Based and Image-Based Retrieval in Multimodal Retrieval Augmented Generation Large Language Model Systems, 2025, [arXiv:cs.CL/2511.16654].
- Gulati, A.; Sen, S.; Sarguroh, W.; Paul, K. From Rows to Reasoning: A Retrieval-Augmented Multimodal Framework for Spreadsheet Understanding, 2026, [arXiv:cs.CL/2601.08741].
- Park, J.S.; O'Brien, J.C.; Cai, C.J.; Morris, M.R.; Liang, P.; Bernstein, M.S. Generative Agents: Interactive Simulacra of Human Behavior. arXiv preprint arXiv:2304.03442 2023. 101. Mem0. Mem0: The Memory Layer for Personalized AI. https://mem0.ai/, 2025. Accessed: 2025-08-06. 102. LlamaIndex. Long-Term Memory in LlamaIndex, 2025. Accessed: 2025-11-08. 103. LangChain AI. LangGraph -Persistence and Checkpointing, 2025. Accessed: 2025-11-08. 104. Python Software Foundation. Python Documentation. https://docs.python.org/3/, 2025. Accessed: 2025-10-07. 105. Microsoft. TypeScript Documentation. https://www.typescriptlang.org/docs/, 2025. Accessed: 2025-10-07. 106. Google DeepMind and Google Research. Announcing the Agent2Agent Protocol (A2A). https://developers. googleblog.com/en/a2a-a-new-era-of-agent-interoperability/, 2025. Accessed: 2025-10-07. 107. IBM Research. The simplest protocol for AI agents to work together. https://research.ibm.com/blog/agent- communication-protocol-ai, 2025. Accessed: 2025-10-07. 108. · arXiv
- Gao, Y.; Xiong, Y.; et al. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint 2024, [2312.10997]. 111. Peng, B.; Zhu, Y.; Liu, Y.; et al. Graph Retrieval-Augmented Generation: A Survey. arXiv preprint 2024, [2408.08921]. 112
- Wu, M.; Zhu, T.; Han, H.; et al. SEAL-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark. arXiv preprint 2024, [2405.08355]. 113. Han, H.; Shomer, H.; Wang, Y.; Lei, Y.; Guo, K.; Hua, Z.; Long, B.; Liu, H.; Tang, J. RAG vs. GraphRAG: A Systematic Evaluation and Key Insights, 2025, [arXiv:cs.IR/2502.11371]. 114. Lumer, E.; Basavaraju, P.H.; Mason, M.; Burke, J.A.; Subbiah, V.K. Graph RAG-Tool Fusion, 2025, [arXiv:cs.CL/2502.07223]. 115. Sarmah, B.; Hall, B.; Rao, R.; Patel, S.; Pasquali, S.; Mehta, D. HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction, 2024, [arXiv:cs.CL/2408.04948]. 116. OpenAI. Embedding Models. https://platform.openai.com/docs/guides/embeddings/embedding- models, 2024. Accessed: 2025-10-08. 117. Cohere. Cohere Embed Models. https://docs.cohere.com/docs/cohere-embed, 2024. Accessed: 2025-10-08. 118. Google Cloud. Vertex AI Embeddings (preview). https://cloud.google.com/vertex-ai/generative-ai/docs/ embeddings, 2025. Accessed: 2025-10-08. 119.
- Shi, J.; Yuan, Z.; Tie, G.; Zhou, P.; Gong, N.Z.; Sun, L. Prompt Injection Attack to Tool Selection in LLM Agents, 2025, [arXiv:cs.CR/2504.19793]. 120. Trivedi, H.; Balasubramanian, N.; Khot, T.; Sabharwal, A. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In Proceedings of the NeurIPS, 2023. 121.
Cited by
Sorted by citations
Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents
Onweller, H., Lumer, E., Huber, A., Ramchandani, P., Subbiah, V. K., & Feld, C. (2026) · arXiv · doi:10.48550/arxiv.2605.06635 · Cited by 2 · see citing works
“Commercial deep research systems such as Perplexity AI, ChatGPT with web search, and Google Gemini with grounding have made citation-generating LLMs widely accessible (Perplexity AI, 2024; OpenAI, 2024; Lumer et al., 2026; 2025d).”
Self-citationLeung, E., Lumer, E., Feld, C., Huber, A., Subbiah, V. K., & Paul, K. (2026) · arXiv · doi:10.48550/arxiv.2607.08700
“RAG evaluation frameworks such as RAGAS [Es et al., 2024] and ARES [Saad-Falcon et al., 2024] assess faithfulness at the answer level, treating the full context-answer pair as a unit rather than scoring individual attribution-citation pairs Lumer et al. [2026a].”
Self-citationMa, J., Wang, C., Zhang, Y., Wang, Q., Yang, W., & Zhao, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3403-5_9
Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding
Gulati, A., Sen, S., Sarguroh, W., & Paul, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36023-6_33
Self-citationChen, Y., Ma, Z., Liu, G., Song, Y., Xiuzhi, D. E. N. G., & Li, J. (2026) · Journal of Construction Engineering and Management · doi:10.1061/jcemd4.coeng-19274
A survey on AI agent security: Reasoning, Acting, and Self-Evolving
Liu, D., Luo, T., Huang, S., Yang, J., Liang, S., Jia, X., Lau, C. P., Dong, J., Liu, Y., Bai, Y., et al. (2026) · AI Plus · doi:10.59717/ipj.aiplus.2026.100004
- Authors
- Lumer, E., Gulati, A., Nizar, F., Hedroits, D., Mehta, A., Hwangbo, H., Subbiah, V. K., Basavaraju, P. H., & Burke, J. A.
- Affiliations
- PricewaterhouseCoopers (United States) (US)
- PricewaterhouseCoopers,Commercial Technology and Innovation Office,U.S.
- Venue
- 2026 IEEE Conference on Artificial Intelligence (CAI)
- Publisher
- IEEE
- Published
- 2026-05-08
- Pages
- 701-708
- OpenAlex
- W4417309755
- Cited by
- 5
- References
- 0
- Citations by year
- 2026: 5
- FWCI
- 44.68
- Type
- conference paper
- Language
- en
- Open access
- green
- doi.org
- Locations
- IEEE Conference on Artificial Intelligence (CAI) (publishedVersion)
- Preprints.org (acceptedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- LLM agents, tool selection, agent selection, production deployment, tool retrieval, retrieval-augmented generation, context engineering, human-in-the-loop, authentication and authorization, agent memory, tool orchestration, Model Context Protocol
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
