Are Tools All We Need? Unveiling the Tool-Use Tax in LLM Agents
Open accessAbstract
Tool-augmented reasoning has become a popular direction for LLM-based agents, and it is widely assumed to improve reasoning and reliability. However, we demonstrate that this consensus does not always hold: in the presence of semantic distractors, tool-augmented reasoning does not necessarily outperform native CoT. To explain this performance gap, we propose a Factorized Intervention Framework that isolates the cost of prompt formatting, the overhead of the tool-calling protocol, and the actual gain from executing tools. Our analysis reveals a critical tradeoff: under semantic noise, the gains from tools often fail to offset the "tool-use tax", which is the performance degradation introduced by the tool-calling protocol itself. To address this, we introduce G-STEP, a lightweight inference-time gate to mitigate protocol-induced errors. While this yields partial recovery, our findings suggest that more substantial improvements still require strengthening the model's intrinsic reasoning and tool-interaction capabilities.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This matches the Capability Overlap Principle and the pass-rate invariance reported under tool ablation.
“This matches the Capability Overlap Principle [45] and the pass-rate Baseline arm - 1 LLM step → 3 tool calls invariance reported under tool ablation in [33].”
From When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation (2026) · cites this paper as [45]
The cited paper does not report pass-rate invariance under tool ablation. Section 3.3 instead says the no-tool condition has the best non-oracle accuracy across its task–model pairs and reports substantial accuracy gaps between tool-using and no-tool conditions. Its conditions also do not establish the stated ‘1 LLM step → 3 tool calls’ baseline. (LLM, conf=0.77)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Part of this paper’s reference list could not be split into single entries.
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 3× · see citing works
“For brevity, we refer to these two augmented benchmarks as GSM8K and HotpotQA in the rest of the paper, unless otherwise specified.”
InfluentialTraining Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · cited 1× · see citing works
“Datasets Using the method in Section 2.1, we augment GSM8K (Cobbe et al., 2021) and HotpotQA (Yang et al., 2018) with GPT-4o-mini-generated distractors, yielding GSM8K-Sem-Distractor and HotpotQA-Sem-Distractor.”
InfluentialQin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16789 · Cited by 71 · cited 2× · see citing works
“In parallel, recent benchmarks have shifted toward realistic and process-oriented evaluation, covering fine-grained tool-use abilities (Ye et al., 2024) and mobile-assistant function calling under multi-turn, imperfect, and shifting user instructions (Wang et al., 2025).”
Self-citationCRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.11738 · Cited by 57 · cited 5× · see citing works
“Across variants, G ATE and G ATE +CRITIC consistently improve over A GENT -”
InfluentialHow Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark
Yang, M., Huang, E., Zhang, L., et al. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2505.18761 · Cited by 49 · cited 1×
“Recent studies show that such distractors can substantially disrupt reasoning-path selection and final accuracy, and that noisy external information may even amplify failures in reasoning and search-augmented systems (Xiong et al., 2025; Yang et al., 2025; Lee et al., 2026; Pham et al., 2026).”
AgentDiagnose: An Open Toolkit for Diagnosing LLM Agent Trajectories
Ou, T.-Y., Guo, W.-Y., Gandhi, A., et al. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-demos.15 · Cited by 35 · cited 2×
“AgentDiagnose (Ou et al., 2025), Zhang et al. (2025), AgentFail (Ma et al., 2026), and Paper-Arena (Wang et al., 2026a) move beyond end-task success toward trajectory-level diagnosis, failure attribution, and root-cause analysis, a trend further summarized by recent survey work on LLM agent…”
AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition
Wang, R.-P., Chen, Y.-X., Wang, Y.-K., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2602.11348 · Cited by 28 · cited 1×
“This gap is especially important for deployment, since failures in tool-augmented systems may stem not only from missing capabilities, but also from the mechanics of tool invocation and interaction itself (Cheng et al., 2026; Wang et al., 2026c).”
Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents
Kim, W., Park, S., In, Y., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2510.02837 · Cited by 24 · cited 1×
“FineLogic (Zhou et al., 2025), Evaluating Step-by-step Reasoning Traces (Lee and Hockenmaier, 2025), Kim et al. (2025) all emphasize that intermediate reasoning traces reveal errors missed by outcome-only evaluation.”
Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study
Zhou, Y.-J., Ye, J.-Y., Ling, Z.-P., et al. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2506.04810 · Cited by 20
Lost in the Noise: How Reasoning Models Fail with Contextual Distractors
Lee, S., Jo, Y., Seo, M., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2601.07226 · Cited by 15 · cited 1×
“Recent studies show that such distractors can substantially disrupt reasoning-path selection and final accuracy, and that noisy external information may even amplify failures in reasoning and search-augmented systems (Xiong et al., 2025; Yang et al., 2025; Lee et al., 2026; Pham et al., 2026).”
A Taxonomy of Failures in Tool-Augmented LLMs
Winston, C., & Just, R. (2025) · International Conference/Workshop on Automation of Software Test · doi:10.1109/ast66626.2025.00019 · Cited by 10 · cited 1× · see citing works
“…call tools, while recent work on reasoning and agent evaluation increasingly argues that final-answer accuracy alone is insufficient for diagnosing intermediate failures (Chen et al., 2025; Lee and Hockenmaier, 2025; Zhou et al., 2025; Hwang et al., 2025; Ou et al., 2025; Winston and Just, 2025).”
Demystifying the Lifecycle of Failures in Platform-Orchestrated Agentic Workflows
Ma, X., Xie, X.-F., Wang, Y., et al. (2025) · doi:10.48550/arxiv.2509.23735 · Cited by 9 · cited 2×
“AgentDiagnose (Ou et al., 2025), Zhang et al. (2025), AgentFail (Ma et al., 2026), and Paper-Arena (Wang et al., 2026a) move beyond end-task success toward trajectory-level diagnosis, failure attribution, and root-cause analysis, a trend further summarized by recent survey work on LLM agent…”
TTPA: Token-level Tool-use Preference Alignment Training Framework with Fine-grained Evaluation
Huang, C.-R., Gao, S., Shi, Z., et al. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2505.20016 · Cited by 9 · cited 1×
“…construction, training, and evaluation over real-world APIs, while later studies improved tool learning through high-quality synthetic data (Liu et al., 2025), structure-aware tool representation (Su et al., 2025), and token-level optimization with fine-grained error scoring (Huang et al., 2025).”
A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
Zhang, K.-T., Hu, M.-Z., Le, H., et al. (2026) · Trans. Mach. Learn. Res. · doi:10.48550/arxiv.2601.17717 · Cited by 6 · cited 1×
“Function calling, search augmentation, and tool-augmented agents are widely regarded as effective ways to provide models with up-to-date information, precise computation, and access to external services (Zhang et al., 2026; Chen et al., 2023; Liu et al., 2025; Chen et al., 2025; Patil et al., 2025).”
Self-citationInvestigating Tool-Memory Conflicts in Tool-Augmented LLMsPreprint
Cheng, J., Pan, R., & Amiri, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.09760 · Cited by 6 · cited 1× · see citing works
“This gap is especially important for deployment, since failures in tool-augmented systems may stem not only from missing capabilities, but also from the mechanics of tool invocation and interaction itself (Cheng et al., 2026; Wang et al., 2026c).”
Assessing LLM Reasoning Steps via Principal Knowledge Grounding
Hwang, H., Cho, Y., Yoon, C., Park, Y., Song, M.-J., Lee, K., Kim, G., & Kang, J. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.findings-emnlp.1085 · Cited by 5 · cited 1×
“…call tools, while recent work on reasoning and agent evaluation increasingly argues that final-answer accuracy alone is insufficient for diagnosing intermediate failures (Chen et al., 2025; Lee and Hockenmaier, 2025; Zhou et al., 2025; Hwang et al., 2025; Ou et al., 2025; Winston and Just, 2025).”
ToolACE: Winning the Points of LLM Function CallingPreprint
Liu, W., Huang, X., Zeng, X., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.00920 · Cited by 4 · see citing works
ACEBench: A Comprehensive Evaluation of LLM Tool Usage
Chen, C., Hao, X., Liu, W., Huang, X. G., Zeng, X., Yu, S., Li, D., Huang, Y., Liu, X., Xinzhi, W., et al. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.697 · Cited by 2 · cited 3× · see citing works
“…call tools, while recent work on reasoning and agent evaluation increasingly argues that final-answer accuracy alone is insufficient for diagnosing intermediate failures (Chen et al., 2025; Lee and Hockenmaier, 2025; Zhou et al., 2025; Hwang et al., 2025; Ou et al., 2025; Winston and Just, 2025).”
Toolscaler: Scalable Generative Tool Calling via Structure-Aware Semantic Tokenization
Su, Y., Jinshuai, Z., Fang, B., Wen, Y., Zhang, J., Song, B., Wang, W., Liu, Q., & Wang, L. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.30 · cited 2×
“AgentDiagnose (Ou et al., 2025), Zhang et al. (2025), AgentFail (Ma et al., 2026), and Paper-Arena (Wang et al., 2026a) move beyond end-task success toward trajectory-level diagnosis, failure attribution, and root-cause analysis, a trend further summarized by recent survey work on LLM agent…”
SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models
Pham, T., Nguyen, N., Zunjare, P., Chen, W., Tseng, Y.-M., & Vu, T. (2025) · arXiv · doi:10.48550/arxiv.2506.01062 · cited 1×
“Recent studies show that such distractors can substantially disrupt reasoning-path selection and final accuracy, and that noisy external information may even amplify failures in reasoning and search-augmented systems (Xiong et al., 2025; Yang et al., 2025; Lee et al., 2026; Pham et al., 2026).”
PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific LiteraturePreprint
Wang, D., Cheng, M., Yu, S., Liu, Z., Guo, Z., Li, X., & Liu, Q. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.10909 · cited 2×
“AgentDiagnose (Ou et al., 2025), Zhang et al. (2025), AgentFail (Ma et al., 2026), and Paper-Arena (Wang et al., 2026a) move beyond end-task success toward trajectory-level diagnosis, failure attribution, and root-cause analysis, a trend further summarized by recent survey work on LLM agent…”
A Survey for LLM Agent Trajectory Analysis: From Failure Attribution to Enhancement
Wang, J., Wang, Y., Chen, M., Xie, X., Chen, C., Mu, F., Liu, Z., & Wang, Q. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3717765 · cited 1×
“…et al. (2025), AgentFail (Ma et al., 2026), and Paper-Arena (Wang et al., 2026a) move beyond end-task success toward trajectory-level diagnosis, failure attribution, and root-cause analysis, a trend further summarized by recent survey work on LLM agent trajectory analysis (Wang et al., 2026b).”
Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
Tie, G., Yuan, Z., Zhao, Z., Hu, C., Gu, T., Zhang, R., Zhang, S., Wu, J., Tu, X., Jin, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.16062 · cited 1×
“Rather than arguing that tools are broadly ineffective, our goal is to identify when tool augmentation breaks down under semantically noisy conditions and to explain why, complementing recent work on inference-time correction and robustness (Tie et al., 2025).”
Self-citation
Flagged first, then by citations
12 of the 35 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Chatcot: Tool-augmented chain-of-thought reason- ing on chat-based large language models. Preprint, arXiv:2305.14323. · arXiv
- Jinu Lee and Julia Hockenmaier. 2025. Evaluating step-by-step reasoning traces: A survey. Preprint, arXiv:2502.12289. · arXiv
- Shishir G Patil, Huanzhi Mao, Fanjia Yan, Char- lie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning.
- Jun Wang, Jiamu Zhou, Muning Wen, Xiaoyun Mo, Haoyu Zhang, Qiqiang Lin, Cheng Jin, Xihuai Wang, Weinan Zhang, Qiuying Peng, and Jun Wang. 2025. Hammerbench: Fine-grained function-calling eval- uation in real mobile device scenarios. Preprint, arXiv:2412.16516. · arXiv
- Zhen Xiong, Yujun Cai, Zhecheng Li, and Yiwei Wang. 2025. Mapping the minds of llms: A graph-based analysis of reasoning llm. Preprint, arXiv:2505.13890. · arXiv
- Junjie Ye, Guanyu Li, Songyang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Tao Ji, Qi Zhang, Tao Gui, and Xuanjing Huang. 2024. Tooleyes: Fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios. Preprint, arXiv:2401.00741. · arXiv
Cited by
Sorted by citations
Wang, Z.-Y., & Vemuri, P. (2026) · arXiv.org · doi:10.48550/arxiv.2606.14476 · Cited by 6
Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability GainsPreprint
Guo, G., Yu, D., Chen, Y., Wang, X., Li, S., Zhao, X., Liu, H., Wang, Q., & Liao, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.02357
“Closer to our setting, MAPO (Yang et al., 2026) identifies a reasoning–action discrepancy in tool-using multi-modal agents under outcome-based rewards, and Tool-Use Tax (Zhang et al., 2026a) pirical characterization of when tool calls fail to expand the set of problems already solvable without tools.”
Chen, Y.-Y., Li, Y., Cui, Y., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.20090 · Cited by 2
“By designing appropriate reward functions and adapting optimization objectives to practical tasks, LLMs can learn to invoke external tools for search, file manipulation, and even complex GUI interactions Zhang et al. (2026b).”
Mehta, A. (2026) · arXiv.org · doi:10.48550/arxiv.2607.02436 · Cited by 2
“Consumption on the same task can differ by a factor of thirty, and spending more does not mean doing better (Bai et al., 2026), while tool calls carry an overhead that can outweigh what the tool contributes (Zhang et al., 2026).”
EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and ReusePreprint
Shi, D., Li, Y., Yi, X., & Wang, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.15161
“Recent advances in large language models (LLMs) have substantially improved their capabilities across a wide range of tasks, particularly in reasoning, planning, and tool use (Yi et al., 2025; Li et al., 2025; Chen et al., 2026; Zhang et al., 2026b).”
ToolGate: Token-Efficient Pre-Call Control for Tool-Augmented Vision-Language Agents
Liu, A.-J., Song, Y., Chen, Z., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.03054 · Cited by 1
“Zhang et al. (2026) identify a tool-use tax in LLM agents, decomposing performance loss into prompt-formatting cost, protocol overhead, and tool-execution gain, while Sun et al. (2026) show that tool necessity can be de-coded from pre-generation hidden states and used to steer tool use.”
When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation
Yang, H., Yu, Q., & Desell, T. (2026) · arXiv · doi:10.48550/arxiv.2607.10569
“This dissociation is what licenses contributions (1) and (2) as cost-engineering claims rather than capability claims, and it is consistent with the Capability Overlap Principle [45]; the load-bearing claim is the empirical agreement-matrix result, not the theoretical frame.”
InfluentialZhang, K.-T., Xiong, Z., Jiang, Z.-M., Zhong, M.-Y., Yuan, Z., Li, Z.-C., Lin, B.-W., Chang, C.-Y., Hu, M.-Z., Wang, H.-Z., et al. (2026) · doi:10.48550/arxiv.2609.32662
Self-citationZheng, S., Shi, D.-S., Li, Y., et al. (2026) · doi:10.48550/arxiv.2608.03794
“…LLMs have increasingly exhibited the potential to serve as autonomous agents capable of planning and executing complex tasks (Shi et al., 2026; Chen et al., 2026; Zhang et al., 2026b), including managing the entire database pipeline (Zhang et al 2026a; Zhou et al., 2024b; Li et al., 2024).”
PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents
Zhang, K., Chundur, S., Qomi, M. J., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.00436
“…work decomposes the cost of tool-augmented reasoning into protocol-and execution-level components and shows that tool calls are not free of error [Zhang et al., 2026], evaluates how LLMs process structured tool outputs across return formats [Kate et al., 2025], and applies item-level…”
Self-citation
- Authors
- Zhang, K., Xiong, Z., Zhong, M., Jiang, Z., Yuan, Z., Li, Z., & Lin, Y.
- Affiliations
- University of Houston (US)
- University of Southern California (US)
- Texas A&M University (US)
- University of California San Diego (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2605.00136
- OpenAlex
- W7160145031
- Semantic Scholar
- b54bc98a951073badd13adf00d72598982fb7e65
- Cited by
- 0
- References
- 35
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- tool-augmented reasoning, LLM agents, chain-of-thought reasoning, semantic noise
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
