CoChat
Sign Up
PreprintCited by 4Natural Language Processing TechniquesTopic ModelingArtificial Intelligence Applications

Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks

Open access
Lumer, Elias, Nizar, Faheem, Jangiti, Akshaya, Frank, Kevin, Gulati, Anmol, Phadate, Mandar, Subbiah, Vamse Kumar·2026·arXiv
DOI 10.48550/arxiv.2601.06007arXiv 2601.06007
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent advancements in Large Language Model (LLM) agents have enabled complex multi-turn agentic tasks requiring extensive tool calling, where conversations can span dozens of API calls with increasingly large context windows. However, although major LLM providers offer prompt caching to reduce cost and latency, its benefits for agentic workloads remain underexplored in the research literature. To our knowledge, no prior work quantifies these cost savings or compares caching strategies for multi-turn agentic tasks. We present a comprehensive evaluation of prompt caching across three major LLM providers (OpenAI, Anthropic, and Google) and compare three caching strategies, including full context caching, system prompt only caching, and caching that excludes dynamic tool results. We evaluate on DeepResearch Bench, a multi-turn agentic benchmark where agents autonomously execute real-world web search tool calls to answer complex research questions, measuring both API cost and time to first token (TTFT) across over 500 agent sessions with 10,000-token system prompts. Our results demonstrate that prompt caching reduces API costs by 41-80% and improves time to first token by 13-31% across providers. We find that strategic prompt cache block control, such as placing dynamic content at the end of the system prompt, avoiding dynamic traditional function calling, and excluding dynamic tool results, provides more consistent benefits than naive full-context caching, which can paradoxically increase latency. An ablation study across prompt sizes (500-50,000 tokens) and tool call counts (3-50) demonstrates universal linear cost and TTFT benefits, after the provider caching token minimum, and reveal provider-specific strategy discrepancies across variants. We provide nuanced discussion and guidance for implementing prompt caching in production agentic systems.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper evaluates prompt caching for long-horizon agentic tasks across OpenAI, Anthropic, and Google models on DeepResearch Bench, comparing no-cache, full-context, system-prompt-only, and exclude-tool-results strategies. It finds prompt caching cuts API costs by 41–80% and can improve TTFT by 13–31% across providers, but strategic cache-boundary control—especially system-prompt-only caching—is more consistent than naive full-context caching, which can increase latency.

Key findings

  • All four tested models show statistically significant cost reductions with prompt caching. Best-mode cost savings in Table 1 are 79.6% for GPT-5.2, 78.5% for Claude Sonnet 4.5, 41.4% for Gemini 2.5 Pro, and 45.9% for GPT-4o; across cache modes, reductions range from 79–81% for GPT-5.2, 78–79% for Claude Sonnet 4.5, 46–48% for GPT-4o, and 28–41% for Gemini 2.5 Pro.
  • TTFT improvements vary more by provider. Best-mode TTFT improvements are 13.0% for GPT-5.2, 22.9% for Claude Sonnet 4.5, 6.1% for Gemini 2.5 Pro, and 30.9% for GPT-4o; the abstract and conclusion summarize TTFT improvements as 13–31% across providers. Full-context caching can regress latency, e.g., GPT-4o shows an 8.8% TTFT regression under full-context caching.
  • System-prompt-only and exclude-tool-results caching generally outperform full-context caching for TTFT, while cost differences between strategies are typically within 2–4 percentage points because the large system prompt drives most savings.
  • Ablation: cost savings scale linearly with prompt size. At 50,000 tokens, GPT-5.2 achieves 89% cost savings ($0.253 to $0.029), Claude Sonnet 4.5 88% ($0.667 to $0.080), and GPT-4o 54% ($0.414 to $0.192); GPT-4o shows 60% TTFT improvement (4,290 ms to 1,699 ms). At 500 tokens, below caching thresholds, GPT-4o, GPT-5.2, and Claude Sonnet 4.5 show TTFT regressions of 10–18%. Across 3–50 tool calls, GPT-5.2 maintains about 77–81% cost savings and GPT-4o 42–53%; Claude Sonnet 4.5 TTFT improvement falls from 19% at 3 tools to 5% at 50 tools.

Methods

  • Design: evaluation across OpenAI, Anthropic, and Google using GPT-5.2, Claude Sonnet 4.5, Gemini 2.5 Pro, and GPT-4o. Four cache conditions are compared: No Cache baseline (UUID prepended to break the cache), Full Context Caching, System Prompt Only Caching (UUID after the system prompt), and Exclude Tool Results Caching (UUIDs after the system prompt and each tool result). Metrics are API cost and time to first token (TTFT).
  • Data/sample: DeepResearch Bench, a multi-turn agentic benchmark where agents execute web search tool calls to answer complex research questions. Each model runs 40 independent agent sessions per cache condition, with a 10,000-token system prompt and a fresh context per session; the abstract reports over 500 agent sessions total. Warmup calls prime the cache, and conditions are separated by more than 24 hours to avoid cross-condition contamination.
  • Analysis: independent samples t-tests compare each cache condition against the no-cache baseline at α = 0.05, with n = 40 per condition; the paper reports mean cost, mean TTFT, percentage improvement, and p-values. An ablation study varies prompt sizes from 500 to 50,000 tokens and tool call counts from 3 to 50.

Limitations

  • Stated in the paper: TTFT measurements exhibit natural variance from server load, network conditions, and provider infrastructure; the ablation study shows higher TTFT variance than the main study. Provider caching thresholds, TTLs, and pricing are subject to change, and practitioners should conduct experiments representative of their own workloads rather than relying solely on published benchmarks.
  • Evident from the text: the main evaluation uses one benchmark (DeepResearch Bench), one agent framework (Deep Agents), four models, 40 sessions per condition, and a fixed 10,000-token system prompt. Sessions start with fresh context, so cross-session cache reuse is not measured, and pricing is taken as of early January 2026.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2601.06007

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 22 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 2.3. Agentic Workloads and Context Engineering

    Modern agentic applications routinely execute 30-50 or more tool calls within a single session.

    “Modern agentic applications including deep research assistants, coding agents such as Claude Code and Cursor, and autonomous task completion systems like Manus routinely execute 30-50 or more tool calls within a single session”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Ji, 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Du et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Mialon et al., 2023 source

      Section 3.3, “Composition of GAIA” (page unavailable), describes benchmark tasks that may require long sequences of actions and multiple tools, but it does not report tool-call counts in application sessions. Searches for “30-50 tool calls” and session-level counts found no passage supporting the claimed routine frequency or magnitude. (LLM, conf=0.77)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 1. Introduction

    Existing work on KV cache optimization focuses primarily on inference-level memory management and compression, rather than evaluating enterprise-grade prompt caching features offered through provider APIs.

    “Existing work on KV cache optimization focuses primarily on inference-level memory management and compression (Kwon et al., 2023; Ge et al., 2023; Shi et al., 2024), rather than evaluating the enterprise-grade prompt caching features offered through provider APIs.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Kwon et al., 2023 source
    • cites Ge et al., 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shi et al., 2024 source
  • Findingreasoning
    § 1. Introduction

    The cache strategy that maximizes cost savings does not always maximize latency improvement.

    “Notably, the cache strategy that maximizes cost savings does not always maximize latency improvement, highlighting the importance of strategy selection based on optimization goals.”
    Quote in the paperSubstantial claim
  • Findingnone given
    § Abstract

    Naive full-context caching can paradoxically increase latency.

    “provides more consistent benefits than naive fullcontext caching, which can paradoxically increase latency.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 4.2. Cost Reduction

    Full context caching exhibits a slight regression of 8.8% for GPT-4o.

    “while full context caching exhibits a slight regression of 8.8%”
    Quote in the paperSubstantial claim
  • May combine two claims. 23
  • The quote only partly states this claim. 3
  • Classified as probably not a substantial claim. 26

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § Abstract

    An ablation study across prompt sizes (500-50,000 tokens) and tool call counts (3-50) demonstrates universal linear cost and TTFT benefits, after the provider caching token minimum.

    “An ablation study across prompt sizes (500-50,000 tokens) and tool call counts (3-50) demonstrates universal linear cost and TTFT benefits, after the provider caching token minimum, and reveal provider-specific strategy discrepancies across variants.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2. Cost Reduction

    GPT-4o shows 28-31% TTFT improvement with system prompt only and exclude tool results strategies.

    “GPT-4o shows 28-31% improvement with system prompt only and exclude tool results strategies”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2. Cost Reduction

    Cost reductions range from 79-81% for GPT-5.2, 78-79% for Claude Sonnet 4.5, 46-48% for GPT-4o, and 28-41% for Gemini 2.5 Pro depending on the cache mode selected.

    “Cost reductions range from 79-81% for GPT-5.2, 78-79% for Claude Sonnet 4.5, 46-48% for GPT-4o, and 28-41% for Gemini 2.5 Pro depending on the cache mode selected.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data
    § 4.1. Overall Results

    Prompt caching reduces API costs by 41-80% across providers.

    “Cost savings range from 41% to 80% across models”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 4.3. Cache Strategy Comparison

    For some models, full context caching shows no improvement or slight regression, while other strategies achieve 28-31% improvement.

    “For some models, full context caching shows no improvement or slight regression, while other strategies achieve 28-31% improvement.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3. Provider Implementation Variability

    Cost savings are consistent across strategies - TTFT varies, with full-context caching sometimes underperforming selective strategies.

    “Cost savings are consistent across strategies - TTFT varies, with full-context caching sometimes underperforming selective strategies.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Prompt caching requires exact prefix matches.

  • May combine two claims.

    Findingown data
    § 6.1. Ablation by Prompt Size

    TTFT improvements are most pronounced at large context sizes, with GPT-4o showing 60% improvement (from 4,290ms to 1,699ms) at 50,000 tokens.

    “TTFT improvements are also most pronounced at large context sizes, with GPT-4o showing 60% improvement (from 4,290ms to 1,699ms) at 50,000 tokens.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given
    § 6.1. Ablation by Prompt Size

    At prompt sizes above the threshold (2,000+ tokens), TTFT improvements become positive and scale with prompt size.

    “At prompt sizes above the threshold (2,000+ tokens), TTFT improvements become positive and scale with prompt size.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 6.1. Ablation by Prompt Size

    Cost savings scale linearly with prompt size across all models and are universally positive at every prompt size tested.

    “As shown in the left columns of Figure 4, cost savings scale linearly with prompt size across all models and are universally positive at every prompt size tested.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data
    § 6.2. Ablation by Tool Count

    GPT-4o achieves 42-53% cost savings across all tool counts.

    “GPT-4o achieves 4253% savings”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 5.1. Strategic Cache Boundary Control

    Providers abstract much of the underlying caching mechanism, automatically triggering cache creation when token thresholds are exceeded.

    “The key insight is that providers abstract much of the underlying caching mechanism, automatically triggering cache creation when token thresholds are exceeded.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1. Introduction

    Concurrent work has audited prompt caching across providers to detect timing side-channel vulnerabilities.

    “Concurrent work has audited prompt caching across providers to detect timing side-channel vulnerabilities (Gu et al., 2025)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gu et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 2.3. Agentic Workloads and Context Engineering

    Agentic workloads feature dynamic, session-specific content that accumulates unpredictably.

    “Unlike static question-answering scenarios where prompts are largely predetermined, agentic workloads feature dynamic, session-specific content that accumulates unpredictably.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.2. Cost Reduction

    The primary driver of cost reduction is caching the large system prompt, which remains stable across all requests within a session.

    “The consistency of cost savings across cache strategies suggests that the primary driver of cost reduction is caching the large system prompt, which remains stable across all requests within a session.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § Abstract

    No prior work quantifies cost savings or compares caching strategies for multi-turn agentic tasks.

    “To our knowledge, no prior work quantifies these cost savings or compares caching strategies for multi-turn agentic tasks.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisenone given
    § 5.1. Strategic Cache Boundary Control

    In agentic applications, the system prompt is the most stable component, containing agent instructions, tool definitions, and persona guidelines that remain constant across sessions.

    “In agentic applications, the system prompt is the most stable component, containing agent instructions, tool definitions, and persona guidelines that remain constant across sessions.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § 3.3. Evaluation Protocol

    Prior to each experimental condition, warmup calls are executed to prime the cache and record cache creation tokens separately from evaluation runs.

    “Prior to each experimental condition, we execute warmup calls to prime the cache and record cache creation tokens separately from evaluation runs.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 3.2. Cache Mode Implementation

    The dual-UUID approach is necessary because provider-level KV-cache handling can vary, and explicit boundaries provide more predictable caching behavior.

    “We found this dual-UUID approach necessary because provider-level KVcache handling can vary, and explicit boundaries provide more predictable caching behavior.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § 7. Conclusion

    Strategic cache boundary control, such as caching only system prompts while excluding dynamic tool results, provides more consistent benefits than naive full context caching, which can paradoxically increase latency.

    “Strategic cache boundary control, such as caching only system prompts while excluding dynamic tool results, provides more consistent benefits than naive full context caching, which can paradoxically increase latency.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Prompt caching requires exact prefix matches.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.2. Prompt Caching in Provider APIs

    Anthropic provides developer-controlled caching through explicit cache breakpoints, allowing users to specify which portions of their prompt should be cached, with configurable TTL options.

    “Anthropic provides developer-controlled caching through explicit cache breakpoints, allowing users to specify which portions of their prompt should be cached, with configurable time-to-live (TTL) options (Anthropic, 2026; 2025b).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Anthropic, 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites 2025b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 6.2. Ablation by Tool Count

    This increased variance likely reflects server load fluctuations during the ablation experiments.

    “This increased variance likely reflects server load fluctuations during the ablation experiments, as the main study (Section 4) showed more consistent TTFT improvements with lower variance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5.3. Provider Implementation Variability

    Our results reflect natural variance in API response times due to factors including server load, geographic distribution, and infrastructure differences across providers.

    “Our results reflect natural variance in API response times due to factors including server load, geographic distribution, and infrastructure differences across providers.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 3.1. Experimental Setup

    Each session starts with a fresh context, ensuring that cache benefits are measured within individual multi-turn conversations rather than across sessions.

    “Each session starts with a fresh context, ensuring that cache benefits are measured within individual multi-turn conversations rather than across sessions.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 3.3. Evaluation Protocol

    Between conditions for different cache modes, the study waits sufficient time (exceeding 24 hours) to ensure cache entries expire based on provider TTL policies, preventing cross-condition cache contamination.

    “Between conditions for different cache modes, we wait sufficient time (exceeding 24 hours) to ensure cache entries expire based on provider TTL policies, preventing cross-condition cache contamination.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 3.2. Cache Mode Implementation

    To control cache boundaries precisely, unique identifiers (UUIDs) are used to break the cache at specific points in the prompt, ensuring that content after the UUID is not cached from previous requests.

    “To control cache boundaries precisely, we use unique identifiers (UUIDs) to break the cache at specific points in the prompt, ensuring that content after the UUID is not cached from previous requests.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § 3.1. Experimental Setup

    DeepResearch Bench is a multi-turn agentic benchmark where agents autonomously execute web search tool calls to answer complex research questions.

    “We use DeepResearch Bench (Du et al., 2025) as our evaluation benchmark, a multi-turn agentic benchmark where agents autonomously execute web search tool calls to answer complex research questions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Du et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.3. Provider Implementation Variability

    Minimum token thresholds for cache eligibility range from 1,024 to 4,096 tokens depending on the provider and model.

    “Minimum token thresholds for cache eligibility range from 1,024 to 4,096 tokens depending on the provider and model (see Appendix A).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Appendix A

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: Prompt caching requires exact prefix matches.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 3.1. Experimental Setup

    All selected models support prompt caching through their respective APIs.

    “All selected models support prompt caching through their respective APIs.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given
    § 7. Conclusion

    We present the first comprehensive evaluation of prompt caching for long-horizon agentic tasks across three major LLM providers.

    “We present the first comprehensive evaluation of prompt caching for long-horizon agentic tasks across three major LLM providers.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.2. Prompt Caching in Provider APIs

    OpenAI offers automatic prompt caching on GPT-4o and newer models, where caching activates automatically for prompts exceeding a minimum token threshold, with cache hits occurring only for exact prefix matches.

    “OpenAI offers automatic prompt caching on GPT-4o and newer models, where caching activates automatically for prompts exceeding a minimum token threshold, with cache hits occurring only for exact prefix matches (OpenAI, 2026; 2024b).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites OpenAI, 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites 2024b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 6.3. Discussion

    The ablation study exhibits higher TTFT variance than the main study, reflecting the challenges of measuring latency in production API environments where server load and infrastructure changes introduce noise.

    “The ablation study exhibits higher TTFT variance than the main study, reflecting the challenges of measuring latency in production API environments where server load and infrastructure changes introduce noise.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.3. Provider Implementation Variability

    Prompt caching can introduce timing side-channels that may leak information about cached content.

    “recent work has demonstrated that prompt caching can introduce timing side-channels that may leak information about cached content (Wu et al., 2025; Gu et al., 2025).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Wu et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Gu et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: Prompt caching requires exact prefix matches.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisereasoning
    § 2.3. Agentic Workloads and Context Engineering

    Tool results often contain user-specific data that will not benefit other sessions, and the interleaving of static system prompts with dynamic tool outputs complicates cache reuse.

    “Tool results often contain userspecific data that will not benefit other sessions, and the interleaving of static system prompts with dynamic tool outputs complicates cache reuse.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § 3.4. Statistical Analysis

    Each cache condition is compared against the no-cache baseline using independent samples t-tests, with statistical significance determined at α = 0.05.

    “We compare each cache condition against the no-cache baseline using independent samples t-tests. Statistical significance is determined at α = 0 . 05 .”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given
    § 5.2. Tool Call Caching Considerations

    Cache creation incurs overhead on the first request, which is only amortized if subsequent requests benefit from cache reads.

    “However, cache creation incurs overhead on the first request, which is only amortized if subsequent requests benefit from cache reads.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Prompt caching requires exact prefix matches.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisereasoning
    § 2.2. Prompt Caching in Provider APIs

    Their focus on security auditing using synthetic prompts and smaller, older generation models does not compare caching strategies or evaluate cost and latency benefits for long-running agentic tasks on modern flagship models.

    “However, their focus on security auditing using synthetic prompts and smaller, older generation models does not compare caching strategies or evaluate cost and latency benefits for long-running agentic tasks on modern flagship models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.2. Prompt Caching in Provider APIs

    Recent work has audited prompt caching across 17 providers, demonstrating that cache hits produce measurable TTFT reductions and identifying security vulnerabilities from timing side-channels.

    “Recent work has audited prompt caching across 17 providers, demonstrating that cache hits produce measurable TTFT reductions and identifying security vulnerabilities from timing side-channels (Gu et al., 2025).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gu et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.2. Prompt Caching in Provider APIs

    Google offers both implicit caching, which activates automatically with no guaranteed cost savings, and explicit context caching, where developers create and reference caches with guaranteed discounts.

    “Google offers both implicit caching, which activates automatically with no guaranteed cost savings, and explicit context caching, where developers create and reference caches with guaranteed discounts (Google Cloud, 2026b; Kilpatrick, 2025).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Google Cloud, 2026b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Kilpatrick, 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 5.2. Tool Call Caching Considerations

    For tool calls that produce variable results or are unlikely to be repeated, caching provides no benefit and may introduce overhead.

    “For tool calls that produce variable results or are unlikely to be repeated, caching provides no benefit and may introduce overhead.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Prompt caching requires exact prefix matches.
39 of 39

  • The paper relies on this without support.

    DeepResearch Bench reflects realistic agentic workloads where context windows grow dynamically through tool invocations and results.

    “This benchmark reflects realistic agentic workloads where context windows grow dynamically through tool invocations and results.”

  • The paper relies on this without support.

    Prompt caching requires exact prefix matches.

    “Prompt caching requires exact prefix matches.”
    “Since prompt caching operates on exact prefix matches, we use UUIDs (indicated by red bars) to control cache boundaries.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prompt-cache economics shows that semantic compaction must account for cache disruption.

    “Promptcache economics further shows that semantic compaction must account for cache disruption (Lumer et al. 2026).”

    From Self-GC: Self-Governing Context for Long-Horizon LLM Agents (2026) · cites this paper as Lumer et al., 2026

    Section 5.1 says changes to a prompt can break its cached prefix and create cache-write overhead without reuse; Section 4.3 reports that caching dynamic content can offset latency benefits. The paper does not examine semantic compaction or establish its effects on cache reuse specifically, so applying these findings to compaction is an inference. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Stated session total does not match the experimental design

    Minor

    Four models, four cache conditions, and 40 sessions per model-condition pair imply 640 sessions, not the stated 500.

  • Reported TTFT improvement range excludes Gemini

    Minor

    The paper says TTFT improves by 13-31% across providers, but Table 1 reports a 6.1% improvement for Gemini 2.5 Pro.

  • The 500-token condition is described as both unable to benefit and cost-saving

    Minor

    The paper calls 500 tokens a control condition where caching cannot provide benefits, but later says cost savings are positive at every tested prompt size, which includes 500 tokens.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Lumer, E., Gulati, A., Subbiah, V. K., Basavaraju, P. H., and Burke, J. A. ScaleMCP: Dynamic and AutoSynchronizing Model Context Protocol Tools for LLM Agents, 2025b. URL https://arxiv.org/abs/ 2505.06416 .

Cited by

Sorted by citations

Authors
Lumer, E., Nizar, F., Jangiti, A., Frank, K., Gulati, A., Phadate, M., & Subbiah, V. K.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2601.06007
arXiv
2601.06007
OpenAlex
W7123254368
Cited by
0
References
31
Type
posted content
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
prompt caching, LLM agents, tool calling, API cost reduction, time-to-first-token

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review