Cited by 3Topic ModelingArtificial Intelligence ApplicationsNatural Language Processing Techniques

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

Smarter AI Agents: Optimizing Tokens the Right Way

Open access
Kommuru, Madhurima·2025·International Journal of Machine Learning and Predictive Analytics

Abstract

AI agents driven by large language models (LLMs) are radically changing industries through methods like automation, decision support, and intelligent interactions. As a result, the efficiency of these systems is as crucial as their capabilities. In fact, one of the most critical factors influencing AI's cost-effectiveness, speed, scalability, and user experience is token optimization, a factor often ignored in performance considerations. Some of the characteristics of the very modern AI workflows that can lead to a token explosion include deep prompting, several agents' interactions, retrieval of memories, and persistent context sharing. Such overuse of tokens has a double effect of continually increasing the expenditure and leading to unpleasant situations like lag, context overflow, deterioration of expected response, and wasting of resources. Alongside the contribution of AI agents to real-time applications, their critical nature is reminding us of the necessity to find ways of managing tokens intelligently to ensure a balance between performance and efficiency. This article presents a series of feasible and potent methods for optimizing token consumption in AI-driven systems at a minimum level without sacrificing the quality of outputs or the extent of contextual understanding. The methods proposed are prompt engineering antediluvian, context compression, memory selection, response generation, retrieval and adaptive token allocation that are area-specific and task-oriente…

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes a modular token-optimization framework for LLM-based AI agents that combines dynamic context filtering, relevance scoring, adaptive token allocation, semantic retrieval, prompt compression, and memory management. In a simulated enterprise customer-support case study, it reports 40–45% token reduction and about 30% latency improvement while maintaining 92–95% response accuracy.

Key findings

  • The optimized AI agent reduced token consumption by 40% on average versus the conventional agent; in long conversation histories and repeated knowledge-retrieval sessions, savings reached almost 45%. Table 2 reports a 40–45% reduction.
  • Average response latency was nearly 30% shorter, which the paper links to smaller, more targeted prompts and removal of irrelevant context before inference.
  • Response accuracy remained around 92%–95%, with better contextual precision and lower hallucination frequency in complex reasoning and multi-step interactions.
  • Operational cost and scalability improved: Table 2 lists significant API cost reduction, better enterprise deployment with large contexts, reduced memory redundancy, and improved multi-agent communication efficiency.

Methods

  • The proposed framework is a modular architecture with input processing, context filtering, prompt optimization, memory management, retrieval, and adaptive response generation. Its token-management strategy includes dynamic context filtering, relevance scoring, priority-based token allocation, sliding context windows, and prompt optimization.
  • The case study evaluates an AI enterprise customer-support assistant using a simulated enterprise dataset containing FAQs, internal documents, support tickets, user histories, and workflow instructions. It compares a conventional LLM-based agent using static prompts and full context against the proposed optimized agent using dynamic token allocation, semantic retrieval, and adaptive context filtering.
  • Implementation used Python-based AI frameworks, embedding models, FAISS and Pinecone vector databases, and LangChain/CrewAI for orchestration. Monitoring tracked token usage, response latency, API cost, and context efficiency; evaluation metrics included token consumption, latency, response quality, and operational cost.

Limitations

  • The paper states that the framework depends heavily on retrieval quality: if embedding generation or relevance scoring is inaccurate, important context may be missed and response quality or reasoning may degrade. It also notes that summarization can remove critical information needed for complex reasoning, and that the optimization layer adds pre-inference computational load, so small applications with limited workloads may not benefit substantially.
  • Evident from the text: the evaluation is a simulated enterprise customer-support case study rather than a real production deployment. The paper does not report sample size, statistical tests, or detailed dataset characteristics.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.67228/3142788x/ijmlpa-2025pi3w9k

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

54 Own interpretation · 13 Own result · 3 No basis given

Show all 16 claims

  • It may state two separate claims as one. 36
  • Probably not a substantial claim. 43
54 of 54
  • The paper relies on this without support.

    Token optimization is a critical factor influencing AI's cost-effectiveness, speed, scalability, and user experience.

    “In fact, one of the most critical factors influencing AI's cost-effectiveness, speed, scalability, and user experience is token optimization, a factor often ignored in performance considerations.”
  • The paper relies on this without support.

    LLM providers bill customers according to the number of tokens consumed.

    “Since most LLM providers bill customers according to the number of tokens consumed, heavy usage will result in higher costs.”
  • The paper relies on this without support.

    There is a lack of standard token optimization frameworks throughout AI pipelines.

    “Another significant problem is the lack of standard token optimization frameworks throughout AI pipelines.”
  • The paper relies on this without support.

    Tokens directly influence API cost, required computation, and inference speed.

    “Since tokens directly influence the cost of APIs, the required computation and the speed of inference, inefficient handling of tokens can lead to significant hikes in the operational expenses of businesses that deploy AI on a large scale.”
  • The paper relies on this without support.

    Enterprises using LLM-powered applications may process millions of tokens daily.

    “Enterprises that make use of LLM-powered applications might be processing millions of tokens each day, thus incurring high infrastructure and subscription costs.”
  • The paper relies on this without support.

    Most existing optimization approaches focus on model capability and ignore operational efficiency.

    “Most of the existing ones just focus on model capability and completely ignore operational efficiency.”
  • The paper relies on this without support.

    Prompt compression reduces non-essential text while retaining the main context.

    “A very popular approach, which is also shared by many is prompt compression, which basically aims at reducing non-essential texts while the main context is retained.”
  • The paper relies on this without support.

    Prompt engineering tactics such as instruction simplification, template refinement, and context summarization do not lead to a drop in response quality, yet reduce token consumption to a minimum level.

    “Prompt engineering are tactics such as instruction simplification, template refinement and context summarization, which do not lead to a drop in response quality, yet they are able to bring down token consumption to a minimum level.”
  • The paper relies on this without support.

    Context pruning removes irrelevant or outdated parts from the input context window.

    “Besides it, another major optimization method is context pruning, which entails the removal of irrelevant or outdated parts in a text context window from the input.”
  • The paper relies on this without support.

    Existing AI workflows are inefficient in token use due to prompt redundancies, holding on to unnecessary memories, and bad context management practices.

    “Unfortunately, a lot of the AI workflows that already exist are very inefficient in terms of how they use tokens, mainly because of prompt redundancies, holding on to unnecessary memories, and bad context management practices.”
  • The paper relies on this without support.

    There is a trade-off between having rich context and staying computationally efficient.

    “Also, there's a considerable trade-off between having a rich context for the information and staying computationally efficient.”
  • The paper relies on this without support.

    When a prompt is overtainted with irrelevant information, the model finds it difficult to spot critical instructions or determine the most useful knowledge to base the response on.

    “When a prompt is overtainted with irrelevant information, the model will find it very difficult to spot critical instructions or it simply may not figure out which piece of knowledge is the most useful one to base the response on.”
  • The paper relies on this without support.

    Large token consumption in enterprise AI contexts strains infrastructure, causes unpredictable expenses, and performance issues.

    “Large token consumption in such contexts will definitely result in the strain of infrastructure, unpredictable expenses and performance issues.”
  • The paper relies on this without support.

    AI agents integrate memory systems to keep track of context and make better decisions.

    “Nowadays, AI agents integrate memory systems to keep track of the context of a situation and to make better decisions.”
  • The paper relies on this without support.

    Working memory stores recent events and details about a task; long-term memory keeps a record of knowledge, likes, and experiences of a user over a long time.

    “Working memory or short-term memory is a storage of recent events and details about a task so that the agents are able to carry on a conversation and reason instantly. Conversely, long-term memory keeps a record of knowledge, likes, and experiences of a user over a long time.”
  • The paper relies on this without support.

    Advanced reasoning, supported by LLMs, enables AI agents to problem-solve, analyze strategically, and learn adaptively during real-time applications.

    “Advanced reasoning, supported by LLMs, enables AI agents to problem-solve, analyze strategically, and learn adaptively during the deployment of real-time applications.”
  • The paper relies on this without support.

    Incorrectly formulated prompts in AI workflows induce repetition of instructions, addition of unnecessary context details, and overformatting, increasing tokens used and computation costs.

    “Typically, in AI workflows, incorrectly formulated prompts induce the repetition of instructions, adding of unnecessary context details, and overformatting, all of which result in an increase of tokens used as well as computation costs.”
  • The paper relies on this without support.

    Working with structured prompts leads to better and reliable answers over multiple communications.

    “Besides, working with structured prompts leads to better and reliable answers over multiple communications.”
  • The paper relies on this without support.

    Similarity searches in vector databases are very rapid and support query-aware context injection, drastically cutting down the need for unnecessary prompt expansion.

    “The similarity searches in these databases are very rapid and they also support query-aware context injection, which drastically cuts down on the need for unnecessary prompt expansion.”
  • The paper relies on this without support.

    API providers charge based on how many tokens are used.

    “Typically, since API providers charge based on how many tokens are used, a decrease in token consumption equals a direct reduction of deployment costs.”
  • The paper relies on this without support.

    The operation of large-scale AI models requires a good deal of computer power and energy during inference.

    “The operation of large-scale AI models requires a good deal of computer power and energy, which is even just during the process of inference.”
  • The paper relies on this without support.

    Using more tokens than necessary increases environmental impact.

    “And when we use even more tokens than necessary for the processing, we increase the environmental impact due to the already existing high level of resource consumption by these models.”
  • The paper relies on this without support.

    AI systems have traditionally found it difficult to handle an extended number of conversation histories, increasing number of users, and rising demand for computing power.

    “AI systems have traditionally found it difficult to handle an extended number of conversation histories, increase the number of users, and the rising demand for computing power.”
23 of 23

Citations

Cites

Show all 28 references

Metadata

Authors
Kommuru, M.
Corresponding author
Kommuru, Madhurima
Venue
International Journal of Machine Learning and Predictive Analytics
Published
2025
Volume
8
Pages
01-19
ISSN
3142-788X (ISSN-L)
OpenAlex
W7203714902
Cited by
3
References
28
Citations by year
2026: 3
FWCI
0.98
Type
journal article
Language
en
Open access
Locations
International Journal of Machine Learning and Predictive Analytics (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
token efficiency, large language models, AI agents, prompt engineering, context compression, memory selection, token allocation, agent cooperation, cost-effectiveness, resource efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review