CoChat
Sign Up
PreprintCited by 15Software Engineering ResearchSoftware Engineering Techniques and PracticesNatural Language Processing Techniques

Reducing Cost of LLM Agents with Trajectory Reduction

Open access
Xiao, Yuan-An, Gao, Pengfei, Peng, Chao, Xiong, Yingfei·2025·arXiv
DOI 10.48550/arxiv.2509.23586arXiv 2509.23586

Preprint — published as 10.1145/3797084

Open this paper in CoChat — free account

Abstract

Multi-turn agent systems based on Large Language Models (LLMs) have become increasingly popular for software engineering tasks. While LLM agents demonstrate promising effectiveness, the high computational cost of input tokens due to ever-growing trajectories remains a significant efficiency concern. Efficiency has been largely overlooked in existing studies and agent products, and this paper addresses this gap by introducing an inference-time trajectory reduction approach that reduces computational costs. By analyzing existing agent trajectories, we demonstrate that useless, redundant, and expired information is widespread across trajectories. Such waste can be identified and reduced without compromising the agent's performance. We propose a simple yet effective trajectory reduction approach, AgentDiet, which automatically removes such waste during agent execution. We implement AgentDiet on a top-performing coding agent, and our evaluation on two LLMs and two benchmarks shows that AgentDiet can reduce input tokens by 39.9%-59.7% and the total computational cost by 21.1%-35.9%, while maintaining the same agent performance. These results indicate that inference-time trajectory reduction is a promising direction for agent systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2509.23586

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Lightweight scorers that filter observations to retain only task-relevant features preserve critical context bandwidth without sacrificing trajectory accuracy.

    “This preserves critical context bandwidth without sacrificing trajectory accuracy, as evidenced in recent programming agents [157].”

    From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [157]

    The paper’s opening summary (p. 1) reports that AgentDiet reduces input tokens while maintaining coding-agent performance, and its case study (p. 6) shows verbose test output removed while the failing test is preserved. But the method uses an LLM-based reflection module (p. 7), not lightweight scorers, and the reported outcome is agent performance rather than a measure of trajectory accuracy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Xiao et al.'s 'Delete' baseline is more efficient than LLM-Summary at a comparable downstream performance.

    “Irrespective of this, their results show that this baseline is more efficient than LLM-Summary at a comparable downstream performance.”

    From The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management (2025) · cites this paper as [34]

    Table 2 (Section 4.2, p. 12) shows that Delete uses fewer input tokens and has lower reported costs than most LLM-based reflection variants, while its 58% pass rate is below their 60–65%. The accompanying discussion calls Delete’s performance reasonably good but warns that it takes more steps and is generally impractical in production. The paper does not identify a comparator named “LLM-Summary,” so the specific comparison is not fully established. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Xiao et al. do not compare with an Observation Masking baseline.

    “However, they do not compare with an Observation Masking baseline.”

    From The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management (2025) · cites this paper as [34]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 35

  1. [1][1] Toufique Ahmed, Kunal Suresh Pai, Premkumar Devanbu, and Earl Barr. 2024. Automatic semantic augmentation of language model prompts (for code summarization). In Proceedings of the IEEE/ACM 46th international conference on software engineering . 1-13. -
  2. [9][9] Zhiyu Fan, Kirill Vasilevski, Dayi Lin, Boyuan Chen, Yihao Chen, Zhiqing Zhong, Jie M Zhang, Pinjia He, and Ahmed E Hassan. 2025. Swe-effi: Re-evaluating software ai agent system effectiveness under resource constraints. arXiv preprint arXiv:2509.09853 (2025). - · arXiv
  3. [10][10] Mahdi Farzandway and Fatemeh Ghassemi. 2025. Automated Repair of C Programs Using Large Language Models. arXiv preprint arXiv:2509.01947 (2025). - · arXiv
  4. [11][11] Pengfei Gao, Zhao Tian, Xiangxin Meng, Xinchen Wang, Ruida Hu, Yuanan Xiao, Yizhou Liu, Zhao Zhang, Junjie Chen, Cuiyun Gao, et al. 2025. Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling. arXiv preprint arXiv:2507.23370 (2025). - · arXiv
  5. [12][12] Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023. Impact of Code Language Models on Automated Program Repair. arXiv:2302.05020 [cs] - · arXiv
  6. [13][13] Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? arXiv:2310.06770 [cs] - · arXiv
  7. [14][14] Sungmin Kang, Bei Chen, Shin Yoo, and Jian-Guang Lou. 2025. Explainable Automated Debugging via Large Language Model-Driven Scientific Debugging. Empirical Software Engineering 30, 2 (March 2025), 45. doi:10.1007/s10664-02410594-x - · DOI
  8. [17][17] Yucheng Li, Bo Dong, Chenghua Lin, and Frank Guerin. 2023. Compressing context to enhance inference efficiency of large language models. arXiv preprint arXiv:2310.06201 (2023). - · arXiv
  9. [19][19] Barys Liskavets, Maxim Ushakov, Shuvendu Roy, Mark Klibanov, Ali Etemad, and Shane K Luke. 2025. Prompt compression with context-aware sentence encoding for fast and improved llm inference. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 39. 24595-24604. -
  10. [20][20] Junwei Liu, Kaixin Wang, Yixuan Chen, Xin Peng, Zhenpeng Chen, Lingming Zhang, and Yiling Lou. 2024. Large Language Model-Based Agents for Software Engineering: A Survey. arXiv:2409.02977 - · arXiv
  11. [21][21] Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172 (2023). - · arXiv
  12. [22][22] Maxime Louis, Hervé Déjean, and Stéphane Clinchant. 2025. Pisco: Pretty simple compression for retrieval-augmented generation. arXiv preprint arXiv:2501.16075 (2025). - · arXiv
  13. [23][23] Piotr Nawrot, Adrian Łańcucki, Marcin Chochowski, David Tarjan, and Edoardo M Ponti. 2024. Dynamic memory compression: Retrofitting llms for accelerated inference. arXiv preprint arXiv:2403.09636 (2024). - · arXiv
  14. [28][28] Justyna Petke, Matias Martinez, Maria Kechagia, Aldeida Aleti, and Federica Sarro. 2024. The patch overfitting problem in automated program repair: Practical magnitude and a baseline for realistic benchmarking. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering . 452-456. -
  15. [29][29] Yihao Qin, Shangwen Wang, Yiling Lou, Jinhao Dong, Kaixin Wang, Xiaoling Li, and Xiaoguang Mao. 2024. Agentfl: Scaling llm-based fault localization to project-level context. arXiv preprint arXiv:2403.16362 (2024). - · arXiv
  16. [30][30] Md Rafiqul Islam Rabin, Aftab Hussain, and Mohammad Amin Alipour. 2022. Syntax-guided program reduction for understanding neural code intelligence models. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming . 70-79. -
  17. [31][31] Mootez Saad, José Antonio Hernández López, Boqi Chen, Dániel Varró, and Tushar Sharma. 2025. An Adaptive Language-Agnostic Pruning Method for Greener Language Models for Code. Proceedings of the ACM on Software Engineering 2, FSE (2025), 1183-1204. -
  18. [32][32] Sahil Suneja, Yunhui Zheng, Yufan Zhuang, Jim A Laredo, and Alessandro Morari. 2021. Probing model signalawareness via prediction-preserving input minimization. In Proceedings of the 29th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering . 945-955. -
  19. [33][33] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017). -
  20. [34][34] Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. Executable Code Actions Elicit Better LLM Agents. arXiv:2402.01030 [cs] - · arXiv
  21. [35][35] Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. 2024. OpenHands: An Open Platform for AI Software Developers as Generalist Agents. arXiv:2407.16741 [cs] - · arXiv
  22. [36][36] Yan Wang, Xiaoning Li, Tien N Nguyen, Shaohua Wang, Chao Ni, and Ling Ding. 2024. Natural is the best: Modelagnostic code simplification for pre-trained large language models. Proceedings of the ACM on Software Engineering 1, FSE (2024), 586-608. -
  23. [37][37] Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, and Graham Neubig. 2023. Learning to filter context for retrieval-augmented generation. arXiv preprint arXiv:2311.08377 (2023). - · arXiv
  24. [38][38] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824-24837. -
  25. [39][39] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. Agentless: Demystifying LLM-based Software Engineering Agents. arXiv:2407.01489 [cs] - · arXiv
  26. [40][40] Chunqiu Steven Xia and Lingming Zhang. 2024. Automated program repair via conversation: Fixing 162 out of 337 bugs for $0.42 each using chatgpt. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis . 819-831. -
  27. [41][41] Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2024. RECOMP: Improving retrieval-augmented LMs with context compression and selective augmentation. In The Twelfth International Conference on Learning Representations . -
  28. [42][42] Guang Yang, Yu Zhou, Wei Cheng, Xiangyu Zhang, Xiang Chen, Terry Yue Zhuo, Ke Liu, Xin Zhou, David Lo, and Taolue Chen. 2024. Less is more: Docstring compression in code generation. arXiv preprint arXiv:2410.22793 (2024). - · arXiv
  29. [43][43] John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. arXiv:2405.15793 [cs] - · arXiv
  30. [44][44] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR) . -
  31. [45][45] Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, et al. 2025. Multi-swe-bench: A multilingual benchmark for issue resolving. arXiv preprint arXiv:2504.02605 (2025). - · arXiv
  32. [46][46] Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. AutoCodeRover: Autonomous Program Improvement. arXiv:2404.05427 [cs] - · arXiv
  33. [47][47] Zhaowei Zhang, Hongyu Zhang, Beijun Shen, and Xiaodong Gu. 2022. Diet code is healthy: Simplifying programs for pre-trained models of code. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 1073-1084. -
  34. [48][48] Shangziqi Zhao, Jiahao Yuan, Guisong Yang, and Usman Naseem. 2025. Can pruning improve reasoning? revisiting long-cot compression with capability in mind for better reasoning. arXiv preprint arXiv:2505.14582 (2025). - · arXiv
  35. [49][49] Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. 2025. MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents. arXiv preprint arXiv:2506.15841 (2025). Received 2025-09-12; accepted 2025-12-22 · arXiv

Cited by

Sorted by citations

Related

Authors
Xiao, Y.-A., Gao, P., Peng, C., & Xiong, Y.
Affiliations
  • Peking University (CN)
  • ByteDance (China) (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Volume
3
Issue
FSE
Pages
1241-1263
ISSN
2994-970X (ISSN-L)
DOI
10.48550/arxiv.2509.23586
arXiv
2509.23586
OpenAlex
W4415332588
Cited by
4
References
0
Citations by year
2026: 4
FWCI
34.03
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the ACM on software engineering. (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (92582202)
  • National Key Research and Development Program of China (2022YFB4501902)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
LLM agents, computational complexity reduction, inference-time optimization, redundancy elimination, coding agents, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review