CoChat
Sign Up
PreprintCited by 1Topic ModelingNatural Language Processing TechniquesArtificial Intelligence Applications

CompactPrompt: A Unified Pipeline for Prompt Data Compression in LLM Workflows

Open access
Choi, Joong Ho, Zhao, Jiayang, Shah, Jeel, Sonawane, Ritvika, Singh, Vedant, Appalla, Avani, Flanagan, Will, Condessa, Filipe·2025·arXiv
DOI 10.48550/arxiv.2510.18043arXiv 2510.18043
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) deliver powerful reasoning and generation capabilities but incur substantial run-time costs when operating in agentic workflows that chain together lengthy prompts and process rich data streams. We introduce CompactPrompt, an end-to-end pipeline that merges hard prompt compression with lightweight file-level data compression. CompactPrompt first prunes low-information tokens from prompts using self-information scoring and dependency-based phrase grouping. In parallel, it applies n-gram abbreviation to recurrent textual patterns in attached documents and uniform quantization to numerical columns, yielding compact yet semantically faithful representations. Integrated into standard LLM agents, CompactPrompt reduces total token usage and inference cost by up to 60% on benchmark dataset like TAT-QA and FinQA, while preserving output quality (Results in less than 5% accuracy drop for Claude-3.5-Sonnet, and GPT-4.1-Mini) CompactPrompt helps visualize real-time compression decisions and quantify cost-performance trade-offs, laying the groundwork for leaner generative AI pipelines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2510.18043

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Part of this paper’s reference list could not be split into single entries.

Research works printed but not linked: 18

  1. [2][2] Tom B. Brown, Ben Mann, Nick Ryder, Mira Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and Dario Amodei. 2020. Language Models are Few- Shot Learners. In Advances in Neural Information Processing Systems, Vol. 33. 1877–1901.
  2. [3][3] Zhiyu Chen,WenhuChen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. FinQA: A Dataset of Nu- merical Reasoning over Financial Data. Proceedings of EMNLP 2021 (2021).
  3. [4][4] Yu-Neng Chuang, Tianwei Xing, Chia-Yuan Chang, Zirui Liu, Xun Chen, and Xia Hu. 2024. Learning to Compress Prompt in Natural Language Formats. arXiv:2402.18700 [cs.CL] https://arxiv.org/abs/ 2402.18700 · arXiv
  4. [5][5] Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. arXiv:2310.05736 https://arxiv.org/abs/2310. 05736 · arXiv
  5. [6][6] Huiqiang Jiang, QianhuiWu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression. arXiv:2310.06839 [cs.CL] https://arxiv.org/abs/2310.06839 · arXiv
  6. [8][8] Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024. Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models. arXiv:2402.14848 [cs.CL] https://arxiv.org/ abs/2402.14848 · arXiv
  7. [9][9] Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024. Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). Association for Computational Linguistics, Bangkok, Thailand, 15339–15353. https: //aclanthology.org/2024.acl-long.818.pdf
  8. [10][10] Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, and Wenhu Chen. 2024. Long-context LLMs Struggle with Long In-context Learning. arXiv:2404.02060 [cs.CL] https://arxiv.org/abs/2404.02060 https://www.anthropic.com/news/claude-3-5-sonnet https://www.anthropic.com/news/claude-3-5-sonnet https://arxiv.org/abs/2402.18700 https://arxiv.org/abs/2402.18700 https://arxiv.org/abs/2402.18700 https://arxiv.org/abs/2310.05736 https://arxiv.org/abs/2310.05736 https://arxiv.org/abs/2310.05736 https://arxiv.org/abs/2310.06839 https://arxiv.org/abs/2310.06839 https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT https://arxiv.org/abs/2402.14848 https://arxiv.org/abs/2402.14848 https://arxiv.org/abs/2402.14848 https://aclanthology.org/2024.acl-long.818.pdf https://aclanthology.org/2024.acl-long.818.pdf https://arxiv.org/abs/2404.02060 https://arxiv.org/abs/2404.02060 CompactPrompt: A Unified Pipeline for Prompt and Data Compression in LLM Workflows ACM ICAIF '25, November 15, Singapore · arXiv
  9. [11][11] Yucheng Li, Bo Dong, Chenghua Lin, and Frank Guerin. 2023. Com- pressing Context to Enhance Inference Efficiency of Large Language Models. arXiv:2310.06201 https://arxiv.org/abs/2310.06201 · arXiv
  10. [12][12] Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. 2024. Prompt Compression for Large Language Models: A Survey. arXiv:2410.12388 [cs.CL] https://arxiv.org/abs/2410.12388 · arXiv
  11. [13][13] Zongqian Li, Yixuan Su, and Nigel Collier. 2024. 500xCompres- sor: Generalized Prompt Compression for Large Language Models. arXiv:2408.03094 https://arxiv.org/abs/2408.03094 · arXiv
  12. [14][14] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172 [cs.CL] https://arxiv.org/abs/2307.03172 · arXiv
  13. [18][18] Jesse Mu, Xiang Lisa Li, and Noah Goodman. 2024. Learning to Com- press Prompts with Gist Tokens. arXiv:2304.08467 https://arxiv.org/ abs/2304.08467 · arXiv
  14. [24][24] X. Shi, C. Zhao, and Y. Liu. 2023. Optimizing Prompt Engineering: A Comprehensive Survey. Journal of Artificial Intelligence Research 78 (2023), 105–134.
  15. [25][25] Terry A. Welch. 1984. A Technique for High-Performance Data Com- pression. Computer 17, 6 (June 1984), 8–19. doi:10.1109/MC.1984. 1659158 · DOI
  16. [26][26] Qianchi Zhang, Hainan Zhang, Liang Pang, Hongwei Zheng, and Zhiming Zheng. 2024. AdaComp: Extractive Context Compression with Adaptive Predictor for Retrieval-Augmented Large Language Models. arXiv:2409.01579 [cs.CL] https://arxiv.org/abs/2409.01579 · arXiv
  17. [27][27] Tinghui Zhang, Yifan Wang, and Daisy Zhe Wang. 2025. SCOPE: A Generative Approach for LLM Prompt Compression. arXiv:2508.15813 [cs.CL] https://arxiv.org/abs/2508.15813 · arXiv
  18. [28][28] Zheng Zhang, Jinyi Li, Yihuai Lan, Xiang Wang, and Hao Wang. 2025. An Empirical Study on Prompt Compression for Large Language Mod- els. arXiv:2505.00019 [cs.CL] https://arxiv.org/abs/2505.00019 · arXiv

Cited by

Sorted by citations

Related

Authors
Choi, J. H., Zhao, J., Shah, J., Sonawane, R., Singh, V., Appalla, A., Flanagan, W., & Condessa, F.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2510.18043
arXiv
2510.18043
OpenAlex
W4416054951
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by-sa/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
prompt compression, large language models, LLM agents, token pruning, uniform quantization, inference cost reduction, cost-performance trade-off, question answering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review