RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
Open accessAbstract
Retrieval-Augmented Generation (RAG) has shown significant improvements in various natural language processing tasks by integrating the strengths of large language models (LLMs) and external knowledge databases. However, RAG introduces long sequence generation and leads to high computation and memory costs. We propose RAGCache, a novel multilevel dynamic caching system tailored for RAG. Our analysis benchmarks current RAG systems, pinpointing the performance bottleneck (i.e., long sequence due to knowledge injection) and optimization opportunities (i.e., caching knowledge's intermediate states). Based on these insights, we design RAGCache, which organizes the intermediate states of retrieved knowledge in a knowledge tree and caches them in the GPU and host memory hierarchy. RAGCache proposes a replacement policy that is aware of LLM inference characteristics and RAG retrieval patterns. It also dynamically overlaps the retrieval and inference steps to minimize the end-to-end latency. We implement RAGCache and evaluate it on vLLM, a state-of-the-art LLM inference system and Faiss, a state-of-the-art vector database. The experimental results show that RAGCache reduces the time to first token (TTFT) by up to 4x and improves the throughput by up to 2.1x compared to vLLM integrated with Faiss.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most prior KV reuse works assume prefix-match, which is impractical for RAG-systems.
“Most of these assume prefix-match [29,41,53,54,85], which is impractical for RAG-systems.”
From Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation (2025) · cites this paper as [41]
Cites
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · see citing works
Malkov, Y. A., & Yashunin, D. A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889473 · Cited by 1,904 · see citing works
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817 · see citing works
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · see citing works
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00638 · Cited by 1,612 · see citing works
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · see citing works
Proceedings of the VLDB Endowment
Alonso, G., Freire, J., Saçan, A., & Tatbul, N. (2011) · Cited by 653 · see citing works
Benchmarking Large Language Models in Retrieval-Augmented Generation
Chen, J., Lin, H., Han, X., & Sun, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29728 · Cited by 454 · see citing works
In-Context Retrieval-Augmented Language Models
Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00605 · Cited by 449 · see citing works
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.557 · Cited by 349 · see citing works
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337 · see citing works
Triton: an intermediate language and compiler for tiled neural network computations
Tillet, P., Kung, H. T., & Cox, D. (2019) · Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages · doi:10.1145/3315508.3329973 · Cited by 318 · see citing works
CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving
Liu, Y., Li, H., Cheng, Y., Ray, S., Huang, Y., Zhang, Q., Du, K., Yao, J., Lu, S., Ananthanarayanan, G., et al. (2024) · Proceedings of the ACM SIGCOMM 2024 Conference · doi:10.1145/3651890.3672274 · Cited by 118 · see citing works
(2023) · doi:10.52202/075280-1506 · Cited by 113 · see citing works
Self-citationImproving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination
Li, C., Zhang, M., Andersen, D. G., & He, Y. (2020) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3318464.3380600 · Cited by 90 · see citing works
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
Yao, J., Li, H., Liu, Y., Ray, S., Cheng, Y., Zhang, Q., Du, K., Lu, S., & Jiang, J. (2025) · Proceedings of the Twentieth European Conference on Computer Systems · doi:10.1145/3689031.3696098 · Cited by 75 · see citing works
Liu, Z., Desai, A., Liao, F., Wang, W., Xie, V., Xu, Z., Kyrillidis, A., & Shrivastava, A. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2279 · Cited by 50 · see citing works
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
Ye, L., Tao, Z., Huang, Y., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.15220 · Cited by 34 · see citing works
Prompt Cache: Modular Attention Reuse for Low-Latency Inference
Gim, I., Chen, G., Lee, S.-S., Sarda, N., Khandelwal, A., & Zhong, L. (2023) · arXiv · doi:10.48550/arxiv.2311.04934 · Cited by 18 · see citing works
MCCS: A Service-based Approach to Collective Communication for Multi-Tenant Cloud
Wu, Y., Xu, Y., Chen, J., Wang, Z., Zhang, Y., Lentz, M., & Zhuo, D. (2024) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3651890.3672252 · Cited by 15 · see citing works
MuxFlow: efficient GPU sharing in production-level clusters with more than 10000 GPUs
Liu, X., Zhao, Y., Liu, S., Li, X., Zhu, Y., Liu, X., & Jin, X. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4227-2 · Cited by 12 · see citing works
Self-citationHeterRAG: Heterogeneous Processing-in-Memory Acceleration for Retrieval-augmented Generation
Liu, C., Liu, H., Chen, D., Huang, Y., Zhang, Y., Xiao, W., Liao, X., & Jin, H. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731089 · Cited by 11 · see citing works
In-Storage Acceleration of Retrieval Augmented Generation as a Service
Mahapatra, R., Santhanam, H., Priebe, C., Xu, H., & Esmaeilzadeh, H. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731032 · Cited by 10 · see citing works
Hermes: Algorithm-System Co-design for Efficient Retrieval-Augmented Generation At-Scale
Shen, M., Umar, M., Maeng, K., Suh, G. E., & Gupta, U. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731076 · Cited by 5 · see citing works
SDCC: software-defined collective communication for distributed training
Jin, X., Zhang, Z., Jia, Y., Ma, Y., & Liu, X. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-3894-4 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
1 of the 26 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2026) · Data Science and Engineering · doi:10.1007/s41019-025-00335-5 · Cited by 549 · see citing works
LLM Inference Serving: Survey of Recent Advances and Opportunities
Li, B., Jiang, Y., Gadepally, V. N., & Tiwari, D. (2024) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec62836.2024.10938426 · Cited by 42 · see citing works
Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation
Agarwal, S., Sundaresan, S., Mitra, S., Mahapatra, D., Gupta, A., Sharma, R., Kapu, N. J., Yu, T., & Saini, S. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3725273 · Cited by 21 · see citing works
Integration of Generative AI and Mobile Networking: A Comprehensive Survey
Li, J., He, M., Zhou, C., Huang, X., Liu, Z., Zhao, L. Y., Wang, C., & Wu, H. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3639401 · Cited by 19 · see citing works
Improving the RAG-based Personalized Discharge Care System by Introducing the Memory Mechanism
Yang, Y., Xu, C., Guo, J., Feng, T., & Ruan, C. (2025) · International Conference on Computer Research and Development · doi:10.1109/iccrd64588.2025.10963086 · Cited by 15 · see citing works
HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems
Li, Y., Gu, R., Huan, C., Wang, Z., Yao, R., Tian, C., & Chen, G. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3749168 · Cited by 9 · see citing works
METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
Ray, S., Pan, R., Gu, Z., Du, K., Feng, S., Ananthanarayanan, G., Netravali, R., & Jiang, J. (2025) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3731569.3764855 · Cited by 17 · see citing works
Accelerating Retrieval Augmented Language Model via PIM and PNM Integration
Jang, J.-W., Oh, J., Kong, Y., Hong, J.-Y., Cho, S.-H., Lee, J., Yang, H., & Yang, J.-S. (2025) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3725843.3756020 · Cited by 5 · see citing works
Arısoy, M. V., Yalçınkaya, M. A., Gürfidan, R., & Arısoy, A. (2025) · Applied Sciences · doi:10.3390/app15041715 · Cited by 4 · see citing works
Hong, G.-H., Ouyang, T., Zhao, K., Zhou, Z., & Chen, X. (2025) · IEEE Real-Time Systems Symposium · doi:10.1109/rtss66672.2025.00022 · Cited by 5 · see citing works
Toward Sustainable Agentic AI Systems: A Survey of Architectures and Methodologies
Lee, Y., & Park, E. (2026) · Sustainable Development · doi:10.1002/sd.70942 · Cited by 5 · see citing works
Ruparel, H., & Patel, T. (2025) · Software Impacts · doi:10.1016/j.simpa.2025.100783 · Cited by 3 · see citing works
Malloy, T., Bernardy, L., Bachyr, O. E., Philippy, F., Samhi, J., Klein, J., & Bissyandé, T. F. (2025) · Electronics · doi:10.3390/electronics14193872 · Cited by 3 · see citing works
From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs
Wu, Y., Liang, S., Zhang, C., Wang, Y., Zhang, Y., Guo, H., Tang, R., & Liu, Y. (2025) · arXiv · doi:10.48550/arxiv.2504.15965 · Cited by 3 · see citing works
CARINA: An Efficient CXL-Oriented Embedding Serving System for Recommendation Models
Yin, P., Zhou, Q., Yan, X., Wang, C., Lo, E., Li, C., Lu, L., Fan, H. T., Zhou, W., Yang, M.-C., et al. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3725274 · Cited by 2 · see citing works
iKnow: an Intent-Guided Chatbot for Cloud Operations with Retrieval-Augmented Generation
Huang, J., Zhong, Y., Yu, G., Jiang, Z., Yan, M., 栾文飞, Yang, T., Ren, R., & Lyu, M. R.-T. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00084 · Cited by 2 · see citing works
VectorLiteRAG: Latency-Aware and Fine-Grained Resource Partitioning for Efficient RAG
Kim, J., & Mahajan, D. (2026) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca68181.2026.11408552 · Cited by 9 · see citing works
From Navigation to Intention: Reframing the Web Experience through Goal-Driven Interfaces
Cordioli, L., & Matera, M. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792479 · Cited by 2 · see citing works
Skyline Retrieval meets Set-Cover Chunk Merging: A Cost-Effective RAG-Sketch for Long-Context LLM QA
Zhu, X., Li, H., Zhang, Y., & Chen, L. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3802111 · Cited by 2 · see citing works
Chimera-VDB: Mixed-Precision Vector Database with HNSW Index for RAG-LLM
Yamane, N., Zielewski, M. R., Nakamura, T., & Suganuma, T. (2025) · ACM SIGOPS Asia-Pacific Workshop on Systems (APSys) · doi:10.1145/3725783.3764411 · Cited by 1 · see citing works
Caching at Scale: Efficiency and Fairness Analysis in Multi-tenant RAG Systems
Ruparel, H., & Patel, T. (2025) · SN Computer Science · doi:10.1007/s42979-025-04467-3 · Cited by 1 · see citing works
Functional Programming Meets Pinecone: Recommending Graph Structured Documents
Δρακόπουλος, Γ., Theodorakopoulos, L., Sioutas, S., & Phivos Mylonas, B. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402214 · Cited by 1 · see citing works
PKAS: Predictive KVCache-Aware Scheduling for Faster LLM and Transformer Inferences
Ye, J., Maurya, A., Chitty-Venkata, K. T., Nicolae, B., Kougkas, A., & Sun, X. (2026) · ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) · doi:10.1145/3806645.3807819 · Cited by 1 · see citing works
Enhancing RAG Systems: A Survey of Optimization Strategies for Performance and Scalability
Nishanth, S., & Swetha, S. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem35402 · Cited by 1 · see citing works
Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents
Zhang, Q., Wornow, M., Wan, G., & Olukotun, K. (2025) · arXiv · doi:10.48550/arxiv.2506.14852
Discovery Omnia: Dynamic RAG for Enhanced Patent Analysis and Systematic Innovation
Avogadri, S., Alzetta, G., & Russo, D. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-08847-5_6 · Cited by 1 · see citing works
Accelerating RAG Systems: A Performance-Oriented Systematic Mapping
da Silva, J. G. J., de Oliveira, S. S. T., & Filho, A. R. G. (2025) · doi:10.5753/eniac.2025.12304
Park, J., & Han, S. (2025) · Journal of Construction Automation and Robotics · doi:10.55785/jcar.4.4.9
Liu, Y., Wang, Z., Chen, Y., He, Y., Lian, J., & Lu, H. (2025) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc67331.2025.00010
Bauhaus: Restructuring Vector Database for LLM Retrieval on CXL-Based Tiered Memory
Kim, K., 안성수, Jeong, W., Kim, J., Choi, S., Gil, M., Kim, M., Jung, D., Hong, Y., Jung, H., et al. (2026) · IEEE Transactions on Computers · doi:10.1109/tc.2026.3656215 · Cited by 1 · see citing works
DKC-LLM: Dynamic Knowledge Caching for Large Language Models in Business Applications
Khaliq, A., & Adebayo, K. J. (2026) · IEEE Access · doi:10.1109/access.2026.3662344
Electricity Standard Knowledge Generation Based on Large Language Model
Fan, X., Zhang, S. Z. F., Liang, X., Wang, Z., & Peng, T. (2025) · International Conference on Power System and Energy Internet (PoSEI) · doi:10.1109/posei67825.2025.11382276
Hybrid Retrieval for Power Equipment Standard Knowledge
Zhang, S. Z. F., Fan, X., Song, B., Liang, X., Zhang, Q., Wang, Z., Xu, X., & Zou, Y. (2025) · International Conference on Smart Grid and Energy Engineering (SGEE) · doi:10.1109/sgee68429.2025.11385482
Venkatesan, V., Saha, P., Vasudevan, G., Balakumar, G., & Balu, G. P. (2026) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc67433.2026.11393771
Bonifazi, G., Buratti, C., Marchetti, M., Parlapiano, F., Traini, D., Domenico Ursino, A., & Virgili, L. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3803797
HaS: Accelerating RAG Through Homology-Aware Speculative Retrieval
Peng, P., Lin, W., Wu, W., Wang, X., & Liu, Y. (2026) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65706.2026.00143
Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
Li, J., Zhu, Y., Chen, B., Lee, E. K., & Nahrstedt, K. (2026) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3805621.3807662
Dealing with the Elephant in the KV Cache: Video Frame Sampling for Multimodal LLM Inference
Papaioannou, K., & Doudali, T. D. (2026) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3805621.3807613
RAG-Cache: Efficient Query-Specific Knowledge Caching for Retrieval Augmented Generation
Jin, H., & Feng, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0378-9_46
VLMCache: Efficient On-Device Vision-Language Model Inference
Zhang, Y., Xu, D., Chen, Z., Wang, C., Zhang, Y., Xu, M., & Huang, G. (2026) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3745756.3809243
RAPO: Retrieval-Augmented Phase Ordering
Yang, J., Lee, J.-H., Hong, Y., & Sung, H. (2026) · ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES) · doi:10.1145/3814943.3816172
CONCORD: Asynchronous Sparse Aggregation for Device-Cloud Rag Under Document Isolation
Hu, X., Tang, Z., Yao, Z., Wang, T., & Jia, W. (2026) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws72778.2026.00134
Load balancing for large language model inference serving: A control and learning oriented survey
Ma, H., Li, Z., Wang, J., Cao, C., Li, D., Yang, J., & Ding, Z. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134973
HeteroRAGCache: Software-Hardware Co-Design for Efficient RAG Caching using Emerging Memories
Park, J., Suh, K., Ponzina, F., & Rosing, T. (2026) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3787109.3815268
Co-Optimizing Request Scheduling and KV Caching for Edge LLM Serving
Li, X., Jiao, W., He, J., & Zhang, S. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3709703
Cache-Enhanced RAG and Graph-RAG for O-RAN
Bano, S., Nezami, Z., Hafeez, M., Zaidi, S. A., & Ahmed, Q. (2025) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps68340.2025.11591230
Omnia: Efficient RAG Serving through Speculative Scheduling
Fu, R., Li, S., Xu, Y., Wu, T., Ma, Z., & Shi, J. (2026) · ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) · doi:10.1145/3806645.3807814
InKubeator: Pre-warming In-Memory KV Caches from Disk for Elastic LLM Serving
Muthukumar, A., Gough, E., & Nadig, D. (2026) · International Conference on Communications · doi:10.1109/icc59461.2026.11587970
Reinforcement Learning for Adaptive Cache Policies in Retrieval-Augmented Generation Systems
Thatha, S., & Mudenagudi, U. (2025) · IEEE Conference on Engineering Informatics (ICEI) · doi:10.1109/icei68356.2025.11603697
Non-Redundant and Hotness-Aware Caching for Low-Latency Retrieval-Augmented Generation
Xie, Z., Wang, H., Zhang, Y.-A., & Jin, P. (2026) · IEEE International Conference on Distributed Computing Systems Workshops (ICDCSW) · doi:10.1109/icdcsw72724.2026.00013
Meridian: In-Memory Acceleration for RAG with Document Attention Decomposition
Liu, C., Huang, Y., Liu, H., Zhang, Y., Qihang, Q. I. U., Li, X., Zheng, L., Liao, X., Jin, H., & Xue, J. (2026) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca66397.2026.00041
TS-Memory: Plug-and-Play Memory for Time Series Foundation Models
Lyu, S., Zhong, S., Chen, T., Ruan, W., Liu, Q., Lv, T., Wen, Q., Wong, R. C.-W., & Liang, Y. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817614
Pick Up Where You Left Off: An Efficient Solution for Continuous Vector Similarity Search
Zheng, Z., Zeng, Y., Chi, Y., & Tong, Y. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817657
Efficient VLM Inference System With LoRA Adapters at the Edge
Wang, W., Mi, L., Chen, J., Yu, T., Cao, T., Li, Y., Li, M., Dai, H., Chen, G., & Liu, Y. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3723365
EC-RAG: Towards Efficient Edge-Cloud Retrieval-Augmented Generation Systems
Wang, L., Wang, K., Jia, R., Lu, K., Wan, J., Huo, H., Zhai, Y., Liang, Z., & Wang, D. (2026) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65706.2026.00082
Pegasus: Accelerating Large Language Model Inference with Stateful Prefix Caching
Chen, F., Li, P., Yu, D., & Guo, S. (2026) · IEEE International Conference on Joint Cloud Computing (JCC) · doi:10.1109/jcc72984.2026.00017
FlowCache: Workflow-Aware Intermediate-State Reuse for Multi-Tool LLM Serving
Wang, F., Li, M., Sheng, F., & Yang, K. (2026) · International Conference on Computer Science and Blockchain (CCSB) · doi:10.1109/ccsb71176.2026.11716474 · Cited by 1 · see citing works
Edge-Cloud Collaborative RAG with Multi-State Semantic Caching
Liu, K., Xu, G., Fu, Y., Ding, M., Wu, Y., Li, J., & Yang, L. (2026) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3832810.3832878
Untariyati, N. A., Adi, K., Widodo, A. P., & Uliniansyah, M. T. (2026) · Research Square · doi:10.21203/rs.3.rs-9879191/v1
Dhakal, A., Pokharel, P., & Adhikari, S. (2026) · European Journal of Applied Science Engineering and Technology · doi:10.59324/ejaset.2026.4(5).11
Gudadhe, S., Rajankar, A., Jaggi, S., & Bhat, K. (2026) · International Conference on Computing Communication Control and automation · doi:10.1109/iccubea69821.2026.11701973
An Evidence-Grounded Multimodal AI Framework for Historical Digital Personae in Cultural Heritage
Lipianina-Honcharenko, K., & Yurkiv, K. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10100351
Anchor-NDP: An importance-based prefix KV management framework for CXL-enabled NDP
Kim, H., Kim, W., & Han, T. H. (2026) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2026.104056
AdaCache: Efficient and Robust Noise-Resistant Semantic Caching for RAG
Chen, S., Zhang, L., & Lee, S.-W. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728681
- Authors
- Jin, C., Zhang, Z., Jiang, X., Liu, F., Liu, X., Liu, X., & Jin, X.
- Affiliations
- Peking University (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Volume
- 44
- Issue
- 1
- Pages
- 1-27
- ISSN
- 0734-2071 (ISSN-L)
- arXiv
- 2404.12457
- OpenAlex
- W4414371540
- Cited by
- 61
- References
- 26
- Citations by year
- 2026: 37 · 2025: 22 · 2024: 2
- FWCI
- 23.66
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Computer Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key Research and Development Program of China
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- retrieval-augmented generation, latency optimization, large language models, tree of knowledge, GPU memory hierarchy, time-to-first-token, vLLM, FAISS
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
