CoChat
Sign Up
PreprintCited by 0Natural Language Processing TechniquesTopic ModelingAdvanced Neural Network Applications

LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents

Open access
Jeon, Hyesung, Ha, Hyeongju, Kim, Jae-Joon·2026·arXiv
DOI 10.48550/arxiv.2602.01053arXiv 2602.01053
Open this paper in CoChat — free accountOpen PDF

Abstract

Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only by lightweight adapters. Despite sharing base model weights, each agent independently builds and stores its own KV cache for the same long, tool-augmented trajectories, incurring substantial memory and compute overhead. Existing KV cache sharing methods largely overlook this multi-LoRA setting. We observe that, cache differences across agents are dominated by adapter outputs, while activations from the shared pretrained backbone remain highly similar. Based on this observation, we propose LRAgent, a KV cache sharing framework for multi-LoRA agents. It decomposes the cache into two components, a shared base component derived from pretrained weights and an adapter-dependent component derived from LoRA weights. LRAgent reduces memory overhead by sharing the base component across agents and storing the adapter component in its inherent low-rank form. It also reduces computational overhead by sharing the low-rank cache, enabled by a shared-A multi-LoRA architecture. This avoids redundant computations for contexts that have already been processed by other agents. To efficiently reconstruct adapter contributions at runtime, we introduce Flash-LoRA-Attention, a kernel that reorders attention computation to avoid materializing the low-rank cache to full dimension. LRAgent achieves throughput and time-to-first-token latency close to fully shared caching, while preserving accuracy near the non-shared caching baseline across agentic question-answering benchmarks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2602.01053

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Research works printed but not linked: 43

  1. Bai, T., Yang, L., Wong, Z. H., Sun, F., Zhuang, X., Peng, J., et al. Efficient pretraining data selection for language models via multi-actor collaboration. In Proceedings of the 63rd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pp. 9465- 9491, Vienna, Austria, jul 2025. Association for Com- putational Linguistics. doi: 10.18653/v1/2025.acl-long. 466. URL https://aclanthology.org/2025. acl-long.466/. · DOI
  2. Bo, X., Chen, X., Dai, Q., Feng, X., Li, R., Wang, L., et al. Reflective multi-agent collaboration based on large language models. In Advances in Neural Information Pro- cessing Systems, volume 37, pp. 138595-138631, 2024. doi: 10.52202/079017-4397. · DOI
  3. Chang, C.-C., Lin, C.-Y., Akhauri, Y., Lin, W.-C., Wu, K.-C., Ceze, L., et al. xkv: Cross-layer svd for kv- cache compression. arXiv preprint arXiv:2503.18893, 2025. doi: 10.48550/arXiv.2503.18893. URL https: //arxiv.org/abs/2503.18893. · DOI · Paper page
  4. Dao, T., Fu, D. Y., Ermon, S., Rudra, A., and Re, C. Flashat- tention: Fast and memory-efficient exact attention with io-awareness. In Advances in Neural Information Pro- cessing Systems, volume 35, pp. 16344-16359, 2022. doi: 10.5555/3600270.3601459. URL https://github. com/Dao-AILab/flash-attention. Paper: arXiv:2205.14135, code repository linked in url field. · DOI
  5. DeepSeek-AI, Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. doi: 10.48550/arXiv.2412. 19437. URL https://arxiv.org/abs/2412. 19437. · DOI · Paper page
  6. Fu, D., He, K., Wang, Y., Hong, W., Gongque, Z., Zeng, W., et al. Agentrefine: Enhancing agent gen- eralization through refinement tuning. arXiv preprint arXiv:2501.01702, 2025a. · arXiv
  7. Fu, T., Min, Z., Zhang, H., Yan, J., Dai, G., Ouyang, W., et al. Cache-to-Cache: Direct semantic communica- tion between large language models. arXiv preprint arXiv:2510.03215, 2025b. doi: 10.48550/arXiv.2510. 03215. URL https://arxiv.org/abs/2510. 03215. · DOI · Paper page
  8. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., et al. LoRA: Low-rank adaptation of large lan- guage models. In International Conference on Learn- ing Representations, 2022. doi: 10.48550/arXiv.2106. 09685. URL https://openreview.net/forum? id=nZeVKeeFYf9. · DOI · Paper page
  9. Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. doi: 10.48550/arXiv. 2310.06825. · DOI · Paper page
  10. Li, B., Wang, Y., Ma, H., Chen, L., Xiao, J., and Wang, S. Mobilora: Accelerating lora-based llm inference on mobile devices via context-aware kv cache optimiza- tion. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 23400-23410, Vienna, Austria, July 2025. Association for Computational Linguistics. doi: 10.18653/v1/2025.acl-long.1140. URL https:// aclanthology.org/2025.acl-long.1140/. · DOI
  11. Liu, S., Chen, T., Liang, Z., Lyu, X., and Amato, C. Llm collaboration with multi-agent reinforcement learning. arXiv preprint arXiv:2508.04652, 2025a. · arXiv
  12. Liu, Y., Lin, K. Q., Chen, C. W., and Shou, M. Z. Video- mind: A chain-of-LoRA agent for long video reason- ing. arXiv preprint arXiv:2503.13444, 2025b. doi: 10.48550/arXiv.2503.13444. URL https://arxiv. org/abs/2503.13444. · DOI
  13. Liu, Y., Huang, Y., Yao, J., Feng, S., Gu, Z., Du, K., et al. DroidSpeak: KV cache sharing for cross-LLM commu- nication and multi-LLM serving. In Proceedings of the 23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI '26). USENIX Association, 2026. doi: 10.48550/arXiv.2411.02820. URL https: //arxiv.org/abs/2411.02820. Accepted for NSDI 2026. · DOI · Paper page
  14. Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, S.-C., Tafjord, O., Clark, P., and Kalyan, A. Learn to explain: Multimodal reasoning via thought chains for science question answering. In Advances in Neural Infor- mation Processing Systems, volume 35, pp. 2507-2521, 2022.
  15. Luong, H.-C. and Chen, B. Why LoRA fails to forget: Regularized low-rank adaptation against backdoors in language models. arXiv preprint arXiv:2601.06305, 2026. doi: 10.48550/arXiv.2601.06305. · DOI
  16. OpenAI, Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. doi: 10.48550/arXiv.2410. 21276. URL https://arxiv.org/abs/2410. 21276. · DOI · Paper page
  17. Pan, Z., Patel, A. D., Shen, Y., Hu, Z., Guan, Y., Li, W.-L., et al. KVFlow: Efficient prefix caching for accelerating LLM-based multi-agent workflows. In Advances in Neural Information Processing Systems (NeurIPS 2025), 2025. URL https://arxiv.org/ abs/2507.07400. · arXiv
  18. Qiao, S., Zhang, N., Fang, R., Luo, Y., Zhou, W., Jiang, Y., et al. AutoAct: Automatic agent learning from scratch for QA via self-planning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3003-3021, Bangkok, Thailand, August 2024. Association for Com- putational Linguistics. doi: 10.18653/v1/2024.acl-long. 165. URL https://aclanthology.org/2024. acl-long.165/. · DOI
  19. Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. In The Twelfth International Conference on Learning Representations, 2024.
  20. Rasal, S. Llm harmony: Multi-agent communication for problem solving. arXiv preprint arXiv:2401.01312, 2024. · arXiv
  21. Shen, W., Li, C., Chen, H., Yan, M., Quan, X., Chen, H., et al. Small llms are weak tool learners: A multi-llm agent. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 16658- 16680, 2024.
  22. Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: Language agents with verbal rein- forcement learning. In Advances in Neural Information Processing Systems, volume 36, pp. 8634-8652, 2023.
  23. Snell, C., Lee, J., Xu, K., and Kumar, A. Scaling llm test- time compute optimally can be more effective than scal- ing model parameters. arXiv preprint arXiv:2408.03314, 2024. doi: 10.48550/arXiv.2408.03314. · DOI · Paper page
  24. Stoica, G., Bolya, D., Bjorck, J., Wang, J., Hsiao, J., Fatemi, M., and Hoffman, J. Model merging with SVD to tie the knots. In The Thirteenth International Conference on Learning Representations, 2025.
  25. Subramaniam, V., Du, Y., Tenenbaum, J. B., Torralba, A., Li, S., and Mordatch, I. Multiagent finetuning: Self improvement with diverse reasoning chains. In International Conference on Learning Representations, 2025. doi: 10.48550/arXiv.2501.05707. URL https: //arxiv.org/abs/2501.05707. · DOI · Paper page
  26. Talebirad, Y. and Nadiri, A. Multi-agent collaboration: Harnessing the power of intelligent llm agents. arXiv preprint arXiv:2306.03314, 2023. · arXiv
  27. Tian, C., Shi, Z., Guo, Z., Li, L., and Xu, C. Hydralora: An asymmetric lora architecture for efficient fine-tuning. In Advances in Neural Information Processing Systems, volume 37, pp. 9565-9584, 2024. doi: 10.5555/3737916. 3738220. URL https://proceedings.neurips. cc/. · DOI
  28. Tomar, A., Hooper, C., Lee, M., Xi, H., Tiwari, R., Kang, W., et al. Xquant: Breaking the memory wall for llm inference with kv cache rematerialization. arXiv preprint arXiv:2508.10395, 2025. doi: 10.48550/arXiv.2508. 10395. URL https://arxiv.org/abs/2508. 10395. · DOI
  29. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288, 2023. doi: 10.48550/arXiv.2307.09288. URL https: //arxiv.org/abs/2307.09288. · DOI · Paper page
  30. Wang, Y., Lin, Y., Zeng, X., and Zhang, G. Multi- lora: Democratizing lora for better multi-task learn- ing. arXiv preprint arXiv:2311.11501, 2023. doi: 10.48550/arXiv.2311.11501. URL https://arxiv. org/abs/2311.11501. · DOI
  31. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Informa- tion Processing Systems, volume 35, pp. 24824-24837, 2022.
  32. Woo, S., Kim, H., Kil, J., Kim, M., Kim, J., Seo, A., et al. Icarus: Identical cache reuse for efficient multi-model inference. OpenReview, ICLR 2026 Conference Sub- mission, 2025. URL https://openreview.net/ forum?id=qrMo6R7lOS. Accessed 2026-01-02.
  33. Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., et al. Autogen: Enabling next-gen llm applications via multi- agent conversations. In First Conference on Language Modeling, jul 2024. URL https://openreview. net/forum?id=BAakY1hNKS.
  34. Xia, Y., Fu, F., Zhang, W., Jiang, J., and Cui, B. Efficient multi-task LLM quantization and serving for multiple LoRA adapters. In Advances in Neural Information Pro- cessing Systems, volume 37, pp. 63686-63714, 2024. doi: 10.5555/3737916.3739950. · DOI
  35. Yang, A., Yu, B., Li, C., Liu, D., Huang, F., Huang, H., et al. Qwen2.5-1m technical report. arXiv preprint arXiv:2501.15383, 2025a. doi: 10.48550/arXiv.2501. 15383. · DOI
  36. Yang, J., Hou, B., Wei, W., Bao, Y., and Chang, S. KVLink: Accelerating large language models via effi- cient KV cache reuse. arXiv preprint arXiv:2502.16002, 2025b. doi: 10.48550/arXiv.2502.16002. URL https: //arxiv.org/abs/2502.16002. · DOI · Paper page
  37. Yang, Y., Muhtar, D., Shen, Y., Zhan, Y., Liu, J., Wang, Y., et al. Mtl-lora: Low-rank adaptation for multi-task learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 22010-22018, 2025c. doi: 10.1609/aaai.v39i20. 35509. URL https://ojs.aaai.org/index. php/AAAI/article/view/35509. · DOI
  38. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhut- dinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369- 2380, Brussels, Belgium, 2018. Association for Compu- tational Linguistics. doi: 10.18653/v1/D18-1259. URL https://aclanthology.org/D18-1259/. · DOI · Paper page
  39. Yao, J., Li, H., Liu, Y., Ray, S., Cheng, Y., Zhang, Q., et al. CacheBlend: Fast large language model serving for RAG with cached knowledge fusion. In Proceed- ings of the Twentieth European Conference on Com- puter Systems (EuroSys '25), pp. 94-109, New York, NY, USA, 2025. Association for Computing Machin- ery. doi: 10.1145/3689031.3696098. URL https: //arxiv.org/abs/2405.16444. · DOI · Paper page
  40. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., et al. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Informa- tion Processing Systems, volume 36, pp. 11809-11822, 2023a.
  41. Ye, H., Gao, Z., Ma, M., Wang, Q., Fu, Y., Chung, M.- Y., et al. KVCOMM: Online cross-context KV-cache communication for efficient LLM-based multi-agent sys- tems. In Advances in Neural Information Processing Sys- tems (NeurIPS 2025), 2025. doi: 10.48550/arXiv.2510. 12872. URL https://arxiv.org/abs/2510. 12872. Accepted for publication in NeurIPS 2025. · DOI · Paper page
  42. Yu, X., Luo, T., Wei, Y., Lei, F., Huang, Y., Peng, H., et al. Neeko: Leveraging dynamic LoRA for efficient multi-character role-playing agent. In Proceedings of the 2024 Conference on Empirical Methods in Natu- ral Language Processing, pp. 12540-12557, Miami, Florida, USA, November 2024. Association for Compu- tational Linguistics. doi: 10.18653/v1/2024.emnlp-main. 697. URL https://aclanthology.org/2024. emnlp-main.697/. · DOI
  43. Yuan, Z., Shang, Y., Song, Y., Wu, Q., Yan, Y., Sun, G., et al. Asvd: Activation-aware singular value decomposition for compressing large language models. arXiv preprint arXiv:2312.05821, 2023. doi: 10.48550/arXiv.2312. 05821. URL https://arxiv.org/abs/2312. 05821. · DOI · Paper page

Authors
Jeon, H., Ha, H., & Kim, J.-J.
Affiliations
University of Seoul (KR)
ORCID
Kim, Jae-Joon
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2602.01053
arXiv
2602.01053
OpenAlex
W7127353950
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Ministry of Science and ICT, South Korea (BK21 FOUR, RS-2023)
  • Institute for Information and Communications Technology Promotion (RS-2025-02273157)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
LLM agents, LoRA adapters, memory efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review