Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference

Feng, Yuan, Lv, Junlin, Cao, Yukun · arXiv · 2024