KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments

Park, Junyoung, Jones, Dalton, Morse, Matthew J · arXiv · 2025