Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time

Liu, Zichang, Desai, Aditya, Liao, Fangshuo · arXiv · 2023