KIVI : Plug-and-play 2bit KV Cache Quantization with Streaming Asymmetric Quantization

Zirui Liu, Jiayi Yuan, Hongye Jin · Unpublished · 2023