Predictive Multi-Tier Memory Management for KV Cache in Large-Scale GPU Inference

Ganjihal, Sanjeev Rao · arXiv · 2026