Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention

Gao, Bin, He, Zhuomin, Sharma, Puru · arXiv · 2024