MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool

Hu, Cunchen, Huang, Heyang, Hu, Junhao · arXiv · 2024