Shan, Yizhou

Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads

Hu, C., Huang, H., Xu, L., Chen, X., Xu, J., Chen, S., Feng, H., Wang, C., Wang, S., Bao, Y., et al. (2024) · arXiv

Cloud Computing and Resource ManagementParallel Computing and Optimization Techniques

Cited by 8

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool

Hu, C., Huang, H., Hu, J., Xu, J., Chen, X., Xie, T., Wang, C., Wang, S., Bao, Y., Sun, N., et al. (2024) · arXiv

Parallel Computing and Optimization TechniquesCloud Computing and Resource Management

Cited by 6

EPIC: Efficient Position-Independent Caching for Serving Large Language Models

Hu, J., Huang, W., Wang, W., Wang, H., Hu, T., Zhang, Q., Feng, H., Chen, X., Shan, Y., & Xie, T. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 2

InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference

Pan, X., Li, E., Li, Q., Liang, S., Shan, Y., Zhou, K., Luo, Y., Wang, X., & Zhang, J. (2024) · arXiv

Advanced Data Storage TechnologiesParallel Computing and Optimization Techniques

Cited by 1