Miao, Xupeng

SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification

Miao, X., Oliaro, G., Zhang, Z., Cheng, X., Wang, Z., Zhang, Z., Wong, R. Y. Y., Zhu, A., Yang, L., Shi, X., et al. (2024) · Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3

Natural Language Processing TechniquesTopic Modeling

Cited by 114

Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow

Mei, Y., Zhuang, Y., Miao, X., Yang, J., Jia, Z., & Vinayak, R. (2025) · Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1

Parallel Computing and Optimization TechniquesCloud Computing and Resource Management

Cited by 35

Official Implementation for "Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow"

Mei, Y., Zhuang, Y., Miao, X., Yang, J., Jia, Z., & Vinayak, R. (2024) · Zenodo

Parallel Computing and Optimization TechniquesCloud Computing and Resource Management

Cited by 0