Abhyankar, Reyna

SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification

Miao, X., Oliaro, G., Zhang, Z., Cheng, X., Wang, Z., Zhang, Z., Wong, R. Y. Y., Zhu, A., Yang, L., Shi, X., et al. (2024) · Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3

Natural Language Processing TechniquesTopic Modeling

Cited by 114

Preble: Efficient Distributed Prompt Scheduling for LLM Serving

Srivatsa, V., He, Z., Abhyankar, R., Li, D., & Zhang, Y. (2024) · arXiv

Cloud Computing and Resource ManagementParallel Computing and Optimization Techniques

Cited by 1

InferCept: Efficient Intercept Support for Augmented Large Language Model Inference

Abhyankar, R., He, Z., Srivatsa, V., Zhang, H., & Zhang, Y. (2024) · arXiv

Natural Language Processing TechniquesParallel Computing and Optimization Techniques

Cited by 0