Llumnix: Dynamic Scheduling for Large Language Model Serving

Sun, Biao, Huang, Ziming, Zhao, Hanyu · arXiv · 2024