Enabling Efficient Batch Serving for LMaaS via Generation Length Prediction

Cheng, Ke, Hu, Wen, Wang, Zhi · arXiv · 2024