DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Holmes, Connor, Tanaka, Masahiro, Wyatt, Michael · arXiv · 2024