Aminabadi, Reza Yazdani

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Aminabadi, R. Y., Rajbhandari, S., Awan, A. A., Li, C., Li, D., Zheng, E., Ruwase, O., Smith, S., Zhang, M., Rasley, J., et al. (2022) · SC22: International Conference for High Performance Computing, Networking, Storage and Analysis

Parallel Computing and Optimization TechniquesAdvanced Neural Network Applications

Cited by 297

DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Aminabadi, R. Y., Rajbhandari, S., Zhang, M., Awan, A. A., Li, C., Li, D., Zheng, E., Rasley, J., Smith, S., Ruwase, O., et al. (2022) · arXiv

Parallel Computing and Optimization TechniquesAdvanced Neural Network Applications

Cited by 297

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 286

ZeRO-Offload: Democratizing Billion-Scale Model Training

Ren, J., Rajbhandari, S., Aminabadi, R. Y., Ruwase, O., Yang, S., Zhang, M., Li, D., & He, Y. (2021) · arXiv

Parallel Computing and Optimization TechniquesAdvanced Neural Network Applications

Cited by 151

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Yao, Z., Aminabadi, R. Y., Zhang, M., Wu, X., Li, C., & He, Y. (2022) · arXiv

Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

Cited by 151

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Holmes, C., Tanaka, M., Wyatt, M., Awan, A. A., Rasley, J., Rajbhandari, S., Aminabadi, R. Y., Qin, H., Bakhtiari, A., Kurilenko, L., et al. (2024) · arXiv

Natural Language Processing TechniquesTopic Modeling

Cited by 10