He, Yuxiong

DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters

Rasley, J., Rajbhandari, S., Ruwase, O., & He, Y. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Parallel Computing and Optimization TechniquesStochastic Gradient Optimization Techniques

Cited by 823

OpenFold: retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

Ahdritz, G., Bouatta, N., Floristean, C., Kadyan, S., Xia, Q., Gerecke, W., O’Donnell, T. J., Berenberg, D., Fisk, I., Zanichelli, N., et al. (2024) · Nature Methods

Protein Structure and DynamicsMachine Learning in Bioinformatics

Cited by 439

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Aminabadi, R. Y., Rajbhandari, S., Awan, A. A., Li, C., Li, D., Zheng, E., Ruwase, O., Smith, S., Zhang, M., Rasley, J., et al. (2022) · SC22: International Conference for High Performance Computing, Networking, Storage and Analysis

Parallel Computing and Optimization TechniquesAdvanced Neural Network Applications

Cited by 297

DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Aminabadi, R. Y., Rajbhandari, S., Zhang, M., Awan, A. A., Li, C., Li, D., Zheng, E., Rasley, J., Smith, S., Ruwase, O., et al. (2022) · arXiv

Parallel Computing and Optimization TechniquesAdvanced Neural Network Applications

Cited by 297

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 286

ZeRO-Offload: Democratizing Billion-Scale Model Training

Ren, J., Rajbhandari, S., Aminabadi, R. Y., Ruwase, O., Yang, S., Zhang, M., Li, D., & He, Y. (2021) · arXiv

Parallel Computing and Optimization TechniquesAdvanced Neural Network Applications

Cited by 151

Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping

Zhang, M., & He, Y. (2020) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 50

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Holmes, C., Tanaka, M., Wyatt, M., Awan, A. A., Rasley, J., Rajbhandari, S., Aminabadi, R. Y., Qin, H., Bakhtiari, A., Kurilenko, L., et al. (2024) · arXiv

Natural Language Processing TechniquesTopic Modeling

Cited by 10

Optimizing Reasoning for Text-to-SQL with Execution Feedback

Zhai, B., Xu, C., He, Y., & Yao, Z. (2025) · Findings of the Association for Computational Linguistics: ACL 2025

Topic ModelingNatural Language Processing Techniques

Cited by 8

NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM

Holmes, C., Zhang, M., He, Y., & Wu, B. (2021) · arXiv

Advanced Neural Network ApplicationsTopic Modeling

Cited by 7