Korthikanti, Vijay

Efficient large-scale language model training on GPU clusters using megatron-LM

Narayanan, D., Shoeybi, M., Casper, J., LeGresley, P., Patwary, M., Korthikanti, V., Vainbrand, D., Kashinkunti, P., Bernauer, J., Catanzaro, B., et al. (2021) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Parallel Computing and Optimization TechniquesTopic Modeling

Cited by 639

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 286

An Empirical Study of Mamba-based Language Models

Waleffe, R., Byeon, W., Riach, D., Norick, B., Korthikanti, V., Dao, T., Gu, A., Hatamizadeh, A., Singh, S., Narayanan, D., et al. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 19

Scalable Training of Mixture-of-Experts Models with Megatron Core

Yan, Z., Bai, H., Yao, X., Liu, D., Liu, T., Liu, H., Li, P., Wu, E., Fan, S., Tao, L., et al. (2026) · arXiv

Parallel Computing and Optimization TechniquesAdvanced Neural Network Applications

Cited by 1

Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models

NVIDIA, :, Blakeman, A., Basant, A., Khattar, A., Renduchintala, A., Bercovich, A., Ficek, A., Bjorlin, A., Taghibakhshi, A., et al. (2025) · arXiv

Advanced Neural Network ApplicationsTopic Modeling

Cited by 0