Patwary, Mostofa

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., & Catanzaro, B. (2019) · arXiv

Topic ModelingParallel Computing and Optimization Techniques

Cited by 893

Efficient large-scale language model training on GPU clusters using megatron-LM

Narayanan, D., Shoeybi, M., Casper, J., LeGresley, P., Patwary, M., Korthikanti, V., Vainbrand, D., Kashinkunti, P., Bernauer, J., Catanzaro, B., et al. (2021) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Parallel Computing and Optimization TechniquesTopic Modeling

Cited by 639

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 286

Factuality Enhanced Language Models for Open-Ended Text Generation

Lee, N., Ping, W., Xu, P., Patwary, M., Fung, P., Shoeybi, M., & Catanzaro, B. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 68

Nemotron-4 340B Technical Report

Nvidia, :, Adler, B., Agarwal, N., Aithal, A., Anh, D. H., Bhattacharya, P., Brundyn, A., Casper, J., Catanzaro, B., et al. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 66

Large scale multi-actor generative dialog modeling,

Boyd, A., Puri, R., Shoeybi, M., Patwary, M., & Catanzaro, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Speech and dialogue systemsTopic Modeling

Cited by 22

MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models

Xu, P., Patwary, M., Shoeybi, M., Puri, R., Fung, P., Anandkumar, A., & Catanzaro, B. (2020) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 12

Nemotron-4 15B Technical Report

Parmar, J., Prabhumoye, S., Jennings, J., Patwary, M., Subramanian, S., Su, D., Zhu, C., Narayanan, D., Jhunjhunwala, A., Dattagupta, A., et al. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 9

Llama-Nemotron: Efficient Reasoning Models

Bercovich, A., Levy, I., Golan, I., Dabbah, M., El-Yaniv, R., Puny, O., Galil, I., Moshe, Z., Ronen, T., Nabwani, N., et al. (2025) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 1

LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts

Elango, V., Bhatia, N., Waleffe, R., Shafipour, R., Asida, T., Khattar, A., Assaf, N., Golub, M., Guman, J., Mitra, T., et al. (2026) · arXiv

Advanced Neural Network ApplicationsTopic Modeling

Cited by 0

Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models

NVIDIA, :, Blakeman, A., Basant, A., Khattar, A., Renduchintala, A., Bercovich, A., Ficek, A., Bjorlin, A., Taghibakhshi, A., et al. (2025) · arXiv

Advanced Neural Network ApplicationsTopic Modeling

Cited by 0