LeGresley, Patrick

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., & Catanzaro, B. (2019) · arXiv

Topic ModelingParallel Computing and Optimization Techniques

Cited by 893

Efficient large-scale language model training on GPU clusters using megatron-LM

Narayanan, D., Shoeybi, M., Casper, J., LeGresley, P., Patwary, M., Korthikanti, V., Vainbrand, D., Kashinkunti, P., Bernauer, J., Catanzaro, B., et al. (2021) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Parallel Computing and Optimization TechniquesTopic Modeling

Cited by 639

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 286

Nemotron-4 340B Technical Report

Nvidia, :, Adler, B., Agarwal, N., Aithal, A., Anh, D. H., Bhattacharya, P., Brundyn, A., Casper, J., Catanzaro, B., et al. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 66

Nemotron-4 15B Technical Report

Parmar, J., Prabhumoye, S., Jennings, J., Patwary, M., Subramanian, S., Su, D., Zhu, C., Narayanan, D., Jhunjhunwala, A., Dattagupta, A., et al. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 9