Phanishayee, Amar

Efficient large-scale language model training on GPU clusters using megatron-LM

Narayanan, D., Shoeybi, M., Casper, J., LeGresley, P., Patwary, M., Korthikanti, V., Vainbrand, D., Kashinkunti, P., Bernauer, J., Catanzaro, B., et al. (2021) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Parallel Computing and Optimization TechniquesTopic Modeling

Cited by 639

PipeDream: Fast and Efficient Pipeline Parallel DNN Training

Harlap, A., Narayanan, D., Phanishayee, A., Seshadri, V., Devanur, N., Ganger, G., & Gibbons, P. (2018) · arXiv

Parallel Computing and Optimization TechniquesStochastic Gradient Optimization Techniques

Cited by 153