The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Optimization of Collective Communication Operations in MPICH
Abstract
We describe our work on improving the performance of collective communication operations in MPICH for clusters connected by switched networks. For each collective operation, we use multiple algorithms depending on the message size, with the goal of minimizing latency for short messages and minimizing bandwidth use for long messages. Although we have implemented new algorithms for all MPI (Message Passing Interface) collective operations, because of limited space we describe only the algorithms for allgather, broadcast, all-to-all, reduce-scatter, reduce, and allreduce. Performance results on a Myrinet-connected Linux cluster and an IBM SP indicate that, in all cases, the new algorithms significantly outperform the old algorithms used in MPICH on the Myrinet cluster, and, in many cases, they outperform the algorithms used in IBM's MPI on the SP. We also explore in further detail the optimization of two of the most commonly used collective operations, allreduce and reduce, particularly for long messages and nonpower-of-two numbers of processes. The optimized algorithms for these operations perform several times better than the native algorithms on a Myrinet cluster, IBM SP, and Cray T3E. Our results indicate that to achieve the best performance for a collective communication operation, one needs to use a number of different algorithms and select the right algorithm for a particular message size and number of processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LogP: towards a realistic model of parallel computation
Culler, D., Karp, R., Patterson, D., Sahay, A., Schauser, K. E., Santos, E., Subramonian, R., & von Eicken, T. (1993) · Proceedings of the fourth ACM SIGPLAN symposium on Principles and practice of parallel programming - PPOPP '93 · doi:10.1145/155332.155333 · Cited by 1,496
Efficient algorithms for all-to-all communications in multiport message-passing systems
Bruck, J., Ho, C.-T., Kipnis, S., Upfal, E., & Weathersby, D. (1997) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/71.642949 · Cited by 281
Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001
(2001) · Proceedings - IEEE International Parallel and Distributed Processing Symposium · doi:10.1109/ipdps.2001.924926 · Cited by 156
Automatically Tuned Collective Communications
Vadhiyar, S. S., Fagg, G. E., & Dongarra, J. J. (2000) · doi:10.5555/370049.370055 · Cited by 147
Interprocessor collective communication library (InterCom)
Barnett, M., Shuler, L., Geijn, R. A., Gupta, S., Payne, D. G., & Watts, J. (2002) · IEEE Scalable High Performance Computing Conference (SHPCC) · doi:10.1109/shpcc.1994.296665 · Cited by 142
Cited by
PipeDream: generalized pipeline parallelism for DNN training
Narayanan, D., Harlap, A., Phanishayee, A., Seshadri, V., Devanur, N. R., Ganger, G. R., Gibbons, P. B., & Zaharia, M. (2019) · Proceedings of the 27th ACM Symposium on Operating Systems Principles · doi:10.1145/3341301.3359646 · Cited by 818
Parallel Spectral Clustering in Distributed Systems
Chen, W.-Y., Song, Y., Bai, H., Lin, C.-J., & Chang, E. Y. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.88 · Cited by 603
Versatile, scalable, and accurate simulation of distributed applications and platforms
Casanova, H., Giersch, A., Legrand, A., Quinson, M., & Suter, F. (2014) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2014.06.008 · Cited by 415
You, Y., Zhang, Z., Hsieh, C.-J., Demmel, J., & Keutzer, K. (2018) · Proceedings of the 47th International Conference on Parallel Processing · doi:10.1145/3225058.3225069 · Cited by 384
FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters
Iandola, F. N., Moskewicz, M. W., Ashraf, K., & Keutzer, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.284 · Cited by 280
Metadata
- Authors
- Thakur, R., Rabenseifner, R., & Gropp, W.
- Venue
- The International Journal of High Performance Computing Applications
- Publisher
- SAGE Publications
- Published
- 2005-02
- Volume
- 19
- Issue
- 1
- Pages
- 49-66
- ISSN
- 0890-2720 (ISSN-L)
- 1094-3420
- 1741-2846
- OpenAlex
- W2131613942
- Cited by
- 911
- References
- 5
- FWCI
- 41.10
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Optimization of Collective Communication Operations in MPICH. https://app.cochat.ai/papers/10.1177/1094342005051521
- The paper
- Thakur, R., Rabenseifner, R., & Gropp, W. (2005). Optimization of Collective Communication Operations in MPICH. The International Journal of High Performance Computing Applications. https://doi.org/10.1177/1094342005051521
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.