Towards a holistic approach to auto-parallelization: integrating profile-driven parallelism detection and machine-learning based mapping
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Compiler-based auto-parallelization is a much studied area, yet has still not found wide-spread application. This is largely due to the poor exploitation of application parallelism, subsequently resulting in performance levels far below those which a skilled expert programmer could achieve. We have identified two weaknesses in traditional parallelizing compilers and propose a novel, integrated approach, resulting in significant performance improvements of the generated parallel code. Using profile-driven parallelism detection we overcome the limitations of static analysis, enabling us to identify more application parallelism and only rely on the user for final approval. In addition, we replace the traditional target-specific and inflexible mapping heuristics with a machine-learning based prediction mechanism, resulting in better mapping decisions while providing more scope for adaptation to different target architectures. We have evaluated our parallelization strategy against the NAS and SPEC OMP benchmarks and two different multi-core platforms (dual quad-core Intel Xeon SMP and dual-socket QS20 Cell blade). We demonstrate that our approach not only yields significant improvements when compared with state-of-the-art parallelizing compilers, but comes close to and sometimes exceeds the performance of manually parallelized codes. On average, our methodology achieves 96% of the performance of the hand-tuned OpenMP NAS and SPEC parallel benchmarks on the Intel Xeon platform and gains a significant speedup for the IBM Cell platform, demonstrating the potential of profile-guided and machine-learning based parallelization for complex multi-core platforms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
A Bridging Model for Parallel Computation
Valiant, L. G. (1990) · ACM eBooks · doi:10.1145/3749740.3749753 · Cited by 3,664
Bailey, D. H., Barszcz, E., Barton, J. T., Browning, D. S., Carter, R. L., Dagum, L., Fatoohi, R. A., Frederickson, P. O., Lasinski, T. A., Schreiber, R. S., et al. (1991) · The International Journal of Supercomputing Applications · doi:10.1177/109434209100500306 · Cited by 2,395 · cited 1×
“More specifically, we have used the NAS NPB (sequential v.2.3) and NPB (OpenMP v.2.3) codes ( 26 ) alongside the SPEC CPU2000 benchmarks and their corresponding SPEC OMP2001”
MethodInfluentialOptimizing Compilers for Modern Architectures: A Dependence-based Approach
Kennedy, K., & Allen, J. R. (2001) · Cited by 1,002
The parallel execution of DO loops
Lamport, L. (1974) · Communications of the ACM · doi:10.1145/360827.360844 · Cited by 611 · cited 3×
“sum0 += A[ Anext ] [ 0 ] [ 0 ] ∗ v [ c o l ] [ 0 ] + A[ Anext ] [ 0 ] [ 1 ] ∗ v [ c o l ] [ 1 ] + A[ Anext ] [ 0 ] [ 2 ] ∗ v [ c o l ] [ 2 ] ; sum1 += .”
Maximizing multiprocessor performance with the SUIF compiler
Hall, M. W., Anderson, J. M., Amarasinghe, S. P., Murphy, B. R., Liao, S.-W., Bugnion, E., & Lam, M. S. (1996) · Computer · doi:10.1109/2.546613 · Cited by 572 · cited 1×
“Polaris (6), SUIF-1 ( 7 ) and, more recently, Open64 (8).”
Optimistic parallelism requires abstractions
Kulkarni, M., Pingali, K., Walter, B., Ramanarayanan, G., Bala, K., & Chew, L. P. (2007) · Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1250734.1250759 · Cited by 441 · cited 1×
“Complementary to the on-going work in auto-parallelization many high-level parallel programming languages ‐ such as Cilk-5 (9), OpenMP, StreamIt (10), UPC (11) and X10 (12) ‐ and programming models ‐ such as Galois ( 14 ), STAPL (15) and HTA (16) ‐ have been proposed.”
BackgroundA stream compiler for communication-exposed architectures
Gordon, M. I., Thies, W., Karczmarek, M., Lin, J., Meli, A. S., Lamb, A. A., Leger, C., Wong, J., Hoffmann, H., Maze, D., et al. (2002) · Proceedings of the 10th international conference on Architectural support for programming languages and operating systems · doi:10.1145/605397.605428 · Cited by 337 · cited 2×
“Complementary to the on-going work in auto-parallelization many high-level parallel programming languages ‐ such as Cilk-5 (9), OpenMP, StreamIt ( 10 ), UPC (11) and X10 (12) ‐ and programming models ‐ such as Galois (14), STAPL (15) and HTA (16) ‐ have been proposed.”
BackgroundOptimizing for reduced code space using genetic algorithms
Cooper, K. D., Schielke, P. J., & Subramanian, D. (1999) · ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES) · doi:10.1145/314403.314414 · Cited by 321 · cited 1×
“For example, Cooper et al. ( 45 ) develop a technique to find “good” compiler optimization sequences for code size reduction.”
MethodSPEComp: A New Benchmark Suite for Measuring Parallel Computer Performance
Aslot, V., Domeika, M., Eigenmann, R., Gaertner, G., Jones, W. B., & Parady, B. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44587-0_1 · Cited by 255 · cited 1×
“Investigations into the caus es of this behavior have revealed that the SPEC OMP codes are not equivalent to the sequential SPEC programs, but have been manually restructured ( 29 ).”
Background
Cited by
Milepost GCC: Machine Learning Enabled Self-tuning Compiler
Fursin, G., Kashnikov, Y., Memon, A. W., Chamski, Z., Temam, O., Namolaru, M., Yom-Tov, E., Mendelson, B., Zaks, A., Courtois, E., et al. (2011) · International Journal of Parallel Programming · doi:10.1007/s10766-010-0161-2 · Cited by 234
“We started combining Milepost technology with machine-learning based autoparallelization and predictive scheduling techniques [60,51,76].”
MethodSelf-citationAdaptive deep learning model selection on embedded systems
Taylor, B., Marco, V. S., Wolff, W., Elkhatib, Y., & Wang, Z. (2018) · Proceedings of the 19th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems · doi:10.1145/3211332.3211336 · Cited by 90
Self-citationLearning Performance-Improving Code Edits
Shypula, A., Madaan, A., Zeng, Y., Alon, U., Gardner, J., Hashemi, M., Neubig, G., Ranganathan, P., Bastani, O., & Yazdanbakhsh, A. (2023) · arXiv · doi:10.48550/arxiv.2302.07867 · Cited by 146
Metadata
- Authors
- Tournavitis, G., Wang, Z., Franke, B., & O'Boyle, M. F. P.
- Venue
- Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation
- Publisher
- ACM
- Published
- 2009-06-15
- Pages
- 177-187
- ISSN
- 1531-7102 (ISSN-L)
- OpenAlex
- W2056579078
- Cited by
- 204
- References
- 26
- FWCI
- 13.72
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.