Scheduling multithreaded computations by work stealing
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
This paper studies the problem of efficiently schedulling fully strict (i.e., well-structured) multithreaded computations on parallel computers. A popular and practical method of scheduling this kind of dynamic MIMD-style computation is “work stealing,” in which processors needing work steal computational threads from other processors. In this paper, we give the first provably good work-stealing scheduler for multithreaded computations with dependencies. Specifically, our analysis shows that the expected time to execute a fully strict computation on P processors using our work-stealing scheduler is T 1 / P + O ( T ∞ , where T 1 is the minimum serial execution time of the multithreaded computation and ( T ∞ is the minimum execution time with an infinite number of processors. Moreover, the space required by the execution is at most S 1 P , where S 1 is the minimum serial space requirement. We also show that the expected total communication of the algorithm is at most O ( PT ∞ ( 1 + n d ) S max ), where S max is the size of the largest activation record of any thread and n d is the maximum number of times that any thread synchronizes with its parent. This communication bound justifies the folk wisdom that work-stealing schedulers are more communication efficient than their work-sharing counterparts. All three of these bounds are existentially optimal to within a constant factor.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bounds on Multiprocessing Timing Anomalies
Graham, R. L. (1969) · SIAM Journal on Applied Mathematics · doi:10.1137/0117039 · Cited by 2,443
Bounds for Certain Multiprocessing Anomalies
Graham, R. L. (1966) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1966.tb01709.x · Cited by 1,647
The Parallel Evaluation of General Arithmetic Expressions
Brent, R. P. (1974) · Journal of the ACM · doi:10.1145/321812.321815 · Cited by 814
Cilk: An Efficient Multithreaded Runtime System
Blumofe, R. D., Joerg, C. F., Kuszmaul, B. C., Leiserson, C. E., Randall, K. H., & Zhou, Y. (1996) · Journal of Parallel and Distributed Computing · doi:10.1006/jpdc.1996.0107 · Cited by 784
Self-citationSpeedup versus efficiency in parallel systems
Eager, D. L., Zahorjan, J., & Lazowska, E. D. (1989) · IEEE Transactions on Computers · doi:10.1109/12.21127 · Cited by 437
I-structures: data structures for parallel computing
Arvind, Nikhil, R. S., & Pingali, K. K. (1989) · ACM Transactions on Programming Languages and Systems · doi:10.1145/69558.69562 · Cited by 360
Department of electrical engineering and computer science
Sarrafzadeh, M. (1990) · ACM SIGDA Newsletter · doi:10.1145/378886.380416 · Cited by 327
Lazy task creation: a technique for increasing the granularity of parallel programs
Mohr, E., Kranz, D. A., & Halstead, R. H. (1991) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/71.86103 · Cited by 307
Detecting data races in Cilk programs that use locks
Cheng, G.-I., Feng, M. D., Leiserson, C. E., Randall, K. H., & Stark, A. F. (1998) · ACM Symposium on Parallel Algorithms and Architectures (SPAA) · doi:10.1145/277651.277696 · Cited by 148
Self-citation
Cited by
Cilk: an efficient multithreaded runtime system
Blumofe, R. D., Joerg, C. F., Kuszmaul, B. C., Leiserson, C. E., Randall, K. H., & Zhou, Y. (1995) · Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming · doi:10.1145/209936.209958 · Cited by 1,021
Self-citationLigra: a lightweight graph processing framework for shared memory
Shun, J., & Blelloch, G. E. (2013) · Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming · doi:10.1145/2442516.2442530 · Cited by 798
Learning scheduling algorithms for data processing clusters
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019) · Proceedings of the ACM Special Interest Group on Data Communication · doi:10.1145/3341302.3342080 · Cited by 723
Demystifying Parallel and Distributed Deep Learning: An In-depth Concurrency Analysis
Ben-Nun, T., & Hoefler, T. (2020) · ACM Computing Surveys · doi:10.1145/3320060 · Cited by 577
The Power of Two Random Choices: A Survey of Techniques and Results
Mitzenmacher, M., Richa, A. W., & Sitaraman, R. (2001) · Combinatorial Optimization · doi:10.1007/978-1-4615-0013-1_9 · Cited by 431
The implementation of the Cilk-5 multithreaded language
Frigo, M., Leiserson, C. E., & Randall, K. H. (1998) · ACM SIGPLAN Notices · doi:10.1145/277652.277725 · Cited by 290
Self-citationRay: A Distributed Framework for Emerging AI Applications
Moritz, P., Nishihara, R., Wang, S. F., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., Jordan, M. I., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05889 · Cited by 247
Augment your batch: Improving generalization through instance repetition
Hoffer, E., Ben-Nun, T., Hubara, I., Giladi, N., Hoefler, T., & Soudry, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00815 · Cited by 166
Metadata
- Authors
- Blumofe, R. D., & Leiserson, C. E.
- Venue
- Journal of the ACM
- Published
- 1999-09
- Volume
- 46
- Issue
- 5
- Pages
- 720-748
- ISSN
- 0004-5411 (ISSN-L)
- 1557-735X
- OpenAlex
- W2016559894
- Cited by
- 1,336
- References
- 14
- FWCI
- 20.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.