Scheduling multithreaded computations by work stealing
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
This paper studies the problem of efficiently schedulling fully strict (i.e., well-structured) multithreaded computations on parallel computers. A popular and practical method of scheduling this kind of dynamic MIMD-style computation is “work stealing,” in which processors needing work steal computational threads from other processors. In this paper, we give the first provably good work-stealing scheduler for multithreaded computations with dependencies. Specifically, our analysis shows that the expected time to execute a fully strict computation on P processors using our work-stealing scheduler is T 1 / P + O ( T ∞ , where T 1 is the minimum serial execution time of the multithreaded computation and ( T ∞ is the minimum execution time with an infinite number of processors. Moreover, the space required by the execution is at most S 1 P , where S 1 is the minimum serial space requirement. We also show that the expected total communication of the algorithm is at most O ( PT ∞ ( 1 + n d ) S max ), where S max is the size of the largest activation record of any thread and n d is the maximum number of times that any thread synchronizes with its parent. This communication bound justifies the folk wisdom that work-stealing schedulers are more communication efficient than their work-sharing counterparts. All three of these bounds are existentially optimal to within a constant factor.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bounds on Multiprocessing Timing Anomalies
Graham, R. L. (1969) · SIAM Journal on Applied Mathematics · doi:10.1137/0117039 · Cited by 2,443
Bounds for Certain Multiprocessing Anomalies
Graham, R. L. (1966) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1966.tb01709.x · Cited by 1,647
The Parallel Evaluation of General Arithmetic Expressions
Brent, R. P. (1974) · Journal of the ACM · doi:10.1145/321812.321815 · Cited by 814
Cilk: An Efficient Multithreaded Runtime System
Blumofe, R. D., Joerg, C. F., Kuszmaul, B. C., Leiserson, C. E., Randall, K. H., & Zhou, Y. (1996) · Journal of Parallel and Distributed Computing · doi:10.1006/jpdc.1996.0107 · Cited by 784
Self-citationSpeedup versus efficiency in parallel systems
Eager, D. L., Zahorjan, J., & Lazowska, E. D. (1989) · IEEE Transactions on Computers · doi:10.1109/12.21127 · Cited by 437
I-structures: data structures for parallel computing
Arvind, Nikhil, R. S., & Pingali, K. K. (1989) · ACM Transactions on Programming Languages and Systems · doi:10.1145/69558.69562 · Cited by 360
Department of electrical engineering and computer science
Sarrafzadeh, M. (1990) · ACM SIGDA Newsletter · doi:10.1145/378886.380416 · Cited by 327
Lazy task creation: a technique for increasing the granularity of parallel programs
Mohr, E., Kranz, D. A., & Halstead, R. H. (1991) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/71.86103 · Cited by 307
Detecting data races in Cilk programs that use locks
Cheng, G.-I., Feng, M. D., Leiserson, C. E., Randall, K. H., & Stark, A. F. (1998) · ACM Symposium on Parallel Algorithms and Architectures (SPAA) · doi:10.1145/277651.277696 · Cited by 148
Self-citationAn analysis of dag-consistent distributed shared-memory algorithms
Blumofe, R. D., Frigo, M., Joerg, C. F., Leiserson, C. E., & Randall, K. H. (1996) · ACM Symposium on Parallelism in Algorithms and Architectures (SPAA) · doi:10.1145/237502.237574 · Cited by 115
Self-citationDAG-consistent distributed shared memory
Blumofe, R. D., Frigo, M., Joerg, C. F., Leiserson, C. E., & Randall, K. H. (2002) · Proceedings of the International Conference on Parallel Processing · doi:10.1109/ipps.1996.508049 · Cited by 81
Self-citationEnumerations of the Hamiltonian walks on a cubic sublattice
Pande, V. S., Joerg, C., Grosberg, A. Y., & Tanaka, T. (1994) · Journal of Physics A Mathematical and General · doi:10.1088/0305-4470/27/18/030 · Cited by 62
Provably efficient scheduling for languages with fine-grained parallelism
Blelloch, G. E., Gibbons, P. B., & Matias, Y. (1995) · ACM Symposium on Parallelism in Algorithms and Architectures (SPAA) · doi:10.1145/215399.215403 · Cited by 54
Cited by
Cilk: an efficient multithreaded runtime system
Blumofe, R. D., Joerg, C. F., Kuszmaul, B. C., Leiserson, C. E., Randall, K. H., & Zhou, Y. (1995) · Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming · doi:10.1145/209936.209958 · Cited by 1,021
Self-citationLigra: a lightweight graph processing framework for shared memory
Shun, J., & Blelloch, G. E. (2013) · Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming · doi:10.1145/2442516.2442530 · Cited by 798
Learning scheduling algorithms for data processing clusters
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019) · Proceedings of the ACM Special Interest Group on Data Communication · doi:10.1145/3341302.3342080 · Cited by 723
Demystifying Parallel and Distributed Deep Learning: An In-depth Concurrency Analysis
Ben-Nun, T., & Hoefler, T. (2020) · ACM Computing Surveys · doi:10.1145/3320060 · Cited by 577
The Power of Two Random Choices: A Survey of Techniques and Results
Mitzenmacher, M., Richa, A. W., & Sitaraman, R. (2001) · Combinatorial Optimization · doi:10.1007/978-1-4615-0013-1_9 · Cited by 431
Metadata
- Authors
- Blumofe, R. D., & Leiserson, C. E.
- Venue
- Journal of the ACM
- Published
- 1999-09
- Volume
- 46
- Issue
- 5
- Pages
- 720-748
- ISSN
- 0004-5411 (ISSN-L)
- 1557-735X
- OpenAlex
- W2016559894
- Cited by
- 1,336
- References
- 14
- FWCI
- 20.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.