The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Parsl: Pervasive Parallel Programming in Python
Open accessAbstract
High-level programming languages such as Python are increasingly used to provide intuitive interfaces to libraries written in lower-level languages and for assembling applications from various components. This migration towards orchestration rather than implementation, coupled with the growing need for parallel computing (e.g., due to big data and the end of Moore's law), necessitates rethinking how parallelism is expressed in programs. Here, we present Parsl, a parallel scripting library that augments Python with simple, scalable, and flexible constructs for encoding parallelism. These constructs allow Parsl to construct a dynamic dependency graph of components that it can then execute efficiently on one or many processors. Parsl is designed for scalability, with an extensible set of executors tailored to different use cases, such as low-latency, high-throughput, or extreme-scale execution. We show, via experiments on the Blue Waters supercomputer, that Parsl executors can allow Python scripts to execute components with as little as 5 ms of overhead, scale to more than 250000 workers across more than 8000 nodes, and process upward of 1200 tasks per second. Other Parsl features simplify the construction and execution of composite programs by supporting elastic provisioning and scaling of infrastructure, fault-tolerant execution, and integrated wide-area data management. We show that these capabilities satisfy the needs of many-task, interactive, online, and machine learning applications in fields such as biology, cosmology, and materials science.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Nextflow enables reproducible computational workflows
Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017) · Nature Biotechnology · doi:10.1038/nbt.3820 · Cited by 4,001
Goecks, J., Nekrutenko, A., Taylor, J., & The Galaxy Team (2010) · Genome Biology · doi:10.1186/gb-2010-11-8-r86 · Cited by 3,453
Pegasus, a workflow management system for science automation
Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P. J., Mayani, R., Chen, W., Ferreira da Silva, R., Livny, M., et al. (2015) · Future Generation Computer Systems · doi:10.1016/j.future.2014.10.008 · Cited by 861 · cited 2×
“This approach to parallelism is in contrast to prior efforts that rely on domain-specific languages (DSL) [25, 27, 39], configuration-based models [1, 24, 31], Python-based graph descriptions [4, 33], and compiled language extensions [19] to support such composition.”
BackgroundResultFireWorks: a dynamic workflow system designed for high‐throughput applications
Jain, A., Ong, S. P., Chen, W., Medasani, B., Qu, X., Kocher, M., Brafman, M., Petretto, G., Rignanese, G., Hautier, G., et al. (2015) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.3505 · Cited by 547
Dalcín, L., Paz, R., & Storti, M. (2005) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2005.03.010 · Cited by 408
Swift: A language for distributed parallel scripting
Wilde, M., Hategan, M., Wozniak, J. M., Clifford, B., Katz, D. S., & Foster, I. (2011) · Parallel Computing · doi:10.1016/j.parco.2011.05.005 · Cited by 402
Self-citationThe Materials Data Facility: Data Services to Advance Materials Science Research
Blaiszik, B., Chard, K., Pruyne, J., Ananthakrishnan, R., Tuecke, S., & Foster, I. (2016) · JOM · doi:10.1007/s11837-016-2001-3 · Cited by 326
Self-citationRay: A Distributed Framework for Emerging AI Applications
Moritz, P., Nishihara, R., Wang, S. F., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., Jordan, M. I., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05889 · Cited by 247
Building a Framework for Predictive Science
McKerns, M., Strand, L., Sullivan, T., Fang, A., & Aivazis, M. (2011) · Proceedings of the Python in Science Conference · doi:10.25080/majora-ebaa42b7-00d · Cited by 171
Efficient and Secure Transfer, Synchronization, and Sharing of Big Data
Chard, K., Tuecke, S. J., & Foster, I. T. (2014) · IEEE Cloud Computing · doi:10.1109/mcc.2014.52 · Cited by 92
Self-citation
Cited by
Huber, S. P., Zoupanos, S., Uhrin, M., Talirz, L., Kahle, L., Häuselmann, R., Gresch, D., Müller, T., Yakutovich, A. V., Andersen, C. W., et al. (2020) · Scientific Data · doi:10.1038/s41597-020-00638-4 · Cited by 314
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2023) · The International Journal of High Performance Computing Applications · doi:10.1177/10943420231201154 · Cited by 120
Self-citationGenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2022) · doi:10.1101/2022.10.10.511571 · Cited by 120
Self-citationgmxapi: A GROMACS-native Python interface for molecular dynamics with ensemble and plugin support
Irrgang, M. E., Davis, C., & Kasson, P. M. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009835 · Cited by 25
Metadata
- Authors
- Babuji, Y., Woodard, A., Li, Z., Katz, D. S., Clifford, B., Kumar, R., Lacinski, L., Chard, R., Wozniak, J. M., Foster, I., Wilde, M., & Chard, K.
- Venue
- Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing
- Publisher
- ACM
- Published
- 2019-06-17
- Pages
- 25-36
- OpenAlex
- W2943761765
- Cited by
- 270
- References
- 11
- FWCI
- 24.92
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Parsl: Pervasive Parallel Programming in Python. https://app.cochat.ai/papers/10.1145/3307681.3325400
- The paper
- Babuji, Y., Woodard, A., Li, Z., Katz, D. S., Clifford, B., Kumar, R., Lacinski, L., Chard, R., Wozniak, J. M., Foster, I., Wilde, M., & Chard, K. (2019). Parsl: Pervasive Parallel Programming in Python. Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing. https://doi.org/10.1145/3307681.3325400
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.