The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
FireWorks: a dynamic workflow system designed for high‐throughput applications
Open accessAbstract
Summary: This paper introduces FireWorks, a workflow software for running high‐throughput calculation workflows at supercomputing centers. FireWorks has been used to complete over 50 million CPU‐hours worth of computational chemistry and materials science calculations at the National Energy Research Supercomputing Center. It has been designed to serve the demanding high‐throughput computing needs of these applications, with extensive support for (i) concurrent execution through job packing, (ii) failure detection and correction, (iii) provenance and reporting for long‐running projects, (iv) automated duplicate detection, and (v) dynamic workflows (i.e., modifying the workflow graph during runtime). We have found that these features are highly relevant to enabling modern data‐driven and high‐throughput science applications, and we discuss our implementation strategy that rests on Python and NoSQL databases (MongoDB). Finally, we present performance data and limitations of our approach along with planned future work. Copyright © 2015 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Commentary: The Materials Project: A materials genome approach to accelerating materials innovation
Jain, A., Ong, S. P., Hautier, G., Chen, W., Richards, W. D., Dacek, S., Cholia, S., Gunter, D., Skinner, D., Ceder, G., et al. (2013) · APL Materials · doi:10.1063/1.4812323 · Cited by 13,094
Self-citationExploring Network Structure, Dynamics, and Function Using NetworkX
Hagberg, A. A., Schult, D. A., & Swart, P. J. (2008) · Proceedings of the Python in Science Conference · doi:10.25080/tcwv9851 · Cited by 7,477
Architectural styles and the design of network-based software architectures
Fielding, R. T., & Taylor, R. N. (2000) · Cited by 5,094
Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis
Ong, S. P., Richards, W. D., Jain, A., Hautier, G., Kocher, M., Cholia, S., Gunter, D., Chevrier, V. L., Persson, K. A., & Ceder, G. (2013) · Computational Materials Science · doi:10.1016/j.commatsci.2012.10.028 · Cited by 4,534
Self-citationIPython: A System for Interactive Scientific Computing
Perez, F., & Granger, B. E. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.53 · Cited by 4,009
Goecks, J., Nekrutenko, A., Taylor, J., & The Galaxy Team (2010) · Genome Biology · doi:10.1186/gb-2010-11-8-r86 · Cited by 3,453
Python for scientific computing
Oliphant, T. E. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.58 · Cited by 3,303 · cited 1×
“[7]–[9] As a high-level language, Python allows scientists (whose primary background is not in computing) to relatively easily”
BackgroundGlobus: a Metacomputing Infrastructure Toolkit
Foster, I., & Kesselman, C. (1997) · The International Journal of Supercomputer Applications and High Performance Computing · doi:10.1177/109434209701100205 · Cited by 3,133
Advances in methods and algorithms in a modern quantum chemistry program package
Shao, Y., Molnar, L. F., Jung, Y., Kussmann, J., Ochsenfeld, C., Brown, S. T., Gilbert, A. T. B., Slipchenko, L. V., Levchenko, S. V., O’Neill, D. P., et al. (2006) · Phys. Chem. Chem. Phys. · doi:10.1039/b517914a · Cited by 2,799
ABINIT: First-principles approach to material and nanosystem properties
Gonze, X., Amadon, B., Anglade, P.-M., Beuken, J.-M., Bottin, F., Boulanger, P., Bruneval, F., Caliste, D., Caracas, R., Côté, M., et al. (2009) · Computer Physics Communications · doi:10.1016/j.cpc.2009.07.007 · Cited by 2,701
Self-citationDistributed computing in practice: the Condor experience
Thain, D., Tannenbaum, T., & Livny, M. (2005) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.938 · Cited by 1,886
Scientific workflow management and the Kepler system
Ludäscher, B., Altıntaş, İ., Berkley, C., Higgins, D., Jaeger-Frank, E., Jones, M. B., Lee, E. A., Tao, J., & Zhao, Y. (2005) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.994 · Cited by 1,719
Taverna: a tool for the composition and enactment of bioinformatics workflows
Oinn, T., Addis, M., Ferris, J., Marvin, D., Senger, M., Greenwood, M., Carver, T., Glover, K., Pocock, M. R., Wipat, A., et al. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth361 · Cited by 1,602
Pegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems
Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G. B., Good, J., et al. (2005) · Scientific Programming · doi:10.1155/2005/128026 · Cited by 1,222
The rechargeable revolution: a better battery
Van Noorden, R. (2014) · Nature · doi:10.1038/507026a · Cited by 1,047
Charting the complete elastic properties of inorganic crystalline compounds
de Jong, M., Chen, W., Angsten, T., Jain, A., Notestine, R., Gamst, A., Sluiter, M., Krishna Ande, C., van der Zwaag, S., Plata, J. J., et al. (2015) · Scientific Data · doi:10.1038/sdata.2015.9 · Cited by 1,005
Self-citationMatchmaking: distributed resource management for high throughput computing
Raman, R., Livny, M., & Solomon, M. (2002) · Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244) · doi:10.1109/hpdc.1998.709966 · Cited by 747
Kepler: An Extensible System for Design and Execution of Scientific Workflows
Altintas, I., Berkley, C., Jaeger, E., Jones, M., Ludäscher, B., & Mock, S. (2004) · Cited by 745
Bell, G., Hey, T., & Szalay, A. (2009) · Science · doi:10.1126/science.1170411 · Cited by 537
Ong, S. P., Cholia, S., Jain, A., Brafman, M., Gunter, D., Ceder, G., & Persson, K. A. (2015) · Computational Materials Science · doi:10.1016/j.commatsci.2014.10.037 · Cited by 480
Self-citationSwift: A language for distributed parallel scripting
Wilde, M., Hategan, M., Wozniak, J. M., Clifford, B., Katz, D. S., & Foster, I. (2011) · Parallel Computing · doi:10.1016/j.parco.2011.05.005 · Cited by 402
Accelerating Electrolyte Discovery for Energy Storage with High-Throughput Screening
Cheng, L., Assary, R. S., Qu, X., Jain, A., Ong, S. P., Rajput, N. N., Persson, K., & Curtiss, L. A. (2015) · The Journal of Physical Chemistry Letters · doi:10.1021/jz502319n · Cited by 356
Self-citationPython: An Ecosystem for Scientific Computing
Perez, F., Granger, B. E., & Hunter, J. D. (2011) · Computing in Science & Engineering · doi:10.1109/mcse.2010.119 · Cited by 241
The Electrolyte Genome project: A big data approach in battery materials discovery
Qu, X., Jain, A., Rajput, N. N., Cheng, L., Zhang, Y., Ong, S. P., Brafman, M., Maginn, E., Curtiss, L. A., & Persson, K. A. (2015) · Computational Materials Science · doi:10.1016/j.commatsci.2015.02.050 · Cited by 205
Self-citationPython Scripting for Computational Science
Langtangen, H. P. (2007) · Texts in computational science and engineering · doi:10.1007/978-3-540-73916-6 · Cited by 190
KNIME: The Konstanz Information Miner
Berthold, M. R., Cebron, N., Dill, F., Fatta, G. D., Gabriel, T. R., Georg, F., Meinl, T., Ohl, P., Sieb, C., & Wiswedel, B. (2006) · CentAUR (University of Reading) · Cited by 151
Couvares, P., Kosar, T., Roy, A., Weber, J., & Wenger, K. (2007) · Springer eBooks · doi:10.1007/978-1-84628-757-2_22 · Cited by 150
High-Throughput ab Initio Screening for Two-Dimensional Electride Materials
Tada, T., Takemoto, S., Matsuishi, S., & Hosono, H. (2014) · Inorganic Chemistry · doi:10.1021/ic501362b · Cited by 140
Workflow Systems for Science: Concepts and Tools
Talia, D. (2013) · ISRN Software Engineering · doi:10.1155/2013/404525 · Cited by 71
NEWT: A RESTful service for building High Performance Computing web applications
Cholia, S., Skinner, D., & Boverhof, J. (2010) · Grid Computing Environments · doi:10.1109/gce.2010.5676125 · Cited by 65
Density functional calculations of the enthalpies of formation of rare-earth orthophosphates
Rustad, J. R. (2012) · American Mineralogist · doi:10.2138/am.2012.3948 · Cited by 56
Adaptive exception handling for scientific workflows
(2009) · doi:10.1002/cpe.1487 · Cited by 25
Cited by
MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations
Gowers, R., Linke, M., Barnoud, J., Reddy, T., Melo, M., Seyler, S., Domański, J., Dotson, D., Buchoux, S., Kenney, I., et al. (2016) · Proceedings of the Python in Science Conference · doi:10.25080/majora-629e541a-00e · Cited by 2,026
Recent developments in the general atomic and molecular electronic structure system
Barca, G. M. J., Bertoni, C., Carrington, L., Datta, D., De Silva, N., Deustua, J. E., Fedorov, D. G., Gour, J. R., Gunina, A. O., Guidez, E., et al. (2020) · The Journal of Chemical Physics · doi:10.1063/5.0005188 · Cited by 1,443
A universal graph deep learning interatomic potential for the periodic table
Chen, C., & Ong, S. P. (2022) · Nature Computational Science · doi:10.1038/s43588-022-00349-3 · Cited by 1,140
Self-citationUnsupervised word embeddings capture latent knowledge from materials science literature
Tshitoyan, V., Dagdelen, J., Weston, L., Dunn, A., Rong, Z., Kononova, O., Persson, K. A., Ceder, G., & Jain, A. (2019) · Nature · doi:10.1038/s41586-019-1335-8 · Cited by 1,123
Self-citationSurface energies of elemental crystals
Tran, R., Xu, Z., Radhakrishnan, B., Winston, D., Sun, W., Persson, K. A., & Ong, S. P. (2016) · Scientific Data · doi:10.1038/sdata.2016.80 · Cited by 1,047
Self-citation
Metadata
- Authors
- Jain, A., Ong, S. P., Chen, W., Medasani, B., Qu, X., Kocher, M., Brafman, M., Petretto, G., Rignanese, G., Hautier, G., Gunter, D., & Persson, K. A.
- Venue
- Concurrency and Computation: Practice and Experience
- Publisher
- Wiley
- Published
- 2015-05-18
- Volume
- 27
- Issue
- 17
- Pages
- 5037-5059
- ISSN
- 1532-0626 (ISSN-L)
- 1532-0634
- OpenAlex
- W1798528872
- Cited by
- 547
- References
- 32
- FWCI
- 61.11
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.