A Survey of Data-Intensive Scientific Workflow Management
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
PHYLIP-Phylogeny inference package (Version 3.2)
Felsenstein, J. (1989) · Cladistics · Cited by 17,682
The Grid 2: Blueprint for a New Computing Infrastructure
Foster, I. T., & Kesselman, C. (1998) · Cited by 7,637
Cloud computing: state-of-the-art and research challenges
Zhang, Q., Cheng, L., & Boutaba, R. (2010) · Journal of Internet Services and Applications · doi:10.1007/s13174-010-0007-6 · Cited by 3,515
Performance-effective and low-complexity task scheduling for heterogeneous computing
Topcuoglu, H., Hariri, S., & Wu, M.-Y. (2002) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/71.993206 · Cited by 3,475
Goecks, J., Nekrutenko, A., Taylor, J., & The Galaxy Team (2010) · Genome Biology · doi:10.1186/gb-2010-11-8-r86 · Cited by 3,453
White, T. (2009) · Cited by 2,800
Dryad: distributed data-parallel programs from sequential building blocks
Isard, M., Budiu, M., Yu, Y., Birrell, A., & Fetterly, D. (2007) · Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 · doi:10.1145/1272996.1273005 · Cited by 2,432
Principles of Distributed Database Systems
Özsu, M. T., & Valduriez, P. (2020) · Springer eBooks · doi:10.1007/978-3-030-26253-2 · Cited by 2,345
Self-citationCondor-a hunter of idle workstations
Litzkow, M. J., Livny, M., & Mutka, M. W. (2003) · [1988] Proceedings. The 8th International Conference on Distributed · doi:10.1109/dcs.1988.12507 · Cited by 2,245
Pig latin: a not-so-foreign language for data processing
Olston, C., Reed, B., Srivastava, U., Kumar, R., & Tomkins, A. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376726 · Cited by 1,730
Scientific workflow management and the Kepler system
Ludäscher, B., Altıntaş, İ., Berkley, C., Higgins, D., Jaeger-Frank, E., Jones, M. B., Lee, E. A., Tao, J., & Zhao, Y. (2005) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.994 · Cited by 1,719
Taverna: a tool for the composition and enactment of bioinformatics workflows
Oinn, T., Addis, M., Ferris, J., Marvin, D., Senger, M., Greenwood, M., Carver, T., Glover, K., Pocock, M. R., Wipat, A., et al. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth361 · Cited by 1,602
Pegasus: A Framework for Mapping Complex Scientific Workflows onto Distributed Systems
Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G. B., Good, J., et al. (2005) · Scientific Programming · doi:10.1155/2005/128026 · Cited by 1,222
Pegasus, a workflow management system for science automation
Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P. J., Mayani, R., Chen, W., Ferreira da Silva, R., Livny, M., et al. (2015) · Future Generation Computer Systems · doi:10.1016/j.future.2014.10.008 · Cited by 861 · cited 7×
“The last method is to cluster the tasks of the same label [42].”
BackgroundMethodInfluentialA Taxonomy of Workflow Management Systems for Grid Computing
Yu, J., & Buyya, R. (2005) · Journal of Grid Computing · doi:10.1007/s10723-005-9010-8 · Cited by 834
Workflows and e-Science: An overview of workflow system features and capabilities
Deelman, E., Gannon, D., Shields, M., & Taylor, I. (2009) · Future Generation Computer Systems · doi:10.1016/j.future.2008.06.012 · Cited by 831
Kepler: An Extensible System for Design and Execution of Scientific Workflows
Altintas, I., Berkley, C., Jaeger, E., Jones, M., Ludäscher, B., & Mock, S. (2004) · Cited by 745
Wolstencroft, K., Haines, R., Fellows, D., Williams, A., Withers, D., Owen, S., Soiland-Reyes, S., Dunlop, I., Nenadic, A., Fisher, P., et al. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt328 · Cited by 677
The cost of doing science on the cloud: the Montage example
Deelman, E., Singh, G., Livny, M., Berriman, G. B., & Good, J. (2008) · doi:10.5555/1413370.1413421 · Cited by 580
Managing data transfers in computer clusters with orchestra
Chowdhury, M., Zaharia, M., Ma, J., Jordan, M. I., & Stoica, I. (2011) · Proceedings of the ACM SIGCOMM 2011 conference · doi:10.1145/2018436.2018448 · Cited by 543
Provenance for Computational Tasks: A Survey
Freire, J., Koop, D., Santos, E., & Silva, C. (2008) · Computing in Science & Engineering · doi:10.1109/mcse.2008.79 · Cited by 510
The cost of doing science on the cloud: The Montage example
Deelman, E., Singh, G., Livny, M., Berriman, B., & Good, J. (2008) · 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1109/sc.2008.5217932 · Cited by 483
Kepler: an extensible system for design and execution of scientific workflows
Altintas, I., Berkley, C., Jaeger, E., Jones, M., Ludascher, B., & Mock, S. (2004) · Proceedings. 16th International Conference on Scientific and Statistical Database Management, 2004. · doi:10.1109/ssdm.2004.1311241 · Cited by 429
Swift: A language for distributed parallel scripting
Wilde, M., Hategan, M., Wozniak, J. M., Clifford, B., Katz, D. S., & Foster, I. (2011) · Parallel Computing · doi:10.1016/j.parco.2011.05.005 · Cited by 402
Principles of Distributed Database Systems, Third Edition
ÖZSU, M. T., & Valduriez, P. (2011) · Springer eBooks · doi:10.1007/978-1-4419-8834-8 · Cited by 369
Self-citationSwift: Fast, Reliable, Loosely Coupled Parallel Computation
Zhao, Y., Hategan, M., Clifford, B., Foster, I., von Laszewski, G., Nefedova, V., Raicu, I., Stef-Praun, T., & Wilde, M. (2007) · 2007 IEEE Congress on Services (Services 2007) · doi:10.1109/services.2007.63 · Cited by 352
Multilevel Algorithms for Multi-Constraint Graph Partitioning
Karypis, G., & Kumar, V. (1998) · doi:10.5555/509058.509086 · Cited by 342
Managing data transfers in computer clusters with orchestra
Chowdhury, M., Zaharia, M., Ma, J., Jordan, M. I., & Stoica, I. (2011) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2043164.2018448 · Cited by 305
Provenance Collection Support in the Kepler Scientific Workflow System
Altintas, I., Barney, O., & Jaeger-Frank, E. (2006) · Lecture Notes in Computer Science · doi:10.1007/11890850_14 · Cited by 269
OpenNebula: A Cloud Management Tool
Milojicic, D. S., Llorente, I. M., & Montero, R. S. (2011) · IEEE Internet Computing · doi:10.1109/mic.2011.44 · Cited by 249
The Triana Workflow Environment: Architecture and Applications
Taylor, I., Shields, M., Wang, I., & Harrison, A. (2007) · Springer eBooks · doi:10.1007/978-1-84628-757-2_20 · Cited by 240
Montage: a grid portal and software toolkit for science-grade astronomical image mosaicking
Jacob, J. C., Katz, D. S., Berriman, G. B., Good, J., Laity, A. C., Deelman, E., Kesselman, C., Singh, G., Su, M.-H., Prince, T. A., et al. (2009) · International Journal of Computational Science and Engineering · doi:10.1504/ijcse.2009.026999 · Cited by 224
Galaxy CloudMan: delivering cloud compute clusters
Afgan, E., Baker, D., Coraor, N., Chapman, B., Nekrutenko, A., & Taylor, J. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-s12-s4 · Cited by 179
Missier, P., Soiland‐Reyes, S., Owen, S., Tan, W., Nenadić, A., Dunlop, I., Williams, A. R., Oinn, T., & Goble, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13818-8_33 · Cited by 178
de Oliveira, D., Ogasawara, E., Baião, F., & Mattoso, M. (2010) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2010.64 · Cited by 176
Self-citationAlbrecht, M., Donnelly, P., Bui, P. J., & Thain, D. (2012) · doi:10.1145/2443416.2443417 · Cited by 155
GroudSim: An Event-Based Simulation Framework for Computational Grids and Clouds
Ostermann, S., Plankensteiner, K., Prodan, R., & Fahringer, T. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21878-1_38 · Cited by 149
Tavaxy: Integrating Taverna and Galaxy workflows with cloud computing support
Abouelhoda, M. I., Issa, S., & Ghanem, M. M. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-77 · Cited by 140
Scheduling Data-IntensiveWorkflows onto Storage-Constrained Distributed Resources
Ramakrishnan, A., Singh, G., Zhao, H., Deelman, E., Sakellariou, R., Vahi, K., Blackburn, K., Meyers, D., & Samidi, M. (2007) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2007.101 · Cited by 133
Multi-Objective Game Theoretic Schedulingof Bag-of-Tasks Workflows on Hybrid Clouds
Duan, R., Prodan, R., & Li, X. (2014) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2014.2303077 · Cited by 120
A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds
de Oliveira, D., Ocaña, K., Baião, F., & Mattoso, M. (2012) · Journal of Grid Computing · doi:10.1007/s10723-012-9227-2 · Cited by 104
Self-citationOpenStack Cloud Computing Cookbook
Jackson, K. L., Bunch, C., & Sigler, E. (2012) · Cited by 103
Common motifs in scientific workflows: An empirical analysis
Garijo, D., Alper, P., Belhajjame, K., Corcho, Ó., Gil, Y., & Goble, C. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2013.09.018 · Cited by 95
Taverna/myGrid: Aligning a Workflow System with the Life Sciences Community
Oinn, T., Li, P., Kell, D. B., Goble, C., Goderis, A., Greenwood, R. M., Hull, D., Stevens, R., Turi, D., & Zhao, J. (2007) · Springer eBooks · doi:10.1007/978-1-84628-757-2_19 · Cited by 93
Provenance trails in the Wings/Pegasus system
Kim, J., Deelman, E., Gil, Y., Mehta, G., & Ratnakar, V. (2008) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.1228 · Cited by 93
A cost-effective strategy for intermediate data storage in scientific cloud workflow systems
Yuan, D., Yang, Y. J., Liu, X., & Chen, J. (2010) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2010.5470453 · Cited by 90
Advanced Topics In Workflow Management: Issues, Requirements, And Solutions
van der Aalst, W. M. P., Weske, M., & Wirtz, G. (2003) · doi:10.5555/1255154.1255158 · Cited by 88
Workflow task clustering for best effort systems with Pegasus
Singh, G., Su, M.-H., Vahi, K., Deelman, E., Berriman, G. B., Good, J., Katz, D. S., & Mehta, G. (2008) · ACM Mardi Gras Conference · doi:10.1145/1341811.1341822 · Cited by 86
Dynamic steering of HPC scientific workflows: A survey
Mattoso, M., Dias, J., Ocaña, K., Ogasawara, E., Costa, F., Horta, F., Silva, V., & de Oliveira, D. (2014) · Future Generation Computer Systems · doi:10.1016/j.future.2014.11.017 · Cited by 80
Self-citationScientific Workflows in the Cloud
Juve, G., & Deelman, E. (2010) · Springer eBooks · doi:10.1007/978-0-85729-049-6_4 · Cited by 78
Scientific Workflow Systems for 21st Century, New Bottle or New Wine?
Zhao, Y., Raicu, I., & Foster, I. T. (2008) · IEEE World Congress on Services (SERVICES) · doi:10.1109/services-1.2008.79 · Cited by 72
Parallelization in Scientific Workflow Management SystemsPreprint
Bux, M., & Leser, U. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1303.7195 · Cited by 52
Balanced Task Clustering in Scientific Workflows
Chen, W., da Silva, R. F., Deelman, E., & Sakellariou, R. (2013) · IEEE International Conference on eScience · doi:10.1109/escience.2013.40 · Cited by 50
Performance evaluation of parallel strategies in public clouds: A study with phylogenomic workflows
de Oliveira, D., Ocaña, K., Ogasawara, E., Dias, J., Gonçalves, J., Baião, F., & Mattoso, M. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2012.12.019 · Cited by 47
Self-citationProvenance for MapReduce-based data-intensive workflows
Crawl, D. L., Wang, J., & Altıntaş, İ. (2011) · Workshop on Workflows in Support of Large-Scale Science (WORKS) · doi:10.1145/2110497.2110501 · Cited by 47
Integration of Workflow Partitioning and Resource Provisioning
Chen, W., & Deelman, E. (2012) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2012.57 · Cited by 47
Anglano, C., & Canonico, M. (2008) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2008.4536445 · Cited by 45
An Essential Guide to the Basic Local Alignment Search Tool
Korf, I. F., Yandell, M. D., Bedell, J. A., et al. (2003) · Cited by 44
An Essential Guide to the Basic Local Alignment Search Tool: BLAST
Konerding, D. (2004) · Briefings in Bioinformatics · doi:10.1093/bib/5.1.93 · Cited by 36
Exploring Molecular Evolution Reconstruction Using a Parallel Cloud Based Scientific Workflow
Ocaña, K., de Oliveira, D., Horta, F., Dias, J., Ogasawara, E., & Mattoso, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-31927-3_16 · Cited by 30
Self-citationOnline workflow management and performance analysis with Stampede
Gunter, D., Deelman, E., Samak, T., Brooks, C. H., Goode, M., Juve, G., Mehta, G., Moraes, P., Silva, F., Martin Swany, A., et al. (2011) · Conference on Network and Service Management · doi:10.5555/2147671.2147695 · Cited by 28
Deploying Bioinformatics Workflows on Clouds with Galaxy and Globus Provision
Liu, B., Sotomayor, B., Madduri, R., Chard, K., & Foster, I. T. (2012) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc.companion.2012.131 · Cited by 27
Turbine: a distributed-memory dataflow engine for extreme-scale many-task applications
Wozniak, J. M., Armstrong, T. G., Maheshwari, K. C., Lusk, E. L., Katz, D. S., Wilde, M., & Foster, I. T. (2012) · doi:10.1145/2443416.2443421 · Cited by 26
Using Domain-Specific Data to Enhance Scientific Workflow Steering Queries
Gonçalves, J. C. D. A. R., de Oliveira, D., Ocaña, K., Ogasawara, E., & Mattoso, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34222-6_12 · Cited by 20
Self-citationScientific Workflow Makespan Reduction through Cloud Augmented Desktop Grids
Reynolds, C. J., Winter, S. C., Terstyanszky, G. Z., Kiss, T., Greenwell, P., Ács, S., & Kacsuk, P. (2011) · IEEE International Conference on Cloud Computing Technology and Science · doi:10.1109/cloudcom.2011.13 · Cited by 18
Discovering drug targets for neglected diseases using a pharmacophylogenomic cloud workflow
Ocaña, K., de Oliveira, D., Dias, J., Ogasawara, E., & Mattoso, M. (2012) · IEEE International Conference on eScience · doi:10.1109/escience.2012.6404431 · Cited by 14
Self-citationLessons learned from Galaxy, a Web-based platform for high-throughput genomic analyses
Goecks, J., The Galaxy Team, Nekrutenko, A., & Taylor, J. (2012) · IEEE International Conference on eScience · doi:10.1109/escience.2012.6404442 · Cited by 8
P2P Techniques for Decentralized Applications
Pacitti, E., Akbarinia, R., & Dick, M. E. (2012) · Synthesis lectures on data management · doi:10.1007/978-3-031-01888-6 · Cited by 7
Self-citationAbstract: Using Provenance to Visualize Data from Large-Scale Experiments
Horta, F., Dias, J., Ocaña, K., de Oliveira, D., Ogasawara, E., & Mattoso, M. (2012) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc.companion.2012.228 · Cited by 5
Self-citationP2P Techniques for Decentralized Applications
Pacitti, E., Akbarinia, R., & Dick, M. E. (2012) · Synthesis lectures on data management · doi:10.2200/s00414ed1v01y201204dtm025 · Cited by 3
Self-citation
Cited by
From distributed machine learning to federated learning: a survey
Liu, J., Huang, J., Zhou, Y., Li, X., Ji, S., Xiong, H., & Dou, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01664-x · Cited by 356
Self-citationKissling, W. D., Ahumada, J. A., Bowser, A., Fernandez, M., Fernández, N., García, E. A., Guralnick, R. P., Isaac, N. J. B., Kelling, S., Los, W., et al. (2018) · Biological Reviews · doi:10.1111/brv.12359 · Cited by 314
Metadata
- Authors
- Liu, J., Pacitti, E., Valduriez, P., & Mattoso, M.
- Venue
- Journal of Grid Computing
- Published
- 2015-03-08
- Volume
- 13
- Issue
- 4
- Pages
- 457-493
- ISSN
- 1570-7873 (ISSN-L)
- 1572-9184
- OpenAlex
- W1974752004
- Cited by
- 297
- References
- 71
- FWCI
- 34.54
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.