How Modeling Standards, Software, and Initiatives Support Reproducibility in Systems Biology and Systems Medicine

An integrated encyclopedia of DNA elements in the human genome

The ENCODE Project Consortium (2012) · Nature · doi:10.1038/nature11247 · Cited by 18,072 · see citing works

The Logic of Scientific Discovery

Bhaskar, R. (2013) · Routledge eBooks · doi:10.4324/9780203090732-9 · Cited by 7,765 · see citing works

Why Most Published Research Findings Are False

Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works

Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences

Goecks, J., Nekrutenko, A., Taylor, J., & The Galaxy Team (2010) · Genome biology · doi:10.1186/gb-2010-11-8-r86 · Cited by 3,437 · see citing works

The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models

Hucka, M., Finney, A., Sauro, H. M., Bolouri, H., Doyle, J. C., Kitano, H., Arkin, A. P., Bornstein, B. J., Bray, D., Cornish-Bowden, A., et al. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg015 · Cited by 3,060 · see citing works

Raise standards for preclinical cancer research

Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1× · see citing works

Promoting an open research culture

Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · see citing works

COPASI—a COmplex PAthway SImulator

Hoops, S., Sahle, S., Gauges, R., Lee, C., Pahle, J., Simus, N., Singhal, M., Xu, L., Mendes, P., & Kummer, U. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl485 · Cited by 2,618 · see citing works

Believe it or not: how much can we rely on published data on potential drug targets?

Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · see citing works

Increasing value and reducing waste in research design, conduct, and analysis

Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · see citing works

Reproducible research in computational science

Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449 · see citing works

Policy: NIH plans to enhance reproducibility

Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · see citing works

The Logic of Scientific Discovery

Popper, K. (2005) · Routledge eBooks · doi:10.4324/9780203994627 · Cited by 1,195 · see citing works

An introduction to Docker for reproducible research

Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059 · see citing works

Reproducibility in Science: Improving the Standard for Basic and Preclinical Research

Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035 · see citing works

Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)

Murphy, S. N., Weber, G. M., Mendis, M., Gainer, V. S., Chueh, H. C., Churchill, S. E., & Kohane, I. S. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.000893 · Cited by 926 · see citing works

The systems biology graphical notation.

Novère, N. L., Hucka, M., Mi, H., Moodie, S., Schreiber, F., Sorokin, A., Demir, E., Wegner, K., Aladjem, M. I., Wimalaratne, S. M., et al. (2009) · Nature Biotechnology · doi:10.1038/nbt.1558 · Cited by 903 · see citing works

Sharing Detailed Research Data Is Associated with Increased Citation Rate

Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000308 · Cited by 881 · see citing works

Ten Simple Rules for Reproducible Computational Research

Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880 · see citing works

How to Make More Published Research True

Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760 · see citing works

The BioPAX community standard for pathway data sharing

Demir, E., Cary, M. P., Paley, S., Fukuda, K., Lemer, C., Vastrik, I., Wu, G., D'Eustachio, P., Schaefer, C., Luciano, J., et al. (2010) · Nature Biotechnology · doi:10.1038/nbt.1666 · Cited by 745 · cited 1× · see citing works

Minimum information requested in the annotation of biochemical models (MIRIAM)

Novère, N. L., Finney, A., Hucka, M., Bhalla, U. S., Campagne, F., Collado-Vides, J., Crampin, E. J., Halstead, M., Klipp, E., Mendes, P., et al. (2005) · Nature Biotechnology · doi:10.1038/nbt1156 · Cited by 622 · see citing works

Phase II failures: 2008–2010

Arrowsmith, J. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439 · Cited by 622 · see citing works

BioModels Database: An enhanced, curated and annotated resource for published quantitative kinetic models

Li, C., Donizelli, M., Rodriguez, N., Dharuri, H., Endler, L., Chelliah, V., Li, L., He, E., Henry, A., Stefan, M. I., et al. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-92 · Cited by 597 · see citing works

WaveLab and Reproducible Research

Buckheit, J. B., & Donoho, D. L. (1995) · Lecture notes in statistics · doi:10.1007/978-1-4612-2544-7_5 · Cited by 551 · see citing works

Repeatability of published microarray gene expression analyses

Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., et al. (2009) · Nature Genetics · doi:10.1038/ng.295 · Cited by 551 · see citing works

Toward interoperable bioscience data

Sansone, S.-A., Rocca-Serra, P., Field, D., Maguire, E., Taylor, C., Hofmann, O., Fang, H., Neumann, S., Tong, W., Amaral-Zettler, L., et al. (2012) · Nature Genetics · doi:10.1038/ng.1054 · Cited by 475 · cited 2× · see citing works

An introduction to Docker for reproducible research, with examples from the R environmentPreprint

Boettiger, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0846 · Cited by 372 · see citing works

A HUPO test sample study reveals common problems in mass spectrometry–based proteomics

HUPO Test Sample Working Group, Bell, A. W., Deutsch, E. W., Au, C. E., Kearney, R. E., Beavis, R., Sechi, S., Nilsson, T., & Bergeron, J. J. M. (2009) · Nature Methods · doi:10.1038/nmeth.1333 · Cited by 351 · see citing works

Why linked data is not enough for scientists

Bechhofer, S., Buchan, I. E., Roure, D. D., Paolo Missier, B., Ainsworth, J. D., Bhagat, J., Couch, P., Cruickshank, D. G., Delderfield, M. R., Dunlop, I., et al. (2011) · Future Generation Computer Systems · doi:10.1016/j.future.2011.08.004 · Cited by 350 · see citing works

ModelDB: A Database to Support Computational Neuroscience

Hines, M. L., Morse, T. M., Migliore, M., Carnevale, N. T., & Shepherd, G. M. (2004) · Journal of Computational Neuroscience · doi:10.1023/b:jcns.0000023869.22017.2e · Cited by 347 · see citing works

An Overview of CellML 1.1, a Biological Model Description Language

Cuellar, A. A., Lloyd, C. M., Nielsen, P. F., Bullivant, D. P., Nickerson, D. P., & Hunter, P. J. (2003) · SIMULATION · doi:10.1177/0037549703040939 · Cited by 345 · see citing works

NeuroML: A Language for Describing Data Driven Models of Neurons and Networks with a High Degree of Biological Detail

Gleeson, P., Crook, S., Cannon, R., Hines, M., Billings, G. O., Farinella, M., Morse, T. M., Davison, A. P., Ray, S., Bhalla, U., et al. (2010) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.1000815 · Cited by 337 · see citing works

Web-based kinetic modelling using JWS Online

Olivier, B. G., & Snoep, J. L. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth200 · Cited by 326 · see citing works

EQUATOR: reporting guidelines for health research

Altman, D. G., Simera, I., Hoey, J., Moher, D., & Schulz, K. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60505-x · Cited by 325 · see citing works

Controlled vocabularies and semantics in systems biology

Courtot, M., Juty, N., Knüpfer, C., Waltemath, D., Zhukova, A., Dräger, A., Dumontier, M., Finney, A., Golebiewski, M., Hastings, J., et al. (2011) · Molecular Systems Biology · doi:10.1038/msb.2011.77 · Cited by 316 · cited 1× · see citing works

An open investigation of the reproducibility of cancer biology research

Errington, T. M., Iorns, E., Gunn, W., Tan, F. E., Lomax, J., & Nosek, B. A. (2014) · eLife · doi:10.7554/elife.04333 · Cited by 312 · see citing works

How scientists fool themselves – and how they can stop

Nuzzo, R. (2015) · Nature · doi:10.1038/526182a · Cited by 296 · see citing works

The Synthetic Biology Open Language (SBOL) provides a community standard for communicating designs in synthetic biology

Galdzicki, M., Clancy, K., Oberortner, E., Pocock, M., Quinn, J. Y., Rodríguez, C., Roehner, N., Wilson, M., Adam, L., Anderson, J. C., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.2891 · Cited by 288 · see citing works

Reproducible computational biology experiments with SED-ML - The Simulation Experiment Description Markup Language

Waltemath, D., Adams, R., Bergmann, F. T., Hucka, M., Kolpakov, F., Miller, A. K., Moraru, I., Nickerson, D., Sahle, S., Snoep, J., et al. (2011) · BMC Systems Biology · doi:10.1186/1752-0509-5-198 · Cited by 269 · see citing works

Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals

Stodden, V., Guo, P., & Ma, Z. (2013) · PLoS ONE · doi:10.1371/journal.pone.0067111 · Cited by 257 · see citing works

EQUATOR: reporting guidelines for health research.

Altman, D. G., Simera, I., Hoey, J., Moher, D., & Schulz, K. (2008) · PubMed Central · Cited by 224 · see citing works

Blind analysis: Hide results to seek the truth

MacCoun, R., & Perlmutter, S. (2015) · Nature · doi:10.1038/526187a · Cited by 218 · see citing works

Identifiers.org and MIRIAM Registry: community resources to provide persistent identification

Juty, N., Novère, N. G. L., & Laibe, C. (2011) · Nucleic Acids Research · doi:10.1093/nar/gkr1097 · Cited by 205 · see citing works

Reproducible research can still be wrong: Adopting a prevention approach

Leek, J. T., & Peng, R. D. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1421412111 · Cited by 193 · see citing works

Incongruence between test statistics and P values in medical papers

García-Berthou, E., & Alcaraz, C. (2004) · BMC Medical Research Methodology · doi:10.1186/1471-2288-4-13 · Cited by 181 · see citing works

COMBINE archive and OMEX format: one file to share all information to reproduce a modeling project

Bergmann, F. T., Adams, R., Moodie, S., Cooper, J., Glont, M., Golebiewski, M., Hucka, M., Laibe, C., Miller, A. K., Nickerson, D. P., et al. (2014) · BMC Bioinformatics · doi:10.1186/s12859-014-0369-z · Cited by 173 · see citing works

The Physiome Model Repository 2

Yu, T., Lloyd, C. M., Nickerson, D. P., Cooling, M. T., Miller, A. K., Garny, A., Terkildsen, J. R., Lawson, J., Britten, R. D., Hunter, P. J., et al. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btq723 · Cited by 169 · see citing works

Minimum Information About a Simulation Experiment (MIASE)

Waltemath, D., Adams, R., Beard, D. A., Bergmann, F. T., Bhalla, U. S., Britten, R., Chelliah, V., Cooling, M. T., Cooper, J., Crampin, E. J., et al. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001122 · Cited by 168 · see citing works

Research Objects: Towards Exchange and Reuse of Digital KnowledgePreprint

Bechhofer, S., Roure, D. D., Gamble, M., Goble, C., & Buchan, I. E. (2010) · Nature Precedings · doi:10.1038/npre.2010.4626.1 · Cited by 168 · see citing works

Tools and techniques for computational reproducibility

Piccolo, S., & Frampton, M. (2016) · GigaScience · doi:10.1186/s13742-016-0135-4 · Cited by 167 · see citing works

A Survey on Data Reproducibility in Cancer Research Provides Insights into Our Limited Ability to Translate Findings from the Laboratory to the Clinic

Mobley, A. K., Linder, S. K., Braeuer, R. R., Ellis, L. M., & Zwelling, L. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0063221 · Cited by 164 · see citing works

Promoting Coordinated Development of Community-Based Information Standards for Modeling in Biology: The COMBINE Initiative

Hucka, M., Nickerson, D. P., Bader, G. D., Bergmann, F., Cooper, J., Demir, E., Garny, A., Golebiewski, M., Myers, C. J., Schreiber, F., et al. (2015) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2015.00019 · Cited by 118 · see citing works

Quantifying Reproducibility in Computational Biology: The Case of the Tuberculosis Drugome

Garijo, D., Kinnings, S., Xie, L., Xie, L., Zhang, Y., Bourne, P. E., & Gil, Y. (2013) · PLoS ONE · doi:10.1371/journal.pone.0080278 · Cited by 114 · see citing works

SEEK: a systems biology data and model management platform

Wolstencroft, K., Owen, S., Krebs, O., Nguyen, Q., Stanford, N., Golebiewski, M., Weidemann, A., Bittkowski, M., An, L., Shockley, D., et al. (2015) · BMC Systems Biology · doi:10.1186/s12918-015-0174-y · Cited by 113 · see citing works

Data archiving is a good investment

Piwowar, H., Vision, T., & Whitlock, M. C. (2011) · Nature · doi:10.1038/473285a · Cited by 108 · see citing works

Annotation and merging of SBML models with semanticSBML

Krause, F., Uhlendorf, J., Lubitz, T., Schulz, M., Klipp, E., & Liebermeister, W. (2010) · Bioinform. · doi:10.1093/bioinformatics/btp642 · Cited by 108 · see citing works

The SBML ODE Solver Library: a native API for symbolic and fast numerical analysis of reaction networks

Machné, R., Finney, A., Müller, S., Lu, J. J.-Q., Stefanie Widder, A., & Christoph Flamm, A. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl086 · Cited by 105 · see citing works

Reliability of 'new drug target' claims called into question

Mullard, A. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3545 · Cited by 98 · see citing works

Model storage, exchange and integration

Novère, N. G. L. (2006) · BMC Neuroscience · doi:10.1186/1471-2202-7-s1-s11 · Cited by 97 · see citing works

Ten Simple Rules for a Computational Biologist’s Laboratory Notebook

Schnell, S. D. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004385 · Cited by 67 · see citing works

BioModels: Content, Features, Functionality, and Use

Juty, N., Ali, R., Glont, M., Keating, S., Rodríguez, N., Swat, M. J., Wimalaratne, S., Hermjakob, H., Novère, N. G. L., Laibe, C., et al. (2015) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.3 · Cited by 66 · see citing works

The Cardiac Electrophysiology Web Lab

Cooper, J., Scharm, M., & Mirams, G. R. (2016) · Biophysical Journal · doi:10.1016/j.bpj.2015.12.012 · Cited by 58 · see citing works

SBW - a modular framework for systems biology

Bergmann, F., & Sauro, H. M. (2006) · Winter Simulation Conference · doi:10.5555/1218112.1218411 · Cited by 57 · see citing works

Combining computational models, semantic annotations and simulation experiments in a graph database

Henkel, R., Wolkenhauer, O., & Waltemath, D. (2015) · Database J. Biol. Databases Curation · doi:10.1093/database/bau130 · Cited by 52 · see citing works

Construction and validation of a detailed kinetic model of glycolysis in Plasmodium falciparum

Penkler, G. P., du Toit, F., Adams, W. W., Rautenbach, M., Palm, D. C., van Niekerk, D. D., & Snoep, J. L. (2015) · FEBS Journal · doi:10.1111/febs.13237 · Cited by 48 · see citing works

Which gene did you mean?

Mons, B. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-142 · Cited by 45 · see citing works

Software review: the KNIME workflow environment and its applications in genetic programming and machine learning

O’Hagan, S., & Kell, D. B. (2015) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-015-9247-3 · Cited by 43 · see citing works

Pharmacometrics Markup Language (PharmML): Opening New Perspectives for Model Exchange in Drug Development

Swat, M. J., Moodie, S., Wimalaratne, S., Kristensen, N. R., Lavielle, M., Mari, A., Magni, P., Smith, M. K., Bizzotto, R., Pasotti, L., et al. (2015) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.57 · Cited by 39 · see citing works

SBW - A Modular Framework for Systems Biology

Bergmann, F., & Sauro, H. M. (2006) · Winter Simulation Conference · doi:10.1109/wsc.2006.322938 · Cited by 38 · see citing works

Simulation Experiment Description Markup Language (SED-ML) Level 1 Version 3 (L1V3)

Bergmann, F. T., Cooper, J., König, M., Moraru, I., Nickerson, D., Le Novère, N., Olivier, B. G., Sahle, S., Smith, L., & Waltemath, D. (2018) · Journal of Integrative Bioinformatics · doi:10.1515/jib-2017-0086 · Cited by 37 · see citing works

A long journey into reproducible computational neuroscience

Topalidou, M., Leblois, A., Boraud, T., & Rougier, N. P. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00030 · Cited by 36 · see citing works

Data management strategies for multinational large-scale systems biology projects

Wruck, W., Peuker, M., & Regenbrecht, C. (2012) · Briefings Bioinform. · doi:10.1093/bib/bbs064 · Cited by 36 · see citing works

The Open Source Brain Initiative: enabling collaborative modelling in computational neuroscience

Gleeson, P., Piasini, E., Crook, S. M., Cannon, R. C., Steuber, V., Jaeger, D., Solinas, S. M. G., D'Angelo, E. U., & Silver, R. A. (2012) · BMC Neuroscience · doi:10.1186/1471-2202-13-s1-o7 · Cited by 35 · see citing works

The systems biology format converter

Rodríguez, N., Pettit, J., Pezze, P. D., Li, L., Henry, A., van Iersel, M., Jalowicki, G., Kutmon, M., Natarajan, K. N., Tolnay, D., et al. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-1000-2 · Cited by 35 · see citing works

Ranked retrieval of Computational Biology models

Henkel, R., Endler, L., Peters, A., Novère, N. G. L., & Waltemath, D. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-423 · Cited by 32 · see citing works

SYCAMORE—a systems biology computational analysis and modeling research environment

Weidemann, A., Richter, S., Stein, M., Sahle, S., Gauges, R., Gabdoulline, R. R., Surovtsova, I., Semmelrock, N., Besson, B., Rojas, I., et al. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn207 · Cited by 30 · see citing works

Meeting report from the fourth meeting of the Computational Modeling in Biology Network (COMBINE)

Waltemath, D., Bergmann, F., Chaouiya, C., Czauderna, T., Gleeson, P., Goble, C., Golebiewski, M., Hucka, M., Juty, N., Krebs, O., et al. (2014) · Standards in Genomic Sciences · doi:10.4056/sigs.5279417 · Cited by 30 · see citing works

Addressing the challenges of multiscale model management in systems biology

Hetherington, J., Bogle, I. D. L., Saffrey, P., Margoninski, O., Li, L., Rey, M., Yamaji, S., Baigent, S., Ashmore, J. F., Page, K. M., et al. (2006) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2006.10.004 · Cited by 28 · see citing works

GigaDB: promoting data dissemination and reproducibility

Sneddon, T. P., Zhe, X. S., Edmunds, S., Li, P., Goodman, L., & Hunter, C. (2014) · Database · doi:10.1093/database/bau018 · Cited by 28 · see citing works

An algorithm to detect and communicate the differences in computational models describing biological systems

Scharm, M., Wolkenhauer, O., & Waltemath, D. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btv484 · Cited by 24 · see citing works

Large-scale data sharing in the life sciences: Data standards, incentives, barriers and funding models (The "Joint Data Standards Study")

Lord, P., Macdonald, A. M., Sinnott, R., Ecklund, D. J., Westhead, M., & Jones, A. (2005) · Cited by 19 · see citing works

Simulation Experiment Description Markup Language (SED-ML) Level 1 Version 2

Bergmann, F. T., Cooper, J., Novère, N. L., Nickerson, D., & Waltemath, D. (2015) · Journal of Integrative Bioinformatics · doi:10.1515/jib-2015-262 · Cited by 15 · see citing works

Reproducibility of Model-Based Results in Systems Biology

Waltemath, D., Henkel, R., Winter, F., & Wolkenhauer, O. (2013) · Springer eBooks · doi:10.1007/978-94-007-6803-1_10 · Cited by 11 · see citing works

A Profile of Today's SBML-Compatible Software

Hucka, M., Bergmann, F., Keating, S., & Smith, L. P. (2011) · IEEE International Conference on e-Science Workshops · doi:10.1109/esciencew.2011.28 · Cited by 10 · see citing works

Reproducibility Undergoes Scrutiny

Blaustein, R. J. (2014) · BioScience · doi:10.1093/biosci/biu017 · Cited by 6 · see citing works

Extracting reproducible simulation studies from model repositories using the combinearchive toolkitPreprint

Scharm, M., & Waltemath, D. (2015) · Datenbanksysteme für Business, Technologie und Web · doi:10.7287/peerj.preprints.792v1 · Cited by 6 · see citing works

Extracting reproducible simulation studies from model repositories using the CombineArchive ToolkitPreprint

Scharm, M., & Waltemath, D. (2015) · doi:10.7287/peerj.preprints.792 · Cited by 2 · see citing works

COMBINE Archive Show Case

Scharm, M., Touré, V., & König, M. (2016) · Figshare · doi:10.6084/m9.figshare.3427271 · Cited by 2 · see citing works

SBML Level 3: an extensible format for the exchange and reuse of biological models

Keating, S. M., Waltemath, D., König, M., Zhang, F.-K., Dräger, A., Chaouiya, C., Bergmann, F. T., Finney, A., Gillespie, C. S., Helikar, T., et al. (2020) · Molecular Systems Biology · doi:10.15252/msb.20199110 · Cited by 270 · see citing works

Towards reproducible computational drug discovery

Schaduangrat, N., Lampa, S., Simeon, S., Gleeson, M. P., Spjuth, O., & Nantasenamat, C. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-0408-x · Cited by 208 · see citing works

Mechanisms of action of sacubitril/valsartan on cardiac remodeling: a systems biology approach

Egea, O. I., Gálvez‐Montón, C., Roura, S., Perea‐Gil, I., Prat‐Vidal, C., Soler‐Botija, C., & Bayés‐Genís, A. (2017) · npj Systems Biology and Applications · doi:10.1038/s41540-017-0013-4 · Cited by 137 · see citing works

Unraveling the Molecular Mechanism of Action of Empagliflozin in Heart Failure With Reduced Ejection Fraction With or Without Diabetes

Egea, O. I., Santiago‐Vacas, E., Yurista, S. R., Lupón, J., Packer, M., Heymans, S. R. B., Zannad, F. M., Butler, J., Pascual-Figal, D. A., Lax, A., et al. (2019) · JACC Basic to Translational Science · doi:10.1016/j.jacbts.2019.07.010 · Cited by 102 · see citing works

Parameter estimation and uncertainty quantification for systems biology models

Mitra, E. D., & Hlavacek, W. S. (2019) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2019.10.006 · Cited by 79 · see citing works

Harmonizing semantic annotations for computational models in biology

Neal, M. L., König, M., Nickerson, D., Mısırlı, G., Kalbasi, R., Dräger, A., Atalag, K., Chelliah, V., Cooling, M. T., Cook, D. L., et al. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bby087 · Cited by 74 · see citing works

Use of nonintrusive sensor‐based information and communication technology for real‐world evidence for clinical trials in dementia

Teipel, S. J., König, A., Hoey, J., Kaye, J. A., Krüger, F., Robillard, J. M., Kirste, T., & Babiloni, C. (2018) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2018.05.003 · Cited by 66 · see citing works

Tellurium notebooks—An environment for reproducible dynamical modeling in systems biology

Medley, J., Choi, K., König, M., Smith, L. P., Gu, S., Hellerstein, J., Sealfon, S., & Sauro, H. (2018) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.1006220 · Cited by 58 · see citing works

Reproducibility of Quantitative Systems Pharmacology Models: Current Challenges and Future Opportunities

Kirouac, D. C., Cicali, B., & Schmidt, S. (2019) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12390 · Cited by 46 · see citing works

PyBioNetFit and the Biological Property Specification Language

Mitra, E. D., Suderman, R., Colvin, J. C., Ionkov, A., Hu, A., Sauro, H. M., Posner, R. G., & Hlavacek, W. S. (2019) · iScience · doi:10.1016/j.isci.2019.08.045 · Cited by 45 · see citing works

Best Practices for Making Reproducible Biochemical Models

Porubsky, V. L., Goldberg, A. P., Rampadarath, A. K., Nickerson, D. P., Karr, J. R., & Sauro, H. M. (2020) · Cell Systems · doi:10.1016/j.cels.2020.06.012 · Cited by 45 · see citing works

Multiscale Modeling of Bone Healing: Toward a Systems Biology Approach

Borgiani, E., Duda, G. N., & Checa, S. (2017) · Frontiers in Physiology · doi:10.3389/fphys.2017.00287 · Cited by 45 · see citing works

Ten simple rules on how to create open access and reproducible molecular simulations of biological systems

Elofsson, A., Hess, B., Lindahl, E. R., Onufriev, A. V., van der Spoel, D., & Wallqvist, A. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006649 · Cited by 43 · see citing works

Microbiome modeling: a beginner's guide

Lange, E., Kranert, L., Krüger, J., Benndorf, D., & Heyer, R. (2024) · Frontiers in Microbiology · doi:10.3389/fmicb.2024.1368377 · Cited by 21 · see citing works

Bayesian probabilistic modeling in pharmaceutical process development

Tábora, J. E., Gonzalez, F. L., & Tom, J. W. (2019) · AIChE Journal · doi:10.1002/aic.16744 · Cited by 21 · see citing works

Notions of similarity for systems biology models

Henkel, R., Hoehndorf, R., Kacprowski, T., Knüpfer, C. R. N., Liebermeister, W., & Waltemath, D. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw090 · Cited by 18 · see citing works

Genome-driven cell engineering review: in vivo and in silico metabolic and genome engineering

Landon, S., Rees-Garbutt, J., Marucci, L., & Grierson, C. S. (2019) · Essays in Biochemistry · doi:10.1042/ebc20180045 · Cited by 16 · see citing works

Improving Process Descriptions in Research by Model-Based Analysis

Shaked, A., & Reich, Y. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.2990488 · Cited by 17 · see citing works

Computational models of melanoma

Albrecht, M., Lucarelli, P., Kulms, D., & Sauter, T. (2020) · Theoretical Biology and Medical Modelling · doi:10.1186/s12976-020-00126-7 · Cited by 17 · see citing works

Progress in integrative systems biology, physiology and medicine: towards a scale-relative biology

Auffray, C., Noble, D., Laurent Nottale, A., & Turner, P. (2020) · The European Physical Journal A · doi:10.1140/epja/s10050-020-00090-3 · Cited by 17 · see citing works

Characteristics of mathematical modeling languages that facilitate model reuse in systems biology: a software engineering perspective

Schölzel, C., Blesius, V., Ernst, G., & Dominik, A. (2021) · npj Systems Biology and Applications · doi:10.1038/s41540-021-00182-w · Cited by 15 · see citing works

Towards a systems approach for chronic diseases, based on health state modeling

Rebhan, M. (2017) · F1000Research · doi:10.12688/f1000research.11085.1 · Cited by 13 · see citing works

Making PBPK models more reproducible in practice

Domínguez-Romero, E., Mazurenko, S., Scheringer, M., dos Santos, V. A. P. M., Evelo, C. T., Anton, M., Hancock, J. M., Županič, A., & Diez, M. S. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae569 · Cited by 12 · see citing works

A fully featured COMBINE archive of a simulation study on syncytial mitotic cycles in Drosophila embryos

Scharm, M., & Waltemath, D. (2016) · F1000Research · doi:10.12688/f1000research.9379.1 · Cited by 11 · see citing works

Executable Simulation Model of the Liver

König, M. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-801238-3.11682-4 · Cited by 10 · see citing works

Characterising reproducibility debt in scientific software: A systematic literature review

Hassan, Z., Treude, C., Norrish, M., Williams, G., & Potanin, A. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112327 · Cited by 12 · see citing works

Characteristics of mathematical modeling languages that facilitate model reuse in systems biology: A software engineering perspectivePreprint

Schölzel, C., Blesius, V., Ernst, G., & Dominik, A. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2019.12.16.875260 · Cited by 9 · see citing works

From FAIR to CURE: guidelines for computational models of biological systems

Sauro, H. M., Agmon, E., Blinov, M., Gennari, J. H., Hellerstein, J. L., Heydarabadipour, A., Jardine, B. E., May, E., Nickerson, D., Smith, L. P., et al. (2026) · npj Systems Biology and Applications · doi:10.1038/s41540-026-00651-0 · Cited by 6 · see citing works

Data Management in Computational Systems Biology: Exploring Standards, Tools, Databases, and Packaging Best Practices

Stanford, N., Scharm, M., Dobson, P. D., Golebiewski, M., Hucka, M., Kothamachu, V. B., Nickerson, D. P., Owen, S., Pahle, J., Wittig, U., et al. (2019) · Methods in molecular biology · doi:10.1007/978-1-4939-9736-7_17 · Cited by 7 · see citing works

Countering reproducibility issues in mathematical models with software engineering techniques: A case study using a one-dimensional mathematical model of the atrioventricular node

Schölzel, C., Blesius, V., Ernst, G., Goesmann, A., & Dominik, A. (2021) · bioRxiv · doi:10.1371/journal.pone.0254749 · Cited by 7 · see citing works

Bayesian estimation reveals that reproducible models in Systems Biology get more citations

Höpfl, S., Pleiss, J., & Radde, N. E. (2023) · Scientific Reports · doi:10.1038/s41598-023-29340-2 · Cited by 6 · see citing works

Reviewability and supportability: New complementary principles to empower research software practices

Zhang, H., Maillo, A., Khan, S. A., Martinez‐de‐Morentin, X., Lehmann, R., Gómez-Cabrero, D., & Tegner, J. N. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.10.034 · Cited by 5 · see citing works

BioArchLinux: community-driven fresh reproducible software repository for life sciences

Zhang, G., Ristola, P., Su, H., Kumar, B., Zhang, B., Hu, Y., Elliot, M. G., Drobot, V., Zhu, J., Staal, J., et al. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf106 · Cited by 5 · see citing works

repo: an R package for data-centered management of bioinformatic pipelines

Napolitano, F. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1510-6 · Cited by 5 · see citing works

A Practical Guide to Reproducible Modeling for Biochemical Networks

Porubsky, V. L., & Sauro, H. M. (2023) · Methods in molecular biology · doi:10.1007/978-1-0716-3008-2_5 · Cited by 4 · see citing works

VeVaPy, a Python Platform for Efficient Verification and Validation of Systems Biology Models with Demonstrations Using Hypothalamic-Pituitary-Adrenal Axis Models

Parker, C., Nelson, E., & Zhang, T. (2022) · Entropy · doi:10.3390/e24121747 · Cited by 4 · see citing works

Reproducibility and FAIR principles: the case of a segment polarity network model

Mendes, P. (2023) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2023.1201673 · Cited by 4 · see citing works

Challenges and opportunities for system biology standards and tools in medical research

König, M., Oellrich, A., Waltemath, D., Dobson, R., Hubbard, T., & Olaf Wolkenhauer, A. (2016) · Research Portal (King's College London) · Cited by 4 · see citing works

Complex Bayesian Modeling Workflows Encoding and Execution Made Easy With a Novel WinBUGS Plugin of the Drug Disease Model Resources Interoperability Framework

Larizza, C., Borella, E., Pasotti, L., Palma, T., Smith, M., Moodie, S., & Magni, P. (2018) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12285 · Cited by 3 · see citing works

Overview: Standards for Modeling in Systems Medicine

Dräger, A., & Waltemath, D. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-816077-0.00001-7 · Cited by 3 · see citing works

Notions of similarity for computational biology modelsPreprint

Henkel, R., Hoehndorf, R., Kacprowski, T., Knüpfer, C. R. N., Liebermeister, W., & Waltemath, D. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/044818 · Cited by 2 · see citing works

Editorial Reproducibility of Computational Models

Erdemir, A., & Sauro, H. M. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2594702 · Cited by 2 · see citing works

Tools for Analyzing R Code the Tidy Way

McGowan, L. D., Kross, S., & Leek, J. T. (2020) · The R Journal · doi:10.32614/rj-2020-011 · Cited by 1 · see citing works

Biomedical Repositories for Simulation Studies

Henkel, R. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-801238-3.11684-8 · Cited by 1 · see citing works

Characterising Reproducibility Debt in Scientific Software:A Systematic Literature ReviewPreprint

Hassan, Z., Treude, C., Norrish, M., Williams, G., & Potanin, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4801433 · Cited by 1 · see citing works

An FSKX compliant source attribution model for salmonellosis and a look at its major hidden pitfalls

Sundermann, E. M., Carreira, G. C., & Käsbohrer, A. (2021) · Food Modelling Journal · doi:10.3897/fmj.2.70008

A Plan for Model Curation at the US Army Armaments Center

Jauregui, C., & Bone, M. A. (2022) · Springer eBooks · doi:10.1007/978-3-030-82083-1_39

Additional file 1 of Computational models of melanoma

Albrecht, M., Lucarelli, P., Kulms, D., & Sauter, T. (2020) · Figshare · doi:10.6084/m9.figshare.12307859

A cellular Digital Twin framework for predictive and mechanistic modeling of drug responses in precision medicine

Danishuddin, Haque, M. A., Madhukar, G., Khan, S., Khan, S. N., & Kim, J.-J. (2026) · Biotechnology Advances · doi:10.1016/j.biotechadv.2026.109040 · Cited by 1 · see citing works

Issues and achievements of the reproducibility movement in systems biology

Waltemath, D. (2016) · Figshare · doi:10.6084/m9.figshare.3383227

Guidelines for reproducibly building and simulating systems biology models

Medley, J., Goldberg, A. P., & Karr, J. R. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2591960 · Cited by 28 · see citing works

Status and Challenges of Reproducibility in Computational Systems and Synthetic Biology

Choi, K., Karr, J. R., & Sauro, H. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-801238-3.11525-9

Practical Resources for Enhancing the Reproducibility of Mechanistic Modeling in Systems Biology

Blinov, M., Gennari, J., Karr, J. R., Moraru, I., Nickerson, D., & Sauro, H. (2021) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2021.06.001 · Cited by 18 · see citing works

Standards, dissemination, and best practices in systems biology

Shin, J., Porubsky, V., Carothers, J., & Sauro, H. M. (2023) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2023.102922 · Cited by 15 · see citing works