Reproducible Research Using Biomodels
Open accessAbstract
Like other types of computational research, modeling and simulation of biological processes (biomodels) is still largely communicated without sufficient detail to allow independent reproduction of results. But reproducibility in this area of research could easily be achieved by making use of existing resources, such as supplying models in standard formats and depositing code, models, and results in public repositories.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Mathematical models from a number of published articles were shown not to reproduce the simulation results described in the article.
“However, mathematical models from a number of published articles were shown not to reproduce the simulation results described in the article (Mendes, 2018).”
From Reproducibility in systems biology modelling (2021) · cites this paper as Mendes, 2018
The Introduction (page unavailable) says many computational articles are hard or impossible to reproduce, and Some recommendations (page unavailable) reports that only a minority of roughly 400 surveyed biomodel articles described their work sufficiently for reproduction. These passages support a reproducibility problem, but do not say that the mathematical models themselves were tested and shown to produce results different from those reported. (LLM, conf=0.84)
Experimental — accuracy not yet measuredBioModels, being the largest repository of open, curated, well-annotated and findable models, contributes significantly to reproducibility.
“BioModels, being the largest repository of open, curated, well‐annotated and findable models, contributes significantly to reproducibility (Mendes, 2018).”
From Reproducibility in systems biology modelling (2021) · cites this paper as Mendes, 2018
In “Publication of electronic materials” (page unavailable), Mendes calls BioModels the most widely used model repository and says its curators verify published results, making submission a major check on reproducibility. The section also says public repositories help materials remain findable. It does not establish that BioModels is the largest repository or that its models are well annotated. (LLM, conf=0.85)
Experimental — accuracy not yet measuredBioModels, being the largest repository of open, curated, well annotated, and findable models, contributes significantly to reproducibility.
“BioModels, being the largest repository of open, curated, well annotated, and findable models, contributes significantly to reproducibility 22 .”
From Reproducibility in systems biology modelling (2020) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Logic of Scientific Discovery.
Hutchison, T. W., & Popper, K. R. (1959) · Economica · doi:10.2307/2550489 · Cited by 7,628 · see citing works
Gillespie, D. T. (1976) · Journal of Computational Physics · doi:10.1016/0021-9991(76)90041-3 · Cited by 6,029 · see citing works
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
Hucka, M., Finney, A., Sauro, H. M., Bolouri, H., Doyle, J. C., Kitano, H., Arkin, A. P., Bornstein, B. J., Bray, D., Cornish-Bowden, A., et al. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg015 · Cited by 3,060 · see citing works
Self-citationThe Logic of Scientific Discovery.
Kendall, M. G., & Popper, K. R. (1959) · Biometrika · doi:10.2307/2332835 · Cited by 2,896 · see citing works
COPASI—a COmplex PAthway SImulator
Hoops, S., Sahle, S., Gauges, R., Lee, C., Pahle, J., Simus, N., Singhal, M., Xu, L., Mendes, P., & Kummer, U. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl485 · Cited by 2,618 · see citing works
Self-citationReproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449 · see citing works
What does research reproducibility mean?
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293 · see citing works
The systems biology graphical notation.
Novère, N. L., Hucka, M., Mi, H., Moodie, S., Schreiber, F., Sorokin, A., Demir, E., Wegner, K., Aladjem, M. I., Wimalaratne, S. M., et al. (2009) · Nature Biotechnology · doi:10.1038/nbt.1558 · Cited by 903 · see citing works
Ten Simple Rules for Reproducible Computational Research
Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880 · see citing works
Le Novere, N. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkj092 · Cited by 762 · see citing works
Minimum information requested in the annotation of biochemical models (MIRIAM)
Novère, N. L., Finney, A., Hucka, M., Bhalla, U. S., Campagne, F., Collado-Vides, J., Crampin, E. J., Halstead, M., Klipp, E., Mendes, P., et al. (2005) · Nature Biotechnology · doi:10.1038/nbt1156 · Cited by 622 · see citing works
Self-citationCellML: its future, present and past
Lloyd, C. M., Halstead, M. D. B., & Nielsen, P. F. (2004) · Progress in Biophysics and Molecular Biology · doi:10.1016/j.pbiomolbio.2004.01.004 · Cited by 499 · see citing works
Reproducibility vs. Replicability: A Brief History of a Confused Terminology
Pleßer, H. E. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2017.00076 · Cited by 426 · see citing works
Enhancing reproducibility for computational methods
Stodden, V., Mcnutt, M. K., Bailey, D. H., Deelman, E., Gil, Y., Hanson, B., Heroux, M. A., Ioannidis, J. P. A., & Taufer, M. (2016) · Science · doi:10.1126/science.aah6168 · Cited by 414 · see citing works
An empirical analysis of journal policy effectiveness for computational reproducibility
Stodden, V., Seiler, J., & Ma, Z. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708290115 · Cited by 369 · see citing works
Gleeson, P., Crook, S., Cannon, R., Hines, M., Billings, G. O., Farinella, M., Morse, T. M., Davison, A. P., Ray, S., Bhalla, U., et al. (2010) · PLoS Comput. Biol. · doi:10.1371/journal.pcbi.1000815 · Cited by 337 · see citing works
Accessible Reproducible Research
Mesirov, J. P. (2010) · Science · doi:10.1126/science.1179653 · Cited by 311 · see citing works
BioModels: ten-year anniversary
Chelliah, V., Juty, N., Ajmera, I., Ali, R., Dumousseau, M., Glont, M., Hucka, M., Jalowicki, G., Keating, S., Knight-Schrijver, V., et al. (2015) · Nucleic Acids Research · doi:10.1093/nar/gku1181 · Cited by 281 · see citing works
Waltemath, D., Adams, R., Bergmann, F. T., Hucka, M., Kolpakov, F., Miller, A. K., Moraru, I., Nickerson, D., Sahle, S., Snoep, J., et al. (2011) · BMC Systems Biology · doi:10.1186/1752-0509-5-198 · Cited by 269 · see citing works
Reproducible research and Biostatistics
Peng, R. D. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp014 · Cited by 244 · see citing works
"High-dilution" experiments a delusion
Maddox, J., Randi, J., & Stewart, W. W. (1988) · Nature · doi:10.1038/334287a0 · Cited by 240 · see citing works
Virtual Cell modelling and simulation software environment
Moraru, I. I., Schaff, J. C., Slepchenko, B. M., Blinov, M. L., Morgan, F., Lakshminarayana, A., Gao, F., Li, Y., & Loew, L. M. (2008) · IET Systems Biology · doi:10.1049/iet-syb:20080102 · Cited by 236 · see citing works
Electronic documents give reproducible research a new meaning
Claerbout, J. F., & Karrenbach, M. (1992) · Second International Meeting for Applied Geoscience & Energy · doi:10.1190/1.1822162 · Cited by 224 · see citing works
Minimum Information About a Simulation Experiment (MIASE)
Waltemath, D., Adams, R., Beard, D. A., Bergmann, F. T., Bhalla, U. S., Britten, R., Chelliah, V., Cooling, M. T., Cooper, J., Crampin, E. J., et al. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001122 · Cited by 168 · see citing works
Tools and techniques for computational reproducibility
Piccolo, S., & Frampton, M. (2016) · GigaScience · doi:10.1186/s13742-016-0135-4 · Cited by 167 · see citing works
Chance, B., Garfinkel, D. J., Higgins, J., Hess, B., & Chance, E. M. (1960) · Journal of Biological Chemistry · doi:10.1016/s0021-9258(18)64638-1 · Cited by 107 · see citing works
Case studies in reproducibility
Hothorn, T., & Leisch, F. (2011) · Briefings in Bioinformatics · doi:10.1093/bib/bbq084 · Cited by 79 · see citing works
SBRML: a markup language for associating systems biology data with models
Dada, J. O., Spasic, I., Paton, N., & Mendes, P. (2010) · Bioinform. · doi:10.1093/bioinformatics/btq069 · Cited by 66 · see citing works
Self-citationApplications and trends in systems biology in biochemistry
Hübner, K., Sahle, S., & Kummer, U. (2011) · FEBS Journal · doi:10.1111/j.1742-4658.2011.08217.x · Cited by 61 · see citing works
Swat, M. J., Moodie, S., Wimalaratne, S., Kristensen, N. R., Lavielle, M., Mari, A., Magni, P., Smith, M. K., Bizzotto, R., Pasotti, L., et al. (2015) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.57 · Cited by 39 · see citing works
Biochemical fluctuations, optimisation and the linear noise approximation
Pahle, J., Challenger, J. D., Mendes, P., & McKane, A. J. (2012) · BMC Systems Biology · doi:10.1186/1752-0509-6-86 · Cited by 26 · see citing works
Self-citationStructuring supplemental materials in support of reproducibility
Greenbaum, D. S., Rozowsky, J. S., Stodden, V., & Gerstein, M. (2017) · Genome biology · doi:10.1186/s13059-017-1205-3 · Cited by 23 · see citing works
Biometrical Journal and Reproducible Research
Hothorn, T., Held, L., & Friede, T. (2009) · Biometrical Journal · doi:10.1002/bimj.200900154 · Cited by 19 · see citing works
Reproducibility of Research in Biophysics
Loew, L. M., Beckett, D., Egelman, E. H., & Scarlata, S. (2015) · Biophysical Journal · doi:10.1016/j.bpj.2015.03.002 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [10]Guerreiro M: Forking software used in eLife papers to GitHub (2017). URL https://elifesciences.org/inside-elife/dbcb6949/forking-software-used-in-elife-papers-to-github
Cited by
Sorted by citations
Reproducibility in systems biology modelling
Tiwari, K., Kananathan, S., Roberts, M. G., Meyer, J. P., Sharif Shohan, M. U., Xavier, A., Maire, M., Zyoud, A., Men, J., Ng, S., et al. (2021) · Molecular Systems Biology · doi:10.15252/msb.20209982 · Cited by 131 · see citing works
Asma, U., Bertotti, M. L., Zamai, S., Arnold, M., Amorati, R., & Scampicchio, M. (2024) · Antioxidants · doi:10.3390/antiox13020222 · Cited by 33 · see citing works
Agent Based Models of Polymicrobial Biofilms and the Microbiome—A Review
Koshy-Chenthittayil, S., Archambault, L. S., Senthilkumar, D., Laubenbacher, R., Mendes, P., & Dongari‐Bagtzoglou, A. I. (2021) · Microorganisms · doi:10.3390/microorganisms9020417 · Cited by 29 · see citing works
Self-citationComputational models in systems biology: standards, dissemination, and best practices
Vieira, L. S., & Laubenbacher, R. (2022) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2022.102702 · Cited by 25 · see citing works
Practical steps to digital organism models, from laboratory model species to ‘Crops in silico
Millar, A. J., Urquiza‐García, U., Freeman, P. L., Hume, A. C., Plotkin, G. D., Sorokina, O., Zardilis, A., & Zieliński, T. (2018) · Journal of Experimental Botany · doi:10.1093/jxb/ery435 · Cited by 15 · see citing works
Masison, J., & Mendes, P. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281401 · Cited by 14 · see citing works
Self-citationRajput, A. J., Aldibani, H. K. A., & Rostami‐Hodjegan, A. (2023) · Biopharmaceutics & Drug Disposition · doi:10.1002/bdd.2357 · Cited by 12 · see citing works
van Kampen, A. H. C., Mahamune, U., Jongejan, A., van Schaik, B. D. C., Balashova, D., Lashgari, D., Pras‐Raves, M. L., Wever, E. J. M., Dane, A. D., García-Valiente, R., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-52446-8 · Cited by 14 · see citing works
From FAIR to CURE: guidelines for computational models of biological systems
Sauro, H. M., Agmon, E., Blinov, M., Gennari, J. H., Hellerstein, J. L., Heydarabadipour, A., Jardine, B. E., May, E., Nickerson, D., Smith, L. P., et al. (2026) · npj Systems Biology and Applications · doi:10.1038/s41540-026-00651-0 · Cited by 6 · see citing works
Self-citationFitzpatrick, R., & Stefan, M. I. (2022) · Neuroinformatics · doi:10.1007/s12021-022-09584-5 · Cited by 6 · see citing works
Reproducibility in systems biology modelling
Tiwari, K., Kananathan, S., Roberts, M. G., Meyer, J. P., Sharif Shohan, M. U., Xavier, A., Maire, M., Zyoud, A., Men, J., Ng, S., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.07.239855 · Cited by 131 · see citing works
Parker, C., Nelson, E., & Zhang, T. (2022) · Entropy · doi:10.3390/e24121747 · Cited by 4 · see citing works
Reproducibility and FAIR principles: the case of a segment polarity network model
Mendes, P. (2023) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2023.1201673 · Cited by 4 · see citing works
Self-citationFAIR Sharing of Reproducible Models of Epidemic and Pandemic Forecast (preprint)Preprint
Ramachandran, K., König, M., Scharm, M., Nguyen, T. V. N., Hermjakob, H., Waltemath, D., & Sheriff, R. S. M. (2022) · Preprints.org · doi:10.20944/preprints202206.0137.v1 · Cited by 2 · see citing works
Advancing Reproducibility Through Shared Data: Bridging Archival and Library Practice
Pakstis, J., Calkins, H., Dobrzynski, C., Lamm, S., & McNamara, L. (2019) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl.2019.00017 · Cited by 2 · see citing works
van Kampen, A. H. C., Mahamune, U., Jongejan, A., van Schaik, B. D. C., Balashova, D., Lashgari, D., Pras‐Raves, M. L., Wever, E. J. M., Dane, A. D., García-Valiente, R., et al. (2023) · doi:10.31222/osf.io/m9yqz · Cited by 2 · see citing works
ShinyCOPASI: a web-based exploratory interface for COPASI modelsPreprint
Gupta, A., & Mendes, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03796
Self-citationDynamic publication media with the COPASI R Connector (CoRC)
Daas, J. C. J., Förster, J. D., & Pahle, J. (2022) · Mathematical Biosciences · doi:10.1016/j.mbs.2022.108822
Mathematical Modelling of MASLD ‐ Towards Digital Twins in Liver Disease
Simonsson, C. (2024) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180757348
Greugny, E. T., Arsène, S., Ratto, N., Éthève, L., Gourlet, J.-B., Cogny, F., & Monteiro, C. (2024) · doi:10.70534/hsrc5460
García-Valiente, R., Langton, S. H., & van Kampen, A. H. C. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.16.724131
- Authors
- Mendes, P.
- Affiliations
- University of Connecticut (US)
- Corresponding author
- Mendes, Pedro
- ORCID
- Mendes, Pedro
- Venue
- Bulletin of Mathematical Biology
- Published
- 2018-09-06
- Volume
- 80
- Issue
- 12
- Pages
- 3081-3087
- ISSN
- 0092-8240 (ISSN-L)
- 1522-9602
- PMC
- PMC6234049
- OpenAlex
- W2805610236
- Cited by
- 20
- References
- 35
- Citations by year
- 2026: 2 · 2024: 4 · 2023: 4 · 2022: 5 · 2021: 3 · 2019: 1 · 2018: 1
- FWCI
- 1.77
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Bulletin of Mathematical Biology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Institute of General Medical Sciences (GM080219)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Keywords
- reproducible research, biomodels, model standardization, software repositories, public repositories, research reproducibility
- MeSH terms
- Animals, Computer Simulation, Humans, Models, Biological, Publishing, Reproducibility of Results, Databases, Factual, Computational Biology, Access to Information, Biomedical Research, Systems Biology, Mathematical Concepts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
