The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Reproducible Research in Computational Science
Open accessAbstract
Computational science has led to exciting new developments, but the nature of the work has exposed limitations in our ability to evaluate published findings. Reproducibility has the potential to serve as a minimum standard for judging scientific claims when full independent replication of a study is not possible.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The difficulty in auditing spreadsheets makes them generally incompatible with the principles of computational reproducibility [5].”
From Gene name errors: Lessons not learned (2021) · cites this paper as [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The rough consensus reduces to three ingredients.”
From Designing reproducible large-language-model-assisted scientific analyses (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredIn the citing paper
“We develop a two-step attribution process which combines a transparent and reproducible 39,40 machine learning approach to identifying studies on observed climate impacts with model-based assessments of detectable anthropogenic contributions to historical temperature and precipitation trends.”
From Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies (2021) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Repeatability of published microarray gene expression analyses
Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., et al. (2009) · Nature Genetics · doi:10.1038/ng.295 · Cited by 551
King, G. (1995) · PS Political Science & Politics · doi:10.1017/s1049096500057607 · Cited by 473
Publish your computer code: it is good enough
Barnes, N. (2010) · Nature · doi:10.1038/467753a · Cited by 317
Accessible Reproducible Research
Mesirov, J. P. (2010) · Science · doi:10.1126/science.1179653 · Cited by 311
Reproducible Epidemiologic Research
Peng, R. D., Dominici, F., & Zeger, S. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj093 · Cited by 292
Self-citationReproducible research and Biostatistics
Peng, R. D. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp014 · Cited by 245
Self-citationMaking scientific computations reproducible
Schwab, M., Karrenbach, N., & Claerbout, J. F. (2000) · Computing in Science & Engineering · doi:10.1109/5992.881708 · Cited by 232
Reproducible Research: Moving toward Research the Public Can Really Trust
Laine, C., Goodman, S. N., Griswold, M. E., & Sox, H. C. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-6-200703200-00154 · Cited by 213
Making Data Maximally Available
Hanson, B., Sugden, A. M., & Alberts, B. (2011) · Science · doi:10.1126/science.1203354 · Cited by 172
King, G. (1995) · PS Political Science & Politics · doi:10.2307/420301 · Cited by 145
Research works printed but not linked: 1
- Yale Law School Roundtable on Data and Code Sharing, Comput. Sci. Eng. 12, 8 (2010).
Cited by
Sorted by citations
phyloseq: An R Package for Reproducible Interactive Analysis and Graphics of Microbiome Census Data
McMurdie, P. J., & Holmes, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0061217 · Cited by 21,942
Radiomics: the bridge between medical imaging and personalized medicine
Lambin, P., Leijenaar, R. T. H., Deist, T. M., Peerlings, J., de Jong, E. E. C., van Timmeren, J., Sanduleanu, S., Larue, R. T. H. M., Even, A. J. G., Jochems, A., et al. (2017) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2017.141 · Cited by 5,688
Waste Not, Want Not: Why Rarefying Microbiome Data Is Inadmissible
McMurdie, P. J., & Holmes, S. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003531 · Cited by 3,034
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035
An introduction to Docker for reproducible research
Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059
A standard protocol for reporting species distribution models
Zurell, D., Franklin, J., König, C., Bouchet, P. J., Dormann, C. F., Elith, J., Fandós, G., Feng, X., Guillera‐Arroita, G., Guisan, A., et al. (2020) · Ecography · doi:10.1111/ecog.04960 · Cited by 1,025
The conundrum of sharing research data
Borgman, C. L. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22634 · Cited by 1,013
Ten Simple Rules for Reproducible Computational Research
Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880
Model Transparency and Validation
Eddy, D. M., Hollingworth, W., Jaime, J., Tsevat, J., McDonald, K. M., & Wong, J. B. (2012) · Medical Decision Making · doi:10.1177/0272989x12454579 · Cited by 803
A perspective on using partial least squares structural equation modelling in data articles
Ringle, C. M., Sarstedt, M., Sinkovics, N., & Sinkovics, R. R. (2023) · Data in Brief · doi:10.1016/j.dib.2023.109074 · Cited by 975
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760
Best practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714
Building a Science of Individual Differences from fMRI
Dubois, J., & Adolphs, R. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.03.014 · Cited by 644
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609
NMRbox: A Resource for Biomolecular NMR Computation
Maciejewski, M. W., Schuyler, A. D., Gryk, M. R., Moraru, I. I., Romero, P. R., Ulrich, E. L., Eghbalnia, H. R., Livny, M., Delaglio, F., & Hoch, J. C. (2017) · Biophysical Journal · doi:10.1016/j.bpj.2017.03.011 · Cited by 520
pyGIMLi: An open-source library for modelling and inversion in geophysics
Rücker, C., Günther, T., & Wagner, F. M. (2017) · Computers & Geosciences · doi:10.1016/j.cageo.2017.07.011 · Cited by 568
Yaniv, Z., Lowekamp, B., Johnson, H. J., & Beare, R. J. (2017) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-017-0037-8 · Cited by 537
Moradi Kashkooli, F., Soltani, M., & Souri, M. (2020) · Journal of Controlled Release · doi:10.1016/j.jconrel.2020.08.012 · Cited by 514
Eddy, D. M., Hollingworth, W., Caro, J. J., Tsevat, J., McDonald, K. M., & Wong, J. B. (2012) · Value in Health · doi:10.1016/j.jval.2012.04.012 · Cited by 466
rfishbase: exploring, manipulating and visualizing FishBase data from R
Boettiger, C., Lang, D. T., & Wainwright, P. C. (2012) · Journal of Fish Biology · doi:10.1111/j.1095-8649.2012.03464.x · Cited by 444
O’Leary, S. J., Puritz, J. B., Willis, S. C., Hollenbeck, C. M., & Portnoy, D. S. (2018) · Molecular Ecology · doi:10.1111/mec.14792 · Cited by 436
Reproducibility vs. Replicability: A Brief History of a Confused Terminology
Pleßer, H. E. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2017.00076 · Cited by 426
Osaba, E., Villar-Rodríguez, E., Ser, J. D., Nebro, A. J., Molina, D., LaTorre, A., Suganthan, P. N., Coello, C. A. C., & Herrera, F. (2021) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2021.100888 · Cited by 436
Convolutional Neural Networks for the Automatic Identification of Plant Diseases
Boulent, J., Foucher, S., Théau, J., & St-Charles, P.-L. (2019) · Frontiers in Plant Science · doi:10.3389/fpls.2019.00941 · Cited by 434
Prabhakar, P. K., Vatsa, S., Srivastav, P. P., & Pathak, S. S. (2020) · Food Research International · doi:10.1016/j.foodres.2020.109157 · Cited by 373
An empirical analysis of journal policy effectiveness for computational reproducibility
Stodden, V., Seiler, J., & Ma, Z. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708290115 · Cited by 369
Evolution of Translational Omics: Lessons Learned and the Path Forward
Micheel, C., Nass, S. J., & Omenn, G. S. (2013) · Cited by 365
Tximeta: Reference sequence checksums for provenance identification in RNA-seq
Love, M. I., Soneson, C., Hickey, P. F., Johnson, L. K., Pierce‐Ward, N. T., Shepherd, L. A., Morgan, M. T., & Patro, R. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007664 · Cited by 361
Exploration and Explanation in Computational Notebooks
Rule, A., Tabard, A., & Hollan, J. D. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173606 · Cited by 347
Huber, S. P., Zoupanos, S., Uhrin, M., Talirz, L., Kahle, L., Häuselmann, R., Gresch, D., Müller, T., Yakutovich, A. V., Andersen, C. W., et al. (2020) · Scientific Data · doi:10.1038/s41597-020-00638-4 · Cited by 401
A review in radiomics: Making personalized medicine a reality via routine imaging
Guiot, J., Vaidyanathan, A., Deprez, L., Zerka, F., Danthine, D., Frix, A.-N., Lambin, P., Bottari, F., Tsoutzidis, N., Miraglio, B., et al. (2021) · Medicinal Research Reviews · doi:10.1002/med.21846 · Cited by 356
Inflammatory Bowel Disease as a Model for Translating the Microbiome
Huttenhower, C., Kostic, A. D., & Xavier, R. J. (2014) · Immunity · doi:10.1016/j.immuni.2014.05.013 · Cited by 332
Reproducible Research Practices and Transparency across the Biomedical Literature
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002333 · Cited by 310
State of the Art: Reproducibility in Artificial Intelligence
Gundersen, O. E., & Kjensmo, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11503 · Cited by 293
Prospects and challenges of multi-omics data integration in toxicology
Canzler, S., Schor, J., Busch, W., Schubert, K. A., Rolle-Kampczyk, U. E., Seitz, H., Kamp, H., von Bergen�, M., Buesen, R., & Hackermüller, J. (2020) · Archives of Toxicology · doi:10.1007/s00204-020-02656-y · Cited by 300
Gilbert, K. J., Andrew, R. L., Bock, D. G., Franklin, M. T., Kane, N. C., Moore, J., Moyers, B. T., Renaut, S., Rennison, D. J., Veen, T., et al. (2012) · Molecular Ecology · doi:10.1111/j.1365-294x.2012.05754.x · Cited by 290
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
Utilizing big data analytics for information systems research: challenges, promises and guidelines
Müller, O., Junglas, I. A., Brocke, J. V., & Debortoli, S. (2016) · European Journal of Information Systems · doi:10.1057/ejis.2016.2 · Cited by 282
OpenSim Moco: Musculoskeletal optimal control
Dembia, C. L., Bianco, N. A., Falisse, A. M. D., Hicks, J. L., & Delp, S. L. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008493 · Cited by 280
pcaExplorer: an R/Bioconductor package for interacting with RNA-seq principal components
Marini, F., & Binder, H. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2879-1 · Cited by 272
Pinder, J. W., Major, G. H., Baer, D. R., Terry, J., Whitten, J. E., Čechal, J., Crossman, J. D., Lizarbe, A. J., Jafari, S., Easton, C. D., et al. (2024) · Applied Surface Science Advances · doi:10.1016/j.apsadv.2023.100534 · Cited by 274
Patil, P., Peng, R. D., & Leek, J. T. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616646366 · Cited by 243
Self-citationPick, J. L., Nakagawa, S., & Noble, D. W. A. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13118 · Cited by 256
AquaCrop-OS: An open source version of FAO's crop water productivity model
Foster, T., Brozović, N., Butler, A. P., Neale, C. M. U., Raes, D., Steduto, P., Fereres, E., & Hsiao, T. C. (2016) · Agricultural Water Management · doi:10.1016/j.agwat.2016.11.015 · Cited by 252
Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies
Callaghan, M., Schleussner, C.-F., Nath, S., Lejeune, Q., Knutson, T. R., Reichstein, M., Hansen, G., Theokritoff, E., Andrijevic, M., Brecha, R. J., et al. (2021) · Nature Climate Change · doi:10.1038/s41558-021-01168-6 · Cited by 251
Marwick, B. (2017) · Journal of Archaeological Method and Theory · doi:10.1007/s10816-015-9272-9 · Cited by 179
Reporting guidelines for simulation‐based research in social sciences
Rahmandad, H., & Sterman, J. D. (2012) · System Dynamics Review · doi:10.1002/sdr.1481 · Cited by 258
Revisiting the serotonin–aggression relation in humans: A meta-analysis.
Duke, A. A., Bègue, L., Bell, R., & Eisenlohr-Moul, T. A. (2013) · Psychological Bulletin · doi:10.1037/a0031544 · Cited by 253
On the Reuse of Scientific Data
Pasquetto, I. V., Randles, B. M., & Borgman, C. L. (2017) · Data Science Journal · doi:10.5334/dsj-2017-008 · Cited by 295
Most computational hydrology is not reproducible, so is it really science?
Hutton, C., Wagener, T., Freer, J. E., Han, D., Duffy, C., & Arheimer, B. (2016) · Water Resources Research · doi:10.1002/2016wr019285 · Cited by 257
Git can facilitate greater reproducibility and increased transparency in science
Ram, K. (2013) · Source Code for Biology and Medicine · doi:10.1186/1751-0473-8-7 · Cited by 226
Performing Skin Microbiome Research: A Method to the Madness
Kong, H. H., Andersson, B., Clavel, T., Common, J., Jackson, S. A., Olson, N. D., Segre, J. A., & Traidl‐Hoffmann, C. (2017) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2016.10.033 · Cited by 232
Open science, reproducibility, and transparency in ecology
Powers, S. M., & Hampton, S. E. (2018) · Ecological Applications · doi:10.1002/eap.1822 · Cited by 242
Yang, P., Fang, H., & Lin, J. (2018) · Journal of Data and Information Quality · doi:10.1145/3239571 · Cited by 222
MultiDIC: An Open-Source Toolbox for Multi-View 3D Digital Image Correlation
Solav, D., Moerman, K. M., Jaeger, A. M., Genovese, K., & Herr, H. (2018) · IEEE Access · doi:10.1109/access.2018.2843725 · Cited by 221
THE MEANING OF FAILED REPLICATIONS: A REVIEW AND PROPOSAL
Clemens, M. A. (2015) · Journal of Economic Surveys · doi:10.1111/joes.12139 · Cited by 246
Reproducibility and replicability of rodent phenotyping in preclinical studies
Kafkafi, N., Agassi, J., Chesler, E. J., Crabbe, J. C., Crusio, W. E., Eilam, D., Gerlai, R., Golani, I., Gomez-Marin, A., Heller, R., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.01.003 · Cited by 215
Student success prediction in MOOCs
Gardner, J., & Brooks, C. (2018) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-018-9203-z · Cited by 210
Towards reproducible computational drug discovery
Schaduangrat, N., Lampa, S., Simeon, S., Gleeson, M. P., Spjuth, O., & Nantasenamat, C. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-0408-x · Cited by 208
Accidental, open and everywhere: Emerging data sources for the understanding of cities
Arribas‐Bel, D. (2013) · Applied Geography · doi:10.1016/j.apgeog.2013.09.012 · Cited by 201
van Atteveldt, W., & Peng, T. (2018) · Communication Methods and Measures · doi:10.1080/19312458.2018.1458084 · Cited by 198
Reproducible research can still be wrong: Adopting a prevention approach
Leek, J. T., & Peng, R. D. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1421412111 · Cited by 207
Self-citationMandated data archiving greatly improves access to research data
Vines, T., Andrew, R. L., Bock, D. G., Franklin, M. T., Gilbert, K. J., Kane, N. C., Moore, J., Moyers, B. T., Renaut, S., Rennison, D. J., et al. (2013) · The FASEB Journal · doi:10.1096/fj.12-218164 · Cited by 188
A survey of tools for the analysis of quantitative PCR (qPCR) data
Pabinger, S., Rödiger, S., Kriegner, A., Vierlinger, K., & Weinhäusel, A. (2014) · Biomolecular Detection and Quantification · doi:10.1016/j.bdq.2014.08.002 · Cited by 189
Usbeck, R., Röder, M., Ngomo, A.-C. N., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti, M., Cherix, D., Eickmann, B., et al. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741626 · Cited by 192
Shining Light into Black Boxes
Morin, A., Urban, J. M., Adams, P. D., Foster, I. T., Šali, A., Baker, D. A., & Piotr Sliz, A. (2012) · Science · doi:10.1126/science.1218263 · Cited by 190
Journal of Computational Neuroscience
(2013) · Journal of Computational Neuroscience · doi:10.1007/10827.1573-6873 · Cited by 192
Essential guidelines for computational method benchmarking
Weber, L. M., Saelens, W., Cannoodt, R., Soneson, C., Hapfelmeier, A., Gardner, P. P., Boulesteix, A.-L., Saeys, Y., & Robinson, M. D. (2019) · Genome Biology · doi:10.1186/s13059-019-1738-8 · Cited by 21
Vision dominates in perceptual language: English sensory vocabulary is optimized for usage
Winter, B., Perlman, M., & Majid, A. (2018) · Cognition · doi:10.1016/j.cognition.2018.05.008 · Cited by 198
High-Throughput Methods in the Discovery and Study of Biomaterials and Materiobiology
Yang, L., Pijuan‐Galito, S., Rho, H. S., Vasilevich, A. S., Eren, A. D., Ge, L., Habibović, P., Alexander, M. R., de Boer, J., Carlier, A., et al. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.0c00752 · Cited by 191
An estimate of the science-wise false discovery rate and application to the top medical literature
Jager, L. R., & Leek, J. T. (2014) · Biostatistics · doi:10.1093/biostatistics/kxt007 · Cited by 179
Punzo, V., Zheng, Z., & Montanino, M. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103165 · Cited by 175
Tools and techniques for computational reproducibility
Piccolo, S., & Frampton, M. (2016) · GigaScience · doi:10.1186/s13742-016-0135-4 · Cited by 179
Ten simple rules for writing and sharing computational analyses in Jupyter Notebooks
Rule, A., Birmingham, A., Zúñiga, C., Altıntaş, İ., Huang, S.-C., Knight, R. D., Moshiri, N., Nguyen, M. H., Rosenthal, S. B., Pérez, F., et al. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007007 · Cited by 169
Pasichnyi, O., Wallin, J., Levihn, F., Shahrokni, H., & Kordas, O. (2019) · Energy Policy · doi:10.1016/j.enpol.2018.11.051 · Cited by 168
A Review of Software for Spatial Econometrics in R
Bivand, R., Millo, G., & Piras, G. (2021) · Mathematics · doi:10.3390/math9111276 · Cited by 187
Shiny-phyloseq: Web application for interactive microbiome analysis with provenance tracking
McMurdie, P. J., & Holmes, S. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu616 · Cited by 166
Changing Trends in Computational Drug Repositioning
Yella, J., Yaddanapudi, S., Wang, Y., & Jegga, A. G. (2018) · Pharmaceuticals · doi:10.3390/ph11020057 · Cited by 171
Uhrin, M., Huber, S. P., Yu, J., Marzari, N., & Pizzi, G. (2020) · Computational Materials Science · doi:10.1016/j.commatsci.2020.110086 · Cited by 177
Winter, B. (2019) · Converging evidence in language and communication research · doi:10.1075/celcr.20 · Cited by 182
Wu, T.-C., Xu, K., Martínek, J., Young, R. R., Banchereau, R., George, J., Turner, J., Kim, K. I., Zurawski, S., Wang, X., et al. (2018) · Cancer Research · doi:10.1158/0008-5472.can-18-0413 · Cited by 168
Major, G. H., Avval, T. G., Moeini, B., Pinto, G., Shah, D., Jain, V., Carver, V., Skinner, W. M., Gengenbach, T. R., Easton, C. D., et al. (2020) · Journal of Vacuum Science & Technology A Vacuum Surfaces and Films · doi:10.1116/6.0000685 · Cited by 166
Stimfit: quantifying electrophysiological data with Python
Guzman, S. J., Schlögl, A., & Schmidt‐Hieber, C. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00016 · Cited by 161
A systematic survey on multi-step attack detection
Navarro, J., Deruyver, A., & Parrend, P. (2018) · Computers & Security · doi:10.1016/j.cose.2018.03.001 · Cited by 159
Using the Jupyter Notebook as a Tool for Open Science: An Empirical Study
Randles, B. M., Pasquetto, I. V., Golshan, M. S., & Borgman, C. L. (2017) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl.2017.7991618 · Cited by 155
Rapid, accurate, precise and reproducible ligand–protein binding free energy prediction
Wan, S., Bhati, A. P., Zasada, S. J., & Coveney, P. V. (2020) · Interface Focus · doi:10.1098/rsfs.2020.0007 · Cited by 158
A Review of Multi‐Compartment Infectious Disease Models
Tang, L., Zhou, Y., Wang, L., Purkayastha, S., Zhang, L., He, J., Wang, F., & Song, P. X. ‐. (2020) · International Statistical Review · doi:10.1111/insr.12402 · Cited by 156
Reproducibility of computational workflows is automated using continuous analysis
Beaulieu‐Jones, B. K., & Greene, C. S. (2017) · Nature Biotechnology · doi:10.1038/nbt.3780 · Cited by 151
Reproducibility in Scientific Computing
Ivie, P., & Thain, D. (2018) · ACM Computing Surveys · doi:10.1145/3186266 · Cited by 157
Rethomics: An R framework to analyse high-throughput behavioural data
Geissmann, Q., Rodríguez, L. G., Beckwith, E. J., & Gilestro, G. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0209331 · Cited by 143
CODATA-ICSTI Task Group on Data Citation Standards and Practices (2013) · Data Science Journal · doi:10.2481/dsj.osom13-043 · Cited by 148
Brokamp, C., Wolfe, C., Lingren, T., Harley, J. B., & Ryan, P. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx128 · Cited by 147
Exploring the impact of analysis software on task fMRI results
Bowring, A., Maumet, C., & Nichols, T. E. (2018) · bioRxiv · doi:10.1002/hbm.24603 · Cited by 166
Philosophy of science and the replicability crisis
Romero, F. (2019) · Philosophy Compass · doi:10.1111/phc3.12633 · Cited by 166
Current Challenges in Plant Eco-Metabolomics
Peters, K., Worrich, A., Weinhold, A., Alka, O., Balcke, G. U., Birkemeyer, C., Bruelheide, H., Calf, O. W., Dietz, S., Dührkop, K., et al. (2018) · International Journal of Molecular Sciences · doi:10.3390/ijms19051385 · Cited by 139
From characterising three years of HRI to methodology and reporting recommendations
Baxter, P. E., Kennedy, J., Senft, E., Lemaignan, S., & Belpaeme, T. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451777 · Cited by 149
Legality and Ethics of Web Scraping
Krotov, V., Johnson, L., & Silva, L. (2020) · Communications of the Association for Information Systems · doi:10.17705/1cais.04724 · Cited by 138
Rethinking Reproducibility as a Criterion for Research Quality
Leonelli, S. (2018) · Research in the history of economic thought and methodology · doi:10.1108/s0743-41542018000036b009 · Cited by 136
Terminologies for Reproducible ResearchPreprint
Barba, L. A. (2018) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2021.ii.2 · Cited by 251
Metadata
- Authors
- Peng, R. D.
- Affiliations
- Johns Hopkins University (US)
- Corresponding author
- Peng, Roger D.
- ORCID
- Peng, Roger D.
- Venue
- Science
- Published
- 2011-12-02
- Volume
- 334
- Issue
- 6060
- Pages
- 1226-1227
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC3383002
- OpenAlex
- W2008985174
- Semantic Scholar
- e4625641d1072542315c14083befb43cd7096aba
- Cited by
- 1,449
- References
- 11
- Citations by year
- 2026: 162 · 2025: 75 · 2024: 63 · 2023: 74 · 2022: 85 · 2021: 121 · 2020: 139 · 2019: 162 · 2018: 138 · 2017: 113 · 2016: 106 · 2015: 82 · 2014: 51 · 2013: 46 · 2012: 29
- FWCI
- 98.43
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Environmental Health Sciences (R01ES019560)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- reproducible research, scientific computing, reproducibility, scientific claims, replication
- MeSH terms
- Computers, Publishing, Research, Reproducibility of Results, Computational Biology, Editorial Policies, Biomedical Research, Biostatistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
