The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Reproducible Research in Computational Science
Open accessAbstract
Computational science has led to exciting new developments, but the nature of the work has exposed limitations in our ability to evaluate published findings. Reproducibility has the potential to serve as a minimum standard for judging scientific claims when full independent replication of a study is not possible.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The difficulty in auditing spreadsheets makes them generally incompatible with the principles of computational reproducibility [5].”
From Gene name errors: Lessons not learned (2021) · cites this paper as [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The rough consensus reduces to three ingredients.”
From Designing reproducible large-language-model-assisted scientific analyses (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredIn the citing paper
“We develop a two-step attribution process which combines a transparent and reproducible 39,40 machine learning approach to identifying studies on observed climate impacts with model-based assessments of detectable anthropogenic contributions to historical temperature and precipitation trends.”
From Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies (2021) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Repeatability of published microarray gene expression analyses
Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., et al. (2009) · Nature Genetics · doi:10.1038/ng.295 · Cited by 551
King, G. (1995) · PS Political Science & Politics · doi:10.1017/s1049096500057607 · Cited by 473
Publish your computer code: it is good enough
Barnes, N. (2010) · Nature · doi:10.1038/467753a · Cited by 317
Accessible Reproducible Research
Mesirov, J. P. (2010) · Science · doi:10.1126/science.1179653 · Cited by 311
Reproducible Epidemiologic Research
Peng, R. D., Dominici, F., & Zeger, S. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj093 · Cited by 292
Self-citationReproducible research and Biostatistics
Peng, R. D. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp014 · Cited by 245
Self-citationMaking scientific computations reproducible
Schwab, M., Karrenbach, N., & Claerbout, J. F. (2000) · Computing in Science & Engineering · doi:10.1109/5992.881708 · Cited by 232
Reproducible Research: Moving toward Research the Public Can Really Trust
Laine, C., Goodman, S. N., Griswold, M. E., & Sox, H. C. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-6-200703200-00154 · Cited by 213
Making Data Maximally Available
Hanson, B., Sugden, A. M., & Alberts, B. (2011) · Science · doi:10.1126/science.1203354 · Cited by 172
King, G. (1995) · PS Political Science & Politics · doi:10.2307/420301 · Cited by 145
Flagged first, then by citations
Research works printed but not linked: 1
- Yale Law School Roundtable on Data and Code Sharing, Comput. Sci. Eng. 12, 8 (2010).
Cited by
Sorted by citations
Items 101–200 of 1,003
Badham, J., Elsawah, S., Guillaume, J. H. A., Hamilton, S. H., Hunt, R. J., Jakeman, A. J., Pierce, S. A., Snow, V. O., Babbar‐Sebens, M., Fu, B., et al. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.02.013 · Cited by 135
Computing environments for reproducibility: Capturing the “Whole Tale”
Brinckman, A., Chard, K., Gaffney, N. I., Hategan, M., Jones, M. B., Kowalik, K., Kulasekaran, S., Ludäscher, B., Mecum, B., Nabrzyski, J., et al. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2017.12.029 · Cited by 151
FLEXI: A high order discontinuous Galerkin framework for hyperbolic–parabolic conservation laws
Krais, N., Beck, A., Bolemann, T., Frank, H. M., Flad, D., Gassner, G. J., Hindenlang, F., Hoffmann, M., Kuhn, T., Sonntag, M., et al. (2020) · Computers & Mathematics with Applications · doi:10.1016/j.camwa.2020.05.004 · Cited by 132
A brief history of forecasting competitions
Hyndman, R. J. (2019) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2019.03.015 · Cited by 131
Irreproducibility in hydrogen storage material research
Broom, D. P., & Hirscher, M. (2016) · Energy & Environmental Science · doi:10.1039/c6ee01435f · Cited by 130
Urgent need for consistent standards in functional enrichment analysis
Wijesooriya, K., Jadaan, S. A., Perera, K. L., Kaur, T., & Ziemann, M. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009935 · Cited by 130
Signer, J., & Balkenhol, N. (2015) · Wildlife Society Bulletin · doi:10.1002/wsb.539 · Cited by 142
MassIVE.quant: a community resource of quantitative mass spectrometry–based proteomics datasets
Choi, M., Carver, J., Chiva, C., Tzouros, M., Huang, T. C., Tsai, T., Pullman, B. S., Bernhardt, O. M., Hüttenhain, R., Teo, G. C., et al. (2020) · Nature Methods · doi:10.1038/s41592-020-0955-0 · Cited by 127
A guide to pre‐processing high‐throughput animal tracking data
Gupte, P. R., Beardsworth, C. E., Spiegel, O., Lourie, E., Toledo, S., Nathan, R., & Bijleveld, A. I. (2021) · Journal of Animal Ecology · doi:10.1111/1365-2656.13610 · Cited by 129
Particle tracking in drug and gene delivery research: State-of-the-art applications and methods
Schuster, B. S., Ensign, L. M., Allan, D., Suk, J. S., & Hanes, J. S. (2015) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2015.03.017 · Cited by 129
BEAPP: The Batch Electroencephalography Automated Processing Platform
Levin, A. R., Leal, A. S. M., Gabard-Durnam, L. J., & O’Leary, H. M. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00513 · Cited by 126
A call for bioimaging software usability
Carpenter, A. E., Kamentsky, L. D., & Eliceiri, K. W. (2012) · Nature Methods · doi:10.1038/nmeth.2073 · Cited by 126
BornAgain : software for simulating and fitting grazing-incidence small-angle scattering
Pospelov, G., Herck, W. V., Burle, J., Loaiza, J. M. C., Durniak, C., Fisher, J. M., Ganeva, M., Yurov, D., & Wuttke, J. (2020) · Journal of Applied Crystallography · doi:10.1107/s1600576719016789 · Cited by 123
ISMRM Raw data format: A proposed standard for MRI raw datasets
Inati, S., Naegele, J., Zwart, N. R., Roopchansingh, V., Lizak, M. J., Hansen, D. C., Liu, C., Atkinson, D., Kellman, P., Kozerke, S., et al. (2016) · Magnetic Resonance in Medicine · doi:10.1002/mrm.26089 · Cited by 120
What’s missing in geographical parsing?
Gritta, M., Pilehvar, M. T., Limsopatham, N., & Collier, N. (2017) · Language Resources and Evaluation · doi:10.1007/s10579-017-9385-8 · Cited by 120
Modelling soil erosion at European scale: towards harmonization and reproducibility
Bosco, C., de Rigo, D., Dewitte, O., Poesen, J., & Panagos, P. (2015) · Natural hazards and earth system sciences · doi:10.5194/nhess-15-225-2015 · Cited by 119
Genomics Virtual Laboratory: A Practical Bioinformatics Workbench for the Cloud
Afgan, E., Sloggett, C., Goonasekera, N., Makunin, I. V., Benson, D., Crowe, M. L., Gladman, S. L., Kowsar, Y., Pheasant, M., Horst, R., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0140829 · Cited by 115
Individual-Based Tracking Systems in Ornithology: Welcome to the Era of Big Data
López‐López, P. (2016) · Ardeola · doi:10.13157/arla.63.1.2016.rp5 · Cited by 117
A quantitative framework for the 3D characterization of the osteocyte lacunar system
Mader, K. S., Schneider, P., Müller, R., & Stampanoni, M. (2013) · Bone · doi:10.1016/j.bone.2013.06.026 · Cited by 115
How to make more published research true
Ioannidis, J. P. A. (2015) · Acimed · Cited by 115
Everything Matters: The ReproNim Perspective on Reproducible Neuroimaging
Kennedy, D. N., Abraham, S. A., Bates, J. F., Crowley, A., Ghosh, S., Gillespie, T., Goncalves, M., Grethe, J. S., Halchenko, Y. O., Hanke, M., et al. (2019) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2019.00001 · Cited by 113
Miłkowski, M., Hensel, W. M., & Hohol, M. (2018) · Journal of Computational Neuroscience · doi:10.1007/s10827-018-0702-z · Cited by 110
Requirements for a global data infrastructure in support of CMIP6
Balaji, V., Taylor, K. E., Juckes, M. N., Lawrence, B., Durack, P. J., Lautenschlager, M., Blanton, C., Cinquini, L., Denvil, S., Elkington, M., et al. (2018) · Geoscientific model development · doi:10.5194/gmd-11-3659-2018 · Cited by 121
Wagener, T., & Pianosi, F. (2019) · Earth-Science Reviews · doi:10.1016/j.earscirev.2019.04.006 · Cited by 110
Analyzing data citation practices using the data citation index
Robinson‐García, N., Jiménez‐Contreras, E., & Torres‐Salinas, D. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23529 · Cited by 126
Energy system modeling: Public transparency, scientific reproducibility, and open development
Morrison, R. (2018) · Energy Strategy Reviews · doi:10.1016/j.esr.2017.12.010 · Cited by 107
Reproducible research and GIScience: an evaluation using AGILE conference papers
Nüst, D., Granell, C., Hofer, B., Konkol, M., Ostermann, F., Šileryté, R., & Cerutti, V. (2018) · PeerJ · doi:10.7717/peerj.5072 · Cited by 122
Species distribution models can be highly sensitive to algorithm configuration
Hallgren, W., Santana, F. S., Low‐Choy, S., Zhao, Y., & Mackey, B. (2019) · Ecological Modelling · doi:10.1016/j.ecolmodel.2019.108719 · Cited by 113
PGP repository: a plant phenomics and genomics data publication infrastructure
Arend, D., Junker, A., Scholz, U., Schüler, D., Wylie, J., & Lange, M. (2016) · Database · doi:10.1093/database/baw033 · Cited by 106
Büttner, F., Howell, D. R., Ardern, C. L., Doherty, C., Blake, C. M., Ryan, J. M., Catena, R. D., Chou, L., Fino, P. C., Rochefort, C., et al. (2019) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-100164 · Cited by 106
Test set bias affects reproducibility of gene signatures
Patil, P., Bachant-Winner, P.-O., Haibe‐Kains, B., & Leek, J. T. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv157 · Cited by 102
DolphinNext: a distributed data processing platform for high throughput genomics
Yukselen, O., Türkyılmaz, O., Ozturk, A. R., Garber, M. G., & Küçükural, A. (2020) · BMC Genomics · doi:10.1186/s12864-020-6714-x · Cited by 99
Processing and Extraction of Seasonal Tree Physiological Parameters from Stem Radius Time Series
Knüsel, S., Peters, R. L., Haeni, M., Wilhelm, M., & Zweifel, R. (2021) · Forests · doi:10.3390/f12060765 · Cited by 103
AMT 1.x: A toolbox for reproducible research in auditory modeling
Majdak, P., Hollomey, C., & Baumgartner, R. (2022) · Acta Acustica · doi:10.1051/aacus/2022011 · Cited by 102
Integrating the discreteness and continuity of intonational categories
Grice, M., Roessig, S., Niemann, H., & Roettger, T. B. (2017) · Journal of Phonetics · doi:10.1016/j.wocn.2017.03.003 · Cited by 100
Krylov, A. I., Windus, T. L., Barnes, T., Marin‐Rimoldi, E., Nash, J. A., Pritchard, B. P., Smith, D. G. A., Altarawy, D., Saxe, P. W., Clementi, C., et al. (2018) · The Journal of Chemical Physics · doi:10.1063/1.5052551 · Cited by 100
Meyer, M., Lamers, D., Kayhan, E., Hunnius, S., & Oostenveld, R. (2021) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2021.101036 · Cited by 107
Nexus between in silico and in vivo models to enhance clinical translation of nanomedicine
Moradi Kashkooli, F., Soltani, M., Souri, M., Meaney, C., & Kohandel, M. (2021) · Nano Today · doi:10.1016/j.nantod.2020.101057 · Cited by 120
Easterbrook, S. (2014) · Nature Geoscience · doi:10.1038/ngeo2283 · Cited by 98
CRISPRdisco: An Automated Pipeline for the Discovery and Analysis of CRISPR-Cas Systems
Crawley, A. B., Henriksen, J. R., & Barrangou, R. (2018) · The CRISPR Journal · doi:10.1089/crispr.2017.0022 · Cited by 97
Progress toward openness, transparency, and reproducibility in cognitive neuroscience
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13325 · Cited by 95
Metaresearch for Evaluating Reproducibility in Ecology and Evolution
Fidler, F. M., Chee, Y. E., Wintle, B. C., Burgman, M. A., Mccarthy, M. A., & Gordon, A. (2016) · BioScience · doi:10.1093/biosci/biw159 · Cited by 106
Hu, K., Gaikwad, S. S., Hulsebos, M., Bakker, M. A., Zgraggen, E., Hidalgo, C. A., Kraska, T., Li, G., Satyanarayan, A., & Demiralp, Ç. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300892 · Cited by 98
Open Science in ArchaeologyPreprint
Marwick, B., Barton, C. M., Bates, L. A., Bollwerk, E., Bocinsky, R. K., Carter, A., Conrad, C., Costa, S., Crema, E. R., Davies, B., et al. (2017) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/72n8g · Cited by 99
Giannoulatou, E., Park, S. H., Humphreys, D. T., & Ho, J. W. K. (2014) · BMC Bioinformatics · doi:10.1186/1471-2105-15-s16-s15 · Cited by 94
MALONE, J. M., Brown, A. J., Lister, A., Ison, J., Hull, D., Parkinson, H., & Stevens, R. (2014) · Journal of Biomedical Semantics · doi:10.1186/2041-1480-5-25 · Cited by 90
Reproducibility and Replicability in Geographical Analysis
Kedron, P., Frazier, A. E., Trgovac, A. B., NELSON, T. A., & Fotheringham, A. S. (2019) · Geographical Analysis · doi:10.1111/gean.12221 · Cited by 105
Data-driven strategic planning of building energy retrofitting: the case of Stockholm
Pasichnyi, O., Levihn, F., Shahrokni, H., Wallin, J., & Kordas, O. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.05.373 · Cited by 92
The fundamental principles of reproducibility
Gundersen, O. E. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0210 · Cited by 121
Proceedings of the 15th Python in Science Conference
(2016) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-629e541a-014 · Cited by 94
Wade, T. L., Sericano, J. L., Sweet, S. T., Knap, A. H., & Guinasso, N. L. (2016) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2015.12.002 · Cited by 92
A statistical definition for reproducibility and replicabilityPreprint
Patil, P., Peng, R. D., & Leek, J. T. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/066803 · Cited by 90
Self-citationIwanaga, T., Wang, H., Hamilton, S. H., Grimm, V., Koralewski, T. E., Salado, A., Elsawah, S., Razavi, S., Yang, J., Glynn, P. D., et al. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104885 · Cited by 88
Elevating The Status of Code in Ecology
Mislan, K. A. S., Heer, J., & White, E. P. (2015) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2015.11.006 · Cited by 85
Integrated assessment models for ecologists: the present and the future
Harfoot, M., Tittensor, D. P., Newbold, T., McInerny, G., Smith, M. J., & Scharlemann, J. P. W. (2013) · Global Ecology and Biogeography · doi:10.1111/geb.12100 · Cited by 90
Dawid, H., Harting, P., van der Hoog, S., & Neugart, M. (2018) · Journal of Evolutionary Economics · doi:10.1007/s00191-018-0594-0 · Cited by 86
A Quick Guide to Software Licensing for the Scientist-Programmer
Morin, A., Urban, J. M., & Piotr Sliz, A. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002598 · Cited by 86
(De) Constructing Bias on Skin Lesion Datasets
Bissoto, A. E., Fornaciali, M. S., Junior, E. A. D. V., & Avila, S. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00335 · Cited by 89
Software Engineering for Computational Science: Past, Present, Future
Johanson, A., & Hasselbring, W. (2018) · Computing in Science & Engineering · doi:10.1109/mcse.2018.021651343 · Cited by 85
Statistical considerations in reporting cardiovascular research
Lindsey, M. L., Gray, G. A., Wood, S. K., & Curran‐Everett, D. (2018) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00309.2018 · Cited by 86
Tan, Y., Worley, R. R., Leonard, J. A., & Fisher, J. W. (2018) · Toxicological Sciences · doi:10.1093/toxsci/kfy010 · Cited by 85
Big Earth data analytics: a survey
Yang, C. P., Yu, M., Li, Y., Hu, F., Jiang, Y., Liu, Q., Sha, D., Xu, M., & Gu, J. (2019) · Big Earth Data · doi:10.1080/20964471.2019.1611175 · Cited by 87
Automated evolutionary approach for the design of composite machine learning pipelines
Nikitin, N. O., Vychuzhanin, P., Sarafanov, M., Polonskaia, I. S., Revin, I., Barabanova, I. V., Maximov, G., Kalyuzhnaya, A. V., & Boukhanovsky, A. V. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.08.022 · Cited by 96
Using R and Bioconductor for proteomics data analysis
Gatto, L., & Christoforou, A. (2013) · Biochimica et Biophysica Acta (BBA) - Proteins and Proteomics · doi:10.1016/j.bbapap.2013.04.032 · Cited by 84
PiGx: reproducible genomics analysis pipelines with GNU Guix
Wurmus, R., Uyar, B., Osberg, B., Franke, V., Blume, A., Wreczycka, K., Ronen, J., & Akalin, A. (2018) · GigaScience · doi:10.1093/gigascience/giy123 · Cited by 82
Big data, integrative omics and network biology
Tolani, P., Gupta, S., Yadav, K., Aggarwal, S., & Yadav, A. K. (2021) · Advances in protein chemistry and structural biology · doi:10.1016/bs.apcsb.2021.03.006 · Cited by 88
Investigating resting-state functional connectivity in the cervical spinal cord at 3 T
Eippert, F., Kong, Y., Winkler, A. M., Andersson, J., Finsterbusch, J., Büchel, C., Brooks, J. C. W., & Tracey, I. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.12.072 · Cited by 78
A Beginner's Guide to Conducting Reproducible Research
Alston, J. M., & Rick, J. A. (2021) · Bulletin of the Ecological Society of America · doi:10.1002/bes2.1801 · Cited by 90
The five pillars of computational reproducibility: bioinformatics and beyond
Ziemann, M., Poulain, P., & Bora, A. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad375 · Cited by 106
Nanoinformatics: a new area of research in nanomedicine
Maojo, V., Fritts, de la Iglesia, D., Cachau, Garcia-Remesal, Mitchell, & Kulikowski (2012) · International Journal of Nanomedicine · doi:10.2147/ijn.s24582 · Cited by 80
The empirical replicability of task-based fMRI as a function of sample size
Bossier, H., Roels, S. P., Seurinck, R., Banaschewski, T., Barker, G. J., Bokde, A. L. W., Quinlan, E. B., Desrivières, S., Flor, H., Grigis, A., et al. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116601 · Cited by 79
Analysis of Gene‐Gene Interactions
Cole, B. S., Hall, M. A., Urbanowicz, R. J., Gilbert‐Diamond, D., & Moore, J. H. (2017) · Current Protocols in Human Genetics · doi:10.1002/cphg.45 · Cited by 78
Lemmon, V., Ferguson, A. R., Popovich, P. G., Xu, X., Snow, D. M., Igarashi, M., Beattie, C. E., & Bixby, J. L. (2014) · Journal of Neurotrauma · doi:10.1089/neu.2014.3400 · Cited by 79
Baer, D. R., McGuire, G. E., Artyushkova, K., Easton, C. D., Engelhard, M., & Shard, A. G. (2021) · Journal of Vacuum Science & Technology A Vacuum Surfaces and Films · doi:10.1116/6.0000873 · Cited by 78
Reproducing Statistical Results
Stodden, V. (2015) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-010814-020127 · Cited by 77
Experimenting with reproducibility: a case study of robustness in bioinformatics
Kim, Y.-M., Poline, J., & Dumas, G. (2018) · GigaScience · doi:10.1093/gigascience/giy077 · Cited by 82
Creating and sharing reproducible research code the workflowr way
Blischak, J., Carbonetto, P. S., & Stephens, M. (2019) · F1000Research · doi:10.12688/f1000research.20843.1 · Cited by 76
Is Most Published Research Really False?
Leek, J. T., & Jager, L. R. (2016) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-060116-054104 · Cited by 77
Earth Observation Open Science: Enhancing Reproducible Science Using Data Cubes
Giuliani, G., Câmara, G., Killough, B., & Minchin, S. (2019) · Data · doi:10.3390/data4040147 · Cited by 76
Konkol, M., Kray, C., & Pfeiffer, M. (2018) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2018.1508687 · Cited by 80
Mechanical energy metamaterials in interstellar travel
Jiao, P. (2023) · Progress in Materials Science · doi:10.1016/j.pmatsci.2023.101132 · Cited by 76
Building Software, Building Community: Lessons from the rOpenSci Project
Boettiger, C., Chamberlain, S., Hart, E., & Ram, K. (2015) · Journal of Open Research Software · doi:10.5334/jors.bu · Cited by 74
Moulds, S., Buytaert, W., & Mijić, A. (2015) · Geoscientific model development · doi:10.5194/gmd-8-3215-2015 · Cited by 79
Transparency: The Emerging Third Dimension of Open Science and Open Data
Lyon, L. (2016) · LIBER Quarterly The Journal of the Association of European Research Libraries · doi:10.18352/lq.10113 · Cited by 82
Big Data for Policymaking: Great Expectations, but with Limited Progress?
Poel, M., Meyer, E. T., & Schroeder, R. (2018) · Policy & Internet · doi:10.1002/poi3.176 · Cited by 78
A longitudinal assessment of the persistence of twitter datasets
Zubiaga, A. (2018) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24026 · Cited by 81
Practical Reproducibility in Geography and Geosciences
Nüst, D., & Pebesma, E. J. (2020) · Annals of the American Association of Geographers · doi:10.1080/24694452.2020.1806028 · Cited by 85
Clinical Trials: Discerning Hype From Substance
Fleming, T. R. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-6-201009210-00008 · Cited by 72
Nyström, P., Falck‐Ytter, T., & Gredebäck, G. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0616-x · Cited by 71
Methods That Support the Validation of Agent-Based Models: An Overview and Discussion
Collins, A. J., Koehler, M., & Lynch, C. J. (2024) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.5258 · Cited by 70
Critical review of reporting of the data analysis step in metabolomics
Considine, E. C., Thomas, G. R., Boulesteix, A., Khashan, A. S., & Kenny, L. C. (2017) · Metabolomics · doi:10.1007/s11306-017-1299-3 · Cited by 73
The nlrx r package: A next‐generation framework for reproducible NetLogo model analyses
Salecker, J., Sciaini, M., Meyer, K., & Wiegand, K. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13286 · Cited by 69
Computational Reproducibility via Containers in Psychology
Clyburne-Sherin, A., Fei, X., & Green, S. A. (2019) · Meta-Psychology · doi:10.15626/mp.2018.892 · Cited by 69
Do machine learning platforms provide out-of-the-box reproducibility?
Gundersen, O. E., Shamsaliei, S., & Isdahl, R. J. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.06.014 · Cited by 84
BIBGT: combining bibliometrics and grounded theory to conduct a literature review
Walsh, I., & Rowe, F. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2022.2039563 · Cited by 82
Saquib, N., Saquib, J., & Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f4313 · Cited by 68
GERBIL – Benchmarking Named Entity Recognition and Linking consistently
Röder, M., Usbeck, R., & Ngomo, A.-C. N. (2017) · Semantic Web · doi:10.3233/sw-170286 · Cited by 68
Major, G. H., Pinder, J. W., Austin, D. E., Baer, D. R., Castle, S. L., Čechal, J., Clark, B., Cohen, H., Counsell, J. D. P., Herrera‐Gómez, A., et al. (2023) · Journal of Vacuum Science & Technology A Vacuum Surfaces and Films · doi:10.1116/6.0002437 · Cited by 67
Baer, D. R. (2018) · Frontiers in Chemistry · doi:10.3389/fchem.2018.00145 · Cited by 67
Understanding experiments and research practices for reproducibility: an exploratory study
Samuel, S., & König‐Ries, B. (2021) · PeerJ · doi:10.7717/peerj.11140 · Cited by 71
Metadata
- Authors
- Peng, R. D.
- Affiliations
- Johns Hopkins University (US)
- Corresponding author
- Peng, Roger D.
- ORCID
- Peng, Roger D.
- Venue
- Science
- Published
- 2011-12-02
- Volume
- 334
- Issue
- 6060
- Pages
- 1226-1227
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC3383002
- OpenAlex
- W2008985174
- Semantic Scholar
- e4625641d1072542315c14083befb43cd7096aba
- Cited by
- 1,449
- References
- 11
- Citations by year
- 2026: 162 · 2025: 75 · 2024: 63 · 2023: 74 · 2022: 85 · 2021: 121 · 2020: 139 · 2019: 162 · 2018: 138 · 2017: 113 · 2016: 106 · 2015: 82 · 2014: 51 · 2013: 46 · 2012: 29
- FWCI
- 98.43
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Environmental Health Sciences (R01ES019560)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- reproducible research, scientific computing, reproducibility, scientific claims, replication
- MeSH terms
- Computers, Publishing, Research, Reproducibility of Results, Computational Biology, Editorial Policies, Biomedical Research, Biostatistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
