The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
An empirical analysis of journal policy effectiveness for computational reproducibility
Open accessAbstract
A key component of scientific communication is sufficient information for other researchers in the field to reproduce published findings. For computational and data-enabled research, this has often been interpreted to mean making available the raw data from which results were generated, the computer code that generated the findings, and any additional information needed such as workflows and input parameters. Many journals are revising author guidelines to include data and code availability. This work evaluates the effectiveness of journal policy that requires the data and code necessary for reproducibility be made available postpublication by the authors upon request. We assess the effectiveness of such a policy by ( i ) requesting data and code from authors and ( ii ) attempting replication of the published findings. We chose a random sample of 204 scientific papers published in the journal Science after the implementation of their policy in February 2011. We found that we were able to obtain artifacts from 44% of our sample and were able to reproduce the findings for 26%. We find this policy—author remission of data and code postpublication upon request—an improvement over no policy, but currently insufficient for reproducibility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Indeed, the reproducibility of published computational research has been reported as similarly dismal (~25%).4”
From Reproducibility of Quantitative Systems Pharmacology Models: Current Challenges and Future Opportunities (2019) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Reproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449
Repeatability of published microarray gene expression analyses
Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., et al. (2009) · Nature Genetics · doi:10.1038/ng.295 · Cited by 551
King, G. (1995) · PS Political Science & Politics · doi:10.1017/s1049096500057607 · Cited by 473
Enhancing reproducibility for computational methods
Stodden, V., Mcnutt, M. K., Bailey, D. H., Deelman, E., Gil, Y., Hanson, B., Heroux, M. A., Ioannidis, J. P. A., & Taufer, M. (2016) · Science · doi:10.1126/science.aah6168 · Cited by 464
Self-citationStatistical Analyses and Reproducible Research
Gentleman, R. C., & Lang, D. T. (2007) · Journal of Computational and Graphical Statistics · doi:10.1198/106186007x178663 · Cited by 364
Gilbert, K. J., Andrew, R. L., Bock, D. G., Franklin, M. T., Kane, N. C., Moore, J., Moyers, B. T., Renaut, S., Rennison, D. J., Veen, T., et al. (2012) · Molecular Ecology · doi:10.1111/j.1365-294x.2012.05754.x · Cited by 290
Reproducible Research in Computational Harmonic Analysis
Donoho, D. L., Maleki, A., Rahman, I. U., Shahram, M., & Stodden, V. (2008) · Computing in Science & Engineering · doi:10.1109/mcse.2009.15 · Cited by 274
Self-citationStodden, V., Guo, P., & Ma, Z. (2013) · PLoS ONE · doi:10.1371/journal.pone.0067111 · Cited by 257
Self-citationIs Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not'
Chang, A. C., & Li, P. (2015) · doi:10.2139/ssrn.2669564 · Cited by 217
Research works printed but not linked: 2
- ; American Association for the Advancement of Science Science journals: Editorial Policies. Available at www.sciencemag.org/authors/science-journals-editorial-policies. Accessed September 11 2011. (2011).
- V Stodden JM Borwein DH Bailey “Setting the default to reproducible’ in Computational Science Research. SIAM News . Available at https://sinews.siam.org/Aboutthe-Author/setting-the-default-to-reproducible-in-computational-science-research. Accessed February 16 2018. (2013).
Cited by
Sorted by citations
Items 201–300 of 369
Ten simple rules for training scientists to make better software
Gallagher, K., Creswell, R., Lambert, B. C., Robinson, M. J., Lei, C. L., Mirams, G. R., & Gavaghan, D. J. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012410 · Cited by 4
Reproducibility, replicability, robustness, and generalizability in corpus linguistics
Flanagan, J. (2025) · International Journal of Corpus Linguistics · doi:10.1075/ijcl.24113.fla · Cited by 6
Pelot, N. A., Wang, B., Marshall, D. P., Hussain, M. A., Musselman, E. D., Yu, G. J., Dale, J., Baumgart, I. W., Dardani, D., Zamani, P. T., et al. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/adb997 · Cited by 4
The case for openness in engineering research
Berg, D. R., & Niemeyer, K. E. (2018) · F1000Research · doi:10.12688/f1000research.14593.2 · Cited by 4
Open Models for Clinical Pharmacology
Kirouac, D. C. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1738 · Cited by 4
Stodden, V. (2020) · International Workshop on Practical Reproducible Evaluation of Computer Systems (P-RECS) · doi:10.1145/3391800.3398172 · Cited by 6
Self-citationScience Capsule: Towards Sharing and Reproducibility of Scientific Workflows
Ghoshal, D., Bianchi, L., Essiari, A., Paine, D., Poon, S. S., Beach, M. W., N’Diaye, A. T., Huck, P., & Ramakrishnan, L. (2021) · doi:10.1109/works54523.2021.00014 · Cited by 3
Parker, C., Nelson, E., & Zhang, T. (2022) · Entropy · doi:10.3390/e24121747 · Cited by 4
Reproducibility and FAIR principles: the case of a segment polarity network model
Mendes, P. (2023) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2023.1201673 · Cited by 5
Gabre, J. L., Merseburger, P., Claeys, A., Siaw, J. T., Bekaert, S.-L., Speleman, F., Hallberg, B., Palmer, R. H., & den Eynden, J. V. (2023) · NAR Cancer · doi:10.1093/narcan/zcad062 · Cited by 4
Zheng, J., Wang, T., Wang, H., Yan, B., Lai, J., Qiu, K., Zhou, X., Tan, J., Wang, S., Ji, H., et al. (2024) · Annals of Surgical Oncology · doi:10.1245/s10434-024-16519-8 · Cited by 4
Efficient Scientific Self-Correction in Times of Crisis
Nuijten, M. B. (2021) · Springer eBooks · doi:10.1007/978-3-030-65355-2_23 · Cited by 4
Interpreting and integrating big data in the life sciences
Mangul, S. (2019) · Emerging Topics in Life Sciences · doi:10.1042/etls20180175 · Cited by 4
The rapid proliferation of solicited content online journals: a quest to disseminate knowledge?
Becker, R. C., Cotarlan, V., & Sadayappan, S. (2019) · Journal of Thrombosis and Thrombolysis · doi:10.1007/s11239-019-01827-8 · Cited by 4
Samota, E. K., & Davey, R. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/581033 · Cited by 4
Huang, Y.-N., Jaiswal, P. V., Rajesh, A., Yadav, A., Yu, D., Liu, F., Scheg, G., Shih, E., Boldirev, G., Nakashidze, I., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.22.469640 · Cited by 4
A Roadmap for Transparent Research in Special Education and Related DisciplinesPreprint
Adelson, J. L., Barton, E. E., Bradshaw, C. P., Bryant, B. R., Bryant, D. P., Cook, B. G., Coyne, M. J., deBettencourt, L. U., DeHaven, A. C., Dymond, S. K., et al. (2019) · doi:10.35542/osf.io/sqfy3 · Cited by 4
Heterogeneity in effect size estimates: Empirical evidence and practical implicationsPreprint
Holzmeister, F., Johannesson, M., BOHM, R. M., Dreber, A., Huber, J., & Kirchler, M. (2024) · doi:10.31222/osf.io/583un · Cited by 4
Enhancing Data Integrity in Data-Driven Environmental Science Research through Blockchain Technology
Li, X., Huang, Y., Liu, Y., Yi, H., Li, L., Wu, Y., Zhang, X., Yang, X., Lin, Y., Lü, D., et al. (2025) · Environmental Science & Technology · doi:10.1021/acs.est.5c03461 · Cited by 3
Ferreira, J. V. A., de Almeida‐Rocha, J. M., Morante‐Filho, J. C., Storck‐Tonon, D., & Benchimol, M. (2025) · Sociobiology · doi:10.13102/sociobiology.v72i3.11410 · Cited by 3
Momeni, F., Khan, M. T., Kiesel, J., & Ross‐Hellauer, T. (2025) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3736731.3746161 · Cited by 7
Dacrema, M. F., Benigni, M., & Ferro, N. (2025) · ACM Transactions on Information Systems · doi:10.1145/3772275 · Cited by 3
From algorithms to systems: integrating computation into drug discovery
Bradley, A. R., Rossall, A., Pairaudeau, G., & Deane, C. M. (2025) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2025.2601102 · Cited by 4
Dudda, L. A., Kozula, M., Ross‐Hellauer, T., Kormann, E., Spijker, R., DeVito, N., Gopalakrishna, G., den Eynden, V. V., Onghena, P., Naudet, F., et al. (2024) · Open Research Europe · doi:10.12688/openreseurope.16567.2 · Cited by 4
Experiences with Reproducibility
Ghoshal, D., Paine, D., Junior, G. Z. P., Elbashandy, A., Gunter, D., Amusat, O., & Ramakrishnan, L. (2020) · International Workshop on Practical Reproducible Evaluation of Computer Systems (P-RECS) · doi:10.1145/3456287.3465478 · Cited by 5
Dudda, L. A., Kozula, M., Ross‐Hellauer, T., Kormann, E., Spijker, R., DeVito, N., Gopalakrishna, G., den Eynden, V. V., Onghena, P., Naudet, F., et al. (2023) · Open Research Europe · doi:10.12688/openreseurope.16567.1 · Cited by 3
Martin, D. L. (2024) · American Antiquity · doi:10.1017/aaq.2024.31 · Cited by 3
Eerola, T. (2024) · Musicae Scientiae · doi:10.1177/10298649241300885 · Cited by 5
eScience Infrastructures in Physical Chemistry
Kanza, S., Willoughby, C., Bird, C., & Frey, J. G. (2021) · Annual Review of Physical Chemistry · doi:10.1146/annurev-physchem-082120-041521 · Cited by 3
An Open Science Workflow for More Credible, Rigorous Research
Corker, K. S. (2022) · Cambridge University Press eBooks · doi:10.1017/9781108903264.012 · Cited by 3
Rclean: A Tool for Writing Cleaner, More Transparent Code
Lau, M. K., Pasquier, T., & Seltzer, M. (2020) · The Journal of Open Source Software · doi:10.21105/joss.01312 · Cited by 3
Reproducible validation and replication studies in nanoscale physics
Clementi, N. C., & Barba, L. A. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0068 · Cited by 3
Pixel: a content management platform for quantitative omics data
Denecker, T., Durand, W., Maupetit, J., Hébert, C., Camadro, J., Poulain, P., & Lelandais, G. (2019) · PeerJ · doi:10.7717/peerj.6623 · Cited by 3
Krieger, N. I., Perzynski, A. T., & Dalton, J. E. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212390 · Cited by 3
The Economics of Computational ReproducibilityPreprint
Colliard, J.-É., Hurlin, C., & Hurlin, C. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3418896 · Cited by 3
Reproducible Research and GIScience: an evaluation using GIScience conference papersPreprint
Ostermann, F., Nüst, D., Granell, C., Hofer, B., & Konkol, M. (2020) · doi:10.31223/x5zk5v · Cited by 3
Wood, J., & van Schalkwyk, I. (2025) · Journal of Transportation Technologies · doi:10.4236/jtts.2025.151010 · Cited by 2
Code sharing and reproducibility in survey-based social research: evidence from a large-scale audit
Krähmer, D., Schächtele, L., & Auspurg, K. (2026) · Royal Society Open Science · doi:10.1098/rsos.251997 · Cited by 3
Díaz, V. D., Belvedere, M., Depraetere, M., Holwerda, F. M., & Schwarz, D. (2026) · Palaeontology · doi:10.1111/pala.70052 · Cited by 2
Brodeur, A., Valenta, D., Marcoci, A., Aparicio, J. P., Mikola, D., Barbarioli, B., Alexander, R., Deer, L., Stafford, T., Vilhuber, L., et al. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2524747123 · Cited by 11
Turoman, N., Heyard, R., Schwab, S., Furrer, E., Vergauwe, E., & Held, L. (2023) · F1000Research · doi:10.12688/f1000research.129814.1 · Cited by 2
gcamreport: An R tool to process and standardize GCAM outputs
Rodés-Bachs, C., Sampedro, J., Horowitz, R., de Ven, D.-J. V., Cui, R. Y., Zhao, A., Zwerling, M., & Khan, Z. (2024) · The Journal of Open Source Software · doi:10.21105/joss.05975 · Cited by 2
Asymmetry in Galaxy Spin Directions: A Fully Reproducible Experiment Using HSC Data
Shamir, L. (2024) · Symmetry · doi:10.3390/sym16101389 · Cited by 2
A beginner's guide to supervised analysis for mass cytometry data in cancer biology
Wlosik, J., Granjeaud, S., Gorvel, L., Olive, D., & Chrétien, A.-S. (2024) · Cytometry Part A · doi:10.1002/cyto.a.24901 · Cited by 2
CompRep: A Dataset For Computational Reproducibility
Costa, L., Barbosa, S., & Cunha, J. (2025) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3736731.3746160 · Cited by 2
Open Science Practices in The Journal Human Factors : 2017–2022
McCarley, J. S., Rose, L. E., Fischer, S., Haney, M. S., Hyk, A., Kuhn, K., Meredith, B., Moussaoui, J. R., Andrade, F. M. G., Pietrok, E. M., et al. (2023) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/21695067231192276 · Cited by 3
Gallagher, R. V., Falster, D. S., Maitner, B., Salguero‐Gómez, R., Vandvik, V., Pearse, W. D., Schneider, F. D., Kattge, J., Poelen, J. H., Madin, J. S., et al. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-1167-9 · Cited by 2
Open science practices in clinical psychology journals: an audit studyPreprint
Cristea, I. A., Naudet, F., Nutu, D., & Gentili, C. (2018) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/t2k56 · Cited by 5
van Kampen, A. H. C., Mahamune, U., Jongejan, A., van Schaik, B. D. C., Balashova, D., Lashgari, D., Pras‐Raves, M. L., Wever, E. J. M., Dane, A. D., García-Valiente, R., et al. (2023) · doi:10.31222/osf.io/m9yqz · Cited by 2
Li, W., Liu, X., Zhang, Q., Shi, L., Zhang, J.-X., Zhang, X., Luan, J., Li, Y., Xu, T., Zhang, R., et al. (2026) · Accountability in Research · doi:10.1080/08989621.2025.2481943 · Cited by 1
A survey of experimental stimulus presentation code sharing in major areas of psychology
Rawal, A., Truong, V., Lo, Y., Tseng, L.-Y., & Duncan, N. W. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02390-8 · Cited by 1
The Observed Availability of Data and Code in Earth Science and Artificial Intelligence
Jones, E. A., McClung, B., Fawad, H., & McGovern, A. (2025) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-24-0147.1 · Cited by 1
It Works (only) on My Machine: A Study on Reproducibility Smells in Ansible Scripts
Sobhani, G., Haque, I., & Sharma, T. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00069 · Cited by 1
Hew, N. L. E., Myers, L. A., van de Walle, A., Shang, S., & Liu, Z. (2025) · Computational Materials Science · doi:10.1016/j.commatsci.2025.114072 · Cited by 1
Data policies in top social science journals: Findings and opportunities for academic libraries
Luo, J. (2025) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2025.103163 · Cited by 1
Unlocking the benefits of transparent and reusable science for climate risk management
Pollack, A. B., Auermuller, L., Burleyson, C., Campbell, J., Condon, M., Cooper, C. M., Coronese, M., Dangendorf, S., Doss‐Gollin, J., Hegde, P., et al. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2422157123 · Cited by 4
Identifying and classifying software mentions in full text scholarly documents
Pride, D., Guenci, M., Dočekal, M., Peroni, S., & Knoth (2025) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl67857.2025.00041 · Cited by 1
Reproducibility: How Can we Make Sure That Research Studies Can be Trusted?
Dudda, L. A., Verheyen, S., & Lindemann, O. (2026) · Frontiers for Young Minds · doi:10.3389/frym.2026.1582084 · Cited by 1
Pejdo, D., Buljan, I., & Marušić, A. (2026) · Research Integrity and Peer Review · doi:10.1186/s41073-026-00217-y · Cited by 1
Reproducibility of Health Claims in Meta-Analysis Studies of COVID Quarantine (Stay-at-Home) Orders
Young, S. S., & Kindzierski, W. B. (2022) · International Journal of Statistics and Probability · doi:10.5539/ijsp.v12n1p54 · Cited by 1
Kindzierski, W. B., Young, S. S., & Dunn, J. D. (2023) · International Journal of Statistics and Probability · doi:10.5539/ijsp.v12n3p40 · Cited by 1
Mapping the Repository Landscape: Harnessing Similarity with RepoSim and RepoSnipy
Li, Z., & Filgueira, R. (2023) · IEEE International Conference on eScience · doi:10.1109/e-science58273.2023.10254873 · Cited by 1
Olagunju, T., Makolo, A. U., & Gisel, A. (2024) · EMBnet journal · doi:10.14806/ej.29.0.1037 · Cited by 1
A Simulation Framework for Function as a Service
Manner, J. (2024) · Schriften aus der Fakultät Wirtschaftsinformatik und Angewandte Informatik der Otto-Friedrich-Universität Bamberg. · doi:10.20378/irb-92913 · Cited by 1
Generative AI for Trustworthy, Open, and Equitable Scholarship
Bourg, C., Kriegsman, S., Lindsay, N., Sardis, H., Stalberg, E., & Altman, M. (2024) · doi:10.21428/e4baedd9.567bfd15 · Cited by 1
On Emergent Limits to Knowledge—Or, How to Trust the Robot Researchers: A Pocket Guide
Stodden, V. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.dcaa63bc · Cited by 1
Self-citationCáceres-Tello, J., Galán-Hernández, J. J., Cevallos, M. B. M., & Menéses, E. L. (2025) · Applied Sciences · doi:10.3390/app152212183 · Cited by 1
A Web-Based Platform for Interactive Parameter Study of Large-Scale Lattice Gas Automata
Gorodnichev, M., & Medvedev, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-25636-4_25 · Cited by 1
Smith, G. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780192868459.003.0002 · Cited by 1
Young, S. S., & Kindzierski, W. B. (2023) · International Journal of Statistics and Probability · doi:10.5539/ijsp.v12n4p1 · Cited by 1
Transcriptome computational workbench (TCW): analysis of single and comparative transcriptomesPreprint
Soderlund, C. A. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/733311 · Cited by 1
Author response: Four erroneous beliefs thwarting more trustworthy research
Yarborough, M. A., Nadon, R., & Karlin, D. (2019) · eLife · doi:10.7554/elife.45261.003 · Cited by 1
Introduction to reproducible psychological researchPreprint
Kunisato, Y. (2019) · doi:10.31234/osf.io/x8js5 · Cited by 1
statacons: An SCons-Based Build Tool for StataPreprint
Guiteras, R. P., Kim, A., Quistorff, B., & Shumway, C. (2022) · doi:10.31222/osf.io/qesx6 · Cited by 1
Open Science in Developmental SciencePreprint
Gennetian, L. A., Frank, M. C., & Tamis‐LeMonda, C. S. (2022) · doi:10.31234/osf.io/uqjf9 · Cited by 1
Analysis reproducibility in mental health research: challenges and solutionsPreprint
Botvinik‐Nezer, R., & Wager, T. D. (2022) · doi:10.31219/osf.io/ujnmp · Cited by 1
Data Reuse in Agricultural Genomics Research: Present Challenges and Future SolutionsPreprint
Hafner, A., DeLeo, V. L., Deng, C. H., Elsik, C. G., Fleming, D. S., Harrison, P. W., Kalbfleisch, T., Petry, B., Pucker, B., Quezada-Rodríguez, E. H., et al. (2024) · Preprints.org · doi:10.20944/preprints202401.0780.v1 · Cited by 1
Taştan, C., & Aydin, B. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.10.698833 · Cited by 1
CODE beyond FAIR: a roadmap for reusable research software
Cosmo, R. D., Granger, S., Hinsen, K., Jullien, N., Berre, D. L., Louvet, V., Maumet, C., Maurice, C., Monat, R., & Rougier, N. P. (2026) · Scientific Data · doi:10.1038/s41597-026-06705-6 · Cited by 1
Cwik, J. C., Cludius, B., Bentz, D., Riehle, M., Ehrenthal, J. C., Haberkamp, A., Clamor, A., Blackwell, S. E., Kämpf, M. S., Nazarenus, E., et al. (2026) · Journal of Clinical Psychology · doi:10.1002/jclp.70162 · Cited by 1
ClaroAI-Bench: Evaluating Agentic Scientific Reproducibility on Real Biomedical Papers
O’Connell, K. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.08.723611
“…imperfect, but this level of spend is modest relative to prior multi-year human-led replication efforts in cancer biology and to survey evidence that computational reproduction routinely requires substantial manual debugging and data acquisition effort (Errington et al., 2021; Stodden et al., 2018).”
The Brazilian Reproducibility Initiative: a systematic assessment of Brazilian biomedical sciencePreprint
Amaral, O. B., Neves, K., Wasilewska‐Sampaio, A. P., & Carneiro, C. F. D. (2018) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ahf7t
Metajelo: a Metadata Package for Journals to Support External Linked Objects
Vilhuber, L., & Lagoze, C. (2021) · International Journal of Digital Curation · doi:10.2218/ijdc.v16i1.600
From the Beginning to the Future
Añel, J. A., Montes, D. P., & Iglesias, J. R. (2020) · Springer eBooks · doi:10.1007/978-3-030-41784-0_3
Preregistration, Data Sharing, and Other Salutary Behaviors
Bausell, R. B. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780197536537.003.0011
Corpora as Evolving Entities: Embedding Corpora in Biomedical Ontologies.
Hobbs, E., Goralski, S. M., Mitchell, A., Simpson, A., Leka, D., Kotey, E., Sekira, M., Munro, J. B., Nadendla, S., Jackson, R., et al. (2020)
Buttenfield, B. P., Stanislawski, L. V., Kronenfeld, B. J., & Shavers, E. J. (2021) · Springer eBooks · doi:10.1007/978-3-030-55462-0_23
An FSKX compliant source attribution model for salmonellosis and a look at its major hidden pitfalls
Sundermann, E. M., Carreira, G. C., & Käsbohrer, A. (2021) · Food Modelling Journal · doi:10.3897/fmj.2.70008
Faculty Opinions recommendation of Systematic benchmarking of omics computational tools.
Thompson, J. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.735401949.793560098
Almgren, H., Parsons, N., Bossche, S. V. D., & Marinazzo, D. (2020) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/9hg3j
Towards reproducible, transparent, and systematic benchmarking of omics computational toolsPreprint
Mangul, S., Martin, L. S., Hill, B., Lam, A. K.-M., DISTLER, M., Zelikovsky, A., Eskin, E., & Flint, J. (2018) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/p8yd9
Turoman, N., Heyard, R., Schwab, S., Furrer, E., Vergauwe, E., & Held, L. (2022) · doi:10.31222/osf.io/nb928
Metadata
- Authors
- Stodden, V., Seiler, J., & Ma, Z.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Columbia University (US)
- Corresponding author
- Stodden, Victoria
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2018-03-12
- Volume
- 115
- Issue
- 11
- Pages
- 2584-2589
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC5856507
- OpenAlex
- W2793993424
- Cited by
- 369
- References
- 26
- Citations by year
- 2026: 45 · 2025: 34 · 2024: 36 · 2023: 53 · 2022: 40 · 2021: 50 · 2020: 36 · 2019: 48 · 2018: 26
- FWCI
- 64.95
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Alfred P Sloan Foundation (PG004545)
- National Science Foundation (1153384)
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- computational reproducibility, editorial policy, data availability, code availability, replication, research artifacts, scientific journals, author guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
