A descriptive analysis of the data availability statements accompanying medRxiv preprints and a comparison with their published counterparts
Open accessThe DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
Objective: To determine whether medRxiv data availability statements describe open or closed data—that is, whether the data used in the study is openly available without restriction—and to examine if this changes on publication based on journal data-sharing policy. Additionally, to examine whether data availability statements are sufficient to capture code availability declarations. Design: Observational study, following a pre-registered protocol, of preprints posted on the medRxiv repository between 25th June 2019 and 1st May 2020 and their published counterparts. Main outcome measures: Distribution of preprinted data availability statements across nine categories, determined by a prespecified classification system. Change in the percentage of data availability statements describing open data between the preprinted and published versions of the same record, stratified by journal sharing policy. Number of code availability declarations reported in the full-text preprint which were not captured in the corresponding data availability statement. Results: 3938 medRxiv preprints with an applicable data availability statement were included in our sample, of which 911 (23.1%) were categorized as describing open data. 379 (9.6%) preprints were subsequently published, and of these published articles, only 155 contained an applicable data availability statement. Similar to the preprint stage, a minority (59 (38.1%)) of these published data availability statements described open data. Of the 151 records eligible for the comparison between preprinted and published stages, 57 (37.7%) were published in journals which mandated open data sharing. Data availability statements more frequently described open data on publication when the journal mandated data sharing (open at preprint: 33.3%, open at publication: 61.4%) compared to when the journal did not mandate data sharing (open at preprint: 20.2%, open at publication: 22.3%). Conclusion: Requiring that authors submit a data availability statement is a good first step, but is insufficient to ensure data availability. Strict editorial policies that mandate data sharing (where appropriate) as a condition of publication appear to be effective in making research data available. We would strongly encourage all journal editors to examine whether their data availability policies are sufficiently stringent and consistently enforced.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Readers of articles from pre-print servers will need to be aware that a pre-print may not follow high standards in terms of clarity, quality of reporting or availability of data, due to a lack of quality-control from an established publisher [6,38].”
From A narrative review of recent tools and innovations toward automating living systematic reviews and evidence syntheses (2023) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Wickham, H. (2016) · Use R! · doi:10.1007/978-3-319-24277-4 · Cited by 33,115
Interrater reliability: The kappa statistic
McHugh, M. L. (2012) · Biochemia Medica · doi:10.11613/bm.2012.031 · Cited by 19,877
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
dplyr: A Grammar of Data Manipulation
Wickham, H., François, R., Henry, L., Müller, K., & Vaughan, D. (2014) · CRAN: Contributed Packages · doi:10.32614/cran.package.dplyr · Cited by 3,175
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · cited 2×
“While requiring data availability statements is an admirable first step for journals to take, and as such is viewed favorably by journal evaluation rubrics such as the Transparency and Openness Promotion [TOP] Guidelines [5], a lack of review of the contents of these statements often leads to issues.”
Backgroundpatchwork: The Composer of Plots
Pedersen, T. L. (2019) · CRAN: Contributed Packages · doi:10.32614/cran.package.patchwork · Cited by 818
stringr: Simple, Consistent Wrappers for Common String Operations
Wickham, H. (2009) · CRAN: Contributed Packages · doi:10.32614/cran.package.stringr · Cited by 618
RColorBrewer: ColorBrewer Palettes
Neuwirth, E. (2002) · CRAN: Contributed Packages · doi:10.32614/cran.package.rcolorbrewer · Cited by 573
Fraser, N., Brierley, L., Dey, G., Polka, J. K., Pálfy, M., Nanni, F., & Coates, J. A. (2021) · PLOS Biology · doi:10.1371/journal.pbio.3000959 · Cited by 555
irr: Various Coefficients of Interrater Reliability and Agreement
Matthias Gamer <m.gamer@uke.uni-hamburg.de>, J. L. <. C. A. (2005) · CRAN: Contributed Packages · doi:10.32614/cran.package.irr · Cited by 540
The Availability of Research Data Declines Rapidly with Article Age
Vines, T. H., Albert, A. Y. K., Andrew, R. L., Débarre, F., Bock, D. G., Franklin, M. T., Gilbert, K. J., Moore, J.-S., Renaut, S., & Rennison, D. J. (2014) · Current Biology · doi:10.1016/j.cub.2013.11.014 · Cited by 482
SARS‐CoV‐2 can be detected in urine, blood, anal swabs, and oropharyngeal swabs specimens
Peng, L., Liu, J., Xu, W., Luo, Q., Chen, D., Lei, Z., Huang, Z., Li, X., Deng, K., Lin, B., et al. (2020) · Journal of Medical Virology · doi:10.1002/jmv.25936 · Cited by 446
The citation advantage of linking publications to research data
Colavizza, G., Hrynaszkiewicz, I., Staden, I., Whitaker, K., & McGillivray, B. (2020) · PLOS ONE · doi:10.1371/journal.pone.0230416 · Cited by 335
Public Data Archiving in Ecology and Evolution: How Well Are We Doing?
Roche, D. G., Kruuk, L. E. B., Lanfear, R., & Binning, S. A. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002295 · Cited by 303
devtools: Tools to Make Developing R Packages Easier
Wickham, H., Hester, J., Chang, W., & Bryan, J. (2011) · CRAN: Contributed Packages · doi:10.32614/cran.package.devtools · Cited by 292
No raw data, no science: another possible source of the reproducibility crisis
Miyakawa, T. (2020) · Molecular Brain · doi:10.1186/s13041-020-0552-2 · Cited by 268
Sharing Clinical Trial Data: A Proposal From the International Committee of Medical Journal Editors
Taichman, D. B., Backus, J., Baethge, C., Bauchner, H., de Leeuw, P. W., Drazen, J. M., Fletcher, J., Frizelle, F. A., Groves, T., Haileamlak, A., et al. (2016) · JAMA · doi:10.1001/jama.2015.18164 · Cited by 254
Naudet, F., Sakarovitch, C., Janiaud, P., Cristea, I., Fanelli, D., Moher, D., & Ioannidis, J. P. A. (2018) · BMJ · doi:10.1136/bmj.k400 · Cited by 227
Data sharing in PLOS ONE: An analysis of Data Availability Statements
Federer, L. M., Belter, C. W., Joubert, D. J., Livinski, A., Lu, Y.-L., Snyders, L. N., & Thompson, H. (2018) · PLOS ONE · doi:10.1371/journal.pone.0194768 · Cited by 208
on behalf of the IICARus Collaboration, Hair, K., Macleod, M. R., & Sena, E. S. (2019) · Research Integrity and Peer Review · doi:10.1186/s41073-019-0069-3 · Cited by 174 · cited 1×
“Recent work has shown that “requiring” a data availability statement was effective in ensuring that this element was completed [4], while “encouraging” authors to follow a reporting checklist (the ARRIVE checklist) had no effect on compliance [50, 51].”
BackgroundReproducible and reusable research: are journal data sharing policies meeting the mark?
Vasilevsky, N. A., Minnier, J., Haendel, M. A., & Champieux, R. E. (2017) · PeerJ · doi:10.7717/peerj.3208 · Cited by 165 · cited 1×
“Of a sample of 318 biomedical journals, only ~20% had a data-sharing policy that mandated data sharing [10].”
BackgroundMüller, K., & Wickham, H. (2016) · CRAN: Contributed Packages · doi:10.32614/cran.package.tibble · Cited by 161
here: A Simpler Way to Find Your Files
Müller, K. (2017) · CRAN: Contributed Packages · doi:10.32614/cran.package.here · Cited by 159
Courbier, S., Dimond, R., & Bros-Facer, V. (2019) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-019-1123-4 · Cited by 141
Preprinting the COVID-19 pandemicPreprint
Fraser, N., Brierley, L., Dey, G. K., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.05.22.111294 · Cited by 120
Low availability of code in ecology: A call for urgent action
Culina, A., van den Berg, I., Evans, S., & Sánchez-Tójar, A. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000763 · Cited by 118
flextable: Functions for Tabular Reporting
Gohel, D., & Skintzos, P. (2017) · doi:10.32614/cran.package.flextable · Cited by 117
Toward standard practices for sharing computer code and programs in neuroscience
Eglen, S. J., Marwick, B., Halchenko, Y. O., Hanke, M., Sufi, S., Gleeson, P., Silver, R. A., Davison, A. P., Lanyon, L., Abrams, M., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4550 · Cited by 111
rvest: Easily Harvest (Scrape) Web Pages
Wickham, H. (2014) · doi:10.32614/cran.package.rvest · Cited by 109
Tools to Make Developing R Packages Easier [R package devtools version 2.3.2]
Wickham, H., Hester, J., Chang, W., & Bryan, J. (2020) · Cited by 107
Barry, A., Bradley, J., Stone, W. J., Guelbeogo, M. W., Lanke, K. H. W., Ouédraogo, A., Soulama, I., Nébié, I., Sermé, S. S., Grignard, L., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-22573-7 · Cited by 102
2019 Novel Coronavirus can be detected in urine, blood, anal swabs and oropharyngeal swabs samplesPreprint
Peng, L., Liu, J., Xu, W., et al. (2020) · medRxiv · doi:10.1101/2020.02.21.20026179 · Cited by 101
Novel delivery of cellular therapy to reduce ischemia reperfusion injury in kidney transplantation
Thompson, E. R., Bates, L., Ibrahim, I., Sewpaul, A., Stenberg, B., McNeill, A., Figueiredo, R., Girdlestone, T., Wilkins, G. C., Irwin, E. A., et al. (2020) · American Journal of Transplantation · doi:10.1111/ajt.16100 · Cited by 88
The jury is still out regarding the generality of adaptive ‘transgenerational’ effects
Sánchez‐Tójar, A., Lagisz, M., Moran, N. P., Nakagawa, S., Noble, D. W. A., & Reinhold, K. (2020) · Ecology Letters · doi:10.1111/ele.13479 · Cited by 82
Meat intake and cancer risk: prospective analyses in UK Biobank
Knuppel, A., Papier, K., Fensom, G. K., Appleby, P. N., Schmidt, J. A., Tong, T. Y. N., Travis, R. C., Key, T. J., & Perez-Cornago, A. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa142 · Cited by 80
Early clinical markers of aggressive multiple sclerosis
Malpas, C. B., Manouchehrinia, A., Sharmin, S., Roos, I., Horáková, D., Havrdová, E., Trojano, M., Izquierdo, G., Eichau, S., Bergamaschi, R., et al. (2020) · Brain · doi:10.1093/brain/awaa081 · Cited by 74
Krawczyk, M., & Reuben, E. (2012) · Accountability in Research · doi:10.1080/08989621.2012.678688 · Cited by 72
New preprint server for medical research
Rawlinson, C., & Bloom, T. (2019) · BMJ · doi:10.1136/bmj.l2301 · Cited by 66
Why researchers should share their analytic code
Goldacre, B., Morton, C. E., & DeVito, N. J. (2019) · BMJ · doi:10.1136/bmj.l6365 · Cited by 61
Simple, Consistent Wrappers for Common String Operations
Wickham, H. (2015) · Cited by 55
Hashmi, M., Taqi, A., Memon, M. I., Ali, S. M., Khaskheli, M. S., Sheharyar, M., Hayat, M., Shiekh, M., Kodippily, C., Dona, D. G., et al. (2020) · Journal of Critical Care · doi:10.1016/j.jcrc.2020.08.017 · Cited by 44
Why data sharing should be the expected norm
Krumholz, H. M. (2015) · BMJ · doi:10.1136/bmj.h599 · Cited by 37
grateful: Facilitate Citation of R Packages
Rodríguez‐Sánchez, F., & Jackson, C. (2023) · doi:10.32614/cran.package.grateful · Cited by 34
officer: Manipulation of Microsoft Word, PowerPoint and Excel Documents
Gohel, D., Moog, S., & Heckmann, M. (2017) · doi:10.32614/cran.package.officer · Cited by 32
Taichman, D. B., Backus, J., Baethge, C., Bauchner, H., Leeuw, P. W. D., Drazen, J. M., Fletcher, J., Frizelle, F. A., Groves, T., Haileamlak, A., et al. (2016) · BMJ · doi:10.1136/bmj.i255 · Cited by 29
medrxivr: Accessing and searching medRxiv and bioRxiv preprint data in R
McGuinness, L., & Schmidt, L. (2020) · Journal of Open Source Software · doi:10.21105/joss.02651 · Cited by 29
Self-citationData sharing in medical research
Packer, M. (2018) · BMJ · doi:10.1136/bmj.k510 · Cited by 27
Hayes, R., Ayles, H., Binka, F. N., Cowan, F. M., Kamali, A., Kapiga, S., Kleinschmidt, I., Mayaud, P., Patel, V., Smith, P. G., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)30682-1 · Cited by 26
rcrossref: Client for Various 'CrossRef' 'APIs'
Chamberlain, S., Zhu, H., Jahn, N., Boettiger, C., & Ram, K. (2014) · doi:10.32614/cran.package.rcrossref · Cited by 24
Solis, J. C. A., Storvoll, I., Vanbelle, S., & Melbye, H. (2020) · Scientific Reports · doi:10.1038/s41598-020-65354-w · Cited by 24
Martin, J., Hosking, G., Wadon, M. E., Agha, S. S., Langley, K., Rees, E., Owen, M. J., O’Donovan, M. C., Kirov, G., & Thapar, A. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-0821-y · Cited by 23
Ebbeling, C. B., Bielak, L., Lakin, P. R., Klein, G. L., Wong, J. M., Luoto, P. K., Wong, W. W., & Ludwig, D. S. (2020) · The Journal of Nutrition · doi:10.1093/jn/nxaa150 · Cited by 22
Du, X., Liu, Y., Chen, J., Peng, L. H., Jin, Y., Cheng, Z., Wang, H., Luo, M., Chen, L., & Zhao, Y. (2020) · medRxiv · doi:10.1101/2020.04.28.20082644 · Cited by 17
Meat intake and cancer risk: prospective analyses in UK BiobankPreprint
Knüppel, A., Papier, K., Fensom, G. K., et al. (2019) · medRxiv · doi:10.1101/19003822 · Cited by 14
A comparison of contemporary versus older studies of aspirin for primary prevention
Moriarty, F., & Ebell, M. H. (2019) · Family Practice · doi:10.1093/fampra/cmz080 · Cited by 12
Practical costs of data sharing
Goodhill, G. J. (2014) · Nature · doi:10.1038/509033b · Cited by 11
Client for Various 'CrossRef' 'APIs' [R package rcrossref version 1.1.0]
Chamberlain, S., Zhu, H., Jahn, N., Boettiger, C., & Ram, K. (2020) · Cited by 11
Early clinical markers of aggressive multiple sclerosisPreprint
Malpas, C. B., Manouchehrinia, A., Sharmin, S., et al. (2019) · medRxiv · doi:10.1101/19002063 · Cited by 7
Novel delivery of cellular therapy to reduce ischaemia reperfusion injury in kidney transplantationPreprint
Thompson, E. R., Bates, L., Ibrahim, I., et al. (2019) · medRxiv · doi:10.1101/19005546 · Cited by 7
Hashmi, M., Taqi, A., Memon, M. I., et al. (2020) · medRxiv · doi:10.1101/2020.04.22.20071555 · Cited by 7
Barry, A., Bradley, J. D., Stone, W. J., et al. (2020) · medRxiv · doi:10.1101/2020.04.08.20057927 · Cited by 7
Ehrlich, O. G., Testaverde, J., Heller, C. A., Daman, S., Anderson, A., & Higgins, P. (2018) · Gastroenterology · doi:10.1053/j.gastro.2017.11.058 · Cited by 7
Crohn’s disease and ulcerative colitis patient perspectives on clinical trials and participationPreprint
Ehrlich, O. G., Testaverde, J., Heller, C. A., et al. (2019) · medRxiv · doi:10.1101/19000273 · Cited by 5
Ebbeling, C. B., Bielak, L., Lakin, P. R., et al. (2019) · medRxiv · doi:10.1101/19001248 · Cited by 4
A Comparison of Contemporary versus Older Studies of Aspirin for Primary PreventionPreprint
Moriarty, F., & Ebell, M. H. (2019) · medRxiv · doi:10.1101/19004267 · Cited by 4
Meat intake and cancer risk: prospective analyses in UK Biobank
Knüppel, A., Papier, K., Appleby, P. N., Key, T. J., & Perez‐Cornago, A. (2020) · Proceedings of The Nutrition Society · doi:10.1017/s0029665120000634 · Cited by 4
Cost-Sensitive Machine Learning Classification for Mass Tuberculosis ScreeningPreprint
Septiandri, A. A., Aditiawarman, Tjiong, R., Burhan, E., & Shankar, A. H. (2019) · medRxiv · doi:10.1101/19000190 · Cited by 3
Martin, J., Hosking, G., Wadon, M. E., et al. (2019) · medRxiv · doi:10.1101/2019.12.12.19014555 · Cited by 3
Solis, J. C. A., Storvoll, I., Vanbelle, S., et al. (2019) · medRxiv · doi:10.1101/19005504 · Cited by 2
Tan, S. C., Flanagan, D., Morris, E., & Graf, C. (2020) · doi:10.22541/au.159422974.49069472 · Cited by 2
Research works printed but not linked: 2
- LA McGuinness, 2020, Protocol for a descriptive analysis of the data availability statements accompanying medRxiv preprints
- C Chan, 2018, Rio: A swiss-army knife for data file i/o
Cited by
Hamilton, D. G., Hong, K., Fraser, H., Rowhani-Farid, A., Fidler, F., & Page, M. J. (2023) · BMJ · doi:10.1136/bmj-2023-075767 · Cited by 86
Page, M. J., Nguyen, P., Hamilton, D. G., Haddaway, N., Kanukula, R., Moher, D., & McKenzie, J. E. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.03.003 · Cited by 59
Jiao, C., Li, K., & Fang, Z. (2022) · Journal of Information Science · doi:10.1177/01655515221101830 · Cited by 45
Anger, M., Wendelborn, C., Winkler, E., & Schickhardt, C. (2022) · PLoS ONE · doi:10.1371/journal.pone.0273259 · Cited by 30
Schmidt, L., Sinyor, M., Webb, R. T., Marshall, C., Knipe, D., Eyles, E. C., John, A., Gunnell, D., & Higgins, J. P. T. (2023) · Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2023.06.007 · Cited by 31
Metadata
- Authors
- McGuinness, L. A., & Sheppard, A. L.
- Affiliations
- University of Bristol (GB)
- University of Leicester (GB)
- Corresponding author
- McGuinness, Luke A.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2021-05-13
- Volume
- 16
- Issue
- 5
- Pages
- e0250887
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC8118451
- OpenAlex
- W3161322225
- Cited by
- 33
- References
- 65
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 2 · 2023: 12 · 2022: 9 · 2021: 2
- FWCI
- 5.34
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- MetArXiv (OSF Preprints) (publishedVersion)
- Explore Bristol Research (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- MetArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- National Institute for Health Research (NIHR; https://www.nihr.ac.uk/) Doctoral Research Fellowship (DRF-2018-11-ST2-048)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- data availability statements, medRxiv, preprints, open data, data sharing policy, research data availability, journal editorial policy, health research data
- MeSH terms
- Data Accuracy, Preprints as Topic, Humans, Peer Review, Research, Information Dissemination, Editorial Policies, Policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
