Formalistic data and code availability policy in high-profile medical journals and pervasive policy-practice gaps in published articles: A meta-research study
Abstract
Background Poor data and code (DAC) sharing undermines open science principles. This study evaluates the stringency of DAC availability policies in high-profile medical journals and identifies policy-practice gaps (PPG) in published articles.Methods DAC availability policies of 931 Q1 medical journals (Clarivate JCR 2021) were evaluated, with PPGs quantified across 3,191 articles from The BMJ, JAMA, NEJM, and The Lancet.Results Only 9.1% (85/931) of journals mandated DAC sharing and availability statements, with 70.6% of these lacking mechanisms to verify authenticity, and 61.2% allowing publication despite invalid sharing. Secondary analysis revealed a disproportionate distribution of policies across subspecialties, with 18.6% (11/59) of subspecialties having >20% journals with mandated policies. Journal impact factors exhibited positive correlations with the stringency of availability statement policies (ρ = 0.20, p < 0.001) but not with sharing policies (ρ = 0.01, p = 0.737). Among the 3,191 articles, PPGs were observed in over 90% of cases. Specifically, 33.7% lacked DAC availability statements, 23.3% refused sharing (58.4% of which without justification in public statements), and 13.5% declared public sharing, with 39.0% being unreachable. Finally, only 0.5% achieved full computational reproducibility.Conclusions Formalistic policies and prevalent PPGs undermine DAC transparency, necessitating a supportive publication ecosystem that empowers authors to uphold scientific responsibility and integrity.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423 · cited 4× · see citing works
“In analyzing paper practices, the DANCE framework was developed through the systematic integration and structuring of the TOP guidelines and the FAIR principles (Nosek et al. 2015; Wilkinson et al. 2016).”
InfluentialSelf-citationA manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · see citing works
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · see citing works
Dolutegravir plus two different prodrugs of tenofovir to treat HIV
Venter, W. D. F., Moorhouse, M., Sokhela, S., Fairlie, L., Mashabane, N., Masenya, M., Serenata, C., Akpomiemie, G., Qavi, A., Chandiwana, N., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1902824 · Cited by 723 · see citing works
Rivaroxaban for Thromboprophylaxis in High-Risk Ambulatory Patients with Cancer
Khorana, A. A., Soff, G. A., Kakkar, A. K., Vadhan‐Raj, S., Riess, H. B., Wun, T., Streiff, M. B., Garcia, D. A., Liebman, H. A., Belani, C. P., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1814630 · Cited by 721 · see citing works
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., et al. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002456 · Cited by 700 · see citing works
(2021) · doi:10.54677/mnmh8546 · Cited by 448 · see citing works
Data sharing practices and data availability upon request differ across scientific disciplines
Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä., Pedaste, M., Raju, M., Сергеевна, А. А., Lukner, H., et al. (2021) · Scientific Data · doi:10.1038/s41597-021-00981-0 · Cited by 397 · see citing works
An empirical analysis of journal policy effectiveness for computational reproducibility
Stodden, V., Seiler, J., & Ma, Z. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708290115 · Cited by 369 · see citing works
Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., et al. (2018) · Royal Society Open Science · doi:10.1098/rsos.180448 · Cited by 336 · see citing works
Gabelica, M., Bojčić, R., & Puljak, L. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.05.019 · Cited by 253 · see citing works
Naudet, F., Sakarovitch, C., Janiaud, P., Cristea, I., Fanelli, D., Moher, D., & Ioannidis, J. P. A. (2018) · BMJ · doi:10.1136/bmj.k400 · Cited by 227 · see citing works
The science of sharing and the sharing of science
Milkman, K. L., & Berger, J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1317511111 · Cited by 154 · see citing works
An Analysis of Altmetrics in Emergency Medicine
Barbic, D., Tubman, M., Lam, H., & Barbic, S. P. (2016) · Academic Emergency Medicine · doi:10.1111/acem.12898 · Cited by 131 · see citing works
Best Practices for Using AI Tools as an Author, Peer Reviewer, or Editor
Leung, T. I., de Azevedo Cardoso, T., Mavragani, A., & Eysenbach, G. (2023) · Journal of Medical Internet Research · doi:10.2196/51584 · Cited by 127 · see citing works
Open data sharing and the Global South—Who benefits?
Serwadda, D., Ndebele, P., Grabowski, M. K., et al. (2018) · Science · doi:10.1126/science.aap8395 · Cited by 118 · see citing works
Data Sharing Statements for Clinical Trials
Taichman, D. B., Sahni, P., Pinborg, A. B., Peiperl, L., Laine, C. M., James, A., Hong, S., Haileamlak, A., Gollogly, L., Godlee, F., et al. (2017) · JAMA · doi:10.1001/jama.2017.6514 · Cited by 113 · see citing works
Improving data access democratizes and diversifies science
Nagaraj, A., Shears, E., & de Vaan, M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2001682117 · Cited by 111 · see citing works
Biomedical Data Sharing and Reuse: Attitudes and Practices of Clinical and Scientific Research Staff
Federer, L., Lu, Y., Joubert, D., Welsh, J. A., & Brandys, B. (2015) · PLoS ONE · doi:10.1371/journal.pone.0129506 · Cited by 105 · see citing works
Hamilton, D. G., Hong, K., Fraser, H., Rowhani-Farid, A., Fidler, F., & Page, M. J. (2023) · BMJ · doi:10.1136/bmj-2023-075767 · Cited by 86 · see citing works
Data management and sharing policy: the first step towards promoting data sharing
Waithira, N., Mutinda, B., & Cheah, P. Y. (2019) · BMC Medicine · doi:10.1186/s12916-019-1315-8 · Cited by 68 · see citing works
Covid-19 vaccines and treatments: we must have raw data, now
Doshi, P., Godlee, F., & Abbasi, K. (2022) · BMJ · doi:10.1136/bmj.o102 · Cited by 63 · see citing works
Media Coverage of Medical Journals: Do the Best Articles Make the News?
Selvaraj, S., Borkar, D. S., & Prasad, V. K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0085355 · Cited by 61 · see citing works
Page, M. J., Nguyen, P., Hamilton, D. G., Haddaway, N., Kanukula, R., Moher, D., & McKenzie, J. E. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.03.003 · Cited by 54 · see citing works
Tan, A. C., Askie, L. M., Hunter, K. E., Barba, A., Simes, R. J., & Seidler, A. L. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1500 · Cited by 48 · see citing works
ODDPub – a Text-Mining Algorithm to Detect Data Sharing in Biomedical Publications
Riedel, N., Kip, M., & Bobrov, E. (2020) · Data Science Journal · doi:10.5334/dsj-2020-042 · Cited by 44 · see citing works
Bergeat, D., Lombard, N., Gasmi, A., Le Floch, B., & Naudet, F. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.15209 · Cited by 41 · see citing works
Offline: Data sharing—why editors may have got it wrong
Horton, R. (2016) · The Lancet · doi:10.1016/s0140-6736(16)31638-5 · Cited by 40 · see citing works
TOP 2025: An Update to the Transparency and Openness Promotion GuidelinesPreprint
Grant, S., Corker, K. S., Mellor, D. T., et al. (2025) · doi:10.31222/osf.io/nmfs6_v2 · Cited by 27 · see citing works
Naaman, K., Grant, S., Kianersi, S., Supplee, L. H., Henschel, B., & Mayo-Wilson, E. (2023) · Royal Society Open Science · doi:10.1098/rsos.221093 · Cited by 25 · see citing works
Tannenbaum, S., Ross, J. S., Krumholz, H. M., Desai, N. R., Ritchie, J. D., Lehman, R., Gamble, G. M., Bachand, J., Schroter, S., Groves, T., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0723 · Cited by 24 · see citing works
Mandatory data and code sharing for research published by <i>The BMJ</i>
Loder, E., Macdonald, H., Bloom, T., & Abbasi, K. (2024) · BMJ · doi:10.1136/bmj.q324 · Cited by 24 · see citing works
Forero, D. A., Curioso, W. H., & Patrinos, G. P. (2021) · BMC Research Notes · doi:10.1186/s13104-021-05817-z · Cited by 23 · see citing works
Morehouse, K. N., Kurdi, B., & Nosek, B. A. (2024) · American Psychologist · doi:10.1037/amp0001346 · Cited by 22 · see citing works
A commitment to act on data sharing
Abbasi, K. A. (2023) · BMJ · doi:10.1136/bmj.p1609 · Cited by 12 · see citing works
Jean-Quartier, C., Stryeck, S., Thien, A., Vrella, B., Kleinschuster, J., Spreitzer, E., Wali, M., Mueller, H., Holzinger, A., & Jeanquartier, F. (2024) · Digital Health · doi:10.1177/20552076241271769 · Cited by 12 · see citing works
Johnson, A. L., Anderson, J. M., Bouvette, M., Pinero, I., Rauh, S., Johnson, B., Kee, M., Heigle, B., Tricco, A. C., Page, M. J., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.11.009 · Cited by 11 · cited 1× · see citing works
“…data-code-sharing practices – is imperative (Milkman and Berger 2014Nagaraj, Shears, and de Vaan 2020; Roche 2017), which can be achieved by optimizing and unifying the DAC sharing policies within the above publishing ecosystem, such as standardizing DAC sharing requirements (Johnson et al. 2023).”
Kianersi, S., Grant, S., Naaman, K., Henschel, B., Mellor, D. T., Apte, S., Deyoe, J. E., Eze, P., Huo, C., Lavender, B. L., et al. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221149735 · Cited by 11 · see citing works
Self-citationCalnan, M., Kirchin, S., Roberts, D. L., Wass, M. N., & Michaelis, M. (2023) · Pharmacological Research · doi:10.1016/j.phrs.2023.107043 · Cited by 11 · see citing works
Lombard, N., Gasmi, A., Sulpice, L., Boudjéma, K., Naudet, F., & Bergeat, D. (2020) · Trials · doi:10.1186/s13063-020-04756-7 · Cited by 10 · see citing works
Hamilton, D. G., Fraser, H. S., Fidler, F. M., McDonald, S. J., Rowhani-Farid, A., Hong, K., & Page, M. J. (2021) · F1000Research · doi:10.12688/f1000research.53874.2 · Cited by 10 · see citing works
Journal requirement for data sharing statements in clinical trials: a cross-sectional study
Zhang, J., Liu, Y., Thabane, L., Li, J., Bai, X., Li, L., Lip, G. Y. H., Sun, X., Xia, M., Spall, H. G. C. V., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111405 · Cited by 10 · see citing works
Self-citationData sharing as the foundation of discovery: ADNI and breakthroughs in Alzheimer's disease
Travaglia, A., & Hoffmann, S. (2024) · Alzheimer s & Dementia · doi:10.1002/alz.14129 · Cited by 10 · see citing works
Evaluating Science's open-data policy
Roche, D. G. (2017) · Science · doi:10.1126/science.aan8158 · Cited by 9 · see citing works
Publishing in the Open Access and Open Science era
Arita, M., Pulverer, B., Uemura, T., et al. (2024) · Genes to Cells · doi:10.1111/gtc.13100 · Cited by 7 · see citing works
Xu, Z., Liu, L., & Meng, Z. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e35034 · Cited by 7 · see citing works
Data Sharing: An Analysis of Medical Faculty Journals and Articles
Wiley, C. (2020) · Science & Technology Libraries · doi:10.1080/0194262x.2020.1781740 · Cited by 4 · see citing works
Koivisto, E., & Mäntylä, E. (2024) · Ecology and Evolution · doi:10.1002/ece3.11698 · Cited by 4 · see citing works
Deeb, H., Zieliński, T., & Millar, A. J. (2024) · BMJ Mental Health · doi:10.1136/bmjment-2024-301333 · Cited by 4 · see citing works
Evaluating Data-Sharing Policies and Author Compliance in Leading Orthopaedic Journals
Major, J., Wild, K., Elfar, A., Dennis, B., Oldham, E. P., Hagood, A., Paul, E., Checketts, J. X., Ford, A. I., & Vassar, M. B. (2025) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.24.00955 · Cited by 4 · see citing works
Investigating the practices and preferences of health scholars in sharing open research data
Wibowo, M. P., & Mon, L. M. (2025) · PLoS ONE · doi:10.1371/journal.pone.0313644 · Cited by 4 · see citing works
Maxwell, L., Shreedhar, P., & Krishnan, A. (2025) · Open Research Europe · doi:10.12688/openreseurope.16063.2 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Still being checked.
- Authors
- Li, W., Liu, X., Zhang, Q., Shi, L., Zhang, J.-X., Zhang, X., Luan, J., Li, Y., Xu, T., Zhang, R., Han, X., Lei, J., Wang, X., Wang, Y., Lan, H., Chen, X., Wu, Y., Wu, Y., Xia, L., Liao, H., Shen, C., Yu, Y., Xu, X., Deng, C., Liu, P., Feng, Z., Huang, C.-J., & Chen, Z.
- Affiliations
- Army Medical University (CN)
- Army Medical University
- Southwest University (CN)
- Sichuan Normal University (CN)
- Chengdu University of Traditional Chinese Medicine (CN)
- Corresponding author
- Chen, Zhiyi
- ORCID
- Venue
- Accountability in Research
- Publisher
- Informa UK Limited
- Published
- 2025-03-25
- Volume
- 33
- Issue
- 2
- Pages
- 2481943-2481943
- ISSN
- 0898-9621 (ISSN-L)
- 1545-5815
- OpenAlex
- W4408845484
- Semantic Scholar
- 9689e75b792fbe2ef503042bb5453be829f6253d
- Cited by
- 1
- References
- 54
- Citations by year
- 2025: 1
- FWCI
- 1.07
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Accountability in Research (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Keywords
- data availability policy, data sharing, open science, policy-practice gap, medical journals, reproducibility, journal impact factor, research transparency, medical subspecialties
- MeSH terms
- Humans, Periodicals as Topic, Publishing, Information Dissemination, Editorial Policies, Journal Impact Factor
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
