Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017
Open accessThe DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index.
Abstract
Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000-2014, the biomedical literature largely lacked transparency in important dimensions. Here, we surveyed a random sample of 149 biomedical articles published between 2015 and 2017 and determined the proportion reporting sources of public and/or private funding and conflicts of interests, sharing protocols and raw data, and undergoing rigorous independent replication and reproducibility checks. We also investigated what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. The majority of the 149 studies disclosed some information regarding funding (103, 69.1% [95% confidence interval, 61.0% to 76.3%]) or conflicts of interest (97, 65.1% [56.8% to 72.6%]). Among the 104 articles with empirical data in which protocols or data sharing would be pertinent, 19 (18.3% [11.6% to 27.3%]) discussed publicly available data; only one (1.0% [0.1% to 6.0%]) included a link to a full study protocol. Among the 97 articles in which replication in studies with different data would be pertinent, there were five replication efforts (5.2% [1.9% to 12.2%]). Although clinical trial identification numbers and funding details were often provided on PubMed, only two of the articles without a full text article in PubMed Central that discussed publicly available data at the full text level also contained information related to data sharing on PubMed; none had a conflicts of interest statement on PubMed. Our evaluation suggests that although there have been improvements over the last few years in certain key indicators of reproducibility and transparency, opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Particular issues are presented by the use of published data, which can be unvalidated or incomplete due to selective reporting26,27.”
From Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparency (2020) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the literature contains studies with questionable methodologies, missing or incomplete data, false or exaggerated effect measures, and many studies do not mention funding or potential conflicts of interest [4–8].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies indicate that, although many authors agree on the importance of these transparent practices, they are often neglected [6, 7].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationEstimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Self-citation1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Self-citationPromoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A Systematic Review
Bekelman, J. E., Li, Y., & Gross, C. P. (2003) · JAMA · doi:10.1001/jama.289.4.454 · Cited by 1,816
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
What does research reproducibility mean?
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293
Self-citation
Research works printed but not linked: 3
- PLoS ONE. Data Availability. Available from: http://journals.plos.org/plosone/s/data-availability.
- BMC Research Notes. Aims and scope. https://bmcresnotes.biomedcentral.com/.
- D de Weerd-Wilson, 2017, How Elsevier is breaking down barriers to reproducibility, How Elsevier is breaking down barriers to reproducibility
Cited by
Items 101–200 of 268
Open Access in language testing and assessment: The case of two flagship journals
Liu, M., Al‐Hoorie, A. H., & Hiver, P. V. (2024) · Language Testing · doi:10.1177/02655322241261417 · Cited by 13
“In another study on a random sample of 149 biomedical articles between 2015 and 2017, Wallach et al. )2018) found that 30% included supplementary materials )though none allowed for a reconstruction of the full protocol), and 18% discussed publicly available data on some level.”
An assessment of transparency and reproducibility‐related research practices in otolaryngology
Johnson, A., Torgerson, T., Skinner, M., Hamilton, T., Tritz, D., & Vassar, M. B. (2019) · The Laryngoscope · doi:10.1002/lary.28322 · Cited by 14
Genomic features of parthenogenetic animalsPreprint
Jaroň, K. S., Bast, J., Nowell, R. W., Ranallo-Benavidez, T. R., Robinson‐Rechavi, M., & Schwander, T. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/497495 · Cited by 15
Reproducible Speech Research With the Artificial Intelligence–Ready PERCEPT Corpora
Benway, N. R., Preston, J. L., Hitchcock, E., Rose, Y., Salekin, A., Liang, W., & McAllister, T. (2023) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2023_jslhr-22-00343 · Cited by 12
Open science practices in criminology and criminal justice journals
Greenspan, R. L., Baggett, L., & Boutwell, B. B. (2024) · Journal of Experimental Criminology · doi:10.1007/s11292-024-09640-x · Cited by 14
Truth, Proof, and Reproducibility: There’s No Counter-Attack for the Codeless
Gray, C. T., & Marwick, B. (2019) · Communications in computer and information science · doi:10.1007/978-981-15-1960-4_8 · Cited by 11
Graf, C., Flanagan, D., Wylie, L., & Silver, D. (2020) · Data Intelligence · doi:10.1162/dint_a_00061 · Cited by 18
Miner, A. S., Fleming, S. L., Haque, A., Fries, J., Althoff, T., Wilfley, D. E., Agras, W. S., Milstein, A. S., Hancock, J. T., Asch, S. M., et al. (2022) · npj Mental Health Research · doi:10.1038/s44184-022-00020-9 · Cited by 28
“Features were selected that maximized reproducibility and transparency 98 .”
BackgroundOpen science practices for eating disorders research
Burke, N. L., Frank, G., Hilbert, A., Hildebrandt, T., Klump, K. L., Thomas, J. J., Wade, T. D., Walsh, B. T., Wang, S. B., & Weissman, R. S. (2021) · International Journal of Eating Disorders · doi:10.1002/eat.23607 · Cited by 13
Chiriboga, L. A., Callis, G. M., Wang, Y., & Chlipala, E. A. (2022) · Journal of Histotechnology · doi:10.1080/01478885.2022.2134022 · Cited by 14
Gorman, D. M., & Ferdinand, A. O. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00098-9 · Cited by 13
Hamilton, D. G., Fraser, H. S., Fidler, F. M., McDonald, S. J., Rowhani-Farid, A., Hong, K., & Page, M. J. (2021) · F1000Research · doi:10.12688/f1000research.53874.2 · Cited by 10
AutoFAIR-A portal for automating FAIR assessments for bioinformatics resources
Bonello, J., Cachia, E., & Alfino, N. (2021) · Biochimica et Biophysica Acta (BBA) - Gene Regulatory Mechanisms · doi:10.1016/j.bbagrm.2021.194767 · Cited by 11
Openness and Computational Reproducibility in Plant Pathology: Where We Stand and a Way Forward
Sparks, A., Ponte, E. M. D., Alves, K. S., Foster, Z., & Grünwald, N. J. (2023) · Phytopathology · doi:10.1094/phyto-10-21-0430-per · Cited by 11
Evaluating and Strengthening the Evidence for Nutritional Bone Research: Ready to Break New Ground?
Lewis, J. R., Voortman, T., & Ioannidis, J. P. A. (2020) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.4236 · Cited by 14
Self-citationBiomedical supervisors’ role modeling of open science practices
Haven, T. L., Abunijela, S., & Hildebrand, N. (2023) · eLife · doi:10.7554/elife.83484 · Cited by 14
“Within biomedical science, examples include open science practices such as openly sharing data and publishing open access, as well as making the underlying methodology openly available, and explicitly acknowledging the limitations of the research findings (Iqbal et al., 2016; Moher et al., 2018; Wallach et al., 2018;…”
BackgroundNault, R., Cave, M. C., Ludewig, G., Moseley, H., Pennell, K. G., & Zacharewski, T. R. (2023) · Environmental Health Perspectives · doi:10.1289/ehp11484 · Cited by 11
Silva, F. M., Adegboye, A. R. A., Lachat, C. K., Curioni, C. C., Gomes, F. D. S., Collins, G. S., Kac, G., de Beyer, J. A., Cook, J. A., Ismail, L. C., et al. (2023) · JMIR Research Protocols · doi:10.2196/43537 · Cited by 10
“3% discussed publicly available data [33].”
Fladie, I. A., Adewumi, M. T., Vo, N. H., Tritz, D., & Vassar, M. B. (2019) · Kidney International Reports · doi:10.1016/j.ekir.2019.11.001 · Cited by 11
Reproducible Research Practices in the Plastic Surgery Literature
Ascha, M., Katabi, L., Stevens, E., Gatherwright, J., & Vassar, M. B. (2022) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000008956 · Cited by 9
Gan, R. W., Sun, D. K., Tatro, A. R., Cohen‐Mekelburg, S., Wiitala, W. L., Zhu, J., & Waljee, A. K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0257520 · Cited by 11
Transparency in Cognitive Training Meta-analyses: A Meta-review
Sandoval‐Lentisco, A., López‐Nicolás, R., Tortajada, M., López-López, J. A., & Sánchez‐Meca, J. (2024) · Neuropsychology Review · doi:10.1007/s11065-024-09638-2 · Cited by 10
“This form was based on previous studies with similar scope (Hardwicke et al., 2020; Koffel & Rethlefsen, 2016; López-Nicolás et al., 2022; Pigott & Polanin, 2020; Wallach et al., 2018) and the PRISMA 2020 guidelines (Page et al., 2021).”
Kemper, J. M., Rolnik, D. L., Mol, B. W., & Ioannidis, J. P. A. (2020) · Fertility and Sterility · doi:10.1016/j.fertnstert.2020.05.020 · Cited by 10
Self-citationGrahe, J. E., & Cuccolo, K. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009000796.014 · Cited by 9
The changing landscape of scientific publishing
Chiriboga, L. A. (2019) · Journal of Histotechnology · doi:10.1080/01478885.2019.1636554 · Cited by 12
Data Management and Sharing: Practices and Perceptions of Psychology ResearchersPreprint
Borghi, J., & Gulick, A. E. V. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7g3ae · Cited by 10
Norris, E., Clark, K., Munafo, M. R., Jay, C., Baldwin, J. R., Lautarescu, A., Pedder, H., Page, M. P. A., Rinke, E. M., Burn, C. C., et al. (2022) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/w48yh · Cited by 18
Data sharing and transparency indicators in published RCTs in Oral Health between 2017 and 2023
Mikelis, F., Karamalaki, D., Mikeli, A., Tzanetakis, G. N., & Koletsi, D. (2024) · Journal of Dentistry · doi:10.1016/j.jdent.2024.105263 · Cited by 8
Automatic transparency evaluation for open knowledge extraction systems
Basereh, M., Caputo, A., & Brennan, R. (2023) · Journal of Biomedical Semantics · doi:10.1186/s13326-023-00293-9 · Cited by 15
Khan, K. S., Fawzy, M. M., CHIEN, P. F. W., Geary, M. P., Bueno‐Cavanillas, A., Núñez‐Núñez, M., Zamora, J., Bedaiwy, M. A., Serour, G. I., D’Hooghe, T., et al. (2025) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.16118 · Cited by 11
Few randomized trials in preterm birth prevention meet predefined usefulness criteria
Hooft, J. V. ’., van Dijk, C. E., Axfors, C., Alfirević, Ž., Oudijk, M. A., Mol, B. W., Bossuyt, P. M., & Ioannidis, J. P. A. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.08.016 · Cited by 9
Self-citationThe Outcomes of Scientific Debates Should Be Published: The Arivale Story
Fiala, C., & Diamandis, E. P. (2020) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfaa110 · Cited by 8
Data sharing statements: impact of journal policies across clinical research disciplines
Archer, D., Barks, N., Chaudhry, M., Dennis, B., Duncan, J., Elfar, A., Gardner, T., Paul, E., Kee, M. D., Ford, A. I., et al. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf359 · Cited by 9
“29,30 These measures, they argue, would deter non-compliance and enhance scientific rigour.”
Peer Review and Transparency in Evidence-Source Selection in Value and Health Technology Assessment
Desai, B., Mattingly, T. J., van den Broek, R. W. M., Pham, N., Frailer, M., Yang, J., & Perfetto, E. M. (2020) · Value in Health · doi:10.1016/j.jval.2020.01.014 · Cited by 8
Gonçalves-Pereira, J., Fernandes, J., Mendes, T. F., Gonzalez, F. A., & Fernandes, S. M. (2024) · Antibiotics · doi:10.3390/antibiotics13090853 · Cited by 8
“Adapted from reference [138].”
Schueller, S., Mikelis, F., Eliades, T., & Koletsi, D. (2024) · European Journal of Orthodontics · doi:10.1093/ejo/cjae064 · Cited by 8
“There is an increasing interest of researchers in transparent research practices that ensure credible and reproducible research [1, 2].”
Holst, M., Faust, A., & Strech, D. (2022) · Health Research Policy and Systems · doi:10.1186/s12961-022-00841-2 · Cited by 9
“The results are unsurprising, given recent findings that practices for robust and transparent research are only very slowly becoming more prevalent [30, 32]; however, they stand in stark contrast to the various experts and institutions that have called for institutions to align their promotion criteria with robust and…”
ResultCHIEN, P. F. W., Elsuity, M. A., Rashwan, M., Núñez‐Núñez, M., Khan, K. S., Zamora, J., Bueno‐Cavanillas, A., & Fawzy, M. M. (2024) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.15488 · Cited by 7
Language Bias in Health Research: External Factors That Influence Latent Language Patterns
Valdez, D., & Goodson, P. (2020) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2020.00004 · Cited by 10
Evidence for Action: Stimulating Research to Address the Global Mental Health Treatment Gap
Pringle, B., Williams, M. J., & Collins, P. Y. (2019) · Springer eBooks · doi:10.1007/978-3-319-70134-9_94-1 · Cited by 10
Advancing physiology through transparency: Celebrating our first registered report
Rasmussen, P., Tipton, M., Stewart, A., & Bailey, D. M. (2024) · Experimental Physiology · doi:10.1113/ep091963 · Cited by 7
Assessing transparency practices in dental randomized controlled trials
Prado, M. C., Dotto, L., Agostini, B., & Sarkis-Onofre, R. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02316-0 · Cited by 6
“Studies indicate that, although many authors agree on the importance of these transparent practices, they are often neglected [6, 7].”
BackgroundReproducibility of Published Meta-Analyses on Clinical-Psychological Interventions
López‐Nicolás, R., Lakens, D., López-López, J. A., Rubio‐Aparicio, M., Sandoval‐Lentisco, A., López‐Ibáñez, C., Blázquez-Rincón, D., & Sánchez‐Meca, J. (2024) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231202929 · Cited by 6
“Although data availability seems to have improved in the last years (Hardwicke et al., 2018; Tedersoo et al., 2021; Wallach et al., 2018), systematic reviews and meta-analyses appear to be a special case.”
BackgroundAre we capable of separating the wheat from the chaff when assessing meta-analyses?
Correia, M. I. T. D. (2019) · Clinical Nutrition · doi:10.1016/j.clnu.2019.04.013 · Cited by 6
Evaluating the reproducibility of research in obstetrics and gynecology
Rauh, S. L., Bowers, A., Rorah, D., Tritz, D., Pate, H., Frye, L., & Vassar, M. B. (2021) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2021.12.021 · Cited by 6
AccTEF: A Transparency and Accountability Evaluation Framework for Ontology-Based Systems
Basereh, M., Caputo, A., & Brennan, R. (2022) · International Journal of Semantic Computing · doi:10.1142/s1793351x22400013 · Cited by 7
Sharing study materials in health and medical research
DeVito, N., Morton, C. E., Cashin, A. G., Richards, G. C., & Lee, H. (2022) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-111987 · Cited by 10
López‐Nicolás, R., López-López, J. A., Rubio‐Aparicio, M., & Sánchez‐Meca, J. (2021) · doi:10.31234/osf.io/dz8cm · Cited by 6
Nicotinamide for Skin Cancer Chemoprevention: The Jury Was Out and Still is
Tan, E., & Williams, H. C. (2026) · American Journal of Clinical Dermatology · doi:10.1007/s40257-025-01005-y · Cited by 5
“Data sharing and methodological transparency are therefore essential to ensure the replicability and credibility of findings [22].”
Ignacio, A. C., Oliveira, N. L., da Silva, L. X. N., Feter, J. S., Nardi, A. T. D., Helal, L. C. A. S., dos Santos, M. R., Soares, D. D. S., Galliano, L. M., Alano, T. S., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309087 · Cited by 8
Fink, L., & Marcus, J. (2024) · Economic Inquiry · doi:10.1111/ecin.13267 · Cited by 16
Chin, J. M., Zeiler, K., Dilevski, N., Holcombe, A. O., Gatfield-Jeffries, R., Bishop, R., Vazire, S., & Schiavone, S. R. (2023) · F1000Research · doi:10.12688/f1000research.127563.1 · Cited by 8
Talimtzi, P., Ntolkeras, A., Kostopoulos, G., Bougioukas, K. I., Pagkalidou, E., Ouranidis, A., Pataka, A., & Haidich, A. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111264 · Cited by 5
Gode, M., & Faggion, C. M. (2024) · Cancer Medicine · doi:10.1002/cam4.7036 · Cited by 5
Advantages and challenges to open science practices
Геннадиевна, Д. И. (2023) · Terra Economicus · doi:10.18522/2073-6606-2023-21-3-70-87 · Cited by 9
“Предвари-тельная регистрация подразумевает, что ученые до начала исследования представляют план своей работы, включая постановку исследовательских вопросов, гипотез, методов анализа, и размещают информацию в специальных репозиториях (Wallach et al., 2018).”
A decentralized framework for cultivating research lifecycle transparency
Jeng, W., Wang, S.-H., Chen, H., Huang, P., Chen, Y., & Hsiao, H. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241496 · Cited by 6
Can Orthopaedics become the Gold Standard for Reproducibility? A Roadmap to SuccessPreprint
Fladie, I. A., Evans, S., Checketts, J. X., Tritz, D., Norris, B. L., & Vassar, M. B. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/715144 · Cited by 6
Barch, D. M. (2021) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2021.07.002 · Cited by 5
Who published what on Somali health issues? Forming the policy for SHAJ through a bibliometric study
Bile, K. M., Emmelin, M., Freij, L., Gustafsson, L. L., Sahlèn, K.-G., Wall, S., & Yusuf, M. W. (2022) · Somali Health Action Journal · doi:10.36368/shaj.v2i1.281 · Cited by 5
Hamilton, D. G., Hong, K., Fraser, H. S., Rowhani-Farid, A., Fidler, F. M., & Page, M. J. (2023) · medRxiv · doi:10.1101/2023.03.22.23287607 · Cited by 5
“Furthermore, the additional searches revealed another 44 eligible reports for inclusion, resulting in a total of 114 eligible meta-research studies examining a combined total of 2,254,031 primary articles for the review [8-15,34-142].”
BackgroundPage, C. E., Anderson, E., & Ainsworth, T. D. (2024) · Ecology and Evolution · doi:10.1002/ece3.11616 · Cited by 6
Improving medical research in the United Kingdom
Bradley, S. H., DeVito, N., Lloyd, K. E., Logullo, P., & Butler, J. E. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06050-y · Cited by 6
Artificial Intelligence in Otolaryngology
Tai, K., Zhao, R., & Rameau, A. (2024) · Otolaryngologic Clinics of North America · doi:10.1016/j.otc.2024.04.008 · Cited by 5
Gorman, L., Browne, W. J., Woods, C. J., Eisler, M. C., van Wijk, M. T., Dowsey, A. W., & Hammond, J. (2021) · Frontiers in Sustainable Food Systems · doi:10.3389/fsufs.2021.727425 · Cited by 4
Storås, A. M., Strümke, I., Riegler, M. A., & Halvorsen, P. (2022) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3524273.3533925 · Cited by 6
“one third of research articles do not share the raw data [15, 34].”
Sofi‐Mahmudi, A., Raittio, E. J., Khazaei, Y., Ashraf, J., Schwendicke, F., Uribe, S. E., & Moher, D. (2024) · PLoS ONE · doi:10.1371/journal.pone.0313991 · Cited by 7
Enhancing Collaboration between Clinician-Researchers and Methodologists in Clinical Research
Amuah, J. E., Mutasingwa, D. R., & Amankwah, E. K. (2020) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2020.02.035 · Cited by 4
Data Communities: Empowering Researcher-Driven Data Sharing in the Sciences
Springer, R., & Cooper, D. (1970) · International Journal of Digital Curation · doi:10.2218/ijdc.v15i1.695 · Cited by 5
Evidence for Action: Stimulating Research to Address the Global Mental Health Treatment Gap
Pringle, B., Williams, M. J., & Collins, P. Y. (2021) · Springer eBooks · doi:10.1007/978-3-030-57296-9_94 · Cited by 5
Evans, S., Fladie, I. A., Anderson, J. M., Tritz, D., & Vassar, M. B. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/773473 · Cited by 3
Hansford, H. J., Wewege, M. A., & Jones, M. D. (2021) · Journal of Physical Activity and Health · doi:10.1123/jpah.2021-0049 · Cited by 4
Anderson, J. M., Niemann, A., Johnson, A., Cook, C., Tritz, D., & Vassar, M. B. (2019) · JMIR Dermatology · doi:10.2196/16078 · Cited by 3
Reporting completeness of scoping reviews in orthodontic literature up to 2022. An empirical study
Mikelis, F., & Koletsi, D. (2023) · European Journal of Orthodontics · doi:10.1093/ejo/cjad022 · Cited by 4
Hughes, A., O’Sullivan, M., Winning, L., Cassetti, O., O’Sullivan, A. M., Madigan, S. M., Egan, B., & Crowe, M. (2024) · Discover Public Health · doi:10.1186/s12982-024-00239-1 · Cited by 3
“The survey forms and data may be shared easily to comply with practices that favour and encourage transparency and reproducibility [25, 26].”
Eerola, T. (2024) · Musicae Scientiae · doi:10.1177/10298649241300885 · Cited by 7
“Overall, transparency and reproducibil-ity-related practices have been only minimally adopted in the fields of biomedicine (Iqbal et al., 2016; Wallach, Boyack & Ioannidis 2018), the social sciences (Hardwicke et al., 2020), and psychology (Hardwicke et al., 2022, 2024).”
InfluentialMiner, A. S., Fleming, S. L., Haque, A., Fries, J., Althoff, T., Wilfley, D. E., Agras, W. S., Milstein, A. S., Hancock, J. T., Ash, S. M., et al. (2022) · medRxiv · doi:10.1101/2022.04.24.22274227 · Cited by 3
Silva, F. M., Adegboye, A. R. A., Curioni, C. C., Gomes, F. D. S., Collins, G. S., Kac, G., Cook, J. A., Ismail, L. C., Page, M. J., Khandpur, N., et al. (2023) · medRxiv · doi:10.1101/2023.06.08.23291109 · Cited by 3
Research Transparency in 59 Disciplines of Clinical Medicine: A Meta-Research StudyPreprint
Sofi‐Mahmudi, A., Raittio, E. J., Uribe, S. E., Khademioore, S., Zeraatkar, D., Mbuagbaw, L. C. E., Bouter, L. M., & Robinson, K. A. (2024) · MetArXiv (OSF Preprints) · doi:10.1101/2024.04.08.24305416 · Cited by 4
“The analyzed articles represent, to some extent, a biased subset of all medical literature, even though earlier research has shown that there are no clear differences in these five transparency practices between articles with and without available full text in the EPMC (23).”
Transparency in Open Science: An Actionable Principle?
Romero, R. C. (2025) · Open Information Science · doi:10.1515/opis-2025-0016 · Cited by 7
Karamitros, G., Bouloukakis, G., Giannas, E., Lamaris, G. A., Thayer, W. P., Perdikis, G., Lineaweaver, W. C., & Assi, P. E. (2026) · Aesthetic Surgery Journal · doi:10.1093/asj/sjag051 · Cited by 2
“11,33 Nonetheless, upper-middle-income economies—especially China, Turkey, and South Korea—demonstrate the steepest relative acceleration, suggesting that global academic equity is gradually improving.”
Transparency of COVID-19-related research: A meta-research study
Sofi‐Mahmudi, A., Raittio, E. J., & Uribe, S. E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0288406 · Cited by 4
UK Psychology PhD researchers’ knowledge, perceptions, and experiences of open science
Pownall, M., Terry, J., Collins, E. C., Sladekova, M., & Jones, A. (2023) · Cogent Psychology · doi:10.1080/23311908.2023.2248765 · Cited by 7
Open Practices in Phytolith Research: A Community Survey
Ruiz-Pérez, J., Karoune, E., Kerfant, C. E., García-Granero, J. J., Madella, M., & Lancelotti, C. (2024) · Open Quaternary · doi:10.5334/oq.125 · Cited by 3
Roettger, T. B., Askelund, A. D., Birkenæs, V., Bjørndal, L. D., Bochyńska, A., Glaser, B. D., Kalandadze, T., Korbmacher, M., Malovic, I., Mayor, J., et al. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02601-6 · Cited by 2
Huang, C. J., Pawel, S., Wever, K. E., Ineichen, B. V., & Heyard, R. (2026) · eLife · doi:10.7554/elife.109853.3 · Cited by 3
Learning from Principles of Evidence-Based Medicine to Optimize Nonclinical Research Practices
Lefevre, I. A., & Balice‐Gordon, R. J. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_276 · Cited by 4
When you are making plans to publish research, you need to plan for data sharing
Bell, R. J., & Haring, R. (2020) · Climacteric · doi:10.1080/13697137.2020.1771302 · Cited by 2
“Along with the growing interest in the spirit of open science, data-sharing practices need to become more widespread and streamlined to increase the credibility and efficiency of our research data 14 .”
BackgroundCreating optimal conditions for reproducible data analysis in R with ‘fertile’
Bertin, A. M., & Baumer, B. S. (2020) · Stat · doi:10.1002/sta4.332 · Cited by 2
A multi-omics digital research object for the genetics of sleep regulationPreprint
Jan, M., Gobet, N., Diessler, S., Franken, P., & Xénarios, I. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/586206 · Cited by 2
Open science practices and risks arising from misuse of biological researchPreprint
Smith, J. A., & Sandbrink, J. B. (2021) · doi:10.31222/osf.io/8afwn · Cited by 2
Bradley, S. H., DeVito, N., Lloyd, K. E., Richards, G. C., Rombey, T., Wayant, C., & Gill, P. J. (2020) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/jy37f · Cited by 2
Palermo, T. M., Bouhassira, D., Davis, K. D., Hemmings, H. C., Hurley, R. W., Katz, J. D., Pandit, J. J., Price, T. J., Schatman, M. E., Schwarz, S. K. W., et al. (2025) · European Journal of Pain · doi:10.1002/ejp.70052 · Cited by 2
“Meta-research studies regarding open science practices high-light critical remaining gaps across many fields in reproducible research practices, open access data, and availability of protocols (e.g., Wallach et al. 2018).”
Palermo, T. M., Bouhassira, D., Davis, K. D., Hemmings, H. C., Hurley, R. W., Katz, J. D., Pandit, J. J., Price, T. J., Schatman, M. E., Schwarz, S. K. W., et al. (2025) · Anaesthesia · doi:10.1111/anae.16655 · Cited by 3
“Across scienti fi c disciplines, concerns have been raised about research quality and trustworthiness [1, 2].”
Palermo, T. M., Bouhassira, D., Davis, K. D., Hemmings, H. C., Hurley, R. W., Katz, J. D., Pandit, J. J., Price, T. J., Schatman, M. E., Schwarz, S. K. W., et al. (2025) · Journal of Pain Research · doi:10.2147/jpr.s381164 · Cited by 4
Palermo, T. M., Bouhassira, D., Davis, K. D., Hemmings, H. C., Hurley, R. W., Katz, J. D., Pandit, J. J., Price, T. J., Schatman, M. E., Schwarz, S. K. W., et al. (2025) · Pain Medicine · doi:10.1093/pm/pnaf062 · Cited by 3
Palermo, T. M., Bouhassira, D., Davis, K. D., Hemmings, H. C., Hurley, R. W., Katz, J. D., Pandit, J. J., Price, T. J., Schatman, M. E., Schwarz, S. K. W., et al. (2025) · Pain · doi:10.1097/j.pain.0000000000003681 · Cited by 5
Palermo, T. M., Bouhassira, D., Davis, K. D., Hemmings, H. C., Hurley, R. W., Katz, J. D., Pandit, J. J., Price, T. J., Schatman, M. E., Schwarz, S. K. W., et al. (2025) · Canadian Journal of Pain · doi:10.1080/24740527.2025.2506941 · Cited by 3
“1,2 While these challenges are not unique to pain and anesthesiology research, we recognize this as a judicious opportunity to raise awareness and collaborate across our journals to align and strengthen initiatives to enhance research integrity, trust, and impact across our field.”
Palermo, T. M., Bouhassira, D., Davis, K. D., Hemmings, H. C., Hurley, R. W., Katz, J. D., Pandit, J. J., Price, T. J., Schatman, M. E., Schwarz, S. K. W., et al. (2025) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000007632 · Cited by 3
Evaluation of Transparency and Openness Guidelines in Physical Therapist Journals
Plante, J., Langerwerf, L., Klopper, M., Rhon, D. I., & Young, J. L. (2023) · Physical Therapy · doi:10.1093/ptj/pzad133 · Cited by 1
A crise na integridade científica e suas implicações
Amorim, M. M. R., & Bacchi, A. D. (2025) · Revista Brasileira de Saúde Materno Infantil · doi:10.1590/1806-9304202500000368-en · Cited by 4
Metadata
- Authors
- Wallach, J. D., Boyack, K. W., & Ioannidis, J. P. A.
- Affiliations
- Yale University (US)
- SciTech Strategies (United States) (US)
- Stanford Medicine (US)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-11-20
- Volume
- 16
- Issue
- 11
- Pages
- e2006930
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC6245499
- OpenAlex
- W2901231347
- Cited by
- 269
- References
- 38
- Citations by year
- 2026: 21 · 2025: 30 · 2024: 30 · 2023: 31 · 2022: 33 · 2021: 42 · 2020: 52 · 2019: 27 · 2018: 2
- FWCI
- 43.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- National Science Foundation
- Laura and John Arnold Foundation
- National Institutes of Health
- National Institute on Drug Abuse (HHSN271201700041C)
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- reproducibility, research transparency, open data, conflicts of interest, funding disclosure, data sharing, replication study, PubMed, biomedical literature
- MeSH terms
- Humans, Publications, Reproducibility of Results, Conflict of Interest, Access to Information, Disclosure, Information Dissemination, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.