Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review
Open accessThe DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


![Figure 3 Status of approved protocols for Chan 2004b study [17].](https://figures.cochat.ai/paper-figures/83600bbc5644401caed195b486efac8f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-3.jpg)
![Figure 4 Status of approved protocols for Easterbrook 1991 study [24].](https://figures.cochat.ai/paper-figures/0a114dad26924772826758941018f42f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-4.jpg)
![Figure 5 Status of approved protocols for Dickersin 1992 study [23].](https://figures.cochat.ai/paper-figures/1453f80dcd014ca3b9b32399777cc8d3/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-5.jpg)
![Figure 6 Status of approved protocols for Dickersin 1993 study [3].](https://figures.cochat.ai/paper-figures/1e175d9e8b2a4f0390577db97b820ca8/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-6.jpg)
![Figure 7 Status of approved protocols for Stern 1997 study [4].](https://figures.cochat.ai/paper-figures/876f2f4683aa40a689e73241856ecb1f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-7.jpg)
![Figure 8 Status of approved protocols for Cooper 1997 study [19].](https://figures.cochat.ai/paper-figures/beceb71fe2a943b2ba35a9fc9d17293b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-8.jpg)
![Figure 9 Status of trials for Wormald 1997 study [30].](https://figures.cochat.ai/paper-figures/f6a16e454252431f84d6cc28737b2e50/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-9.jpg)
![Figure 10 Status of approved protocols for Ioannidis 1998 study [5].](https://figures.cochat.ai/paper-figures/261662c83e664841aaa276cbc7c62394/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-10.jpg)
![Figure 11 Status of approved protocols for Pich 2003 study [27].](https://figures.cochat.ai/paper-figures/48fe85db48fc4b6ca22f25bb5580c0a7/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-11.jpg)
![Figure 12 Status of approved protocols for Cronin 2004 study [20].](https://figures.cochat.ai/paper-figures/8d211a89740e4b67b0c80783ab9c0582/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-12.jpg)
![Figure 13 Status of approved protocols for Decullier 2005 study [7].](https://figures.cochat.ai/paper-figures/c7868be91276448192f71187c860c7a5/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-13.jpg)
![Figure 14 Status of approved protocols for Decullier 2006 study [22].](https://figures.cochat.ai/paper-figures/4859f8920e714475a966997babd4cb7a/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-14.jpg)
![Figure 15 Status of approved protocols for Hahn 2002 study [26].](https://figures.cochat.ai/paper-figures/69f971391f504ed4bae3d24d4a05f2c1/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-15.jpg)
![Figure 16 Status of approved protocols for Chan 2004a study [18].](https://figures.cochat.ai/paper-figures/933f83ab86634ea581351235ceea1e11/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-16.jpg)
![Figure 17 Status of approved protocols for Ghersi 2006 study [25].](https://figures.cochat.ai/paper-figures/97881d16700f4ed48102dec8fc05b22b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-17.jpg)
![Figure 18 Status of approved protocols for von Elm 2008 study [29].](https://figures.cochat.ai/paper-figures/90d1c2ab102849719682b2f8f8409206/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-18.jpg)
![Figure 19 Status of approved protocols for Turer 2007 study [28].](https://figures.cochat.ai/paper-figures/ef99b99be5ff4105970df8c239918ce4/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-19.jpg)
![Figure 20 Status of approved protocols for De Jong 2010 study [21].](https://figures.cochat.ai/paper-figures/bf16aa205a3c4592aa0e050025754624/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-20.jpg)
![Figure 21 Status of approved protocols for Blumle 2008 study [31].](https://figures.cochat.ai/paper-figures/f70bc7510f114d9ba3b2e1e1e342ba98/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-21.jpg)
![Figure 22 Status of approved protocols for Hall 2007 study [32].](https://figures.cochat.ai/paper-figures/577511f26e0441a6bbc7643a3998c183/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-22.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinical trial registries provide the ability to measure biases in the reporting of clinical trials that arise due to non-publication, delayed publication, or incomplete publication of results [14].”
From A systematic review of the processes used to link clinical trial registrations to their published results (2017) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many publications of randomized clinical trials of hypnosis interventions have been unclear or incomplete in reporting of the design elements that are necessary for the evaluation of the quality of evidence, 13 or have not provided sufficient details for possible direct replication.14,15”
From Best Practice Recommendations for Conducting and Reporting Controlled Trials in Clinical Hypnosis Research (2024) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“studies with statistically non-significant results may not have been submitted for publication (publication bias), or particular results that were statistically non-significant may have been omitted from study reports (selective non-reporting bias).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [83]
The cited source says what the claim attributes to it.
In the citing paper
“Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias.”
From Systematic review protocol assessing the processes for linking clinical trial registries and their published results (2016) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Despite providing important information relevant to patients, trials with statistically non-significant results are more prone to non-publication,114 or, when published, to a long delay to publication114 115 116—often referred to as publication bias.”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [116]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 12×
“Easterbrook et al [24] found that compared with unfunded studies, government funded studies were more likely to yield statistically significant results but government sponsorship was not found to have a statistically significant effect on the likelihood of publication and company sponsored trials were less likely to b…”
BackgroundMethodResultInfluentialSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · cited 1×
“Several studies investigated a cohort of trials submitted to drug licensing authorities [38,40,41,65] and all found that many of these trials remain unpublished, with one study demonstrating that trials with positive outcomes resulted more often in submission of a final report to the regulatory authority [40].”
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,412
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · cited 2×
“While much effort has been invested in trying to identify the former [2], it is equally important to understand the nature and frequency of missing data from the latter level.”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 29×
“The main objective of the study by Blumle et al [31] was to consider how eligibility criteria stated in protocols was reported in subsequent reports, in doing so they noted that 52% of studies in their cohort were published, decreasing to 48% for RCTs only.”
BackgroundMethodInfluentialSystematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 6×
“Since the original version of this review was published [13], a Cochrane methodology review is now available on publication bias [64].”
BackgroundMethodInfluentialSelf-citationPublication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · cited 2×
“While much effort has been invested in trying to identify the former [1,2], it is equally important to understand the nature and frequency of missing data from the latter level.”
Background
Research works printed but not linked: 2
- Menzel S, Uebing B, Hucklenbroich P, Schober O (2007) Evaluation of clinical trials following an approval from a research ethics committee. Dtsch Med Wochenschr 132: 2313– 2317. 17957593
- Chappell L, Alfirevic Z, Chien P, Jarvis S, Thornton JG (2005) A comparison of the published version of randomized controlled trials in a specialist clinical journal with the original trial protocols; Chicago.
Cited by
Items 201–300 of 1,001
Journals’ instructions to authors: A cross-sectional study across scientific disciplines
Malički, M., Aalbersberg, I. J., Bouter, L., & ter Riet, G. (2019) · PLOS ONE · doi:10.1371/journal.pone.0222157 · Cited by 56
Büttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2019-101863 · Cited by 47
Akomea-Frimpong, I., Jin, X., & Osei‐Kyei, R. (2022) · Sustainability · doi:10.3390/su14127174 · Cited by 68
Wang, X., Husu, J., & Toom, A. (2024) · Teaching and Teacher Education · doi:10.1016/j.tate.2024.104822 · Cited by 64
Toews, I., Glenton, C., Lewin, S. A., Berg, R. C., Noyes, J., Booth, A., Marušić, A., Malički, M., Munthe‐Kaas, H. M., & Meerpohl, J. J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0159290 · Cited by 62
Turnbull, S., Cabral, C., Hay, A. D., & Lucas, P. J. (2020) · Journal of Medical Internet Research · doi:10.2196/17849 · Cited by 58
Effect of Yoga on Frailty in Older Adults: A Systematic Review
Loewenthal, J., Innes, K. E., Mitzner, M. G., Mita, C., & Orkaby, A. R. (2023) · Annals of Internal Medicine · doi:10.7326/m22-2553 · Cited by 69
“6%) had published results, suggesting potential publication bias in that trials with nonpositive results are more likely to remain unpublished (74).”
BackgroundMyers, S. P., & Vigar, V. (2019) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2018.0340 · Cited by 55
Buljac‐Samardžić, M., Doorn, C. M. D.-V., & Maynard, M. T. (2021) · Journal of Patient Safety · doi:10.1097/pts.0000000000000816 · Cited by 56
Ioannidis, J. P. A. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.04.029 · Cited by 58
Côté, P., Hartvigsen, J., Axén, I., Leboeuf‐Yde, C., Corso, M., Shearer, H. M., Wong, J. J., Marchand, A.-A., Cassidy, J. D., French, S., et al. (2021) · Chiropractic & Manual Therapies · doi:10.1186/s12998-021-00362-9 · Cited by 58
Guimarães, J. R., Oliveira, K. S. G. C., Gonçalves, M. C. P., Romanelli, J. P., Lopes, L. A., Berenguer‐Murcia, Á., Fernández‐Lafuente, R., & Tardioli, P. W. (2023) · Reaction Chemistry & Engineering · doi:10.1039/d3re00420a · Cited by 56
Savina, M., Gourgou, S., Italiano, A., Dinart, D., Rondeau, V., Penel, N., Mathoulin‐Pélissier, S., & Bellera, C. A. (2017) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2017.11.014 · Cited by 54
Sivaramalingam, B., Young, I., Pham, M. T., Waddell, L., Greig, J., Mascarenhas, M., & Papadopoulos, A. (2015) · Foodborne Pathogens and Disease · doi:10.1089/fpd.2014.1927 · Cited by 47
Meta‐analysis: Exercise intervention for sleep problems in cancer patients
Fang, Y., Hung, C., Chan, J., Huang, S., & Lee, Y. (2019) · European Journal of Cancer Care · doi:10.1111/ecc.13131 · Cited by 48
Bias in Peripheral Depression Biomarkers
Carvalho, A. F., Köhler, C. A., Brunoni, A. R., Miskowiak, K. W., Herrmann, N., Lanctôt, K. L., Hyphantis, T. N., Quevedo, J., Fernandes, B. S., & Berk, M. (2016) · Psychotherapy and Psychosomatics · doi:10.1159/000441457 · Cited by 52
Risk Factors for Ovarian Cancer: An Umbrella Review of the Literature
Whelan, E., Kalliala, I. E. J., Semertzidou, A., Raglan, O., Bowden, S. J., Kechagias, K. S., Markozannes, G., Cividini, S., McNeish, I. A., Marchesi, J. R., et al. (2022) · Cancers · doi:10.3390/cancers14112708 · Cited by 64
Progress on core outcome sets for critical care research
Blackwood, B., Marshall, J. C., & Rose, L. (2015) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000000232 · Cited by 54
on behalf of the Taskforce Team of the Obesity Fact Sheet of the Korean Society for the Study of Obesity, Lee, K. R., Seo, M. H., Han, K., Jung, J.-H., & Hwang, I. C. (2018) · British Journal of Cancer · doi:10.1038/s41416-018-0214-7 · Cited by 56
Yao, C., Duan, Z. D., & Baruch, Y. (2020) · International Journal of Management Reviews · doi:10.1111/ijmr.12223 · Cited by 61
Godard-Sebillotte, C., Le Berre, M., Schuster, T., Trottier, M., & Vedel, I. (2019) · PLOS ONE · doi:10.1371/journal.pone.0218426 · Cited by 50
Wood, W., & Carden, L. (2014) · Psychological Bulletin · doi:10.1037/a0036722 · Cited by 50
Hodkinson, A., Dietz, K. C., Lefebvre, C., Golder, S., Jones, M., Doshi, P., Heneghan, C., Jefferson, T., Boutron, I., & Stewart, L. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0766-x · Cited by 49
Cure-all or curse? A meta-regression on the effect of tourism development on poverty alleviation
Zhang, D., Wang, Q., & Yang, Y. (2022) · Tourism Management · doi:10.1016/j.tourman.2022.104650 · Cited by 52
The Place of Experimental Design and Statistics in the 3Rs
Parker, R. M., & Browne, W. J. (2014) · ILAR Journal · doi:10.1093/ilar/ilu044 · Cited by 53
“…0.05 (Masicampo and Lalande 2012), results not in keeping with the experimental hypothesis residing in metaphorical file drawers rather than the pages of journals (e.g., Dwan et al. 2013; Rosenthal 1979; ter Riet et al. 2012) likely reflect a variety of investigator-level and more systemic factors.”
ResultInfluentialRisk of bias assessments for selective reporting were inadequate in the majority of Cochrane reviews
Saric, F., Barcot, O., & Puljak, L. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.04.007 · Cited by 48
Outcome reporting recommendations for clinical trial protocols and reports: a scoping review
Butcher, N. J., Mew, E. J., Monsour, A., Chan, A.-W., Moher, D., & Offringa, M. (2020) · Trials · doi:10.1186/s13063-020-04440-w · Cited by 48
Trial registration 10 years on
Weber, W. E. J., Merino, J. G., & Loder, E. (2015) · BMJ · doi:10.1136/bmj.h3572 · Cited by 46
Bashir, R., Bourgeois, F. T., & Dunn, A. G. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0518-3 · Cited by 47
“Clinical trial registries provide the ability to measure biases in the reporting of clinical trials that arise due to non-publication, delayed publication, or incomplete publication of results [14].”
Backgroundde Barra, M. (2017) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.01.033 · Cited by 51
“RCTs of commercial health products are themselves subject to a publication bias, with “positive” outcomes more likely to be published (Dwan et al., 2013).”
BackgroundOh, S., Ji, C., & Azad, S. (2020) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13272 · Cited by 47
A scoping review on interventions to promote physical activity among adults with disabilities
Castro, Ó., Ng, K., Novoradovskaya, E., Bosselut, G., & Hassandra, M. (2017) · Disability and health journal · doi:10.1016/j.dhjo.2017.10.013 · Cited by 49
Publication and Reporting of Test Accuracy Studies Registered in ClinicalTrials.gov
Korevaar, D. A., Ochodo, E. A., Bossuyt, P. M. M., & Hooft, L. (2014) · Clinical Chemistry · doi:10.1373/clinchem.2013.218149 · Cited by 47
Lee, J.-Y., & Baek, M.-K. (2023) · Sustainability · doi:10.3390/su151411325 · Cited by 50
van den Akker, O. R., van Assen, M. A. L. M., Enting, M., de Jonge, M., Ong, H. H., Rüffer, F. F., Schoenmakers, M., Stoevenbelt, A. H., Wicherts, J. M., & Bakker, M. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231187988 · Cited by 51
Herrmann, D., Sinnett, P., Holmes, J., Khan, S. H., Koller, C., & Vassar, M. B. (2016) · Annals of Oncology · doi:10.1093/annonc/mdw691 · Cited by 52
“Such bias compromises the validity of systematic review outcomes and may lead to inaccurate clinical decisions [4].”
BackgroundAyorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01010-1 · Cited by 49
“extensively in the clinical research literature [2, 4, 5], and several high-profile cases of non-publication of”
BackgroundMethodResultvan den Akker, O. R., Bakker, M., van Assen, M. A. L. M., Pennington, C. R., Verweij, L., Elsherif, M. M., Claesen, A., Gaillard, S. D. M., Yeung, S. K., Frankenberger, J.-L., et al. (2024) · Psychological Methods · doi:10.1037/met0000687 · Cited by 52
Kohler, M., Haag, S., Biester, K., Brockhaus, A. C., McGauran, N., Grouven, U., Kolsch, H., Seay, U., Horn, H., Moritz, G., et al. (2015) · BMJ · doi:10.1136/bmj.h796 · Cited by 43
Evaluation of Research Ethics Committees: Criteria for the Ethical Quality of the Review Process
Scherzinger, G., & Bobbert, M. (2017) · Accountability in Research · doi:10.1080/08989621.2016.1273778 · Cited by 49
“By so doing, it helps reduce the publication bias (Dwan et al. 2013).”
BackgroundIMI – Industry Guidelines and Ethical Considerations for Myopia Control Report
Jones, L., Drobe, B., González‐Méijome, J. M., Gray, L. S., Kratzer, T., Newman, S., Nichols, J. J., Ohlendorf, A., Ramdass, S., Santodomingo‐Rubido, J., et al. (2019) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.18-25963 · Cited by 42
Qin, X., Jia, G., Zhou, X., & Yang, Z. (2022) · Advances in Nutrition · doi:10.1093/advances/nmac087 · Cited by 51
Saltaji, H., Armijo-Olivo, S., Cummings, G. G., Amin, M., & Flores-Mir, C. (2017) · PLoS ONE · doi:10.1371/journal.pone.0190089 · Cited by 43
Knowledge and attitude towards mpox: Systematic review and meta-analysis
León‐Figueroa, D. A., Barboza, J. J., Siddiq, A., Sah, R., Valladares-Garrido, M. J., & Rodriguez-Morales, A. J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0308478 · Cited by 56
Zareba, P., Wu, C., Agzarian, J., Rodríguez, D., & Kearon, C. (2014) · British journal of surgery · doi:10.1002/bjs.9527 · Cited by 45
Methodology of clinical trials for rare diseases
Smith, C. T., Williamson, P. R., & Beresford, M. W. (2014) · Best Practice & Research Clinical Rheumatology · doi:10.1016/j.berh.2014.03.004 · Cited by 48
Self-citationQuality of family relationships and outcomes of dementia: a systematic review
Edwards, H. B., Ijaz, S., Whiting, P. F., Leach, V. A., Richards, A., Cullum, S. J., Cheston, R., & Savović, J. (2018) · BMJ Open · doi:10.1136/bmjopen-2016-015538 · Cited by 44
“It is known that studies with null findings are less likely to be published, and authors can selectively report their ‘more interesting’ findings.(54) This means it is possible there are other relevant results that have not been reported and so would not be available to a review.”
Individual consequences of age stereotypes on older workers: A systematic review
Weber, J., Angerer, P., & Müller, A. (2019) · Zeitschrift für Gerontologie und Geriatrie · doi:10.1007/s00391-019-01506-6 · Cited by 50
Shang, Z., Wanyan, P., Zhang, B., Wang, M., & Wang, X. (2023) · Stem Cell Research & Therapy · doi:10.1186/s13287-023-03332-5 · Cited by 45
A brief introduction of meta‐analyses in clinical practice and research
Wang, X., Zhang, X., Li, Z., Zhong, W., Yang, P., & Mao, C. (2021) · The Journal of Gene Medicine · doi:10.1002/jgm.3312 · Cited by 45
“of bias that is recognized as a potential threat to the validity of results.(79) The real research effect may be exaggerated or even falsely positive if only published articles are included.”
BackgroundBrief Behavioral Treatment for Insomnia: A Meta-Analysis
Kwon, M., Wang, J., Wilding, G. E., Dickerson, S. S., & Dean, G. E. (2021) · Behavioral Sleep Medicine · doi:10.1080/15402002.2021.1982715 · Cited by 45
Lepore, S. J., Revenson, T. A., Roberts, K. J., Pranikoff, J. R., & Davey, A. (2015) · Psychology & Health · doi:10.1080/08870446.2014.971798 · Cited by 41
Cumpston, M., Brennan, S., Ryan, R. E., & McKenzie, J. E. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.02.003 · Cited by 43
Barranco-Merino, R., Higuera-Trujillo, J. L., & Llinares, C. (2023) · Applied Sciences · doi:10.3390/app132413095 · Cited by 47
Hou, S., & Roberson, K. (2014) · Journal of Cancer Education · doi:10.1007/s13187-014-0723-x · Cited by 41
Pierzycki, R. H., McNamara, A. J., Hoare, D. J., & Hall, D. A. (2015) · Hearing Research · doi:10.1016/j.heares.2015.11.003 · Cited by 43
“That is to say, we assert that reports concerning tinnitus and oscillatory measures of brain activity are subject to considerable confirmation bias (Dwan et al., 2013).”
BackgroundDiscontinuation and non-publication of clinical trials in cardiovascular medicine
Roddick, A., Chan, F., Stefaniak, J. D., & Zheng, S. L. (2017) · International Journal of Cardiology · doi:10.1016/j.ijcard.2017.06.020 · Cited by 42
Reducing falls among people living with dementia: A systematic review
Peek, K., Bryant, J., Carey, M. L., Dodd, N., Freund, M., Lawson, S., & Meyer, C. (2018) · Dementia · doi:10.1177/1471301218803201 · Cited by 45
Transparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 47
Pranić, S. M., & Marušić, A. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.07.007 · Cited by 46
Monson, F. K. S. (2024) · The Journal of High Technology Management Research · doi:10.1016/j.hitech.2024.100516 · Cited by 47
Anderson, A., Alexanders, J., Sinani, C., Hayes, S. C., & Fogarty, M. (2014) · Physiotherapy · doi:10.1016/j.physio.2014.07.006 · Cited by 39
Clifford, A., & Shakeshaft, A. P. (2017) · Drug and Alcohol Review · doi:10.1111/dar.12510 · Cited by 41
Clinical trial registration and reporting: a survey of academic organizations in the United States
Mayo-Wilson, E., Heyward, J., Keyes, A., Reynolds, J., White, S., Atri, N., Alexander, G. C., Omar, A., & Ford, D. E. (2018) · BMC Medicine · doi:10.1186/s12916-018-1042-6 · Cited by 43
Parental Involvement and Student Achievement: A Meta-analysis of Publications in the Scopus Database
Universitas Negeri Yogyakarta, I., Sujarwo, S., Herwin, H., & Universitas Negeri Yogyakarta, I. (2023) · International Journal of Instruction · doi:10.29333/iji.2023.1627a · Cited by 59
López‐Pernas, S. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3328913 · Cited by 49
“This phenomenon occurs when the probability of a study being published is contingent on its positive or statistically significant results [71], [72].”
BackgroundSearching For the Optimal Fluid to Restore Microcirculatory Flow Dynamics After Haemorrhagic Shock
Naumann, D. N., Beaven, A., Dretzke, J., Hutchings, S., & Midwinter, M. J. (2016) · Shock · doi:10.1097/shk.0000000000000687 · Cited by 39
“This is a notable limitation since significant results are more likely to be published (124), and it is likely that unpublished and unavailable studies have not been included in the current systematic review.”
BackgroundMoxibustion for the treatment of osteoarthritis: An updated systematic review and meta-analysis
Choi, T., Lee, M. S., Kim, J. I., & Zaslawski, C. (2017) · Maturitas · doi:10.1016/j.maturitas.2017.03.314 · Cited by 41
Age of Data at the Time of Publication of Contemporary Clinical Trials
Welsh, J. W., Lu, Y., Dhruva, S. S., Bikdeli, B., Desai, N. R., Benchetrit, L., Zimmerman, C. O., Mu, L., Ross, J. S., & Krumholz, H. M. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.1065 · Cited by 46
Mayo-Wilson, E., Fusco, N., Hong, H., Li, T., Canner, J. K., & Dickersin, K. (2019) · Trials · doi:10.1186/s13063-019-3581-3 · Cited by 35
Zhang, S., Liang, F., & Li, W. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.07.010 · Cited by 40
Page, M. J., Huang, H., Verhagen, A. P., Gagnier, J. J., & Buchbinder, R. (2018) · Arthritis Care & Research · doi:10.1002/acr.23254 · Cited by 38
“Selective reporting of only the positive/statistically significant outcomes is common in clinical trials (26), and so by relying on what was reported rather than asking trialists about any nonreported domains and instruments, we may have underestimated the frequency of outcome domains and instruments that were actuall…”
BackgroundCatalá-López, F., Aleixandre-Benavent, R., Caulley, L., Hutton, B., Tabarés-Seisdedos, R., Moher, D., & Alonso-Arroyo, A. (2020) · Trials · doi:10.1186/s13063-019-3944-9 · Cited by 38
Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review
Kempf, E., de Beyer, J. A., Cook, J. A., Holmes, J., Mohammed, S. B., Nguyen, T., Simera, I., Trivella, M., Altman, D. G., Hopewell, S., et al. (2018) · British Journal of Cancer · doi:10.1038/s41416-018-0305-5 · Cited by 41
“This result agrees with studies of other types of research: half of all RCTs report at least one outcome that either does not appear in the study protocol or is omitted or is changed when compared to their protocols.(26) Statistically significant and positive results in RCTs are more likely to be reported than non-sig…”
BackgroundResultList, F., Tarone, A. M., Zhu‐Salzman, K., & Vargo, E. L. (2022) · Pest Management Science · doi:10.1002/ps.6884 · Cited by 40
“90 However, another explanation for a bias in the literature is that researchers gravitate toward what appears to have already worked.”
BackgroundAll preclinical trials should be registered in advance in an online registry
Lorkeers, S. J. J. O., Doevendans, P. A., & Chamuleau, S. A. J. (2014) · European Journal of Clinical Investigation · doi:10.1111/eci.12299 · Cited by 39
Funnels for publication bias – have we lost the plot?
Choi, S. W., & Lam, D. M. H. (2015) · Anaesthesia · doi:10.1111/anae.13355 · Cited by 40
Manole, A., Cârciumaru, R., Brînzaș, R., & Manole, F. (2024) · Information · doi:10.3390/info16010011 · Cited by 44
Qin, X., Chen, J., Jia, G., & Yang, Z. (2023) · Advances in Nutrition · doi:10.1016/j.advnut.2023.02.004 · Cited by 39
Evaluation of publication bias in response interruption and redirection: A meta‐analysis
Dowdy, A., Tincani, M., & Schneider, W. J. (2020) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.724 · Cited by 37
Zou, C. X., Becker, J. E., Phillips, A. T., Garritano, J. M., Krumholz, H. M., Miller, J. E., & Ross, J. S. (2018) · Trials · doi:10.1186/s13063-018-2957-0 · Cited by 36
Ricci-Cabello, I., Bobrow, K., Islam, S. M. S., Chow, C. K., Maddison, R., Whittaker, R., & Farmer, A. J. (2019) · JMIR mHealth and uHealth · doi:10.2196/12191 · Cited by 36
Open Science Needs No Martyrs, but We Must Recognize the Need for Reform
Susi, T. (2022) · doi:10.4000/books.oep.16156 · Cited by 36
Hazards of current concentration-setting practices in environmental toxicology studies
Wolf, J. C., & Segner, H. E. (2023) · Critical Reviews in Toxicology · doi:10.1080/10408444.2023.2229372 · Cited by 39
“This preponderance of false-positive results could be a function of investigator bias (Ernst and Canter 2003), which itself may stem from the pressure to report and publish positive study outcomes (Joober et al. 2012; Dwan et al. 2013).”
BackgroundJafari, A., Abbastabar, M., Alaghi, A., Heshmati, J., Crowe, F. L., & Sepidarkish, M. (2024) · Phytotherapy Research · doi:10.1002/ptr.8340 · Cited by 38
Cao, X., Lei, L., & Wen, J. (2021) · Learned Publishing · doi:10.1002/leap.1322 · Cited by 35
“Learned Publishing © 2020 ALPSP Learned Publishing 2021; 34: 82–88 results in academic literature (Dwan et al., 2013).”
BackgroundResultInfluentialYuan, Z.-M., & Yao, M. (2022) · Scientometrics · doi:10.1007/s11192-022-04515-2 · Cited by 32
“Third, the increasing linguistic positivity bias in academic writing may also be attributed to positive outcome reporting bias (Dwan et al., 2013) or positive publication bias (Mlinarić et al., 2017).”
BackgroundResultA model-based correction for outcome reporting bias in meta-analysis
Copas, J., Dwan, K., Kirkham, J., & Williamson, P. (2014) · Biostatistics · doi:10.1093/biostatistics/kxt046 · Cited by 36
Self-citationAcetyl-L-carnitine for patients with hepatic encephalopathy
Martí-Carvajal, A. J., Gluud, C. N., Arévalo-Rodríguez, I., & Martí-Amarista, C. E. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011451.pub2 · Cited by 43
“This approach may reduce the impact of outcome reporting bias (Kirkham 2010; Dwan 2013).”
BackgroundGrey matters; on the importance of publication bias in systematic reviews
Winters, M., & Weir, A. (2016) · British Journal of Sports Medicine · doi:10.1136/bjsports-2016-096679 · Cited by 36
Useche, S. A., Colomer-Pérez, N., Alonso, F., & Faus, M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0296830 · Cited by 37
Schepers, W., Gryp, D., Häussermann, F., Heylen, L., Regenmortel, T. V., & Witte, J. D. (2025) · Health & Place · doi:10.1016/j.healthplace.2025.103491 · Cited by 43
Qureshı, R., Mayo-Wilson, E., Rittiphairoj, T., McAdams‐DeMarco, M., Güallar, E., & Li, T. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.10.024 · Cited by 39
Consent and confidentiality in the light of recent demands for data sharing
Williams, G. A., & Pigeot, I. (2016) · Biometrical Journal · doi:10.1002/bimj.201500044 · Cited by 47
“…considerable bias in research publication—for example, well-known patterns of under- and overpublication (McGauran et al., 2010; Jones et al., 2013; Dwan et al., 2013; Page et al., 2014) create bias by the relative absence of negative results and waste study participants’ time and energies…”
Backgroundvan den Bogert, C. A., Souverein, P. C., Brekelmans, C. T. M., Janssen, S. W. J., Koëter, G. H., Leufkens, H. G. M., & Bouter, L. M. (2016) · PLoS ONE · doi:10.1371/journal.pone.0167709 · Cited by 39
“The waste and bias implicated in clinical research caused by non-publication over the past years [3,6,7,8,9,10,11,12,13,14,15,16,17,18] has strengthened the view of several organizations and governments that all clinical trials must be published [19,20,21,22,23].”
Probabilistic forecasting of replication studies
Pawel, S., & Held, L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231416 · Cited by 46
Lee, H., Choi, T., Myung, C., Lee, J. A., & Lee, M. S. (2016) · Maturitas · doi:10.1016/j.maturitas.2016.01.012 · Cited by 35
Saldanha, I. J., Scherer, R. W., Rodriguez-Barraquer, I., Jampel, H. D., & Dickersin, K. (2016) · Trials · doi:10.1186/s13063-016-1343-z · Cited by 31
“Indeed, differences between abstracts and other sources have been found for results of RCTs [9, 10].”
BackgroundImpact of industry collaboration on randomised controlled trials in oncology
Linker, A. S., Yang, A. W., Roper, N., Whitaker, E. M., & Korenstein, D. R. (2016) · European Journal of Cancer · doi:10.1016/j.ejca.2016.11.005 · Cited by 34
Sun, X., Zhou, X., Yu, Y., & Liu, H. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0622-7 · Cited by 34
Metadata
- Authors
- Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group
- Affiliations
- University of Liverpool (GB)
- Corresponding author
- Dwan, Kerry
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-05
- Volume
- 8
- Issue
- 7
- Pages
- e66844
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3702538
- OpenAlex
- W2002062153
- Cited by
- 1,127
- References
- 69
- Citations by year
- 2026: 75 · 2025: 84 · 2024: 64 · 2023: 71 · 2022: 127 · 2021: 98 · 2020: 112 · 2019: 89 · 2018: 83 · 2017: 102 · 2016: 89 · 2015: 70 · 2014: 57 · 2013: 5
- FWCI
- 15.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- SERVAL (Université de Lausanne) (publishedVersion, cc-by)
- IRIS (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Tufts Medical Center
- King's College London
- University of Oxford
- University of Bristol
- University of Sydney
- Queen's University
- Women's College Hospital
- Centre Hospitalier Universitaire Vaudois
- Hospices Civils de Lyon
- Moorfields Eye Hospital NHS Foundation Trust
- School of Medicine, Stanford University
- Tufts University School of Medicine
- Medical Research Council
- National Health and Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- publication bias, selective reporting, randomized controlled trials, meta-analysis validity, trial reporting quality, primary outcome reporting
- MeSH terms
- Humans, Publications, Cohort Studies, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.