Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review
Open accessThe DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


![Figure 3 Status of approved protocols for Chan 2004b study [17].](https://figures.cochat.ai/paper-figures/83600bbc5644401caed195b486efac8f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-3.jpg)
![Figure 4 Status of approved protocols for Easterbrook 1991 study [24].](https://figures.cochat.ai/paper-figures/0a114dad26924772826758941018f42f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-4.jpg)
![Figure 5 Status of approved protocols for Dickersin 1992 study [23].](https://figures.cochat.ai/paper-figures/1453f80dcd014ca3b9b32399777cc8d3/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-5.jpg)
![Figure 6 Status of approved protocols for Dickersin 1993 study [3].](https://figures.cochat.ai/paper-figures/1e175d9e8b2a4f0390577db97b820ca8/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-6.jpg)
![Figure 7 Status of approved protocols for Stern 1997 study [4].](https://figures.cochat.ai/paper-figures/876f2f4683aa40a689e73241856ecb1f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-7.jpg)
![Figure 8 Status of approved protocols for Cooper 1997 study [19].](https://figures.cochat.ai/paper-figures/beceb71fe2a943b2ba35a9fc9d17293b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-8.jpg)
![Figure 9 Status of trials for Wormald 1997 study [30].](https://figures.cochat.ai/paper-figures/f6a16e454252431f84d6cc28737b2e50/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-9.jpg)
![Figure 10 Status of approved protocols for Ioannidis 1998 study [5].](https://figures.cochat.ai/paper-figures/261662c83e664841aaa276cbc7c62394/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-10.jpg)
![Figure 11 Status of approved protocols for Pich 2003 study [27].](https://figures.cochat.ai/paper-figures/48fe85db48fc4b6ca22f25bb5580c0a7/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-11.jpg)
![Figure 12 Status of approved protocols for Cronin 2004 study [20].](https://figures.cochat.ai/paper-figures/8d211a89740e4b67b0c80783ab9c0582/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-12.jpg)
![Figure 13 Status of approved protocols for Decullier 2005 study [7].](https://figures.cochat.ai/paper-figures/c7868be91276448192f71187c860c7a5/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-13.jpg)
![Figure 14 Status of approved protocols for Decullier 2006 study [22].](https://figures.cochat.ai/paper-figures/4859f8920e714475a966997babd4cb7a/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-14.jpg)
![Figure 15 Status of approved protocols for Hahn 2002 study [26].](https://figures.cochat.ai/paper-figures/69f971391f504ed4bae3d24d4a05f2c1/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-15.jpg)
![Figure 16 Status of approved protocols for Chan 2004a study [18].](https://figures.cochat.ai/paper-figures/933f83ab86634ea581351235ceea1e11/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-16.jpg)
![Figure 17 Status of approved protocols for Ghersi 2006 study [25].](https://figures.cochat.ai/paper-figures/97881d16700f4ed48102dec8fc05b22b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-17.jpg)
![Figure 18 Status of approved protocols for von Elm 2008 study [29].](https://figures.cochat.ai/paper-figures/90d1c2ab102849719682b2f8f8409206/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-18.jpg)
![Figure 19 Status of approved protocols for Turer 2007 study [28].](https://figures.cochat.ai/paper-figures/ef99b99be5ff4105970df8c239918ce4/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-19.jpg)
![Figure 20 Status of approved protocols for De Jong 2010 study [21].](https://figures.cochat.ai/paper-figures/bf16aa205a3c4592aa0e050025754624/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-20.jpg)
![Figure 21 Status of approved protocols for Blumle 2008 study [31].](https://figures.cochat.ai/paper-figures/f70bc7510f114d9ba3b2e1e1e342ba98/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-21.jpg)
![Figure 22 Status of approved protocols for Hall 2007 study [32].](https://figures.cochat.ai/paper-figures/577511f26e0441a6bbc7643a3998c183/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-22.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinical trial registries provide the ability to measure biases in the reporting of clinical trials that arise due to non-publication, delayed publication, or incomplete publication of results [14].”
From A systematic review of the processes used to link clinical trial registrations to their published results (2017) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many publications of randomized clinical trials of hypnosis interventions have been unclear or incomplete in reporting of the design elements that are necessary for the evaluation of the quality of evidence, 13 or have not provided sufficient details for possible direct replication.14,15”
From Best Practice Recommendations for Conducting and Reporting Controlled Trials in Clinical Hypnosis Research (2024) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“studies with statistically non-significant results may not have been submitted for publication (publication bias), or particular results that were statistically non-significant may have been omitted from study reports (selective non-reporting bias).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [83]
The cited source says what the claim attributes to it.
In the citing paper
“Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias.”
From Systematic review protocol assessing the processes for linking clinical trial registries and their published results (2016) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Despite providing important information relevant to patients, trials with statistically non-significant results are more prone to non-publication,114 or, when published, to a long delay to publication114 115 116—often referred to as publication bias.”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [116]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 12×
“Easterbrook et al [24] found that compared with unfunded studies, government funded studies were more likely to yield statistically significant results but government sponsorship was not found to have a statistically significant effect on the likelihood of publication and company sponsored trials were less likely to b…”
BackgroundMethodResultInfluentialSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · cited 1×
“Several studies investigated a cohort of trials submitted to drug licensing authorities [38,40,41,65] and all found that many of these trials remain unpublished, with one study demonstrating that trials with positive outcomes resulted more often in submission of a final report to the regulatory authority [40].”
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,412
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · cited 2×
“While much effort has been invested in trying to identify the former [2], it is equally important to understand the nature and frequency of missing data from the latter level.”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 29×
“The main objective of the study by Blumle et al [31] was to consider how eligibility criteria stated in protocols was reported in subsequent reports, in doing so they noted that 52% of studies in their cohort were published, decreasing to 48% for RCTs only.”
BackgroundMethodInfluentialSystematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 6×
“Since the original version of this review was published [13], a Cochrane methodology review is now available on publication bias [64].”
BackgroundMethodInfluentialSelf-citationPublication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · cited 2×
“While much effort has been invested in trying to identify the former [1,2], it is equally important to understand the nature and frequency of missing data from the latter level.”
Background
Research works printed but not linked: 2
- Menzel S, Uebing B, Hucklenbroich P, Schober O (2007) Evaluation of clinical trials following an approval from a research ethics committee. Dtsch Med Wochenschr 132: 2313– 2317. 17957593
- Chappell L, Alfirevic Z, Chien P, Jarvis S, Thornton JG (2005) A comparison of the published version of randomized controlled trials in a specialist clinical journal with the original trial protocols; Chicago.
Cited by
Items 501–600 of 1,001
Systematic reviews in restorative dentistry: discussing relevant aspects
Sarkis‐Onofre, R., Pereira‐Cenci, T., Tricco, A. C., Demarco, F. F., Moher, D., & Cenci, M. S. (2019) · Journal of Esthetic and Restorative Dentistry · doi:10.1111/jerd.12463 · Cited by 17
Overall bias and sample sizes were unchanged in ICU trials over time: a meta-epidemiological study
Anthon, C. T., Granholm, A., Perner, A., Laake, J. H., & Møller, M. H. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.021 · Cited by 16
Salholz‐Hillel, M., Strech, D., & Carlisle, B. G. (2022) · Clinical Trials · doi:10.1177/17407745221087456 · Cited by 16
Tijdink, J. K., Smulders, Y. M., Bouter, L. M., & Vinkers, C. H. (2019) · BMC Medical Ethics · doi:10.1186/s12910-019-0405-7 · Cited by 15
Conservation of semi-natural grasslands: Effects of different management methods on biodiversity
Tälle, M. (2018) · doi:10.3384/diss.diva-144796 · Cited by 17
Quality of meta‐analyses in freshwater ecology: A systematic review
Lodi, S., Godoy, B., Ortega, J., & Bini, L. (2021) · Freshwater Biology · doi:10.1111/fwb.13695 · Cited by 10
“This subject has been extensively debated (Dwan et al., 2013; Møller & Jennions, 2001; Rothstein et al., 2006) due to its effects on systematic reviews and meta- analyses (and also on narrative reviews).”
BackgroundCorrecting for outcome reporting bias in a meta-analysis: A meta-regression approach
van Aert, R. C. M., & Wicherts, J. M. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02132-2 · Cited by 20
“A systematic review (Dwan et al., 2008; Dwan et al., 2013) identified five articles that studied ORB (Chan, Hróbjartsson, et al., 2004a; Chan, Krleža-Jeri´c et al., 2004b; Elm et al., 2008; Ghersi, 2006; Hahn et al., 2002).”
BackgroundReporting trends of p values in the neurosurgical literature
To, M., & Jukes, A. (2019) · Journal of neurosurgery · doi:10.3171/2018.8.jns172897 · Cited by 13
Davidović, N., Komić, L., Mešin, I., Kotarac, M., Okmažić, D., & Franić, T. (2021) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-020-01710-5 · Cited by 14
Core Outcome Sets Relevant to Maternity Service Users: A Scoping Review
Slavin, V., Creedy, D. K., & Gamble, J. A. (2021) · Journal of Midwifery & Women s Health · doi:10.1111/jmwh.13195 · Cited by 15
Holliman, R., & Foster, R. D. (2023) · Journal of Child and Adolescent Counseling · doi:10.1080/23727810.2023.2232142 · Cited by 13
“…results to be published in journals (Stanley, 2005) and can lead to an artificial inflation of evidence regarding interventions, whereas studies with less impressive or significant results remain unpublished, creating inaccurate perceptions as to the efficacy of an intervention (Dwan et al., 2013).”
ResultMathur, M. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1667 · Cited by 15
“15,39,40 However, these estimates refer to individual studies rather than meta-analyses.”
BackgroundIntroducing Prospective Manuscript Review to Address Publication Bias
Mell, L. K., & Zietman, A. L. (2014) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2014.07.052 · Cited by 14
Mechler, K., Hoffmann, G. F., Dittmann, R., & Ries, M. (2016) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1546 · Cited by 13
Neurodevelopmental outcome descriptions in cohorts of extremely preterm children
Ding, S., Mew, E. J., Chee-A-Tow, A., Offringa, M., Butcher, N. J., & Moore, G. P. (2020) · Archives of Disease in Childhood Fetal & Neonatal · doi:10.1136/archdischild-2019-318144 · Cited by 17
Silveira, K., García-Barrera, M. A., & Smart, C. M. (2020) · Neuropsychology Review · doi:10.1007/s11065-020-09444-6 · Cited by 16
“The paucity of research reporting standardized test scores – even when using standardized tests – may be related to publication bias and outcome reporting bias, both of which are prevalent in healthcare research (e.g., Dwan et al. 2013).”
BackgroundMathur, M., & VanderWeele, T. (2019) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/p3xyd · Cited by 32
Vollgraff Heidweiller-Schreurs, C. A., Korevaar, D. A., Mol, B. W. J., Bax, C. J., de Groot, C. J. M., de Boer, M. A., & Bossuyt, P. M. M. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.07.016 · Cited by 12
Kekecs, Z., Moss, D., Whorwell, P. J., Varga, K., Terhune, D. B., Shenefelt, P. D., Palsson, O. S., Benedittis, G. D., & Elkins, G. (2024) · Journal of Evidence-Based Integrative Medicine · doi:10.1177/2515690x241274538 · Cited by 17
PROTOCOL: Assessment of outcome reporting bias in studies included in Campbell systematic reviews
Littell, J. H., Gorman, D. M., Valentine, J. C., & Pigott, T. (2023) · Campbell Systematic Reviews · doi:10.1002/cl2.1332 · Cited by 16
“Trials, 11(1), 52. https://doi. org/10.1186/1745-6215-11-52 Dwan et al., 2013 Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & Reporting Bias Group.”
ResultInfluentialKrešo, A., Grahovac, M., Znaor, L., & Marušić, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-79394-z · Cited by 13
A scoping review of activities intended to reduce publication bias in randomised trials
Hohlfeld, A., Kredo, T., & Clarke, M. J. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02728-5 · Cited by 13
Balanced examination of positive publication bias impact
Ramos, M. (2025) · Accountability in Research · doi:10.1080/08989621.2025.2538066 · Cited by 12
“These and others (Dowdy, Tincani, and Schneider 2020; Driessen et al. 2015; Dwan et al. 2013; Franco, Malhotra, and Simonovits 2014; Olsson, Sundell, and Rowe 2023; Sham and Smith 2014) show how studies on positive publication bias seem largely focused on demonstrating the overrepresentation of…”
Menopause and suicide: A systematic review
Hendriks, O., McIntyre, J. C., Rose, A. K., Sambrook, L., Reisel, D., Crockett, C., Newson, L. R., & Saini, P. (2025) · Women s Health · doi:10.1177/17455057251360517 · Cited by 13
“116,117 Additionally, most included studies were observational, with the majority being cross-sectional and lacking comparator groups (see Table 2).”
Shaw, S. R., D'Intino, J., & Lysenko, E. (2019) · Canadian Journal of School Psychology · doi:10.1177/0829573519843027 · Cited by 12
Showell, M. G., Buckman, S., Berber, S., Allah, N. A., Patterson, B., Cole, S., Farquhar, C. M., & Jordan, V. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279926 · Cited by 14
“A prospectively registered trial would have more hurdles to overcome, and many trials may never start or may be terminated early [16,17].”
BackgroundMerabtine, T., Tarhini, Z., Preux, P., Christou, N., & Jost, J. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.115946 · Cited by 13
“Finally because studies that report significant or positive results are more likely to be published than studies that report non-significant or negative results, we searched for publication bias (Dwan et al., 2013).”
Quality and Quantity of Information in Summary Basis of Decision Documents Issued by Health Canada
Habibi, R., & Lexchin, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092038 · Cited by 14
Kirkham, J. J., Dwan, K. M., Blümle, A., von Elm, E. B., & Williamson, P. R. (2016) · PLoS ONE · doi:10.1371/journal.pone.0157883 · Cited by 12
“Previously, we identified four additional studies that covered the entire process from the publication of the study protocol to the publication of the study outcomes that could have also been used as potential data sources for comparison [1].”
BackgroundSelf-citationSmith, T. D. H., Watt, H., Gunn, L., Car, J., & Boyle, R. J. (2016) · The Annals of Family Medicine · doi:10.1370/afm.1970 · Cited by 12
“9) Age when asthma diagnosed, median (range),a y 31 (11-52) 33 (11-50) 46 (30-59)b 47 (31-61)b”
Ziogas, I. A., Johnson, W. R., Matsuoka, L. K., Rauf, M. A., Thurm, C. W., Hall, M., Bacchetta, M. D., Godown, J. A., & Alexopoulos, S. P. (2020) · Transplantation · doi:10.1097/tp.0000000000003414 · Cited by 12
Ma, C., Liang, N., Gao, L., & Jia, C. (2020) · Journal of Obstetrics and Gynaecology · doi:10.1080/01443615.2020.1820461 · Cited by 13
“They might have been subjected to selection bias which was likely to have generated exaggerated treatment effects (Savovi c et al. 2012; Dwan et al. 2013).”
BackgroundAggarwal, R., & Oremus, M. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.10.010 · Cited by 12
Comparing the Value of Data Visualization Methods for Communicating Harms in Clinical Trials
Qureshı, R., Chen, X., Goerg, C., Mayo‐Wilson, E., Dickinson, S., Golzarri‐Arroyo, L., Hong, H., Phillips, R., Cornelius, V. R., McAdams‐DeMarco, M., et al. (2022) · Epidemiologic Reviews · doi:10.1093/epirev/mxac005 · Cited by 12
Tibbetts, J., Chapman, C., O’Dean, S., Butterworth, P., & Slade, T. N. (2025) · SSM - Mental Health · doi:10.1016/j.ssmmh.2025.100481 · Cited by 17
Misreporting of Results of Research in Psychiatry
Bowcut, J., Levi, L. W., Livnah, O., Ross, J. S., Knable, M. B., Davidson, M., Davis, J. M., & Weiser, M. (2021) · Schizophrenia Bulletin · doi:10.1093/schbul/sbab040 · Cited by 12
Principles of Systematic Reviewing
Higgins, J. P. T., Smith, G. D., Altman, D. G., & Egger, M. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch2 · Cited by 14
Deeks, J. J., Ashby, D., Takwoingi, Y., Perera, R., Evans, S. J. W., & Bird, S. M. (2024) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1093/jrsssa/qnae053 · Cited by 11
Krstičević, M., Saric, D., Šarić, F., Slapnicar, E., Borić, K., Došenović, S., Kadić, A. J., Kegalj, M. J., & Puljak, L. (2019) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2019-0068 · Cited by 12
Pires, D., Cruz, E. B., Gomes, L. A., & Nunes, C. (2020) · Physical Therapy · doi:10.1093/ptj/pzaa030 · Cited by 14
O’Connor, C. M. C., Gresham, M. D., Poulos, R. G., Hudson, W., Jackman, J., Clemson, L. M., McGilton, K. S., Cameron, I. D., Radoslovich, H., & Poulos, C. J. (2020) · Disability and Rehabilitation · doi:10.1080/09638288.2020.1797910 · Cited by 12
“Further, the exclusion of ineffective intervention trials during the literature review may have introduced selection bias into the developed programs; there is pressing need for this project to be followed up by comprehensive systematic reviews and meta-analyses as further study data become available [47,48].”
BackgroundIs the Psychopathic Brain an Artifact of Coding Bias? A Systematic Review
Jalava, J., Griffiths, S. Y., Larsen, R. R., & Alcott, B. E. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.654336 · Cited by 13
“The biases toward positive findings include the file drawer problem (only studies with positive findings are submitted to journals; Rosenthal, 1979), publication and reporting bias (journals are more likely to publish and authors to report positive than null-findings; Jennings and Van Horn, 2012; David et al., 2013, 2…”
BackgroundZhang, X., Deng, J., Tang, Y., Guan, X.-X., Chen, X., & Fan, J. (2022) · Phytotherapy Research · doi:10.1002/ptr.7500 · Cited by 13
Conducting separate reviews of benefits and harms could improve systematic reviews and meta-analyses
Mayo-Wilson, E., Qureshı, R., & Li, T. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02234-0 · Cited by 11
Norris, S. L., Holmer, H. K., Fu, R. F., Ogden, L. A., Viswanathan, M., & Abou-Setta, A. M. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1113 · Cited by 11
Dibao-Dina, C., Caille, A., & Giraudeau, B. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.007 · Cited by 11
Dune, T. M., Ullman, J., Ferfolja, T., Thepsourinthone, J., Garga, S., & Mengesha, Z. B. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17113861 · Cited by 13
“The review included both peer reviewed articles as well as unpublished reports that contain information related to the health, social and aged care experiences of GSD women, which makes publication bias unlikely [48].”
BackgroundHarvey, L. A. (2015) · Spinal Cord · doi:10.1038/sc.2015.89 · Cited by 11
Paludan-Müller, A. S., Ogden, M. C., Marquardsen, M., Vive, J., Jørgensen, K. J., & Gøtzsche, P. C. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026661 · Cited by 10
Bashir, R., & Dunn, A. G. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-013048 · Cited by 10
Report on a pilot project to introduce a publications officer
Cobey, K. D., Galipeau, J., Shamseer, L., & Moher, D. (2016) · Canadian Medical Association Journal · doi:10.1503/cmaj.151340 · Cited by 10
Innovations in infertility: a comprehensive analysis of the ClinicalTrials.gov database
Peipert, B. J., Spinosa, D., Howell, E., Weber, J. M., Truong, T., & Harris, B. S. (2021) · Fertility and Sterility · doi:10.1016/j.fertnstert.2021.06.013 · Cited by 10
Kim, W., Kim, J. H., Ki, Y., Chong, S., & Kim, T. J. (2022) · Academic Radiology · doi:10.1016/j.acra.2022.03.028 · Cited by 13
Rivero‐de‐Aguilar, A., Pérez‐Ríos, M., Ruano‐Raviña, A., Candal‐Pedreira, C., Puente-Hernandez, M., Ross, J. S., & Varela‐Lema, L. (2023) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp-2023-331132 · Cited by 12
The Logic of Generalization From Systematic Reviews and Meta-Analyses of Impact Evaluations
Littell, J. H. (2024) · Evaluation Review · doi:10.1177/0193841x241227481 · Cited by 11
“Reporting, publication, and dissemination biases affect study availability (Dwan et al., 2013, 2014; McLeod & Weisz, 2004; Pigott et al., 2013; Rothstein et al., 2005; Song et al., 2009, 2010).”
BackgroundMethodological quality of behavioural weight loss studies: a systematic review
Lemon, S. C., Wang, M. L., Haughton, C. F., Estabrook, D. P., Frisard, C., & Pagoto, S. (2016) · Obesity Reviews · doi:10.1111/obr.12412 · Cited by 12
“Publication bias is a well-known limiting factor in synthesizing research evidence that could impact the conclusions of this study (42).”
BackgroundShepshelovich, D., Yahav, D., Tibau, A., & Amir, E. (2019) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2019.08.009 · Cited by 10
A systematic analysis of diagnostic performance for Alzheimer's disease using structural MRI
Wu, J., Zhao, K., Li, Z., Wang, D., Ding, Y., Wei, Y., Zhang, H., & Liu, Y. (2022) · Psychoradiology · doi:10.1093/psyrad/kkac001 · Cited by 12
“Reporting bias is known as publication bias, which has an enormous impact on the effectiveness of a systematic review and its meta-analysis (Dwan et al. , 2013).”
Nasri, T., Sangmaneedet, S., Nam, N. H., Worawong, K., Taweenan, W., & Sukon, P. (2022) · Veterinary Parasitology · doi:10.1016/j.vetpar.2022.109724 · Cited by 10
Exposição ocupacional e câncer: uma revisão guarda-chuva
Guimarães, R. M., Dutra, V. G. P., Ayres, A. R. G., Garbin, H. B. D. R., Martins, T. C. D. F., & Meira, K. C. (2022) · Revista Brasileira de Saúde Ocupacional · doi:10.1590/2317-6369/37620pt2022v47e14 · Cited by 14
Zou, J., Zhou, J., Faller, S., Brown, R., Sankararaman, S., & Eskin, E. (2022) · G3 Genes Genomes Genetics · doi:10.1093/g3journal/jkac261 · Cited by 12
Silva, N. S., Elkins, M. R., Lemes, Í. R., Stubbs, P. W., Franco, M. R., & Pinto, R. Z. (2023) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2023.102854 · Cited by 11
Parissopoulos, S., Mpouzika, M. D. A., & Timmins, F. (2015) · Nursing in Critical Care · doi:10.1111/nicc.12205 · Cited by 10
“Thus, low TVs, pressure limitation and prone positioning interventions are the only recommendations presently receiving rigorous support (Dwan et al., 2013).”
BackgroundSingle-center trials in neonatology: Issues to consider
Sinha, I. P., & Sinha, S. K. (2015) · Seminars in Fetal and Neonatal Medicine · doi:10.1016/j.siny.2015.08.003 · Cited by 11
Karlsen, A. P. H., Dahl, J. B., & Mathiesen, O. (2018) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.13072 · Cited by 11
Dhruva, S. S., Shah, N. D., & Ross, J. S. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2020.04.013 · Cited by 10
Effects of selective outcome reporting on risk perception
Freudenstein, F., Croft, R. J., Loughran, S., Zeleke, B. M., & Wiedemann, P. M. (2021) · Environmental Research · doi:10.1016/j.envres.2021.110821 · Cited by 10
Composite Scores and Other Outcome Measures in Stroke Trials
Pistoia, F., Sacco, S., Ornello, R., Degan, D., Tiseo, C., & Carolei, A. (2016) · Monographs in clinical neuroscience/Frontiers of neurology and neuroscience/Monographs in neural sciences · doi:10.1159/000445413 · Cited by 11
Cariaga-Martínez, A. E., & Alelú‐Paz, R. (2016) · Journal of Negative Results in BioMedicine · doi:10.1186/s12952-016-0064-x · Cited by 11
Lind, J., Sabates‐Wheeler, R., & Szyp, C. (2022) · OpenDocs (Institute of Development Studies) · doi:10.19088/basic.2022.008 · Cited by 13
Risk of bias assessment tool for systematic review and meta-analysis of the gut microbiome
Lampeter, T., Love, C., Tang, T. T., Marella, A. S., Lee, H. Y., Oganyan, A. K., Moffat, D., Kareem, A., Rusling, M., Massmann, A., et al. (2023) · Gut Microbiome · doi:10.1017/gmb.2023.12 · Cited by 13
“Often, results are selected for significance, omitted for non-significance, or omitted for adverse effect of intervention (Dwan et al., 2013; Hedin et al., 2016; Van der Steen et al., 2019).”
BackgroundWan, X., Mo, H., Chen, J., Zhou, Q., Qiu, S., Zou, Y., Wu, W., Tang, Q., & Wang, S. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-087181 · Cited by 9
“6 Therefore, it is crucial to evaluate the rigour and reliability of the evidence”
Therapeutic potential of polysaccharides in inflammation: current insights and future directions
Balasubramaniyam, T., Ahn, H.-B., Lim, J., Basu, A., Kubiak, J. Z., Rampogu, S. D., & Nathan, V. K. (2025) · International Immunopharmacology · doi:10.1016/j.intimp.2025.115538 · Cited by 10
Metrological uncertainty and the value of informative nulls in marketing research
Sarstedt, M. (2026) · Journal of Marketing Analytics · doi:10.1057/s41270-026-00465-3 · Cited by 10
Matvienko‐Sikar, K., O'Shea, J., Kennedy, S., Thomas, S. D., AVERY, K. N. L., Byrne, M., McHugh, S., Connor, D. B. O., Saldanha, I. J., Smith, V. A., et al. (2024) · Health Psychology Review · doi:10.1080/17437199.2024.2367613 · Cited by 11
“For example, in a systematic review of studies examining ORB in reports of randomised controlled trials (RCTs), 40 – 62% of studies reported at least one outcome that was changed, omitted, or introduced (Dwan et al., 2013).”
Self-citationTiming, modifications, and tenor of firms' mandatory R&D disclosures: The role of competition
Rathee, C., Malik, S., & Salandra, R. (2024) · Research Policy · doi:10.1016/j.respol.2024.105112 · Cited by 10
Dal‐Ré, R. (2014) · Medicina Clínica · doi:10.1016/j.medcli.2014.01.030 · Cited by 10
Boden, C., Bidonde, J., & Busch, A. J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.04.021 · Cited by 11
Littlewood, N. A., Rocha, R., Smith, R. K., Martin, P. A., Lockhart, S. L., Schoonover, R. F., Wilman, E. N., Bladon, A. J., Sainsbury, K. A., Pimm, S. L., et al. (2020) · doi:10.17863/cam.57654 · Cited by 16
Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study
Moran, J. L. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01336-4 · Cited by 10
“Confounding such effects is the selective reporting of outcomes, or outcome reporting bias (ORB), whereby secondary outcomes are selectively reported based upon outcome results [3, 4].”
BackgroundInvestigating and Dealing with Publication Bias and Other Reporting Biases
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch5 · Cited by 14
Telemedicine in France: A review of registered clinical trials from 2000 to 2015
Ohannessian, R., Yaghobian, S., Chaleuil, M., & Salles, N. (2016) · European Research in Telemedicine / La Recherche Européenne en Télémédecine · doi:10.1016/j.eurtel.2016.04.001 · Cited by 10
Cook, B. G. (2016) · Behavioral Disorders · doi:10.17988/0198-7429-41.3.161 · Cited by 9
The use of study registration and protocols in plastic surgery research: A systematic review
Pidgeon, T. E., Limb, C., Agha, R., Whitehurst, K., Chandrakumar, C., Wellstead, G., Fowler, A. J., & Orgill, D. P. (2017) · International Journal of Surgery · doi:10.1016/j.ijsu.2017.06.035 · Cited by 9
An Integrative Review of Interventions to Support Parents When Managing Their Child's Pain at Home
Parker, R., McKeever, S., Wiseman, T., & Twycross, A. M. (2017) · Pain Management Nursing · doi:10.1016/j.pmn.2017.10.006 · Cited by 12
Gutierres, B. D. S., Aguiar, P. N., Dourado, B. B., Alves, A. L., Matsas, S., Simões, A. R., & del Giglio, A. (2020) · Journal of Thoracic Oncology · doi:10.1016/j.jtho.2020.03.004 · Cited by 9
Riley, W. T., Bibb, K., Hargrave, S. L., & Fearon, P. (2020) · PLoS ONE · doi:10.1371/journal.pone.0242271 · Cited by 9
A Scoping Review of Associations Between Ethno-Cultural Context and Mental Health in Canada
Emerson, S. D., Ritland, L., & Guhn, M. (2021) · Canadian Journal of Community Mental Health · doi:10.7870/cjcmh-2021-010 · Cited by 10
Is the replication crisis a base-rate fallacy?
Autzen, B. (2021) · Metamedicine · doi:10.1007/s11017-022-09561-8 · Cited by 9
Kjeldsen, M. Z., Bricca, A., Liu, X., Frøkjær, V. G., Madsen, K. B., & Munk‐Olsen, T. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01952-1 · Cited by 13
“As we only include case-control and cohort studies, we do not expect to be able to identify prespecified protocols on the included primary studies, and therefore, we will not investigate selective reporting bias through published protocols [44].”
Improving publication rates in a collaborative clinical trials research network
Archer, S. W., Carlo, W. A., Truog, W. E., Stevenson, D. K., Meurs, K. P. V., Sánchez, P. J., Das, A., Devaskar, U. P., Nelin, L. D., Huitema, C. M. P., et al. (2016) · Seminars in Perinatology · doi:10.1053/j.semperi.2016.05.003 · Cited by 10
Transparency in risk regulation : the case of the European Medicines Agency
Balog‐Way, D. H. P. (2017) · Research Portal (King's College London) · Cited by 14
Estimating the prevalence of missing experiments in a neuroimaging meta-analysisPreprint
Samartsidis, P., Montagna, S., Laird, A. R., Fox, P. T., Johnson, T. D., & Nichols, T. E. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/225425 · Cited by 10
“Evidence of the file drawer problem has been found in many fields of sci-entific research, including psychology (K¨uhberger et al. , 2014), public health (Dwan et al. , 2008, 2013) and the social sciences (Sterling et al. , 1995).”
Lind, J., Sabates‐Wheeler, R., & Szyp, C. (2023) · OpenDocs (Institute of Development Studies) · doi:10.19088/basic.2023.001 · Cited by 16
Comparison of protocols and registry entries to published reports for systematic reviews
Pieper, D., Allers, K., Mathes, T., Hoffmann, F., Klerings, I., Rombey, T., & Nussbaumer-Streit, B. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000053 · Cited by 8
Wu, D., Wu, H., & Li, J. (2024) · Scientometrics · doi:10.1007/s11192-024-05074-4 · Cited by 8
“Rather, two primary factors are widely recognized as driving this surge in the use of positive expressions: a competitive academic environment (Millar et al., 2019) and inherent favouritism towards positive results (Dwan et al., 2013).”
Evangelista, L. S., Meraz, R., Wierenga, K. L., Nguyen, A. P.-U., Angosta, A. D., & Kawi, J. (2025) · Nutrients · doi:10.3390/nu17142361 · Cited by 8
“Publication bias is also a consideration—trials with neutral or negative results (if any exist in this area) might be less likely to be published, potentially overstating the benefits in the literature [37].”
Akudjedu, T. N., Ago, J. L., Iweka, E., & Dushimirimana, S. (2025) · Radiography · doi:10.1016/j.radi.2025.103238 · Cited by 12
Declaración de transparencia para las publicaciones científicas
Catalá-López, F., Hutton, B., & Moher, D. (2014) · Medicina Clínica · doi:10.1016/j.medcli.2014.01.035 · Cited by 9
Zhu, H., & Liu, X. (2023) · Library Hi Tech · doi:10.1108/lht-01-2023-0022 · Cited by 9
Metadata
- Authors
- Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group
- Affiliations
- University of Liverpool (GB)
- Corresponding author
- Dwan, Kerry
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-05
- Volume
- 8
- Issue
- 7
- Pages
- e66844
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3702538
- OpenAlex
- W2002062153
- Cited by
- 1,127
- References
- 69
- Citations by year
- 2026: 75 · 2025: 84 · 2024: 64 · 2023: 71 · 2022: 127 · 2021: 98 · 2020: 112 · 2019: 89 · 2018: 83 · 2017: 102 · 2016: 89 · 2015: 70 · 2014: 57 · 2013: 5
- FWCI
- 15.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- SERVAL (Université de Lausanne) (publishedVersion, cc-by)
- IRIS (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Tufts Medical Center
- King's College London
- University of Oxford
- University of Bristol
- University of Sydney
- Queen's University
- Women's College Hospital
- Centre Hospitalier Universitaire Vaudois
- Hospices Civils de Lyon
- Moorfields Eye Hospital NHS Foundation Trust
- School of Medicine, Stanford University
- Tufts University School of Medicine
- Medical Research Council
- National Health and Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- publication bias, selective reporting, randomized controlled trials, meta-analysis validity, trial reporting quality, primary outcome reporting
- MeSH terms
- Humans, Publications, Cohort Studies, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.