Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review
Open accessThe DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


![Figure 3 Status of approved protocols for Chan 2004b study [17].](https://figures.cochat.ai/paper-figures/83600bbc5644401caed195b486efac8f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-3.jpg)
![Figure 4 Status of approved protocols for Easterbrook 1991 study [24].](https://figures.cochat.ai/paper-figures/0a114dad26924772826758941018f42f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-4.jpg)
![Figure 5 Status of approved protocols for Dickersin 1992 study [23].](https://figures.cochat.ai/paper-figures/1453f80dcd014ca3b9b32399777cc8d3/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-5.jpg)
![Figure 6 Status of approved protocols for Dickersin 1993 study [3].](https://figures.cochat.ai/paper-figures/1e175d9e8b2a4f0390577db97b820ca8/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-6.jpg)
![Figure 7 Status of approved protocols for Stern 1997 study [4].](https://figures.cochat.ai/paper-figures/876f2f4683aa40a689e73241856ecb1f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-7.jpg)
![Figure 8 Status of approved protocols for Cooper 1997 study [19].](https://figures.cochat.ai/paper-figures/beceb71fe2a943b2ba35a9fc9d17293b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-8.jpg)
![Figure 9 Status of trials for Wormald 1997 study [30].](https://figures.cochat.ai/paper-figures/f6a16e454252431f84d6cc28737b2e50/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-9.jpg)
![Figure 10 Status of approved protocols for Ioannidis 1998 study [5].](https://figures.cochat.ai/paper-figures/261662c83e664841aaa276cbc7c62394/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-10.jpg)
![Figure 11 Status of approved protocols for Pich 2003 study [27].](https://figures.cochat.ai/paper-figures/48fe85db48fc4b6ca22f25bb5580c0a7/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-11.jpg)
![Figure 12 Status of approved protocols for Cronin 2004 study [20].](https://figures.cochat.ai/paper-figures/8d211a89740e4b67b0c80783ab9c0582/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-12.jpg)
![Figure 13 Status of approved protocols for Decullier 2005 study [7].](https://figures.cochat.ai/paper-figures/c7868be91276448192f71187c860c7a5/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-13.jpg)
![Figure 14 Status of approved protocols for Decullier 2006 study [22].](https://figures.cochat.ai/paper-figures/4859f8920e714475a966997babd4cb7a/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-14.jpg)
![Figure 15 Status of approved protocols for Hahn 2002 study [26].](https://figures.cochat.ai/paper-figures/69f971391f504ed4bae3d24d4a05f2c1/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-15.jpg)
![Figure 16 Status of approved protocols for Chan 2004a study [18].](https://figures.cochat.ai/paper-figures/933f83ab86634ea581351235ceea1e11/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-16.jpg)
![Figure 17 Status of approved protocols for Ghersi 2006 study [25].](https://figures.cochat.ai/paper-figures/97881d16700f4ed48102dec8fc05b22b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-17.jpg)
![Figure 18 Status of approved protocols for von Elm 2008 study [29].](https://figures.cochat.ai/paper-figures/90d1c2ab102849719682b2f8f8409206/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-18.jpg)
![Figure 19 Status of approved protocols for Turer 2007 study [28].](https://figures.cochat.ai/paper-figures/ef99b99be5ff4105970df8c239918ce4/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-19.jpg)
![Figure 20 Status of approved protocols for De Jong 2010 study [21].](https://figures.cochat.ai/paper-figures/bf16aa205a3c4592aa0e050025754624/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-20.jpg)
![Figure 21 Status of approved protocols for Blumle 2008 study [31].](https://figures.cochat.ai/paper-figures/f70bc7510f114d9ba3b2e1e1e342ba98/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-21.jpg)
![Figure 22 Status of approved protocols for Hall 2007 study [32].](https://figures.cochat.ai/paper-figures/577511f26e0441a6bbc7643a3998c183/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-22.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinical trial registries provide the ability to measure biases in the reporting of clinical trials that arise due to non-publication, delayed publication, or incomplete publication of results [14].”
From A systematic review of the processes used to link clinical trial registrations to their published results (2017) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many publications of randomized clinical trials of hypnosis interventions have been unclear or incomplete in reporting of the design elements that are necessary for the evaluation of the quality of evidence, 13 or have not provided sufficient details for possible direct replication.14,15”
From Best Practice Recommendations for Conducting and Reporting Controlled Trials in Clinical Hypnosis Research (2024) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“studies with statistically non-significant results may not have been submitted for publication (publication bias), or particular results that were statistically non-significant may have been omitted from study reports (selective non-reporting bias).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [83]
The cited source says what the claim attributes to it.
In the citing paper
“Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias.”
From Systematic review protocol assessing the processes for linking clinical trial registries and their published results (2016) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Despite providing important information relevant to patients, trials with statistically non-significant results are more prone to non-publication,114 or, when published, to a long delay to publication114 115 116—often referred to as publication bias.”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [116]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 12×
“Easterbrook et al [24] found that compared with unfunded studies, government funded studies were more likely to yield statistically significant results but government sponsorship was not found to have a statistically significant effect on the likelihood of publication and company sponsored trials were less likely to b…”
BackgroundMethodResultInfluentialSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · cited 1×
“Several studies investigated a cohort of trials submitted to drug licensing authorities [38,40,41,65] and all found that many of these trials remain unpublished, with one study demonstrating that trials with positive outcomes resulted more often in submission of a final report to the regulatory authority [40].”
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,412
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · cited 2×
“While much effort has been invested in trying to identify the former [2], it is equally important to understand the nature and frequency of missing data from the latter level.”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 29×
“The main objective of the study by Blumle et al [31] was to consider how eligibility criteria stated in protocols was reported in subsequent reports, in doing so they noted that 52% of studies in their cohort were published, decreasing to 48% for RCTs only.”
BackgroundMethodInfluentialSystematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 6×
“Since the original version of this review was published [13], a Cochrane methodology review is now available on publication bias [64].”
BackgroundMethodInfluentialSelf-citationPublication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · cited 2×
“While much effort has been invested in trying to identify the former [1,2], it is equally important to understand the nature and frequency of missing data from the latter level.”
Background
Research works printed but not linked: 2
- Menzel S, Uebing B, Hucklenbroich P, Schober O (2007) Evaluation of clinical trials following an approval from a research ethics committee. Dtsch Med Wochenschr 132: 2313– 2317. 17957593
- Chappell L, Alfirevic Z, Chien P, Jarvis S, Thornton JG (2005) A comparison of the published version of randomized controlled trials in a specialist clinical journal with the original trial protocols; Chicago.
Cited by
Items 401–500 of 1,001
Identifying evidence for five realist reviews in primary health care: A comparison of search methods
Duddy, C., & Roberts, N. W. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1523 · Cited by 26
Rothmund, M., Lehmann, J., Moser, W., de Rojas, T., Sodergren, S. C., Darlington, A., & Riedl, D. (2022) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2022.103755 · Cited by 24
Retracting Inconclusive Research: Lessons from the Séralini GM Maize Feeding Study
Resnik, D. B. (2015) · Journal of Agricultural and Environmental Ethics · doi:10.1007/s10806-015-9546-y · Cited by 25
“Studies of peer review have found that reviewers and editors often fail to catch obvious errors, disagree about the quality of the manuscript, and are susceptible to various biases (Schroter et al. 2008; Bornmann et al. 2010; Lee et al. 2012; Dwan et al. 2013).”
Publication bias in the neighbourhood effects literature
Nieuwenhuis, J. (2016) · Geoforum · doi:10.1016/j.geoforum.2016.02.017 · Cited by 26
Hall, S., Oldfield, P., Mullins, B. J., Pollard, B., & Criado‐Perez, C. (2017) · Procedia Engineering · doi:10.1016/j.proeng.2017.04.341 · Cited by 24
Salandra, R. (2018) · Research Policy · doi:10.1016/j.respol.2018.04.005 · Cited by 24
“The most recent systematic review of the evidence on selective reporting discusses the increasing proportion of studies in which at least one outcome is changed or omitted, with some analyses concluding that up to 62% of the investigated trials had major discrepancies in the outcomes (Dwan et al., 2013).”
BackgroundInfluentialBrown, N. D., & Ashcroft, K. (2025) · Behavioral Sciences · doi:10.3390/bs15081031 · Cited by 24
“The English-language limitation may have introduced bias, affecting the representativeness of the included evidence (Dwan et al., 2013).”
Journal editors impasse with outcome reporting bias
Dal‐Ré, R., & Caplan, A. L. (2015) · European Journal of Clinical Investigation · doi:10.1111/eci.12484 · Cited by 21
Paladin, I., & Pranić, S. M. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01730-6 · Cited by 24
Kien, C., Nußbaumer-Streit, B., Thaler, K. J., Griebler, U., Noord, M. V., Wagner, P., & Gartlehner, G. (2014) · BMC Health Services Research · doi:10.1186/s12913-014-0551-z · Cited by 24
“Information contained in registries is often incomplete [6,7].”
BackgroundKorevaar, D. A., van Es, N., Zwinderman, A. H., Cohen, J. F., & Bossuyt, P. M. M. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0177-4 · Cited by 20
“There is now overwhelming evidence that studies with statistically non-significant results are less likely to result in a publication in a peer-reviewed journal than those with statistically significant results [1, 2, 6, 7].”
BackgroundHaworth, M., Hoshika, Y., & Killi, D. (2016) · Frontiers in Plant Science · doi:10.3389/fpls.2016.01153 · Cited by 27
“…with a large effect size are more likely to be written up in comparison to those where the replication is insufficient to demonstrate a significant effect) and reviewer bias (where reviewers favor manuscripts reporting strong treatment effects confirming a prevailing consensus; Dwan et al., 2013).”
BackgroundResearch, regulatory, and clinical decision-making: the importance of scientific integrity
Wallach, J. D., Gonsalves, G., & Ross, J. S. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.021 · Cited by 25
Students With Emotional–Behavioral Disorders as Cross-Age Tutors: A Synthesis of the Literature
Watts, G. W., Bryant, D. P., & Carroll, M. L. (2018) · Behavioral Disorders · doi:10.1177/0198742918771914 · Cited by 24
Germain, D. S., Denicoff, A. M., Torres, A., Kelaghan, J. W., McCaskill‐Stevens, W., Mishkin, G. E., O’Mara, A. M., & Minasian, L. M. (2020) · Cancer · doi:10.1002/cncr.32765 · Cited by 22
Thompson, T., Dias, S., Poulter, D. R., Weldon, S. M., Marsh, L. L., Rossato, C. J. L., Shin, J. I., Firth, J., Veronese, N., Dragioti, E., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01398-3 · Cited by 24
DeVito, N., & Goldacre, B. M. (2022) · Clinical Trials · doi:10.1177/17407745211073483 · Cited by 22
Dunn, A. G., Coiera, E. W., & Bourgeois, F. T. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.007 · Cited by 24
“A number of studies have used ClinicalTrials.gov to examine reporting biases, including both publication bias and outcome reporting bias [6,52].”
BackgroundIntroduction to Special Issue: Null Effects and Publication Bias in Learning Disabilities Research
Therrien, W. J., & Cook, B. G. (2018) · Learning Disabilities Research and Practice · doi:10.1111/ldrp.12163 · Cited by 22
Khan, M. S., Shaikh, A., Ochani, R. K., Akhtar, T., Fatima, K., Khan, S. U., Mookadam, F., Murad, M. H., Figueredo, V. M., Doukky, R., et al. (2019) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.118.005260 · Cited by 21
Tellum, T., Naftalin, J., Chapron, C., Dueholm, M., Guo, S., Hirsch, M., Larby, E. R., Munro, M. G., Sarıdoğan, E., van der Spuy, Z. M., et al. (2022) · Human Reproduction · doi:10.1093/humrep/deac166 · Cited by 22
Publication bias in psychology: A closer look at the correlation between sample size and effect size
Linden, A. H., Pollet, T. V., & Hönekopp, J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0297075 · Cited by 22
Murphy, E., Poudel, G. R., Ganesan, S., Suo, C., Manning, V., Beyer, E., Clemente, A., Moffat, B. A., Zalesky, A., & Lorenzetti, V. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105865 · Cited by 24
“Second, we excluded grey literature (e.g., pre-prints, dis-sertations, conferences) meaning that publication biases might have led to the current literature review over-representing significant findings (Dwan et al., 2013).”
BackgroundMueck, L. (2013) · Nature Nanotechnology · doi:10.1038/nnano.2013.204 · Cited by 21
Making protocols available with the article improved evaluation of selective outcome reporting
Calméjane, L., Dechartres, A., Tran, V.-T., & Ravaud, P. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.08.020 · Cited by 20
“Such practices referred as selective outcome reporting, tend to favor positive findings[6] which could distort the body of evidence available to clinicians and patients.”
BackgroundSutterland, A. L., Mounir, D. A., Ribbens, J. J., Kuiper, B., van Gool, T., & de Haan, L. (2020) · Schizophrenia Bulletin Open · doi:10.1093/schizbullopen/sgaa042 · Cited by 22
Tan, J., Li, L., Huang, X., Yang, C., Liang, X., Zhao, Y., Xie, J., Chen, R., Wang, D., & Xie, S. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-038450 · Cited by 22
Masouleh, S. K., Eickhoff, S. B., Balajoo, S. M., Nicolaisen‐Sobesky, E., Thirion, B., & Genon, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-17556-7 · Cited by 20
Trial Registration: Understanding and Preventing Reporting Bias in Social Work Research
Harrison, B. A., & Mayo-Wilson, E. (2014) · Research on Social Work Practice · doi:10.1177/1049731513512374 · Cited by 18
Caballero, D. C., Castillo, C. A., Ballesteros‐Yáñez, I., & Juárez, L. M. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0161-2 · Cited by 20
Grover, S., & Dalton, N. (2020) · Indian Journal of Psychiatry · doi:10.4103/psychiatry.indianjpsychiatry_320_19 · Cited by 20
Chugh, A., Patnana, A. K., Kumar, P., Chugh, V. K., Khera, D., & Singh, S. (2020) · Journal of Oral Biology and Craniofacial Research · doi:10.1016/j.jobcr.2020.07.011 · Cited by 26
“The deficiency in reporting publication bias in the included trials may overestimate the treatment effects and result in faulty evidence-based conclusions.(43)”
BackgroundBelgers, V., Röttgering, J. G., Douw, L., Klein, M., Ket, J. C. F., van de Ven, P. M., Würdinger, T., Linde, M. E. V., Niers, J. M., Weber, M., et al. (2023) · Cannabis and Cannabinoid Research · doi:10.1089/can.2021.0187 · Cited by 20
Brown tumors in nuclear medicine: a systematic review
Jacquet-Francillon, N., & Prévôt, N. (2023) · Annals of Nuclear Medicine · doi:10.1007/s12149-023-01832-1 · Cited by 19
“Case reports are known to have biases such as the lack of ability to generalize, lack of ability to establish cause-effect relationship, publication bias, and all biases of a retrospective design [75, 76].”
BackgroundCan a good tree bring forth evil fruit? The funding of medical research by industry
Capps, B. (2016) · British Medical Bulletin · doi:10.1093/bmb/ldw014 · Cited by 21
Caveat emptor: the combined effects of multiplicity and selective reporting
Li, T., Mayo-Wilson, E., Fusco, N., Hong, H., & Dickersin, K. (2018) · Trials · doi:10.1186/s13063-018-2888-9 · Cited by 21
“In addition, trials may not be reproducible because trialists have reported their studies selectively [4].”
BackgroundBagg, M. K., O’Hagan, E. T., Zahara, P., Wand, B. M., Hübscher, M., Moseley, G. L., & McAuley, J. H. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.12.006 · Cited by 24
Héroux, M. E., Butler, A. A., Cashin, A. G., McCaughey, E. J., Affleck, A. J., Green, M. A., Cartwright, A., Jones, M. D., Kiely, K. M., van Schooten, K. S., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-060976 · Cited by 21
Reporting bias in imaging: higher accuracy is linked to faster publication
Sharifabadi, A. D., Korevaar, D. A., McGrath, T. A., van Es, N., Frank, R. A., Cherpak, L. A., Dang, W., Salameh, J., Nguyen, F., Stanley, C., et al. (2018) · European Radiology · doi:10.1007/s00330-018-5354-x · Cited by 21
Poor outcome reporting in medical research; building practice on spoilt grounds
Mantziari, S., & Demartines, N. (2017) · Annals of Translational Medicine · doi:10.21037/atm.2017.03.75 · Cited by 21
“Moreover, it has been repeatedly observed that significant results and the presence of external funding interfere greatly with the choice of outcomes to be reported in final publication (3,6).”
BackgroundResultInfluentialMeursinge Reynders, R., Ladu, L., & Di Girolamo, N. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.07.001 · Cited by 18
Publication rates of abstracts presented at major interventional radiology conferences
Shergill, R., Kaka, H., Kennedy, S. A., & Baerlocher, M. O. (2017) · Diagnostic and Interventional Radiology · doi:10.5152/dir.2017.16499 · Cited by 19
“those with nonsignificant results (2).”
Kosa, S. D., Mbuagbaw, L. C. E., Debono, V. B., Bhandari, M., Dennis, B. B., Ene, G. E. V., Leenus, A., Shi, D., Thabane, M., Valvasori, S., et al. (2017) · Contemporary Clinical Trials · doi:10.1016/j.cct.2017.12.011 · Cited by 21
Hasan, H., Muhammed, T., Yu, J., Taguchi, K., Samargandi, O. A., Howard, A. F., Lo, A., Olson, R. A., & Goddard, K. (2017) · Cancer Epidemiology · doi:10.1016/j.canep.2017.08.013 · Cited by 18
The File Drawer Problem in Nanomedicine
Ashkarran, A. A., Swann, J. M., Hollis, L. P., & Mahmoudi, M. (2021) · Trends in biotechnology · doi:10.1016/j.tibtech.2021.01.009 · Cited by 18
Preobrazenski, N., Islam, H., & Gurd, B. J. (2021) · European Journal of Applied Physiology · doi:10.1007/s00421-021-04669-6 · Cited by 20
Gallinat, C., Moessner, M., Apondo, S., Thomann, P. A., Herpertz, S. C., & Bauer, S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182312391 · Cited by 20
“The present RCT provides important insights and may inform other researchers about potential challenges in conducting such a study, and helps to minimize publication bias [40,41], even though it was not completed in accordance to the study plan.”
BackgroundVilela, N., Nicolini, A. C., dos Reis, I. N. R., Sendyk, D. I., Cavagni, J., & Pannuti, C. M. (2022) · Journal of Periodontal Research · doi:10.1111/jre.13066 · Cited by 20
Christensen, R., Maxwell, L. J., Jüni, P., Tovey, D. I., Williamson, P. R., Boers, M., Goel, N., Buchbinder, R., MARCH, L. M., Terwee, C. B., et al. (2015) · The Journal of Rheumatology · doi:10.3899/jrheum.141384 · Cited by 18
Self-citationVaganay, A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0163702 · Cited by 20
The Self-Referencing task: Theoretical overview and empirical evidence
Mattavelli, S., Richetin, J., Gallucci, M., & Perugini, M. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.02.006 · Cited by 20
Papageorgiou, S. N., Antonoglou, G. N., Sándor, G. K., & Eliades, T. (2017) · PLOS ONE · doi:10.1371/journal.pone.0182785 · Cited by 17
Tong, A., Crowe, S., Gill, J. S., Harris, T., Hemmelgarn, B. R., Manns, B., Pecoits‐Filho, R., Tugwell, P., Biesen, W. V., Wang, A. Y.-M., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-021198 · Cited by 20
“Sex Men 43 (74) Women 15 (26) Age group (years) 30–39 14 (24) 40–49 20 (34) 50–59 18 (31) ≥60–69 6 (10) Years of experience in haemodialysis ≤10 14 (24) 11–20 22 (38) >20 22 (38) Country UK 14 (24) Australia 13 (22) Belgium 8 (14) USA 7 (12) Austria 5 (7) Canada 4 (7) Germany 4 (7) Singapore 2 (3) New Zealand 1 (2) Si…”
Jia, Y., Huang, D., Wen, J., Liang, J., Qureshı, R., Wang, Y., Rosman, L. M., Chen, Q., Robinson, K. A., Gagnier, J. J., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.11.023 · Cited by 17
Performance slumps in sport: A systematic review
Stead, J., Poolton, J., & Alder, D. B. (2022) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2022.102136 · Cited by 21
Health Associations of Positive Childhood Experiences: A Scoping Review of the Literature
Hero, J. O., Gallant, L., Burstein, D., Newberry, S. J., Qureshi, N., Feistel, K., Anderson, K. N., Hannan, K., & Sege, R. D. (2025) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph22010059 · Cited by 17
Can UK NHS research ethics committees effectively monitor publication and outcome reporting bias?
Begum, R., & Kolstoe, S. E. (2015) · BMC Medical Ethics · doi:10.1186/s12910-015-0042-8 · Cited by 22
“It is a well-recognized issue [2] that has recently come to the forefront of the public agenda in the UK due to the activities of the international “AllTrials” petition [3], and the writings of populist science authors such as Ben Goldacre [4].”
BackgroundMaggin, D. M. (2015) · Journal of Behavioral Education · doi:10.1007/s10864-015-9239-7 · Cited by 17
“The reason for this recommendation relates to the conceptually important and empirically established notion of publication bias which refers to the phenomenon that those studies with positive results are more likely than those without positive results to be published (e.g., Dwan et al. 2013).”
ResultПантели, Д., Nolting, A., Eckhardt, H., Kulig, M., & Busse, R. (2016) · Health Research Policy and Systems · doi:10.1186/s12961-016-0080-9 · Cited by 18
Korevaar, D. A., Cohen, J. F., Spijker, R., Saldanha, I. J., Dickersin, K., Virgili, G., Hooft, L., & Bossuyt, P. M. M. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.06.002 · Cited by 17
“There is overwhelming evidence of publication bias, caused by an overrepresentation of positive and favorable results in full-text publications, and leading to an overoptimistic literature base [4,6].”
Dyson, M. P., Shave, K., Gates, A., Fernandes, R. M., Scott, S. D., & Hartling, L. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-018199 · Cited by 19
Rigor and reproducibility for data analysis and design in the behavioral sciences
Hildebrandt, T., & Prenoveau, J. M. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103552 · Cited by 19
The Effect of Transcranial Random Noise Stimulation on Cognitive Training Outcome in Healthy Aging
Brambilla, M., Dinkelbach, L., Bigler, A., Williams, J., Zokaei, N., Kadosh, R. C., & Brem, A. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.625359 · Cited by 18
Paludan‐Müller, A. S., Créquit, P., & Boutron, I. (2021) · BMC Medicine · doi:10.1186/s12916-021-01955-0 · Cited by 21
“A lack of transparency or missing information on harms is common [3, 5].”
BackgroundSurian, D., Dunn, A. G., Orenstein, L., Bashir, R., Coiera, E. W., & Bourgeois, F. T. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.01.008 · Cited by 17
“About half of all trials remain unpublished two years after trial completion, and of those that are published, around half have missing or changed outcomes [12,13].”
BackgroundSelective primary outcome reporting in high-impact journals of anaesthesia and pain
Bonnot, B., Yavchitz, A., Mantz, J., Paugam-Burtz, C., & Boutron, I. (2016) · British Journal of Anaesthesia · doi:10.1093/bja/aew280 · Cited by 17
Weston, J., Dwan, K., Altman, D., Clarke, M., Gamble, C., Schroter, S., Williamson, P., & Kirkham, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010075 · Cited by 16
“When comparing trial publications with protocols, it was found that 40–62% of studies had at least one primary outcome that was changed, introduced or omitted.(4) Previous qualitative research in this field has found that the prevalence of incomplete outcome reporting is high and researchers were generally unaware of…”
BackgroundResultSelf-citationMorgan, B., Hejdenberg, J., Kuleszewicz, K., Armstrong, D., & Ziebland, S. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00931-y · Cited by 16
Mehrabi, F., & Ghezelbash, A. (2025) · The Gerontologist · doi:10.1093/geront/gnaf219 · Cited by 21
Alqaidoom, Z., Nguyen, P., Awadh, M., & Page, M. J. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1583 · Cited by 18
“Publication bias and selective non-reporting bias are two major threats to the validity of systematic reviews and meta-analyses of randomised controlled trials.(1) Publication bias occurs when journals tend to accept trials based on the statistical significance, magnitude or direction of effect estimates.”
BackgroundMaternal sensitivity, early interactions and infant attachment: a systematic review
Montealegre-Ramón, M. D. P., Martínez-Fuentes, M. T., López, J. P., & García, P. S. (2024) · Early Child Development and Care · doi:10.1080/03004430.2024.2386324 · Cited by 19
“There is evidence that results with statistically signi fi - cant and positive data (supporting the hypothesis) are more likely to be published (Dwan, Gamble, Williamson, Kirkham, & Reporting Bias Group, 2013).”
Chen, D. C., Arnold, K., Sukhdeo, R., Alla, J. F., & Raman, S. (2025) · BMJ Oncology · doi:10.1136/bmjonc-2025-000733 · Cited by 16
SALVAGING PSYCHOTHERAPY RESEARCH: A MANIFESTO
Coyne, J. C., & Kok, R. N. (2014) · Journal of Evidence-Based Psychotherapies · Cited by 19
Evoniuk, G. E., Mansi, B. A., DeCastro, B., & Sykes, J. A. (2017) · BMJ · doi:10.1136/bmj.j1726 · Cited by 16
Brosens, L., Robinet, J., Pelckmans, I., Ameijeiras‐Mariño, Y., Govers, G., Opfergelt, S., Minella, J. P. G., & Vanderborght, J. (2021) · CATENA · doi:10.1016/j.catena.2021.105698 · Cited by 17
Turner, L. G., Munsie, M., Levine, A. D., & Ikonomou, L. (2021) · Stem Cell Reports · doi:10.1016/j.stemcr.2021.09.005 · Cited by 16
“‘‘Negative’’ results also need to be disclosed, even though researchers sometimes encounter pressures to publish ‘‘positive’’ results and not report that particular investigational products are unsafe or ineffective (Dwan et al., 2013).”
BackgroundSelective outcome reporting and sponsorship in randomized controlled trials in IVF and ICSI
Braakhekke, M., Scholten, I., Mol, F., Limpens, J., Mol, B. W., & van der Veen, F. (2017) · Human Reproduction · doi:10.1093/humrep/dex273 · Cited by 18
Retailing science: genre hybridization in online science news stories
Zhang, Y. (2018) · Text and Talk · doi:10.1515/text-2017-0040 · Cited by 17
Cherpak, L. A., Korevaar, D. A., McGrath, T. A., Dang, W., Walker, D., Salameh, J., Sharifabadi, A. D., & McInnes, M. D. F. (2019) · Radiology · doi:10.1148/radiol.2019182206 · Cited by 17
Peng, S., Liu, S., Lei, J., Ren, W., Xiao, L., Liu, X., Lü, M., & Zhou, K. (2022) · PLoS ONE · doi:10.1371/journal.pone.0266780 · Cited by 16
Psychology’s Replication Crisis and Clinical Psychological SciencePreprint
Tackett, J. L., Brandes, C. M., King, K. M., & Markon, K. E. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/kc8xt · Cited by 16
Addressing selective reporting of experiments through predefined exclusion criteria
Neves, K., & Amaral, O. B. (2020) · eLife · doi:10.7554/elife.56626 · Cited by 15
More confident, less formal: stylistic changes in academic psychology writing from 1970 to 2016
Wheeler, M. A., Vylomova, E., McGrath, M. J., & Haslam, N. (2021) · Scientometrics · doi:10.1007/s11192-021-04166-9 · Cited by 15
Miskovic‐Wheatley, J., Vatter, S., Ong, S. H., Carroll, B., Touyz, S. W., Griffiths, K. R., & Maguire, S. (2025) · International Journal of Eating Disorders · doi:10.1002/eat.24503 · Cited by 19
Tan, Z., Tanner‐Smith, E. E., Walters, S. T., Tan, L., Huh, D., Zhou, Z., Luningham, J. M., Larimer, M. E., & Mun, E. (2023) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.15126 · Cited by 21
Berger, V. W. (2014) · Science and Engineering Ethics · doi:10.1007/s11948-014-9576-2 · Cited by 15
“The abuses are well documented, and would fill an encyclopedia, so it is not possible to be comprehensive here, but one recent example is the overwhelming propensity of researchers to modify the primary outcomes between the protocol stage and the final report stage (Dwan et al., 2013).”
Xie, S., & Mi, C. (2023) · Learned Publishing · doi:10.1002/leap.1515 · Cited by 15
“…of positive words in high ranking journals is possibly associated with citation bias (Gøtzsche, 2022), which is first nested within the context of the publication bias and outcome report bias (Dwan et al., 2013) and then lead to linguistic positivity bias in academic publications (Wen & Lei, 2022).”
BackgroundInfluentialInverse publication reporting bias favouring null, negative results
Ioannidis, J. P. A. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112292 · Cited by 16
Massoudi, B. L., & Chester, K. (2017) · Yearbook of Medical Informatics · doi:10.1055/s-0037-1606510 · Cited by 18
Salandra, R., Criscuolo, P., & Salter, A. J. (2020) · Research Policy · doi:10.1016/j.respol.2020.104130 · Cited by 16
Counting What Counts: The Case for Harmonized Outcomes in Child and Youth Mental Health Research
Szatmári, P., Offringa, M., Butcher, N. J., & Monga, S. (2019) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2019.02.016 · Cited by 17
Xiong, X., Zhang, S., Liao, X., Du, J.-J., Zheng, W., Hu, S., Wei, Q., & Yang, L. (2024) · Environmental Pollution · doi:10.1016/j.envpol.2024.123531 · Cited by 12
Forrester, E. J., Henke, J., Knab, J., & Cole, R. P. (2025) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2024.12.002 · Cited by 14
Guidance for reporting outcomes in clinical trials: scoping review protocol
Butcher, N. J., Mew, E. J., Saeed, L. H., Monsour, A., Chee-A-Tow, A., Chan, A., Moher, D., & Offringa, M. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-023001 · Cited by 16
“Notably, 40%-60% of trials have been found to have at least one primary outcome that was changed, introduced or omitted between protocol and publication.(7) Without clear and complete reporting of trial outcomes, researchers cannot adequately appraise, consolidate or replicate findings, hindering the translation of ev…”
BackgroundQuinn, E., Hsiao, K. H., Maitland-Scott, I., Gomez, M., Baysari, M. T., Najjar, Z., & Gupta, L. (2020) · JMIR Public Health and Surveillance · doi:10.2196/24330 · Cited by 15
Registered Reports in Special Education: Introduction to the Special Series
Cook, B. G., Maggin, D. M., & Robertson, R. E. (2021) · Remedial and Special Education · doi:10.1177/0741932521996459 · Cited by 14
“Moreover, evidence of publication bias and scarcity of published null findings has been documented in many fields (Dwan et al., 2013; Fanelli, 2012; Polanin et al., 2016), including special education (Chow & Ekholm, 2018; Therrien & Cook, 2018; Gage et al., 2017).”
BackgroundSupporting study registration to reduce research waste
Purgar, M., Glasziou, P., Klanjšček, T., Nakagawa, S., & Čulina, A. (2024) · Nature Ecology & Evolution · doi:10.1038/s41559-024-02433-5 · Cited by 17
“OSF might thus be among the options for preregistration in ecology.”
Hopewell, S., Witt, C. M., Linde, K., Icke, K., Adedire, O., Kirtley, S., & Altman, D. G. (2018) · Trials · doi:10.1186/s13063-017-2395-4 · Cited by 15
“A number of studies have looked at the selective reporting of primary outcomes [11] or switching of outcomes from either the trial registry [17–19], or trial protocol [20] and the published manuscript.”
BackgroundJia, P., Tang, L., Yu, J., Lee, A. H., Zhou, X., Kang, D., Luo, Y., Liu, J., & Sun, X. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019847 · Cited by 17
Differences in the placebo response in duloxetine and venlafaxine trials
Breilmann, J., Furukawa, T. A., Becker, T. M., & Koesters, M. (2018) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12881 · Cited by 14
Metadata
- Authors
- Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group
- Affiliations
- University of Liverpool (GB)
- Corresponding author
- Dwan, Kerry
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-05
- Volume
- 8
- Issue
- 7
- Pages
- e66844
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3702538
- OpenAlex
- W2002062153
- Cited by
- 1,127
- References
- 69
- Citations by year
- 2026: 75 · 2025: 84 · 2024: 64 · 2023: 71 · 2022: 127 · 2021: 98 · 2020: 112 · 2019: 89 · 2018: 83 · 2017: 102 · 2016: 89 · 2015: 70 · 2014: 57 · 2013: 5
- FWCI
- 15.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- SERVAL (Université de Lausanne) (publishedVersion, cc-by)
- IRIS (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Tufts Medical Center
- King's College London
- University of Oxford
- University of Bristol
- University of Sydney
- Queen's University
- Women's College Hospital
- Centre Hospitalier Universitaire Vaudois
- Hospices Civils de Lyon
- Moorfields Eye Hospital NHS Foundation Trust
- School of Medicine, Stanford University
- Tufts University School of Medicine
- Medical Research Council
- National Health and Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- publication bias, selective reporting, randomized controlled trials, meta-analysis validity, trial reporting quality, primary outcome reporting
- MeSH terms
- Humans, Publications, Cohort Studies, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.