Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review
Open accessThe DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


![Figure 3 Status of approved protocols for Chan 2004b study [17].](https://figures.cochat.ai/paper-figures/83600bbc5644401caed195b486efac8f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-3.jpg)
![Figure 4 Status of approved protocols for Easterbrook 1991 study [24].](https://figures.cochat.ai/paper-figures/0a114dad26924772826758941018f42f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-4.jpg)
![Figure 5 Status of approved protocols for Dickersin 1992 study [23].](https://figures.cochat.ai/paper-figures/1453f80dcd014ca3b9b32399777cc8d3/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-5.jpg)
![Figure 6 Status of approved protocols for Dickersin 1993 study [3].](https://figures.cochat.ai/paper-figures/1e175d9e8b2a4f0390577db97b820ca8/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-6.jpg)
![Figure 7 Status of approved protocols for Stern 1997 study [4].](https://figures.cochat.ai/paper-figures/876f2f4683aa40a689e73241856ecb1f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-7.jpg)
![Figure 8 Status of approved protocols for Cooper 1997 study [19].](https://figures.cochat.ai/paper-figures/beceb71fe2a943b2ba35a9fc9d17293b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-8.jpg)
![Figure 9 Status of trials for Wormald 1997 study [30].](https://figures.cochat.ai/paper-figures/f6a16e454252431f84d6cc28737b2e50/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-9.jpg)
![Figure 10 Status of approved protocols for Ioannidis 1998 study [5].](https://figures.cochat.ai/paper-figures/261662c83e664841aaa276cbc7c62394/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-10.jpg)
![Figure 11 Status of approved protocols for Pich 2003 study [27].](https://figures.cochat.ai/paper-figures/48fe85db48fc4b6ca22f25bb5580c0a7/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-11.jpg)
![Figure 12 Status of approved protocols for Cronin 2004 study [20].](https://figures.cochat.ai/paper-figures/8d211a89740e4b67b0c80783ab9c0582/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-12.jpg)
![Figure 13 Status of approved protocols for Decullier 2005 study [7].](https://figures.cochat.ai/paper-figures/c7868be91276448192f71187c860c7a5/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-13.jpg)
![Figure 14 Status of approved protocols for Decullier 2006 study [22].](https://figures.cochat.ai/paper-figures/4859f8920e714475a966997babd4cb7a/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-14.jpg)
![Figure 15 Status of approved protocols for Hahn 2002 study [26].](https://figures.cochat.ai/paper-figures/69f971391f504ed4bae3d24d4a05f2c1/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-15.jpg)
![Figure 16 Status of approved protocols for Chan 2004a study [18].](https://figures.cochat.ai/paper-figures/933f83ab86634ea581351235ceea1e11/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-16.jpg)
![Figure 17 Status of approved protocols for Ghersi 2006 study [25].](https://figures.cochat.ai/paper-figures/97881d16700f4ed48102dec8fc05b22b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-17.jpg)
![Figure 18 Status of approved protocols for von Elm 2008 study [29].](https://figures.cochat.ai/paper-figures/90d1c2ab102849719682b2f8f8409206/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-18.jpg)
![Figure 19 Status of approved protocols for Turer 2007 study [28].](https://figures.cochat.ai/paper-figures/ef99b99be5ff4105970df8c239918ce4/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-19.jpg)
![Figure 20 Status of approved protocols for De Jong 2010 study [21].](https://figures.cochat.ai/paper-figures/bf16aa205a3c4592aa0e050025754624/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-20.jpg)
![Figure 21 Status of approved protocols for Blumle 2008 study [31].](https://figures.cochat.ai/paper-figures/f70bc7510f114d9ba3b2e1e1e342ba98/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-21.jpg)
![Figure 22 Status of approved protocols for Hall 2007 study [32].](https://figures.cochat.ai/paper-figures/577511f26e0441a6bbc7643a3998c183/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-22.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinical trial registries provide the ability to measure biases in the reporting of clinical trials that arise due to non-publication, delayed publication, or incomplete publication of results [14].”
From A systematic review of the processes used to link clinical trial registrations to their published results (2017) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many publications of randomized clinical trials of hypnosis interventions have been unclear or incomplete in reporting of the design elements that are necessary for the evaluation of the quality of evidence, 13 or have not provided sufficient details for possible direct replication.14,15”
From Best Practice Recommendations for Conducting and Reporting Controlled Trials in Clinical Hypnosis Research (2024) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“studies with statistically non-significant results may not have been submitted for publication (publication bias), or particular results that were statistically non-significant may have been omitted from study reports (selective non-reporting bias).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [83]
The cited source says what the claim attributes to it.
In the citing paper
“Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias.”
From Systematic review protocol assessing the processes for linking clinical trial registries and their published results (2016) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Despite providing important information relevant to patients, trials with statistically non-significant results are more prone to non-publication,114 or, when published, to a long delay to publication114 115 116—often referred to as publication bias.”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [116]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 12×
“Easterbrook et al [24] found that compared with unfunded studies, government funded studies were more likely to yield statistically significant results but government sponsorship was not found to have a statistically significant effect on the likelihood of publication and company sponsored trials were less likely to b…”
BackgroundMethodResultInfluentialSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · cited 1×
“Several studies investigated a cohort of trials submitted to drug licensing authorities [38,40,41,65] and all found that many of these trials remain unpublished, with one study demonstrating that trials with positive outcomes resulted more often in submission of a final report to the regulatory authority [40].”
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,412
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · cited 2×
“While much effort has been invested in trying to identify the former [2], it is equally important to understand the nature and frequency of missing data from the latter level.”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 29×
“The main objective of the study by Blumle et al [31] was to consider how eligibility criteria stated in protocols was reported in subsequent reports, in doing so they noted that 52% of studies in their cohort were published, decreasing to 48% for RCTs only.”
BackgroundMethodInfluentialSystematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 6×
“Since the original version of this review was published [13], a Cochrane methodology review is now available on publication bias [64].”
BackgroundMethodInfluentialSelf-citationPublication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · cited 2×
“While much effort has been invested in trying to identify the former [1,2], it is equally important to understand the nature and frequency of missing data from the latter level.”
Background
Research works printed but not linked: 2
- Menzel S, Uebing B, Hucklenbroich P, Schober O (2007) Evaluation of clinical trials following an approval from a research ethics committee. Dtsch Med Wochenschr 132: 2313– 2317. 17957593
- Chappell L, Alfirevic Z, Chien P, Jarvis S, Thornton JG (2005) A comparison of the published version of randomized controlled trials in a specialist clinical journal with the original trial protocols; Chicago.
Cited by
Items 301–400 of 1,001
Phlebitis in Intravenous Amiodarone Administration: Incidence and Contributing Factors
Oragano, C. A., Patton, D., & Moore, Z. E. H. (2019) · Critical Care Nurse · doi:10.4037/ccn2019381 · Cited by 39
Frantsve‐Hawley, J., Abt, E., Carrasco‐Labra, A., Dawson, T., Michaels, M., Pahlke, S. C., Rindal, D. B., Spallek, H., & Weyant, R. J. (2022) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2022.07.012 · Cited by 39
Understanding and evaluating meta-analysis
Dawson, D. V., Pihlstrom, B. L., & Blanchette, D. R. (2015) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2015.10.023 · Cited by 36
Neamah, H. H., Kuhlmann, A. S., & Tabak, R. G. (2016) · The Science of Diabetes Self-Management and Care · doi:10.1177/0145721716630386 · Cited by 38
TARG Meta-Research Group & Collaborators (2023) · BMJ Open · doi:10.1136/bmjopen-2023-076264 · Cited by 33
Core outcomes measures in dental computer vision studies (DentalCOMS)
Büttner, M., Rokhshad, R., Brinz, J., Issa, J., Chaurasia, A., Uribe, S. E., Karteva, T., Chala, S., Tichý, A., & Schwendicke, F. (2024) · Journal of Dentistry · doi:10.1016/j.jdent.2024.105318 · Cited by 33
Time to publication for results of clinical trials
Showell, M. G., Cole, S., Clarke, M. J., DeVito, N., Farquhar, C. M., & Jordan, V. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000011.pub3 · Cited by 57
Primary outcome reporting in adolescent depression clinical trials needs standardization
Monsour, A., Mew, E. J., Patel, S., Chee-A-Tow, A., Saeed, L., Santos, L., Courtney, D., Watson, P., Monga, S., Szatmári, P., et al. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01019-6 · Cited by 33
The Curious Case of Centrality Measures: A Large-Scale Empirical Investigation
Saqr, M., & López-Pernas, S. (2022) · Journal of Learning Analytics · doi:10.18608/jla.2022.7415 · Cited by 32
“Such an environment makes researchers more likely to submit positive findings knowing that they may receive preferential treatment (Dwan et al., 2013; Joober et al., 2012).”
BackgroundAl-Durra, M., Nolan, R. P., Seto, E., Cafazzo, J. A., & Eysenbach, G. (2018) · Journal of Medical Internet Research · doi:10.2196/11924 · Cited by 32
“Our finding is substantially different and should not be compared to findings from other studies that reported that 40%–62% of clinical trials had at least one change in primary outcome when comparing trial publications and protocols [12,13,15].”
McKenzie, J. E., Taljaard, M., Hemming, K., Arnup, S. J., Giraudeau, B., Eldridge, S. M., Hooper, R., Kahan, B. C., Li, T., Moher, D., et al. (2025) · BMJ · doi:10.1136/bmj-2024-080472 · Cited by 36
Ioannidis, J. P. A., Horbar, J. D., Ovelman, C., Brosseau, Y., Thorlund, K., Buus‐Frank, M. E., Mills, E. J., & Soll, R. (2015) · BMJ · doi:10.1136/bmj.h72 · Cited by 34
Valkenburg, C., Slot, D. E., & van der Weijden, G. A. (2019) · International Journal of Dental Hygiene · doi:10.1111/idh.12423 · Cited by 38
Condon, T. M., Sexton, R. W., Wells, A. J., & To, M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237879 · Cited by 33
Messaging in Biological Psychiatry: Misrepresentations, Their Causes, and Potential Consequences
Dumas-Mallet, E., & Gonon, F. (2020) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000276 · Cited by 35
Littlewood, N. A., Rocha, R., Smith, R. K., Martin, P. A., Lockhart, S. L., & Schoonover, R. F. (2020) · Open Book Publishers · doi:10.11647/obp.0234 · Cited by 37
Liu, W., Wang, T., Zhu, M., & Jin, G. (2023) · Nutrients · doi:10.3390/nu15061344 · Cited by 38
Liberati, G., Hünefeldt, T., & Belardinelli, M. O. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00865 · Cited by 34
Integrative Review of Nurse-Delivered Physical Activity Interventions in Primary Care
Richards, E. A., & Cai, Y. (2015) · Western Journal of Nursing Research · doi:10.1177/0193945915581861 · Cited by 35
de Vries, Y. A., Roest, A. M., Beijers, L., Turner, E. H., & de Jonge, P. (2016) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2016.09.370 · Cited by 34
Latifi‐Navid, H., Latifi‐Navid, S., Mostafaiy, B., Jamalkandi, S. A., & Ahmadi, A. (2018) · Scientific Reports · doi:10.1038/s41598-018-29280-2 · Cited by 34
Perlmutter, A. S., Tran, V.-T., Dechartres, A., & Ravaud, P. (2017) · Annals of Oncology · doi:10.1093/annonc/mdw682 · Cited by 31
“Studies usually compared publications with (i) protocols available from ethics committees [3, 10, 11], (ii) protocols publicly available as supplemental material from journals [2] or (iii) public clinical-trial registry entries [4, 6, 12].”
BackgroundMoore, T. H., Higgins, J., & Dwan, K. (2023) · Cochrane evidence synthesis and methods · doi:10.1002/cesm.12031 · Cited by 34
“For more information see section 8.3.2 of the detailed guidance [14] and a review on bias in the selection of the result [20].”
InfluentialSelf-citationWuytack, F., Smith, V. A., Clarke, M. J., Williamson, P. R., & Gargon, E. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0060-0 · Cited by 31
“Outcome reporting bias, that is, the selection of reported outcomes based on the outcome results, may significantly impact on the findings of studies and Cochrane reviews [1].”
BackgroundSelf-citationHuber, H. F., Jenkins, S. L., Li, C., & Nathanielsz, P. W. (2019) · Journal of Developmental Origins of Health and Disease · doi:10.1017/s2040174419000539 · Cited by 32
Gonçalves, M. C. P., Romanelli, J. P., Guimarães, J. R., Vieira, A. C., de Azevedo, B. P., & Tardioli, P. W. (2021) · Critical Reviews in Biotechnology · doi:10.1080/07388551.2021.1888071 · Cited by 31
Wu, W. K., Grogan, W. M., Ziogas, I. A., Patel, Y. J., Bacchetta, M. D., & Alexopoulos, S. P. (2022) · Transplantation Reviews · doi:10.1016/j.trre.2022.100693 · Cited by 31
Jamjoom, A. A. B., Gane, A. B., & Demetriades, A. K. (2016) · Journal of neurosurgery · doi:10.3171/2016.8.jns16765 · Cited by 31
Wiewel, E. W., Borrell, L. N., Jones, H. E., Maroko, A., & Torian, L. V. (2017) · AIDS and Behavior · doi:10.1007/s10461-017-1700-2 · Cited by 34
“Finally, publication bias may have limited the publication of previous null findings on neighborhood characteristics and HIV outcomes [55, 56].”
BackgroundCho, Y., Sautenet, B., Rangan, G. K., Craig, J. C., Ong, A. C. M., Chapman, A. B., Ahn, C., Chen, D., Coolican, H., Kao, J. T., et al. (2017) · Trials · doi:10.1186/s13063-017-2298-4 · Cited by 30
The ethics of reporting all the results of clinical trials
Brassington, I. (2017) · British Medical Bulletin · doi:10.1093/bmb/ldw058 · Cited by 30
“Though some have claimed to be unable to find compelling evidence of this phenomenon,(2) others have found that it is real.(3,4) To whatever extent that it does occur, several sources insist that the non-publication of data is in some sense ‘unethical’, or that publication of all results is an ethical imperative.”
BackgroundHenry, A., Alphonse, J., Tynan, D., & Welsh, A. W. (2018) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.17476 · Cited by 34
“44 Australia 28 (54) 29 (56) Overseas 24 (46) 23 (44) Smoking during pregnancy 4 (8) 2 (4) 0.”
Shinohara, K., Tajika, A., Imai, H., Takeshima, N., Hayasaka, Y., & Furukawa, T. A. (2015) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12502 · Cited by 29
“A recent systematic review concluded 40–62% of studies had primary outcome discrepancies between the protocols and published reports (21).”
Sundin, J., Amcoff, M., Mateos‐González, F., Raby, G. D., Jutfelt, F., & Clark, T. D. (2017) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-017-2337-x · Cited by 33
“, the observation that studies reporting negative results are typically more difficult to publish) (Browman 1999; Dwan et al. 2013).”
Won, J., Kim, S.-Y., Bae, I., & Lee, H. (2019) · PLoS ONE · doi:10.1371/journal.pone.0223305 · Cited by 31
Bias and Groupthink in Science’s Peer-Review System
Resnik, D. B., & Smith, E. M. R. (2020) · Springer eBooks · doi:10.1007/978-3-030-36822-7_9 · Cited by 35
Stoll, M., Mancini, A. L., Hubenschmid, L., Dreimüller, N., König, J., Cuijpers, P. J., Barth, J., & Lieb, K. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.08.013 · Cited by 31
Eijgermans, D. G. M., Fang, Y., Jansen, D. E. M. C., Bramer, W. M., Raat, H., & Jansen, W. (2021) · Children and Youth Services Review · doi:10.1016/j.childyouth.2021.106288 · Cited by 32
Small Study Effects in Diagnostic Imaging Accuracy: A Meta-Analysis
Lu, L., Phua, Q. S., Bacchi, S., Goh, R., Gupta, A. K., Kovoor, J. G., Ovenden, C. D., & To, M. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.28776 · Cited by 33
Baudart, M., Ravaud, P., Baron, G., Dechartres, A., Haneef, R., & Boutron, I. (2016) · BMC Medicine · doi:10.1186/s12916-016-0551-4 · Cited by 32
“gov [5, 6, 21–24].”
BackgroundCanestaro, W. J., Hendrix, N., Bansal, A., Sullivan, S. D., Devine, E. B., & Carlson, J. J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.004 · Cited by 32
Bechini, A., Lorini, C., Zanobini, P., Tacconi, F. M., Boccalini, S., Grazzini, M., Bonanni, P., & Bonaccorsi, G. (2020) · Vaccines · doi:10.3390/vaccines8020165 · Cited by 31
Enzyme-inducing antiepileptic drugs and fractures in people with epilepsy: A systematic review
Fraser, L.-A., Burneo, J. G., & Fraser, J. A. (2015) · Epilepsy Research · doi:10.1016/j.eplepsyres.2015.07.003 · Cited by 29
Dudda, L., Kormann, E., Kozula, M., DeVito, N. J., Klebel, T., Dewi, A. P. M., Spijker, R., Stegeman, I., Van den Eynden, V., Ross-Hellauer, T., et al. (2025) · Royal Society Open Science · doi:10.1098/rsos.242057 · Cited by 28
de Vries, Y. A., Roest, A. M., Turner, E. H., & de Jonge, P. (2019) · Psychological Medicine · doi:10.1017/s0033291718002805 · Cited by 28
“, 2015), medicine (McGauran et al., 2010; Hart et al., 2012; Dwan et al., 2013), and science in general (Fanelli, 2012).”
The commercialization of the biomedical sciences: (mis)understanding bias
de Melo‐Martín, I. (2019) · History & Philosophy of the Life Sciences · doi:10.1007/s40656-019-0274-x · Cited by 30
Ferran, J.-M., & Nevitt, S. J. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0836-3 · Cited by 28
Bonafiglia, J. T., Islam, H., Preobrazenski, N., & Gurd, B. J. (2021) · Journal of sport and health science/Journal of Sport and Health Science · doi:10.1016/j.jshs.2021.03.005 · Cited by 30
Machielsen, A. J. H. M., Iqbal, N., Kimman, M., Sahnan, K., Adegbola, S. O., Kane, G., Woodcock, R., Kleijnen, J., Grossi, U., Breukink, S. O., et al. (2021) · Techniques in Coloproctology · doi:10.1007/s10151-021-02452-5 · Cited by 33
“Furthermore, selective reporting of outcomes based on significant results is a recognised problem and can overestimate the size of the treatment effect [8, 9].”
BackgroundLittell, J. H., Pigott, T., Nilsen, W., Roberts, J., & Labrum, T. (2023) · Campbell Systematic Reviews · doi:10.1002/cl2.1324 · Cited by 33
Acetyl-L-carnitine for patients with hepatic encephalopathy
Martí-Carvajal, A. J., Gluud, C. N., & Arévalo-Rodríguez, I. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011451 · Cited by 30
Registration status and outcome reporting of trials published in core headache medicine journals
Rayhill, M. L., Sharon, R., Burch, R. C., & Loder, E. (2015) · Neurology · doi:10.1212/wnl.0000000000002127 · Cited by 28
Saldanha, I. J., Li, T., Yang, P., Owczarzak, J., Williamson, P. R., & Dickersin, K. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.02.005 · Cited by 31
Self-citationLeviton, A. (2018) · Nutrients · doi:10.3390/nu10091152 · Cited by 27
“This appears to happen most commonly when reviewers and editors view “positive” findings as more attractive for publication than “negative” (or non-significant) findings [150].”
BackgroundBorysowski, J., Wnukiewicz‐Kozłowska, A., & Górski, A. (2020) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.14223 · Cited by 32
“This requirement is very important because it is known that risk of nonpublication is higher for studies with negative results.(70) To increase”
BackgroundLiao, Z., Fang, Z., Gou, S., Luo, Y., Liu, Y., He, Z., Li, X., Peng, Y., Fu, Z., Li, D., et al. (2022) · BMC Medicine · doi:10.1186/s12916-021-02229-5 · Cited by 30
Korevaar, D. A., Salameh, J., Vali, Y., Cohen, J. F., McInnes, M. D. F., Spijker, R., & Bossuyt, P. M. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1389 · Cited by 27
Does ‘playtime’ reduce stimulus-seeking and other boredom-like behaviour in laboratory ferrets?
Burn, C. C., Raffle, J., & Bizley, J. K. (2020) · Animal Welfare · doi:10.7120/09627286.29.1.019 · Cited by 26
Akomea-Frimpong, I., Jin, X., Osei‐Kyei, R., & Pariafsai, F. (2023) · Journal of Financial Management of Property and Construction · doi:10.1108/jfmpc-12-2021-0070 · Cited by 25
Soils as Carbon Stores and Sinks: Expectations, Patterns, Processes, and Prospects of Transitions
van Noordwijk, M., Aynekulu, E., Hijbeek, R., Milne, E., Minasny, B., & Saputra, D. D. (2023) · Annual Review of Environment and Resources · doi:10.1146/annurev-environ-112621-083121 · Cited by 37
Alayche, M., Cobey, K. D., Ng, J. Y., Ardern, C. L., Khan, K., Chan, A., Chow, R., Masalkhi, M., Ayala, A. P., Ebrahimzadeh, S., et al. (2023) · FACETS · doi:10.1139/facets-2022-0208 · Cited by 29
Trends in Heart Failure Clinical Trials From 2001–2012
Tahhan, A. S., Vaduganathan, M., Kelkar, A. A., Georgiopoulou, V. V., Kalogeropoulos, A. P., Greene, S. J., Fonarow, G. C., Gheorghiade, M., & Butler, J. (2015) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2015.06.014 · Cited by 27
Johnston, B. C., Miller, P. A., Agarwal, A., Mulla, S. M., Khokhar, R., Oliveira, K. D., Hitchcock, C. L., Sadeghirad, B., Mohiuddin, M., Sekercioglu, N., et al. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.06.010 · Cited by 29
Contacting of authors by systematic reviewers: protocol for a cross-sectional study and a survey
Reynders, R. M., Ladu, L., & Girolamo, N. D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0643-z · Cited by 29
Standards for design and measurement would make clinical research reproducible and usable
Dickersin, K., & Mayo‐Wilson, E. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708273114 · Cited by 27
Steinfath, M., Vogl, S., Violet, N., Schwarz, F., Mielke, H., Selhorst, T., Greiner, M., & Schönfelder, G. (2018) · PLoS ONE · doi:10.1371/journal.pone.0202762 · Cited by 26
Witt, K., Townsend, E., Arensman, E., Gunnell, D., Hazell, P., Salisbury, T. T., van Heeringen, K., & Hawton, K. E. (2019) · Archives of Suicide Research · doi:10.1080/13811118.2019.1592043 · Cited by 27
George, J. M., Pannek, K., Rose, S., Ware, R. S., Colditz, P. B., & Boyd, R. N. (2017) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.13611 · Cited by 26
Comparison of reporting phase I trial results in ClinicalTrials.gov and matched publications
Shepshelovich, D., Goldvaser, H., Wang, L., Razak, A. R. A., & Bedard, P. L. (2017) · Investigational New Drugs · doi:10.1007/s10637-017-0510-8 · Cited by 25
Wild Listening: Ecology of a Science Podcast
Barrios-O’Neill, D. (2018) · Springer eBooks · doi:10.1007/978-3-319-90056-8_8 · Cited by 25
West, N. X., He, T., Zou, Y., DiGennaro, J., Biesbrock, A. R., & Davies, M. (2020) · Journal of Dentistry · doi:10.1016/j.jdent.2020.103566 · Cited by 27
Mayo‐Wilson, E., Grant, S., & Supplee, L. H. (2021) · Prevention Science · doi:10.1007/s11121-021-01284-x · Cited by 27
“Many studies have shown that journal articles tend to overestimate intervention effects and underestimate their harms (Dwan et al., 2013; Song et al., 2010).”
BackgroundLiu, X., & Zhu, H. (2023) · Scientometrics · doi:10.1007/s11192-023-04679-5 · Cited by 26
“That is, studies with positive or significant results have higher odds of being published (Dwan et al., 2013), while descriptions of negative results such as “nonsignificant difference’’ have declined in article abstracts (Fanelli, 2012; Pautasso, 2010).”
ResultThe standards of reporting trials in pets (PetSORT): Explanation and elaboration
Sargeant, J. M., Ruple, A., Selmic, L. E., & O'Connor, A. M. C. (2023) · Frontiers in Veterinary Science · doi:10.3389/fvets.2023.1137781 · Cited by 27
Publication Bias & Small-study Effects in Pediatric Dentistry Meta-analyses
Papageorgiou, S. N., Dimitraki, D., Coolidge, T., & Kotsanos, N. (2014) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2014.09.001 · Cited by 28
The use of systemic antibiotics in the treatment of refractory periodontitis
Santos, R. S., Macedo, R. F., Albuquerque‐Souza, E., Soares, R. S. C., Feitosa, D. D. S., & Sarmento, C. (2016) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2016.02.013 · Cited by 28
Transcranial Random Noise Stimulation Does Not Enhance the Effects of Working Memory Training
Holmes, J., Byrne, E. M., Gathercole, S. E., & Ewbank, M. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00993 · Cited by 32
“Their history also shows that they are marked by high levels of early positive results that are typically not sustained over longer periods, probably because of publication bias (Dwan, Gamble, Williamson, & Kirkham, 2013; Scherer, Langenberg, & von Elm, 2007).”
Blümle, A., Schandelmaier, S., Oeller, P., Kasenda, B., Briel, M., von Elm, E. B., & on behalf of the DISCO study group (2016) · PLoS ONE · doi:10.1371/journal.pone.0165605 · Cited by 27
van Hoorn, C. E., Hoeks, S. E., Essink, H. M., Tibboel, D., & de Graaff, J. C. (2018) · Pediatric Anesthesia · doi:10.1111/pan.13553 · Cited by 26
English studies: Decolonisation, deparochialising knowledge and the null curriculum
Chaka, C., Lephalala, M., & Ngesi, N. (2017) · Perspectives in Education · doi:10.18820/2519593x/pie.v35i2.16 · Cited by 32
Do not make clinical decisions based on abstracts of healthcare research: A systematic review
Nascimento, D. P., Ostelo, R., van Tulder, M. W., Gonzalez, G. Z., Araújo, A. C., Vanin, A. A., & Costa, L. O. P. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.030 · Cited by 30
Thomson, H. M., Labuschagne, I., Greenwood, L., Robinson, E., Sehl, H., Suo, C., & Lorenzetti, V. (2021) · Psychopharmacology · doi:10.1007/s00213-021-05938-0 · Cited by 27
“, positive findings are more likely to be published and have a higher chance of being reported) and consequently over-represent positive findings (Dwan et al. 2013).”
BackgroundMuradchanian, J., Hoekstra, R., Kiers, H. A. L., & van Ravenzwaaij, D. (2023) · PLoS ONE · doi:10.1371/journal.pone.0292279 · Cited by 26
“Publication bias has been reported in a variety of disciplines within social sciences (including economics [1, 3, 4, 6, 10]) and medical sciences [1, 7, 11, 12].”
BackgroundCommon Mistakes in Statistical and Methodological Practices of Sport Management Research
Kim, Y., & Lee, J. L. (2018) · Measurement in Physical Education and Exercise Science · doi:10.1080/1091367x.2018.1537278 · Cited by 26
Martin, P. B., Surian, D., Bashir, R., Bourgeois, F. T., & Dunn, A. G. (2019) · JAMIA Open · doi:10.1093/jamiaopen/ooy062 · Cited by 26
“Around half of all clinical trials remain unpublished after 2 years, and of those that are published, around half have missing or changed outcomes.(17) Safety outcomes in particular tend to be left out of published trial reports, introducing a challenge for systematic reviewers wanting to provide a timely and complete…”
BackgroundPage, M. J., Forbes, A., Chau, M., Green, S. E., & McKenzie, J. E. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011863 · Cited by 23
Copas, J., Marson, A., Williamson, P., & Kirkham, J. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280217738546 · Cited by 23
“To date, the risk of bias assessments have only been validated for use in randomised controlled trials for benefit outcomes, and applied in only a handful of non-randomised reports for harm outcomes ([3], [4], [5]).”
Self-citationSchulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. L. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059347 · Cited by 23
“79 p-Hacking Describes the process of analysing the data in multiple ways until statistically significant results are found.”
Fang, Y., Lee, Y., Chan, J., Chiou, P., Chou, X.-Y., Chiu, W.-T., & Hung, C. (2020) · Supportive Care in Cancer · doi:10.1007/s00520-019-05278-y · Cited by 25
Julien, A., Danet, L., Loisel, M., Brauge, D., Pariente, J., Péran, P., & Planton, M. (2022) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2022.07.007 · Cited by 26
Risk of bias in exercise science: A systematic review of 340 studies
Preobrazenski, N., McCaig, A., Turner, A., Kushner, M., Pacitti, L. J., Mendolia, P., MacDonald, B., Storoschuk, K. L., Bouck, T., Zaza, Y., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109010 · Cited by 25
“71 This can lead to an overrepresentation of positive results, creating an inflated perception of effectiveness.”
BackgroundStatistical and methodological considerations for reporting RCTs in medical literature
Lee, S., & Kang, H. (2015) · Korean Journal of Anesthesiology · doi:10.4097/kjae.2015.68.2.106 · Cited by 24
“This also includes cases in which unplanned outcomes are added; or in which unplanned statistics and analyses of sub-categories or sub-groups are applied [6-8].”
BackgroundResultSystematic reviews of therapeutic interventions frequently consider patient-important outcomes
Ameur, H., Ravaud, P., Fayard, F., Riveros, C., & Dechartres, A. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.01.006 · Cited by 23
Open Science and Single-Case Design Research
Cook, B. G., Johnson, A. H., Maggin, D. M., Therrien, W. J., Barton, E. E., Lloyd, J. W., Reichow, B., Talbott, E., & Travers, J. C. (2021) · Remedial and Special Education · doi:10.1177/0741932521996452 · Cited by 23
“Indeed, evidence suggests publication bias—or the over-representation of studies with positive findings in the published literature—exists in many research bases (Dwan et al., 2013; Ferguson & Heene, 2012; Vera-Badillo & Riechelmann, 2018), including in education and special education (Chow & Ekholm, 2018; Gage et al.”
BackgroundFrosi, G., Riley, R. D., Williamson, P. R., & Kirkham, J. J. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.017 · Cited by 27
“Empirical evidence has shown that outcomes which are statistically significant are more likely to be fully reported compared to non-significant outcomes [2].”
BackgroundSelf-citationMaternal adiposity and perinatal and offspring outcomes: an umbrella review
Yang, Z., Feng, G., Gao, X., Yan, X., Li, Y., Wang, Y., Li, S., Jiang, Y., Zhao, S., Zhao, H., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01994-6 · Cited by 22
“Further, the results from individual primary studies may be skewed by residual confounding, excess significance bias and selective reporting, potentially exaggerating the observed effect sizes 16–18 .”
Schneider, A. R., & Hering, D. (2024) · Applied Vegetation Science · doi:10.1111/avsc.70003 · Cited by 22
Levey, C., Innes, N., Schwendicke, F., Lamont, T., & Göstemeyer, G. (2017) · Trials · doi:10.1186/s13063-017-2256-1 · Cited by 27
From registration to publication: A study on Dutch academic randomized controlled trials
Huiskens, J., Kool, B. R. J., Bakker, J.-M., Bruns, E. R. J., de Jonge, S. W., Olthof, P. B., van Rosmalen, B. V., van Gulik, T. M., Hooft, L., & Punt, C. J. A. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1379 · Cited by 25
“VIII 18 5 (28) 14 (78) 10 (56) 13 (72) 1 (6)”
InfluentialVereijken, A. J., Aerts, I., Jetten, J., Tassignon, B., Verschueren, J., MEEUSEN, R. R., & van Trijffel, E. (2020) · VUBIR (Vrije Universiteit Brussel) · Cited by 37
Metadata
- Authors
- Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group
- Affiliations
- University of Liverpool (GB)
- Corresponding author
- Dwan, Kerry
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-05
- Volume
- 8
- Issue
- 7
- Pages
- e66844
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3702538
- OpenAlex
- W2002062153
- Cited by
- 1,127
- References
- 69
- Citations by year
- 2026: 75 · 2025: 84 · 2024: 64 · 2023: 71 · 2022: 127 · 2021: 98 · 2020: 112 · 2019: 89 · 2018: 83 · 2017: 102 · 2016: 89 · 2015: 70 · 2014: 57 · 2013: 5
- FWCI
- 15.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- SERVAL (Université de Lausanne) (publishedVersion, cc-by)
- IRIS (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Tufts Medical Center
- King's College London
- University of Oxford
- University of Bristol
- University of Sydney
- Queen's University
- Women's College Hospital
- Centre Hospitalier Universitaire Vaudois
- Hospices Civils de Lyon
- Moorfields Eye Hospital NHS Foundation Trust
- School of Medicine, Stanford University
- Tufts University School of Medicine
- Medical Research Council
- National Health and Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- publication bias, selective reporting, randomized controlled trials, meta-analysis validity, trial reporting quality, primary outcome reporting
- MeSH terms
- Humans, Publications, Cohort Studies, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.