Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review
Open accessThe DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


![Figure 3 Status of approved protocols for Chan 2004b study [17].](https://figures.cochat.ai/paper-figures/83600bbc5644401caed195b486efac8f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-3.jpg)
![Figure 4 Status of approved protocols for Easterbrook 1991 study [24].](https://figures.cochat.ai/paper-figures/0a114dad26924772826758941018f42f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-4.jpg)
![Figure 5 Status of approved protocols for Dickersin 1992 study [23].](https://figures.cochat.ai/paper-figures/1453f80dcd014ca3b9b32399777cc8d3/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-5.jpg)
![Figure 6 Status of approved protocols for Dickersin 1993 study [3].](https://figures.cochat.ai/paper-figures/1e175d9e8b2a4f0390577db97b820ca8/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-6.jpg)
![Figure 7 Status of approved protocols for Stern 1997 study [4].](https://figures.cochat.ai/paper-figures/876f2f4683aa40a689e73241856ecb1f/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-7.jpg)
![Figure 8 Status of approved protocols for Cooper 1997 study [19].](https://figures.cochat.ai/paper-figures/beceb71fe2a943b2ba35a9fc9d17293b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-8.jpg)
![Figure 9 Status of trials for Wormald 1997 study [30].](https://figures.cochat.ai/paper-figures/f6a16e454252431f84d6cc28737b2e50/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-9.jpg)
![Figure 10 Status of approved protocols for Ioannidis 1998 study [5].](https://figures.cochat.ai/paper-figures/261662c83e664841aaa276cbc7c62394/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-10.jpg)
![Figure 11 Status of approved protocols for Pich 2003 study [27].](https://figures.cochat.ai/paper-figures/48fe85db48fc4b6ca22f25bb5580c0a7/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-11.jpg)
![Figure 12 Status of approved protocols for Cronin 2004 study [20].](https://figures.cochat.ai/paper-figures/8d211a89740e4b67b0c80783ab9c0582/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-12.jpg)
![Figure 13 Status of approved protocols for Decullier 2005 study [7].](https://figures.cochat.ai/paper-figures/c7868be91276448192f71187c860c7a5/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-13.jpg)
![Figure 14 Status of approved protocols for Decullier 2006 study [22].](https://figures.cochat.ai/paper-figures/4859f8920e714475a966997babd4cb7a/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-14.jpg)
![Figure 15 Status of approved protocols for Hahn 2002 study [26].](https://figures.cochat.ai/paper-figures/69f971391f504ed4bae3d24d4a05f2c1/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-15.jpg)
![Figure 16 Status of approved protocols for Chan 2004a study [18].](https://figures.cochat.ai/paper-figures/933f83ab86634ea581351235ceea1e11/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-16.jpg)
![Figure 17 Status of approved protocols for Ghersi 2006 study [25].](https://figures.cochat.ai/paper-figures/97881d16700f4ed48102dec8fc05b22b/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-17.jpg)
![Figure 18 Status of approved protocols for von Elm 2008 study [29].](https://figures.cochat.ai/paper-figures/90d1c2ab102849719682b2f8f8409206/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-18.jpg)
![Figure 19 Status of approved protocols for Turer 2007 study [28].](https://figures.cochat.ai/paper-figures/ef99b99be5ff4105970df8c239918ce4/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-19.jpg)
![Figure 20 Status of approved protocols for De Jong 2010 study [21].](https://figures.cochat.ai/paper-figures/bf16aa205a3c4592aa0e050025754624/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-20.jpg)
![Figure 21 Status of approved protocols for Blumle 2008 study [31].](https://figures.cochat.ai/paper-figures/f70bc7510f114d9ba3b2e1e1e342ba98/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-21.jpg)
![Figure 22 Status of approved protocols for Hall 2007 study [32].](https://figures.cochat.ai/paper-figures/577511f26e0441a6bbc7643a3998c183/systematic-review-of-the-empirical-evidence-of-study-publication-bias-and-figure-22.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinical trial registries provide the ability to measure biases in the reporting of clinical trials that arise due to non-publication, delayed publication, or incomplete publication of results [14].”
From A systematic review of the processes used to link clinical trial registrations to their published results (2017) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many publications of randomized clinical trials of hypnosis interventions have been unclear or incomplete in reporting of the design elements that are necessary for the evaluation of the quality of evidence, 13 or have not provided sufficient details for possible direct replication.14,15”
From Best Practice Recommendations for Conducting and Reporting Controlled Trials in Clinical Hypnosis Research (2024) · cites this paper as [13]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“studies with statistically non-significant results may not have been submitted for publication (publication bias), or particular results that were statistically non-significant may have been omitted from study reports (selective non-reporting bias).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [83]
The cited source says what the claim attributes to it.
In the citing paper
“Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias.”
From Systematic review protocol assessing the processes for linking clinical trial registries and their published results (2016) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Despite providing important information relevant to patients, trials with statistically non-significant results are more prone to non-publication,114 or, when published, to a long delay to publication114 115 116—often referred to as publication bias.”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [116]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 12×
“Easterbrook et al [24] found that compared with unfunded studies, government funded studies were more likely to yield statistically significant results but government sponsorship was not found to have a statistically significant effect on the likelihood of publication and company sponsored trials were less likely to b…”
BackgroundMethodResultInfluentialSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · cited 1×
“Several studies investigated a cohort of trials submitted to drug licensing authorities [38,40,41,65] and all found that many of these trials remain unpublished, with one study demonstrating that trials with positive outcomes resulted more often in submission of a final report to the regulatory authority [40].”
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,412
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · cited 2×
“While much effort has been invested in trying to identify the former [2], it is equally important to understand the nature and frequency of missing data from the latter level.”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 29×
“The main objective of the study by Blumle et al [31] was to consider how eligibility criteria stated in protocols was reported in subsequent reports, in doing so they noted that 52% of studies in their cohort were published, decreasing to 48% for RCTs only.”
BackgroundMethodInfluentialSystematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 6×
“Since the original version of this review was published [13], a Cochrane methodology review is now available on publication bias [64].”
BackgroundMethodInfluentialSelf-citationPublication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · cited 2×
“While much effort has been invested in trying to identify the former [1,2], it is equally important to understand the nature and frequency of missing data from the latter level.”
Background
Research works printed but not linked: 2
- Menzel S, Uebing B, Hucklenbroich P, Schober O (2007) Evaluation of clinical trials following an approval from a research ethics committee. Dtsch Med Wochenschr 132: 2313– 2317. 17957593
- Chappell L, Alfirevic Z, Chien P, Jarvis S, Thornton JG (2005) A comparison of the published version of randomized controlled trials in a specialist clinical journal with the original trial protocols; Chicago.
Cited by
Items 601–700 of 1,001
A systematic review of the relationship between portion size and indexes of adiposity in children
Dobrescu, A. T., Porter, A., Ferriday, D., & Rogers, P. J. (2025) · Obesity Reviews · doi:10.1111/obr.13928 · Cited by 8
Mooney, L. A., & Fay, L. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012362 · Cited by 10
The Need for Formal Evidence Synthesis in Food Policy: A Case Study of Willingness-to-Pay
Clark, B., Frewer, L. J., Panzone, L., & Stewart, G. B. (2017) · Animals · doi:10.3390/ani7030023 · Cited by 8
“This failure to obtain a true representative sample of all those addressing a specific topic can threaten the validity of the meta-analysis [15–17], and can lead to the formation of unreliable conclusions [18,19], including a distorting effect on cumulative knowledge and evidence-based practice.”
BackgroundState of reporting of primary biomedical research: a scoping review protocol
Li, G., Mbuagbaw, L. C. E., Samaan, Z., Jin, Y., Nwosu, I., Levine, M., Adachi, J. D., & Thabane, L. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-014749 · Cited by 8
The use and abuse of meta‐analysis
Sotiriadis, A., Chatzakis, C., & Odibo, A. O. (2020) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.22060 · Cited by 9
Study design, result posting, and publication of late-stage cardiovascular trials
Kapelios, C. J., Naci, H., Vardas, P. E., & Mossialos, E. A. (2020) · European Heart Journal - Quality of Care and Clinical Outcomes · doi:10.1093/ehjqcco/qcaa080 · Cited by 8
Psychopathy: Neurohype and Its Consequences
Jalava, J., & Griffiths, S. Y. (2021) · History, philosophy and theory of the life sciences · doi:10.1007/978-3-030-82454-9_6 · Cited by 8
Liu, S., Bourgeois, F. T., & Dunn, A. G. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1545 · Cited by 9
“Up to half of all registered trials remain unpublished in the 2 years after trial completion.(10) Among the set of trials that are registered on ClinicalTrials.”
BackgroundWang, A., Menon, R., Li, T., Harris, L. A., Harris, I. A., Naylor, J., & Adie, S. (2023) · ANZ Journal of Surgery · doi:10.1111/ans.18273 · Cited by 12
“In previous studies, one of the main reasons consistently provided by authors for omitting outcomes was a lack of statistical significance, along with word limitations, perceived lack of clinical importance of results, fear of rejection, and publication in a different journal article.(12,20,21) Although unreported out…”
BackgroundReal-world public health interventions demonstrate how research evidence informs program scale-up
Crane, M. A., Lee, K., Wolfenden, L., Phongsavan, P., & Bauman, A. E. (2024) · Health Promotion International · doi:10.1093/heapro/daae111 · Cited by 9
“This is perhaps a product of evidence bias towards the reporting of positive impacts (Dwan et al ., 2013).”
BackgroundExercise reduces depressive symptoms in adults with arthritis: Evidential value
Kelley, G. A., & Kelley, K. S. (2016) · World Journal of Rheumatology · doi:10.5499/wjr.v6.i2.23 · Cited by 10
Tzanetakis, G. N., Tzimpoulas, N., Floratos, S. G., Agrafioti, A., Kontakiotis, E. G., & Shemesh, H. (2017) · International Endodontic Journal · doi:10.1111/iej.12805 · Cited by 8
Ruan, X., & Kaye, A. D. (2015) · American Journal of Medical Quality · doi:10.1177/1062860615599838 · Cited by 8
Lactobacillus for Gastroenteritis in Children. Reply.
Schnadower, D., Tarr, P. I., & Freedman, S. B. (2019) · PubMed · doi:10.1056/nejmc1900909 · Cited by 4
Dehghan, S., Moghaddasi, H., Rabiei, R., Choobineh, H., Maghooli, K., & Vahidi-Asl, M. (2025) · Journal of Education and Health Promotion · doi:10.4103/jehp.jehp_1798_23 · Cited by 10
“[56] Selective outcome reporting may have affected our findings since we found no studies reporting failed predictive models.”
Nunes, C. B., Lim, W. Y., Dy, Q., Ong, J., & Hwang, L. (2025) · European Journal of Nutrition · doi:10.1007/s00394-025-03718-6 · Cited by 8
“Reporting bias towards significant findings, even without an actual effect, may exacerbate these discrepancies [37].”
Kohal, R., Komine, F., Lith, A., Spies, B. C., Burkhardt, F., & Vach, K. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.105911 · Cited by 8
Barker, L., Wang, S. S., Huda, M. N., John, J. R., Wu, W. T., Lin, P., & Eapen, V. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-07527-3 · Cited by 9
“…standards: primary outcomes are typically used for sample-size determination and analytic hierarchy, whereas secondary outcomes are often exploratory, underpowered, vulnerable to selective reporting, and therefore, contribute greater to indi-rectness and imprecision in pooled estimates [57–61].”
van den Bogert, C. A., Souverein, P. C., Brekelmans, C. T. M., Janssen, S. W. J., van Hunnik, M., Koëter, G. H., Leufkens, H. G. M., & Bouter, L. M. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-007827 · Cited by 10
“To date, few studies have been carried out using this approach.(47) We report the design of a study that aims to evaluate reporting practices in an inception cohort of clinical”
Janssen, J. H. M., Châtel, B. D. L., Heijer, N. D., Tieben, R., Deen, M., Corten, R., Peeters, G. M. E. E., & Rikkert, M. G. M. O. (2023) · JMIR Formative Research · doi:10.2196/45173 · Cited by 8
“Terminated studies are underrepresented due to publication bias [26,27], while their evaluation can reveal pitfalls avoidable in the future.”
BackgroundTrends of Publication of Negative Trials Over Time
Laviolle, B., Locher, C., Allain, J., Cornu, Q. L., Charpentier, P., Lefebvre, M., Pape, C. L., Leven, C., Palpacuer, C., Pontoizeau, C., et al. (2025) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3535 · Cited by 8
Protocol publication rate and comparison between article, registry and protocol in RCTs
Mathieu, S., Bouillon‐Minois, J., Triché, L. R., Coudeyre, E., de Chazeron, I., Finotto, T., Laporte, C., Moisset, X., Samalin, L., Villatte, G., et al. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02471-y · Cited by 10
Colchicine for the primary prevention of cardiovascular events
Martí-Carvajal, A. J., Gemmato-Valecillos, M. A., Martín, D. M., Sanctis, J. B. D., Martí-Amarista, C. E., Hidalgo, R., Alegría‐Barrero, E., Lizardo, R. J. R., & Correa-Pérez, A. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015003.pub2 · Cited by 15
Weinberg, T., Wang, G., Lam, K. F. Y., Kitchen, J., & Chan, A. (2018) · British Journal of Dermatology · doi:10.1111/bjd.17332 · Cited by 7
Hugues, A., Marco, J. D., Bonan, I., Rode, G., Cucherat, M., & Gueyffier, F. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229822 · Cited by 7
Changes in the Serum Metabolome of Patients Treated With Broad-Spectrum Antibiotics
Jaskiw, G. E., Obrenovich, M. E., Kundrapu, S., & Donskey, C. J. (2020) · Pathogens and Immunity · doi:10.20411/pai.v5i1.394 · Cited by 7
“Given the bias against publishing negative results [41], investigators may be discouraged from conducting investigations of antibiotic effects on the serum metabolome or from publishing ostensibly negative data from completed studies.”
BackgroundMalone, E. R., Barua, R., Meti, N., Li, X., Fazelzad, R., & Hansen, A. R. (2021) · Cancer Medicine · doi:10.1002/cam4.4086 · Cited by 8
Feng, Z., Oberije, C., van de Wetering, A. J. P., Koch, A., Wouters, K. A. D., Vaes, N., Masclee, A., Carvalho, B. G., Meijer, G. A., Zeegers, M. P., et al. (2022) · Clinical and Translational Gastroenterology · doi:10.14309/ctg.0000000000000499 · Cited by 9
“tion bias still represents a ubiquitous problem in many scientific fields (46,47).”
BackgroundMunezero, E., Behan, N. A., Stephanie Diaz, G., Neumann, E.-M., & MacFarlane, A. J. (2022) · Advances in Nutrition · doi:10.1093/advances/nmac056 · Cited by 7
Sistematik Derleme Metodolojisi ve Yazımı
Cinar, N. D., & Suzan, Ö. K. (2022) · Akademisyen Kitabevi eBooks · doi:10.37609/akya.2053 · Cited by 7
Seehra, J., Khraishi, H., & Pandis, N. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01795-3 · Cited by 7
“Introduction It is well established that studies reporting statistically significant findings are more likely to be published compared to studies reporting non-significant or equivalent findings [1, 2].”
ResultRandomized controlled trials and alternative study designs in surgical oncology
Ceelen, W. P., & Søreide, K. (2023) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2023.03.001 · Cited by 8
Ibeneme, S. C., Mah, J., Omeje, C., Fortwengel, G., Nwosu, A. O., Irem, F. O., Ibeneme, G. C., Myezwa, H., & Nweke, M. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-05069-z · Cited by 7
“Publication Bias : Studies involving healthier participants may be more likely to be published, potentially leading to publication bias [137].”
BackgroundEffect of Randomized Clinical Trial Findings on Emergency Management
Hudgins, J. D., Fine, A. M., & Bourgeois, F. T. (2015) · Academic Emergency Medicine · doi:10.1111/acem.12840 · Cited by 7
Author Reply: Once Again, Menstrual Cycles and Mate Preferences
Wood, W. (2014) · Emotion Review · doi:10.1177/1754073914523053 · Cited by 7
Dechartres, A., Bond, E. G., Scheer, J., Riveros, C., Atal, I., & Ravaud, P. (2016) · BMC Medicine · doi:10.1186/s12916-016-0740-1 · Cited by 10
“Evidence of such reporting bias has been well documented [3–6], with many studies showing that trials with statistically significant results are more likely to be published and to be published more quickly than those with no statistical difference [3, 6–8].”
ResultIncluding non-public data and studies in systematic reviews and systematic maps
Haddaway, N., Collins, A., Coughlin, D., & Kohl, C. (2016) · Environment International · doi:10.1016/j.envint.2016.12.003 · Cited by 7
Interpreting evidence in general practice: Bias and conflicts of interest
Parker, L. M., Grundy, Q., & Bero, L. A. (2018) · Australian Journal of General Practice · doi:10.31128/ajgp-12-17-4432 · Cited by 7
Anderson, L. N., & Ledford, C. J. W. (2021) · Family Medicine · doi:10.22454/fammed.2021.246112 · Cited by 7
Jefferson, L., Fairhurst, C. M., Cooper, E., Hewitt, C. E., Torgerson, T., Cook, L., Tharmanathan, P., Cockayne, S., & Torgerson, D. J. (2016) · JRSM Open · doi:10.1177/2054270416649283 · Cited by 8
Indirect evidence of reporting biases was found in a survey of medical research studies
Albarqouni, L., López-López, J. A., & Higgins, J. P. T. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.11.013 · Cited by 7
“Reporting biases may impact scientific reports in different ways.[6-9] First, a whole study may 56 be suppressed, or harder to find, or published with delay, if its results are not considered to 57 be interesting.”
BackgroundNutritional epidemiology, extinction or evolution? It is all about balance and moderation
Verhoog, S., Marques-Vidal, P. M., & Franco, O. H. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00514-5 · Cited by 7
Stakeholder views on publication bias in health services research
Williams, I., Ayorinde, A., Mannion, R., Skrybant, M., Song, F., Lilford, R. J., & Chen, Y. (2020) · Journal of Health Services Research & Policy · doi:10.1177/1355819620902185 · Cited by 7
“Triangulation and/or large-scale investigation is therefore required before definitive claims can be made as to the presence and impact of publication and related biases in HSR.”
BackgroundInfluentialBlümle, A., Haag, T., Balmford, J., Rücker, G., Schumacher, M. C., & Binder, N. (2020) · PLoS ONE · doi:10.1371/journal.pone.0230797 · Cited by 9
Trial registration in pediatric surgery trials
Rokhsefat, S., Morra, D. E., Offringa, M., Askie, L. M., & Kelly, L. E. (2017) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2017.10.049 · Cited by 7
Prospective registration of clinical trials published in the B ritish J ournal of D ermatology
Batchelor, J., & Spuls, P. I. (2014) · British Journal of Dermatology · doi:10.1111/bjd.13314 · Cited by 7
Open Access Journals in Ophthalmology and Vision Science: All That Glitters Is Not Gold
Lê, J. T., Qureshı, R., & Li, T. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.06.019 · Cited by 8
A plea for preregistration in taxonomy
Conix, S., Cuypers, V., Zachos, F. E., Artois, T., & Monnens, M. (2023) · Megataxa · doi:10.11646/megataxa.10.1.1 · Cited by 7
Kohal, R., Lith, A., Komine, F., Honda, J., Spies, B. C., Burkhardt, F., & Vach, K. (2025) · Clinical Oral Implants Research · doi:10.1111/clr.70049 · Cited by 6
“Negative or non-confirmatory data are essential for building a complete and unbiased evidence base, especially in clinical fields where positive outcomes are disproportionately represented in the literature (Cairo et al. 2012; Dwan et al. 2013).”
Ebenezer, V., Ganesh, P., & Vinayagar, M. (2026) · Frontiers in Oral Health · doi:10.3389/froh.2026.1790589 · Cited by 8
“Studies reporting negative results or high complication rates may be underrepresented, potentially inflating apparent success rates (104).”
Norris, E., O’Mahony, A., Coyne, R., Varol, T., Green, J., Reynolds, J. P., & Toomey, E. C. (2024) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2024.2351939 · Cited by 6
“Preprints and Registered Reports (discussed in more detail later in this paper) also serve to reduce publication bias, which can lead to over-estimates of effect sizes and decreases the value of meta-analyses where studies with null findings are less likely to be included (Dwan et al., 2013).”
BackgroundLlopis, P., Oviedo-Trespalacios, Ó., Alonso, F., Faus, M., & Useche, S. A. (2025) · Sustainable Futures · doi:10.1016/j.sftr.2025.100982 · Cited by 7
Kelley, G. A., & Kelley, K. S. (2015) · BioMed Research International · doi:10.1155/2015/151985 · Cited by 11
Investigating systematic review for multi-disciplinary research in the built environment
Parida, S., & Brown, K. A. (2017) · Built Environment Project and Asset Management · doi:10.1108/bepam-10-2016-0056 · Cited by 7
Core Outcome Sets for Studies of Diabetes in Pregnancy: A Review
Egan, A. M., Bogdanet, D., Biesty, L., Kgosidialwa, O., McDonagh, C., O'Shea, C., O’Shea, P. M., Devane, D., & Dunne, F. (2020) · Diabetes Care · doi:10.2337/dc20-1621 · Cited by 9
Branagan, A., Canty, N., O’Halloran, E., Madden, M., & O‘Neill, M. F. (2021) · World Journal of Pediatrics · doi:10.1007/s12519-021-00483-4 · Cited by 8
Boesen, K., Jørgensen, K. J., & Gøtzsche, P. C. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.10.027 · Cited by 6
Quintero, M. J., López, F. J., Vadillo, M. A., & Morís, J. (2024) · Cognitive Therapy and Research · doi:10.1007/s10608-024-10497-7 · Cited by 7
“Lastly, we only included published articles (see Eligibility criteria ), but there may be laboratory and clinical studies that have not been published yet due to publication bias (Dwan et al., 2013; Franco et al., 2014), and, therefore, were not included in this review.”
Vaughn, K., Skinner, M., Vaughn, V., Wayant, C., & Vassar, M. B. (2018) · Journal of Hypertension · doi:10.1097/hjh.0000000000001870 · Cited by 6
“Inadequate publication bias assessments decrease confidence in the summary effect estimate, as unpublished studies with negative results may alter the level of significance reported in the systematic review [41].”
ResultMheissen, S., Khan, H., Almuzian, M., Alzoubi, E. E., & Pandis, N. (2021) · European Journal of Orthodontics · doi:10.1093/ejo/cjab069 · Cited by 6
Approaches to Assessing and Adjusting for Selective Outcome Reporting in Meta-analysis
Jackson, J. L., Balk, E. M., Hyun, N., & Kuriyama, A. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-07135-3 · Cited by 10
La crisis de reproducibilidad de la ciencia y la necesidad de publicar los resultados negativos
Martín-Moro, J. G. (2017) · Archivos de la Sociedad Española de Oftalmología · doi:10.1016/j.oftal.2017.07.009 · Cited by 6
Outcome Reporting Bias in Randomized Experiments on Substance Use Disorders
Wooditch, A., Sloas, L. B., Wu, X., & Key, A. (2018) · Journal of Quantitative Criminology · doi:10.1007/s10940-018-9388-7 · Cited by 8
Loézar, C., Madrid, E., Jahr, C., Daviú, A., Ahumada, H., Hernandez, H. P., Keller, E., & Bonfill, X. (2018) · Ophthalmic Epidemiology · doi:10.1080/09286586.2018.1503688 · Cited by 6
“CCTs are also the raw material for systematic reviews, which aim to integrate the best available evidence, and therefore require an exhaustive approach to identify them all on a given topic of interest.(5) The most common approach for identifying CCTs is the implementation of a sensitive electronic search strategy.”
BackgroundTrial Registration and Outcome Reporting in Child and Pediatric Psychology: A Systematic Review
Hildenbrand, A. K., Conour, C. N., Straus, J. A., Moufarrej, S., & Palermo, T. M. (2019) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsz054 · Cited by 7
Berkowitz, A. (2019) · Journal of Neurogenetics · doi:10.1080/01677063.2019.1706093 · Cited by 7
“A positive result (e.g. a statistically significant effect of a treatment or a gene) is much easier to publish, and to publish in a high-profile journal, than a negative result (Carroll, 2018a, 2018b; Dwan et al., 2008, 2013; Kicinski, 2013, 2014).”
BackgroundDeVito, N., Drysdale, H., McKee, M., & Goldacre, B. M. (2021) · Tobacco Control · doi:10.1136/tobaccocontrol-2020-056221 · Cited by 7
Reporting results in manual therapy clinical trials: A need for improvement
Núñez‐Cortés, R., Álvarez, G., Bracchiglione, J., Cabanas‐Valdés, R., Sanz, J. C., Bonfill, X., & Urrútia, G. (2021) · International journal of osteopathic medicine · doi:10.1016/j.ijosm.2021.06.002 · Cited by 7
Chong, S. W., Imberger, G. L., Karahalios, A., Wang, A., Burggraf, M., Louis, M., Liskaser, G. M., Bianco, A., & Peyton, P. J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0270841 · Cited by 6
“When studies are prospectively registered, there are still often major discrepancies between protocols and the corresponding published papers [3,4,6,10,14].”
ResultSargeant, J. M., O'Connor, A. M. C., O’Sullivan, T. L., & Ramirez, A. (2023) · Journal of Swine Health and Production · doi:10.54846/jshap/1300 · Cited by 6
Advancing research by publishing research protocols and negative studies
Agha, R. (2016) · International Journal of Surgery Protocols · doi:10.1016/j.isjp.2016.06.001 · Cited by 6
“conducted a systematic review of publication and outcome reporting bias [4].”
BackgroundThe need for clarity and consistency in defining and reporting primary outcomes
Griffiths, P., & Norman, I. J. (2021) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.103967 · Cited by 6
“Studies with ‘negative’ (i.e. not statistically significant) results are less likely to be published because authors are less likely to submit such studies for publication (Dwan et al, 2013).”
BackgroundInfluentialHeravi, A. T., Gryaznov, D., Busse, J. W., Schönenberger, C., von Niederhäusern, B., Hausheer, L., Covino, M., Schwenke, J. M., Epp, S., Griessbach, A., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111822 · Cited by 6
Lonjon, G., Ross, N., Dhenin, A., Vassal, M., Gourari, A., Bouic, N., & Bonnal, A. (2025) · Brain and Spine · doi:10.1016/j.bas.2025.104379 · Cited by 5
“One article (Dwan et al., 2013) showed the opposite: positive clinical results and publications that seemed strongly related.”
Zhu, L. M., Mendel, A., Ross, C., & Makhzoum, J. (2025) · Rheumatology Advances in Practice · doi:10.1093/rap/rkaf128 · Cited by 6
“Outcome reporting bias was evaluated using the Outcome Reporting Bias in Trials (ORBIT) tool [27].”
Grenier, J., & Thiel, A. (2025) · European Journal of Pain · doi:10.1002/ejp.70150 · Cited by 6
Littell, J. H., & Gorman, D. M. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01998-1 · Cited by 6
Hove, L. V., Baetens, I., Hamza, C. A., Dierckx, E., Haekens, A., Fieremans, L., & Vanderstichelen, S. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197611272.013.29 · Cited by 7
Methods for assessing inverse publication bias of adverse events
Xing, X., Xu, C. Q., Amer, F. M. A., Shi, L., Zhu, J., & Lin, L. (2024) · Contemporary Clinical Trials · doi:10.1016/j.cct.2024.107646 · Cited by 6
Adu, Y., Ring, D., & Teunis, T. (2024) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000003273 · Cited by 7
“The greater reporting of positive effects in lower quality studies might be related to the tendency to favor publishing statistically signi fi cant results [24].”
Identification of home bias in journal ranking lists
Wu, D., Su, Q., & Li, J. (2025) · Journal of Informetrics · doi:10.1016/j.joi.2025.101707 · Cited by 6
Mayer, M. G. (2015) · F1000Research · doi:10.12688/f1000research.7037.1 · Cited by 5
Mergenthaler, P., & Meisel, A. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-800687-0.00007-4 · Cited by 5
Faggion, C. M. (2022) · Accountability in Research · doi:10.1080/08989621.2022.2028625 · Cited by 5
Malmsiø, D., Frost, A. D., & Hróbjartsson, A. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.12.012 · Cited by 5
Uptake of the core outcome set on polycystic ovary syndrome before and after its publication
Li, W., Li, G., Chi, H., Wang, H., & Zeng, L. (2023) · Human Reproduction · doi:10.1093/humrep/dead149 · Cited by 6
Beyond journal publications – a new format for the publication of clinical trials
Wieseler, B. (2016) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2016.11.003 · Cited by 6
Wright, D. N. M., Williams, E., Bryce, C., le May, A., Stein, K., Milne, R., & Walley, T. (2018) · Health Research Policy and Systems · doi:10.1186/s12961-018-0339-4 · Cited by 5
What Is and What Is Not a Conflict of Interest in the Sphere of Public Health
Royo‐Bordonada, M. Á., & López, F. J. G. (2019) · Springer eBooks · doi:10.1007/978-3-030-28626-2_14 · Cited by 5
Animal models: value and translational potency
Mergenthaler, P., & Meisel, A. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-820493-1.00028-3 · Cited by 5
ClinicalTrials.Gov: Pitfalls for pregnant women looking to enroll in studies
Franceschini, R., & Wicks, S. L. (2022) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2022.100890 · Cited by 5
Frank, R. A., Fabiano, N., Hallgrimson, Z., Korevaar, D. A., Cohen, J. F., Bossuyt, P. M., Leeflang, M., Moher, D., McInnes, M. D. F., Treanor, L. M., et al. (2022) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.28046 · Cited by 7
Choi, J., Park, J., Kim, J.-E., Lee, M., Kang, D., Shin, A., & Choi, J. (2023) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-22-0466 · Cited by 6
Ho, L. T. F., Cheung, Y. M. K., Choi, C. C. C., Wu, I. X., Mao, C., & Chung, V. C. (2024) · Journal of Dermatological Treatment · doi:10.1080/09546634.2024.2343072 · Cited by 5
“Risk of bias refers to the potential for systematic errors in a study’s design, conduct, or analysis that can influence its results (10,28).”
InfluentialConcealing research outcomes: Missing data, negative results and missed publications
Bhaskar, S. (2017) · Indian Journal of Anaesthesia · doi:10.4103/ija.ija_361_17 · Cited by 6
Joober, R. (2022) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.220222 · Cited by 6
Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. (2021) · doi:10.1101/2021.07.20.21260565 · Cited by 23
“Outcomes that are statistically significant have higher odds of being fully reported than non-significant outcomes (29,30).”
BackgroundKim, S., Bae, S. H., & Hyun, M. S. (2025) · Journal of Korean Academy of Nursing · doi:10.4040/jkan.24072 · Cited by 4
Misplaced Protections: How IRB Review is the Wrong Gatekeeper for EBP Initiatives
Edmonds, S. W., Fowler, C., & Cullen, L. (2025) · Journal of PeriAnesthesia Nursing · doi:10.1016/j.jopan.2025.07.021 · Cited by 4
Toward a Philosophy of Error in Science
Allchin, D. (2025) · doi:10.1093/9780197827703.001.0001 · Cited by 10
Metadata
- Authors
- Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group
- Affiliations
- University of Liverpool (GB)
- Corresponding author
- Dwan, Kerry
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-05
- Volume
- 8
- Issue
- 7
- Pages
- e66844
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3702538
- OpenAlex
- W2002062153
- Cited by
- 1,127
- References
- 69
- Citations by year
- 2026: 75 · 2025: 84 · 2024: 64 · 2023: 71 · 2022: 127 · 2021: 98 · 2020: 112 · 2019: 89 · 2018: 83 · 2017: 102 · 2016: 89 · 2015: 70 · 2014: 57 · 2013: 5
- FWCI
- 15.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- SERVAL (Université de Lausanne) (publishedVersion, cc-by)
- IRIS (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Tufts Medical Center
- King's College London
- University of Oxford
- University of Bristol
- University of Sydney
- Queen's University
- Women's College Hospital
- Centre Hospitalier Universitaire Vaudois
- Hospices Civils de Lyon
- Moorfields Eye Hospital NHS Foundation Trust
- School of Medicine, Stanford University
- Tufts University School of Medicine
- Medical Research Council
- National Health and Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- publication bias, selective reporting, randomized controlled trials, meta-analysis validity, trial reporting quality, primary outcome reporting
- MeSH terms
- Humans, Publications, Cohort Studies, Randomized Controlled Trials as Topic, Outcome Assessment, Health Care, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.