Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The reporting of outcomes within published randomized trials has previously been shown to be incomplete, biased and inconsistent with study protocols. We sought to determine whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review. METHODS: We compared protocols for randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications. Characteristics of reported and unreported outcomes were recorded from the protocols and publications. Incompletely reported outcomes were defined as those with insufficient data provided in publications for inclusion in meta-analyses. An overall odds ratio measuring the association between completeness of reporting and statistical significance was calculated stratified by trial. Finally, primary outcomes specified in trial protocols were compared with those reported in publications. RESULTS: We identified 48 trials with 68 publications and 1402 outcomes. The median number of participants per trial was 299, and 44% of the trials were published in general medical journals. A median of 31% (10th-90th percentile range 5%-67%) of outcomes measured to assess the efficacy of an intervention (efficacy outcomes) and 59% (0%-100%) of those measured to assess the harm of an intervention (harm outcomes) per trial were incompletely reported. Statistically significant efficacy outcomes had a higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95% confidence interval 1.5-5.0). Primary outcomes differed between protocols and publications for 40% of the trials. INTERPRETATION: Selective reporting of outcomes frequently occurs in publications of high-quality government-funded trials.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our findings echo previous research that suggests that the effects of education on behaviour may be limited [5,78,80,82-85], but may represent a necessary ingredient or first step in the process of change.”
From Systematic review of knowledge translation strategies in the allied health professions (2012) · cites this paper as [84]
In the citing paper
“There is emerging evidence that this is a common problem in the literature, yet it is a difficult problem to pinpoint [84].”
From Systematic review of knowledge translation strategies in the allied health professions (2012) · cites this paper as [84]
In the citing paper
“Although up to 38% of trials defined multiple primary outcomes,249 250 251 252”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [250]
In the citing paper
“These substantive modifications are rarely acknowledged in the final trial reports.200 249 250 480”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [250]
In the citing paper
“Outcome reporting bias (i.e., within study) occurs when authors select specific outcomes to include in the report of their research; such selection is associated with those outcomes that appear to reach statistical significance.”
From Reporting research results: a moral obligation for all researchers. (2007) · cites this paper as [11]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Deeks, J. J., Altman, D. G., & Bradburn, M. J. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch15 · Cited by 2,163
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 3×
“Deviations from trial protocols should be described in the published reports so that readers can assess the potential for bias.(1,13) At a minimum, they should be declared at the time of submission to journals.”
BackgroundResultSelf-citationMeta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
Melander, H., Ahlqvist-Rastad, J., Meijer, G., & Beermann, B. (2003) · BMJ · doi:10.1136/bmj.326.7400.1171 · Cited by 635
Publication and Related Biases
Song, F., Eastwood, A., Gilbody, S., Duley, L., & Sutton, A. (2001) · The Advanced Handbook of Methods in Evidence Based Healthcare · doi:10.4135/9781848608344.n21 · Cited by 455
Publication and related biases.
Song, Eastwood, Gilbody, Duley, & Sutton (2000) · Health Technology Assessment · doi:10.3310/hta4100 · Cited by 427
How important is publication bias? A synthesis of available data.
Dickersin, K. (1997) · PubMed · Cited by 311
Bias in meta-analytic research
Felson, D. T. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90072-u · Cited by 231
Underpowering in randomized trials reporting a sample size calculation
Vickers, A. J. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00141-0 · Cited by 102
Hahn, S., Williamson, P. R., & Hutton, J. L. (2002) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2002.00314.x · Cited by 97 · cited 1×
“Other literature on outcome reporting bias is limited to case reports and a small pilot study that required permission from researchers to access their ethics protocols.(8) Comparisons between journal publications and final reports submitted to drug-approval agencies have also revealed discrepancies in data reporting.”
Background
Cited by
Items 401–444 of 444
Pharmaceutical Company Sponsorship of Research: A Review
Sismondo, S. (2006)
Validation and Confirmation of Associations
Ioannidis, J. (2008) · CRC Press eBooks · doi:10.3109/9781420052923-17
Reporting and Interpreting Results
Little, J. (2008) · CRC Press eBooks · doi:10.3109/9781420052923-19
Boutefnouchet, T., Laios, T., & Mathur, K. (2017) · Journal of Sports Medicine and Therapy · doi:10.29328/journal.jsmt.1001012
Conflict of interest policies and disclosures in Journals
Alfonso, F., Timmis, A., Ambrosio, G., Ector, H., Kulakowski, P., Vardas, P. E., & Pinto, J. (2012) · Cardiovascular Medicine · doi:10.4414/cvm.2012.01657
Alfonso, F., Timmis, A., Pinto, F. J., & Ambrosio, G. (2012) · Journal of the Hong Kong College of Cardiology · doi:10.55503/2790-6744.1048
Ponce, M. F. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3164259
Chalut, F., Chalut, F., & LeLorier, J. (2019) · Journal of Clinical Cardiology and Cardiovascular Therapy · doi:10.31546/jcccvt.1005
Biggane, A. M. (2020) · theses.fr (ABES) · doi:10.70675/7ce817c3ze739z4f55za028z9ae90e5e7d82 · Cited by 2
A reader responds to "The myth of equipoise in phase 1 clinical trials".
Noble, J. H. (2008) · PubMed · Cited by 1
Helal, L. C. A. S., da Costa, G. G., Alves, C. P. D. L., Luizeti, B. O., Freccia, G. W., Gonçalves, L. S., Nascimento, D. P., Rice, D. B., Mayo‐Wilson, E., & Hoffmann, T. (2020) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/uegmb
(2005) · BMJ · doi:10.1136/bmj.330.7499.1052-b
Klonoff, D. C. (2005) · The American Journal of Medicine · doi:10.1016/j.amjmed.2005.04.009
Lecture: Quality of Randomised Clinical Trials in Portal Hypertension and Other Fields of Hepatology
Gluud, C. N., Klingenberg, S. L., & Gluud, L. L. (2006) · Wiley eBooks · doi:10.1002/9780470988831.ch13
La Cochrane Collaboration a 10 ans : realisations et defis
(2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041478
ABC of adolescence: Substance misuse: alcohol, tobacco, inhalants, and other drugs
British Medical Journal Publishing Group (2005) · BMJ · doi:10.1136/bmj.330.7499.1052-a
Vedula, S. S., Hróbjartsson, A., & Page, M. J. (2021) · Springer eBooks · doi:10.1007/978-3-319-52677-5_183-1
Nazarian, B., Cheraghi, Z., & Farhanchi, A. (2022) · مجله پزشکی بالینی ابن سینا · doi:10.52547/ajcm.29.1.57
The applicability and utilization of systematic reviews for clinical practice
Lindsley, K. B. (2022) · Utrecht University Repository (Utrecht University) · doi:10.33540/1513
Designing and Deciphering Clinical Trials: Methodological Ingredients to Nourish a Healthy Study
Glantz, M., & Mansouri, A. (2023) · doi:10.1093/med/9780197573778.003.0028
Improving risk of bias assessment and quality of reporting in clinical research
Bertizzolo, L. (2021) · theses.fr (ABES) · doi:10.70675/7c89a3e9zfac2z4b5bza6a5z77fb27b5a29b
RAAB, G. M. (2024) · Springer eBooks · doi:10.1007/978-3-031-25859-6_2
Hirt, J., Pieper, D., Becker, M., Breuing, J., Marshall, M. R., & Gooßen, K. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111890
Milan, N., Woolley, K. A., Master, Z., & Feeley, B. T. (2025) · Orthopaedic Journal of Sports Medicine · doi:10.1177/23259671251374306 · Cited by 1
Addressing Outcome Reporting Bias in Meta‐Analysis: A Selection Model Perspective
Saracini, A. G., & Held, L. (2025) · Statistics in Medicine · doi:10.1002/sim.70238 · Cited by 3
Korjonen, M. H. (2012) · UCL Discovery (University College London)
Lazzarini, S. G., Stella Yousif, M., Bargeri, S., Castellini, G., & Gianola, S. (2024) · Figshare · doi:10.6084/m9.figshare.24417626
Lazzarini, S. G., Stella Yousif, M., Bargeri, S., Castellini, G., & Gianola, S. (2023) · Figshare · doi:10.6084/m9.figshare.24417576
Publication bias in clinical trials
Fenner, M. (2009) · doi:10.53731/46tpv-p8c75
Registry History Changes and Results Reporting of Clinical Trials
Holst, M. (2025) · Refubium (Universitätsbibliothek der Freien Universität Berlin) · doi:10.17169/refubium-49008
Outcome reporting bias in ophthalmology clinical trials between 2014 and 2023
Li, X., Tien, C.-W., Huang, R. S., Hanna, A., Acharya, S., Lusterio, A., Hsiao, H., Tao, B. K.-L., Xie, J. S., & Margolin, E. (2026) · British Journal of Ophthalmology · doi:10.1136/bjo-2025-328466
Le gaspillage évitable de la recherche dans les essais cliniques
Yordanov, Y. (2019) · theses.fr (ABES) · doi:10.70675/130e8831z862fz47f2z8000z2f04c8541788
Burgwinkel, C., Gicquel, M., Boutron, I., Chevret, S., Held, L., Moher, D., Naudet, F., & Locher, C. (2026) · BMC Medicine · doi:10.1186/s12916-026-04831-x
Nguyen, P.-Y., Turner, S., Korevaar, E., Page, M. J., & McKenzie, J. E. (2026) · BMC Public Health · doi:10.1186/s12889-026-27475-1
Martinić-Cezar, I. J., Pranić, S. M., Tavra, A., & Marušić, A. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15103981
Deep Learning Approaches for Intrusion Detection in IoT Environments: A Systematic Literature Review
Fernández-Morales, E., García-Merino, J. C., Tobarra, L., Robles-Gómez, A., & Vargas, R. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3719346
Publication Bias: Impact on Evidence Synthesis, Detection Methods, and Prevention Strategies
Hussain, R., Ahmad, N. A., Alshorman, K., & Babar, Z.-U.-D. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-26472-6.00176-5
de Sousa, D. F., de Menezes, R. N., Vieira, M. E. N. P., Oliveira, S. D. F., & Vieira, N. B. (2026) · doi:10.62642/tec-978-65-83199-83-6_015
IDENTIFYING AND OVERCOMING OBSTACLES TO SAMPLE SIZE AND POWER CALCULATIONS IN FMRI STUDIES
Guo, Q. (2013) · MacSphere (McMaster University) · doi:10.71548/22074
Metadata
- Authors
- Chan, A.-W.
- Affiliations
- Canadian Institutes of Health Research (CA)
- Corresponding author
- Chan, A.-W.
- Venue
- Canadian Medical Association Journal
- Publisher
- CMA Impact Inc.
- Published
- 2004-09-28
- Volume
- 171
- Issue
- 7
- Pages
- 735-740
- ISSN
- 0820-3946 (ISSN-L)
- 1488-2329
- PMC
- PMC517858
- OpenAlex
- W2159096825
- Cited by
- 444
- References
- 18
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 9 · 2023: 14 · 2022: 14 · 2021: 15 · 2020: 13 · 2019: 16 · 2018: 21 · 2017: 17 · 2016: 20 · 2015: 23 · 2014: 16 · 2013: 27 · 2012: 42
- FWCI
- 25.89
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Canadian Medical Association Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- DigitalGeorgetown (Georgetown University Library) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- selective reporting, selective outcome reporting, Canadian Institutes of Health Research, primary outcome reporting, efficacy outcomes, publication bias
- MeSH terms
- Canada, Government Agencies, Humans, Research Support as Topic, Randomized Controlled Trials as Topic, Treatment Outcome, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.