Selective reporting of outcomes in randomised controlled trials in systematic reviews of cystic fibrosis
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Background: Outcome reporting bias (ORB) in randomised trials has been identified as a threat to the validity of systematic reviews. Previous work highlighting this problem is limited to considering a single primary review outcome. The aim of this study was to assess ORB across all efficacy outcomes in the Cochrane systematic reviews of cystic fibrosis. Methods: Systematic reviews of interventions for cystic fibrosis published on the Cochrane Library by the Cochrane Cystic Fibrosis and Genetic Disorders Group before 2010 were assessed for discrepancies in outcomes between review protocol and full review. ORB in eligible trials was also assessed for all efficacy review outcomes. Two authors independently classified each outcome using a nine-point classification system developed by the Outcome Reporting Bias In Trials study. These classifications were used to inform the assessment of the risk of bias for selective outcome reporting for each trial. Results: –46 Cochrane cystic fibrosis systematic reviews were included. The median number of primary outcomes, number of trials and participants per trial in the reviews were 3 (IQR 2, 3), 4 (IQR 2, 8) and 21 (IQR 14, 41), respectively. 18 reviews (39%, 18/46) had a discrepancy in outcomes between protocol and full review. 37 reviews were eligible to be included in the ORB assessment. When considering review primary outcomes and all review outcomes, ORB was suspected in at least one trial in 86% and 100%, respectively. Conclusions: Assessment of ORB within a systematic review of a single primary outcome underestimates the risk of ORB in comparison to the assessment of multiple primary and secondary outcomes. ORB in trials is highly prevalent within systematic reviews of cystic fibrosis when assessed across all outcomes. This could be reduced by the development of a core outcome set for trials and systematic reviews in cystic fibrosis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies examining selective reporting of results in systematic reviews have found that 11% to 22% of reviews did not present results for at least one pre-specified outcome of the review.140 141 142 143”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [142]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Self-citationKirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationStandardising outcomes for clinical trials and systematic reviews
Clarke, M. (2007) · Trials · doi:10.1186/1745-6215-8-39 · Cited by 483
Sinha, I., Jones, L., Smyth, R. L., & Williamson, P. R. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050096 · Cited by 202
Self-citationRamsey, B. W., & Boat, T. F. (1994) · The Journal of Pediatrics · doi:10.1016/s0022-3476(94)70301-9 · Cited by 200
Bias in Meta-Analysis Due to Outcome Variable Selection Within Studies
Hutton, J. L., & Williamson, P. R. (2000) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00197 · Cited by 194
Self-citationOutcome selection bias in meta-analysis
Williamson, P. R., Gamble, C., Altman, D. G., & Hutton, J. L. (2005) · Statistical Methods in Medical Research · doi:10.1191/0962280205sm415oa · Cited by 188
Self-citationBias Due to Changes in Specified Outcomes during the Systematic Review Process
Kirkham, J. J., Altman, D. G., & Williamson, P. R. (2010) · PLoS ONE · doi:10.1371/journal.pone.0009810 · Cited by 164 · cited 2×
“This compares with 22% of reviews (64/288) that contained a discrepancy in at least one outcome measure in the main ORBIT study which looked at reviews covering all 50 Cochrane review groups.(2) However, this is confounded by the different publication date ranges of the reviews (assessed as up to date between 2006 and…”
ResultSelf-citationUsing observational data from registries to compare treatments: The fallacy of omnimetrics
Green, S. B., & Byar, D. P. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030413 · Cited by 161 · cited 1×
“BACKGROUND The value of systematic reviews in establishing an evidence base is widely acknowledged with well-conducted systematic reviews of randomised controlled trials (RCTs) being placed at the top of the hierarchy of evidence.(1) It is essential, when conducting systematic reviews, to consider the potential for bi…”
Background
Research works printed but not linked: 2
- Von Mosch S Dwan K Reporting of harm and efficacy outcomes of randomised clinical trials in cystic fibrosis before and after the publication of the revised CONSORT (CONsolidated Standards Of Reporting Trials) statement in 2001—a comparison of trial reports published from 1994 and from 2008 [ thesis]. The University of Liverpool, 2011
- The Cochrane Library, Issue 4, Chichester: Wiley, 2009
Cited by
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 13,262
Page, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916
Page, M. J., McKenzie, J. E., Kirkham, J., Dwan, K., Kramer, S., Green, S., & Forbes, A. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000035.pub2 · Cited by 281
Self-citationDimairo, M., Pallmann, P., Wason, J., Todd, S., Jaki, T., Julious, S. A., Mander, A. P., Weir, C. J., Koenig, F., Walton, M. K., et al. (2020) · BMJ · doi:10.1136/bmj.m115 · Cited by 149
Boney, O., Moonesinghe, R., Myles, P. S., Grocott, M. P. W., Bartoszko, J., Beattie, W. S., Bellomo, R., Buggy, D. J., Cabrini, L., Canet, J., et al. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.09.027 · Cited by 124
Metadata
- Authors
- Dwan, K., Kirkham, J. J., Williamson, P. R., & Gamble, C.
- Affiliations
- University of Liverpool (GB)
- Corresponding author
- Dwan, Kerry
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2013-05
- Volume
- 3
- Issue
- 6
- Pages
- e002709
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC3669728
- OpenAlex
- W2084159208
- Semantic Scholar
- aee6386dd505c80bb0b60c6abc1c2f0e0b2c08c3
- Cited by
- 55
- References
- 20
- Citations by year
- 2026: 1 · 2025: 2 · 2023: 5 · 2022: 2 · 2021: 6 · 2020: 16 · 2019: 3 · 2018: 5 · 2017: 2 · 2016: 3 · 2015: 4 · 2014: 3 · 2013: 3
- FWCI
- 0.89
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (submittedVersion)
- Funders
- National Institute for Health and Care Research
- Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- selective reporting, selective outcome reporting, cystic fibrosis, systematic reviews, risk of bias assessment, core outcome set, randomized controlled trials, primary and secondary outcomes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Selective reporting of outcomes in randomised controlled trials in systematic reviews of cystic fibrosis. https://app.cochat.ai/papers/10.1136/bmjopen-2013-002709
- The paper
- Dwan, K., Kirkham, J. J., Williamson, P. R., & Gamble, C. (2013). Selective reporting of outcomes in randomised controlled trials in systematic reviews of cystic fibrosis. BMJ Open. https://doi.org/10.1136/bmjopen-2013-002709
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
