CoChat

Assessing the potential for outcome reporting bias in a review: a tutorial

Open access
Dwan, Kerry, Gamble, Carrol, Kolamunnage-Dona, Ruwanthi, Mohammed, Shabana, Powell, Colin, Williamson, Paula R · 2010 · Trials
Cited by 63

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Outcome reporting bias (ORB) occurs when variables are selected for publication based on their results. This can impact upon the results of a meta-analysis, biasing the pooled treatment effect estimate.The aim of this paper is to show how to assess a systematic review and corresponding trial reports for ORB using an example review of intravenous and nebulised magnesium in the treatment of asthma. METHODS: The review was assessed for ORB by 1) checking the reasons, when available, for excluding studies to ensure that no studies were excluded because they did not report the outcomes of interest in the review; 2) assessing the eligible studies as to whether the review outcomes of interest were reported. Each study was classified using a system developed in the ORBIT (Outcome Reporting Bias In Trials) project to indicate whether ORB was suspected and a reason for the suspicion. Authors of trials that did not report the outcomes of interest were contacted for information. A sensitivity analysis was performed to assess the robustness of the conclusions of the review to this potential source of bias. RESULTS: Twenty-four studies were included in the review; two studies had been excluded for not reporting either of the two outcomes of interest. Six included studies did not report hospital admission and two did not report pulmonary function. There was high suspicion of outcome reporting bias in four studies. Results from the sensitivity analysis indicate that review conclusions were not overturned. CONCLUSION: This paper demonstrates, with the example of the magnesium review, how to assess a review for outcome reporting bias. A review should not exclude studies if they have not reported the outcomes of interest and should consider the potential for outcome reporting bias in all included studies.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/1745-6215-11-52

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 9 references

Research works printed but not linked: 26

  1. Jones R: Strength of evidence in qualitative research. J Clin Epidemiol. 2007, 60 (4): 321-3. 10.1016/j.jclinepi.2006.06.001. · DOI
  2. Higgins JPT, Green S: Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0. 2008, The Cochrane Collaboration · DOI · Paper page
  3. Hutton JL, Williamson PR: Bias in meta-analysis due to outcome variable selection within studies. Applied Statistics. 2000, 49 (3): 359-370.
  4. Higgins JPT, Altman DG: Chapter 8: Assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0. Edited by: Higgins JPT, Green S. 2008, The Cochrane Collaboration · DOI · Paper page
  5. Mohammed S, Goodacre S: Intravenous and nebulised magnesium sulphate for acute asthma: systematic review and meta-analysis. Emerg Med J. 2007, 24 (12): 823-30. 10.1136/emj.2007.052050. · DOI
  6. Basic Asthma Research Strategy II: The Second Asthma UK Consultation. Clinical and Experimental Allergy. 2006, 1310-
  7. British Thoracic Society and Scottish Intercollegiate Guidelines Network: British Guideline on the Management of Asthma. A national clinical guideline. Thorax. 2008, 63 (Supplement IV): iv1-iv121. · DOI · Paper page
  8. Coates JE, Chinn DJ, Miller MR: Lung Function: Physiology, Measurement and Application in Medicine. 2006, Wiley, Sixth · DOI
  9. Gorelick MH, Stevens MW, Schultz T, Scribano PV: Difficulty in Obtaining Peak Expiratory Flow Measurements in Children With Acute Asthma. Pediatric Emergency Care. 2004, 20 (1): 22-26. 10.1097/01.pec.0000106239.72265.16. · doi:10.1097/01.pec.0000106239.72265.16
  10. Dadhich P, Vats M, Lokendra D, Gupta RC, Gupta ML, Gupta N: Magnesium Sulphate Nebulization in Acute Severe Asthma. 2003, 107S-
  11. Santana JC, Barreto SS, Piva JP, Garcia PC: Randomized clinical trial of intravenous magnesium sulphate versus salbutamol in early management of severe acute asthma in children. J Pediatr. 2001, 77 (4): 279-87. · doi:10.2223/jped.235
  12. Boonyavorakul C, Thakkinstian A, Charoenpan P: Intravenous magnesium sulfate in acute severe asthma. 2000, 221-225.
  13. Scarfone RJ, Loiselle JM, Joffe MD, Mull CC, Stiller S, Thompson K, Gracely EJ: A randomized trial of magnesium in the emergency department treatment of children with asthma. Ann Emerg Med. 2000, 36 (6): 572-8. 10.1067/mem.2000.111060. · doi:10.1067/mem.2000.111060
  14. Bessmertny O, DiGregorio RV, Cohen H, Becker E, Looney D, Golden J, Kohl L, Johnson T: A randomized clinical trial of nebulized magnesium sulfate in addition to albuterol in the treatment of acute mild-to-moderate asthma exacerbations in adults. Annals of Emergency Medicine. 2002, 39 (6): 585-591. 10.1067/mem.2002.123300. · doi:10.1067/mem.2002.123300
  15. Bijani K, Moghadamnia AA, Islami Khalili E: Intravenous magnesium sulphate as an adjunct in the treatment of severe asthmatic patients non-responding to conventional therapy. The Internet Journal of Asthma, Allergy and Immunology. 2002, 2 (1): · doi:10.5580/2381
  16. Devi PR, Kumar L, Singhi SC, Prasad R, Singh M: Intravenous magnesium sulfate in acute severe asthma not responding to conventional therapy. Indian Pediatr. 1997, 34 (5): 389-97.
  17. Gürkan F, Haspolat K, Boşnak M, Dikici B, Derman O, Ece A: Intravenous magnesium sulphate in the management of moderate to severe acute asthmatic children nonresponding to conventional therapy. Eur J Emerg Med. 1999, 6 (3): 201-5. · doi:10.1097/00063110-199909000-00005
  18. Meral A, Coker M, Tanac R: Inhalation therapy with magnesium sulphate and salbutamol sulphate in bronchial asthma. Turkish Journal of Pediatrics. 1996, 38: 169-75.
  19. Tiffany BR, Berk WA, Todd IK, White SR: Magnesium bolus or infusion fails to improve expiratory flow in acute asthma exacerbations. 1993, 831-834.
  20. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG: Epidemiology and reporting characteristics of systematic reviews. PLoS Medicine/Public Library of Science. 2007, 4 (3): e78-
  21. Higgins JP, Thompson SG, Deeks JJ, Altman DG: Measuring inconsistency in meta-analyses. Bmj. 2003, 327 (7414): 557-60. 10.1136/bmj.327.7414.557. · DOI · Paper page
  22. Porter RS, Nester , Braitman LE, Geary U, Dalsey WC: Intravenous magnesium is ineffective in adult asthma, a randomized trial. Eur J Emerg Med. 2001, 8 (1): 9-15. 10.1097/00063110-200103000-00003. · doi:10.1097/00063110-200103000-00003
  23. Sinha I, Jones L, Smyth RL, Williamson PR: A systematic review of studies that aim to determine which outcomes to measure in clinical trials in children. PLoS Med. 2008, 5 (4): e96-10.1371/journal.pmed.0050096. · DOI · Paper page
  24. Clarke M: Standardising Outcomes in Paediatric Clinical Trials. PLoS Medicine. 2008, 5 (4): e102-10.1371/journal.pmed.0050102. · DOI · Paper page
  25. Ciarallo L, Sauer AH, Shannon MW: Intravenous magnesium therapy for moderate to severe pediatric asthma: Results of a randomized, placebo-controlled trial. The Journal of Pediatrics. 1996, 129 (6): 809-814. 10.1016/S0022-3476(96)70023-9. · DOI · Paper page
  26. Ciarallo L, Brousseau D, Reinert S: Higher-Dose Intravenous Magnesium Therapy for Children With Moderate to Severe Acute Asthma. 2000, 979-983.

Cited by

See all 63

Metadata

Authors
Dwan, K., Gamble, C., Kolamunnage-Dona, R., Mohammed, S., Powell, C., & Williamson, P. R.
Venue
Trials
Published
2010-05-12
Volume
11
Issue
1
Pages
52-52
ISSN
1745-6215 (ISSN-L)
OpenAlex
W2170245994
Cited by
63
References
9
FWCI
4.63
Type
journal article
Language
en
Open access
gold
Licence
creativecommons.org/licenses/by/2.0 (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Assessing the potential for outcome reporting bias in a review: a tutorial. https://app.cochat.ai/papers/10.1186/1745-6215-11-52
The paper
Dwan, K., Gamble, C., Kolamunnage-Dona, R., Mohammed, S., Powell, C., & Williamson, P. R. (2010). Assessing the potential for outcome reporting bias in a review: a tutorial. Trials. https://doi.org/10.1186/1745-6215-11-52

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.