CoChat

Impact of analysing continuous outcomes using final values, change scores and analysis of covariance on the performance of meta‐analytic methods: a simulation study

Open access
McKenzie, Joanne E., Herbison, G. Peter, Deeks, Jonathan J. · 2016 · Research Synthesis Methods
Cited by 40

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Abstract

When meta‐analysing intervention effects calculated from continuous outcomes, meta‐analysts often encounter few trials, with potentially a small number of participants, and a variety of trial analytical methods. It is important to know how these factors affect the performance of inverse‐variance fixed and DerSimonian and Laird random effects meta‐analytical methods. We examined this performance using a simulation study. Meta‐analysing estimates of intervention effect from final values, change scores, ANCOVA or a random mix of the three yielded unbiased estimates of pooled intervention effect. The impact of trial analytical method on the meta‐analytic performance measures was important when there was no or little heterogeneity, but was of little relevance as heterogeneity increased. On the basis of larger than nominal type I error rates and poor coverage, the inverse‐variance fixed effect method should not be used when there are few small trials. When there are few small trials, random effects meta‐analysis is preferable to fixed effect meta‐analysis. Meta‐analytic estimates need to be cautiously interpreted; type I error rates will be larger than nominal, and confidence intervals will be too narrow. Use of trial analytical methods that are more efficient in these circumstances may have the unintended consequence of further exacerbating these issues. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/jrsm.1196

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 11 references

Research works printed but not linked: 36

  1. Armitage P, 2002, Statistical Methods in Medical Research, 10.1002/9780470773666 · DOI · Paper page
  2. Banerjee S, 2008, Caveats in the Meta‐analyses of Continuous Data: A Simulation Study
  3. 10.1002/pon.3205 · DOI · Paper page
  4. 10.1201/9781420035926 · DOI · Paper page
  5. 10.1002/jrsm.12 · DOI · Paper page
  6. 10.1016/j.jclinepi.2007.02.006 · DOI · Paper page
  7. 10.1002/sim.2673 · DOI · Paper page
  8. Cohen J, 1988, Statistical Power Analysis for the Behavioural Sciences
  9. 10.1016/0197-2456(86)90046-2 · DOI · Paper page
  10. 10.1111/j.0006-341X.1999.00732.x · DOI · Paper page
  11. 10.1002/sim.4780111304 · DOI · Paper page
  12. 10.1111/j.0006-341X.2004.00235.x · doi:10.1111/j.0006-341x.2004.00235.x
  13. 10.1002/(SICI)1097-0258(19960330)15:6<619::AID-SIM188>3.0.CO;2-A · DOI · Paper page
  14. 10.1002/(SICI)1521-4036(199912)41:8<901::AID-BIMJ901>3.0.CO;2-W · DOI
  15. 10.1002/sim.1186 · DOI · Paper page
  16. Larsen RJ, 1986, An Introduction to Mathematical Statistics and Its Applications
  17. MckenzieJE.2011.Methodological issues in meta‐analysis of randomised controlled trials with continuous outcomes (PhD thesis). Doctor of Philosophy Monash University.
  18. 10.1002/(SICI)1097-0258(19990215)18:3<321::AID-SIM28>3.0.CO;2-P · DOI · Paper page
  19. 10.6028/jres.087.022 · DOI · Paper page
  20. Raudenbush SW, 2009, The Handbook of Research Synthesis and Meta‐analysis
  21. 10.1016/j.jclinepi.2010.08.002 · DOI
  22. 10.1002/sim.5726 · DOI · Paper page
  23. 10.1037/1082-989X.13.1.31 · DOI · Paper page
  24. 10.1016/S0197-2456(00)00105-7 · doi:10.1016/s0197-2456(00)00105-7
  25. 10.1002/sim.2682 · DOI · Paper page
  26. 10.1002/9780470723586 · DOI · Paper page
  27. 10.1002/9780470723586.ch16 · doi:10.1002/9780470723586.ch16
  28. 10.1002/sim.4780080410 · DOI · Paper page
  29. 10.1080/10543409308835065 · DOI
  30. Sharp S, 1997, sbe16: meta‐analysis, Stata Technical Bulletin, 38, 9
  31. Sharp S, 1998, sbe16.1: new syntax and output for the meta‐analysis command, Stata Technical Bulletin, 42, 6
  32. Sharp S, 1998, sbe16.2: corrections to the meta‐analysis command, Stata Technical Bulletin, 43, 15
  33. 10.1111/j.1467-9876.2005.00489.x · DOI · Paper page
  34. STATACORP, 2007, Stata Statistical Software: Release 10
  35. 10.1177/009286150103500417 · DOI
  36. Ziegler S, 2001, Deficits and remedy of the standard random effects methods in meta‐analysis, Methods of Information in Medicine, 40, 148, 10.1055/s-0038-1634478 · DOI

Cited by

Metadata

Authors
McKenzie, J. E., Herbison, G. P., & Deeks, J. J.
Venue
Research Synthesis Methods
Publisher
Wiley
Published
2015-12-29
Volume
7
Issue
4
Pages
371-386
ISSN
  • 1759-2879 (ISSN-L)
  • 1759-2887
OpenAlex
W2219274198
Cited by
40
References
11
FWCI
1.79
Type
journal article
Language
en
Open access
hybrid
Funders
University of Otago

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.