A multivariate meta‐analysis approach for reducing the impact of outcome reporting bias in systematic reviews
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Multivariate meta‐analysis allows the joint synthesis of multiple correlated outcomes from randomised trials, and is an alternative to a separate univariate meta‐analysis of each outcome independently. Usually not all trials report all outcomes; furthermore, outcome reporting bias (ORB) within trials, where an outcome is measured and analysed but not reported on the basis of the results, may cause a biased set of the evidence to be available for some outcomes, potentially affecting the significance and direction of meta‐analysis results. The multivariate approach, however, allows one to ‘borrow strength’ across correlated outcomes, to potentially reduce the impact of ORB. Assuming ORB missing data mechanisms, we aim to investigate the magnitude of bias in the pooled treatment effect estimates for multiple outcomes using univariate meta‐analysis, and to determine whether the ‘borrowing of strength’ from multivariate meta‐analysis can reduce the impact of ORB. A simulation study was conducted for a bivariate fixed effect meta‐analysis of two correlated outcomes. The approach is illustrated by application to a Cochrane systematic review. Results show that the ‘borrowing of strength’ from a multivariate meta‐analysis can reduce the impact of ORB on the pooled treatment effect estimates. We also examine the use of the Pearson correlation as a novel approach for dealing with missing within‐study correlations, and provide an extension to bivariate random‐effects models that reduce ORB in the presence of heterogeneity. Copyright © 2012 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 1×
“Empirical research provides strong evidence that outcome reporting bias is a significant problem in randomised controlled trials [1, 2].”
BackgroundSelf-citationKirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationMultivariate meta‐analysis: Potential and promise
Jackson, D., Riley, R., & White, I. R. (2011) · Statistics in Medicine · doi:10.1002/sim.4172 · Cited by 530
Self-citationComparison of protocols and registry entries to published reports for randomised controlled trials
Dwan, K., Altman, D. G., Cresswell, L., Blundell, M., Gamble, C. L., & Williamson, P. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000031.pub2 · Cited by 243 · cited 1×
“Empirical research provides strong evidence that outcome reporting bias is a significant problem in randomised controlled trials [1, 2].”
BackgroundSelf-citationFrequency and reasons for outcome reporting bias in clinical trials: interviews with trialists
Smyth, R. M. D., Kirkham, J. J., Jacoby, A., Altman, D. G., Gamble, C., & Williamson, P. R. (2011) · BMJ · doi:10.1136/bmj.c7153 · Cited by 220 · cited 2×
“findings from a series of semistructured interviews with trialists, which found that in nearly a quarter of trials the direction of the main findings influenced the trialists decision not to analyse and hence report on prespecified outcome data [3].”
BackgroundSelf-citationIdentification and impact of outcome selection bias in meta‐analysis
Williamson, P. R., & Gamble, C. (2005) · Statistics in Medicine · doi:10.1002/sim.2025 · Cited by 132 · cited 1×
“A previous study investigated the impact of selective outcome reporting in a review comparing beta-lactam and beta-lactam–aminoglycoside combination therapy in the treatment of cancer patients with neutropenia [10].”
BackgroundSelf-citationApplication and investigation of a bound for outcome reporting bias
Williamson, P. R., & Gamble, C. (2007) · Trials · doi:10.1186/1745-6215-8-9 · Cited by 26 · cited 2×
“In the recently published Outcome Reporting Bias in Trials (ORBIT) study [5], the maximum bias bound approach was used in a sensitivity analysis [6] to estimate the impact of outcome reporting bias on the review meta-analysis.”
MethodSelf-citation
Cited by
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127
Self-citationGetReal in network meta-analysis: a review of the methodology
Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., Salanti, G., & on behalf of GetReal Methods Review Group (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1195 · Cited by 401
“On the other hand, a joint meta-analysis of all outcomes, which incorporates possible correlations, can increase precision by ‘borrowing of strength’ across outcomes and may reduce the impact of outcome reporting bias (Jackson et al., 2011; Kirkham et al., 2012).”
BackgroundDealing with effect size multiplicity in systematic reviews and meta‐analyses
López‐López, J. A., Page, M. J., Lipsey, M. W., & Higgins, J. P. T. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1310 · Cited by 294
A practical introduction to multivariate meta-analysis
Mavridis, D., & Salanti, G. (2013) · Statistical Methods in Medical Research · doi:10.1177/0962280211432219 · Cited by 255
The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles
O’Boyle, E. H., Banks, G. C., & Gonzalez-Mulé, E. (2017) · Journal of Management · doi:10.1177/0149206314527133 · Cited by 246
Metadata
- Authors
- Kirkham, J. J., Riley, R. D., & Williamson, P. R.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2012-04-25
- Volume
- 31
- Issue
- 20
- Pages
- 2179-2195
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2165762701
- Cited by
- 119
- References
- 7
- FWCI
- 3.89
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A multivariate meta‐analysis approach for reducing the impact of outcome reporting bias in systematic reviews. https://app.cochat.ai/papers/10.1002/sim.5356
- The paper
- Kirkham, J. J., Riley, R. D., & Williamson, P. R. (2012). A multivariate meta‐analysis approach for reducing the impact of outcome reporting bias in systematic reviews. Statistics in Medicine. https://doi.org/10.1002/sim.5356
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.