A practical introduction to multivariate meta-analysis
The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
Multivariate meta-analysis is becoming increasingly popular and official routines or self-programmed functions have been included in many statistical software. In this article, we review the statistical methods and the related software for multivariate meta-analysis. Emphasis is placed on Bayesian methods using Markov chain Monte Carlo, and codes in WinBUGS are provided. The various model-fitting options are illustrated in two examples and specific guidance is provided on how to run a multivariate meta-analysis using various software packages.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For this situation, a range of methods are available that appropriately model or account for the dependency of the effect estimates. These methods include multivariate meta-analysis,110 multilevel models,111 or robust variance estimation.112”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [110]
In the citing paper
“Multivariate meta–analysis differs from univariate meta–analysis in that it takes into account within–and between–study correlations, reducing bias, and improving precision [63].”
Citations
Cites
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,368
Gelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369 · cited 1×
“Synthesis of diagnostic test accuracy studies is a popular application area of multivariate metaanalysis.(30) Diagnostic test accuracy studies typically report the number of true positives (diseased people correctly diagnosed), false positives (non-diseased people incorrectly diagnosed as diseased), true negatives and…”
BackgroundGRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review
Cowles, M. K., & Carlin, B. P. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476956 · Cited by 2,056
Advanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,580
Unbalanced Repeated-Measures Models with Structured Covariance Matrices
Jennrich, R. I., & Schluchter, M. D. (1986) · Biometrics · doi:10.2307/2530695 · Cited by 1,325
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133 · cited 1×
“This is an important advantage because trial investigators typically compute many outcomes but decide to publish only those indicating a significant treatment.(48) As systematic reviewers and meta-analysts increasingly recognize the selective reporting bias as a major threat to the validity of the meta-analysis result…”
BackgroundEvaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078
Self-citationEstimation of a Single Effect Size: Parametric and Nonparametric Methods
Hedges, L. V., & Olkin, I. (1985) · Statistical Methods for Meta-Analysis · doi:10.1016/b978-0-08-057065-5.50010-5 · Cited by 983
THEORY TESTING: COMBINING PSYCHOMETRIC META‐ANALYSIS AND STRUCTURAL EQUATIONS MODELING
VISWESVARAN, C., & ONES, D. S. (1995) · Personnel Psychology · doi:10.1111/j.1744-6570.1995.tb01784.x · Cited by 926
A unification of models for meta-analysis of diagnostic accuracy studies
Harbord, R. M., Deeks, J. J., Egger, M., Whiting, P., & Sterne, J. A. C. (2007) · Biostatistics · doi:10.1093/biostatistics/kxl004 · Cited by 751
(1999) · Cited by 751
Meta-analytic structural equation modeling: A two-stage approach.
Cheung, M. W.-L., & Chan, W. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.1.40 · Cited by 637
Extending DerSimonian and Laird's methodology to perform multivariate random effects meta‐analyses
Jackson, D., White, I. R., & Thompson, S. G. (2010) · Statistics in Medicine · doi:10.1002/sim.3602 · Cited by 594
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568 · cited 2×
“The use of priors that give support to unrealistically large values of heterogeneity should be avoided and it is suggested to investigate vague prior distributions within a realistic range of values for the data set under consideration in a sensitivity analysis.(47) A major difficulty encountered in practice is that t…”
BackgroundResultBORROWING STRENGTH FROM EXTERNAL TRIALS IN A META-ANALYSIS
Higgins, J. P. T., & WHITEHEAD, A. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961230)15:24<2813::aid-sim366>3.0.co;2-u · Cited by 567
BORROWING STRENGTH FROM EXTERNAL TRIALS IN A META-ANALYSIS
HIGGINS, J. P. T., & WHITEHEAD, A. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961230)15:24<2733::aid-sim562>3.0.co;2-0 · Cited by 557
Multivariate meta‐analysis: Potential and promise
Jackson, D., Riley, R., & White, I. R. (2011) · Statistics in Medicine · doi:10.1002/sim.4172 · Cited by 530
Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511
metandi: Meta-analysis of diagnostic accuracy using hierarchical logistic regression
Harbord, R. M., & Whiting, P. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900203 · Cited by 453 · cited 1×
“Finally, the command metandi performs bivariate meta-analysis of diagnostic test accuracy studies using hierarchical logistic regression.(39) The function takes as arguments the number of true positives (tp), the number of false positives (fp), the number of true negatives (tn) and the number of false negatives (fn).”
MethodStatistical Meta‐Analysis with Applications
Hartung, J., Knapp, G., & Sinha, B. K. (2008) · Wiley Series in Probability and Statistics · doi:10.1002/9780470386347 · Cited by 418
Multivariate Random-effects Meta-regression: Updates to Mvmeta
White, I. R. (2011) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x1101100206 · Cited by 400
Meta-analytic procedures for combining studies with multiple effect sizes.
Rosenthal, R., & Rubin, D. B. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.3.400 · Cited by 394
A bivariate approach to meta-analysis
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122405 · Cited by 327 · cited 1×
“As the studies are assumed independent experiments, likelihood methods can be employed to estimate model parameters.(13) The likelihood is given by:”
MethodDevelopment, feasibility and performance of a health risk appraisal questionnaire for older persons
Stuck, A. E., Kharicha, K., Dapp, U., Anders, J., von Renteln-Kruse, W., Meier-Baumgartner, H. P., Harari, D., Swift, C. G., Ivanova, K., Egger, M., et al. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-1 · Cited by 324
Statistical meta-analysis with applications
(2009) · Choice Reviews Online · doi:10.5860/choice.46-4497 · Cited by 311
Multivariate Random-effects Meta-analysis
White, I. R. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900103 · Cited by 286
Meta-analysis of multiple outcomes by regression with random effects
Berkey, C. S., Hoaglin, D. C., Antczak-Bouckoms, A., Mosteller, F., & Colditz, G. A. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19981130)17:22<2537::aid-sim953>3.0.co;2-c · Cited by 280
Bivariate random-effects meta-analysis and the estimation of between-study correlation
Riley, R. D., Abrams, K. R., Sutton, A. J., Lambert, P. C., & Thompson, J. R. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-3 · Cited by 244
Multivariate Meta-Analysis: The Effect of Ignoring Within-Study Correlation
Riley, R. D. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00593.x · Cited by 225
Riley, R. D., Abrams, K. R., Lambert, P. C., Sutton, A. J., & Thompson, J. R. (2007) · Statistics in Medicine · doi:10.1002/sim.2524 · Cited by 199
Modeling between-trial variance structure in mixed treatment comparisons
Lu, G., & Ades, A. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp032 · Cited by 196
Using Results from Replicated Studies to Estimate Linear Models
Becker, B. J. (1992) · Journal of Educational Statistics · doi:10.2307/1165128 · Cited by 181
Cheung, M. W.-L. (2008) · Psychological Methods · doi:10.1037/a0013163 · Cited by 175
Riley, R. D., Thompson, J. R., & Abrams, K. R. (2008) · Biostatistics · doi:10.1093/biostatistics/kxm023 · Cited by 167
Golfinopoulos, V., Salanti, G., Pavlidis, N., & Ioannidis, J. P. (2007) · The Lancet Oncology · doi:10.1016/s1470-2045(07)70281-4 · Cited by 159
Self-citationNam, I., Mengersen, K., & Garthwaite, P. (2003) · Statistics in Medicine · doi:10.1002/sim.1410 · Cited by 153
Attention should be given to multiplicity issues in systematic reviews
Bender, R., Bunce, C., Clarke, M., Gates, S., Lange, S., Pace, N. L., & Thorlund, K. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.03.004 · Cited by 121
Kirkham, J. J., Riley, R. D., & Williamson, P. R. (2012) · Statistics in Medicine · doi:10.1002/sim.5356 · Cited by 119
Averaging Dependent Effect Sizes in Meta-Analysis: a Cautionary Note about Procedures
Marín-Martínez, F., & Sánchez-Meca, J. (1999) · The Spanish Journal of Psychology · doi:10.1017/s1138741600005436 · Cited by 99
MetaDAS: A SAS macro for meta-analysis of diagnostic accuracy studies
Takwoingi, Y., & Deeks, J. J. (2010) · Cited by 80
Yanez, N. D., Kronmal, R. A., & Shemanski, L. R. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19981130)17:22<2597::aid-sim940>3.0.co;2-g · Cited by 77
Impact of approximating or ignoring within‐study covariances in multivariate meta‐analyses
Ishak, K. J., Platt, R. W., Joseph, L., & Hanley, J. A. (2008) · Statistics in Medicine · doi:10.1002/sim.2913 · Cited by 57
Fixed-Effects Meta-Analyses as Multiple-Group Structural Equation Models
Cheung, M. W. ‐. (2010) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2010.489367 · Cited by 31
Research works printed but not linked: 6
- 10.1037/0033-2909.99.3.400 · doi:10.1037/0033-2909.99.3.400 · Paper page
- Houwelingen HC, 1993, Stat Med, 12, 2272
- Riley RD, 2007, BMC Med Res Methodol, 7, 1
- 10.3102/10769986017004341 · doi:10.3102/10769986017004341 · Paper page
- 10.1002/sim.2112 · DOI · Paper page
- Kirkham JJ, 2010, Br Med J, 15, 340
Cited by
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 13,262
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473
“An appropriate analysis of multiple outcomes should be undertaken using multivariate methods to account for the dependency between outcomes [45,46].”
Self-citationROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272
“However, there are many variants and extensions to this, with the options of modeling outcome data explicitly (for example, taking a logistic regression approach for binary data [21], of modeling two or more outcomes simultaneously (bivariate or multivariate metaanalysis [22], of modeling multiple treatment effects si…”
Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729
“2012, 3 80–97 (Jackson et al., 2011; Mavridis and Salanti, 2012).”
Self-citationPage, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916
Metadata
- Authors
- Mavridis, D., & Salanti, G.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Salanti, Georgia
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2012-01-23
- Volume
- 22
- Issue
- 2
- Pages
- 133-158
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2009253656
- Semantic Scholar
- 19a357072ef5a449dfa8a641dce69f5c7777f631
- Cited by
- 255
- References
- 44
- Citations by year
- 2026: 8 · 2025: 27 · 2024: 12 · 2023: 17 · 2022: 38 · 2021: 28 · 2020: 23 · 2019: 13 · 2018: 17 · 2017: 10 · 2016: 17 · 2015: 15 · 2014: 15 · 2013: 10 · 2012: 5
- FWCI
- 16.44
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- multivariate meta-analysis, Bayesian analysis, Markov chain Monte Carlo, WinBUGS, statistical software, model fitting
- MeSH terms
- Bayes Theorem, Humans, Markov Chains, Monte Carlo Method, Software, Meta-Analysis as Topic, Multivariate Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.