Multivariate meta‐analysis: Potential and promise
Open accessAbstract
The multivariate random effects model is a generalization of the standard univariate model. Multivariate meta‐analysis is becoming more commonly used and the techniques and related computer software, although continually under development, are now in place. In order to raise awareness of the multivariate methods, and discuss their advantages and disadvantages, we organized a one day ‘Multivariate meta‐analysis’ event at the Royal Statistical Society. In addition to disseminating the most recent developments, we also received an abundance of comments, concerns, insights, critiques and encouragement. This article provides a balanced account of the day's discourse. By giving others the opportunity to respond to our assessment, we hope to ensure that the various view points and opinions are aired before multivariate meta‐analysis simply becomes another widely used de facto method without any proper consideration of it by the medical statistics community. We describe the areas of application that multivariate meta‐analysis has found, the methods available, the difficulties typically encountered and the arguments for and against the multivariate methods, using four representative but contrasting examples. We conclude that the multivariate methods can be useful, and in particular can provide estimates with better statistical properties, but also that these benefits come at the price of making more assumptions which do not result in better inference in every case. Although there is evidence that multivariate meta‐analysis has considerable potential, it must be even more carefully applied than its univariate counterpart in practice. Copyright © 2011 John Wiley & Sons, Ltd.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pairwise meta-analytic estimates are usually reported in addition to network estimates.
“As interventions are by definition heterogeneous and pairwise meta‐analytic estimates are usually reported in addition to the network estimates, 30 random effects pairwise meta‐analyses were first used to obtain pooled SMDs (with associated CIs) for weight loss, BMI, WC, %BF and fitness, respectively, with the I 2 statistic used to quantify heterogeneity.”
From What exercise prescription is optimal to improve body composition and cardiorespiratory fitness in adults living with obesity? A network meta‐analysis (2021) · cites this paper as [30]
Only 1 study measured well-being in the comparison of MBPs with passive controls, so the result is unreliable.
“but only 1 study measured it, so although multivariate meta–analysis “borrows strength” from other outcomes and studies through their correlations, this result is unreliable [282].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · see citing works
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · see citing works
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
Statistical Methods for Meta‐Analysis
Zhou, X.-H., Obuchowski, N., & McClish, D. (2008) · Wiley series in probability and statistics · doi:10.1002/9780470317082.ch12 · Cited by 6,648 · see citing works
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,372 · see citing works
The Emerging Risk Factors Collaboration (2010) · The Lancet · doi:10.1016/s0140-6736(10)60484-9 · Cited by 5,125 · see citing works
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552 · see citing works
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369 · cited 3× · see citing works
“Perhaps the most common medical application of multivariate meta-analysis is the bivariate metaanalysis of studies of diagnostic test accuracy [5, 28].”
BackgroundResultA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1× · see citing works
“Similarly, it allows a joint prediction region [63] for the true sensitivity and specificity in an individual study setting.”
BackgroundRobust variance estimation in meta‐regression with dependent effect size estimates
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.5 · Cited by 2,086 · see citing works
The Handbook of Research Synthesis.
Light, R. J., Cooper, H., & Hedges, L. V. (1994) · Journal of the American Statistical Association · doi:10.2307/2291021 · Cited by 1,880 · see citing works
Advanced methods in meta‐analysis: multivariate approach and meta‐regression
van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1040 · Cited by 1,574 · see citing works
A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations
Rutter, C. M., & Gatsonis, C. (2001) · Statistics in Medicine · doi:10.1002/sim.942 · Cited by 1,510 · see citing works
Unbalanced repeated-measures models with structured covariance matrices.
Jennrich, R., & Schluchter, M. (1986) · Biometrics · doi:10.2307/2530695 · Cited by 1,382 · see citing works
Network meta‐analysis for indirect treatment comparisons
Lumley, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1201 · Cited by 1,206 · see citing works
Evaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078 · see citing works
Assessing Evidence Inconsistency in Mixed Treatment Comparisons
Lu, G., & Ades, A. E. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000001302 · Cited by 756 · see citing works
A unification of models for meta-analysis of diagnostic accuracy studies
Harbord, R. M., Deeks, J. J., Egger, M., Whiting, P. F., & Sterne, J. A. C. (2006) · Biostatistics · doi:10.1093/biostatistics/kxl004 · Cited by 753 · see citing works
Chu, H., & Cole, S. R. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.06.011 · Cited by 708 · see citing works
Principles of Statistical Inference
Cox, D. R. (2006) · doi:10.1017/cbo9780511813559 · Cited by 670 · see citing works
<i>Meta-analysis:</i> Potentials and promise
Egger, M., & Smith, G. D. (1997) · BMJ · doi:10.1136/bmj.315.7119.1371 · Cited by 646 · see citing works
Extending DerSimonian and Laird's methodology to perform multivariate random effects meta‐analyses
Jackson, D., White, I. R., & Thompson, S. G. (2009) · Statistics in Medicine · doi:10.1002/sim.3602 · Cited by 591 · see citing works
Self-citationA LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
Hardy, R., & Thompson, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6 · Cited by 517 · see citing works
Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511 · see citing works
Egger, M., & Smith, G. D. (1995) · BMJ · doi:10.1136/bmj.310.6982.752 · Cited by 508 · see citing works
Stochastically dependent effect sizes.
Gleser, L. J., & Olkin, I. (1994) · Cited by 485 · see citing works
metandi: Meta-analysis of diagnostic accuracy using hierarchical logistic regression
Harbord, R. M., & Whiting, P. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900203 · Cited by 453 · cited 3× · see citing works
“The Stata program metandi [30] was used to analyse our first example bivariately using maximum likelihood.”
BackgroundMethodInfluentialThe binomial distribution of meta-analysis was preferred to model within-study variability
Hamza, T. H., van Houwelingen, H. C., & Stijnen, T. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.016 · Cited by 388 · see citing works
A bivariate approach to meta-analysis
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122405 · Cited by 327 · see citing works
Systematic Reviews: Meta-analysis and its problems
Eysenck, H. J. (1994) · BMJ · doi:10.1136/bmj.309.6957.789 · Cited by 295 · see citing works
Multivariate Random-effects Meta-analysis
White, I. R. (2009) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900103 · Cited by 289 · see citing works
Self-citationKertai, M. D., Boersma, E., Bax, J. J., Heijenbrok‐Kal, M. H., Hunink, M. G. M., L’talien, G. J., Roelandt, J. R. T. C., van Urk, H., & Poldermans, D. (2003) · Heart · doi:10.1136/heart.89.11.1327 · Cited by 280 · see citing works
Meta-analysis of multiple outcomes by regression with random effects
Berkey, C. S., Hoaglin, D. C., Antczak‐Bouckoms, A., Mosteller, F. C., & Colditz, G. A. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19981130)17:22 · Cited by 274 · see citing works
Meta‐analysis of continuous outcome data from individual patients
Higgins, J. P. T., Whitehead, A., Turner, R., Omar, R., & Thompson, S. G. (2001) · Statistics in Medicine · doi:10.1002/sim.918 · Cited by 265 · see citing works
Bivariate random-effects meta-analysis and the estimation of between-study correlation
Riley, R. D., Abrams, K. R., Sutton, A. J., Lambert, P. C., & Thompson, J. R. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-3 · Cited by 243 · see citing works
Self-citationA multivariate mixed linear model for meta-analysis.
Kalaian, H. A., & Raudenbush, S. W. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.3.227 · Cited by 229 · see citing works
Multivariate Meta-Analysis: The Effect of Ignoring Within-Study Correlation
Riley, R. D. (2009) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2008.00593.x · Cited by 227 · see citing works
Self-citationJackson, D., Bowden, J., & Baker, R. (2010) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2009.09.017 · Cited by 226 · see citing works
Self-citationModeling multivariate effect sizes.
Raudenbush, S. W., Becker, B. J., & Kalaian, H. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.1.111 · Cited by 195 · see citing works
Riley, R. D., Abrams, K. R., Lambert, P. C., Sutton, A. J., & Thompson, J. R. (2006) · Statistics in Medicine · doi:10.1002/sim.2524 · Cited by 195 · see citing works
Self-citationInvestigating heterogeneity in an individual patient data meta-analysis of time to event outcomes
Smith, C. T., Williamson, P. R., & Marson, A. G. (2005) · Statistics in Medicine · doi:10.1002/sim.2050 · Cited by 173 · see citing works
Riley, R. D., Thompson, J. R., & Abrams, K. R. (2007) · Biostatistics · doi:10.1093/biostatistics/kxm023 · Cited by 167 · see citing works
Self-citationNam, I., Mengersen, K., & Garthwaite, P. (2003) · Statistics in Medicine · doi:10.1002/sim.1410 · Cited by 153 · see citing works
On meta-analytic assessment of surrogate outcomes
Gail, M. H. (2000) · Biostatistics · doi:10.1093/biostatistics/1.3.231 · Cited by 149 · see citing works
Flexible parametric models for random‐effects distributions
Lee, K. J., & Thompson, S. G. (2007) · Statistics in Medicine · doi:10.1002/sim.2897 · Cited by 148 · see citing works
Evaluating diagnostic tests: The area under the ROC curve and the balance of errors
Hand, D. J. (2010) · Statistics in Medicine · doi:10.1002/sim.3859 · Cited by 140 · see citing works
Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials
Jones, A. P., Riley, R. D., Williamson, P. R., & Whitehead, A. (2009) · Clinical Trials · doi:10.1177/1740774508100984 · Cited by 137 · see citing works
Self-citationTHE INCIDENCE OF CYSTIC FIBROSIS
Kosorok, M. R., WEI, W.-H., & Farrell, P. M. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960315)15:5 · Cited by 132 · see citing works
Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds
Hamza, T. H., Arends, L. R., van Houwelingen, H. C., & Stijnen, T. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-73 · Cited by 129 · see citing works
Paul, M., Riebler, A., Bachmann, L. M., Rue, H., & Held, L. (2010) · Statistics in Medicine · doi:10.1002/sim.3858 · Cited by 98 · see citing works
Meta‐analysis of diagnostic test studies using individual patient data and aggregate data
Riley, R. D., Dodd, S. R., Craig, J. V., Thompson, J. R., & Williamson, P. R. (2008) · Statistics in Medicine · doi:10.1002/sim.3441 · Cited by 98 · see citing works
Self-citationTandon, S., Smith, C. T., Riley, R. D., Boyd, M. T., & Jones, T. M. (2010) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-09-0981 · Cited by 95 · see citing works
Self-citationMeta‐analysis of ordinal outcomes using individual patient data
Whitehead, A., Omar, R., Higgins, J. P. T., Savaluny, E., Turner, R., & Thompson, S. G. (2001) · Statistics in Medicine · doi:10.1002/sim.919 · Cited by 93 · see citing works
Chu, H., Nie, L., Cole, S. R., et al. (2009) · Statistics in Medicine · doi:10.1002/sim.3627 · Cited by 91 · see citing works
A new approach to outliers in meta-analysis
Baker, R., & Jackson, D. (2007) · Health Care Management Science · doi:10.1007/s10729-007-9041-8 · Cited by 90 · see citing works
Self-citationMeta‐analysis of summary survival curve data
Arends, L. R., Hunink, M. G. M., & Stijnen, T. (2008) · Statistics in Medicine · doi:10.1002/sim.3311 · Cited by 88 · see citing works
Meta-analysis of genetic studies using Mendelian randomization—a multivariate approach
Thompson, J. R., Minelli, C., Abrams, K. R., Tobin, M. D., & Riley, R. D. (2005) · Statistics in Medicine · doi:10.1002/sim.2100 · Cited by 85 · see citing works
Self-citationMetaDAS: A SAS macro for meta-analysis of diagnostic accuracy studies
Takwoingi, Y., & Deeks, J. J. (2010) · Cited by 80 · see citing works
Simel, D. L., & Bossuyt, P. M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.02.007 · Cited by 77 · see citing works
Impact of approximating or ignoring within‐study covariances in multivariate meta‐analyses
Ishak, K. J., Platt, R. W., Joseph, L., & Hanley, J. A. (2007) · Statistics in Medicine · doi:10.1002/sim.2913 · Cited by 57 · see citing works
The Fibrinogen Studies Collaboration (2009) · Statistics in Medicine · doi:10.1002/sim.3540 · Cited by 56 · see citing works
Riley, R. D., Sutton, A. J., Abrams, K. R., & Lambert, P. C. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.01.018 · Cited by 53 · see citing works
Self-citationA method for the meta‐analysis of mutually exclusive binary outcomes
Trikalinos, T. A., & Olkin, I. (2008) · Statistics in Medicine · doi:10.1002/sim.3299 · Cited by 46 · see citing works
Meta‐Analysis of Diagnostic Test Accuracy Studies with Multiple Thresholds using Survival Methods
Putter, H., Fiocco, M. F., & Stijnen, T. (2009) · Biometrical Journal · doi:10.1002/bimj.200900073 · Cited by 44 · see citing works
Generalized least squares for the synthesis of correlated information
de González, A. B. (2003) · Biostatistics · doi:10.1093/biostatistics/4.3.423 · Cited by 41 · see citing works
Regression analysis of discrete time survival data under heterogeneity
Xue, X., & Brookmeyer, R. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970915)16:17 · Cited by 34 · see citing works
Multiple-outcomes Meta-analysis of Treatments for Periodontal Disease
Berkey, C. S., Antczak‐Bouckoms, A., Hoaglin, D. C., Mosteller, E., & Pihlstrom, B. L. (1995) · Journal of Dental Research · doi:10.1177/00220345950740040201 · Cited by 31 · see citing works
Ritz, J., Demidenko, E., & Spiegelman, D. L. (2007) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2007.07.004 · Cited by 31 · see citing works
Multivariate meta analysis with potentially correlated marketing study results
Sohn, S. Y. (2000) · Naval Research Logistics (NRL) · doi:10.1002/1520-6750(200009)47:6 · Cited by 22 · see citing works
Komuro, H., Valentine, M. B., Rowe, S. T., et al. (1998) · Journal of Pediatric Surgery · doi:10.1016/s0022-3468(98)90612-1 · Cited by 21 · see citing works
The Significance Level of the Standard Test for a Treatment Effect in Meta-analysis
Jackson, D. (2009) · Statistics in Biopharmaceutical Research · doi:10.1198/sbr.2009.0009 · Cited by 12 · see citing works
Self-citationModelling Reporting Bias: The Operative Mortality Rate for Ruptured Abdominal Aortic Aneurysm Repair
Jackson, D., Copas, J. B., & Sutton, A. J. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2005.00375.x · Cited by 11 · see citing works
Self-citationThe Significance Level of Meta-Regression's Standard Hypothesis Test
Jackson, D. (2008) · Communication in Statistics- Theory and Methods · doi:10.1080/03610920801893863 · Cited by 8 · see citing works
Self-citationMeta-analysis: formulating, evaluating, combining, and reporting
Normand, S.-L. T. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990215)18:3
Hahn, S., Williamson, P. R., Hutton, J. L., Garner, P., & Flynn, E. V. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24
Explaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Flagged first, then by citations
Research works printed but not linked: 2
- [14]White IR Multivariate meta-analysis The Stata Journal 2009 9 40 56
- [69]Raudenbush SW Becker BJ Kalaian H Modeling multivariate effect sizes Psychological Bulletin 1988 103 111 120
Cited by
Sorted by citations
Graphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · see citing works
Sordo, L., Barrio, G., Bravo, M. J., Indave, B. I., Degenhardt, L., Wiessing, L., Ferri, M., & Pastor-Barriuso, R. (2017) · BMJ · doi:10.1136/bmj.j1550 · Cited by 1,816 · see citing works
Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729 · see citing works
Cheung, M. W.-L. (2014) · Psychological Methods · doi:10.1037/a0032968 · Cited by 1,342 · see citing works
White, I. R., Barrett, J. K., Jackson, D., & Higgins, J. P. T. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1045 · Cited by 1,114 · see citing works
Self-citationMethodological issues and advances in biological meta-analysis
Nakagawa, S., & Santos, E. S. A. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9555-5 · Cited by 1,032 · see citing works
Reducing and meta-analysing estimates from distributed lag non-linear models
Gasparrini, A., & Armstrong, B. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-1 · Cited by 1,081 · see citing works
Small sample adjustments for robust variance estimation with meta-regression.
Tipton, E. (2015) · Psychological Methods · doi:10.1037/met0000011 · Cited by 810 · see citing works
Multivariate meta-analysis for non-linear and other multi-parameter associations
Gasparrini, A., Armstrong, B., & Kenward, M. G. (2012) · Statistics in Medicine · doi:10.1002/sim.5471 · Cited by 772 · see citing works
A meta-analysis of procedures to change implicit measures
Forscher, P. S., Lai, C. K., Axt, J. R., Ebersole, C. R., Herman, M., Devine, P. G., & Nosek, B. A. (2019) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000160 · Cited by 700 · see citing works
Tanner-Smith, E. E., & Tipton, E. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1091 · Cited by 588 · see citing works
Meta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579 · see citing works
Quantifying the impact of between-study heterogeneity in multivariate meta-analyses
Jackson, D., White, I. R., & Riley, R. D. (2012) · Statistics in Medicine · doi:10.1002/sim.5453 · Cited by 577 · see citing works
Self-citationBurke, D. L., Ensor, J., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7141 · Cited by 538 · see citing works
Self-citation<b>netmeta</b> : An <i>R</i> Package for Network Meta-Analysis Using Frequentist Methods
Balduzzi, S., Rücker, G., Nikolakopoulou, A., Papakonstantinou, T., Salanti, G., Efthimiou, O., & Schwarzer, G. (2023) · Journal of Statistical Software · doi:10.18637/jss.v106.i02 · Cited by 532 · see citing works
A graphical tool for locating inconsistency in network meta-analyses
Krahn, U., Binder, H., & König, J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-35 · Cited by 458 · see citing works
Meta-analysis of multiple outcomes: a multilevel approach
Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0527-2 · Cited by 450 · see citing works
Multivariate Random-effects Meta-regression: Updates to Mvmeta
White, I. R. (2011) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x1101100206 · Cited by 400 · see citing works
Self-citationGetReal in network meta-analysis: a review of the methodology
Efthimiou, O., Debray, T. P. A., van Valkenhoef, G., Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., Salanti, G., & on behalf of GetReal Methods Review Group (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1195 · Cited by 401 · see citing works
Handling Complex Meta-analytic Data Structures Using Robust Variance Estimates: a Tutorial in R
Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016) · Journal of Developmental and Life-Course Criminology · doi:10.1007/s40865-016-0026-5 · Cited by 390 · see citing works
Testing for funnel plot asymmetry of standardized mean differences
Pustejovsky, J. E., & Rodgers, M. A. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1332 · Cited by 354 · see citing works
Synthesising quantitative evidence in systematic reviews of complex health interventions
Higgins, J. P. T., López-López, J. A., Becker, B. J., Davies, S. R., Dawson, S., Grimshaw, J. M., McGuinness, L. A., Moore, T. H. M., Rehfuess, E. A., Thomas, J., et al. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000858 · Cited by 322 · see citing works
Galante, J., Friedrich, C., Dawson, A. F., Modrego-Alarcón, M., Gebbing, P., Delgado-Suárez, I., Gupta, R., Dean, L., Dalgleish, T., White, I. R., et al. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003481 · Cited by 319 · see citing works
Self-citationA practical introduction to multivariate meta-analysis
Mavridis, D., & Salanti, G. (2013) · Statistical Methods in Medical Research · doi:10.1177/0962280211432219 · Cited by 255 · see citing works
Meta-analytic structural equation modeling with moderating effects on SEM parameters.
Jak, S., & Cheung, M. W. L. (2020) · Psychological Methods · doi:10.1037/met0000245 · Cited by 262 · see citing works
O'Donoghue, G., Blake, C., Cunningham, C., Lennon, O., & Perrotta, C. (2021) · Obesity Reviews · doi:10.1111/obr.13137 · Cited by 190 · see citing works
A historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165 · see citing works
Meta-analytic insights into evolutionary ecology: an introduction and synthesis
Nakagawa, S., & Poulin, R. (2012) · Evolutionary Ecology · doi:10.1007/s10682-012-9593-z · Cited by 77 · see citing works
Meta-Analysis of Complex Interventions
Tanner-Smith, E. E., & Grant, S. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014112 · Cited by 62 · see citing works
Xu, Y., Ding, W., Wang, Y., Tan, Y., Xi, C., Ye, N., Wu, D., & Xu, X. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246892 · Cited by 42 · see citing works
Bayesian multivariate meta-analysis of multiple factors
Lin, L., & Chu, H. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1293 · Cited by 19 · see citing works
Lu, J., Yang, Y., Chen, H., Ma, H., & Tan, Y. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1362127 · Cited by 11 · see citing works
- Authors
- Jackson, D., Riley, R., & White, I. R.
- Affiliations
- MRC Biostatistics Unit (GB)
- University of Birmingham (GB)
- Corresponding author
- Jackson, Dan
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2011-01-26
- Volume
- 30
- Issue
- 20
- Pages
- 2481-2498
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- PMC
- PMC3470931
- OpenAlex
- W2147527890
- Cited by
- 530
- References
- 76
- Citations by year
- 2026: 32 · 2025: 46 · 2024: 41 · 2023: 42 · 2022: 43 · 2021: 44 · 2020: 41 · 2019: 38 · 2018: 31 · 2017: 29 · 2016: 25 · 2015: 39 · 2014: 26 · 2013: 30 · 2012: 17
- FWCI
- 26.56
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Statistics in Medicine (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Funders
- Medical Research Council (U.1052.00.006)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- multivariate meta-analysis, statistical inference, methodological critique
- MeSH terms
- N-Myc Proto-Oncogene Protein, Carcinoma, Squamous Cell, Cardiovascular Diseases, Chromosome Deletion, Chromosomes, Human, Pair 1, Computer Simulation, Electrocardiography, Humans, Neuroblastoma, Nuclear Proteins, Oropharyngeal Neoplasms, Meta-Analysis as Topic, Models, Statistical, Oncogene Proteins, Multivariate Analysis, Disease-Free Survival, Chromosome Disorders
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
