The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Recent developments in meta‐analysis
Open accessAbstract
The art and science of meta‐analysis, the combination of results from multiple independent studies, is now more than a century old. In the last 30 years, however, as the need for medical research and clinical practice to be based on the totality of relevant and sound evidence has been increasingly recognized, the impact of meta‐analysis has grown enormously. In this paper, we review highlights of recent developments in meta‐analysis in medical research. We outline in particular how emphasis has been placed on (i) heterogeneity and random‐effects analyses; (ii) special consideration in different areas of application; (iii) assessing bias within and across studies; and (iv) extension of ideas to complex evidence synthesis. We conclude the paper with some remarks on ongoing challenges and possible directions for the future. Copyright © 2007 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Self-citationMeta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Self-citationOperating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Statistical Aspects of the Analysis of Data From Retrospective Studies of Disease
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.4.719 · Cited by 14,689
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“With the ever-expanding literature of medical research studies, and the suggestion that the majority of such findings may be false [6], the need for critical evaluations and considered syntheses of multiple studies is an easy case to make.”
BackgroundFunnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369 · cited 1×
“More recent work has included the development of a hierarchical SROC approach [232, 233] that incorporates random effects to allow for among-study heterogeneity and an approach treating sensitivity and specificity as bivariate normally distributed [234].”
BackgroundTowards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · Radiology · doi:10.1148/radiol.2261021292 · Cited by 3,215
Beta blockade during and after myocardial infarction: An overview of the randomized trials
Yusuf, S., Peto, R., Lewis, J., Collins, R., & Sleight, P. (1985) · Progress in Cardiovascular Diseases · doi:10.1016/s0033-0620(85)80003-7 · Cited by 3,205
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Methods for Trend Estimation from Summarized Dose-Response Data, with Applications to Meta-Analysis
Greenland, S., & Longnecker, M. P. (1992) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116237 · Cited by 2,353
Meta-analyses involving cross-over trials: methodological issues
Elbourne, D. R., Altman, D. G., Higgins, J. P., Curtin, F., Worthington, H. V., & Vail, A. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.140 · Cited by 2,305
Self-citationPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · cited 1×
“While parametric tests generally appear to be more powerful than non-parametric [121], Egger’s test has highly inflated type 1 errors in some circumstances when binary outcomes are considered [122–124].”
MethodA modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267 · cited 1×
“This has led to a number of modified tests being developed for binary outcomes [122, 123, 125, 126], and more generally [127].”
BackgroundReporting Recommendations for Tumor Marker Prognostic Studies (REMARK)
McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2005) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/dji237 · Cited by 2,162
Synthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132
Self-citationImproved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075 · cited 1×
“Recent developments in meta-regression have seen an improved variance estimator for association [61, 62], a permutation test to establish the true statistical significance of a positive finding [63] and discussions of advanced models, such as fractional polynomials and splines [64] to quantify non-linear associations.”
BackgroundThe Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
What to add to nothing? Use and avoidance of continuity corrections in meta‐analysis of sparse data
J. Sweeting, M., J. Sutton, A., & C. Lambert, P. (2004) · Statistics in Medicine · doi:10.1002/sim.1761 · Cited by 1,734
Simultaneous comparison of multiple treatments: Combining direct and indirect evidence.
Caldwell, D. M., Ades, A. E., & Higgins, J. P. T. (2005) · BMJ · doi:10.1136/bmj.331.7521.897 · Cited by 1,600 · cited 1×
“The use of MTC to compare all trials in an area is starting to be recognized by prominent general medical journals [172].”
MethodSelf-citationImputing missing standard deviations in meta-analyses can provide accurate results
Furukawa, T. A., Barbui, C., Cipriani, A., Brambilla, P., & Watanabe, N. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.06.006 · Cited by 1,568 · cited 1×
“A variety of methods for imputing these variances have been observed [98, 99], and methods for imputing them for the specific case of change-from-baseline measures have been proposed in a Bayesian framework [80].”
MethodA comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267 · cited 1×
“Recent developments in meta-regression have seen an improved variance estimator for association [61, 62], a permutation test to establish the true statistical significance of a positive finding [63] and discussions of advanced models, such as fractional polynomials and splines [64] to quantify non-linear associations.”
BackgroundSelf-citationMendelian randomization: prospects, potentials, and limitations
Smith, G. D. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh132 · Cited by 1,222
Network meta‐analysis for indirect treatment comparisons
Lumley, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1201 · Cited by 1,206
Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Thompson, S. G. (1994) · BMJ · doi:10.1136/bmj.309.6965.1351 · Cited by 1,017 · cited 1×
“A notable paradigm shift in recent years has been the movement away from simple meta-analyses in favour of explorations of among-study differences [25, 50].”
BackgroundMuch ado about nothing: a comparison of the performance of meta-analytical methods with rare events
Bradburn, M. J., Deeks, J. J., Berlin, J. A., & Russell Localio, A. (2007) · Statistics in Medicine · doi:10.1002/sim.2528 · Cited by 938
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809
A refined method for the meta-analysis of controlled clinical trials with binary outcome
Hartung, J., & Knapp, G. (2001) · Statistics in Medicine · doi:10.1002/sim.1009 · Cited by 775
Issues in the meta-analysis of cluster randomized trials
Donner, A., & Klar, N. (2002) · Statistics in Medicine · doi:10.1002/sim.1301 · Cited by 734 · cited 1×
“Another practically important area of the literature, although still quite small, is specific methodological guidance for non-standard trial designs, such as cross-over trials [187–190] and cluster-randomized trials [191].”
BackgroundThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 2×
“The power of standard meta-analytic methods has been studied by Hedges and Pigott [29, 72].”
BackgroundMethodThe power of statistical tests for moderators in meta-analysis
Hedges, L. V., & Pigott, T. D. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.426 · Cited by 672
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568 · cited 1×
“incorporate in a natural way the uncertainty in estimation of the among-study variance have been described [46].”
BackgroundA brief history of research synthesis
Chalmers, I., Hedges, L. V., & Cooper, H. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001003 · Cited by 552
Bax, L., Yu, L.-M., Ikeda, N., Tsuruta, H., & Moons, K. G. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-50 · Cited by 514 · cited 1×
“The Cochrane Collaboration software, RevMan [270], continues to be developed and a new freely downloadable Excel add-in MIX [271] offers excellent functionality and educational content for those on a tight budget.”
MethodLambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511
Self-citationMultiple-bias modelling for analysis of observational data
Greenland, S. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.00349.x · Cited by 497
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461 · cited 1×
“While it has been suggested that correcting for publication bias is not possible without making untestable assumptions [130], a bound for the bias as a function of the fraction of missing studies has been proposed [131], as has the use of a priori specified selection models [132].”
BackgroundOutcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444
A simple confidence interval for meta-analysis
Sidik, K., & Jonkman, J. N. (2002) · Statistics in Medicine · doi:10.1002/sim.1262 · Cited by 431 · cited 1×
“(However, concerns have been expressed at the substitution of estimates for unknown quantities [44].”
BackgroundPogue, J. M., & Yusuf, S. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00051-2 · Cited by 415
A comparison of heterogeneity variance estimators in combining results of studies
Sidik, K., & Jonkman, J. N. (2007) · Statistics in Medicine · doi:10.1002/sim.2688 · Cited by 380
Sterne, J. A. C., Becker, B. J., & Egger, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch5 · Cited by 341
Imputing response rates from means and standard deviations in meta-analyses
Furukawa, T. A., Cipriani, A., Barbui, C., Brambilla, P., & Watanabe, N. (2005) · International Clinical Psychopharmacology · doi:10.1097/00004850-200501000-00010 · Cited by 337
A bivariate approach to meta-analysis
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122405 · Cited by 327
Modelling publication bias in meta-analysis: a review
Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R. (2000) · Statistical Methods in Medical Research · doi:10.1177/096228020000900503 · Cited by 288
Self-citationWarn, D. E., Thompson, S. G., & Spiegelhalter, D. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1189 · Cited by 277
Simpson's paradox and calculation of number needed to treat from meta-analysis
Cates, C. J. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-1 · Cited by 251
Heterogeneity in Meta-analysis of Data from Epidemiologic Studies: A Commentary
Colditz, G. A., Burdick, E., & Mosteller, F. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117644 · Cited by 245
Obstacles and opportunities in meta-analysis of genetic association studies
Salanti, G., Sanderson, S., & Higgins, J. P. T. (2005) · Genetics in Medicine · doi:10.1097/01.gim.0000151839.12032.1a · Cited by 196
Self-citationNam, I., Mengersen, K., & Garthwaite, P. (2003) · Statistics in Medicine · doi:10.1002/sim.1410 · Cited by 153
Meta-analysis of rare and adverse event data
Sutton, A. J., Cooper, N. J., Lambert, P. C., Jones, D. R., Abrams, K. R., & Sweeting, M. J. (2002) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/14737167.2.4.367 · Cited by 141
Self-citationIdentification and impact of outcome selection bias in meta‐analysis
Williamson, P. R., & Gamble, C. (2005) · Statistics in Medicine · doi:10.1002/sim.2025 · Cited by 132 · cited 1×
“Statistical methods have been put forward to assess the likely impact of selective outcome [143–145] and subgroup [146] reporting.”
MethodHedges, L. V., & Vevea, J. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch9 · Cited by 123
Ades, A. E., & Cliffe, S. (2002) · Medical Decision Making · doi:10.1177/027298902400448920 · Cited by 104
Hahn, S., Williamson, P. R., & Hutton, J. L. (2002) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2002.00314.x · Cited by 97
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Effective techniques in healthy eating and physical activity interventions: A meta-regression.
Michie, S., Abraham, C., Whittington, C., McAteer, J., & Gupta, S. (2009) · Health Psychology · doi:10.1037/a0016136 · Cited by 2,229
Hayes, S. A., & Watson, S. L. (2013) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-012-1604-y · Cited by 1,847
Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729
Quantifying publication bias in meta-analysis
Lin, L., & Chu, H. (2018) · Biometrics · doi:10.1111/biom.12817 · Cited by 1,674
Metadata
- Authors
- Sutton, A. J., & Higgins, J. P. T.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2007-06-25
- Volume
- 27
- Issue
- 5
- Pages
- 625-650
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2001855995
- Cited by
- 590
- References
- 67
- FWCI
- 28.93
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
