Meta-analysis: Potentials and promise
Open accessAbstract
The number of papers published on meta-analyses in medical research has increased sharply in the past 10 years (fig 1). The merits and perils of the somewhat mysterious procedure of meta-analysis, however, continue to be debated in the medical community.1 23 What, then, is meta-analysis? A useful definition was given by Huque: “A statistical analysis that combines or integrates the results of several independent clinical trials considered by the analyst to be ‘combinable.’” 4 The terminology, however, is still debated, and expressions used concurrently include “overview,” “pooling,” and “quantitative synthesis.” We believe that the term meta-analysis should be used to describe the statistical integration of separate studies, whereas “systematic review” is most appropriate for denoting any review of a body of data that uses clearly defined methods and criteria (box). Systematic reviews can include meta-analyses, appraisals of single trials, and other sources of evidence.6 In this article we examine the potentials and promise of meta-analysis of randomised controlled trials. In later articles of this series we will consider the practical steps involved in meta-analysis,7 examine various extensions beyond the calculation of a combined estimate,8 address potential biases and discuss strategies to detect and minimise the influence of these in meta-analysis of randomised trials9 and of observational studies.10 We will conclude with a discussion of unresolved issues and future developments.11 Details of relevant software will appear on the BMJ's website at the end of the series. Fig 1 Number of publications about meta-analysis, 1987–96 (results from Medline search using text word and medical subject heading “meta-analysis”) #### What's in a name? The case for “meta-analysis” The term meta-analysis for statistically combining and analysing data from separate studies is appropriate because:
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Egger and Davey Smith showed the same with respect to beta-blockers and mortality after acute myocardial infarction.
“In 1997, Egger and Davey Smith showed the same with respect to beta-blockers and mortality after acute myocardial infarction.”
From Commentary: Two-sample Mendelian randomization: opportunities and challenges (2016) · cites this paper as [23]
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
Self-citationPrimary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890 · see citing works
Beta blockade during and after myocardial infarction: An overview of the randomized trials
Yusuf, S., Peto, R., Lewis, J., Collins, R., & Sleight, P. (1985) · Progress in Cardiovascular Diseases · doi:10.1016/s0033-0620(85)80003-7 · Cited by 3,205 · see citing works
EFFECTIVENESS OF INTRAVENOUS THROMBOLYTIC TREATMENT IN ACUTE MYOCARDIAL INFARCTION
GRUPPOITALIANOPERLOSTUDIODELL (1986) · The Lancet · doi:10.1016/s0140-6736(86)92368-8 · Cited by 2,626 · see citing works
Meta-analysis: Principles and procedures
Egger, M., Smith, G. D., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7121.1533 · Cited by 2,415 · see citing works
Self-citationBeecher, H. K. (1955) · JAMA · doi:10.1001/jama.1955.02960340022006 · Cited by 1,850 · see citing works
QUANTITATIVE METHODS IN THE REVIEW OF EPIDEMIOLOGIC LITERATURE1
GREENLAND, S. (1987) · Epidemiologic Reviews · doi:10.1093/oxfordjournals.epirev.a036298 · Cited by 1,644 · see citing works
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628 · see citing works
Antman, E. M. (1992) · JAMA · doi:10.1001/jama.268.2.240 · Cited by 1,535 · see citing works
<i>meta-analysis</i> bias in location and selection of studies
Egger, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7124.61 · Cited by 1,374 · see citing works
Self-citationCumulative Meta-Analysis of Therapeutic Trials for Myocardial Infarction
Lau, J., Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992) · New England Journal of Medicine · doi:10.1056/nejm199207233270406 · Cited by 1,140 · see citing works
<i>Meta-analysis</i> Spurious precision? Meta-analysis of observational studies
Egger, M., Schneider, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7125.140 · Cited by 997 · see citing works
Self-citationEffects of Adjuvant Tamoxifen and of Cytotoxic Therapy on Mortality in Early Breast Cancer
Early Breast Cancer Trialists' Collaborative Group (1988) · New England Journal of Medicine · doi:10.1056/nejm198812293192601 · Cited by 934 · see citing works
The Medical Review Article: State of the Science
MULROW, C. D. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-106-3-485 · Cited by 856 · see citing works
Nurmohamed, M. T., Buller, H. R., Dekker, E., Hommes, D. W., Rosendaal, F. R., Briët, E., Rosendaal, F. R., & Vandenbroucke, J. P. (1992) · The Lancet · doi:10.1016/0140-6736(92)93223-a · Cited by 664 · see citing works
Meta-analysis: State-of-the-Science
Dickersin, K., & Berlin, J. A. (1992) · Epidemiologic Reviews · doi:10.1093/oxfordjournals.epirev.a036084 · Cited by 502 · see citing works
Felson, D. T., Anderson, J. J., & Meenan, R. F. (1990) · Arthritis & Rheumatism · doi:10.1002/art.1780331001 · Cited by 472 · see citing works
Statistical Inference: A Commentary for the Social and Behavioural Sciences
Oakes, M. W. (1986) · Medical Entomology and Zoology · Cited by 392 · see citing works
Getting to grips with Archie Cochrane's agenda.
Chalmers, I., Dickersin, K., & Chalmers, T. C. (1992) · BMJ · doi:10.1136/bmj.305.6857.786 · Cited by 376 · see citing works
Low molecular weight heparin in prevention of perioperative thrombosis.
Leizorovicz, A., Haugh, M., Chapuis, F. R., Samama, M. M., & Boissel, J. (1992) · BMJ · doi:10.1136/bmj.305.6859.913 · Cited by 375 · see citing works
Bero, L. A. (1995) · JAMA · doi:10.1001/jama.274.24.1935 · Cited by 373 · see citing works
Cholesterol lowering trials in coronary heart disease: Frequency of citation and outcome.
Ravnskov, U. (1992) · BMJ · doi:10.1136/bmj.305.6844.15 · Cited by 313 · see citing works
Cornell, R. G., Wolf, F. M., Bailar, J. C., & Mosteller, F. C. (1989) · Journal of the American Statistical Association · doi:10.2307/2289981 · Cited by 295 · see citing works
Report on Certain Enteric Fever Inoculation Statistics
Simpson, L. (1904) · British medical journal · doi:10.1136/bmj.2.2288.1243 · Cited by 283 · see citing works
Minimizing the Three Stages of Publication Bias
Chalmers, T. C. (1990) · JAMA · doi:10.1001/jama.1990.03440100104016 · Cited by 226 · see citing works
Reference bias in reports of drug trials.
Gøtzsche, P. C. (1987) · BMJ · doi:10.1136/bmj.295.6599.654 · Cited by 224 · see citing works
Meta-analysis and the meta-epidemiology of clinical research
Naylor, C. D. (1997) · BMJ · doi:10.1136/bmj.315.7109.617 · Cited by 209 · see citing works
Meta-analysis: Unresolved issues and future developments
Smith, G. D., & Egger, M. (1998) · BMJ · doi:10.1136/bmj.316.7126.221 · Cited by 175 · see citing works
Self-citationMeta-analysis. Beyond the grand mean?
Smith, G. D., Egger, M., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7122.1610 · Cited by 174 · see citing works
Self-citationTHE COMPARATIVE EFFICACY AND TOXICITY OF SECOND LINE DRUGS IN RHEUMATOID ARTHRITIS
Felson, D. T., Anderson, J., & Meenan, R. F. (1990) · Cited by 153 · see citing works
Meta-analysis in medicine Where we are and where we want to go
Jenicek, M. (1989) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(89)90023-1 · Cited by 141 · see citing works
Land, K. C., Wachter, K. W., & Straf, M. L. (1991) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2071894 · Cited by 97 · see citing works
United States District Court for the District of Columbia
(1974) · Journal of the American Pharmaceutical Association (1961) · doi:10.1016/s0003-0465(16)34346-4 · Cited by 73 · see citing works
Promoting clinical effectiveness
Hayward, J. (1996) · BMJ · doi:10.1136/bmj.312.7045.1491 · Cited by 66 · see citing works
Cholesterol and total mortality: need for larger trials.
Collins, R., KEECH, A. C., Peto, R., Sleight, P., KJEKSHUS, J. K., Wilhelmsen, L., MacMahon, S. W., Shaw, J., Simes, J., & D, E. B. M. (1992) · BMJ · doi:10.1136/bmj.304.6843.1689 · Cited by 61 · see citing works
Meta‐Analysis: Whither Narrative Review?
Teagarden, J. R. (1989) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1002/j.1875-9114.1989.tb04139.x · Cited by 53 · see citing works
Lancet, T. (1997) · The Lancet · doi:10.1016/s0140-6736(97)21036-6 · Cited by 52 · see citing works
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (2019) · CRC Press eBooks · doi:10.1201/9780429187445-19 · Cited by 39 · see citing works
Timolol after myocardial infarction: an answer or a new set of questions?
Mitchell, J. R. A. (1981) · BMJ · doi:10.1136/bmj.282.6276.1565 · Cited by 37 · see citing works
Interpretation of results from subset analyses within overviews of randomized clinical trials
Gelber, R. D., & Goldhirsch, A. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060331 · Cited by 37 · see citing works
Minimizing the three stages of publication bias
Chalmers, T. C. (1990) · JAMA · doi:10.1001/jama.263.10.1392 · Cited by 28 · see citing works
Meta-analysis of second-line antirheumatic drugs: sample size bias and uncertain benefit
Gøtzsche, P. C. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90130-f · Cited by 25 · see citing works
Setting limits in clinical medicine.
Murphy, D. J., Povar, G. J., & Pawlson, L. G. (1994) · PubMed · Cited by 24 · see citing works
WHERE WE ARE AND WHERE WE WANT TO GO
Jenicek, M. (1989) · Cited by 17 · see citing works
Setting limits in clinical medicine
Murphy, D. J. (1994) · Archives of Internal Medicine · doi:10.1001/archinte.154.5.505 · Cited by 14 · see citing works
[Disclosure of medical records].
Solbach, G. (1998) · PubMed · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884 · see citing works
Meta-analysis: Principles and procedures
Egger, M., Smith, G. D., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7121.1533 · Cited by 2,415 · see citing works
Self-citationPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · see citing works
Self-citationA Systematic Review of the Prevalence of Schizophrenia
Saha, S., Chant, D., Welham, J., & McGrath, J. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020141 · Cited by 2,168 · see citing works
Interpretation of the evidence for the efficacy and safety of statin therapy
Collins, R., Reith, C., Emberson, J., Armitage, J., Baigent, C., Blackwell, L., Blumenthal, R., Danesh, J., Smith, G. D., DeMets, D., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)31357-5 · Cited by 1,857 · see citing works
Self-citationMorley, S., Eccleston, C., & Williams, A. (1999) · Pain · doi:10.1016/s0304-3959(98)00255-3 · Cited by 1,597 · see citing works
Sterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466 · see citing works
How Can Research Organizations More Effectively Transfer Research Knowledge to Decision Makers?
LAVIS, J. N., ROBERTSON, D., WOODSIDE, J. M., McLEOD, C. B., & ABELSON, J. (2003) · The Milbank Quarterly · doi:10.1111/1468-0009.t01-1-00052 · Cited by 1,176 · see citing works
Can There Ever Be Too Many Options? A Meta-Analytic Review of Choice Overload
Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2010) · Journal of Consumer Research · doi:10.1086/651235 · Cited by 1,070 · see citing works
McGrath, J., Saha, S., Welham, J., El Saadi, O., MacCauley, C., & Chant, D. (2004) · BMC Medicine · doi:10.1186/1741-7015-2-13 · Cited by 1,072 · see citing works
<i>Meta-analysis</i> Spurious precision? Meta-analysis of observational studies
Egger, M., Schneider, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7125.140 · Cited by 997 · see citing works
Self-citationCommentary: Two-sample Mendelian randomization: opportunities and challenges
Lawlor, D. A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw127 · Cited by 810 · see citing works
Bias in clinical intervention research.
Gluud, L. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj069 · Cited by 636 · see citing works
The effect of acute aerobic exercise on positive activated affect: A meta-analysis
Reed, J., & Ones, D. S. (2006) · Psychology of Sport and Exercise · doi:10.1016/j.psychsport.2005.11.003 · Cited by 622 · see citing works
Multivariate meta‐analysis: Potential and promise
Jackson, D., Riley, R., & White, I. R. (2011) · Statistics in Medicine · doi:10.1002/sim.4172 · Cited by 530 · see citing works
Cytokines in bipolar disorder vs. healthy control subjects: A systematic review and meta-analysis
Munkholm, K., Braüner, J. V., Kessing, L. V., & Vinberg, M. (2013) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2013.05.018 · Cited by 402 · see citing works
Psychological interventions for women with metastatic breast cancer
Mustafa, M., Carson-Stevens, A., Gillespie, D., & Edwards, A. G. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004253.pub4 · Cited by 244 · see citing works
Husebø, A. M. L., Dyrstad, S. M., Søreide, J. A., & Bru, E. (2013) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2012.04322.x · Cited by 177 · see citing works
Meta-analysis. Beyond the grand mean?
Smith, G. D., Egger, M., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7122.1610 · Cited by 174 · see citing works
Self-citationPractical recommendations to conduct a neuroimaging meta-analysis for neuropsychiatric disorders
Tahmasian, M., Sepehry, A. A., Samea, F., Khodadadifar, T., Soltaninejad, Z., Javaheripour, N., Khazaie, H., Zarei, M., Eickhoff, S. B., & Eickhoff, C. R. (2019) · Human Brain Mapping · doi:10.1002/hbm.24746 · Cited by 126 · see citing works
Stafford, J., Chung, W. T., Sommerlad, A., Kirkbride, J. B., & Howard, R. (2022) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.5711 · Cited by 126 · see citing works
JOURNAL QUALITY, EFFECT SIZE, AND PUBLICATION BIAS IN META-ANALYSIS
Murtaugh, P. A. (2002) · Ecology · doi:10.1890/0012-9658(2002)083[1162:jqesap]2.0.co;2 · Cited by 126 · see citing works
Systematic reviews and meta-analysis for the surgeon scientist
Mahid, S. S., Hornung, C. A., Minor, K. S., Turina, M., & Galandiuk, S. (2006) · British Journal of Surgery · doi:10.1002/bjs.5596 · Cited by 109 · see citing works
Subclinical Thyroid Dysfunction and the Risk for Fractures: A Systematic Review and Meta-analysis
Wirth, C. D., Blum, M. R., da Costa, B. R., Baumgartner, C., Collet, T.-H., Medici, M., Peeters, R. P., Aujesky, D., Bauer, D. C., & Rodondi, N. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0125 · Cited by 85 · see citing works
da Costa, B. R., Beckett, B., Diaz, A., Resta, N. M., Johnston, B. C., Egger, M., Jüni, P., & Armijo-Olivo, S. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0441-7 · Cited by 62 · see citing works
Self-citationBasic concepts in meta-analysis: a primer for clinicians
KHOSHDEL, A., ATTIA, J., & CARNEY, S. L. (2006) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2006.01078.x · Cited by 55 · see citing works
Tak, L. M., Meijer, A., Manoharan, A., de Jonge, P., & Rosmalen, J. G. M. (2010) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3181d714e1 · Cited by 23 · see citing works
Brown, H. E., Atkin, A. J., Panter, J., Corder, K., Wong, G., Chinapaw, M. J. M., & van Sluijs, E. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005439 · Cited by 20 · see citing works
Systematic reviews and meta‐analyses in coloproctology: interpretation and potential pitfalls
Wille‐Jørgensen, P., & Renehan, A. G. (2008) · Colorectal Disease · doi:10.1111/j.1463-1318.2007.01421.x · Cited by 19 · see citing works
Systematic review in diagnostic radiology
MacDonald-Jankowski, D. S., & Dozier, M. F. (2001) · Dentomaxillofacial Radiology · doi:10.1038/sj.dmfr.4600586 · Cited by 18 · see citing works
Random or Fixed? An Empirical Examination of Meta-Analysis Model Choices
Slaney, K. L., Tafreshi, D., & Hohn, R. (2018) · Review of General Psychology · doi:10.1037/gpr0000140 · Cited by 13 · see citing works
- Authors
- Egger, M., & Smith, G. D.
- Affiliations
- University of Bristol (GB)
- Corresponding author
- Egger, Matthias
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 1997-11-22
- Volume
- 315
- Issue
- 7119
- Pages
- 1371-1374
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- PMC
- PMC2127866
- OpenAlex
- W2084254640
- Cited by
- 646
- References
- 47
- Citations by year
- 2026: 11 · 2025: 22 · 2024: 29 · 2023: 34 · 2022: 19 · 2021: 28 · 2020: 19 · 2019: 16 · 2018: 24 · 2017: 25 · 2016: 20 · 2015: 22 · 2014: 28 · 2013: 26 · 2012: 27
- FWCI
- 5.77
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation
- University of Bristol
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- meta-analysis, systematic reviews, randomized controlled trials, terminology, evidence synthesis, meta-analysis bias
- MeSH terms
- Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
