The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Open accessAbstract
Although meta-analysis is now well established as a method of reviewing evidence, an uncritical use of the technique can be very misleading. One common problem is the failure to investigate appropriately the sources of heterogeneity, in particular the clinical differences between the studies included. This paper distinguishes between the concepts of clinical and statistical heterogeneity and exemplifies the importance of investigating heterogeneity by using published meta-analyses of epidemiological studies of serum cholesterol concentration and clinical trials of its reduction. Although not without some dangers of speculative conclusions, prompted by overzealous inspection of the data to hand, a sensible investigation of sources of heterogeneity should increase both the scientific and the clinical relevance of the results of meta-analyses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Practical Statistics for Medical Research.
Walter, S., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,666
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668
Practical Statistics for Medical Research.
Bartko, J. J., & Altman, D. G. (1992) · Journal of the American Statistical Association · doi:10.2307/2290246 · Cited by 4,834
Altman, R., Carreras, L., Díaz, R. S., Figueroa, E. B., Paolasso, E., Parodi, J. C., Cade, J. F., Donnan, G. A., Eadie, M. J., Gavaghan, T. P., et al. (1994) · Cited by 3,322
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
Systemic treatment of early breast cancer by hormonal, cytotoxic, or immune therapy
Early Breast Cancer Trialist's Collaborative Group (1992) · The Lancet · doi:10.1016/0140-6736(92)90997-h · Cited by 2,169
Law, M., Wald, N., & Thompson, S. (1994) · British medical journal · doi:10.1136/bmj.308.6925.367 · Cited by 1,306
Yusuf, S. (1991) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1991.03470010097038 · Cited by 1,014
Research works printed but not linked: 4
- Oxman A .Preparing and maintaining systematic reviews. In: Sackett D , ed Cochrane Collaboration Handbook.Section VI. Oxford:Cochrane Collaboration,1994.
- Meier, 1987, Meta-analysis of clinical trials as a scientific discipline [commentary]., Statist Med, 6, 329, 10.1002/sim.4780060322 · DOI
- 10.1136/bmj.308.6921.81 · DOI
- Schulz KF Chalmers I Hayes RJ Altman DG .Empirical evidence of bias: dimensions of methodologic quality are associated with estimates of treatment effects in 250 controlled trials from 33 meta-analyses. JAMA (in press).
Cited by
Sorted by citations
Items 901–1,000 of 1,000
Thomason, J. M. (2010) · Journal of Evidence-Based Dental Practice (Brazil Edition) · doi:10.1016/s1877-069x(10)00097-5
Wu, B. (2012) · African Journal of Microbiology Research · doi:10.5897/ajmr11.604
Harrington, R., Whittaker, J. F., Shoebridge, P., & Campbell, F. (1999) · RET: revista de toxicomanías
Narrative or systematic reviews: can we be more ‘evidence-based’?
Barer, D. H. (1995) · Reviews in Clinical Gerontology · doi:10.1017/s0959259800004810
The role of meta-analysis in defining clinical practice in advanced non-small-cell lung cancer
Maïo, M. D. (2013) · Clinical Investigation · doi:10.4155/cli.13.44
Newer anticonvulsants in the treatment of anxiety disorders
Ipser, J., & Stein, D. J. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006241
Patient-Level Data Provide Additional Insight for Tigecycline All-Cause Mortality Meta-analysis
McGovern, P. C., & Quintana, A. (2012) · Clinical Infectious Diseases · doi:10.1093/cid/cis760
Begum, F., Ghosh, D., Tseng, G. C., & Feingold, E. (2012)
Ho, M.-W., Ghosh, P., Elasoff, R. M., & Tiwari, R. C. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2124798
Fuchs, S. C. P. C., & Paim, B. S. (2010)
Study Design, Statistical Inference and Literature Search in Surgical Research
Skapinakis, P., & Athanasiou, T. (2010) · Springer eBooks · doi:10.1007/978-3-540-71915-1_4 · Cited by 1
Imaging biomarkers for detection of coronary artery disease
den Dekker, M. A. M. (2014) · University of Groningen research database (University of Groningen / Centre for Information Technology)
Understanding and using systematic literature reviews
Cipriani, A., & Barbui, C. (2011) · doi:10.1093/med/9780199565498.003.0167
Bayesian probability encoding in medical decision analysis
Pang, C. S. (2008) · National University of Singapore
Methods for exploring heterogeneity in systematic reviews of randomized controlled trials.
Gagnier, J. J. (2010) · TSpace (University of Toronto)
Chapter 7: Systematic Reviews and Meta-Analyses
Schumacher, G. E. (2017) · The American Pharmacists Association eBooks · doi:10.21019/9781582122823.ch7
The importance of dietary factors in knee osteoarthritis: an evidence-based approach
Christensen, R. (2009)
The Mortality and Morbidity of Alcohol-attributable Injury
Taylor, B. J. (2014) · TSpace (University of Toronto)
Shared decision-making tools: do they really involve patients?
Walsh, K. A., & Sandars, J. E. (2017) · British Journal of Hospital Medicine · doi:10.12968/hmed.2017.78.6.304
The Methodology of Meta-Analyses and Its Potential Contribution to Patient Care
Gandini, S. (2017) · Springer eBooks · doi:10.1007/978-3-319-48848-6_80
Wu, F., Ye, Y., & Zhou, X. (2018) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2018.06.027
Translating Experimental Evidence Into Clinical Decision Making
Healy, E. W., & Wasfy, J. H. (2019) · Cardiovascular revascularization medicine · doi:10.1016/j.carrev.2018.12.004
Stroup, D. F., & Thacker, S. B. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat05179
Napoli, A. D., & Nichelatti, M. (2018) · Giornale di Clinica Nefrologica e Dialisi · doi:10.33393/gcnd.2014.899
On ratio measures of population heterogeneity for meta-analysesPreprint
Cairns, M., & Prendergast, L. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.10332
Boaful, G., & George, P. (2021) · Consultant · doi:10.25270/con.2021.03.00008
Different Biomarker Modalities and Response to Anti–PD-1/PD-L1 Therapies
Liu, S., & Niu, W. (2019) · JAMA Oncology · doi:10.1001/jamaoncol.2019.5148
Improved meta analysis using predicted relative risk
Khan, S., & Hasan, M. T. (2008)
COVID-19 in heart transplant patients: is there a cause for concern?
Cheema, H. A., & Shahid, A. (2022) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwac001
Petticrew, M. P., & Roberts, H. M. (2006) · Wiley eBooks · doi:10.1002/9780470754887.biblio
Friedman, L. M., Furberg, C. D., & DeMets, D. L. (1998) · Springer eBooks · doi:10.1007/978-1-4757-2915-3_16
Good, P. I., & Hardin, J. W. (2012) · Wiley eBooks · doi:10.1002/9781118360125.biblio
Good, P. I., & Hardin, J. W. (2008) · doi:10.1002/9780470473924.biblio
Hotopf, M. H., Hardy, R., & Lewis, G. (1997) · The British Journal of Psychiatry · doi:10.1192/s0007125000147282
Peacock, J. L., & Peacock, P. J. (2020) · doi:10.1093/med/9780198743583.003.0013
Marcus, M. W., de Vries, M. M., Montolio, F. G. J., & Jansonius, N. M. (2012) · Ophthalmology · doi:10.1016/j.ophtha.2012.06.018
Moayyedi, P. (2007) · doi:10.1002/9780470692189.ch10
Napoli, A. D., & Nichelatti, M. (2014) · Giornale di Techniche Nefrologiche e Dialitiche · doi:10.5301/039493621402600302
(1977) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1467-842x.1977.tb01000.x
Dose Effects of Steroids on Survival in Sepsis
Deans, K. J., Natanson, C., & Minneci, P. C. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-11-200412070-00027
Leandro, G., & Gallus, G. (2005) · doi:10.1002/9780470994894.refs
Senn, S. J. (2021) · doi:10.1002/9781119238614.ch16
Are antioxidant supplements dangerous?
(2007) · Focus on Alternative and Complementary Therapies · doi:10.1211/fact.12.3.0015
Response to the Comment by Yang X et al.
Caini, S., Riccio, M. D., & Gandini, S. (2023) · Journal of Thoracic Oncology · doi:10.1016/j.jtho.2022.11.017
Adaptation of CERES plot to meta-regression
Avcı, E. (2023) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2023.2250890
Medical students’ mental wellbeing during COVID-19 pandemic: A commentary on Ayubi et al. (2023)
Elawfi, B., & Nashwan, A. J. (2023) · Personalized Medicine in Psychiatry · doi:10.1016/j.pmip.2023.100110
PREVALENCE OF EQUINE HERPESVIRUSES (EHVS) INFECTION IN EGYPT: SYSTEMATIC META-ANALYSIS
Marzok, M., Al-Mubarak, A., Elgioushy, M. M., & El-Khodery, S. (2025) · The Journal of Animal and Plant Sciences · doi:10.36899/japs.2025.1.0018
Alwisi, N., Ismail, R., Al-Kuwari, H., Al-Ansari, K. H., Al-Matwi, M. A., Aweer, N. A., Al-Marri, W. N., Al-Kubaisi, Y., Almohannadi, M., Hamran, S., et al. (2025) · Gastroenterology Insights · doi:10.3390/gastroent16020012
Zhao, F.-Y., Fu, Q.-Q., & Ho, J. Y. S. (2025) · Nature and Science of Sleep · doi:10.2147/nss.s547791
Kim, H. S., Cho, Y. J., Goh, T. S., & Lee, J. S. (2025) · Journal of Korean society of spine surgery · doi:10.4184/jkss.2025.32.3.70
Karpova, I. V. (2006) · D-Scholarship@Pitt (University of Pittsburgh)
Lærum‐Onsager, E., Molin, M., Olsen, C. F., Bye, A., Debesay, J., Hestevik, C. H., Bjerk, M., & Pripp, A. H. (2021) · Figshare · doi:10.6084/m9.figshare.14570006
Nicolotti, D., Grossi, S., Palermo, V., Pontone, F., Maglietta, G., Diodati, F., Puntoni, M., Rossi, S., & Caminiti, C. (2024) · Figshare · doi:10.6084/m9.figshare.26678162
Nicolotti, D., Grossi, S., Palermo, V., Pontone, F., Maglietta, G., Diodati, F., Puntoni, M., Rossi, S., & Caminiti, C. (2024) · Figshare · doi:10.6084/m9.figshare.26678174
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26562004
Backenroth, D., Royce, T., Pinheiro, J., Samant, M. K., & Humblet, O. (2023) · Figshare · doi:10.6084/m9.figshare.24030852
Wirbel, J., Zych, K., Essex, M., Karcher, N., Kartal, E., Salazar, G., Bork, P., Sunagawa, S., & Zeller, G. (2021) · Figshare · doi:10.6084/m9.figshare.14344169
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26562001
Nicolotti, D., Grossi, S., Palermo, V., Pontone, F., Maglietta, G., Diodati, F., Puntoni, M., Rossi, S., & Caminiti, C. (2024) · Figshare · doi:10.6084/m9.figshare.26678168
Nicolotti, D., Grossi, S., Palermo, V., Pontone, F., Maglietta, G., Diodati, F., Puntoni, M., Rossi, S., & Caminiti, C. (2024) · Figshare · doi:10.6084/m9.figshare.26678171
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561986
Ho, L. T. F., Ke, F. Y. T., Wong, C. H. L., Wu, I. X. Y., Cheung, A. K. L., Mao, C., & Chung, V. C. (2021) · Figshare · doi:10.6084/m9.figshare.16909666
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561980
Wirbel, J., Zych, K., Essex, M., Karcher, N., Kartal, E., Salazar, G., Bork, P., Sunagawa, S., & Zeller, G. (2021) · Figshare · doi:10.6084/m9.figshare.14344172
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561989
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561983
Nicolotti, D., Grossi, S., Palermo, V., Pontone, F., Maglietta, G., Diodati, F., Puntoni, M., Rossi, S., & Caminiti, C. (2024) · Figshare · doi:10.6084/m9.figshare.26678165
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561992
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561995
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561998
Additional file 1 of Vitamin D in adolescent idiopathic scoliosis: a meta-analysis
Kim, D. S., & Lee, J. S. (2023) · Figshare · doi:10.6084/m9.figshare.24054633
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561977
Lærum‐Onsager, E., Molin, M., Olsen, C. F., Bye, A., Debesay, J., Hestevik, C. H., Bjerk, M., & Pripp, A. H. (2021) · Figshare · doi:10.6084/m9.figshare.14570009
Wirbel, J., Zych, K., Essex, M., Karcher, N., Kartal, E., Salazar, G., Bork, P., Sunagawa, S., & Zeller, G. (2021) · Figshare · doi:10.6084/m9.figshare.14344175
Ho, L. T. F., Ke, F. Y. T., Wong, C. H. L., Wu, I. X. Y., Cheung, A. K. L., Mao, C., & Chung, V. C. (2021) · Figshare · doi:10.6084/m9.figshare.16909660
Blázquez-Rincón, D., Sánchez-Meca, J., Botella, J., & Suero, M. (2024) · Figshare · doi:10.6084/m9.figshare.26561974
Schlattmann, P. (2025) · Springer eBooks · doi:10.1007/978-1-4614-6625-3_21-1
Atypical antipsychotics in the treatment of schizophrenia
Prior, C., Clements, J., & Rowett, M. (2001) · BMJ · doi:10.1136/bmj.322.7291.924/a
Liu, Z., Ji, C., Wang, R., Liu, Q., Guo, X., Li, F., Kong, Y., Chen, L., Yang, X., & Gao, S. (2025) · BMC Endocrine Disorders · doi:10.1186/s12902-025-02103-y
Yáñez-Dávila, D., Camprubí, A., Cano, N. A., Partida, E. G., & Santoyo, E. (2025) · Enseñanza y Comunicación de las Geociencias · doi:10.22201/cgeo.29928087e.2025.4.2.101
Kim, K., Cho, Y. J., Goh, T. S., & Lee, J. S. (2025) · Journal of Korean society of spine surgery · doi:10.4184/jkss.2025.32.4.112
Marzok, M. A., Ashraf, G., Alkuwayti, M. A., Elgioushy, M. M., Babiker, H., & El‐Khodery, S. (2026) · Veterinary Research Communications · doi:10.1007/s11259-026-11088-5
Wang, X. (2012) · D-Scholarship@Pitt (University of Pittsburgh)
Meta-analyses of Clinical Drug Trials—Gold Standard Reviews or Statistical Alchemy?
Chan, E. (2000) · Annals of the Academy of Medicine Singapore · doi:10.47102/annals-acadmedsg.v29n5p610
Marzok, M., Ashraf, G., Almubarak, A. I., Babiker, H., & El‐Khodery, S. (2026) · Veterinary Sciences · doi:10.3390/vetsci13040316
Li, H., Huo, H.-D., Wang, S., & Yang, J. (2026) · BMC Cancer · doi:10.1186/s12885-025-15278-3 · Cited by 1
Comparison of the Combined and Posterior Approaches for Adult Spinal Deformity: A Meta-Analysis
Kim, K., Cho, Y. J., Goh, T. S., & Lee, J. S. (2026) · Journal of Korean society of spine surgery · doi:10.4184/jkss.2026.33.1.5
Contribution des études observationnelles dans la méta-analyse
Cheurfa, C. (2023) · theses.fr (ABES) · doi:10.70675/ecaa782bz76f7z4c43za043ze1d3e0cfbfaa
Walsche, A. D. (2025) · theses.fr (ABES) · doi:10.70675/0df76955z9a01z470dzbd40z8105df32d440
Soslu, R., Çuvalcıoğlu, İ. C., Uysal, H. Ş., Uysal, A., Thapa, R. K., Akgül, M. Ş., & García‐Ramos, A. (2026) · International Journal of Sports Science & Coaching · doi:10.1177/17479541261452541 · Cited by 1
Baumgartner, K. T., Denny, K., Montgomery, E., Patton, C., Shedd, M., Yeary, J. C., Yaeger, L. H., Griffey, R. T., & Fuller, B. M. (2026) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxag173
Deng, X., Li, Y., Wu, Z., & Wang, Z. (2026) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-026-09953-0
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070
Explaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Hahn, S., Williamson, P. R., Hutton, J. L., Garner, P., & Flynn, E. V. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24
Metadata
- Authors
- Thompson, S. G.
- Affiliations
- London School of Hygiene & Tropical Medicine (GB)
- Corresponding author
- Thompson, S G
- ORCID
- Thompson, S G
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 1994-11-19
- Volume
- 309
- Issue
- 6965
- Pages
- 1351-1355
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- PMC
- PMC2541868
- OpenAlex
- W1511524925
- Cited by
- 1,017
- References
- 32
- Citations by year
- 2026: 23 · 2025: 35 · 2024: 38 · 2023: 29 · 2022: 30 · 2021: 47 · 2020: 38 · 2019: 32 · 2018: 36 · 2017: 26 · 2016: 24 · 2015: 24 · 2014: 31 · 2013: 48 · 2012: 51
- FWCI
- 3.70
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- meta-analysis heterogeneity, clinical heterogeneity, statistical heterogeneity, serum cholesterol, cholesterol lowering, meta-analysis methodology, overfitting
- MeSH terms
- Adult, Aged, Aging, Cholesterol, Clinical Trials as Topic, Humans, Middle Aged, Neoplasms, Research Design, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review