The DOI matches its Crossref record.
Invited Commentary: A Critical Look at Some Popular Meta-Analytic Methods
Abstract
Meta-analysis is essential for obtaining reproducible summaries of study results and valuable for discovering patterns among study results. A good meta-analysis will highlight and delineate the subjective components of these processes and vigorously search for sources of heterogeneity. Unfortunately, these objective are not always met by common techniques. For example, a scatterplot is an objective summarization if the data are uncensored, but inferred patterns should be regarded as subjective recognitions of the analyst, not objective data properties. Random-effects summaries encourage averaging over important data patterns, divert attention from key sources of heterogeneity, and can amplify distortions produced by publication bias; such summaries should only be used when important heterogeneity remains after a thorough search for the sources of such heterogeneity. Quality scoring adds the analyst's subjective bias to the results, wastes information, and can prevent the recognition of key sources of heterogeneity; it should be completely replaced by meta-regression on quality items (the score components).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Of note, although random-effects models allow for between-study heterogeneity, they do not help to understand the sources of heterogeneity [71].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [71]
In the citing paper
“Since “journals are typically more likely to publish results that establish a difference than those that do not” [48], and publication bias is difficult to eliminate, different articles advise to graphically examine its presence using funnel plots, or to assess it with statistical tests [16, 36, 48, 55, 85, 97, 99].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [85]
In the citing paper
“Although a random-effects model takes heterogeneity into account statistically, the drawback is that it may “divert attention from key sources of heterogeneity, […], such summaries should only be used when important heterogeneity remains after thorough search of the sources of heterogeneity” [85].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [85]
In the citing paper
“Authors recommended not to pool if heterogeneity is too high [35, 38, 47, 85, 103], and use statistical measures of heterogeneity to choose between random-effects (in case of heterogeneity) and fixed-effect models [55, 66].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [85]
In the citing paper
“Although many authoritative articles providing sound advice exist, such as those included in this review by Greenland [85], Egger et al. [29, 97], and a series in Research Synthesis Methods [37, 40, 65, 71, 75], the inexperienced reviewer may find them difficult to identify among the many other papers with contradictory advice.”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [85]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Still being checked.
Cited by
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
“As discussed in Box 4, scales that numerically summarize multiple components into a single number are misleading and unhelpful (92, 93).”
BackgroundLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Metadata
- Authors
- Greenland, S.
- Affiliations
- University of California, Los Angeles (US)
- Corresponding author
- Greenland, Sander
- ORCID
- Greenland, Sander
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 1994-08-01
- Volume
- 140
- Issue
- 3
- Pages
- 290-296
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W1953850870
- Cited by
- 511
- References
- 0
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 21 · 2023: 2 · 2022: 11 · 2021: 7 · 2020: 7 · 2019: 12 · 2018: 14 · 2017: 16 · 2016: 16 · 2015: 19 · 2014: 20 · 2013: 22 · 2012: 21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- meta-analytic methods, heterogeneity, publication bias, random effects models, quality scoring, meta-regression, scatter plots, reproducibility, study quality
- MeSH terms
- Data Interpretation, Statistical, Epidemiologic Methods, Humans, Quality Control, Regression Analysis, Meta-Analysis as Topic, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Invited Commentary: A Critical Look at Some Popular Meta-Analytic Methods. https://app.cochat.ai/papers/10.1093/oxfordjournals.aje.a117248
- The paper
- Greenland, S. (1994). Invited Commentary: A Critical Look at Some Popular Meta-Analytic Methods. American Journal of Epidemiology. https://doi.org/10.1093/oxfordjournals.aje.a117248
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.