The DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index, and 1 are not checked yet.
Meta-analysis: Neither quick nor easy
Open accessAbstract
BACKGROUND: Meta-analysis is often considered to be a simple way to summarize the existing literature. In this paper we describe how a meta-analysis resembles a conventional study, requiring a written protocol with design elements that parallel those of a record review. METHODS: The paper provides a structure for creating a meta-analysis protocol. Some guidelines for measurement of the quality of papers are given. A brief overview of statistical considerations is included. Four papers are reviewed as examples. The examples generally followed the guidelines we specify in reporting the studies and results, but in some of the papers there was insufficient information on the meta-analysis process. CONCLUSIONS: Meta-analysis can be a very useful method to summarize data across many studies, but it requires careful thought, planning and implementation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We followed a rigorous inclusion/exclusion protocol to ensure minimisation of confounding factors amongst the RCT populations [112].”
From Estimating the Effect of Aerobic Exercise Training on Novel Lipid Biomarkers: A Systematic Review and Multivariate Meta-Analysis of Randomized Controlled Trials (2023) · cites this paper as [112]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Primary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 2×
“As we mentioned in the introduction, there is some disagreement on this subject [5,6,9,11,12,15, 16 ].”
Meta-Analysis: Quantitative Methods for Research Synthesis
Rosenthal, R., & Wolf, F. M. (1987) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2071646 · Cited by 1,371
A method for assessing the quality of a randomized control trial
Chalmers, T. C., Smith, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., & Ambroz, A. (1981) · Controlled Clinical Trials · doi:10.1016/0197-2456(81)90056-8 · Cited by 1,349
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308
Review papers : The statistical basis of meta-analysis
Fleiss, J. (1993) · Statistical Methods in Medical Research · doi:10.1177/096228029300200202 · Cited by 1,286
Meta-Analysis in Clinical Research
L'ABBÉ, K. A., DETSKY, A. S., & O'ROURKE, K. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-107-2-224 · Cited by 1,267
Meta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,257
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A. A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,164
Factors Influencing Publication of Research Results
Dickersin, K. (1992) · JAMA · doi:10.1001/jama.1992.03480030052036 · Cited by 931
Glucosamine and Chondroitin for Treatment of Osteoarthritis
McAlindon, T. E., LaValley, M. P., Gulin, J. P., & Felson, D. T. (2000) · JAMA · doi:10.1001/jama.283.11.1469 · Cited by 854
Incorporating variations in the quality of individual randomized trials into meta-analysis
Detsky, A. S., Naylor, C. D., O'Rourke, K., McGeer, A. J., & L'Abbé, K. A. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90085-2 · Cited by 640
Brand, J. L. (1997) · Digital Commons - Andrews University (Andrews University) · Cited by 537
Grégoire, G., Derderian, F., & LeLorier, J. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00098-b · Cited by 521
Meta-analysis: State-of-the-Science
Dickersin, K., & Berlin, J. A. (1992) · Epidemiologic Reviews · doi:10.1093/oxfordjournals.epirev.a036084 · Cited by 495
Thompson, S. G., & Pocock, S. (1991) · The Lancet · doi:10.1016/0140-6736(91)91975-z · Cited by 489
Approaches to heterogeneity in meta‐analysis
Petitti, D. B. (2001) · Statistics in Medicine · doi:10.1002/sim.1091 · Cited by 469
A comparison of statistical methods for combining event rates from clinical trials
Berlin, J. A., Laird, N. M., Sacks, H. S., & Chalmers, T. C. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080202 · Cited by 418
The Promise and Problems of Meta-Analysis
Bailar, J. C. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370810 · Cited by 407
Ischemic Stroke Risk With Oral Contraceptives
Gillum, L. A., Mamidipudi, S. K., & Johnston, S. C. (2000) · JAMA · doi:10.1001/jama.284.1.72 · Cited by 395
Meta-analysis: Statistical alchemy for the 21st century
Feinstein, A. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00110-c · Cited by 335
Gray, A., Berlin, J. A., McKinlay, J. B., & Longcope, C. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90028-8 · Cited by 315
Thacker, S. B. (1988) · JAMA · doi:10.1001/jama.259.11.1685 · Cited by 280
INCORPORATING VARIABILITY IN ESTIMATES OF HETEROGENEITY IN THE RANDOM EFFECTS MODEL IN META-ANALYSIS
Biggerstaff, B. J., & Tweedie, R. L. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970415)16:7 · Cited by 272
Chalmers, T. C., Levin, H. R., Sacks, H. S., Reitman, D., Berrier, J., & Nagalingam, R. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060320 · Cited by 250
Caubet, J.-F., Tosteson, T. D., Dong, E. W., Naylon, E. M., Whiting, G. W., Ernstoff, M. S., & Ross, S. D. (1997) · Urology · doi:10.1016/s0090-4295(96)00325-1 · Cited by 203
Dickersin, K. (1992) · JAMA · doi:10.1001/jama.267.3.374 · Cited by 193
Problems induced by meta‐analyses
Chalmers, T. C. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100618 · Cited by 144
Meta-analysis of randomized controlled trials. A concern for standards;
Moher, D. (1995) · JAMA · doi:10.1001/jama.274.24.1962 · Cited by 132
The potential and limitations of meta-analysis.
Spector, T. D., & Thompson, S. G. (1991) · Journal of Epidemiology & Community Health · doi:10.1136/jech.45.2.89 · Cited by 125
Villar, J., Mackey, M. E., Carroli, G., & Donner, A. (2001) · Statistics in Medicine · doi:10.1002/sim.1096 · Cited by 96
Keynote addresses. Meta‐analysis: Reconciling the results of independent studies
Olkin, I. (1995) · Statistics in Medicine · doi:10.1002/sim.4780140507 · Cited by 84
Clinical Trials and Meta-Analysis
Kassirer, J. P. (1992) · New England Journal of Medicine · doi:10.1056/nejm199207233270411 · Cited by 80
Meta-analysis of Randomized Controlled Trials
Moher, D. (1995) · JAMA · doi:10.1001/jama.1995.03530240072044 · Cited by 76
Meta-meth-analysis: Unanswered questions about aggregating data
Spitzer, W. O. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90258-b · Cited by 67
DIFFERENTIAL RECALL BIAS AND SPURIOUS ASSOCIATIONS IN CASE/CONTROL STUDIES
Barry, D. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961215)15:23 · Cited by 39
Liberati, A. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00115-7 · Cited by 30
A consumer's guide to meta‐analysis
LaValley, M. P. (1997) · Arthritis & Rheumatism · doi:10.1002/art.1790100309 · Cited by 12
Research works printed but not linked: 2
- Stein RA: Meta-analysis from one FDA reviewer's perspective. 1988 Proceedings of the American Statistical Association Biopharmaceutical Section. 1988, 34-38.
- Hedges LV, Olkin I: Statistical Methods for Meta-Analysis. London: Academic Press. 1985
Cited by
Sorted by citations
Psychotherapy Relationships That Work
Norcross, J. C. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199737208.001.0001 · Cited by 969
Roelfs, D. J., Shor, E., Davidson, K. W., & Schwartz, J. E. (2011) · Social Science & Medicine · doi:10.1016/j.socscimed.2011.01.005 · Cited by 759
Mantyka‐Pringle, C., Martin, T. G., & Rhodes, J. R. (2011) · Global Change Biology · doi:10.1111/j.1365-2486.2011.02593.x · Cited by 802
Elliott, R., Bohart, A. C., Watson, J. C., & Greenberg, L. S. (2011) · Psychotherapy · doi:10.1037/a0022187 · Cited by 511
Publication Bias in the Organizational Sciences
Kepes, S., Banks, G. C., McDaniel, M. A., & Whetzel, D. L. (2012) · Organizational Research Methods · doi:10.1177/1094428112452760 · Cited by 416
Metadata
- Authors
- Berman, N. G., & Parker, R. A.
- Affiliations
- Harbor–UCLA Medical Center (US)
- Beth Israel Deaconess Medical Center (US)
- Corresponding author
- Berman, Nancy G
- ORCID
- Parker, Robert A
- Venue
- BMC Medical Research Methodology
- Published
- 2002-08-09
- Volume
- 2
- Issue
- 1
- Pages
- 10-10
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC122061
- OpenAlex
- W2123303485
- Cited by
- 270
- References
- 43
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 13 · 2023: 15 · 2022: 9 · 2021: 14 · 2020: 10 · 2019: 13 · 2018: 10 · 2017: 17 · 2016: 20 · 2015: 11 · 2014: 20 · 2013: 15 · 2012: 16
- FWCI
- 2.67
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Research Methodology (publishedVersion)
- PubMed Central (publishedVersion, other-oa)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institutes of Health (RR00425, RR 01032)
- National Center for Research Resources (RR 01032, M01RR001032, RR00425, M01RR000425)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- meta-analysis, protocol design, study quality assessment, reporting guidelines, statistical considerations, research planning
- MeSH terms
- Data Collection, Demography, Humans, Meta-Analysis as Topic, Treatment Outcome, Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
