The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews
Abstract
▪ Abstract We describe the history and current status of the meta-analytic enterprise. The advantages and historical criticisms of meta-analysis are described, as are the basic steps in a meta-analysis and the role of effect sizes as chief coins of the meta-analytic realm. Advantages of the meta-analytic procedures include seeing the “landscape” of a research domain, keeping statistical significance in perspective, minimizing wasted data, becoming intimate with the data summarized, asking focused research questions, and finding moderator variables. Much of the criticism of meta-analysis has been based on simple misunderstanding of how meta-analyses are actually carried out. Criticisms of meta-analysis that are applicable are equally applicable to traditional, nonquantitative, narrative reviews of the literature. Much of the remainder of the chapter deals with the processes of effect size estimation, the understanding of the heterogeneity of the obtained effect sizes, and the practical and scientific importance of the effect sizes obtained.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There was some disagreement (Table 4) whether the research question should be specific (narrowly formulated) [51, 53], or general (broadly formulated) [35, 47, 49].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
In the citing paper
“The majority of articles discussed aspects general to systematic reviews including advantages and limitations of different literature sources and databases, search tools, the importance of identifying unpublished studies, literature searching techniques including how to build a search string and reporting [12, 29, 33–36, 46, 48, 50, 51, 53, 54, 57, 59–62, 67, 68].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
In the citing paper
“Most articles considered the combination of RCTs and non-randomised designs [29, 35, 38, 39, 49, 51, 56, 61, 75, 81].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
In the citing paper
“Two articles recommended stratifying by study design if heterogeneity is large and adjusting for study design if heterogeneity is small [51, 77].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
In the citing paper
“Some authors suggested that results should be combined in a single meta-analysis only in the absence of marked heterogeneity [34, 51].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
In the citing paper
“One common recommendation was to explore heterogeneity in sensitivity analyses, comparing studies stratified by design or clinical features [12, 16, 29, 34, 39, 51, 55, 56, 66, 77, 85, 100, 102].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
Citations
Cites
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,707
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Self-citationMethods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Self-citationMeta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249
Self-citationDiMatteo, M. R., Lepper, H. S., & Croghan, T. W. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.14.2101 · Cited by 4,177
Self-citationCohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,896
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,368
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154
Yusuf, S., Zucker, D. M., Passamani, E. R., Peduzzi, P. N., Takaro, T., Fisher, L. D., Kennedy, J. W., Davis, K. B., Killip, T., Norris, R. M., et al. (1994) · The Lancet · doi:10.1016/s0140-6736(94)91963-1 · Cited by 2,125
Smith, M. L., Glass, G. V., & Miller, T. I. (1980) · Medical Entomology and Zoology · Cited by 2,123
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 2,003
Self-citationEagly, A. H., Ashmore, R. D., Makhijani, M. G., & Longo, L. C. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.1.109 · Cited by 1,875
Comorbidity of anxiety and unipolar mood disorders
Mineka, S., Watson, D., & Clark, L. A. (1998) · Annual Review of Psychology · doi:10.1146/annurev.psych.49.1.377 · Cited by 1,573
Summing up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422
Contrast Analysis: Focused Comparisons in the Analysis of Variance.
Levin, J. R., Rosenthal, R., & Rosnow, R. L. (1987) · Journal of the American Statistical Association · doi:10.2307/2289419 · Cited by 1,252
Self-citationA simple, general purpose display of magnitude of experimental effect.
Rosenthal, R., & Rubin, D. B. (1982) · Journal of Educational Psychology · doi:10.1037//0022-0663.74.2.166 · Cited by 1,233
Self-citationContrasts and Effect Sizes in Behavioral Research
Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511804403 · Cited by 1,147
Self-citationWriting meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108
Self-citationA META‐ANALYSIS OF SELF‐SUPERVISOR, SELF‐PEER, AND PEER‐SUPERVISOR RATINGS
Harris, M. M., & Schaubroeck, J. M. (1988) · Personnel Psychology · doi:10.1111/j.1744-6570.1988.tb00631.x · Cited by 1,080
The Handbook of Research Synthesis
Ziegel, E. R., Cooper, H., & Hedges, L. V. (1995) · Technometrics · doi:10.2307/1269651 · Cited by 936
The Effects of Television Violence on Antisocial Behavior: A Meta-Analysis1
Paik, H., & Comstock, G. A. (1994) · Communication Research · doi:10.1177/009365094021004004 · Cited by 857
Contrasts and Effect Sizes in Behavioral Research: A Correlational Approach
Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (1999) · Cited by 854
Self-citationBook Review: Integrating Research: A Guide for Literature Reviews
Kolbe, R. H., & Cooper, H. M. (1991) · Journal of Marketing Research · doi:10.2307/3172876 · Cited by 598
Preliminary Report: Findings from the Aspirin Component of the Ongoing Physicians' Health Study
The Steering Committee of the Physicians' Health Study Research Group (1988) · New England Journal of Medicine · doi:10.1056/nejm198801283180431 · Cited by 522
Conway, J. M., & Huffcutt, A. I. (1997) · Human Performance · doi:10.1207/s15327043hup1004_2 · Cited by 465
How hard is hard science, how soft is soft science? The empirical cumulativeness of research.
Hedges, L. V. (1987) · American Psychologist · doi:10.1037/0003-066x.42.5.443 · Cited by 453
Meta-Analysis for Explanation: A Casebook.
Jackson, S., Cook, T. D., Cooper, H. M., Cordray, D. S., Hartmann, H. I., Hedges, L. V., Light, R. J., Louis, T. A., & Mosteller, F. C. (1993) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2074578 · Cited by 434
Meta-Analytic Procedures for Combining Studies With Multiple Effect Sizes
Rosenthal, R., & Rubin, D. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.3.400 · Cited by 373
Self-citationStudies of educational interventions and outcomes in diabetic adults: A meta-analysis revisited
Brown, S. A. (1990) · Patient Education and Counseling · doi:10.1016/0738-3991(90)90070-2 · Cited by 361
Baum, M. L., Anish, D. S., Chalmers, T. C., Sacks, H. S., Smith, H., & Fagerstrom, R. M. (1981) · New England Journal of Medicine · doi:10.1056/nejm198110013051404 · Cited by 340
Effectiveness of cognitive-behavior therapy for maladapting children: A meta-analysis.
Durlak, J. A., Fuhrman, T., & Lampman, C. B. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.204 · Cited by 325
Achieving patient compliance : the psychology of the medical practitioner's role
DiMatteo, M. R., & DiNicola, D. D. (1982) · DigitalGeorgetown (Georgetown University Library) · Cited by 274
Self-citationContrasts and Effect Sizes in Behavioral Research: A Correlational Approach
Boik, R. J. (2001) · Journal of the American Statistical Association · doi:10.1198/jasa.2001.s432 · Cited by 257
Cesarean childbirth and psychosocial outcomes: A meta-analysis.
DiMatteo, M. R., Morton, S. C., Lepper, H. S., Damush, T. M., Carney, M. F., Pearson, M., & Kahn, K. L. (1996) · Health Psychology · doi:10.1037/0278-6133.15.4.303 · Cited by 247
Self-citationJudgment Studies: Design, Analysis, and Meta-Analysis.
Ridgeway, C. L., & Rosenthal, R. (1989) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2074155 · Cited by 235
Self-citationRosenthal, R. (1987) · Cambridge University Press eBooks · doi:10.1017/cbo9780511527807 · Cited by 219
Self-citationQuantitative methods for literature reviews
Green, B. F., & Hall, J. A. (1984) · Annual Review of Psychology · doi:10.1146/annurev.ps.35.020184.000345 · Cited by 213
A meta-analysis of controlled trials of cardiac patient education
Mullen, P. D., Mains, D. A., & Vélez, R. (1992) · Patient Education and Counseling · doi:10.1016/0738-3991(92)90194-n · Cited by 212
Meta-Analysis for Explanation: A Casebook
Spilker, B. (1993) · JAMA · doi:10.1001/jama.1993.03500230123042 · Cited by 199
Achieving Patient Compliance: The Psychology of the Medical Practitioner's Role
Smith, C. K. (1983) · JAMA · doi:10.1001/jama.1983.03340170098042 · Cited by 191
The Counternull Value of an Effect Size: A New Statistic
Rosenthal, R., & Rubin, D. B. (1994) · Psychological Science · doi:10.1111/j.1467-9280.1994.tb00281.x · Cited by 170
Self-citationMeta-analysis of the effects of psychoeducational care in adults with asthma
Devine, E. C. (1996) · Research in Nursing & Health · doi:10.1002/(sici)1098-240x(199610)19:5 · Cited by 118
Testing for moderator variables in meta‐analysis: Issues and methods
Hall, J. A., & Rosenthal, R. (1991) · Communication Monographs · doi:10.1080/03637759109376240 · Cited by 116
Self-citationHow often are our numbers wrong?
Rosenthal, R. (1978) · American Psychologist · doi:10.1037/0003-066x.33.11.1005 · Cited by 115
Self-citationParadoxical interventions: A meta-analysis.
Shoham-Salomon, V., & Rosenthal, R. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.55.1.22 · Cited by 100
Self-citationHypotheses and problems in research synthesis.
Hall, J. A., Tickle‐Degnen, L., Rosenthal, R., & Mosteller, F. C. (1994) · Cited by 98
Self-citationLand, K. C., Wachter, K. W., & Straf, M. L. (1991) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2071894 · Cited by 97
Franck, L. S. (1990) · Nursing Research · doi:10.1097/00006199-199011000-00015 · Cited by 94
Fabio, R. A. (1995) · Physical Therapy · doi:10.1093/ptj/75.10.865 · Cited by 94
Devine, E. C., & Pearcy, J. M. (1996) · Patient Education and Counseling · doi:10.1016/0738-3991(96)00862-2 · Cited by 93
Meyer, R. G., & Smith, S. R. (1977) · American Psychologist · doi:10.1037//0003-066x.32.8.638 · Cited by 92
A Meta-Analysis of the Effects Of Psychoeducational Care in Adults with Hypertension
Devine, E. C., & REIFSCHNEIDER, E. (1995) · Nursing Research · doi:10.1097/00006199-199507000-00009 · Cited by 80
Interpreting and Evaluating Meta-Analysis
Hall, J. A., & Rosenthal, R. (1995) · Evaluation & the Health Professions · doi:10.1177/016327879501800404 · Cited by 75
Self-citationProgress in clinical psychology: Is there any?
Rosenthal, R. (1995) · Clinical Psychology Science and Practice · doi:10.1111/j.1468-2850.1995.tb00035.x · Cited by 75
Self-citationHow Hard Is Hard Science, How Soft Is Soft Science?
Hedges, L. V. (1987) · Cited by 71
Quality-weighting of studies in meta-analytic research
Rosenthal, R. (1991) · Psychotherapy Research · doi:10.1080/10503309112331334031 · Cited by 32
Self-citationMatters of Life and Death: Risks vs. Benefits of Medical Care
Evans, R. W. (1985) · JAMA · doi:10.1001/jama.1985.03360060139050 · Cited by 27
A New Goal: Preventing Disease, Not Infection
Cohen, J. (1993) · Science · doi:10.1126/science.8266069 · Cited by 26
DiMatteo, M. R. (2000) · Cited by 20
Self-citationMatters of Life & Death: Risks vs. Benefits of Medical Care
Robin, E. D. (1984) · Medical Entomology and Zoology · Cited by 14
Paradoxical interventions: A meta-analysis.
Shoham-Salomon, V., & Rosenthal, R. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.55.1.22 · Cited by 4
Self-citation
Research works printed but not linked: 11
- 10.1037//0033-2909.111.2.256 · DOI
- Chalmers I, 1995, Systematic Reviews.
- 10.1037//0033-2909.110.1.109 · DOI · Paper page
- 10.1037//0003-066X.42.5.443 · DOI
- Hunt M, 1997, How Science Takes Stock.
- Light RJ, Singer JD, Willett JB. 1994. The visual presentation and interpretation of meta-analyses. SeeCooper & Hedges 1994, pp. 439–53
- Pearson K, 1904, Br. Med. J. Nov., 5, 1243
- 10.1037//0003-066X.33.11.1005 · DOI · Paper page
- 10.1037//0033-2909.86.3.638 · DOI · Paper page
- 10.1037//0033-2909.118.2.183 · DOI · Paper page
- Rosenthal R, 1991, Essentials of Behavioral Research.
Cited by
Sorted by citations
Corporate Social and Financial Performance: A Meta-Analysis
Orlitzky, M., Schmidt, F. L., & Rynes, S. L. (2003) · Organization Studies · doi:10.1177/0170840603024003910 · Cited by 7,641
Emotion-regulation strategies across psychopathology: A meta-analytic review
Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.11.004 · Cited by 6,686
Acceptance and commitment therapy: model, processes and outcomes
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2005.06.006 · Cited by 6,587
An effect size primer: A guide for clinicians and researchers.
Ferguson, C. J. (2009) · Professional Psychology: Research and Practice · doi:10.1037/a0015808 · Cited by 3,467
Predictors of posttraumatic stress disorder and symptoms in adults: a meta-analysis
Ozer, E. J., Best, S. R., Lipsey, T. L., & Weiss, D. S. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.1.52 · Cited by 3,322
Metadata
- Authors
- Rosenthal, R., & DiMatteo, M. R.
- Affiliations
- University of California, Riverside (US)
- Venue
- Annual Review of Psychology
- Publisher
- Annual Reviews
- Published
- 2001-02
- Volume
- 52
- Issue
- 1
- Pages
- 59-82
- ISSN
- 0066-4308 (ISSN-L)
- 1545-2085
- OpenAlex
- W1870939480
- Cited by
- 1,780
- References
- 67
- Citations by year
- 2026: 41 · 2025: 62 · 2024: 97 · 2023: 99 · 2022: 75 · 2021: 98 · 2020: 109 · 2019: 103 · 2018: 89 · 2017: 83 · 2016: 75 · 2015: 81 · 2014: 85 · 2013: 79 · 2012: 91
- FWCI
- 7.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annual Review of Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- meta-analysis, effect size, effect size estimation, effect size heterogeneity, moderators, history of meta-analysis
- MeSH terms
- Humans, Research, Science, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
