The DOI matches its Crossref record; 45 of the 49 references it cites were found in a registry or an index.
Writing meta-analytic reviews.
Abstract
This article describes what should typically be included in the introduction, method, results, and discussion sections of a meta-analytic review. Method sections include information on literature searches, criteria for inclusion of studies, and a listing of the characteristics recorded for each study. Results sections include information describing the distribution of obtained effect sizes, central ten-dencies, variability, tests of significance, confidence intervals, tests for heterogeneity, and contrasts (univariate or multivariate). The interpretation of meta-analytic results is often facilitated by the inclusion of the binomial effect size display procedure, the coefficient of robustness, file drawer analysis, and, where overall results are not significant, the counternull value of the obtained effect size and power analysis. The purpose of this article is to provide some guidelines for the preparation of meta-analytic reviews of literature. Meta-an-alytic reviews are quantitative summaries of research domains that describe the typical strength of the effect or phenomenon, its variability, its statistical significance, and the nature of the moderator variables from which one can predict the relative strength of the effect or phenomenon (Cooper, 1989; Glass,
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Heterogeneity of effect size and signi fi cance of categorical moderators were determined by Q -statistics, 38,39 and dimensional moderators were determined by mixed-effects -model metaregressions. 40”
From Parenting Behavior and Child Language: A Meta-analysis. (2019) · cites this paper as [38]
In the citing paper
“In 1 study, a nonsigni fi cant fi nding was reported without accompanying statistical information, and therefore, P = .50 was entered into the Comprehensive Meta-Analysis Software. 38”
From Parenting Behavior and Child Language: A Meta-analysis. (2019) · cites this paper as [38]
In the citing paper
“Calculations were based on a random-effects model to account for existing heterogeneity among studies. 38”
From Parenting Behavior and Child Language: A Meta-analysis. (2019) · cites this paper as [38]
In the citing paper
“A common problem in performing a meta‐analysis is that studies may not have been published because of non‐significant or unfavorable findings, the so called “publication or file drawer bias” (Rosenthal, 1995).”
From Does Natural Mentoring Matter? A Multilevel Meta‐analysis on the Association Between Natural Mentoring and Youth Outcomes (2018) · cites this paper as Rosenthal, 1995
In the citing paper
“Low reliability has also been shown to affect Type II-not Type I-error, which makes it less likely to observe significant results but does not cause spuriously significant findings [50,51]”
From Gender Differences in Coping and Psychological Adaptation during the COVID-19 Pandemic (2023) · cites this paper as [51]
In the citing paper
“As Rosenthal (1995) has pointed out, it is rather likely that published studies have reported lower probabilities of a Type I er ror than have those studies 'squirreled away in fi le drawers.'”
From A meta-analytic review of the effects of high stress on eyewitness memory (2004) · cites this paper as Rosenthal, 1995
In the citing paper
“However, before concluding that the null hypothesis that Z ma = 0.00 might have validity for children, one should consider that in this instance the counternull hypothesis (Rosenthal, 1995), Z ma = -2.02, is just as likely to be true.”
From A meta-analytic review of the effects of high stress on eyewitness memory (2004) · cites this paper as Rosenthal, 1995
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,707
The Handbook of Social Psychology
Gravatt, A. E., Lindzey, G. E., & Aronson, F. (1969) · The Family Coordinator · doi:10.2307/582242 · Cited by 12,518
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
The visual display of quantitative information
(1985) · General Pharmacology The Vascular System · doi:10.1016/0306-3623(85)90230-7 · Cited by 5,090
Applied Nonparametric Statistics
Wozniak, P. J., & Daniel, W. W. (1991) · Technometrics · doi:10.2307/1268795 · Cited by 5,008
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249
Self-citationCohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Essentials of Behavioral Research: Methods and Data Analysis
Rosenthal, R., & Rosnow, R. L. (1984) · Cited by 3,925
Self-citationMeta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,896
BARNETT, V. D., Lewis, T., & Abeles, F. F. (1979) · Physics Today · doi:10.1063/1.2995737 · Cited by 2,834
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 2,003
Self-citationParametric measures of effect size.
Rosenthal, R. (1994) · Cited by 1,856
Self-citationSumming up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486
Cleveland, W. S. (1985) · Medical Entomology and Zoology · Cited by 1,425
Contrast Analysis: Focused Comparisons in the Analysis of Variance.
Levin, J. R., Rosenthal, R., & Rosnow, R. L. (1987) · Journal of the American Statistical Association · doi:10.2307/2289419 · Cited by 1,252
Self-citationA simple, general purpose display of magnitude of experimental effect.
Rosenthal, R., & Rubin, D. B. (1982) · Journal of Educational Psychology · doi:10.1037//0022-0663.74.2.166 · Cited by 1,233
Self-citationInterpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,158
Self-citationEstimation of effect size from a series of independent experiments.
Hedges, L. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.92.2.490 · Cited by 1,030
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · cited 2×
“file drawers (Sterling, 1959).”
BackgroundBook Review: Integrating Research: A Guide for Literature Reviews
Kolbe, R. H., & Cooper, H. M. (1991) · Journal of Marketing Research · doi:10.2307/3172876 · Cited by 598
A Handbook for data analysis in the behavioral sciences: methodological issues
(1993) · Choice Reviews Online · doi:10.5860/choice.31-0602 · Cited by 544
Lindzey, G. E., & Aronson, E. (1985) · Random House eBooks · Cited by 508
Stochastically dependent effect sizes.
Gleser, L. J., & Olkin, I. (1994) · Cited by 485
(1994) · Cited by 463
Vevea, J. L., Zelinsky, N. A. M., & Orwin, R. G. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.13 · Cited by 417
Comparing effect sizes of independent studies.
Rosenthal, R., & Rubin, D. B. (1982) · Psychological Bulletin · doi:10.1037//0033-2909.92.2.500 · Cited by 406
Self-citationHedges, L. V. (1994) · Cited by 397
Meta-Analytic Procedures for Combining Studies With Multiple Effect Sizes
Rosenthal, R., & Rubin, D. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.3.400 · Cited by 373
Self-citationBuchanan, R., & Kosslyn, S. M. (1996) · Design Issues · doi:10.2307/1511748 · Cited by 251
Measures of effect size for categorical data.
Fleiss, J. L. (1994) · Cited by 248
SCIENTIFIC COMMUNICATION AND LITERATURE RETRIEVAL
White, H. D. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.7 · Cited by 221
Replication in behavioral research.
Rosenthal, R. (1990) · Journal of social behavior and personality · Cited by 214
Self-citationSensitivity analysis and diagnostics.
Greenhouse, J. B., & Iyengar, S. (1994) · Cited by 192
The Counternull Value of an Effect Size: A New Statistic
Rosenthal, R., & Rubin, D. B. (1994) · Psychological Science · doi:10.1111/j.1467-9280.1994.tb00281.x · Cited by 170
Self-citationCooper, H. M., & Hedges, L. V. (1994) · Journal of International Crisis and Risk Communication Research · Cited by 170
Wortman, P. M. (1994) · Cited by 154
Combining significance levels.
Becker, B. J. (1994) · Cited by 115
IDENTIFYING POTENTIALLY INTERESTING VARIABLES AND ANALYSIS OPPORTUNITIES
Lipsey, M. W. (2019) · Russell Sage Foundation eBooks · doi:10.7758/9781610448864.11 · Cited by 100
Comparing significance levels of independent studies.
Rosenthal, R., & Rubin, D. B. (1979) · Psychological Bulletin · doi:10.1037//0033-2909.86.5.1165 · Cited by 99
Self-citationHypotheses and problems in research synthesis.
Hall, J. A., Tickle‐Degnen, L., Rosenthal, R., & Mosteller, F. C. (1994) · Cited by 98
Self-citationKoplan, S. M. (1979) · Community Mental Health Review · doi:10.1300/j257v03n05_04 · Cited by 53
Hedges, L. V., Cooper, H. M., & Bushman, B. J. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.1.188 · Cited by 46
Flagged first, then by citations
4 of the 49 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A meta-analytic test of intergroup contact theory.
Pettigrew, T. F., & Tropp, L. R. (2006) · Journal of personality and social psychology · doi:10.1037/0022-3514.90.5.751 · Cited by 9,708
Corporate Social and Financial Performance: A Meta-Analysis
Orlitzky, M., Schmidt, F. L., & Rynes, S. L. (2003) · Organization Studies · doi:10.1177/0170840603024003910 · Cited by 7,641
The job satisfaction–job performance relationship: A qualitative and quantitative review.
Judge, T. A., Thoresen, C. J., Bono, J. E., & Patton, G. K. (2001) · Psychological Bulletin · doi:10.1037/0033-2909.127.3.376 · Cited by 4,760
Sin, N. L., & Lyubomirsky, S. (2009) · Journal of Clinical Psychology · doi:10.1002/jclp.20593 · Cited by 2,955
Outlier and influence diagnostics for meta-analysis
Viechtbauer, W., & Cheung, M. W.-L. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.11 · Cited by 2,949
Metadata
- Authors
- Rosenthal, R.
- Affiliations
- Harvard University (US)
- Corresponding author
- Rosenthal, Robert
- Venue
- Psychological Bulletin
- Published
- 1995-09
- Volume
- 118
- Issue
- 2
- Pages
- 183-192
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2109747954
- Cited by
- 1,108
- References
- 49
- Citations by year
- 2026: 16 · 2025: 30 · 2024: 38 · 2023: 40 · 2022: 35 · 2021: 47 · 2020: 38 · 2019: 55 · 2018: 60 · 2017: 32 · 2016: 41 · 2015: 46 · 2014: 51 · 2013: 34 · 2012: 48
- FWCI
- 31.75
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- Funders
- James McKeen Cattell Fund
- Harvard University
- Faculty of Arts and Sciences
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- effect size, heterogeneity, power analysis, moderators
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
