Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews
Abstract
▪ Abstract We describe the history and current status of the meta-analytic enterprise. The advantages and historical criticisms of meta-analysis are described, as are the basic steps in a meta-analysis and the role of effect sizes as chief coins of the meta-analytic realm. Advantages of the meta-analytic procedures include seeing the “landscape” of a research domain, keeping statistical significance in perspective, minimizing wasted data, becoming intimate with the data summarized, asking focused research questions, and finding moderator variables. Much of the criticism of meta-analysis has been based on simple misunderstanding of how meta-analyses are actually carried out. Criticisms of meta-analysis that are applicable are equally applicable to traditional, nonquantitative, narrative reviews of the literature. Much of the remainder of the chapter deals with the processes of effect size estimation, the understanding of the heterogeneity of the obtained effect sizes, and the practical and scientific importance of the effect sizes obtained.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There was some disagreement among the articles whether the research question should be specific (narrowly formulated) or general (broadly formulated).
“There was some disagreement (Table 4) whether the research question should be specific (narrowly formulated) [51, 53], or general (broadly formulated) [35, 47, 49].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
The majority of articles discussed aspects general to systematic reviews including advantages and limitations of different literature sources and databases, search tools, the importance of identifying unpublished studies, literature searching techniques including how to build a search string and reporting.
“The majority of articles discussed aspects general to systematic reviews including advantages and limitations of different literature sources and databases, search tools, the importance of identifying unpublished studies, literature searching techniques including how to build a search string and reporting [12, 29, 33–36, 46, 48, 50, 51, 53, 54, 57, 59–62, 67, 68].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
Most articles considered the combination of RCTs and non-randomised designs.
“Most articles considered the combination of RCTs and non-randomised designs [29, 35, 38, 39, 49, 51, 56, 61, 75, 81].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
Two articles recommended stratifying by study design if heterogeneity is large and adjusting for study design if heterogeneity is small.
“Two articles recommended stratifying by study design if heterogeneity is large and adjusting for study design if heterogeneity is small [51, 77].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
Some authors suggested that results should be combined in a single meta-analysis only in the absence of marked heterogeneity.
“Some authors suggested that results should be combined in a single meta-analysis only in the absence of marked heterogeneity [34, 51].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
One common recommendation was to explore heterogeneity in sensitivity analyses, comparing studies stratified by design or clinical features.
“One common recommendation was to explore heterogeneity in sensitivity analyses, comparing studies stratified by design or clinical features [12, 16, 29, 34, 39, 51, 55, 56, 66, 77, 85, 100, 102].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [51]
Cites
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,707 · see citing works
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · see citing works
Self-citationMethods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,832 · see citing works
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · see citing works
Self-citationMeta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249 · see citing works
Self-citationDiMatteo, M. R., Lepper, H. S., & Croghan, T. W. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.14.2101 · Cited by 4,177 · see citing works
Self-citationThe earth is round (p < .05).
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003 · see citing works
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,896 · see citing works
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,363 · see citing works
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154 · see citing works
Yusuf, S., Zucker, D. M., Passamani, E. R., Peduzzi, P. N., Takaro, T., Fisher, L. D., Kennedy, J. W., Davis, K. B., Killip, T., Norris, R. M., et al. (1994) · The Lancet · doi:10.1016/s0140-6736(94)91963-1 · Cited by 2,129 · see citing works
Smith, M. L., Glass, G. V., & Miller, T. I. (1980) · Medical Entomology and Zoology · Cited by 2,123 · see citing works
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 1,925 · see citing works
Self-citationEagly, A. H., Ashmore, R. D., Makhijani, M. G., & Longo, L. C. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.1.109 · Cited by 1,875 · see citing works
Comorbidity of anxiety and unipolar mood disorders
Mineka, S., Watson, D., & Clark, L. A. (1998) · Annual Review of Psychology · doi:10.1146/annurev.psych.49.1.377 · Cited by 1,573 · see citing works
Summing up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486 · see citing works
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422 · see citing works
Contrast Analysis: Focused Comparisons in the Analysis of Variance.
Levin, J. R., Rosenthal, R., & Rosnow, R. L. (1987) · Journal of the American Statistical Association · doi:10.2307/2289419 · Cited by 1,252 · see citing works
Self-citationA simple, general purpose display of magnitude of experimental effect.
Rosenthal, R., & Rubin, D. B. (1982) · Journal of Educational Psychology · doi:10.1037//0022-0663.74.2.166 · Cited by 1,233 · see citing works
Self-citationContrasts and Effect Sizes in Behavioral Research
Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511804403 · Cited by 1,144 · see citing works
Self-citationWriting meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108 · see citing works
Self-citationA META‐ANALYSIS OF SELF‐SUPERVISOR, SELF‐PEER, AND PEER‐SUPERVISOR RATINGS
Harris, M. M., & Schaubroeck, J. M. (1988) · Personnel Psychology · doi:10.1111/j.1744-6570.1988.tb00631.x · Cited by 1,080 · see citing works
The Handbook of Research Synthesis
Ziegel, E. R., Cooper, H., & Hedges, L. V. (1995) · Technometrics · doi:10.2307/1269651 · Cited by 936 · see citing works
The Effects of Television Violence on Antisocial Behavior: A Meta-Analysis1
Paik, H., & Comstock, G. A. (1994) · Communication Research · doi:10.1177/009365094021004004 · Cited by 857 · see citing works
Contrasts and Effect Sizes in Behavioral Research: A Correlational Approach
Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (1999) · Cited by 854 · see citing works
Self-citationIntegrating Research: A Guide for Literature Reviews
Kolbe, R. H., & Cooper, H. M. (1991) · Journal of Marketing Research · doi:10.2307/3172876 · Cited by 598 · see citing works
Preliminary Report: Findings from the Aspirin Component of the Ongoing Physicians' Health Study
The Steering Committee of the Physicians' Health Study Research Group (1988) · New England Journal of Medicine · doi:10.1056/nejm198801283180431 · Cited by 522 · see citing works
Conway, J. M., & Huffcutt, A. I. (1997) · Human Performance · doi:10.1207/s15327043hup1004_2 · Cited by 465 · see citing works
How hard is hard science, how soft is soft science? The empirical cumulativeness of research.
Hedges, L. V. (1987) · American Psychologist · doi:10.1037/0003-066x.42.5.443 · Cited by 435 · see citing works
Meta-Analysis for Explanation: A Casebook.
Jackson, S., Cook, T. D., Cooper, H. M., Cordray, D. S., Hartmann, H. I., Hedges, L. V., Light, R. J., Louis, T. A., & Mosteller, F. C. (1993) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2074578 · Cited by 434 · see citing works
Meta-analytic procedures for combining studies with multiple effect sizes.
Rosenthal, R., & Rubin, D. B. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.3.400 · Cited by 394 · see citing works
Self-citationStudies of educational interventions and outcomes in diabetic adults: A meta-analysis revisited
Brown, S. A. (1990) · Patient Education and Counseling · doi:10.1016/0738-3991(90)90070-2 · Cited by 361 · see citing works
Baum, M. L., Anish, D. S., Chalmers, T. C., Sacks, H. S., Smith, H., & Fagerstrom, R. M. (1981) · New England Journal of Medicine · doi:10.1056/nejm198110013051404 · Cited by 340 · see citing works
Effectiveness of cognitive-behavior therapy for maladapting children: A meta-analysis.
Durlak, J. A., Fuhrman, T., & Lampman, C. B. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.204 · Cited by 318 · see citing works
Achieving patient compliance : the psychology of the medical practitioner's role
DiMatteo, M. R., & DiNicola, D. D. (1982) · DigitalGeorgetown (Georgetown University Library) · Cited by 274 · see citing works
Self-citationContrasts and Effect Sizes in Behavioral Research: A Correlational Approach
Boik, R. J. (2001) · Journal of the American Statistical Association · doi:10.1198/jasa.2001.s432 · Cited by 257 · see citing works
Cesarean childbirth and psychosocial outcomes: A meta-analysis.
DiMatteo, M. R., Morton, S. C., Lepper, H. S., Damush, T. M., Carney, M. F., Pearson, M., & Kahn, K. L. (1996) · Health Psychology · doi:10.1037/0278-6133.15.4.303 · Cited by 247 · see citing works
Self-citationJudgment Studies: Design, Analysis, and Meta-Analysis.
Ridgeway, C., & Rosenthal, R. (1989) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2074155 · Cited by 236 · see citing works
Self-citationRosenthal, R. (1987) · Cambridge University Press eBooks · doi:10.1017/cbo9780511527807 · Cited by 219 · see citing works
Self-citationQuantitative methods for literature reviews
Green, B. F., & Hall, J. A. (1984) · Annual Review of Psychology · doi:10.1146/annurev.ps.35.020184.000345 · Cited by 213 · see citing works
A meta-analysis of controlled trials of cardiac patient education
Mullen, P. D., Mains, D. A., & Vélez, R. (1992) · Patient Education and Counseling · doi:10.1016/0738-3991(92)90194-n · Cited by 212 · see citing works
Meta-Analysis for Explanation: A Casebook
Spilker, B. (1993) · JAMA · doi:10.1001/jama.1993.03500230123042 · Cited by 199 · see citing works
Achieving Patient Compliance: The Psychology of the Medical Practitioner's Role
Smith, C. K. (1983) · JAMA · doi:10.1001/jama.1983.03340170098042 · Cited by 191 · see citing works
The Counternull Value of an Effect Size: A New Statistic
Rosenthal, R., & Rubin, D. B. (1994) · Psychological Science · doi:10.1111/j.1467-9280.1994.tb00281.x · Cited by 170 · see citing works
Self-citationMeta-analysis of the effects of psychoeducational care in adults with asthma
Devine, E. C. (1996) · Research in Nursing & Health · doi:10.1002/(sici)1098-240x(199610)19:5 · Cited by 118 · see citing works
Testing for moderator variables in meta‐analysis: Issues and methods
Hall, J. A., & Rosenthal, R. (1991) · Communication Monographs · doi:10.1080/03637759109376240 · Cited by 116 · see citing works
Self-citationHow often are our numbers wrong?
Rosenthal, R. (1978) · American Psychologist · doi:10.1037/0003-066x.33.11.1005 · Cited by 115 · see citing works
Self-citationParadoxical interventions: A meta-analysis.
Shoham-Salomon, V., & Rosenthal, R. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.55.1.22 · Cited by 100 · see citing works
Self-citationHypotheses and problems in research synthesis.
Hall, J. A., Tickle‐Degnen, L., Rosenthal, R., & Mosteller, F. C. (1994) · Cited by 98 · see citing works
Self-citationLand, K. C., Wachter, K. W., & Straf, M. L. (1991) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2071894 · Cited by 97 · see citing works
Franck, L. S. (1990) · Nursing Research · doi:10.1097/00006199-199011000-00015 · Cited by 94 · see citing works
Fabio, R. A. (1995) · Physical Therapy · doi:10.1093/ptj/75.10.865 · Cited by 94 · see citing works
Devine, E. C., & Pearcy, J. M. (1996) · Patient Education and Counseling · doi:10.1016/0738-3991(96)00862-2 · Cited by 93 · see citing works
Meyer, R. G., & Smith, S. R. (1977) · American Psychologist · doi:10.1037//0003-066x.32.8.638 · Cited by 92 · see citing works
A Meta-Analysis of the Effects Of Psychoeducational Care in Adults with Hypertension
Devine, E. C., & REIFSCHNEIDER, E. (1995) · Nursing Research · doi:10.1097/00006199-199507000-00009 · Cited by 80 · see citing works
Interpreting and Evaluating Meta-Analysis
Hall, J. A., & Rosenthal, R. (1995) · Evaluation & the Health Professions · doi:10.1177/016327879501800404 · Cited by 75 · see citing works
Self-citationProgress in clinical psychology: Is there any?
Rosenthal, R. (1995) · Clinical Psychology Science and Practice · doi:10.1111/j.1468-2850.1995.tb00035.x · Cited by 75 · see citing works
Self-citationHow Hard Is Hard Science, How Soft Is Soft Science?
Hedges, L. V. (1987) · Cited by 71 · see citing works
Quality-weighting of studies in meta-analytic research
Rosenthal, R. (1991) · Psychotherapy Research · doi:10.1080/10503309112331334031 · Cited by 31 · see citing works
Self-citationMatters of Life and Death: Risks vs. Benefits of Medical Care
Evans, R. W. (1985) · JAMA · doi:10.1001/jama.1985.03360060139050 · Cited by 27 · see citing works
A New Goal: Preventing Disease, Not Infection
Cohen, J. (1993) · Science · doi:10.1126/science.8266069 · Cited by 25 · see citing works
DiMatteo, M. R. (2000) · Cited by 20 · see citing works
Self-citationMatters of Life & Death: Risks vs. Benefits of Medical Care
Robin, E. D. (1984) · Medical Entomology and Zoology · Cited by 14 · see citing works
Paradoxical interventions: A meta-analysis.
Shoham-Salomon, V., & Rosenthal, R. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.55.1.22 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Corporate Social and Financial Performance: A Meta-Analysis
Orlitzky, M., Schmidt, F. L., & Rynes, S. L. (2003) · Organization Studies · doi:10.1177/0170840603024003910 · Cited by 7,641 · see citing works
Emotion-regulation strategies across psychopathology: A meta-analytic review
Aldao, A., Nolen-Hoeksema, S., & Schweizer, S. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.11.004 · Cited by 6,686 · see citing works
Acceptance and commitment therapy: model, processes and outcomes
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2005.06.006 · Cited by 6,587 · see citing works
An effect size primer: A guide for clinicians and researchers.
Ferguson, C. J. (2009) · Professional Psychology: Research and Practice · doi:10.1037/a0015808 · Cited by 3,467 · see citing works
Predictors of posttraumatic stress disorder and symptoms in adults: a meta-analysis
Ozer, E. J., Best, S. R., Lipsey, T. L., & Weiss, D. S. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.1.52 · Cited by 3,322 · see citing works
Sin, N. L., & Lyubomirsky, S. (2009) · Journal of Clinical Psychology · doi:10.1002/jclp.20593 · Cited by 2,955 · see citing works
A meta-analysis of the technology acceptance model
King, W. R., & He, J. (2006) · Information & Management · doi:10.1016/j.im.2006.05.003 · Cited by 2,906 · see citing works
A meta-analysis of the five-factor model of personality and academic performance.
Poropat, A. E. (2009) · Psychological Bulletin · doi:10.1037/a0014996 · Cited by 2,729 · see citing works
Physician Communication and Patient Adherence to Treatment
Haskard‐Zolnierek, K. B., & DiMatteo, M. R. (2009) · Medical Care · doi:10.1097/mlr.0b013e31819a5acc · Cited by 2,522 · see citing works
Self-citationEllis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461 · see citing works
DiMatteo, M. R. (2004) · Medical Care · doi:10.1097/01.mlr.0000114908.90348.f9 · Cited by 2,169 · see citing works
Self-citationSocial support and patient adherence to medical treatment: a meta-analysis.
DiMatteo, M. R. (2004) · Health Psychology · doi:10.1037/0278-6133.23.2.207 · Cited by 2,107 · see citing works
Self-citationIntrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis.
Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014) · Psychological Bulletin · doi:10.1037/a0035661 · Cited by 1,981 · see citing works
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987 · see citing works
A Guide to Conducting a Systematic Literature Review of Information Systems Research
Okoli, C., & Schabram, K. (2011) · doi:10.2139/ssrn.1954824 · Cited by 1,849 · see citing works
How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence
Cummings, J. J., & Bailenson, J. N. (2016) · Media Psychology · doi:10.1080/15213269.2015.1015740 · Cited by 1,749 · see citing works
Broadbent, J., & Poon, W. L. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.04.007 · Cited by 1,692 · see citing works
On the Use of Beta Coefficients in Meta-Analysis.
Peterson, R. A., & Brown, S. P. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.1.175 · Cited by 1,643 · see citing works
The efficacy of motivational interviewing: a meta-analysis of controlled clinical trials
Burke, B. L., Arkowitz, H., & Menchola, M. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.5.843 · Cited by 1,575 · see citing works
Martin, B. C., McNally, J. J., & Kay, M. J. (2013) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2012.03.002 · Cited by 1,565 · see citing works
“Meta-analyses are sometimes criticized for mixing good and bad studies (Rosenthal & DiMatteo, 2001).”
Patient Adherence and Medical Treatment Outcomes: A Meta-Analysis
Robin DiMatteo, M., Giordani, P. J., Lepper, H. S., & Croghan, T. W. (2002) · Medical Care · doi:10.1097/00005650-200209000-00009 · Cited by 1,490 · see citing works
Self-citationThe efficacy of psychodynamic psychotherapy
Shedler, J. (2010) · American Psychologist · doi:10.1037/a0018378 · Cited by 1,452 · see citing works
A Guide to Conducting a Standalone Systematic Literature Review
Okoli, C. (2015) · Communications of the Association for Information Systems · doi:10.17705/1cais.03743 · Cited by 1,413 · see citing works
Cheung, M. W.-L. (2014) · Psychological Methods · doi:10.1037/a0032968 · Cited by 1,342 · see citing works
Motor coordination in autism spectrum disorders: A synthesis and meta-analysis
Fournier, K. A., Hass, C. J., Naik, S. K., Lodha, N., & Cauraugh, J. H. (2010) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-010-0981-3 · Cited by 1,271 · see citing works
Gender differences in emotion expression in children: A meta-analytic review.
Chaplin, T. M., & Aldao, A. (2013) · Psychological Bulletin · doi:10.1037/a0030737 · Cited by 1,247 · see citing works
“Therefore, we conducted random effects models, as they assume that effect sizes differ from the population by sampling error plus random variability among the studies (Field, 2003; Hedges & Vevea, 1998; Hunter & Schmidt, 1990; Lipsey & Wilson, 2001; Rosenthal & DiMatteo, 2001).”
MethodPredictors of posttraumatic stress disorder and symptoms in adults: A meta-analysis.
Ozer, E. J., Best, S. R., Lipsey, T. L., & Weiss, D. S. (2008) · Psychological Trauma Theory Research Practice and Policy · doi:10.1037/1942-9681.s.1.3 · Cited by 1,252 · see citing works
Westen, D., Novotny, C. M., & Thompson-Brenner, H. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.4.631 · Cited by 1,212 · see citing works
Two Decades of Terror Management Theory: A Meta-Analysis of Mortality Salience Research
Burke, B. L., Martens, A., & Faucher, E. H. (2010) · Personality and Social Psychology Review · doi:10.1177/1088868309352321 · Cited by 1,188 · see citing works
Field, A. P., & Gillett, R. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x502733 · Cited by 1,162 · see citing works
A Systematic Literature Review on Fault Prediction Performance in Software Engineering
Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.103 · Cited by 1,154 · see citing works
Meta-analysis of 41 functional neuroimaging studies of executive function in schizophrenia
Minzenberg, M. J., Laird, A. R., Thelen, S., Carter, C. S., & Glahn, D. C. (2009) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2009.91 · Cited by 1,158 · see citing works
Taras, V., Kirkman, B. L., & Steel, P. (2010) · Journal of Applied Psychology · doi:10.1037/a0018938 · Cited by 1,160 · see citing works
What Schools Need to Know About Fostering School Belonging: a Meta-analysis
Allen, K., Kern, M. L., Vella-Brodrick, D., Hattie, J., & Waters, L. (2018) · Educational Psychology Review · doi:10.1007/s10648-016-9389-8 · Cited by 1,141 · see citing works
Kremen, C., & Miles, A. (2012) · Ecology and Society · doi:10.5751/es-05035-170440 · Cited by 1,126 · see citing works
Perceptual-Cognitive Expertise in Sport: A Meta-Analysis
Mann, D. T. Y., Williams, A. M., Ward, P., & Janelle, C. M. (2007) · Journal of Sport and Exercise Psychology · doi:10.1123/jsep.29.4.457 · Cited by 1,425 · see citing works
Destination image and tourist loyalty: a meta-analysis
Zhang, H., Fu, X., Cai, L. A., & Lu, L. (2014) · Tourism Management · doi:10.1016/j.tourman.2013.06.006 · Cited by 1,098 · see citing works
A systematic review of systematic review process research in software engineering
Kitchenham, B., & Brereton, P. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.07.010 · Cited by 1,094 · see citing works
Variables associated with achievement in higher education: A systematic review of meta-analyses.
Schneider, M., & Preckel, F. (2017) · Psychological Bulletin · doi:10.1037/bul0000098 · Cited by 1,162 · see citing works
Can There Ever Be Too Many Options? A Meta-Analytic Review of Choice Overload
Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2010) · Journal of Consumer Research · doi:10.1086/651235 · Cited by 1,070 · see citing works
A comprehensive meta-analysis of the relationship between Emotional Intelligence and health
Martins, A., Ramalho, N., & Morin, E. (2010) · Personality and Individual Differences · doi:10.1016/j.paid.2010.05.029 · Cited by 1,070 · see citing works
A meta-analysis of crop pest and natural enemy response to landscape complexity
Chaplin‐Kramer, R., O’Rourke, M. E., Blitzer, E. J., & Kremen, C. (2011) · Ecology Letters · doi:10.1111/j.1461-0248.2011.01642.x · Cited by 1,037 · see citing works
Hartnell, C. A., Ou, A. Y., & Kinicki, A. J. (2011) · Journal of Applied Psychology · doi:10.1037/a0021987 · Cited by 1,088 · see citing works
Lorains, F. K., Cowlishaw, S., & Thomas, S. A. (2011) · Addiction · doi:10.1111/j.1360-0443.2010.03300.x · Cited by 1,006 · see citing works
Nguyen, A. D., & Benet‐Martínez, V. (2012) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022111435097 · Cited by 1,021 · see citing works
Emotion regulation strategies in depressive and anxiety symptoms in youth: A meta-analytic review
Schäfer, J. Ö., Naumann, E., Holmes, E. A., Tuschen-Caffier, B., & Samson, A. C. (2017) · Journal of Youth and Adolescence · doi:10.1007/s10964-016-0585-0 · Cited by 941 · see citing works
Ruffman, T., Henry, J., Livingstone, V., & Phillips, L. (2008) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2008.01.001 · Cited by 944 · see citing works
A meta-analysis of the effect of cognitive bias modification on anxiety and depression
Hallion, L. S., & Ruscio, A. M. (2011) · Psychological Bulletin · doi:10.1037/a0024355 · Cited by 891 · see citing works
The five-factor model and personality disorder empirical literature: A meta-analytic review
Saulsman, L. M., & Page, A. C. (2004) · Clinical Psychology Review · doi:10.1016/j.cpr.2002.09.001 · Cited by 915 · see citing works
Specificity of cognitive emotion regulation strategies: a transdiagnostic examination
Aldao, A., & Nolen-Hoeksema, S. (2010) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.06.002 · Cited by 860 · see citing works
Deficits in facial affect recognition among antisocial populations: A meta-analysis
Marsh, A. A., & Blair, J. (2007) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2007.08.003 · Cited by 849 · see citing works
The effect of contact with natural environments on positive and negative affect: A meta-analysis
McMahan, E. A., & Estes, D. (2015) · The Journal of Positive Psychology · doi:10.1080/17439760.2014.994224 · Cited by 833 · see citing works
A meta-analysis of risk factors for post-traumatic stress disorder in children and adolescents
Trickey, D., Siddaway, A. P., Meiser-Stedman, R., Serpell, L., & Field, A. P. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2011.12.001 · Cited by 828 · see citing works
Taylor, G., Jungert, T., Mageau, G. A., Schattke, K., Dedic, H., Rosenfield, S., & Koestner, R. (2014) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2014.08.002 · Cited by 816 · see citing works
Exploring the relationship between celebrity endorser effects and advertising effectiveness
Amos, C., Holmes, G. R., & Strutton, D. R. (2008) · International Journal of Advertising · doi:10.1080/02650487.2008.11073052 · Cited by 835 · see citing works
What type of leadership behaviors are functional in teams? A meta-analysis
Burke, C. S., Stagl, K. C., Klein, C., Goodwin, G. F., Salas, E., & Halpin, S. M. (2006) · The Leadership Quarterly · doi:10.1016/j.leaqua.2006.02.007 · Cited by 1,107 · see citing works
Chida, Y., & Steptoe, A. (2010) · Hypertension · doi:10.1161/hypertensionaha.109.146621 · Cited by 787 · see citing works
Predictors of individual-level innovation at work: A meta-analysis.
Hammond, M. M., Neff, N. L., Farr, J. L., Schwall, A. R., & Zhao, X. (2011) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0018556 · Cited by 807 · see citing works
Does Team Training Improve Team Performance? A Meta-Analysis
Eduardo Salas, A., DiazGranados, D., Klein, C. R., Burke, C. S., Stagl, K. C., Goodwin, G. F., & Halpin, S. M. (2008) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x375009 · Cited by 779 · see citing works
The use (and abuse) of the new environmental paradigm scale over the last 30 years: A meta-analysis
Hawcroft, L. J., & Milfont, T. L. (2009) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2009.10.003 · Cited by 767 · see citing works
Yildiz, P. D., Ayers, S., & Phillips, L. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.10.009 · Cited by 768 · see citing works
The road to forgiveness: A meta-analytic synthesis of its situational and dispositional correlates
Fehr, R., Gelfand, M. J., & Nag, M. (2010) · Psychological Bulletin · doi:10.1037/a0019993 · Cited by 749 · see citing works
Role of Stress, Arousal, and Coping Skills in Primary Insomnia
Morin, C. M., Rodrigue, S., & Ivers, H. (2003) · Psychosomatic Medicine · doi:10.1097/01.psy.0000030391.09558.a3 · Cited by 739 · see citing works
Brinckmann, J., Grichnik, D., & Kapsa, D. (2008) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2008.10.007 · Cited by 736 · see citing works
Food cue reactivity and craving predict eating and weight gain: A meta-analytic review
Boswell, R. G., & Kober, H. (2016) · Obesity Reviews · doi:10.1111/obr.12354 · Cited by 703 · see citing works
Gegenfurtner, A., Lehtinen, E., & Säljö, R. (2011) · Educational Psychology Review · doi:10.1007/s10648-011-9174-7 · Cited by 715 · see citing works
Climate for Creativity: A Quantitative Review
Hunter, S. T., Bedell, K. E., & Mumford, M. D. (2007) · Creativity Research Journal · doi:10.1080/10400410709336883 · Cited by 709 · see citing works
Bai, S., Hew, K. F., & Huang, B. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100322 · Cited by 689 · see citing works
Sex differences in impulsivity: A meta-analysis.
Cross, C. P., Copping, L. T., & Campbell, A. (2011) · Psychological Bulletin · doi:10.1037/a0021591 · Cited by 866 · see citing works
Tharenou, P., Donohue, R., & Cooper, B. K. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810527 · Cited by 687 · see citing works
The Empirical Status of Social Learning Theory: A Meta‐Analysis
Pratt, T. C., Cullen, F. T., Sellers, C. S., Winfree, L. T., Madensen, T. D., Daigle, L. E., Fearn, N. E., & Gau, J. M. (2009) · Justice Quarterly · doi:10.1080/07418820903379610 · Cited by 681 · see citing works
Organizational Culture and Innovation: A Meta‐Analytic Review
Büschgens, T., Bausch, A., & Balkin, D. B. (2013) · Journal of Product Innovation Management · doi:10.1111/jpim.12021 · Cited by 678 · see citing works
Health literacy and adherence to medical treatment in chronic and acute illness: A meta-analysis
Miller, T. A. (2016) · Patient Education and Counseling · doi:10.1016/j.pec.2016.01.020 · Cited by 662 · see citing works
Perceived Racism and Mental Health Among Black American Adults: A Meta-Analytic Review
Pieterse, A. L., Todd, N. R., Neville, H. A., & Carter, R. T. (2012) · Journal of Counseling Psychology · doi:10.1037/a0026208 · Cited by 649 · see citing works
The Empirics of Wetland Valuation: A Comprehensive Summary and a Meta-Analysis of the Literature
Brander, L., Florax, R. J. G. M., & Vermaat, J. E. (2006) · Environmental and Resource Economics · doi:10.1007/s10640-005-3104-4 · Cited by 645 · see citing works
META-ANALYSES OF FINANCIAL PERFORMANCE AND EQUITY: FUSION OR CONFUSION?
Dalton, D. R., Daily, C. M., Certo, S. T., & Roengpitya, R. (2003) · Academy of Management Journal · doi:10.2307/30040673 · Cited by 643 · see citing works
Meta-analysis of genetic association studies.
Munafo, M., & Flint, J. (2004) · Trends in Genetics · doi:10.1016/j.tig.2004.06.014 · Cited by 346 · see citing works
Meta-analytic structural equation modeling: A two-stage approach.
Cheung, M. W. ‐., & Chan, W. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.1.40 · Cited by 646 · see citing works
A meta-analysis of the effects of cell phones on driver performance
Caird, J. K., Willness, C. R., Steel, P., & Scialfa, C. (2008) · Accident Analysis & Prevention · doi:10.1016/j.aap.2008.01.009 · Cited by 634 · see citing works
Health benefits: Meta-analytically determining the impact of well-being on objective health outcomes
Howell, R. T., Kern, M. L., & Lyubomirsky, S. (2007) · Health Psychology Review · doi:10.1080/17437190701492486 · Cited by 627 · see citing works
Field, M., Munafò, M. R., & Franken, I. H. A. (2009) · Psychological Bulletin · doi:10.1037/a0015843 · Cited by 602 · see citing works
Examining the Impact of Cell Phone Conversations on Driving Using Meta-Analytic Techniques
Horrey, W. J., & Wickens, C. D. (2006) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006776412135 · Cited by 604 · see citing works
Impact of narratives on persuasion in health communication: a meta-analysis
Shen, F., Sheer, V. C., & Li, R. (2015) · Journal of Advertising · doi:10.1080/00913367.2015.1018467 · Cited by 574 · see citing works
The psychophysiology of posttraumatic stress disorder: a meta-analysis
Pole, N. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.5.725 · Cited by 582 · see citing works
Witt, K., Dorn, R. A. V., & Fazel, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0055942 · Cited by 585 · see citing works
Narcissism and Leadership: A Meta‐Analytic Review of Linear and Nonlinear Relationships
Grijalva, E., Harms, P. D., Newman, D. A., Gaddis, B. H., & Fraley, R. C. (2015) · Personnel Psychology · doi:10.1111/peps.12072 · Cited by 572 · see citing works
Dew, M. A., DiMartini, A. F., Dabbs, A. D. V., Myaskovsky, L., Steel, J. L., Unruh, M. L., Switzer, G. E., Zomak, R. A., Kormos, R. L., & Greenhouse, J. B. (2007) · Transplantation · doi:10.1097/01.tp.0000258599.65257.a6 · Cited by 571 · see citing works
Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010) · Psychological Medicine · doi:10.1017/s0033291709006114 · Cited by 533 · see citing works
Insight in schizophrenia: a meta-analysis
Mintz, A. R., Dobson, K. S., & Romney, D. M. (2003) · Schizophrenia Research · doi:10.1016/s0920-9964(02)00316-x · Cited by 563 · see citing works
Principles Underlying the Use of Multiple Informants' Reports
Reyes, A. D. L., Thomas, S. A., Goodman, K. L., & Kundey, S. M. A. (2012) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185617 · Cited by 569 · see citing works
Neurocognitive outcomes and recovery after pediatric TBI: meta-analytic review of the literature
Babikian, T., & Asarnow, R. (2009) · Neuropsychology · doi:10.1037/a0015268 · Cited by 557 · see citing works
Social Projection to Ingroups and Outgroups: A Review and Meta-Analysis
Robbins, J. M., & Krueger, J. I. (2005) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0901_3 · Cited by 556 · see citing works
Lawson, G. M., Hook, C. J., & Farah, M. J. (2018) · Developmental Science · doi:10.1111/desc.12529 · Cited by 556 · see citing works
Quantifying construct validity: Two simple measures.
Westen, D., & Rosenthal, R. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.84.3.608 · Cited by 543 · see citing works
Self-citationSmall Effects: The Indispensable Foundation for a Cumulative Psychological Science
Götz, F. M., Gosling, S. D., & Rentfrow, P. J. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620984483 · Cited by 553 · see citing works
Ferguson, C. J. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615592234 · Cited by 524 · see citing works
French, D., Olander, E. K., Chisholm, A., & McSharry, J. (2014) · Annals of Behavioral Medicine · doi:10.1007/s12160-014-9593-z · Cited by 531 · see citing works
Customer engagement in social media: a framework and meta-analysis
de Oliveira Santini, F., Ladeira, W. J., Pinto, D. C., Herter, M. M., Sampaio, C. H., & Babin, B. J. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-020-00731-5 · Cited by 523 · see citing works
City branding: a state‐of‐the‐art review of the research domain
Lucarelli, A., & Berg, P. O. (2011) · Journal of Place Management and Development · doi:10.1108/17538331111117133 · Cited by 522 · see citing works
Daskalopoulou, C., Stubbs, B., Kralj, C., Koukounari, A., Prince, M., & Prina, A. M. (2017) · Ageing Research Reviews · doi:10.1016/j.arr.2017.06.003 · Cited by 501 · see citing works
“However, rarely are studies identical replications of one another, so including studies that are diverse in methodology, measures, and sample within our meta-analysis exhibits the advantage of improving the generalisability of our conclusions (Rosenthal and DiMatteo, 2001).”
- Authors
- Rosenthal, R., & DiMatteo, M. R.
- Affiliations
- University of California, Riverside (US)
- Venue
- Annual Review of Psychology
- Publisher
- Annual Reviews
- Published
- 2001-02
- Volume
- 52
- Issue
- 1
- Pages
- 59-82
- ISSN
- 0066-4308 (ISSN-L)
- 1545-2085
- OpenAlex
- W1870939480
- Cited by
- 1,780
- References
- 67
- Citations by year
- 2026: 41 · 2025: 62 · 2024: 97 · 2023: 99 · 2022: 75 · 2021: 98 · 2020: 109 · 2019: 103 · 2018: 89 · 2017: 83 · 2016: 75 · 2015: 81 · 2014: 85 · 2013: 79 · 2012: 91
- FWCI
- 7.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annual Review of Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Keywords
- meta-analysis, effect size, effect size estimation, effect size heterogeneity, moderators, history of meta-analysis
- MeSH terms
- Humans, Research, Science, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
