Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors.
Abstract
A Monte Carlo study was conducted to determine Types I and II error rates of the Schmidt and Hunter (S&H) meta-analysis method and the U statistic for assessing homogeneity within a set of correlations. One thousand samples of correlations were generated randomly to fill each of 450 cells of an 18 × 5 × 5 (Underlying Population Correlations × Numbers of Correlations Compared × Sample Size Per Correlation) design. To assess Type I error rates, correlations were drawn from the same population. To assess power, correlations were drawn from two different populations. As compared with U, which was uniformly robust, the Type I error rate for the S&H method was unacceptably high in many cells, particularly when the criterion for determining homogeneity was set at a highly conservative level. Power for the S&H method increased with increasing size of population differences, sample size per correlation, and in some cases, number of correlations compared. The U statistic did more poorly in most conditions in protecting from Type II errors.
Cites (1)
- Primary, Secondary, and Meta-Analysis of ResearchGLASS, GENE V · 1976 · Educational Researcher
Cited by (19)
- The Handbook of Research Synthesis and Meta-Analysis2019
- Assessing heterogeneity in meta-analysis: Q statistic or I² index?Huedo-Medina, Tania B., Sánchez-Meca, Julio, Marín-Martínez, Fulgencio, Botella, Juan · 2006 · Psychological Methods
- The Role of Justice in Organizations: A Meta-AnalysisCohen-Charash, Yochi, Spector, Paul E. · 2001 · Organizational Behavior and Human Decision Processes
- Meta-Analytic review of leader–member exchange theory: Correlates and construct issues.Gerstner, Charlotte R., Day, David V. · 1997 · Journal of Applied Psychology
- Career Benefits Associated With Mentoring for Proteges: A Meta-Analysis.Allen, Tammy D., Eby, Lillian T., Poteet, Mark L., Lentz, Elizabeth, Lima, Lizzette · 2004 · Journal of Applied Psychology
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors. https://app.cochat.ai/papers/10.1037//0021-9010.72.1.3
- The paper
- Spector, P. E., & Levine, E. L. (1987). Meta-analysis for integrating study outcomes: A Monte Carlo study of its susceptibility to Type I and Type II errors. Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.72.1.3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.