Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods.
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
The efficacy of the Hedges and colleagues, Rosenthal-Rubin, and Hunter-Schmidt methods for combining correlation coefficients was tested for cases in which population effect sizes were both fixed and variable. After a brief tutorial on these meta-analytic methods, the author presents two Monte Carlo simulations that compare these methods for cases in which the number of studies in the meta-analysis and the average sample size of studies were varied. In the fixed case the methods produced comparable estimates of the average effect size; however, the Hunter-Schmidt method failed to control the Type I error rate for the associated significance tests. In the variable case, for both the Hedges and colleagues and Hunter-Schmidt methods, Type I error rates were not controlled for meta-analyses including 15 or fewer studies and the probability of detecting small effects was less than .3. Some practical recommendations are made about the use of meta-analysis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Eight studies in Table 3 involving a total of 444 participants reported actual correlation sizes and these were combined meta-analytically (Field, 2001; Hunter & Schmidt, 1990). Where more than one effect was reported separately within a single study, their average was used. This yielded an average r = .02, which did not differ significantly from zero, z = .29, p > .70.”
From Episodic memory, perceptual memory, and their interaction: Foundations for a theory of posttraumatic stress disorder. (2014) · cites this paper as Field, 2001
In the citing paper
“Moreover, Field (2001) shows that using the Fisher z transformation for heterogeneous effect sizes overestimates the true effect size by about 15 % to 45 % .”
From A Meta-Analysis of Organizational Complaint Handling and Customer Responses (2011) · cites this paper as Field (2001)
In the citing paper
“we did not apply a Fisher z-transformation when integrating effect sizes, because these correlations would underestimate the coefficients' actual variance and overestimate the effect size of heterogeneous correlations (as are typical in marketing studies) by 15% to 45%.”
From Understanding and managing customer relational benefits in services: a meta-analysis (2020) · cites this paper as Field 2001
Citations
Cites
Discovering Statistics Using SPSS
Field, A. P., & Miles, J. N. V. (2000) · Medical Entomology and Zoology · Cited by 28,288
Self-citationStatistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META‐ANALYSIS
BARRICK, M. R., & MOUNT, M. K. (1991) · Personnel Psychology · doi:10.1111/j.1744-6570.1991.tb00688.x · Cited by 8,689
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
(2006) · Technometrics · doi:10.1198/tech.2006.s417 · Cited by 4,950
Primary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · The Statistician · doi:10.2307/2348600 · Cited by 4,249
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,853
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Aldenderfer, M. S., & Blashfield, R. K. (1984) · Cited by 2,358
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,327
Goldstein, H. (2010) · Wiley Series in Probability and Statistics · doi:10.1002/9780470973394 · Cited by 1,568
Meta-Analysis: Cumulating Research Findings across Studies
Feldman, J. M., Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1984) · The Academy of Management Review · doi:10.2307/258247 · Cited by 1,199
Interpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,158
Meta-Analysis: Cumulating Research Findings across Studies
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1986) · Educational Researcher · doi:10.2307/1175263 · Cited by 792
Wolf, F. (1986) · doi:10.4135/9781412984980 · Cited by 784
Discovering Statistics Using SPSS for Windows: Advanced Techniques for Beginners
Field, A. P. (2000) · Sage eBooks · Cited by 737
Self-citationMooney, C. (1997) · doi:10.4135/9781412985116 · Cited by 677
Averaging correlation coefficients: Should Fisher’s z transformation be used?
Silver, N. C., & Dunlap, W. P. (1987) · Journal of Applied Psychology · doi:10.1037/0021-9010.72.1.146 · Cited by 606
What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology.
Schmidt, F. L. (1992) · American Psychologist · doi:10.1037/0003-066x.47.10.1173 · Cited by 564 · cited 1×
“Hunter & Schmidt (1990) discuss ways in which these”
BackgroundComparing effect sizes of independent studies.
Rosenthal, R., & Rubin, D. B. (1982) · Psychological Bulletin · doi:10.1037//0033-2909.92.2.500 · Cited by 406
Hedges, L. V. (1994) · Cited by 397
Overton, R. C. (1998) · Psychological Methods · doi:10.1037//1082-989x.3.3.354 · Cited by 356
Combining Information: Statistical Issues and Opportunities for Research
Draper, D. L., Gaver, D. P., Goel, P., Greenhouse, J. B., Hedges, L. V., Morris, C., Tucker, J., & Waternaux, C. (1993) · Cited by 279
(2024) · Dictionary of Toxicology · doi:10.1007/978-981-99-9283-6_1776 · Cited by 276
Validity generalization results for computer programmers.
Schmidt, F. L., Gast-Rosenberg, I., & Hunter, J. E. (1980) · Journal of Applied Psychology · doi:10.1037//0021-9010.65.6.643 · Cited by 166
Meta-Analysis Using Multilevel Models with an Application to the Study of Class Size Effects
Goldstein, H., Yang, M., Omar, R., Turner, R., & Thompson, S. (2000) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00200 · Cited by 135
Comparison of three major meta-analytic approaches.
Johnson, B. T., Mullen, B., & Salas, E. (1995) · Journal of Applied Psychology · doi:10.1037//0021-9010.80.1.94 · Cited by 133
Comment: Assumptions and Procedures in the File Drawer Problem
Rosenthal, R., & Rubin, D. (1988) · Statistical Science · doi:10.1214/ss/1177013014 · Cited by 93
Schmidt, F. L., Hunter, J. E., & Raju, N. S. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.4.665 · Cited by 74
Schmidt, F. L., & Hunter, J. E. (1999) · Journal of Applied Psychology · doi:10.1037//0021-9010.84.1.144 · Cited by 61
Osburn, H. G., & Callender, J. (1992) · Journal of Applied Psychology · doi:10.1037//0021-9010.77.2.115 · Cited by 60
Nonlinearity of range corrections in meta-analysis: Test of an improved procedure.
Law, K. S., Schmidt, F. L., & Hunter, J. E. (1994) · Journal of Applied Psychology · doi:10.1037//0021-9010.79.3.425 · Cited by 48
Hedges, L. V. (1992) · Journal of Educational Statistics · doi:10.2307/1165125 · Cited by 46
Special Issue on Meta-Analysis
Becker, B. J., & Hedges, L. V. (1992) · Journal of Educational Statistics · doi:10.3102/10769986017004277 · Cited by 42
Averaging correlation coefficients: Influence of heterogeneity and set size.
Strube, M. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.3.559 · Cited by 41
Research works printed but not linked: 14
- Becker, 1996, The generalizability of empirical research results., 363
- Cohen, 1988, Statistical power analysis for the behaviouralsciences.
- 10.1037/0033-2909.112.1.155 · doi:10.1037/0033-2909.112.1.155 · Paper page
- Glass, 1981, Meta-analysis in socialresearch.
- Hedges, 1985, Statistical methods formeta-analysis.
- Hunter, 1982, Meta-analysis: Cumulating research findingsacross studies.
- 10.1037/0021-9010.80.1.94 · doi:10.1037/0021-9010.80.1.94 · Paper page
- 10.1037/0021-9010.79.3.425 · doi:10.1037/0021-9010.79.3.425 · Paper page
- 1992, Combining information: Statistical issues andopportunities for research.
- 10.1037/0021-9010.77.2.115 · doi:10.1037/0021-9010.77.2.115 · Paper page
- 10.1037/1082-989X.3.3.354 · doi:10.1037/1082-989x.3.3.354 · Paper page
- 10.1037/0033-2909.92.2.500 · doi:10.1037/0033-2909.92.2.500 · Paper page
- 10.1037/0021-9010.84.1.144 · doi:10.1037/0021-9010.84.1.144 · Paper page
- 10.1037/0021-9010.65.6.643 · doi:10.1037/0021-9010.65.6.643 · Paper page
Cited by
Assessing heterogeneity in meta-analysis: Q statistic or I² index?
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.193 · Cited by 4,256
A meta-analysis of the technology acceptance model
King, W. R., & He, J. (2006) · Information & Management · doi:10.1016/j.im.2006.05.003 · Cited by 2,906
Distinct brain networks for adaptive and stable task control in humans
Dosenbach, N. U. F., Fair, D. A., Miezin, F. M., Cohen, A. L., Wenger, K. K., Dosenbach, R. A. T., Fox, M. D., Snyder, A. Z., Vincent, J. L., Raichle, M. E., et al. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0704320104 · Cited by 2,632
DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.1.74 · Cited by 2,479
“*Fiedler, K., & Walka, I. (1993). Training lie detectors to use nonverbal cues instead of global heuristics.”
BackgroundA meta-analytic review of work–family conflict and its antecedents
Byron, K. (2005) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2004.08.009 · Cited by 2,336
Metadata
- Authors
- Field, A. P.
- Affiliations
- University of Sussex (GB)
- Corresponding author
- Field, Andy P.
- ORCID
- Field, Andy P.
- Venue
- Psychological Methods
- Published
- 2001
- Volume
- 6
- Issue
- 2
- Pages
- 161-180
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2103027218
- Cited by
- 725
- References
- 39
- Citations by year
- 2026: 13 · 2025: 29 · 2024: 25 · 2023: 34 · 2022: 41 · 2021: 36 · 2020: 36 · 2019: 33 · 2018: 33 · 2017: 37 · 2016: 35 · 2015: 43 · 2014: 41 · 2013: 41 · 2012: 37
- FWCI
- 2.10
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Licence
- cc-by
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- fixed effects model, random effects models, type I error rate, sample size, Monte Carlo simulation
- MeSH terms
- Humans, Monte Carlo Method, Psychometrics, Meta-Analysis as Topic, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.