Meta-analysis of correlations revisited: Attempted replication and extension of Field's (2001) simulation studies.
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
In 2 Monte Carlo studies of fixed- and random-effects meta-analysis for correlations, A. P. Field (2001) ostensibly evaluated Hedges-Olkin-Vevea Fisher-z and Schmidt-Hunter Pearson-r estimators and tests in 120 conditions. Some authors have cited those results as evidence not to meta-analyze Fisher-z correlations, especially with heterogeneous correlation parameters. The present attempt to replicate Field's simulations included comparisons with analytic values as well as results for efficiency and confidence-interval coverage. Field's results under homogeneity were mostly replicable, but those under heterogeneity were not: The latter exhibited up to over .17 more bias than ours and, for tests of the mean correlation and homogeneity, respectively, nonnull rejection rates up to .60 lower and .65 higher. Changes to Field's observations and conclusions are recommended, and practical guidance is offered regarding simulation evidence and choices among methods. Most cautions about poor performance of Fisher-z methods are largely unfounded, especially with a more appropriate z-to-r transformation. The Appendix gives a computer program for obtaining Pearson-r moments from a normal Fisher-z distribution, which is used to demonstrate distortion due to direct z-to-r transformation of a mean Fisher-z correlation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hunter, J. E., & Schmidt, F. L. (2000) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00156 · Cited by 833
The power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716
The power of statistical tests for moderators in meta-analysis
Hedges, L. V., & Pigott, T. D. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.426 · Cited by 672
A simple confidence interval for meta-analysis
Sidik, K., & Jonkman, J. N. (2002) · Statistics in Medicine · doi:10.1002/sim.1262 · Cited by 431
The Problems in Using Fixed-Effects Models of Meta-Analysis on Real-World Data
Field, A. P. (2003) · Understanding Statistics · doi:10.1207/s15328031us0202_02 · Cited by 237
Cited by
Nahrgang, J. D., Morgeson, F. P., & Hofmann, D. A. (2011) · Journal of Applied Psychology · doi:10.1037/a0021484 · Cited by 1,632
Field, A. P., & Gillett, R. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x502733 · Cited by 1,162
The relationship between materialism and personal well-being: A meta-analysis
Dittmar, H., Bond, R., Hurst, M., & Kasser, T. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0037409 · Cited by 890
“…2009; Shadish & Haddock, 2009) for the meta-analysis, in which correlations are transformed to Fisher’s z to stabilize the variance, and followed Hafdahl’s (2009, 2010; Hafdahl & Williams, 2009) recommendation to use an integral z-to-r transformation for converting our results back to the rmetric.”
Haedt-Matt, A. A., & Keel, P. K. (2011) · Psychological Bulletin · doi:10.1037/a0023660 · Cited by 799
“…fixedeffects analyses due to concerns about overly narrow confidence intervals and inflated Type I error rates associated with fixed-effects, Monte Carlo simulations have found that a random-effects model is biased when based on few studies (i.e., k = 5–10; Field, 2001; Hafdahl & Williams, 2009).”
Meta-analysis of the relations of anxiety sensitivity to the depressive and anxiety disorders.
Naragon-Gainey, K. (2010) · Psychological Bulletin · doi:10.1037/a0018055 · Cited by 555
Hurst, M., Dittmar, H., Bond, R., & Kasser, T. (2013) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2013.09.003 · Cited by 416
Murayama, K., & Elliot, A. J. (2012) · Psychological Bulletin · doi:10.1037/a0028324 · Cited by 341
Degner, J., & Dalege, J. (2013) · Psychological Bulletin · doi:10.1037/a0031436 · Cited by 300
Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H.-D. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2011.01.006 · Cited by 290
Imhoff, R., & Lamberty, P. (2018) · European Journal of Social Psychology · doi:10.1002/ejsp.2494 · Cited by 267
ADHD and personality: a meta-analytic review
Gomez, R., & Corr, P. J. (2014) · Clinical Psychology Review · doi:10.1016/j.cpr.2014.05.002 · Cited by 151
“This property renders the RE model more desirable for a metaanalysis, when there is a reasonable number of data set, such as five or more (Field, 2001; Hafdahl & Williams, 2009).”
BackgroundKrieglstein, F., Beege, M., Rey, G. D., Ginns, P., Krell, M., & Schneider, S. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09683-4 · Cited by 118
Scott, J. L., Dawkins, S., Quinn, M. G., Sanderson, K., Elliott, K.-E. J., Stirling, C., Schüz, B., & Robinson, A. (2016) · Aging & Mental Health · doi:10.1080/13607863.2015.1040724 · Cited by 77
Metadata
- Authors
- Hafdahl, A. R., & Williams, M. A.
- Venue
- Psychological Methods
- Published
- 2009
- Volume
- 14
- Issue
- 1
- Pages
- 24-42
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2099704999
- Cited by
- 55
- References
- 5
- FWCI
- 0.35
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.