On the robustness, bias, and stability of statistics from meta-analysis of correlation coefficients: Some initial Monte Carlo findings.

Oswald, Frederick L., Johnson, Jeff W.·1998·Journal of Applied Psychology

Abstract

Each of several Monte Carlo simulations generated 100 sets of observed study correlations based on normal, heteroscedastic, or slightly nonlinear bivariate distributions, with one population correlation coefficient and true variance of 0. A version of J. E. Hunter and F. L. Schmidt's (1990b) meta-analysis was applied to each study set. Within simulations, ρ was accurate on average. σ ρ 2 was biased; one would correctly conclude more than half the time that no moderator effects existed. However, cases of variation in ρ and especially in σ ρ 2 indicated that results from individual meta-analyses could deviate substantially from what was found on average. Findings for these no-moderator cases offer applied psychologists some guidelines and cautions when drawing conclusions about true population correlations and true moderator effects (e.g., situational specificity, validity generalization) from meta-analytic results.

Cites (2)

  1. THEORY TESTING: COMBINING PSYCHOMETRIC META‐ANALYSIS AND STRUCTURAL EQUATIONS MODELING
    VISWESVARAN, CHOCKALINGAM, ONES, DENIZ S. · 1995 · Personnel Psychology
  2. Mediation of interpersonal expectancy effects: 31 meta-analyses
    Harris, Monica J., Rosenthal, Robert · 1985 · Psychological Bulletin

Show all 2 references

Cited by (13)

  1. Trust in leadership: Meta-analytic findings and implications for research and practice.
    Dirks, Kurt T., Ferrin, Donald L. · 2002 · Journal of Applied Psychology
  2. Relations between personality and coping: A meta-analysis.
    Connor-Smith, Jennifer K., Flachsbart, Celeste · 2007 · Journal of Personality and Social Psychology
  3. A Meta-Analysis of Work Demand Stressors and Job Performance: Examining Main and Moderating Effects
    GILBOA, SIMONA, SHIROM, ARIE, FRIED, YITZHAK, COOPER, CARY · 2008 · Personnel Psychology
  4. Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies.
    Oswald, Frederick L., Mitchell, Gregory, Blanton, Hart, Jaccard, James, Tetlock, Philip E. · 2013 · Journal of Personality and Social Psychology
  5. The Affective Underpinnings of Job Perceptions and Attitudes: A Meta-Analytic Review and Integration.
    Thoresen, Carl J., Kaplan, Seth A., Barsky, Adam P., Warren, Christopher R., de Chermont, Kelly · 2003 · Psychological Bulletin

See all 49

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: On the robustness, bias, and stability of statistics from meta-analysis of correlation coefficients: Some initial Monte Carlo findings. https://app.cochat.ai/papers/10.1037//0021-9010.83.2.164
The paper
Oswald, F. L., & Johnson, J. W. (1998). On the robustness, bias, and stability of statistics from meta-analysis of correlation coefficients: Some initial Monte Carlo findings. Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.83.2.164

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.