Refinements in validity generalization methods: Implications for the situational specificity hypothesis.
Abstract
Using a large database, this study examined three refinements of validity generalization procedures: (a) a more accurate procedure for correcting the residual SD for range restriction to estimate SDP, (b) use of f instead of study-observed rs in the formula for sampling error variance, and (c) removal of non-Pearson rs. The first procedure does not affect the amount of variance accounted for by artifacts. The addition of the second and third procedures increased the mean percentage of validity variance accounted for by artifacts from 70% to 82%, a 17% increase. The cumulative addition of all three procedures decreased the mean SDf estimate from .150 to .106, a 29% decrease. Six additional variance-producing artifacts were identified that could not be corrected for. In light of these, we concluded that the obtained estimates of mean SDP and mean validity variance accounted for were consistent with the hypothesis that the true mean SDP value is close to zero. These findings provide further evidence against the situational specificity hypothesis. The first published validity generalization research study (Schmidt & Hunter, 1977) hypothesized that if all sources of artifactual variance in cognitive test validities could be controlled methodologically through study design (e.g., construct validity of tests and criterion measures, computational errors) or corrected for (e.g., sampling error, measurement error), there might be no remaining variance in validities across settings. That is, not only would validity be generalizable based on 90% credibility values in the estimated true validity distributions, but all observed variance in validities would be shown to be artifactual and the situational specificity hypothesis would be shown to be false even in its limited form. However, subsequent validity generalization research (e.g., Pearlman, Schmidt, & Hunter, 1980; Schmidt, Gast-Rosenberg, & Hunter, 1980; Schmidt, Hunter, Pearlman, & Shane, 1979) was based on data drawn from the general published and unpublished research literature, and therefore it was not possible to control or correct for the sources of artifactual variance that can generally be controlled for only through study design and execution (e.g., computational and typographical errors, study differences in criterion contamination). Not unexpectedly, many of these meta-analyses accounted for less than 100% of observed validity variance, and the average across studies was also less than 100% (e.g., see Pearlman et al., 1980; Schmidt et al., 1979). The conclusion that the validity of cognitive abilities tests in employment is generalizable is now widely accepted (e.g., see
Cites (1)
- Development and test of a new model for validity generalization.Callender, John C., Osburn, H. G. · 1980 · Journal of Applied Psychology
Cited by (26)
- The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings.Schmidt, Frank L., Hunter, John E. · 1998 · Psychological Bulletin
- The five factor model of personality and job performance in the European Community.Jesús F. Salgado · 1997 · Journal of Applied Psychology
- The role of social support in the process of work stress: a meta-analysisViswesvaran, Chockalingam, Sanchez, Juan I., Fisher, Jeffrey · 1999 · Journal of Vocational Behavior
- General Mental Ability in the World of Work: Occupational Attainment and Job Performance.Schmidt, Frank L., Hunter, John · 2004 · Journal of Personality and Social Psychology
- Emotional intelligence: A meta-analytic investigation of predictive validity and nomological netVan Rooy, David L, Viswesvaran, Chockalingam · 2004 · Journal of Vocational Behavior
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Refinements in validity generalization methods: Implications for the situational specificity hypothesis. https://app.cochat.ai/papers/10.1037//0021-9010.78.1.3
- The paper
- Schmidt, F. L., Law, K., Hunter, J. E., & Rothstein, H. R. (1993). Refinements in validity generalization methods: Implications for the situational specificity hypothesis. Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.78.1.3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.