High stakes testing in higher education and employment: Appraising the evidence for validity and fairness.
Abstract
The authors review criticisms commonly leveled against cognitively loaded tests used for employment and higher education admissions decisions, with a focus on large-scale databases and meta-analytic evidence. They conclude that (a) tests of developed abilities are generally valid for their intended uses in predicting a wide variety of aspects of short-term and long-term academic and job performance, (b) validity is not an artifact of socioeconomic status, (c) coaching is not a major determinant of test performance, (d) tests do not generally exhibit bias by underpredicting the performance of minority group members, and (e) test-taking motivational mechanisms are not major determinants of test performance in these high-stakes settings.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The power of standardized achievement tests to predict later academic and occupational outcomes is well-established.
“The power of standardized achievement tests to predict later academic and occupational outcomes is well-established (Kuncel and Hezlett [2007]; Sackett, Borneman and Connelly [2008]; Willingham [1985]).”
From Personality Psychology and Economics (2011) · cites this paper as Sackett, Borneman and Connelly [2008]
Cites
Sample Selection Bias as a Specification Error
Heckman, J. J. (1979) · Econometrica · doi:10.2307/1912352 · Cited by 28,779 · see citing works
Stereotype threat and the intellectual test performance of African Americans.
Steele, C. M., & Aronson, J. (1995) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.69.5.797 · Cited by 8,181 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
Cohen, D. (1998) · The Women s Review of Books · doi:10.2307/4022859 · Cited by 4,323 · see citing works
Schmidt, F. L., & Hunter, J. E. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.2.262 · Cited by 4,180 · see citing works
Do Psychosocial and Study Skill Factors Predict College Outcomes? A Meta-Analysis.
Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.2.261 · Cited by 2,479 · see citing works
Self-discipline outdoes IQ in predicting academic performance of adolescents.
Duckworth, A. L., & Seligman, M. E. P. (2005) · Psychological Science · doi:10.1111/j.1467-9280.2005.01641.x · Cited by 2,253 · see citing works
Contending with group image: The psychology of stereotype and social identity threat
Steele, C. M., Spencer, S. J., & Aronson, J. (2002) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(02)80009-0 · Cited by 2,201 · see citing works
Gender differences in mathematics performance: A meta-analysis.
Hyde, J. S., Fennema, E., & Lamon, S. J. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.139 · Cited by 1,732 · see citing works
The Black-White Test Score Gap
Jencks, C., & Phillips, M. (2011) · Rowman & Littlefield Publishers eBooks · doi:10.5771/9780815746119 · Cited by 1,718 · see citing works
Rotundo, M., & Sackett, P. R. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.1.66 · Cited by 1,545 · see citing works
Self-citationGender differences in verbal ability: A meta-analysis.
Hyde, J. S., & Linn, M. C. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.104.1.53 · Cited by 1,405 · see citing works
An Introduction to Sample Selection Bias in Sociological Data
Berk, R. A. (1983) · American Sociological Review · doi:10.2307/2095230 · Cited by 1,213 · see citing works
Sex Differences in Mental Test Scores, Variability, and Numbers of High-Scoring Individuals
Hedges, L. V., & Nowell, A. (1995) · Science · doi:10.1126/science.7604277 · Cited by 1,112 · see citing works
Comparative analysis of the reliability of job performance ratings.
Viswesvaran, C., Öneş, D. S., & Schmidt, F. L. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.5.557 · Cited by 774 · see citing works
Psychological Tests and Personnel Decisions.
Ashton, G. C., Cronbach, L. J., & Gleser, C. (1968) · Biometrics · doi:10.2307/2528052 · Cited by 715 · see citing works
TEST BIAS: PREDICTION OF GRADES OF NEGRO AND WHITE STUDENTS IN INTEGRATED COLLEGES
Cleary, T. A. (1968) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1968.tb00613.x · Cited by 658 · see citing works
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001) · Psychological Bulletin · doi:10.1037/0033-2909.127.1.162 · Cited by 639 · cited 3× · see citing works
“For example, the Kuncel et al. (2001) meta-analysis of relationships between the GRE and various outcomes reports a range-restriction-corrected correlation of .41 with first-year grades but a much smaller corrected correlation of .22 with degree completion.”
BackgroundMethodInfluentialSternberg, R. J., Wagner, R. K., Williams, W. M., Horvath, J. A., & al, E. (1995) · American Psychologist · doi:10.1037/0003-066x.50.11.912 · Cited by 604 · see citing works
Frey, M. C., & Detterman, D. K. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00687.x · Cited by 585 · see citing works
Taylor, H. C., & Russell, J. T. (1939) · Journal of Applied Psychology · doi:10.1037/h0057079 · Cited by 493 · see citing works
Willingham, W. W., & Cole, N. S. (1997) · Choice Reviews Online · doi:10.5860/choice.35-2259 · Cited by 438 · see citing works
Correction for range restriction: An expanded typology.
Sackett, P. R., & Yang, H. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.1.112 · Cited by 399 · see citing works
Self-citationSackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001) · American Psychologist · doi:10.1037/0003-066x.56.4.302 · Cited by 387 · see citing works
Self-citationPROJECT A VALIDITY RESULTS: THE RELATIONSHIP BETWEEN PREDICTOR AND CRITERION DOMAINS
McHenry, J. J., Hough, L. M., TOQUAM, J. L., Hanson, M. A., & Ashworth, S. D. (1990) · Personnel Psychology · doi:10.1111/j.1744-6570.1990.tb01562.x · Cited by 385 · see citing works
Standardized Tests Predict Graduate Students' Success
Kuncel, N. R., & Hezlett, S. A. (2007) · Science · doi:10.1126/science.1136618 · Cited by 336 · see citing works
ROTH, P. L., BEVIER, C. A., BOBKO, P., SWITZER, F. S., & TYLER, P. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00094.x · Cited by 325 · see citing works
MOTIVATIONAL COMPONENTS OF TEST TAKING
ARVEY, R. D., STRICKLAND, W., DRAUDEN, G., & MARTIN, C. (1990) · Personnel Psychology · doi:10.1111/j.1744-6570.1990.tb00679.x · Cited by 324 · see citing works
Implications of direct and indirect range restriction for meta-analysis methods and findings.
Hunter, J. E., Schmidt, F. L., & Le, H. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.3.594 · Cited by 315 · see citing works
Geiser, S., & Studley, W. R. (2002) · Educational Assessment · doi:10.1207/s15326977ea0801_01 · Cited by 293 · see citing works
Toward a Broader View of Social Stereotyping.
Biernat, M. (2003) · American Psychologist · doi:10.1037/0003-066x.58.12.1019 · Cited by 282 · see citing works
THORNDIKE, R. L. (1971) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1971.tb00907.x · Cited by 265 · see citing works
Affirmative action: Psychological data and the policy debates.
Crosby, F. J., Iyer, A., Clayton, S., & Downing, R. A. (2003) · American Psychologist · doi:10.1037/0003-066x.58.2.93 · Cited by 233 · see citing works
Within-group norming and other forms of score adjustment in preemployment testing.
Sackett, P. R., & Wilk, S. L. (1994) · American Psychologist · doi:10.1037/0003-066x.49.11.929 · Cited by 230 · see citing works
Self-citationOn the interpretation of the correlation coefficient as a measure of predictive efficiency.
Brogden, H. E. (1946) · Journal of Educational Psychology · doi:10.1037/h0061548 · Cited by 221 · see citing works
Linn, R. L. (1973) · Review of Educational Research · doi:10.3102/00346543043002139 · Cited by 200 · see citing works
Tracking Exceptional Human Capital Over Two Decades
Lubinski, D., Benbow, C., Webb, R., & Bleske-Rechek, A. (2006) · Psychology Science · doi:10.1111/j.1467-9280.2006.01685.x · Cited by 194 · see citing works
Evaluating Gender Biases on Actual Job Performance of Real People: A Meta‐Analysis1
Bowen, C., Swim, J. K., & Jacobs, R. R. (2000) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2000.tb02432.x · Cited by 193 · see citing works
When your top choice turns you down: Effect of rejected offers on the utility of selection tests.
Murphy, K. R. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.1.133 · Cited by 191 · see citing works
Role of recruitment in staffing utility analysis.
Boudreau, J. W., & Rynes, S. L. (1985) · Journal of Applied Psychology · doi:10.1037//0021-9010.70.2.354 · Cited by 174 · see citing works
Ethnic group differences in measures of job performance: a new meta-analysis.
Roth, P. L., Huffcutt, A. I., & Bobko, P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.4.694 · Cited by 171 · see citing works
Cullen, M. J., Hardison, C. M., & Sackett, P. R. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.2.220 · Cited by 165 · see citing works
Self-citationA reexamination of black-white mean differences in work performance: More data, more moderators.
McKay, P. F., & McDaniel, M. A. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.3.538 · Cited by 143 · see citing works
Fair game? The use of standardized admissions tests in higher education
Zwick, R. (2003) · PsycEXTRA Dataset · doi:10.1037/e672192007-009 · Cited by 142 · see citing works
Effects of Coaching on SAT I: Reasoning Test Scores
Powers, D. E., & Rock, D. A. (1999) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1999.tb00549.x · Cited by 137 · see citing works
Fairness in Employment Testing
Hartigan, J. A., & Wigdor, A. K. (1989) · Science · doi:10.1126/science.2740906 · Cited by 127 · see citing works
Coaching for the Scholastic Aptitude Test: Further Synthesis and Appraisal
Becker, B. J. (1990) · Review of Educational Research · doi:10.3102/00346543060003373 · Cited by 122 · see citing works
Young, J. W., & Kobrin, J. L. (2001) · Cited by 120 · see citing works
Predicting College Grades: An Analysis of Institutional Trends over Two Decades
Stage, F. K., Willingham, W. W., Lewis, C., Morgan, R., & Ramist, L. (1993) · The Journal of Higher Education · doi:10.1080/00221546.1993.11778415 · Cited by 119 · see citing works
Linearity of ability-performance relationships: A reconfirmation.
Coward, W. M., & Sackett, P. R. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.3.297 · Cited by 101 · see citing works
Self-citationBartlett, C. J., Bobko, P., Mosier, S., & HANNAN, R. (1978) · Personnel Psychology · doi:10.1111/j.1744-6570.1978.tb00442.x · Cited by 101 · see citing works
Fairness is not validity or cultural bias in racial-group assessment: A quantitative perspective.
Helms, J. E. (2006) · American Psychologist · doi:10.1037/0003-066x.61.8.845 · Cited by 92 · see citing works
Sex differences in predictions of college grades from scholastic aptitude test scores.
Stricker, L. J., Rock, D. A., & Burton, N. W. (1993) · Journal of Educational Psychology · doi:10.1037/0022-0663.85.4.710 · Cited by 91 · see citing works
(2008) · Encyclopedia of Special Education · doi:10.1002/9780470373699.speced1719 · Cited by 86 · see citing works
Gender Differences on the Scholastic Aptitude Test, the American College Test and College Grades
Mau, W.-C., & Lynn, R. (2001) · Educational Psychology · doi:10.1080/01443410020043832 · Cited by 80 · see citing works
A New Coefficient: Application to Biserial Correlation and to Estimation of Selective Efficiency
Brogden, H. E. (1949) · Psychometrika · doi:10.1007/bf02289151 · Cited by 72 · see citing works
GENDER BIAS AND THE COLLEGE PREDICTIONS OF THE SATS: A Cry of Despair
Leonard, D. K., & Jiang, J. (1999) · Research in Higher Education · doi:10.1023/a:1018759308259 · Cited by 68 · see citing works
Farrell, J. N., & McDaniel, M. A. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.1.60 · Cited by 58 · see citing works
Increasing the number of psychologists of color: Public policy issues for affirmative diversity.
Vásquez, M. J. T., & Jones, J. M. (2006) · American Psychologist · doi:10.1037/0003-066x.61.2.132 · Cited by 44 · see citing works
Stricker, L. J., Rock, D. A., Burton, N. W., et al. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.2.178 · Cited by 35 · see citing works
Waters, S. D., Cooper, S. R., Kuncel, N. R., Sackett, P. R., & Arneson, J. J. (2009) · Cited by 30 · see citing works
Self-citationZwick, R. (2002) · Phi Delta Kappan · doi:10.1177/003172170208400411 · Cited by 26 · see citing works
Racial groups and test fairness, considering history and construct validity.
Newman, D. A., Hanges, P. J., & Outtz, J. L. (2007) · American Psychologist · doi:10.1037/0003-066x.62.9.1082 · Cited by 19 · see citing works
Expectancy Charts II. Their Theoretical Development
Lawshe, C. H., Bolda, R. A., Brune, R. L., et al. (1958) · Personnel Psychology · doi:10.1111/j.1744-6570.1958.tb00040.x · Cited by 19 · see citing works
CALIFORNIA AND THE SAT: A REANALYSIS OF UNIVERSITY OF CALIFORNIA ADMISSIONS DATA *Preprint
Zwick, R., Brown, T., & Sklar, J. C. (2004) · RePEc: Research Papers in Economics · Cited by 17 · see citing works
Sackett, P. R., Kuncel, N. R., Arneson, J. J., et al. (2009) · College Board · Cited by 9 · see citing works
Self-citationNot all Affirmative Action rewards merit.
JOHNSON, J. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.123 · Cited by 5 · see citing works
Speaking of fairness in testing.
Griffore, R. J. (2007) · American Psychologist · doi:10.1037/0003-066x.62.9.1081 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Personality Psychology and Economics
Almlund, M., Duckworth, A. L., Heckman, J., & Kautz, T. (2011) · National Bureau of Economic Research · doi:10.3386/w16822 · Cited by 540 · see citing works
Duckworth, A. L., Quinn, P. D., & Tsukayama, E. (2012) · Journal of Educational Psychology · doi:10.1037/a0026280 · Cited by 416 · see citing works
Berry, C. M., Clark, M. A., & McClure, T. K. (2011) · Journal of Applied Psychology · doi:10.1037/a0023222 · Cited by 99 · see citing works
Is stereotype threat a useful construct for organizational psychology research and practice?
Kalokerinos, E. K., von Hippel, C., & Zacher, H. (2014) · Industrial and Organizational Psychology · doi:10.1111/iops.12167 · Cited by 90 · see citing works
Nakano, S., & Watkins, M. W. (2013) · Psychology in the Schools · doi:10.1002/pits.21724 · Cited by 27 · see citing works
Motivated cultural worldview adherence and culturally loaded test performance
Landau, M. J., Greenberg, J., & Rothschild, Z. K. (2009) · Personality and Social Psychology Bulletin · doi:10.1177/0146167208329630 · Cited by 17 · see citing works
- Authors
- Sackett, P. R., Borneman, M. J., & Connelly, B. S.
- Affiliations
- University of Minnesota (US)
- Venue
- American Psychologist
- Published
- 2008
- Volume
- 63
- Issue
- 4
- Pages
- 215-227
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2167792530
- Cited by
- 233
- References
- 68
- Citations by year
- 2026: 9 · 2025: 12 · 2024: 5 · 2023: 13 · 2022: 7 · 2021: 11 · 2020: 11 · 2019: 7 · 2018: 8 · 2017: 13 · 2016: 13 · 2015: 14 · 2014: 26 · 2013: 23 · 2012: 19
- FWCI
- 22.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Psychologist (publishedVersion)
- PubMed (publishedVersion)
- Funders
- College Board
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Medical Education and Admissions
- Keywords
- high-stakes testing, test validity, test fairness, university admission, employment testing, cognitive ability tests, socioeconomic status, test-taking motivation, student performance prediction, job performance prediction
- MeSH terms
- Educational Status, Employment, Humans, Psychological Tests, Universities, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
