Individual Differences in Course Choice Result in Underestimation of the Validity of College Admissions Systems
Abstract
We demonstrate that the validity of SAT scores and high school grade point averages (GPAs) as predictors of academic performance has been underestimated because of previous studies' reliance on flawed performance indicators (i.e., college GPA) that are contaminated by the effects of individual differences in course choice. We controlled for this contamination by predicting individual course grades, instead of GPAs, in a data set containing more than 5 million college grades for 167,816 students. Percentage of variance accounted for by SAT scores and high school GPAs was 30 to 40% lower when the criteria were freshman and cumulative GPAs than when the criteria were individual course grades. SAT scores and high school GPAs together accounted for between 44 and 62% of the variance in college grades. This study provides new estimates of the criterion-related validity of SAT scores and high school GPAs, and highlights the care that must be taken in choosing appropriate criteria in validity studies.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Standardized tests and high school GPA show substantial validity for predicting first-year college GPA and degree completion.
“These predictors show substantial validity for predicting various relevant outcomes, including first-year college GPA and degree completion (Berry & Sackett, 2009;Kuncel & Hezlett, 2010).”
From An Omitted Variables Approach to Disentangling Differential Prediction in College Admissions (2026) · cites this paper as Berry & Sackett, 2009
The criterion (FGPA) is an imperfect measure of academic success; it aggregates performance across heterogeneous majors, courses, and instructors, reflecting not only students' knowledge and skills but also course composition and grading practices.
“Second, the criterion is an imperfect measure of academic success; FGPA aggregates performance across heterogeneous majors, courses, and instructors, reflecting not only students' knowledge and skills but also course composition and grading practices (Berry & Sackett, 2009).”
From An Omitted Variables Approach to Disentangling Differential Prediction in College Admissions (2026) · cites this paper as Berry & Sackett, 2009
Cites
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
The Chronicle of Higher Education
Herndon, G. A. (1977) · Serials Review · doi:10.1016/s0098-7913(77)80012-9 · Cited by 4,377 · see citing works
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023 · see citing works
Gulliksen, H. O. (1950) · John Wiley & Sons, Inc. eBooks · doi:10.1037/13240-000 · Cited by 2,184 · see citing works
Kuncel, N. R., Credé, M., & Thomas, L. L. (2005) · Review of Educational Research · doi:10.3102/00346543075001063 · Cited by 1,137 · see citing works
Measurement Theory for the Behavioral Sciences.
Elliott, G. C., Ghiselli, E. Ε., Campbell, J. P., & Zedeck, S. (1982) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2067145 · Cited by 1,011 · see citing works
Schmidt, F. L., Oh, I., & Hayes, T. L. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711007x255327 · Cited by 713 · see citing works
Correction for range restriction: An expanded typology.
Sackett, P. R., & Yang, H. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.1.112 · Cited by 399 · see citing works
Self-citationBritish Journal of Mathematical and Statistical Psychology
British Psychological Society (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/(issn)2044-8317 · Cited by 389 · see citing works
Sackett, P. R., Kuncel, N. R., Arneson, J. J., Cooper, S. R., & Waters, S. D. (2009) · Psychological Bulletin · doi:10.1037/a0013978 · Cited by 241 · see citing works
Self-citationSuccess in College: The Role of Personal Qualities and Academic Ability
Pascarella, E. T., Willingham, W. W., Young, J. W., & Morris, M. M. (1987) · The Journal of Higher Education · doi:10.2307/1981398 · Cited by 163 · see citing works
Predicting College Grades: An Analysis of Institutional Trends over Two Decades
Stage, F. K., Willingham, W. W., Lewis, C., Morgan, R., & Ramist, L. (1993) · The Journal of Higher Education · doi:10.1080/00221546.1993.11778415 · Cited by 119 · see citing works
Hawkins, D. A., & Lautz, J. (2005) · Cited by 117 · see citing works
Bridgeman, B., McCamley‐Jenkins, L., & Ervin, N. (2000) · Cited by 111 · see citing works
Elliott, R., & Strenta, A. C. (1988) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1988.tb00312.x · Cited by 110 · see citing works
Predicting college grades: An analysis of institutional trends over two decades
Eysenck, H. J. (1991) · Personality and Individual Differences · doi:10.1016/0191-8869(91)90049-h · Cited by 82 · see citing works
Theory-Based University Admissions Testing for a New Millennium
Sternberg, R. J., The Rainbow Project Collaborators, & The University of Michigan Business (2004) · Educational Psychologist · doi:10.1207/s15326985ep3903_4 · Cited by 62 · see citing works
Choosing students : higher education admissions tools for the 21st century
Camara, W. J., & Kimmel, E. W. (2005) · Cited by 59 · see citing works
Rooney, C., & Schaeffer, B. (1998) · Cited by 30 · see citing works
(2005) · Routledge eBooks · doi:10.4324/9781410612533 · Cited by 30 · see citing works
An empirical comparison of the results of five validity generalization procedures.
Burke, M. J., Raju, N. S., & Pearlman, K. (1986) · Journal of Applied Psychology · doi:10.1037//0021-9010.71.2.349 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Westrick, P. A., Le, H., Robbins, S. B., Radunzel, J., & Schmidt, F. L. (2015) · Educational Assessment · doi:10.1080/10627197.2015.997614 · Cited by 222 · see citing works
Fact and Fiction in Cognitive Ability Testing for Admissions and Hiring Decisions
Kuncel, N. R., & Hezlett, S. A. (2010) · Current Directions in Psychological Science · doi:10.1177/0963721410389459 · Cited by 200 · see citing works
Adjusting medical school admission: assessing interpersonal skills using situational judgement tests
Lievens, F. (2013) · Medical Education · doi:10.1111/medu.12089 · Cited by 142 · see citing works
Galla, B. M., Shulman, E. P., Plummer, B. D., Gardner, M., Hutt, S. J., Goyer, J. P., D’Mello, S. K., Finn, A. S., & Duckworth, A. L. (2019) · American Educational Research Journal · doi:10.3102/0002831219843292 · Cited by 139 · see citing works
Keiser, H. N., Sackett, P. R., Kuncel, N. R., & Brothen, T. (2015) · Journal of Applied Psychology · doi:10.1037/apl0000069 · Cited by 127 · see citing works
Self-citationIndividual differences as predictors of work, educational, and broad life outcomes
Kuncel, N. R., Öneş, D. S., & Sackett, P. R. (2010) · Personality and Individual Differences · doi:10.1016/j.paid.2010.03.042 · Cited by 118 · see citing works
Self-citationThe Role of Socioeconomic Status in SAT-Grade Relationships and in College Admissions Decisions
Sackett, P. R., Kuncel, N. R., Beatty, A. S., Rigdon, J. L., Shen, W., & Kiger, T. B. (2012) · Psychological Science · doi:10.1177/0956797612438732 · Cited by 88 · see citing works
Self-citationA Dispositional Approach to Work–School Conflict and Enrichment
McNall, L. A., & Michel, J. S. (2010) · Journal of Business and Psychology · doi:10.1007/s10869-010-9187-0 · Cited by 60 · see citing works
Short-Term and Long-Term Effects of POGIL in a Large-Enrollment General Chemistry Course
Vincent‐Ruz, P., Meyer, T. Y., Garrett-Roe, S., & Schunn, C. D. (2020) · Journal of Chemical Education · doi:10.1021/acs.jchemed.9b01052 · Cited by 55 · see citing works
Correspondence Bias in Performance Evaluation: Why Grade Inflation Works
Moore, D. A., Swift, S. A., Sharek, Z. S., & Francesca Gino, B. (2010) · Personality and Social Psychology Bulletin · doi:10.1177/0146167210371316 · Cited by 52 · see citing works
Using personality and cognitive ability to predict academic achievement in a young adult sample
Beaujean, A. A., Firmin, M. W., Attai, S. L., Johnson, C. B., Firmin, R. L., & Mena, K. E. (2011) · Personality and Individual Differences · doi:10.1016/j.paid.2011.06.023 · Cited by 51 · see citing works
Graduate Admissions Practices: A Targeted Review of the Literature
Michel, R. S., Belur, V. K., Naemi, B., & Kell, H. J. (2019) · ETS Research Report Series · doi:10.1002/ets2.12271 · Cited by 47 · see citing works
Mattern, K. D., & Patterson, B. F. (2012) · Journal of Applied Psychology · doi:10.1037/a0030610 · Cited by 45 · see citing works
Fluid ability, crystallized ability, and performance across multiple domains: A meta-analysis
Postlethwaite, B. E. (2011) · doi:10.17077/etd.zopi8wvs · Cited by 58 · see citing works
Briley, D. A., Domiteaux, M., & Tucker–Drob, E. M. (2014) · Learning and Individual Differences · doi:10.1016/j.lindif.2014.03.010 · Cited by 39 · see citing works
Core self-evaluations interact with cognitive ability to predict academic achievement
Rosopa, P. J., & Schroeder, A. N. (2009) · Personality and Individual Differences · doi:10.1016/j.paid.2009.07.028 · Cited by 38 · see citing works
Differential prediction generalization in college admissions testing.
Aguinis, H., Culpepper, S. A., & Pierce, C. A. (2016) · Journal of Educational Psychology · doi:10.1037/edu0000104 · Cited by 37 · see citing works
Bardi, M., Koone, T., Mewaldt, S. P., & O'Connor, K. (2011) · Stress · doi:10.3109/10253890.2011.571322 · Cited by 32 · see citing works
Realistic job expectations predict academic achievement
Krammer, G., Sommer, M., & Arendasy, M. E. (2016) · Learning and Individual Differences · doi:10.1016/j.lindif.2016.09.010 · Cited by 30 · see citing works
The role of DHEA in relation to problem solving and academic performance
Wemm, S., Koone, T., Blough, E. R., Mewaldt, S. P., & Bardi, M. (2010) · Biological Psychology · doi:10.1016/j.biopsycho.2010.05.003 · Cited by 27 · see citing works
Reeve, C. L. L., & Bonaccio, S. (2011) · Intelligence · doi:10.1016/j.intell.2011.06.009 · Cited by 26 · see citing works
Development of STEM Readiness Benchmarks to Assist Educational and Career Decision Making
Mattern, K. D., Radunzel, J., & Westrick, P. A. (2015) · Cited by 25 · see citing works
Abad, F. J., Sorrel, M. A., Román, F. J., & Colom, R. (2016) · Psychological Assessment · doi:10.1037/pas0000228 · Cited by 24 · see citing works
Dropping the GRE, keeping the GRE, or GRE-optional admissions? Considering tradeoffs and fairness
Newman, D. A., Tang, C., Song, Q. C., & Wee, S. (2022) · International Journal of Testing · doi:10.1080/15305058.2021.2019750 · Cited by 23 · see citing works
Interest–Ability Profiles: An Integrative Approach to Knowledge Acquisition
Hyland, W. E., Hoff, K. A., & Rounds, J. (2022) · Journal of Intelligence · doi:10.3390/jintelligence10030043 · Cited by 21 · see citing works
Nurudeen, A. H., Fakhrou, A. A., Lawal, N., & Ghareeb, S. A. (2023) · Engineering Reports · doi:10.1002/eng2.12766 · Cited by 21 · see citing works
Noncognitive skills and critical thinking predict undergraduate academic performance
Martín‐Raugh, M. P., Kell, H. J., Ling, G., Fishtein, D., & Yang, Z. (2022) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2022.2073964 · Cited by 16 · see citing works
Higdem, J. L., Kostal, J. W., Kuncel, N. R., Sackett, P. R., Shen, W., Beatty, A. S., & Kiger, T. B. (2016) · Educational Measurement Issues and Practice · doi:10.1111/emip.12103 · Cited by 16 · see citing works
Self-citationA Snapshot in Time: Industrial–Organizational Psychology Today
Highhouse, S., & Schmitt, N. (2012) · Handbook of Psychology, Second Edition · doi:10.1002/9781118133880.hop212001 · Cited by 13 · see citing works
Dahlke, J. A., Kostal, J. W., Sackett, P. R., & Kuncel, N. R. (2018) · Journal of Applied Psychology · doi:10.1037/apl0000316 · Cited by 12 · see citing works
Self-citationDahlke, J. A., Sackett, P. R., & Kuncel, N. R. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000382 · Cited by 12 · see citing works
Self-citationBerry, C. M., Sackett, P. R., & Sund, A. E. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-012-9284-3 · Cited by 11 · see citing works
Self-citationKim, A., & Berry, C. M. (2013) · Journal of Personality · doi:10.1111/jopy.12078 · Cited by 10 · see citing works
Self-citationIntelligence and the Workplace
Drasgow, F. (2012) · Handbook of Psychology, Second Edition · doi:10.1002/9781118133880.hop212008 · Cited by 10 · see citing works
Shewach, O. R., McNeal, K. D., Kuncel, N. R., & Sackett, P. R. (2018) · Educational Measurement Issues and Practice · doi:10.1111/emip.12212 · Cited by 8 · see citing works
Self-citationFischer, F. T., Schult, J., & Hell, B. (2014) · Diagnostica · doi:10.1026/0012-1924/a000120 · Cited by 8 · see citing works
Paris, J. H., Beckowski, C. P., & Fiorot, S. (2023) · Research in Higher Education · doi:10.1007/s11162-023-09743-w · Cited by 7 · see citing works
Sex‐specific Differential Prediction of Academic Achievement by G erman Ability Tests
Schult, J., Hell, B., Päßler, K., & Schuler, H. (2013) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12023 · Cited by 7 · see citing works
How Useful Are Traditional Admission Measures in Predicting Graduation Within Four Years
Wyatt, J. N., Mattern, K. D., & Patterson, B. F. (2013) · Cited by 7 · see citing works
Thinking at work: Intelligence, critical thinking, job knowledge, and reasoning.
Kuncel, N. R., & Beatty, A. S. (2013) · American Psychological Association eBooks · doi:10.1037/14047-024 · Cited by 7 · see citing works
Data Mining the University: College GPA Predictions from SAT ScoresPreprint
Hsu, S. D. H., & Schombert, J. M. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1589792 · Cited by 7 · see citing works
Gong, T., Li, J., Yeung, J. Y., & Zhang, X. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2310151 · Cited by 6 · see citing works
A validation review of the SAT and ACT for college and university admissions decisions
Amrein‐Beardsley, A., Azizova, Z. T., Gibbs, N. P., Ikegwuonu, E., Kim, J., Torre, D. L., Lavery, M. R., Pivovarova, M., & Zheng, Y. (2025) · Education Policy Analysis Archives · doi:10.14507/epaa.33.8734 · Cited by 6 · see citing works
Tests of Scholastic Aptitude Cover Reasoning Facets Sufficiently
Schult, J., Fischer, F. T., & Hell, B. (2015) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000247 · Cited by 6 · see citing works
Napovedna veljavnost slovenske mature za študijsko uspešnost v humanistiki in družboslovju
Sočan, G., Krebl, M., Špeh, A., & Kutin, A. (2016) · Psihološka obzorja · doi:10.20419/2016.25.449 · Cited by 4 · see citing works
Becker, H. (2014) · Journal of International Crisis and Risk Communication Research · Cited by 4 · see citing works
Revisiting predictive bias of cognitive ability tests against Hispanic American job applicants
Berry, C. M., Zhao, P., Batarse, J. C., & Reddock, C. M. (2019) · Personnel Psychology · doi:10.1111/peps.12378 · Cited by 3 · see citing works
Self-citationDear admission committee…: Which moves in application essays predict student master grades?
de Boer, T., van Rijnsoever, F. J., & de Bresser, H. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304394 · Cited by 2 · see citing works
Effects of curricular emphasis in college on the GRE and its impact on the gender gap in performance
Tock, J. L., & Ericsson, K. A. (2018) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2018.11.003 · Cited by 2 · see citing works
Greco, A. M., Guilera, G., Maldonado-Murciano, L., Gómez‐Benito, J., & Barrios, M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19159402 · Cited by 2 · see citing works
Data Mining Functions in Advanced Education
Trivedi, N., Ahuja, S., Gupta, J. K., Mittal, V., & Jain, D. (2013) · Cited by 2 · see citing works
Saved by the biodata: Meta-analytic relationships between biodata scores and student success
Speer, A. B., Tenbrink, A. P., Wegmeyer, L. J., & Sendra, C. C. (2023) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2023.103863 · Cited by 1 · see citing works
Wittman, D. (2024) · Educational Measurement: Issues and Practice · doi:10.1111/emip.12598 · Cited by 1 · see citing works
Sackett, P. R., Sharpe, M. S., & Kuncel, N. R. (2021) · Applied Measurement in Education · doi:10.1080/08957347.2021.1987903 · Cited by 1 · see citing works
Self-citationThe SAT as a Predictor of College Success: Evidence from a Selective University
Rask, K. N., & Tiefenthaler, J. (2009) · Cited by 1 · see citing works
Validity Decay in STEM and Non-STEM Fields of Study
Westrick, P. A. (2014) · Cited by 1 · see citing works
Estimating Incremental Validity Under Missing Data
Fife, D. A., Mendoza, J. L., & Berry, C. M. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2016.1259099 · Cited by 1 · see citing works
Self-citationWartość prognostyczna wyników rekrutacyjnych dla powodzenia w studiowaniu
Zając, T. (2016) · Jagiellonian Digital Library (Jagiellonian University) · Cited by 1 · see citing works
Nelson, C. D. (2015) · Digital Commons - University of South Florida (University of South Florida) · Cited by 1 · see citing works
Steinert, J. K. (2011) · doi:10.25148/etd.fi11041505 · Cited by 1 · see citing works
Beretti, M. A. (2010) · Open University of Cape Town (University of Cape Town) · Cited by 1 · see citing works
An Omitted Variables Approach to Disentangling Differential Prediction in College Admissions
Demeke, S., Sackett, P. R., Kuncel, N. R., & Lee, S. D. (2026) · Educational Measurement: Issues and Practice · doi:10.1111/emip.70048
Self-citationAllgemeine Zutrittsberechtigung mit der Matura versus spezifische Zugangsprüfungen
Trost, G. (2014) · Springer eBooks · doi:10.1007/978-3-658-06090-9_9
Expanding Merit Through Communitarianism: Racial and Ethnic Implications
Griffin, J. D. (2016) · eScholarship (California Digital Library)
Larson, E. (2019) · CUNY Academic Works (City University of New York)
STUDENT-ATHLETE EDUCATIONAL OUTCOMES: A RETROSPECTIVE EVALUATION
Dressler, C. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/r701-v466
Holmes, J. D. (2016) · Wiley eBooks · doi:10.1002/9781118760499.ch15
College Entrance Exams: A Modern “Shibboleth”?
Robinson, D. H., & Wainer, H. (2022) · Significance · doi:10.1111/1740-9713.01650
A competency-based digital medical education model in conflict-affected settings
Qayumi, K. A., Omar, F. A., Noora, M., Hosseini, S. T. M., Hosseini, M. M., Naebkhilh, A., Aziz, S., Mansurih, P., Waqef, S., Oryakhil, N., et al. (2026) · EClinicalMedicine · doi:10.1016/j.eclinm.2026.104011
- Authors
- Berry, C. M., & Sackett, P. R.
- Affiliations
- Wayne State University (US)
- University of Minnesota (US)
- Venue
- Psychological Science
- Publisher
- SAGE Publications
- Published
- 2009-07
- Volume
- 20
- Issue
- 7
- Pages
- 822-830
- ISSN
- 0956-7976 (ISSN-L)
- 1467-9280
- OpenAlex
- W2093150504
- Cited by
- 68
- References
- 21
- Citations by year
- 2026: 1 · 2024: 3 · 2023: 3 · 2022: 5 · 2021: 1 · 2020: 1 · 2019: 6 · 2018: 4 · 2016: 8 · 2015: 6 · 2014: 5 · 2013: 6 · 2012: 5
- FWCI
- 26.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Higher Education Research Studies
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Medical Education and Admissions
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- course selection, individual differences, criterion validity, SAT scores, high school grade point average, college grades, university admission, student performance prediction, validity evidence, course grades
- MeSH terms
- Career Choice, Choice Behavior, College Admission Test, Curriculum, Educational Status, Humans, School Admission Criteria, Students, Universities, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
