Effect of rater race on conclusions regarding differential prediction in cognitive ability tests.
Abstract
The possibility of predictive bias by race in employment tests is commonly examined by across-group comparisons of the slopes and intercepts of regression lines using test scores to predict performance measures. This research assumed that the criteria, primarily supervisory ratings, were unbiased. However, a concern is that the apparent lack of differential prediction in cognitive ability tests may be an artifact of the predominant use of performance ratings provided by supervisors who are members of the majority group; a criterion that is potentially biased against members of the minority group. We posited that ratings by a supervisor of the same race as the employee being rated would be less open to claims of bias. We compared ability-performance relationships in samples of Black and White employees that allowed for between-subjects and within-subjects comparisons under 2 conditions: when all employees were rated by a White supervisor and when each employee was rated by a supervisor of the same race. Neither analysis found evidence of predictive bias against Black employees.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some papers found no predictive bias but found tests were less valid for Black applicants.
“found no predictive bias but found tests were less valid for Black applicants (Rotundo & Sackett, 1999)”
From Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review (2025) · cites this paper as Rotundo & Sackett, 1999
Cites
CONSIDERATIONS FOR STUDIES OF TEST BIAS
LINN, R. L., & WERTS, C. E. (1971) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1971.tb00898.x · Cited by 113 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Organizational responsibility: Doing good and doing well.
Aguinis, H. (2010) · American Psychological Association eBooks · doi:10.1037/12171-024 · Cited by 661 · see citing works
ROTH, P. L., BEVIER, C. A., BOBKO, P., SWITZER, F. S., & TYLER, P. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00094.x · Cited by 325 · see citing works
General Mental Ability and Selection in Private Sector Organizations: A Commentary
Gottfredson, L. (2002) · doi:10.1207/s15327043hup1501 · Cited by 315
Ethnic group differences in measures of job performance: a new meta-analysis.
Roth, P. L., Huffcutt, A. I., & Bobko, P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.4.694 · Cited by 171 · see citing works
“We chose not to include the data from two studies of the U.S. Employment Service database analyzing the GATB (Rotundo & Sackett, 1999; Sackett & DuBois, 1991) because we had already included a similar database in which the vast majority of the same subjects were analyzed (Waldman & Avolio, 1991).”
BackgroundMethodInfluentialAN EXAMINATION OF WHETHER AND HOW RACIAL AND GENDER BIASES INFLUENCE CUSTOMER SATISFACTION
Hekman, D. R., Aquino, K. F., Owens, B. P., Mitchell, T. R., Schilpzand, P., & Leavitt, K. (2010) · Academy of Management Journal · doi:10.5465/amj.2010.49388763 · Cited by 172 · see citing works
“This feature of our data represents a methodological improvement over studies that only measure employee performance with a single subjective rating and are therefore unable to determine if the rating is biased (Rotundo & Sackett, 1999).”
BackgroundThe Existence and Nature of Racial Bias in Supervisory Ratings.
Stauffer, J. M., & Buckley, M. R. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.3.586 · Cited by 127 · see citing works
“For example, despite the fact that they found significant rater–ratee race effects (which normally would serve to indicate bias), Rotundo and Sackett (1999) dismissed these effects as evidence of bias because of the following: It is not clear whether a rater–ratee race effect reflects inflated…”
BackgroundInfluentialARE HIGHLY STRUCTURED JOB INTERVIEWS RESISTANT TO DEMOGRAPHIC SIMILARITY EFFECTS
McCarthy, J., Iddekinge, C. H. V., & Campion, M. A. (2010) · Personnel Psychology · doi:10.1111/j.1744-6570.2010.01172.x · Cited by 112 · see citing works
VAN IDDEKINGE, C. H., & PLOYHART, R. E. (2008) · Personnel Psychology · doi:10.1111/j.1744-6570.2008.00133.x · Cited by 101 · see citing works
Berry, C. M., Clark, M. A., & McClure, T. K. (2011) · Journal of Applied Psychology · doi:10.1037/a0023222 · Cited by 99 · see citing works
Revival of test bias research in preemployment testing.
Aguinis, H., Culpepper, S. A., & Pierce, C. A. (2010) · Journal of Applied Psychology · doi:10.1037/a0018714 · Cited by 107 · see citing works
“As an example, consider a study by Rotundo and Sackett (1999). These authors conducted differential prediction analyses of data collected by the U.”
BackgroundAguinis, H., & Smith, M. A. (2007) · Personnel Psychology · doi:10.1111/j.1744-6570.2007.00069.x · Cited by 73 · see citing works
The Role of Cognitive Ability Tests in Employment Selection
Outtz, J. L. (2002) · Human Performance · doi:10.1080/08959285.2002.9668089 · Cited by 69 · see citing works
“The database included test scores, performance ratings, and demographic data on 36,614 individuals in 171 different jobs representing 653 different employers (Rotundo & Sackett, 1999).”
BackgroundMethodViswesvaran, C., & Ones, D. (2002) · Human Performance · doi:10.1080/08959285.2002.9668092 · Cited by 56 · see citing works
“If this were not the case, it would actually be an indication of differential prediction and test bias (Rotundo & Sackett, 1999).”
BackgroundA comparison of MMPI-2 validity in African American and Caucasian psychiatric inpatients.
Arbisi, P. A., Ben-Porath, Y., & McNulty, J. L. (2002) · Psychological Assessment · doi:10.1037/1040-3590.14.1.3 · Cited by 54
Berry, C. M. (2015) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-032414-111256 · Cited by 66 · see citing works
Sauer, S. J., Thomas-Hunt, M. C., & Morris, P. A. (2010) · Organ. Sci. · doi:10.1287/orsc.1090.0523 · Cited by 41 · see citing works
Meade, A. W., & Fetzer, M. (2009) · Organizational Research Methods · doi:10.1177/1094428109331487 · Cited by 48 · see citing works
“Ostensibly, objective measures of performance should be less prone to potential stereotyping and racial bias than are subjective performance ratings (Ford, Kraiger, & Schechtman, 1986; Rotundo & Sackett, 1999; Schmidt & Hunter, 1981).”
BackgroundBalboni, G., Naglieri, J. A., & Cubelli, R. (2009) · Journal of Psychoeducational Assessment · doi:10.1177/0734282909343763 · Cited by 32 · see citing works
Wilson, F. (2002) · doi:10.1016/s0263-2373(02)00113-5 · Cited by 35
Monnot, M. J., Quirk, S. W., Hoerger, M. P., & Brewer, L. (2009) · Psychological Assessment · doi:10.1037/a0015316 · Cited by 25 · see citing works
“A step-down hierarchical regression procedure (Lautenschlager & Mendoza, 1986) has been used previously to identify racial bias in clinical predictions (Arbisi et al., 2002) and employment settings (Rotundo & Sackett, 1999).”
MethodPsychometric Accuracy and (the Continuing Need for) Quality Thinking in Meta-Analysis
Bobko, P., & Roth, P. L. (2007) · Organizational Research Methods · doi:10.1177/1094428107303155 · Cited by 26 · see citing works
“For example, Waldman and Avolio (1991) appear to have used GATB data from 1970 to 1984, whereas Rotundo and Sackett (1999) appear to have used GATB data from 1972 to 1987.”
MethodCognitive Ability in Personnel Selection Decisions
Öneş, D. S., Viswesvaran, C., & Dilchert, S. (2017) · Wiley eBooks · doi:10.1002/9781405164221.ch7 · Cited by 42 · see citing works
Implications of research staff demographics for psychological science.
Does, S., Ellemers, N., Dovidio, J. F., Norman, J. B., Mentovich, A., van der Lee, R., & Goff, P. A. (2018) · American Psychologist · doi:10.1037/amp0000199 · Cited by 28 · see citing works
An Examination of Gender-Moderated Test Bias on the Levenson Self-Report Psychopathy Scale
Marion, B. E., & Sellbom, M. (2011) · Journal of Personality Assessment · doi:10.1080/00223891.2011.558873 · Cited by 26
“This modified multiple regression procedure has been used by Rotundo and Sackett (1999) to investigate racial bias in employment tests, and by Arbisi et al (2002) to assess ethnic test bias in the MMPI–2.”
MethodDahlke, J. A., & Sackett, P. R. (2017) · Journal of Applied Psychology · doi:10.1037/apl0000234 · Cited by 28 · see citing works
“Consistent with previous research using these data (e.g., Rotundo & Sackett, 1999), the occupational categories in our data set include those with meaningful numbers of both Black and White employees.”
MethodSelf-citationSellbom, M. (2015) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2015.1048392 · Cited by 24 · see citing works
“…hierarchical regression procedure was used to examine not only classic moderation (i.e., significant differences in slope) but also systematic intercept differences in prediction of PTSD symptoms (see Lautenschlager & Mendoza, 1986; Rotundo & Sackett, 1999, for specific details of this procedure).”
MethodThe Effects of Team Leader Race on Performance Evaluations
Ellis, A. P. J., Ilgen, D. R., & Hollenbeck, J. (2006) · Small Group Research · doi:10.1177/1046496406288978 · Cited by 20 · see citing works
Criterion‐Related Validity of Dutch Police‐Selection Measures and Differences Between Ethnic Groups
De Meijer, L. A. L., Born, M. P., Terlouw, G., & Van Der Molen, H. T. (2008) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2008.00438.x · Cited by 18 · see citing works
“Most research in this area has been conduced in North America (e.g., Hunter, Schmidt, & Hunter, 1979; Rotundo & Sackett, 1999 ) and has used cognitive ability tests as predictors.”
MethodNew Effect Sizes for Tests of Categorical Moderation and Differential Prediction
Nye, C. D., & Sackett, P. (2017) · Organizational Research Methods · doi:10.1177/1094428116644505 · Cited by 8 · see citing works
“However, consistent with past research using this data set (Rotundo & Sackett, 1999), we used only a subset of the data for our analyses.”
MethodSelf-citationÖneş, D. S., & Viswesvaran, C. (2002) · Human Performance · doi:10.1080/08959285.2002.9668080 · Cited by 7 · see citing works
Moderation of selection procedure validity by employee race
Gardner, D. G., & Deadrick, D. L. (2012) · Journal of Managerial Psychology · doi:10.1108/02683941211220180 · Cited by 6 · see citing works
“In a differential validity study in which the researchers concluded that there were no slope differences (Rotundo and Sackett, 1999), Aguinis et al.”
BackgroundMind the Gap: Between-Group Differences and Fair Test Use
Gamliel, E., & Cahan, S. (2007) · doi:10.1111/j.1468-2389.2007.00387.x · Cited by 5
Reflections on the Journal of Applied Psychology for 1997 to 2002.
Murphy, K. R. (2017) · Journal of Applied Psychology · doi:10.1037/apl0000054 · Cited by 3 · see citing works
“There were also important articles dealing with the nature and influence of contextual performance (e.g., Conway, 1999; Rotundo & Sackett, 1999).”
BackgroundEVALUATING EMPIRICAL RELATIONSHIPS AMONG PREDICTION, MEASUREMENT, AND SCALING INVARIANCE
Moses, T. P. (2011) · ETS Research Report Series · doi:10.1002/j.2333-8504.2011.tb02242.x · Cited by 2 · see citing works
“Dorans, N. J., & Holland, P. W. (2000).”
InfluentialRevisiting predictive bias of cognitive ability tests against Hispanic American job applicants
Berry, C. M., Zhao, P., Batarse, J. C., & Reddock, C. M. (2019) · Personnel Psychology · doi:10.1111/peps.12378 · Cited by 3 · see citing works
Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review
Cuppello, S., Zibarras, L. D., & Corr, P. J. (2025) · Trends in Psychology · doi:10.1007/s43076-025-00504-5
Berry, C. M., Sackett, P. R., & Sund, A. E. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-012-9284-3 · Cited by 11 · see citing works
Self-citationReconsidering the search for alternatives to general mental ability tests
Cucina, J. M. (2025) · Intelligence · doi:10.1016/j.intell.2024.101892
- Authors
- Rotundo, M., & Sackett, P. R.
- Affiliations
- University of Minnesota (US)
- ORCID
- Sackett, Paul R.
- Venue
- Journal of Applied Psychology
- Published
- 1999-10
- Volume
- 84
- Issue
- 5
- Pages
- 815-822
- ISSN
- 0021-9010 (ISSN-L)
- 1939-1854
- OpenAlex
- W2002449559
- Cited by
- 45
- References
- 0
- Citations by year
- 2025: 3 · 2020: 1 · 2019: 1 · 2018: 1 · 2017: 4 · 2016: 1 · 2015: 5 · 2012: 3
- FWCI
- 3.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Applied Psychology (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Keywords
- predictive bias, differential prediction, cognitive ability tests, employment testing, supervisor ratings, racial bias, Black workers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
