Potential sources of criterion bias in supervisor ratings used for test validation
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Lefkowitz and Battista (1995) found that supervisors gave higher performance ratings to same-ethnicity staff.
“Lefkowitz and Battista (1995) found that supervisors gave higher performance ratings to same-ethnicity staff”
From Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review (2025) · cites this paper as Lefkowitz and Battista (1995)
When controlling for same-ethnicity supervisor ratings, the criterion validity of tests reduced.
“and when controlling for this, the criterion validity of tests reduced.”
From Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review (2025) · cites this paper as Lefkowitz and Battista (1995)
Cites
Research in Organizational Behavior
Hammer, E. R., Cummings, L. L., & Staw, B. M. (1984) · Academy of Management Review · doi:10.2307/258300 · Cited by 11,176 · see citing works
Feeling and thinking: Preferences need no inferences.
Zajonc, R. B. (1980) · American Psychologist · doi:10.1037/0003-066x.35.2.151 · Cited by 8,094 · see citing works
Tests for comparing elements of a correlation matrix
Steiger, J. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.87.2.245 · Cited by 4,621 · see citing works
A vertical dyad linkage approach to leadership within formal organizations
Dansereau, F., Graen, G., & Haga, W. J. (1975) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(75)90005-7 · Cited by 2,890 · see citing works
EFFECTS OF RACE ON ORGANIZATIONAL EXPERIENCE, JOB PERFORMANCE EVALUATIONS, AND CAREER OUTCOMES.
Greenhaus, J. H., Parasuraman, S., & Wormley, W. M. (1990) · Academy of Management Journal · doi:10.2307/256352 · Cited by 2,735 · see citing works
Merton, R. K. (1948) · The Antioch Review · doi:10.2307/4609267 · Cited by 2,145 · see citing works
Validity and utility of alternative predictors of job performance.
Hunter, J. E., & Hunter, R. F. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.96.1.72 · Cited by 2,107 · see citing works
Blaney, P. H. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.2.229 · Cited by 1,739 · see citing works
Experimenter Effects in Behavioral Research
McGuigan, F. J., & Rosenthal, R. (1968) · The American Journal of Psychology · doi:10.2307/1420816 · Cited by 1,639 · see citing works
Personal Networks of Women and Minorities in Management: A Conceptual Framework
Ibarra, H. (1993) · Academy of Management Review · doi:10.5465/amr.1993.3997507 · Cited by 1,630 · see citing works
水口, 礼. (2017) · The SAGE Encyclopedia of Industrial and Organizational Psychology, 2nd edition · doi:10.4135/9781483386874.n283 · Cited by 1,455 · see citing works
Positive Affect, Cognitive Processes, and Social Behavior
Isen, A. M. (1987) · Advances in experimental social psychology · doi:10.1016/s0065-2601(08)60415-3 · Cited by 1,443 · see citing works
Merton, R. K. (2016) · The Antioch Review · doi:10.7723/antiochreview.74.3.0504 · Cited by 1,371 · see citing works
Experimenter Effects in Behavioral Research.
Titchener, J. L. (1967) · Archives of Internal Medicine · doi:10.1001/archinte.1967.00300050109027 · Cited by 1,207 · see citing works
Putting gender into context: An interactive model of gender-related behavior.
Deaux, K., & Major, B. (1987) · Psychological Review · doi:10.1037/0033-295x.94.3.369 · Cited by 1,173 · see citing works
The influence of affect on categorization.
Isen, A. M., & Daubman, K. A. (1984) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.47.6.1206 · Cited by 1,112 · see citing works
Test validity and the ethics of assessment.
Messick, S. (1980) · American Psychologist · doi:10.1037/0003-066x.35.11.1012 · Cited by 922 · see citing works
Beyond attribution theory: Cognitive processes in performance appraisal.
Feldman, J. M. (1981) · Journal of Applied Psychology · doi:10.1037/0021-9010.66.2.127 · Cited by 871 · see citing works
Wayne, S. J., & Ferris, G. R. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.5.487 · Cited by 860 · see citing works
Landy, F. J., & Farr, J. L. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.87.1.72 · Cited by 805 · see citing works
The influence of positive affect on the unusualness of word associations.
Isen, A. M., Johnson, M. M. S., Mertz, E., & Robinson, G. F. (1985) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.48.6.1413 · Cited by 802 · see citing works
Schmidt, F. L., Hunter, J. E., & Outerbridge, A. N. (1986) · Journal of Applied Psychology · doi:10.1037/0021-9010.71.3.432 · Cited by 758 · see citing works
Generalizing From Laboratory to Field Settings.
Saxton, S. L., & Locke, E. A. (1987) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2069857 · Cited by 753 · see citing works
From individual differences to social categories: Analysis of a decade's research on gender.
Deaux, K. (1984) · American Psychologist · doi:10.1037/0003-066x.39.2.105 · Cited by 653 · see citing works
Schmitt, N., Gooding, R. Z., Noe, R. A., & Kirsch, M. P. (1984) · Personnel Psychology · doi:10.1111/j.1744-6570.1984.tb00519.x · Cited by 554 · see citing works
Supervisor-subordinate similarity: Types, effects, and mechanisms.
Turban, D. B., & Jones, A. P. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.2.228 · Cited by 503 · see citing works
A cognitive view of the performance appraisal process: A model and research propositions
DeNisi, A. S., Cafferty, T. P., & Meglino, B. M. (1984) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(84)90029-1 · Cited by 444 · see citing works
Journal of Organizational Behavior
(2007) · Journal of Organizational Behavior · doi:10.1002/job.467 · Cited by 431 · see citing works
The criterion problem: 1917-1992.
Austin, J. T., & Villanova, P. D. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.6.836 · Cited by 403 · see citing works
A meta-analysis of ratee race effects in performance ratings.
Kraiger, K., & Ford, J. K. (1985) · Journal of Applied Psychology · doi:10.1037/0021-9010.70.1.56 · Cited by 376 · see citing works
Handbook of methods for detecting test bias
Berk, R. A. (1982) · Johns Hopkins University Press eBooks · Cited by 367 · see citing works
Performance Measurement and Theory
(2017) · Routledge eBooks · doi:10.4324/9781315211947 · Cited by 349 · see citing works
Sackett, P. R., DuBois, C. L., & Noe, A. W. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.2.263 · Cited by 287 · see citing works
(1989) · Choice Reviews Online · doi:10.5860/choice.27-1601 · Cited by 271 · see citing works
Pulakos, E. D., & Wexley, K. N. (1983) · Academy of Management Journal · doi:10.2307/256139 · Cited by 242 · see citing works
Heilman, M. E., & Stopeck, M. H. (1985) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(85)90035-4 · Cited by 236 · see citing works
Examination of race and sex effects on performance ratings.
Pulakos, E. D., White, L. A., Oppler, S. H., & Borman, W. C. (1989) · Journal of Applied Psychology · doi:10.1037//0021-9010.74.5.770 · Cited by 211 · see citing works
Some Neglected Variables in Research on Discrimination in Appraisals
Dipboye, R. L. (1985) · Academy of Management Review · doi:10.5465/amr.1985.4277365 · Cited by 205 · see citing works
Bazerman, M. H., Beekun, R. I., & Schoorman, F. D. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.6.873 · Cited by 166 · see citing works
Schoorman, F. D. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.1.58 · Cited by 152 · see citing works
Performance appraisal decisions as a function of rater training and purpose of the appraisal.
Zedeck, S., & Cascio, W. F. (1982) · Journal of Applied Psychology · doi:10.1037//0021-9010.67.6.752 · Cited by 140 · see citing works
Rater-ratee race effects on performance evaluation: Challenging meta-analytic conclusions.
Sackett, P. R., & DuBois, C. L. Z. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.6.873 · Cited by 133 · see citing works
Effects of previous performance on evaluations of present performance.
Murphy, K. R., Balzer, W. K., Lockhart, M. C., & Eisenman, E. J. (1985) · Journal of Applied Psychology · doi:10.1037//0021-9010.70.1.72 · Cited by 122 · see citing works
Race effects in performance evaluations: Controlling for ability, education, and experience.
Waldman, D., & Avolio, B. (1991) · Journal of Applied Psychology · doi:10.1037/0021-9010.76.6.897 · Cited by 106 · see citing works
EFFECTS OF GENDER ON SELF- AND SUPERVISORY RATINGS.
Shore, L. M., & Thornton, G. C. (1986) · Academy of Management Journal · doi:10.2307/255863 · Cited by 101 · see citing works
Dobbins, G. H., Cardy, R. L., & Truxillo, D. M. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.3.551 · Cited by 95 · see citing works
AN INVESTIGATION OF THE RATER-RATEE ACQUAINTANCE AND RATER BIAS.
Kingstorm, P. O., & Mainstone, L. E. (1985) · Academy of Management Journal · doi:10.2307/256119 · Cited by 85 · see citing works
The role of appraisal purpose: Effects of purpose on information acquisition and utilization
Williams, K. J., DeNisi, A. S., Blencoe, A. G., & Cafferty, T. P. (1985) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(85)90027-5 · Cited by 77 · see citing works
Dobbins, G. H., & Russell, J. M. (1986) · Personnel Psychology · doi:10.1111/j.1744-6570.1986.tb00593.x · Cited by 73 · see citing works
Smither, J. W., Reilly, R. R., & Buda, R. (1988) · Journal of Applied Psychology · doi:10.1037//0021-9010.73.3.487 · Cited by 71 · see citing works
Race and sex as determinants of the mean and variance of performance ratings.
Schmitt, N., & Lappin, M. (1980) · Journal of Applied Psychology · doi:10.1037//0021-9010.65.4.428 · Cited by 71 · see citing works
A Causal Analysis of Cognitive Ability, Job Knowledge, Job Performance, and Supervisor Ratings
Hunter, J. E. (2017) · Routledge eBooks · doi:10.4324/9781315211947-13 · Cited by 63 · see citing works
Effects of Prior Expectations on Performance Ratings: A Longitudinal Study
Hogan, E. A. (1987) · Academy of Management Journal · doi:10.5465/256279 · Cited by 61 · see citing works
Oppler, S. H., Campbell, J. P., Pulakos, E. D., & Borman, W. C. (1992) · Journal of Applied Psychology · doi:10.1037//0021-9010.77.2.201 · Cited by 59 · see citing works
Performance Rating Validity: The Relationship of Objective and Subjective Measures of Performance
Alexander, E. R., & Wilkins, R. D. (1982) · Group & Organization Studies · doi:10.1177/105960118200700410 · Cited by 55 · see citing works
EMPLOYMENT DISCRIMINATION: AN ORGANIZATIONAL MODEL.
Haberfeld, Y. (1992) · Academy of Management Journal · doi:10.2307/256476 · Cited by 53 · see citing works
RACE AS A FACTOR IN JOB PLACEMENT: SERENDIPITOUS FINDINGS OF “ETHNIC DRIFT”
Lefkowitz, J. M. (1994) · Personnel Psychology · doi:10.1111/j.1744-6570.1994.tb01734.x · Cited by 46 · see citing works
Self-citationSex Segregation and Women at Work: A Selective Review
Gutek, B. A. (1988) · Applied Psychology · doi:10.1111/j.1464-0597.1988.tb01130.x · Cited by 41 · see citing works
Changes in raters' perceptions of subordinates: A catastrophe model.
Hanges, P. J., Braverman, E. P., & Rentsch, J. R. (1991) · Journal of Applied Psychology · doi:10.1037//0021-9010.76.6.878 · Cited by 30 · see citing works
Principles for the validation and use of personnel selection procedures
(1975) · PsycEXTRA Dataset · doi:10.1037/e475802008-001 · Cited by 30 · see citing works
Employee performance evaluations: Note Effects of ratee age, rater age, and rater gender
Griffeth, R. W., & Bedeian, A. G. (1989) · Journal of Organizational Behavior · doi:10.1002/job.4030100107 · Cited by 27 · see citing works
McClelland, L. (1974) · Journal of Personality and Social Psychology · doi:10.1037/h0036008 · Cited by 20 · see citing works
Sex differences in performance rating: A research note
WILLIAMS, R. S., & Walker, J. W. (1985) · Journal of Occupational Psychology · doi:10.1111/j.2044-8325.1985.tb00206.x · Cited by 20 · see citing works
Lefkowitz, J. M., & Fraser, A. W. (1980) · Journal of Applied Psychology · doi:10.1037/0021-9010.65.6.685 · Cited by 19 · see citing works
Self-citationAn Investigation of the Rater-Ratee Acquaintance and Rater Bias
Kingstrom, P. O., & Mainstone, L. E. (1985) · Academy of Management Journal · doi:10.5465/256119 · Cited by 17 · see citing works
Check-reading as a function of pointer symmetry and uniform alignment.
Johnsgård, K. W. (1953) · Journal of Applied Psychology · doi:10.1037/h0055556 · Cited by 7 · see citing works
Sex Effects in Performance--Related Judgments
Northcraft, G. B., Huber, V. L., & Neale, M. A. (1988) · Human Performance · doi:10.1207/s15327043hup0103_2 · Cited by 5 · see citing works
Adjustments of Minority Group Test Scores Used in Employment Referrals
Hartigan, J. A. (1990) · CHANCE · doi:10.1080/09332480.1990.10554969 · Cited by 3 · see citing works
Flagged first, then by citations
2 of the 70 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Translating self-efficacy in job performance over time: The role of job crafting
Miraglia, M., Cenciotti, R., Alessandri, G., & Боргогни, Л. (2017) · Human Performance · doi:10.1080/08959285.2017.1373115 · Cited by 85 · see citing works
Varma, A., Pichler, S. M., & Srinivas, E. S. (2005) · The International Journal of Human Resource Management · doi:10.1080/09585190500314904 · Cited by 77 · see citing works
Workplace racial composition, perceived discrimination, and organizational attachment
Stainback, K., & Irvin, M. (2011) · Social Science Research · doi:10.1016/j.ssresearch.2011.11.016 · Cited by 67 · see citing works
Advances in the Science of Performance Appraisal: Implications for Practice
Latham, G. P., & Mann, S. L. (2006) · Wiley eBooks · doi:10.1002/9780470696378.ch7 · Cited by 66 · see citing works
Baltes, B. B., Bauer, C. C., & Frensch, P. A. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.1.151 · Cited by 49 · see citing works
RACE AS A FACTOR IN JOB PLACEMENT: SERENDIPITOUS FINDINGS OF “ETHNIC DRIFT”
Lefkowitz, J. M. (1994) · Personnel Psychology · doi:10.1111/j.1744-6570.1994.tb01734.x · Cited by 46 · see citing works
Self-citationAn analysis of bias in supervisor narrative comments in performance appraisal
Wilson, K. Y. (2010) · Human Relations · doi:10.1177/0018726710369396 · Cited by 41 · see citing works
A Meta-Analysis of the Relationship Between Rater Liking and Performance Ratings
Sutton, A. W., Baldwin, S., Wood, L. E., & Hoffman, B. J. (2013) · Human Performance · doi:10.1080/08959285.2013.836523 · Cited by 40 · see citing works
Liking and similarity as predictors of multi‐source ratings
Bates, R. A. (2002) · Personnel Review · doi:10.1108/00483480210438753 · Cited by 40 · see citing works
Gender differences in leadership competencies, expatriate readiness, and performance
Connerley, M. L., Mecham, R. L., & Strauss, J. P. (2008) · Gender in Management An International Journal · doi:10.1108/17542410810887347 · Cited by 37 · see citing works
Antecedents and outcomes of Hungarian nurses’ career adaptability
Pajic, S., Keszler, Á., Kismihók, G., Mol, S. T., & Hartog, D. N. D. (2018) · International Journal of Manpower · doi:10.1108/ijm-10-2018-0334 · Cited by 35 · see citing works
Tracking job performance trajectories over time: A six-year longitudinal study
Alessandri, G., Боргогни, Л., & Truxillo, D. M. (2014) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2014.949679 · Cited by 34 · see citing works
Performance disparities between whites and ethnic minorities: Real differences or assessment bias?
Dewberry, C. (2001) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317901167578 · Cited by 34 · see citing works
Xu, X. V., Peng, Y., Ma, J., & Jalil, D. (2023) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12441 · Cited by 17 · see citing works
Interpersonal Affect: Does It Really Bias Performance Appraisals?
Varma, A., & Pichler, S. M. (2007) · Journal of Labor Research · doi:10.1007/bf03380053 · Cited by 14 · see citing works
Meta-Analysis of Clerical Performance Predictors: Still stable after all these years
Whetzel, D. L., McCloy, R. A., Hooper, A. C., Russell, T. L., Waters, S. D., Campbell, W. J., & Ramos, R. A. (2011) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2010.00533.x · Cited by 12 · see citing works
RACE DIFFERENCES IN TERMINATION AT WORK: THE ROLE OF EDUCATIONAL INEQUALITY
Hargis, M. B., Baltes, B. B., Fried, Y., & Levi, A. S. (2005) · Journal of Business and Psychology · doi:10.1007/s10869-005-9001-6 · Cited by 9 · see citing works
Cenciotti, R., Alessandri, G., Боргогни, Л., & Consiglio, C. (2021) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12357 · Cited by 8 · see citing works
Park, H.-S., & Blenkinsopp, J. (2016) · International Review of Administrative Sciences · doi:10.1177/0020852315615197 · Cited by 6 · see citing works
Artificial or intelligent? Machine learning and medical selection: possibilities and risks
Tiffin, P. A., & Paton, L. W. (2018) · MedEdPublish · doi:10.15694/mep.2018.0000256.1 · Cited by 4 · see citing works
Cornish, T., Wilson, K. Y., & Jones, R. (2017) · doi:10.1057/978-1-137-33014-7_4 · Cited by 1 · see citing works
Grigg, K. (2017) · Figshare · doi:10.4225/03/58a65f877f3dd · Cited by 1 · see citing works
Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review
Cuppello, S., Zibarras, L. D., & Corr, P. J. (2025) · Trends in Psychology · doi:10.1007/s43076-025-00504-5
Siqueira, D. D., & Lucena, W. G. L. (2023) · Revista de Negócios · doi:10.7867/1980-4431.2022v27n2p50-68
- Authors
- Lefkowitz, J., & Battista, M.
- Affiliations
- Baruch College (US)
- ORCID
- Lefkowitz, Joel
- Venue
- Journal of Business and Psychology
- Published
- 1995-06
- Volume
- 9
- Issue
- 4
- Pages
- 389-414
- ISSN
- 0889-3268 (ISSN-L)
- 1573-353X
- OpenAlex
- W2057801357
- Cited by
- 25
- References
- 70
- Citations by year
- 2025: 1 · 2023: 2 · 2021: 1 · 2018: 2 · 2017: 3 · 2016: 1 · 2014: 1 · 2013: 1
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Journal of Business and Psychology (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Employer Branding and e-HRM
- Social Sciences › Psychology › Applied Psychology › Human Resource Development and Performance Evaluation
- Keywords
- supervisor ratings, performance evaluation (human resources), criterion validity, test validation, ethnic similarity, sex bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
