Exploring upper limits of reliability and validity in job performance ratings.
Abstract
Explored possible upper bound limits in reliability and validity of performance ratings by creating a nearly ideal environment for obtaining performance evaluations and then examining closely the precision of these ratings. 16 scripts describing persons performing on 2 jobs–recruiting interviewer and manager—were prepared in such a way that the performers' effectiveness on various dimensions of performance approached a preset, realistic level. Five- to 7-min performances of these scripts were videotaped, and 14 raters evaluated the effectiveness of each performer on each performance dimension. Raters were very knowledgeable about the 2 jobs, had ample opportunity to observe the performers, and used a relatively advanced behavior scale type of rating form. Thus, the opportunity for obtaining highly reliable and valid ratings was excellent. Levels of convergent and discriminant validities were high compared to other studies, but interrater agreement was far from perfect. An analysis of the performance rating process reveals the possible reasons for error in ratings, and research approaches to learn more about this process are discussed.
Cited by (12)
- Estimating within-group interrater reliability with and without response biasJames, Lawrence R., Demaree, Robert G., Wolf, Gerrit · 1984 · Journal of Applied Psychology
- Beyond attribution theory: Cognitive processes in performance appraisal.Jack M. Feldman · 1981 · Journal of Applied Psychology
- COMPETITIVE PRIORITIES: INVESTIGATING THE NEED FOR TRADE‐OFFS IN OPERATIONS STRATEGYBOYER, KENNETH K., LEWIS, MARIANNE W. · 2002 · Production and Operations Management
- SOCIAL CONTEXT OF PERFORMANCE EVALUATION DECISIONS.Judge, T. A., Ferris, G. R. · 1993 · Academy of Management Journal
- KEEN TO HELP? MANAGERS' IMPLICIT PERSON THEORIES AND THEIR SUBSEQUENT EMPLOYEE COACHINGHESLIN, PETER A., VANDEWALLE, DON, LATHAM, GARY P. · 2006 · Personnel Psychology
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Exploring upper limits of reliability and validity in job performance ratings. https://app.cochat.ai/papers/10.1037//0021-9010.63.2.135
- The paper
- Borman, W. C. (1978). Exploring upper limits of reliability and validity in job performance ratings. Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.63.2.135
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.