Format and training effects on rating accuracy and rater errors.

Borman, Walter C.·1979·Journal of Applied Psychology

Abstract

Examined the effects of rater training and rating format on halo and accuracy in performance ratings. A training program previously found to reduce the number of rating errors was adapted for use here, and 5 rating formats that have shown promise in helping to reduce rating errors were developed for 2 jobs. As in previous research, 5- to 9-min videotapes of actors performing on these 2 jobs according to scripts depicting realistic levels of effectiveness on the various performance dimensions provided the ratees for Ss to evaluate. 123 college students were assigned randomly to 1 of 10 cells defined by 5 format and 2 training treatments (trained vs untrained), and raters evaluated the effectiveness of the videotaped ratees' performances. Results show that training significantly reduced halo, but this finding was uneven across the 2 jobs. Training did not improve accuracy. Significant format effects were found for both halo and accuracy, but a significant job by format effect on accuracy suggested that no one format is consistently best across situations. Possible training strategies to enhance rating reliability and accuracy are discussed.

Cited by (13)

  1. Estimating within-group interrater reliability with and without response bias
    James, Lawrence R., Demaree, Robert G., Wolf, Gerrit · 1984 · Journal of Applied Psychology
  2. On the accuracy of personality judgment: A realistic approach
    Funder, David C. · 1995 · Psychological Review
  3. Errors and mistakes: Evaluating the accuracy of social judgment.
    Funder, David C. · 1987 · Psychological Bulletin
  4. Item Parceling Strategies in SEM: Investigating the Subtle Effects of Unmodeled Secondary Constructs
    Hall, Rosalie J., Snell, Andrea F., Foust, Michelle Singer · 1999 · Organizational Research Methods
  5. Performance appraisal and performance management: 100 years of progress?
    DeNisi, Angelo S., Murphy, Kevin R. · 2017 · Journal of Applied Psychology

See all 245

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Format and training effects on rating accuracy and rater errors. https://app.cochat.ai/papers/10.1037//0021-9010.64.4.410
The paper
Borman, W. C. (1979). Format and training effects on rating accuracy and rater errors. Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.64.4.410

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.