Are performance appraisal ratings from different rating sources comparable?

Facteau, Jeffrey D., Craig, S. Bartholomew·2001·Journal of Applied Psychology

Abstract

The purpose of this study was to test whether a multisource performance appraisal instrument exhibited measurement invariance across different groups of raters. Multiple-groups confirmatory factor analysis as well as item response theory (IRT) techniques were used to test for invariance of the rating instrument across self, peer, supervisor, and subordinate raters. The results of the confirmatory factor analysis indicated that the rating instrument was invariant across these rater groups. The IRT analysis yielded some evidence of differential item and test functioning, but it was limited to the effects of just 3 items and was trivial in magnitude. Taken together, the results suggest that the rating instrument could be regarded as invariant across the rater groups, thus supporting the practice of directly comparing their ratings. Implications for research and practice are discussed, as well as for understanding the meaning of between-source rating discrepancies.

Cites (2)

  1. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance.
    Byrne, Barbara M., Shavelson, Richard J., Muthén, Bengt · 1989 · Psychological Bulletin
  2. On the Meaning of within-Factor Correlated Measurement Errors
    Gerbing, David W., Anderson, James C. · 1984 · Journal of Consumer Research

Show all 2 references

Cited by (14)

  1. Power and sensitivity of alternative fit indices in tests of measurement invariance.
    Meade, Adam W., Johnson, Emily C., Braddy, Phillip W. · 2008 · Journal of Applied Psychology
  2. A competence‐based and multidimensional operationalization and measurement of employability
    Heijde, Claudia M. Van Der, Van Der Heijden, Beatrice I. J. M. · 2006 · Human Resource Management
  3. 12 Structural Equation Modeling in Management Research: A Guide for Improved Analysis
    Larry J. Williams, Robert J. Vandenberg, Jeffrey R. Edwards · 2009 · Academy of Management Annals
  4. The Social Context of Performance Appraisal: A Review and Framework for the Future
    Levy, Paul E., Williams, Jane R. · 2004 · Journal of Management
  5. DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL, META‐ANALYSIS, AND REVIEW OF EMPIRICAL FINDINGS
    SMITHER, JAMES W., LONDON, MANUEL, REILLY, RICHARD R. · 2005 · Personnel Psychology

See all 169

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Are performance appraisal ratings from different rating sources comparable? https://app.cochat.ai/papers/10.1037//0021-9010.86.2.215
The paper
Facteau, J. D., & Craig, S. B. (2001). Are performance appraisal ratings from different rating sources comparable? Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.86.2.215

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.