Are performance appraisal ratings from different rating sources comparable?
Abstract
The purpose of this study was to test whether a multisource performance appraisal instrument exhibited measurement invariance across different groups of raters. Multiple-groups confirmatory factor analysis as well as item response theory (IRT) techniques were used to test for invariance of the rating instrument across self, peer, supervisor, and subordinate raters. The results of the confirmatory factor analysis indicated that the rating instrument was invariant across these rater groups. The IRT analysis yielded some evidence of differential item and test functioning, but it was limited to the effects of just 3 items and was trivial in magnitude. Taken together, the results suggest that the rating instrument could be regarded as invariant across the rater groups, thus supporting the practice of directly comparing their ratings. Implications for research and practice are discussed, as well as for understanding the meaning of between-source rating discrepancies.
Cites (2)
- Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance.Byrne, Barbara M., Shavelson, Richard J., Muthén, Bengt · 1989 · Psychological Bulletin
- On the Meaning of within-Factor Correlated Measurement ErrorsGerbing, David W., Anderson, James C. · 1984 · Journal of Consumer Research
Cited by (14)
- Power and sensitivity of alternative fit indices in tests of measurement invariance.Meade, Adam W., Johnson, Emily C., Braddy, Phillip W. · 2008 · Journal of Applied Psychology
- A competence‐based and multidimensional operationalization and measurement of employabilityHeijde, Claudia M. Van Der, Van Der Heijden, Beatrice I. J. M. · 2006 · Human Resource Management
- 12 Structural Equation Modeling in Management Research: A Guide for Improved AnalysisLarry J. Williams, Robert J. Vandenberg, Jeffrey R. Edwards · 2009 · Academy of Management Annals
- The Social Context of Performance Appraisal: A Review and Framework for the FutureLevy, Paul E., Williams, Jane R. · 2004 · Journal of Management
- DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL, META‐ANALYSIS, AND REVIEW OF EMPIRICAL FINDINGSSMITHER, JAMES W., LONDON, MANUEL, REILLY, RICHARD R. · 2005 · Personnel Psychology
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Are performance appraisal ratings from different rating sources comparable? https://app.cochat.ai/papers/10.1037//0021-9010.86.2.215
- The paper
- Facteau, J. D., & Craig, S. B. (2001). Are performance appraisal ratings from different rating sources comparable? Journal of Applied Psychology. https://doi.org/10.1037//0021-9010.86.2.215
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.