A meta-analysis of interrater and internal consistency reliability of selection interviews.

Abstract

A meta-analysis of 111 interrater reliability coefficients and 49 coefficient alphas from selection interviews was conducted. Moderators of interrater reliability included study design, interviewer training, and 3 dimensions of interview structure ( standardization of questions, of response evaluation, and of combining multiple ratings). Interactions showed that standardizing questions had a stronger moderating effect on reliability when coefficients were from separate ( rather than panel) interviews, and multiple ratings were useful when combined mechanically (there was no evidence of usefulness when combined subjectively). Average correlations (derived from alphas) between ratings were moderated by standardization of questions and number of ratings made. Upper limits of validity were estimated to be.67 for highly structured interviews and.34 for unstructured interviews.

Cites (25)

1. Review of Psychometric theory (2nd ed.).
   SITGREAVES, ROSEDITH · 1979 · Contemporary Psychology: A Journal of Reviews
2. What is coefficient alpha? An examination of theory and applications
   Cortina, Jose M. · 1993 · Journal of Applied Psychology
3. Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
   Schmidt, Frank L., Hunter, John E. · 2015
4. The robust beauty of improper linear models in decision making.
   Dawes, Robyn M. · 1979 · American Psychologist
5. Personnel Selection in Organizations
   Murphy, Kevin R., Schmitt, Neal, Borman, Walter C. · 1993 · The Academy of Management Review
6. The validity of employment interviews: A comprehensive review and meta-analysis.
   Michael A. McDaniel, Deborah L. Whetzel, Frank L. Schmidt, Steven D. Maurer · 1994 · Journal of Applied Psychology
7. Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs.
   Allen I. Huffcutt, Winfred Arthur · 1994 · Journal of Applied Psychology
8. A meta‐analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview*
   Wiesner, Willi H., Cronshaw, Steven F. · 1988 · Journal of Occupational Psychology
9. The situational interview.
   Latham, Gary P., Saari, Lise M., Pursell, Elliott D., Campion, Michael A. · 1980 · Journal of Applied Psychology
10. The problem of analyzing multiplicative composites: Interactions revisited.
    Evans, Martin G. · 1991 · American Psychologist

Show all 25 references

Cited by (11)

1. Information sharing and team performance: A meta-analysis
   Mesmer-Magnus, Jessica R., DeChurch, Leslie A. · 2009 · Journal of Applied Psychology
2. An other perspective on personality: Meta-analytic integration of observers' accuracy and predictive validity.
   Connelly, Brian S., Ones, Deniz S. · 2010 · Psychological Bulletin
3. Influence tactics and work outcomes: a meta analysis
   Higgins, Chad A., Judge, Timothy A., Ferris, Gerald R. · 2003 · Journal of Organizational Behavior
4. Stubborn Reliance on Intuition and Subjectivity in Employee Selection
   Highhouse, Scott · 2008 · Industrial and Organizational Psychology
5. BEYOND EMPLOYMENT INTERVIEW VALIDITY: A COMPREHENSIVE NARRATIVE REVIEW OF RECENT RESEARCH AND TRENDS OVER TIME
   Posthuma, Richard A., Morgeson, Frederick P., Campion, Michael A. · 2002 · Personnel Psychology

See all 318