Cited by 33

Robots Are Judging Me: Perceived Fairness of Algorithmic Recruitment Tools

Hilliard, Airlie, Guenole, Nigel, Leutner, Franziska·2022·Frontiers in Psychology

Abstract

Recent years have seen rapid advancements in selection assessments, shifting away from human and toward algorithmic judgments of candidates. Indeed, algorithmic recruitment tools have been created to screen candidates’ resumes, assess psychometric characteristics through game-based assessments, and judge asynchronous video interviews, among other applications. While research into candidate reactions to these technologies is still in its infancy, early research in this regard has explored user experiences and fairness perceptions. In this article, we review applicants’ perceptions of the procedural fairness of algorithmic recruitment tools based on key findings from seven key studies, sampling over 1,300 participants between them. We focus on the sub-facets of behavioral control, the extent to which individuals feel their behavior can influence an outcome, and social presence, whether there is the perceived opportunity for a social connection and empathy. While perceptions of overall procedural fairness are mixed, we find that fairness perceptions concerning behavioral control and social presence are mostly negative. Participants feel less confident that they are able to influence the outcome of algorithmic assessments compared to human assessments because they are more objective and less susceptible to manipulation. Participants also feel that the human element is lost when these tools are used since there is a lack of perceived empathy and interpersonal warmth. Since this field of research is relatively under-explored, we end by proposing a research agenda, recommending that future studies could examine the role of individual differences, demographics, and neurodiversity in influencing fairness perceptions of algorithmic recruitment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2022.940456

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Algorithmic selection is generally seen as less fair than human or human-assisted processes, with the most consistent negative effects on perceived social presence and interpersonal treatment.

    “Algorithmic selection is generally seen as less fair than human or human-assisted processes, with the most This preprint research paper has not been peer reviewed.Electronic copy available at: https://ssrn.com/abstract=7489654P r e p r i n t n o t p e e r r e v i e w e d consistent negative effects on perceived social presence and interpersonal treatment [37].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. Albert E. T. (2019). AI in talent acquisition: a review of AI-applications used in recruitment and selection. Strateg. HR Rev. 18 215– 221. 10.1108/SHR-04-2019-0024 · DOI
  2. Atkins S. M. Sprenger A. M. Colflesh G. J. H. Briner T. L. Buchanan J. B. Chavis S. E. (2014). Measuring working memory is all fun and games. Exp. Psychol. 61 417– 438. 10.1027/1618-3169/a000262 24962121 · DOI · Paper page
  3. Bertrand M. Mullainathan S. (2004). Are Emily and greg more employable than lakisha and jamal? A field experiment on labor market discrimination. Am. Econ. Rev. 94 991– 1013. 10.1257/0002828042002561 · DOI · Paper page
  4. Buolamwini J. (2018). Gender shades: intersectional accuracy disparities in commercial gender classification. Proc. Mach. Learn. Res. 81 1– 15.
  5. Cheng M. M. Hackett R. D. (2021). A critical review of algorithms in HRM: definition, theory, and practice. Hum. Resour. Manag. Rev. 31: 100698. 10.1016/j.hrmr.2019.100698 · DOI · Paper page
  6. Ellison L. J. McClure Johnson T. Tomczak D. Siemsen A. Gonzalez M. F. (2020). Game on! Exploring reactions to game-based selection assessments. J. Manag. Psychol. 35 241– 254. 10.1108/JMP-09-2018-0414 · DOI
  7. Fontechia S. A. Miltenberger R. G. Smith T. J. Berkman K. (2019). Evaluating video modeling for teaching professional E-mailing skills in transition-age job seekers with autism. J. Appl. Rehabil. Couns. 50 73– 90. 10.1891/0047-2220.50.1.73 11261958 · DOI
  8. Fritts M. Cabrera F. (2021). AI recruitment algorithms and the dehumanization problem. Ethics Inf. Technol. 23 791– 801. 10.1007/s10676-021-09615-w · DOI
  9. Georgiou K. Nikolaou I. (2020). Are applicants in favor of traditional or gamified assessment methods? Exploring applicant reactions towards a gamified selection method. Comput. Hum. Behav. 109: 106356. 10.1016/j.chb.2020.106356 · DOI
  10. Gilliland S. W. (1993). The perceived fairness of selection systems: an organizational justice perspective. Acad. Manag. Rev. 18 694– 734. 10.5465/amr.1993.9402210155 · DOI · Paper page
  11. Guenole N. Svensson C. Wille B. Aloyan K. Saville P. (eds). (2022). “ A European perspective on Psychometric Measurement Technology,” in Technology and Measurement Around the Globe, (Cambridge: Cambridge University Press).
  12. Guidotti R. Monreale A. Ruggieri S. Turini F. Giannotti F. Pedreschi D. (2018). A survey of methods for explaining black box models. ACM Comput. Surv. 51: 93. 10.1145/3236009 · DOI · Paper page
  13. Hauk N. Hüffmeier J. Krumm S. (2018). Ready to be a silver surfer? A meta-analysis on the relationship between chronological age and technology acceptance. Comput. Hum. Behav. 84 304– 319. 10.1016/j.chb.2018.01.020 · DOI
  14. Hilliard A. Kazim E. Bitsakis T. Leutner F. (2022a). Measuring personality through images: validating a forced-choice image-based assessment of the big five personality traits. J. Intell. 10: 12. 10.3390/jintelligence10010012 35225927 PMC8883940 · DOI
  15. Hilliard A. Kazim E. Koshiyama A. Zannone S. Trengove M. Kingsman N. (2022b). Regulating the robots: NYC mandates bias audits for ai-driven employment decisions. SSRN Electronic J. 14 1– 14. 10.2139/SSRN.4083189 · DOI
  16. Kaibel C. Koch-Bayram I. Biemann T. Mühlenbock M. (2019). “ Applicant perceptions of hiring algorithms - Uniqueness and discrimination experiences as moderators,” in Proceedings of the 79th Annual Meeting of the Academy of Management 2019: Understanding the Inclusive Organization (Briarcliff Manor, NY: AoM). 10.5465/AMBPP.2019.210 · DOI
  17. Kessler S. R. Bandelli A. C. Spector P. E. Borman W. C. Nelson C. E. Penney L. M. (2010). Re-Examining machiavelli: a three-dimensional model of machiavellianism in the workplace. J. Appl. Soc. Psychol. 40 1868– 1896. 10.1111/J.1559-1816.2010.00643.X · DOI
  18. Kilduff M. Chiaburu D. S. Menges J. I. (2010). Strategic use of emotional intelligence in organizational settings: exploring the dark side. Res. Organ. Behav. 30 129– 152. 10.1016/j.riob.2010.10.002 · DOI
  19. Köchling A. Wehner M. C. (2020). Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development. Bus. Res. 13 795– 848. 10.1007/S40685-020-00134-W/FIGURES/2 · DOI
  20. Köchling A. Wehner M. C. Warkocz J. (2022). Can I show my skills? Affective responses to artificial intelligence in the recruitment process. Rev. Manag. Sci. 2022 1– 30. 10.1007/S11846-021-00514-4/FIGURES/3 · DOI
  21. Landers R. N. Armstrong M. B. Collmus A. B. Mujcic S. Blaik J. (2021). Theory-driven game-based assessment of general cognitive ability: design theory, measurement, prediction of performance, and test fairness. J. Appl. Psychol. 10.1037/apl0000954 [Epub ahead of print]. 34672652 · DOI
  22. Langer M. Baum K. König C. J. Hähne V. Oster D. Speith T. (2021). Spare me the details: how the type of information about automated interviews influences applicant reactions. Int. J. Sel. Assess. 29 154– 169. 10.1111/ijsa.12325 · DOI
  23. Langer M. König C. J. Fitili A. (2018). Information as a double-edged sword: the role of computer experience and information on applicant reactions towards novel technologies for personnel selection. Comput. Hum. Behav. 81 19– 30. 10.1016/j.chb.2017.11.036 · DOI · Paper page
  24. Langer M. König C. J. Papathanasiou M. (2019). Highly automated job interviews: acceptance under the influence of stakes. Int. J. Sel. Assess. 27 217– 234. 10.1111/ijsa.12246 · DOI · Paper page
  25. Lepri B. Oliver N. Letouzé E. Pentland A. Vinck P. (2018). Fair, transparent, and accountable algorithmic decision-making processes: the premise, the proposed solutions, and the open challenges. Philos. Technol. 31 611– 627. 10.1007/s13347-017-0279-x · DOI · Paper page
  26. Leutner F. Sonia-Cristina S.-C. Liff J. Mondragon N. (2020). The potential of game- and video-based assessments for social attributes: examples from practice. J. Manag. Psychol. 36 533– 547. 10.1108/JMP-01-2020-0023 · DOI
  27. Li L. Lassiter T. Lee M. K. Oh J. (2021). “ Algorithmic hiring in practice: recruiter and hr professional’s perspectives on AI use in hiring; algorithmic hiring in practice: recruiter and HR professional’s perspectives on AI use in hiring,” in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (New York, NY: Association for Computing Machinery). 10.1145/3461702 · DOI
  28. Mirowska A. Mesnet L. (2021). Preferring the devil you know: potential applicant reactions to artificial intelligence evaluation of interviews. Hum. Resour. Manag. J. 32: 12393. 10.1111/1748-8583.12393 · DOI
  29. Salovey P. Mayer J. D. (1990). Emotional Intelligence. Imag. Cogn. Pers. 9 185– 211. 10.2190/DUGG-P24E-52WK-6CDG 22612255 · DOI · Paper page
  30. Society for Industrial and Organizational Psychology (2018). Principles for the Validation and Use of Personnel Selection Procedures, 5th Edn. Bowling Green, OH: Society for Industrial and Organizational Psychology. 10.1017/iop.2018.195 · DOI
  31. Suen H. Y. Chen M. Y. C. Lu S. H. (2019). Does the use of synchrony and artificial intelligence in video interviews affect interview ratings and applicant attitudes? Comput. Hum. Behav. 98 93– 101. 10.1016/j.chb.2019.04.012 · DOI
  32. Weiss B. Feldman R. S. (2006). Looking good and lying to do it: deception as an impression management strategy in job interviews. J. Appl. Soc. Psychol. 36 1070– 1086. 10.1111/J.0021-9029.2006.00055.X · DOI · Paper page
  33. Willis C. Powell-Rudy T. Colley K. Prasad J. (2021). Examining the use of game-based assessments for hiring autistic job seekers. J. Intell. 9: 53. 10.3390/JINTELLIGENCE9040053 34842751 PMC8628896 · DOI

Cited by

Sorted by citations

Authors
Hilliard, A., Guenole, N., & Leutner, F.
Venue
Frontiers in Psychology
Published
2022-07-25
Volume
13
ISSN
1664-1078
Cited by
33
References
4

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review