CoChat
Sign Up
Cited by 37Explainable Artificial Intelligence (XAI)AI and HR TechnologiesHuman-Automation Interaction and Safety

Explainability does not mitigate the negative impact of incorrect AI advice in a personnel selection task

Open access
Cecil, Julia, Lermer, Eva, Hudecek, Matthias F. C., Sauer, Jan, Gaube, Susanne·2024·Scientific Reports
DOI 10.1038/s41598-024-60220-5PMC PMC11056364
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite the rise of decision support systems enabled by artificial intelligence (AI) in personnel selection, their impact on decision-making processes is largely unknown. Consequently, we conducted five experiments (N = 1403 students and Human Resource Management (HRM) employees) investigating how people interact with AI-generated advice in a personnel selection task. In all pre-registered experiments, we presented correct and incorrect advice. In Experiments 1a and 1b, we manipulated the source of the advice (human vs. AI). In Experiments 2a, 2b, and 2c, we further manipulated the type of explainability of AI advice (2a and 2b: heatmaps and 2c: charts). We hypothesized that accurate and explainable advice improves decision-making. The independent variables were regressed on task performance, perceived advice quality and confidence ratings. The results consistently showed that incorrect advice negatively impacted performance, as people failed to dismiss it (i.e., overreliance). Additionally, we found that the effects of source and explainability of advice on the dependent variables were limited. The lack of reduction in participants’ overreliance on inaccurate advice when the systems’ predictions were made more explainable highlights the complexity of human-AI interaction and the need for regulation and quality standards in HRM.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-024-60220-5

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing studies often rely on vignette-based designs, samples of students or laypeople rather than professional recruiters, or indirect indicators of influence such as confidence ratings and ranking scores.

    “Existing studies often rely on vignette-based designs (Oberst et al. 2020), samples of students (Cecil et al. 2024) or laypeople (Neumann and Kuncel 2025) rather than professional recruiters, or indirect indicators of influence such as confidence ratings and ranking scores.”

    From When Recruiters Meet Algorithms: Advice Taking and Bias in AI‐Assisted Recruitment (2026) · cites this paper as Cecil et al. 2024

    The Experiment 1a Method identifies a sample of 370 university students, supporting the part attributed to Cecil et al. But the Experiment 1b Method reports a replication with HRM employees, 54.5% of whom worked in selection or recruiting. The paper therefore does not support characterizing its samples as students rather than professional recruiters without qualification. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • This pattern is consistent with recent findings in personnel selection and with the broader literature on overreliance on automated systems.

    “This pattern is consistent with recent findings in personnel selection (Cecil et al. 2024) and with the broader literature on overreliance on automated systems (Parasuraman and Manzey 2010;Parasuraman and Riley 1997;Skitka et al. 1999), although it awaits confirmation through formal inferential comparison.”

    From When Recruiters Meet Algorithms: Advice Taking and Bias in AI‐Assisted Recruitment (2026) · cites this paper as Cecil et al. 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 43 · Cited by 37

Research works printed but not linked: 33

  1. [1]Helm JM Machine learning and Artificial Intelligence: Definitions, applications, and future directions Curr. Rev. in Musculoskel. Med. 2020 13 69 76 10.1007/s12178-020-09600-8 PMC7083992 31983042 · DOI
  2. [2]Campion MA Campion ED Machine learning applications to personnel selection: Current illustrations, lessons learned, and future research Pers. Psychol. 2023 00 1 17 10.1111/peps.1262 · DOI
  3. [3]Allal-Chérif O Yela Aránega A Castaño Sánchez R Intelligent recruitment: How to identify, select Campion & Campion, 2023, and retain talents from around the world using artificial intelligence Technol. Forecast. Soc. Change 2021 169 120822 10.1016/j.techfore.2021.120822 · DOI
  4. [4]Giermindl LM Strich F Christ O Leicht-Deobald U Redzepi A The dark sides of people analytics: Reviewing the perils for organisations and employees Eur. J. Inf. Syst. 2022 31 410 435 10.1080/0960085X.2021.1927213 · DOI · Paper page
  5. [7]Black JS Esch P AI-enabled recruiting in the war for talent Bus. Horiz. 2021 64 513 524 10.1016/j.bushor.2021.02.015 · DOI
  6. [8]Larkin C Drummond Otten C Árvai J Paging Dr JARVIS! Will people accept advice from artificial intelligence for consequential risk management decisions J. Risk Res. 2021 25 407 422 10.1080/13669877.2021.1958047 · DOI
  7. [9]Gino F Brooks AW Schweitzer ME Anxiety, advice, and the ability to discern: Feeling anxious motivates individuals to seek and use advice J. Pers. Soc. Psychol. 2012 102 497 512 10.1037/a0026413 22121890 · DOI
  8. [10]Schultze T Mojzisch A Schulz-Hardt S On the inability to ignore useless advice: A case for anchoring in the judge-advisor-system Exp. Psychol. 2017 64 170 183 10.1027/1618-3169/a000361 28633625 · DOI
  9. [11]Vodrahalli, K., Daneshjou, R., Gerstenberg, T. & Zou, J. Do humans trust advice more if it comes from AI?: An Analysis of Human-AI Interactions. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society 763–777 10.1145/3514094.3534150 (AIES, 2022). · DOI · Paper page
  10. [12]Furnham A Boo HC A literature review of the anchoring effect J. Socio-Econ. 2011 40 35 42 10.1016/j.socec.2010.10.008 · DOI · Paper page
  11. [13]Will P Krpan D Lordan G People versus machines: Introducing the HIRE framework Artif. Intell. Rev. 2022 10.1007/s10462-022-10193-6 · DOI
  12. [16]Bogert E Lauharatanahirun N Schecter A Human preferences toward algorithmic advice in a word association task Sci. Rep. 2022 12 14501 10.1038/s41598-022-18638-2 36008508 PMC9411628 · DOI
  13. [19]Goodyear K Advice taking from humans and machines: An fMRI and effective connectivity study Front. Hum. Neurosci. 2016 10.3389/fnhum.2016.00542 27867351 PMC5095979 · DOI
  14. [20]Strickland L Inhibitory cognitive control allows automated advice to improve accuracy while minimizing misuse Psychol. Sci. 2021 32 1768 1781 10.1177/09567976211012676 34570615 · DOI
  15. [22]Jones-Jang SM Park YJ How do people react to AI failure? Automation bias, algorithmic aversion, and perceived controllability J. Comput.-Mediat. Commun. 2022 10.1093/jcmc/zmac029 · DOI · Paper page
  16. [24]Gerlings, J., Shollo, A. & Constantiou, I. Reviewing the need for explainable Artificial Intelligence (xAI). In Hawaii International Conference on System Sciences. 10.24251/HICSS.2021.156 (HICSS, 2021). · DOI
  17. [27]Antognini D Musat C Faltings B Multi-dimensional explanation of target variables from documents Proc. AAAI Conf. Artif. Intell. 2021 35 14 12507 12515 10.1609/aaai.v35i14.17483 · DOI
  18. [28]Harbecke, D., Schwarzenberg, R. & Alt, C. Learning explanations from language data. Preprint at https://arxiv.org/pdf/1808.04127.pdf (2018). · arXiv
  19. [30]Schwarzenberg, R., Harbecke, D., Macketanz, V., Avramidis, E. & Möller, S. Train, sort, explain: Learning to diagnose translation models. arXiv, 10.18653/v1/N19-4006 (2019). · DOI
  20. [31]Wallace, E., Feng, S. & Boyd-Graber, J. Interpreting neural networks with nearest neighbors. Preprint at https://arxiv.org/pdf/1809.02847.pdf (2018). · arXiv
  21. [32]Van der Waa J Nieuwburg E Cremers A Neerincx M Evaluating XAI: A comparison of rule-based and example-based explanations Artif. Intell. 2021 291 103404 10.1016/j.artint.2020.103404 · DOI · Paper page
  22. [33]Langer M Oster D Speith T Hermanns H Kästner L Schmidt E Sesing A Baum K What do we want from Explainable artificial intelligence (XAI)? A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research Artif. Intell. 2021 296 103473 10.1016/j.artint.2021.103473 · DOI · Paper page
  23. [34]Van Iddekinge CH Lievens F Sackett PR Personnel selection: A review of ways to maximize validity, diversity, and the applicant experience Pers. Psychol. 2023 76 651 686 10.1111/peps.12578 · DOI
  24. [36]Langer M Oster D Speith T Hermanns H Kästner L Schmidt E Sesing A Baum K What do we want from Explainable artificial intelligence (XAI)? A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research Artif. Intell. 2021 10.1016/j.artint.2021.103473 · DOI · Paper page
  25. [37]Castelo N Bos MW Lehmann DR Task-dependent algorithm aversion J. Mark. Res. 2019 56 809 825 10.1177/0022243719851788 · DOI · Paper page
  26. [38]Chong L Zhang G Goucher-Lambert K Kotovsky K Cagan J Human confidence in artificial intelligence and in themselves: The evolution and impact of confidence on adoption of AI advice Comp. Hum. Behav. 2022 127 107018 10.1016/j.chb.2021.107018 · DOI · Paper page
  27. [40]Schlegel, U., Arnout, H., El-Assady, M., Oelke, D. & Keim, D. A. Towards a rigorous evaluation of XAI methods on time series. Preprint at https://arxiv.org/pdf/1909.07082.pdf (2019). · arXiv
  28. [41]Bansal, G. et al. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing vol 7, pp 2–11 (HCOMP, 2019).
  29. [42]Westphal M Vössing M Satzger G Yom-Tov GB Rafaeli A Decision control and explanations in human-AI collaboration: Improving user perceptions and compliance Comp. Hum. Behav. 2023 144 107714 10.1016/j.chb.2023.107714 · DOI
  30. [43]Spatola N Marchesi S Wykowska A Cognitive load affects early processes involved in mentalizing robot behaviour Sci. Rep. 2022 12 14924 10.1038/s41598-022-19213-5 36056165 PMC9440196 · DOI
  31. [44]Vasconcelos, H. et al. Explanations can reduce overreliance on AI systems during decision-making. Preprint at https://arxiv.org/pdf/2212.06823.pdf (2023). · arXiv
  32. [46]König N Improving measurement and prediction in personnel selection through the application of machine learning Pers. Psychol. 2023 00 1 63 10.1111/peps.12608 · DOI
  33. [48]Buçinca Z Malaya MB Gajos KZ To Trust or to Think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making Proc. ACM Human-Comput. Interact. 2021 5 1 21 10.1145/3449287(HCI · DOI

Authors
Cecil, J., Lermer, E., Hudecek, M. F. C., Sauer, J., & Gaube, S.
Affiliations
  • Ludwig-Maximilians-Universität München (DE)
  • Technische Hochschule Augsburg (DE)
  • University of Regensburg (DE)
  • Ostbayerische Technische Hochschule Amberg-Weiden (DE)
  • University College London (GB)
Corresponding author
Cecil, Julia
ORCID
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2024-04-28
Volume
14
Issue
1
Pages
9736-9736
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/s41598-024-60220-5
PMC
PMC11056364
OpenAlex
W4395958655
Cited by
37
References
43
Citations by year
2026: 20 · 2025: 12 · 2024: 5
FWCI
5.46
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Volkswagen Foundation (98525)
  • Ludwig-Maximilians-Universität München
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
Keywords
explainable AI, personnel selection, human-AI interaction, overreliance on AI, decision support system, advice quality, heatmaps, human resource management, trust in AI
MeSH terms
Adult, Artificial Intelligence, Decision Making, Female, Humans, Male, Personnel Selection, Task Performance and Analysis, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review