CoChat
Sign Up
Cited by 46Human-Automation Interaction and SafetyExplainable Artificial Intelligence (XAI)AI and HR Technologies

Optimizing human-AI collaboration: Effects of motivation and accuracy information in AI-supported decision-making

Open access
Eisbach, Simon, Langer, Markus, Hertel, Guido·2023·Computers in Human Behavior: Artificial Humans
DOI 10.1016/j.chbah.2023.100015
Open this paper in CoChat — free account

Abstract

Artificial intelligence (AI) systems increasingly support human decision-making in fields like medicine, management, and finance. However, such human-AI (HAI) collaboration is often less effective than AI systems alone. Moreover, efforts to make AI recommendations more transparent have failed to improve the decision quality of HAI collaborations. Based on dual process theories of cognition, we hypothesized that suboptimal HAI collaboration is partly due to heuristic information processing of humans, creating a trust imbalance towards the AI system. In an online experiment with 337 participants, we investigated motivation and accuracy information as potential factors to induce more deliberate elaboration of AI recommendations, and thus improve HAI collaboration. Participants worked on a simulated personnel selection task and received recommendations from a simulated AI system. Participants' motivation was varied through gamification, and accuracy information through additional information from the AI system. Results indicate that both motivation and accuracy information positively influenced HAI performance, but in different ways. While high motivation primarily increased humans’ use in high-quality recommendations only, accuracy information improved both the use of low- and high-quality recommendations. However, a combination of high motivation and accuracy information did not yield additional improvement of HAI performance.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.chbah.2023.100015

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing research has focused primarily on attitudes, acceptance intentions, fairness perceptions, and self-reported trust, while fewer studies examine whether recruiters actually revise their decisions after receiving external recommendations.

    “Existing research has focused primarily on attitudes, acceptance intentions, fairness perceptions, and selfreported trust, while fewer studies examine whether recruiters actually revise their decisions after receiving external recommendations”

    From When Recruiters Meet Algorithms: Advice Taking and Bias in AI‐Assisted Recruitment (2026) · cites this paper as Eisbach et al. 2023

    Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 54 · Cited by 46

Research works printed but not linked: 30

  1. Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021). Does the whole exceed its parts? The effect of AI explanations on complementary team performance. In Y. Kitamura, A. Quigley, K. Isbister, T. Igarashi, P. Bjørn, & S. Drucker (Eds.), Proceedings of the 2021 CHI conference on human factors in computing systems (pp. 1–16). ACM. https://doi.org/10.1145/ 3411764.3445717.
  2. Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1–21. https://doi.org/10.1145/ 3449287
  3. Bunt, A., Lount, M., & Lauzon, C. (2012). Are explanations always important?. In C. Duarte, L. Carriço, J. Jorge, S. Oviatt, & D. Gonçalves (Eds.), Proceedings of the 2012 ACM international conference on intelligent user interfaces - IUI (Vol. 12, p. 169) ACM Press. https://doi.org/10.1145/2166966.2166996. · DOI · Paper page
  4. Burton, J. W., Stein, M., & Jensen, T. B. (2020). A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220–239. https://doi.org/10.1002/bdm.2155 · DOI · Paper page
  5. Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42(1), 116–131. https://doi.org/10.1037/0022-3514.42.1.116 · DOI · Paper page
  6. Chae, Y. (2020). US AI regulation guide: Legislative overview and practical considerations. The Journal of Robotics, Artificial Intelligence & Law, 3(1), 17–40.
  7. Chaiken, S. (1980). Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39(5), 752–766.
  8. Gregor, S., & Benbasat, I. (1999). Explanations from intelligent systems: Theoretical foundations and implications for practice. MIS Quarterly, 23(4), 497. https://doi. org/10.2307/249487 · DOI · Paper page
  9. Hertel, G., Fisher, S. L., & Van Fossen, J. (2023). Motivated trust in AI: An integrative model considering multiple stakeholder views in HRM. In B. Murray, J. H. Dulebohn, D. L. Stone, & K. M. Lukaszewski (Eds.), Research in human resource management: The future of human resource management. Charlotte, NC: Information Age Publishing. Research in Human Resource Management.
  10. Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/ 10.1177/0018720814547570 · DOI · Paper page
  11. Hunkenschroer, A. L., & Luetge, C. (2022). Ethics of AI-enabled recruiting and selection: A review and research agenda. Journal of Business Ethics, 1–31. https://doi.org/ 10.1007/s10551-022-05049-6 · DOI · Paper page
  12. Iyer, L. S. (2021). AI enabled applications towards intelligent transportation. Transportation Engineering, 5, Article 100083. https://doi.org/10.1016/j. treng.2021.100083 · DOI
  13. Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021). Formalizing trust in artificial intelligence. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. ACM. https://doi.org/10.1145/3442188.3445923. · DOI · Paper page
  14. Kaber, D. B., & Endsley, M. R. (2004). The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theoretical Issues in Ergonomics Science, 5(2), 113–153.
  15. Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020). Interpreting interpretability: Understanding data scientists' use of interpretability tools for machine learning. In Proceedings of the 2020 CHI conference on human factors in computing systems. ACM. https://doi.org/10.1145/3313831.3376219. · DOI · Paper page
  16. Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013). Too much, too little, or just right? Ways explanations impact end users' mental models. In 2013 IEEE symposium on visual languages and human centric computing (pp. 3–10). IEEE. https://doi.org/10.1109/VLHCC.2013.6645235. · DOI · Paper page
  17. Lange, D., & Pfarrer, M. D. (2017). Editors' comments: Sense and structure—the core building blocks of an AMR article. Academy of Management Review, 42(3), 407–416. https://doi.org/10.5465/amr.2016.0225 · DOI
  18. Langer, M., König, C. J., Back, C., & Hemsing, V. (2022). Trust in Artificial Intelligence: Comparing trust processes between human and automated trustees in light of unfair bias. Journal of Business and Psychology, 38(3), 493–508. https://doi.org/10.1007/ s10869-022-09829-9
  19. Langer, M., König, C. J., & Busch, V. (2021). Changing the means of managerial work: Effects of automated decision support systems on personnel selection tasks. Journal of Business and Psychology, 36(5), 751–769. https://doi.org/10.1007/s10869-020- 09711-6 · DOI
  20. Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., & Lee, S.-I. (2018). Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature Biomedical Engineering, 2(10), 749–760. https://doi.org/10.1038/s41551- 018-0304-0 · DOI
  21. Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. https://doi. org/10.5465/amr.1995.9508080335 · DOI · Paper page
  22. Milana, C., & Ashta, A. (2021). Artificial intelligence techniques in finance and financial markets: A survey of the literature. Strategic Change, 30(3), 189–209. https://doi. org/10.1002/jsc.2403 · DOI
  23. Mosier, K. L., & Manzey, D. (2019). Humans and automated decision aids: A match made in heaven?. In Human performance in automated and autonomous systems (pp. 19–42). CRC Press.
  24. Ngo, T., & Krämer, N. (2022). I humanize, therefore I understand? Effects of explanations and humanization of intelligent systems on perceived and objective user understanding. https://doi.org/10.31234/osf.io/6az2h · DOI
  25. Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020). A governance model for the application of AI in health care. Journal of the American Medical Informatics Association : JAMIA, 27(3), 491–497. https://doi.org/10.1093/jamia/ocz192 · DOI · Paper page
  26. Solberg, E., Kaarstad, M., Eitrheim, M. H. R., Bisio, R., Reegård, K., & Bloch, M. (2022). A conceptual model of trust, perceived risk, and reliance on AI decision aids. Group & Organization Management, 47(2), 187–222. https://doi.org/10.1177/ 10596011221081238
  27. Computers in Human Behavior: Artificial Humans 1 (2023) 100015 and transparency. https://doi.org/10.1145/3531146.3534639. June 21-24, 2022, Seoul, South Korea (pp. 2239–2250). The Association for Computing Machinery. · DOI
  28. Strack, F., & Deutsch, R. (2004). Reflective and impulsive determinants of social behavior. Personality and Social Psychology Review : An Official Journal of the Society for Personality and Social Psychology, Inc, 8(3), 220–247. https://doi.org/10.1207/ s15327957pspr0803_1
  29. Zerilli, J., Bhatt, U., & Weller, A. (2022). How transparency modulates trust in artificial intelligence. Patterns (New York, N.Y.), 3(4), Article 100455. https://doi.org/ 10.1016/j.patter.2022.100455 · DOI · Paper page
  30. Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In M. Hildebrandt, C. Castillo, E. Celis, S. Ruggieri, L. Taylor, & G. Zanfir-Fortuna (Eds.), Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 295–305). ACM. https://doi.org/10.1145/3351095.3372852. · DOI · Paper page

Authors
Eisbach, S., Langer, M., & Hertel, G.
Affiliations
  • University of Münster (DE)
  • Philipps University of Marburg (DE)
Corresponding author
Hertel, Guido
ORCID
Venue
Computers in Human Behavior: Artificial Humans
Publisher
Elsevier BV
Published
2023-08
Volume
1
Issue
2
Pages
100015
ISSN
2949-8821 (ISSN-L)
DOI
10.1016/j.chbah.2023.100015
OpenAlex
W4386825977
Cited by
46
References
54
Citations by year
2026: 19 · 2025: 20 · 2024: 7
FWCI
13.64
Type
journal article
Language
en
Update notices
Open access
Licence
Locations
  • Computers in Human Behavior Artificial Humans (publishedVersion, cc-by-nc-nd)
  • FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
Topics
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
Keywords
human-AI collaboration, AI-supported decision-making, dual-process theory, heuristic processing, motivation, gamification, trust in AI, personnel selection, deliberative processes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review