Challenges of human—machine collaboration in risky decision-making
Open accessAbstract
The purpose of this paper is to delineate the research challenges of human—machine collaboration in risky decision-making. Technological advances in machine intelligence have enabled a growing number of applications in human—machine collaborative decision-making. Therefore, it is desirable to achieve superior performance by fully leveraging human and machine capabilities. In risky decision-making, a human decision-maker is vulnerable to cognitive biases when judging the possible outcomes of a risky event, whereas a machine decision-maker cannot handle new and dynamic contexts with incomplete information well. We first summarize features of risky decision-making and possible biases of human decision-makers therein. Then, we argue the necessity and urgency of advancing human—machine collaboration in risky decision-making. Afterward, we review the literature on human—machine collaboration in a general decision context, from the perspectives of human—machine organization, relationship, and collaboration. Lastly, we propose challenges of enhancing human—machine communication and teamwork in risky decision-making, followed by future research avenues.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
According to Xiong et al., machines will be helpful in gathering data and conveying key messages, while humans can obligate emotional influences and make an unbiased judgment.
“According to Xiong et al. [168], machines will be helpful in gathering data and conveying key messages, while humans can obligate emotional influences and make an unbiased judgment.”
From Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2024) · cites this paper as [168]
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524 · see citing works
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · cited 1× · see citing works
“…the likelihoods and consequences of a risk event, people are vulnerable to representativeness bias, availability bias, and anchoring effect (Tversky and Kahneman, 1974; Bier, 2004; Kahneman and Frederick, 2002), and their judgments do not always follow the Bayesian rule (Grether, 1992;…”
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1× · see citing works
“For example, Parasuraman et al. (2000) proposed the level of automation framework to specify the levels of automation for independent functions and the extent and process of human involvement.”
BackgroundAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
Duan, Y., Edwards, J. S., & Dwivedi, Y. K. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.01.021 · Cited by 2,628 · cited 3× · see citing works
“…(shared) mental models (Cannon-Bowers et al., 1993; Flemisch et al., 2012; Goodrich and Yi, 2013), situation awareness (Salmon et al., 2008; Ososky et al., 2012), measurement (Damacharla et al., 2018), and practice in decision-making (Jarrahi, 2018; Duan et al., 2019; Haesevoets et al., 2021).”
Background“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298 · see citing works
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · see citing works
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · see citing works
How to grow a mind: Statistics, structure, and abstraction
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011) · Science · doi:10.1126/science.1192788 · Cited by 1,815 · see citing works
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364 · cited 3× · see citing works
“In a hybrid system, machine behaviors in fl uence and shape human behaviors and vice versa (Rahwan et al., 2019).”
BackgroundInfluentialS-R compatibility: spatial characteristics of stimulus and response codes.
Fitts, P. M., & Seeger, C. M. (1953) · Journal of Experimental Psychology · doi:10.1037/h0062827 · Cited by 1,088 · see citing works
Understanding, explaining, and utilizing medical artificial intelligence
Cadario, R., Longoni, C., & Morewedge, C. K. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01146-0 · Cited by 342 · cited 8× · see citing works
“, 2020), explainability (Gunning, 2016; Degani et al., 2017; DARPA, 2018; Amann et al., 2020; Cadario et al., 2021), task allocation (van Maanen and van Dongen, 2005; Roth et al.”
BackgroundInfluentialThe effect of cognitive load on economic decision making: A survey and new experiments
Deck, C., & Jahedi, S. (2015) · European Economic Review · doi:10.1016/j.euroecorev.2015.05.004 · Cited by 304 · see citing works
Human-machine collaboration in managerial decision making
Haesevoets, T., De Cremer, D., Dierckx, K., & Van Hiel, A. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106730 · Cited by 280 · see citing works
Kraus, J., Scholz, D., Stiegemeier, D., & Baumann, M. (2020) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819853686 · Cited by 276 · cited 2× · see citing works
“…and developed, including impact factors (Lee and See, 2004; Hancock et al., 2011; Hoff and Bashir, 2015; Salem et al., 2015; Schaefer et al., 2016), and trust dynamics and calibration (McGuirl and Sarter, 2006; Madhavan and Wiegmann, 2007; Wang et al., 2016; Akash et al., 2017; Kraus et al., 2020).”
BackgroundMcGuirl, J. M., & Sarter, N. B. (2006) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006779166334 · Cited by 212 · see citing works
Fixing the program my computer learned: barriers for end users, challenges for the machine
Kulesza, T., Wong, W.-K., Stumpf, S., Perona, S., White, R., Burnett, M. M., Oberst, I., & Ko, A. J. (2009) · Proceedings of the 14th international conference on Intelligent user interfaces · doi:10.1145/1502650.1502678 · Cited by 83 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,062 · see citing works
“This refers to some risky decisions that are made using AI [177] (Fig.”
BackgroundZhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99 · see citing works
Human in the AI Loop via xAI and Active Learning for Visual Inspection
Rožanec, J. M., Montini, E., Cutrona, V., Papamartzivanos, D., Klemenčič, T., Fortuna, B., Mladenić, D., Veliou, E., Giannetsos, T., & Emmanouilidis, C. (2024) · Artificial Intelligence in Manufacturing · doi:10.1007/978-3-031-46452-2_22 · Cited by 17 · see citing works
- Authors
- Xiong, W., Fan, H., Ma, L., & Wang, C.
- Affiliations
- Tsinghua University (CN)
- Corresponding author
- Wang, Chen
- ORCID
- Wang, Chen
- Venue
- Frontiers of Engineering Management
- Published
- 2022-01-27
- Volume
- 9
- Issue
- 1
- Pages
- 89-103
- ISSN
- 2095-7513 (ISSN-L)
- 2096-0255
- OpenAlex
- W4213319945
- Cited by
- 95
- References
- 17
- Citations by year
- 2026: 27 · 2025: 23 · 2024: 16 · 2023: 21 · 2022: 8
- FWCI
- 16.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Frontiers of Engineering Management (publishedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- human-machine collaboration, risky decision making, cognitive biases, human-machine communication, incomplete information, decision making under uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
