From Trust in Automation to Decision Neuroscience: Applying Cognitive Neuroscience Methods to Understand and Improve Interaction Decisions Involved in Human Automation Interaction
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Human automation interaction (HAI) systems have thus far failed to live up to expectations mainly because human users do not always interact with the automation appropriately. Trust in automation (TiA) has been considered a central influence on the way a human user interacts with an automation; if TiA is too high there will be overuse, if TiA is too low there will be disuse. However, even though extensive research into TiA has identified specific HAI behaviors, or trust outcomes, a unique mapping between trust states and trust outcomes has yet to be clearly identified. Interaction behaviors have been intensely studied in the domain of HAI and TiA and this has led to a reframing of the issues of problems with HAI in terms of reliance and compliance. We find the behaviorally defined terms reliance and compliance to be useful in their functionality for application in real-world situations. However, we note that once an inappropriate interaction behavior has occurred it is too late to mitigate it. We therefore take a step back and look at the interaction decision that precedes the behavior. We note that the decision neuroscience community has revealed that decisions are fairly stereotyped processes accompanied by measurable psychophysiological correlates. Two literatures were therefore reviewed. TiA literature was extensively reviewed in order to understand the relationship between TiA and trust outcomes, as well as to identify gaps in current knowledge. We note that an interaction decision precedes an interaction behavior and believe that we can leverage knowledge of the psychophysiological correlates of decisions to improve joint system performance. As we believe that understanding the interaction decision will be critical to the eventual mitigation of inappropriate interaction behavior, we reviewed the decision making literature and provide a synopsis of the state of the art understanding of the decision process from a decision neuroscience perspective. We forward hypotheses based on this understanding that could shape a research path toward the ability to mitigate interaction behavior in the real world.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 4×
“…automations in general, and initial observations about the design of the automation (Muir and Moray, 1996; Nass et al., 1996; Dzindolet et al., 2002; Lee and See, 2004; Parasuraman and Miller, 2004; Miller, 2005; Merritt and Ilgen, 2008; Merritt, 2011; Merritt et al., 2012; Pak et al., 2012).”
BackgroundInfluentialHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 5×
“Misuse refers to instances when the automation is used without undo skepticism, tending to result in overuse (Parasuraman and Riley, 1997; Bahner et al., 2008; Parasuraman and Manzey, 2010).”
BackgroundInfluentialA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 8×
“In these situations, TiA then develops and shows dynamic changes from the ongoing comparison of the expectations about the automation’s behavior and observations by the human user about the automation’s performance weighted heavily on the risk borne by the human user (Sheridan and Hennessy, 1984; Muir, 1994; Muir and…”
BackgroundMethodInfluentialThe Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · cited 2×
“…compared value A cortical area involved in gaze fixation, saccade, and attention, underlying evidence accumulation Coe et al. (2002) and Goldberg et al. (2006) signals, has been observed as preceeding the conscious decision to act (Gold and Shadlen, 2007; Soon et al., 2013; Perez et al., 2015).”
BackgroundA framework for studying the neurobiology of value-based decision making
Rangel, A., Camerer, C., & Montague, P. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2357 · Cited by 2,184 · cited 11×
“In the same fMRI study the ventral striatum was implicated in the formation of representations of the expected value or reward of an alternative (Yacubian et al., 2006; Kable and Glimcher, 2007; Rangel et al., 2008; Basten et al., 2010; Lim et al., 2011).”
BackgroundInfluentialA Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 2×
“Such complex interactions involve perceived risk, self-confidence, workload, and even personality type (Lee and Moray, 1994; Muir, 1994; Parasuraman and Riley, 1997; Lee and See, 2004; Merritt and Ilgen, 2008; Hancock et al., 2011; Schaefer et al., 2012; Merritt et al., 2014).”
BackgroundUnconscious determinants of free decisions in the human brain
Soon, C. S., Brass, M., Heinze, H.-J., & Haynes, J.-D. (2008) · Nature Neuroscience · doi:10.1038/nn.2112 · Cited by 1,741 · cited 3×
“Further, the ability to predict decision outcomes has been pursued by both the cognitive neuroscience (Soon et al., 2008; Haynes, 2011; Perez et al., 2015) and brain computer interface (Musallam et al., 2004) communities.”
BackgroundNeural correlates of decision variables in parietal cortex
Platt, M. L., & Glimcher, P. W. (1999) · Nature · doi:10.1038/22268 · Cited by 1,721 · cited 2×
“…unfolds, some have hypothesized that the comparative value, also known as the decision variable, is accumulated in the lateral intraparietal cortex (LIP) until a decision threshold is reached, bringing about a decision (Platt and Glimcher, 1999; Kiani and Shadlen, 2009; Mulder et al., 2014).”
BackgroundResultComplacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 7×
“Misuse refers to instances when the automation is used without undo skepticism, tending to result in overuse (Parasuraman and Riley, 1997; Bahner et al., 2008; Parasuraman and Manzey, 2010).”
BackgroundInfluentialDecision making, the P3, and the locus coeruleus–norepinephrine system
Nieuwenhuis, S., Aston-Jones, G., & Cohen, J. D. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.4.510 · Cited by 1,678
Trust, control strategies and allocation of function in human-machine systems
LEE, J., & MORAY, N. (1992) · Ergonomics · doi:10.1080/00140139208967392 · Cited by 1,459
Visual fixations and the computation and comparison of value in simple choice
Krajbich, I., Armel, C., & Rangel, A. (2010) · Nature Neuroscience · doi:10.1038/nn.2635 · Cited by 1,408
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 3×
“…of automation performance and then observes unexpected errors or has more self confidence in her ability to perform the task than confidence in the efficacy of the automation for the same task (Lee and Moray, 1994; Parasuraman and Riley, 1997; Moray et al., 2000; Dzindolet et al., 2003).”
BackgroundOn-line trust: concepts, evolving themes, a model
Corritore, C. L., Kracher, B., & Wiedenbeck, S. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00041-7 · Cited by 1,200 · cited 1×
“There are aspects of interpersonal trust that are analogous to TiA, in particular that there needs to be a sense of risk or vulnerability on the part of the trustor for trust to develop (Lee and Moray, 1994; Muir, 1994; Corritore et al., 2003; Lee and See, 2004; Evans and Krueger, 2011).”
BackgroundRepresentation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027 · cited 1×
“…unfolds, some have hypothesized that the comparative value, also known as the decision variable, is accumulated in the lateral intraparietal cortex (LIP) until a decision threshold is reached, bringing about a decision (Platt and Glimcher, 1999; Kiani and Shadlen, 2009; Mulder et al., 2014).”
BackgroundMUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993
MUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 6×
“There are aspects of interpersonal trust that are analogous to TiA, in particular that there needs to be a sense of risk or vulnerability on the part of the trustor for trust to develop (Lee and Moray, 1994; Muir, 1994; Corritore et al., 2003; Lee and See, 2004; Evans and Krueger, 2011).”
BackgroundInfluentialHare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.1309-08.2008 · Cited by 792
Perceptions of moral character modulate the neural systems of reward during the trust game
Delgado, M. R., Frank, R. H., & Phelps, E. A. (2005) · Nature Neuroscience · doi:10.1038/nn1575 · Cited by 729 · cited 1×
“These stimuli will often produce an orienting response (Sokolov et al., 2002; Glimcher and Rustichini, 2004; Delgado et al., 2005), characterized in humans by a measurable increase in tonic skin conductance (SC) levels and a decrease in heart rate variability (Figner and Murphy, 2011).”
BackgroundDoes automation bias decision-making?
SKITKA, L. J., MOSIER, K. L., & BURDICK, M. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0252 · Cited by 694 · cited 1×
“…to choose the route of least cognitive effort, making it easier, or at least preferable, to accept that feedback from an automation as correct (Dzindolet et al., 1999, 2001; Skitka et al., 1999, 2000; Wang et al., 2008; Parasuraman and Manzey, 2010; Goddard et al., 2014; Mosier and Skitka, 1996).”
BackgroundEye tracking in HCI and usability research
Poole, A., & Ball, L. J. (2006) · Encyclopedia of Human Computer Interaction · doi:10.4018/978-1-59140-562-7.ch034 · Cited by 690 · cited 1×
“For example, gaze fixation, critical for evidence evaluation (Poole and Ball, 2006) in visually based decisions, is controlled by the LIP in monkeys (Coe et al., 2002).”
BackgroundCognitive Control Signals for Neural Prosthetics
Musallam, S., Corneil, B. D., Greger, B., Scherberger, H., & Andersen, R. A. (2004) · Science · doi:10.1126/science.1097938 · Cited by 663 · cited 2×
“Further, the ability to predict decision outcomes has been pursued by both the cognitive neuroscience (Soon et al., 2008; Haynes, 2011; Perez et al., 2015) and brain computer interface (Musallam et al., 2004) communities.”
BackgroundChib, V. S., Rangel, A., Shimojo, S., & O'Doherty, J. P. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2575-09.2009 · Cited by 643 · cited 1×
“That is, value based comparison has been suggested as supported by activation in the ventral medial prefrontal cortex (vmPFC; Chib et al., 2009; Gläscher et al., 2009; Basten et al., 2010), whereas the ‘‘comparator’’ function in perceptual decisions has been associated with increased activity in…”
BackgroundSimilarities and differences between human–human and human–automation trust: an integrative review
Madhavan, P., & Wiegmann, D. A. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500337708 · Cited by 577
Yacubian, J., Gläscher, J., Schroeder, K., Sommer, T., Braus, D. F., & Büchel, C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2915-06.2006 · Cited by 537
Neural computations associated with goal-directed choice
Rangel, A., & Hare, T. (2010) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2010.03.001 · Cited by 532
Merritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527 · cited 2×
“…automations in general, and initial observations about the design of the automation (Muir and Moray, 1996; Nass et al., 1996; Dzindolet et al., 2002; Lee and See, 2004; Parasuraman and Miller, 2004; Miller, 2005; Merritt and Ilgen, 2008; Merritt, 2011; Merritt et al., 2012; Pak et al., 2012).”
BackgroundMidfrontal conflict-related theta-band power reflects neural oscillations that predict behavior
Cohen, M. X., & Donner, T. H. (2013) · Journal of Neurophysiology · doi:10.1152/jn.00479.2013 · Cited by 494 · cited 1×
“…decision process was accompanied by a general shift in power between lower bands (delta, 1–4 Hz and theta, 5–7 Hz) to higher frequency bands (alpha, 8–12 Hz and beta, 13–35 Hz), as well as a broadband increase in cross-trial phase coherence at about 220 ms post stimulus (Cohen and Donner, 2013).”
BackgroundThe benefits of imperfect diagnostic automation: a synthesis of the literature
Wickens, C. D., & Dixon, S. R. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500370105 · Cited by 423 · cited 2×
“…is important enough that as long as errors are predictable and the automation error rate is at a consistent rate of approximately 30% or less, most human users will decide to continue to use and benefit from the automation (Parasuraman et al., 2000; Wickens and Dixon, 2007; Wang et al., 2009).”
BackgroundGläscher, J., Hampton, A. N., & O'Doherty, J. P. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhn098 · Cited by 387 · cited 1×
“That is, value based comparison has been suggested as supported by activation in the ventral medial prefrontal cortex (vmPFC; Chib et al., 2009; Gläscher et al., 2009; Basten et al., 2010), whereas the ‘‘comparator’’ function in perceptual decisions has been associated with increased activity in…”
BackgroundThe Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386 · cited 2×
“…from biases toward automations in general, and initial observations about the design of the automation (Muir and Moray, 1996; Nass et al., 1996; Dzindolet et al., 2002; Lee and See, 2004; Parasuraman and Miller, 2004; Miller, 2005; Merritt and Ilgen, 2008; Merritt, 2011; Merritt et al., 2012;…”
BackgroundLim, S.-L., O'Doherty, J. P., & Rangel, A. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1246-11.2011 · Cited by 329
Ploran, E. J., Nelson, S. M., Velanova, K., Donaldson, D. I., Petersen, S. E., & Wheeler, M. E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3522-07.2007 · Cited by 282 · cited 2×
“However, there has been evidence from fMRI studies that the human parietal cortex is also involved in accumulating the decision variable (Ploran et al., 2007; Heekeren et al., 2008).”
BackgroundBahner, J. E., Hüper, A.-D., & Manzey, D. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2008.06.001 · Cited by 279
Automation failures on tasks easily performed by operators undermine trust in automated aids
Madhavan, P., Wiegmann, D. A., & Lacson, F. C. (2006) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006777724408 · Cited by 268 · cited 1×
“However, it is worth noting that for various reasons, human users are notoriously poor at making accurate judgments of competence (Sheridan and Hennessy, 1984; Lee and Moray, 1992; Dzindolet et al., 2002, 2003; Madhavan et al., 2006; Verberne et al., 2012; Merritt et al., 2014).”
BackgroundAffective Processes in Human–Automation Interactions
Merritt, S. M. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811411912 · Cited by 260 · cited 1×
“…automations in general, and initial observations about the design of the automation (Muir and Moray, 1996; Nass et al., 1996; Dzindolet et al., 2002; Lee and See, 2004; Parasuraman and Miller, 2004; Miller, 2005; Merritt and Ilgen, 2008; Merritt, 2011; Merritt et al., 2012; Pak et al., 2012).”
BackgroundAutomation bias: Empirical results assessing influencing factors
Goddard, K., Roudsari, A., & Wyatt, J. C. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.01.001 · Cited by 163
Automation Bias and Errors: Are Crews Better Than Individuals?
Skitka, L. J., Mosier, K. L., Burdick, M., & Rosenblatt, B. (2000) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1001_5 · Cited by 127
Trust as a Construct for Evaluation of Automated Aids: Past and Future Theory and Research
Masalonis, A. J., & Parasuraman, R. (1999) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129904300312 · Cited by 36 · cited 1×
“…frequent failures to trigger alarms, the frequency of monitoring the automation will increase, thereby reducing reliance on the automated agent (Masalonis and Parasuraman, 1999; Bagheri and Jamieson, 2004; Meyer, 2004; Madhavan and Wiegmann, 2007; Parasuraman and Manzey, 2010; Geels-Blair et…”
Manzey, D., Bahner, J. E., & Hueper, A.-D. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000303 · Cited by 33 · cited 1×
“Misuse has two related causes; automation bias and complacency (Manzey et al., 2006; Parasuraman and Manzey, 2010).”
BackgroundContinuous Calibration of Trust in Automated Systems
Merritt, S. M., Huber, K., LaChapell-Unnerstall, J., & Lee, D. (2014) · doi:10.21236/ada606748 · Cited by 20
Cited by
Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317
Measurement of trust in automation: A narrative review and reference guide
Kohn, S. C., de Visser, E. J., Wiese, E., Lee, Y.-C., & Shaw, T. H. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.604977 · Cited by 315
Adaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239
“Behaviors might not be observable if there is no interaction; nevertheless, behavioral measures are practical and can easily be used as a basis for modeling and prediction [25].”
A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives
Gebru, B., Zeleke, L., Blankson, D., Nabil, M., Nateghi, S., Homaifar, A., & Tunstel, E. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2022.3144956 · Cited by 101
“Measurements using physiological traits, such as employing heart-rate variability (HRV) measurement [55], evaluation of brain activity using fMRI [56], [57], and fNIRS [58], and electroencephalogram (EEG) [59]–[63], have been used to study and evaluate trust via its psychophysiological correlates.”
MethodLearning From the Slips of Others: Neural Correlates of Trust in Automated Agents
de Visser, E. J., Beatty, P. J., Estepp, J. R., Kohn, S., Abubshait, A., Fedota, J. R., & McDonald, C. G. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00309 · Cited by 73
Metadata
- Authors
- Drnec, K., Marathe, A. R., Lukos, J. R., & Metcalfe, J. S.
- Venue
- Frontiers in Human Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2016-06-30
- Volume
- 10
- Pages
- 290-290
- ISSN
- 1662-5161 (ISSN-L)
- OpenAlex
- W2470063825
- Cited by
- 72
- References
- 41
- FWCI
- 7.02
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.