CoChat
Sign Up
Cited by 73Human-Automation Interaction and SafetyEEG and Brain-Computer InterfacesNeural and Behavioral Psychology Studies

Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents

Open access
de Visser, Ewart J., Beatty, Paul J., Estepp, Justin R., Kohn, Spencer, Abubshait, Abdulaziz, Fedota, John R., McDonald, Craig G.·2018·Frontiers in Human Neuroscience
DOI 10.3389/fnhum.2018.00309PMC PMC6095965
Open this paper in CoChat — free accountOpen PDF

Abstract

With the rise of increasingly complex artificial intelligence (AI), there is a need to design new methods to monitor AI in a transparent, human-aware manner. Decades of research have demonstrated that people, who are not aware of the exact performance levels of automated algorithms, often experience a mismatch in expectations. Consequently, they will often provide either too little or too much trust in an algorithm. Detecting such a mismatch in expectations, or trust calibration, remains a fundamental challenge in research investigating the use of automation. Due to the context-dependent nature of trust, universal measures of trust have not been established. Trust is a difficult construct to investigate because even the act of reflecting on how much a person trusts a certain agent, can change the perception of that agent. We hypothesized that electroencephalograms (EEG) would be able to provide such a universal index of trust without the need of self-report. In this work, EEGs were recorded for 21 participants (mean age = 22.1; 13 females) while they observed a series of algorithms perform a modified version of a flanker task. Each algorithm’s degree of credibility and reliability were manipulated. We hypothesized that neural markers of action monitoring, such as the observational error-related negativity (oERN) and observational error positivity (oPe), are potential candidates for monitoring computer algorithm performance. Our findings demonstrate that (1) it is possible to reliably elicit both the oERN and oPe while participants monitored these computer algorithms, (2) the oPe, as opposed to the oERN, significantly distinguished between high and low reliability algorithms and that 3) the oPe significantly correlated with subjective measures of trust. This work provides the first evidence for the utility of neural correlates of error monitoring for examining trust in computer algorithms.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fnhum.2018.00309

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Perceived expertise of a system had no effects on trust in cases when the system was inaccurate.

    “In addition to the anthropomorphism of a system, the perceived expertise of a system had no efects on trust in cases when the system was inaccurate [25].”

    From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [25]

    The Discussion (page unavailable) reports a strong effect of actual reliability but no interaction between credibility and reliability, consistent with perceived expertise having little additional effect once participants observed performance. The Results, however, say the expert and novice cover stories initially established credibility; the paper does not establish that expertise had no effect on trust at any point specifically for inaccurate systems. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 58

  1. Abubshait A. Wiese E. (2017). You look human, but act like a machine: agent appearance and behavior modulate different aspects of human–robot interaction. Front. Psychol. 8: 1393. 10.3389/fpsyg.2017.01393 28878703 PMC5572356 · DOI
  2. Adolphs R. (2002). Trust in the brain. Nat. Neurosci. 5 192– 193. 10.1038/nn0302-192 11865307 · DOI
  3. Berberian B. Somon B. Sahaï A. Gouraud J. (2017). The out-of-the-loop Brain: a neuroergonomic approach of the human automation interaction. Annu. Rev. Control 44 303– 315. 10.1016/J.ARCONTROL.2017.09.010 · DOI
  4. Bryk A. S. Raudenbush S. W. (1987). Application of hierarchical linear models to assessing change. Psychol. Bull. 101 147– 158. 10.1037/0033-2909.101.1.147 · DOI
  5. Carp J. Halenar M. J. Quandt L. C. Sklar A. Compton R. J. (2009). Perceived similarity and neural mirroring: evidence from vicarious error processing. Soc. Neurosci. 4 85– 96. 10.1080/17470910802083167 18633841 · DOI
  6. Chavarriaga R. del Millán R. J. (2010). Learning from EEG error-related potentials in noninvasive brain-computer interfaces. IEEE Trans. Neural. Syst. Rehabil. Eng. 18 381– 388. 10.1109/TNSRE.2010.2053387 20570777 · DOI · Paper page
  7. Chavarriaga R. Sobolewski A. del R. Millán J. (2014). Errare machinale est: the use of error-related potentials in brain-machine interfaces. Front. Neurosci. 8: 208. 10.3389/fnins.2014.00208 25100937 PMC4106211 · DOI
  8. Chen J. Y. C. Barnes M. J. Wright J. L. Stowers K. Lakhmani S. G. (2017). “Situation awareness-based agent transparency for human-autonomy teaming effectiveness,” in International Society for Optics and Photonics Vol. 10194 eds George T. Dutta A. K. Islam M. S. (Bellingham, WA: SPIE), 101941V. 10.1117/12.2263194 · DOI
  9. Delorme A. Makeig S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134 9– 21. 10.1016/j.jneumeth.2003.10.009 15102499 · DOI · Paper page
  10. Desmet C. Deschrijver E. Brass M. (2014). How social is error observation? The neural mechanisms underlying the observation of human and machine errors. Soc. Cogn. Affect. Neurosci. 9 427– 435. 10.1093/scan/nst002 23314011 PMC3989123 · DOI
  11. Drnec K. Marathe A. R. Lukos J. R. Metcalfe J. S. (2016). From trust in automation to decision neuroscience: applying cognitive neuroscience methods to understand and improve interaction decisions involved in human automation interaction. Front. Hum. Neurosci. 10: 290. 10.3389/fnhum.2016.00290 27445741 PMC4927573 · DOI
  12. Erchov S. (2017). Reconceptualizing Trust: Defining, Modeling, and Measuring Trust. Doctoral dissertation, Fairfax, VA, George Mason University.
  13. Eriksen B. A. Eriksen C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept. Psychophys. 16 143– 149. 10.3758/BF03203267 · DOI · Paper page
  14. Falkenstein M. (2004). Errors, conflicts, and the brain. J. Psychophysiol. 18 153– 163. 10.1027/0269-8803.18.4.153 · DOI
  15. Fedota J. R. Parasuraman R. (2010). Neuroergonomics and human error. Theor. Issues Ergon. Sci. 11 402– 421. 10.1080/14639220902853104 · DOI
  16. Fischer A. G. Klein T. A. Ullsperger M. (2017). Comparing the error-related negativity across groups: the impact of error- and trial-number differences. Psychophysiology 54 998– 1009. 10.1111/psyp.12863 28369880 · DOI
  17. Gehring W. J. Goss B. Coles M. G. H. Meyer D. E. Donchin E. (1993). A neural system for error detection and compensation. Psychol. Sci. 4 385– 390. 10.1118/1.4868459 24694128 · DOI
  18. Goodyear K. Parasuraman R. Chernyak S. de Visser E. Madhavan P. Deshpande G. (2017). An fMRI and effective connectivity study investigating miss errors during advice utilization from human and machine agents. Soc. Neurosci. 12 570– 581. 10.1080/17470919.2016.1205131 27409387 · DOI
  19. Goodyear K. Parasuraman R. Chernyak S. Madhavan P. Deshpande G. Krueger F. (2016). Advice taking from humans and machines: an fMRI and effective connectivity study. Front. Hum. Neurosci. 10: 542. 10.3389/fnhum.2016.00542 27867351 PMC5095979 · DOI
  20. Gramann K. Fairclough S. H. Zander T. O. Ayaz H. (2017). Editorial: trends in neuroergonomics. Front. Hum. Neurosci. 11: 165. 10.3389/fnhum.2017.00165 28424601 PMC5380744 · DOI
  21. Grissmann S. Spuler M. Faller J. Krumpe T. Zander T. Kelava A. (2017). Context sensitivity of EEG-based workload classification under different affective valence. IEEE Trans. Affect. Comput. 1. 10.1109/TAFFC.2017.2775616 · DOI
  22. Hoff K. Bashir M. (2015). Trust in automation: integrating empirical evidence on factors that influence trust. Hum. Factors 57 407– 434. 10.1177/0018720814547570 25875432 · DOI · Paper page
  23. King-Casas B. Tomlin D. Anen C. Camerer C. F. Quartz S. R. Montague P. R. (2005). Getting to know you: reputation and trust in a two-person economic exchange. Science 308 78– 83. 10.1126/science.1108062 15802598 · DOI · Paper page
  24. Koban L. Pourtois G. (2014). Brain systems underlying the affective and social monitoring of actions: an integrative review. Neurosci. Biobehav. Rev. 46 71– 84. 10.1016/j.neubiorev.2014.02.014 24681006 · DOI
  25. Krach S. Hegel F. Wrede B. Sagerer G. Binkofski F. Kircher T. (2008). Can machines think? Interaction and perspective taking with robots investigated via fMRI. PLoS One 3: e2597. 10.1371/journal.pone.0002597 18612463 PMC2440351 · DOI
  26. Krueger F. McCabe K. Moll J. Kriegeskorte N. Zahn R. Strenziok M. (2007). Neural correlates of trust. Proc. Natl. Acad. Sci. U.S.A. 104 20084– 20089. 10.1073/pnas.0710103104 18056800 PMC2148426 · DOI
  27. Lee J. Moray N. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35 1243– 1270. 10.1080/00140139208967392 1516577 · DOI · Paper page
  28. Lyons J. Havig P. (2014). “Transparency in a human-machine context: approaches for fostering shared awareness/intent,” in Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments. VAMR 2014. Lecture Notes in Computer Science Vol. 8525 eds Shumaker R. Lackey S. (Cham: Springer), 10.1007/978-3-319-07458-0_18 · DOI · Paper page
  29. Madhavan P. Wiegmann D. (2005). Cognitive anchoring on self-generated decisions reduces operator reliance on automated diagnostic aids. Hum. Factors 47 332– 341. 10.1518/0018720054679489 16170942 · DOI
  30. Madhavan P. Wiegmann D. (2007a). Effects of information source, pedigree, and reliability on operator interaction with decision support system. Hum. Factors 49 773– 785. 10.1518/001872007X230154 17915596 · DOI
  31. Madhavan P. Wiegmann D. (2007b). Similarities and differences between human–human and human–automation trust: an integrative review. Theor. Issues Ergon. Sci. 8 277– 301. 10.1080/14639220500337708 · DOI · Paper page
  32. Madhavan P. Wiegmann D. Lacson F. (2006). Automation failures on tasks easily performed by operators undermine trust in automated aids. Hum. Factors 48 241– 256. 10.1518/001872006777724408 16884046 · DOI · Paper page
  33. Mayer R. Davis J. Schoorman F. (1995). An integrative model of organizational trust. Acad. Manag. Rev. 20 709– 734. 10.5465/amr.1995.9508080335 · DOI · Paper page
  34. Merritt S. Heimbaugh H. LaChapell J. Lee D. (2013). I trust it, but I don’t know why: effects of implicit attitudes toward automation on trust in an automated system. Hum. Factors 55 520– 534. 10.1177/0018720812465081 23829027 · DOI
  35. Merritt S. Lee D. Unnerstall J. L. Huber K. (2014). Are well-calibrated users effective users? Associations between calibration of trust and performance on an automation-aided task. Hum. Factors 57 34– 47. 10.1177/0018720814561675 25790569 · DOI · Paper page
  36. Merritt S. Unnerstall J. L. Lee D. Huber K. (2015). Measuring individual differences in the perfect automation schema. Hum. Factors 57 740– 753. 10.1177/0018720815581247 25882304 · DOI
  37. Mirnig N. Stollnberger G. Miksch M. Stadler S. Giuliani M. Tscheligi M. (2017). To Err is robot: how humans assess and act toward an erroneous social robot. Front. Robot. AI 4: 21 10.3389/frobt.2017.00021 · DOI · Paper page
  38. Moray N. (2000). Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. J. Exp. Psychol. Appl. 6 44– 58. 10.1037//0278-7393.6.1.44 10937311 · DOI
  39. Parasuraman R. (2003). Neuroergonomics: research and practice. Theor. Issues Ergon. Sci. 4 5– 20. 10.1080/14639220210199753 · DOI
  40. Parasuraman R. (2011). Neuroergonomics: brain, cognition, and performance at work. Curr. Dir. Psychol. Sci. 20 181– 186. 10.1177/0963721411409176 · DOI
  41. Pop V. L. Shrewsbury A. Durso F. T. (2015). Individual differences in the calibration of trust in automation. Hum. Factors 57 545– 556. 10.1177/0018720814564422 25977317 · DOI · Paper page
  42. Prinzel L. J. Freeman F. G. Scerbo M. Mikulka P. J. Pope A. T. (2003). Effects of a psychophysiological system for adaptive automation on performance, workload, and the event-related potential P300 component. Hum. Factors 45 601– 614. 10.1518/hfes.45.4.601.27092 15055457 · DOI
  43. Rice S. (2009). Examining single- and multiple-process theories of trust in automation. J. Gen. Psychol. 136 303– 319. 10.3200/GENP.136.3.303-322 19650524 · DOI
  44. Rovira E. McGarry K. Parasuraman R. (2007). Effects of imperfect automation on decision making in a simulated command and control task. Hum. Factors 49 76– 87. 10.1518/001872007779598082 17315845 · DOI
  45. Salazar-Gomez A. F. Delpreto J. Gil S. Guenther F. H. Rus D. (2017). “Correcting robot mistakes in real time using EEG signals,” in Proceeding of the International Conference on Robotics and Automation, (Piscataway, NJ: IEEE). 10.1109/ICRA.2017.7989777 · DOI
  46. Sarter N. Woods D. Billings C. (1997). “Automation surprises,” in Handbook of Human Factors & Ergonomics, 2nd Edn, ed. Salvendy G. (Hoboken, NJ: Wiley Publishing), 1926– 1943.
  47. Scerbo M. Freeman F. G. Mikulka P. J. (2003). A brain-based system for adaptive automation. Theor. Issues Ergon. Sci. 4 200– 219. 10.1080/1463922021000020891 · DOI
  48. Somon B. Campagne A. Delorme A. Berberian B. (2017). Performance monitoring applied to system supervision. Front. Hum. Neurosci. 11: 360. 10.3389/fnhum.2017.00360 28744209 PMC5504305 · DOI
  49. Steinhauser M. Yeung N. (2010). Decision processes in human performance monitoring. J. Neurosci. 30 15643– 15653. 10.1523/JNEUROSCI.1899-10.2010 21084620 PMC3073548 · DOI · Paper page
  50. Ullsperger M. Fischer A. G. Nigbur R. Endrass T. (2014b). Neural mechanisms and temporal dynamics of performance monitoring. Trends Cogn. Sci. 18 259– 267. 10.1016/j.tics.2014.02.009 24656460 · DOI
  51. van Schie H. T. Mars R. B. Coles M. G. H. Bekkering H. (2004). Modulation of activity in medial frontal and motor cortices during error observation. Nat. Neurosci. 7 549– 554. 10.1038/nn1239 15107858 · DOI
  52. Wessel J. R. Danielmeier C. Morton J. B. Ullsperger M. (2012). Surprise and error: common neuronal architecture for the processing of errors and novelty. J. Neurosci. 32 7528– 7537. 10.1523/JNEUROSCI.6352-11.2012 22649231 PMC6703591 · DOI
  53. Wessel J. R. (2017). An adaptive orienting theory of error processing. Psychophysiology 55: e13041. 10.1111/psyp.13041 29226960 · DOI
  54. Wiese E. Metta G. Wykowska A. (2017). Robots as intentional agents: using neuroscientific methods to make robots appear more social. Front. Psychol. 8: 1663. 10.3389/fpsyg.2017.01663 29046651 PMC5632653 · DOI
  55. Winkler I. Debener S. Müller K. R. Tangermann M. (2015). “On the influence of high-pass filtering on ICA-based artifact reduction in EEG-ERP,” in Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, 4101– 4105. 10.1109/EMBC.2015.7319296 26737196 · DOI
  56. Zander T. Kothe C. (2011). Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J. Neural Eng. 8: 025005. 10.1088/1741-2560/8/2/025005 21436512 · DOI
  57. Zander T. Kothe C. Jatzev S. Gaertner M. (2010). “Enhancing human-computer interaction with input from active and passive brain-computer interfaces,” in Brain-Computer Interfaces. Human-Computer Interaction Series, eds Tan D. Nijholt A. (London: Springer).
  58. Zander T. Krol L. R. Birbaumer N. P. Gramann K. (2016). Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activity. Proc. Natl. Acad. Sci. U.S.A. 113 14898– 14903. 10.1073/pnas.1605155114 27956633 PMC5206562 · DOI

Cited by

Sorted by citations

Authors
de Visser, E. J., Beatty, P. J., Estepp, J. R., Kohn, S., Abubshait, A., Fedota, J. R., & McDonald, C. G.
Affiliations
  • George Mason University (US)
  • United States Air Force Research Laboratory (US)
  • National Institutes of Health (US)
ORCID
Venue
Frontiers in Human Neuroscience
Publisher
Frontiers Media SA
Published
2018-08-10
Volume
12
Pages
309-309
ISSN
1662-5161 (ISSN-L)
DOI
10.3389/fnhum.2018.00309
PMC
PMC6095965
OpenAlex
W2886380230
Cited by
73
References
16
Citations by year
2026: 10 · 2025: 16 · 2024: 12 · 2023: 9 · 2022: 6 · 2021: 12 · 2020: 4 · 2019: 4
FWCI
6.77
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Human Neuroscience (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Air Force Office of Scientific Research (15RHCOR234, 16RT0881)
Topics
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Keywords
trust calibration, error monitoring, EEG, flanker task, algorithm reliability, human-AI interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review