I can do better than your AI: expertise and explanations
The DOI matches its Crossref record; 59 of the 60 references it cites were found in a registry or an index.
Abstract
Intelligent assistants, such as navigation, recommender, and expert systems, are most helpful in situations where users lack domain knowledge. Despite this, recent research in cognitive psychology has revealed that lower-skilled individuals may maintain a sense of illusory superiority, which might suggest that users with the highest need for advice may be the least likely to defer judgment. Explanation interfaces - a method for persuading users to take a system's advice - are thought by many to be the solution for instilling trust, but do their effects hold for self-assured users? To address this knowledge gap, we conducted a quantitative study (N=529) wherein participants played a binary decision-making game with help from an intelligent assistant. Participants were profiled in terms of both actual (measured) expertise and reported familiarity with the task concept. The presence of explanations, level of automation, and number of errors made by the intelligent assistant were manipulated while observing changes in user acceptance of advice. An analysis of cognitive metrics lead to three findings for research in intelligent assistants: 1) higher reported familiarity with the task simultaneously predicted more reported trust but less adherence, 2) explanations only swayed people who reported very low task familiarity, and 3) showing explanations to people who reported more task familiarity led to automation bias.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“several studies suggest that overloading users with information about the system could potentially harm people's situation awareness and lead to worse performance or decision-making outcome [6, 27, 29].”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [29]
In the citing paper
“Prior work suggests that subjective self-reported trust may not be a reliable indicator for trusting behaviors [16, 29], which are what ultimately matter in AI-assisted decision tasks.”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [29]
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Axelrod, R., & Hamilton, W. D. (1981) · Science · doi:10.1126/science.7466396 · Cited by 20,065
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Making sense of cronbach’s alpha.
Tavakol, M., & Dennick, R. (2011) · International Journal of Medical Education · doi:10.5116/ijme.4dfb.8dfd · Cited by 15,126
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731
The structure of phenotypic personality traits.
Goldberg, L. R. (1993) · American Psychologist · doi:10.1037/0003-066x.48.1.26 · Cited by 4,816 · cited 1×
“There are still more intricacies to explanations, for instance, recent research in intelligent user interfaces [4] has found that explanations interact with user characteristics [25] to affect trusting beliefs about a recommender system, implying that not all explanations are equal for all users.”
BackgroundLord, C. G., Ross, L., & Lepper, M. R. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.11.2098 · Cited by 4,583 · cited 1×
“Another option is to appeal to emotions [22], rather than rationality [5, 44].”
Background(2011) · Springer eBooks · doi:10.1007/978-0-387-85820-3 · Cited by 3,761
Eppler, M. J., & Mengis, J. (2008) · Kommunikationsmanagement im Wandel · doi:10.1007/978-3-8349-9772-2_15 · Cited by 2,177
Situation Awareness Analysis and Measurement
(2000) · CRC Press eBooks · doi:10.1201/b12461 · Cited by 2,046
Persuasive technology: using computers to change what we think and do
Fogg, B. J. (2002) · Ubiquity · doi:10.1145/764008.763957 · Cited by 1,554 · cited 1×
“Another option is to appeal to emotions [22], rather than rationality [5, 44].”
BackgroundTrust in a specific technology: An investigation of its components and measures
Mcknight, D. H., Carter, M., Thatcher, J. B., & Clay, P. F. (2011) · ACM Transactions on Management Information Systems · doi:10.1145/1985347.1985353 · Cited by 1,185 · cited 1×
“More generally, research by Mcknight ties the concepts of trust in technology and trust in other people [46], showing that people can discriminate between the two.”
BackgroundSituation awareness global assessment technique (SAGAT)
Endsley, M. R. (2003) · Proceedings of the IEEE 1988 National Aerospace and Electronics Conference · doi:10.1109/naecon.1988.195097 · Cited by 956 · cited 1×
“On the negative side of things, theories of human awareness [17] and multitasking [23] predict that users who spend time perceiving and understanding explanations from intelligent assistants may have lower awareness of other objects in their environment (or must necessarily take more time to…”
BackgroundKruger, J. (1999) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.77.2.221 · Cited by 898
O'Donovan, J., & Smyth, B. (2005) · Proceedings of the 10th international conference on Intelligent user interfaces · doi:10.1145/1040830.1040870 · Cited by 865
Self-citationPrisoner's Dilemma: A Study in Conflict and Co-operation.
Flook, A. J. M., Rapoport, A., & Chammah, A. M. (1970) · The Philosophical Quarterly · doi:10.2307/2218423 · Cited by 855
Initial trust and online buyer behaviour
Chen, Y.-H., & Barnes, S. (2007) · Industrial Management & Data Systems · doi:10.1108/02635570710719034 · Cited by 841
Dunning, D. (2011) · Advances in Experimental Social Psychology · doi:10.1016/b978-0-12-385522-0.00005-6 · Cited by 806
(2002) · Cited by 770
Advances in Collaborative Filtering
Koren, Y., & Bell, R. (2011) · Recommender Systems Handbook · doi:10.1007/978-0-387-85820-3_5 · Cited by 745
Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice1
Gregor, S., & Benbasat, I. (1999) · MIS Quarterly · doi:10.2307/249487 · Cited by 535 · cited 1×
“Research into explanations started with expert systems studies [1, 28], and more recently their trust and persuasion effects have been studied in recommender systems [36, 50].”
BackgroundHuman–Agent Teaming for Multirobot Control: A Review of Human Factors Issues
Chen, J. Y. C., & Barnes, M. J. (2014) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2013.2293535 · Cited by 522
Personality, Organizational Culture, and Cooperation: Evidence from a Business Simulation
Chatman, J. A., & Barsade, S. G. (1995) · Administrative Science Quarterly · doi:10.2307/2393792 · Cited by 506
Recommendation Agents for Electronic Commerce: Effects of Explanation Facilities on Trusting Beliefs
Wang, W., & Benbasat, I. (2007) · Journal of Management Information Systems · doi:10.2753/mis0742-1222230410 · Cited by 498
Salem, M., Lakatos, G., Amirabdollahian, F., & Dautenhahn, K. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2696454.2696497 · Cited by 494 · cited 1×
“Although system effectiveness continues to improve (e.g. [39]), it is not conclusive that improved algorithms result in improved adherence to system recommendations, for instance, no effect of system error was found in Salem et al. [52].”
BackgroundCognitive Psychology: Connecting Mind, Research and Everyday Experience
Goldstein, E. B. (2004) · Cited by 486
A systematic review and taxonomy of explanations in decision support and recommender systems
Nunes, I., & Jannach, D. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-017-9195-0 · Cited by 395
Direct Measurement of Situation Awareness: Validity and Use of SAGAT
Endsley, M. R. (2017) · Situational Awareness · doi:10.4324/9781315087924-9 · Cited by 334
Situation Awareness-Based Agent Transparency
Chen, J. Y., Procci, K., Boyce, M., Wright, J., Garcia, A., & Barnes, M. (2014) · doi:10.21236/ada600351 · Cited by 293
Self-enhancement and Superiority Biases in Social Comparison
Hoorens, V. (1993) · European Review of Social Psychology · doi:10.1080/14792779343000040 · Cited by 273
Automation Bias in Intelligent Time Critical Decision Support Systems
Cummings, M. (2004) · AIAA 1st Intelligent Systems Technical Conference · doi:10.2514/6.2004-6313 · Cited by 269 · cited 3×
“Effects related to automation bias [13] were reproduced here.”
BackgroundInfluentialAutomation bias in intelligent time critical decision support systems
Cummings, M. L. (2017) · Decision Making in Aviation · doi:10.4324/9781315095080-17 · Cited by 233
Lin, G.-C., Wen, Z., Marsh, H., & Lin, H.-S. (2010) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2010.488999 · Cited by 216
The Dynamics of Social Dilemmas
Glance, N. S., & Huberman, B. A. (1994) · Scientific American · doi:10.1038/scientificamerican0394-76 · Cited by 210
Engeln–Maddox, R. (2005) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.2005.24.8.1114 · Cited by 187
Explainable Artificial Intelligence (XAI)
Sewada, R., Jangid, A., Kumar, P., & Mishra, N. (2023) · Journal of Nonlinear Analysis and Optimization · doi:10.36893/jnao.2022.v13i02.041-047 · Cited by 172
Arnold, V., Clark, N., Collier, P. A., Leech, S. A., & Sutton, S. G. (2006) · MIS Quarterly · doi:10.2307/25148718 · Cited by 166
Multiplicity Control in Structural Equation Modeling
Cribbie, R. A. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510709336738 · Cited by 158
Trust in information sources: Seeking information from people, documents, and virtual agents
Hertzum, M., Andersen, H. H. K., Andersen, V., & Hansen, C. B. (2002) · Interacting with Computers · doi:10.1016/s0953-5438(02)00023-1 · Cited by 157 · cited 1×
“These explanations could be considered “rational” in that they explain the mathematical or algorithmic properties of the systems they explain, which contrasts with more emotional explanations, such as those that are employed by virtual agents [31, 38].”
BackgroundCUSTOMER TRUST ONLINE: EXAMINING THE ROLE OF THE EXPERIENCE WITH THE WEB SITE
Koufaris, M., & Hampton-Sosa, W. (2002) · Cited by 150
Inspectability and control in social recommenders
Knijnenburg, B. P., Bostandjiev, S., O'Donovan, J., & Kobsa, A. (2012) · Proceedings of the sixth ACM conference on Recommender systems · doi:10.1145/2365952.2365966 · Cited by 137
Self-citationMaking Decisions about Privacy: Information Disclosure in Context-Aware Recommender Systems
Knijnenburg, B. P., & Kobsa, A. (2013) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2499670 · Cited by 135
Contemporary Models of Intelligence
Davidson, J. E., & Downing, C. L. (2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511807947.004 · Cited by 135
Lomax, R. G. (2018) · Routledge eBooks · doi:10.4324/9781315755649-34 · Cited by 109
Boddy, J., Carver, A., & Rowley, K. (1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90111-x · Cited by 101
Personality and cooperation in finitely repeated prisoner’s dilemma games
Kagel, J. H., & McGee, P. J. (2014) · Economics Letters · doi:10.1016/j.econlet.2014.05.034 · Cited by 91
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90
How to Recommend?: User Trust Factors in Movie Recommender Systems
Berkovsky, S., Taib, R., & Conway, D. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025209 · Cited by 77
AN INVESTIGATION OF SELF-REPORTED COMPUTER LITERACY: IS IT RELIABLE?
(2005) · Issues In Information Systems · doi:10.48009/1_iis_2005_289-295 · Cited by 75
(2006) · Springer eBooks · doi:10.1007/978-1-4020-4938-5 · Cited by 75
Division of Attention: The Single-Channel Hypothesis Revisited
Gladstones, W. H., Regan, M. A., & Lee, R. B. (1989) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748908402350 · Cited by 61
TRUST IN TECHNOLOGY: DEVELOPMENT OF A SET OF CONSTRUCTS AND MEASURES
McKnight, H., Carter, M., & Clay, P. F. (2009) · AIS Electronic Library (AISeL) · Cited by 56
Dynamics of human trust in recommender systems
Harman, J. L., O’Donovan, J., Abdelzaher, T., & González, C. (2014) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2645710.2645761 · Cited by 43
Self-citationType of Automation Failure: The Effects on Trust and Reliance in Automation
Johnson, J. D., Sanchez, J., Fisk, A. D., & Rogers, W. A. (2004) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120404801807 · Cited by 41 · cited 1×
“” This is sometimes called “overtrusting” [34] and can, in some instances, lead to catastrophic failure1.”
BackgroundHankins, F. H., & Reinheimer, H. (1914) · The Journal of Race Development · doi:10.2307/29738031 · Cited by 26
Physiological evidence for a dual process model of the social effects of emotion in computers
Choi, A., de Melo, C. M., Khooshabeh, P., Woo, W., & Gratch, J. S. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.10.006 · Cited by 19
A study of dynamic information display and decision-making in abstract trust games
Schaffer, J., O’Donovan, J., Marusich, L., Yu, M., Gonzalez, C., & Höllerer, T. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.01.002 · Cited by 10 · cited 2×
“Moreover, this version of the game has been previously validated in terms of understandability and information requirements [53].”
BackgroundSelf-citationEngineering Psychology and Cognitive Ergonomics: Volume Five - Aerospace and Transportation Systems
(2017) · Routledge eBooks · doi:10.4324/9781315094472 · Cited by 8
The Effects of Individual and Context on Aggression in Repeated Social Interaction
Martin, J. M., Juvina, I., Lebiere, C. J., & González, C. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21741-8_47 · Cited by 6
Research works printed but not linked: 4
- Robyn M Dawes and Richard H Thaler . 1988. Anomalies: cooperation. The Journal of Economic Perspectives ( 1988 ), 187--197. Robyn M Dawes and Richard H Thaler. 1988. Anomalies: cooperation. The Journal of Economic Perspectives (1988), 187--197.
- Mica R Endsley and Daniel J Garland . 2000. Situation awareness analysis and measurement . CRC Press . Mica R Endsley and Daniel J Garland. 2000. Situation awareness analysis and measurement. CRC Press.
- C Gonzalez , N Ben-Asher , JM Martin , and V Dutt . 2013. Emergence of cooperation with increased information: Explaining the process with instance-based learning models. Unpublished manuscript under review ( 2013 ). C Gonzalez, N Ben-Asher, JM Martin, and V Dutt. 2013. Emergence of cooperation with increased information: Explaining the process with instance-based learning models. Unpublished manuscript under review (2013).
- Sherrie YX Komiak and Izak Benbasat . 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly ( 2006 ), 941--960. Sherrie YX Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly (2006), 941--960.
Cited by
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Three Challenges for AI-Assisted Decision-Making
Steyvers, M., & Kumar, A. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231181102 · Cited by 217
Interactive and Explainable Region-guided Radiology Report Generation
Tanida, T., Müller, P., Kaissis, G., & Rueckert, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00718 · Cited by 205
Metadata
- Authors
- Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T.
- Affiliations
- DEVCOM Army Research Laboratory (US)
- University of California Santa Barbara
- University of California, Santa Barbara (US)
- Venue
- Proceedings of the 24th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2019-03-17
- Pages
- 240-251
- OpenAlex
- W2915124689
- Cited by
- 119
- References
- 60
- Citations by year
- 2026: 22 · 2025: 16 · 2024: 20 · 2023: 23 · 2022: 15 · 2021: 12 · 2020: 9 · 2019: 2
- FWCI
- 15.69
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- intelligent assistants, automation bias, trust in AI, task familiarity, advice acceptance, binary decision making, AI errors, levels of automation, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.