How do visual explanations foster end users' appropriate trust in machine learning?
Published version of a preprint
The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index.
Abstract
We investigated the effects of example-based explanations for a machine learning classifier on end users' appropriate trust. We explored the effects of spatial layout and visual representation in an in-person user study with 33 participants. We measured participants' appropriate trust in the classifier, quantified the effects of different spatial layouts and visual representations, and observed changes in users' trust over time. The results show that each explanation improved users' trust in the classifier, and the combination of explanation, human, and classification algorithm yielded much better decisions than the human and classification algorithm separately. Yet these visual explanations lead to different levels of trust and may cause inappropriate trust if an explanation is difficult to understand. Visual representation and performance feedback strongly affect users' trust, and spatial layout shows a moderate effect. Our results do not support that individual differences (e.g., propensity to trust) affect users' trust in the classifier. This work advances the state-of-the-art in trust-able machine learning and informs the design and appropriate use of automated systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, many empirical studies on human-AI decision-making did not observe complementary performance [6,16,37,52,58,61,68,82,84,87].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [84]
In the citing paper
“Existing literature is ambiguous with regard to effects of explanations [3, 6, 74]: While in some experiments, explanations support AR [74, 75], in others they cause 'blind trust' [3, 6] in AI advice.”
From Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations (2023) · cites this paper as [75]
In the citing paper
“In addition, some studies also employed linear trust meters [137], or visual analog scales [139].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [137]
In the citing paper
“A third way to measure the trust calibration was to measure over-, under-, and appropriate trust as the behavioral deviation (reliance) from the ideal behavior [27,50,55,104,110,132,137].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [137]
In the citing paper
“In contrast, Yang et al. [137] found that visual explanations supported an appropriate trust calibration.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [137]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 1×
“While many definitions and models of trust were proposed[81, 119] in different contexts (e.g., human-human[70, 77], human-automation[37, 62, 93], and e-commerce[32, 64]), we use the definition for human-computer trust from Madsen and Gregor[72] which was adapted from McAllister[78]: “…”
MethodGeurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300 · cited 1×
“…coefficients for a linear support-vector classifier[28], showing the distribution of selected features across the random trees for the importance of features in an extra-trees classifier[33], and explaining the inner working of an artificial neural network for overall variable contributions[95].”
BackgroundScikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
Affect and Cognition Based Trust as Foundations for Interpersonal Cooperation in Organizations
McAllister, D. J. (1995) · Academy of Management Journal · doi:10.2307/256727 · Cited by 7,346
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 5×
“While many definitions and models of trust were proposed[81, 119] in different contexts (e.g., human-human[70, 77], human-automation[37, 62, 93], and e-commerce[32, 64]), we use the definition for human-computer trust from Madsen and Gregor[72] which was adapted from McAllister[78]: “…”
BackgroundMethodInfluentialDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“Other techniques delved into the black-box to explain the classifier[106], assisted machine learning experts to understand the classifier’s internal representation[26, 73, 124], and deep neural networks’ architecture[50, 121].”
MethodExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
E-commerce: the role of familiarity and trust.
Gefen, D. (2000) · Omega · doi:10.1016/s0305-0483(00)00021-9 · Cited by 3,529 · cited 1×
“We referred to two ways in the literature to measure trust: (1) directly reporting the level of trust using subjective rating scales (e.g.,[14, 29, 31, 60, 126]); and (2) measuring through a multi-item questionnaire (e.g.,[47, 78, 82, 85, 87, 100, 104]).”
MethodAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 2×
“Empirically, example-based explanations helped users with diverse backgrounds better understand and access an intelligent system[8, 11, 18].”
Background
Research works printed but not linked: 32
- Jaegul Choo and Shixia Liu . 2018. Visual Analytics for Explainable Deep Learning. arXiv preprint arXiv:1804.02527 ( 2018 ). Jaegul Choo and Shixia Liu. 2018. Visual Analytics for Explainable Deep Learning. arXiv preprint arXiv:1804.02527 (2018). · arXiv
- Sven Coppers , Jan Van den Bergh , Kris Luyten, Karin Coninx, Iulianna van der Lek-Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018 . Intellingo : An Intelligible Translation Environment (CHI '18). ACM, New York, NY, USA, Article 524, 13 pages. Sven Coppers, Jan Van den Bergh, Kris Luyten, Karin Coninx, Iulianna van der Lek-Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018. Intellingo: An Intelligible Translation Environment (CHI '18). ACM, New York, NY, USA, Article 524, 13 pages.
- Finale Doshi-Velez and Been Kim . 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 ( 2017 ). Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017). · arXiv
- John Ellson , Emden Gansner , Lefteris Koutsofios , Stephen C North , and Gordon Woodhull . 2001 . Graphviz---open source graph drawing tools . In International Symposium on Graph Drawing. Springer, 483--484 . John Ellson, Emden Gansner, Lefteris Koutsofios, Stephen C North, and Gordon Woodhull. 2001. Graphviz---open source graph drawing tools. In International Symposium on Graph Drawing. Springer, 483--484.
- Amos Freedy , Ewart DeVisser , Gershon Weltman , and Nicole Coeyman . 2007 . Measurement of trust in human-robot collaboration. In Collaborative Technologies and Systems , International Symposium on. IEEE, 106--114 . Amos Freedy, Ewart DeVisser, Gershon Weltman, and Nicole Coeyman. 2007. Measurement of trust in human-robot collaboration. In Collaborative Technologies and Systems, International Symposium on. IEEE, 106--114.
- David Gefen , Elena Karahanna , and Detmar W Straub . 2003. Trust and TAM in online shopping: An integrated model. MIS quarterly 27, 1 ( 2003 ), 51--90. David Gefen, Elena Karahanna, and Detmar W Straub. 2003. Trust and TAM in online shopping: An integrated model. MIS quarterly 27, 1 (2003), 51--90.
- Ian J Goodfellow , Jonathon Shlens , and Christian Szegedy . 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 ( 2014 ). Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014). · arXiv
- David Gunning . 2017. Explainable artificial intelligence (XAI) . Defense Advanced Research Projects Agency (DARPA) , 2nd Web ( 2017 ). David Gunning. 2017. Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), 2nd Web (2017).
- 10.1037/0278-7393.25.4.923 · doi:10.1037/0278-7393.25.4.923 · Paper page
- Mohammad T Khasawneh , Shannon R Bowling , Xiaochun Jiang , Anand K Gramopadhye , and Brian J Melloy . 2003. A model for predicting human trust in automated systems. Origins 5 ( 2003 ). Mohammad T Khasawneh, Shannon R Bowling, Xiaochun Jiang, Anand K Gramopadhye, and Brian J Melloy. 2003. A model for predicting human trust in automated systems. Origins 5 (2003).
- Sherrie YX Komiak and Izak Benbasat . 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly ( 2006 ), 941--960. Sherrie YX Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly (2006), 941--960.
- Josua Krause , Aritra Dasgupta , Jordan Swartz , Yindalon Aphinyanaphongs , and Enrico Bertini . 2017. A workflow for visual diagnostics of binary classifiers using instance-level explanations. arXiv preprint arXiv:1705.01968 ( 2017 ). Josua Krause, Aritra Dasgupta, Jordan Swartz, Yindalon Aphinyanaphongs, and Enrico Bertini. 2017. A workflow for visual diagnostics of binary classifiers using instance-level explanations. arXiv preprint arXiv:1705.01968 (2017). · arXiv
- Josua Krause , Adam Perer , and Enrico Bertini . 2018 . A user study on the effect of aggregating explanations for interpreting machine learning models . In KDD Workshop on Interactive Data Exploration and Analytics (IDEA). Josua Krause, Adam Perer, and Enrico Bertini. 2018. A user study on the effect of aggregating explanations for interpreting machine learning models. In KDD Workshop on Interactive Data Exploration and Analytics (IDEA).
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- Brian Y Lim , Qian Yang , Ashraf M Abdul , and Danding Wang . 2019 . Why these Explanations? Selecting Intelligibility Types for Explanation Goals. In IUI Workshops. Brian Y Lim, Qian Yang, Ashraf M Abdul, and Danding Wang. 2019. Why these Explanations? Selecting Intelligibility Types for Explanation Goals. In IUI Workshops.
- Maria Madsen and Shirley Gregor . 2000 . Measuring human-computer trust . In 11th Australasian Conference on Information Systems , Vol. 53 . Citeseer, 6--8. Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In 11th Australasian Conference on Information Systems, Vol. 53. Citeseer, 6--8.
- 10.5555/1077316.1077319 · doi:10.5555/1077316.1077319
- 10.5465/amr.1995.9508080335 · doi:10.5465/amr.1995.9508080335 · Paper page
- Maranda McBride and Shona Morgan . 2010. Trust calibration for automated decision aids . Institute for Homeland Security Solutions (2010) . Maranda McBride and Shona Morgan. 2010. Trust calibration for automated decision aids. Institute for Homeland Security Solutions (2010).
- D Harrison McKnight and Norman L Chervany . 2001. Trust and distrust definitions: One bite at a time . In Trust in Cyber-societies . Springer , 27--54. D Harrison McKnight and Norman L Chervany. 2001. Trust and distrust definitions: One bite at a time. In Trust in Cyber-societies. Springer, 27--54.
- D Harrison McKnight , Vivek Choudhury , and Charles Kacmar . 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research 13, 3 ( 2002 ), 334--359. D Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research 13, 3 (2002), 334--359.
- David L McLain and Katarina Hackman . 1999. Trust , risk, and decision-making in organizational change. Public Administration Quarterly ( 1999 ), 152--176. David L McLain and Katarina Hackman. 1999. Trust, risk, and decision-making in organizational change. Public Administration Quarterly (1999), 152--176.
- Stephanie M Merritt , Heather Heimbaugh , Jennifer LaChapell , and Deborah Lee . 2013. I trust it, but I don't know why: Effects of implicit attitudes toward automation on trust in an automated system. Human factors 55, 3 ( 2013 ), 520--534. Stephanie M Merritt, Heather Heimbaugh, Jennifer LaChapell, and Deborah Lee. 2013. I trust it, but I don't know why: Effects of implicit attitudes toward automation on trust in an automated system. Human factors 55, 3 (2013), 520--534.
- Christopher A Miller . 2005 . Trust in adaptive automation: the role of etiquette in tuning trust via analogic and affective methods . In Proceedings of the 1st International Conference on Augmented Cognition. Citeseer, 22--27 . Christopher A Miller. 2005. Trust in adaptive automation: the role of etiquette in tuning trust via analogic and affective methods. In Proceedings of the 1st International Conference on Augmented Cognition. Citeseer, 22--27.
- Raja Parasuraman and Victor Riley . 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors 39, 2 ( 1997 ), 230--253. Raja Parasuraman and Victor Riley. 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors 39, 2 (1997), 230--253.
- Simone Stumpf , Adrian Bussone , and Dympna O'Sullivan . 2016. Explanations considered harmful? user interactions with machine learning systems . In Human Centred Machine Learning at CHI 2016 . Simone Stumpf, Adrian Bussone, and Dympna O'Sullivan. 2016. Explanations considered harmful? user interactions with machine learning systems. In Human Centred Machine Learning at CHI 2016.
- Sandra Wachter Brent Mittelstadt and Chris Russell. [n.d.]. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GPDR. Harvard Journal of Law & Technology 31 2 ([n. d.]). Sandra Wachter Brent Mittelstadt and Chris Russell. [n.d.]. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GPDR. Harvard Journal of Law & Technology 31 2 ([n. d.]).
- Dayong Wang , Aditya Khosla , Rishab Gargeya , Humayun Irshad , and Andrew H Beck . 2016. Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 ( 2016 ). Dayong Wang, Aditya Khosla, Rishab Gargeya, Humayun Irshad, and Andrew H Beck. 2016. Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016). · arXiv
- 10.5555/2906831.2906852 · doi:10.5555/2906831.2906852
- Scott Cheng-Hsin Yang and Patrick Shafto . 2017 . Explainable artificial intelligence via bayesian teaching . In NIPS 2017 Workshop on Teaching Machines, Robots, and Humans. Scott Cheng-Hsin Yang and Patrick Shafto. 2017. Explainable artificial intelligence via bayesian teaching. In NIPS 2017 Workshop on Teaching Machines, Robots, and Humans.
- Jason Yosinski , Jeff Clune , Anh Nguyen , Thomas Fuchs , and Hod Lipson . 2015. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 ( 2015 ). Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 (2015). · arXiv
- Jason Yosinski , Jeff Clune , Anh Mai Nguyen , Thomas J. Fuchs, and Hod Lipson. 2015 . Understanding Neural Networks Through Deep Visualization. CoRR abs/1506.06579 (2015). arXiv:1506.06579 Jason Yosinski, Jeff Clune, Anh Mai Nguyen, Thomas J. Fuchs, and Hod Lipson. 2015. Understanding Neural Networks Through Deep Visualization. CoRR abs/1506.06579 (2015). arXiv:1506.06579 · arXiv
Cited by
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
“mes paired with importance weights [8, 21, 47, 52, 63]. On vision tasks, some work takes the demonstrate-by-example approach, i.e., by explaining an example with its closest match in the training set [48, 67]. These two methods are typically less suitable for domains involving unstructured text, as the text features a…”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
“mes paired with importance weights [8, 21, 47, 52, 63]. On vision tasks, some work takes the demonstrate-by-example approach, i.e., by explaining an example with its closest match in the training set [48, 67]. These two methods are typically less suitable for domains involving unstructured text, as the text features a…”
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538
“There have been rather mixed results on whether current XAI techniques could appropriately enhance user trust [15, 80, 107] or the intended task performance, whether for decision making [12, 58, 111], model evaluation [6, 13, 22], or model development [50].”
ResultExploring students' perspectives on generative AI-assisted academic writing
Kim, J., Yu, S., Detrick, R., & Li, N. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12878-7 · Cited by 438
“…perspectives and demonstrate a range of explainable AI techniques and methodologies to better assist in sensemaking (Alqaraawi et al., 2020), enhancing user trust (Yang et al., 2020), or enabling shared decision making between student and AI system (Kim et al., 2024; Liao et al., 2020,).”
Metadata
- Authors
- Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L.
- Affiliations
- Brown University (US)
- Pacific Northwest National Laboratory (US)
- Venue
- Proceedings of the 25th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2020-03-17
- Pages
- 189-201
- OpenAlex
- W3009578469
- Cited by
- 200
- References
- 97
- Citations by year
- 2026: 21 · 2025: 29 · 2024: 56 · 2023: 34 · 2022: 25 · 2021: 30 · 2020: 4
- FWCI
- 8.87
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Keywords
- explainable machine learning, example-based explanations, trust in machine learning, spatial configuration, visual representation, performance feedback, human-machine decision-making, end users
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How do visual explanations foster end users' appropriate trust in machine learning? https://app.cochat.ai/papers/10.1145/3377325.3377480
- The paper
- Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020). How do visual explanations foster end users' appropriate trust in machine learning? Proceedings of the 25th International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3377325.3377480
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.