CoChat

How do visual explanations foster end users' appropriate trust in machine learning?

Yang, Fumeng, Huang, Zhuanyi, Scholtz, Jean, Arendt, Dustin L. · 2020 · Proceedings of the 25th International Conference on Intelligent User Interfaces

Published version of a preprint

The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index.

Abstract

We investigated the effects of example-based explanations for a machine learning classifier on end users' appropriate trust. We explored the effects of spatial layout and visual representation in an in-person user study with 33 participants. We measured participants' appropriate trust in the classifier, quantified the effects of different spatial layouts and visual representations, and observed changes in users' trust over time. The results show that each explanation improved users' trust in the classifier, and the combination of explanation, human, and classification algorithm yielded much better decisions than the human and classification algorithm separately. Yet these visual explanations lead to different levels of trust and may cause inappropriate trust if an explanation is difficult to understand. Visual representation and performance feedback strongly affect users' trust, and spatial layout shows a moderate effect. Our results do not support that individual differences (e.g., propensity to trust) affect users' trust in the classifier. This work advances the state-of-the-art in trust-able machine learning and informs the design and appropriate use of automated systems.

Overview

Integrity signals

Registry

Crossref · 10.1145/3377325.3377480

Cites

97 of 97 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 97 references

Research works printed but not linked: 32

  1. Jaegul Choo and Shixia Liu . 2018. Visual Analytics for Explainable Deep Learning. arXiv preprint arXiv:1804.02527 ( 2018 ). Jaegul Choo and Shixia Liu. 2018. Visual Analytics for Explainable Deep Learning. arXiv preprint arXiv:1804.02527 (2018). · arXiv
  2. Sven Coppers , Jan Van den Bergh , Kris Luyten, Karin Coninx, Iulianna van der Lek-Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018 . Intellingo : An Intelligible Translation Environment (CHI '18). ACM, New York, NY, USA, Article 524, 13 pages. Sven Coppers, Jan Van den Bergh, Kris Luyten, Karin Coninx, Iulianna van der Lek-Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018. Intellingo: An Intelligible Translation Environment (CHI '18). ACM, New York, NY, USA, Article 524, 13 pages.
  3. Finale Doshi-Velez and Been Kim . 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 ( 2017 ). Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017). · arXiv
  4. John Ellson , Emden Gansner , Lefteris Koutsofios , Stephen C North , and Gordon Woodhull . 2001 . Graphviz---open source graph drawing tools . In International Symposium on Graph Drawing. Springer, 483--484 . John Ellson, Emden Gansner, Lefteris Koutsofios, Stephen C North, and Gordon Woodhull. 2001. Graphviz---open source graph drawing tools. In International Symposium on Graph Drawing. Springer, 483--484.
  5. Amos Freedy , Ewart DeVisser , Gershon Weltman , and Nicole Coeyman . 2007 . Measurement of trust in human-robot collaboration. In Collaborative Technologies and Systems , International Symposium on. IEEE, 106--114 . Amos Freedy, Ewart DeVisser, Gershon Weltman, and Nicole Coeyman. 2007. Measurement of trust in human-robot collaboration. In Collaborative Technologies and Systems, International Symposium on. IEEE, 106--114.
  6. David Gefen , Elena Karahanna , and Detmar W Straub . 2003. Trust and TAM in online shopping: An integrated model. MIS quarterly 27, 1 ( 2003 ), 51--90. David Gefen, Elena Karahanna, and Detmar W Straub. 2003. Trust and TAM in online shopping: An integrated model. MIS quarterly 27, 1 (2003), 51--90.
  7. Ian J Goodfellow , Jonathon Shlens , and Christian Szegedy . 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 ( 2014 ). Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014). · arXiv
  8. David Gunning . 2017. Explainable artificial intelligence (XAI) . Defense Advanced Research Projects Agency (DARPA) , 2nd Web ( 2017 ). David Gunning. 2017. Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), 2nd Web (2017).
  9. 10.1037/0278-7393.25.4.923 · doi:10.1037/0278-7393.25.4.923 · Paper page
  10. Mohammad T Khasawneh , Shannon R Bowling , Xiaochun Jiang , Anand K Gramopadhye , and Brian J Melloy . 2003. A model for predicting human trust in automated systems. Origins 5 ( 2003 ). Mohammad T Khasawneh, Shannon R Bowling, Xiaochun Jiang, Anand K Gramopadhye, and Brian J Melloy. 2003. A model for predicting human trust in automated systems. Origins 5 (2003).
  11. Sherrie YX Komiak and Izak Benbasat . 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly ( 2006 ), 941--960. Sherrie YX Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly (2006), 941--960.
  12. Josua Krause , Aritra Dasgupta , Jordan Swartz , Yindalon Aphinyanaphongs , and Enrico Bertini . 2017. A workflow for visual diagnostics of binary classifiers using instance-level explanations. arXiv preprint arXiv:1705.01968 ( 2017 ). Josua Krause, Aritra Dasgupta, Jordan Swartz, Yindalon Aphinyanaphongs, and Enrico Bertini. 2017. A workflow for visual diagnostics of binary classifiers using instance-level explanations. arXiv preprint arXiv:1705.01968 (2017). · arXiv
  13. Josua Krause , Adam Perer , and Enrico Bertini . 2018 . A user study on the effect of aggregating explanations for interpreting machine learning models . In KDD Workshop on Interactive Data Exploration and Analytics (IDEA). Josua Krause, Adam Perer, and Enrico Bertini. 2018. A user study on the effect of aggregating explanations for interpreting machine learning models. In KDD Workshop on Interactive Data Exploration and Analytics (IDEA).
  14. 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
  15. Brian Y Lim , Qian Yang , Ashraf M Abdul , and Danding Wang . 2019 . Why these Explanations? Selecting Intelligibility Types for Explanation Goals. In IUI Workshops. Brian Y Lim, Qian Yang, Ashraf M Abdul, and Danding Wang. 2019. Why these Explanations? Selecting Intelligibility Types for Explanation Goals. In IUI Workshops.
  16. Maria Madsen and Shirley Gregor . 2000 . Measuring human-computer trust . In 11th Australasian Conference on Information Systems , Vol. 53 . Citeseer, 6--8. Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In 11th Australasian Conference on Information Systems, Vol. 53. Citeseer, 6--8.
  17. 10.5555/1077316.1077319 · doi:10.5555/1077316.1077319
  18. 10.5465/amr.1995.9508080335 · doi:10.5465/amr.1995.9508080335 · Paper page
  19. Maranda McBride and Shona Morgan . 2010. Trust calibration for automated decision aids . Institute for Homeland Security Solutions (2010) . Maranda McBride and Shona Morgan. 2010. Trust calibration for automated decision aids. Institute for Homeland Security Solutions (2010).
  20. D Harrison McKnight and Norman L Chervany . 2001. Trust and distrust definitions: One bite at a time . In Trust in Cyber-societies . Springer , 27--54. D Harrison McKnight and Norman L Chervany. 2001. Trust and distrust definitions: One bite at a time. In Trust in Cyber-societies. Springer, 27--54.
  21. D Harrison McKnight , Vivek Choudhury , and Charles Kacmar . 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research 13, 3 ( 2002 ), 334--359. D Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research 13, 3 (2002), 334--359.
  22. David L McLain and Katarina Hackman . 1999. Trust , risk, and decision-making in organizational change. Public Administration Quarterly ( 1999 ), 152--176. David L McLain and Katarina Hackman. 1999. Trust, risk, and decision-making in organizational change. Public Administration Quarterly (1999), 152--176.
  23. Stephanie M Merritt , Heather Heimbaugh , Jennifer LaChapell , and Deborah Lee . 2013. I trust it, but I don't know why: Effects of implicit attitudes toward automation on trust in an automated system. Human factors 55, 3 ( 2013 ), 520--534. Stephanie M Merritt, Heather Heimbaugh, Jennifer LaChapell, and Deborah Lee. 2013. I trust it, but I don't know why: Effects of implicit attitudes toward automation on trust in an automated system. Human factors 55, 3 (2013), 520--534.
  24. Christopher A Miller . 2005 . Trust in adaptive automation: the role of etiquette in tuning trust via analogic and affective methods . In Proceedings of the 1st International Conference on Augmented Cognition. Citeseer, 22--27 . Christopher A Miller. 2005. Trust in adaptive automation: the role of etiquette in tuning trust via analogic and affective methods. In Proceedings of the 1st International Conference on Augmented Cognition. Citeseer, 22--27.
  25. Raja Parasuraman and Victor Riley . 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors 39, 2 ( 1997 ), 230--253. Raja Parasuraman and Victor Riley. 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors 39, 2 (1997), 230--253.
  26. Simone Stumpf , Adrian Bussone , and Dympna O'Sullivan . 2016. Explanations considered harmful? user interactions with machine learning systems . In Human Centred Machine Learning at CHI 2016 . Simone Stumpf, Adrian Bussone, and Dympna O'Sullivan. 2016. Explanations considered harmful? user interactions with machine learning systems. In Human Centred Machine Learning at CHI 2016.
  27. Sandra Wachter Brent Mittelstadt and Chris Russell. [n.d.]. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GPDR. Harvard Journal of Law & Technology 31 2 ([n. d.]). Sandra Wachter Brent Mittelstadt and Chris Russell. [n.d.]. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GPDR. Harvard Journal of Law & Technology 31 2 ([n. d.]).
  28. Dayong Wang , Aditya Khosla , Rishab Gargeya , Humayun Irshad , and Andrew H Beck . 2016. Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 ( 2016 ). Dayong Wang, Aditya Khosla, Rishab Gargeya, Humayun Irshad, and Andrew H Beck. 2016. Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016). · arXiv
  29. 10.5555/2906831.2906852 · doi:10.5555/2906831.2906852
  30. Scott Cheng-Hsin Yang and Patrick Shafto . 2017 . Explainable artificial intelligence via bayesian teaching . In NIPS 2017 Workshop on Teaching Machines, Robots, and Humans. Scott Cheng-Hsin Yang and Patrick Shafto. 2017. Explainable artificial intelligence via bayesian teaching. In NIPS 2017 Workshop on Teaching Machines, Robots, and Humans.
  31. Jason Yosinski , Jeff Clune , Anh Nguyen , Thomas Fuchs , and Hod Lipson . 2015. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 ( 2015 ). Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 (2015). · arXiv
  32. Jason Yosinski , Jeff Clune , Anh Mai Nguyen , Thomas J. Fuchs, and Hod Lipson. 2015 . Understanding Neural Networks Through Deep Visualization. CoRR abs/1506.06579 (2015). arXiv:1506.06579 Jason Yosinski, Jeff Clune, Anh Mai Nguyen, Thomas J. Fuchs, and Hod Lipson. 2015. Understanding Neural Networks Through Deep Visualization. CoRR abs/1506.06579 (2015). arXiv:1506.06579 · arXiv

Cited by

See all 206

Metadata

Authors
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L.
Affiliations
  • Brown University (US)
  • Pacific Northwest National Laboratory (US)
Venue
Proceedings of the 25th International Conference on Intelligent User Interfaces
Publisher
ACM
Published
2020-03-17
Pages
189-201
OpenAlex
W3009578469
Cited by
200
References
97
Citations by year
2026: 21 · 2025: 29 · 2024: 56 · 2023: 34 · 2022: 25 · 2021: 30 · 2020: 4
FWCI
8.87
Type
conference paper
Language
en
Open access
closed
Locations
  • ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
  • OSF Preprints (OSF Preprints) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
Keywords
explainable machine learning, example-based explanations, trust in machine learning, spatial configuration, visual representation, performance feedback, human-machine decision-making, end users

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How do visual explanations foster end users' appropriate trust in machine learning? https://app.cochat.ai/papers/10.1145/3377325.3377480
The paper
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020). How do visual explanations foster end users' appropriate trust in machine learning? Proceedings of the 25th International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3377325.3377480

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.