Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 134 of the 139 references it cites were found in a registry or an index.

Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance

Open access
Hoffman, Robert R., Mueller, Shane T., Klein, Gary, Litman, Jordan · 2023 · Frontiers in Computer Science

Abstract

If a user is presented an AI system that portends to explain how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? This question entails some key concepts of measurement such as explanation goodness and trust. We present methods for enabling developers and researchers to: (1) Assess the a priori goodness of explanations, (2) Assess users' satisfaction with explanations, (3) Reveal user's mental model of an AI system, (4) Assess user's curiosity or need for explanations, (5) Assess whether the user's trust and reliance on the AI are appropriate, and finally, (6) Assess how the human-XAI work system performs. The methods we present derive from our integration of extensive research literatures and our own psychometric evaluations. We point to the previous research that led to the measurement scales which we aggregated and tailored specifically for the XAI context. Scales are presented in sufficient detail to enable their use by XAI researchers. For Mental Model assessment and Work System Performance, XAI researchers have choices. We point to a number of methods, expressed in terms of methods' strengths and weaknesses, and pertinent measurement issues.

Overview

Integrity signals

Registry

Crossref · 10.3389/fcomp.2023.1096257

Cites

134 of 134 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 139 references

Research works printed but not linked: 20

  1. Adams, 2003, Trust in automated systems
  2. Biran, 2017, “Explanation and Justification in Machine Learning: A Survey,”, IJCAI-17 Workshop on Explainable Artificial Intelligence (XAI)
  3. Cañas, 2003, A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support
  4. Carroll, 1984, Minimalist training, Datamation, 1, 125
  5. Doyle, 2002, “Mental models of dynamic systems,”, System Dynamics and Integrated Modeling, Encyclopedia of Life Support Systems
  6. Ford, 1993, “Participatory explanation,”, Presented at the FLAIRS 93: Sixth Florida Artificial Intelligence Research Symposium (FLAIRS), 111
  7. Gentner, 1983, “Flowing waters or teem crowds: Mental models of electricity,”, Mental models, 99
  8. Gentner, 1983, Mental Models
  9. Goodstein, 1988, Tasks, errors and mental models.
  10. Harris, 1984, “Machine intelligence in real systems: Some ergonomic issues,”, Human-Computer Interaction, 267
  11. Hilton, 1996, Mental models and causal explanation: Judgements of probable cause and explanatory relevance, Thinking Reason., 2, 273, 10.1080/135467896394447 · DOI
  12. Hoffman, 1989, “Whom (or what) do you trust: Historical reflections on the psychology and sociology of information technology,”, Proceedings of the Fourth Annual Symposium on Human Interaction with Complex Systems, 28
  13. Hoffman, 2018, Literature Review and Integration of Key Ideas for Explainable AI
  14. Kucala, 2013, The truthiness of trustworthiness., 57
  15. MillerT. Explanation in artificial intelligence: insights from the social sciences. ArXiv:1706.07269 [Cs]2017 · arXiv
  16. Ritter, 1988, “Teaching real-time tactical thinking,”, Intelligent Tutoring Systems: Lessons Learned, 285
  17. Samurcay, 1996, Causal versus topographical support for diagnosis in a dynamic situation, Le Travail Humain, 59, 45
  18. Sarter, 1997, “Automation surprises,”, Handbook of Human Factors/Ergonomics, 1926
  19. Taylor, 1988, “Using cognitive models to make plants safer: Experimental and practical approaches,”, Tasks, errors and mental models, 233
  20. van der Veer, 2003, “Mental models,”, The human-computer interaction handbook, 52

Cited by

Metadata

Authors
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J.
Affiliations
  • Florida Institute for Human and Machine Cognition (US)
  • Michigan Technological University (US)
  • University of Maine at Machias (US)
Corresponding author
Hoffman, Robert R.
Venue
Frontiers in Computer Science
Published
2023-02-06
Volume
5
ISSN
2624-9898 (ISSN-L)
OpenAlex
W4319321305
Cited by
272
References
139
Citations by year
2026: 117 · 2025: 97 · 2024: 46 · 2023: 12
FWCI
25.64
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Computer Science (publishedVersion, cc-by)
  • Digital Commons - Michigan Tech (Michigan Technological University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Defense Advanced Research Projects Agency
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
explainable AI, user satisfaction, mental models, curiosity, trust in AI, measurement scales, psychometric assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.