Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance
Open accessThe DOI matches its Crossref record; 134 of the 139 references it cites were found in a registry or an index.
Abstract
If a user is presented an AI system that portends to explain how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? This question entails some key concepts of measurement such as explanation goodness and trust. We present methods for enabling developers and researchers to: (1) Assess the a priori goodness of explanations, (2) Assess users' satisfaction with explanations, (3) Reveal user's mental model of an AI system, (4) Assess user's curiosity or need for explanations, (5) Assess whether the user's trust and reliance on the AI are appropriate, and finally, (6) Assess how the human-XAI work system performs. The methods we present derive from our integration of extensive research literatures and our own psychometric evaluations. We point to the previous research that led to the measurement scales which we aggregated and tailored specifically for the XAI context. Scales are presented in sufficient detail to enable their use by XAI researchers. For Mental Model assessment and Work System Performance, XAI researchers have choices. We point to a number of methods, expressed in terms of methods' strengths and weaknesses, and pertinent measurement issues.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“Trust in computers is typically understood in terms of the metaphor of “calibration” (Lee and See, 2004).”
BackgroundConflict, arousal, and curiosity.
Berlyne, D. E. (1960) · McGraw-Hill Book Company · doi:10.1037/11164-000 · Cited by 5,995 · cited 1×
“Epistemic curiosity is the general desire for knowledge, a motive to learn new ideas, resolve knowledge gaps, and solve problems, even though this may entail effortful cognitive activity (Berlyne, 1960, 1978; Loewenstein, 1994; Litman and Lunsford, 2010).”
BackgroundCognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Categorization and Representation of Physics Problems by Experts and Novices*
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) · Cognitive Science · doi:10.1207/s15516709cog0502_2 · Cited by 5,248 · cited 1×
“The review article by van der Veer and Melguzio (2003) highlights this method (see also Chi et al., 1981; Evans et al., 2001; St-Cyr and Burns, 2002; van der Veer and Melguzio, 2003).”
Novak, J. D., Gowin, D. B., & Kahle, J. B. (1984) · doi:10.1017/cbo9781139173469 · Cited by 5,199
Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness
Ford, M., & Johnson-Laird, P. N. (1985) · Language · doi:10.2307/414498 · Cited by 4,419 · cited 2×
“There is a consensus that mental models can be inferred from empirical evidence (Johnson-Laird, 1983; Zhang and Wickens, 1987; Glenberg and Langston, 1992; Qin and Simon, 1992; Bogacz and Trafton, 2004; Clement, 2004; Heiser and Tversky, 2006; Klein and Hoffman, 2008).”
BackgroundThe Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913 · cited 1×
“, 2004), Cacioppo’s Need for Cognition scale (Cacioppo et al., 1984), and the I-Type/D-Type Curiosity Scales (Litman and Jimerson, 2004) consider curiosity to be a pervasive style or personality trait.”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · cited 2×
“satisfaction: understandability, feeling of satisfaction, sufficiency of detail, completeness, usefulness, accuracy, and trustworthiness (Muir and Moray, 1996).”
BackgroundMethod
Research works printed but not linked: 20
- Adams, 2003, Trust in automated systems
- Biran, 2017, “Explanation and Justification in Machine Learning: A Survey,”, IJCAI-17 Workshop on Explainable Artificial Intelligence (XAI)
- Cañas, 2003, A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support
- Carroll, 1984, Minimalist training, Datamation, 1, 125
- Doyle, 2002, “Mental models of dynamic systems,”, System Dynamics and Integrated Modeling, Encyclopedia of Life Support Systems
- Ford, 1993, “Participatory explanation,”, Presented at the FLAIRS 93: Sixth Florida Artificial Intelligence Research Symposium (FLAIRS), 111
- Gentner, 1983, “Flowing waters or teem crowds: Mental models of electricity,”, Mental models, 99
- Gentner, 1983, Mental Models
- Goodstein, 1988, Tasks, errors and mental models.
- Harris, 1984, “Machine intelligence in real systems: Some ergonomic issues,”, Human-Computer Interaction, 267
- Hilton, 1996, Mental models and causal explanation: Judgements of probable cause and explanatory relevance, Thinking Reason., 2, 273, 10.1080/135467896394447 · DOI
- Hoffman, 1989, “Whom (or what) do you trust: Historical reflections on the psychology and sociology of information technology,”, Proceedings of the Fourth Annual Symposium on Human Interaction with Complex Systems, 28
- Hoffman, 2018, Literature Review and Integration of Key Ideas for Explainable AI
- Kucala, 2013, The truthiness of trustworthiness., 57
- MillerT. Explanation in artificial intelligence: insights from the social sciences. ArXiv:1706.07269 [Cs]2017 · arXiv
- Ritter, 1988, “Teaching real-time tactical thinking,”, Intelligent Tutoring Systems: Lessons Learned, 285
- Samurcay, 1996, Causal versus topographical support for diagnosis in a dynamic situation, Le Travail Humain, 59, 45
- Sarter, 1997, “Automation surprises,”, Handbook of Human Factors/Ergonomics, 1926
- Taylor, 1988, “Using cognitive models to make plants safer: Experimental and practical approaches,”, Tasks, errors and mental models, 233
- van der Veer, 2003, “Mental models,”, The human-computer interaction handbook, 52
Cited by
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Hur, S., Lee, Y., Park, J., Jeon, Y. J., Cho, J. H., Cho, D., Lim, D., Hwang, W., Cha, W. C., & Yoo, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01958-8 · Cited by 118
Measuring trust in artificial intelligence: validation of an established scale and its short form
McGrath, M. J., Lack, O., Tisch, J., & Duenser, A. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1582880 · Cited by 135
Explainable AI framework for reliable and transparent automated energy management in buildings
Teixeira, B., Carvalhais, L., Pinto, T. D. O., & Vale, Z. (2025) · Energy and Buildings · doi:10.1016/j.enbuild.2025.116246 · Cited by 93
Ethical implications of AI in the Metaverse
Zhuk, A. (2024) · AI and Ethics · doi:10.1007/s43681-024-00450-5 · Cited by 78
“AI algorithms should be designed to provide clear and understandable explanations for their decisions and recommendations [41].”
Metadata
- Authors
- Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J.
- Affiliations
- Florida Institute for Human and Machine Cognition (US)
- Michigan Technological University (US)
- University of Maine at Machias (US)
- Corresponding author
- Hoffman, Robert R.
- Venue
- Frontiers in Computer Science
- Publisher
- Frontiers Media SA
- Published
- 2023-02-06
- Volume
- 5
- ISSN
- 2624-9898 (ISSN-L)
- OpenAlex
- W4319321305
- Cited by
- 272
- References
- 139
- Citations by year
- 2026: 117 · 2025: 97 · 2024: 46 · 2023: 12
- FWCI
- 25.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Computer Science (publishedVersion, cc-by)
- Digital Commons - Michigan Tech (Michigan Technological University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- explainable AI, user satisfaction, mental models, curiosity, trust in AI, measurement scales, psychometric assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance. https://app.cochat.ai/papers/10.3389/fcomp.2023.1096257
- The paper
- Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2023). Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance. Frontiers in Computer Science. https://doi.org/10.3389/fcomp.2023.1096257
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.