Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance
Open accessThe DOI matches its Crossref record; 134 of the 139 references it cites were found in a registry or an index.
Abstract
If a user is presented an AI system that portends to explain how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? This question entails some key concepts of measurement such as explanation goodness and trust. We present methods for enabling developers and researchers to: (1) Assess the a priori goodness of explanations, (2) Assess users' satisfaction with explanations, (3) Reveal user's mental model of an AI system, (4) Assess user's curiosity or need for explanations, (5) Assess whether the user's trust and reliance on the AI are appropriate, and finally, (6) Assess how the human-XAI work system performs. The methods we present derive from our integration of extensive research literatures and our own psychometric evaluations. We point to the previous research that led to the measurement scales which we aggregated and tailored specifically for the XAI context. Scales are presented in sufficient detail to enable their use by XAI researchers. For Mental Model assessment and Work System Performance, XAI researchers have choices. We point to a number of methods, expressed in terms of methods' strengths and weaknesses, and pertinent measurement issues.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“Trust in computers is typically understood in terms of the metaphor of “calibration” (Lee and See, 2004).”
BackgroundConflict, arousal, and curiosity.
Berlyne, D. E. (1960) · McGraw-Hill Book Company · doi:10.1037/11164-000 · Cited by 5,995 · cited 1×
“Epistemic curiosity is the general desire for knowledge, a motive to learn new ideas, resolve knowledge gaps, and solve problems, even though this may entail effortful cognitive activity (Berlyne, 1960, 1978; Loewenstein, 1994; Litman and Lunsford, 2010).”
BackgroundCognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Categorization and Representation of Physics Problems by Experts and Novices*
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) · Cognitive Science · doi:10.1207/s15516709cog0502_2 · Cited by 5,248 · cited 1×
“The review article by van der Veer and Melguzio (2003) highlights this method (see also Chi et al., 1981; Evans et al., 2001; St-Cyr and Burns, 2002; van der Veer and Melguzio, 2003).”
Novak, J. D., Gowin, D. B., & Kahle, J. B. (1984) · doi:10.1017/cbo9781139173469 · Cited by 5,199
Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness
Ford, M., & Johnson-Laird, P. N. (1985) · Language · doi:10.2307/414498 · Cited by 4,419 · cited 2×
“There is a consensus that mental models can be inferred from empirical evidence (Johnson-Laird, 1983; Zhang and Wickens, 1987; Glenberg and Langston, 1992; Qin and Simon, 1992; Bogacz and Trafton, 2004; Clement, 2004; Heiser and Tversky, 2006; Klein and Hoffman, 2008).”
BackgroundThe Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913 · cited 1×
“, 2004), Cacioppo’s Need for Cognition scale (Cacioppo et al., 1984), and the I-Type/D-Type Curiosity Scales (Litman and Jimerson, 2004) consider curiosity to be a pervasive style or personality trait.”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · cited 2×
“satisfaction: understandability, feeling of satisfaction, sufficiency of detail, completeness, usefulness, accuracy, and trustworthiness (Muir and Moray, 1996).”
BackgroundMethodThe psychology of curiosity: A review and reinterpretation.
Loewenstein, G. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.1.75 · Cited by 2,738 · cited 4×
“Such situations lead people to engage in effortful processing, and motivates them to seek out additional knowledge in order to gain an insight and resolve the incongruency (Loewenstein, 1994).”
BackgroundMethodInfluentialEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“A proposed regulation before the European Union (Goodman and Flaxman, 2016) prohibits “automatic processing” unless user’s rights are safeguarded.”
BackgroundFoundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799 · cited 1×
“The report by Jian et al. (2000) illustrates those psychometric analyses.”
BackgroundSelf‐Explanations: How Students Study and Use Examples in Learning to Solve Problems
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989) · Cognitive Science · doi:10.1207/s15516709cog1302_1 · Cited by 1,714 · cited 3×
“This can be corrected by asking the learner to explicitly express their understanding or reasoning (Chi et al., 1989; Van Lehn et al., 1990; Chi and Van Lehn, 1991; O’Reilly et al., 1998; Rosenblit and Kein, 2002; Mills and Keil, 2004; Rittle-Johnson, 2006; Fernbach et al., 2012; Bjork et al., 2013).”
BackgroundSelf-Regulated Learning: Beliefs, Techniques, and Illusions
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143823 · Cited by 1,468 · cited 1×
“…can be corrected by asking the learner to explicitly express their understanding or reasoning (Chi et al., 1989; Van Lehn et al., 1990; Chi and Van Lehn, 1991; O’Reilly et al., 1998; Rosenblit and Kein, 2002; Mills and Keil, 2004; Rittle-Johnson, 2006; Fernbach et al., 2012; Bjork et al., 2013).”
BackgroundToward an Epistemology of Physics
diSessa, A. A. (1993) · Cognition and Instruction · doi:10.1080/07370008.1985.9649008 · Cited by 1,352 · cited 2×
“To be sure, people’s mental models are rarely complete and coherent (diSessa, 1993), or even consistently correct.”
BackgroundThe role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 2×
“Trust in automation can rapidly break down under conditions of time pressure, or when there are conspicuous system faults or errors, or when there is a high false alarm rate (Dzindolet et al., 2003; Madhavan and Wiegmann, 2007).”
BackgroundMental Models in Cognitive Science
Johnson-Laird, P. N. (1980) · Cognitive Science · doi:10.1016/s0364-0213(81)80005-5 · Cited by 1,273
Protocol Analysis: Verbal Reports as Data
Danks, J. H., Ericsson, K. A., & Simon, H. A. (1985) · The American Journal of Psychology · doi:10.2307/1422516 · Cited by 1,218
Bouwman, M. J. (1983) · European Journal of Operational Research · doi:10.1016/0377-2217(83)90060-7 · Cited by 1,064
The misunderstood limits of folk science: An illusion of explanatory depth
Rozenblit, L., & Keil, F. (2002) · Cognitive Science · doi:10.1207/s15516709cog2605_1 · Cited by 1,032 · cited 1×
“This can be corrected by asking the learner to explicitly express their understanding or reasoning (Chi et al., 1989; Van Lehn et al., 1990; Chi and Van Lehn, 1991; O’Reilly et al., 1998; Rosenblit and Kein, 2002; Mills and Keil, 2004; Rittle-Johnson, 2006; Fernbach et al., 2012; Bjork et al., 2013).”
BackgroundMUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993
(1995) · Choice Reviews Online · doi:10.5860/choice.32-2769 · Cited by 993
Information Processing and Human-Machine Interaction
Rasmussen, J. (1986) · Medical Entomology and Zoology · Cited by 907
Kashdan, T. B., Rose, P., & Fincham, F. D. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8203_05 · Cited by 888 · cited 1×
“Available scales of curiosity, such as the Curiosity Exploration Inventory (Kashdan et al., 2004), Cacioppo’s Need for Cognition scale (Cacioppo et al., 1984), and the I-Type/D-Type Curiosity Scales (Litman and Jimerson, 2004) consider curiosity to be a pervasive style or personality trait.”
BackgroundMUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 1×
“Based on our review of the psychological literature, including theoretical and empirical work by Muir (1987, 1994) and by Cahour and Forzy (2009), we identified several key attributes of explanation 10.3389 /fcomp.”
MethodInfluentialMaheswaran, D., & Chaiken, S. (1991) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.61.1.13 · Cited by 831 · cited 1×
“Curiosity is also triggered in circumstances where one experiences a violated expectation (Maheswaran and Chaiken, 1991).”
BackgroundAn integrated trust and reputation model for open multi-agent systems
Huynh, T. D., Jennings, N. R., & Shadbolt, N. R. (2006) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-005-6825-4 · Cited by 791
Trust between humans and machines, and the design of decision aids
Muir, B. M. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80013-5 · Cited by 762 · cited 1×
“Based on our review of the psychological literature, including theoretical and empirical work by Muir (1987, 1994) and by Cahour and Forzy (2009), we identified several key attributes of explanation 10.3389 /fcomp.”
MethodInfluentialEliciting Self‐Explanations Improves Understanding
Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994) · Cognitive Science · doi:10.1207/s15516709cog1803_3 · Cited by 757
Working Minds: A Practitioner's Guide to Cognitive Task Analysis
Crandall, B., Klein, G. A., & Hoffman, R. R. (2006) · The MIT Press eBooks · doi:10.7551/mitpress/7304.001.0001 · Cited by 728
Self-citationFoundations of Intelligent Tutoring Systems
(2013) · Psychology Press eBooks · doi:10.4324/9780203761557 · Cited by 610 · cited 1×
“Additionally, explanation is what Intelligent Tutoring Systems were (and are) all about (Sleeman and Brown, 1982; Polson and Richardson, 1988; Psotka et al., 1988; Ritter and Feurzeig, 1988; Anderson et al., 1990; Lesgold et al., 1992; Forbus and Feltovich, 2001).”
BackgroundExplanation, imagination, and confidence in judgment.
Koehler, D. J. (1991) · Psychological Bulletin · doi:10.1037//0033-2909.110.3.499 · Cited by 561 · cited 1×
“Explanations can entail what will happen in the future (Mitchell et al., 1989; Koehler, 1991; Lombrozo and Carey, 2006).”
BackgroundCognitive modeling and intelligent tutoring
Anderson, J. R., Boyle, C. F., Corbett, A. T., & Lewis, M. W. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90093-f · Cited by 534 · cited 1×
“Additionally, explanation is what Intelligent Tutoring Systems were (and are) all about (Sleeman and Brown, 1982; Polson and Richardson, 1988; Psotka et al., 1988; Ritter and Feurzeig, 1988; Anderson et al., 1990; Lesgold et al., 1992; Forbus and Feltovich, 2001).”
BackgroundMerritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527 · cited 1×
“…engineering, as well as in the popular media (e.g., Hoffman, 1989; Shadbolt, 2002; Lee and See, 2004; Huynh et al., 2006; Woods and Hollnagel, 2006; Merritt and Ilgen, 2008; Hoffman et al., 2009, 2013; Naone, 2009; Montague, 2010; Fitzhugh et al., 2011; Kucala, 2013; Merritt et al., 2013, 2015;…”
BackgroundComprehension of illustrated text: Pictures help to build mental models
Glenberg, A. M., & Langston, W. E. (1992) · Journal of Memory and Language · doi:10.1016/0749-596x(92)90008-l · Cited by 497 · cited 1×
“There is a consensus that mental models can be inferred from empirical evidence (Johnson-Laird, 1983; Zhang and Wickens, 1987; Glenberg and Langston, 1992; Qin and Simon, 1992; Bogacz and Trafton, 2004; Clement, 2004; Heiser and Tversky, 2006; Klein and Hoffman, 2008).”
BackgroundInteractive tasks and the implicit‐explicit distinction
Berry, D. C., & Broadbent, D. E. (1988) · British Journal of Psychology · doi:10.1111/j.2044-8295.1988.tb02286.x · Cited by 483 · cited 1×
“Performance on a simulated industrial process control task can be good and yet the operator’s understanding can be limited and even incorrect (see Berry and Broadbent, 1988).”
BackgroundJoint Cognitive Systems: Patterns in Cognitive Systems Engineering
Woods, D. D., & Hollnagel, E. (2006) · CRC Press eBooks · doi:10.1201/9781420005684 · Cited by 482 · cited 2×
“The complexity of performance at the work system level requires analysis based on this and other trade-offs (Woods and Hollnagel, 2006).”
BackgroundThe Measurement of Curiosity As a Feeling of Deprivation
Litman, J. A., & Jimerson, T. L. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8202_3 · Cited by 478 · cited 2×
“Available scales of curiosity, such as the Curiosity Exploration Inventory (Kashdan et al., 2004), Cacioppo’s Need for Cognition scale (Cacioppo et al., 1984), and the I-Type/D-Type Curiosity Scales (Litman and Jimerson, 2004) consider curiosity to be a pervasive style or personality trait.”
BackgroundSelf-citationIntelligent tutoring systems : lessons learned
Psotka, J., Massey, L., & Mutter, S. A. (1988) · Lawrence Erlbaum Associates eBooks · Cited by 457
Holzinger, A., Carrington, A., & Müller, H. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00636-z · Cited by 406 · cited 1×
“Holzinger et al. (2020) proposed a 10-item scale called the System Causability Scale.”
MethodCounterfactual Visual ExplanationsPreprint
Goyal, Y., Wu, Z., Ernst, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07451 · Cited by 397 · cited 1×
“The task retrospections included the asking of counterfactual and contrastive explanations, to ask why a software agent did not decide in a particular way, or ask why decided one way when it should have decided another (see also Goyal et al., 2019).”
BackgroundPromoting Transfer: Effects of Self-Explanation and Direct Instruction
Rittle-Johnson, B. (2006) · Child Development · doi:10.1111/j.1467-8624.2006.00852.x · Cited by 395 · cited 4×
“Self-generated deductions and generalizations help learners refine their knowledge (Chi et al., 1989, 1994; Chi and Van Lehn, 1991; Rittle-Johnson, 2006; Molinaro and Garcia-Madruga, 2011; Lombrozo, 2016).”
BackgroundInfluentialMerritt, S. M., Heimbaugh, H., LaChapell, J., & Lee, D. (2013) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812465081 · Cited by 329
Berlyne, D. E. (1978) · Motivation and Emotion · doi:10.1007/bf00993037 · Cited by 326 · cited 1×
“Epistemic curiosity is the general desire for knowledge, a motive to learn new ideas, resolve knowledge gaps, and solve problems, even though this may entail effortful cognitive activity (Berlyne, 1960, 1978; Loewenstein, 1994; Litman and Lunsford, 2010).”
BackgroundThe Content of Physics Self-Explanations
Chi, M. T. H., & VanLehn, K. A. (1991) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0101_4 · Cited by 306 · cited 2×
“This can be corrected by asking the learner to explicitly express their understanding or reasoning (Chi et al., 1989; Van Lehn et al., 1990; Chi and Van Lehn, 1991; O’Reilly et al., 1998; Rosenblit and Kein, 2002; Mills and Keil, 2004; Rittle-Johnson, 2006; Fernbach et al., 2012; Bjork et al.,…”
BackgroundFunctional explanation and the function of explanation
LOMBROZO, T., & CAREY, S. (2006) · Cognition · doi:10.1016/j.cognition.2004.12.009 · Cited by 300 · cited 1×
“Explanations can entail what will happen in the future (Mitchell et al., 1989; Koehler, 1991; Lombrozo and Carey, 2006).”
BackgroundMental models: concepts for human-computer interaction research
Staggers, N., & Norcio, A. F. (1993) · International Journal of Man-Machine Studies · doi:10.1006/imms.1993.1028 · Cited by 294 · cited 2×
“Staggers and Norcio (1993) provide a good summary of the conceptual and theoretical issues.”
BackgroundExtracting, Representing, and Analyzing Mental Models
Carley, K., & Palmquist, M. (1992) · Social Forces · doi:10.2307/2579746 · Cited by 292
Affective Processes in Human–Automation Interactions
Merritt, S. M. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811411912 · Cited by 260 · cited 2×
“For example, an XAI system that drives the work efficiency (say, to deal with data overload issues) may not make for a very contented workplace (Merritt, 2011).”
BackgroundTasks, errors and mental models
Broadbent, D. E. (1988) · Knowledge-Based Systems · doi:10.1016/0950-7051(88)90045-7 · Cited by 253
Johnson-Laird, P. N. (1989) · The Foundations of Cognitive Science · doi:10.7551/mitpress/3072.003.0014 · Cited by 230
Hoffman, R. R., Johnson, M., Bradshaw, J. M., & Underbrink, A. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.24 · Cited by 229
Self-citationExplanatory Preferences Shape Learning and Inference
Lombrozo, T. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.08.001 · Cited by 229 · cited 1×
“Self-generated deductions and generalizations help learners refine their knowledge (Chi et al., 1989, 1994; Chi and Van Lehn, 1991; Rittle-Johnson, 2006; Molinaro and Garcia-Madruga, 2011; Lombrozo, 2016).”
BackgroundThe relation between problem categorization and problem solving among experts and novices
Hardiman, P. T., Dufresne, R., & Mestre, J. P. (1989) · Memory & Cognition · doi:10.3758/bf03197085 · Cited by 227
An explainable artificial intelligence system for small-unit tactical behavior
van Lent, M., Fisher, W. M., & Mancuso, M. R. (2004) · Cited by 215
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G. (2019) · arXiv · doi:10.48550/arxiv.1902.01876 · Cited by 215 · cited 2×
“For reviews of the literature, see Byrne (2017), Miller (2017), Hoffman et al. (2018), and Mueller et al. (2019).”
BackgroundSelf-citationAutomated rationale generation: a technique for explainable AI and its effects on human perceptions
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., & Riedl, M. O. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302316 · Cited by 210 · cited 2×
“Ehsan et al. (2019) focused on the “human-likeness” of explanations, that is, the degree to which a machine-generated explanation seems like the sort of thing a human might say.”
BackgroundThe Perception And Measurement Of Human-robot Trust
Schaefer, K. E. (2013) · STARS (University of Central Florida) · Cited by 192
Mills, C. M., & Keil, F. C. (2004) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2003.09.003 · Cited by 186
Complacency and Automation Bias in the Use of Imperfect Automation
Wickens, C. D., Clegg, B. A., Vieane, A. Z., & Sebok, A. L. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815581940 · Cited by 176 · cited 1×
“Trust in automation is of concern in computer science and cognitive systems engineering, as well as in the popular media (e.g., Hoffman, 1989; Shadbolt, 2002; Lee and See, 2004; Huynh et al., 2006; Woods and Hollnagel, 2006; Merritt and Ilgen, 2008; Hoffman et al., 2009, 2013; Naone, 2009; Montague, 2010; Fitzhugh et…”
BackgroundHarford, T. (2014) · Significance · doi:10.1111/j.1740-9713.2014.00778.x · Cited by 175
Arrows in Comprehending and Producing Mechanical Diagrams
Heiser, J., & Tversky, B. (2006) · Cognitive Science · doi:10.1207/s15516709cog0000_70 · Cited by 167
Madhavan, P., & Wiegmann, D. A. (2007) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x230154 · Cited by 157 · cited 1×
“Trust in automation can rapidly break down under conditions of time pressure, or when there are conspicuous system faults or errors, or when there is a high false alarm rate (Dzindolet et al., 2003; Madhavan and Wiegmann, 2007).”
BackgroundWork-arounds, make-work, and kludges
Koopman, P., & Hoffman, R. R. (2003) · IEEE Intelligent Systems · doi:10.1109/mis.2003.1249172 · Cited by 154 · cited 1×
“With regard to performance at the work system level, many considerations other than raw efficiency come into play (Koopman and Hoffman, 2003; Hoffman and Hancock, 2017).”
BackgroundSelf-citationFrom Guidon to Neomycin and Heracles in twenty short lessons
Clancey, W. J. (1986) · AI Magazine · doi:10.1609/aimag.v7i3.548 · Cited by 153 · cited 1×
“Expert Systems researchers implemented methods of explanation (Clancey, 1984, 1986; McKeown and Swartout, 1987; Moore and Swartout, 1990).”
MethodLin, T. Y. (2002) · Educational Technology & Society · Cited by 149
Mental Models and Causal Explanation: Judgements of Probable Cause and Explanatory Relevance
Hilton, D. (1996) · Thinking & Reasoning · doi:10.1080/135467896394447 · Cited by 144
Johnson, D. S. (2007) · Journal of Interactive Marketing · doi:10.1002/dir.20091 · Cited by 141 · cited 1×
“The scale developed by Johnson (2007) asks only about reliance and the rareness of errors.”
BackgroundHow it works: a field study of non-technical users interacting with an intelligent system
Tullio, J., Dey, A. K., Chalecki, J., & Fogarty, J. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240630 · Cited by 137 · cited 2×
“A focus for instructional design has been to develop methods to get people to recognize when they are employing a knowledge shield that prevents them from developing richer mental models (Hilton, 1996; Hilton and Erb, 1996; Prietula et al., 2000; Tullio et al., 2007; Schaffernicht and Groesser, 2011).”
BackgroundMethods and Tactics in Cognitive Science
Kintsch, W., Miller, J. R., & Polson, P. G. (2014) · Psychology Press eBooks · doi:10.4324/9781315802619 · Cited by 135 · cited 1×
“There is a large body of psychological research on mental models (Johnson-Laird, 1980, 1989; Gentner and Gentner, 1983; Greeno, 1983; Carroll, 1984; Kintsch et al., 1984).”
BackgroundDoes projection into use improve trust and exploration? An example with a cruise control system
Cahour, B., & Forzy, J.-F. (2009) · Safety Science · doi:10.1016/j.ssci.2009.03.015 · Cited by 134 · cited 2×
“Some of the scale items can be adapted to make them appropriate to the XAI context, but the result of this modification is just a few items, which are ones that are in the Cahour-Fourzy Scale (Cahour and Forzy, 2009) (e.g., Can you trust the decisions the [system] will make?”
BackgroundMethodInfluentialApplied Concept Mapping: Capturing, Analyzing, and Organizing Knowledge
(2011) · CRC Press eBooks · doi:10.1201/b10716 · Cited by 134 · cited 1×
“Novak and Gowin, 1984; Evans et al., 2001; Cañas et al., 2003; Moon et al., 2011; Hoffman and Hancock, 2017.”
Methodology for building an intelligent tutoring system
Clancey, W. J. (1987) · Stanford University eBooks · Cited by 123
Back to the future: Temporal perspective in the explanation of events
Mitchell, D. J., Edward Russo, J., & Pennington, N. (1989) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960020103 · Cited by 119 · cited 1×
“Explanations can entail what will happen in the future (Mitchell et al., 1989; Koehler, 1991; Lombrozo and Carey, 2006).”
BackgroundVerbal reports as evidence of the process operator's knowledge
Bainbridge, L. (1979) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(79)80035-8 · Cited by 110
Explanation Fiends and Foes: How Mechanistic Detail Determines Understanding and Preference
Fernbach, P. M., Sloman, S. A., Louis, R. D. S., & Shube, J. N. (2012) · Journal of Consumer Research · doi:10.1086/667782 · Cited by 109 · cited 1×
“This can be corrected by asking the learner to explicitly express their understanding or reasoning (Chi et al., 1989; Van Lehn et al., 1990; Chi and Van Lehn, 1991; O’Reilly et al., 1998; Rosenblit and Kein, 2002; Mills and Keil, 2004; Rittle-Johnson, 2006; Fernbach et al., 2012; Bjork et al., 2013).”
BackgroundIndividual Differences in the Calibration of Trust in Automation
Pop, V. L., Shrewsbury, A., & Durso, F. T. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814564422 · Cited by 109 · cited 1×
“Trust in automation is of concern in computer science and cognitive systems engineering, as well as in the popular media (e.g., Hoffman, 1989; Shadbolt, 2002; Lee and See, 2004; Huynh et al., 2006; Woods and Hollnagel, 2006; Merritt and Ilgen, 2008; Hoffman et al., 2009, 2013; Naone, 2009; Montague, 2010; Fitzhugh et…”
Background(2018) · doi:10.1201/9781315137957-2 · Cited by 108 · cited 2×
“Only if this trusting relation is achieved can the user’s reliance on the computer be confident (Riley, 1996).”
BackgroundTransparent Queries: investigation users' mental models of search engines
Muramatsu, J., & Pratt, W. (2001) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/383952.383991 · Cited by 101 · cited 1×
“Muramatsu and Pratt, 2001.”
Voosen, P. (2017) · Science · doi:10.1126/science.357.6346.22 · Cited by 101 · cited 1×
“AI systems are receiving considerable attention in the recent popular press (Harford, 2014; Bornstein, 2016; Alang, 2017; Champlin et al., 2017; Hawkins, 2017; Kuang, 2017; Pavlus, 2017; Pinker, 2017; Schwiep, 2017; Voosen, 2017; Weinberger, 2017).”
Hoffman, R. R., & Hancock, P. A. (2017) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816686248 · Cited by 98 · cited 1×
“With regard to performance at the work system level, many considerations other than raw efficiency come into play (Koopman and Hoffman, 2003; Hoffman and Hancock, 2017).”
BackgroundSelf-citationA comprehensive method for comparing mental models of dynamic systems
Schaffernicht, M. F. G., & Groesser, S. N. (2010) · European Journal of Operational Research · doi:10.1016/j.ejor.2010.09.003 · Cited by 95 · cited 1×
“A focus for instructional design has been to develop methods to get people to recognize when they are employing a knowledge shield that prevents them from developing richer mental models (Hilton, 1996; Hilton and Erb, 1996; Prietula et al., 2000; Tullio et al., 2007; Schaffernicht and Groesser, 2011).”
BackgroundFeltovich, P. J., Coulson, R. L., & Spiro, R. J. (2001) · The MIT Press eBooks · Cited by 85
Intelligent aids, mental models, and the theory of machines
Moray, N. P. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80020-2 · Cited by 84 · cited 1×
“…systems (Bainbridge, 1979, 1988; de Kleer and Brown, 1983; Williams et al., 1983; Young, 1983; Harris and Helander, 1984; Rasmussen, 1986; Moray, 1987; Goodstein et al., 1988; Carberry, 1990; May et al., 1993; Staggers and Norcio, 1993; Rasmussen et al., 1994; Samurcay and Hoc, 1996;…”
Mental Models of Mere Mortals with Explanations of Reinforcement Learning
Anderson, A. W., Dodge, J. E., Sadarangani, A., Juozapaitis, Z., Newman, E., Irvine, J., Chattopadhyay, S., Olson, M., Fern, A., & Burnett, M. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3366485 · Cited by 84 · cited 1×
“Compelling evidence of the value of task reflection comes from a set of studies that used methods of knowledge elicitation for the evaluation of explainable AI systems (Anderson et al., 2020; Dodge et al., 2021; Khanna et al., 2021; Tabatabai et al., 2021).”
Merritt, S. M., Lee, D., Unnerstall, J. L., & Huber, K. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814561675 · Cited by 83
Pointing: a way toward explanation dialogue
Moore, J. D., & Swartout, W. (1990) · National Conference on Artificial Intelligence · Cited by 82
The Role of Trust as a Mediator Between System Characteristics and Response Behaviors
Chancey, E. T., Bliss, J. P., Proaps, A. B., & Madhavan, P. (2015) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815582261 · Cited by 73 · cited 1×
“…Lee and See, 2004; Huynh et al., 2006; Woods and Hollnagel, 2006; Merritt and Ilgen, 2008; Hoffman et al., 2009, 2013; Naone, 2009; Montague, 2010; Fitzhugh et al., 2011; Kucala, 2013; Merritt et al., 2013, 2015; Chancey et al., 2015; Hoff and Bashir, 2015; Pop et al., 2015; Wickens et al., 2015).”
4. Some guidelines for conducting a cognitive task analysis
Klein, G., & Militello, L. (2004) · Advances in Human Performance and Cognitive Engineering Research · doi:10.1016/s1479-3601(01)01006-2 · Cited by 67
Self-citationA Comparison of Self-Explanation and Elaborative Interrogation
O’Reilly, T., Symons, S., & MacLatchy-Gaudet, H. (1998) · Contemporary Educational Psychology · doi:10.1006/ceps.1997.0977 · Cited by 66 · cited 1×
“…can be corrected by asking the learner to explicitly express their understanding or reasoning (Chi et al., 1989; Van Lehn et al., 1990; Chi and Van Lehn, 1991; O’Reilly et al., 1998; Rosenblit and Kein, 2002; Mills and Keil, 2004; Rittle-Johnson, 2006; Fernbach et al., 2012; Bjork et al., 2013).”
BackgroundCounterfactual Thinking: From Logic to Morality
Byrne, R. M. J. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721417695617 · Cited by 65 · cited 1×
“For reviews of the literature, see Byrne (2017), Miller (2017), Hoffman et al. (2018), and Mueller et al. (2019).”
BackgroundEveryday Expertise: Cognitive Demands in Diabetes Self-Management
Lippa, K. D., Klein, H. A., & Shalin, V. L. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x250601 · Cited by 62
The Role of Encoding in the Self-Explanation Effect
Calin-Jageman, R. J., & Ratner, H. H. (2005) · Cognition and Instruction · doi:10.1207/s1532690xci2304_4 · Cited by 56
Surrogates and Mappings: Two Kinds of Conceptual Models for Interactive Devices
Young, R. M. (2014) · Psychology Press eBooks · doi:10.4324/9781315802725-7 · Cited by 56
The ShadowBox Approach to Cognitive Skills Training
Klein, G. A., & Borders, J. (2016) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343416636515 · Cited by 55 · cited 4×
“Klein and Borders, 2016.”
BackgroundMethodInfluentialSelf-citationValidation of a trust in medical technology instrument
Montague, E. (2010) · Applied Ergonomics · doi:10.1016/j.apergo.2010.01.009 · Cited by 54 · cited 2×
“…Shadbolt, 2002; Lee and See, 2004; Huynh et al., 2006; Woods and Hollnagel, 2006; Merritt and Ilgen, 2008; Hoffman et al., 2009, 2013; Naone, 2009; Montague, 2010; Fitzhugh et al., 2011; Kucala, 2013; Merritt et al., 2013, 2015; Chancey et al., 2015; Hoff and Bashir, 2015; Pop et al., 2015;…”
BackgroundLanguage Generation and Explanation
McKeown, K. R., & Swartout, W. (1987) · Annual Review of Computer Science · doi:10.1146/annurev.cs.02.060187.002153 · Cited by 54 · cited 1×
“Expert Systems researchers implemented methods of explanation (Clancey, 1984, 1986; McKeown and Swartout, 1987; Moore and Swartout, 1990).”
MethodSpecial report : Can we copy the brain? - What intelligent machines need to learn from the Neocortex
Hawkins, J. (2017) · IEEE Spectrum · doi:10.1109/mspec.2017.7934229 · Cited by 53 · cited 1×
“AI systems are receiving considerable attention in the recent popular press (Harford, 2014; Bornstein, 2016; Alang, 2017; Champlin et al., 2017; Hawkins, 2017; Kuang, 2017; Pavlus, 2017; Pinker, 2017; Schwiep, 2017; Voosen, 2017; Weinberger, 2017).”
A Taxonomy of Emergent Trusting in the Human–Machine Relationship
Hoffman, R. R. (2017) · Cognitive Systems Engineering · doi:10.1201/9781315572529-8 · Cited by 51
Self-citationThe dynamics of trust in cyberdomains
Hoffman, R. R., Lee, J. D., Woods, D. D., Shadbolt, N., Miller, J., & Bradshaw, J. M. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.124 · Cited by 43
Self-citationHuman Reasoning About a Simple Physical System
Williams, M. D., Hollan, J. D., & Stevens, A. L. (2014) · Psychology Press eBooks · doi:10.4324/9781315802725-11 · Cited by 40 · cited 4×
“…process control systems, complex computational systems, decision aids, and intelligent systems (Bainbridge, 1979, 1988; de Kleer and Brown, 1983; Williams et al., 1983; Young, 1983; Harris and Helander, 1984; Rasmussen, 1986; Moray, 1987; Goodstein et al., 1988; Carberry, 1990; May et al.,…”
BackgroundMethodInfluentialGreeno, J. G. (1982) · Defense Technical Information Center (DTIC) · doi:10.21236/ada123387 · Cited by 40
Macrocognition, Mental Models, and Cognitive Task Analysis Methodology
Maarten, S. J., Militello, L. G., Ormerod, T., & Raanan, L. (2017) · CRC Press eBooks · doi:10.1201/9781315597584-6 · Cited by 34 · cited 2×
“There is a consensus that mental models can be inferred from empirical evidence (Johnson-Laird, 1983; Zhang and Wickens, 1987; Glenberg and Langston, 1992; Qin and Simon, 1992; Bogacz and Trafton, 2004; Clement, 2004; Heiser and Tversky, 2006; Klein and Hoffman, 2008).”
BackgroundSchraagen, J. M., Elsasser, P., Fricke, H., Hof, M., & Ragalmuto, F. (2020) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181320641077 · Cited by 28 · cited 2×
“Some of the measures have been adopted by researchers outside of that Program (Schraagen et al., 2020).”
MethodRepresenting, Running, and Revising Mental Models: A Computational Model
Friedman, S., Forbus, K. D., & Sherin, B. L. (2017) · Cognitive Science · doi:10.1111/cogs.12574 · Cited by 27
Using structural descriptions of interfaces to automate the modelling of user cognition
May, J., Barnard, P. J., & BLANDFORD, A. E. (1993) · User Modeling and User-Adapted Interaction · doi:10.1007/bf01099424 · Cited by 26 · cited 1×
“…de Kleer and Brown, 1983; Williams et al., 1983; Young, 1983; Harris and Helander, 1984; Rasmussen, 1986; Moray, 1987; Goodstein et al., 1988; Carberry, 1990; May et al., 1993; Staggers and Norcio, 1993; Rasmussen et al., 1994; Samurcay and Hoc, 1996; Doyle et al., 2002; Mueller and Klein, 2011).”
BackgroundPrietula, M. J., Feltovich, P. J., & Marchak, F. M. (2000) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872000779656589 · Cited by 26 · cited 1×
“A focus for instructional design has been to develop methods to get people to recognize when they are employing a knowledge shield that prevents them from developing richer mental models (Hilton, 1996; Hilton and Erb, 1996; Prietula et al., 2000; Tullio et al., 2007; Schaffernicht and Groesser, 2011).”
BackgroundStorm-LK: A Human-Centered Knowledge Model for Weather Forecasting
Hoffman, R. R., Coffey, J. W., Ford, K. M., & Carnot, M. J. (2001) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120104500807 · Cited by 26
Self-citationUnderstanding dynamic and static displays: using images to reason dynamically
Bogacz, S., & Trafton, J. G. (2005) · Cognitive Systems Research · doi:10.1016/j.cogsys.2004.11.007 · Cited by 26 · cited 1×
“There is a consensus that mental models can be inferred from empirical evidence (Johnson-Laird, 1983; Zhang and Wickens, 1987; Glenberg and Langston, 1992; Qin and Simon, 1992; Bogacz and Trafton, 2004; Clement, 2004; Heiser and Tversky, 2006; Klein and Hoffman, 2008).”
BackgroundAfter-Action Review for AI (AAR/AI)
Dodge, J. E., Khanna, R., Irvine, J., Lam, K.-H., Mai, T., Lin, Z., Kiddle, N. G., Newman, E., Anderson, A. W., Raja, S., et al. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3453173 · Cited by 25 · cited 3×
“Gentner and Gentner, 1983; Williams et al., 1983; Dodge et al., 2021; Khanna et al., 2021.”
BackgroundThe Stakeholder Playbook for Explaining AI SystemsPreprint
Hoffman, R., Klein, G. L., Mueller, S. T., Jalaeian, M., & Tate, C. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9pqez · Cited by 21
Self-citationImproving Users' Mental Models of Intelligent Software Tools
Mueller, S. T., & Klein, G. A. (2011) · IEEE Intelligent Systems · doi:10.1109/mis.2011.32 · Cited by 20 · cited 1×
“…de Kleer and Brown, 1983; Williams et al., 1983; Young, 1983; Harris and Helander, 1984; Rasmussen, 1986; Moray, 1987; Goodstein et al., 1988; Carberry, 1990; May et al., 1993; Staggers and Norcio, 1993; Rasmussen et al., 1994; Samurcay and Hoc, 1996; Doyle et al., 2002; Mueller and Klein, 2011).”
BackgroundSelf-citationExplanation-Based Learning of Correctness: Towards a Model of the Self-Explanation Effect
VanLehn, K. A., Ball, W. E., & Kowalski, B. (1990) · Defense Technical Information Center (DTIC) · doi:10.21236/ada225644 · Cited by 18 · cited 3×
“This can be corrected by asking the learner to explicitly express their understanding or reasoning (Chi et al., 1989; Van Lehn et al., 1990; Chi and Van Lehn, 1991; O’Reilly et al., 1998; Rosenblit and Kein, 2002; Mills and Keil, 2004; Rittle-Johnson, 2006; Fernbach et al., 2012; Bjork et al., 2013).”
BackgroundFinding AI’s Faults with AAR/AI: An Empirical Study
Khanna, R., Dodge, J. E., Anderson, A. W., Dikkala, R., Irvine, J., Shureih, Z., Lam, K.-H., Matthews, C., Lin, Z., Kahng, M., et al. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3487065 · Cited by 16 · cited 1×
“Compelling evidence of the value of task reflection comes from a set of studies that used methods of knowledge elicitation for the evaluation of explainable AI systems (Anderson et al., 2020; Dodge et al., 2021; Khanna et al., 2021; Tabatabai et al., 2021).”
BackgroundMental Model Assessments: Is There Convergence Among Different Methods?
Evans, A. W., Jentsch, F. G., Hitt, J. M., Bowers, C. A., & Salas, E. (2001) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120104500406 · Cited by 15 · cited 4×
“There is sufficient evidence to conclude that different methods for eliciting mental models can converge (Evans et al., 2001; van der Veer and Melguzio, 2003).”
InfluentialVerbal reports: a problem in research design
Praetorius, N., & Duncan, K. (1988) · Taylor & Francis, Inc. eBooks · Cited by 14
AI medicine comes to Africa's rural clinics
Champlin, C., Bell, D., & Schocken, C. (2017) · IEEE Spectrum · doi:10.1109/mspec.2017.7906899 · Cited by 14 · cited 1×
“AI systems are receiving considerable attention in the recent popular press (Harford, 2014; Bornstein, 2016; Alang, 2017; Champlin et al., 2017; Hawkins, 2017; Kuang, 2017; Pavlus, 2017; Pinker, 2017; Schwiep, 2017; Voosen, 2017; Weinberger, 2017).”
Imagistic Simulation and Physical Intuition in Expert Problem Solving
Clement, J. J. (2019) · Routledge eBooks · doi:10.4324/9781315789354-35 · Cited by 13 · cited 2×
“There is a consensus that mental models can be inferred from empirical evidence (Johnson-Laird, 1983; Zhang and Wickens, 1987; Glenberg and Langston, 1992; Qin and Simon, 1992; Bogacz and Trafton, 2004; Clement, 2004; Heiser and Tversky, 2006; Klein and Hoffman, 2008).”
BackgroundEpistemic Strategies Causal Thinking in Expert and Nonexpert Judgment
Beach, L. R. (2007) · Springer eBooks · doi:10.1007/978-0-585-34290-0_6 · Cited by 11
Incurious Motives to Seek Information about Potential Threats
Litman, J. A., & Lunsford, G. D. (2010) · European Journal of Personality · doi:10.1002/per.766 · Cited by 9 · cited 1×
“Epistemic curiosity is the general desire for knowledge, a motive to learn new ideas, resolve knowledge gaps, and solve problems, even though this may entail effortful cognitive activity (Berlyne, 1960, 1978; Loewenstein, 1994; Litman and Lunsford, 2010).”
BackgroundSelf-citationSecond International Workshop on User Modeling
Carberry, S. (1991) · AI Magazine · doi:10.1609/aimag.v11i4.870 · Cited by 9
Mental Models and Ecological Interface Design: An Experimental Investigation
St-Cyr, O., & Burns, C. M. (2002) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120204600311 · Cited by 8
Zhang, K., & Wickens, C. D. (1987) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193128703100123 · Cited by 8
Shadbolt, N. (2002) · IEEE Intelligent Systems · doi:10.1109/mis.2002.988440 · Cited by 8 · cited 1×
“Trust in automation is of concern in computer science and cognitive systems engineering, as well as in the popular media (e.g., Hoffman, 1989; Shadbolt, 2002; Lee and See, 2004; Huynh et al., 2006; Woods and Hollnagel, 2006; Merritt and Ilgen, 2008; Hoffman et al., 2009, 2013; Naone, 2009; Montague,…”
BackgroundTheory → Concepts → Measures but Policies → Metrics
Hoffman, R. R. (2018) · CRC Press eBooks · doi:10.1201/9781315593173-2 · Cited by 5
Self-citationFitzhugh, E. W., Hoffman, R. R., & Miller, J. E. (2018) · CRC Press eBooks · doi:10.1201/9781315549637-11 · Cited by 5
Self-citationWard, P., Wilson, K., Suss, J., Woody, W. D., & Hoffman, R. R. (2019) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198795872.013.17 · Cited by 4
Self-citation“Why did my AI agent lose?”: Visual Analytics for Scaling Up After-Action Review
Tabatabai, D., Ruangrotsakun, A., Irvine, J., Dodge, J. E., Shureih, Z., Lam, K.-H., Burnett, M., Fern, A., & Kahng, M. (2021) · Visualization · doi:10.1109/vis49827.2021.9623268 · Cited by 4 · cited 2×
“Compelling evidence of the value of task reflection comes from a set of studies that used methods of knowledge elicitation for the evaluation of explainable AI systems (Anderson et al., 2020; Dodge et al., 2021; Khanna et al., 2021; Tabatabai et al., 2021).”
BackgroundThe Second International Workshop on User Modeling
McTear, M. (1991) · AI Communications · doi:10.3233/aic-1991-4106 · Cited by 3 · cited 2×
“These considerations pertain directly to how people understand machines, spanning process control systems, complex computational systems, decision aids, and intelligent systems (Bainbridge, 1979, 1988; de Kleer and Brown, 1983; Williams et al., 1983; Young, 1983; Harris and Helander, 1984; Rasmussen, 1986; Moray, 1987…”
Post Quantification of Introspective Data
Fryer, D. (1939) · The American Journal of Psychology · doi:10.2307/1416744 · Cited by 3 · cited 2×
“Fryer (1939) provides a clear and succinct presentation of a method that combines retrospection with Likert scale questionnaire to quantify participants’ reasoning.”
MethodLesgold, A. M., Lajoie, S. P., Bunzo, M., & Eggan, G. (2021) · Routledge eBooks · doi:10.4324/9781315044361-8 · Cited by 2
Imagery as Process Representation in Problem Solving
Qin, Y., & Simon, H. A. (1992) · eScholarship (California Digital Library) · Cited by 2
European Union Intellectual Property Office (n.d.) · doi:10.2814/062663 · Cited by 1
Research works printed but not linked: 20
- Adams, 2003, Trust in automated systems
- Biran, 2017, “Explanation and Justification in Machine Learning: A Survey,”, IJCAI-17 Workshop on Explainable Artificial Intelligence (XAI)
- Cañas, 2003, A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support
- Carroll, 1984, Minimalist training, Datamation, 1, 125
- Doyle, 2002, “Mental models of dynamic systems,”, System Dynamics and Integrated Modeling, Encyclopedia of Life Support Systems
- Ford, 1993, “Participatory explanation,”, Presented at the FLAIRS 93: Sixth Florida Artificial Intelligence Research Symposium (FLAIRS), 111
- Gentner, 1983, “Flowing waters or teem crowds: Mental models of electricity,”, Mental models, 99
- Gentner, 1983, Mental Models
- Goodstein, 1988, Tasks, errors and mental models.
- Harris, 1984, “Machine intelligence in real systems: Some ergonomic issues,”, Human-Computer Interaction, 267
- Hilton, 1996, Mental models and causal explanation: Judgements of probable cause and explanatory relevance, Thinking Reason., 2, 273, 10.1080/135467896394447 · DOI
- Hoffman, 1989, “Whom (or what) do you trust: Historical reflections on the psychology and sociology of information technology,”, Proceedings of the Fourth Annual Symposium on Human Interaction with Complex Systems, 28
- Hoffman, 2018, Literature Review and Integration of Key Ideas for Explainable AI
- Kucala, 2013, The truthiness of trustworthiness., 57
- MillerT. Explanation in artificial intelligence: insights from the social sciences. ArXiv:1706.07269 [Cs]2017 · arXiv
- Ritter, 1988, “Teaching real-time tactical thinking,”, Intelligent Tutoring Systems: Lessons Learned, 285
- Samurcay, 1996, Causal versus topographical support for diagnosis in a dynamic situation, Le Travail Humain, 59, 45
- Sarter, 1997, “Automation surprises,”, Handbook of Human Factors/Ergonomics, 1926
- Taylor, 1988, “Using cognitive models to make plants safer: Experimental and practical approaches,”, Tasks, errors and mental models, 233
- van der Veer, 2003, “Mental models,”, The human-computer interaction handbook, 52
Cited by
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Hur, S., Lee, Y., Park, J., Jeon, Y. J., Cho, J. H., Cho, D., Lim, D., Hwang, W., Cha, W. C., & Yoo, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01958-8 · Cited by 118
Measuring trust in artificial intelligence: validation of an established scale and its short form
McGrath, M. J., Lack, O., Tisch, J., & Duenser, A. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1582880 · Cited by 135
Explainable AI framework for reliable and transparent automated energy management in buildings
Teixeira, B., Carvalhais, L., Pinto, T. D. O., & Vale, Z. (2025) · Energy and Buildings · doi:10.1016/j.enbuild.2025.116246 · Cited by 93
Ethical implications of AI in the Metaverse
Zhuk, A. (2024) · AI and Ethics · doi:10.1007/s43681-024-00450-5 · Cited by 78
“AI algorithms should be designed to provide clear and understandable explanations for their decisions and recommendations [41].”
Metadata
- Authors
- Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J.
- Affiliations
- Florida Institute for Human and Machine Cognition (US)
- Michigan Technological University (US)
- University of Maine at Machias (US)
- Corresponding author
- Hoffman, Robert R.
- Venue
- Frontiers in Computer Science
- Publisher
- Frontiers Media SA
- Published
- 2023-02-06
- Volume
- 5
- ISSN
- 2624-9898 (ISSN-L)
- OpenAlex
- W4319321305
- Cited by
- 272
- References
- 139
- Citations by year
- 2026: 117 · 2025: 97 · 2024: 46 · 2023: 12
- FWCI
- 25.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Computer Science (publishedVersion, cc-by)
- Digital Commons - Michigan Tech (Michigan Technological University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- explainable AI, user satisfaction, mental models, curiosity, trust in AI, measurement scales, psychometric assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.