The challenge of crafting intelligible intelligence
Open accessAbstract
To trust the behavior of complex AI algorithms, especially in mission-critical settings, they must be made intelligible.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Providing system explanations may undermine user perceptions when users lack the attentional capacity to process complex explanations, for example when executing a demanding task.
“Providing system explanations may also undermine user perceptions when users lack the attentional capacity to process complex explanations, for example when executing a demanding task [1,12,43,85].”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [85]
Section 6, “Towards Interactive Explanation” (page unavailable), says explanations should suit the audience’s background, and “Ranking Intelligible Models” reports a preference for simpler explanations. Neither passage says that limited attention during a demanding task causes explanations to undermine user perceptions. (LLM, conf=0.78)
Experimental — accuracy not yet measuredGreater transparency potentially increases end user control and improves acceptance of complex algorithmic systems.
“Greater transparency potentially increases end user control and improves acceptance of complex algorithmic systems [31,44,57,85].”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [85]
Users anticipated a preference for transparency before using the system.
“Consistent with our initial expectations and the prior literature on transparency [28,49,85], users anticipated a preference for transparency before using the system.”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [85]
The dependence of ML model-based program behavior on training data introduces various uncertainties into the system's outcomes.
“This phenomenon introduces various uncertainties into the system's outcomes [4,5].”
From Software engineering challenges for machine learning applications: A literature review (2020) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 2× · see citing works
“Indeed, all methods for rendering an inscrutable model intelligible require mapping the complex model to a simpler one.(28) Several high-level approaches to mapping have been proposed.”
BackgroundMethodGrice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1× · see citing works
“We conclude Our final output is a sampled sentence, so we backpropagate the discriminative loss Laysan Albatross Description: However, as this description does not Laysan Albatross Description: This observed that 6 Class Definition: and white belly. the water Class Definition: The detectors [40].”
MethodMastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 2× · see citing works
“Since many AI systems, e.g. , AlphaGo [35], combine deep search and machine learning, additional challenges will result from the need to explain interactions between combinatorics and learned models.”
BackgroundExplaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 2× · see citing works
“visual features from a network pre-trained on ImageNet [6], our model also includes a Þne-grained recognition pipeline to produce strong image features [3]. Like many contemporary description models [7,8,9,10,11], our model learns to generate a sequence of words using an LSTM [12]. However, we design a novel loss func…”
MethodExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 3× · see citing works
“visual features from a network pre-trained on ImageNet [6], our model also includes a Þne-grained recognition pipeline to produce strong image features [3]. Like many contemporary description models [7,8,9,10,11], our model learns to generate a sequence of words using an LSTM [12]. However, we design a novel loss func…”
BackgroundMethodEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 2× · see citing works
“ity.” However, how does one express this trade off? Other examples include balancing training error while uncovering causality in medicine and balancing accuracy and fairness in recidivism prediction [12]. For the latter, a simplified objective function such as accuracy combined with historically biased training data…”
BackgroundMethodAnchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1× · see citing works
“describe a similar optimization algorithm that balances faithfulness and coverage in its search for summary rules.(34) Indeed, all methods for rendering an inscrutable model intelligible require mapping the complex model to a simpler one.”
BackgroundIntelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · cited 2× · see citing works
“er is paying more attention to the background than 7 to the fish itself, and asks to see the training data that influenced the classifier; the nearest neighbors are computed using influence functions [20]. While there are anemones in those images, it also seems as if the system is recognizing clownfish. 4) To gain con…”
BackgroundMethodPower to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · cited 1× · see citing works
“Here, we expect to build on tools for interactive machine learning [1] and explanatory debugging [20, 21], which have explored interactions for adding new training examples, correcting erroneous labels in existing data, specifying new features, and modifying shape functions.”
Anderson, J. R., Boyle, C., & Reiser, B. (1985) · Science · doi:10.1126/science.228.4698.456 · Cited by 699 · see citing works
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · cited 1× · see citing works
“Here, we expect to build on tools for interactive machine learning [1] and explanatory debugging [20, 21], which have explored interactions for adding new training examples, correcting erroneous labels in existing data, specifying new features, and modifying shape functions.”
Explanation, imagination, and confidence in judgment.
Koehler, D. J. (1991) · Psychological Bulletin · doi:10.1037//0033-2909.110.3.499 · Cited by 561 · cited 2× · see citing works
“Koehler found that simply presenting an explanation for a proposition makes people think that it is more likely to be true [18].”
BackgroundIntelligible models for classification and regression
Lou, Y., Caruana, R., & Gehrke, J. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339556 · Cited by 547 · cited 2× · see citing works
“Lou et al. observed that among methods available for learning GA2M models, the version with bagged shallow regression tree shape functions learned via gradient boosting achieves the highest accuracy [27].”
MethodGenerating Visual Explanations
Hendricks, L. A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., & Darrell, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_1 · Cited by 534 · see citing works
Conversational processes and causal explanation.
Hilton, D. J. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.1.65 · Cited by 456 · cited 1× · see citing works
“Social science research suggests that an explanation is best considered a social process, a conversation between explainer and explainee [15, 30].”
BackgroundSimplicity and probability in causal explanation.
Lombrozo, T. (2007) · Cognitive Psychology · doi:10.1016/j.cogpsych.2006.09.006 · Cited by 430 · cited 2× · see citing works
“According to Lombrozo, humans prefer explanations that are simpler ( i.e. , contain fewer clauses), more general, and coherent ( i.e. , consistent with what the human’s prior beliefs) [26].”
XPLAIN: a system for creating and explaining expert consulting programs
Swartout, W. (1983) · Artificial Intelligence · doi:10.1016/s0004-3702(83)80014-9 · Cited by 394 · cited 1× · see citing works
“n practice in AI systems. For example, early rule-based expert systems included explanation systems that augmented a trace of the system’s reasoning — on a particular case — with background knowledge [38]. Recommender systems, which were one of the first deployed uses of machine learning, also induced demand for expla…”
BackgroundAssessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246 · cited 2× · see citing works
“In brief, we seek AI systems where A) it is clear what factors caused the system’s action [24], allowing the users to predict how changes to the situation would have led to alternative behaviors, and B) permits effective control of the AI by enabling interaction.”
BackgroundNeural-Symbolic Learning and Reasoning: A Survey and InterpretationPreprint
Besold, T. R., Garcez, A. D., Bader, S., Bowman, H., Domingos, P., Hitzler, P., Kühnberger, K., Lamb, L. C., Lowd, D., Lima, P. M. V., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03902 · Cited by 243 · cited 2× · see citing works
“s are occupied.” While quantified and relational abstractions are very natural for people, progress in statisticalrelationallearninghasbeenslowandtherearemanyopenquestions for neuro-symbolic learning [3]. MappingControlActions:Generating an explanation by mapping an inscrutable model into a simpler, explanatory model…”
BackgroundMethodTRIPs: an integrated intelligent problem-solving assistant
Ferguson, G., & Allen, J. F. (1998) · Cited by 207 · see citing works
Lewis, D. D. (1987) · doi:10.1093/0195036468.003.0007 · Cited by 206 · see citing works
Explainable PlanningPreprint
Fox, M., Long, D. P., & Magazzeni, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.10256 · Cited by 148 · see citing works
Steps Toward Robust Artificial Intelligence
Dietterich, T. G. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2756 · Cited by 139 · cited 1× · see citing works
“In order to trust deployed AI systems, we must not only improve their robustness [5], but also develop ways to make their reasoning intelligible.”
BackgroundA generalized taxonomy of explanations styles for traditional and social recommender systems
Papadimitriou, A., Symeonidis, P., & Manolopoulos, Y. (2011) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-011-0215-0 · Cited by 126 · cited 1× · see citing works
“…systems, one of the first deployed uses of machine learning, also induced demand for explanations of their specific recommendations; the most satisfying answers combined justifications based on the user’s previous choices, ratings of similar users, and features of the items being recommended [32].”
BackgroundA Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations
Krause, J., Dasgupta, A., Swartz, J. L., Aphinyanaphongs, Y., & Bertini, E. (2017) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2017.8585720 · Cited by 87 · see citing works
Explanatory Coherence and the Induction of Properties
Sloman, S. A. (1997) · Thinking & Reasoning · doi:10.1080/135467897394374 · Cited by 56 · cited 1× · see citing works
“Furthermore, explaining a fact in the same way as previous facts have been explained amplifies this effect [36].”
ResultCauses and Explanations: A Structural-Model Approach
Halpern, J. Y., & Pearl, J. (2000) · eScholarship (California Digital Library) · Cited by 39 · see citing works
Hierarchical Expertise-Level Modeling for User Specific Robot-Behavior ExplanationsPreprint
Sreedharan, S., Srivastava, S., & Kambhampati, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06895 · Cited by 6 · cited 1× · see citing works
“Sreedharan et al. describe an algorithm for generating the minimal explanation that patches a user’s partial understanding of a domain [37].”
Background
Flagged first, then by citations
Research works printed but not linked: 5
- L. A. Hendricks, R. Hu, T. Darrell, and Z. Akata. 2017. Grounding Visual Expla- nations. ArXiv (2017). arXiv:1711.06465 · arXiv
- B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, and R. Sayres. 2017. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). ArXiv e-prints (Nov. 2017). arXiv:stat.ML/1711.11279
- H. Lakkaraju, E. Kamar, R. Caruana, and J. Leskovec. 2017. Interpretable & Explorable Approximations of Black Box Models. KDD-FATML (2017).
- Z. Lipton. 2016. The Mythos of Model Interpretability. In ICML Workshop on Human Interpretability in ML.
- Donald A Norman. 2014. Some Observations on Mental Models. In Mental Models. Psychology Press, 15-22.
Cited by
Sorted by citations
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Explainable machine learning for scientific insights and discoveries
Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J. (2020) · IEEE Access · doi:10.1109/access.2020.2976199 · Cited by 1,040 · see citing works
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811 · see citing works
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768 · see citing works
“Interactive HCAI approaches are endorsed by Weld and Bansal [118], who recommend that designers should “make the explanation system interactive so users can drill down until they are satisfied with their understanding.””
BackgroundInfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · see citing works
“y one is increasing the effectiveness of decision making, while reducing the cost of operations. But increased use of AI brings several well-known dangers as well, including bias and erratic behavior [64]. Many researchers have argued that if AI could explain its reasoning, these , Vol. 1, No. 1, Article . Publication…”
BackgroundInfluentialAI applications to medical images: From machine learning to deep learning
Castiglioni, I., Rundo, L., Codari, M., Leo, G. D., Salvatore, C., Interlenghi, M., Gallivanone, F., Cozzi, A., D’Amico, N. C., & Sardanelli, F. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.02.006 · Cited by 737 · see citing works
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · see citing works
“Intelligibility supports building a mental model of the system that enables user to predict system behavior [207].”
BackgroundInfluentialTrustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612 · see citing works
“Various approaches of interaction have been studied to enhance the explainability of AI, including the visualization of ML models [72] and interactive parameter-tuning [351].”
BackgroundMethodCao, G., Duan, Y., Edwards, J. S., & Dwivedi, Y. K. (2021) · Technovation · doi:10.1016/j.technovation.2021.102312 · Cited by 571 · see citing works
“…(e.g. Agrawal et al., 2017a; Duan et al., 2019; Dwivedi et al., 2021) and concerns about the “ dark side ” of AI (e.g. Shrestha et al., 2019; Weld and Bansal, 2019), there seems to still be a need for a theoretical model to facilitate the integration of conceptual development and empirical…”
BackgroundInfluentialKaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · see citing works
Stephanidis, C., Salvendy, G., Antona, M., Chen, J. Y. C., Dong, J., Duffy, V. G., Fang, X., Fidopiastis, C., Fragomeni, G., Fu, L. P., et al. (2019) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2019.1619259 · Cited by 524 · see citing works
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · see citing works
“Recent work has also begun to explore interactive explanations that could address users’ follow-up questions as a way to fill individual knowledge gaps [91, 104].”
BackgroundExplaining a series of models by propagating Shapley values
Chen, H., Lundberg, S., & Lee, S. (2022) · Nature Communications · doi:10.1038/s41467-022-31384-3 · Cited by 289 · see citing works
Sokol, K., & Flach, P. A. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372870 · Cited by 278 · see citing works
“Adjusting an explanation to users requires the explainability technique to model their background knowledge and mental model [48, 55].”
BackgroundMethodInfluentialSurvey on Explainable AI: From Approaches, Limitations and Applications Aspects
Yang, W., Wei, Y.-C., Wei, H., Chen, Y., Huang, G., Li, X., Li, R., Yao, N., Wang, X., Gu, X., et al. (2023) · Human-Centric Intelligent Systems · doi:10.1007/s44230-023-00038-y · Cited by 275 · see citing works
“Additionally, some regulations such as European General Data Protection Regulation (general data protection regulation (GDPR))[4] have been introduced to drive further XAI research, demanding the important ethics [5], justifications [6], trust [7] and bias [8] to explore reliable XAI solutions.”
BackgroundZhang (2021) · doi:10.1145/3432945 · Cited by 225 · see citing works
“Prior work indicates evidence that AI with intelligible, accountable, and explainable features are more likely to build trust with humans [62].”
BackgroundRapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · see citing works
“26 The training data itself, in addition to the mechanics of training, also contribute to the objective mismatch problem.”
BackgroundSokol, K., & Flach, P. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00637-y · Cited by 186 · see citing works
“Secondly, the work of Krause et al. [16] and Weld and Bansal [44] focused on interactive (multi-modal) explainability systems.”
BackgroundMethodInfluentialMolina, M. D., & Sundar, S. S. (2022) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmac010 · Cited by 179 · see citing works
From explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180 · see citing works
Ethics of Artificial Intelligence in Medicine and Ophthalmology
Abdullah, Y. I., Schuman, J. S., Shabsigh, R., Caplan, A. L., & Al‐Aswad, L. A. (2021) · Asia-Pacific Journal of Ophthalmology · doi:10.1097/apo.0000000000000397 · Cited by 159 · see citing works
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144 · see citing works
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129 · see citing works
“The idea of explaining not only the AI's prediction but also alternative possibilities appeared in several papers [12], [34], [42] as a way to counter automation bias.”
BackgroundMethodInfluentialExplaining machine learning models with interactive natural language conversations using TalkToModel
Slack, D., Krishna, S., Lakkaraju, H., & Singh, S. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00692-8 · Cited by 125 · see citing works
“Yet, recent work suggests practitioners often have difficulty using explainability techniques [35, 31, 79].”
BackgroundIs the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Bansal, G., Nushi, B., Kamar, E. S., Horvitz, E., & Weld, D. S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17359 · Cited by 129 · see citing works
“…aimed at enhancing the understandability of complex AI inference (Horvitz et al. 1986), interpretability of machine-learned models (Caruana et al. 2015; Weld and Bansal 2019), and performing post-hoc explanations of the output of models (Ribeiro, Singh, and Guestrin 2016; Lund-berg and Lee 2017).”
MethodBuilding Human Values into Recommender Systems: An Interdisciplinary Synthesis
Stray, J., Halevy, A. Y., Assar, P., Hadfield-Menell, D., Boutilier, C. E., Ashar, A., Bakalar, C., Beattie, L., Ekstrand, M. D., Leibowicz, C., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3632297 · Cited by 145 · see citing works
“From a technical perspective, explanations can be tricky to generate because many recommender techniques (including deep learning) do not lend themselves to easy explanations (Weld and Bansal, 2019).”
BackgroundInteractive machine teaching: a human-centered approach to building machine-learned models
Ramos, G., Meek, C., Simard, P., Suh, J., & Ghorashi, S. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1734931 · Cited by 120 · see citing works
“From a growing body of research in Explainable AI (XAI), there are two main approaches for developing explainable machine learning models (Weld & Bansal, 2019).”
BackgroundMatthews, G., Hancock, P., Lin, J., Panganiban, A., Reinerman-Jones, L., Szalma, J., & Wohleber, R. W. (2021) · Personality and Individual Differences · doi:10.1016/j.paid.2020.109969 · Cited by 91 · see citing works
Trusting Intelligent Machines: Deepening Trust Within Socio-Technical Systems
Andras, P., Esterle, L., Guckert, M., Han, T. A., Lewis, P. R., Milanovic, K., Payne, T., Perret, C., Pitt, J., Powers, S. T., et al. (2018) · IEEE Technology and Society Magazine · doi:10.1109/mts.2018.2876107 · Cited by 153 · see citing works
“In some areas, for example Deep Mind's Alpha Go Zero, the results appear to exceed human ability; some moves are made that are novel and inexplicable to human Goplaying experts and yet are effective, leading to more wins and new insights into the game [16].”
BackgroundTowards Trustable Explainable AI
Ignatiev, A. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/726 · Cited by 114 · see citing works
“Explainable AI is an emerging field, with a growing number of areas of research [ Weld and Bansal, 2019 ] .”
BackgroundAnchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · see citing works
“Theoretically, explanations should help users build a better mental model of an intelligent system [48].”
BackgroundMeddage, D. P. P., Ekanayake, I. U., Weerasuriya, A. U., Lewangamage, C., Tse, K. T., Miyanawala, T. P., & Ramanayaka, C. E. (2022) · Journal of Wind Engineering and Industrial Aerodynamics · doi:10.1016/j.jweia.2022.105027 · Cited by 99 · see citing works
AlQuraishi, M., & Sorger, P. K. (2021) · Nature Methods · doi:10.1038/s41592-021-01283-4 · Cited by 96 · see citing works
Neuroprediction and A.I. in Forensic Psychiatry and Criminal Justice: A Neurolaw Perspective
Tortora, L., Meynen, G., Bijlsma, J., Tronci, E., & Ferracuti, S. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00220 · Cited by 99 · see citing works
“algorithms can be made intelligible, in order to be accountable for their decisions (Weld and Bansal, 2019).”
BackgroundAn Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction
Paranjape, B., Joshi, M., Thickstun, J., Hajishirzi, H., & Zettlemoyer, L. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.153 · Cited by 94 · see citing works
Software engineering challenges for machine learning applications: A literature review
Kumeno, F. (2020) · Intelligent Decision Technologies · doi:10.3233/idt-190160 · Cited by 81 · see citing works
“Developing the skill set for ML applications is the major challenge related to this KA [15,16,27,32,39,40,59].”
A Survey of Human‐Centered Evaluations in Human‐Centered Machine Learning
Sperrle, F., El‐Assady, M., Guo, G., Borgo, R., Chau, D. H., Endert, A., & Keim, D. A. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14329 · Cited by 79 · see citing works
Amiri, S., Mottahedi, S., Lee, E., & Hoque, S. (2021) · Computers, Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2021.101647 · Cited by 75 · see citing works
Artificial intelligence and administrative evil
Young, M. M., Himmelreich, J., Bullock, J. B., & Kim, K.-C. (2021) · Perspectives on Public Management and Governance · doi:10.1093/ppmgov/gvab006 · Cited by 76 · see citing works
False Positives vs. False Negatives
Lafreniere, B., Jonker, T. R., Santosa, S., Parent, M., Glueck, M., Grossman, T., Benko, H., & Wigdor, D. (2021) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3472749.3474735 · Cited by 76 · see citing works
“The intelligibility and interpretability of software that uses machine learning has been identified as key factors to user tolerance of errors [5, 24].”
MethodShallow and deep learning classifiers in medical image analysis
Prinzi, F., Currieri, T., Gaglio, S., & Vitabile, S. (2024) · European Radiology Experimental · doi:10.1186/s41747-024-00428-2 · Cited by 72 · see citing works
“Fortunately, in recent years, explainable AI has emerged to address the problem of poor interpretability, to make the learned logic accessible and the process understandable by humans [80–83].”
BackgroundRight for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations
Stammer, W., Schramowski, P., & Kersting, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00362 · Cited by 73 · see citing works
“These contributions are important to make progress towards creating conversational explanations between machines and human users [53, 33].”
BackgroundShneiderman, B. (2021) · Communications of the ACM · doi:10.1145/3445973 · Cited by 71 · see citing works
Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
Kaur, H., Adar, E., Gilbert, É., & Lampe, C. A. C. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533135 · Cited by 73 · see citing works
“, [12, 25, 32, 82, 83, 121, 126]) with the motivation that by translating ideas from how people explain things to each other, we can design better solutions for how ML models can be explained to people.”
BackgroundWagan, S. A., Koo, J., Siddiqui, I. F., Attique, M., Shin, D. R., & Qureshi, N. M. F. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.09.005 · Cited by 76 · see citing works
Nagahisarchoghaei, M., Nur, N., Cummins, L., Nur, N., Karimi, M. M., Nandanwar, S., Bhattacharyya, S., & Rahimi, S. (2023) · Electronics · doi:10.3390/electronics12051092 · Cited by 76 · see citing works
NICE: an algorithm for nearest instance counterfactual explanations
Brughmans, D., Leyman, P., & Martens, D. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00930-y · Cited by 64 · see citing works
“Alternatively, Weld & Bansal [69] argue in favor of interactive explanation systems, as users may have follow-up questions and more detailed concerns once an (initial) explanation has been provided.”
BackgroundPose Tutor: An Explainable System for Pose Correction in the Wild
Dittakavi, B., Bavikadi, D., Desai, S. V., Chakraborty, S., Reddy, N., Balasubramanian, V. N., Callepalli, B., & Sharma, A. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00398 · Cited by 64 · see citing works
Mental Models of Mere Mortals with Explanations of Reinforcement Learning
Anderson, A. W., Dodge, J. E., Sadarangani, A., Juozapaitis, Z., Newman, E., Irvine, J., Chattopadhyay, S., Olson, M., Fern, A., & Burnett, M. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3366485 · Cited by 63 · see citing works
Ferrario, A., Facchini, A., & Termine, A. (2024) · Minds and Machines · doi:10.1007/s11023-024-09681-1 · Cited by 64 · see citing works
“Yet, as testified by several critical works, e.g., (Weld & Bansal, 2019; Krishnan, 2020; Watson, 2022; Miller, 2023), there are serious doubts that such move indeed constitutes a successful ‘socio-technical fix.’”
Explainable machine learning for project management control
Santos, J. I., Pereda, M., Ahedo, V., & Galán, J. M. (2023) · Computers & Industrial Engineering · doi:10.1016/j.cie.2023.109261 · Cited by 61 · see citing works
Liao, Q. V., Zhang, Y., Luss, R., Doshi-Velez, F., & Dhurandhar, A. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21995 · Cited by 69 · see citing works
Stakeholder Perceptions of the Safety and Assurance of Artificial Intelligence in Healthcare
Sujan, M. A., White, S., Habli, I., & Reynolds, N. (2022) · Safety Science · doi:10.1016/j.ssci.2022.105870 · Cited by 58 · see citing works
In defence of machine learning: Debunking the myths of artificial intelligence
Laurent, C. D. S. (2018) · Europe’s Journal of Psychology · doi:10.5964/ejop.v14i4.1823 · Cited by 56 · see citing works
Eight human factors and ergonomics principles for healthcare artificial intelligence
Sujan, M. A., Pool, R., & Salmon, P. M. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100516 · Cited by 54 · see citing works
Fok, R., & Weld, D. S. (2024) · AI Magazine · doi:10.1002/aaai.12182 · Cited by 81 · see citing works
Fok, R., & Weld, D. S. (2023) · arXiv · doi:10.48550/arxiv.2305.07722 · Cited by 54 · see citing works
Self-citationExplainExplore: Visual Exploration of Machine Learning Explanations
Collaris, D., & van Wijk, J. J. (2020) · IEEE Pacific Visualization Symposium · doi:10.1109/pacificvis48177.2020.7090 · Cited by 52 · see citing works
“There are two main approaches in this field [21, 57]: either a model is used that is inherently interpretable, or an explanation is generated by means of a surrogate model.”
BackgroundMethodResultInfluentialFoundations of explanations as model reconciliation
Sreedharan, S., Chakraborti, T., & Subbarao Kambhampati, B. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103558 · Cited by 51 · see citing works
Expert Level Evaluations for Explainable AI (XAI) Methods in the Medical Domain
Muddamsetty, S. M., Jahromi, M. N. S., & Moeslund, T. B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68796-0_3 · Cited by 51 · see citing works
Zouinar, M. (2020) · Activites · doi:10.4000/activites.4941 · Cited by 54 · see citing works
Principles and Practice of Explainable Machine Learning
Belle, V., & Papantonis, I. (2021) · Frontiers in Big Data · doi:10.3389/fdata.2021.688969 · Cited by 99 · see citing works
“She again tries various candidates and she finds out that Random Forests achieve the 5Note that this informal view encourages a notional plot of explainability versus accuracy, as is common in informal discussions on the challenge of XAI [32, 88].”
BackgroundNaveed, S., Stevens, G., & Robin-Kern, D. (2024) · Applied Sciences · doi:10.3390/app142311288 · Cited by 54 · see citing works
Human-centered explainability for life sciences, healthcare, and medical informatics
Dey, S., Chakraborty, P., Kwon, B. C., Dhurandhar, A., Ghalwash, M. F., Suárez, F., Ng, K., Sow, D., Varshney, K. R., & Meyer, P. (2022) · Patterns · doi:10.1016/j.patter.2022.100493 · Cited by 47 · see citing works
“82 We will now present the four different types of visualization that exist and their motivation (Figure 5), while also discussing the use of these types of interactive visualizations for three kinds of personas: researchers/data scientists, clinical researchers, and clinicians (Figure 6).”
BackgroundResearching AI Legibility through Design
Lindley, J., Akmal, H. A., Pilling, F. L., & Coulton, P. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376792 · Cited by 45 · see citing works
Machine Thinking, Fast and Slow
Bonnefon, J., & Rahwan, I. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.007 · Cited by 45 · see citing works
“Operating on such a sparse network would also help the machine to express its conclusions in terms of explicit causal models with rules and reasons, something that Fast Thinking can struggle with [19,20].”
Backgrounddel Gobbo, E., Guarino, A., Cafarelli, B., & Grilli, L. (2023) · Knowledge and Information Systems · doi:10.1007/s10115-023-01892-9 · Cited by 41 · see citing works
Personalized chatbot trustworthiness ratings
Srivastava, B., Rossi, F., Usmani, S., & Bernagozzi, M. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.3023919 · Cited by 40 · see citing works
“For example, trust in an AI model is often related (also) to the possibility to get an explanation for its output (see [11], [25], [34]), so explainability could be another trust issue to consider in the future.”
BackgroundOn Efficiently Explaining Graph-Based Classifiers
Huang, X., Izza, Y., Ignatiev, A., & Marques-Silva, J. P. (2021) · Principles of Knowledge Representation and Reasoning · doi:10.24963/kr.2021/34 · Cited by 38 · see citing works
Prinzi, F., Militello, C., Scichilone, N., Gaglio, S., & Vitabile, S. (2023) · IEEE Access · doi:10.1109/access.2023.3327808 · Cited by 38 · see citing works
“Although stingy validation protocols may allow detecting the overfitting problem on the training distribution, the model may perform poorly when a distribution encountered during deployment is slightly different [65].”
BackgroundNon-perturbative renormalization for the neural network-QFT correspondence
Erbin, H., Lahoche, V., & Samary, D. O. (2022) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ac4f69 · Cited by 35 · see citing works
“Our paper is part of the lively field of explainable AI [5, 6], where physics do have a role to play [7].”
BackgroundBorrego-Díaz, J., & Galán-Páez, J. (2022) · Minds and Machines · doi:10.1007/s11023-022-09603-z · Cited by 34 · see citing works
“This is the case with complex deep neural networks or enhanced decision forests (Weld & Bansal, 2019).”
BackgroundHernandez-Bocanegra, D. C., & Ziegler, J. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3579541 · Cited by 33 · see citing works
“Despite the potential benefit of using these models to increase users’ understanding of intelligent systems [82, 117], their practical implementation in RS (and in XAI in general) still lacks sufficient empirical validation [77, 82, 103].”
Tractable Explanations for d-DNNF Classifiers
Huang, X., Izza, Y., Ignatiev, A., Cooper, M., Asher, N. M., & Marques-Silva, J. P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20514 · Cited by 32 · see citing works
“Although recent years have witnessed a growing interest in finding explanations of machine learning (ML) models (Lip-ton 2018; Guidotti et al. 2019; Weld and Bansal 2019; Mon-roe 2021), explanations have been studied from different perspectives and in different branches of AI at least since the 80s…”
BackgroundWhy we do need explainable AI for healthcare
Cinà, G., Röber, T. E., Goedhart, R., & Birbil, Ş. İ. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00209-4 · Cited by 39 · see citing works
A model validation of robo-advisers for stock investment
Shiva, A., Kushwaha, B. P., & Rishi, B. (2023) · Borsa Istanbul Review · doi:10.1016/j.bir.2023.09.005 · Cited by 38 · see citing works
Zhang, R., Flathmann, C., Musick, G., Schelble, B. G., McNeese, N. J., Knijnenburg, B. P., & Duan, W. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3635474 · Cited by 33 · see citing works
“As such, much of explainable AI research has focused on how to create human-friendly explanations, which often require a directional conversation between collaborators [1, 2, 107].”
BackgroundLeveraging explainable AI for sustainable agriculture: a comprehensive review of recent advances
Rajbongshi, A., Johora, F. T., Hossain, A., Sarker, M. S., Rahman, M. H., Rahman, M. W., Alotaibi, F. T., & Moni, M. A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11459-5 · Cited by 39 · see citing works
“(Stahl and Wright 2018 ; Weld and Bansal 2019; Lui and Lamb 2018; Hengstler et al. 2016 ; Chen 2019 ).”
Jin, W., Fatehi, M., Guo, R., & Hamarneh, G. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102751 · Cited by 31 · see citing works
On the Ethical and Epistemological Utility of Explicable AI in Medicine
Herzog, C. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00546-y · Cited by 30 · see citing works
“In this vein, the notion “intelligibility” is often regarded as a generic term, encompassing the possibility of referring to “interpretability” or “explainability” (or both), cf. (Marcinkevičs and Vogt 2020; Weld and Bansal 2018), Weld and Bansal (2018) acknowledge a necessity of interdisciplinary cooperation when try…”
BackgroundInfluentialA Computational Inflection for Scientific Discovery
Hope, T., Downey, D., Weld, D. S., Etzioni, O., & Horvitz, E. (2023) · Communications of the ACM · doi:10.1145/3576896 · Cited by 34 · see citing works
“One limitation is that the representations learned by these models are currently far from understood and lack “hooks” for control and interpretability, critical in human-AI collaboration [60].”
BackgroundNourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E. D., & Gogate, V. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3531066 · Cited by 30 · see citing works
“Theoretically, explanations should help users build a better mental model of an intelligent system [79].”
BackgroundExplainable zero-shot learning via attentive graph convolutional network and knowledge graphs
Geng, Y.-X., Chen, J.-Y., Ye, Z.-Q., Yuan, Z.-G., Zhang, W., & Chen, H. (2021) · Semantic Web · doi:10.3233/sw-210435 · Cited by 36 · see citing works
“Early algorithms focus on utilizing class attributes to model the semantic relationship of classes [6, 12– 14].”
BackgroundEverett, S., Bunning, B., Jain, P., López, I., Agarwal, A., Desai, M., Gallo, R. J., Goh, E., Kadiyala, V. B., Kanjee, Z., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02545-1 · Cited by 56 · see citing works
Artificial Intelligence and Machine Learning
Mich, L. (2020) · Springer eBooks · doi:10.1007/978-3-030-05324-6_25-1 · Cited by 30 · see citing works
The “Collections as ML Data” checklist for machine learning and cultural heritage
Lee, B. C. G. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24765 · Cited by 27 · see citing works
“Indeed, the research fields of explainable AI and human-AI interaction are devoted to understanding the implications of relying on such models (Weld & Bansal, 2019).”
EEG Waveform Analysis of P300 ERP with Applications to Brain Computer Interfaces
Ramele, R., Villar, A. J., & Santos, J. M. (2018) · Brain Sciences · doi:10.3390/brainsci8110199 · Cited by 27 · see citing works
“This brings the possibility to use these techniques to implement intelligible [84] automatic detection procedures, i.”
MethodInterpretable Radiomic Signature for Breast Microcalcification Detection and Classification
Prinzi, F., Orlando, A. A. M., Gaglio, S., & Vitabile, S. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01012-1 · Cited by 26 · see citing works
“As shown in [51] there are important technical and social reasons to prefer inherently intelligible AI models over deep neural models.”
On the Intersection of Explainable and Reliable AI for Physical Fatigue Prediction
Narteni, S., Orani, V., Cambiaso, E., Rucco, M., & Mongelli, M. (2022) · IEEE Access · doi:10.1109/access.2022.3191907 · Cited by 26 · see citing works
Recio-García, J. A., Díaz-Agudo, B., & Pino-Castilla, V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58342-2_12 · Cited by 25 · see citing works
From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence
Alvarez-Melis, D., Kaur, H., Daumé, H., Wallach, H., & Vaughan, J. W. (2021) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v9i1.18938 · Cited by 25 · see citing works
“This is in line with prior work on human-centered design principles for ML (Amershi et al. 2019; Hohman et al. 2019; Weld and Bansal 2019).”
ResultRobo-advisory financial services and the dynamics of new innovation in Saudi Arabia
Ansari, Y., & Bansal, R. (2024) · Journal of Open Innovation Technology Market and Complexity · doi:10.1016/j.joitmc.2024.100397 · Cited by 24 · see citing works
Different "Intelligibility" for Different Folks
Zhou, Y., & Danks, D. J. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375810 · Cited by 24 · see citing works
“For example, Engineers often need information about counterfactual performance or differencemaking relationships [20], and that is exactly what someone with Affectee goals requires.”
BackgroundToward Affective XAI: Facial Affect Analysis for Understanding Explainable Human-AI Interactions
Guerdan, L., Raymond, A., & Gunes, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00423 · Cited by 24 · see citing works
“While approaches such as these can improve human decisionmaking and increase trust in AI systems [41], many existing XAI methods remain centered on technical aspects of explanations and fail to account for cognitive-emotional processes [35].”
BackgroundLi, Y., Gupta, V., Kilic, M. N. T., Choudhary, K. K., Wines, D., Liao, W., Choudhary, A., & Agrawal, A. (2024) · Digital Discovery · doi:10.1039/d4dd00199k · Cited by 26 · see citing works
A social evaluation of the perceived goodness of explainability in machine learning
Wanner, J., Herm, L.-V., Heinrich, K., & Janiesch, C. (2021) · Journal of Business Analytics · doi:10.1080/2573234x.2021.1952913 · Cited by 22 · see citing works
“Therefore, if an algo-rithm uses an unintuitive learning behaviour, this leads to a higher scepticism of the user (Weld & Bansal, 2019).”
Expert-Informed, User-Centric Explanations for Machine Learning
Pazzani, M. J., Soltani, S., Kaufman, R., Qian, S., & Hsiao, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21491 · Cited by 20 · see citing works
“However, much of recent XAI for inscrutable models (Weld & Bansal, 2019) such as deep learning has used the intuitions of developers as a guide for creating explanations.”
BackgroundGAM Coach: Towards Interactive and User-centered Algorithmic Recourse
Wang, Z. J., Vaughan, J. W., Caruana, R., & Chau, D. H. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580816 · Cited by 20 · see citing works
“” Our findings confirm hypotheses that recourse preferences can be incorporated to identify satisfactory plans [3, 94], and these preferences are idiosyncratic [42, 87].”
BackgroundSuper-machines or sub-humans: Mental models and trust in intelligent autonomous systems
Matthews, G., Panganiban, A. R., Lin, J., Long, M. D., & Schwing, M. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-819472-0.00003-4 · Cited by 20 · see citing works
- Authors
- Weld, D. S., & Bansal, G.
- Affiliations
- University of Washington (US)
- Venue
- Communications of the ACM
- Published
- 2019-05-21
- Volume
- 62
- Issue
- 6
- Pages
- 70-79
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2811033754
- Cited by
- 253
- References
- 29
- Citations by year
- 2026: 25 · 2025: 26 · 2024: 25 · 2023: 42 · 2022: 37 · 2021: 45 · 2020: 39 · 2019: 10 · 2018: 4
- FWCI
- 14.78
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Communications of the ACM (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-1420667)
- Future of Life Institute (2015-144577)
- Washington Research Foundation (WRF/TJ Cable Professorship)
- Office of Naval Research (N00014-15-1-2774)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- explainable AI, trust in AI, mission-critical systems, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
