Building Trust in Interactive Machine Learning via User Contributed Interpretable Rules
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Machine learning technologies are increasingly being applied in many different domains in the real world. As autonomous machines and black-box algorithms begin making decisions previously entrusted to humans, great academic and public interest has been spurred to provide explanations that allow users to understand the decision-making process of the machine learning model. Besides explanations, Interactive Machine Learning (IML) seeks to leverage user feedback to iterate on an ML solution to correct errors and align decisions with those of the users. Despite the rise in explainable AI (XAI) and Interactive Machine Learning (IML) research, the links between interactivity, explanations, and trust have not been comprehensively studied in the machine learning literature. Thus, in this study, we develop and evaluate an explanation-driven interactive machine learning (XIML) system with the Tic-Tac-Toe game as a use case to understand how a XIML mechanism improves users’ satisfaction with the machine learning system. We explore different modalities to support user feedback through visual or rules-based corrections. Our online user study (n = 199) supports the hypothesis that allowing interactivity within this XIML system causes participants to be more satisfied with the system, while visual explanations play a less prominent (and somewhat unexpected) role. Finally, we leverage a user-centric evaluation framework to create a comprehensive structural model to clarify how subjective system aspects, which represent participants’ perceptions of the implemented interaction and visualization mechanisms, mediate the influence of these mechanisms on the system’s user experience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“…of explanatory machine learning (XML) mechanisms, including in the area of music and movie recommendation, mortgage qualification, visual cues to find the “focus” of deep neural networks in image recognition, and proxy methods to simplify the output of complex systems [Gilpin et al. 2018].”
MethodAnchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“…data instances from past predictions [Gurumoorthy et al. 2019], identifying what features are present or missing to support the prediction for building contrastive explanations [Dhurandhar et al. 2018], and generating interpretable rule-based representations [Dash et al. 2018; Ribeiro et al. 2018].”
BackgroundExplaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 2×
“Herlocker et al. presented an evaluation of a “white box” conceptual model of recommendations, as opposed to the typical black box approach [Herlocker et al. 2000].”
MethodPower to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 2×
“presented the results of a sequence of pre-registered experiments that focus on the number of features and the transparency of themodel [Poursabzi-Sangdeh et al. 2021].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · cited 1×
“For instance, Kaur et al. [Kaur et al. 2020] mentioned that visualizations may help data scientists uncover issues with datasets or models, although the existence of visualizations can also lead to cases of over-trust.”
BackgroundDARPA's explainable artificial intelligence (XAI) program
Gunning, D. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3308446 · Cited by 546
To be or not to be algorithm aware: a question of a new digital divide?
Gran, A.-B., Booth, P., & Bucher, T. (2021) · Information, Communication & Society · doi:10.1080/1369118x.2020.1736124 · Cited by 410 · cited 1×
“…and this may raise questions regarding the equitable distribution of the benefits of XIML systems: if only users who are well-versed in the concept of XAI can benefit from such systems, XIML could become the context for a new digital divide between expert and novice users (cf. [Gran et al. 2020]).”
BackgroundDesign, development and evaluation of a human-computer trust scale
Gulati, S., Sousa, S., & Lamas, D. (2019) · Behaviour & Information Technology · doi:10.1080/0144929x.2019.1656779 · Cited by 229 · cited 1×
“0.844 Perceived system trust[Gulati et al. 2019] (EXP) AVE: NA I believe that there could be negative consequences when using the system.”
BackgroundInfluentialWilder, B., Horvitz, E., & Kamar, E. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/212 · Cited by 118
The Algorithmic Automation Problem: Prediction, Triage, and Human Effort
Raghu, M., Blumer, K., Corrado, G., Kleinberg, J., Obermeyer, Z., & Mullainathan, S. (2019) · arXiv · doi:10.48550/arxiv.1903.12220 · Cited by 58
Interpretable & Explorable Approximations of Black Box Models
Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01154 · Cited by 23 · cited 1×
“In [Lakkaraju et al. 2017], authors built an explainable version of a model and allowed users to interact with the explanations.”
Background
Cited by
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Bertrand, A., Viard, T., Belloum, R., Eagan, J. R., & Maxwell, W. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581314 · Cited by 71
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
“Recent studies have explored variables that contribute to the fairness [31, 76, 124, 130] and trustworthiness [34, 48, 80, 139] of AI systems, as well as the impact of assigning different decision-making roles to humans and AI on the reliance on such systems [52, 103, 122, 144].”
BackgroundBrachman, M., Pan, Q., Do, H. J., Dugan, C., Chaudhary, A., Johnson, J. M., Rai, P., Chakraborti, T., Gschwind, T., Laredo, J. A., et al. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584088 · Cited by 11
“These explanations often focus on feature relevance [19, 52, 129] and the impact of explanations on trust [100, 136], persuasiveness [62], and acceptance of systems [50].”
Background
Metadata
- Authors
- Guo, L., Daly, E. M., Alkan, O., Mattetti, M., Cornec, O., & Knijnenburg, B.
- Venue
- 27th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2022-03-22
- Pages
- 537-548
- OpenAlex
- W4220893555
- Cited by
- 43
- References
- 17
- FWCI
- 5.54
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.