The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Abstract
The wide adoption of Machine Learning (ML) technologies has created a growing demand for people who can train ML models. Some advocated the term "machine teacher'' to refer to the role of people who inject domain knowledge into ML models. This "teaching'' perspective emphasizes supporting the productivity and mental wellbeing of machine teachers through efficient learning algorithms and thoughtful design of human-AI interfaces. One promising learning paradigm is Active Learning (AL), by which the model intelligently selects instances to query a machine teacher for labels, so that the labeling workload could be largely reduced. However, in current AL settings, the human-AI interface remains minimal and opaque. A dearth of empirical studies further hinders us from developing teacher-friendly interfaces for AL algorithms. In this work, we begin considering AI explanations as a core element of the human-AI interface for teaching machines. When a human student learns, it is a common pattern to present one's own reasoning and solicit feedback from the teacher. When a ML model learns and still makes mistakes, the teacher ought to be able to understand the reasoning underlying its mistakes. When the model matures, the teacher should be able to recognize its progress in order to trust and feel confident about their teaching outcome. Toward this vision, we propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the surging field of explainable AI (XAI) into an AL setting. We conducted an empirical study comparing the model learning outcomes, feedback content and experience with XAL, to that of traditional AL and coactive learning (providing the model's prediction without explanation). Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and additional cognitive workload. Our study also reveals important individual factors that mediate a machine teacher's reception to AI explanations, including task knowledge, AI experience and Need for Cognition. By reflecting on the results, we suggest future directions and design implications for XAL, and more broadly, machine teaching through AI explanations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The authors suggest two types of human intuition useful for detecting model errors-intuition about feature relevance and intuition about feature mechanism (e.g., weight), which align with prior findings about how experts critique model explanations in interactive ML settings [33,75].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [33]
In the citing paper
“Some user groups are more vulnerable to these harms than others, such as AI novices [60], people working in cognitively constrained settings [52], and even those with certain personality traits [17].”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [17]
In the citing paper
“For example, recent studies have repeatedly found that XAI features bring less benefit, or even harms (leading to over-trust), to certain user groups such as AI novices or users working in cognitively constrained settings [17, 52, 60].”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [17]
In the citing paper
“This task is suitable for our study since it requires little domain expertise and imposes relatively limited risks, and thus is amenable for non-expert participants [39].”
From Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making (2023) · cites this paper as [39]
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 3×
“[62, 72]) is a cluster of XAI techniques that explain how the model arrived at a particular decision.”
BackgroundMethod"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Developing and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550 · cited 3×
“Trust in technologies is frequently measured based on McKnight’s framework on Trust [63, 64], which considers the dimensions of capability, benevolence, integrity for trust belief, and multiple actionbased items (e.”
MethodInfluentialLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 3×
“The first two categories, model and outcome explanations, are also referred as global and local explanations [6, 59, 66].”
BackgroundCacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359 · cited 2×
“We also explore the effect of Need for cognition [12], defined as an individual’s tendency to engage in thinking or complex cognitive activities.”
BackgroundMethod(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 4×
“While Machine Learning technologies are increasingly used in a wide variety of domains ranging from critical systems to everyday consumer products, currently only a small group of people with formal training possess the skills to develop these technologies.”
BackgroundMethodInfluentialIBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use
Lewis, J. R. (1995) · International Journal of Human-Computer Interaction · doi:10.1080/10447319509526110 · Cited by 2,429 · cited 2×
“Satisfaction of the annotation experience, by five items adapted from After-Scenario Questionnaire [54] and User Engagement Scale [68] (e.”
MethodAnchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“Other design choices for local explanations include presenting similar examples with the same known outcome [9, 37], and rules that the model believes to guarantee the prediction [73] (e.”
BackgroundMachine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · cited 3×
“Common goals of AI explanations, as reflected in much of the XAI literature, are to support a complete and sound understanding of the model [16, 48], and to foster trust in the AI [19, 69].”
BackgroundA sequential algorithm for training text classifiers
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,731
Seung, H. S., Opper, M., & Sompolinsky, H. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130417 · Cited by 1,651 · cited 1×
“To select the next instance for labeling, multiple query sampling strategies have been proposed in the literature [25, 26, 33, 41, 53, 79, 80].”
BackgroundImproving generalization with active learning
Cohn, D., Atlas, L., & Ladner, R. (1994) · Machine Learning · doi:10.1007/bf00993277 · Cited by 1,591 · cited 1×
“AL can be used in different scenarios like stream based [23] (from a stream of incoming data), pool based [52] (from a large set of unlabeled instances), etc.”
BackgroundInitial trust formation in new organizational relationships
McKnight, D. H., Cummings, L. L., & Chervany, N. L. (1998) · The Academy of Management Review · doi:10.2307/259290 · Cited by 1,296 · cited 1×
“Trust in technologies is frequently measured based on McKnight’s framework on Trust [63, 64], which considers the dimensions of capability , benevolence , integrity for trust belief, and multiple action-based items (e.g., "I will be able to rely on the system for the target task") for trust…”
MethodSelective Sampling Using the Query by Committee Algorithm
Freund, Y., Seung, H. S., Shamir, E., & Tishby, N. (1997) · Machine Learning · doi:10.1023/a:1007330508534 · Cited by 1,097 · cited 1×
“To select the next instance for labeling, multiple query sampling strategies have been proposed in the literature [25, 26, 33, 41, 53, 79, 80].”
BackgroundPower to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · cited 6×
“For example, seeing suboptimal model performance, a data scientist has to spend extensive time obtaining additional labeled data from the SMEs, or gathering other feedback which helps in feature engineering or other steps in the ML development process [3, 10].”
BackgroundInfluentialO’Brien, H. L., Cairns, P., & Hall, M. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.01.004 · Cited by 1,014 · cited 2×
“Satisfaction of the annotation experience, by five items adapted from After-Scenario Questionnaire [54] and User Engagement Scale [68] (e.”
MethodAn analysis of active learning strategies for sequence labeling tasks
Settles, B., & Craven, M. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613855 · Cited by 972 · cited 3×
“Different AL algorithms exploit different notions of uncertainty, e.g. entropy [79], confidence [25], margin [7], etc.”
BackgroundMethodQuestioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · cited 4×
“The HCI community have defined explanation taxonomies based on different types of user needs, often referred as intelligibility types [55, 57, 58] .”
BackgroundInfluentialSelf-citationDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · cited 1×
“There have been many recent efforts to categorize the ever-growing collection of explanation techniques [5, 6, 35, 58, 59, 66, 94].”
BackgroundZhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 3×
“Explanations of AI are sought for various reasons, such as by regulators to assess model compliance, or by end users to support their decision-making [55, 93, 100].”
BackgroundSelf-citationEffects of need for cognition on message evaluation, recall, and persuasion.
Cacioppo, J. T., Petty, R. E., & Morris, K. J. (1983) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.45.4.805 · Cited by 849 · cited 1×
“Need for cognition has been extensively researched in social and cognitive psychology as a mediating factor for how one responds to cognitively demanding tasks (e.g. [13, 39]).”
BackgroundA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
The Oxford Handbook of Thinking and Reasoning
(2012) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199734689.001.0001 · Cited by 697 · cited 1×
“The explanations also enable the teacher to gauge the students’ grasp of new concepts, reinforce successful learning, correct misunderstanding, repair gaps, as well as adjust the teaching strategies [61].”
BackgroundFails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633 · cited 3×
“However, conventional AL systems, with a minimum interface asking for labels, lack the fundamental element in iML–a tight interaction loop that transparently presents how every human input impacts the model, so that the non-ML-experts could adapt their input to drive the model into desired directions [3, 31].”
BackgroundA Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 4×
“, example or rule based explanations [35]), given the former approach’s popularity and intuitiveness– it reflects how the model weighs different features and gives people direct access to the inner working of the model.”
BackgroundInfluentialPrinciples of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · cited 10×
“, output, performance, features used [31, 32, 48, 75]).”
BackgroundMethodInfluentialWhy and why not explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · cited 2×
“The HCI community have defined explanation taxonomies based on different types of user needs, often referred as intelligibility types [55, 57, 58] .”
BackgroundThe What-If Tool: Interactive Probing of Machine Learning Models
Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viegas, F., & Wilson, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934619 · Cited by 505 · cited 1×
“This idea underlies recent research into visual analytical tools that support debugging or feature engineering work [40, 47, 96].”
BackgroundHaugtvedt, C. P., & Petty, R. E. (1992) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.63.2.308 · Cited by 491 · cited 1×
“Need for cognition has been extensively researched in social and cognitive psychology as a mediating factor for how one responds to cognitively demanding tasks (e.g. [13, 39]).”
BackgroundHierarchical sampling for active learning
Dasgupta, S., & Hsu, D. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390183 · Cited by 444 · cited 1×
“To select the next instance for labeling, multiple query sampling strategies have been proposed in the literature [25, 26, 33, 41, 53, 79, 80].”
BackgroundActive Learning by Querying Informative and Representative Examples
Huang, S.-J., Jin, R., & Zhou, Z.-H. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2307881 · Cited by 440 · cited 1×
“To select the next instance for labeling, multiple query sampling strategies have been proposed in the literature [25, 26, 33, 41, 53, 79, 80].”
BackgroundInteracting with Predictions: Visual Inspection of Black-box Machine Learning Models
Krause, J., Perer, A., & Ng, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858529 · Cited by 390 · cited 1×
“This idea underlies recent research into visual analytical tools that support debugging or feature engineering work [40, 47, 96].”
BackgroundToo much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · cited 3×
“An important lesson from the bulk of iML work is that users value transparency beyond performance [49, 75], such as descriptive information about how the algorithm works or what features are used [48, 75].”
BackgroundLai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · cited 1×
“, only a subset of top features [51]), or have people first make their own judgment before seeing the explanation.”
BackgroundReducing Labeling Effort for Structured Prediction Tasks
Culotta, A., & McCallum, A. (2005) · doi:10.21236/ada440382 · Cited by 356 · cited 3×
“To select the next instance for labeling, multiple query sampling strategies have been proposed in the literature [25, 26, 33, 41, 53, 79, 80].”
BackgroundMethodTheoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317 · cited 1×
“We also consulted a recent paper on trust scale for automation [45] and added the dimension of predictability for trust belief.”
MethodProxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · cited 1×
“While recent studies of XAI often found explanations to improve users’ understanding of AI systems [11, 19, 43], empirical results regarding its impact on users’ subjective experience such as trust [19, 69, 100] and acceptance [43] have been mixed.”
BackgroundExplaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 3×
“Common goals of AI explanations, as reflected in much of the XAI literature, are to support a complete and sound understanding of the model [16, 48], and to foster trust in the AI [19, 69].”
BackgroundExplaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286
Self-citationProceedings of the 17th international conference on World Wide Web
(2008) · doi:10.1145/1367497 · Cited by 281
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277 · cited 1×
“While recent studies of XAI often found explanations to improve users’ understanding of AI systems [11, 19, 43], empirical results regarding its impact on users’ subjective experience such as trust [19, 69, 100] and acceptance [43] have been mixed.”
BackgroundBalcan, M.-F., Broder, A., & Zhang, T. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-72927-3_5 · Cited by 272 · cited 2×
“entropy [79], confidence [25], margin [7], etc.”
MethodExplainability fact sheets: a framework for systematic assessment of explainable approaches
Sokol, K., & Flach, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372870 · Cited by 264 · cited 1×
“For the machine teacher, desired properties of explanations may include easiness to judge, non-repetitiveness, tailored to their preferences and tendency to provide feedback, etc. [85].”
BackgroundUTOPIAN: User-Driven Topic Modeling Based on Interactive Nonnegative Matrix Factorization
Choo, J., Lee, C., Reddy, C. K., & Park, H. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.212 · Cited by 262 · cited 2×
“Broadly, iML encompasses all kinds of ML tasks including supervised ML, unsupervised ML (e.g., clustering [21, 83]) and reinforcement learning [14].”
BackgroundModelTracker: Redesigning Performance Analysis Tools for Machine Learning
Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702509 · Cited by 262 · cited 1×
“Most iML systems present users with performance information as impacted by their input, either performance metrics [4, 42], or model output, for example by visualizing the output for a batch of instances [32] or allowing users to select instances to inspect.”
BackgroundPrototype selection for interpretable classification
Bien, J., & Tibshirani, R. (2011) · The Annals of Applied Statistics · doi:10.1214/11-aoas495 · Cited by 245 · cited 1×
“Other design choices for local explanations include presenting similar examples with the same known outcome [9, 37], and rules that the model believes to guarantee the prediction [73] (e.”
BackgroundActive Learning with Feedback on Features and Instances
Raghavan, H., Madani, O., & Jones, R. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248608 · Cited by 232
Proactive learning: cost-sensitive active learning with multiple imperfect oracles
Donmez, P., & Carbonell, J. G. (2008) · Proceedings of the 17th ACM conference on Information and knowledge management · doi:10.1145/1458082.1458165 · Cited by 228 · cited 1×
“For example, by solving a multi-instance, multi-oracle optimization problem, proactive learning [28] relaxes the assumptions that the annotator is infallible, indefatigable (always answers with the same level of quality), individual (only one oracle), and insensitive to costs.”
BackgroundEnsembleMatrix: interactive visualization to support machine learning with multiple classifiers
Talbot, J., Lee, B., Kapoor, A., & Tan, D. S. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518895 · Cited by 226 · cited 1×
“iML research has studied a variety of user input into the model such as providing labels, training examples [31], as well as specifying model and algorithm choice [91], parameters, error preferences [42], etc.”
BackgroundGamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models
Hohman, F., Head, A., Caruana, R., DeLine, R., & Drucker, S. M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225 · cited 3×
“This idea underlies recent research into visual analytical tools that support debugging or feature engineering work [40, 47, 96].”
BackgroundMethodExplainability in human–agent systems
Rosenfeld, A., & Richardson, A. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09408-y · Cited by 224 · cited 1×
“We focus on those explaining ML classifiers (as opposed to other types of AI system such as planning [17] or multi-agent systems [74]).”
BackgroundManifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models
Zhang, J., Wang, Y., Molino, P., Li, L., & Ebert, D. S. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864499 · Cited by 220 · cited 1×
“Explanations are therefore increasingly incorporated in ML development tools supporting debugging tasks such as performance analysis [71], interactive debugging [48], feature engineering [46], instance inspection and model comparison [40, 99].”
MethodThe effects of example-based explanations in a machine learning interface
Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220
Settles, B. (2011) · Cited by 218
Designing robot learners that ask good questions
Cakmak, M., & Thomaz, A. L. (2012) · Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction · doi:10.1145/2157689.2157693 · Cited by 214 · cited 3×
“One exception is in the field of human-robot interaction (HRI), where AL algorithms were used to develop robots that continuously learn by asking humans questions [14, 15, 18, 34, 76].”
BackgroundMethodOpen user profiles for adaptive news systems: help or harm?
Ahn, J.-W., Brusilovsky, P., Grady, J., He, D., & Syn, S. Y. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242575 · Cited by 200
Squares: Supporting Interactive Performance Analysis for Multiclass Classifiers
Ren, D., Amershi, S., Lee, B., Suh, J., & Williams, J. D. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598828 · Cited by 200 · cited 1×
“Explanations are therefore increasingly incorporated in ML development tools supporting debugging tasks such as performance analysis [71], interactive debugging [48], feature engineering [46], instance inspection and model comparison [40, 99].”
MethodThe Sage dictionary of statistics : a practical resource for students in the social sciences
Cramer, D., & Howitt, D. (2004) · Cited by 197
Interacting meaningfully with machine learning systems: Three experiments
Stumpf, S., Rajaram, V., Li, L., Wong, W.-K., Burnett, M., Dietterich, T., Sullivan, E., & Herlocker, J. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.03.004 · Cited by 184 · cited 4×
“It may be interesting to explore approaches that take human feedback as weighted signals, constraints, a part of a co-training model or ensemble [90] , or impacting the data [92] or the sampling strategy.”
BackgroundMethodInfluentialCueFlik: interactive concept learning in image search
Fogarty, J., Tan, D., Kapoor, A., & Winder, S. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357061 · Cited by 178 · cited 2×
“, output, performance, features used [31, 32, 48, 75]).”
BackgroundToward harnessing user feedback for machine learning
Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T. G., Sullivan, E., Drummond, R., & Herlocker, J. L. (2007) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1216295.1216316 · Cited by 177 · cited 3×
“It is worth pointing out that all of the above-mentioned iML and AL systems supporting featurelevel input are for text-based models [48, 70, 78, 84, 89].”
BackgroundMethodINFUSE: Interactive Feature Selection for Predictive Modeling of High Dimensional Data
Krause, J., Perer, A., & Bertini, E. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346482 · Cited by 169 · cited 1×
“Explanations are therefore increasingly incorporated in ML development tools supporting debugging tasks such as performance analysis [71], interactive debugging [48], feature engineering [46], instance inspection and model comparison [40, 99].”
MethodActive learning by labeling features
Druck, G., Settles, B., & McCallum, A. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 1 - EMNLP '09 · doi:10.3115/1699510.1699522 · Cited by 162 · cited 2×
“3, proposed methods to incorporate feedback on top features or boosting the importance of features [30, 70, 78, 89], and removing features [48, 92].”
BackgroundMethodDesigning Interactions for Robot Active Learners
Çakmak, M., Chao, C., & Thomaz, A. L. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051030 · Cited by 159 · cited 4×
“One exception is in the field of human-robot interaction (HRI), where AL algorithms were used to develop robots that continuously learn by asking humans questions [14, 15, 18, 34, 76].”
BackgroundMethodInfluentialNarayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153
A benchmark and comparison of active learning for logistic regression
Yang, Y., & Loog, M. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.06.004 · Cited by 149 · cited 2×
“For this study, we used entropy-based uncertainty sampling to select the next instance to query, as it is the most commonly used sampling strategy [98] and also computationally inexpensive.”
MethodToolkit to support intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864353 · Cited by 143 · cited 1×
“Based on Lim and Dey’s foundational work [56, 57], intelligibility types can be represented by prototypical user questions to understand the AI, including inputs, outputs, certainty, why, why not, how to, what if and when.”
BackgroundTheory of mind for learning and teaching: the nature and role of explanation
Wellman, H. M., & Lagattuta, K. H. (2004) · Cognitive Development · doi:10.1016/j.cogdev.2004.09.003 · Cited by 140 · cited 1×
“Explanations play a critical role in human teaching and learning [65, 95].”
BackgroundInteractive optimization for steering machine classification
Kapoor, A., Lee, B., Tan, D. S., & Horvitz, E. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753529 · Cited by 138 · cited 3×
“iML research has studied a variety of user input into the model such as providing labels, training examples [31], as well as specifying model and algorithm choice [91], parameters, error preferences [42], etc.”
BackgroundSage Dictionary of Statistics: A Practical Resource for Students in the Social Sciences
Broughton, V. (2005) · Reference Reviews · doi:10.1108/09504120510580208 · Cited by 123 · cited 1×
“10 as marginally significant, following statistical convention [24]”
BackgroundSpringer, A., & Whittaker, S. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302322 · Cited by 115 · cited 1×
“One potential solution is progressive disclosure by starting from simplified explanations and progressively provide more details [87].”
BackgroundSmith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112 · cited 2×
“It is worth pointing out that all of the above-mentioned iML and AL systems supporting feature-level input are for text-based models [48, 70, 78, 84, 89].”
BackgroundNorris, S. P. (1997) · Science Education · doi:10.1002/(sici)1098-237x(199704)81:2<239::aid-sce7>3.0.co;2-g · Cited by 107
FeatureInsight: Visual support for error-driven feature ideation in text classification
Brooks, M., Amershi, S., Lee, B., Drucker, S. M., Kapoor, A., & Simard, P. (2015) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2015.7347637 · Cited by 96 · cited 3×
“First, we present the model’s own reasoning for a particular instance to query user feedback instead of requesting global feature weights from people [10, 48, 70, 78].”
BackgroundMethodClosing the Loop: User-Centered Design and Evaluation of a Human-in-the-Loop Topic Modeling System
Smith, A., Kumar, V. R., Boyd-Graber, J. L., Seppi, K. D., & Findlater, L. (2018) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3172944.3172965 · Cited by 88 · cited 2×
“Broadly, iML encompasses all kinds of ML tasks including supervised ML, unsupervised ML (e.g., clustering [21, 83]) and reinforcement learning [14].”
BackgroundMeasuring Crowdsourcing Effort with Error-Time Curves
Cheng, J., Teevan, J. B., & Bernstein, M. S. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702145 · Cited by 83 · cited 1×
“But labeling errors are inevitable, even for simple perceptual judgment tasks [20].”
BackgroundWhy these Explanations? Selecting Intelligibility Types for Explanation Goals.
Lim, B. Y., Yang, Q., Abdul, A., & Wang, D. (2019) · IUI Workshops · Cited by 79
Theoretical considerations and development of a questionnaire to measure trust in automation
Körber, M. (2018) · doi:10.31234/osf.io/nfc45 · Cited by 71
On active learning for data acquisition
Zheng, Z., & Padmanabhan, B. (2003) · Proceedings · doi:10.1109/icdm.2002.1184002 · Cited by 71 · cited 1×
“Other AL paradigms include active class selection [60] and active feature acquisition [101], which query the annotator for additional training examples and missing features, respectively.”
BackgroundTransparent active learning for robots
Chao, C., Çakmak, M., & Thomaz, A. L. (2010) · SMARTech Repository (Georgia Institute of Technology) · doi:10.5555/1734454.1734562 · Cited by 71
Why-oriented end-user debugging of naive Bayes text classification
Kulesza, T., Stumpf, S., Wong, W., Burnett, M., Perona, S., Ko, A. J., & Oberst, I. (2011) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2030365.2030367 · Cited by 69 · cited 1×
“Recent work demonstrated that, while ML experts may be able to reason with model features globally, lay people prefer local explanations grounded in specific cases [6, 40, 49, 50].”
BackgroundLomasky, R., Brodley, C. E., Aernecke, M. J., Walt, D. R., & Friedl, M. A. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74958-5_63 · Cited by 64 · cited 1×
“Other AL paradigms include active class selection [60] and active feature acquisition [101], which query the annotator for additional training examples and missing features, respectively.”
BackgroundShivaswamy, P., & Joachims, T. (2015) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4539 · Cited by 57 · cited 1×
“Therefore, we compare the model training results with XAL to two baseline conditions: traditional AL and coactive learning (CL) [81].”
MethodConsensually driven explanation in science teaching
Meyer, K., & Woodruff, E. (1997) · Science Education · doi:10.1002/(sici)1098-237x(199704)81:2<173::aid-sce4>3.0.co;2-c · Cited by 48 · cited 1×
“Explanations play a critical role in human teaching and learning [65, 95].”
BackgroundExplainable Artificial Intelligence (XAI)
(2023) · International Journal of Food and Nutritional Sciences · doi:10.48047/ijfans/v12/i1/271 · Cited by 43
Wu, T., Weld, D. S., & Heer, J. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3319616 · Cited by 39 · cited 2×
“For example, some have warned against “local decision pitfalls” [97] of human feedback in iML that overly focuses on modifying a subset of model features, commonly resulting in an overfitted model that fails to generalize.”
BackgroundActive learning with confidence-based answers for crowdsourcing labeling tasks
Song, J., Wang, H., Gao, Y., & An, B. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.07.010 · Cited by 39 · cited 1×
“Future work could explore utilizing weak rejection to improve model learning, for example, with AL algorithms that can consider probabilistic annotations [86].”
Towards maximizing the accuracy of human-labeled sensor data
Rosenthal, S. L., & Dey, A. K. (2010) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1719970.1720006 · Cited by 35 · cited 7×
“For example, AL can be used to solicit users’ labels for their own records to train an email spam classifier or context-aware sensors [42, 75], but a regular user may lack the knowledge or contextual information to make all judgments correctly.”
BackgroundInfluentialImpact of Batch Size on Stopping Active Learning for Text Classification
Beatty, G., Kochis, E., & Bloodgood, M. (2018) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2018.00059 · Cited by 17 · cited 1×
“We used a batch size of 1 [8], meaning the model was retrained after each new queried label.”
Method"Why Should I Trust Interactive Learners?" Explaining Interactive Queries of Classifiers to UsersPreprint
Teso, S., & Kersting, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08578 · Cited by 13 · cited 6×
“These benefits were discussed in a very recent paper by Teso and Kersting [92], which explored soliciting corrections for the model’s explanation, specifically feedback that a mentioned feature should be considered irrelevant instead.”
BackgroundMethodResultInfluentialHanafi, M. F., Abouzied, A., Chiticariu, L., & Li, Y. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025540 · Cited by 11 · cited 1×
“The rule-based explanation, on the other hand, could be an interesting design for future work to explore, as annotators may be able to approve or disapprove the rules, or judge between multiple candidate rules [38].”
BackgroundGeneration of meaningful robot expressions with active learning
Saponaro, G., & Bernardino, A. J. M. (2011) · doi:10.1145/1957656.1957752 · Cited by 5 · cited 1×
“One exception is in the field of human-robot interaction (HRI), where AL algorithms were used to develop robots that continuously learn by asking humans questions [14, 15, 18, 34, 76].”
MethodAsking rank queries in pose learning
González-Pacheco, V., Malfáz, M., & Salichs, M. A. (2014) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2559636.2559845 · Cited by 2 · cited 1×
“One exception is in the field of human-robot interaction (HRI), where AL algorithms were used to develop robots that continuously learn by asking humans questions [14, 15, 18, 34, 76].”
Method
Research works printed but not linked: 14
- David A. Anisi . 2003. Optimal Motion Control of a Ground Vehicle. Master's thesis . Royal Institute of Technology (KTH) , Stockholm, Sweden . David A. Anisi. 2003. Optimal Motion Control of a Ground Vehicle. Master's thesis. Royal Institute of Technology (KTH), Stockholm, Sweden.
- Vijay Arya , Rachel KE Bellamy , Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilović, et almbox. 2019 . One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques . arXiv preprint arXiv:1909.03012 (2019). Vijay Arya, Rachel KE Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilović, et almbox. 2019. One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. arXiv preprint arXiv:1909.03012 (2019). · arXiv
- 10.1037/0022-3514.45.4.805 · doi:10.1037/0022-3514.45.4.805 · Paper page
- Tathagata Chakraborti , Sarath Sreedharan , and Subbarao Kambhampati . 2020. The Emerging Landscape of Explainable AI Planning and Decision Making. arXiv preprint arXiv:2002.11697 ( 2020 ). Tathagata Chakraborti, Sarath Sreedharan, and Subbarao Kambhampati. 2020. The Emerging Landscape of Explainable AI Planning and Decision Making. arXiv preprint arXiv:2002.11697 (2020). · arXiv
- Sophie Chou , William Li , and Ramesh Sridharan . 2014 . Democratizing data science . In Proceedings of the KDD 2014 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , New York, NY, USA. Citeseer, 24--27. Sophie Chou, William Li, and Ramesh Sridharan. 2014. Democratizing data science. In Proceedings of the KDD 2014 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA. Citeseer, 24--27.
- Karthik S Gurumoorthy , Amit Dhurandhar , and Guillermo Cecchi . 2017 . Protodash: Fast interpretable prototype selection. arXiv preprint arXiv:1707.01212 (2017). Karthik S Gurumoorthy, Amit Dhurandhar, and Guillermo Cecchi. 2017. Protodash: Fast interpretable prototype selection. arXiv preprint arXiv:1707.01212 (2017). · arXiv
- 10.1037/0022-3514.63.2.308 · doi:10.1037/0022-3514.63.2.308 · Paper page
- David D Lewis and William A Gale . 1994 b. A sequential algorithm for training text classifiers . In SIGIR?94 . Springer , 3--12. David D Lewis and William A Gale. 1994 b. A sequential algorithm for training text classifiers. In SIGIR?94. Springer, 3--12.
- Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). · arXiv
- 10.5555/1613715.1613855 · doi:10.5555/1613715.1613855
- 10.5555/2831071.2831072 · doi:10.5555/2831071.2831072
- Patrice Y Simard , Saleema Amershi , David M Chickering , Alicia Edelman Pelton , Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et almbox. 2017 . Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017). Patrice Y Simard, Saleema Amershi, David M Chickering, Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et almbox. 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017). · arXiv
- Simone Stumpf , Adrian Bussone , and Dympna O?sullivan. 2016 . Explanations considered harmful? user interactions with machine learning systems . In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) . Simone Stumpf, Adrian Bussone, and Dympna O?sullivan. 2016. Explanations considered harmful? user interactions with machine learning systems. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) .
- Richard Tomsett , Dave Braines , Dan Harborne , Alun Preece , and Supriyo Chakraborty . 2018. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552 ( 2018 ). Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, and Supriyo Chakraborty. 2018. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552 (2018). · arXiv
Cited by
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
“Because prior work suggested that people with high Need for Cognition (NFC, a stable personality trait that captures one’s motivation to engage in effortful mental activities) tend to benefit more from complex user interface features [12, 22, 24], we did so by disaggregating our results by NFC level.”
BackgroundExpanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538
Self-citationHuman-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.011 · Cited by 475
Self-citationSurvey on Explainable AI: From Approaches, Limitations and Applications Aspects
Yang, W., Wei, Y., Wei, H., Chen, Y., Huang, G., Li, X., Li, R., Yao, N., Wang, X., Gu, X., et al. (2023) · Human-Centric Intelligent Systems · doi:10.1007/s44230-023-00038-y · Cited by 240
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Metadata
- Authors
- Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K.
- Affiliations
- Stony Brook University (US)
- IBM (United States) (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-01-05
- Volume
- 4
- Issue
- CSCW3
- Pages
- 1-28
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3120532656
- Cited by
- 106
- References
- 95
- Citations by year
- 2026: 12 · 2025: 24 · 2024: 21 · 2023: 25 · 2022: 9 · 2021: 15
- FWCI
- 6.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- explainable AI, machine teaching, human-AI interface, AI explanations, active learning (machine learning), trust calibration, teacher feedback, anchoring effect, mental workload, need for cognition, AI experience, task knowledge, human-in-the-loop learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.