Data-Centric Explanations: Explaining Training Data of Machine Learning Systems to Promote Transparency
Open accessAbstract
Training datasets fundamentally impact the performance of machine learning (ML) systems. Any biases introduced during training (implicit or explicit) are often reflected in the system's behaviors leading to questions about fairness and loss of trust in the system. Yet, information on training data is rarely communicated to stakeholders. In this work, we explore the concept of data-centric explanations for ML systems that describe the training data to end-users. Through a formative study, we investigate the potential utility of such an approach, including the information about training data that participants find most compelling. In a second study, we investigate reactions to our explanations across four different system scenarios. Our results suggest that data-centric explanations have the potential to impact how users judge the trustworthiness of a system and to assist users in assessing fairness. We discuss the implications of our findings for designing explanations to support users’ perceptions of ML systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Practical approaches to implement such ideas typically revolve around the systematic collection and curation of data on various aspects of systems.
“Practical approaches to implement such ideas typically revolve around the systematic collection and curation of data on various aspects of systems [2, 28, 46].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [2]
Section 2.3 (p. 11) describes standardized documentation of training datasets, covering their creation, composition, uses, distribution, and maintenance. The Overview describes selecting that information for a data-centric explanation. This supports an example of systematically organizing information for transparency, but not the broader claim that practical approaches typically collect and curate data about various aspects of systems. (LLM, conf=0.82)
Experimental — accuracy not yet measuredWork on AI safety and reproducibility has long pointed out the crucial importance of training data for understanding model performance, ensuring reproducibility, and assessing legal exposure.
“Work on AI safety and reproducibility has long pointed out the crucial importance of training data for understanding model performance, ensuring reproducibility, and assessing legal exposure [2, 25, 29, 46, 58].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [2]
The Introduction says training-dataset quality and characteristics are fundamental to system performance, and §2.3 (p. 11) says dataset documentation helps assess quality and fitness for a machine-learning system. The passages do not address ensuring reproducibility or assessing legal exposure; searches for those topics found no supporting passage in our copy. (LLM, conf=0.87)
Experimental — accuracy not yet measuredPrevious studies have shown that data-centric explanations have the potential to assist users in assessing fairness.
“For instance, previous studies have shown that data-centric explanations [7] have the potential to assist users in assessing fairness.”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Generalized expectancies for internal versus external control of reinforcement.
Rotter, J. B. (1966) · Psychological Monographs: General and Applied · doi:10.1037/h0092976 · Cited by 21,937 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
Basics of Qualitative Research (3rd ed.): Techniques and Procedures for Developing Grounded Theory
Corbin, J., & Strauss, A. (2008) · SAGE · doi:10.4135/9781452230153 · Cited by 12,136 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
On the dimensionality of organizational justice: A construct validation of a measure.
Colquitt, J. A. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.386 · Cited by 5,989 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445 · see citing works
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796 · see citing works
(2018) · Cited by 3,510 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1× · see citing works
“The opacity of these systems can also hurt a user’s ability to form meaningful trust relationships with machine learning systems [28, 75, 87, 89] and to hold the systems properly accountable for their decisions [11, 65].”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
VoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256 · see citing works
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799 · see citing works
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · see citing works
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · see citing works
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338 · see citing works
Techniques for interpretable machine learning
Du, M., Liu, N., & Hu, X. (2019) · Communications of the ACM · doi:10.1145/3359786 · Cited by 1,268 · see citing works
Evaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164 · see citing works
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · see citing works
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Consumer Credit Risk Models Via Machine-Learning AlgorithmsPreprint
Khandani, A., Kim, A., & Lo, A. W. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1568864 · Cited by 895 · see citing works
Consumer credit-risk models via machine-learning algorithms
Khandani, A. E., Kim, A. J., & Lo, A. W. (2010) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2010.06.001 · Cited by 864 · see citing works
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1× · see citing works
“For example, if the training dataset is not representative of the target population, certain groups can be disadvantaged [3], and any biases in the training data [81] are ultimately reflected and aggravated in the deployed system [3,106].”
BackgroundImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · see citing works
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · see citing works
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · cited 1× · see citing works
“For example, if the training dataset is not representative of the target population, certain groups can be disadvantaged [3], and any biases in the training data [81] are ultimately reflected and aggravated in the deployed system [3,106].”
BackgroundSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763 · see citing works
Algorithmic Accountability: Journalistic investigation of computational power structures
Diakopoulos, N. (2015) · Digital Journalism · doi:10.1080/21670811.2014.976411 · Cited by 733 · see citing works
QuAC: Question Answering in Context
Choi, E., He, H., Iyyer, M., Yatskar, M., Yih, W.-T., Choi, Y., Liang, P., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1241 · Cited by 714 · cited 1× · see citing works
“Many machine learning researchers have begun adopting this procedure when releasing their datasets [21,93,100] and this approach is starting to gain traction in some organizations (e.”
BackgroundLepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0279-x · Cited by 709 · see citing works
Model-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699 · see citing works
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 7× · see citing works
“Evaluating the Impact of Explanations In parallel to developing different explanation approaches, numerous studies have investigated the impact of explanations on user perceptions of and interactions with the systems [5,14,20,33,59,60,86,102,105].”
BackgroundMethodInfluentialExplainable Machine Learning in Credit Risk Management
Bussmann, N., Giudici, P., Marinelli, D., & Papenbrock, J. (2020) · Computational Economics · doi:10.1007/s10614-020-10042-0 · Cited by 635 · see citing works
How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615 · see citing works
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · see citing works
González-Gallardo, C. E., Montes, A., Sierra, G., Núñez-Juárez, J. A., Salinas-López, A. J., & Ek, J. (2015) · conference · Cited by 559 · see citing works
<i>Why and why not</i> explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · see citing works
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 3× · see citing works
“Evaluating the Impact of Explanations In parallel to developing different explanation approaches, numerous studies have investigated the impact of explanations on user perceptions of and interactions with the systems [5,14,20,33,59,60,86,102,105].”
Background(2014) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 524 · see citing works
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515 · see citing works
How the Machine 'Thinks:' Understanding Opacity in Machine Learning AlgorithmsPreprint
Burrell, J. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2660674 · Cited by 511 · see citing works
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448 · see citing works
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · see citing works
Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models
Krause, J., Perer, A., & Ng, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858529 · Cited by 390 · see citing works
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · see citing works
Explanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388 · see citing works
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386 · see citing works
The Frontiers of Fairness in Machine LearningPreprint
Chouldechova, A., & Roth, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08810 · Cited by 372 · see citing works
Trust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · see citing works
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 6× · see citing works
“Evaluating the Impact of Explanations In parallel to developing different explanation approaches, numerous studies have investigated the impact of explanations on user perceptions of and interactions with the systems [5,14,20,33,59,60,86,102,105].”
BackgroundInfluentialThe Impact of Explanation Facilities on User Acceptance of Expert Systems Advice
Ye, L. R., & Johnson, P. E. (1995) · MIS Quarterly · doi:10.2307/249686 · Cited by 290 · see citing works
Explaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286 · see citing works
A Principled Way of Assessing Visualization Literacy
Boy, J., Rensink, R. A., Bertini, E., & Fekete, J. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346984 · Cited by 280 · see citing works
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277 · see citing works
Transcending Boundaries: Improvisation and Disability in Dance
Whatley, S. (2019) · Pure (Coventry University) · doi:10.1093/oxfordhb · Cited by 265 · see citing works
de Laat, P. B. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0293-z · Cited by 251 · see citing works
Bringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250 · see citing works
Assessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246 · see citing works
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225 · see citing works
Hohman, F., Head, A., Caruana, R., DeLine, R. A., & Drucker, S. M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225 · see citing works
Interpretable classification models for recidivism prediction
Zeng, J., Ustun, B., & Rudin, C. (2017) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12227 · Cited by 220 · see citing works
The effects of example-based explanations in a machine learning interface
Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220 · see citing works
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207 · see citing works
"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196 · see citing works
Productivity and Selection of Human Capital with Machine Learning
Chalfin, A. J., Danieli, O., Hillis, A., Jelveh, Z., Luca, M., Ludwig, J., & Mullainathan, S. (2016) · American Economic Review · doi:10.1257/aer.p20161029 · Cited by 190 · see citing works
Srivastava, M., Heidari, H., & Krause, A. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330664 · Cited by 179 · see citing works
Lacerda, A. M., de Cristo, M. A. P., Gonçalves, M. A., Fan, W., Ziviani, N., & Ribeiro-Neto, B. A. (2006) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1148170.1148265 · Cited by 158 · see citing works
Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., & Ziegler, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300717 · Cited by 156 · see citing works
Measuring Justice and Fairness
Cropanzano, R., Ambrose, M. L., Colquitt, J. A., & Rodell, J. B. (2015) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199981410.013.8 · Cited by 149 · see citing works
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302313 · Cited by 148 · see citing works
Why Unbiased Computational Processes Can Lead to Discriminative Decision Procedures
Calders, T., & Žliobaitė, I. (2013) · Studies in Applied Philosophy, Epistemology and Rational Ethics · doi:10.1007/978-3-642-30487-3_3 · Cited by 128 · see citing works
Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital PathologyPreprint
Holzinger, A., Malle, B., Kieseberg, P., Roth, P., Müller, H., Reihs, R., & Zatloukal, K. (2017) · arXiv.org · doi:10.48550/arxiv.1712.06657 · Cited by 113 · see citing works
Are explanations always important?: a study of deployed, low-cost intelligent interactive systems
Bunt, A., Lount, M., & Lauzon, C. (2012) · Proceedings of the 2012 ACM international conference on Intelligent User Interfaces · doi:10.1145/2166966.2166996 · Cited by 106 · see citing works
Self-citationLiem, C. C. S., Langer, M., Demetriou, A. M., Hiemstra, A. M. F., Wicaksana, A. S., Born, M. P., & König, C. J. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4_9 · Cited by 101 · see citing works
In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction
Wang, C., Han, B., Patel, B., & Rudin, C. (2023) · Journal of Quantitative Criminology · doi:10.1007/s10940-022-09545-w · Cited by 101 · see citing works
Applicability of Demographic Recommender System to Tourist Attractions: A Case Study on Trip Advisor
Wang, Y., Chan, S. C. F., & Ngai, G. (2012) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2012.133 · Cited by 91 · see citing works
A Case-Based Explanation System for Black-Box Systems
Nugent, C., & Cunningham, P. (2005) · Artificial Intelligence Review · doi:10.1007/s10462-005-4609-5 · Cited by 84 · see citing works
Interpretable Credit Application Predictions With Counterfactual ExplanationsPreprint
McGrath, R., Costabello, L., Van, C. L., Sweeney, P., Kamiab, F., Shen, Z., & Lécué, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05245 · Cited by 78 · see citing works
Interpreting Black-Box Classifiers Using Instance-Level Visual Explanations
Tamagnini, P., Krause, J., Dasgupta, A., & Bertini, E. (2017) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/3077257.3077260 · Cited by 70 · see citing works
Design of an explainable machine learning challenge for video interviews
Escalante, H. J., Guyon, I., Escalera, S., Jacques, J. C. S., Madadi, M., Baró, X., Ayache, S., Viegas, E., Güçlütürk, Y., Güçlü, U., et al. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966320 · Cited by 66 · see citing works
A Review of Explanation and Explanation in Case-Based Reasoning
Doyle, D., Tsymbal, A., & Cunningham, P. (2003) · Trinity's Access to Research Output (TARA) (Trinity College Dublin) · Cited by 61 · see citing works
Context-aware movie recommendation based on signal processing and machine learning
Biancalana, C., Gasparetti, F., Micarelli, A., Miola, A. M., & Sansonetti, G. (2011) · doi:10.1145/2096112.2096114 · Cited by 59 · see citing works
Algorithmic Transparency via Quantitative Input Influence
Datta, A., Sen, S., & Zick, Y. (2017) · Studies in big data · doi:10.1007/978-3-319-54024-5_4 · Cited by 43 · see citing works
Open the Black Box Data-Driven Explanation of Black Box Decision SystemsPreprint
Pedreschi, D., Giannotti, F., Guidotti, R., Monreale, A., Pappalardo, L., Ruggieri, S., & Turini, F. (2018) · arXiv.org · doi:10.48550/arxiv.1806.09936 · Cited by 39 · see citing works
AQUA: Questions that drive the explanation process
Ram, A. (1993) · Cited by 24 · see citing works
Halpern, N., Goldberg, Y., Kadouri, L., Duvdevani, M., Hamburger, T., Peretz, T., & Hubert, A. (2017) · OncoTargets and Therapy · doi:10.2147/ott.s126905 · Cited by 23 · see citing works
Machine learning-based product recommendation using Apache Spark
Chen, L., Li, R., Liu, Y., Zhang, R., & Woodbridge, D. M.-K. (2017) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/uic-atc.2017.8397470 · Cited by 23 · see citing works
Understanding the Utility of Rationale in a Mixed-Initiative System for GUI Customization
Bunt, A., McGrenere, J., & Conati, C. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73078-1_18 · Cited by 21 · see citing works
Self-citationFahner, G. (2018) · Cited by 18 · see citing works
Explaining Recommendations by Means of User Reviews.
Donkers, T., Loepp, B., & Ziegler, J. (2018) · IUI Workshops · Cited by 13 · see citing works
Baselines and a datasheet for the Cerema AWP datasetPreprint
Seck, I., Dahmane, K., Duthon, P., & Loosli, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04016 · Cited by 12 · see citing works
In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism PredictionPreprint
Wang, C., Han, B., Patel, B., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04176 · Cited by 10 · see citing works
Fair-by-design explainable models for prediction of recidivism
Soares, E., & Angelov, P. (2019) · arXiv · doi:10.48550/arxiv.1910.02043 · Cited by 9 · see citing works
Targeted Advertising Based on Browsing HistoryPreprint
Zhang, Y., Zhou, H., Tan, N., Bagheri, S., & Er, M. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04498 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [18]18. Zhengping Che, Sanjay Purushotham, Robinder Khemani, and Yan Liu. 2016. Interpretable Deep Models for ICU Outcome Prediction. AMIA ... Annual Symposium proceedings. AMIA Symposium 2016: 371-380.
- [45]45. Ben Green and Lily Hu. 2018. The Myth in the Methodology: Towards a Recontextualization of Fairness in Machine Learning. the ICML 2018 Debates Workshop.
- [72]72. C C S Liem, Markus Langer, Andrew Demetriou, Annemarie M F Hiemstra, A S Wicaksana, M Ph. Born, and Cornelius J. König. 2018. Explainable and interpretable models in computer vision and machine learning. https://doi.org/10.1007/978-3-319- 98131-4 · DOI
- [83]83. Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and Measuring Model Interpretability. Retrieved from http://arxiv.org/abs/1802.07810 · arXiv
Cited by
Sorted by citations
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278 · see citing works
“Usability workload [3, 16, 21, 48, 66] helpfulness [13, 45, 46, 48, 56, 65, 67, 68] satisfaction [1, 6, 7, 16, 18, 19, 29, 47, 69, 70] undesired behavior detection [2, 24, 27, 38, 53, 57, 71, 72, 73, 74, 75, 76, 77, 78, 79] ease of use and others [1, 3, 13, 20, 21, 24, 30, 32, 37, 48, 65, 66, 71, 80, 81, 82, 83, 84, 8…”
BackgroundMethodInfluentialLai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204 · see citing works
“Subjective metrics of understanding typically ask participants to directly rate their understanding of the AI [5, 12, 15, 18, 23, 75, 91, 105, 108], or some variations of it such as confidence in understanding [57], ease of understanding [46, 83], or confidence in simulation [2, 80].”
BackgroundInfluentialA literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180 · see citing works
“[2], [3], [4], [5], [6], [7], [8].”
Generative AI for visualization: State of the art and future directions
Ye, Y., Hao, J., Hou, Y., Wang, Z., Xiao, S., Luo, Y., & Zeng, W. (2024) · Visual Informatics · doi:10.1016/j.visinf.2024.04.003 · Cited by 147 · see citing works
BreastScreening-AI: Evaluating medical intelligent agents for human-AI interactions
Calisto, F. M., Santiago, C., Nunes, N., & Nascimento, J. C. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102285 · Cited by 125 · see citing works
Systematic review of data-centric approaches in artificial intelligence and machine learning
Singh, P. (2023) · Data Science and Management · doi:10.1016/j.dsm.2023.06.001 · Cited by 124 · see citing works
Improved Alzheimer’s Disease Detection by MRI Using Multimodal Machine Learning Algorithms
Battineni, G., Hossain, M. A., Chintalapudi, N., Traini, E., Dhulipalla, V. R., Mariappan, R., & Amenta, F. (2021) · Diagnostics · doi:10.3390/diagnostics11112103 · Cited by 93 · see citing works
“It is all about the data quality which helps to accurate data labelling [38].”
BackgroundA Survey of Data Quality Requirements That Matter in ML Development Pipelines
Priestley, M., O’donnell, F., & Simperl, E. (2023) · Journal of Data and Information Quality · doi:10.1145/3592616 · Cited by 86 · see citing works
“T r a n s p a r e n c y o f t h e m o d e l c a n b e s u pp o r t e d b y s e n s i t i v i t y t e s t i n g a n d e x p l a n a t i o n s [ 1 , 2 , 35 ] .”
BackgroundResultYurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581161 · Cited by 73 · see citing works
“For instance, previous studies have shown that data-centric explanations [7] have the potential to assist users in assessing fairness.”
ResultRethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 67 · see citing works
“Practical approaches to implement such ideas typically revolve around the systematic collection and curation of data on various aspects of systems [2, 28, 46].”
BackgroundBertrand, A., Viard, T., Belloum, R., Eagan, J. R., & Maxwell, W. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581314 · Cited by 59 · see citing works
“Anik and Bunt [7] is an example of this interactivity type.”
BackgroundInfluentialZheng, C., Wang, D., Wang, A. Y., & Ma, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517615 · Cited by 54 · see citing works
“The formative study has two parts: we first thoroughly surveyed relevant literature [5, 29, 31, 35, 37, 38, 42, 55] and industrial standard documentations [1, 2, 18, 36] to understand how a presentation deck looks like, and what contents are usually included.”
BackgroundSelective Explanations: Leveraging Human Input to Align Explainable AI
Lai, V. W.-M., Zhang, Y., Chen, C., Liao, Q. V., & Tan, C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610206 · Cited by 64 · see citing works
“…the effects of explanations on people’s task performance [6, 57, 108], efficiency [1, 17, 21, 34, 36, 39, 54, 55, 59, 64, 88, 99, 103], cognitive load [1, 37], understanding [4, 8, 11, 14, 21, 68, 88, 98, 103], subjective perceptions of AI [9, 15, 25, 37, 54, 55, 68, 75, 92] among others.”
BackgroundMitigating Bias in Algorithmic Systems—A Fish-eye View
Orphanou, K., Otterbacher, J., Kleanthous, S., Batsuren, K., Giunchiglia, F., Bogina, V., Tal, A. S., Hartman, A., & Kuflik, T. (2022) · ACM Computing Surveys · doi:10.1145/3527152 · Cited by 54 · see citing works
“Explainability Management Domain Problem Solution Space Reference(s) ML Model Model Expainability Use of decision tree [38, 54, 75, 114, 178, 194, 231] Model Use of decision rules [45, 99, 129] Model Ontologies [19, 42, 167] Output Outcome Explainability Local explanations [168, 169, 201] Output/User Visualization met…”
MethodA Critical Survey on Fairness Benefits of Explainable AI
Deck, L., Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658990 · Cited by 86 · see citing works
Aoki, N., Tatsumi, T., Naruse, G., & Maeda, K. (2024) · Government Information Quarterly · doi:10.1016/j.giq.2024.101965 · Cited by 51 · see citing works
A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities
Tocchetti, A., Corti, L., Balayn, A., Yurrita, M., Lippmann, P., Brambilla, M., & Yang, J. (2024) · ACM Computing Surveys · doi:10.1145/3665926 · Cited by 60 · see citing works
“Other works build tools, e.g., documentation or checklists, [7, 29, 74, 144] and workflows [205] to support making and documenting relevant choices when building or evaluating a model.”
Naveed, S., Stevens, G., & Robin-Kern, D. (2024) · Applied Sciences · doi:10.3390/app142311288 · Cited by 54 · see citing works
Bhattacharya, A., Ooge, J., Štiglic, G., & Verbert, K. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584075 · Cited by 44 · see citing works
“ferent explanation methods, including data-centric approaches [3], feature importance [2], and example-based methods [2].”
BackgroundMethodEffect of Explanation Conceptualisations on Trust in AI-assisted Credibility Assessment
Pareek, S., van Berkel, N., Velloso, E., & Goncalves, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686922 · Cited by 42 · see citing works
Towards a multi-stakeholder value-based assessment framework for algorithmic systems
Yurrita, M., Murray-Rust, D., Balayn, A., & Bozzon, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533118 · Cited by 42 · see citing works
“3314243 [11] Ariful IslamAnik and Andrea Bunt.”
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42 · see citing works
“4% set up their studies in both conditions, either by changing the decision task [5] or artificially modifying the scenario.”
BackgroundSilva, D. E., Chen, C., & Zhu, Y. (2022) · New Media & Society · doi:10.1177/14614448221098042 · Cited by 46 · see citing works
“…to find methods that increase the transparency of algorithmic systems in the hopes of increasing understanding (Liao et al., 2020; Polack, 2020), improving product design (Eiband et al., 2018; Kizilcec, 2016), and improving attitudes toward those systems (Anik and Bunt, 2021; Eslami et al., 2019).”
Methodvan Berkel, N., Sarsenbayeva, Z., & Gonçalves, J. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102954 · Cited by 41 · see citing works
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41 · see citing works
“For example, in high-stake scenarios such as in the medical [39, 61, 67, 97], legal [6, 81, 86, 131], and financial [27, 36, 43, 45, 46] domains, individuals tend to exhibit a preference for human decision-makers over AI systems.”
BackgroundBalakrishnan, M., Ferreira, K., & Tong, J. D. (2025) · Management Science · doi:10.1287/mnsc.2022.03850 · Cited by 40 · see citing works
Wang, X., & Yin, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581366 · Cited by 38 · see citing works
“To this end, one promising direction is to supplement explanations of AI models with explanations of the underlying data [4], which in e�ect may help people establish some “knowledge” or data-driven insights about the domain.”
BackgroundChen, C., & Sundar, S. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580805 · Cited by 38 · see citing works
“Aside from communicating training data demographics [2] and labelers’ racial backgrounds [9], designers can convey the extent to which the labels match the raw training data.”
BackgroundInfluentialEXMOS: Explanatory Model Steering through Multifaceted Explanations and Data Configurations
Bhattacharya, A., Stumpf, S., Gosak, L., Štiglic, G., & Verbert, K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642106 · Cited by 38 · see citing works
“Explanation methods have been categorised as model-specific or model-agnostic based on the degree of specificity [2, 5].”
InfluentialVisser, R. W., Peters, T. M., Scharlau, I., & Hammer, B. (2025) · Cognitive Systems Research · doi:10.1016/j.cogsys.2025.101357 · Cited by 47 · see citing works
“Anik and Bunt (2021), Liao and Varshney (2021), Mohseni et al. (2021) give an overview of different explanation approaches, or types, that are already used to answer such questions.”
InfluentialLewicki, K., Lee, M. S. A., Cobbe, J., & Singh, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581463 · Cited by 30 · see citing works
“Indeed, such activities also presents an opportunity for the HCI community, leveraging and extending work on how to provide means and tooling for meaningful transparency to support accountability [11, 27, 46, 62, 107].”
BackgroundThe Past, Present, and Prospective Future of XAI: A Comprehensive Review
Islam, M. U., Mottalib, M. M., Hassan, M., Alam, Z. I., Zobaed, S., & Rabby, M. F. (2022) · Studies in computational intelligence · doi:10.1007/978-3-030-96630-0_1 · Cited by 30 · see citing works
A domain-specific language for describing machine learning datasets
Giner-Miguelez, J., Gómez, A., & Cabot, J. (2023) · Journal of Computer Languages · doi:10.1016/j.cola.2023.101209 · Cited by 29 · see citing works
An Agile Software Development Life Cycle Model for Machine Learning Application Development
Ranawana, R., & Karunananda, A. S. (2021) · SLAAI International Conference on Artificial Intelligence (SLAAI-ICAI) · doi:10.1109/slaai-icai54477.2021.9664736 · Cited by 27 · see citing works
Ground Truth Or Dare: Factors Affecting The Creation Of Medical Datasets For Training AI
Zając, H. D., Avlona, N.-R., Kensing, F., Andersen, T. O., & Shklovski, I. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604766 · Cited by 25 · see citing works
“This has implications for scholarly discussions that focus on developing documentation frameworks that support the responsible and informed use of complex datasets [2, 5, 20, 31, 44].”
BackgroundCorbett, E., & Denton, E. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594104 · Cited by 25 · see citing works
“find quite a few studies that empirically examine how end-users understand and use existing transparency mechanisms [5, 27, 41, 65]”
How can Explainability Methods be Used to Support Bug Identification in Computer Vision Models?
Balayn, A., Rikalo, N., Lofi, C., Yang, J., & Bozzon, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517474 · Cited by 24 · see citing works
“We draw inspiration from works situated at the intersection of machine learning and HCI that investigate how machine learning or related tools (e.g. explainability, debugging user-interfaces, etc.) are used [35], or could be used [20], and how to design them [4, 52].”
BackgroundCorti, L., Oltmans, R., Jung, J., Balayn, A., Wijsenbeek, M. S., & Yang, J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642551 · Cited by 24 · see citing works
The landscape of data and AI documentation approaches in the European policy context
Micheli, M., Hupont, I., Delipetrev, B., & Soler-Garrido, J. (2023) · Ethics and Information Technology · doi:10.1007/s10676-023-09725-7 · Cited by 23 · see citing works
“Initiatives for distinct data types include Natural Language Processing (NLP) (Bender & Friedman, 2018; Lhoest et al., 2021; McMillan-Major et al., 2021; Shimorina & Belz, 2021), population data (Anik & Bunt, 2021), or images datasets (Prabhu and Birhane, 2020).”
BackgroundSchelenz, L., Segal, A., Adelio, O., & Gal, K. (2023) · ACM Journal on Responsible Computing · doi:10.1145/3636508 · Cited by 24 · see citing works
“The results of the study on user perceptions in popular real-world AI-based systems that use personalization show the discontents of users in several areas of transparency along the IPO + user control model.”
BackgroundOkolo, C. T., Agarwal, D., Dell, N. L., & Vashistha, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637348 · Cited by 21 · see citing works
“…on interviewing users and practitioners (UX designers, data scientists, researchers, etc.) to understand gaps in existing XAI tools [33, 36, 56, 67], evaluating XAI methods with various stakeholders [5, 17, 22, 35, 38, 59, 110], and introducing participatory co-design of XAI systems [76, 127, 132].”
Black, E., Naidu, R. C. A., Ghani, R., Rodolfa, K. T., Ho, D. E., & Heidari, H. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623259 · Cited by 19 · see citing works
Chen, C., Liao, M., & Sundar, S. S. (2024) · International Symposium on Trustworthy Autonomous Systems (TAS) · doi:10.1145/3686038.3686066 · Cited by 19 · see citing works
“…to lay users, researchers have adopted the documentation approach by detailing training data-related information before AI interaction, and it has demonstrated effectiveness in helping users identify bias in machine learning algorithms [3] and shaping appropriate expectations and trust in AI [10].”
Chen, C., & Sundar, S. S. (2024) · Human-Computer Interaction · doi:10.1080/07370024.2024.2392494 · Cited by 17 · see citing works
Salimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659567 · Cited by 11 · see citing works
“…increasing capabilities of AI systems to perform tasks with high accuracy have led to increasing interest in incorporating these systems into human decision-making processes across various fields, such as finance [24, 32, 42, 44], healthcare [37, 62, 65, 84], and the legal domain [6, 71, 76, 106].”
BackgroundBrachman, M., Pan, Q., Do, H. J., Dugan, C., Chaudhary, A., Johnson, J. M., Rai, P., Chakraborti, T., Gschwind, T., Laredo, J. A., et al. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584088 · Cited by 11 · see citing works
“However, much of the work in the design and evaluation of human-centric XAI focuses on decision-based systems, such as medical decision systems [56, 96, 120, 135], bail decision systems [4, 33, 77], and recommendation systems [62, 86, 93].”
BackgroundTowards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions
Bhattacharya, A. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3638177 · Cited by 10 · see citing works
“Moreover, recent works have emphasised the need for Data-Centric AI (DCAI) [32] approaches for model improvement by improving the quality of the training data for building more reliable and trustworthy prediction models [5].”
Towards Unlocking the Hidden Potentials of the Data-Centric AI Paradigm in the Modern Era
Majeed, A., & Hwang, S. O. (2024) · Applied System Innovation · doi:10.3390/asi7040054 · Cited by 10 · see citing works
Balayn, A., Rikalo, N., Yang, J., & Bozzon, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581555 · Cited by 10 · see citing works
“…a tool suggesting a workfow and artifacts, etc. Previous works around software debugging and machine teaching provide hints for its design, highlighting the importance of structured steps [84, 89]; structured documentation [6, 18, 33, 43, 81]; or warnings against graphical user-interfaces [128].”
BackgroundMethodXAI in Automated Fact-Checking? The Benefits Are Modest and There's No One-Explanation-Fits-All
Lim, G., & Perrault, S. T. (2023) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/3638380.3638388 · Cited by 9 · see citing works
“One aspect examines how people understand, interact with, accept, and trust XAI [8, 31, 75, 81], where we place our work.”
BackgroundAn Explanatory Model Steering System for Collaboration between Domain Experts and AI
Bhattacharya, A., Stumpf, S., & Verbert, K. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3631700.3664886 · Cited by 8 · see citing works
The Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 8 · see citing works
Representation Debiasing of Generated Data Involving Domain Experts
Bhattacharya, A., Stumpf, S., & Verbert, K. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3631700.3664910 · Cited by 8 · see citing works
Yurrita, M., Verma, H., Balayn, A., Gadiraju, U., Pont, S. C., & Bozzon, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713145 · Cited by 8 · see citing works
“Even if the effect found by Anik and Bunt [5] referred to training data rather than input data, we hypothesize that the type of input data will affect decision-subjects’ perceptions.”
InfluentialA Human-Centered Approach to Interpretable Machine Learning in Clinical Decision Support Systems
Nimbalkar, V., Chawla, L., Adnan, M. M., Bhansali, A., Gupta, M., & Kalra, R. (2025) · International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI) · doi:10.1109/icaiihi67124.2025.11403473 · Cited by 7 · see citing works
“The major weakness in most works is the compromise of model accuracy and interpretability which typically leaves the developers to choose between performance and transparency [12,19].”
"This means nothing to me": Building credibility in conversational systems
Candello, H. C. D. S. P., Soella, G. M., Sanctos, C. S., Grave, M., & de Brito-Filho, A. A. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3603759 · Cited by 7 · see citing works
“Between values that could a priori guide the realworld observation was fairness [2, 10, 12, 15, 16], explainability [10, 11], accountability [1, 9, 11, 16], and trust [2, 8, 9, 20, 25].”
BackgroundTowards a Non-Ideal Methodological Framework for Responsible ML
Mothilal, R. K., Guha, S., & Ahmed, S. I. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642501 · Cited by 9 · see citing works
“…at various phases of ML lifecyle for data collection and processing [32, 96, 140], disaggregated evaluation of models, explaining model outcomes [4, 33, 79], ethical risk assessment [115, 123], and accountability [109], etc. Researchers also develop frameworks to evaluate ethical or fairness…”
BackgroundPOP: Mining POtential Performance of New Fashion Products via Webly Cross-modal Query Expansion
Joppi, C., Skenderi, G., & Cristani, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_3 · Cited by 7 · see citing works
Bhattacharya, A., Stumpf, S., Croon, R. D., & Verbert, K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713497 · Cited by 6 · see citing works
Unlocking the Tacit Knowledge of Data Work in Machine Learning
Cha, I., Oh, J., Park, C. Y., Han, J., & Lee, H. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585616 · Cited by 6 · see citing works
“On the other hand, much work in the field of HCI is bridging the gap between the technical aspect of building ML datasets and understanding the roles and influences of humans making ML data [3, 5, 28, 30, 32].”
BackgroundData journeys: Explaining AI workflows through abstraction
Daga, E., & Groth, P. (2023) · Semantic Web · doi:10.3233/sw-233407 · Cited by 7 · see citing works
“While much of the XAI literature has focused on the need to explain the outcomes of machine learning and specifically deep learning models [9], a new line of work has begun to emphasise the need to explain how systems are built and trained [6,24,60].”
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 9 · see citing works
“Recent studies have proposed integrating actionable data-centric explanations (Anik and Bunt 2021; Liao and Varshney 2021; Yurrita et al. 2023; Esfahani et al. 2024b; Bhattacharya et al. 2025) alongside model-centric ones, offering potential benefits for both AI experts and lay users by connecting…”
A new hybrid data and model-centric predictive approach dedicated to industrial pipe maintenance
Braydi, A., Fossat, P., Casaburo, A., Pernet, V., Zwick, C., Ardabilian, M., & Bareille, O. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110821 · Cited by 5 · see citing works
“However, there has been a recent shift in focus towards the data-centric approach (Zajam et al., 2019; Anik and Bunt, 2021).”
Nieberl, M., Zeiser, A., & Timinger, H. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00018 · Cited by 5 · see citing works
“Conversely, revealing problems with the training data can negatively impact trust [28].”
BackgroundIdentifying Algorithmic Decision Subjects' Needs for Meaningful Contestability
Yurrita, M., Verma, H., Balayn, A., Alfrink, K., Gadiraju, U., & Bozzon, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757415 · Cited by 7 · see citing works
“Limiting information to output explanations might, therefore, hinder decision subjects’ ability to question structural aspects (e.g., data sources [7]) of the decision-making process [41].”
Mapping the Smart City: Participatory approaches to XAI
Gupta, S. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563703.3593063 · Cited by 4 · see citing works
“Several methods and tools, such as model [44] and data documentation [4, 7, 22, 29] frameworks, have been designed to explain AI to experts [17].”
MethodPrecel, H., McDonald, A., Hecht, B. J., & Vincent, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642749 · Cited by 4 · see citing works
“For instance, Anik et al. [6] investigate how explanations of training data in ML systems can increase trans-parency and influence trustworthiness of systems.”
BackgroundA Learning Approach for Increasing AI Literacy via XAI in Informal Settings
Sneirson, M., Chai, J., & Howley, I. K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-64299-9_29 · Cited by 4 · see citing works
Candello, H. C. D. S. P., Soella, G. M., & Nascimento, L. D. C. (2024) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3672406.3672409 · Cited by 4 · see citing works
“In order to unveil values in real-world situations, for instance: fairness [3, 19, 22, 31, 38], explainability [1, 19, 20], accountability [2, 17, 20, 38], trust [3, 12, 17, 25, 45, 50], and privacy [30, 42, 43, 49].”
Toward Patient-Centered AI Fact Labels: Leveraging Extrinsic Trust Cues
Yoo, D. W., Stroud, A. M., Zhu, X., Miller, J. E., & Barry, B. A. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735758 · Cited by 4 · see citing works
“Concurrently, the human-computer interaction (HCI) communities have extensively explored strategies to ensure AI model fairness, accountability, transparency, and trust [2, 6, 15, 17, 46].”
Elements that Influence Transparency in Artificial Intelligent Systems - A Survey
Muralidhar, D., Belloum, R., de Oliveira, K. M., & Ashok, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42280-5_21 · Cited by 4 · see citing works
Dissecting users' needs for search result explanations
Juneja, P., Zhang, W., Smith, A., Lamba, H., Tetreault, J., & Jaimes, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642059 · Cited by 7 · see citing works
Chen, Y., Verbert, K., Gerling, K., Abeele, M.-E. V., & Abeele, V. V. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712139 · Cited by 5 · see citing works
“In accordance with the data-centric approach [8], it furnishes details about the dataset and models used for training, along with model accuracy and basic demographic information such as age.”
InfluentialErlei, A., Cau, F. M., Georgiev, R., Kumar, S. C., Bizer, K., & Gadiraju, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791420 · Cited by 4 · see citing works
“Furthermore, additional information may also fail to improve human reliance if they cannot interpret the signals correctly or experience information overload [112].”
Social Transparency in Network Monitoring and Security Systems
Soroko, D., Savino, G.-L., Gray, N., & Schöning, J. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627773 · Cited by 3 · see citing works
Making Smart Cities Explainable: What XAI Can Learn from the “Ghost Map”
Gupta, S., & Loukissas, Y. A. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585847 · Cited by 3 · see citing works
“Alongside the creation of model documentation frameworks, XAI researchers have also designed systems to specifically document data creation processes such as “Data-centric Explanations” [2], “Datasheets” [16], “Dataset Nutrition Labels” [23], and “Data Statements” [4].”
MethodQu, J., Arguello, J., & Wang, Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658547 · Cited by 3 · see citing works
Anik, A. I., & Bunt, A. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00024 · Cited by 3 · see citing works
“Information about the training data is a central component of training dataset explanations [19], where the primary goal is to increase system transparency by informing the user about the formation of training datasets and the training process.”
BackgroundMethodInfluentialSelf-citationGenerative AI for Visualization
Jhanjhi, N. Z., Shah, I. A., & Nawaz, S. (2024) · Advances in web technologies and engineering book series · doi:10.4018/979-8-3693-3703-5.ch003 · Cited by 3 · see citing works
Thinking Like a Scientist: Can Interactive Simulations Foster Critical AI Literacy?
Zhao, Y., Michal, A. L., Thain, N., & Subramonyam, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98417-4_5 · Cited by 3 · see citing works
“Others have focused on understanding training data to explain how biases in training data can negatively affect the fairness of AI decisions (i.e., Explainable AI) [2].”
Chen, C., Liao, M., Phadnis, A. A., Li, Y., High, A. C., Abdullah, S., & Sundar, S. S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791542 · Cited by 2 · see citing works
“In-formed by the documentation paradigm [41, 58], several studies have sought to disclose training data characteristics to help lay users recognize biases in machine learning models [3, 20, 21].”
HAEM: Obtaining Higher-Quality Classification Task Results with AI Workers
Yamashita, Y., Ito, H., Wakabayashi, K., Kobayashi, M., & Morishima, A. (2022) · ACM Web Science Conference (WebSci) · doi:10.1145/3501247.3531580 · Cited by 3 · see citing works
“For example, law [15, 19, 36], medicine [10, 32], finance [8, 20], education [3, 14], and leisure [29, 30].”
BackgroundDiversity Challenges in Recruiting for Human-Centered Explainable AI Studies
Anik, A. I., & Bunt, A. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00067 · Cited by 2 · see citing works
“Our research specifically explores how to design AI explanations that communicate information about the training datasets used to train the systems [13], [14].”
BackgroundSelf-citationAI Systems Quality: A Data-Centric Perspective
Belghiti, A. E., Sbaï, H., & Asri, H. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-95326-2_8 · Cited by 1 · see citing works
Filtering Instances and Rejecting Predictions to Obtain Reliable Models in Healthcare
Valeriano, M. G., Marzagão, D. K., Montelongo, A., Kiffer, C. R. V., Katz, N., & Lorena, A. C. (2026) · Machine Learning · doi:10.1007/s10994-025-06941-8 · Cited by 1 · see citing works
“Biases, inconsistencies, and a lack of standardization can severely compromise model performance and reliability [4].”
Anik, A. I., & Bunt, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789087 · Cited by 1 · see citing works
“Thisgapis especially relevant for Training dataset explanations [5, 6], which are typically longer than other types of AI explanations studied to date.”
InfluentialSelf-citationHow Cultural Cognition Affects Trust and Perceived Quality of AI Explanations
Yuan, C.-W., & Lee, Y.-H. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101021 · Cited by 2 · see citing works
Krämer, N. C., Wischnewski, M., & Müller, E. (2023) · doi:10.31234/osf.io/h32ze · Cited by 1 · see citing works
Investigating Explanations that Target Training Data.
Anik, A. I., & Bunt, A. (2021) · IUI Workshops
Self-citationNailing Prediction: Experimental Evidence on the Value of Tools in Predictive Model DevelopmentPreprint
Yue, D., Hamilton, P., & Bojinov, I. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4300501
Data and databases in cardiovascular medicine and surgery
Shamavu, K. Y., & Tcheng, J. E. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-90534-3.00030-5
Desirability vs. feasibility: a research through design inquiry of explainable AI
Corti, L., & Yang, J. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-340553-2.00010-1
Responsible Artificial Intelligence for Healthcare: Trends, Challenges, and Opportunities
Baker, S., & Ahn, E. (2025) · Springer eBooks · doi:10.1007/978-3-031-88024-7_1
Huang, P., & Yang, Y. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1443
“While data plays a crucial role in the development, evaluation, and deployment of machine learning (ML) models, it is also the key component in ensuring the fairness, transparency in the models (Anik & Bunt, 2021; Holstein et al., 2019).”
Bertrand, A. (2024) · doi:10.70675/216d3647z78bcz4c68z9ba9z2d0a029cf1b4 · Cited by 1 · see citing works
Sensemaking in Multi-Agent LLM Interfaces: How Users Interpret Transparency and Trustworthiness Cues
Pareek, S., Govers, J., Kollerup, N. K., Wong, E., Velloso, E., & Gonçalves, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791157
“…or reliable the AI is, through accuracy scores [32, 102, 104], confidence estimates [107], or uncertainty markers [7, 41, 81], while process transparency focuses on making the AI’s decision-making more interpretable, often via explanations [3, 67, 69] and insights into model internals [66, 70].”
Scheuerman, M. K., Hutiri, W. T., Rahmattalabi, A., Matthews, V., Xiang, A., & Andrews, J. T. A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790364
What Happened to Scenario-Based Design in HCI?: A Scoping Review
Carroll, J. M., Jo, J., Kim, J., Lin, Y.-F., & Chen, E. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790899
Ooge, J., Faik, A., & Verbert, K. (2026) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3785022.3785087 · Cited by 1 · see citing works
- Authors
- Anik, A. I., & Bunt, A.
- Affiliations
- University of Manitoba (CA)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-13
- OpenAlex
- W3160343688
- Cited by
- 103
- References
- 99
- Citations by year
- 2026: 12 · 2025: 17 · 2024: 29 · 2023: 30 · 2022: 12 · 2021: 3
- FWCI
- 8.55
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Mspace (University of Manitoba) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- training data transparency, explainable machine learning, algorithmic fairness, training data bias, stakeholder communication, formative research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
