Cited by 119Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AISoftware Testing and Debugging Techniques

The DOI matches its Crossref record; 85 of the 89 references it cites were found in a registry or an index, and 4 are not checked yet.

Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs

Open access

Suresh, Harini, Gomez, Steven R., Nam, Kevin K., Satyanarayan, Arvind·2021·Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

DOI 10.1145/3411764.3445088

Open this paper in CoChat — free account Open PDF

Abstract

To ensure accountability and mitigate harm, it is critical that diverse stakeholders can interrogate black-box automated systems and find information that is understandable, relevant, and useful to them. In this paper, we eschew prior expertise- and role-based categorizations of interpretability stakeholders in favor of a more granular framework that decouples stakeholders’ knowledge from their interpretability needs. We characterize stakeholders by their formal, instrumental, and personal knowledge and how it manifests in the contexts of machine learning, the data domain, and the general milieu. We additionally distill a hierarchical typology of stakeholder needs that distinguishes higher-level domain goals from lower-level interpretability tasks. In assessing the descriptive, evaluative, and generative powers of our framework, we find our more nuanced treatment of stakeholders reveals gaps and opportunities in the interpretability literature, adds precision to the design and comparison of user studies, and facilitates a more reflexive approach to conducting this research.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1145/3411764.3445088

Cites

85 of 89 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“In this case, instead of starting with a tabula rasa, perhaps an interpretability interface would be initialized with summaries of the model’s strengths and weaknesses (akin to a model card [84]).”
Method
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507
Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 6×
““anticipate ethics-related failures before launch" [97], bias or mistake detection [89, 101, 103, 125], understand skewness and biases in input data [35], find unknown vulnerabilities and flaws [5, 38] (8/58) T3: Understand the extent of the information the model is using data entanglement [97], be…”
BackgroundInfluential
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743
Why and Where: A Characterization of Data Provenance
Buneman, P., Khanna, S., & Wang-Chiew, T. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44503-x_20 · Cited by 1,334

Show all 81 references

Research works printed but not linked: 52

2020. ML Interpretability for Scientific Discovery (MLI4SD) Workshop. https://sites.google.com/view/mli4sd-icml2020/home. Accessed: 2020-09-16.
Agnar Aamodt and Enric Plaza. 1994. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI communications 7, 1 (1994), 39–59.
10.1109/ACCESS.2018.2870052Conference · DOI
Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilović, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, and Yunfeng Zhang. 2019. One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. arXiv:1909.03012 [cs, stat] (Sept. 2019). http://arxiv.org/abs/1909.03012 arXiv:1909.03012. · arXiv
Robert K Atkinson, Sharon J Derry, Alexander Renkl, and Donald Wortham. 2000. Learning from examples: Instructional principles from the worked examples research. Review of educational research 70, 2 (2000), 181–214.
10.1145/3377325.3377498arXiv:2001.08298 · DOI
Chaofan Chen Oscar Li Daniel Tao Alina Barnett Cynthia Rudin and Jonathan K Su. 2019. This looks like that: deep learning for interpretable image recognition. In Advances in neural information processing systems. 8930–8941.
Michael Chromik, Malin Eiband, Sarah Theres Völkel, and Daniel Buschek. 2019. Dark Patterns of Explainability, Transparency, and User Control for Intelligent Systems. Los Angeles (2019), 6.
Gloria Dall’Alba and Jörgen Sandberg. 2006. Unveiling professional development: A critical review of stage models. Review of educational research 76, 3 (2006), 383–412.
Arun Das and Paul Rad. 2020. Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey. arXiv:2006.11371 [cs] (June 2020). http://arxiv.org/abs/2006.11371 arXiv:2006.11371. · arXiv
Amit Dhurandhar Vijay Iyengar Ronny Luss and Karthikeyan Shanmugam. 2017. A formal framework to characterize interpretability of procedures. (2017). arXiv:1707.03886. · arXiv
Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arXiv:1702.08608 [cs, stat] (March 2017). http://arxiv.org/abs/1702.08608 arXiv:1702.08608. · arXiv
10.5555/7916 · DOI
Johanna Drucker. 2012. Humanistic theory and digital scholarship. Debates in the digital humanities 150 (2012), 85–95.
Michael Eraut. 2010. Knowledge, working practices, and learning. In Learning through practice. Springer, 37–58.
James Fleck. 1998. Expertise: knowledge, power and tradeability. In Exploring expertise. Springer, 143–171.
Ruth C. Fong and Andrea Vedaldi. 2017. Interpretable Explanations of Black Boxes by Meaningful Perturbation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Counterfactual Visual Explanations. In Proceedings of the 36th International Conference on Machine Learning, Vol. 97. Long Beach, California, USA. http://proceedings.mlr.press/v97/goyal19a.html arXiv:1904.07451. · arXiv
Donna Haraway. 1988. Situated knowledges: The science question in feminism and the privilege of partial perspective. Feminist studies 14, 3 (1988), 575–599.
Johanna Hartelius. 2008. The Rhetoric of Expertise. Ph.D. Dissertation. The University of Texas at Austin.
10.1080/02691728.2011.578301Publisher: · DOI
Sam Hepenstal and David McNeish. 2020. Explainable Artificial Intelligence: What Do You Need to Know?. In Augmented Cognition. Theoretical and Technological Approaches, Dylan D. Schmorrow and Cali M. Fidopiastis (Eds.). Springer International Publishing, Cham, 266–275.
Jessica Hullman and Nick Diakopoulos. 2011. Visualization rhetoric: Framing effects in narrative visualization. IEEE transactions on visualization and computer graphics 17, 12(2011), 2231–2240.
Bogdan Kulynych, David Madras, Smitha Milli, Inioluwa Deborah Raji, Angela Zhou, and Richard Zemel. 2020. Participatory Approaches to Machine Learning. International Conference on Machine Learning Workshop.
10.1609/hcomp.v7i1.5280 · DOI
10.1145/3233231 · DOI · Paper page
Scott M Lundberg, Bala Nair, Monica S Vavilala, Mayumi Horibe, Michael J Eisses, Trevor Adams, David E Liston, Daniel King-Wai Low, Shu-Fang Newman, Jerry Kim, 2018. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature biomedical engineering 2, 10 (2018), 749–760.
George A Miller. 1956. The magical number seven, plus or minus two: Some limits on our capacity for processing information.Psychological review 63, 2 (1956), 81.
10.1007/s13347-020-00405-8arXiv:2007.04068 · DOI
Sina Mohseni, Jeremy E. Block, and Eric D. Ragan. 2020. A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning. arXiv:1801.05075 [cs] (June 2020). http://arxiv.org/abs/1801.05075 arXiv:1801.05075. · arXiv
Sina Mohseni, Niloofar Zarei, and Eric D. Ragan. 2020. A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. arXiv:1811.11839 [cs] (April 2020). http://arxiv.org/abs/1811.11839 arXiv:1811.11839. · arXiv
Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Gershman, and Finale Doshi-Velez. 2018. How do humans understand explanations from machine learning systems? an evaluation of the human-interpretability of explanation. arXiv:1802.00682 (2018). · arXiv
Prajwal Paudyal, Junghyo Lee, Azamat Kamzin, Mohamad Soudki, Ayan Banerjee, and Sandeep KS Gupta. 2019. Learn2Sign: Explainable AI for Sign Language Learning.. In IUI Workshops.
Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Manipulating and Measuring Model Interpretability. arXiv:1802.07810 [cs] (Nov. 2019). http://arxiv.org/abs/1802.07810 arXiv:1802.07810. · arXiv
Alun Preece, Dan Harborne, Dave Braines, Richard Tomsett, and Supriyo Chakraborty. 2018. Stakeholders in Explainable AI. arXiv:1810.00184 [cs] (Sept. 2018). http://arxiv.org/abs/1810.00184 arXiv:1810.00184. · arXiv
Alexander Renkl. 2014. Toward an instructionally oriented theory of example-based learning. Cognitive science 38, 1 (2014), 1–37.
Marco Tulio Ribeiro Sameer Singh and Carlos Guestrin. 2016. Model-agnostic interpretability of machine learning. (2016). arXiv:1606.05386. · arXiv
Mireia Ribera and Agata Lapedriza. 2019. Can we do better explanations? A proposal of user-centered explainable AI.. In IUI Workshops.
Wojciech Samek Thomas Wiegand and Klaus-Robert Müller. 2017. Explainable artificial intelligence: Understanding visualizing and interpreting deep learning models. (2017). arXiv:1708.08296. · arXiv
Udo Schlegel, Hiba Arnout, Mennatallah El-Assady, Daniela Oelke, and Daniel A. Keim. 2019. Towards a Rigorous Evaluation of XAI Methods on Time Series. arXiv:1909.07082 [cs] (Sept. 2019). http://arxiv.org/abs/1909.07082 arXiv:1909.07082. · arXiv
Johannes Schneider and Joshua Handali. 2019. PERSONALIZED EXPLANATION FOR MACHINE LEARNING: A CONCEPTUALIZATION. In Proceedings of the 27th European Conference on Information Systems (ECIS). Stockholm & Uppsala, Sweden. https://aisel.aisnet.org/ecis2019_rp/171
10.1109/TVCG.2019.2934629Conference · DOI
Clay Spinuzzi. 2005. The methodology of participatory design. Technical communication 52, 2 (2005), 163–174.
Mukund Sundararajan, Jinhua Xu, Ankur Taly, Rory Sayres, and Amir Najmi. 2019. Exploring Principled Visualizations for Deep Network Attributions.. In IUI Workshops, Vol. 4.
Richard Tomsett Dave Braines Dan Harborne Alun Preece and Supriyo Chakraborty. 2018. Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems. In arXiv:1806.07552 [cs]. http://arxiv.org/abs/1806.07552 arXiv:1806.07552. · arXiv
Ryan Weber. 2012. Review of The Rhetoric of Expertise. Rhetoric and Public Affairs 15, 1 (2012), 193–196. https://www.jstor.org/stable/41955617 Publisher: Michigan State University Press.
Adrian Weller. 2019. Transparency: Motivations and Challenges. arXiv:1708.01870 [cs] (Aug. 2019). http://arxiv.org/abs/1708.01870 arXiv:1708.01870. · arXiv
Jacob O Wobbrock and Julie A Kientz. 2016. Research contributions in human-computer interaction. interactions 23, 3 (2016), 38–44.
Yao Xie Ge Gao and Xiang ’Anthony’ Chen. 2019. Outlining the Design Space of Explainable Intelligent Systems for Medical Diagnosis. arxiv:1902.06019 [cs.HC] · arXiv
Jill Yielder. 2001. Professional Expertise: A Model for Integration and Change. Thesis. ResearchSpace@Auckland. https://researchspace.auckland.ac.nz/handle/2292/2340Accepted: 2008-01-30T02:04:57Z.
10.1007/978-3-319-10590-1_53Series · DOI
Ruijing Zhao, Izak Benbasat, and Hasan Cavusoglu. 2019. Transparency in Advice-Giving Systems: A Framework and a Research Model for Transparency Provision.. In IUI Workshops.

Sorted by citations

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review
Chen, H., Gomez, C., Huang, C.-M., & Unberath, M. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00699-2 · Cited by 300
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 493
“Planning is a task-oriented metacognitive strategy entailing both the definition of clear goals (i.e., self-awareness) and devising a comprehensive approach for achieving them by breaking them down into smaller, manageable steps (i.e., task decomposition) [21].”
Background
Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI
Pushkarna, M., Zaldivar, A., & Kjartansson, O. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533231 · Cited by 231
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 233
From explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180
The coming of age of interpretable and explainable machine learning models
Lisboa, P. J. G., Saralajew, S., Vellido, A., Fernández-Domenech, R., & Villmann, T. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.02.040 · Cited by 167
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
The role of explainable AI in the context of the AI Act
Panigutti, C., Hamon, R., Hupont, I., Llorca, D. F., Yela, D. F., Junklewitz, H., Scalzo, S., Mazzini, G., Sánchez, I., Garrido, J. S., et al. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594069 · Cited by 201
Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making
Ma, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143
Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141
Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience
Liao, Q. V., Subramonyam, H., Wang, J., & Vaughan, J. W. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580652 · Cited by 120
Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability
Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581161 · Cited by 108
“Such strategies should be adapted to decision subjects’ insight needs [88] and designed in a way that they would understand [11, 52].”
Background
Fairness via Explanation Quality
Dai, J. C., Upadhyay, S., Aïvodji, U., Bach, S. H., & Lakkaraju, H. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534159 · Cited by 72
Exploring the effects of human-centered AI explanations on trust and reliance
Scharowski, N., Perrig, S. A. C., Svab, M., Opwis, K., & Brühlmann, F. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1151150 · Cited by 57
Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work
Zheng, C., Wang, D., Wang, A. Y., & Ma, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517615 · Cited by 54
Selective Explanations: Leveraging Human Input to Align Explainable AI
Lai, V. W.-M., Zhang, Y., Chen, C., Liao, Q. V., & Tan, C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610206 · Cited by 75
Explainable reinforcement learning (XRL): a systematic literature review and taxonomy
Bekkemoen, Y. (2023) · Machine Learning · doi:10.1007/s10994-023-06479-7 · Cited by 74
Mitigating knowledge imbalance in AI-advised decision-making through collaborative user involvement
Gómez, C., Unberath, M., & Huang, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102977 · Cited by 57
Machine Learning Security in Industry: A Quantitative Survey
Grosse, K., Bieringer, L., Besold, T. R., Biggio, B., & Krombholz, K. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3251842 · Cited by 55
What Are the Users’ Needs? Design of a User-Centered Explainable Artificial Intelligence Diagnostic System
He, X., Hong, Y., Xi, Z., & Zhang, Y. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2095093 · Cited by 49
How mature is requirements engineering for AI-based systems? A systematic mapping study on practices, challenges, and future research directions
Habiba, U.-E., Haug, M., Bogner, J., & Wagner, S. (2024) · Requirements Engineering · doi:10.1007/s00766-024-00432-3 · Cited by 51
Towards a multi-stakeholder value-based assessment framework for algorithmic systems
Yurrita, M., Murray-Rust, D., Balayn, A., & Bozzon, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533118 · Cited by 50
Seamful XAI: Operationalizing Seamful Design in Explainable AI
Ehsan, U., Liao, Q. V., Passi, S., Riedl, M. O., & Daumé, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637396 · Cited by 81
Slide4N: Creating Presentation Slides from Computational Notebooks with Human-AI Collaboration
Wang, F., Liu, X., Liu, O., Neshati, A., Ma, T., Zhu, M., & Zhao, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580753 · Cited by 38
Human‐centered explainable artificial intelligence: An Annual Review of Information Science and Technology (ARIST) paper
Ridley, M. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24889 · Cited by 61
Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities
Li, H. X., Appleby, G., Brumar, C. D., Chang, R., & Suh, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3326904 · Cited by 56
User-Centric Explainability in Healthcare: A Knowledge-Level Perspective of Informed Machine Learning
Oberste, L., & Heinzl, A. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3227225 · Cited by 35
A Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?
Wang, Z., Huang, C., & Yao, X. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3702004 · Cited by 38
Toward Practical Usage of the Attention Mechanism as a Tool for Interpretability
Tutek, M., & Šnajder, J. (2022) · IEEE Access · doi:10.1109/access.2022.3169772 · Cited by 35
Whose Knowledge is Valued? Epistemic Injustice in CSCW Applications
Ajmani, L. H., Foriest, J. C., Taylor, J., Pittman, K., Gilbert, S., & DeVito, M. A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687062 · Cited by 30
Let's Talk Futures: A Literature Review of HCI's Future Orientation
Sanchez, C., Wang, S., Savolainen, K., Epp, F. A., & Salovaara, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713759 · Cited by 30
Using human-in-the-loop and explainable AI to envisage new future work practices
Tsiakas, K., & Murray-Rust, D. (2022) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3529190.3534779 · Cited by 38
Toward Human-centered XAI in Practice: A survey
Kong, X., Liu, S., & Zhu, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-022-1407-3 · Cited by 35
Questioning the ability of feature-based explanations to empower non-experts in robo-advised financial decision-making
Bertrand, A., Eagan, J., & Maxwell, W. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594053 · Cited by 27
How can Explainability Methods be Used to Support Bug Identification in Computer Vision Models?
Balayn, A., Rikalo, N., Lofi, C., Yang, J., & Bozzon, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517474 · Cited by 26
How to Choose an Explainability Method? Towards a Methodical Implementation of XAI in Practice
Vermeire, T., Laugel, T., Renard, X., Martens, D., & Detyniecki, M. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-93736-2_39 · Cited by 23
A Human–AI interaction paradigm and its application to rhinocytology
Desolda, G., Dimauro, G., Esposito, A., Lanzilotti, R., Matera, M., & Zancanaro, M. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102933 · Cited by 25
"If it is easy to understand then it will have value": Examining Perceptions of Explainable AI with Community Health Workers in Rural India
Okolo, C. T., Agarwal, D., Dell, N. L., & Vashistha, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637348 · Cited by 28
Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts
Suh, A., Appleby, G., Anderson, E. W., Finelli, L. A., Chang, R., & Cashman, D. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3259341 · Cited by 20
Rideshare Transparency: Translating Gig Worker Insights on AI Platform Design to Policy
Rao, V. N., Dalal, S., Agarwal, E., Calacci, D., & Monroy‐Hernández, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711059 · Cited by 32
GAM Coach: Towards Interactive and User-centered Algorithmic Recourse
Wang, Z. J., Vaughan, J. W., Caruana, R., & Chau, D. H. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580816 · Cited by 20
Can Requirements Engineering Support Explainable Artificial Intelligence? Towards a User-Centric Approach for Explainability Requirements
Habiba, U.-E., Bogner, J., & Wagner, S. (2022) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew56159.2022.00038 · Cited by 25
Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI
Donoso-Guzmán, I., Ooge, J., Parra, D., & Verbert, K. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44070-0_10 · Cited by 23
How do ML practitioners perceive explainability? an interview study of practices and challenges
Habiba, U.-E., Habib, M. K., Bogner, J., Fritzsch, J., & Wagner, S. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10565-2 · Cited by 28
Unpacking Trust Dynamics in the LLM Supply Chain: An Empirical Exploration to Foster Trustworthy LLM Production & Use
Balayn, A., Yurrita, M., Rancourt, F., Casati, F., & Gadiraju, U. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713787 · Cited by 36
Improving Governance Outcomes Through AI Documentation: Bridging Theory and Practice
Winecoff, A., & Bogen, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713814 · Cited by 39
Beyond Visual Analytics: Human-Machine Teaming for AI-Driven Data Sensemaking
Wenskovitch, J. E., Fallon, C. K., Miller, K. H., & Dasgupta, A. (2021) · IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX) · doi:10.1109/trex53765.2021.00012 · Cited by 15
Saliency Cards: A Framework to Characterize and Compare Saliency Methods
Boggust, A., Suresh, H., Strobelt, H., Guttag, J. V., & Satyanarayan, A. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593997 · Cited by 15
Self-citation
Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bank
Sipos, L., Schäfer, U., Glinka, K., & Müller-Birn, C. (2023) · Mensch und Computer (MuC) · doi:10.1145/3603555.3608551 · Cited by 12
Systemization of Knowledge (SoK): Creating a Research Agenda for Human-Centered Real-Time Risk Detection on Social Media Platforms
Alsoubai, A., Park, J., Qadir, S., Stringhini, G., Razi, A., & Wiśniewski, P. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642315 · Cited by 13
Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency
Ajmani, L., Abdelkadir, N. A., & Chancellor, S. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732071 · Cited by 15
(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
Jin, A., & Salehi, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641902 · Cited by 18
Towards Visual Analytics for Explainable AI in Industrial Applications
Kucher, K., Zohrevandi, E., & Westin, C. (2025) · Analytics · doi:10.3390/analytics4010007 · Cited by 14
Faulty or Ready? Handling Failures in Deep-Learning Computer Vision Models until Deployment: A Study of Practices, Challenges, and Needs
Balayn, A., Rikalo, N., Yang, J., & Bozzon, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581555 · Cited by 13
Kaleidoscope: Semantically-grounded, context-specific ML model evaluation
Suresh, H., Shanmugam, D., Chen, T., Bryan, A. G., D’Amour, A., Guttag, J. V., & Satyanarayan, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581482 · Cited by 11
Self-citation
Minimum levels of interpretability for artificial moral agents
Vijayaraghavan, A., & Badea, C. (2024) · AI and Ethics · doi:10.1007/s43681-024-00536-0 · Cited by 12
AI is Entering Regulated Territory: Understanding the Supervisors' Perspective for Model Justifiability in Financial Crime Detection
Bertrand, A., Eagan, J., Maxwell, W., & Brand, J. L. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642326 · Cited by 8
Enhancing Transparency through Explainable Artificial Intelligence: An Exploratory Analysis on Collusion and Corruption Scenario in Digital Government
Sampaio, I. G. B., Fontes, S., Andrade, E. D. O., Bernardini, F., & Viterbo, J. (2024) · Annual International Conference on Digital Government Research · doi:10.1145/3657054.3657274 · Cited by 7
Ethical and epistemic implications of artificial intelligence in medicine: a stakeholder-based assessment
Adams, J. (2025) · AI & Society · doi:10.1007/s00146-025-02398-4 · Cited by 11
Aspects and Views on Responsible Artificial Intelligence
Brumen, B., Göllner, S., & Tropmann-Frick, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25599-1_29 · Cited by 7
FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines
Barker, M., Kallina, E., Ashok, D., Collins, K. M., Casovan, A., Weller, A., Talwalkar, A., Chen, V., & Bhatt, U. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623239 · Cited by 8
Developing science teachers’ enacted pedagogical content knowledge through integration of student feedback into the Refined Consensus Model of pedagogical content knowledge
Hartmuth, D., Amin, R. M., Schlichting, D., Schött, S., Haudenschild, J., Küchenhoff, H., & Neuhaus, B. J. (2025) · Cogent Education · doi:10.1080/2331186x.2025.2470563 · Cited by 7
FLAN: feature-wise latent additive neural models for biological applications
Nguyen, A.-P., Vasilaki, S., & Martínez, M. R. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad056 · Cited by 8
From Discovery to Adoption: Understanding the ML Practitioners’ Interpretability Journey
Ashtari, N., Mullins, R. S., Qian, C., Wexler, J., Tenney, I., & Pushkarna, M. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596046 · Cited by 9
Perception-Centric Explainable AI: Bridging Cognitive Theories and HCI Design for Enhanced User Experience
Alhasan, S., & Alnanih, R. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.03.040 · Cited by 5
Enhancing AI Explainability Through the EXACT Framework: A User-Centric Approach
Alhasan, S., & Alnanih, R. A. (2025) · IEEE Access · doi:10.1109/access.2025.3576234 · Cited by 5
Introducing the ethical-epistemic matrix: a principle-based tool for evaluating artificial intelligence in medicine
Adams, J. (2024) · AI and Ethics · doi:10.1007/s43681-024-00597-1 · Cited by 5
Visually Explaining Uncertain Price Predictions in Agrifood: A User-Centred Case-Study
Ooge, J., & Verbert, K. (2022) · Agriculture · doi:10.3390/agriculture12071024 · Cited by 6
Tensions Between the Proxies of Human Values in AI
Datta, T., Nissani, D., Cembalest, M., Khanna, A., Massa, H., & Dickerson, J. P. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00049 · Cited by 7
The explanation dialogues: an expert focus study to understand requirements towards explanations within the GDPR
State, L., Colmenarejo, A. B., Beretta, A., Ruggieri, S., Turini, F., & Law, S. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09430-w · Cited by 6
Can AI Model the Complexities of Human Moral Decision-making? A Qualitative Study of Kidney Allocation Decisions
Keswani, V., Conitzer, V., Sinnott-Armstrong, W. P., Nguyen, B. K., Heidari, H., & Borg, J. S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714167 · Cited by 8
Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making
Lee, M. H., & Tok, M. Z. Y. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732155 · Cited by 8
Explainable AI for Daily Scenarios from End-Users’ Perspective: Non-Use, Concerns, and Ideal Design
Wang, L., Anyi, C. L., Xu, K., Liu, Y., Arriaga, R. I., & Goel, A. K. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735796 · Cited by 7
Modular oversight methodology: a framework to aid ethical alignment of algorithmic creations
Kyriakou, K., & Otterbacher, J. (2024) · Design Science · doi:10.1017/dsj.2024.23 · Cited by 5
A Study of Explainability Features to Scrutinize Faceted Filtering Results
Qu, J., Arguello, J., & Wang, Y. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482409 · Cited by 5
Communicating Performance of Regression Models Using Visualization in Pharmacovigilance
Suh, A., Appleby, G., Anderson, E. W., Finelli, L. A., & Cashman, D. (2021) · Workshop on Visual Analytics in Healthcare (VAHC) · doi:10.1109/vahc53616.2021.00006 · Cited by 4
Requirements Engineering for Explainable AI
Umm-e-Habiba (2023) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re57278.2023.00058 · Cited by 4
Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships
Boggust, A., Bang, H., Strobelt, H., & Satyanarayan, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713406 · Cited by 12
Self-citation
Exploring Subjective Notions of Explainability through Counterfactual Visualization of Sentiment Analysis
Crisan, A., Butters, N., & Zoe (2024) · IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV) · doi:10.1109/beliv64461.2024.00007 · Cited by 3
f-RecX: A framework for designing effective textual explanations in recommender systems’ user interfaces
Al-Hazwani, I., Morgenshtern, G., El‐Assady, M., & Bernard, J. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103627 · Cited by 3
Explainable AI for Hate Speech Moderation: A Stakeholder‐Centered and Sociotechnical Review
Qureshi, M. D. M., Qureshi, M. A., & Rashwan, W. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70076 · Cited by 6
VIME: Visual Interactive Model Explorer for Identifying Capabilities and Limitations of Machine Learning Models for Sequential Decision-Making
Antar, A. D., Molaei, S., Chen, Y.-Y., Lee, M. L., & Banović, N. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676323 · Cited by 9
ABC-GAN: Spatially Constrained Counterfactual Generation for Image Classification Explanations
Mindlin, D., Schilling, M., & Cimiano, P. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_15 · Cited by 3
Explainable AI For Financial and Operational Decision Support: Improving Transparency and Managerial Trust
Chy, M. T. I. (2024) · American Journal of Advanced Technology and Engineering Solutions · doi:10.63125/kwfvzw68 · Cited by 15
Human-AI Interfaces are a Central Component of Trustworthy AI
Plass, M., Kargl, M., Evans, T., Brčić, L., Regitnig, P., Geißler, C., Carvalho, R., Jansen, C., Zerbe, N., Holzinger, A., et al. (2022) · Intelligent systems reference library · doi:10.1007/978-3-031-12807-3_11 · Cited by 2
Persona Adaptable Strategies Make Large Language Models Tractable
Hattab, G., Anžel, A., Dubey, A., Ezekannagha, C., Yang, Z., & İlgen, B. (2024) · International Conference on Natural Language Processing and Information Retrieval (NLPIR) · doi:10.1145/3711542.3711581 · Cited by 2
Human-Centered Explainable AI (XAI)
Liao, Q. V., & Varshney, K. R. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.011 · Cited by 448
Toward User-centered Explainable Displays for Complex Machine Learning Models in Healthcare: A Case Study of Heart Disease Prediction
Nuamah, J. K. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241264252 · Cited by 1
"Good" XAI Design: For What? In Which Ways?
Wang, L., Liu, Y., & Goel, A. K. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720036 · Cited by 1
Disentangling Stakeholder Role and Expertise in User-Centered Explainable AI
Szymanski, M., Abeele, V. V., & Verbert, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728351 · Cited by 1
Biasing Rule-Based Explanations Towards User Preferences
Mahya, P., & Fürnkranz, J. (2025) · Information · doi:10.3390/info16070535 · Cited by 2
Towards Explainable AI: A Framework for Interpretable Deep Learning in High-Stakes Domains
Manga, I. (2025) · International Conference on Soft Computing for Security Applications (ICSCSA) · doi:10.1109/icscsa66339.2025.11170778 · Cited by 1
Domain Experience and Expertise in Explainable AI Applications: A Bearing Fault Diagnosis Case Study
Zhao, Z., Castelle, M., & Turkay, Ç. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757583 · Cited by 1
SQeX: a scenario-based and question-driven framework for designing explanatory interfaces in autonomous vehicles
Zhang, Y., Wang, W., Li, J., Guo, W., & Sun, X. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2626745 · Cited by 1
Understanding Movement Disorder Specialists’ Workflow for Designing Digital Health Tools
Ikhile, J. A., & Pandey, V. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3803795 · Cited by 1
XAgen: An Explainability Tool for Identifying and Correcting Failures in Multi-Agent Workflows
Wang, X., Yin, M., Koh, E., & Dogan, M. D. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798531 · Cited by 2
Speeding Things Up. Can Explainability Improve Human Learning?
Mannmeusel, J., Rothfelder, M., & Khoshrou, S. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_4 · Cited by 2
Towards Trustworthy and Human-Centred AI Explanations
Manresa-Yee, C., Ramis, S., Gaya-Morey, F. X., & Rubio, J. M. B. (2024) · Springer eBooks · doi:10.1007/978-3-031-71697-3_4 · Cited by 2
Trustworthiness Evaluation of Large Language Models
Chen, P., & Liu, S. (2025) · Springer eBooks · doi:10.1007/978-3-031-76770-8_12 · Cited by 2

Authors: Suresh, H., Gomez, S. R., Nam, K. K., & Satyanarayan, A.
Affiliations: Massachusetts Institute of Technology (US)
ORCID: Suresh, Harini
Gomez, Steven R.
Nam, Kevin K.
Satyanarayan, Arvind
Venue: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Publisher: ACM
Published: 2021-05-06
Pages: 1-16
DOI: 10.1145/3411764.3445088
OpenAlex: W3123497755
Cited by: 117
References: 81
Citations by year: 2026: 15 · 2025: 24 · 2024: 28 · 2023: 32 · 2022: 14 · 2021: 4
FWCI: 9.01
Type: conference paper
Language: en
Open access: bronze
doi.org
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor, 1 d)
Locations: CHI Conference on Human Factors in Computing Systems (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Air Force Research Laboratory (FA8750-19-2-1000)
NSF (National Science Foundation) (1900991)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords: model interpretability, stakeholder needs, stakeholder typology, black-box systems, accountability, formal knowledge, instrumental knowledge, personal knowledge, user study

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks