The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle
Abstract
The interpretability or explainability of AI systems (XAI) has been a topic gaining renewed attention in recent years across AI and HCI communities. Recent work has drawn attention to the emergent explainability requirements of in situ, applied projects, yet further exploratory work is needed to more fully understand this space. This paper investigates applied AI projects and reports on a qualitative interview study of individuals working on AI projects at a large technology and consulting company. Presenting an empirical understanding of the range of stakeholders in industrial AI projects, this paper also draws out the emergent explainability practices that arise as these projects unfold, highlighting the range of explanation audiences (who), as well as how their explainability needs evolve across the AI project lifecycle (when). We discuss the importance of adopting a sociotechnical lens in designing AI systems, noting how the “AI lifecycle” can serve as a design metaphor to further the XAI design field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Participants' questions reflected their desire for an actionable or utility-oriented understanding to support their end goal of optimizing code generation and programming productivity [18,50,52], such as asking the Input, Output, How and How-to XAI questions (described in Section 4) to facilitate strategies to get better outputs from the AI.”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [18]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Hutchins, E. (1995) · doi:10.7551/mitpress/1881.001.0001 · Cited by 10,801
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“We must also take into account individuals’ AI literacy [30].”
BackgroundHow the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“The contemporary AI landscape has been aptly characterized as opaque[6] with its ever-increasing interdependencies on multiple components and algorithmic sophistication.”
BackgroundModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Context and consciousness: activity theory and human-computer interaction
(1996) · Choice Reviews Online · doi:10.5860/choice.33-5756 · Cited by 2,511
Research works printed but not linked: 7
- 10.5555/1597122.1597135 · doi:10.5555/1597122.1597135
- Shipi Dhanorkar , Christine T Wolf , Kun Qian , Anbang Xu , Lucian Popa , and Yunyao Li. 2020. Tutorial on Explainability for Natural Language Processing. Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing(2020). Shipi Dhanorkar, Christine T Wolf, Kun Qian, Anbang Xu, Lucian Popa, and Yunyao Li. 2020. Tutorial on Explainability for Natural Language Processing. Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing(2020).
- Upol Ehsan and Mark O. Riedl . 2020. Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach . In HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence, Constantine Stephanidis, Masaaki Kurosu, Helmut Degen, and Lauren Reinerman-Jones (Eds.) . Springer International Publishing , Cham , 449–466. Upol Ehsan and Mark O. Riedl. 2020. Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach. In HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence, Constantine Stephanidis, Masaaki Kurosu, Helmut Degen, and Lauren Reinerman-Jones (Eds.). Springer International Publishing, Cham, 449–466.
- Riccardo Guidotti , Anna Monreale , Salvatore Ruggieri , Franco Turini , Fosca Giannotti , and Dino Pedreschi . 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR) 51, 5 ( 2018 ), 1–42. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR) 51, 5 (2018), 1–42.
- Karen Holtzblatt Jessamyn Wendell and Shelley Wood. 2004. Rapid Contextual Design. Morgan Kaufmann. https://www.elsevier.com/books/rapid-contextual-design/holtzblatt/978-0-12-354051-5 Karen Holtzblatt Jessamyn Wendell and Shelley Wood. 2004. Rapid Contextual Design. Morgan Kaufmann. https://www.elsevier.com/books/rapid-contextual-design/holtzblatt/978-0-12-354051-5
- 10.5555/2891730.2891924 · doi:10.5555/2891730.2891924
- William R Swartout . 1983 . XPLAIN: A system for creating and explaining expert consulting programs. Artificial intelligence 21, 3 (1983), 285–325. William R Swartout. 1983. XPLAIN: A system for creating and explaining expert consulting programs. Artificial intelligence 21, 3 (1983), 285–325.
Cited by
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2021) · arXiv · doi:10.48550/arxiv.2112.11561 · Cited by 336
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2024) · IEEE Access · doi:10.1109/access.2024.3431437 · Cited by 323
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
“…desire for an actionable or utility-oriented understanding to support their end goal of optimizing code generation and programming productivity [18, 50, 52], such as asking the Input, Output, How and How-to XAI questions (described in Section 4) to facilitate strategies to get better outputs…”
Background
Metadata
- Authors
- Dhanorkar, S., Wolf, C. T., Qian, K., Xu, A., Popa, L., & Li, Y.
- Affiliations
- Pennsylvania State University (US)
- Independent Researcher, United States
- Amazon (United States) (US)
- IBM Research - Almaden (US)
- IBM (United States) (US)
- Venue
- Designing Interactive Systems Conference 2021
- Publisher
- ACM
- Published
- 2021-06-28
- Pages
- 1591-1602
- OpenAlex
- W3173721282
- Cited by
- 119
- References
- 51
- Citations by year
- 2026: 12 · 2025: 28 · 2024: 35 · 2023: 20 · 2022: 20 · 2021: 4
- FWCI
- 9.13
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Designing Interactive Systems Conference (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- explainable AI, AI lifecycle, stakeholder needs, socio-technical design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle. https://app.cochat.ai/papers/10.1145/3461778.3462131
- The paper
- Dhanorkar, S., Wolf, C. T., Qian, K., Xu, A., Popa, L., & Li, Y. (2021). Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle. Designing Interactive Systems Conference 2021. https://doi.org/10.1145/3461778.3462131
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.