"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making
Open accessThe DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Abstract
Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant. In a qualitative lab study, we interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) predictions for prostate cancer diagnosis, to learn the types of information that they desired about the AI assistant. Our findings reveal that, far beyond understanding the local, case-specific reasoning behind any model decision, clinicians desired upfront information about basic, global properties of the model, such as its known strengths and limitations, its subjective point-of-view, and its overall design objective--what it's designed to be optimized for. Participants compared these information needs to the collaborative mental models they develop of their medical colleagues when seeking a second opinion: the medical perspectives and standards that those colleagues embody, and the compatibility of those perspectives with their own diagnostic patterns. These findings broaden and enrich discussions surrounding AI transparency for collaborative decision-making, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The training included learning optimal ways of using the AI in their settings and pointing out possible errors [18].”
From Explainable recommendation: when design meets trust calibration (2021) · cites this paper as [18]
In the citing paper
“This finding relates to earlier work in Human-AI trust calibration [18], which showed that training clinicians to use the AI-based decision-making tool reduced users’ errors and increased the Human-AI collaborative decision-making performance.”
From Explainable recommendation: when design meets trust calibration (2021) · cites this paper as [18]
In the citing paper
“there has been increasing recent research efforts [8, 15, 22, 41, 58, 65] that highlight the importance of involving stakeholders to understand their practices and needs [15, 41, 65] and socio-environmental factors [8] for the design and evaluation of a system.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [15]
In the citing paper
“researchers have discussed the importance of an AI explanation to communicate an AI output to a user [5, 20, 29] and realize human-AI collaboration [48] for a decision-making task [15, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [15]
In the citing paper
“Third, this allows uncertainty to be expressed seamlessly within the natural language interactions of LLM-infused applications, rather than on the side or in onboarding materials [21, 74, 79] that users might overlook.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [21]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Braun, V., & Clarke, V. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-004 · Cited by 10,299
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Recent work on fairness and accountability has proposed the use of short documents accompanying trained ML models to disclose the intended use of models, details of their performance evaluation procedures, and the potential biases that they may embody [25, 51].”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439
Shared mental models in expert team decision making.
Cannon‐Bowers, J. A., Salas, E., & Converse, S. A. (1993) · Cited by 1,545
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467
Research works printed but not linked: 7
- Wouter Bulten , Hans Pinckaers , Hester van Boven , Robert Vink , Thomas de Bel , Bram van Ginneken , Jeroen van der Laak , Christina Hulsbergen-van de Kaa, and Geert Litjens . 2019 . Automated Gleason Grading of Prostate Biopsies using Deep Learning . arXiv preprint arXiv:1907.07980 (2019). Wouter Bulten, Hans Pinckaers, Hester van Boven, Robert Vink, Thomas de Bel, Bram van Ginneken, Jeroen van der Laak, Christina Hulsbergen-van de Kaa, and Geert Litjens. 2019. Automated Gleason Grading of Prostate Biopsies using Deep Learning. arXiv preprint arXiv:1907.07980 (2019). · arXiv
- Dumitru Erhan , Yoshua Bengio , Aaron Courville , and Pascal Vincent . 2009 . Visualizing higher-layer features of a deep network . University of Montreal 1341 , 3 (2009), 1 . Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. University of Montreal 1341, 3 (2009), 1.
- Timnit Gebru , Jamie Morgenstern , Briana Vecchione , Jennifer Wortman Vaughan , Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018 . Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). · arXiv
- Ken Hinckley ( Ed .). 2018 . ACM Trans. Comput.-Hum. Interact. 25 , 1 (2018). Ken Hinckley (Ed.). 2018. ACM Trans. Comput.-Hum. Interact. 25, 1 (2018).
- 10.5555/3305381.3305576 · doi:10.5555/3305381.3305576
- Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). · arXiv
- Ming Yin , Jennifer Wortman Vaughan, and Hanna Wallach . 2019 . Understanding the Effect of Accuracy on Trust in Machine Learning Models . (2019). Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the Effect of Accuracy on Trust in Machine Learning Models. (2019).
Cited by
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
“Indeed, at first glance, it appears that researchers have already confirmed the utility of explanations on tasks ranging from medical diagnosis [10, 47] and data annotation [57] to deception detection [36].”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614
“In subsequent work, Cai et al. examined what information pathologists (in a lab setting) found to be important when being introduced to AI assistants, before integrating these assistants into routine prostate cancer screening practice [9].”
BackgroundSelf-citationHow Machine Learning Will Transform Biomedicine
Goecks, J., Jalili, V., Heiser, L. M., & Gray, J. W. (2020) · Cell · doi:10.1016/j.cell.2020.03.022 · Cited by 582
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543
“This may take the form of a human-AI calibration phase, where a library of curated test cases is used to demonstrate the system ’ s strengths and weaknesses, or an AI primer during on-boarding 40 .”
MethodSharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.011 · Cited by 472
Rezwana, J., & Maher, M. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3519026 · Cited by 342
Towards Accurate Differential Diagnosis with Large Language Models
McDuff, D., Schaekermann, M., Tu, T., Palepu, A., Wang, A., Garrison, J., Singhal, K., Sharma, Y., Azizi, S., Kulkarni, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.00164 · Cited by 311
Towards accurate differential diagnosis with large language models
McDuff, D., Schaekermann, M., Tu, T., Palepu, A., Wang, A., Garrison, J., Singhal, K., Sharma, Y., Azizi, S., Kulkarni, K., et al. (2025) · Nature · doi:10.1038/s41586-025-08869-4 · Cited by 311
Chen, H., Gomez, C., Huang, C.-M., & Unberath, M. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00699-2 · Cited by 300
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376739 · Cited by 271
Self-citationSezgin, E. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231186520 · Cited by 261
Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000016 · Cited by 251
“When looking at the third layer, we consider that explainability can promote the use of the system as originally intended, namely as a CDSS that promotes algorithm-based decision making [75].”
BackgroundHaque, A. B., Islam, A. K. M. N., & Mikalef, P. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122120 · Cited by 240
Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 233
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Zhang (2021) · doi:10.1145/3432945 · Cited by 225
Guidelines and evaluation of clinical explainable AI in medical image analysis
Jin, W., Li, X., Fatehi, M., & Hamarneh, G. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102684 · Cited by 199
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08155 · Cited by 193
Utilization of Self-Diagnosis Health Chatbots in Real-World Settings: Case Study
Fan, X., Chao, D., Zhang, Z., Wang, D., Li, X., & Tian, F. (2020) · Journal of Medical Internet Research · doi:10.2196/19928 · Cited by 192
Should AI models be explainable to clinicians?
Abgrall, G., Holder, A. L., Chelly Dagdia, Z., Zeitouni, K., & Monnet, X. (2024) · Critical Care · doi:10.1186/s13054-024-05005-y · Cited by 175
Bulten, W., Balkenhol, M., Belinga, J.-J. A., Brilhante, A., Çakır, A., Egevad, L., Eklund, M., Farré, X., Geronatsiou, K., Molinié, V., et al. (2021) · Modern Pathology · doi:10.1038/s41379-020-0640-y · Cited by 174
Herrmann, T., & Pfeiffer, S. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01391-5 · Cited by 173
“Cai et al. (2019a) also support creating possibilities for humans to interactively refine AI results.”
BackgroundInfluentialThieme, A., Hanratty, M., Lyons, M., Palacios, J., Marques, R. F., Morrison, C., & Doherty, G. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3564752 · Cited by 171
Reviewing the Need for Explainable Artificial Intelligence (xAI)
Gerlings, J., Shollo, A., & Constantiou, I. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.156 · Cited by 189
You, S., Yang, C., & Li, X. (2022) · Journal of Management Information Systems · doi:10.1080/07421222.2022.2063553 · Cited by 192
Applications, Challenges, and Future Directions of Human-in-the-Loop Learning
Kumar, S., Datta, S., Singh, V., Datta, D., Kumar Singh, S., & Sharma, R. (2024) · IEEE Access · doi:10.1109/access.2024.3401547 · Cited by 168
Panigutti, C., Beretta, A., Giannotti, F., & Pedreschi, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502104 · Cited by 214
Jain, A., Way, D., Gupta, V., Gao, Y., de Oliveira Marinho, G., Hartford, J., Sayres, R., Kanada, K., Eng, C., Nagpal, K., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.7249 · Cited by 164
Hogg, H. D. J., Al-Zubaidy, M., Technology Enhanced Macular Services Study Reference Group, Talks, J., Denniston, A. K., Kelly, C. J., Malawana, J., Papoutsi, C., Teare, M. D., Keane, P. A., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/39742 · Cited by 163
Where is the human in human-centered AI? Insights from developer priorities and user experiences
Bingley, W., Curtis, C., Lockey, S., Bialkowski, A., Gillespie, N. A., Haslam, S. A., Ko, R. K. L., Steffens, N. K., Wiles, J., & Worthy, P. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107617 · Cited by 178
PromptMaker: Prompt-based Prototyping with Large Language Models
Jiang, E., Olson, K., Toh, E., Molina, A., Donsbach, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503564 · Cited by 157
Self-citationMetahuman systems = humans + machines that learn
Lyytinen, K., Nickerson, J. V., & King, J. L. (2020) · Journal of Information Technology · doi:10.1177/0268396220915917 · Cited by 187
How to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155
Wang, L., Zhang, Z., Wang, D., Cao, W., Zhou, X., Zhang, P., Liu, J., Fan, X., & Tian, F. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1187299 · Cited by 170
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Lai, V., Carton, S., Bhatnagar, R., Liao, Q. V., Zhang, Y., & Tan, C. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501999 · Cited by 145
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144
Ma, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143
Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517439 · Cited by 143
Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645199 · Cited by 142
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141
A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning
Kaluarachchi, T. I., Reis, A., & Nanayakkara, S. C. (2021) · Sensors · doi:10.3390/s21072514 · Cited by 159
Artificial Intelligence for Social Good: A SurveyPreprint
Shi, Z. R., Wang, C., & Fang, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01818 · Cited by 150
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137
Hou, Y. T.-Y., & Jung, M. F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479864 · Cited by 173
The explainability paradox: Challenges for xAI in digital pathology
Evans, T., Retzlaff, C. O., Geißler, C., Kargl, M., Plass, M., Müller, H., Kiehl, T.-R., Zerbe, N., & Holzinger, A. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.03.009 · Cited by 135
Steiner, D. F., Nagpal, K., Sayres, R., Foote, D., Wedin, B., Pearce, A., Cai, C. J., Winter, S. R., Symonds, M., Yatziv, L., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.23267 · Cited by 135
Self-citationZhang, R., Duan, W., Flathmann, C., McNeese, N., Freeman, G., & Williams, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610072 · Cited by 134
Gmeiner, F., Yang, H., Yao, L., Holstein, K., & Martelaro, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580999 · Cited by 130
Feuston, J. L., & Brubaker, J. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479856 · Cited by 128
Co-design of human-centered, explainable AI for clinical decision support
Panigutti, C., Beretta, A., Fadda, D., Giannotti, F., Pedreschi, D., Perotti, A., & Rinzivillo, S. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3587271 · Cited by 127
Beyond AI-powered context-aware services: the role of human–AI collaboration
Jiang, N., Liu, X., Liu, H., Lim, E. T. K., Tan, C., & Gu, J. (2022) · Industrial Management & Data Systems · doi:10.1108/imds-03-2022-0152 · Cited by 157
BreastScreening-AI: Evaluating medical intelligent agents for human-AI interactions
Calisto, F. M., Santiago, C., Nunes, N., & Nascimento, J. C. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102285 · Cited by 125
Rajashekar, N., Shin, Y. E., Pu, Y., Chung, S., You, K., Giuffrè, M., Chan, C., Saarinen, T., Hsiao, A., Sekhon, J. S., et al. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642024 · Cited by 156
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120
Dhanorkar, S., Wolf, C. T., Qian, K., Xu, A., Popa, L., & Li, Y. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462131 · Cited by 119
Suresh, H., Gomez, S. R., Nam, K. K., & Satyanarayan, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445088 · Cited by 117
Assertiveness-based Agent Communication for a Personalized Medicine on Medical Imaging Diagnosis
Calisto, F. M., Fernandes, J., Morais, M., Santiago, C., Abrantes, J., Nunes, N., & Nascimento, J. C. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580682 · Cited by 137
Nourani, M., King, J., & Ragan, E. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7469 · Cited by 116
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
Wang, Z., Huang, Y., Song, D., Ma, L., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642803 · Cited by 115
Wong, A. K. C., Lee, J. H. T., Zhao, Y., Lu, Q., Yang, S., & Hui, V. C. C. (2025) · JMIR Aging · doi:10.2196/66778 · Cited by 114
A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images
Messina, P., Pino, P., Parra, D., Soto, Á., Besa, C., Uribe, S. A., Andía, M. E., Cristián Tejos, N., Prieto, C., & Daniel Capurro, N. (2022) · ACM Computing Surveys · doi:10.1145/3522747 · Cited by 142
Bridging the “last mile” gap between AI implementation and operation: “data awareness” that matters
Cabitza, F., Campagner, A., & Balsano, C. (2020) · Annals of Translational Medicine · doi:10.21037/atm.2020.03.63 · Cited by 113
Effect of AI Explanations on Human Perceptions of Patient-Facing AI-Powered Healthcare Systems
Zhang, Z., Genc, Y., Wang, D., Ahsen, M. E., & Fan, X. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01743-6 · Cited by 113
AI in medical diagnosis: AI prediction & human judgment
Göndöcs, D., & Dörfler, V. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102769 · Cited by 130
How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions
Cheng, H.-F., Stapleton, L., Kawakami, A., Sivaraman, V., Cheng, Y., Qing, D., Perer, A., Holstein, K., Wu, Z. S., & Zhu, H. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501831 · Cited by 134
Too Late to be Creative? AI-Empowered Tools in Creative Processes
Hwang, A. H.-C. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503549 · Cited by 103
“Who goes first? Influences of human-AI workflow on decision making in clinical imaging,”
Fogliato, R., Chappidi, S., Lungren, M., Fisher, P., Wilson, D., Fitzke, M., Parkinson, M., Horvitz, E., Inkpen, K., & Nushi, B. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533193 · Cited by 103
Okolo, C. T., Kamath, S., Dell, N. L., & Vashistha, A. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445420 · Cited by 123
Closing the translation gap: AI applications in digital pathology
Steiner, D. F., Chen, P.-H. C., & Mermel, C. H. (2020) · Biochimica et Biophysica Acta (BBA) - Reviews on Cancer · doi:10.1016/j.bbcan.2020.188452 · Cited by 101
Self-citationHuman-AI Collaboration in a Cooperative Game Setting: Measuring Social Perception and Outcomes
Ashktorab, Z., Liao, Q., Dugan, C., Johnson, J. M., Pan, Q., Zhang, W., Kumaravel, S., & Campbell, M. S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415167 · Cited by 119
Human-AI Collaboration in Healthcare: A Review and Research Agenda
Lai, Y. F., Kankanhalli, A., & Ong, D. C. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.046 · Cited by 100
Zhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99
“For example, such a new paradigm would introduce additional burden to experts [13, 56, 72].”
BackgroundInfluentialShifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services
Holten Møller, N., Shklovski, I., & Hildebrandt, T. T. (2020) · Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society · doi:10.1145/3419249.3420149 · Cited by 97
Explainable AI, But Explainable to Whom? An Exploratory Case Study of xAI in Healthcare
Gerlings, J., Jensen, M. S., & Shollo, A. (2021) · Intelligent systems reference library · doi:10.1007/978-3-030-83620-7_7 · Cited by 123
Caldwell, S., Sweetser, P., O’Donnell, N., Knight, M. J., Aitchison, M., Gedeon, T., Johnson, D. M., Brereton, M. F., Gallagher, M. R., & Conroy, D. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3514257 · Cited by 109
Harnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Yang, Q., Hao, Y., Quan, K., Yang, S., Zhao, Y., Kuleshov, V., & Wang, F. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581393 · Cited by 113
Trustworthy and ethical AI-enabled cardiovascular care: a rapid review
Mooghali, M., Stroud, A. M., Yoo, D. W., Barry, B. A., Grimshaw, A. A., Ross, J. S., Zhu, X., & Miller, J. E. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02653-6 · Cited by 91
The state of human-centered NLP technology for fact-checking
Das, A., Liu, H., Kovatchev, V., & Lease, M. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103219 · Cited by 134
The Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641022 · Cited by 121
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 115
Zając, H. D., Li, D., Dai, X., Carlsen, J. F., Kensing, F., & Andersen, T. O. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582430 · Cited by 96
Improving Human-AI Collaboration With Descriptions of AI Behavior
Cabrera, Á. A., Perer, A., & Hong, J. I. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579612 · Cited by 87
“Human-AI collaboration is finding real-world use in tasks ranging from diagnosing prostate cancer [12] to screening calls to child welfare hotlines [19, 27].”
BackgroundHuman-in-the-loop machine learning: Reconceptualizing the role of the user in interactive approaches
Gómez–Carmona, O., Casado–Mansilla, D., López–de–Ipiña, D., & García-Zubía, J. (2024) · Internet of Things · doi:10.1016/j.iot.2023.101048 · Cited by 101
Enhancing trust in AI through industry self-governance.
Roski, J., Maier, E. J., Vigilante, K., Kane, E. A., & Matheny, M. E. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab065 · Cited by 83
Street-Level Algorithms and AI in Bureaucratic Decision-Making: A Caseworker Perspective
Ammitzbøll Flügge, A., Hildebrandt, T., & Møller, N. H. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449114 · Cited by 83
Suresh, H., Movva, R., Doğan, A. L., Bhargava, R., Cruxên, I. A., Cuba, A. M., Taurino, G., So, W., & D’Ignazio, C. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533132 · Cited by 83
Groh, M., Harris, C., Daneshjou, R., Badri, O., & Koochek, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555634 · Cited by 101
Extending the Nested Model for User-Centric XAI: A Design Study on GNN-based Drug Repurposing
Wang, Q., Huang, K., Chandak, P., Zitnik, M., & Gehlenborg, N. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209435 · Cited by 82
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82
“Reliable uncertainty estimates may allow decision-makers to determine how much confidence to place in a model’s competence [20, 99].”
BackgroundMethodYildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82
Healthcare AI Treatment Decision Support: Design Principles to Enhance Clinician Adoption and Trust
Burgess, E. R., Jankovic, I., Austin, M., Cai, N., Kapuścińska, A., Currie, S., Overhage, J. M., Poole, E. S., & Kaye, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581251 · Cited by 95
Human-AI Collaboration for UX Evaluation: Effects of Explanation and Synchronization
Fan, M., Yang, X., Yu, T., Liao, Q. V., & Zhao, J. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512943 · Cited by 81
Jiang, E., Toh, E., Molina, A., Olson, K., Kayacik, C., Donsbach, A., Cai, C. J., & Terry, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501870 · Cited by 79
Self-citationJin, W., Li, X., & Hamarneh, G. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21452 · Cited by 21
Metadata
- Authors
- Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M.
- Affiliations
- Google (United States) (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2019-11-07
- Volume
- 3
- Issue
- CSCW
- Pages
- 1-24
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2970837303
- Cited by
- 594
- References
- 79
- Citations by year
- 2026: 84 · 2025: 158 · 2024: 97 · 2023: 75 · 2022: 80 · 2021: 70 · 2020: 28 · 2019: 2
- FWCI
- 20.77
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-AI collaboration, physicians, prostate cancer diagnosis, pathologists, AI transparency, model limitations, second opinion, deep neural network inference, clinical decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.