"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making
Open accessThe DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Abstract
Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant. In a qualitative lab study, we interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) predictions for prostate cancer diagnosis, to learn the types of information that they desired about the AI assistant. Our findings reveal that, far beyond understanding the local, case-specific reasoning behind any model decision, clinicians desired upfront information about basic, global properties of the model, such as its known strengths and limitations, its subjective point-of-view, and its overall design objective--what it's designed to be optimized for. Participants compared these information needs to the collaborative mental models they develop of their medical colleagues when seeking a second opinion: the medical perspectives and standards that those colleagues embody, and the compatibility of those perspectives with their own diagnostic patterns. These findings broaden and enrich discussions surrounding AI transparency for collaborative decision-making, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The training included learning optimal ways of using the AI in their settings and pointing out possible errors [18].”
From Explainable recommendation: when design meets trust calibration (2021) · cites this paper as [18]
In the citing paper
“This finding relates to earlier work in Human-AI trust calibration [18], which showed that training clinicians to use the AI-based decision-making tool reduced users’ errors and increased the Human-AI collaborative decision-making performance.”
From Explainable recommendation: when design meets trust calibration (2021) · cites this paper as [18]
In the citing paper
“there has been increasing recent research efforts [8, 15, 22, 41, 58, 65] that highlight the importance of involving stakeholders to understand their practices and needs [15, 41, 65] and socio-environmental factors [8] for the design and evaluation of a system.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [15]
In the citing paper
“researchers have discussed the importance of an AI explanation to communicate an AI output to a user [5, 20, 29] and realize human-AI collaboration [48] for a decision-making task [15, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [15]
In the citing paper
“Third, this allows uncertainty to be expressed seamlessly within the natural language interactions of LLM-infused applications, rather than on the side or in onboarding materials [21, 74, 79] that users might overlook.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [21]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Braun, V., & Clarke, V. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-004 · Cited by 10,299
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Recent work on fairness and accountability has proposed the use of short documents accompanying trained ML models to disclose the intended use of models, details of their performance evaluation procedures, and the potential biases that they may embody [25, 51].”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439
Shared mental models in expert team decision making.
Cannon‐Bowers, J. A., Salas, E., & Converse, S. A. (1993) · Cited by 1,545
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467
Research works printed but not linked: 7
- Wouter Bulten , Hans Pinckaers , Hester van Boven , Robert Vink , Thomas de Bel , Bram van Ginneken , Jeroen van der Laak , Christina Hulsbergen-van de Kaa, and Geert Litjens . 2019 . Automated Gleason Grading of Prostate Biopsies using Deep Learning . arXiv preprint arXiv:1907.07980 (2019). Wouter Bulten, Hans Pinckaers, Hester van Boven, Robert Vink, Thomas de Bel, Bram van Ginneken, Jeroen van der Laak, Christina Hulsbergen-van de Kaa, and Geert Litjens. 2019. Automated Gleason Grading of Prostate Biopsies using Deep Learning. arXiv preprint arXiv:1907.07980 (2019). · arXiv
- Dumitru Erhan , Yoshua Bengio , Aaron Courville , and Pascal Vincent . 2009 . Visualizing higher-layer features of a deep network . University of Montreal 1341 , 3 (2009), 1 . Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. University of Montreal 1341, 3 (2009), 1.
- Timnit Gebru , Jamie Morgenstern , Briana Vecchione , Jennifer Wortman Vaughan , Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018 . Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). · arXiv
- Ken Hinckley ( Ed .). 2018 . ACM Trans. Comput.-Hum. Interact. 25 , 1 (2018). Ken Hinckley (Ed.). 2018. ACM Trans. Comput.-Hum. Interact. 25, 1 (2018).
- 10.5555/3305381.3305576 · doi:10.5555/3305381.3305576
- Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). · arXiv
- Ming Yin , Jennifer Wortman Vaughan, and Hanna Wallach . 2019 . Understanding the Effect of Accuracy on Trust in Machine Learning Models . (2019). Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the Effect of Accuracy on Trust in Machine Learning Models. (2019).
Cited by
Items 601–666 of 666
On the purpose of artificial intelligence in critical care medicine
Drudi, C., Matta, S. C., Lee, H., O’Donoghue, S. C., D’Couto, H. T., Hamza, A., Gutiérrez, C. A., Nakasi, R., Byers, J., Tumukunde, M., et al. (2026) · Journal of Translational Critical Care Medicine · doi:10.1097/jtccm-d-25-00021
Weng, N., Duan, W., Guynup, C., Flathmann, C., Basappa, R., & McNeese, N. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6511199
A software architecture for verifiable and explainable classification
Barbosa, R., Rinzivillo, S., Robin, J., Beretta, A., & Madeira, H. (2026) · Machine Learning · doi:10.1007/s10994-026-07006-0
Vereschak, O. (2022) · theses.fr (ABES) · doi:10.70675/7ecd53bfza7ccz4edfz8f1azdb8052ba1233 · Cited by 2
Understanding, Mitigating, and Leveraging Cognitive Biases to Calibrate Trust in Evolving AI Systems
Pareek, S., Boonprakong, N., Kollerup, N. K., Chen, S., Hosio, S. J., Yatani, K., Lee, Y., Gadiraju, U., van Berkel, N., & Gonçalves, J. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3778722 · Cited by 1
Jung, Y., Vo, T. H. A., Moon, H. S., Choi, J. Y., Oh, H., Lee, U., Kim, E. J., Lee, T. Y., & Lee, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790970
Friend, Foe, or Bot? Exploring Intergroup Dynamics in Hybrid Human-Bot Teams
Zhunis, A., Pan, Z., Zhang, Y., & Ma, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791939
Dental Provider Experiences with AI Radiograph Annotation: A Qualitative Case Study
Schroeder, K. L., Slashcheva, L. D., Heaton, L. J., Cheung, H. J., Prosa, B., Ferrian, N., Grantz, J., Jacobi, D., Helgeson, M. J., & Tranby, E. P. (2026) · JDR Clinical & Translational Research · doi:10.1177/23800844261429908
Thavanesan, N. (2026) · ePrints Soton (University of Southampton) · doi:10.5258/soton/pg/t246
“I Cannot Teach it Anything”: Human-in-the-Loop AI After Deployment in Clinical Documentation
Vase, S., & Andersen, T. O. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3815172 · Cited by 1
Deng, W. H., Yan, M., Kim, S. S. Y., Jha, A., Wilcox, L. G., Holstein, K., Eslami, M., & Gatys, L. A. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830632
Self-citationOrganizational Practices and Socio-Technical Design of Human-Centered AI
Herrmann, T. (2026) · Springer eBooks · doi:10.1007/978-981-95-3658-0_66
Between automation and authenticity: The hidden emotional costs of generative AI in service work
Zha, B., Zhang, Y., & Jun, Z. (2026) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2026.104878
-, V. A., Himanshu, H., Rani, N., Semwal, A., Singh, S., & Rajput, P. K. (2026) · International Journal of Drug Delivery Technology · doi:10.25258/ijddt.16.49s.40
Stepping Into the Black Box: Opening Up LLMs to Public Exploration Through Discursive Design
Immel, S., Rajani, N., Verweij, R., Li, J., Nissen, B., Soares, L. L., & Taylor, A. S. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812976 · Cited by 1
Sumiari, K. N., Parnata, I. K., Pramitari, I. G. A. A., & Putrayasa, I. M. A. (2026) · Greenation International Journal of Economics and Accounting · doi:10.38035/gijea.v4i2.901
Haas, S., Hammer, J., Hettler, F. M., Rodrigues, P. K., Teuteberg, F., & Thomas, O. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1850272
Krishnan, M., & Karthik, R. (2026) · International Journal of Information Technology Research Studies (IJITRS) · doi:10.63090/ijitrs/3139.3209.0027
VILLAVICENCIO, M., Pan, S., & Wang, Q. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812329
Human–AI teams for decision-making in banking and finance: A systematic scoping review
Aksu, F., Morandini, S., & Pietrantoni, L. (2026) · Human Systems Management · doi:10.1177/01672533261461402
Muñoz, Á. A., Yamamoto, Y., & Sandström, K. (2026) · Proceedings of the Design Society · doi:10.1017/pds.2026.10373
Human-AI Interaction in Healthcare - A Manifesto for Improved Usability Evaluation
Oliveira, M., Cruz, C. F., & Matela, N. (2026) · ACM Interactive Health Conference · doi:10.1145/3786579.3804937
Bridging AI and Clinical Reasoning: Abductive Explanations for Alignment on Critical Symptoms
Sonna, B., & Grastien, A. (2026) · Australasian Computer Science Week Multiconference (ACSW) · doi:10.1145/3793811.3793822
Keleher, M., Morgan, A., Baig, K., Tsang, A., & Chiasson, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29723-5_26
Thinking HATs: A Mixed-Methods Study on Team Cognition and Dynamics in Human-Agent Teams
Yousefi, M., Shahi, A., Topliss, J., Sharifi, M., Romera, A. J. J., Hoermann, S., & Piumsomboon, T. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2694626
Moores, A., Dabbous, M., Acampora, M., Sparrow, B., & Lorencatto, F. (2026) · doi:10.2196/preprints.106471
Adel, A., Al‐Rawi, M., Shillabeer, A., Shearman, P., Yalda, R., Jan, T., Al-Moisheer, A. S., & Moni, M. A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11646-y
Schäfer, U., Saakyan, W., Norden, M., Kuruc, F., Sörries, P., Dziobek, I., Müller-Birn, C., & Drimalla, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.22005
Rahin, S. A., Li, W., & McKinney, B. A. (2026) · International Conference on Bioinformatics and Computational Biology (ICBCB) · doi:10.1109/icbcb69424.2026.11621303
Artificial wisdom in gastrointestinal diagnostics: A new frontier in translational reasoning
Issa, I. A., & Issa, T. (2026) · Artificial Intelligence in Gastroenterology · doi:10.35712/aig.v7.i2.118884
Otify, A., & Nabney, I. T. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1878569
The Role of AI in Shaping Competitiveness in Finnish companies
Nousiainen, M. (2026) · Osuva (University of Vaasa)
Kumari, J., Yadav, P., & Bhargava, C. P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8988-2.ch001
AI-Driven Decision Support Systems for Infrastructure Management: A Systematic Review
Tusar, T. (2026) · Pacific Journal of Advanced Engineering Innovations · doi:10.70818/pjaei.v03i01.0235 · Cited by 1
Schöttler, R., & Kuhlmann, A. (2026) · RENATE · doi:10.34657/38504
AI-Supported Adaptive Interfaces in Radio Test Systems
Gu, Q. (2026) · Aaltodoc (Aalto University)
Chanda, M., Mazumder, O., Rahman, T., Deb, N., FERDOUS, J., & Kushal, K. S. (2026) · Expert Systems · doi:10.1111/exsy.70402
Singh, D. B., Dangi, Y., & Pokharel, B. P. (2026) · Health Science Reports · doi:10.1002/hsr2.73117
AMINA: The Inclusive and Accountable AI for Marginalized Immigrant Nonprofit AssistancePreprint
Mokhberi, M., Das, D., & Ahmed, S. I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.30084
Li, Z. "., Freeman, B., Raman, A., Rajkumar, K. A., Fang, X., Srivastava, R., Feiner, S., & Thakoor, K. A. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830722
Darla, M., Miltz, D., Chandnani, K., Purkayastha, S., Diehl, J. W., & Bhavani, S. V. (2026) · JMIR Human Factors · doi:10.2196/95472
Fiebrink, R., Gillies, M. F. P., & Ramos, G. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108872225.010
Bosri, R., Lorimer, A. H., Hossain, A., Duddu, V., & Kacsmar, B. (2026) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.48550/arxiv.2609.12288
What Makes a Great Co-Worker in an AI-Native Workplace?Preprint
Choudhuri, R., Meijer, M., Lee, S. Y.-T., Lee, C., Mannion, C., Jahn, P., Sarma, A., Bird, C., & Ferng, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.13786 · Cited by 1
xAARA: Explainable and Augmented Automated Rehabilitation Assessment
Ahmed, T., Rikakis, T., Kelliher, A., Khan, M. S. S., & Guo, Z. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3848511
Anik, A. I. (2026) · Mspace (University of Manitoba)
Ala-Luopa, S., Jarrahi, M. H., & Olsson, T. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816898
Balayn, A., Rancourt, F., Casati, F., & Gadiraju, U. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816909
Rethinking Objectivity in Clinical AI: A Qualitative Study of Concussion Evaluators
Zaidi, A. M., Saw, J. J.-W., Fu, L., Arneson, K., Kim, I., Cross, A., & Karahalios, K. G. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816992
Ontika, N. N., Saßmannshausen, S. M., Rohde, M., & de Carvalho, A. F. P. (2026) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3829807.3829885
Trust in LLM-Supported Engineering Work: The Role of Downstream DependencyPreprint
Herath, M., & Palmquist, A. (2026) · Research Square · doi:10.21203/rs.3.rs-11038705/v1
Sethi, S., & Singh, S. (2026) · ACM Transactions on Social Computing · doi:10.1145/3849483
Entanglement. Trasformazioni nelle pratiche del Design.
Bianco, A. (2025) · Archivio Istituzionale della Ricerca - Università degli Studi della Campania "Luigi Vanvitelli"
"I Am the Gatekeeper": Clinician Visions of AI Collaboration in Exposure Therapy
Moradzadeh, S., Sherrill, A. M., Wiese, C. W., Arriaga, R. I., & Abdullah, S. (2026) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3785651.3831522
The Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000166086
Beyond Detection: A Critical Perspective on User Agency in Digital Mental Health
Chen, S., & Zhao, Z. (2026) · Graphics Interface · doi:10.1145/3831423.3831450
Designing and Evaluating a Clinical Decision Support System to Assess Mild Cognitive Impairment
Chen, Y., Rekkas, P., Mehdiyev, R., Verbert, K., & Abeele, V. V. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3859240
Kendjaeva, D. (2026) · AGROBUSINESS SCIENCE AND TECHNOLOGY · doi:10.68372/sdjhp873
Por que projetos de IA agêntica falham nas empresas brasileiras?
Coelho, L. B., Boeira, J. P. D., Bonin, L., & de Carvalho, A. C. P. L. F. (2026) · GV executivo \b (Impresso)/GV Executivo · doi:10.12660/gvexec.v25n4.2026.97666
Detjen, H., Pfrang, D., Westphal, M., Finster, M., Corr, D., & Wenzel, M. A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.13386
Kui, Z., Onishchenko, D., Manuel, K., Gorodilov, M., Farseev, A., & Makarov, I. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3839014
Jesso, S. T., Kelliher, A., Sanghavi, H., Martin, T., & Parker, S. H. (2022) · doi:10.3389/fpsyg.2022.830345.s001
Metadata
- Authors
- Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M.
- Affiliations
- Google (United States) (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2019-11-07
- Volume
- 3
- Issue
- CSCW
- Pages
- 1-24
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2970837303
- Cited by
- 594
- References
- 79
- Citations by year
- 2026: 84 · 2025: 158 · 2024: 97 · 2023: 75 · 2022: 80 · 2021: 70 · 2020: 28 · 2019: 2
- FWCI
- 20.77
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-AI collaboration, physicians, prostate cancer diagnosis, pathologists, AI transparency, model limitations, second opinion, deep neural network inference, clinical decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.