CoChat

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

Open access
Cai, Carrie J., Winter, Samantha, Steiner, David, Wilcox, Lauren, Terry, Michael · 2019 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.

Abstract

Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant. In a qualitative lab study, we interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) predictions for prostate cancer diagnosis, to learn the types of information that they desired about the AI assistant. Our findings reveal that, far beyond understanding the local, case-specific reasoning behind any model decision, clinicians desired upfront information about basic, global properties of the model, such as its known strengths and limitations, its subjective point-of-view, and its overall design objective--what it's designed to be optimized for. Participants compared these information needs to the collaborative mental models they develop of their medical colleagues when seeking a second opinion: the medical perspectives and standards that those colleagues embody, and the compatibility of those perspectives with their own diagnostic patterns. These findings broaden and enrich discussions surrounding AI transparency for collaborative decision-making, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice.

Overview

Integrity signals

Registry

Crossref · 10.1145/3359206

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

4 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 81 references

Research works printed but not linked: 7

  1. Wouter Bulten , Hans Pinckaers , Hester van Boven , Robert Vink , Thomas de Bel , Bram van Ginneken , Jeroen van der Laak , Christina Hulsbergen-van de Kaa, and Geert Litjens . 2019 . Automated Gleason Grading of Prostate Biopsies using Deep Learning . arXiv preprint arXiv:1907.07980 (2019). Wouter Bulten, Hans Pinckaers, Hester van Boven, Robert Vink, Thomas de Bel, Bram van Ginneken, Jeroen van der Laak, Christina Hulsbergen-van de Kaa, and Geert Litjens. 2019. Automated Gleason Grading of Prostate Biopsies using Deep Learning. arXiv preprint arXiv:1907.07980 (2019). · arXiv
  2. Dumitru Erhan , Yoshua Bengio , Aaron Courville , and Pascal Vincent . 2009 . Visualizing higher-layer features of a deep network . University of Montreal 1341 , 3 (2009), 1 . Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. University of Montreal 1341, 3 (2009), 1.
  3. Timnit Gebru , Jamie Morgenstern , Briana Vecchione , Jennifer Wortman Vaughan , Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018 . Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). · arXiv
  4. Ken Hinckley ( Ed .). 2018 . ACM Trans. Comput.-Hum. Interact. 25 , 1 (2018). Ken Hinckley (Ed.). 2018. ACM Trans. Comput.-Hum. Interact. 25, 1 (2018).
  5. 10.5555/3305381.3305576 · doi:10.5555/3305381.3305576
  6. Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). · arXiv
  7. Ming Yin , Jennifer Wortman Vaughan, and Hanna Wallach . 2019 . Understanding the Effect of Accuracy on Trust in Machine Learning Models . (2019). Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the Effect of Accuracy on Trust in Machine Learning Models. (2019).

Cited by

Items 401–500 of 666

Metadata

Authors
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M.
Affiliations
Google (United States) (US)
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2019-11-07
Volume
3
Issue
CSCW
Pages
1-24
ISSN
2573-0142 (ISSN-L)
OpenAlex
W2970837303
Cited by
594
References
79
Citations by year
2026: 84 · 2025: 158 · 2024: 97 · 2023: 75 · 2022: 80 · 2021: 70 · 2020: 28 · 2019: 2
FWCI
20.77
Type
journal article
Language
en
Open access
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Keywords
human-AI collaboration, physicians, prostate cancer diagnosis, pathologists, AI transparency, model limitations, second opinion, deep neural network inference, clinical decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.