The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Open accessAbstract
Machine learning (ML) is increasingly being used in image retrieval systems for medical decision making. One application of ML is to retrieve visually similar medical images from past patients (e.g. tissue from biopsies) to reference when making a medical decision with a new patient. However, no algorithm can perfectly capture an expert's ideal notion of similarity for every case: an image that is algorithmically determined to be similar may not be medically relevant to a doctor's specific diagnostic needs. In this paper, we identified the needs of pathologists when searching for similar images retrieved using a deep learning algorithm, and developed tools that empower users to cope with the search algorithm on-the-fly, communicating what types of similarity are most important at different moments in time. In two evaluations with pathologists, we found that these tools increased the diagnostic utility of images found and increased user trust in the algorithm. The tools were preferred over a traditional interface, without a loss in diagnostic accuracy. We also observed that users adopted new strategies when using refinement tools, re-purposing them to test and understand the underlying algorithm and to disambiguate ML errors from their own errors. Taken together, these findings inform future human-ML collaborative systems for expert decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In particular, humans can review AI explanations to understand how the AI models generate outputs [61] and identify whether an AI output is right or not [14, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In order to better understand factors impacting the adoption of CDS, previous studies of deployed systems have used post-hoc surveys or interviews to gauge provider impressions 28,44,53 ; however, provider impressions can differ from their actual use 4,18,27 .”
From Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing (2022) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, ease of use and integration into the workflow have been noted as important factors influencing adoption 26,28,47,48 .”
From Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing (2022) · cites this paper as [28]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Many researchers have discussed that the explainability [61] of a system is critical for humanAI collaborative decision-making [14, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [14]
The cited source says what the claim attributes to it.
In the citing paper
“These mechanisms include providing interpretable explanations for AI advice [21, 72, 123], using cognitive forcing functions[17, 47, 99], and incorporating feedback loops to enhance the interaction between humans and AI systems [10, 11, 139].”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [21]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,050
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
Query by image and video content: the QBIC system
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petković, D., et al. (1995) · Computer · doi:10.1109/2.410146 · Cited by 3,097
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,080
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · Cited by 2,913
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial NetworksPreprint
Zhu, J.-Y., Park, T., Isola, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10593 · Cited by 2,278
QBIC project: querying images by content, using color, texture, and shape
Niblack, C. W., Barber, R., Equitz, W., Flickner, M., Glasman, E. H., Petković, D., Yanker, P., Faloutsos, C., & Taubin, G. (1993) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.143648 · Cited by 1,747
Müller, H., Michoux, N., Bandon, D., & Geissbühler, A. (2004) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2003.11.024 · Cited by 1,490
Network Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389
A Contemporary Prostate Cancer Grading System: A Validated Alternative to the Gleason Score
Epstein, J. I., Zeléfsky, M. J., Sjoberg, D. D., Nelson, J. B., Egevad, L., Magi‐Galluzzi, C., Vickers, A. J., Parwani, A. V., Reuter, V. E., Fine, S. W., et al. (2015) · European Urology · doi:10.1016/j.eururo.2015.06.046 · Cited by 1,375
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240
Self-citationPower to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Deep Learning for Content-Based Image Retrieval
Wan, J., Wang, D., Hoi, S. C. H., Wu, P., Zhu, J., Zhang, Y., & Li, J. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654948 · Cited by 832
A roadmap for national action on clinical decision support
Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2334 · Cited by 707
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615
CNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607
Clinical Decision-Support Systems
Musen, M. A., Middleton, B., & Greenes, R. A. (2013) · Springer eBooks · doi:10.1007/978-1-4471-4474-8_22 · Cited by 588
Understanding intermediate layers using linear classifier probes
Alain, G., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1610.01644 · Cited by 581
From generic to specific deep representations for visual recognition
Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., & Carlsson, S. (2015) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301270 · Cited by 404
Content-based image retrieval in radiology: current status and future directions
Akgül, C. B., Rubin, D. L., Napel, S., Beaulieu, C. F., Greenspan, H., & Acar, B. (2011) · Journal of Digital Imaging · doi:10.1007/s10278-010-9290-9 · Cited by 388
First I "like" it, then I hide it: Folk Theories of Social Feeds
Eslami, M., Karahalios, K., Sandvig, C., Vaccaro, K., Rickman, A., Hamilton, K., & Kirlik, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858494 · Cited by 375
Reasons For Physicians Not Adopting Clinical Decision Support Systems: Critical Analysis
Khairat, S., Marc, D. T., Crosby, W., & Sanousi, A. A. (2018) · JMIR Medical Informatics · doi:10.2196/medinform.8912 · Cited by 346
Clinical Decision Support Systems
Helmons, P. J. (2009) · CRC Press eBooks · doi:10.1201/9781420071764-c15 · Cited by 295
Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision
Sittig, D. F., Wright, A. T., & Middleton, B. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s034 · Cited by 287
AUTOMATED GRADING OF PROSTATE CANCER USING ARCHITECTURAL AND TEXTURAL IMAGE FEATURES
Doyle, S. T., Hwang, M. E., Shah, K., Madabhushi, A., Feldman, M., & Tomaszeweski, J. E. (2007) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2007.357094 · Cited by 235
Machine Teaching: A New Paradigm for Building Machine Learning SystemsPreprint
Simard, P. Y., Amershi, S., Chickering, D. M., Pelton, A. E., Ghorashi, S., Meek, C., Ramos, G., Suh, J., Verwey, J., Wang, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.06742 · Cited by 220
The effects of example-based explanations in a machine learning interface
Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220
Self-citationFogarty, J. A., Tan, D. S., Kapoor, A., & Winder, S. A. J. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357061 · Cited by 186
Amershi, S., Fogarty, J. A., & Weld, D. S. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207680 · Cited by 145
Storage and Retrieval for Image and Video Databases
Oakley, J., Davis, D., & Shann, R. (1993) · Research Explorer (The University of Manchester) · Cited by 143
MOSQUERA-LOPEZ, C. M., Agaian, S. С., Hoyos, A. V., & Thompson, I. M. (2014) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2014.2340401 · Cited by 116
Kohli, A., & Jha, S. (2018) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2017.12.029 · Cited by 114
Yang, Q., Zimmerman, J., Steinfeld, A., Carey, L., & Antaki, J. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858373 · Cited by 105
Structured labeling for facilitating concept evolution in machine learning
Kulesza, T., Amershi, S., Caruana, R., Fisher, D., & Charles, D. X. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557238 · Cited by 104
Latent Constraints: Learning to Generate Conditionally from Unconditional Generative ModelsPreprint
Engel, J., Hoffman, M. D., & Roberts, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05772 · Cited by 93
The Illusion of Control: Placebo Effects of Control Settings
Vaccaro, K., Huang, D., Eslami, M., Sandvig, C., Hamilton, K., & Karahalios, K. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173590 · Cited by 65
Intelligent Image Retrieval Techniques: A Survey
Yasmin, M., Mohsin, S., & Sharif, M. (2014) · Journal of Applied Research and Technology · doi:10.1016/s1665-6423(14)71609-8 · Cited by 56
Toward content-based image retrieval with deep convolutional neural networks
Sklan, J. E. S., Plassard, A. J., Fabbri, D., & Landman, B. A. (2015) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2081551 · Cited by 56
Content-based image retrieval from large medical databases
Kak, A. C., & Pavlopoulou, C. (2003) · International Symposium on 3D Data Processing, Visualization and Transmission · doi:10.1109/tdpvt.2002.1024053 · Cited by 42
Mehta, N., Alomari, R. S., & Chaudhary, V. (2009) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2009.5334811 · Cited by 37
(2015) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 34
Self-citationZhang, H., & Su, Z. (2002) · Springer eBooks · doi:10.1007/978-0-387-35592-4_3 · Cited by 24
Visual and Multimedia Information Management
Xiaofang, Z., & Pu, P. (2002) · Springer eBooks · doi:10.1007/978-0-387-35592-4 · Cited by 6
Research works printed but not linked: 2
- Matthew Chalmers and Ian MacColl. 2003. Seamful and seamless design in ubiquitous computing. In Workshop at the crossroads: The interaction of HCI and systems issues in UbiComp, Vol. 8.
- 10.5555/647251.720114 · DOI
Cited by
Sorted by citations
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Self-citationA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
Six Human-Centered Artificial Intelligence Grand Challenges
Ozmen Garibay, O., Winslow, B., Andolina, S., Antona, M., Bodenschatz, A., Coursaris, C., Falco, G., Fiore, S. M., Garibay, I., Grieman, K., et al. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2153320 · Cited by 501
Metadata
- Authors
- Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., & Terry, M.
- Affiliations
- Google (United States) (US)
- Google Brain, Mountain View, CA, USA
- ORCID
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-14
- OpenAlex
- W2918341242
- Cited by
- 405
- References
- 50
- Citations by year
- 2026: 27 · 2025: 51 · 2024: 80 · 2023: 73 · 2022: 73 · 2021: 54 · 2020: 38 · 2019: 9
- FWCI
- 37.29
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- medical image retrieval, pathologists, human-machine collaboration, trust in machine learning, diagnostic accuracy, human-centered design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
