CoChat

Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetrics

Open access
Zhu, Yinghao, Sui, Dehao, Wang, Zixiang, Hu, Xuning, Gu, Lei, Qi, Yifan, Wu, Tianchen, Wang, Ling, Wei, Yuan, Tang, Wen, Cui, Zhihan, Wang, Yasha, Yu, Lequan, Harrison, Ewen M, Gao, Junyi, Ma, Liantao · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems

The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.

Abstract

Clinician skepticism toward opaque AI hinders adoption in high-stakes healthcare. We present AICare, an interactive and interpretable AI copilot for collaborative clinical decision-making. By analyzing longitudinal electronic health records, AICare grounds dynamic risk predictions in scrutable visualizations and LLM-driven diagnostic recommendations. Through a within-subjects counterbalanced study with 16 clinicians across nephrology and obstetrics, we comprehensively evaluated AICare using objective measures (task completion time and error rate), subjective assessments (NASA-TLX, SUS, and confidence ratings), and semi-structured interviews. Our findings indicate AICare’s reduced cognitive workload. Beyond performance metrics, qualitative analysis reveals that trust is actively constructed through verification, with interaction strategies diverging by expertise: junior clinicians used the system as cognitive scaffolding to structure their analysis, while experts engaged in adversarial verification to challenge the AI’s logic. This work offers design implications for creating AI systems that function as transparent partners, accommodating diverse reasoning styles to augment rather than replace clinical judgment.

Overview

Integrity signals

Registry

Crossref · 10.1145/3772318.3791458

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 74 references

Research works printed but not linked: 5

  1. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning. PMLR, 1321–1330.
  2. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017).
  3. John D Manning Beatriz E Marciano and James J Cimino. 2013. Visualizing the data–using lifelines2 to gain insights from data drawn from a clinical data repository. AMIA summits on translational science proceedings 2013 (2013) 168.
  4. National Academy of Medicine; The Learning Health System Series. 2023. Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril. National Academies Press (US), Washington (DC).
  5. Sana Tonekaboni, Shalmali Joshi, Melissa D McCradden, and Anna Goldenberg. 2019. What clinicians want: contextualizing explainable machine learning for clinical end use. In Machine learning for healthcare conference. PMLR, 359–380.

Cited by

See all 4

Metadata

Authors
Zhu, Y., Sui, D., Wang, Z., Hu, X., Gu, L., Qi, Y., Wu, T., Wang, L., Wei, Y., Tang, W., Cui, Z., Wang, Y., Yu, L., Harrison, E. M., Gao, J., & Ma, L.
Affiliations
  • Peking University (CN)
  • Hong Kong University of Science and Technology (HK)
  • Peking University Third Hospital (CN)
  • Xuzhou Medical College (CN)
  • University of Hong Kong (HK)
  • University of Edinburgh (GB)
  • Health Data Research UK (GB)
Venue
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2026-04-13
Pages
1-26
OpenAlex
W7154143872
Cited by
3
References
69
Citations by year
2026: 3
FWCI
25.86
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
Funders
  • National Natural Science Foundation of China (62402017, 82470774, 82561128248, 82504426)
  • Research Grants Council of Hong Kong (27206123, 17200125, C5055-24G, T45-401/22-N)
  • Hong Kong Innovation and Technology Fund (GHP/318/22GD)
  • Beijing Natural Science Foundation (QY25224, L244063, L244025)
  • Peking University (Clinical Medicine Plus X Pilot Project) (2024YXXLHGG007)
  • Peking University (\"TengYun\" Clinical Research Program) (TY2025015)
  • Beijing Traditional Chinese Medicine Science and Technology Development Fund (BJZYZD-2025-13)
  • Beijing Municipal Health Commission Research Ward Excellence Clinical Research Program (BRWEP2024W032150205)
  • Xuzhou Scientific Technological Projects (KC23143)
  • Health Data Research UK-The Alan Turing Institute Wellcome PhD Programme in Health Data Science (218529/Z/19/Z)
  • Baidu Scholarship (/)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
AI copilots, explainable AI, clinical decision-making, clinician trust in AI, mental workload, nephrology, obstetrics, cognitive scaffolding, adversarial verification, AI adoption in healthcare, NASA-TLX, System Usability Scale

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetrics. https://app.cochat.ai/papers/10.1145/3772318.3791458
The paper
Zhu, Y., Sui, D., Wang, Z., Hu, X., Gu, L., Qi, Y., Wu, T., Wang, L., Wei, Y., Tang, W., Cui, Z., Wang, Y., Yu, L., Harrison, E. M., Gao, J., & Ma, L. (2026). Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetrics. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3772318.3791458

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.