Who should do as AI say? Only non-task expert physicians benefit from correct explainable AI advice
Open accessAbstract
Artificial intelligence (AI)-generated clinical advice is becoming more prevalent in healthcare. However, the impact of AI-generated advice on physicians’ decision-making is underexplored. In this study, physicians received X-rays with correct diagnostic advice and were asked to make a diagnosis, rate the advice’s quality, and judge their own confidence. We manipulated whether the advice came with or without a visual annotation on the X-rays and whether it was labeled as coming from an AI or a human radiologist. Overall, receiving annotated advice from an AI resulted in the highest diagnostic accuracy. Physicians rated the quality of AI advice higher than human advice. Neither manipulation had strong effects on participants’ confidence. Importantly, the results varied among task experts and non-task experts, with only the latter considerably benefiting from correct explainable AI advice. These findings raise important considerations for the deployment of diagnostic advice in healthcare.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
As clinicians gain experience, they develop a deeper understanding of clinical contexts and decision-making, which may influence trust in and reliance on clinical decision support systems.
“As clinicians gain experience, they develop a deeper understanding of clinical contexts and decision-making processes, which may influence both trust in and reliance on CDSSs [ 8 , 15 - 17 ].”
From Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists (2025) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · see citing works
Organizational behavior and human decision processes
Armstrong, J. S. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90125-2 · Cited by 1,766 · see citing works
Rajpurkar, P., Irvin, J., Ball, R. L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C. P., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002686 · Cited by 1,449 · see citing works
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · see citing works
Self-citationClinical Gastroenterology and Hepatology
Ghosh, S. (2006) · Gut · doi:10.1136/gut.2006.097675 · Cited by 512 · see citing works
Lee, H., Yune, S., Mansouri, M., Kim, M., Tajmir, S. H., Guerrier, C. E., Ebert, S. A., Pomerantz, S. R., Romero, J. M., Kamalian, S., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0324-9 · Cited by 441 · see citing works
How does artificial intelligence in radiology improve efficiency and health outcomes?
van Leeuwen, K. G., de Rooij, M., Schalekamp, S., van Ginneken, B., & Rutten, M. J. C. M. (2022) · Pediatric Radiology · doi:10.1007/s00247-021-05114-8 · Cited by 236 · see citing works
Interpretation of Plain Chest Roentgenogram
Raoof, S., Feigin, D. S., Sung, A. W., Raoof, S., Irugulpati, L., & Rosenow, E. C. (2012) · CHEST Journal · doi:10.1378/chest.10-1302 · Cited by 218 · see citing works
Journal of the American College of Radiology
Bell, D. A., & Campos, A. (2020) · Radiopaedia.org · doi:10.53347/rid-76888 · Cited by 162 · see citing works
Comparing comprehension measured by multiple-choice and open-ended questions.
Ozuru, Y., Briner, S., Kurby, C. A., & McNamara, D. S. (2013) · Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale · doi:10.1037/a0032918 · Cited by 134 · see citing works
Added value of double reading in diagnostic radiology,a systematic review
Geijer, H., & Geijer, M. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0599-0 · Cited by 117 · see citing works
AI outperforms radiologists in mammographic screening
Killock, D. (2020) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-020-0329-7 · Cited by 86 · see citing works
Professional commitment among US physician executives in managed care
Hoff, T. J. (2000) · Social Science & Medicine · doi:10.1016/s0277-9536(99)00410-4 · Cited by 57 · see citing works
Johnson, A. E. W., Pollard, T., Mark, R., Berkowitz, S., & Horng, S. (2019) · physionet.org · doi:10.13026/c2jt1q · Cited by 42 · see citing works
Mathematics Strategy Use in Solving Test Items in Varied Formats
Bonner, S. M. (2013) · The Journal of Experimental Education · doi:10.1080/00220973.2012.727886 · Cited by 20 · see citing works
Common Errors and Pitfalls in Interpretation of the Adult Chest Radiograph
Kanne, J. P., Thoongsuwan, N., & Stern, E. J. (2005) · Clinical Pulmonary Medicine · doi:10.1097/01.cpm.0000156704.33941.e2 · Cited by 17 · see citing works
Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions
Vodrahalli, K., Daneshjou, R., Gerstenberg, T., & Zou, J. (2021) · arXiv · doi:10.48550/arxiv.2107.07015 · Cited by 16 · see citing works
Curbside Consultations: The Good, the Bad, and the Ugly
Lin, M., Pappas, S. C., Sellin, J., & El-Serag, H. B. (2016) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2015.09.026 · Cited by 13 · see citing works
Where Are They All Hiding? Common Blind Spots on Chest Radiography
Humphrey, K. L., Wu, C. C., Gilman, M. D., El-Sherief, A. H., Shepard, J.-A. O., & Abbott, G. F. (2011) · Contemporary Diagnostic Radiology · doi:10.1097/01.cdr.0000406727.37058.bc · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- M. Jacobs, M. F. Pradier, T. H. McCoy, R. H. Perlis, F. Doshi-Velez, K. Z. Gajos, How machine-learning recommendations in uence clinician treatment selections: the example of antidepressant selection. Transl Psychiatry. 11, 108 (2021).
- D. Li, L. M. Pehrson, C. A. Lauridsen, L. Tøttrup, M. Fraccaro, D. Elliott, H. D. Zając, S. Darkner, J. F. Carlsen, M. B. Nielsen, The added effect of arti cial intelligence on physicians' performance in detecting thoracic pathologies on CT and chest X-ray: A systematic review. Diagnostics. 11, 2206 (2021).
- A. Bussone, S. Stumpf, D. O'Sullivan, (IEEE, 2015), pp. 160-169.
- A. E. W. Johnson, T. J. Pollard, S. J. Berkowitz, N. R. Greenbaum, M. P. Lungren, C. Deng, R. G. Mark, S. Horng, MIMIC-CXR, a de-identi ed publicly available database of chest radiographs with free-text reports. Sci Data. 6, 317 (2019).
- D. R. Hekman, H. K. Steensma, G. A. Bigley, J. F. Hereford, Effects of organizational and professional identi cation on the relationship between administrators' social in uence and professional employees' adoption of new work behavior. Journal of Applied Psychology. 94, 1325-1335 (2009).
Cited by
Sorted by citations
Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
Agarwal, N., Moehring, A., Rajpurkar, P., & Salz, T. (2023) · doi:10.3386/w31422 · Cited by 161 · see citing works
Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49 · see citing works
Küper, A., Lodde, G. C., Livingstone, E., Schadendorf, D., & Krämer, N. (2025) · Journal of Medical Internet Research · doi:10.2196/58660 · Cited by 31 · see citing works
- Authors
- Gaube, S., Suresh, H., Raue, M., Lermer, E., Koch, T., Hudecek, M., Ackery, A. D., Grover, S. C., Coughlin, J. F., Frey, D., Kitamura, F., Ghassemi, M., & Colak, E.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Massachusetts Institute of Technology (US)
- Ansbach University of Applied Sciences (DE)
- University of Regensburg (DE)
- St. Michael's Hospital (CA)
- Universidade Federal de São Paulo (BR)
- ORCID
- Venue
- Research Square
- Published
- 2022-06-01
- OpenAlex
- W4281783041
- Cited by
- 3
- References
- 20
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 1
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- Research Square (acceptedVersion, cc-by)
- Funders
- Volkswagen Foundation (98525)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- explainable AI, physician decision-making, diagnostic accuracy, radiology, X-ray interpretation, visual annotation, human-AI collaboration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
