Cited by 188Artificial Intelligence in Healthcare and EducationMultimodal Machine Learning ApplicationsRadiology practices and education

Collaboration between clinicians and vision–language models in radiology report generation

Open access
Tanno, Ryutaro, Barrett, David G. T., Sellergren, Andrew, Ghaisas, Sumedh, Dathathri, Sumanth, See, Abigail, Welbl, Johannes, Lau, Charles +21 more·2025·Nature Medicine

Published version of a preprint

Abstract

Abstract Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment. The worldwide shortage of radiologists, however, restricts access to expert care and imposes heavy workloads, contributing to avoidable errors in report delivery. While recent progress in automated report generation with vision-language models offers clear potential to ameliorate this situation, the path toward real-world adoption has been stymied by the challenge of evaluating the clinical quality of AI-generated reports. In this study, we build a state-of-the-art report generation system for chest radiographs, Flamingo-CXR, by fine-tuning a well-known vision-language foundation model on radiology data. To measure the quality of the AI-generated reports, we perform an expert evaluation, that is largest in scale and diversity to date, by engaging a group of 27 certified radiologists in the United States and India to provide detailed assessment of AI-generated and human written reports from an intensive care setting as well as an inpatient setting. We observe a wide distribution of preferences across the panel, ranging from full consensus to dissensus, across clinical settings and regions, with 55.4% of Flamingo-CXR intensive care reports evaluated to be preferable or equivalent to clinician reports, by half or more of the panel, rising to 77.7% for outpatient x-rays overall and to 94% for the subset of cases with no pertinent abnormal findings. For reports that contain errors we develop an assistive setting, the first demonstration of clinician-AI collaboration for radiology report composition, and we observe a synergistic improvement across all clinical settings. Altogether, these nuanced evaluations reveal disparities between the AI system and radiologists, identify areas for potential clinical utility and pave the way toward a collaborative system that enhances clinical accuracy of radiology reporting.

Integrity signals

Registry

Crossref · 10.1038/s41591-024-03302-1

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Tanno, R., Barrett, D. G. T., Sellergren, A., Ghaisas, S., Dathathri, S., See, A., Welbl, J., Lau, C., Tu, T., Azizi, S., Singhal, K., Schaekermann, M., May, R., Lee, R., Man, S., Mahdavi, S., Ahmed, Z., Matias, Y., Barral, J., Eslami, S. M. A., Belgrave, D., Liu, Y., Kalidindi, S. R., Shetty, S., Natarajan, V., Kohli, P., Huang, P.-S., Karthikesalingam, A., & Ktena, I.
Affiliations
  • Google DeepMind (United Kingdom) (GB)
  • Google (United Kingdom) (GB)
  • GlaxoSmithKline (United Kingdom) (GB)
  • Apollo Hospitals (IN)
Corresponding author
  • Tanno, Ryutaro
  • Barrett, David G. T.
  • Karthikesalingam, Alan
Venue
Nature Medicine
Published
2024-11-07
Volume
31
Issue
2
Pages
599-608
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
OpenAlex
W4404143589
Cited by
186
References
54
Citations by year
2026: 106 · 2025: 77 · 2024: 2 · 2023: 1
FWCI
17.30
Type
journal article
Language
en
Open access
Locations
  • Nature Medicine (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Funders
DeepMind
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
Keywords
automated radiology report generation, vision-language models, chest radiograph, clinician-AI collaboration, radiologist workload
MeSH terms
Radiologists, Artificial Intelligence, Humans, Radiology, Radiography, Thoracic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review