Enhancing Patient Communication With Chat-GPT in Radiology: Evaluating the Efficacy and Readability of Answers to Common Imaging-Related Questions
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
PURPOSE To assess ChatGPT's accuracy, relevance, and readability in answering patients' common imaging-related questions and examine the effect of a simple prompt. METHODS 22 imaging-related questions were developed from categories previously described as important to patients: safety, the radiology report, the procedure, preparation before imaging, meaning of terms, and medical staff. These questions were posed to ChatGPT with and without a short prompt instructing the model to provide an accurate and easy-to-understand response for the average person. Four board-certified radiologists evaluated the answers for accuracy, consistency, and relevance. Two patient advocates also reviewed responses for their utility for patients. Readability was assessed by Flesch Kincaid Grade Level (FKGL). Statistical comparisons were performed using chi-square and paired t-tests. RESULTS 264 answers were assessed for both unprompted and prompted questions. Unprompted responses were accurate 83% (218/264) of the time, which did not significantly change for prompted responses (87% [229/264]; P=0.2). The consistency of the responses increased from 72%f (63/88) to 86% (76/88) when prompted (P=0.02). Nearly all responses (99% [261/264]) were at least partially relevant for both question types. Fewer unprompted responses were considered fully relevant at 67% (176/264), though this increased significantly to 80% when prompted (210/264) (P=0.001). The average FKGL was high at 13.6 [12.9-14.2], unchanged with the prompt (13.0 [12.41-13.60], P=0.2). None of the responses reached the eighth-grade readability recommended for patient-facing materials. CONCLUSIONS ChatGPT demonstrates the potential to respond accurately, consistently, and relevantly to patients' imaging-related questions. However, imperfect accuracy and high complexity necessitate oversight before implementation. Prompts reduced response variability and yielded more targeted information but did not improve readability. RELEVANCE AND APPLICATION ChatGPT has the potential to increase accessibility to health information and to streamline the production of patient-facing educational materials, though its current limitations require cautious implementation and further research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738
Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT
Haver, H. L., Ambinder, E. B., Bahl, M., Oluyemi, E. T., Jeudy, J., & Yi, P. H. (2023) · Radiology · doi:10.1148/radiol.230424 · Cited by 247
Cited by
Tam, T. Y. C., Sivarajkumar, S., Kapoor, S., Stolyar, A. V., Polanska, K., McCarthy, K. R., Osterhoudt, H., Wu, X., Visweswaran, S., Fu, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01258-7 · Cited by 424
Higher education students’ perceptions of ChatGPT: A global study of early reactions
Ravšelj, D., Keržič, D., Tomaževič, N., Umek, L., Brezovar, N., A. Iahad, N., Abdulla, A. A., Akopyan, A., Aldana Segura, M. W., AlHumaid, J., et al. (2025) · PLOS ONE · doi:10.1371/journal.pone.0315011 · Cited by 376
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Large language models in patient education: a scoping review of applications in medicine
Aydin, S., Karabacak, M., Vlachos, V., & Margetis, K. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1477898 · Cited by 288
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283
Metadata
- Authors
- Gordon, E. B., Towbin, A. J., Wingrove, P., Shafique, U., Haas, B., Kitts, A. B., Feldman, J., & Furlan, A.
- Venue
- Journal of the American College of Radiology
- Publisher
- Elsevier BV
- Published
- 2024-02
- Volume
- 21
- Issue
- 2
- Pages
- 353-359
- ISSN
- 1546-1440 (ISSN-L)
- OpenAlex
- W4387747149
- Cited by
- 100
- References
- 2
- FWCI
- 3.46
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.