Performance of an Open-Source Large Language Model in Extracting Information from Free-Text Radiology Reports
Open accessAbstract
“Just Accepted” papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Purpose To assess the performance of a local open-source large language model (LLM) on various information extraction tasks from real-life emergency brain MRI reports. Materials and Methods All consecutive emergency brain MRI reports written in 2022 from a French quaternary center were retrospectively reviewed. Two radiologists identified MRIs that were performed for headaches. Four radiologists scored reports' conclusions as normal or abnormal. Abnormalities were labeled as either headache-causing or incidental. Vicuna, an open-source LLM, performed the same tasks. Vicuna’s performance metrics were evaluated using the radiologists’ consensus as the reference standard. Results Among the 2398 reports during the study period, radiologists identified 595 that included headaches in their indication (median age of patients, 35 years [IQR, 26–51], 68% (403/595) female). A positive finding was reported in 227/595 (38%) cases, 136 of which could explain the headache. The LLM had a sensitivity/specificity (95%CI), respectively, of 98% (583/595)(97–99)/99% (1791/1803)(99–100) for detecting the presence of headache in the clinical context, 99% (514/517)(98–100)/99% (68/69)(92–100) for the use of contrast medium injection, 97% (219/227)(93–99)/99% (364/368)(97–100) for study categorization as normal or abnormal and 88% (120/136)(82– 93)/73% (66/91)(62–81) for causal inference between MRI findings and headache. Conclusion An open-source LLM was able to extract information from free-text radiology reports with excellent accuracy without requiring further training. ©RSNA, 2024
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most studies applying large language models on clinical notes have been conducted in the US without individual consent, using de-identified data, with only one study from France utilising retrospective MRI reports without identifiable data.
“Most studies applying LLM on clinical notes have been conducted in the US without individual consent, using de-identified data, with only one study from France utilising retrospective MRI reports without identifiable data [5, 6, 13, 14, 17–20].”
From A pilot feasibility study comparing large language models in extracting key information from ICU patient text records from an Irish population (2024) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738 · see citing works
Buderer, N. M. F. (1996) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.1996.tb03538.x · Cited by 912 · see citing works
Explainability for Large Language Models: A Survey
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681 · see citing works
Natural language processing in radiology: a systematic review
Pons, E., Braun, L. M. M., Hunink, M. G. M., & Kors, J. A. (2016) · Radiology · doi:10.1148/radiol.16142770 · Cited by 573 · see citing works
Bhayana, R., Krishna, S., & Bleakney, R. R. (2023) · Radiology · doi:10.1148/radiol.230582 · Cited by 460 · see citing works
Emergent analogical reasoning in large language models
Webb, T., Holyoak, K. J., & Lu, H. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01659-w · Cited by 404 · see citing works
Adams, L. C., Truhn, D., Busch, F., Kader, A., Niehues, S. M., Makowski, M. R., & Bressem, K. K. (2023) · Radiology · doi:10.1148/radiol.230725 · Cited by 339 · see citing works
Ethics of large language models in medicine and medical research
Li, H., Moon, J. T., Purkayastha, S., Celi, L. A., Trivedi, H., & Gichoya, J. W. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00083-3 · Cited by 286 · see citing works
Potential of ChatGPT and GPT-4 for Data Mining of Free-Text CT Reports on Lung Cancer
Fink, M. A., Bischoff, A., Fink, C. A., Moll, M., Kroschke, J., Dulz, L., Heußel, C. P., Kauczor, H., & Weber, T. F. (2023) · Radiology · doi:10.1148/radiol.231362 · Cited by 278 · see citing works
Jiang, Z., Araki, J., Ding, H., & Neubig, G. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00407 · Cited by 177 · see citing works
AI Chatbots, Health Privacy, and Challenges to HIPAA Compliance
Marks, M., & Haupt, C. E. (2023) · JAMA · doi:10.1001/jama.2023.9458 · Cited by 174 · see citing works
Mukherjee, P., Hou, B., Lanfredi, R. B., & Summers, R. M. (2023) · Radiology · doi:10.1148/radiol.231147 · Cited by 117 · see citing works
Artificial Intelligence in Clinical Diagnosis: Opportunities, Challenges, and Hype
Kulkarni, P. A., & Singh, H. (2023) · JAMA · doi:10.1001/jama.2023.11440 · Cited by 112 · see citing works
Why open-source generative AI models are an ethical way forward for science
Spirling, A. (2023) · Nature · doi:10.1038/d41586-023-01295-4 · Cited by 104 · see citing works
Accuracy of ChatGPT, Google Bard, and Microsoft Bing for Simplifying Radiology Reports
Amin, K., Davis, M. A., Doshi, R. H., Haims, A. H., Khosla, P., & Forman, H. P. (2023) · Radiology · doi:10.1148/radiol.232561 · Cited by 104 · see citing works
A Systematic Review of Interventions to Reduce Computed Tomography Usage in the Emergency Department
Dunne, C. L., Elzinga, J. L., Vorobeichik, A., Sudershan, S., Keto‐Lambert, D., Lang, E., & Dowling, S. K. (2022) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2022.06.001 · Cited by 61 · see citing works
Langlotz, C. P. (2002) · Radiology · doi:10.1148/radiol.2241020415 · Cited by 57 · see citing works
The Future of AI and Informatics in Radiology: 10 Predictions
Langlotz, C. P. (2023) · Radiology · doi:10.1148/radiol.231114 · Cited by 56 · see citing works
Radiology in the era of large language models: the near and the dark side of the moon
Úbeda, P. L., Martín‐Noguerol, T., & Luna, A. (2023) · European Radiology · doi:10.1007/s00330-023-09901-9 · Cited by 27 · see citing works
Diagnostic Neuroimaging in Headache Patients: A Systematic Review and Meta-Analysis
Jang, Y. E., Cho, E. Y., Choi, H. Y., Kim, S. M., & Park, H. Y. (2019) · Psychiatry Investigation · doi:10.30773/pi.2019.04.11 · Cited by 24 · see citing works
Trofimova, A. V., Duszak, R., Kadom, N., & Sadigh, G. (2020) · Headache The Journal of Head and Face Pain · doi:10.1111/head.14020 · Cited by 14 · see citing works
Jupin-Delevaux, É., Djahnine, A., Talbot, F., Richard, A., Gouttard, S., Mansuy, A., Douek, P., Si-Mohamed, S. A., & Boussel, L. (2023) · Research in Diagnostic and Interventional Imaging · doi:10.1016/j.redii.2023.100027 · Cited by 13 · see citing works
Lungren, M. P., Amrhein, T. J., Paxton, B. E., Srinivasan, R. C., Collins, H. R., Eastwood, J. D., & Kilani, R. K. (2013) · Radiology · doi:10.1148/radiol.13130281 · Cited by 13 · see citing works
Evaluating Large Language Models in Extracting Cognitive Exam Dates and ScoresPreprint
Zhang, H., Jethani, N., Jones, S., et al. (2023) · medRxiv · doi:10.1101/2023.07.10.23292373 · Cited by 13 · see citing works
Diagnostic yield of emergency MRI in non-traumatic headache
Happonen, T., Nyman, M. J., Ylikotila, P., Merisaari, H., Mattila, K., & Hirvonen, J. (2022) · Neuroradiology · doi:10.1007/s00234-022-03044-2 · Cited by 11 · see citing works
Factors associated with significant MRI findings in medical walk-in patients with acute headache
Budweg, J., Sprenger, T., Vere-Tyndall, A. D., Hagenkord, A., Stippich, C., & Berger, C. T. (2016) · Swiss Medical Weekly · doi:10.4414/smw.2016.14349 · Cited by 10 · see citing works
Transforming primary healthcare through natural language processing and big data analytics
Uddin, Y., Nair, A., Shariq, S., & Hannan, S. H. (2023) · BMJ · doi:10.1136/bmj.p948 · Cited by 5 · see citing works
Sebro, R., & Kahn, C. E. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad119 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [17]Zheng L, Chiang WL, Sheng Y, et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. ArXiv 2306.05685 [preprint] https://arxiv.org/abs/2306.05685. Posted June 9, 2023. Accessed May 29, 2024. · arXiv
- [19]Agrawal M, Hegselmann S, Lang H, Kim Y, Sontag D. Large Language Models are Few-Shot Clinical Information Extractors. ArXiv 2205.12689 [preprint] https://arxiv.org/abs/2205.12689. Posted May 25, 2022. Accessed May 29, 2024. · arXiv
- [21]Gutiérrez BJ, McNeal N, Washington C, et al. Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again. ArXiv 2203.08410 [preprint] https://arxiv.org/abs/2203.08410. Posted March 16, 2022. Accessed May 29, 2024. · arXiv
- [22]Gero Z, Singh C, Cheng H, et al. Self-Verification Improves Few-Shot Clinical Information Extraction. ArXiv 2306.00024 [preprint] https://arxiv.org/abs/2306.00024. Posted May 30, 2023. Accessed May 29, 2024. · arXiv
- [36]Lai VD, Ngo NT, Ben Veyseh AP, et al. ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning. ArXiv 2304.05613 [preprint] https://arxiv.org/abs/2304.05613. Posted April 12, 2023. Accessed May 29, 2024. · arXiv
Cited by
Sorted by citations
Large Language Models and Large Multimodal Models in Medical Imaging: A Primer for Physicians
Bradshaw, T., Tie, X., Warner, J., Hu, J., Li, Q., & Li, X. (2025) · Journal of Nuclear Medicine · doi:10.2967/jnumed.124.268072 · Cited by 58 · see citing works
Comprehensive testing of large language models for extraction of structured data in pathology
Grothey, B., Odenkirchen, J., Brkic, A., Schömig-Markiefka, B., Quaas, A., Büttner, R., & Tolkach, Y. (2025) · Communications Medicine · doi:10.1038/s43856-025-00808-8 · Cited by 54 · see citing works
Salam, B., Stüwe, C., Nowak, S., Sprinkart, A. M., Theis, M., Kravchenko, D., Mesropyan, N., Dell, T., Endler, C., Pieper, C. C., et al. (2025) · European Radiology · doi:10.1007/s00330-025-11438-y · Cited by 43 · see citing works
Benchmarking the diagnostic performance of open source LLMs in 1933 Eurorad case reports
Kim, S. H., Schramm, S., Adams, L. C., Braren, R., Bressem, K. K., Keicher, M., Platzek, P., Paprottka, K. J., Zimmer, C., Hedderich, D. M., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01488-3 · Cited by 41 · see citing works
Lee, J. E., Park, K. S., Kim, Y., Song, H.-C., Park, B., & Jeong, Y. J. (2024) · American Journal of Roentgenology · doi:10.2214/ajr.24.31696 · Cited by 41 · see citing works
Benchmarking Large Language Models in Evidence-Based Medicine
Li, J., Deng, Y., Sun, Q., Zhu, J., Tian, Y., Li, J., & Zhu, T. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3483816 · Cited by 35 · see citing works
Hein, D., Christie, A. L., Holcomb, M. J., Xie, B., Jain, A., Vento, J. A., Rakheja, N., Shakur, A. H., Christley, S., Cowell, L. G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01686-z · Cited by 30 · see citing works
Meddeb, A., Ebert, P., Bressem, K. K., Desser, D., Dell'Orco, A., Bohner, G., Kleine, J. F., Siebert, E., Grauhan, N., Brockmann, M. A., et al. (2025) · Journal of NeuroInterventional Surgery · doi:10.1136/jnis-2024-022078 · Cited by 27 · see citing works
The performance of large language models in dentomaxillofacial radiology: a systematic review
Liu, Z., Nalley, A., Hao, J., Ai, Q. Y. H., Yeung, A. W. K., Tanaka, R., & Hung, K. F. (2025) · Dentomaxillofacial Radiology · doi:10.1093/dmfr/twaf060 · Cited by 31 · see citing works
Use of Large Language Models on Radiology Reports: A Scoping Review
Lee, R. C., Hadidchi, R., Coard, M. C., Rubinov, Y., Alamuri, T. T., Liaw, A., Chandrupatla, R. S., & Duong, T. Q. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.10.005 · Cited by 26 · see citing works
Li, D. K. B., Gupta, K., Bhaduri, M., Sathiadoss, P., Bhatnagar, S., & Chong, J. (2024) · Clinical Imaging · doi:10.1016/j.clinimag.2024.110382 · Cited by 23 · see citing works
Large language models for bioinformatics
Ruan, W., Lyu, Y., Zhang, J., Cai, J., Shu, P., Yang, G., Lu, Y., Gao, S., Wang, Y., Wang, P., et al. (2025) · Quantitative Biology · doi:10.1002/qub2.70014 · Cited by 35 · see citing works
Urquhart, E., Ryan, J., Hartigan, S., Nita, C., Hanley, C., Moran, P., Bates, J., Jooste, R., Judge, C., Laffey, J. G., et al. (2024) · Intensive Care Medicine Experimental · doi:10.1186/s40635-024-00656-1 · Cited by 18 · see citing works
Drafting the Future: The Dawn of AI Report Generation in Radiology
Seah, J. C. Y., Tang, J. S. N., & Tran, A. (2025) · Radiology · doi:10.1148/radiol.243378 · Cited by 19 · see citing works
Wihl, J., Rosenkranz, E., Schramm, S., Berberich, C., Griessmair, M., Woźnicki, P., Pinto, F., Ziegelmayer, S., Adams, L. C., Bressem, K. K., et al. (2025) · European Radiology Experimental · doi:10.1186/s41747-025-00600-2 · Cited by 18 · see citing works
In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
Jiménez-Sánchez, A., Avlona, N.-R., de Boer, S., Campello, V. M., Feragen, A., Ferrante, E., Ganz, M., Gichoya, J. W., González, C., Groefsema, S., et al. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732035 · Cited by 21 · see citing works
Bentegeac, R., Guellec, B. L., Kuchcinski, G., Amouyel, P., & Hamroun, A. (2025) · Journal of Medical Internet Research · doi:10.2196/64348 · Cited by 19 · see citing works
Self-citationReal-World Monitoring of Artificial Intelligence in Radiology: Challenges and Best Practices
Dean, G., Montañà, E., Kyriazi, S., Shelmerdine, S. C., Buckens, C. F. M., Agrell, H., Ranschaert, E. R., Marušić, A., Davies, G. J., Wardle, P., et al. (2025) · Korean Journal of Radiology · doi:10.3348/kjr.2025.0962 · Cited by 15 · see citing works
Jabal, M. S., Warman, P. I., Zhang, J., Gupta, K., Jain, A., Mazurowski, M. A., Wiggins, W. F., Magudia, K. A., & Calabrese, E. (2025) · Radiology Artificial Intelligence · doi:10.1148/ryai.240551 · Cited by 18 · see citing works
Sorin, V., Collins, J. D., Bratt, A. K., Kusmirek, J. E., Mugu, V. K., Kline, T. L., Butler, C. S., Wood, N. G., Cook, C. J., & Korfiatis, P. D. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf073 · Cited by 13 · see citing works
Sollini, M., Pini, C., Lazăr, A. M., Gelardi, F., Ninatti, G., Bauckneht, M., Chiti, A., & Kirienko, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-28993-5 · Cited by 11 · see citing works
Fathi, M., Vakili, K. G., Hajibeygi, R., Bahrami, A., Behzad, S., Tafazolimoghadam, A., Aghabozorgi, H., Eshraghi, R., Bhatt, V. V., & Gholamrezanezhad, A. (2024) · Clinical Imaging · doi:10.1016/j.clinimag.2024.110356 · Cited by 12 · see citing works
Poole, S. T., Sisodia, N., Koshal, K., Henderson, K., Wijangco, J., Paredes, D., Chen, C., Rowles, W. M., Akula, A. V., Wuerfel, J. T., et al. (2025) · Annals of Neurology · doi:10.1002/ana.27251 · Cited by 11 · see citing works
Human level information extraction from clinical reports with finetuned language models
Liu, L., Lian, L., Hao, Y., Pace, A., Kim, E., Homsi, N., Pershad, Y., Lai, L., Gracie, T. J., Kishtagari, A., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-28767-z · Cited by 14 · see citing works
Yang, L., Zhou, Y., Qi, J., Zhen, X., Sun, L., Shi, S. R., Su, Q., & Yang, X. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.111984 · Cited by 10 · see citing works
Liu, Z., Ai, Q. Y. H., Yeung, A. W. K., Tanaka, R., Nalley, A., & Hung, K. F. (2025) · Diagnostics · doi:10.3390/diagnostics15182315 · Cited by 9 · see citing works
Fang, Z., Johnston, A., Cheuy, L. Y., Na, H. S., Paschali, M., González, C., Armstrong, B. A., Koirala, A., Laurel, D., Campion, A., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02007-0 · Cited by 11 · see citing works
Dreizin, D., Khatri, G. D., Staziaki, P. V., Buch, K. A., Unberath, M., Mohammed, M., Sodickson, A. D., Khurana, B., Agrawal, A., Spann, J. S., et al. (2024) · Emergency Radiology · doi:10.1007/s10140-024-02306-1 · Cited by 8 · see citing works
Ito, R., Kato, K., Nanataki, K., Abe, Y., Ogawa, H., Minamimoto, R., Kato, K., Taoka, T., & Naganawa, S. (2025) · EJNMMI Reports · doi:10.1186/s41824-025-00246-8 · Cited by 7 · see citing works
Yang, Q., Jiang, J., Dong, X., Yang, H., Wang, Q., Yang, Z., Yang, D., & Liu, P. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01728-8 · Cited by 6 · see citing works
Li, P., Xu, Y., Liu, X., Shen, Z., Wang, Y., Lv, X., Lu, Z., Wu, H., Zhuang, J., & Chen, Y. (2026) · JMIR Medical Informatics · doi:10.2196/81422 · Cited by 6 · see citing works
Dehdab, R., Mankertz, F., Brendel, J. M., Maalouf, N., Kaya, K., Afat, S., Kolahdoozan, S., & Radmard, A. R. (2025) · Academic Radiology · doi:10.1016/j.acra.2025.07.041 · Cited by 6 · see citing works
Leveraging Large Language Models for Accurate AO Fracture Classification from CT Text Reports
Mergen, M., Spitzl, D., Ketzer, C., Strenzke, M., Marka, A. W., Makowski, M. R., Bressem, K. K., Adams, L. C., & Gassert, F. T. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01603-6 · Cited by 6 · see citing works
Sugawara, H., Takada, A., & Kato, S. (2025) · Japanese Journal of Radiology · doi:10.1007/s11604-025-01884-5 · Cited by 5 · see citing works
Yao, Y., Cen, X., Gan, L., Jiang, J., Wang, M., Xu, Y., & Yuan, J. (2025) · JMIR Medical Informatics · doi:10.2196/75556 · Cited by 5 · see citing works
Dennstädt, F., Lerch, L., Schmerder, M., Čihorić, N., Cereghetti, G. M., Gaio, R., Bonél, H., Filchenko, I., Hastings, J., Dammann, F., et al. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf128 · Cited by 7 · see citing works
Data mining in pediatric radiology in the era of artificial intelligence
Guarnera, A., Ghosh, A., Gikera, R., Vahdati, S., Zhang, K., & Gupta, A. (2026) · Pediatric Radiology · doi:10.1007/s00247-026-06551-z · Cited by 5 · see citing works
Khosravi, B., Dapamede, T., Li, F., Chisango, Z. M., Bikmal, A., Garg, S., Owosela, B., Khosravi, A., Chavoshi, M., Trivedi, H., et al. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01505-7 · Cited by 6 · see citing works
Hein, D., Christie, A., Holcomb, M. J., Xie, B., Jain, A., Vento, J., Rakheja, N., Shakur, A. H., Christley, S., Cowell, L. G., et al. (2025) · medRxiv · doi:10.1101/2025.02.11.25322107 · Cited by 6 · see citing works
Demystifying Mental Health Reports Through an LLM-based Approach
Srinivasan, S. S. K., Bahadur, A., Singh, S., Gupta, S., Jain, V., Deb, K. S., Kumar, M. J., & Singh, P. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720208 · Cited by 4 · see citing works
Kanani, M. M., Monawer, A. H., Brown, L., King, W. E., Miller, Z. D., Venugopal, N., Heagerty, P. J., Jarvik, J. G., Cohen, T., & Cross, N. M. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01530-6 · Cited by 4 · see citing works
Prucker, P., Bressem, K. K., Kim, S. H., Weller, D., Kader, A., Dorfner, F. J., Ziegelmayer, S., Graf, M. M., Lemke, T., Gassert, F. T., et al. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01618-z · Cited by 6 · see citing works
NLP-Driven Image Label Extraction from Radiology Reports: A Review
Sivakrishna, K., Swathi, P., Lakshmi, T., Lakshmi, M. N., Obulesu, D., & Rani, S. S. (2025) · International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI) · doi:10.1109/icoici65217.2025.11252535 · Cited by 4 · see citing works
Clinical AI in Radiology: Foundations, Trends, Applications, and Emerging Directions
Hartsock, I., Koutsoubis, N., Ahmed, S., Parker, N. H., Schabath, M. B., Araujo, C. R., Qayyum, A., Lam, C. A., Gatenby, R. A., & Rasool, G. (2026) · Cancers · doi:10.3390/cancers18060942 · Cited by 4 · see citing works
Schramm, S., Guellec, B. L., Topka, M., Svec, M., Backhaus, P., Eisenkolb, V. M., Riedel, E. O., Beyrle, M., Platzek, P., Ramschütz, C., et al. (2026) · Radiology · doi:10.1148/radiol.253477 · Cited by 4 · see citing works
Klang, E., Alper, L., Sorin, V., Barash, Y., Nadkarni, G. N., & Zimlichman, E. (2023) · BJR|Open · doi:10.1093/bjro/tzae022 · Cited by 4 · see citing works
Moassefi, M., Houshmand, S., Faghani, S., Chang, P., Sun, S. H., Khosravi, B., Triphati, A. G., Rasool, G., Bhatia, N. K., Folio, L. R., et al. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01523-5 · Cited by 4 · see citing works
Novoa-Laurentiev, J., Curtin-Bowen, M., Pullman, A., Song, W., Syrowatka, A., Chen, J., Sainlaire, M., CHANG, F. Y., Gray, K., Panta, P., et al. (2025) · JMIR Medical Informatics · doi:10.2196/63720 · Cited by 4 · see citing works
Automated cardiac magnetic resonance interpretation derived from prompted large language models
Wang, L., Peng, L., Wan, Y., Li, X., Chen, Y., Wang, L., Gong, X., Zhao, X., Yu, L., Zhao, S., et al. (2025) · Cardiovascular Diagnosis and Therapy · doi:10.21037/cdt-2025-112 · Cited by 4 · see citing works
Navigating Artificial Intelligence in Scientific Manuscript Writing: Tips and Traps
Kumar, I., Yadav, N., & Verma, A. K. (2025) · Indian journal of radiology and imaging - new series/Indian journal of radiology and imaging/Indian Journal of Radiology & Imaging · doi:10.1055/s-0044-1800801 · Cited by 8 · see citing works
Dennstädt, F., Fauser, S., Čihorić, N., Schmerder, M., Lombardo, P., Cereghetti, G. M., von Däniken, S. U., Minder, T., Meyer, J., Chiang, L., et al. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01659-4 · Cited by 5 · see citing works
A New Era of Text Mining in Radiology with Privacy-Preserving LLMs
D’Antonoli, T. A., & Blüthgen, C. (2024) · Radiology Artificial Intelligence · doi:10.1148/ryai.240261 · Cited by 4 · see citing works
Sales, A. H. A., Gizaw, C. J., Beck, J., & Grauvogel, J. (2025) · Diagnostics · doi:10.3390/diagnostics15222841 · Cited by 3 · see citing works
RadCLARE: an automated clinical language engine for detecting semantic errors in radiology reports
Pan, F. S., Lou, J., Guo, Y., Wang, D., Wang, Z., Fan, Q., Wang, H., Zheng, C., & Yang, L. (2025) · European Radiology Experimental · doi:10.1186/s41747-025-00659-x · Cited by 3 · see citing works
Large Language Models: A Comprehensive Guide for Radiologists
Kim, S., Lee, C., & Kim, S. (2024) · Journal of the Korean Society of Radiology · doi:10.3348/jksr.2024.0080 · Cited by 3 · see citing works
Natural language processing and LLMs in liver imaging: a practical review of clinical applications
Úbeda, P. L., Martín‐Noguerol, T., & Luna, A. (2025) · Abdominal Radiology · doi:10.1007/s00261-025-05127-z · Cited by 3 · see citing works
Khazanchi, R., Soetikno, A. G., Chopra, A., Herrera, D. E., Staub, J. R., Balu, A. R., Kumar, D., Chen, A. R., Hsu, W. K., Patel, A. A., et al. (2025) · Journal of Clinical Neuroscience · doi:10.1016/j.jocn.2025.111648 · Cited by 2 · see citing works
Sheng, W., Wang, Y., Xiao, L., Guo, B., Zhang, Y., Qiao, L., Li, Q., Yang, L., & Zhang, Y. (2025) · European Radiology · doi:10.1007/s00330-025-12147-2 · Cited by 2 · see citing works
Odisho, A. Y., Liu, A. W., Pace, W. A., Carlisle, M. N., Krumm, R., Cowan, J. E., Carroll, P. R., & Cooperberg, M. R. (2026) · JCO Clinical Cancer Informatics · doi:10.1200/cci-24-00334 · Cited by 2 · see citing works
Zhu, J., Li, J., Zhao, S., Deng, Y., Miao, Y., & Xu, J. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-025-08182-x · Cited by 2 · see citing works
Kim, S. H., Schramm, S., Adams, L. C., Braren, R., Bressem, K. K., Keicher, M., Zimmer, C., Hedderich, D. M., & Wiestler, B. (2024) · medRxiv · doi:10.1101/2024.09.04.24313026 · Cited by 2 · see citing works
Dennstädt, F., Fauser, S., Čihorić, N., Schmerder, M., Lombardo, P., Cereghetti, G. M., von Däniken, S., Minder, T., Meyer, J., Chiang, L., et al. (2025) · medRxiv · doi:10.1101/2025.04.08.25325371 · Cited by 2 · see citing works
Akbasli, I. T., Beck, K. L., Liou, W., Du, S., Nemer, G., Baloğlu, O., Latifi, S. Q., Marino, B. S., Albahra, S., & Tandon, A. A. (2026) · medRxiv · doi:10.64898/2026.01.21.26344531 · Cited by 2 · see citing works
Ng, Y. S., Kanani, M. M., King, W. E., Miller, Z. D., Brown, L., Wishart, J., Lindgren-Ruby, A., Medverd, J. R., & Cross, N. M. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.11.024 · Cited by 1 · see citing works
Large language models and their impact in medical imaging education
Zhu, J., & Cai, H. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.3433 · Cited by 2 · see citing works
Adamson, S., Berry, C., Adler, N. R., Christian, T., Librata, W., & Mar, V. J. (2025) · Australasian Journal of Dermatology · doi:10.1111/ajd.70034 · Cited by 1 · see citing works
Banh, L., Stangl, F. J., Strobel, G., & Riedl, R. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-032-00815-2_3 · Cited by 1 · see citing works
Alwakeel, M., Buck, E., Martin, J. G., Aslam, I., Rajagopal, S., Pei, J., Podgoreanu, M. V., Lindsell, C. J., & Wong, A.-K. I. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-09569-5_21 · Cited by 1 · see citing works
Large Language Models in Radiologic Numerical Tasks: A Thorough Evaluation and Error Analysis
Nowroozi, A., Bondarenko, M., Serapio, A., Schnitzler, T., Brar, S. S., & Sohn, J. H. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01824-9 · Cited by 2 · see citing works
Evaluating the role of ChatGPT in structured radiology reporting: A systematic review
Alalawi, S. A., Alchoghari, R., Hakami, A. M., Alhazmi, A., Almutari, E., Asseiri, R. A., Alzahrani, A. M., Aladhyani, M., Alqarni, S., & Al-Sharydah, A. M. (2026) · Medicine · doi:10.1097/md.0000000000047541 · Cited by 1 · see citing works
Semantic Similarity in Radiology Reports via LLMs and NER
Pearson, B., Adnan, A., & Abdallah, Z. S. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17216-7_10 · Cited by 1 · see citing works
Yu, H., Bondarenko, M., Nowroozi, A., Lee, Y. J., Serapio, A., Kaveti, P., & Sohn, J. H. (2026) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.70336 · Cited by 1 · see citing works
Who labels best? Radiologists, rules, or large language models for CT reports on pulmonary embolism
Fink, M. A., Bischoff, A., Atsiatorme, E., Kremer, A., Kroschke, J., Moll, M., Stein, P., Riebl, V., Leichenich, T., Kauczor, H., et al. (2026) · European Radiology Experimental · doi:10.1186/s41747-026-00738-7 · Cited by 1 · see citing works
Fink, M. A., Bischoff, A., Atsiatorme, E., Kremer, A., Kroschke, J., Moll, M., Stein, P., Riebl, V., Leichenich, T., Kauczor, H., et al. (2026) · Insights into Imaging · doi:10.1186/s13244-026-02289-2 · Cited by 1 · see citing works
D’Antonoli, T. A., Adams, L. C., Lübberstedt, J., Graf, M. M., Mertens, C. J., Busch, F., Ziegelmayer, S., Makowski, M. R., Bressem, K. K., & Luiken, I. (2026) · European Radiology · doi:10.1007/s00330-026-12697-z · Cited by 2 · see citing works
Novoa-Laurentiev, J., Curtin-Bowen, M., Pullman, A., Song, W., Syrowatka, A., Chen, J., Sainlaire, M., CHANG, F. Y., Gray, K., Panta, P., et al. (2024) · doi:10.2196/preprints.63720 · Cited by 1 · see citing works
Gee, J. C., & Yao, M. S. (2025) · Radiology · doi:10.1148/radiol.243659 · Cited by 1 · see citing works
Dennstädt, F., Lerch, L., Schmerder, M., Čihorić, N., Cereghetti, G. M., Gaio, R., Bonél, H., Filchenko, I., Hastings, J., Dammann, F., et al. (2025) · medRxiv · doi:10.1101/2025.06.01.25328636 · Cited by 1 · see citing works
Corso, F., Peppoloni, V., Mazzeo, L., Leone, G., Passos, L., Miskovic, V., Armanini, J., Ferrarin, A., Wiest, I. C., Wolf, F., et al. (2025) · medRxiv · doi:10.1101/2025.10.16.25337917 · Cited by 1 · see citing works
Empowering Patient Wellness: Automated Radiology Report Generation Using Deep Learning and NLG
Mishra, A., Nauhwar, H., & Jindal, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93982-2_19
Sollini, M., Pini, C., Lazăr, A. M., Gelardi, F., Ninatti, G., Bauckneht, M., Chiti, A., & Kirienko, M. (2025) · Research Square · doi:10.21203/rs.3.rs-6863946/v1
Assessing GPT as a Weak Oracle for Annotating Radiological Studies
de Ferrari, J., Ñanculef, R., Benoit, D., Araya, M., & Solar, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-95838-0_10
HONGSLO, A., Gupta, A., Nguyen, Q., Caldwell, J., Choi, B., Harvey, C. J., Thompson, J., Mazzotti, D. R., Yao, Z., & Noheria, A. (2025) · medRxiv · doi:10.1101/2025.10.05.25337366
Chang, E., Xie, K., & Ellis, C. A. (2025) · Seminars in Neurology · doi:10.1055/a-2742-2430
Bonney, L. M., Brown, S. L., Bradley, K. M., Hassan, A. B., & McGowan, D. R. (2025) · medRxiv · doi:10.1101/2025.11.27.25339233
Yang, L., Liang, X., Wang, Z., Diao, Z., Huang, X., Shen, D., Tan, X., Li, H., Chen, Z., Qiu, S., et al. (2025) · Radiology Cardiothoracic Imaging · doi:10.1148/ryct.250033
Urquhart, E., Ryan, J., Hartigan, S. D., Niță, C., Hanley, C., Moran, P., Bates, J. J., Jooste, R., Judge, C. S., Laffey, J. G., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26777725
Application of Foundation Models in Emergency and Critical Care: A Scoping ReviewPreprint
Kong, Y., Mai, X., Meyer, A., FANG, Y. C., Xu, Z., Song, Z., Gong, C., Wacker, D. A., Heneghan, J. A., Wang, Q., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-8338830/v1
Maiter, A., & Zapaishchykova, A. (2026) · Radiology Artificial Intelligence · doi:10.1148/ryai.251133
Verifying LLM-extracted text with token alignmentPreprint
Booeshaghi, A. S., & Streets, A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.06.704502
Hanna, J. J., Evans, C. S., Dennis, C. R., Lee, K. S., Lehmann, C. U., & Medford, R. J. (2025) · Methods of Information in Medicine · doi:10.1055/a-2797-4295
Costa, M., Khan, M. R., Vedovelli, L., Gregori, D., Azzolina, D., & Bressan, S. (2026) · BMC Emergency Medicine · doi:10.1186/s12873-025-01468-6
Wang, Y., Alejandrez, A., Stewart, C., Malekhedayat, M., Luong, J., Smith‐Bindman, R., & Mongan, J. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01882-7
Birry, M., Roach, C., Hunter‐Philpott, D. A., & Doyle, A. (2026) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.70103
Amagai, S., Powell, E. C., Alpern, E. R., Florin, T. A., Li, Y., Liebovitz, D., Luo, Y., Espinoza, J., Wong, A. F., & Ramgopal, S. (2026) · Pediatric Emergency Care · doi:10.1097/pec.0000000000003616
Artificial Intelligence Tools for Dental Clinics
Sesma, N., Barros, L. F., Teixeira, M. L., Matias, B. B. R., Dias, S. C., Costa, A. D. M. E., Dutra, V., Saavedra, G. [., Pereira, T. C., Laurino, F. A. R., et al. (2026) · Springer eBooks · doi:10.1007/978-3-032-24895-4_6
Özenbaş, C., & Şükun, A. (2026) · The Egyptian Journal of Radiology and Nuclear Medicine · doi:10.1186/s43055-026-01766-4
Zhong, W., Yan, H., Liu, Y., Liu, Y., Yang, K., Gao, H., Yao, Z., Hao, W., Yan, Y., & Yin, C. (2026) · Journal of Medical Internet Research · doi:10.2196/91399
Khan, H., Mahdian, G., & Jalooli, A. (2026) · IEEE World AI IoT Congress (AIIoT) · doi:10.1109/aiiot68874.2026.11569596
Zhang, Z., Jiang, Z., Qi, Y., Hu, Z., Zhuang, M., Li, J., & Wang, L. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02068-x
- Authors
- Le Guellec, B., Lefèvre, A., Geay, C., Shorten, L., Bruge, C., Hacein-Bey, L., Amouyel, P., Pruvo, J.-P., Kuchcinski, G., & Hamroun, A.
- Affiliations
- Inserm (FR)
- ORCID
- Venue
- Radiology: Artificial Intelligence
- Published
- 2024-07-01
- Volume
- 6
- Issue
- 4
- Pages
- e230364-e230364
- ISSN
- 2638-6100 (ISSN-L)
- PMC
- PMC11294959
- OpenAlex
- W4396722287
- Cited by
- 115
- References
- 28
- Citations by year
- 2026: 42 · 2025: 60 · 2024: 12 · 2023: 1
- FWCI
- 10.66
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- cc-by
- Locations
- Radiology Artificial Intelligence (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (submittedVersion, cc-by)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- LillOA (Université de Lille (University Of Lille)) (submittedVersion)
- LillOA (Université de Lille (University Of Lille)) (submittedVersion, other-oa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- open-source large language models, vicuña, headache, contrast medium injection, incidental finding, radiologist consensus, local LLM deployment, radiology education, radiology workforce shortage
- MeSH terms
- Adult, Brain, Emergency Service, Hospital, Female, Headache, Humans, Magnetic Resonance Imaging, Male, Middle Aged, Natural Language Processing, Retrospective Studies, Sensitivity and Specificity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
