Scientific evidence for clinical text summarization using large language models: scoping review
Open accessPublished version of a preprint
Abstract
BACKGROUND: Information overload in electronic health records requires effective solutions to alleviate clinicians' administrative tasks. Automatically summarizing clinical text has gained significant attention with the rise of large language models. While individual studies show optimism, a structured overview of the research landscape is lacking. OBJECTIVE: This study aims to present the current state of the art on clinical text summarization using large language models, evaluate the level of evidence in existing research and assess the applicability of performance findings in clinical settings. METHODS: This scoping review complied with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Literature published between January 1, 2019, and June 18, 2024, was identified from 5 databases: PubMed, Embase, Web of Science, IEEE Xplore, and ACM Digital Library. Studies were excluded if they did not describe transformer-based models, did not focus on clinical text summarization, did not engage with free-text data, were not original research, were nonretrievable, were not peer-reviewed, or were not in English, French, Spanish, or German. Data related to study context and characteristics, scope of research, and evaluation methodologies were systematically collected and analyzed by 3 authors independently. RESULTS: A total of 30 original studies were included in the analysis. All used observational retrospective designs, mainly using real patient data (n=28, 93%). The research landscape demonstrated a narrow research focus, often centered on summarizing radiology reports (n=17, 57%), primarily involving data from the intensive care unit (n=15, 50%) of US-based institutions (n=19, 73%), in English (n=26, 87%). This focus aligned with the frequent reliance on the open-source Medical Information Mart for Intensive Care dataset (n=15, 50%). Summarization methodologies predominantly involved abstractive approaches (n=17, 57%) on single-document inputs (n=4, 13%) with unstructured data (n=13, 43%), yet reporting on methodological details remained inconsistent across studies. Model selection involved both open-source models (n=26, 87%) and proprietary models (n=7, 23%). Evaluation frameworks were highly heterogeneous. All studies conducted internal validation, but external validation (n=2, 7%), failure analysis (n=6, 20%), and patient safety risks analysis (n=1, 3%) were infrequent, and none reported bias assessment. Most studies used both automated metrics and human evaluation (n=16, 53%), while 10 (33%) used only automated metrics, and 4 (13%) only human evaluation. CONCLUSIONS: Key barriers hinder the translation of current research into trustworthy, clinically valid applications. Current research remains exploratory and limited in scope, with many applications yet to be explored. Performance assessments often lack reliability, and clinical impact evaluations are insufficient raising concerns about model utility, safety, fairness, and data privacy. Advancing the field requires more robust evaluation frameworks, a broader research scope, and a stronger focus on real-world applicability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Current benchmarking efforts do not quantify the downstream impact on clinical workflows.
“Nevertheless, these benchmarking efforts, like earlier studies, do not quantify the downstream impact on clinical workflows.”
From Implementation of large language models in electronic health records (2025) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1× · see citing works
“While recent studies have attempted to establish taxonomies for hallucinations [79,80], failures may be inherently task-specific, underscoring the need for systematic error analysis and categorization based on existing research.”
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944 · see citing works
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · cited 1× · see citing works
“While recent studies have attempted to establish taxonomies for hallucinations [79,80], failures may be inherently task-specific, underscoring the need for systematic error analysis and categorization based on existing research.”
Recommendations for the Extraction, Analysis, and Presentation of Results in Scoping Reviews
Pollock, D., Peters, M. D. J., Khalil, H., McInerney, P., Alexander, L., Tricco, A. C., Evans, C., de Moraes, É. B., Godfrey, C. M., Pieper, D., et al. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-22-00123 · Cited by 1,975 · see citing works
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047 · see citing works
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045 · see citing works
Addressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903 · see citing works
Adapted large language models can outperform medical experts in clinical text summarization
Van Veen, D., Van Uden, C., Blankemeier, L., Delbrouck, J.-B., Aali, A., Bluethgen, C., Pareek, A., Polacin, M., Reis, E. P., Seehofnerová, A., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02855-5 · Cited by 891 · see citing works
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 665 · see citing works
Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review
Bedi, S., Liu, Y., Orr-Ewing, L., Dash, D., Koyejo, S., Callahan, A., Fries, J. A., Wornow, M., Swaminathan, A., Lehmann, L. S., et al. (2025) · JAMA · doi:10.1001/jama.2024.21700 · Cited by 662 · see citing works
The TRIPOD-LLM reporting guideline for studies using large language models
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03425-5 · Cited by 615 · see citing works
SummEval: Re-evaluating Summarization Evaluation
Fabbri, A. R., Kryściński, W., McCann, B., Xiong, C., Socher, R., & Radev, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00373 · Cited by 587 · see citing works
Privacy-preserving artificial intelligence in healthcare: Techniques and applications
Khalid, N., Qayyum, A., Bilal, M., Al-Fuqaha, A., & Qadir, J. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106848 · Cited by 581 · see citing works
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · see citing works
The myth of generalisability in clinical research and machine learning in health care
Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., & Celi, L. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30186-2 · Cited by 467 · see citing works
Wohlin, C., Kalinowski, M., Felizardo, K. R., & Mendes, E. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.106908 · Cited by 437 · see citing works
Tam, T. Y. C., Sivarajkumar, S., Kapoor, S., Stolyar, A. V., Polanska, K., McCarthy, K. R., Osterhoudt, H., Wu, X., Visweswaran, S., Fu, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01258-7 · Cited by 424 · see citing works
Kusmaryono, I., Wijayanti, D., & Maharani, H. R. (2022) · International Journal of Educational Methodology · doi:10.12973/ijem.8.4.625 · Cited by 418 · see citing works
Benchmarking Large Language Models for News Summarization
Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00632 · Cited by 415 · see citing works
The application of large language models in medicine: A scoping review
Meng, X., Yan, X., Zhang, K., Liu, D., Cui, X., Yang, Y., Zhang, M., Cao, C., Wang, J., Wang, X., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109713 · Cited by 414 · see citing works
Science in the age of large language models
Birhane, A., Kasirzadeh, A., Leslie, D., & Wachter, S. (2023) · Nature Reviews Physics · doi:10.1038/s42254-023-00581-4 · Cited by 344 · see citing works
Ethical and regulatory challenges of large language models in medicine
Ong, J. C. L., Chang, S. Y.-H., William, W., Butte, A. J., Shah, N. H., Chew, L. S. T., Liu, N., Doshi-Velez, F., Lu, W., Savulescu, J., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00061-x · Cited by 342 · see citing works
Cabitza, F., Campagner, A., Soares, F., García de Guadiana-Romualdo, L., Challa, F., Sulejmani, A., Seghezzi, M., & Carobene, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106288 · Cited by 284 · see citing works
Evaluation framework to guide implementation of AI systems into healthcare settings
Reddy, S., Rogers, W., Makinen, V.-P., Coiera, E., Brown, P., Wenzel, M., Weicken, E., Ansari, S., Mathur, P., Casey, A., et al. (2021) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100444 · Cited by 230 · see citing works
Assessing the research landscape and clinical utility of large language models: a scoping review
Park, Y.-J., Pillai, A., Deng, J., Guo, E., Gupta, M., Paget, M., & Naugler, C. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02459-6 · Cited by 187 · see citing works
Yang, J., Soltan, A. A. S., & Clifton, D. A. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00614-9 · Cited by 181 · see citing works
Human evaluation of automatically generated text: Current trends and best practice guidelines
van der Lee, C., Albert Gatt, A., van Miltenburg, E., & Krahmer, E. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101151 · Cited by 178 · see citing works
RadBERT: Adapting Transformer-based Language Models to Radiology
Yan, A., McAuley, J., Lü, X., Du, J., Chang, E., Gentili, A., & Hsu, C. (2022) · Radiology Artificial Intelligence · doi:10.1148/ryai.210258 · Cited by 171 · see citing works
Maleki, F., Ovens, K. L., Gupta, R., Reinhold, C., Spatz, A., & Forghani, R. (2022) · Radiology Artificial Intelligence · doi:10.1148/ryai.220028 · Cited by 162 · see citing works
A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models
Zhang, H., Yu, P. S., & Zhang, J. (2025) · ACM Computing Surveys · doi:10.1145/3731445 · Cited by 159 · see citing works
Wang, H., Wu, W., Dou, Z., He, L., & Yang, L. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105173 · Cited by 156 · see citing works
Alkhalaf, M., Yu, P., Yin, M., & Deng, C. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104662 · Cited by 156 · see citing works
Chevrier, R., Foufi, V., Gaudet-Blavignac, C., Robert, A., & Lovis, C. (2019) · Journal of Medical Internet Research · doi:10.2196/13484 · Cited by 150 · see citing works
Self-citationSummarization of clinical information: A conceptual model
Feblowitz, J. C., Wright, A. T., Singh, H., Samal, L., & Sittig, D. F. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.03.008 · Cited by 124 · see citing works
Cohen, R., Elhadad, M., & Elhadad, N. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-10 · Cited by 121 · see citing works
An Iterative Optimizing Framework for Radiology Report Summarization With ChatGPT
Ma, C., Wu, Z., Wang, J., Xu, S., Wei, Y., Liu, Z., Zeng, F., Jiang, X., Guo, L., Cai, X., et al. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3364586 · Cited by 113 · see citing works
A toolbox for surfacing health equity harms and biases in large language models
Pfohl, S. R., Cole-Lewis, H., Sayres, R., Neal, D., Asiedu, M., Dieng, A., Tomasev, N., Rashid, Q. M., Azizi, S., Rostamzadeh, N., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03258-2 · Cited by 113 · see citing works
Clinical and Surgical Applications of Large Language Models: A Systematic Review
Pressman, S. M., Borna, S., Gomez-Cabello, C. A., Haider, S. A., Haider, C. R., & Forte, A. J. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13113041 · Cited by 107 · see citing works
Reddy, S. (2023) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2023.101304 · Cited by 99 · see citing works
Abstractive text summarization: State of the art, challenges, and improvements
Shakil, H., Farooq, A., & Kalita, J. K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128255 · Cited by 89 · see citing works
A survey of text summarization: Techniques, evaluation and challenges
Supriyono, S., Wibawa, A. P., Suyono, & Kurniawan, F. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100070 · Cited by 85 · see citing works
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Liu, Y., Fabbri, A. R., Liu, P., Zhao, Y., Nan, L., Han, R., Han, S., Joty, S., Wu, C.-S., Xiong, C., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.228 · Cited by 75 · see citing works
Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model.
Röösli, E., Bozkurt, S., & Hernandez-Boussard, T. (2022) · Scientific Data · doi:10.1038/s41597-021-01110-7 · Cited by 73 · see citing works
ChestXRayBERT: A Pretrained Language Model for Chest Radiology Report Summarization
Cai, X., Liu, S., Han, J., Yang, L., Liu, Z., & Liu, T. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3132724 · Cited by 72 · see citing works
de Hond, A., Leeuwenberg, T., Bartels, R., van Buchem, M., Kant, I., Moons, K. G., & van Smeden, M. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00111-0 · Cited by 53 · see citing works
Searle, T., Ibrahim, Z. M., Teo, J., & Dobson, R. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104358 · Cited by 52 · see citing works
Effective aggregation of various summarization techniques
Mehta, P., & Majumder, P. (2017) · Information Processing & Management · doi:10.1016/j.ipm.2017.11.002 · Cited by 50 · see citing works
Goldstein, A., & D, Y. S. M. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.03.022 · Cited by 48 · see citing works
(2011) · PubMed Central · Cited by 46 · see citing works
Chuang, Y.-N., Tang, R., Jiang, X., & Hu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104606 · Cited by 46 · see citing works
Goswami, J., Prajapati, K. K., Saha, A., & Saha, A. K. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111531 · Cited by 45 · see citing works
A method to automate the discharge summary hospital course for neurology patients
Hartman, V., Bapat, S. S., Weiner, M. G., Navi, B. B., Sholle, E., & Campion, T. R. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad177 · Cited by 44 · see citing works
Developing robust benchmarks for driving forward AI innovation in healthcare
Mincu, D., & Roy, S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00559-4 · Cited by 41 · see citing works
A Systematic Review of ChatGPT and Other Conversational Large Language Models in HealthcarePreprint
Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y., Clayton, E. W., Malin, B., & Yin, Z. (2024) · medRxiv · doi:10.1101/2024.04.26.24306390 · Cited by 41 · see citing works
Chen, Y.-P., Chen, Y.-P., Lin, J.-J., Huang, C., & Lai, F. (2020) · JMIR Medical Informatics · doi:10.2196/17787 · Cited by 33 · see citing works
Leveraging Summary Guidance on Medical Report Summarization
Zhu, Y., Yang, X., Wu, Y., & Zhang, W. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3304376 · Cited by 33 · see citing works
The testing of AI in medicine is a mess. Here’s how it should be done
Lenharo, M. (2024) · Nature · doi:10.1038/d41586-024-02675-0 · Cited by 31 · see citing works
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Krishna, K., Bransom, E., Kuehl, B., Iyyer, M., Dasigi, P., Cohan, A., & Lo, K. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.121 · Cited by 29 · see citing works
Application of generative language models to orthopaedic practice
Caterson, J., Ambler, O., Cereceda‐Monteoliva, N., Horner, M., Jones, A., & Poacher, A. T. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-076484 · Cited by 27 · see citing works
From Softmax to Nucleusmax: A Novel Sparse Language Model for Chinese Radiology Report Summarization
Zhao, S., Li, Q., Yang, Y., Wen, J., & Luo, W. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3596219 · Cited by 26 · see citing works
RadAdapt: Radiology Report Summarization via Lightweight Domain Adaptation of Large Language Models
Van Veen, D., Van Uden, C., Attias, M., Pareek, A., Bluethgen, C., Polacin, M., Chiu, W., Delbrouck, J.-B., Zambrano Chaves, J., Langlotz, C., et al. (2023) · The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks · doi:10.18653/v1/2023.bionlp-1.42 · Cited by 25 · see citing works
Patient Information Summarization in Clinical Settings: Scoping Review
Keszthelyi, D., Gaudet-Blavignac, C., Bjelogrlic, M., & Lovis, C. (2023) · JMIR Medical Informatics · doi:10.2196/44639 · Cited by 24 · see citing works
Self-citationChien, A., Tang, H., Jagessar, B., Chang, K.-W., Peng, N., Nael, K., & Salamon, N. (2024) · American Journal of Neuroradiology · doi:10.3174/ajnr.a8102 · Cited by 24 · see citing works
Úbeda, P. L., Martín‐Noguerol, T., Díaz-Angulo, C., & Luna, A. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105443 · Cited by 24 · see citing works
Fine-tuning the BERTSUMEXT model for Clinical Report Summarization
Vinod, P., Safar, S., Mathew, D., Venugopal, P., Joly, L. M., & George, J. (2020) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet49848.2020.9154087 · Cited by 22 · see citing works
BDKG at MEDIQA 2021: System Report for the Radiology Report Summarization Task
Dai, S., Wang, Q., Lyu, Y., & Zhu, Y. T. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.11 · Cited by 21 · see citing works
Gao, Y., Dligach, D., Miller, T. A., Xu, D., Churpek, M. M., & Afshar, M. (2022) · PubMed · doi:10.48550/arxiv.2208.08408 · Cited by 20 · see citing works
Empowering Patients in the Therapeutic Decision-Making Process: A Glance Into Behçet's Syndrome
Marinello, D., Cianni, F. D., Bianco, A. D., Mattioli, I., Sota, J., Cantarini, L., Emmi, G., Leccese, P., Lopalco, G., Mosca, M., et al. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.769870 · Cited by 19 · see citing works
iSEA: An Interactive Pipeline for Semantic Error Analysis of NLP Models
Yuan, J., Vig, J., & Rajani, N. F. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511146 · Cited by 19 · see citing works
Wu, D., & Bibault, J. (2023) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2023.109978 · Cited by 19 · see citing works
Large language models for reducing clinicians’ documentation burden
Roberts, K. E. (2024) · Nature Medicine · doi:10.1038/s41591-024-02888-w · Cited by 18 · see citing works
Medical Reports Summarization Using Text-To-Text Transformer
Helwan, A., Azar, D., & Ozsahin, D. U. (2023) · Advances in Science and Engineering Technology International Conferences (ASET) · doi:10.1109/aset56582.2023.10180671 · Cited by 17 · see citing works
Alambo, A., Banerjee, T., Thirunarayan, K., & Cajita, M. (2022) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc48229.2022.9871798 · Cited by 14 · see citing works
Learning to Summarize Chinese Radiology Findings With a Pre-Trained Encoder
Jiang, Z., Cai, X., Yang, L., Gao, D., Zhao, W., Han, J., Liu, J., Shen, D. G., & Liu, T. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3280987 · Cited by 14 · see citing works
Why Mayo Clinic Is Embracing the Cloud and What This Means for Clinicians and Researchers
Ryu, A. J., Magnuson, D. R., & Kingsley, T. C. (2021) · Mayo Clinic Proceedings Innovations Quality & Outcomes · doi:10.1016/j.mayocpiqo.2021.08.010 · Cited by 13 · see citing works
A Framework to Assess Clinical Safety and Hallucination Rates of LLMs for Medical Text SummarisationPreprint
Asgari, E., Khalil, S., Montaña-Brown, N., et al. (2024) · medRxiv · doi:10.1101/2024.09.12.24313556 · Cited by 13 · see citing works
Early prediction of sepsis using chatGPT-generated summaries and structured data
Li, Q., Ma, H., Song, D., Bai, Y., Zhao, L., & Xie, K. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18378-7 · Cited by 12 · see citing works
Optum at MEDIQA 2021: Abstractive Summarization of Radiology Reports using simple BART Finetuning
Kondadadi, R., Manchanda, S., Ngo, J. A., & McCormack, R. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.32 · Cited by 8 · see citing works
Fine-tuning BERT Models for Summarizing German Radiology Findings
Liang, S., Kades, K., Fink, M., Full, P., Weber, T., Kleesiek, J., Strube, M., & Maier-Hein, K. (2022) · Proceedings of the 4th Clinical Natural Language Processing Workshop · doi:10.18653/v1/2022.clinicalnlp-1.4 · Cited by 8 · see citing works
In the AI science boom, beware: your results are only as good as your data
Moseley, H. (2024) · Nature · doi:10.1038/d41586-024-00306-2 · Cited by 8 · see citing works
A Survey on Evaluation of Out-of-Distribution GeneralizationPreprint
Yu, H., Liu, J., Zhang, X., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.01874 · Cited by 7 · see citing works
The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-DeterminismPreprint
Song, Y., Wang, G., Li, S., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.10457 · Cited by 7 · see citing works
A Day-to-Day Approach for Automating the Hospital Course Section of the Discharge Summary.
Hartman, V., & Campion, T. R. (2022) · PubMed Central · Cited by 6 · see citing works
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Krishna, K., Bransom, E., Kuehl, B., Iyyer, M., Dasigi, P., Cohan, A., & Lo, K. (2023) · arXiv · doi:10.48550/arxiv.2301.13298 · Cited by 3
EU AI Act faces enforcement challenges
(2024) · Emerald expert briefings · doi:10.1108/oxan-db287242 · Cited by 2 · see citing works
Rad-Former: Structuring Radiology Reports using Transformers*
Ajad, A., Saini, T., & Niranjan, K. M. (2023) · International Conference on Recent Advances in Information Technology · doi:10.1109/rait57693.2023.10127096 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [18]Vaswani A Shazeer N Parmar N Attention is All you Need Advances in Neural Information Processing Systems 2017 2024-07-23 United Kingdom Curran Associates, Inc https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
- [23]Ajad A Saini T Niranjan K Rad-formertructuring radiology reports using transformers* 2023 5th International Conference on Recent Advances in Information Technology (RAIT) March 3, 2023 Dhanbad, India 1 6
- [44]Cai P Liu F Bajracharya A Sills J Kapoor A Liu W Berlowitz D Levy D Pradhan R Yu H Generation of patient after-visit summaries to support physicians 2022 Proceedings of the 29th International Conference on Computational Linguistics October 1, 2022 Gyeongju, Republic of Korea 6234 6247
- [76]Bommasani R Hudson DA Adeli E Altman R Arora S von Arx S Bernstein MS Bohg J Bosselut A Brunskill E Brynjolfsson E Buch S Card D Castellon R Chatterji N Chen A Creel K Davis JQ Demszky D Donahue C Doumbouya M Durmus E Ermon S Etchemendy J Ethayarajh K Fei-Fei L Finn C Gale T Gillespie L Goel K Goodman N On the opportunities and risks of foundation models ArXiv. Preprint posted on July 12, 2022 2021
Cited by
Sorted by citations
Implementation of large language models in electronic health records
Griot, M., Vanderdonckt, J., & Yuksel, D. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0001141 · Cited by 23 · see citing works
- Authors
- Bednarczyk, L., Reichenpfader, D., Gaudet-Blavignac, C., Ette, A. K., Zaghir, J., Zheng, Y., Bensahla, A., Bjelogrlic, M., & Lovis, C.
- Affiliations
- University Hospital of Geneva (CH)
- University of Geneva (CH)
- Corresponding author
- Bednarczyk, Lydie
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-05-15
- Volume
- 27
- Pages
- e68998
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC12123242
- OpenAlex
- W4408354096
- Cited by
- 83
- References
- 89
- Citations by year
- 2026: 68 · 2025: 15
- FWCI
- 7.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Archive ouverte UNIGE (University of Geneva) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- BFH: ARBOR
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- clinical text summarization, large language models, electronic health records, information overload, radiology report summarization, intensive care unit, abstractive summarization, transformer models, MIMIC-III, evaluation framework, clinical implementation, patient safety, clinical decision support, administrative burden reduction, failure analysis, external validation
- MeSH terms
- Large Language Models, Humans, Language, Natural Language Processing, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
