The DOI matches its Crossref record; 34 of the 36 references it cites were found in a registry or an index.
Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format
Open accessAbstract
Importance: By law, patients have immediate access to discharge notes in their medical records. Technical language and abbreviations make notes difficult to read and understand for a typical patient. Large language models (LLMs [eg, GPT-4]) have the potential to transform these notes into patient-friendly language and format. Objective: To determine whether an LLM can transform discharge summaries into a format that is more readable and understandable. Design, Setting, and Participants: This cross-sectional study evaluated a sample of the discharge summaries of adult patients discharged from the General Internal Medicine service at NYU (New York University) Langone Health from June 1 to 30, 2023. Patients discharged as deceased were excluded. All discharge summaries were processed by the LLM between July 26 and August 5, 2023. Interventions: A secure Health Insurance Portability and Accountability Act-compliant platform, Microsoft Azure OpenAI, was used to transform these discharge summaries into a patient-friendly format between July 26 and August 5, 2023. Main Outcomes and Measures: Outcomes included readability as measured by Flesch-Kincaid Grade Level and understandability using Patient Education Materials Assessment Tool (PEMAT) scores. Readability and understandability of the original discharge summaries were compared with the transformed, patient-friendly discharge summaries created through the LLM. As balancing metrics, accuracy and completeness of the patient-friendly version were measured. Results: Discharge summaries of 50 patients (31 female [62.0%] and 19 male [38.0%]) were included. The median patient age was 65.5 (IQR, 59.0-77.5) years. Mean (SD) Flesch-Kincaid Grade Level was significantly lower in the patient-friendly discharge summaries (6.2 [0.5] vs 11.0 [1.5]; P < .001). PEMAT understandability scores were significantly higher for patient-friendly discharge summaries (81% vs 13%; P < .001). Two physicians reviewed each patient-friendly discharge summary for accuracy on a 6-point scale, with 54 of 100 reviews (54.0%) giving the best possible rating of 6. Summaries were rated entirely complete in 56 reviews (56.0%). Eighteen reviews noted safety concerns, mostly involving omissions, but also several inaccurate statements (termed hallucinations). Conclusions and Relevance: The findings of this cross-sectional study of 50 discharge summaries suggest that LLMs can be used to translate discharge summaries into patient-friendly language and formats that are significantly more readable and understandable than discharge summaries as they appear in electronic health records. However, implementation will require improvements in accuracy, completeness, and safety. Given the safety concerns, initial implementation will require physician review.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, variability in output quality, including hallucinations and incomplete contextualization, necessitates human oversight.42”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [42]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738
Shoemaker, S. J., Wolf, M. S., & Brach, C. (2014) · Patient Education and Counseling · doi:10.1016/j.pec.2014.05.027 · Cited by 1,422
Interventions to Reduce 30-Day Rehospitalization: A Systematic Review
Hansen, L. O., Young, R. S., Hinami, K., Leung, A., & Williams, M. V. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00008 · Cited by 1,182
Greene, J., & Hibbard, J. H. (2011) · Journal of General Internal Medicine · doi:10.1007/s11606-011-1931-2 · Cited by 1,028
Do increases in patient activation result in improved self-management behaviors?
Hibbard, J. H., Mahoney, E. R., Stock, R., & Tusler, M. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00669.x · Cited by 906
Addressing bias in artificial intelligence in health care
Parikh, R. B., Teeple, S., & Navathe, A. S. (2019) · JAMA · doi:10.1001/jama.2019.18058 · Cited by 764
Readability Standards for Informed-Consent Forms as Compared with Actual Readability
Paasche‐Orlow, M. K., Taylor, H. A., & Brancati, F. L. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa021212 · Cited by 763
Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., Donald, R., Chang, S., Berkowitz, S., Finn, A., Jahangir, E., et al. (2023) · doi:10.21203/rs.3.rs-2566942/v1 · Cited by 598
Diamond, L. C., Izquierdo, K., Canfield, D. R., Matsoukas, K., & Gany, F. M. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-04847-5 · Cited by 591
Using ChatGPT to write patient clinic letters
Ali, S. R., Dobbs, T. D., Hutchings, H. A., & Whitaker, I. S. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00048-1 · Cited by 375
Assessing reading levels of health information: uses and limitations of flesch formula
Jindal, P., & MacDermid, J. (2017) · Education for Health · doi:10.4103/1357-6283.210517 · Cited by 275
Using Patient Portals to Improve Patient Outcomes: Systematic Review
Han, H., Gleason, K. T., Sun, C., Miller, H., Kang, S. J., Chow, S., Anderson, R., Nagy, P. G., & Bauer, T. (2019) · JMIR Human Factors · doi:10.2196/15038 · Cited by 182
AI chatbots not yet ready for clinical use.
Au Yeung, J., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1161098 · Cited by 176
Generative AI in Health Care and Liability Risks for Physicians and Safety Concerns for Patients
Duffourc, M., & Gerke, S. (2023) · JAMA · doi:10.1001/jama.2023.9630 · Cited by 132
Wolff, J. L., Darer, J. D., Berger, A. L., Clarke, D. N., Green, J. A., Stametz, R. A., Delbanco, T., & Walker, J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw108 · Cited by 124
Mac, O. A., Ayre, J., Bell, K., McCaffery, K. J., & Muscat, D. M. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.46051 · Cited by 92
Hospital Discharge Instructions: Comprehension and Compliance Among Older Adults
Albrecht, J. S., Gruber‐Baldini, A. L., Hirshon, J. M., Brown, C. H., Goldberg, R. W., Rosenberg, J. H., Comer, A. C., & Furuno, J. P. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-2956-0 · Cited by 81
Prevalence and Sources of Duplicate Information in the Electronic Medical Record
Steinkamp, J., Kantrowitz, J. J., & Airan-Javia, S. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.33348 · Cited by 74
Eppler, M., Ganjavi, C., Knudsen, J., Davis, R. J., Ayo‐Ajibola, O., Desai, A., Ramacciotti, L. S., Chen, A. T.-Y., Abreu, A. D. C., Desai, M. M., et al. (2023) · Urology Practice · doi:10.1097/upj.0000000000000428 · Cited by 66
Readability of discharge summaries: with what level of information are we dismissing our patients?
Choudhry, A. J., Baghdadi, Y. M. K., Wagie, A. E., Habermann, E. B., Heller, S. F., Jenkins, D. H., Cullinane, D. C., & Zielinski, M. D. (2015) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2015.12.005 · Cited by 64
Lye, C. T., Forman, H. P., Daniel, J. G., & Krumholz, H. M. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy065 · Cited by 59
Nayak, A., Alkaitis, M. S., Nayak, K., Nikolov, M. C., Weinfurt, K. P., & Schulman, K. A. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.2561 · Cited by 58
Bala, S., Keniston, A., & Burden, M. A. (2020) · JMIR Formative Research · doi:10.2196/16670 · Cited by 57
Unaka, N. I., Statile, A. M., Haney, J., Beck, A. F., Brady, P. W., & Jerardi, K. E. (2017) · Journal of Hospital Medicine · doi:10.12788/jhm.2688 · Cited by 57
A method to automate the discharge summary hospital course for neurology patients
Hartman, V., Bapat, S. S., Weiner, M. G., Navi, B. B., Sholle, E., & Campion, T. R. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad177 · Cited by 44
Choudhry, A. J., Younis, M., Ray-Zack, M. D., Glasgow, A. E., Haddad, N. N., Habermann, E. B., Jenkins, D. H., Heller, S. F., Schiller, H. J., & Zielinski, M. D. (2018) · Surgery · doi:10.1016/j.surg.2018.10.014 · Cited by 39
Use and Maturity of Electronic Patient Portals
Gheorghiu, B., & Hagens, S. (2017) · Studies in health technology and informatics · doi:10.3233/978-1-61499-742-9-136 · Cited by 24
Readability of Patient Discharge Instructions
Burns, S., Amobi, N., Chen, J., O’Brien, M., & Haber, L. A. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-06988-y · Cited by 21
Smolle, C., Schwarz, C., Hoffmann, M. M., Kamolz, L., Sendlhofer, G., & Brunner, G. A. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06468-3 · Cited by 16
Khoong, E. C., Sherwin, E. B., Harrison, J. D., Wheeler, M. M., Shah, S. J., Mourad, M., & Khanna, R. R. (2023) · Journal of Hospital Medicine · doi:10.1002/jhm.13172 · Cited by 12
Open Notes: New Federal Rules Promoting Open and Transparent Communication
Delbanco, T., & Wachenheim, D. (2021) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/j.jcjq.2021.02.004 · Cited by 8
“Was hab’ ich?” Makes Medical Specialist Language Understandable for Patients
Bittner, A., Bittner, J., & Jonietz, A. (2014) · Springer eBooks · doi:10.1007/978-3-319-12178-9_26 · Cited by 4
Flagged first, then by citations
2 of the 36 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Weiss, 2003, Health Literacy: A Manual for Clinicians, 51
Cited by
Sorted by citations
The TRIPOD-LLM reporting guideline for studies using large language models
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03425-5 · Cited by 615
Self-citationLarge language models in patient education: a scoping review of applications in medicine
Aydin, S., Karabacak, M., Vlachos, V., & Margetis, K. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1477898 · Cited by 318
Invasion and metastasis in cancer: molecular insights and therapeutic targets
Li, Y., Liu, F., Cai, Q., Deng, L., Ouyang, Q., Zhang, X. H.-F., & Zheng, J. (2025) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-025-02148-4 · Cited by 314
Liu, S., McCoy, A. B., & Wright, A. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf008 · Cited by 200
Vrdoljak, J., Boban, Z., Vilović, M., Kumrić, M., & Božić, J. (2025) · Healthcare · doi:10.3390/healthcare13060603 · Cited by 202
Metadata
- Authors
- Zaretsky, J., Kim, J. M., Baskharoun, S., Zhao, Y., Austrian, J., Aphinyanaphongs, Y., Gupta, R., Blecker, S. B., & Feldman, J.
- Affiliations
- NYU Langone Health (US)
- NYU Grossman Long Island School of Medicine (US)
- Corresponding author
- Zaretsky, Jonah
- ORCID
- Venue
- JAMA Network Open
- Publisher
- American Medical Association (AMA)
- Published
- 2024-03-11
- Volume
- 7
- Issue
- 3
- Pages
- e240357
- ISSN
- 2574-3805 (ISSN-L)
- PMC
- PMC10928500
- OpenAlex
- W4392643126
- Cited by
- 303
- References
- 36
- Citations by year
- 2026: 131 · 2025: 136 · 2024: 36
- FWCI
- 28.33
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JAMA Network Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Social Sciences › Linguistics and Language › Text Readability and Simplification
- Keywords
- discharge summaries, large language models, generative AI, readability, comprehensibility, patient education materials, clinical documentation, GPT-4, hallucinations, general internal medicine, patient access to medical records, health information literacy, Flesch-Kincaid Grade Level, PEMAT, natural language processing of clinical notes
- MeSH terms
- Adult, Aged, Artificial Intelligence, Cross-Sectional Studies, Female, Humans, Inpatients, Language, Male, Middle Aged, Patient Discharge, United States, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
