Improving Patient Engagement: Is There a Role for Large Language Models?
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs are being explored for utility in spine surgery research and patient education.
“LLMs are being explored for their utility in spine surgery research and patient education, offering to simplify complex medical concepts for patients trying to understand their own conditions, improving healthcare literacy.7,8”
From Large Language Model Hallucinations in Spine Surgery: A Comparative Analysis of Clinician vs Patient-Level Prompts (2026) · cites this paper as [7]
The “Accessible LLMs as a Parallel to Patient Engagement” section discusses patients using LLMs to ask about their care, summarize information, and formulate questions for clinicians. The paper focuses on radiation oncology and patient engagement, however; it does not discuss spine surgery research or establish that LLMs improve healthcare literacy by simplifying complex concepts. (LLM, conf=0.86)
Experimental — accuracy not yet measuredPatients could be misled by fabricated citations or directed to nonscientific sources.
“A patient seeking to be well-informed about their spinal condition could be misled by fabricated citations or directed to nonscientific sources.7”
From Large Language Model Hallucinations in Spine Surgery: A Comparative Analysis of Clinician vs Patient-Level Prompts (2026) · cites this paper as [7]
The “Accessible LLMs as a Parallel to Patient Engagement” section says LLMs can give patients authoritative-sounding but incorrect answers and relay internet misinformation. The “Scalable Engagement” section also warns about hallucinations. Neither section specifically discusses fabricated citations, directing patients to nonscientific sources, or spinal conditions. (LLM, conf=0.85)
Experimental — accuracy not yet measuredLLMs tend to present fabrications with an authoritative tone, mixing them with factual information, making errors difficult to detect.
“This risk is amplified by the tendency for LLMs to present these fabrications with an authoritative tone, mixing them with factual information, in a way that makes errors difficult to detect, presenting a potential threat to patient safety.7,11”
From Large Language Model Hallucinations in Spine Surgery: A Comparative Analysis of Clinician vs Patient-Level Prompts (2026) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Large language models to identify social determinants of health in electronic health records
Guevara, M., Chen, S., Thomas, S., Chaunzwa, T. L., Franco, I., Kann, B. H., Moningi, S., Qian, J. M., Goldstein, M., Harper, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00970-0 · Cited by 278 · see citing works
Self-citationThe effect of using a large language model to respond to patient messages
Chen, S., Guevara, M., Moningi, S., Hoebers, F., Elhalawani, H., Kann, B. H., Chipidza, F. E., Leeman, J., Aerts, H. J. W. L., Miller, T., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00060-8 · Cited by 147 · see citing works
Self-citationToward Clinical-Grade Evaluation of Large Language Models
Moreno, A. C., & Bitterman, D. S. (2024) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2023.11.012 · Cited by 28 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 13
- [1]Kinney AP, Sankaranarayanan B. Effects of Patient Portal Use on Patient Satisfaction: Survey and Partial Least Squares Analysis. J Med Internet Res 2021; 23(8): e19820. 34448712 PMC8433860
- [2]Richwine C, Johnson C, Patel V. Disparities in patient portal access and the role of providers in encouraging access and use. J Am Med Inform Assoc 2023; 30(2): 308– 317. 36451262 PMC9846679
- [3]Sadasivaiah S, Lyles CR, Kiyoi S, Wong P, Ratanawongsa N. Disparities in patient-reported interest in Web-based patient portals: Survey at an urban academic safety-net hospital. J Med Internet Res 2019; 21(3): e11421. 30912747 PMC6454341
- [4]Murphy DR, Meyer AND, Russo E, Sittig DF, Wei L, Singh H. The Burden of Inbox Notifications in Commercial Electronic Health Records. JAMA Intern Med 2016; 176(4): 559– 560. 26974737 PMC4860883
- [5]Lieu TA, Altschuler A, Weiner JZ, Primary care physicians’ experiences with and strategies for managing electronic messages. JAMA Netw Open. 2019; 2(12): e1918287. 31880798 PMC6991215
- [6]Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: Time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27(4): 531– 538. 32016375 PMC7647261
- [7]Hilliard RW, Haskell J, Gardner RL. Are specific elements of electronic health record use associated with clinician burnout more than others? J Am Med Inform Assoc 2020; 27(9): 1401– 1410. 32719859 PMC7647296
- [8]Rittenberg E, Liebman JB, Rexrode KM. Primary Care Physician Gender and Electronic Health Record Workload. J Gen Intern Med 2022; 37(13): 3295– 3301. 34993875 PMC9550938
- [9]Garcia P, Ma SP, Shah S, Artificial Intelligence-Generated Draft Replies to Patient Inbox Messages. JAMA Netw Open. 2024; 7(3): e243201. 38506805 PMC10955355
- [13]Chen S, Gallifant J, Gao M, Cross-Care: Assessing the healthcare implications of pre-training data on language model bias. arXiv [csCL]. Published online May 8, 2024. Accessed June 2, 2024. http://arxiv.org/abs/2405.05506 PMC12045606 40313683 · arXiv
- [14]Zack T, Lehman E, Suzgun M, Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health. 2024; 6(1): e12– e22. 38123252
- [16]Tai-Seale M, Baxter SL, Vaida F, AI-Generated Draft Replies Integrated Into Health Records and Physicians’ Electronic Communication. JAMA Netw Open. 2024; 7(4): e246565. 38619840 PMC11019394
- [18]Chen S, Kann BH, Foote MB, Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol 2023; 9(10): 1459– 1462. 37615976 PMC10450584
Cited by
Sorted by citations
McLaughlin, N. D., Srinivas, A. N., Lowe, Z. F., Botterbush, K. S., Patel, M. S., & Avila, M. J. (2026) · Neurosurgery Practice · doi:10.1227/neuprac.0000000000000244 · Cited by 1 · see citing works
“7,11 The focus on citation generation is therefore essential.”
- Authors
- Kouzy, R., & Bitterman, D. S.
- Affiliations
- The University of Texas MD Anderson Cancer Center (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Bitterman, Danielle S.
- Venue
- International Journal of Radiation Oncology*Biology*Physics
- Publisher
- Elsevier BV
- Published
- 2024-11
- Volume
- 120
- Issue
- 3
- Pages
- 639-641
- ISSN
- 0360-3016 (ISSN-L)
- PMC
- PMC12235655
- OpenAlex
- W4402757258
- Cited by
- 3
- References
- 3
- Citations by year
- 2026: 1 · 2025: 2
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Radiation Oncology*Biology*Physics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Cancer Institute (U54CA274516)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › General Health Professions › Patient-Provider Communication in Healthcare
- Keywords
- patient engagement, large language models, patient portal messaging, radiation oncology, electronic health records, patient-provider communication
- MeSH terms
- Humans, Language, Patient Participation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
