Can Artificial Intelligence Improve the Readability of Patient Education Materials on Aortic Stenosis? A Pilot Study
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
INTRODUCTION: The advent of generative artificial intelligence (AI) dialogue platforms and large language models (LLMs) may help facilitate ongoing efforts to improve health literacy. Additionally, recent studies have highlighted inadequate health literacy among patients with cardiac disease. The aim of the present study was to ascertain whether two freely available generative AI dialogue platforms could rewrite online aortic stenosis (AS) patient education materials (PEMs) to meet recommended reading skill levels for the public. METHODS: Online PEMs were gathered from a professional cardiothoracic surgical society and academic institutions in the USA. PEMs were then inputted into two AI-powered LLMs, ChatGPT-3.5 and Bard, with the prompt "translate to 5th-grade reading level". Readability of PEMs before and after AI conversion was measured using the validated Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook Index (SMOGI), and Gunning-Fog Index (GFI) scores. RESULTS: Overall, 21 PEMs on AS were gathered. Original readability measures indicated difficult readability at the 10th-12th grade reading level. ChatGPT-3.5 successfully improved readability across all four measures (p < 0.001) to the approximately 6th-7th grade reading level. Bard successfully improved readability across all measures (p < 0.001) except for SMOGI (p = 0.729) to the approximately 8th-9th grade level. Neither platform generated PEMs written below the recommended 6th-grade reading level. ChatGPT-3.5 demonstrated significantly more favorable post-conversion readability scores, percentage change in readability scores, and conversion time compared to Bard (all p < 0.001). CONCLUSION: AI dialogue platforms can enhance the readability of PEMs for patients with AS but may not fully meet recommended reading skill levels, highlighting potential tools to help strengthen cardiac health literacy in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Flesch, R. (1948) · Journal of Applied Psychology · doi:10.1037/h0057532 · Cited by 5,391 · cited 2×
“…English/6th-grade level; 80–70, fairly easy to read/7th-grade level; 70–60, plain English/8th-to 9th-grade level; 60–50, fairly difficult/10th-to 12th-grade level; 50–30, difficult/college level; 30–10, very difficult/college graduate level; 10–0, extremely difficult/professional level) [15].”
MethodKincaid, J. P., Fishburne, Jr., Robert P., R., Richard L., C., & Brad, S. (1975) · doi:10.21236/ada006655 · Cited by 3,497
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“5 and Bard have been utilized throughout medical education, research, and practice, with over 50 recent studies describing the great potential of these emerging platforms in healthcare [20].”
BackgroundCreation and Adoption of Large Language Models in Medicine
Shah, N. H., Entwistle, D., & Pfeffer, M. A. (2023) · JAMA · doi:10.1001/jama.2023.14217 · Cited by 435
Shahsavar, Y., & Choudhury, A. (2023) · JMIR Human Factors · doi:10.2196/47564 · Cited by 341 · cited 1×
“Others have similarly questioned the trustworthiness and overall appropriateness of AI-generated responses to medical questions posed by patients [31, 32].”
BackgroundReadability of online health information: A meta-narrative systematic review
Daraz, L., Morrow, A. S., Ponce, O. J., Farah, W., Katabi, A., Majzoub, A., Seisa, M. O., Benkhadra, R., Alsawas, M., Larry, P., et al. (2018) · American Journal of Medical Quality · doi:10.1177/1062860617751639 · Cited by 278
Can Artificial Intelligence Improve the Readability of Patient Education Materials?
Kirchner, G. J., Kim, R. Y., Weddle, J. B., & Bible, J. E. (2023) · Clinical Orthopaedics & Related Research · doi:10.1097/corr.0000000000002668 · Cited by 208
Li, R., Kumar, A., & Chen, J. H. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1835 · Cited by 104
On the cusp: considering the impact of artificial intelligence language models in healthcare
Goodman, R. S., Patrinely, J. R., Osterman, T., Wheless, L., & Johnson, D. B. (2023) · Med · doi:10.1016/j.medj.2023.02.008 · Cited by 100
Leonard Grabeel, K., Russomanno, J., Oelschlegel, S., Tester, E., & Heidel, R. E. (2018) · Journal of the Medical Library Association · doi:10.5195/jmla.2018.262 · Cited by 96
Cited by
Large language models in patient education: a scoping review of applications in medicine
Aydin, S., Karabacak, M., Vlachos, V., & Margetis, K. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1477898 · Cited by 288
Thomae, A. V., Witt, C. M., & Barth, J. (2024) · JMIR Medical Education · doi:10.2196/50545 · Cited by 36
Assessing the Capability of Large Language Model Chatbots in Generating Plain Language Summaries
Mondal, H., Gupta, G., Sarangi, P. K., Sharma, S., Choudhary, P. K., Juhi, A., Kumari, A., & Mondal, S. (2025) · Cureus · doi:10.7759/cureus.80976 · Cited by 14
Leon, M., Ruaengsri, C., Pelletier, G., Bethencourt, D., Shibata, M., Flores, M. Q., & Shudo, Y. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13216543 · Cited by 12
Deng, J., Li, L., Oosterhof, J. J., Malliaras, P., Silbernagel, K. G., Breda, S. J., Eygendaal, D., Oei, E. H., & de Vos, R.-J. (2025) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2025.103275 · Cited by 12
Metadata
- Authors
- Rouhi, A. D., Ghanem, Y. K., Yolchieva, L., Saleh, Z., Joshi, H., Moccia, M. C., Suarez-Pierre, A., & Han, J. J.
- Venue
- Cardiology and Therapy
- Published
- 2024-01-09
- Volume
- 13
- Issue
- 1
- Pages
- 137-147
- ISSN
- 2193-6544 (ISSN-L)
- 2193-8261
- OpenAlex
- W4390771646
- Cited by
- 117
- References
- 10
- FWCI
- 63.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Can Artificial Intelligence Improve the Readability of Patient Education Materials on Aortic Stenosis? A Pilot Study. https://app.cochat.ai/papers/10.1007/s40119-023-00347-0
- The paper
- Rouhi, A. D., Ghanem, Y. K., Yolchieva, L., Saleh, Z., Joshi, H., Moccia, M. C., Suarez-Pierre, A., & Han, J. J. (2024). Can Artificial Intelligence Improve the Readability of Patient Education Materials on Aortic Stenosis? A Pilot Study. Cardiology and Therapy. https://doi.org/10.1007/s40119-023-00347-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.