Cited by 208Health Literacy and Information AccessibilityText Readability and SimplificationArtificial Intelligence in Healthcare and Education

Can Artificial Intelligence Improve the Readability of Patient Education Materials?

Open access
Kirchner, Gregory J., Kim, Raymond Y., Weddle, John B., Bible, Jesse E.·2023·Clinical Orthopaedics & Related Research

Abstract

Background: The recommended readability of online health education materials is at or below the sixth- to eighth-grade level. Nevertheless, more than a decade of research has demonstrated that most online education materials pertaining to orthopaedic surgery do not meet these recommendations. The repeated evidence of this limited progress underscores that unaddressed barriers exist to improving readability, such as the added time and cost associated with writing easily readable materials that cover complex topics. Freely available artificial intelligence (AI) platforms might facilitate the conversion of patient-education materials at scale, but to our knowledge, this has not been evaluated in orthopaedic surgery. Questions/purposes: (1) Can a freely available AI dialogue platform rewrite orthopaedic patient education materials to reduce the required reading skill level from the high-school level to the sixth-grade level (which is approximately the median reading level in the United States)? (2) Were the converted materials accurate, and did they retain sufficient content detail to be informative as education materials for patients? Methods: Descriptions of lumbar disc herniation, scoliosis, and spinal stenosis, as well as TKA and THA, were identified from educational materials published online by orthopaedic surgery specialty organizations and leading orthopaedic institutions. The descriptions were entered into an AI dialogue platform with the prompt “translate to fifth-grade reading level” to convert each group of text at or below the sixth-grade reading level. The fifth-grade reading level was selected to account for potential variation in how readability is defined by the AI platform, given that there are several widely used preexisting methods for defining readability levels. The Flesch Reading Ease score and Flesch-Kincaid grade level were determined for each description before and after AI conversion. The time to convert was also recorded. Each education material and its respective conversion was reviewed for factual inaccuracies, and each conversion was reviewed for its retention of sufficient detail for intended use as a patient education document. Results: As presented to the public, the current descriptions of herniated lumbar disc, scoliosis, and stenosis had median (range) Flesch-Kincaid grade levels of 9.5 (9.1 to 10.5), 12.6 (10.8 to 15), and 10.9 (8 to 13.6), respectively. After conversion by the AI dialogue platform, the median Flesch-Kincaid grade level scores for herniated lumbar disc, scoliosis, and stenosis were 5.0 (3.3 to 8.2), 5.6 (4.1 to 7.3), and 6.9 (5 to 7.8), respectively. Similarly, descriptions of TKA and THA improved from 12.0 (11.2 to 13.5) to 6.3 (5.8 to 7.6) and from 11.6 (9.5 to 12.6) to 6.1 (5.4 to 7.1), respectively. The Flesch Reading Ease scores followed a similar trend. Seconds per sentence conversion was median 4.5 (3.3 to 4.9) and 4.5 (3.5 to 4.8) for spine conditions and arthroplasty, respectively. Evaluation of the materials that were converted for ease of reading still provided a sufficient level of nuance for patient education, and no factual errors or inaccuracies were identified. Conclusion: We found that a freely available AI dialogue platform can improve the reading accessibility of orthopaedic surgery online patient education materials to recommended levels quickly and effectively. Professional organizations and practices should determine whether their patient education materials exceed current recommended reading levels by using widely available measurement tools, and then apply an AI dialogue platform to facilitate converting their materials to more accessible levels if needed. Additional research is needed to determine whether this technology can be applied to additional materials meant to inform patients, such as surgical consent documents or postoperative instructions, and whether the methods presented here are applicable to non–English language materials.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1097/corr.0000000000002668

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • By equipping patients with knowledge, ChatGPT plays a vital role in facilitating informed decision-making, as patients can weigh the benefits and risks more confidently.

    “By equipping patients with this knowledge, ChatGPT plays a vital role in facilitating informed decision-making, as patients can weigh the benefits and risks more confidently.”
  • The empathetic and reassuring nature of ChatGPT interactions can help alleviate patient anxiety, foster trust, and strengthen doctor-patient relationships.

    “Furthermore, the empathetic and reassuring nature of ChatGPT interactions can help alleviate patient anxiety, foster trust, and strengthen doctor-patient relationships.28)”
  • During orthopedic procedures, ChatGPT can act as a virtual surgical assistant by processing real-time data from various sources including surgical instruments and patient vital signs.

    “During orthopedic procedures, ChatGPT can act as a virtual surgical assistant by processing real-time data from various sources including surgical instruments and patient vital signs.”
  • ChatGPT can monitor patient vital signs, detect anomalies or deviations from the norm, and promptly alert the surgical team to potential complications.

    “Simultaneously, ChatGPT can monitor patient vital signs, detect anomalies or deviations from the norm, and promptly alert the surgical team to potential complications.”
  • This finding affirmed the potential of AI as a reliable tool in both diagnosis and treatment.

    “This finding affirmed the potential of AI as a reliable tool in both diagnosis and treatment.2831)”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Kirchner, G. J., Kim, R. Y., Weddle, J. B., & Bible, J. E.
Affiliations
Penn State Milton S. Hershey Medical Center (US)
Venue
Clinical Orthopaedics & Related Research
Published
2023-04-28
Volume
481
Issue
11
Pages
2260-2267
ISSN
  • 0009-921X (ISSN-L)
  • 1528-1132
OpenAlex
W4367307887
Cited by
208
References
27
Citations by year
2026: 71 · 2025: 74 · 2024: 53 · 2023: 10
FWCI
72.21
Type
journal article
Language
en
Open access
Locations
  • Clinical Orthopaedics and Related Research (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
  • Physical Sciences › Computer Science › Artificial Intelligence › Text Readability and Simplification
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
readability, patient education materials, artificial intelligence, AI-generated text, orthopedic surgery, plain language, Flesch-Kincaid Grade Level, Flesch Reading Ease, total knee arthroplasty, total hip arthroplasty, lumbar disc herniation, scoliosis, spinal stenosis, online health information, factual accuracy, conversion time
MeSH terms
Artificial Intelligence, Constriction, Pathologic, Humans, Patient Education as Topic, Scoliosis, United States, Internet, Comprehension, Health Literacy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review