CoChat
Sign Up
Cited by 12Artificial Intelligence in Healthcare and EducationHealth Literacy and Information AccessibilitySocial Media in Health Education

ChatGPT is a comprehensive education tool for patients with patellar tendinopathy, but it currently lacks accuracy and readability

Open access
Deng, Jie, Li, Lun, Oosterhof, Jelle J., Malliaras, Peter, Silbernagel, Karin Grävare, Breda, Stephan J., Eygendaal, Denise, Oei, Edwin HG. +1 more·2025·Musculoskeletal Science and Practice
DOI 10.1016/j.msksp.2025.103275
Open this paper in CoChat — free account

Abstract

BACKGROUND: Generative artificial intelligence tools, such as ChatGPT, are becoming increasingly integrated into daily life, and patients might turn to this tool to seek medical information. OBJECTIVE: To evaluate the performance of ChatGPT-4 in responding to patient-centered queries for patellar tendinopathy (PT). METHODS: Forty-eight patient-centered queries were collected from online sources, PT patients, and experts and were then submitted to ChatGPT-4. Three board-certified experts independently assessed the accuracy and comprehensiveness of the responses. Readability was measured using the Flesch-Kincaid Grade Level (FKGL: higher scores indicate a higher grade reading level). The Patient Education Materials Assessment Tool (PEMAT) evaluated understandability, and actionability (0-100%, higher scores indicate information with clearer messages and more identifiable actions). Semantic Textual Similarity (STS score, 0-1; higher scores indicate higher similarity) assessed variation in the meaning of texts over two months (including ChatGPT-4o) and for different terminologies related to PT. RESULTS: Sixteen (33%) of the 48 responses were rated accurate, while 36 (75%) were rated comprehensive. Only 17% of treatment-related questions received accurate responses. Most responses were written at a college reading level (median and interquartile range [IQR] of FKGL score: 15.4 [14.4-16.6]). The median of PEMAT for understandability was 83% (IQR: 70%-92%), and for actionability, it was 60% (IQR: 40%-60%). The medians of STS scores in the meaning of texts over two months and across terminologies were all ≥ 0.9. CONCLUSIONS: ChatGPT-4 provided generally comprehensive information in response to patient-centered queries but lacked accuracy and was difficult to read for individuals below a college reading level.

Integrity signals

Registry

Crossref · 10.1016/j.msksp.2025.103275

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

1 of the 36 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Deng, J., Li, L., Oosterhof, J. J., Malliaras, P., Silbernagel, K. G., Breda, S. J., Eygendaal, D., Oei, E. H., & de Vos, R.-J.
Affiliations
  • Erasmus MC (NL)
  • University of Groningen (NL)
  • Stichting Haaglanden Medisch Centrum (NL)
  • Monash University (AU)
  • University of Delaware (US)
Corresponding author
Deng, Jie
ORCID
Venue
Musculoskeletal Science and Practice
Publisher
Elsevier BV
Published
2025-04
Volume
76
Pages
103275
ISSN
2468-7812 (ISSN-L)
DOI
10.1016/j.msksp.2025.103275
OpenAlex
W4407025693
Cited by
12
References
36
Citations by year
2026: 6 · 2025: 6
FWCI
2.87
Type
journal article
Language
en
Open access
Licence
Locations
  • Musculoskeletal Science and Practice (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, cc-by)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
  • Social Sciences › Health › Social Media in Health Education
Keywords
ChatGPT-4, patellar tendinopathy, generative AI, patient education, health information accuracy, readability, semantic textual similarity, PEMAT, GPT-4o
MeSH terms
Generative Artificial Intelligence, Adult, Female, Humans, Male, Middle Aged, Patient Education as Topic, Patellar Ligament, Comprehension, Tendinopathy, Health Literacy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review