A cross-sectional study on ChatGPT’s alignment with clinical practice guidelines in musculoskeletal rehabilitation
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
BACKGROUND: AI models like ChatGPT have the potential to support musculoskeletal rehabilitation by providing clinical insights. However, their alignment with evidence-based guidelines needs evaluation before integration into physiotherapy practice. OBJECTIVE: To evaluate the performance of ChatGPT (GPT-4 model) in generating responses to musculoskeletal rehabilitation queries by comparing its recommendations with evidence-based clinical practice guidelines (CPGs). DESIGN: This study was designed as a cross-sectional observational study. METHODS: Twenty questions covering disease information, assessment, and rehabilitation were developed by two experienced physiotherapists specializing in musculoskeletal disorders. The questions were distributed across three anatomical regions: upper extremity (7 questions), lower extremity (9 questions), and spine (4 questions). ChatGPT's responses were obtained and evaluated independently by two raters using a 5-point Likert scale assessing relevance, accuracy, clarity, completeness, and consistency. Weighted kappa values were calculated to assess inter-rater agreement and consistency within each category. RESULTS: ChatGPT's responses received the highest average score for clarity (4.85), followed by accuracy (4.62), relevance (4.50), and completeness (4.20). Consistency received the lowest score (3.85). The highest agreement (weighted kappa = 0.90) was observed in the disease information category, whereas rehabilitation displayed relatively lower agreement (weighted kappa = 0.56). Variability in consistency and moderate weighted kappa values in relevance and clarity highlighted areas requiring improvement. CONCLUSIONS: This study demonstrates ChatGPT's potential in providing guideline-aligned information in musculoskeletal rehabilitation. However, due to observed limitations in consistency, completeness, and the ability to replicate nuanced clinical reasoning, its use should remain supplementary rather than as a primary decision-making tool. While it performed better in disease information, as evidenced by higher inter-rater agreement and scores, its performance in the rehabilitation category was comparatively lower, highlighting challenges in addressing complex, nuanced therapeutic interventions. This variability in consistency and domain-specific reasoning underscores the need for further refinement to ensure reliability in complex clinical scenarios. CLINICAL TRIAL NUMBER: Not applicable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Retracted: Implementing AI-driven Diagnostic Decision Support Systems for Smart Healthcare
Neravetla, A. R., Nomula, V. K., Mohammed, A. S., & Dhanasekaran, S. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10725323 · Cited by 16
RetractedChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
Lin, I., Wiles, L., Waller, R., Goucke, R., Nagree, Y., Gibberd, M., Straker, L., Maher, C. G., & O’Sullivan, P. P. B. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099878 · Cited by 1,001
Blanpied, P. R., Gross, A. R., Elliott, J. M., Devaney, L. L., Clewley, D., Walton, D. M., Sparks, C., & Robertson, E. K. (2017) · Journal of Orthopaedic & Sports Physical Therapy · doi:10.2519/jospt.2017.0302 · Cited by 979
Kreiner, D. S., Hwang, S. W., Easa, J. E., Resnick, D. K., Baisden, J. L., Bess, S., Cho, C. H., DePalma, M. J., Dougherty, P., Fernand, R., et al. (2014) · The Spine Journal · doi:10.1016/j.spinee.2013.08.003 · Cited by 724
Role of Chat GPT in public health
Biswas, S. S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03172-7 · Cited by 673 · cited 1×
“Among AI applications, natural language processing (NLP) models such as Chat-GPT have garnered significant attention due to their ability to generate human-like responses to complex medical inquiries [3, 4].”
George, S. Z., Fritz, J. M., Silfies, S. P., Schneider, M. J., Beneciuk, J. M., Lentz, T. A., Gilliam, J. R., Hendren, S., & Norman, K. S. (2021) · Journal of Orthopaedic & Sports Physical Therapy · doi:10.2519/jospt.2021.0304 · Cited by 607
Kelley, M. J., Shaffer, M. A., Kuhn, J. E., Michener, L. A., Seitz, A. L., Uhl, T. L., Godges, J. J., & McClure, P. (2013) · Journal of Orthopaedic & Sports Physical Therapy · doi:10.2519/jospt.2013.0302 · Cited by 468
Willy, R. W., Hoglund, L. T., Barton, C. J., Bolgla, L. A., Scalzitti, D. A., Logerstedt, D. S., Lynch, A. D., Snyder-Mackler, L., & McDonough, C. M. (2019) · Journal of Orthopaedic & Sports Physical Therapy · doi:10.2519/jospt.2019.0302 · Cited by 418
Kotsifaki, R., Korakakis, V., King, E., Barbosa, O., Maree, D., Pantouveris, M., Bjerregaard, A., Luomajoki, J., Wilhelmsen, J., & Whiteley, R. (2023) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-106158 · Cited by 303
Research works printed but not linked: 1
- Jain K, Prajapati V. NLP/deep learning techniques in healthcare for decision making. Prim Health Care: Open Access. 2021;11(3):373–80.
Cited by
Ngo, K., Mekhail, S., Chan, V., Li, X., Yin, A., Choi, H. Y., Allman-Farinelli, M. A., & Chen, J. (2025) · Nutrients · doi:10.3390/nu17223515 · Cited by 19
Arzu, U., & Gencer, B. (2025) · Diagnostics · doi:10.3390/diagnostics15141834 · Cited by 9
AI-Powered Physiotherapy: Evaluating LLMs Against Students in Clinical Rehabilitation Scenarios
Michou, I., Fouras, A., Chrysanthakopoulou, D., Theodoritsi, M., Stellatou, S., Mariettou, S., & Koutsojannis, C. M. (2026) · Applied Sciences · doi:10.3390/app16031165 · Cited by 8
Safran, E., & Çalı, A. (2025) · Anatolian Current Medical Journal · doi:10.38053/acmj.1746227 · Cited by 4
Self-citationFairley, J. L., Kapoor, M., & Sharma, D. (2026) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2026.03.001 · Cited by 4
Metadata
- Authors
- Safran, E., & Yildirim, S.
- Affiliations
- Bezmiâlem Vakıf Üniversitesi (TR)
- Corresponding author
- Safran, Ertuğrul
- Venue
- BMC Musculoskeletal Disorders
- Published
- 2025-04-24
- Volume
- 26
- Issue
- 1
- Pages
- 411-411
- ISSN
- 1471-2474 (ISSN-L)
- PMC
- PMC12023614
- OpenAlex
- W4409737737
- Cited by
- 21
- References
- 35
- Citations by year
- 2026: 15 · 2025: 6
- FWCI
- 5.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- BMC Musculoskeletal Disorders (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Medicine › Rehabilitation › Musculoskeletal Disorders and Rehabilitation
- Keywords
- ChatGPT, large language models, musculoskeletal rehabilitation, physiotherapy, guideline alignment, upper limb, lower extremity, spine, interobserver agreement, clinical reasoning
- MeSH terms
- Generative Artificial Intelligence, Cross-Sectional Studies, Humans, Musculoskeletal Diseases, Practice Guidelines as Topic, Physical Therapy Modalities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A cross-sectional study on ChatGPT’s alignment with clinical practice guidelines in musculoskeletal rehabilitation. https://app.cochat.ai/papers/10.1186/s12891-025-08650-8
- The paper
- Safran, E., & Yildirim, S. (2025). A cross-sectional study on ChatGPT’s alignment with clinical practice guidelines in musculoskeletal rehabilitation. BMC Musculoskeletal Disorders. https://doi.org/10.1186/s12891-025-08650-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.