Exploring the Potential of ChatGPT as a Supplementary Tool for Providing Orthopaedic Information
Open accessAbstract
PURPOSE: To investigate the potential use of large language models (LLMs) in orthopaedics by presenting queries pertinent to anterior cruciate ligament (ACL) surgery to generative pre-trained transformer (ChatGPT, specifically using its GPT-4 model of March 14th 2023). Additionally, this study aimed to evaluate the depth of the LLM's knowledge and investigate its adaptability to different user groups. It was hypothesized that the ChatGPT would be able to adapt to different target groups due to its strong language understanding and processing capabilities. METHODS: ChatGPT was presented with 20 questions and response was requested for two distinct target audiences: patients and non-orthopaedic medical doctors. Two board-certified orthopaedic sports medicine surgeons and two expert orthopaedic sports medicine surgeons independently evaluated the responses generated by ChatGPT. Mean correctness, completeness, and adaptability to the target audiences (patients and non-orthopaedic medical doctors) were determined. A three-point response scale facilitated nuanced assessment. RESULTS: ChatGPT exhibited fair accuracy, with average correctness scores of 1.69 and 1.66 (on a scale from 0, incorrect, 1, partially correct, to 2, correct) for patients and medical doctors, respectively. Three of the 20 questions (15.0%) were deemed incorrect by any of the four orthopaedic sports medicine surgeon assessors. Moreover, overall completeness was calculated to be 1.51 and 1.64 for patients and medical doctors, respectively, while overall adaptiveness was determined to be 1.75 and 1.73 for patients and doctors, respectively. CONCLUSION: Overall, ChatGPT was successful in generating correct responses in approximately 65% of the cases related to ACL surgery. The findings of this study imply that LLMs offer potential as a supplementary tool for acquiring orthopaedic knowledge. However, although ChatGPT can provide guidance and effectively adapt to diverse target audiences, it cannot supplant the expertise of orthopaedic sports medicine surgeons in diagnostic and treatment planning endeavours due to its limited understanding of orthopaedic domains and its potential for erroneous responses. LEVEL OF EVIDENCE: V.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ChatGPT provides rapid updates, preventive measures, and medical guidelines to the public during health crises.
“Specific advantages include its ability to provide rapid updates, preventive measures, and medical guidelines to the public.”
ChatGPT plays a crucial role in pandemic management by aiding in the swift and accurate dissemination of critical information.
“ChatGPT plays a crucial role in pandemic management by aiding in the swift and accurate dissemination of critical information.”
ChatGPT's natural language processing capabilities enable it to generate coherent responses, making it a valuable tool for healthcare organisations.
“Its natural language processing capabilities enable it to generate coherent responses, making it a valuable tool for healthcare organisations.”
ChatGPT can offer standardised and accurate responses that prepare patients for consultations and help them make informed decisions.
“ChatGPT can offer standardised and accurate responses, which prepare patients for their consultations and help them make informed decisions [5,6,7,8,9,10,42,43].”
ChatGPT can function as a virtual assistant, offering patients crucial preoperative information for cosmetic orthognathic surgery.
“Within the realm of surgical procedures, such as cosmetic orthognathic surgery, ChatGPT can function as a virtual assistant, offering patients crucial preoperative information.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 21 · Cited by 117
- Authors
- Kaarre, J., Feldt, R., Keeling, L. E., Dadoo, S., Zsidai, B., Hughes, J. D., Samuelsson, K., & Musahl, V.
- Affiliations
- University of Pittsburgh (US)
- Chalmers University of Technology (SE)
- University of Gothenburg (SE)
- Sahlgrenska University Hospital (SE)
- Corresponding author
- Kaarre, Janina
- ORCID
- Venue
- Knee Surgery Sports Traumatology Arthroscopy
- Publisher
- Wiley
- Published
- 2023-08-08
- Volume
- 31
- Issue
- 11
- Pages
- 5190-5198
- ISSN
- 0942-2056 (ISSN-L)
- 1433-7347
- PMC
- PMC10598178
- OpenAlex
- W4385667643
- Cited by
- 117
- References
- 21
- Citations by year
- 2026: 23 · 2025: 43 · 2024: 46 · 2023: 5
- FWCI
- 4.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Knee Surgery Sports Traumatology Arthroscopy (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Chalmers Research (Chalmers University of Technology) (submittedVersion, other-oa)
- Funders
- University of Gothenburg
- Göteborgs Universitet
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- ChatGPT, large language models, anterior cruciate ligament surgery, patient education, medical education, response accuracy, orthopaedic sports medicine, AI-generated medical information
- MeSH terms
- Orthopedic Surgeons, Humans, Language, Orthopedics, Anterior Cruciate Ligament, Orthopedic Procedures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
