The use of ChatGPT and other large language models in surgical science
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
ChatGPT, a large language model (LLM), was released on November 30, 2022. This model can generate seemingly intelligent writing by comprehensively answering prompts after being trained on vast amounts of text data1. Its disruptive potential quickly caught the public’s attention, leading to many using it for writing tasks. As a result, media outlets have provided extensive coverage of ChatGPT and other LLMs, sparking discussions about their potential uses and controversies2,3. While some view LLMs as valuable tools that could revolutionize science, others express scepticism and concern. However, their potential impact on surgical science remains unexplored. In this article, the potential role of LLMs like ChatGPT in surgical science, from the authors’ perspective as surgical scientists involved in both clinical trials and artificial intelligence-based surgical research, will be examined. ChatGPT and similar models are a type of artificial intelligence that are designed to understand and generate natural language text. They belong to the category of transformer-based neural networks and are trained on a vast amount of text data, including books, articles, and websites. During the training process, the model analyses the text to learn the structure and patterns of the language. Once trained, the model can be used to generate new text that is comparable in style and content to the training data, based on the given prompts. There are several ways that LLMs can be applied in surgical science. One of the most noteworthy is for writing tasks, which can enhance the productivity and efficiency of surgeon-scientists and editors. Additionally, in theory, LLMs have the potential to be used for data extraction and clinical decision-making. Currently, LLMs can be used to generate drafts for various types of written materials, including study ideas, research protocols, manuscripts, grant proposals, instructional materials, and patient education materials. For example, a prompt to an LLM could be: ‘Write an outline for a grant application on the role of operation (X) for patients with (Y)’ (Fig. 1). LLMs can also be used to enhance the quality of existing text and eliminate errors, particularly for non-native speakers. This can ensure that written materials produced in surgical science are of higher quality and more understandable for the intended audience. For example, a prompt to the LLM could be: ‘Improve the text in this manuscript draft on (X): (text)’. In future, this text improvement could also be applied to clinical notes, such as surgical reports, if LLMs are integrated into an electronic health record environment. Example of the use of ChatGPT for surgical science In theory, if LLMs were integrated into clinical environments, they could be used to extract information from electronic health records or other data repositories. This would automate data collection for research purposes and potentially mitigate the challenges associated with manual extraction, such as time consumption and human error. For example, a researcher might prompt the system to extract surgical outcome measures from a surgical report. Additionally, LLMs could be used to answer clinician queries directly by analysing the available patient data and providing a comprehensive answer. This could promote more efficient patient management, such as when a clinician needs a summary of a patient’s therapy proceedings. To consider the potential of LLMs in clinical decision-making, it is important to recognize the significance of language models like PubMed GPT and BioGPT, which are trained on medical knowledge4,5. In a mature setting, LLMs could be utilized in clinical workflows, where they automatically evaluate patient information and produce a patient management plan that surgeons can use as a reference point or consider in their decision-making process. This has the potential to streamline the decision-making process, increase efficiency, and ensure that patients receive optimal care. Although LLMs have the potential to be highly beneficial, there are several important considerations to keep in mind. For example, in terms of text generation and improvement, it is crucial for surgical scientists to understand the limitations of current LLMs. One of the most significant issues is known as ‘neural hallucinations,’ where the model generates text that is factually incorrect or nonsensical, despite appearing confident in its ability6. For instance, an LLM might suggest using churros, a type of fried pastry, as surgical instruments due to their size and flexibility7. Additionally, because LLMs are trained on specific datasets, there is a risk of introducing biases into the model’s output. Therefore, it is essential to thoroughly evaluate the output of an LLM before incorporating it into any work. When using these models for writing tasks, surgeon-scientists should be aware of these limitations and carefully check the validity of the model’s output. To ensur
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
Cited by
Artificial intelligence in surgery
Varghese, C., Harrison, E. M., O’Grady, G., & Topol, E. J. (2024) · Nature Medicine · doi:10.1038/s41591-024-02970-3 · Cited by 287
Eppler, M., Ganjavi, C., Ramacciotti, L. S., Piazza, P., Rodler, S., Checcucci, E., Gomez Rivas, J., Kowalewski, K. F., Belenchón, I. R., Puliatti, S., et al. (2024) · European Urology · doi:10.1016/j.eururo.2023.10.014 · Cited by 117
Liu, X., Wu, J., Shao, A., Shen, W., Ye, P., Wang, Y., Ye, J., Jin, K., & Yang, J. (2024) · Journal of Medical Internet Research · doi:10.2196/51926 · Cited by 52
ChatGPT applications in Academic Research: A Review of Benefits, Concerns, and Recommendations
AlZaabi, A., ALAmri, A., Albalushi, H., Aljabri, R., & AalAbdulsalam, A. (2023) · doi:10.1101/2023.08.17.553688 · Cited by 32
Guthrie, E., Levy, D., & del Carmen, G. (2024) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2024.02.016 · Cited by 22
Klotz, R., Pausch, T. M., Kaiser, J., Joos, M. C., Hecktor, R., Ahmed, A., Dörr-Harim, C., Mehrabi, A., Loos, M., Roth, S., et al. (2025) · HPB · doi:10.1016/j.hpb.2024.11.012 · Cited by 18
Metadata
- Authors
- Janssen, B. V., Kazemier, G., & Besselink, M. G.
- Venue
- BJS Open
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-03-07
- Volume
- 7
- Issue
- 2
- ISSN
- 2474-9842 (ISSN-L)
- OpenAlex
- W4360806508
- Cited by
- 53
- References
- 3
- FWCI
- 1.80
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.