CoChat
Sign Up
Cited by 19Artificial Intelligence in Healthcare and EducationElectronic Health Records SystemsTechnology and Human Factors in Education and Health

Clinical and economic impact of a large language model in perioperative medicine: a randomized crossover trial

Open access
Ke, Yu He, Yang Ong, Bernard Soon, Jin, Liyuan, Sim, Jacqueline Xiu Ling, Chan, Chi Ho, Soh, Chai Rick, Wong, Danny Jon Nian, Liu, Nan +5 more·2025·npj Digital Medicine
DOI 10.1038/s41746-025-01858-xPMC PMC12280117
Open this paper in CoChat — free accountOpen PDF

Abstract

Preoperative assessment is a critical but time-consuming component of perioperative care, often hindered by poor guideline adherence and high documentation burdens. This study evaluates the impact of PEACH (PErioperative AI CHatbot), an LLM-based clinical decision support system, on documentation efficiency, quality, user acceptance, and cost-effectiveness in preoperative consultations. PEACH did not significantly reduce overall documentation time in this randomized crossover trial involving resident physicians at Singapore General Hospital. However, subgroup analyses showed time savings for moderate-complexity patients (5.77 min, p = 0.010) and experienced physicians (4.6 min, p = 0.040). Evaluators preferred PEACH-assisted documentation in 57.1% of cases, with improved inclusion of issue lists (p = 0.05). Economic modeling projected annual institutional savings of SGD197,501 (USD146,297), with sensitivity analyses ranging from SGD 48,979 to 197,499 (USD36,280 to 146,295). These findings suggest that LLM-based tools like PEACH may enhance preoperative documentation efficiency and offer economic value.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-025-01858-x

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • These AI technologies have improved diagnostic accuracy, medication safety, workflow efficiency, and access to specialist care.

    “They have improved diagnostic accuracy, medication safety, workflow efficiency, and access to specialist care 5,6 .”

    From AI-induced never-skilling in medical education (2026) · cites this paper as [6]

    The Results (page unavailable) report shorter documentation time with PEACH in certain subgroups, supporting a limited improvement in workflow efficiency; the overall difference was not statistically significant. The paper does not show improvements in diagnostic accuracy, medication safety, or access to specialist care. Its Results report similar error rates across groups and mention referral drafting, not increased specialist access. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • AI systems have improved medication safety through decision-support tools.

    “improved medication safety through decision-support tools 6,13 .”

    From AI-induced never-skilling in medical education (2026) · cites this paper as [6]

    The Introduction (page not specified) identifies PEACH as an AI clinical decision-support system. The Results (page not specified) report that minor and major error rates were low and similar with and without PEACH; they do not show improved medication safety specifically. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 22

  1. [1]Haana, V., Sethuraman, K., Stephens, L., Rosen, H. & Meara, J. G. Case cancellations on the day of surgery: an investigation in an Australian paediatric hospital. ANZ J. Surg. 79, 636–640 (2009). 19895520
  2. [2]Koushan, M., Wood, L. C. & Greatbanks, R. Evaluating factors associated with the cancellation and delay of elective surgical procedures: a systematic review. Int. J. Qual. Health Care33, mzab092 (2021). 34100548
  3. [3]Thirunavukarasu, A. J. et al. Large language models in medicine. Nat. Med. 29, 1930–1940 (2023). 37460753
  4. [4]Ke, Y. H. et al. Development and testing of retrieval augmented generation in large language models—a case study report. arXiv: https://arxiv.org/abs/2402.01733 (2024). · arXiv
  5. [5]Ke, Y. H. et al. Retrieval augmented generation for 10 large language models and its generalizability in assessing medical fitness. NPJ Digit. Med. 8, 187 (2025). 40185842 PMC11971376
  6. [6]Lim, D. Y. Z. et al. Large language models in anaesthesiology: use of ChatGPT for American Society of Anesthesiologists physical status classification. Br. J. Anaesth. 10.1016/j.bja.2023.06.052 (2023). 37474421 · DOI · Paper page
  7. [7]Ismaiel, N. et al. The evaluation of the performance of ChatGPT in the management of labor analgesia. J. Clin. Anesth. 98, 111582 (2024). 39167880
  8. [8]Turan, E. İ, Baydemir, A. E., Özcan, F. G. & Şahin, A. S. Evaluating the accuracy of ChatGPT-4 in predicting ASA scores: A prospective multicentric study ChatGPT-4 in ASA score prediction. J. Clin. Anesth. 96, 111475 (2024). 38657530
  9. [14]Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023). 37438534 PMC10396962
  10. [15]Chung, P. et al. Large language model capabilities in perioperative risk prediction and prognostication. JAMA Surg. 159, 928–937 (2024). 38837145 PMC11154375
  11. [16]Woo, J. J. et al. Custom Large Language Models improve accuracy: Comparing Retrieval Augmented Generation and Artificial Intelligence agents to non-custom models for evidence-based medicine. Arthroscopy 10.1016/j.arthro.2024.10.042 (2024). 39521391 · DOI
  12. [17]Klang, E. et al. A strategy for cost-effective large language model use at health system-scale. NPJ Digit. Med. 7, 320 (2024). 39558090 PMC11574261
  13. [19]Kim, Y. J., Choi, J. H. & Fotso, G. M. N. Medical professionals’ adoption of AI-based medical devices: UTAUT model with trust mediation. J. Open Innov. 10, 100220 (2024).
  14. [20]Bergsland, J., Elle, O. J. & Fosse, E. Barriers to medical device innovation. Med. Devices 7, 205–209 (2014). PMC4063798 24966699
  15. [21]Renukappa, S., Mudiyi, P., Suresh, S., Abdalla, W. & Subbarao, C. Evaluation of challenges for adoption of smart healthcare strategies. Smart Health 26, 100330 (2022).
  16. [22]Giannakopoulos, K., Kavadella, A., Aaqel Salim, A., Stamatopoulos, V. & Kaklamanos, E. G. Evaluation of the performance of generative AI large language models ChatGPT, Google Bard, and Microsoft Bing Chat in supporting evidence-based dentistry: comparative mixed methods study. J. Med. Internet Res. 25, e51580 (2023). 38009003 PMC10784979
  17. [23]Haider, S. A. et al. Evaluating large language model (LLM) performance on established breast classification systems. Diagnostics 14, 1491 (2024). 39061628 PMC11275570
  18. [24]Sim, X. L. et al. Transforming the perioperative medicine care model: the Singapore experience. Anaesth. Intensive Care 51, 96–106 (2023). 36688348
  19. [25]Xiao, G. et al. DuoAttention: efficient long-context LLM inference with retrieval and Streaming Heads. ArXiv: https://arxiv.org/abs/2410.10819 (2024). · arXiv
  20. [26]Peng, H. et al. When does in-context learning fall short and why? A study on specification-heavy tasks. arXiv: https://arxiv.org/abs/2311.08993 (2023). · arXiv
  21. [28]Godbole, A., George, J. G. & Shandilya, S. Leveraging long-context Large Language Models for multi-document understanding and summarization in enterprise applications. arXiv: https://arxiv.org/abs/2409.18454 (2024). · arXiv
  22. [29]Meskó, B. Prompt engineering as an important emerging skill for medical professionals: tutorial. J. Med. Internet Res. 25, e50638 (2023). 37792434 PMC10585440

Cited by

Sorted by citations

Authors
Ke, Y. H., Yang Ong, B. S., Jin, L., Sim, J. X. L., Chan, C. H., Soh, C. R., Wong, D. J. N., Liu, N., Sng, B. L., Ting, D. S. W., Yeo, S. Q., Ong, M. E. H., & Abdullah, H. R.
Affiliations
  • SingHealth (SG)
  • Singapore General Hospital (SG)
  • Duke-NUS Medical School (SG)
  • KK Women's and Children's Hospital (SG)
  • Singapore National Eye Center (SG)
Corresponding author
Ke, Yu He
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2025-07-21
Volume
8
Issue
1
Pages
462-462
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-025-01858-x
PMC
PMC12280117
OpenAlex
W4412535374
Cited by
19
References
6
Citations by year
2026: 17 · 2025: 2
FWCI
4.57
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Medical Research Council (HCSAINV22jul-0002)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
Keywords
large language models, clinical decision support, preoperative assessment, preoperative documentation, documentation efficiency, cost-effectiveness, user acceptance, medical residents, AI chatbots, perioperative medicine, economic modeling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review