The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Outpatient reception via collaboration between nurses and a large language model: a randomized controlled trial
Abstract
Reception is an essential process for patients seeking medical care and a critical component influencing the healthcare experience. However, current communication systems rely mainly on human efforts, which are both labor and knowledge intensive. A promising alternative is to leverage the capabilities of large language models (LLMs) to assist the communication in medical center reception sites. Here we curated a unique dataset comprising 35,418 cases of real-world conversation audio corpus between outpatients and receptionist nurses from 10 reception sites across two medical centers, to develop a site-specific prompt engineering chatbot (SSPEC). The SSPEC efficiently resolved patient queries, with a higher proportion of queries addressed in fewer rounds of queries and responses (Q&Rs; 68.0% ≤2 rounds) compared with nurse-led sessions (50.5% ≤2 rounds) (P = 0.009) across administrative, triaging and primary care concerns. We then established a nurse-SSPEC collaboration model, overseeing the uncertainties encountered during the real-world deployment. In a single-center randomized controlled trial involving 2,164 participants, the primary endpoint indicated that the nurse-SSPEC collaboration model received higher satisfaction feedback from patients (3.91 ± 0.90 versus 3.39 ± 1.15 in the nurse group, P < 0.001). Key secondary outcomes indicated reduced rate of repeated Q&R (3.2% versus 14.4% in the nurse group, P < 0.001) and reduced negative emotions during visits (2.4% versus 7.8% in the nurse group, P < 0.001) and enhanced response quality in terms of integrity (4.37 ± 0.95 versus 3.42 ± 1.22 in the nurse group, P < 0.001), empathy (4.14 ± 0.98 versus 3.27 ± 1.22 in the nurse group, P < 0.001) and readability (3.86 ± 0.95 versus 3.71 ± 1.07 in the nurse group, P = 0.006). Overall, our study supports the feasibility of integrating LLMs into the daily hospital workflow and introduces a paradigm for improving communication that benefits both patients and nurses. Chinese Clinical Trial Registry identifier: ChiCTR2300077245 .
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our system employs local language models to preserve data privacy, which comes at some cost to performance – the local model may not match that of commercial models58,59.”
From A large language model digital patient system enhances ophthalmology history taking skills (2025) · cites this paper as [58]
In the citing paper
“Our system employs local language models to preserve data privacy, which comes at some cost to performance – the local model may not match that of commercial models58,59.”
From A large language model digital patient system enhances ophthalmology history taking skills (2025) · cites this paper as [58]
In the citing paper
“in the largest randomised controlled trial identified in this review, published in Nature Medicine and involving 2,164 patients-to significantly enhance patient satisfaction when integrated into nurse-led communication workflows (Wan et al., 2024).”
From When the algorithm enters the classroom: A critical integrative review of large language models, nursing education structural gaps, and the reconstitution of professional identity (2026) · cites this paper as Wan et al., 2024
In the citing paper
“suggesting LLMs may function effectively as cognitive offloading tools for highly routinised communication tasks, potentially preserving nurses' attentional resources for higher-order clinical judgement (Wan et al., 2024).”
From When the algorithm enters the classroom: A critical integrative review of large language models, nursing education structural gaps, and the reconstitution of professional identity (2026) · cites this paper as Wan et al., 2024
In the citing paper
“Of the participating nurses, 95% reported reduced workload and 90% reported reduced occupational stress”
From When the algorithm enters the classroom: A critical integrative review of large language models, nursing education structural gaps, and the reconstitution of professional identity (2026) · cites this paper as Wan et al. (2024)
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · cited 1×
“Although close-to-human performance of LLMs has been achieved in answering United States Medical Licensing Examination (USMLE)-style questions and patient questions on a public forum 11,12 , several issues have constrained the implementation of LLMs 13 .”
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738
Kwame, A., & Petrucka, P. M. (2021) · BMC Nursing · doi:10.1186/s12912-021-00684-2 · Cited by 1,067
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847
Active Retrieval Augmented Generation
Jiang, Z., Xu, F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.495 · Cited by 497
Research works printed but not linked: 6
- Guo, S., Yang, T. & Dong, S. Research advances on cost-efficiency measurement and evaluation of public hospitals in China. Chin. J. Health Policy 13, 45–51 (2020).
- Zeng, D. The comparison and implication on the cost management of public hospitals between China and the US. Chin. J. Health Policy 12, 13–17 (2012).
- Loh, S. B. & Raamkumar, A. S. Harnessing large language models’ empathetic response generation capabilities for online mental health counselling support. Preprint at https://arxiv.org/abs/2310.08017 (2023). · arXiv
- Martin-Maroto, F. & de Polavieja, G. G. Semantic embeddings in semilattices. Preprint at https://arxiv.org/abs/2205.12618 (2022). · arXiv
- Mao, R., Chen, G., Zhang, X., Guerin, F. & Cambria, E. GPTEval: a survey on assessments of ChatGPT and GPT-4. In Proc. 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) (eds Calzolari, N. et al.) 7844–7866 (ELRA and ICCL, 2024).
- Es, S., James, J., Espinosa-Anke, L. & Schockaert, S. RAGAS: Automated Evaluation of Retrieval Augmented Generation. In Proc. 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (eds Aletras, N. & De Clercq, O.) 150–158 (Association for Computational Linguistics, 2024).
Cited by
Artsi, Y., Sorin, V., Glicksberg, B. S., Korfiatis, P. D., Nadkarni, G. N., & Klang, E. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1659134 · Cited by 126
Howcroft, A., Bennett‐Weston, A., Khan, A. H., Griffiths, J., Gay, S. P., & Howick, J. (2025) · British Medical Bulletin · doi:10.1093/bmb/ldaf017 · Cited by 120
Evaluating large language models and agents in healthcare: key challenges in clinical applications
Chen, X., Xiang, J., Lu, S., Liu, Y., He, M., & Shi, D. (2025) · Intelligent Medicine · doi:10.1016/j.imed.2025.03.002 · Cited by 100
The Roles of Artificial Intelligence in Teaching Anatomy: A Systematic Review
Joseph, T. S., Gowrie, S., Montalbano, M. J., Bandelow, S., Clunes, M. T., Dumont, A. S., Iwanaga, J., Tubbs, R. S., & Loukas, M. (2025) · Clinical Anatomy · doi:10.1002/ca.24272 · Cited by 67
Collaborative, T. C. (2025) · BMJ · doi:10.1136/bmj-2024-083305 · Cited by 62
Metadata
- Authors
- Wan, P., Huang, Z., Tang, W., Nie, Y., Pei, D., Deng, S., Chen, J., Zhou, Y., Duan, H., Chen, Q., & Long, E.
- Affiliations
- National Institutes of Health (US)
- Chinese Academy of Medical Sciences & Peking Union Medical College (CN)
- Southern University of Science and Technology (CN)
- Renmin Hospital of Wuhan University (CN)
- Yale University (US)
- Corresponding author
- Chen, Qingyu
- Long, Erping
- Venue
- Nature Medicine
- Published
- 2024-07-15
- Volume
- 30
- Issue
- 10
- Pages
- 2878-2885
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W4400657362
- Cited by
- 97
- References
- 31
- Citations by year
- 2026: 44 · 2025: 47 · 2024: 6
- FWCI
- 9.07
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- large language models, hospital reception, chatbots, prompt engineering, patient reception, patient satisfaction, nurse-patient communication, triage, response quality, empathy, negative emotions, hospital workflow
- MeSH terms
- Adult, Communication, Cooperative Behavior, Female, Humans, Language, Male, Middle Aged, Nurses, Outpatients, Patient Satisfaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Outpatient reception via collaboration between nurses and a large language model: a randomized controlled trial. https://app.cochat.ai/papers/10.1038/s41591-024-03148-7
- The paper
- Wan, P., Huang, Z., Tang, W., Nie, Y., Pei, D., Deng, S., Chen, J., Zhou, Y., Duan, H., Chen, Q., & Long, E. (2024). Outpatient reception via collaboration between nurses and a large language model: a randomized controlled trial. Nature Medicine. https://doi.org/10.1038/s41591-024-03148-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.