The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge
Open accessAbstract
Objective The primary aim of this research was to address the limitations observed in the medical knowledge of prevalent large language models (LLMs) such as ChatGPT, by creating a specialized language model with enhanced accuracy in medical advice. Methods We achieved this by adapting and refining the large language model meta-AI (LLaMA) using a large dataset of 100,000 patient-doctor dialogues sourced from a widely used online medical consultation platform. These conversations were cleaned and anonymized to respect privacy concerns. In addition to the model refinement, we incorporated a self-directed information retrieval mechanism, allowing the model to access and utilize real-time information from online sources like Wikipedia and data from curated offline medical databases. Results The fine-tuning of the model with real-world patient-doctor interactions significantly improved the model's ability to understand patient needs and provide informed advice. By equipping the model with self-directed information retrieval from reliable online and offline sources, we observed substantial improvements in the accuracy of its responses. Conclusion Our proposed ChatDoctor, represents a significant advancement in medical LLMs, demonstrating a significant improvement in understanding patient inquiries and providing accurate advice. Given the high stakes and low error tolerance in the medical field, such enhancements in providing accurate and reliable information are not only beneficial but essential.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.754 · Cited by 1,243
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.45 · Cited by 682 · cited 1×
“We proposed an autonomous ChatDoctor model that can retrieve online and offline medical domain knowledge to answer medical questions on up-to-date medical terms and diseases, which can potentially reduce the errors and hallucinations of LLMs [9-11].”
MethodGessain, A., Nakoune, E., & Yazdanpanah, Y. (2022) · New England Journal of Medicine · doi:10.1056/nejmra2208860 · Cited by 409
ChatGPT: Is this version good for healthcare and research?
Vaishya, R., Misra, A., & Vaish, A. (2023) · Diabetes & Metabolic Syndrome: Clinical Research & Reviews · doi:10.1016/j.dsx.2023.102744 · Cited by 293
MEANS: A medical question-answering system combining NLP techniques and semantic Web technologies
Ben Abacha, A., & Zweigenbaum, P. (2015) · Information Processing & Management · doi:10.1016/j.ipm.2015.04.006 · Cited by 230
Hatherley, J. J. (2020) · Journal of Medical Ethics · doi:10.1136/medethics-2019-105935 · Cited by 230
Research works printed but not linked: 2
- Bertscore. Evaluating text generation with bert. (2020). Accessed: April 21, 2020: http://arXiv:1904.09675.
- Raise a child in large language model. towards effective and generalizable fine-tuning. (2021). Accessed: September 13, 2021: http://arXiv:2109.05687.
Cited by
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 4,688
Large language models in health care: Development, applications, and challenges
Yang, R., Tan, T. F., Lu, W., Thirunavukarasu, A. J., Ting, D. S. W., & Liu, N. (2023) · Health Care Science · doi:10.1002/hcs2.61 · Cited by 366
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health
Tian, S., Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad493 · Cited by 359
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
Xu, X., Yao, B., Dong, Y., Gabriel, S., Yu, H., Hendler, J., Ghassemi, M., Dey, A. K., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3643540 · Cited by 278
“Researchers have explored the capability of these LLMs in health fields [54, 61, 62, 73, 98, 109].”
BackgroundRetrieval augmented generation for large language models in healthcare: A systematic review
Amugongo, L. M., Mascheroni, P., Brooks, S., Doering, S., & Seidel, J. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000877 · Cited by 267
Yu, P., Xu, H., Hu, X., & Deng, C. (2023) · Healthcare · doi:10.3390/healthcare11202776 · Cited by 257
Large language models in medical and healthcare fields: applications, advances, and challenges
Wang, D., & Zhang, S. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10921-0 · Cited by 246
Moulaei, K., Yadegari, A., Baharestani, M., Farzanbakhsh, S., Sabet, B., & Reza Afrash, M. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105474 · Cited by 244
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge
Zhou, H., Liu, F., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Zhou, P., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05112 · Cited by 240
“The details of existing general and medical large language models are shown in Table 1 and Table 2, respectively.”
BackgroundSelf-citationA Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242
Cascella, M., Semeraro, F., Montomoli, J., Bellini, V., Piazza, O., & Bignami, E. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02045-3 · Cited by 219
Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI
Abbasian, M., Khatibi, E., Azimi, I., Oniani, D., Shakeri Hossein Abad, Z., Thieme, A., Sriram, R., Yang, Z., Wang, Y., Lin, B., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01074-z · Cited by 213
Med-HALT: Medical Domain Hallucination Test for Large Language Models
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2023) · Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/2023.conll-1.21 · Cited by 152
Med-HALT: Medical Domain Hallucination Test for Large Language Models
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15343 · Cited by 152
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)
Zeng, S., Zhang, J., He, P., Liu, Y., Xing, Y., Xu, H., Ren, J., Chang, Y., Wang, S., Yin, D., et al. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.267 · Cited by 149
Towards building multilingual language model for medicine
Qiu, P., Wu, C., Zhang, X., Lin, W., Wang, H., Zhang, Y., Wang, Y., & Xie, W. (2024) · Nature Communications · doi:10.1038/s41467-024-52417-z · Cited by 147
Self-citationHe, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102963 · Cited by 147
A Survey on the Memory Mechanism of Large Language Model based Agents
Zhang, Z., Bo, X., Ma, C., Li, R., Chen, X., Dai, Q., Zhu, J., Dong, Z., & Wen, J.-R. (2024) · arXiv · doi:10.48550/arxiv.2404.13501 · Cited by 149
Ng, K. K. Y., Matsuba, I., & Zhang, P. C. (2025) · NEJM AI · doi:10.1056/aira2400380 · Cited by 107
DRG-LLaMA : tuning LLaMA model to predict diagnosis-related group for hospitalized patients
Wang, H., Gao, C., Dantona, C., Hull, B., & Sun, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00989-3 · Cited by 99
Application of large language models in medicine
Liu, F., Zhou, H., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Hua, Y., Zhou, P., et al. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00279-5 · Cited by 94
Self-citationWang, D., Liang, J., Ye, J., Li, J., Li, J., Zhang, Q., Hu, Q., Pan, C., Wang, D., Liu, Z., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/58041 · Cited by 76
Medical foundation large language models for comprehensive text analysis and beyond
Xie, Q., Chen, Q., Chen, A., Peng, C., Hu, Y., Lin, F., Peng, X., Huang, J., Zhang, J., Keloth, V., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01533-1 · Cited by 74
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69
A bibliometric analysis of the rise of ChatGPT in medical research
Barrington, N. M., Gupta, N., Musmar, B., Doyle, D., Panico, N., Godbole, N., Reardon, T., & D’Amico, R. S. (2023) · Medical Sciences · doi:10.3390/medsci11030061 · Cited by 69
MedExpQA: Multilingual benchmarking of Large Language Models for Medical Question Answering
Alonso, I., Oronoz, M., & Agerri, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102938 · Cited by 62
Conversational health agents: a personalized large language model-powered agent framework
Abbasian, M., Azimi, I., Rahmani, A. M., & Jain, R. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf067 · Cited by 52
Wang, L., Li, J., Zhuang, B., Huang, S., Fang, M., Wang, C., Li, W., Zhang, M., & Gong, S. (2025) · Journal of Medical Internet Research · doi:10.2196/64486 · Cited by 51
Leveraging Large Language Models for Generating Responses to Patient Messages
Liu, S., McCoy, A. B., Wright, A. P., Carew, B., Genkins, J. Z., Huang, S. S., Peterson, J. F., Steitz, B., & Wright, A. (2023) · doi:10.1101/2023.07.14.23292669 · Cited by 43
MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
Cai, Y., Wang, L., Wang, Y., De Melo, G., Zhang, Y., Wang, Y., & He, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29723 · Cited by 42
Self-citationCevasco, K. E., Morrison Brown, R. E., Woldeselassie, R., & Kaplan, S. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02059-x · Cited by 35
Evaluating search engines and large language models for answering health questions
Fernández-Pichel, M., Pichel, J. C., & Losada, D. E. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01546-w · Cited by 33
Rossettini, G., Bargeri, S., Cook, C., Guida, S., Palese, A., Rodeghiero, L., Pillastrini, P., Turolla, A., Castellini, G., & Gianola, S. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1574287 · Cited by 31
Goel, I., Bhaskar, Y., Kumar, N., Singh, S., Amanullah, M., Dhar, R., & Karmakar, S. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1550407 · Cited by 31
Dronkers, E. A. C., Geneid, A., al Yaghchi, C., & Lechien, J. R. (2025) · Journal of Voice · doi:10.1016/j.jvoice.2024.02.020 · Cited by 20
Chan, S., Li, J., Yao, B., Mahmood, A., Huang, C.-M., Jimison, H., Mynatt, E. D., & Wang, D. (2023) · arXiv · doi:10.48550/arxiv.2310.05853 · Cited by 16
Medical multimodal large language models: A systematic review
Hu, Y., Xu, C., Lin, B., Yang, W., & Tang, Y. Y. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.09.005 · Cited by 14
Bedi, P., Thukral, A., & Dhiman, S. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02465-8 · Cited by 10
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Xu, W., Chan, H. P., Li, L., Aljunied, M., Yuan, R., Wang, J., Xiao, C., Chen, G., Liu, C., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.07044 · Cited by 4
Self-citationExternal Data Extraction Attacks Against Retrieval-Augmented Large Language Models
He, Y., Chen, Y., Li, Y., Shao, S., Qi, L., Li, B., Tao, D., & Qin, Z. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3705326 · Cited by 4
Self-citationTrustworthy Medical Question Answering: An Evaluation-Centric Survey
Wang, Y., Wang, B., Mercer, R. E., Rudzicz, F., Roy, S. S., Ren, P., Chen, Z., & Wang, X. (2025) · arXiv · doi:10.48550/arxiv.2506.03659
“In healthcare, specialized variants such as Med-PaLM (Singhal et al., 2023) and Chat-Doctor (Li et al., 2023b) have even matched or ✉ Corresponding author exceeded human performance on professional exams —Med-PaLM achieved a passing score of 67.6% on USMLE-style MedQA questions and Med-PaLM 2 reached 86.5% accuracy— a…”
Influential
Metadata
- Authors
- Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S., & Zhang, Y.
- Venue
- Cureus
- Published
- 2023-06-24
- Volume
- 15
- Issue
- 6
- Pages
- e40895-e40895
- ISSN
- 2168-8184 (ISSN-L)
- OpenAlex
- W4381930847
- Cited by
- 505
- References
- 15
- FWCI
- 17.47
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.